Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: compliment with latest update 2.2 startuptime lot faster! 2GB lib. (Read 2166 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Hoi FooBar 2000,

thank for amazing work!
All the best from Amsterdam!
Enjoy!
FJ

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #1 –
I just tried 2.24 preview 64 and startup time is greatly improved - 3-5min instead of 30-60.

RAM usage (due to refacets?) is still a non-starter.

I'm using more components and more complicated theme w 32bit but ram usage is still ~%50 less?

Both accessing the same library.

UPDATE: 2.24 did not close cleanly - window closed but background process hung and required kill process.
I disabled ALL components and startup is back to previous slowness.
Took 26min for a responsive FB window to appear. RAM usage with no components: 5.65GB
Exited again and it hung again as background process (no components!)


32bit 2.1.6 (metadb.sqlite 2.25GB):
Core (2024-09-19 09:01:40 UTC)
    foobar2000 core 2.1.6
foo_bpm (2015-03-01 03:39:58 UTC)
    BPM Analyser 0.2.4.6
foo_converter (2024-09-19 09:02:10 UTC)
    Converter 2.1.6
foo_discogger (2024-09-13 04:08:14 UTC)
    Discogger 1.0.21.1
foo_dsp_eq (2024-09-19 09:02:14 UTC)
    Equalizer 1.2.3
foo_dsp_std (2024-09-19 09:02:16 UTC)
    Standard DSP Array 2.1.6
foo_facets (2011-08-27 18:05:16 UTC)
    Facets 1.0
foo_fileops (2024-09-19 09:02:20 UTC)
    File Operations 2.1.6
foo_flowin (2022-10-17 15:54:26 UTC)
    Flowin 0.2.0 beta
foo_freedb2 (2024-09-19 09:02:24 UTC)
    Online Tagger 0.10
foo_input_std (2024-09-19 09:02:04 UTC)
    CD Audio Decoder 2.1.6
    FFmpeg Decoders 6.0
    FLAC Decoder 1.4.3
    Monkey's Audio Decoder 10.30
    Opus Decoder 1.4
    Standard Input Array 2.1.6
foo_playcount (2023-03-14 18:03:58 UTC)
    Playback Statistics 3.1.5
foo_run (2022-09-19 16:12:20 UTC)
    Run services 0.4.5
foo_scrobble (2022-09-06 05:43:00 UTC)
    Scrobble 1.6.0.22456
foo_simplaylist (2018-09-05 06:56:25 UTC)
    SimPlaylist 1.0
foo_ui_std (2024-09-19 09:01:50 UTC)
    Album List 2.1.6
    Decoding Speed Test 2.1.6
    Default User Interface 2.1.6
    File Integrity Verifier 2.1.6
foo_uie_jsplitter (2024-10-05 04:31:02 UTC)
    JSplitter 3.6.1.10
foo_uie_webview (2024-08-10 14:11:28 UTC)
    WebView 0.1.8.0
foo_unpack (2024-09-19 09:02:32 UTC)
    ZIP/GZIP/RAR/7-Zip Reader 2.1.6
foo_verifier (2023-04-17 18:43:48 UTC)
    File Integrity Verifier 1.4.3
foo_wave_minibar_mod (2024-01-16 17:24:18 UTC)
    Waveform Minibar (mod) 1.2.58

64bit 2.24 preview (metadb.sqlite 1.95GB):
Core (2024-10-11 20:50:00 UTC)
    foobar2000 core 2.24 preview 2024-10-11
foo_converter (2024-10-11 20:50:32 UTC)
    Converter 2.24 preview 2024-10-11
foo_discogger (2024-08-25 16:46:44 UTC)
    Discogger 1.0.20.5
foo_dsp_eq (2024-10-11 20:50:36 UTC)
    Equalizer 1.2.3
foo_dsp_std (2024-10-11 20:50:38 UTC)
    Standard DSP Array 2.24 preview 2024-10-11
foo_fileops (2024-10-11 20:50:42 UTC)
    File Operations 2.24 preview 2024-10-11
foo_flowin (2022-10-17 15:55:30 UTC)
    Flowin 0.2.0 beta
foo_freedb2 (2024-10-11 20:50:46 UTC)
    Online Tagger 0.10
foo_input_std (2024-10-11 20:50:28 UTC)
    CD Audio Decoder 2.24 preview 2024-10-11
    FFmpeg Decoders 6.0
    FLAC Decoder 1.4.3
    Monkey's Audio Decoder 10.61
    Opus Decoder 1.5.2
    Standard Input Array 2.24 preview 2024-10-11
    WavPack Decoder 5.7.0
foo_jscript_panel3 (2024-08-31 23:46:22 UTC)
    JScript Panel 3 3.7.3
foo_playcount (2023-03-14 18:04:18 UTC)
    Playback Statistics 3.1.5
foo_run (2022-09-19 15:14:04 UTC)
    Run services 0.4.5
foo_scrobble (2022-09-06 04:43:00 UTC)
    Scrobble 1.6.0.22456
foo_ui_std (2024-10-11 20:50:14 UTC)
    Album List 2.24 preview 2024-10-11
    Decoding Speed Test 2.24 preview 2024-10-11
    Default User Interface 2.24 preview 2024-10-11
    File Integrity Verifier 2.24 preview 2024-10-11
foo_uie_webview (2024-08-10 14:09:06 UTC)
    WebView 0.1.8.0
foo_unpack (2024-10-11 20:50:56 UTC)
    ZIP/GZIP/RAR/7-Zip Reader 2.24 preview 2024-10-11
foo_wave_minibar_mod (2024-01-16 17:24:36 UTC)
    Waveform Minibar (mod) 1.2.58

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #3 –
I think Peter needs a memory dump of the frozen process to see what is going on. I would be curious to have your config to see how on earth anything can be that slow.
The idea with sqlite database was to decrease memory use as not all data needs to be in memory. But I fear the compatibility with legacy components ruined that benefit.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #4 –
mine was 2tb music startuptime used to b 12-14 seconds.
now it is 3.6 sec.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #5 –
Thank you Peter and Team!

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #6 –
@JKFJ not trying to jack your thread, v2.2 started faster for me too, unfortunately it doesn't quit properly

Quote from: Case on
I think Peter needs a memory dump of the frozen process to see what is going on. I would be curious to have your config to see how on earth anything can be that slow.

v2 does not have a core.cfg file; how specifically do I share my config?

Also do you mean dump via Process Manager or is there another means to do so?
mini dump attached; full dump was 3.79GB (not attached)

In my experience, anytime FB must be force closed, the next run is extremely slow to open. I assume this is because I keep Libary>rescan on startup active.
It used to open and then continue scanning library. Is that still a thing, or does it scan before opening?

That v.2.24 only takes 30+ min to relaunch (image) is a huge update from the 6.5 hours it previously took with v2.16, which was documented (on a fresh 2.16 64 install) here:
https://hydrogenaud.io/index.php/topic,126548.msg1051202.html#msg1051202

I lost a long weekend to that, so clearly I'm committed to see this improved.

No user components
Core (2024-10-11 20:50:00 UTC)
    foobar2000 core 2.24 preview 2024-10-11
foo_converter (2024-10-11 20:50:32 UTC)
    Converter 2.24 preview 2024-10-11
foo_dsp_eq (2024-10-11 20:50:36 UTC)
    Equalizer 1.2.3
foo_dsp_std (2024-10-11 20:50:38 UTC)
    Standard DSP Array 2.24 preview 2024-10-11
foo_fileops (2024-10-11 20:50:42 UTC)
    File Operations 2.24 preview 2024-10-11
foo_freedb2 (2024-10-11 20:50:46 UTC)
    Online Tagger 0.10
foo_input_std (2024-10-11 20:50:28 UTC)
    CD Audio Decoder 2.24 preview 2024-10-11
    FFmpeg Decoders 6.0
    FLAC Decoder 1.4.3
    Monkey's Audio Decoder 10.61
    Opus Decoder 1.5.2
    Standard Input Array 2.24 preview 2024-10-11
    WavPack Decoder 5.7.0
foo_ui_std (2024-10-11 20:50:14 UTC)
    Album List 2.24 preview 2024-10-11
    Decoding Speed Test 2.24 preview 2024-10-11
    Default User Interface 2.24 preview 2024-10-11
    File Integrity Verifier 2.24 preview 2024-10-11
foo_unpack (2024-10-11 20:50:56 UTC)
    ZIP/GZIP/RAR/7-Zip Reader 2.24 preview 2024-10-11

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #7 –
Wouldn't it be better to get it checked once?

CPU-Z
https://www.cpuid.com/softwares/cpu-z.html

GPU-Z
https://www.techpowerup.com/download/techpowerup-gpu-z/

CrystalDiskInfo
https://sourceforge.net/projects/crystaldiskinfo/files/9.4.4/CrystalDiskInfo9_4_4.exe/download
SHURE SRH1840, SENNHEISER HD660S2, SENNHEISER HD620S, SENNHEISER HD 490 Pro Plus, beyerdynamic DT 1990 PRO, HiFiMAN Edition XS, Bowers & Wilkins P7, FiiO FT5, 水月雨 (MOONDROP) 空鳴 - VOID, Nakamichi Elite FIVE ANC, SONY WH1000XM5 (made a Upgrade/Balanced Cable by myself)

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #8 –
Quote from: Air KEN on
Wouldn't it be better to get it checked once?


Not really sure what we are checking? Is this the config @Case was referring to?

This is a year old machine w SSD, should certainly run FB?

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #9 –
I checked the machine specs and storage status.
SHURE SRH1840, SENNHEISER HD660S2, SENNHEISER HD620S, SENNHEISER HD 490 Pro Plus, beyerdynamic DT 1990 PRO, HiFiMAN Edition XS, Bowers & Wilkins P7, FiiO FT5, 水月雨 (MOONDROP) 空鳴 - VOID, Nakamichi Elite FIVE ANC, SONY WH1000XM5 (made a Upgrade/Balanced Cable by myself)

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #10 –
Quote from: mjm716 on
v2 does not have a core.cfg file; how specifically do I share my config?
Core.cfg would not have been enough with older versions either for this situation. The contents of the profile directory would be ideal. Just metadata contents alone aren't enough to simulate what is done with them during startup.

Quote from: mjm716 on
Also do you mean dump via Process Manager or is there another means to do so?
Task Manager memory dump. Should be identical to Process Explorer's dump. I fear minidump isn't enough, but I linked this to Peter just in case. We can wait and see if that minidump is usable for finding out the cause.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #11 –
Thanks for the dump.
It's a metadb garbage collect in progress, triggered once in a while to optimize metadb performance. Unfortunately it takes a while to finish on huge data set.
By design this is attempted once a week, I'll re-evaluate this since it's obviously not acceptable / makes the app seem stuck in your case.
Microsoft Windows: We can't script here, this is bat country.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #12 –
The garbage purge itself is needed and appreciated.  Perhaps it could be made an "on-demand" option, with a warning when excessive time may be involved?

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #13 –
Quote from: Case on
The contents of the profile directory would be ideal.

Task Manager memory dump.

Full FB64 directory attached.

Quote from: Peter on
Thanks for the dump.
It's a metadb garbage collect in progress, triggered once in a while to optimize metadb performance. Unfortunately it takes a while to finish on huge data set.

I did leave it for more than an hour before killing the process, so not sure how long the garbage collection should take?

The full dump with PM was 3.8 GB. I also tried with TM and size is 4.3GB.
I could take the effort to upload either somewhere, if anyone is actually going to download it?

I appreciate everyone's input.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #14 –
I didn't mean a list of contents. I meant the actual files so one could test the poor performance. Huge libraries to test with have historically been used in foobar2000 development to optimize performance.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #15 –
If you upload your metadb.sqlite, I'll download and investigate it.
Best send link to me via private message only, no need to post it in public.
Thanks.
Microsoft Windows: We can't script here, this is bat country.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #16 –
I don't mind, I'm scrobbling daily. Happy to share if the dataset is useful here (and if anyone points out efficiencies I can use)

90% compression rate on the db is amazing... seems like an opportunity.

profile: https://drive.google.com/file/d/1pm7tkLW0eCplyBmXsu9RpZj-YK-pJJRZ/view?usp=drive_link

Main thing I'd like to see is the 5.5GB ram usage of 64 get down towards the 2.5GB usage of 32 (per image in reply #1 above).
Both are using that dataset, main difference is refacets/facets.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #17 –
Thanks.
Profile downloaded, will look into tomorrow.
Microsoft Windows: We can't script here, this is bat country.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #18 –
I haven't been able to provoke anything remotely as horrible on my machine using usual procedures.
However one important fact caught my attention: your portable install + profile is not on your shiny fast Samsung SSD, but on something else. What exactly is your S: volume?
If it's a spinning drive, or worse, a network share, it would explain horrible performance, especially of garbage collect operations.

Please use your SSD for foobar2000 install + profile, or else random access to profile data will suffer.
Old versions of foobar2000 may handle this better, never doing garbage collect but still rewriting large files much on shutdown, just in a more sequential manner which works better in your case.
Microsoft Windows: We can't script here, this is bat country.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #19 –
Update,

Today's build introduces emergency toggle for cases like this: create blank file named "suppress-garbage-collect", without the quotation marks, in your profile folder.

This will casue foobar2000 to stop trying to perform garbage collection on the metadb.sqlite file, effectively working around this particular scenario.
Microsoft Windows: We can't script here, this is bat country.

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #20 –
Quote from: Peter on
I haven't been able to provoke anything remotely as horrible on my machine using usual procedures.

The process to recreate is set library to "rescan at startup", then either add new items to library and restart, or force quit. Either option results in long start time.
Of course using my data, FB would not be reading actual files, so it is more of a virtual test.

I imagine ram usage should still be approximate in that scenario - what sort of ram usage did you experience?

Quote from: Peter on
Please use your SSD for foobar2000 install + profile, or else random access to profile data will suffer.

Thanks, this is useful and I have been overlooking that obvious point!
I try to keep my music portable, but will look for an alternative workflow to accommodate.


Quote from: Peter on
This will casue foobar2000 to stop trying to perform garbage collection on the metadb.sqlite file, effectively working around this particular scenario.

Does this eliminate clean-up entirely, or will clean-up happen at a more opportune time e.g. at next run?

thanks again!

Re: compliment with latest update 2.2 startuptime lot faster! 2GB lib.

Reply #21 –
Hi All,
Peter mine works great all together!
fast and responsive.
Garbage collect.. woks fine.

Once Again Thank you Peter!!


Jeroen.