Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Large Library Performance (Read 2510 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Large Library Performance

I'm trying to improve my FB performance (if possible) and would welcome any help/suggestions.

For my testing, I'm using 1.4.2 foobar portable and I disabled ALL user components except for Facets.

Here is a real-time video of my issue - It twice illustrates a 30 second FB freeze when changing selection from a single artist to the entire library (notice beach ball cursor and delay before FB responds and facets window actually scrolls)
dropbox video screen grab

maybe it's not an issue at all and simply the best FB can do under the Library size circumstances (1.73TB)?
Certainly some additional Console feedback during the WIN freeze time would be helpful.

Also, a side question: It appears that FB is using FFmpeg 3.2.4 when I have 4.1.1 installed?


thanks in advance!

Re: Large Library Performance

Reply #1 –
How many tracks total?

How much complex titleformatting are you using in facets or in your playlist viewer?

Are any facets filters activated (which will filter across all your panes)?

What drive type? Remote or local? Does it go into sleep mode?

Have you tried adjusting LargeFieldsConfig.txt AND also getting rid of junk tags in general (if you have any)?

CPU type, load figures during all this? RAM?

I'm assuming the custom look is from your windows theme and not related to your foobar config?

Did you actually remove Simplaylist and the other components which are visible on your skin (at least as tabs hinting at them)?

To me it looks like facets might finish rendering and only then foobar freezes. You could check this by looking at the height of the scrollbar. If it changes first then freezes, presumably facets finished then something else happened (that caused the freeze). If the size only changes after the freeze ends, it's more likely related to facets itself. Given the scrollbar most likely has a minimum width with enough tracks, you might have to experiment with just the right amount of tracks to cause a freeze while still being able to tell the difference.

I would not rule out an overly zealous antivirus either.

Although my library presumably isn't as big as yours, it's not a small one either and I have found facets to be especially performant, compared to something like CUI filter panels or just in general. I would primarily suspect something that has to parse the data after facets updated, such as your playlist viewer.

Re: Large Library Performance

Reply #2 –
Quote from: Daeron on
How many tracks total?
226K

How much complex titleformatting are you using in facets or in your playlist viewer?
Facets only active component, only using a few ifs, eg:
$if($strstr(%album artist%,Various Artists),$meta(album artist,0),%artist%)

Are any facets filters activated (which will filter across all your panes)?
no

What drive type? Remote or local? Does it go into sleep mode?
external on USB3, no sleep mode

Have you tried adjusting LargeFieldsConfig.txt AND also getting rid of junk tags in general (if you have any)?
all tags cleaned
# Size limit for meta fields that appear in the basic fields list
basicMetaMax=2000
# Size limit for meta fields that do not appear in either list
defaultMetaMax=1000
# Size limit for tech info fields
infoMax=200

CPU type, load figures during all this? RAM?
intel i3 1.7ghz; 10GB RAM

I'm assuming the custom look is from your windows theme and not related to your foobar config?
dark theme via OS, DUI with Facets only installed for the test

Did you actually remove Simplaylist and the other components which are visible on your skin (at least as tabs hinting at them)?
everything removed except for facets

To me it looks like facets might finish rendering and only then foobar freezes.
yes, seems to - Facets renders with counts, and then the freeze happens for additional 25seconds - therein lies the problem - nothing in console, so why is the app hanging after Facets renders list?

I would not rule out an overly zealous antivirus either.
tested with AV disabled - same issue

Although my library presumably isn't as big as yours, it's not a small one either and I have found facets to be especially performant, compared to something like CUI filter panels or just in general. I would primarily suspect something that has to parse the data after facets updated, such as your playlist viewer.
yes, but with no other components or scripting running, it must be FB app?

Re: Large Library Performance

Reply #3 –
Quote from: mjm716 on
CPU type, load figures during all this? RAM?
intel i3 1.7ghz; 10GB RAM
Hey there, you skipped how much CPU load - right-click taskbar > Task Manager and see how much CPU is working during the freeze. Further, you could try Appendix B of the pinned support topic and post results for developers to look at.

Re: FFmpeg, I think foobar uses it own internal version for compatibility reasons.

Possible bugs aside, I wonder if the i3 (1.7GHz seems low) might be a bottleneck. In a recent thread, davideleo had some amazing results upgrading the CPU, although a specific case of auto-playlists at startup.

Cheers

Re: Large Library Performance

Reply #4 –
Quote from: anamorphic on
Quote from: mjm716 on
CPU type, load figures during all this? RAM?
intel i3 1.7ghz; 10GB RAM
Hey there, you skipped how much CPU load - right-click taskbar > Task Manager and see how much CPU is working
FB is running robustly around 750MB/below 5% CPU. When I do click from single artist to total library, RAM use maxes to about 825MB and CPU spikes only to about 75%, although Task Manager does flash 'not responding'; numbers drop back immediately once control returns.

Quote from: anamorphic on

Possible bugs aside, I wonder if the i3 (1.7GHz seems low) might be a bottleneck.
This laptop is a dog relatively, and it's what I'm stuck with for the time being. :(


thanks for the other tips.

the only process blowing up when clicking from single artist to Library is Facets. Here is the stack:
Code: [Select]
ntoskrnl.exe!KeSynchronizeExecution+0x5be6
ntoskrnl.exe!KeWaitForSingleObject+0x12e6
ntoskrnl.exe!KeWaitForSingleObject+0xadb
ntoskrnl.exe!KeWaitForSingleObject+0x1ff
ntoskrnl.exe!KeCheckProcessorGroupAffinity+0x98a
ntoskrnl.exe!ExIsResourceAcquiredSharedLite+0x7cb
ntoskrnl.exe!KeWaitForSingleObject+0x1521
ntoskrnl.exe!KeWaitForSingleObject+0xadb
ntoskrnl.exe!KeWaitForMultipleObjects+0x467
ntoskrnl.exe!ObWaitForMultipleObjects+0x2a0
ntoskrnl.exe!RtlUnicodeStringToInteger+0x6d9
ntoskrnl.exe!_setjmpex+0x6f83
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x547
wow64cpu.dll!TurboDispatchJumpAddressEnd+0x9e
wow64cpu.dll!BTCpuSimulate+0x9
wow64.dll!Wow64LdrpInitialize+0x236
wow64.dll!Wow64LdrpInitialize+0x120
ntdll.dll!LdrInitializeThunk+0x179
ntdll.dll!LdrInitializeThunk+0x5b
ntdll.dll!LdrInitializeThunk+0xe

I also made a video of Process Explorer activity during the freeze. Nothing jumps out at me, but maybe someone with better skills will see something?

Re: Large Library Performance

Reply #5 –
Given it's a laptop i3 I'm assuming it's 2 cores 4 threads? I'm not 100% confident on this but as I recall the load numbers are skewed when Hyper Threading is in place, meaning 75% might as well be full load.

Does the entire OS freeze or just foobar? I'm guessing yes, if Task Manager is affected.

So it's almost certainly your weak CPU.

Re: Large Library Performance

Reply #7 –
I've had a larger library and it loaded to playlist in about 1-2 seconds
edit: not with facets

Re: Large Library Performance

Reply #8 –
Quote from: mjm716 on
Here is a real-time video of my issue - It twice illustrates a 30 second FB freeze when changing selection from a single artist to the entire library (notice beach ball cursor and delay before FB responds and facets window actually scrolls)
I just tested a ~223.000 tracks library on a fresh foobar2000 portable installation, with only facets as a user-component: it's lightning-fast.


Quote from: anamorphic on
Possible bugs aside, I wonder if the i3 (1.7GHz seems low) might be a bottleneck. In a recent thread, davideleo had some amazing results upgrading the CPU, although a specific case of auto-playlists at startup.

Yes, @Case wrote more than once that my set up was limited by the CPU, but I wasn't quite convinced because my previous CPU (also a 2 cores/4 threads i3) was never loaded over 30% by foobar2000. However, my new 6 core CPU spikes to almost 100% on start-up for a few seconds. This is probably the main reason why the start-up time is so dramatically reduced.
What I mean is that a low CPU load does not necessarily mean that it is powerful enough, it might simply be bottlenecked by other hardware components. My poor i3 had no reason to run faster, given the slow sata II connections and DDR3 RAM.
I'm late

Re: Large Library Performance

Reply #9 –
If one watches the video of process activity I previously posted, it indicates several things.

First the set-up is a fresh portable 1.42 install with only DUI/Facets.

For the video:
I click between artists and then show FB in process explorer as baseline.
Then I click back to FB, select the entire library and quickly switch back to FB process explorer activity.
One can see that Facets does render the library list quickly (3-4 seconds) in the background FB window.
However the list and FB remain unresponsive for another 25sec, also indicated by the blue beach ball spinning cursor over the FB window.
During that freeze, one can monitor FB process explorer activity*. I grab the Facet stack during that time, so my machine is responding normally during the FB freeze.

*The FB process explorer shows many threads starting and stopping (green and red color). These say "ntdll.dll!RtlUser Thread Start" but only give "Unable to access thread" messages when trying to get Stack/info.

Re: Large Library Performance

Reply #10 –
I'd like to point out that the current implementation of "Selection Properties" element is inefficient for large track groups and stalls the UI doing its job. You can try getting rid of any instances of it to see if it helps.

This has been addressed for upcoming foobar2000 v1.5, it will no longer block the UI while updating as well as use multiple CPU cores.
Microsoft Windows: We can't script here, this is bat country.

Re: Large Library Performance

Reply #11 –
1.5 is an exciting prospect - sign me up for beta test!

Quote from: Peter on
You can try getting rid of any instances of it to see if it helps.

Sorry, I'm not following - I'm only aware of selecting tracks and getting properties (e.g. alt-enter)

What would be an 'instance of selection properties'?
 

Re: Large Library Performance

Reply #12 –
Quote from: mjm716 on
What would be an 'instance of selection properties'?
The DUI element named Selection Properties. Perhaps, what element do you have in your "metadata" tab? (Seen in your first video)

Re: Large Library Performance

Reply #13 –
a ha, thanks! forgot about that one; unfortunately removing it does not affect the delay.