Problem: installed plugins limit decreases over time

Topic: Problem: installed plugins limit decreases over time (Read 2642 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

Problem: installed plugins limit decreases over time

2015-09-26 09:39:33

HIYAS

I have problem! With almost every update of 3rd party plugins, there apppear additional amount of not loaded plugins.
Note: I don't add new plugins, just update existing.
To date I can count in foobar dir only 109 plugins and 4 are refused on start, totals 105 acceptable plugins now.
I have read that foobar can load up to 127 plugins (this amount formerly was accepted)
Where's the problem that foobar refuses to load still more plugins?

Problem: installed plugins limit decreases over time

Reply #1 – 2015-09-26 20:14:39

I'll be that guy and ask. How many of those plugins do you actually use?

Problem: installed plugins limit decreases over time

Reply #2 – 2015-09-26 22:22:42

Quote from: Anakunda on 2015-09-26 09:39:33

Where's the problem that foobar refuses to load still more plugins?

Windows itself is this problem. http://www.hydrogenaud.io/forums/index.php?showtopic=103210, https://social.msdn.microsoft.com/Forums/en...ng-loadlibrary-
And there was no such problem on XP.

Problem: installed plugins limit decreases over time

Reply #3 – 2015-09-26 23:27:24

Your post makes it sound as if Microsoft broke something after Windows XP. However this is wrong. The truth is not as simple as that. It is rather a combination of several factors:

Microsoft added a new feature after Windows XP (fibers). This feature comes with a resource limit (128 FLS entries, FLS=fiber local storage).
The C/C++ runtime library supports this new feature on Windows versions where it is available. However each instance of the runtime library requires FLS entry.
Each DLL which is compiled with the static runtime library has its own instance of the runtime library. All DLLs which are compiled with the DLL runtime library share one instance of the runtime library.
The recommended approach for foobar2000 components is to use the static runtime library because this allows an easy installation without requiring a separate installer for each component.
There are a lot more components available than when Windows XP was still the dominant version of Windows.

Problem: installed plugins limit decreases over time

Reply #4 – 2015-09-27 02:50:52

Quote from: Daeron on 2015-09-26 20:14:39

I'll be that guy and ask. How many of those plugins do you actually use?

Oh ! But that doesn't solve the issue at all.

Problem: installed plugins limit decreases over time

Reply #5 – 2015-09-27 02:55:19

Quote from: Anakunda on 2015-09-27 02:50:52

Quote from: Daeron on 2015-09-26 20:14:39
I'll be that guy and ask. How many of those plugins do you actually use?

Oh ! But that doesn't solve the issue at all.

Actually it does and Daeron probably was trying to make a point. Why would you need that many plugins working?

Problem: installed plugins limit decreases over time

Reply #6 – 2015-09-27 03:29:14

@Rollin:
Does that mean updated plugins use more FLS entries than old versions?

@jazzthieve:
Asking for that doesn't lead to answer . Being that limitation lower you would face that issue with more "reasonable" amount of plugins.

Problem: installed plugins limit decreases over time

Reply #7 – 2015-09-27 05:30:28

Quote from: Anakunda on 2015-09-27 03:29:14

Does that mean updated plugins use more FLS entries than old versions?

Maybe: http://blogs.msdn.com/b/vcblog/archive/201...spx?PageIndex=2

Quote

Additionally, each copy of the CRT has its own state, including one or two FLS slots and per-thread allocations that store various runtime state (e.g., errno).

Problem: installed plugins limit decreases over time

Reply #8 – 2015-09-27 09:40:35

Quote from: lvqcl on 2015-09-27 05:30:28

Quote from: Anakunda on 2015-09-27 03:29:14
Does that mean updated plugins use more FLS entries than old versions?

Maybe: http://blogs.msdn.com/b/vcblog/archive/201...spx?PageIndex=2
Quote
Additionally, each copy of the CRT has its own state, including one or two FLS slots and per-thread allocations that store various runtime state (e.g., errno).

Indeed, one for Universal CRT, one for VCRuntime. Each of them seems to require it's own FLS slot.
Moreover, the behavior of the failure might have changed. Previously, CRT startup routine failed and plug-in DLL just didn't load. Now, the failure seems to be delayed. So, plugin might load but fb2k itself might be killed later (by abort() function). Haven't actually tested, though. I've just looked at the CRT source.

Problem: installed plugins limit decreases over time

Reply #9 – 2015-09-27 11:50:02

Quote from: foosion on 2015-09-26 23:27:24

There are a lot more components available than when Windows XP was still the dominant version of Windows.

Yes. But on XP I could run foobar2000 with 130+ plugins loaded with no problem. And I switched from XP to 7 only in october 2013 ) And faced this problem the same day I switched to 7.

Problem: installed plugins limit decreases over time

Reply #10 – 2015-09-27 12:15:21

^perhaps you could download the foobar2000 SDK, fire up visual studio and incorporate all the features you need into 1 plugin? or 50 or 100 if that still works...

Problem: installed plugins limit decreases over time

Reply #11 – 2015-09-27 13:45:14

Thanks folkz for explaining the issue. The answer is then to not update plugins, quite disappointing.

Problem: installed plugins limit decreases over time

Reply #12 – 2015-09-27 14:02:05

Yes as often updates come with additional dlls loaded for each plugin (newer VCRT, dll for emulating certain DSP architecture inside plugin, etc). You have to remember, that some plugins consist of more than one dll, which also counts towards the limit.

Problem: installed plugins limit decreases over time

Reply #13 – 2015-09-27 15:28:16

Quote from: EpicForever on 2015-09-27 14:02:05

Yes as often updates come with additional dlls loaded for each plugin

As far as I understand a single DLL plugin requires 2 FLS slots if compiled with MSVC 2015 (and only one slot if compiled with previous versions of MSVC).

Quote from: Anakunda on 2015-09-27 13:45:14

Thanks folkz for explaining the issue. The answer is then to not update plugins, quite disappointing.

There are several other solutions:
1) stop keeping unused plugins
2) ask fb2k developer to redistribute MSVC CRT files, and ask plugin developers to use dynamic CRT linking
3) when plugin sources are available: compile them yourself with dynamic CRT linking
4) downgrade to Windows XP
5) another possibility (mentioned presumably by the author of foo_ui_hacks): open a DLL in a hex editor, find "FlsAlloc" string and change it to a name of a non-existing function.

edit: fix numbers out of order

Problem: installed plugins limit decreases over time

Reply #14 – 2015-09-27 21:21:43

Quote from: lvqcl on 2015-09-27 15:28:16

5) ask fb2k developer to redistribute MSVC CRT files, and ask plugin developers to use dynamic CRT linking

...and the same version of the compiler/runtime library as fb2k itself. Nevertheless this would be the preferred solution for me. Even if only half of the available components switched to this it would (almost) double the number of components you could load.

Quote from: lvqcl on 2015-09-27 15:28:16

4) ... open a DLL in a hex editor, find "FlsAlloc" string and change it to a name of a non-existing function.

I don't know the details but that approach is probably only safe as long as neither fb2k nor any of the loaded DLLs uses fibers. Although it is probably safe enough for all practical purposes.

Problem: installed plugins limit decreases over time

Reply #15 – 2015-09-27 23:23:21

Quote from: lvqcl on 2015-09-27 15:28:16

5) another possibility (mentioned presumably by the author of foo_ui_hacks): open a DLL in a hex editor, find "FlsAlloc" string and change it to a name of a non-existing function.

As a component developer, I will personally hunt down and murderinate anyone that does this to any of my components. The crash reports are already full of broken computers and deployments. I don't need binary modifications to further taint things.

You cannot realistically as a third party determine if a component uses functionality that requires FLS, nor can you know that future updates of the runtime will not change this assumption.

The proper long-term solution to these problems is plugin containers for isolation and scaling; until then, curate your set of available plugins.

Problem: installed plugins limit decreases over time

Reply #16 – 2015-09-27 23:31:15

Quote from: Anakunda on 2015-09-27 13:45:14

Thanks folkz for explaining the issue. The answer is then to not update plugins, quite disappointing.

There's a special circle in hell reserved for people who encourage others to silently avoid updates and cause more support and maintenance problems down the line.

Problem: installed plugins limit decreases over time

Reply #17 – 2015-09-28 00:18:30

Quote from: Zao on 2015-09-27 23:23:21

The proper long-term solution to these problems is plugin containers for isolation and scaling

100 fb2k processes for 100 plugin DLLs? IPC instead of virtual function calls? Not sure that it's a good idea.

Problem: installed plugins limit decreases over time

Reply #18 – 2015-09-28 01:37:02

Taken to that extreme, yes. However nothing much stops you from sharing a host, like Windows services do and Firefox plugin-container.exe.
You would distribute relevant parts of the fb2k API into the processes at suitable boundaries.

I don't expect existing components to take well to being segregated, but if there was a way to volunteer components to use a limited or full API for separation in the future, I'd sure consider authoring separable components. Some types of components are prime candidates for separation like inputs, DSPs and outputs.

The thing is, current-day foobar2000 architecture is butting up against several limits to in-process hosting. Real-life configurations run out of address space, we run into OS limitations on handle counts and exhaust scarce resources like FLS slots.

Component authors have to not only deal with their own problems, but with the problems inherent in sharing a process with other parties. You cannot as a component reliably spawn processes with inheritable handles, as that requires full control over all inheritable handles in the process. You can also not mess with other global state, even on a per-thread basis as you don't own much at all.

All this reflects in the crash reports we get. Most of the crashes I have are impossible to debug due to the wide variety and interaction of components inside the process.

I don't know what design would better serve the community in the future, but I know it's getting rather tiresome having someone reorganize your sock drawer whenever you look away.