Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Collection folder monitoring does not work over samba (Read 18306 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Collection folder monitoring does not work over samba

Reply #25
I have just upgraded Samba on the Debian side : from 3.0.4 to 3.3.4 (April 29, 2009).
FYI there is also a 3.4.0 pre2 version, but as you may figure it's still in early beta stage.

I'll report any improvements on my next addition to my foobar library.

Collection folder monitoring does not work over samba

Reply #26
You're completely welcome to go and do it if you want these bugs fixed. None of the developers care enough about Samba to spend their time on this. There are plenty of records of things not working here on the forums, so go right ahead and link them to various threads.

Collection folder monitoring does not work over samba

Reply #27
When I was looking at this, the problem seemed to be generally with Linux kernel, particularly its filesystem change notification API inotify.

On Windows, since some old NT version, you can ask to be notified of any changes in a directory, or whole directory sub-tree. foobar2000 uses the latter feature to a great extent, because a Media Library directory naturally needs to be watched including all its subdirectories.

inotify works only per-inode basis, which means only for one directory. Leaving all subdirectories opened to watch them would create big overhead and at least wine developers didn't want to go that way. I don't know about Samba, but I guess the emulation works also only partially, because of the performance reasons.
Full-quoting makes you scroll past the same junk over and over.

Collection folder monitoring does not work over samba

Reply #28
@Canar : do you really have to behave in such a prepotent way ? I don't think it's necessary here, plus it's totally useless.

Besides, what do you have against Samba ? For me things are very simple :
- If you know a better way to communicate between a Windows machine and a Linux machine, you are more than welcome to share it.
- If not, you are more than welcome to develop it.
- And if you're not up to the challenge, then it looks a bit "easy" for me to say that Samba doesn't even deserve the attention of any of the developers here, just as if it was a "lesser thing".

May I remind you that Samba is open source / free software ? So everybody here, including you Canar, is welcome to help it become better.
More info is available at http://www.samba.org

@Yirkha : if you say "for performance reasons", that would mean it's not a bug but an intended behaviour. Correct me if I'm wrong, but in that case we couldn't put the blame on Samba/Linux anymore, could we ?
If inotify works only per-inode basis for performance reasons, then maybe (I say maybe) foobar should take that into consideration in order to preserve some compatibility.

Anyway, I'm willing to spend some time on this issue, but I NEED TO KNOW if the problem comes from Samba, from Linux, or from somewhere else. I'm not an expert, I just want to sort this out. Thanks to those who are willing to help in a constructive way.

Here is some tech-info about inotify :
http://en.wikipedia.org/wiki/Inotify (Yirkha, are you sure you are not talking about dnotify instead of inotify ?)
http://inotify-tools.sourceforge.net/

Collection folder monitoring does not work over samba

Reply #29
if you say "for performance reasons", that would mean it's not a bug but an intended behaviour. Correct me if I'm wrong, but in that case we couldn't put the blame on Samba/Linux anymore, could we ?
Windows platform - whole sub-tree folder watching works.
Linux platform - whole sub-tree folder watching does not work, therefore it's a lesser thing, "why?" is irrelevant.

As you said, everyone is free to contribute to make the emulation better.
Don't expect that from people who don't care, those who are affected by it should take that challenge you speak of.
Full-quoting makes you scroll past the same junk over and over.

Collection folder monitoring does not work over samba

Reply #30
Yirkha, in the link I provided in my previous post ( http://en.wikipedia.org/wiki/Inotify ) it is said that "Inotify has many advantages over dnotify, the module that it replaced. With the older module, a program had to use one file descriptor for each directory that it was monitoring. This can become a bottleneck since the limit of file descriptors per process could be reached. The use of file descriptors along with dnotify also proved to be a problem when using removable media. Devices could not be unmounted since file descriptors kept the resource busy".

Wouldn't that mean that inotify doesn't work on a per-directory basis, but rather on a sub-branch basis ?

Collection folder monitoring does not work over samba

Reply #31
I just had a quick look at samba code and ReadChangeDirectoryW() is somewhat supported through inotify,  but it does not seem to register
recursively the watched directory for changes (I could be wrong here), which is against the ReadChangeDirectoryW() api spec.
For it to notify something at all, samba must be compiled with inotify support (should be the case in all binary package) and smb.conf must not contain 'kernel change notify = no'

OT: Contrary to popular belief, wine *do* watches recursively a directory in its ReadChangeDirectoryW implementation, but this implementation is semi-broken (or half-working if you prefer):

- move and rename change notifications are not done at all
- It doesn't do multiple change notifications returned in a single call to  ReadChangeDirectoryW () or GetOverlappedResult().

I've actually done a patch that fixes both issues. I may resubmit that pacth to wine, but I have little hope it will be accepted (as it is more than a trivial 10 lines fix).

Collection folder monitoring does not work over samba

Reply #32
I've got a quick reply from John McCutchan, Inotify's main creator !

"Hi,

Yes, that is correct. Inotify does not support recursive directory monitoring. You must write this manually, the algorithm is trivial. Having many Inotify watches is okay.

John
"

I hope this can be useful...

Collection folder monitoring does not work over samba

Reply #33
That's what wine do. Now you need to convince samba developpers to do the same .

Moderation: Removed useless full quote of the preceding post.

Collection folder monitoring does not work over samba

Reply #34
...or, since Samba is open-source, any willing developer could go and do it (create a "foobar-compliant" version of Samba). Could you do that bubbleguuum ?

I guess that's one avantage of open-source : you don't necessarily have to "convince" any developer to get what you want / need, if you can do it yourself of course.

Collection folder monitoring does not work over samba

Reply #35
foorious, if you're willing to pay me to spend my time on something I consider completely extraneous I might consider doing it. Until then, I'll recommend you use Windows File Sharing, which is what defines the standard.

I'm not sure precisely what part of my previous post you find so prepotent. Apparently, stating the obvious is considered prepotent by you.

Collection folder monitoring does not work over samba

Reply #36
Like I said, I won't comment further on that matter.

Canar, since our "Problem with Media Library and Samba share" is of no interest to you, I hope you'll agree that we keep on discussing it without you, so you can spend your precious time on things that look more important to you.

Sorry for the OT and thank you for your help anyway.


@Everyone else : it looks like we're moving towards the solution, which is a good thing. Now we know where the "problem" resides and how to solve it.

Collection folder monitoring does not work over samba

Reply #37
"Inotify has many advantages over dnotify, the module that it replaced. (...)".

Wouldn't that mean that inotify doesn't work on a per-directory basis, but rather on a sub-branch basis ?
dnotify works with file descriptors ("opened files"), inotify works with inodes ("files at the filesystem level").
I thought having a few thousands of inodes watched would be a performance hit anyway, however if the developer doesn't think so, I'm happy to be wrong.

@Everyone else : it looks like we're moving towards the solution, which is a good thing. Now we know where the "problem" resides and how to solve it.
Now we know exactly as much as we did a year ago when 0.9.6 was being developed. Let's celebrate...
Full-quoting makes you scroll past the same junk over and over.

 

Collection folder monitoring does not work over samba

Reply #38
You are right Yirkha : let's celebrate. I have some good news to report, but they need to be confirmed.

My Samba upgrade on the Linux site seems to have improved, and maybe even corrected the problem !!

For the moment I have added two whole albums [EDIT - four whole albums !] to my library (File ops > Move). Both albums were added to the library almost in real time, with no missing tracks. All relevant autoplaylists were also updated. No need to restart foobar anymore. It just seems to work !

Here's what I have done since yesterday :
- Updated Samba to 3.3.4 ( http://news.samba.org/releases/3.3.4 ),
- Deleted all monitored folders within foobar 0.9.6.8 prefs,
- Added all monitored folders again, so they could be rescanned properly with the new Samba version.

Voilà.

Of course these preliminary results don't mean nothing, and they NEED to be confirmed. So all other Linux users out there, please upgrade Samba to 3.3.4 and report results. Thank you.

Collection folder monitoring does not work over samba

Reply #39
Maybe it's not so surprising considering inotify support has been apparently added in samba 3.0.25 and you've been using version 3.0.4
Full-quoting makes you scroll past the same junk over and over.

Collection folder monitoring does not work over samba

Reply #40
It was a type error : my previous version was 3.0.24. But you are right, it's still inferior to 25.

Collection folder monitoring does not work over samba

Reply #41
So all other Linux users out there, please upgrade Samba to 3.3.4 and report results. Thank you.
Good luck with this. See my above post.
elevatorladylevitateme

Collection folder monitoring does not work over samba

Reply #42
So the problem with recursive directory watching is virtually nonexistent (any reasonably fresh release of Samba with inotify support enabled implements it good enough to work with foobar's media library).

Still it does not solve my original problem (rescan after reconnect).
What exactly tells foobar that all directories in reconnected share are "new"? And why they are not "new" anymore after foobar restart?

Collection folder monitoring does not work over samba

Reply #43
I've got this same problem too and am still seeing it with 9.6.7 - is this even an acknowledged bug?

Collection folder monitoring does not work over samba

Reply #44
*more samba threads merged*

Unkosibomvu: See Post #39
elevatorladylevitateme

Collection folder monitoring does not work over samba

Reply #45
I guess that's one avantage of open-source : you don't necessarily have to "convince" any developer to get what you want / need, if you can do it yourself of course.

True, true. You want/need this feature, you can do it, provided you can do it yourself, of course. 

I don't think I need this feature, so... I don't have to concern myself with Samba code, open-source or not. 

Collection folder monitoring does not work over samba

Reply #46
i'm running a debian fileserver with samba shares and have windows vista clients. these clients mount the share as a network drive. i never had any problem with foobar monitoring. it notices any changes within seconds and my library is rather large.

// i have not read the whole thread but i can provide my config files if the problem still exists and no solution has been found so far

Collection folder monitoring does not work over samba

Reply #47
Xenion, the solution for foobar monitoring and Samba has already been found : Samba version needs to be superior or equal to 3.0.25.

Only Messer's "rescan after reconnect" problem doesn't have a solution so far.

Collection folder monitoring does not work over samba

Reply #48
My first post here, let me just say thanks to all the foobar2000 core and component devs for an excellent piece of software.

I also have the Samba monitoring issue, and I've read this thread along with anything I've been able to find on Google.

These are my specs:
Server - Debian Lenny, Samba version 3.2.5 - updated every morning using apt-get
Client - Windows XP sp3, all current updates
foobar2000:
Core (2009-08-22 06:25:22) foobar2000 core 0.9.6.9
foo_abx.dll (2009-06-07 08:25:26) ABX Comparator 1.3.4
foo_albumlist.dll (2009-08-22 06:23:44) Album List 4.3.1
foo_audioscrobbler.dll (2009-08-20 22:15:56) Audioscrobbler 1.4.2
foo_cdda.dll (2009-08-22 06:23:34)CD Audio Decoder 2.1.4
foo_converter.dll (2009-08-22 06:23:26) Converter 1.2.1
foo_dsp_std.dll (2009-08-22 06:23:48) Standard DSP Array 1.0
foo_fileops.dll (2009-08-22 06:22:36) File Operations 2.1.2
foo_freedb2.dll (2009-08-22 06:22:52) freedb Tagger 0.6.1
foo_input_std.dll (2009-08-22 06:23:28) Standard Input Array 1.0
foo_rgscan.dll (2009-08-22 06:23:20) ReplayGain Scanner 2.0.9
foo_ui_std.dll (2009-08-22 06:23:54) Default User Interface 0.9.5
foo_unpack.dll (2009-08-22 06:22:20)RAR reader 1.2 ZIP/GZIP reader 1.0

The samba share is mounted as a drive letter, the M: drive is a 300gb IDE drive on my server dedicated solely to music.

Ever since foobar2000 changed to this new monitoring scheme, any changes on the server are not reflected in the library. The only way to get the library changes to take effect are a) restart samba, b) restart foobar2000.

Honestly, for me, it's not a such a big issue but I signed up and posted just to say that this issue still persists and I'm willing to try out anything that is suggested, other than move my music collection from the safety of an ext3 journaling file system.

Thanks.