Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Auto-Rating (variables) component (Read 15541 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Auto-Rating (variables) component

As Peter, foosion et al have made abundantly clear many times:

1) Title formatting is not suitable for calculations
2) System date/time messes with foobar's UI

What is clear is that to create an auto-rating scheme, the relations between date (added, first played etc.) and some form of play-count are required. The most accurate play-count is: "total seconds played" / "track length in seconds".

So, if there was a little man with a stop watch in one hand and a clock in the other, and all he did was watch foobar2k's header, and every time it changed he noted the filename/path in a notebook and hit his stopwatch. That would record the perfect play-count and would simultaneously get rid of the whole %skip% issue.

So let's say this little man is a component, and his note book is an SQL Database.
In the DB the component records the following info:

1) Filename/Path
2) Track Length (in secs)
3) Total Recorded Played Time (in secs)  - (keeping a running total)
4) Current Time

The component looks up the following info from the official Playback Statistics component.

- Added
- First Played
- Last Played

And calculates the following:

- Number of Days Since Added
- Number of Days First Played
- Number of Days Last Played

And stores all these values in the DB.

NOTE:
1) Any system time function is occuring completely outside foobar's core.
2) No %title_formatting% is being used

Then, either the same or a complimentary component (perhaps with a calculating script??) takes the following numbers:

Track Length (secs)
Running Total Recorded Played Time (secs)
Number of Days Since Added (days)
Number of Days First Played (days)
Number of Days Last Played (days)

And does some calculation (which is open to scripting input by users???):

The component outputs the result as a meta-data value in the form of %title_formatting_result% which can then be picked up by the foobar2k core as any other piece of meta-data. A further discussion is how often the component would offer up the data to foobar (hourly, daily, on startup etc ...).

If this was possible it would mean that (AFAIK) topdownjimmy's hotness algorithm, foo_DAR, and also (if he gave permission) big_berny's Media Monkey AutoRateAccurate  ratings schemes (as well as others, no doubt) would be all potentially be available to the foobar user in one component.

So with possibly a single component, users would be able to select from a range of auto-rating schemes and, in addition, be able to tune the ratings algorithms to their satisfaction [AFAIK all the above allow (or plan to allow) tuning, so users could easily (with sliders) select the weightings of key variables].

Questions:

1) Is this possible?
2) Would this avoid the system date-time repaint glitch issues Peter discussed?
3) Could it be done in such a way that avoids really annoying the foobar2k devs?

C.
PC = TAK + LossyWAV  ::  Portable = Opus (130)

Auto-Rating (variables) component

Reply #1
I can't believe such a well written and sensible post has no replies.

I don't know much about programming, and the f2k core. In fact, I know nothing. What I do know is that carpman's code for autorating (fooDAR) is amazing, and gives foobar a really useful feature, to know the popularity of files within your personal library. Since I adapted it to my foobar, I've re-discovered music I had forgotten, I've seen how obsessed I get with some songs, I've spent many time looking at the numbers. :P what can I say.. I love statistics. and I love them even more if they make sense, and are not just a simple play count.

And now that I had got used to it, I decided to switch to the last version only to find out cwb-hooks (needed for DAR to work) is not supported. I guess I should have checked before...  :(

Anyway, as I said I'm pretty illiterate in computer programming, but it kind of surprises me that such a piece of software as foobar is doesn't support system date/time.

I would like to support carpman on this ideas (specially as he seems to have it all thought about), because I think we can all agree that the support of this has obvouis possibilities for rating schemes and maybe other kind of components.

:)

Auto-Rating (variables) component

Reply #2
sune,

Thanks for the kind words re. foo_DAR. I was a little surprised about the lack of response (I dropped another bomb recently in the Playback Statistics component thread which met the same fate, so I seem to have the gift). Primarily I was hoping for an answer to a pretty straight forward technical question, though one which would require knowledge of foobar's inner workings.

I find it funny that programmers don't seem interested in ratings, and those who write ratings formulas/algorithms don't appear to be programmers. Such is life. 

Cheers,

C.
PC = TAK + LossyWAV  ::  Portable = Opus (130)

Auto-Rating (variables) component

Reply #3
Note: I'm not a developer, and although I do have some development knowledge, I'm nowhere near familiar enough with the foobar2000 API to know exactly what's going on here. I'll try explain things to the best of my understanding.

The issue, as far as I can discern, is that the function that describes the rating would change every time the date/time changes, which is a continuous process.

The reason that date/time stuff is so resisted is simply because it is hard to do right. It would require a massive overhaul of the underlying APIs, and the cost isn't worth the gain.

Extending title formatting is not a bad idea, but one of the underlying assumptions made in places is that the contents of these variables is constant until the file being referred to is modified. If that's true, perhaps your idea could be implemented as some kind of tagging extension. Click on an entry in the main menu, and the rater iterates through the entire media library, changing the tags to be most up-to-date based on your rating scheme.

Does that make sense?

Auto-Rating (variables) component

Reply #4
Couldn't this currently be implemented with autoplaylist Queries?

If say you wanted to autorate on a 5 point scale, it would involve 5 autoplaylists.
A little clunky maybe, but at least do-able.
elevatorladylevitateme

Auto-Rating (variables) component

Reply #5
The issue, as far as I can discern, is that the function that describes the rating would change every time the date/time changes, which is a continuous process.

The reason that date/time stuff is so resisted is simply because it is hard to do right. It would require a massive overhaul of the underlying APIs, and the cost isn't worth the gain.

Extending title formatting is not a bad idea, but one of the underlying assumptions made in places is that the contents of these variables is constant until the file being referred to is modified. If that's true, perhaps your idea could be implemented as some kind of tagging extension. Click on an entry in the main menu, and the rater iterates through the entire media library, changing the tags to be most up-to-date based on your rating scheme.

Thanks Canar for the reply.

I understand the time/update issue.
But clearly fb2k has no problem reading from a DB and everytime you play a track the playcount changes and fb2k can deal with such changes.

What I'm suggesting is pretty much the same, except that the component would offer up a %calculation% for fb2k to read, and the frequency of updates from the database is something that could be negotiated to avoid the core problem which is:

Potentially with a time-based autorating scheme, every second which passes can mean a change in rating.
Clearly this is not desirable, but my point was that a "off-site" component could manage such changes, and only offer up %results% to fb2k as and when fb2k is happy to manage such an influx of new data. Rather like "update media library now" - let's face it, fb2k hardly craps itself when dealing with such updates.

Wouldn't this bypass the re-paint problems that Peter was talking about?


[EDIT]
An alternative way around this is to include a form of system time which = %system day%.
Date-based autoratings schemes like "hotness" and foo_DAR only need system day. Would that solve the constant update problem?

As far as such dynamic rating schemes embedding their result in each file's tags - I don't like that idea, as:
a) DB's are meant for constant read/write access,whereas file tags are not (tags are suited to occasional write; constant read)
b) foo_DAR never needs to be rendered as a static number in a tag, as really it's just a calculation based on other variables - those other variables are often better stored in a DB for the read/write issues mentioned above.
[/EDIT]

@ shakey_snake -- I'm afraid I didn't understand this. Sorry.

C.
PC = TAK + LossyWAV  ::  Portable = Opus (130)

Auto-Rating (variables) component

Reply #6
But clearly fb2k has no problem reading from a DB and everytime you play a track the playcount changes and fb2k can deal with such changes.
Yeah, because in case of %title%, %rating% or %isplaying%, the part of fb2k which changed that field can send a change notification to just one (or several tracks). With time based formula, you either issue 10000+ such notifications each second for great performance or suffer from the repaint problems.

What I'm suggesting is pretty much the same, except that the component would offer up a %calculation% for fb2k to read, and the frequency of updates from the database is something that could be negotiated to avoid the core problem. [...] Wouldn't this bypass the re-paint problems that Peter was talking about?
Yes, you understand the problem well. Such a solution would work - it could be either a manual "Recalculate" command, it could happen automatically "when computer is idle" in the background, ...

An alternative way around this is to include a form of system time which = %system day%. Date-based autoratings schemes like "hotness" and foo_DAR only need system day. Would that solve the constant update problem?
Partialy, but it's a bad design anyway. Because you couldn't link %system day% to a particular track(s), the refresh command would need to be send about all 10000+ tracks each midnight, even for people who don't care at all.
Full-quoting makes you scroll past the same junk over and over.

Auto-Rating (variables) component

Reply #7
Partialy, but it's a bad design anyway. Because you couldn't link %system day% to a particular track(s), the refresh command would need to be send about all 10000+ tracks each midnight, even for people who don't care at all.
Might a solution be a %startup time/day%? It would be constant through the foobar instance solving the non-repeatability problem, but you would have to restart foobar to update the field. A cross of this and the "recalculate" idea could become a full solution.

Auto-Rating (variables) component

Reply #8
Might a solution be a %startup time/day%? It would be constant through the foobar instance solving the non-repeatability problem, but you would have to restart foobar to update the field. A cross of this and the "recalculate" idea could become a full solution.

%foobar startup day% or %foobar initiation/instance day% would be fine. If you did as you suggested no one would expect the field to update without restarting foobar - to that's not a problem and quite a nice solution. That way, as you say, we're no longer dealing with a moveable %system time%, but rather a static %foobar instance day%.

It's worth remembering that the sole point of this (as far as auto-rating schemes go) is to calculate the following:

Number of Days Since Added
Number of Days Since First Played
Number of Days Since Last Played

With outputs as integers.

Thus if Playback Statistics registered the startup or %foobar initiation/instance day% (i.e. current day)
Then surely it wouldn't be too hard to get the above numbers.

Then the only problem that would need to be resolved is what Yirkha pointed to:
Such a solution would work - it could be either a manual "Recalculate" command, it could happen automatically "when computer is idle" in the background, ...

I agree with Yirkha, that BOTH a background "when computer is idle" scan and also a manual update scan would be good options.

The only other thing that I'd like to add at this point is the need for a more accurate playcount.

Below is my suggestion [EDITED]:
a) record "Total Played Time per Track", or
b) allow an option whereby Playback Statistics offers an additional (alternative) playcount based on:
Total Played Time per Track / Track Length

Currently, if I play the first 1 min of a 10 min song 10 times, foo_playcount says I've played it 10 times, whereas "total played time" / "track length" would say I've played it once. Though this is not entirely accurate, it's far, far closer to the truth.

Benefits:
a) Eliminates the need for skip counts.
b) It would not affect those users who always play their files all the way through, but it would beneficially affect those users who skip from track to track and still require an accurate reading of how often they've listened to each track.
c) Eliminates the need for any "count if played when played x %" or "count if played when played > 2 mins" kind of options.


What's hard to figure is how much of the job can be done with the Playback Statistics component, and how much needs to be done with a standalone component.

C.
PC = TAK + LossyWAV  ::  Portable = Opus (130)

 

Auto-Rating (variables) component

Reply #9
The most accurate play-count is: "totalseconds played" / "track length in seconds".


I suggest "totalseconds played" (e.g. some Playback statistics database field) would update on song change. When you seek, it'll save seconds played so far and position after seek. When you seek again or song changes, you would do "current position"-"saved position" and add this to seconds played so far and update field in database. It shouldn't be hard to implement (I'm not C++ developer so I leave this to devs or somebody else).


Regarding other things suggested here, I mostly agree. But autoplaylists updates themselfs (evaluating query), so it should be problem with that.

Auto-Rating (variables) component

Reply #10
Hey I really like this topic. At the moment I don't update to new foobar versions because I need cwb_hooks for those things: (systemdate, -time, datediff and random)

1. displaying some kind of auto-rating (I've my own formular)
2. sorting the playlist by (1) with addition of a random factor and rating
3. displaying the time since last played (in days, weeks or months depending on the value)
4. highliting tracks I played in the last 24 hours

so if you find a solution it would be nice if all of that would be possible...

I don't really understand why cwb_hooks should be so bad that we're not allowed to use it anymore, but I think it's because of these per second updates.





I like the idea with an extern database. I've thought about it a bit and have some ideas:
1. we need an extern database (sql or whatever)
2. we need a component which can update foobars database with the extern database. this could happen on the start of foobar, on track change or through a user action like a menu item "update virtual tags"
3. we could have components or even extern applications that could change the data in the database whatever they like

I would prefer if the database could not only save tags for files using its path, but could store those tags for files with some kind of fingerprint, so that you could rename your files without being afraid to loose data like playcounts, or you would even be able to share your database between different pcs without having the same file structure or format for your music.

then we would not only be able to have all kind of auto-rating or similar algorithms not written in title formatting, you could then use every programming language you prefer to change the virtual tags in the database. I for example would like to write an application in ruby, which could download last.fm charts and tags and save them in the database and with that you could built very intelligent playlists in foobar. Maybe someone else would write an application or component for storing lyrics, wikipedia articels, album art, rating, mood-tags and all other information without chaning the music files. you see you could do everything with it;)

Auto-Rating (variables) component

Reply #11
The most accurate play-count is: "totalseconds played" / "track length in seconds".


I suggest "totalseconds played" (e.g. some Playback statistics database field) would update on song change. When you seek, it'll save seconds played so far and position after seek. When you seek again or song changes, you would do "current position"-"saved position" and add this to seconds played so far and update field in database. It shouldn't be hard to implement (I'm not C++ developer so I leave this to devs or somebody else).

Couldn't this be simplified to:

A "PLAY SESSION" is defined by 2 factors:

1) PLAY being active
2) FILENAME being played remaining the same

Where these 2 factors remain unchanged count the number of seconds, and that yields a FILENAME's "PLAY SESSION TIME". 

So the component would count the file's "PLAY SESSION TIME" and when this is over as defined by either:
a) PLAY becoming inactive or,
b) another FILENAME becoming active / played
It would add "PLAY SESSION TIME" in seconds to "FILE'S TOTAL PLAYED TIME in SECONDS" and store this value in its database.

The only question I have about this is:

How does foobar treat seeking. Does foobar treat this is a STOP and a RESTART, if so this would simply mean that you'd have lots of small "PLAY SESSION TIMES" being written to the DB. I don't see this should be a problem.

Then, of course one could do:

"FILE'S TOTAL PLAYED TIME in SECONDS" / %length_seconds%

and then we get a decent play count.

C.
PC = TAK + LossyWAV  ::  Portable = Opus (130)

Auto-Rating (variables) component

Reply #12
Okay, how about this as a work-around for all the lovers of 0.9.5.3+.
WARNING: This quasi "solution" is not going to please the purists, but perhaps it might be so damn ugly that it can prompt a few tech-types to provide a real solution (see the ideas floating around this thread).

Anyway, here goes:

foo_run component runs a script on fb2k start-up or tied to a hot-key, that alters the date modified header of all the files in the user's audio directory(s). This is preferable to actually altering the files (and much faster).

I can then use TheQwerty's date difference solution, posted here.

And use %last_modified% as the static system date time against which to calculate all the relevant date differences.

Then I would strongly recommend people try to encourage Peter et al to improve the playcount component because at present, compared to the rest of foobar2000, which is seriously impressive, the playcount method employed is rather medieval IMHO. See here for details.

Any thoughts welcome.

By the way, is it possible to hack cwb_hooks and for someone to provide a simple plugin that can provide cwb_datediff. I have zero idea about the IP rights / license situation here.

C.

[EDIT: typo]
PC = TAK + LossyWAV  ::  Portable = Opus (130)

Auto-Rating (variables) component

Reply #13
Hi carpman. I've been away from the forums for a while and I had totally missed this post.

Since I'm not in the least a purist  and I've missed your rating system since I upgraded my foobar, I'd love that you'd implement this workaround to make it work with  0.9.5.3 version or older. I hope you haven't completely forgot about this, being this post a few months old

Apart from this, like I've already told you other times, I fully support your requests to the developers regarding system time and date, to finally get that amazing rating system of yours to work properly. It's really too bad your requests don't get all the attention they deserve.

sune

Auto-Rating (variables) component

Reply #14
Hi Sune, good to hear from you.

I'd be a little wary of that "work-around" - it was a theoretical-only work-around and it may not "work around" the problem at all.  For a more up-to-date status of fooDAR and fb2k 0.9.5.3+ see here.

As I said in that post, I've provided solutions to how this can be done (which are linked to in the above post), but I'm not a programmer, if people are really interested then it can be done, but I also think one of the major obstacles is the current official playback statistics component, which IMO is poor (I do not consider a 30 min piece of music "played" after it has been playing for 1 min!).

That said, I'll keep an open mind. I don't believe that the foobar developers are against ratings per se, it's clearly just very very low down on their list of priorities.  I think the most useful suggestion I've made are for these per session fields. If these were implimented and the foobar playback stats component allowed users to set a % played then I'd  have fooDAR up and running in the latest versions.

That would be nice, but to be honest I'm very happy with 0.9.4.3 which works perfectly AND I can still scan folders manually! 

C.

[EDIT: FYI I'm still working on fooDAR and so if/when it is compatible with the latest version it will at least be a little better than before. The penalty from the delay in first playing a song now reduces exponentially as a function of the playcount, which is a nice improvement.]

PC = TAK + LossyWAV  ::  Portable = Opus (130)