Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: [fb2k v2] SQLite Utilities (foo_sqlite) (Read 68418 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #150
I tried to insert a record with the SQLite console, but it didn't work.
Of course, not. The lookup tables are not meant to be filled by the user. There is a reason why the tags database is not exposed to the user. The values in these table require a specific format.
OK, I think I'm starting to grasp the logic of the component. I was expecting the lookup value tag to be a virtual tag, but if I understand correctly, the only way to insert values in any field of a lookup table is by copying it from an actual file tag. An actual tag can hold inconsistent values, tough. I made some tests and I've seen that once a lookup value is copied in the SQLite db, it is not overwritten by subsequent attempts to copy a different value. Nevertheless, the actual tag has precedence over the SQLite tag, so how can I grant referential integrity in the properties dialog? Also, how do I modify an already existing lookup value in the lookup table?
I'm late

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #151
I was expecting the lookup value tag to be a virtual tag,
As long as you don't activate "Forward read, if empty" and "Forward write" it is. However, for existing tagged files, the file tag values are cached and won't be neither automatically changed by just defining a SQLite tag for them nor the values will be written to the database. And as long as there is no entry for a file in the database, also no lookup values will be used.

but if I understand correctly, the only way to insert values in any field of a lookup table is by copying it from an actual file tag.
No, this is only one possibility. You could also for example just enter a value in the properties dialog or any other means, which let you write tag, e.g. masstagger scripts, using updatable virtual tables (MediaLibraryUpdatable, PlaylistUpdatable), etc. But for initializing the tag database it is probably the best way just to copy the actual file tags to the it.

An actual tag can hold inconsistent values, tough. I made some tests and I've seen that once a lookup value is copied in the SQLite db, it is not overwritten by subsequent attempts to copy a different value.
Also this depends on the configuration of the SQLite tag, see later.

Nevertheless, the actual tag has precedence over the SQLite tag, so how can I grant referential integrity in the properties dialog?
In general by not activating "Forward read, if empty" for the relevant SQLite tag. In this case the actual file tag is ignored. But if you copy all file tags to the db using the default configuration for a certain lookup value, the last copied value will become the lookup value for all files.

Also, how do I modify an already existing lookup value in the lookup table?
Also this depends on the configuration of the SQLite tag. The relevant options are "Overwrite without marker" and "Only if empty" (see help file for further information), but in any case you can overwrite it by prepending the overwrite marker (default @@) to the new value.

Consider the following example:

Given are 3 tagged tracks without any SQLite tag or lookup table defined so far:
track1: Artist=Rockband; Genre=Rock
track2: Artist=Rockband; Genre=Rock
track3: Artist=Rockband; Genre=Pop

Now execute the following steps:
  • create 2 SQLite tags "artist" and "genre" without changing the default settings.
  • define a lookup table "artist" with the key tag "artist".
  • edit the SQLite tag "genre", set the lookup table name to "artist" and keep the automatically set lookup table column "genre".

A possible scenario for initializing the tag database and adjusting the genre tag afterwards could be:

  • initialize the tag database by copying the file tags to it using the context menu: "Tagging -> SQLite tags -> Copy cached tags to the  SQLite db. If you don't have any other tag info filters active you could also use "Copy unfiltered tags to the SQLite db". The result will be in general the same. All three files should now have the genre "Pop" while actual file tags for track1 and track2 are still "Rock" as you can see in the details section for each file
  • open the properties dialog for track2 and change the value of genre to "Pop Rock". This becomes now an individuell tag value for track2 as you also can see in the details section: genre is listed under "Tag values" while it is listed under "Lookup values" for the other two tracks which have still the genre "Pop"
  • open the properties dialog for track1 and change the value of genre to "@@Rock". Now the genre lookup value for the artist "Rockband" is "Rock", so the genre for track3 is now also "Rock", while it is still "Pop Rock" for track2
  • open the properties dialog for track2 and remove the genre tag value for it. Now the genre value for track2 becomes the lookup value "Rock"






Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #152
Hi everyone,

In a separate topic I've asked if it is possible to 'automatically' rate an album based on the individual songs ratings - some kind of averaging - and I've been suggested to ask my question here since sqlite seems to be a powerful toolbox, so here I am.

Would you think it would even be possible to achieve such a task with sqlite ? I must admit I'm not familiar at all with it

thanks a lot for your feedback

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #153
In a separate topic I've asked if it is possible to 'automatically' rate an album based on the individual songs ratings - some kind of averaging - and I've been suggested to ask my question here since sqlite seems to be a powerful toolbox, so here I am.
Once you have the plugin installed:
  • Go to 'File > Preferences > Media Library > SQLite utilities > Playlist table' click 'Add defaults', then add the tag that you want to store the album rating in (just put the name in the Name property and click OK). The following sql assumes albumRating.
  • Drag files from a number of albums into a new playlist.
  • Select 'Library > SQLite console' and a popup window should appear.
  • Copy the following code into the window and Execute it.
Code: [Select]
select album,
       [album artist],
       date,
       avg(rating) albumrating
  from Playlist
  where playlist_index = active_playlist()
  group by 1, 2, 3;
NOTE:
The code runs against the tracks in the active playlist and produces a summary for you to verify.
I've assumed the current tag holding your track rating is called RATING, change that in the sql above if it is not.
I have no idea about how ratings are stored so at the moment it simply averages the tag.
It assumes the combination of <album,[album artist],date> uniquely identifies an album in your collection.

The above simply produces a listing showing the albumrating that could be stored.
I did see some discussion about how best it could be calculated, so I'm assuming this first pass will generate further discussion and evolve the sql.
When you're happy with the calculation we can modify the code to update your files, but I would want you to test that against a subset of your data first.

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #154
Hi SimBun,

thanks for the support!
Just had a try and it looks OK. The only obvious improvement I see would be to round up or truncate the returned results to 2 digits after the decimal point like 2.08 instead of 2.07692307692308

This could be enough for what I'm looking for. Even if weighting based on songs length - as I was suggested - could be fancier, I would prefer to keep it simple.

However I'm wondering what to do about songs I've rated 1/5 because usually those are intro/interlude/outro tracks I want to skip with foo_skip when listening to shuffled playlist. They could degrade the album rating though I do not care about them. Anyway it's something I could consider later

thanks again

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #155
However I'm wondering what to do about songs I've rated 1/5 because usually those are intro/interlude/outro tracks I want to skip with foo_skip when listening to shuffled playlist. They could degrade the album rating though I do not care about them. Anyway it's something I could consider later
If you want to exclude ALL tracks with a rating of 1 then it's trivial to do in the sql (I've excluded them in the sql below - "and rating <> 1"), otherwise, maybe you could base it on the length of the tracks.

Code: [Select]
drop table if exists albumSummary;
create table albumSummary as
  select album,
         [album artist],
         date,
         round(avg(rating), 2) albumrating
  from Playlist
  where playlist_index = active_playlist()
        and rating <> 1
  group by 1, 2, 3;

update PlaylistUpdatable as a
  set albumrating = b.albumrating
  from albumSummary as b
  where       playlist_index = active_playlist()
        and a.album          = b.album
        and a.[album artist] = b.[album artist]
        and a.date           = b.date;
drop table if exists albumSummary;

I've tested it myself and it looks fine, but please try this on a copy first and always make sure you have an up to date backup.

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #156
I've just tested your script and it works great, many thanks!

I have basic knowledge in scripting and none in sql syntax but I would have expected some iteration loop but can't see any obvious one, where is the magic ? Definitely I need to read some sql 101  ;)

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #157
otherwise, maybe you could base it on the length of the tracks.

I guess the part of the script to be modified would be
Code: [Select]
round(avg(rating), 2) albumrating

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #158
I have basic knowledge in scripting and none in sql syntax but I would have expected some iteration loop but can't see any obvious one, where is the magic ?
The magic is that database operations act on an entire database and not just one entry in the database, in other words iteration is built into the "select... from" structure.
It's your privilege to disagree, but that doesn't make you right and me wrong.

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #159
I've just tested your script and it works great, many thanks!

I have basic knowledge in scripting and none in sql syntax but I would have expected some iteration loop but can't see any obvious one, where is the magic ? Definitely I need to read some sql 101  ;)
SQL reads row by row but you can produce aggregations using 'group' and other window functions.
There's really not a lot to the basic sql syntax so it's definitely worth getting comfortable with.

otherwise, maybe you could base it on the length of the tracks.

I guess the part of the script to be modified would be
Code: [Select]
round(avg(rating), 2) albumrating
I was actually thinking more about how to exclude the intro/interlude/outro if it wasn't just these that were rated 1. Are they generally shorter, maybe the replaygain figures would be a good way to identify them.

The section you highlighted is the code that computes and rounds the rating, so if you wanted to change the logic it would go there.

If you run the following code it'll show you everything you could reference in your calculation - although likely it's just length_seconds that would be of interest.
Code: [Select]
select *
  from Playlist
  where playlist_index = active_playlist()




Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #160
Good to know.
Sure there are likely several ways to filter out unneeded tracks and calculate an 'accurate' average.
I'll be travelling for work this week so not much time to think about it but hopefully the week after I'll be able to

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #161
The section you highlighted is the code that computes and rounds the rating, so if you wanted to change the logic it would go there.
If you run the following code it'll show you everything you could reference in your calculation - although likely it's just length_seconds that would be of interest.

Finally I've been thinking about averaging the rating weighted on track length (sometimes I do have instrumental only tracks which can be >3 mins on which i defined a rating of 1)
I tried to play with length_seconds and with the sum() function to get total album length but without succes so far, still investigating  ;)

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #162
The section you highlighted is the code that computes and rounds the rating, so if you wanted to change the logic it would go there.
If you run the following code it'll show you everything you could reference in your calculation - although likely it's just length_seconds that would be of interest.

Finally I've been thinking about averaging the rating weighted on track length (sometimes I do have instrumental only tracks which can be >3 mins on which i defined a rating of 1)
I tried to play with length_seconds and with the sum() function to get total album length but without succes so far, still investigating  ;)
To sum length_seconds it's simply:
Code: [Select]
sum(length_seconds) as name_of_calculated_column
For the average you'd just need to use "avg" instead of "sum".

So in the original sql it would be:

Code: [Select]
  select album,
         [album artist],
         date,
         round(avg(rating), 2) albumrating,
         sum(length_seconds) as total_album_length_seconds
  from Playlist
  where playlist_index = active_playlist()
        and rating <> 1
  group by 1, 2, 3;

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #163
Thanks, I hope I got it now. Here should be the final code:
Code: [Select]
drop table if exists albumSummary;
create table albumSummary as
  select album,
         [album artist],
         date,
         round(sum(rating*length_seconds)/sum(length_seconds), 2) as albumrating
  from Playlist
  where playlist_index = active_playlist()
  group by 1, 2, 3;

update PlaylistUpdatable as a
  set albumrating = b.albumrating
  from albumSummary as b
  where       playlist_index = active_playlist()
        and a.album          = b.album
        and a.[album artist] = b.[album artist]
        and a.date           = b.date;
drop table if exists albumSummary;


Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #165
Thanks! Following this discussion:
- As far as I have tested this saves the tag to the files; is it possible to save them on the database only?
- Is it possible to automatically re-calculate this tag when new files are added to the library?

Btw, I have found round does not output numbers in a consistent format. Had to use padr() too
Code: [Select]
...
       padr(round(sum(rating*length_seconds)/sum(length_seconds),2),4,'0') as albumrating
...

Otherwise I keep getting ratings as 2.0 and other times as 2.72, ... instead of always 2 decimals.

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #166
Hi Regor,
thanks for the correction. Unfortunately I'm not skilled enough to answer your questions but likely someone else will

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #167
- As far as I have tested this saves the tag to the files; is it possible to save them on the database only?
Yes, with the tagging functionality of foo_sqlite by defining the tag as an SQLite tag.

- Is it possible to automatically re-calculate this tag when new files are added to the library?
Only indirectly by using the possibility to materialize the virtual media library table. You could add one ore more triggers to the materialized table which are doing the calculation. Not only when adding new files to the library, also when updating the rating tag. But these triggers need to be carefully designed to avoid endless recursions or too long running updates.

Code: [Select]
...
       padr(round(sum(rating*length_seconds)/sum(length_seconds),2),4,'0') as albumrating
...
A better solution would be to use the format() function:
Code: [Select]
...
       format('%.2f',round(sum(rating*length_seconds)/sum(length_seconds),2)) as albumrating
...

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #168
Thanks!
Quote
Yes, with the tagging functionality of foo_sqlite by defining the tag as an SQLite tag.
Ok, just seen it. Will try experimenting with it.

Quote
Only indirectly by using the possibility to materialize the virtual media library table. You could add one ore more triggers to the materialized table which are doing the calculation. Not only when adding new files to the library, also when updating the rating tag. But these triggers need to be carefully designed to avoid endless recursions or too long running updates.
Too complex for me at this moment; I barely understand the basics right now. I appreciate the component capabilities but functionality/docs are abstract as hell (for me).

Quote
A better solution would be to use the format() function:
Code: [Select]
...
       format('%.2f',round(sum(rating*length_seconds)/sum(length_seconds),2)) as albumrating
...

I tried the format function as first solution and it didn't work at all for me. Maybe I didn't use it properly though.
In docs I always find that SQL format works like this:
FORMAT (value, format [, culture])

Buy you are reversing the args there (?) Just checking now, the reversed version works xd so that was the problem.

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #169
Quote
Only indirectly by using the possibility to materialize the virtual media library table. You could add one ore more triggers to the materialized table which are doing the calculation. Not only when adding new files to the library, also when updating the rating tag. But these triggers need to be carefully designed to avoid endless recursions or too long running updates.
Too complex for me at this moment; I barely understand the basics right now. I appreciate the component capabilities but functionality/docs are abstract as hell (for me).
Actually, currently it wouldn't work anyway. After checking the code I saw, that the virtual tables are not visible for the db connection, which is running the materialization. Will be changed for the next version.

I tried the format function as first solution and it didn't work at all for me. Maybe I didn't use it properly though.
In docs I always find that SQL format works like this:
FORMAT (value, format [, culture])
That were the wrong docs. You should always check only the SQLite docs as especially function implementations are not really standardized.

Buy you are reversing the args there (?) Just checking now, the reversed version works xd so that was the problem.
It's documented like that here.


Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #170
Does this plugin have any documentation? I'm hoping I can use this to sanitize my library tags without modifying the original files, but the question mark in the program leads to a file not found error and the documentation link on the download page leads to a blank wiki entry. Is there a site for this that I'm just not seeing? Id like to know what this component is capable of!

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #171
Does this plugin have any documentation?
Yes via the menu "Help -> SQLite utilities".

but the question mark in the program leads to a file not found error
If you mean the question mark on the titlebar of the preferences dialog, it's indeed not implemented for the main entry. However, for all four subpages clicking on the question mark will lead you to the relevant page of the documentation.

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #172
Does this plugin have any documentation?
Yes via the menu "Help -> SQLite utilities".
File not found.
but the question mark in the program leads to a file not found error
If you mean the question mark on the titlebar of the preferences dialog, it's indeed not implemented for the main entry. However, for all four subpages clicking on the question mark will lead you to the relevant page of the documentation.
File not found for all these as well. Exception being the one you called out, that one produces a little box that says "No help available for this page." Where do I download the documentation files?

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #173
Where do I download the documentation files?
They are shipped with the component and you can find them in the components folder in your profile folder. But I guess the problem is caused by a wrong system setting. Clicking on the question mark or the help menu entry executes the application which is associated in your system with the file extension html. If this application can't be find, you will get a "File not found" error.

 

Re: [fb2k v2] SQLite Utilities (foo_sqlite)

Reply #174
They are shipped with the component and you can find them in the components folder in your profile folder.
Thanks! Found them. Took a little digging but the component appears coded to look for the documentation in:
AppData\Roaming\foobar2000-v2\user-components-x64\foo_sqlite\help\

However, when Foobar2000 is installed from the Microsoft Store, the equivalent location is actually:
AppData\Local\Packages\Resolute.foobar2000modern_cg7j1awqsza28\LocalCache\Roaming\foobar2000-v2\user-components-x64\foo_sqlite\help

Looks like a minor bug.
Workaround might be to symlink the actual help folder to the location where the component is looking. That's non-trivial, but I might give it a try for convenience's sake if time allows.

Edit: I found some time and set up the symlink. It works! Question Mark in the app leads to the documentation as expected. The join is:
"\Appdata\Roaming\foobar2000-v2" to "\Appdata\Local\Packages\Resolute.foobar2000modern_cg7j1awqsza28\LocalCache\Roaming\foobar2000-v2"
I didn't want to just copy/paste the docs to the location since there's no telling if other functionality of the component is broken because of this. This way, any incorrect locations in the code will point to the things they're supposed to.