Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: [fb2k v2] SQL Tree (foo_uie_sql_tree) (Read 144089 times) previous topic - next topic
0 Members and 2 Guests are viewing this topic.

Re: foo_uie_sql_tree

Reply #150
I'm wondering if I can use this component to display %album rating%, preferably by writing a tag to the files, in ELPlaylist, as averaged from individual ratings thorugh a calculation posted earlier. I've reached the thread by looking up "calculate album rating" on the searchbar, and though I know my way around foobar pretty well, I don't get how this works. Could someone quickly guide me through what I want? All I figured out is I can open a terminal, a blank view window, and add columns from preferences (which keep disappearing after I clumsily add them...)

Re: foo_uie_sql_tree

Reply #151
I'm wondering if I can use this component to display %album rating%, preferably by writing a tag to the files,
Yes, writing a tag for album rating to the files is possible. But first you should be more specific on this:
All I figured out is I can open a terminal, a blank view window, and add columns from preferences (which keep disappearing after I clumsily add them...)
Please, describe exactly, what you are doing, best with screen shots.

Re: foo_uie_sql_tree

Reply #152
Thanks a lot for the quick assistance.

The way I understand this component functionality is, from a list of tags stored in rows on the MediaLibrary window, it can retrieve data with which it can make advanced calculations, which can then be turned into new tags. Am I getting it right?

So far I've:
1. Installed the foo_sqlite and foo_uie_sql_tree components
2. Added a SQL Tree panel to my layout, which shows <Examples> > Library folders
3. From the Preferences > MediaLibrary, clicked "Add defaults", resulting in a long list of rows presumably read from my library. After I click "OK" something off happens - the Preferences window closes, and when I open it again the rows are empty. If I click again on "Add defaults," and then click "Apply", I get further on what's wrong with an error message. Here I thought one of my tags contains a faulty value, so I try to add only the tags I'm actually gonna be using, like %rating%. I tried 3 ways to do it, but they all result in the same aforementioned error when I press apply.

After getting this solved I believe I'd have to open SQLite console, paste the code I linked in my previous post, click "Execute" (at which point it would generate the values), and then write these values to the files in some way.

Hopefully I'm on the right path!

Re: foo_uie_sql_tree

Reply #153
The way I understand this component functionality is, from a list of tags stored in rows on the MediaLibrary window, it can retrieve data with which it can make advanced calculations, which can then be turned into new tags. Am I getting it right?
I wouldn't describe it in this way, but basically your understanding is correct.

Here I thought one of my tags contains a faulty value, so I try to add only the tags
The problem here are the tag names in the list which starts with a double quote, which are shown in your first screen shot. The empty tag name at the beginning of the list could also cause some troubles. In a future version, such kind of tags will be ignored.

so I try to add only the tags I'm actually gonna be using, like %rating%. I tried 3 ways to do it, but they all result in the same aforementioned error when I press apply
This is a bit strange as if the tags with the double quotes are not in the list, the mentioned error message can normally not occur. You also shouldn't change the settings for the multivalue handling of the rating tag. Just keep the default values and leave the values in the Tag group untouched. The meaning of the entries are described in the help file (Help -> SQLite viewer).

You also need to define a tag for the album rating, e.g. "album rating" (without the double quotes)

After getting this solved I believe I'd have to open SQLite console, paste the code I linked in my previous post, click "Execute" (at which point it would generate the values), and then write these values to the files in some way.
Actually, it's not that easy:

To avoid messing up completely your media library, you should use only the active playlist for updating your tags. Therefore you also need to set up the tags properly for the playlist table like for the media library table in the preferences.

After this is done you need to create an instance of the virtual table for the playlist table, which is updatable as described in the help file:
Code: [Select]
CREATE VIRTUAL TABLE Playlist_Updatable USING MetaDB_Module(no_multivalue_split);
This is a one time task and you can run this statement in the SQLite console.

You can update your tags with the following SQL statement (I didn't test it, but I use similar statements, which are working in this way).
Code: [Select]
-- Creating a temporary table for the album ratings for performance reasons is at least necessary for larger playlist
DROP TABLE IF EXISTS tmp_AlbumRating;
CREATE TABLE tmp_AlbumRating AS
SELECT "%album artist%" AS album_artist, -- %album artist% needs to be defined as a TFExpression type in the preferences. This is automatically done, when using the default values
           album,
           avg(CAST(rating AS REAL)) AS "album rating" -- as tags are always text fields, the rating tag needs to be explicitely casted to a numeric value
FROM Playlist_View
WHERE playlist_index=active_playlist() -- use only the entries from the active playlist to avoid completely messing up the whole library
GROUP BY 1,2;

CREATE UNIQUE INDEX tmp_AlbumRating_pk ON tmp_AlbumRating(album_artist,album);

UPDATE Playlist_Updatable
SET "album rating"= (SELECT "album rating"
                                 FROM tmp_AlbumRating
                                 WHERE album_artist=Playlist_Updatable."%album artist%"
                                       AND album=Playlist_Updatable.album
WHERE playlist_index=active_playlist();

In the SQL tree you could create a node then, which contains the script above in the batch section (the query section remains empty), use "Execute SQL" as click action and "Send to SQLite console" as middle click action.


Re: foo_uie_sql_tree

Reply #154
Hi,

since a couple of days I am getting this message in foo_sqlite and the tree component will not show anything (likely because of this issue).

Code: [Select]
Execution error:
SQLite Error: (1) Unable to declare virtual table MediaLibrary:
SQLite Error: (1) duplicate column name: #helium albumid

Error while preparing the first statement of:

SELECT * FROM MEDIALIBRARY

Any ideas what to do? I have verified the library for the column but without success.

Thanks in advance

Re: foo_uie_sql_tree

Reply #155
Check the entries in the preferences under "Media Library -> SQLite viewer -> Medialibrary table". Although it should not be possible if you don't change the internal sqlite tables manually, it seems that you have two or more entries with the same name here. If this is the case, delete all but one of them and the problem should disappear.

Re: foo_uie_sql_tree

Reply #156
Thanks for the hint I wasn't aware of that setup. Strangely all entries were duplicated. I created an auto hotkey script to have all duplicates removed. Now it works again.

Have a nice day!

Re: foo_uie_sql_tree

Reply #157
Hi,
I have been trying to create a 25 most played tracks playlist with this code:

Code: [Select]
SELECT DISTINCT title
FROM medialibrary
ORDER BY play_count
DESC LIMIT 25

I never used SQL before, but as far as I understand, select distinct should retrieve only different values from the table. In the SQL console it works, and 25 different tracks appear.
X

But as soon as I use that as a query and send it to a playlist, duplicate tracks are retrieved. They are tracks with same title but different path, etc. (i.e. 2 masterings or versions of the same album). Check capture 2.
X

Since the playback statistics plugin adds play_count according to "track - tittle", obviously 2 versions of the same track/album get the same statistics. Foobar don't consider them as duplicates, neither does your plugin option "remove duplicates before sending...".

Is there any way to filter those duplicates and get the same result than the console? I don't really care if duplicates are discarded just by selecting only the first result or whatever.

Thanks!

Re: foo_uie_sql_tree

Reply #158
This SQL statement should do what you want:

Code: [Select]
SELECT a.title
FROM MediaLibrary a JOIN (SELECT title,max(path) path,max(subsong) subsong
                                            FROM MediaLibrary
                                            GROUP BY title
                                            ORDER BY play_count DESC
                                            LIMIT 25) b ON a.path=b.path AND a.subsong=b.subsong

Re: foo_uie_sql_tree

Reply #159
Nope. Now all tracks with "duplicates" are simply omitted. Note the yellow tracks are in both queries (yours and mine).
X.
X

Re: foo_uie_sql_tree

Reply #160
Nope. Now all tracks with "duplicates" are simply omitted.
Actually is this by chance, but anyway the query is indeed wrong.

Put this into the batch section of the query due to performance reasons:
Code: [Select]
DROP TABLE IF EXISTS tmp_paths;
CREATE TABLE tmp_paths AS
SELECT max(path) path
FROM MediaLibrary
GROUP BY title;

CREATE UNIQUE INDEX tmp_paths_in_uq ON tmp_paths(path);

Then use this query:
Code: [Select]
SELECT title
FROM MediaLibrary a JOIN tmp_paths b ON a.path=b.path
ORDER BY play_count DESC
LIMIT 25

This will not work in case you are using files with multiple subsongs, like for example cuesheets, but without the possibility to use window functions, which are not supported by the currently used SQLite version, it's a bit difficult to get what you want.

Re: foo_uie_sql_tree

Reply #161
Now I get this error:
X

Btw I only have single track files, no cue files or subsongs. I do have multiple files named "01 - blablabla.flac", on different albums and/or different masterings of the same album.

If there is no "easy" solution, no worries. I already found how to do it on JavaScript with a SMP script (the "remove duplicates" part).

Re: foo_uie_sql_tree

Reply #162
Now I get this error:
[attach type=thumb]18423[/attach]
As you didn't mention it, I can only guess, that you tried to validate the batch. As mentioned in the help file, this is not possible, if DDL statements are involved. Try just to execute it or run it in the console.

Re: foo_uie_sql_tree

Reply #163
I just got this installed as I read on different threads that this can find duplicates.

I not that great at SQL queries so any help would be appreciated.

Getting %artist% and %title% duplicates would be what I'm after. There is a post from 2007 seems to be an old syntax for finding duplicates. https://hydrogenaud.io/index.php?topic=51307.msg505964#msg505964

Thanks for this @fbuser‍.

Re: foo_uie_sql_tree

Reply #164
I just got this installed as I read on different threads that this can find duplicates.
Yes, it is possible.

There is a post from 2007 seems to be an old syntax for finding duplicates. https://hydrogenaud.io/index.php?topic=51307.msg505964#msg505964
It's because it's a different component.

Getting %artist% and %title% duplicates would be what I'm after.
Put this into the batch section of the query:
Code: [Select]
DROP TABLE IF EXISTS tmp_artist_title;
CREATE TABLE tmp_artist_title AS
SELECT strfilter(lower(unaccent(artist)),'abcdefghijklmnopqrstuvwxyz0123456789') AS "artist",
       strfilter(lower(unaccent(title)),'abcdefghijklmnopqrstuvwxyz0123456789') AS "title"
FROM MediaLibrary
WHERE title IS NOT NULL
  AND strfilter(lower(unaccent(title)),'abcdefghijklmnopqrstuvwxyz0123456789')<>''
GROUP BY 1,2;

CREATE UNIQUE INDEX tmp_artist_title_pk ON tmp_artist_title(title,artist);

DROP TABLE IF EXISTS tmp_artist_title_path;
CREATE TABLE tmp_artist_title_path AS
SELECT path,
       subsong,
       strfilter(lower(unaccent(artist)),'abcdefghijklmnopqrstuvwxyz0123456789') AS "artist",
       strfilter(lower(unaccent(title)),'abcdefghijklmnopqrstuvwxyz0123456789') AS "title"
FROM MediaLibrary a
WHERE title IS NOT NULL
  AND strfilter(lower(unaccent(title)),'abcdefghijklmnopqrstuvwxyz0123456789')<>''
  AND EXISTS (SELECT NULL
              FROM tmp_artist_title
              WHERE artist=strfilter(lower(unaccent(a.artist)),'abcdefghijklmnopqrstuvwxyz0123456789')
                AND title=strfilter(lower(unaccent(a.title)),'abcdefghijklmnopqrstuvwxyz0123456789'))
ORDER BY 3,4;

DROP TABLE IF EXISTS tmp_paths;
CREATE TABLE tmp_paths AS
WITH Paths AS (
SELECT a.path path1,
       a.subsong subsong1,
       b.path path2,
       b.subsong subsong2
FROM tmp_artist_title_path a JOIN tmp_artist_title_path b ON a.rowid=b.rowid+1
WHERE a.artist=b.artist
  AND a.title=b.title)
SELECT path1 path,
       subsong1 subsong
FROM Paths
UNION
SELECT path2, subsong2
FROM Paths;

CREATE UNIQUE INDEX tmp_paths_pk ON tmp_paths(path,subsong);

Then use this query:
Code: [Select]
SELECT upper(substr(strfilter(lower(unaccent(artist)),'abcdefghijklmnopqrstuvwxyz0123456789'),1,1)) Letter,
       proper(artist) Artist
FROM MediaLibrary a
WHERE EXISTS (SELECT NULL
              FROM tmp_paths
              WHERE path=a.path
                AND subsong=a.subsong)
ORDER BY upper(substr(strfilter(lower(unaccent(artist)),'abcdefghijklmnopqrstuvwxyz0123456789'),1,1)),
         strfilter(lower(unaccent(artist)),'abcdefghijklmnopqrstuvwxyz0123456789') COLLATE NaturalNoCase,
         CAST(length_seconds AS INTEGER),
         title

Note: this might also find tracks, which are actually no duplicates, because all non-alphanumeric characters and diacritics are removed from artist and title before they are compared case insensitive.

Remove all strfilter functions, if you don't want to filter out non-alphanumeric characters, e.g. lower(unaccent(artist)) for the artist.

Remove all unaccent functions, if you don't want diacritics to be removed, e.g. strfilter(lower(artist),'abcdefghijklmnopqrstuvwxyz0123456789') for the artist.

Remove all lower functions, if you want a case sensitive comparison, e.g. strfilter(unaccent(artist),'abcdefghijklmnopqrstuvwxyz0123456789') for the artist.

If you don't want all of them above just use the relevant column, e.g. just artist for the artist.

Re: foo_uie_sql_tree

Reply #165
@fbuser

wow, that's intense!! Thanks so much.

I'm getting an error for the query and the batch validate like this for query:


And for the batch when I try to execute it:
Code: [Select]
Execution error:
SQLite Error: (1) no such table: MediaLibrary_View

Error while preparing the first statement of:
CREATE TABLE tmp_artist_title_path AS
SELECT path,
      subsong,
      strfilter(lower(unaccent(artist)),'abcdefghijklmnopqrstuvwxyz0123456789') AS "artist",
      strfilter(lower(unaccent(title)),'abcdefghijklmnopqrstuvwxyz0123456789') AS "title"
FROM MediaLibrary_View a
WHERE title IS NOT NULL
  AND strfilter(lower(unaccent(title)),'abcdefghijklmnopqrstuvwxyz0123456789')<>''
  AND EXISTS (SELECT NULL
              FROM tmp_artist_title
              WHERE artist=strfilter(lower(unaccent(a.artist)),'abcdefghijklmnopqrstuvwxyz0123456789')
                AND title=strfilter(lower(unaccent(a.title)),'abcdefghijklmnopqrstuvwxyz0123456789'))
ORDER BY 3,4;

CREATE UNIQUE INDEX tmp_artist_title_path_pk ON tmp_artist_title_path(path, subsong);

DROP TABLE IF EXISTS tmp_paths;
CREATE TABLE tmp_paths AS
WITH Paths AS (
SELECT a.path path1,
      a.subsong subsong1,
      b.path path2,
      b.subsong subsong2
FROM tmp_artist_title_path a JOIN tmp_artist_title_path b ON a.rowid=b.rowid+1
WHERE a.artist=b.artist
  AND a.title=b.title)
SELECT path1 path,
      subsong1 subsong
FROM Paths
UNION
SELECT path2, subsong2
FROM Paths;

CREATE UNIQUE INDEX tmp_paths_pk ON tmp_paths(path);


Re: foo_uie_sql_tree

Reply #167
@fbuser‍ I'm getting an error for the query and the batch validate like this for query:
When I try to validate batch, I get:
SQLLite Error table tmp_artist_title already exists
As mentioned in the help file, this is not possible, if DDL statements are involved. Try just to execute it or run it in the console.


Re: foo_uie_sql_tree

Reply #169
If I execute, I get error:
Code: [Select]
Execution error: SQLite Error: (1) no such table: MediaLibrary_View
Ok, it's an adaption of my query to detect duplicates, which contains some specialties for my own purposes. I forgot to replace  a reference to my MediaLibrary view. Use MediaLibrary instead of MediaLibrary_View. It's corrected in my post.



Re: foo_uie_sql_tree

Reply #172
Now I get error:
Code: [Select]
Execution error:
SQLite Error: (19) UNIQUE constraint failed: tmp_paths.path

Error while executing:
CREATE UNIQUE INDEX tmp_paths_pk ON tmp_paths(path)
Thanks for testing it. It's just another error, which can't occur on my collection as I don't use cue sheets or other files with more than one subsong. It's corrected too in my post. The relevant part needs to be CREATE UNIQUE INDEX tmp_paths_pk ON tmp_paths(path, subsong)

Re: foo_uie_sql_tree

Reply #173
I omitted last line in batch
Now it gives a result (letter, artist) I don't really know, what that means.
Indexes are used to speed up queries.