Skip to main content


If you are using a Hotmail or Outlook email address, please change it now, as Microsoft is rejecting all email from our service outright.
Topic: [SMP] Music Graph development (Read 2017 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

[SMP] Music Graph development

First my request: I'm looking for a few users to try and test a SMP collection of scrips I have been working on. When they are ready, I will public them on github and here, but right now I'm just looking for betatesting. This is my first time coding on JS and after +2000 lines of code some help would be appreciated testing it!

Back to the topic. Some time ago, while tinkering with foobar I found it was really difficult to create autplaylist based on similarity checks. Things like these:
  • Similar Artist
  • Date
  • Genre similarity
  • Style similarity
  • Key
  • BPM
  • Moods
  • Top 25 most played tracks

Things like Random Pools component, current SMP scripts or Search for same components (I know 2) have many limitations for those things:
  • Genre similarity: For example, you can create a random pool of Rock and Alt. Rock. And it works great! But... you can not create dynamically a random pool for the genre of the currently selected track. Random pools are meant to be pre-set. You would need to create a random pool for every genre on your library if you want those playlist....
  • Date: The same. You can put a date on random pools. But it's a static one. Every time you want to change the date of your pool, you have to reconfigure the pool.
  • Moods: any plugin it's either checking if Moods are equal or not (search for same). But you can not check 4 equal moods for ex.
  • Top 25 most played tracks: Random pools and other plugins can provide a top 25 tracks. But if you have tracks with same title but different album versions, they are duplicated! Foobar remove duplicates only remove the same exact tracks. You could have the same track appearing 3 times, rendering that playlist useless. There is no workaround for this with any plugin I know.

  • Similar Artist: Yep, we got Lastfm scripts BUT you can only create an auto-playlist with just 1 similar artist. Not all of them! The point to have similar artists auto-playlist is creating playlist with artist similar to the currently selected track, all of them. Not only one.. I want to select Led Zep, press a button and create a playlist with all those similar artists.

I could continue...  but essentially, foobar is missing the functionality spottily gives, or allmusic, or musicbee, etc. Essentially the ability to create playlist dynamically based on tags or internet info of the currently selected track. There are some things done on that field but really limited on functionality.

There is also a music server app name MusicIp which works similarly to the things mentioned. Short story: it requires days to scan 50k tracks, it's an unsupported program from 10 years ago,  requires the server to be loaded along foobar, CMD scripts to send tracks to server and load playlists AND... it doesn't support multi tags. So all auto-playlist created rely on the musical analysis, but tags become useless. Also rating is not loaded, you have to create scripts to load foobar ratings to the server, manually! The program accepts some scripting and advanced commands, but there is no documentation and it's closed source... so good luck with it. Btw, someday I will post the things I managed to make work properly (only thing missing is multi-tag support).

All these findings lead me to AI reasearch. I mean... MusicIp "works" but it uses musical analysis from a decade ago. Nowadays we have machine learning models giving us things like genre or moods. That's the thing Spotify, Allmusic or any other network player uses. So we have 2 problems to solve: first, finding how to analyze music using modern methods. And tag it! Second, use foobar to create dynamic playlist with that info!

The first one is partially solved using Musicbrainz Picard. You can get mood, BPM and key tags from that program. And any file missing those variables can be analyzed with an exe which uploads the analysis to the server; 1 day later you re-load those files in Picard and magic. The missing tags are there.
I have managed to tag  (mood, occasion, BPM, Key) 99% of my files with that method, while adding manually things like genre and style ("subgenre"). Things like similar artists are provided by the component (can be saved to local tags!) or SMP scripts.

Tagging is mostly done. but the second one, previously discussed, its unresolved up to now. We have 2 things to consider now: dynamic playlist creating and genre similarity, whatever that means. Back to AI research, I thought I could find something useful on that field... without luck. "Solutions" require complicate scripting, a range of tools and then finding a good set of reference... but someday I found this Music map. There have been other similar maps, most interactive, giving you youtube videos of the selected style.

So, considering automated music genre mapping was out of question, I thought that could be translated to code. At the same time I was beginning to discover SMP scripts, so I tried to translate that map to js. This is my fist time on js, so it has been a hard road.

My first attempt is what I call "DYNGENRE". It's a pretty simple concept: let those 18 Supergenres (following the web convention) be assigned a number from 0 to 18. Any genre tag or style tag should fall into one of those supergenres. We can simplify it even more and not make any difference between "modal jazz" or "cool Jazz", they are just jazz (9)! And assign to those styles which are a mix of some supergenres their set of numbers: "Trip-Hop" (12,18) would be "Hip-Hop" (12) and "Downtempo" (18).

That assigns any track to a Supergenre (or a mix) according to genre tags and styles, and the best part: it's done on the fly. No need to re-tag. No need to recheck files when changing scripts. The DYNGENRE map can be set as you wish just by tweaking the js file.

The other problem was dynamic playlist creation, but I managed to code that for the currently selected track. It takes the mood, BPM, Genre, Style, Date and key tags and create playlists based on those tags. That means no more statics pools. You select rock from the 90s, and it outputs things according to that directly to a playlist (bypassing the need to press 20 buttons using the "search for same" plugins).

While working on that, I got the "duplication tracks" problem again... essentially, if I looked for things similar to a  Led Zeppelin track then it would output things from Led Zep,  Deep Purple, etc. but If I had 4 versions of "You shook me"... then all 4 got selected on the final playlist! That was one of the biggest problems with other plugins too. I want a playlist with things appearing once, not the same track all around from different albums, compilations, live sets, etc. So I created some scripts to solve that too.

The "remove duplicates script" looks for tracks on the handle list or playlist with same title/artist/date and remove all but one. It can be configurable to any other tag too and maintains the order. The last part is important! The script greatly surpasses the built in foobar remove duplicates functionality and the one on SMP methods too.

Other things I added are the combination of multitags. i.e. "I want only tracks with 5 similar moods", but if the tracks have 8 different moods, then you have to create k-combinations (k=5) of the entire set and check for every combinantion via queries. That's impossible to do on static pools or manually. Then we have user adjustable weighting of tags, filtering of any track below a minimum score, pool selection by lowest distance or random, etc
With all the problems solved, the final result is a playlist generated based on the tags of currently selected track with no duplicates and following similarity rules. All user adjustable.

And that's the "DYNGENRE" method which would be a more complex version than the "WEIGHT" method. My collection of scripts, ready to use, offer that (search by distance), along remove duplicate scripts (ready to be used standalone on playlist), Top 25 Tracks (without the duplicates problem), search similar style + k-com of moods and Search for same style (multitag).

This is a screenshot of the thing working. I took a folk track, and 857 tracks were eligible on the pool (score over 65%). Final selection, 50 tracks.

Re: [SMP] Music Graph development

Reply #1
After all this discussion, here it comes the second part. Along the "DYNGENRE" and "WEIGHT" methods, there is a third one: "GRAPH". Remember the previous music map examples... I simplified the thing assigning main genres/styles to numbers and not much more! But what about a real map (graph) with genres and styles? This:

Well, after coding all the previous things I tried creating a real map. With distances and links. I found this js library which does exactly what I wanted: graphs, nodes, links and the most important thing, pathsearch. But it was for node.js. I find how the hell put that thing to work within foobar and got a graph created using arrays with all the genres and styles. And I mean all. Then adding supergenres, clusters, style clusters, etc.

I put all that on the "search by distance" script, configured some limits for the paths and got it working. Now "Coutry Blues" is not just "Blues" (dyngenre method), but a node within blues supergenre. And it would be more distant to Electric blues than to Delta blues, for ex. Then we got style clusters and primary origins/derivations to traverse between different super-genres.. and all that. It takes 2-5 seg to search 40k tracks and calculate paths and distances within foobar. It works, and creates playlists.

My final step on the development have been the graphical part. Coding +60 arrays with nodes and links is great but it becomes a crazy thing, to the point I don't know if I'm duplicating things or setting things wrong. And here the graphical representation comes to help. I managed to render the graph within firefox using other JS library and creating an HTML file pointing to the foobar js scripts. Whenever you edit something within foobar scripts, it gets automatically loaded on the html. And that's the first image, you can zoom, move, hover the nodes and look for the IDs for debugging. With that tool I have managed to solve some problems I had at the graph today and added Folk and Classical clusters too, so that should make all popular music up to date (but the "World music" at musicmap web).

For that part, I would gladly accept some help rendering the map according to link weighting (i.e. the real precomputed distance) instead of the viva methods by link centrality. That goes beyond my pretty limited knowledge of js and would be the final piece.

Re: [SMP] Music Graph development

Reply #2
First my request: I'm looking for a few users to try and test a SMP collection of scrips I have been working on.

I wasted a bunch of time with MusicIP back when it was beta - a lot of potential, as it generated the best related mixes I ever encountered. However it had the slowest track scanning ever and proprietary model that rendered it quickly useless.

I'm interested in yours - count me in. How?

It sounds great and also like it could lead to some personally long wanted FB statistical functions that have been out of my depth so far.

Re: [SMP] Music Graph development

Reply #3
Oh thanks! Just check your pm inbox in a few days ;)

This one generates mixes as good or better than musicIp but you need to have variety on your library. i.e. if you only have 20 folk tracks, then "folky playlists" will be really limited... while musicIp didn't care and filled the gaps with anything until it filled the minimum playlist length. The good part is that anything is configurable here, so you can lower the score filtering too...

This script also can be set to just output similar style/genre tracks or by mood or whatever, so you can also emulate those musicIp remixes which had different genres but "natural" variations.

I would want to recreate dynamic variations too. MusicIp was able to use one track as reference, output some similar tracks, use one of them as new reference for the rest of the playlist and so on. Also I could add more control to the content of the playlist, like X 80% similar tracks, Y 50% similar tracks, etc.Currently it's totally random, by score order (only higher scored tracks get to playlists), or by score order with random probability of being picked (it starts at high scored tracks but there is some randomness). MusicIp had so many details that I could work on this during so much time.

I want to implement fingerprinting comparison too at some point. Using either AcousticID fingerprints or libfooid. To find duplicates, similar track versions, ... but that's currently delayed since my js knowledge is pretty limited right now hahaha If someone points me to JS code to de-structure the fp I would be so grateful!

I'm currently polishing a few things so I can export all the files to you or anyone. Also managed to put custom images or shapes as nodes on rendering.

I have to create some buttons too (since I'm currently using the scripts as  "File\SMP" menus) and I will be ready for betatesting.

Re: [SMP] Music Graph development

Reply #4
I feel like a lot of this could be simplifed/made redundant by just smart tagging and understanding how we tend to categorize things in our heads.

I defaulted to just tagging common elements in music instead of following a rigid tree branching structure that people tend to follow. Each artist just has a bunch of stickers that range from very broad to oddly specific. The specificity is increased in areas of your library where you have the most knowledge/gathered the most amount of songs. And the only purpose is to help you quickly gather any subset of artists that have something in common.

Example stickers, ranging from more broad to oddly specific:

Basic genre of the song:
- This is always filled out as your starting point
- This is just to get the most basic idea of how the music sounds like.
- Lots of artists don't fit just one genre, so I often tag them with 1-3.

Subgenres equivalent:
- This is for areas of your library that has grown significantly and you need to break them down into smaller groups
- This requires that you actually learned to spot the small nuances of a genre and you actually care about the distictions
- If all "Electronic" music sounds the same to you, there's no point in meticulously tagging its subgenres
- But if "Melodic" or "Gothic" Metal has meaning to you, those subset of artists are worth tagging more precisely
- More generic terms tend to work better. E.g. "Melodic" can refer to Metal, or Rock, or even Classical music, or Industrial can refer to Rock, Metal or even Electronic music (if you approach it in a specific way).
- This way you tend to get a lot of cross-pollination between genres when building playlists

- This is fairly personal choice. It's more like a twist or "angle of attack" to further differentiate tracks within the same genre and subgenre.
- Words like "Modern", "Bleak", "Upbeat", "Downtempo/minimal" come to my mind. These are similar to subgenres but I'd say more abstract.
- They are more easily repeated across a wider arrays of genres/subgenres that don't otherwise have much to do with each other (you can have an upbeat song from any Electronic, Rock, Pop or even Metal song etc.)

Instruments used/location specific
- If you want you can specifically mark artists or song that emphasize an instrument or lack of
- For example music with standout drum usage (regardless of whether it's EDM, Classical or Metal music)
- Territory specific music that tend to use a similar set of instruments/style of music (E.g. Arabic or Asian)
- Instrumental songs vs songs with Female or Male vocal lead

Easy listening:
- This is to simply mark songs that are specifically easy, or hard to listen to, to filter potentially problematic songs from your autoplaylists
- Tracks don't tagged are counted as neutral

With this kind of tagging you can quickly get different kind of sets of music (either very broad, or very specific), or use them to generate nice autoplaylists with not that many rules in the query:
- All electronic music
- Only Melodic Metal music
- All gothic sounding music (ranging from Pop, to Metal, to Rock, to even Classical music)
- Electronic or Rock music that has no vocals but is generally upbeat
- Film scores that sound "Modern" and has a heavy emphasis on drums
- Anything that has an "Industrial" sound to it
- Japanese Pop music with female vocals
- etc.

Apart from these I also tend to use ratings in two ways:

Ratings supplied by the listener:
- People are not very good at rating objectively in a capped system (e.g. people usually rate from 7 to 10 in a 10 based system, unless the track is completely terrible)
- People's taste changes over time so what was rated 5/5 might not be subjectively thought as a 5/5 later
- In my system you can press a button to increase the songs rating by 1.
- Any song that's 1 or above is considered a favorite and can be included in autoplaylists
- Songs that are rated below 0 are banned and won't be part of normal listening
- Ratings don't have an upper cap so it's a numbers race. After sufficient amount of time passed you get a few tracks that are rated absurdly high and you can make queries based on this to filter out the "best" songs

Ratings tracked by the computer and the listener can't tamper with:
- Using the DAR system for this
- People tend to cheat with the ratings and artifically inflate them, if they get the choice (such as pressing the button multiple times)
- This represents the real time listened vs the subjective rating
- Using both numbers you can reasonably estimate the actual top songs
- Or just gradually phase out songs that have been listened to a lot, or force yourself to listen to those that got not much playback time

With the combination of the two things above you can make really well personalized sets of tracks that fit your current mood really well (via Facets or CUI Filter Panels).

My only gripe is that the random pools component (which has an option to basically normalize how often an artist's songs are presented) doesn't really let you save different presets of finely-tuned playlist generation rules, or that foobar can't really keep a running tally of "this artist has been played before in the last 20 minutes", to give equal chance to artists who only have a couple tracks.

Re: [SMP] Music Graph development

Reply #5
Btw it's already done and working on my side so there is no need to simplify things; I'm just looking for people to test it.

Your approach is the base of the things I'm discussing, but not the end. Yes, you can manually create playlists if you have tags set like that. That's what I have done on my library and it's what it's required for the scripts to work!

But that's not the point. I have my library tagged exactly like your description and I miss tons of things. Specially creating playlists dynamically. If I told you to select tracks similar in mood to the current one, you could spend 10 minutes creating combinations of moods...What is similar? 1 mood? 2? 5? Then the same with genres, with styles, etc. You know British Folk is similar to Folk Baroque, but .. that's in your head. Foobar knows nothing about "genres". You have to put that in tags in some way.

Autoplaylists, facets and all that have many weak points:
-Require direct user interaction. Queries become unmanageable for complex things. The user must check every playlist created to be sure they are right (duplications, ratings, etc.).
-They can not be changed dynamically. Yep, you can create a query for rock with moods "acoustic, sad, agressive, blabla", But then if you want the same for Folk, from specific dates you have to recreated the entire thing manually
-It's impossible for an human being to translate combinations of tags into queries manually. Check this image with a 15 lines query:;attach=18631;image
-They can not be set according to a source track. Yep, you can look for Rock or Glam Rock and all that. But you can not create an autoplaylist or Facets filter with "x same tags" to the current selected track.
-Can not be saved. Facets solves that partially.
-Can not filter tracks. You apply a query and that's all. Forget about filtering duplicates or other similar things. In fact, queries are created -by definition- to give you all the tracks found. Duplicate detection is out of the equation. I have many versions of the same songs, I don't want to listen to an autoplaylist with the same track 5 times.  "Top 25 tracks" playlist is a clear example of this problem.
-Exclusions are always manually set: Heavy Metal but not Black Metal and Folk Metal is OK but Folk Metal and Folk-Rock is not, and...
-Can not set playlist lenght.
-Require a crazy amount of work to make it work properly. Yes, you can add more and more and more tags for corner cases and complicate queries... but it makes no sense at some point. I want a system which identifies folk sub-genres and their relations, not having to add a trillion of tags to differentiate them (Instruments used/location specific & Easy listening:); work is done once, etc.
-What about the day you just want to listen to music and not to spend time with all those silly things?

Essentially it's simple. Foobar already has all the ingredients to do things like modern music players, but ironically they have not been mixed yet. Random pools is an example of that, it's development is against that aim of fine-tuning playlists. For ex. it has already been asked to assign pools to playlists names, but that request has been rejected due to being "too complicated".

Even foobar, which already has many title-format variables, omits the current selected track or playing track data. If foobar gives us %currentplaying_genre" or things like that, it would be a joy to create dynamic auto-playlists or random pools according to the selected track. You can not create playlists using other playlist as source, etc. Check MusicBee for those details missing.

We are at a point where machine learning, automatic tagging or music recognition are "simple" and userfriendly, so what I expect about a modern player is applying those concepts locally. I don't like Spotify because it only works via internet. Do you remember Genius from itunes? The same. I want the same things found on those players but working on my PC. I don't want to learn 8000 queries and tags to "teach" foobar to make playlists, I want foobar to show me my local music the same other players do... finding related things automatically. And in the process, to offer many more things.

Re: [SMP] Music Graph development

Reply #6
Just finished making buttons to test it easily. Also for the other functions, with hover text, and ui changing according to preferences.
For that part, I would gladly accept some help rendering the map according to link weighting (i.e. the real precomputed distance) instead of the viva methods by link centrality. That goes beyond my pretty limited knowledge of js and would be the final piece.
And somehow I managed to solve that.

I want to implement fingerprinting comparison too at some point. Using either AcousticID fingerprints or libfooid. To find duplicates, similar track versions, ... but that's currently delayed since my js knowledge is pretty limited right now hahaha If someone points me to JS code to de-structure the fp I would be so grateful!
But that still applies.

Sent you a PM mjm716

Re: [SMP] Music Graph development

Reply #7
Development continues and I have added some new things to the set of scripts.

-Buttons framework to merge/add/create new buttons seamlessly to any SMP bar. That means... you create a button file for whatever you want (sort the playlist for example) and then you simply "include it" on a main file. The main buttons file loads all included buttons, sets position, etc. Zero coding (and no need to edit the other files to merge them). Enabling or disabling buttons just takes 1 line.

-Included some example buttons. An user configurable button within foobar (using properties panel) and 2 new buttons with new functionality (some auto-tagging processing and playlist filtering).

-Updated the HTML rendering file with a few things.... it precomputes the graph before rendering it, pins the node info, etc. Font and label sizes are now set according to window size.

-Added extensive comments over the scripts.

-Added a remove duplicates function which can be used as playlist filtering, setting how many duplicates are allowed... you can filter by date and only allow 3 tracks per date or things like that. And speed improvements (thanks to snotlicker)

Betatesting still required! Thanks :)

I want to implement fingerprinting comparison too at some point. Using either AcousticID fingerprints or libfooid. To find duplicates, similar track versions, ... but that's currently delayed since my js knowledge is pretty limited right now hahaha If someone points me to JS code to de-structure the fp I would be so grateful!
And help required for that yet!

Re: [SMP] Music Graph development

Reply #8
Wow! That's really impressive!
I could not even imagine that such cool stuff would be performed via my component!
Keep it up! ;)

Re: [SMP] Music Graph development

Reply #9
I want to implement fingerprinting comparison too at some point. Using either AcousticID fingerprints or libfooid. To find duplicates, similar track versions, ... but that's currently delayed since my js knowledge is pretty limited right now hahaha If someone points me to JS code to de-structure the fp I would be so grateful!
And help required for that yet!

You would need raw output of fpcalc for the files.

These links may help !

Re: [SMP] Music Graph development

Reply #10
Wow! That's really impressive!
I could not even imagine that such cool stuff would be performed via my component!
Keep it up! ;)
Thanks! Well the html rendering part is done outside foobar with a browser, so it's just using standard js code. I'm not sure if it could be integrated in any way within foobar as a html popup though (?) On github, you told me IE is used for html rendering, so I could try that path once I install it. I don't use fancy things, just standard js, SVG rendering and jquery as a library. It would be awesome: I could do things like showing tracks from the library matching the node selected, playlists by node and "radius", etc.

The graph creation is done with a node.js library, but works within foobar and browsers; I simply had to change a few things on the ngraph library to make it work.

The distance calcs and playlist and all that is done within foobar, your component is fantastic I think any functionality from modern web players can be replicated locally without problems with js. This is just an example. The only limitation I have found is processing time. Spotify has tons of servers calculating these same things! It takes some seconds on our PCs to create a graph or distance calculations for all the library. I even think machine learning models can be paired with your component for auto-tagging or things like that. We just have to see js as a door which allows us to load arbitrary things within foobar and manipulate the library and the playlist with all that info!

You would need raw output of fpcalc for the files.

These links may help !

Let's see if I find a way to calc the raw fp within foobar (so users don't have to analyze things with other programs*) and the comparison logic. Thanks! I will take a look at that.

*Foobar has a component but uses other fp method. Ideally, I would support both.

Re: [SMP] Music Graph development

Reply #11
Some advances:

-Managed to create buttons bar as easy as just include the button file. 1 line.

-Buttons bar can be merged too -> You just include the button bar path, and it's merged the same than standalone buttons.

-Buttons and bars can be duplicated just by including them twice. This is important since they act like different instances, with their own properties at the panel (they get duplicated with prefixes), variables and all that! (other implementations required iDs to be different, properties description adjusted before merging, creating new files for "copies", etc.)

-All these things mixed means... you can just copy the "include" line of a button example 10 times and configure each button within foobar (properties panel) as you wish. In other words, you can create customized buttons using a predefined template without coding a line. You only need to copy/paste paths at the js file and the rest is adjusted on the fly.




In this example, every button does the "search by distance" thing explained on the thread according to different user set variables. That means I don't have to create a predefined set of buttons which creates playlists according to my liking... but the user can create their own in a few seconds.

The great part of the framework is that it's universal. It can be applied to anything, not just my intended use.

-"Similar by" arbitrary tags / number of coincidences. If date is used, then it checks by range instead of number of coincidences. It's a generalization of "search same" functions, user can put whichever tag they want and number of coincidences required for the query. Using it, creates a dynamic query according to the current selected track, and creates a playlist without duplicates (using my remove duplicates function) with random entries from that query. It would be a query equivalent to the "search by distance" implementation. Obviously the query is limited to "equal tags" and simple things while the other uses a graph to find relations between genres/styles and more complex things...

Tooltip is adjusted according to the panel properties and tags set.
Query example created.
Properties panel.

Re: [SMP] Music Graph development

Reply #12
-Remapping and merging tags feature for similarity checks. So not only you can use your own tag names, but also merge different fields into one before evaluating them. This allows us to use the scripts with any tag scheme without needing to re-tag.
"genre tag" -> genre, allmusic_genre, ...

-Added 2 custom tags for checking. There is specific logic for genre, style, date, etc. but you can also add 2 custom tags to be checked by matching (strings) or within a range (numbers) according to their own weights. Why? Let's say some user have it's own "occasion" tag, then it can be used on the search by similarity buttons/functions without changing the code. It's just a placeholder for user set tags, appart from the main ones.

-Some speed improvements (even with the tag merge/remap feature added).

-Filter for the final playlist. Apart from removing duplicates (by title-date-artist, not just duplicated files),  also a filter can be set to only allow X tracks per artist, per date, etc. Fully configurable (tags and number of allowed tracks per check).

Work is mostly done on this thing, I'm now working on a playlist manager which doesn't force you to have playlist loaded within foobar, so you only load whatever you want. (like a library viewer / album list for playlists).

Right now if you have 100 playlist, you must have them loaded on the program, cluttering the playlist tabs... well it makes no sense to me. Other playlist managers alleviate the issue, but you still need to have them all loaded and then "explore" the playlists list using the manager instead of the tabs. Tabs become totally useless and unmanageable:

My approach is different, you have your list of playlist and you simply load the ones you want as new tabs. That way you have the best of both worlds: playlist tab functionality (without cluttering) and a playlist manager. There is something similar for autoplaylists (by saving queries) on SMP, but it's limited to that: auto-playlists.

SimplePortal 1.0.0 RC1 © 2008-2021