Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: How does the Database Store Info (Read 10177 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

How does the Database Store Info

I've just started working on a project similar in some respects to foobar, but different in a lot of others.  I don't want to say to much in case it doesn't go anywhere, but lets just say it has different goals from foobar.

Anyway, I was wondering how foobar stored information in it's database.  More specifically, what data structures does it use?

For example, say you wanted to store tag information.  You need to store the name of the tag (e.g. "Artist"), the data the tag contains (e.g. "Jeff Buckley"), and In my case I want to store another value (e.g. "blah").  Now this could be done with a multi-dimensional array.  In C# this would look like:

Code: [Select]
private string[,] tagInfo = new string[30,3];

In this case, you could store 30 different tags, with 3 collumns in the array.

Here's my (very) crude ASCII table (feature request for next IPB  ):

-------------------------------
Artist | Jeff Buckley | Blah
Title | Halleluiah    | Blah
-------------------------------

And on it goes...

Problems:
1.  Multi-dimensional arrays are slow in C#.  Very slow, at least compared to C++.
2.  No Type Checking as such.  All values have to be strings.  This isn't to bad, but would still require some casting.
3.  Probably about a million other reasons why this is a bad idea.

So my questions:
1.  What sort of data structure does foobar use in it's database?
2.  Is it still suitable if another value needs to be stored?

How does the Database Store Info

Reply #1 –
Tags are stored in a file_info class that implements a map from strings to lists of strings. In versions prior to 0.9 this is obfuscated a bit: the implementation maps strings to strings, but allows duplicate keys.

Storing everything as string is not that bad as that is the usual way to store tags in files anyway, except for binary data which is currently ignored. Even if it is supported in the future, it would not be cached.

How does the Database Store Info

Reply #2 –
So you mean something like (where items listed as -> are the mapped list of strings):

Artist
-> Jeff Buckley

Title
-> Halleluiah

Genre
-> Rock
-> Blues

I don't understand how this can work.  What about custom field names?  I am obviously missing something.  And what if you wanted to store more information related to each field.  For the sake of an example:

Title; Halleluiah; String
Cover; Blah; Binary Data

How does the Database Store Info

Reply #3 –
You are missing the fact that a custom field name is just another string, and as I said foobar2000 does not store binary data in its metadata cache. It might be possible to load such data on demand in the future. You'll have to figure out for yourself, if that way of handling things is suitable for your own application.

How does the Database Store Info

Reply #4 –
The binary data was just an example.  I know foobar doesn't store it in it's cache.

I still don't understand how this works though.  You say there are strings mapped to a list of strings.

What data does the first string point to?
What's data does each string in the mapped list point to?

For example.  Say there's a string called a1, which is mapped to a list of strings called b1 in some kind of hash table.  If a1 stores the field name, then you would need a lot of variables to store each field name and it's appropriate data.  Does that mean there is a maximum number of field names foobar can store in it's cache?

Edit: I'm not trying to discover the secrets of foobar so I can recreate my own version or anything.  It will be quite a long time before I can develop an appliction anywhere near as sophisticated as foobar.  I'm just interested to know how foobar does it.

How does the Database Store Info

Reply #5 –
A key-value pair corresponds to a tag: the key is the tag name, the value is the list of tag values. I thought that would have been clear, but maybe I should have explicitly said that before.

The number of tag names and values foobar2000 can store is only limited by available memory.

How does the Database Store Info

Reply #7 –
I think I'm starting to understand this.

So, to put it visually, there is a two-column table.  Column A contains the key (field name) and column b contains the name of a list (that stores strings, i.e. the field data).  If this is correct (which seems doubtful cause my head seems to be falling apart), is it correct to assume that this table and the list (which must be some sort of array of lists) grow automatically, thus limitless field names are possible?

How does the Database Store Info

Reply #8 –
@kjoonlee
That will be extremely helpful later, thanks very much.

How does the Database Store Info

Reply #9 –
Quote
Tags are stored in a file_info class that implements a map from strings to lists of strings. In versions prior to 0.9 this is obfuscated a bit: the implementation maps strings to strings, but allows duplicate keys.[a href="index.php?act=findpost&pid=319288"][{POST_SNAPBACK}][/a]

Does this mean that prior to 0.9 it was like this:

ARTIST=foo
ARTIST=bar
ARTIST=2000
(edit: somewhat like Vorbis comments)

but in 0.9 it's like this?

ARTIST=foo|bar|2000
(edit: somewhat like APEv2)

I'm making blind guesses, please bear with me

How does the Database Store Info

Reply #10 –
Somewhat like that. The concept can be visualized with pseudocode like this:

Old:
Code: [Select]
struct entry {
   string name;
   string value;
}

entry[] metadata = {
    {"ARTIST", "foo"},
    {"ARTIST", "bar"},
    {"TITLE", "2000"}
};


New:
Code: [Select]
struct entry {
   string name;
   string[] values;
}

entry[] metadata = {
    {"ARTIST", {"foo", "bar"}},
    {"TITLE", {"2000"}}
};


The actual implementation is of course a bit different, but organizes data in the same way.

How does the Database Store Info

Reply #11 –
Cool, thanks foosion.  I think I understand it now.

I think the solution for my program will be to write a seperate class that can store three different values, and then create an array of objects of that class for each tag in the file.