Over three years ago I tested the LAME 3.97b1 VBR bitrates (link (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=37011&view=findpost&p=328558)). For that test I gathered a set of 25 various "reference" tracks. The tracks were specifically selected so that they could more or less represent an average large media library of pop/rock/jazz etc genres. Since then the same set has served well in many occasions. During the preparations of the new public 128 kbps VBR MP3 listening test I gathered another set of classical tracks.
Now I have tested all LAME 3.98.2 VBR settings (in 0.5 step increments) using these two reference sets. For this test I encoded 1050 MP3 files (21x50). I analyzed the files with EncSpot Pro v.2.1 and prosessed the exported data with Excel.
Here are the results:
Some general details: (the frequency values are kHz)
(http://i238.photobucket.com/albums/ff132/alexb2k/HA/properties.png)
Bitrates "Various":
(http://i238.photobucket.com/albums/ff132/alexb2k/HA/various.png)
Bitrates "Classical":
(http://i238.photobucket.com/albums/ff132/alexb2k/HA/classical.png)
Average Bitrates:
(http://i238.photobucket.com/albums/ff132/alexb2k/HA/bitrates.png)
(http://i238.photobucket.com/albums/ff132/alexb2k/HA/chart.png)
The test tracks:
(http://i238.photobucket.com/albums/ff132/alexb2k/HA/tracks.png)
EDIT:
The classical track #4 has a slightly incorrect name in the table. It should be "Berlin Radio Symphony Orchestra - Mahler, Symphony No 8, 2. Ewiger Wonnebrand".
The source data is available here: http://www.hydrogenaudio.org/forums/index....st&p=601002 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=67522&view=findpost&p=601002) (Excel sheet)
Excellent and useful work. This table should appear on the wiki.
On my own classical music test set (bigger: 150 tracks from different discs / 16 hours of music), results on the few settings I tested before are very similar:
-V5: 133.55 kbps (+2.7 kbps)
-V4.5: 145.36kbps (+2.8 kbps)
Thanks for sharing your helpfull information. The data presented together with my personal listening experience is that V3 is a very nice spot concerning quality versus quantity (lame 3.97 and 3.98).
I wonder what the plot (or chart) would look like adding the -Y switch above -V3
Thank you for this interesting bitrate table. Very useful and helpful for the community.
Nice presentation too.
On my own classical music test set (bigger: 150 tracks from different discs / 16 hours of music), results on the few settings I tested before are very similar:
-V5: 133.55 kbps (+2.7 kbps)
-V4.5: 145.36kbps (+2.8 kbps)
Interestingly the average values that EncSpot reported directly are even closer to your values:
Classical -V5 133.1 kbps
Classical -V4.5 144.7kbps
I noticed that the average values calculated by Excel always differed slightly from the EncSpot averages. Probably that is caused by rounding. EncSpot may internally use greater precision than it shows for individual tracks. I could have used the EncSpot averages, but it would have looked odd if the average values could not have been verified by calculating them from the given track values.
In any case this kind of a test can only give a rough estimation, so the exact precision does not matter that much.
I wonder what the plot (or chart) would look like adding the -Y switch above -V3
It would be more useful to test that with tracks that really exploit the SFB21 bitrate bloat problem. I avoided the most extreme tracks when I gathered the test sets. Perhaps someone who has a nice media library of Metal/Industrial/etc genres could do the comparison.
I wonder what the plot (or chart) would look like adding the -Y switch above -V3
It would be more useful to test that with tracks that really exploit the SFB21 bitrate bloat problem. I avoided the most extreme tracks when I gathered the test sets. Perhaps someone who has a nice media library of Metal/Industrial/etc genres could do the comparison.
I have plenty of tracks that suffer from the sfb21 bloat problem.
I noticed that the average values calculated by Excel always differed slightly from the EncSpot averages. Probably that is caused by rounding. EncSpot may internally use greater precision than it shows for individual tracks. I could have used the EncSpot averages, but it would have looked odd if the average values could not have been verified by calculating them from the given track values.
I'd guess that EncSpot takes into account padding or maybe the id3 buffer space, and just calculates the bitrate on actual audio data. I'm probably wrong though
I saw that the Evanescence track suffered from the bitrate bloat. Both /mnt and I have plenty of tracks that Lame does this with. I can suggest nearly anything by Lamb Of God, All That Remains, Hatebreed, or In Flames. There are also some Nine Inch Nails tracks off of [With_Teeth] and Year Zero that get bloated.
And for some reason Powerslave by Iron Maiden really suffers from sbf21 bloating, like it was a very brutal Death Metal album.
Actually I really would not like to start testing these things again. It was a bit exhausting experience. So could someone else test the -Y switch?
In general it might be a good idea to publish a separate metal/industrial/punk/etc (you name it) table in which the -V0 bitrates would probably be near 300 kbps.
Regarding to the bitrate diffrences between the EncSpot calculated and Excel calculated average values I dont understand how the difference can be more than 0.5 kbps if it is caused by rounding. A difference of 0.5 kbps could be possible in the worst case scenario when all tracks happen to be internally something like xxx.499 kbps when EncSpot displays & exports rounded integer numbers (which I used in my Excel table). Maybe EncSpot is buggy afterall.
I updated the bitrate estimates on the LAME (http://wiki.hydrogenaudio.org/index.php?title=LAME) page a bit, according to your results.
I have made a table of 29 sfb21 bloated tracks at V3 - V0 and the -Y switch aswell.
(http://img224.imageshack.us/img224/7485/sfb21bitratetablerr1.png)
A zipped Excel document version can be found here. (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=4755)
Great info guys! the ~60kbps savings with -Y are impressive, thanks for taking the time to make these tests
I noticed that the average values calculated by Excel always differed slightly from the EncSpot averages. Probably that is caused by rounding. EncSpot may internally use greater precision than it shows for individual tracks. I could have used the EncSpot averages, but it would have looked odd if the average values could not have been verified by calculating them from the given track values.
Perhaps encspot uses a weighted average, weighted on the track durations that is?
Thanks tosse. I did some tests and I think you are right. I should have thought about that.
When EncSpot calculates the track specific average bitrates it naturally takes the amount of each differently sized frame into account. Logically and practically it is correct to do the same when the average bitrate of a bunch of files is calculated.
If there is a 30 s 80 kbps file and a 2 min 320 kbps file EncSpot correctly reports that the average bitrate is 272 kbps instead of 200 kbps.
I'll add the EncSpot calculated average bitrates to my test results.
EDIT
On a second thought, for the purpose of this test, I am not interested about the mathematically correct disk space usage / audio duration ratio of my complete test file sets.
I am trying find out how the encoder behaves with various kinds of source material. In that sense each source track should probably be weighted equally. For instance, I may have a difficult to encode shorter track that represents a certain type of source material and an easy to encode longer track that represents another type of source material.
In any case this is not going to be an exact science.
Hello,
A bit surprised by the low target bitrate of the -V5 setting in the wiki, which is 130 kbps, I also ran a small test in order to check Lame 3.98.2 bitrates.
I chose 10 tracks from various styles. It was before finding this thread, and I got results very similar to yours, exept for a small difference for the -V2 setting.
Our results are indeed a bit different from what is given in the wiki.
V6 V5 V4 V3 V2
1 110.4 120.4 142.9 156.8 170.7
2 114.9 131.5 150.9 164.4 181.1
3 112.5 124.6 154.8 174.3 192.2
4 124 136.6 162.5 177.2 191.6
5 121.8 138.6 156.6 171 184
6 135.8 151.5 167.4 181.5 201.4
7 129.8 143.4 160 174.9 210
8 129.5 143.4 157.2 171.4 239.6
9 127.4 141.1 155.2 169 200.5
10 125.5 139.1 152.4 166.2 229.8
-----------------------------------------------
Avg 123.16 137.02 155.99 170.67 200.09
The tracks are :
1 : Ashram - All'Imbruinire (Soft piano and voice)
2 : Grieg - Peer Gynt - I Dovregubbens Hall, Neeme Järvi (Classical, symphonic, extremely dynamic)
3 : Graduel d'Aliénor de Bretagne - Kyrie, Ensemble Organum (Classical, choir)
4 : Faun - Isis (folk)
5 : Princess Mononoke soundtrack - The Journey to the West (orchestral soundtrack)
6 : Kraftwerk - Trans Europ Express / Metal on Metal / Franz Shubert (electro pop, 1977)
7 : Madonna - Hung Up (mainstream pop, 2005)
8 : Nightwish - Whishmaster (metal)
9 : Talk Talk - Such a Shame (new wave pop, 1984)
10 : Within Temptation - What Have You Done (clipressed pop)
On my collection of 12579 files I got an average bitrate of 228.23 kbps with -V 0 and 144.31 kbps with -V 4. That is with track length not weighed in and all lame 3.98.2. The flac source files are at 759.02 kbps. I guess the classical and spoken word brings it down that far (about 2000 files). Plus not that much loud metal, hard rock, etc.