HydrogenAudio

Hydrogenaudio Forum => Uploads => Topic started by: Alex B on 2011-04-12 14:13:21

Title: Multiformat 64 kbps test - result files
Post by: Alex B on 2011-04-12 14:13:21
The related thread: http://www.hydrogenaudio.org/forums/index....showtopic=88023 (http://www.hydrogenaudio.org/forums/index.php?showtopic=88023)

[attachment=6433:Multifor...4_chunky.rar]
EDIT

Fixed the thread link. It pointed to an old test results thread from the year 2007: http://www.hydrogenaudio.org/forums/index....showtopic=56851 (http://www.hydrogenaudio.org/forums/index.php?showtopic=56851). A problem with using chunky was discussed in that thread. Here is how it worked for me: http://www.hydrogenaudio.org/forums/index....st&p=510734 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=56851&view=findpost&p=510734).
Title: Multiformat 64 kbps test - result files
Post by: Alex B on 2011-04-12 20:28:43
Data for the bitrate table.

[attachment=6435:bitrates_xls.zip]
Title: Multiformat 64 kbps test - result files
Post by: _m²_ on 2011-04-12 21:10:18
Reorganized the bitrates spreadsheet, so it's more workable.
EDIT: Got a bug in the file, corrected.
EDIT: Minor improvements.
Title: Multiformat 64 kbps test - result files
Post by: Alex B on 2011-04-12 22:18:08
Thanks for your help.


Here's the finished table:

(http://i224.photobucket.com/albums/dd212/AB2K/ha/bitrate_table.png)

In Excel format:

[attachment=6444:bitrate_table_xls.zip]

Title: Multiformat 64 kbps test - result files
Post by: Jillian on 2011-04-13 06:17:05
Vorbis nearly caught Nero even with 6.9 kbps advantage.
Title: Multiformat 64 kbps test - result files
Post by: IgorC on 2011-04-13 07:00:40
Vorbis nearly caught Nero even with 6.9 kbps advantage.

No, there is no 6.9 kbps.

Average bitrate of tested samples has nothing to do with average bitrate for large collection of music.
Encode a few albums with Aotuv -q0.1 and Nero -q0.245 and you will see that the difference is tiny.

Before the conduction of the test a large number of files were encoded by Aotuv and Nero encoders. The difference between total filesizes was less than 0.5% (a small fraction of 1 kbps)
Title: Multiformat 64 kbps test - result files
Post by: Jillian on 2011-04-13 08:42:28
Vorbis nearly caught Nero even with 6.9 kbps advantage.

No, there is no 6.9 kbps.

Average bitrate of tested samples has nothing to do with average bitrate for large collection of music.
Encode a few albums with Aotuv -q0.1 and Nero -q0.245 and you will see that the difference is tiny.

Before the conduction of the test a large number of files were encoded by Aotuv and Nero encoders. The difference between total filesizes was less than 0.5% (a small fraction of 1 kbps)

But this test is based on 30 samples not music albums you've encoded before test isn't it? Sorry if i'm violated TOS.
Title: Multiformat 64 kbps test - result files
Post by: Garf on 2011-04-13 09:45:46
But this test is based on 30 samples not music albums you've encoded before test isn't it?


No, the bitrate or quality settings used for the test are based on a large set of music, not the samples on the test. Those produce ~64kbps (actually 68kbps, IIRC) files, i.e. identical for all codecs. The analysis of that is in another thread.

So, no codec has an advantage: they all run at 64kbps settings. The fact that some may use a lower or higher bitrate for some samples in the test just means their VBR algorithms differ. I somewhat dislike posting the bitrate table in the results because it confuses people that don't read the explanation; the only people that table is useful for is for the codec developers that might want to improve their VBR in some cases.

If you look at the table more closely, you will see that Vorbis has to "pay back" the large bitrate it uses on some samples by using the lowest rate on others. In the end, the average ends up a bit above 68kbps, but this is because the test samples are only 30 - if we'd have tested an infinite number of samples, all codecs would have averaged exactly 68kbps.

All of this is explained on the test webpage, BTW.
Title: Multiformat 64 kbps test - result files
Post by: Alex B on 2011-04-13 11:58:17
Jillian,

See my post here: http://www.hydrogenaudio.org/forums/index....st&p=751888 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=88023&view=findpost&p=751888)
Title: Multiformat 64 kbps test - result files
Post by: Jillian on 2011-04-13 12:49:16
But this test is based on 30 samples not music albums you've encoded before test isn't it?


No, the bitrate or quality settings used for the test are based on a large set of music, not the samples on the test. Those produce ~64kbps (actually 68kbps, IIRC) files, i.e. identical for all codecs. The analysis of that is in another thread.

So, no codec has an advantage: they all run at 64kbps settings. The fact that some may use a lower or higher bitrate for some samples in the test just means their VBR algorithms differ. I somewhat dislike posting the bitrate table in the results because it confuses people that don't read the explanation; the only people that table is useful for is for the codec developers that might want to improve their VBR in some cases.

If you look at the table more closely, you will see that Vorbis has to "pay back" the large bitrate it uses on some samples by using the lowest rate on others. In the end, the average ends up a bit above 68kbps, but this is because the test samples are only 30 - if we'd have tested an infinite number of samples, all codecs would have averaged exactly 68kbps.

All of this is explained on the test webpage, BTW.

Ok, I get it. I feel bad for Vorbis because of it's unlucky, it get samples that force it's VBR algorithm to use higher bitrate. No advantage here. Thank you for your explanation 

Jillian,

See my post here: http://www.hydrogenaudio.org/forums/index....st&p=751888 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=88023&view=findpost&p=751888)

Thanks you, so I see no problem in real world use here. But I think OPUS comes a little bit late. I doubt if it not just another high performance experimental open source codec that is not intend to used widely.
Title: Multiformat 64 kbps test - result files
Post by: Garf on 2011-04-13 13:19:24
Ok, I get it. I feel bad for Vorbis because of it's unlucky, it get samples that force it's VBR algorithm to use higher bitrate.


I don't think "luck" has anything to do with it - Vorbis has a very good VBR implementation, and correctly recognizes that the test samples are more difficult than average music (that's why they were picked for the test!).

Most probably, if the Vorbis VBR didn't work as well and the bitrates used were lower, Vorbis would have done far worse than it did.

It looks like Nero VBR might actually have failed here. It did not recognize the samples were difficult (unlike Apple), and was consequently beaten (by Apple).

Quote
Thanks you, so I see no problem in real world use here. But I think OPUS comes a little bit late. I doubt if it not just another high performance experimental open source codec that is not intend to used widely.


I don't know. Opus was designed for very-low latency, which is a very different application than the music archival which is more common on HA. I don't think any real good alternatives exist for its application area.

The cool thing (and to me, quite surprising!) appears to be that despite being very-low latency, it's still good, and hence usable for other applications. Whether people find it interesting to use outside that area, I don't know.

But if you need to stream over the internet and patent royalties are a concern, it looks like the prime contender right now.
Title: Multiformat 64 kbps test - result files
Post by: Jillian on 2011-04-13 13:39:01
But if you need to stream over the internet and patent royalties are a concern, it looks like the prime contender right now.

Thanks, so is it time to replace wikipedia audio(vorbis) with opus?
Title: Multiformat 64 kbps test - result files
Post by: Garf on 2011-04-13 13:53:36
But if you need to stream over the internet and patent royalties are a concern, it looks like the prime contender right now.

Thanks, so is it time to replace wikipedia audio(vorbis) with opus?


Vorbis is supported by some HTML5 browsers, whereas Opus support is nonexistant.
Title: Multiformat 64 kbps test - result files
Post by: IgorC on 2011-04-13 21:49:29
Bitrate verification on my set of albums:
(http://i55.tinypic.com/fkbgqa.png)
Title: Multiformat 64 kbps test - result files
Post by: Xanikseo on 2011-04-20 16:33:05
IgorC, in your set of albums, which version of aoTuV did you use to encode to vorbis? I assume that as it is your own collection, you may not always re-encode your collection when new versions of the encoder come out (I certainly don't). I find that aoTuV b6.02 produce files consistently larger than those produced by aotuv b5.7, which may explain why the average bitrate in the listening test is so high. I wouldn't be surprised if b5.7 produce files on average ~64-68kbps, but I would be less certain about b6.02... Then again, it could just be coincidence, that for the times I check, b6.02 tends to produce larger files...

EDIT: I just realised I'm being a bit silly, who listens to 64kbps files for pleasure anyway? You must have used the current version.
Title: Multiformat 64 kbps test - result files
Post by: IgorC on 2011-04-20 18:24:33
I have answered here http://www.hydrogenaudio.org/forums/index....st&p=752847 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=88023&view=findpost&p=752847)