Skip to main content
Topic: New Public Multiformat Listening Test (Jan 2014) (Read 81742 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

New Public Multiformat Listening Test (Jan 2014)

Reply #275
lvqcl is right. It's 1000.

Gecko, can You recalculate it again? Thanks.

New Public Multiformat Listening Test (Jan 2014)

Reply #276

I will ask people also report their bitrate on Vorbis -q2.2.

If we will go with plan B (~ 100 kbps) then we have a fair set of settings:
apple CVBR 96 - 101.1 kpbs
Opus 96 - 100.9 kbps
Vorbis - somewhere near to -q2.2 (?)

P.S. I'm updating the same table here

New Public Multiformat Listening Test (Jan 2014)

Reply #277
@Kamedo2:
My test set consists of 24 tracks which I chose from old (pre-loudness war) pop music as well as new one, of rather 'hard and wild' pop music as well as rather slow ballads.

My preferred settings are Apple cvbr 96, Opus bitrate 96,  aoTuV q2.
lame3995o -Q1.7
opus --bitrate 140

New Public Multiformat Listening Test (Jan 2014)

Reply #278
Thanks lvqcl, IgorC. Fixed!

Does using 2^10 for kilo in an IT context make me a nerd?


New Public Multiformat Listening Test (Jan 2014)

Reply #280
Update.
Link to table.



New Public Multiformat Listening Test (Jan 2014)

Reply #281
Vorbis aoTuV [20110424], q2.2: 99.6

New Public Multiformat Listening Test (Jan 2014)

Reply #282
aotuv -q 2,2: 100.9 kbps

New Public Multiformat Listening Test (Jan 2014)

Reply #283
Link


New Public Multiformat Listening Test (Jan 2014)

Reply #285
Based on that table so far I would go with:
Apple TVBR 45 | ~94.9Kbps
Opus 92 | ~96.6Kbps
Vorbis q1.99 | ~91.7Kbps

OR

Apple CVBR 96 | ~101.1Kbps
Opus 97 | ~101.8Kbps
Vorbis q2.4 | ~102.7Kbps
,but at these bitrates the tests might be too difficult?

New Public Multiformat Listening Test (Jan 2014)

Reply #286
Based on that table so far I would go with:
Apple TVBR 45 | ~94.9Kbps
Opus 92 | ~96.6Kbps
Vorbis q1.99 | ~91.7Kbps

I'm against of this set.
It's possible to low bitrate for Opus, like --bitrate 89. But there still a significant  bitrate advantage for TVBR (+3.5%). No go.
Let's wait for Kamedo2 and Kohlrabi to submit their rates for Vorbis -q2.2


It's more like
Apple CVBR 96 - 101.1 kbps
Opus 96 - 100.8 kbps
Vorbis -q2.2 - ~101 kbps (?)

New Public Multiformat Listening Test (Jan 2014)

Reply #287
Sorry,  I still think that not equal averages of bitrates for selected sound  samples is a flaw of the test, because it means that this combination  of settings and samples favors one codecs and is unlucky for the  others. And the only reasonable excuse for this situation could be the  use of natural (integer) settings. The  explanation that those "equal" settings are calculated using some  big music library is not valid because we do not use that big library  in our test, we do not use even its representative sub-set, we use a  limited set of some marginal samples which have nothing in common  with that big library. So, why should we set codecs with one samples  and test them with another?

keeping audio clear together - soundexpert.org

New Public Multiformat Listening Test (Jan 2014)

Reply #288
So basically you think that VBR is a useless thing. Right?

New Public Multiformat Listening Test (Jan 2014)

Reply #289
IMO the average bitrate of an encoder setting should be taken from a test set. This is a general strategy which has nothing to do with a specific listening test.
lame3995o -Q1.7
opus --bitrate 140

New Public Multiformat Listening Test (Jan 2014)

Reply #290
So an encoder with good VBR algorithm isn't better than an encoder with bad VBR algorithm?

New Public Multiformat Listening Test (Jan 2014)

Reply #291
IMO the average bitrate of an encoder setting should be taken from a test set. This is a general strategy which has nothing to do with a specific listening test.

I'm not sure I 100% understand this but I think I am with you here. I believe the test, if ~96 kbps is chosen, must be done with the setting the developer or encoder gives us.

For example:
AAC-LC Apple CVBR 96 (~96 kbps) or TVBR 36 (~95 kbps)
AAC-LC Fraunhofer/fdk VBR 3 (~96-112 kbps)
Ogg Vorbis Q2 (~96-112 kbps)
Opus VBR 96 (~96 kbps)

If the bitrates are lower or higher I think it means the encoder doesn't need more in the first case or it does in the second. It depends how the developer tunes the encoder IMO.

If we are looking for a precise bitrate let's just use CBR or ABR.

New Public Multiformat Listening Test (Jan 2014)

Reply #292
IMO the average bitrate of an encoder setting should be taken from a test set. ... .

Why are You saying it now and not a few days ago?


It's contrary what You've said a few days ago:
I prefer using a test set of hopefully representative music for deciding upon which settings to use.
And from the various results given here it looks like Apple --cvbr 96, Opus --bitrate 96, aoTuv -q2 give these contenders equal chances (maybe with a tiny bit stronger aoTuv setting). These are 'natural' choices moreover.

How do You expect people should get your posts if You quickly change your mind?  Huh?


...This is a general strategy which has nothing to do with a specific listening test

This is a general ... what?
How many times we've heared You saying  "bitrate increase on some particular hard sample/samples, but no significant bitrate increase overall" in your LAME extension threads?
And now what You suggest goes completely contrary of yourself.



@ Serge Smirnoff
Robert has crystal clearly answered your question here
There is nothing to add.

New Public Multiformat Listening Test (Jan 2014)

Reply #293
I prefer using a test set of hopefully representative music for deciding upon which settings to use.
And from the various results given here it looks like Apple --cvbr 96, Opus --bitrate 96, aoTuv -q2 give these contenders equal chances (maybe with a tiny bit stronger aoTuv setting). These are 'natural' choices moreover.


I'm not sure this is a good idea. If the last big 96kbps test consisted of challenging or "killer" samples, it was already quite difficult to detect fault. Simpler samples may make the test longer and more tedious, as well as making the opportunity to detect flaws less frequent.

New Public Multiformat Listening Test (Jan 2014)

Reply #294
So basically you think that VBR is a useless thing. Right?

No, I think that we should use different set of samples for choosing vbr settings, the samples that are actually participate in test, not some external. Encoder will choose how to distribute bitrate among those samples, so there is enough room to test the efficiency of vbr algorithm.
keeping audio clear together - soundexpert.org

New Public Multiformat Listening Test (Jan 2014)

Reply #295
@ Serge Smirnoff
Robert has crystal clearly answered your question here
There is nothing to add.

I saw. I don't agree with this answer. And that's why I posted my opposite opinion with supporting arguments.
keeping audio clear together - soundexpert.org


New Public Multiformat Listening Test (Jan 2014)

Reply #297
so there is enough room to test the efficiency of vbr algorithm.

I don't think so.

What is your reasoning?

If  some vbr algorithm chooses lower/higher bitrates for our test set, what does it mean, is it more smarter or less smarter, more efficient or less efficient?

In other words, how non-equal overall bitrates of codecs with some test set helps to reveal effectiveness of vbr algorithms?
keeping audio clear together - soundexpert.org

New Public Multiformat Listening Test (Jan 2014)

Reply #298
These are just my opinions:
  • I prefer bitrate settings to be determined on as large and as varied a corpus of music as possible.
  • I prefer using floating point quality settings to get as close as possible to 96kbps for the large corpus of music.
  • I prefer allowing the test samples to include a disproportionately large number of challenging samples.
  • I am okay with encoders using unconstrained VBR resulting in test sample bitrates which differ significantly from the larger corpus average bitrate.


New Public Multiformat Listening Test (Jan 2014)

Reply #299
The average bitrate of my library, 122 tracks, speed (x realtime)
88547 27.1 venc603(aoTuV) -q1.9 %i %o
88957 28.3 venc603(aoTuV) -q1.95 %i %o
89326 26.3 venc603(aoTuV) -q1.99 %i %o
89417 27.2 venc603(aoTuV) -q1.9999 %i %o
95359 29.2 venc603(aoTuV) -q2 %i %o
96169 26.1 venc603(aoTuV) -q2.1 %i %o
97673 25.9 venc603(aoTuV) -q2.2 %i %o
99763 27.7 venc603(aoTuV) -q2.4 %i %o

 
SimplePortal 1.0.0 RC1 © 2008-2019