Request for a listening test

Reply #25 – 2004-02-06 13:55:58

Quote

This just means "which codec is more safe to use".

Well said

Request for a listening test

Reply #26 – 2004-02-06 14:08:15

I'm tempted to perform something like a ~200 kbps listening test. Not with killer-samples (as Garf said, there are not representative at all), with just with musical/common one. Recently, I've tried to oppose mpc --standard, vorbis GT3b1 Q6 and iTunes AAC 192 kbps (I haven't the latest Nero builds anymore) on a nice part of a Vivaldi concerto. I was able to ABX two of the three files: vorbis (added noise) and iTunes AAC (distorted sound). I've tried many times with mpc standard, without succes... It was only one sample, but it was the first I've selected. With some chances, a whole test is possible.

Lossy encoding for archiving purpose isn't critical anymore for my own purpose: I've HDD space enough for lossless, for ~1000 CD. I'm just curious. There are now promising encoders:
· wmaPRO
· AAC (iTunes & Nero)
· Vorbis
· mpc
and hybrid for very high bitrate. It's exciting

Request for a listening test

Reply #27 – 2004-02-06 15:42:28

Quote

Quote

So as soon as you provide, say, ABX results proving you (or anyone else, let's not be nitpicking) can actually hear the difference between a reference, an mpc q7, lame ape and aac at around 200kbps or whatever else you're requesting, then discussion can begin.

Even for MPC this has been done before, see for instance this thread. There are also existing problems for LAME alt-presets, various AAC implementations and for Ogg Vorbis even at bitrates higher than 200 kbps.

Yes, I know there are problem samples for any codec, but being able to ABX one specific encoder against the reference on some specific sample is far from a multi-format comparison.

But seriously, I think there are some misconceptions which lead to the pretty frequent requests of high-bitrate tests.

Most importantly I believe, modern audio encoders are much better than most people think. As Roberto's previous multiformat listening test showed, and most people who ever tried an ABX can attest, modern formats are pretty astonishing at bitrates as "low" as 128 kbit/s. The general doubt towards the capabilities of codecs at that bitrate is probably due to a kind of "boomerang" reaction from the discovery that 128 kbit/s mp3s are in fact not as good as it gets in terms of transparency (which seemed to be a general consensus among the masses for some time). Nowadays the trend definitely goes towards 192kbit/s as the new "128" but we shouldn't forget that modern formats today are a hell of a lot better than early mp3s.

What I tried to suggest with my previous post is that most of the people who root for high bitrate listening tests probably haven't tried an ABX at such bitrates for some time. Because, if they did, they'd probably realize that it's asking much more from the participants than could be hoped for in a public listening test (and I don't mean to discredit the more experienced users on this board, who have tried seriously to make plans about such tests, of course; but just proposing such a test and hoping everything would work out from there is a bit naive, to say the least).

Another thing I'd find difficult in a "golden ear" test is the doubtful validity of extrapolating the results. Interpreting the results of a listening test will always involve a fair bit of extrapolation (for example the results of such a test are usually taken as a statement about general quality, not as a statement about this particular set of 12 samples). At low bitrates this is not as problematic. There are lots of artifacts virtually everyone is able to hear, so if a majority of participants say something sounds "metallic", there's a good chance you'll be able to hear it, too. This is not the case in a high bitrate test. If you tested two or three samples of a particular genre with some codec at a low bitrate, and you can hear specific artifacts, chances are those artifacts will also be present at most other, similar samples. At high bitrates, on the other hand, you have to be much more careful about such statements.
Thus, you'd either need to have lots and lots of participants (as Roberto said), or you'd have to be really careful in concluding anything from the results.

In the end I suppose the current method of doing public listening tests at reasonable bitrates combined with the conlusions we can draw from individual problem samples and individual listening tests by some of the restless "golden ears" around here (you know who I'm looking at ), is probably more useful than anything a public high bitrate test would produce.

Request for a listening test

Reply #28 – 2004-02-07 06:16:29

Quote

You should cut and paste your usual response, every time someone says there was a winner in that test.

http://www.rjamorim.com/test/128extension/results.html

I had to hold real hard to avoid being blunt there :B

Request for a listening test

Reply #29 – 2004-02-07 15:12:37

Schnofler wrote:

> Nowadays the trend definitely goes towards 192kbit/s as the new "128" but we shouldn't forget that modern formats today are a hell of a lot better than early mp3s.

This is an interesting opinion (please read the original message for the whole statement).
I think that a 192Kbps test would have a follow. As pointed in the 128Kbps extension test (http://www.rjamorim.com/test/128extension/results.html) the competition at 128Kbps is non existant because modern codecs perform the same way. However, the differences and the artifacts perceived by the listeners in that test could be the ground for this other listening test.

Guruboolez wrote:

> Recently, I've tried to oppose mpc --standard, vorbis GT3b1 Q6 and iTunes AAC 192 kbps (I haven't the latest Nero builds anymore) on a nice part of a Vivaldi concerto. I was able to ABX two of the three files: vorbis (added noise) and iTunes AAC (distorted sound). I've tried many times with mpc standard, without succes... It was only one sample, but it was the first I've selected. With some chances, a whole test is possible.

As I have written, someone has already successfully ABXed at that bitrate and found some answers.

I change the target of my proposal to from MPC Q7 vs. the rest of the world to a "simple" 192Kbps listening test (MPC Q6).

Hecatombles

Request for a listening test

Reply #30 – 2004-02-07 15:14:37

Quote

Dream on.

quoted for truth

Notice