Public Listening Test [2010]
Reply #37 – 2010-01-03 16:34:23
I am not a statistician -- and certainly not an applied one -- so it might very well be that this is a fair compromise to do to eliminate at least some randomguessers and also those who are able to tell differences but actually prefer the compression artifacts. Yes, that's the basic idea. If a listener grades consistent with the majority of non-post-screened listeners, there is no way to tell from the results whether (s)he guessed or actually heard a difference. So you have to include that listener.This is Hydrogenaudio, why not using LossyWAV for the high anchor? Let's get the word out! Since people have been almost flooding this thread with proposals for encoders to be tested in a single test , it's time for me to give my 2 cents. The question is what we want. Of course you could put all codecs of interest (LossyWav, iTunes CVBR and TVBR, nero 1.3.3 and 1.5.3, CT/Winamp, LAME, WMA, etc.) into one test. But trust me, if you do that, you will get mostly inconclusive results, i.e. waste a lot of listening effort, due to listener overload, as I already explained. LossyWav's objective is to be transparent. AAC at 96 kbps usually is not transparent, so its objective is to be near-transparent, i.e. "as good as possible". If you want to check whether LossyWav is transparent, do a separate ABX test against the unprocessed original (or maybe an ABX-HR test including AAC at 256 kbps or so, if you want.) Then people can focus on whether the codecs under test really are transparent, without being distracted by at the same time having to evaluate the quality of lower-bit-rate codecs. If you want to check whether there's a statistically significant improvement of iTunes TVBR over CVBR, propose or conduct a separate public ABX-HR or MUSHRA test for those two encoders. Then people can focus on sonic differences (if any) between those two encoders. The same applies to nero 1.5.3 vs. 1.3.3.Then, once you finished those last two tests , you can take the "winners" of those tests and conduct the test which we are promoting (under the title "AAC test") in this thread. Yes, it's a lot of work, but it's the only way to get meaningful results.If you don't want to do those last two tests , just choose one encoder from each test based on certain non-quality considerations (e.g. nero 1.5.3 because it's a newer release, iTunes CVBR because it's also available for Windows). This should be fine since most likely, there are only minor sonic differences at 96 kbps between nero 1.3.3 and 1.5.3 and between iTunes CVBR and TVBR. You can always do said tests later, of course also with the exact same test material. Chris