I'm finishing the plans of a 128kbps listening test.
Here are the ideas I have:
- I'll first conduct a listening test of AAC only at 128kbps. After this test is finished, I would take the winner and conduct a general 128kbps listening test, comparing AAC, MPC, MP3, Vorbis, and maybe Atrac3 and/or WMA.
-These are the samples I'm thinking of using:
Samples from ff123's 64kbps listening test:
ATrain.wav
BeautySlept.wav
Blackwater.wav
FloorEssence.wav
Layla.wav
LifeShatters.wav
LisztBMinor.wav
MidnightVoyage.wav
thear1.wav
TheSource.wav
Waiting.wav
BachS1007.wav would be replaced by BWW1011.wav, suggested by GuruBoolez
the following problem cases would be added:
41_30sec.wav
death2.wav
IMO, there are too many samples. I would like to stay at 10 max. So, I'd like to receive suggestions about dropping and/or replacing some of them.
If you want to check them for yourself, they are available at ff123's samples (http://www.ff123.net/samples/) page.
- The encoders and settings tested would be
Psytel AACenc 2.15 -streaming
latest FAAC -a 64
latest Nero - streaming
Sorenson Squeeze 3.5 (FhG Pro) 128kbps
QuickTime 6.2 (Dolby) 128kbps
- Testing setup would be standard ABC/HR, double blind, just like the 64kbps test.
Please, comment/criticize/suggest now or shut up forever.
Thank-you very much.
Regards;
Roberto.
I think this question may be off topic but are you also planning a higher bitrate test as well?
rjamorim,
As you are no doubt aware, Quicktime Pro 6.2 has three quality levels at any given bitrate, "good", "better", and "best". iTunes uses "good", and there is a significant difference betwen "good" and "best" at 128. Other reviews/tests I've seen have been comparing both. I would suggest testing both since iTunes has a high profile and QT Pro "best" has its evangelists.
Thanks for taking the time to create and manage the test.
I think this question may be off topic but are you also planning a higher bitrate test as well?
Maybe, but I'm still wondering if there's any point. Most samples get transparent on 160kbps+, except for golden eared people. And if I limit the test to killer samples, then the results will be useless, since a small amount of people listem to these.
As you are no doubt aware, Quicktime Pro 6.2 has three quality levels at any given bitrate, "good", "better", and "best".
It's the same in QuickTime 6.1, actually. But they are called "low", "medium" and "high"
I plan to use "high"/"best", of course.
iTunes uses "good", and there is a significant difference betwen "good" and "best" at 128. Other reviews/tests I've seen have been comparing both. I would suggest testing both since iTunes has a high profile and QT Pro "best" has its evangelists.
Hrm. I don't know. I know iTunes is the way to go if you plan to perform batch encodings, but people really concerned about quality should go with QuickTime (even if using a tool like AACelerator (http://www.macmethod.com/AACelerator.php)). Biggest problem is that another encode for each sample would make a test that's already big enough even bigger, not to mention it would kill more bandwidth. :-/
I just want to say that listening tests on AAC encoded files are much anticipated (and appreciated!). Zillions of tests have been carried out on mp3s so it's about time..
By the way, I'm cheering for the underdog FAAC! (at least the test might give the developer some hints where to go). I happen to favor small free encoders:)
I think your general concept looks pretty good Roberto, with a mix of problem and real life music samples. I'm not familiar with all the samples listed, but I think you should bias the selection towards real music rather than problem samples.
I question the need for higher bit rates too, as +160 kbits, there are many codecs that approach transparency. 128 kbits is a reasonable area for those with storage limitations, and gives you more room to be able hear differences in how each of the encoders behaves with certain situations.
Bring it on.
Den.
As a MacOS X user, I have to note that there is a big quality difference between the iTunes 4.01 AAC encoding which uses the lowest quality setting of QuickTime Pro 6.2 and the encoding using directly QuickTime Pro 6.2 at the highest quality setting.
Personally, I encode my iPod music using the excellent Ovolab AAChoo front-end for the QuickTime Pro AAC codec that allows the use of the highest quality setting of QuickTime Pro
http://www.ovolab.com/ (http://www.ovolab.com/)
and also automatically updates the iTunes music library after an encoding session. This utility can be very helpful for the AAC tests and I think it is essential for MacOS X + iPod users who want the best available quality for their encodings.