Lame.exe and Nero.exe consume about 5% or less

For Windows users, I made a tiny tool to access the QuickTime AAC encoder from the command-line.
qtaacenc

Quote from: jido link=msg=0 date=

This is Hydrogenaudio, why not using LossyWAV for the high anchor? Let's get the word out!

Since people have been almost flooding this thread with proposals for encoders to be tested in a single test, it's time for me to give my 2 cents.

The question is what we want. Of course you could put all codecs of interest (LossyWav, iTunes CVBR and TVBR, nero 1.3.3 and 1.5.3, CT/Winamp, LAME, WMA, etc.) into one test. But trust me, if you do that, you will get mostly inconclusive results, i.e. waste a lot of listening effort, due to listener overload, as I already explained.

LossyWav's objective is to be transparent. AAC at 96 kbps usually is not transparent, so its objective is to be near-transparent, i.e. "as good as possible". If you want to check whether LossyWav is transparent, do a separate ABX test against the unprocessed original (or maybe an ABX-HR test including AAC at 256 kbps or so, if you want.) Then people can focus on whether the codecs under test really are transparent, without being distracted by at the same time having to evaluate the quality of lower-bit-rate codecs.

A 7-kHz anchor (and 48-kbps LC) should be about mid-way between "very bad quality" and the quality of the 96-kbps LC encoders.

Are you sure 48 kbps LC is not too exaggerated? Do you expect any contender to be worse or as bad as 64 kbps?

The idea was not to test LossyWAV transparency, but to download smaller files. BTW that could also apply to the proposed 7 kHz-filtered low anchor.

For Windows users, I made a tiny tool to access the QuickTime AAC encoder from the command-line.
qtaacenc

...
Agreed. But for completeness sake: Fraunhofer is currently finalizing quality tunings on their encoder which have been going on for about two years. Release is scheduled for end of January. If there is any interest, I can ask if it's possible to provide an evaluation encoder for this test.

Chris

dBpowerAMP reports the bitrate as being 0kbps.

Can you specify also samplerate because all encoded files are 32 khz?

I might have missed something, but where to get the Fraunhofer encoder?

Maybe you better should've offered it in an own thread instead of hiding it in this "public listening test" thread.

... we should now focus onto what encoder knows less killer samples. A listening test in its traditional form is not suited to evaluate that. The limited, initial choice of test samples will determine the outcome. ...

Quote from: rpp3po on 2010-01-08 15:57:52

... we should now focus onto what encoder knows less killer samples. A listening test in its traditional form is not suited to evaluate that. The limited, initial choice of test samples will determine the outcome. ...

Yes, the value of the outcome of such listening tests is often rated too high IMO. Especially at such a pretty low bitrate. Especially as many people just take the average score of each encoder to get a quality order for the encoders.
But: Despite this listening tests do have a value IMO, though a restricted one.

As for your approach: this would give valuable information, though restricted information as well because the universe of killer samples is not known.

So, rpp3po, I don't understand what you mean by "A listening test in its traditional form is not suited to evaluate that". I think it is. You just need to know the abovementioned categories.

And the more 'emesesk' samples the set includes the worse AAC will look in comparison to MP3 and vice versa.

You can intentionally exclude problematic samples from the test, but what are you going to get then? Good results for all encoders for most music. But we already know that. My point is, isn't it time to move on with testing to compare which encoders have the lowest probability of failing over broad collections of music, even if that would be harder to accomplish than with traditional 8-16 sample shootouts?

A question from the peanut gallery.

Do different codecs have specifiably different types of killer samples?

My thought is that we know that a number of good codecs give excellent results for most music, but it might be possible, perhaps, to give specific recommendations matching codec to genre (codec A is good for harpsichord, for example, but falls over on distorted guitars, on which codec B does better).

IF this were possible, it might be more practically useful for listeners than another very tight general comparison of good codecs over "average" music.

Edit: punctuation

The encoder with the least problematic samples in the set wins. You can intentionally exclude problematic samples from the test, but what are you going to get then? Good results for all encoders for most music. But we already know that. My point is, isn't it time to move on with testing to compare which encoders have the lowest probability of failing over broad collections of music, even if that would be harder to accomplish than with traditional 8-16 sample shootouts?

For the individual developer, who knows its encoder inside out, a "traditional" listening test with select samples can be of great value, no question.

[...I'm in favor to rise bitrate to 128 kbps and include more difficult samples randomly (and don't include samples which are only difficult for one specific competitor). Emese-like samples are horrible at this bitrate.

Quote from: rpp3po on 2010-01-08 21:14:35

The encoder with the least problematic samples in the set wins. You can intentionally exclude problematic samples from the test, but what are you going to get then? Good results for all encoders for most music. But we already know that. My point is, isn't it time to move on with testing to compare which encoders have the lowest probability of failing over broad collections of music, even if that would be harder to accomplish than with traditional 8-16 sample shootouts?

For the individual developer, who knows its encoder inside out, a "traditional" listening test with select samples can be of great value, no question.

It's just different approach. Good one actually.

Somebody would say that testing killer samples wouldn't be representative but the same way light samples aren't representative neither.

I'm in favor to rise bitrate to 128 kbps and include more difficult samples randomly (and don't include samples which are only difficult for one specific competitor). Emese-like samples are horrible at this bitrate.