Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: AAC at 128kbps v2 listening test - FINISHED (Read 63888 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

AAC at 128kbps v2 listening test - FINISHED

Reply #125 –
Quote
Not sure if I have that old Nero encoders anywhere, but there's no doubt that Nero has clearly improved.

Oh, it did improve. It surely got closer to QuickTime.

But, until another test result is posted proving it improved a lot, my test results prove it didn't improve all that much.

AAC at 128kbps v2 listening test - FINISHED

Reply #126 –
Quote
So that warrants allowing a codec to use more bits than allowed by the test setup?


Hmm.. we had this discussion before - and if I remember correctly, in the time of 128 extension test  it was decided to use values for near 128 kb/s on average content, not on test items.

The fact that encoder used more bits on this particular sample set just means that it judged them as "hard to encode"

Of course, you could fine-scale encoder for each sample to even give you 128 kb/s for each sample with VBR - but what's the use of that - unless encoder has 2-pass VBR supported and used by most users?

Quote
What are you trying to say there?


That linear scaling of quality to projected bit rate is dead flat wrong.

AAC at 128kbps v2 listening test - FINISHED

Reply #127 –
Quote
But, until another test result is posted proving it improved a lot, my test results prove it didn't improve all that much.

..for the tested 12 samples... Ideal would be to test something like 120 samples, but this is impossible.
Juha Laaksonheimo

AAC at 128kbps v2 listening test - FINISHED

Reply #128 –
Quote
..for the tested 12 samples...

Right, but at least this is a proof, I'm not breaking rule #8 or anything

besides, I test few samples but with a wide variety of styles and a wide variety of listeners, each one of them with their specific sensibilities to artifacts. IMO that is more representative of current state of codec technology than a test consisting of hundreds or samples and only one or two listeners.

AAC at 128kbps v2 listening test - FINISHED

Reply #129 –
Quote
IMO that is more representative of current state of codec technology than a test consisting of hundreds or samples and only one or two listeners.

I agree and disagree. Group testing is the only way to indicate somekind of average, but 12 samples is way too little to give full picture of codec qualities, it gives somekind of indication.
And eventually there's no objective full picture, it's always subjective.
Juha Laaksonheimo

AAC at 128kbps v2 listening test - FINISHED

Reply #130 –
Quote
The fact that encoder used more bits on this particular sample set just means that it judged them as "hard to encode"

Sure. But other encoders probably also found this sample hard to encode. Still, they behaved and didn't allow bitrate to fluctuate wildly.

Quote
That linear scaling of quality to projected bit rate is dead flat wrong.


Hrm... OK, but I still see no substantial proofs of this. You mention some "JAES" quality scales that can't be published, and some speculations about what would happen at listening tests comparing other bitrates, and there are some claims without proof that Nero got much better.

AAC at 128kbps v2 listening test - FINISHED

Reply #131 –
Quote
But if the average bitrate over hundreds of samples is 131-132kbps, on another batch of samples the bitrates would maybe be lower.

Or lower....
Don't forget that I've launched a long batch encoding (1500 tracks, randomly listed from ~250 CD), and average bitrate was something between 136 and 139 kbps. It was only what people call "classical", but there's a great variety within this genre.


But I also agree that we can't balance final score by bitrate. Especially if the bitrate is the average one for short samples. And especially if the samples were selected with difficulty in mind. In these particular conditions, VBR encoders are always bigger than CBR encoders (VBR+difficulty=high bitrate).
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

AAC at 128kbps v2 listening test - FINISHED

Reply #132 –
Quote
but 12 samples is way too little to give full picture of codec qualities

Of course it is. But it gives a picture. Better than having only personal tests comparing codecs of different versions, with different samples, and with different methodology  - and with results that can't be compared amongst themselves.

AAC at 128kbps v2 listening test - FINISHED

Reply #133 –
Quote
In these particular conditions, VBR encoders are always bigger than CBR encoders (VBR+difficulty=high bitrate).

Nope. Faac and Compaact! are VBR, the settings I used output average 128kbps over a very big amount of tracks, and still they behaved, staying at 128kbps +-4

I need to publish the bitrate deviation table ASAP >_<

AAC at 128kbps v2 listening test - FINISHED

Reply #134 –
Quote
I think Guru can verify this too although only for the kind of music he listens.

Yes, Nero AAC improvments are real. At least with the music I listen to.
But I must add that old Nero AAC at ~128 kbps sounded really bad to my ears, clearly lower than lame mp3 for exemple. Now, Nero AAC is much better, though I could still complain about some problems (don't know what's the exact name of the problem -ringing? noise pumping, distorted background?-, but I also hear it on non-classical samples).

I must add that Nero AAC have some other advantages. I could transcode various lossless & lossy formats through Nero AAC frontend or foobar2000. I have gapless playback with this encoder too.
Quality is a very important thing, but handling & extra-fature are important too.
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

AAC at 128kbps v2 listening test - FINISHED

Reply #135 –
i just wonder: is it possible to calculate what's the possibility that with another 12 samples the results would be completely different?
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung

AAC at 128kbps v2 listening test - FINISHED

Reply #136 –
Quote
i just wonder: is it possible to calculate what's the possibility that with another 12 samples the results would be completely different?

Not at all. Any claim about it would be just wild speculation.

The only way to know for sure would be conducing a listening test with the different samples.

AAC at 128kbps v2 listening test - FINISHED

Reply #137 –
Quote
Quote
In these particular conditions, VBR encoders are always bigger than CBR encoders (VBR+difficulty=high bitrate).

Nope. Faac and Compaact! are VBR, the settings I used output average 128kbps over a very big amount of tracks, and still they behaved, staying at 128kbps +-4

I need to publish the bitrate deviation table ASAP >_<

If you want samples which make Compaact behive badly bitrate wise, I have tons of those. 
It depends quite a lot on the music style really. For this "effect music" which for example Dibrom listens sometimes, Compaact's bitrate usually is sky high.
Same with any track of any style of music with lots of sharp attacks practically.
Juha Laaksonheimo

AAC at 128kbps v2 listening test - FINISHED

Reply #138 –
Quote
Nope. Faac and Compaact! are VBR, the settings I used output average 128kbps over a very big amount of tracks, and still they behaved, staying at 128kbps +-4

I need to publish the bitrate deviation table ASAP >_<

I've also tried to batch encode a lot of tracks (the same than for Nero AAC) with faac -q115. I've stoped at ~300 files this time.
Average bitrate was between 111 and 114 kbps (compare it to the 136-139 kbps I obtained for Nero: not the same class with classical).

I don't have the extreme in mind, but I'm quite sure that the higher bitrate reached 140-150 kbps (but it was rare, probably the solo harpsichord tracks). A lot of files were < to 110 kbps.
It means that Faac is highly responsive too. Maybe too much (that's why I've used ABR on my second classical music listening test).
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

AAC at 128kbps v2 listening test - FINISHED

Reply #139 –
Ok, I used ODG here, but it should be clear what I mean:




This is ODG scaling (si02.wav - hard pre-echo case) from 64 to 256 kb/s for plain-vanilla LC-AAC, no PNS, no nothing, Nero flavour -  and compared with bit rate ratio (64 kb/s is 1.0 and 256 kb/s is 4.0)

You see that curves are nowhere near identical - which means that quality does not scale with bit rate in a linear fashion.

AAC at 128kbps v2 listening test - FINISHED

Reply #140 –
Quote
Average bitrate was between 111 and 114 kbps (compare it to the 136-139 kbps I obtained for Nero: not the same class with classical).

Yes, but classical account for only 2 of the tested samples.

I quote Spoon:

Quote
First Results for FAAC are in, for 2GB of encodings, quality 100 was 5% below and quality 125 was 5% above, I am thinking quality 115 would be spot on the ball.

AAC at 128kbps v2 listening test - FINISHED

Reply #141 –
Quote
What if we slide bit rate to, say, 192 - so, QT would get 6.3 rating, right?

Wrong.

Wrong indeed.  If the target bitrate was 192, and QT hit 192, then its unadjusted and composite ratings would be the same.  If QT, however, encoded to a target of 192kbps at an actual (who knows how) of 128kbps, then yes, it would have a composite rating of 6.3. 

As I stated, I'm sure my concept is flawed, and if so there are two things that need to be done. 

1.  Tell me how it's wrong.  That's been addressed to some extent (though I'm not enough of an expert to know what the answer really means)... and ...

2. Based on the afforementioned demand for this kind of information, propose an alternative system that would tell which codec provides (on average) the best quality at a fixed filesize target.  Concerning this "fixed target", yes, VBR may "not work that way", but hard drives do.  A file size is a file size, and the larger the average filesize, the less encoded music you can keep in a fixed amount of space.  This is what people want to know.  And also, yes, there will be variation, since using quality-managed VBR encoding produces files of various average bitrates.  But a test like this one can give us a scale of averages for the samples tested (the "qualification" I mentioned before).

From among the Eternal Questions of Psychoacoustic Audio Encoding™:  "My quality target is X.  How many files encoded to my quality target with Nero AAC can I put on my 20GB hard drive?  OK, how many files encoded to the same quality target with QT AAC can I put in the same space?"

Is there no system that can be incorporated to answer this essential question so many expert and non-expert music encoders have?

AAC at 128kbps v2 listening test - FINISHED

Reply #142 –
Quote
You see that curves are nowhere near identical - which means that quality does not scale with bit rate in a linear fassion.

OK. It's obviously a curve. But approximations can be done, specially considering the bitrate difference is only (?) 13kbps, not 32 as it would be in 128kbps vs. 160.

AAC at 128kbps v2 listening test - FINISHED

Reply #143 –
Quote
Quote
Average bitrate was between 111 and 114 kbps (compare it to the 136-139 kbps I obtained for Nero: not the same class with classical).

Yes, but classical account for only 2 of the tested samples.

I quote Spoon:

Quote
First Results for FAAC are in, for 2GB of encodings, quality 100 was 5% below and quality 125 was 5% above, I am thinking quality 115 would be spot on the ball.

Yes. I was really interested by spoon results.
What I'm trying to show is that faac also correspond to my previous equation:
In these particular conditions, VBR encoders are always bigger than CBR encoders (VBR+difficulty=high bitrate).
150 kbps for full complex tracks (harpsichord)
100 kbps for low complex tracks (voice, piano...)
and with short samples, values are even more contrasted

On a listening test based on difficult samples, VBR encodings will probably have an average bitrate superior to common encodings.
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

AAC at 128kbps v2 listening test - FINISHED

Reply #144 –
Quote
On a listening test based on difficult samples, VBR encodings will probably have an average bitrate superior to common encodings.

That's OK. But what if, in a listening test based on difficult samples, a VBR encoder manages to behave and stay close to the target bitrate?

AAC at 128kbps v2 listening test - FINISHED

Reply #145 –
Quote
Roberto
OK. It's obviously a curve. But approximations can be done, specially considering the bitrate difference is only 13kbps, not 32 as it would be in 128kbps vs. 160.


Yes, I said that it could be done, but only by making the bit-rate vs. quality curve based on average content for certain encoder -  it can't be linear approximation because your results would be flawed.

Quote
ScorLibran
1. Tell me how it's wrong. That's been addressed to some extent (though I'm not enough of an expert to know what the answer really means)... and ...


Psychoacoustic coders do not scale quality with bit rate in a linear fashion - meaning that you can't just project quality result to some other bit rate by simple proportion.

Quote
From among the Eternal Questions of Psychoacoustic Audio Encoding™: "My quality target is X. How many files encoded to my quality target with Nero AAC can I put on my 20GB hard drive? OK, how many files encoded to the same quality target with QT AAC can I put in the same space?"


Well - we did these tests before listening test, and Nero ended up in 130.5 (correct me if I am wrong) in average - which indicates how much stuf you would be able  store.

Of course, it depends on signal statistics - so if you want to know exactly how much, you'll either have to encode files,  or to use CBR.  Different music has different masking patterns, and if you use 'real' VBR - it is not easy to predict final bit rate without, at least, psychoacoustic processing.

AAC at 128kbps v2 listening test - FINISHED

Reply #146 –
Quote
Quote
You see that curves are nowhere near identical - which means that quality does not scale with bit rate in a linear fassion.

OK. It's obviously a curve. But approximations can be done, specially considering the bitrate difference is only (?) 13kbps, not 32 as it would be in 128kbps vs. 160.

Not anykind of reliable approximation of cbr quality from vbr. The coding methods are so different. CBR has often different kind of issues than VBR, and CBR can avoid undercoding problems which may happen with VBR especially at mid-low bitrates like near 128kbps average. The tweaking of CBR and VBR profiles must be done totally separately.
Juha Laaksonheimo

AAC at 128kbps v2 listening test - FINISHED

Reply #147 –
Quote
But what if, in a listening test based on difficult samples, a VBR encoder manages to behave and stay close to the target bitrate?

Criticism will shut up
But very ponctuel bitrate explosion might help encoders. See MPC --standard: short frames are reaching 700 kbps with castanets.wav
Imagine the average bitrate on micro samples with VBR encodings...
Wavpack Hybrid: one encoder for all scenarios
WavPack -c4.5hx6 (44100Hz & 48000Hz) ≈ 390 kbps + correction file
WavPack -c4hx6 (96000Hz) ≈ 768 kbps + correction file
WavPack -h (SACD & DSD) ≈ 2400 kbps at 2.8224 MHz

AAC at 128kbps v2 listening test - FINISHED

Reply #148 –
Quote
it can't be linear approximation because your results would be flawed.

Hrm... no. They could be slightly imprecise, but considering we are extrapolating few kbps - not tens or hundreds - I doubt the results would be much different than if that codec behaved and output test samples at 128kbps average.

AAC at 128kbps v2 listening test - FINISHED

Reply #149 –
Quote
Not anykind of reliable approximation of cbr quality from vbr. The coding methods are so different. CBR has often different kind of issues than VBR, and CBR can avoid undercoding problems which may happen with VBR especially at mid-low bitrates like near 128kbps average. The tweaking of CBR and VBR profiles must be done totally separately.

We are not trying to approximate CBR from VBR. We are approximating a theoretical VBR mode that stayed at an average bitrate of 128kbps across test samples.