Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Pre-echo: Castanets, --api vs. iTunes AAC @ 320 (Read 15656 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

I've played around with iTunes AAC but ended up settling on ALAC, because I had this nagging feeling that I was hearing pre-echo on very sharp attacks that I did not hear with LAME. Tonight, I fired up Virtual PC (Java ABC/HR is goofy on my machine) and did two quick ABX tests using a castanets sample, encoded by LAME 3.96.1 and iTunes 4.6, both @ 320kbps CBR.

(Edit: re-ran test for 16 trials each. Not done sequentially as to avoid alpha inflation (increased Type I error rate).

The results for AAC @320:

1R = C:\Documents and Settings\Angelo\Desktop\castanets aac.wav

---------------------------------------
General Comments:

---------------------------------------
ABX Results:
Original vs C:\Documents and Settings\Angelo\Desktop\castanets aac.wav
    16 out of 16, pval < 0.001


Now for LAME:


1R = C:\Documents and Settings\Angelo\Desktop\castanets insane 320.wav

---------------------------------------
General Comments:

---------------------------------------
ABX Results:
Original vs C:\Documents and Settings\Angelo\Desktop\castanets insane 320.wav
    11 out of 16, pval = 0.105

This is VERY surprising. Actually, I almost hit significance at the .05 level and somehow lost my concentration or couldn't tell the difference anymore (just like the last time). In any case, this was much harder than the AAC test. So, what's wrong with my ears...aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?

And finally, LAME vs. AAC:

1L = C:\Documents and Settings\Angelo\Desktop\castanets aac.wav

---------------------------------------
General Comments:

---------------------------------------
ABX Results:
Original vs C:\Documents and Settings\Angelo\Desktop\castanets aac.wav
    16 out of 16, pval < 0.001

Results: I could tell the AAC encoding apart from the original, p<.001. I could NOT tell LAME --insane from the original, p>.10. And finally, I could tell the LAME encoding apart from the AAC one, p<.001.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #1
It would be more interesting if you did a 16/16 or more test to be sure Lame was really better.

Quote
aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?

Not all MP3 encoders are created equally.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #2
Quote
It would be more interesting if you did a 16/16 or more test to be sure Lame was really better.

Quote
aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?

Not all MP3 encoders are created equally.
[a href="index.php?act=findpost&pid=229878"][{POST_SNAPBACK}][/a]


I think that can be arranged. I'm assuming you mean 16 trials of each codec compared to the reference? I also might try comparing LAME to the AAC file as well...I've already established that I can hear a difference with the AAC file, but not LAME, so that might be interesting.

 

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #3
Quote
Quote
It would be more interesting if you did a 16/16 or more test to be sure Lame was really better.

Quote
aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?

Not all MP3 encoders are created equally.
[a href="index.php?act=findpost&pid=229878"][{POST_SNAPBACK}][/a]


I think that can be arranged. I'm assuming you mean 16 trials of each codec compared to the reference? I also might try comparing LAME to the AAC file as well...I've already established that I can hear a difference with the AAC file, but not LAME, so that might be interesting.
[a href="index.php?act=findpost&pid=229880"][{POST_SNAPBACK}][/a]

Yes 16 trials, I think it would also be interesting & I would apprecate it, while your at it to chuck Nero's AAC encoder & FhG fastenc -hq.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #4
Ok, have a look at my edited results...significance level increased to p<.001.

By the way, I'm on OS X and don't have a copy of Nero to run on VPC. Does the trial version of Nero come with the AAC encoder?

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #5
Yes.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #6
Quote
So, what's wrong with my ears...aren't I supposed to be able to perceive pre-echo better with MP3 as opposed to AAC?


MP3 encoders, notably LAME with presets, have had much more tuning than the relatively newer AAC encoders. So stuff like this can happen, but it should not be the general case.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #7
Your results surprised me. I did some pre-echo tests in the past with QT AAC encoder, and results were really good. MP3 have inherent flaws, and honestly, I would be surprised if one day one MP3 encoder could achieve this quality, even at 640 kbps. Maybe is latest iTunes encoder broken somewhere? To be sure, I've quickly compared iTunes 4.6 AAC at 320 kbps to lame 3.97a3 at the same bitrate.

Code: [Select]
foo_abx v1.2 report
foobar2000 v0.8.3
2004/07/30 02:31:34

File A: file://C:\ABX\castanets-lame3.97a3-320.mp3
File B: file://C:\ABX\castanets-iTunes320.m4a

02:31:35 : Test started.
02:32:00 : 01/01  50.0%
02:32:08 : 02/02  25.0%
02:32:16 : 03/03  12.5%
02:32:24 : 04/04  6.3%
02:32:31 : 05/05  3.1%
02:32:43 : 06/06  1.6%
02:32:50 : 07/07  0.8%
02:32:54 : 08/08  0.4%
02:32:58 : 09/09  0.2%
02:33:02 : 10/10  0.1%
02:33:07 : 11/11  0.0%
02:33:11 : 12/12  0.0%
02:33:15 : 13/13  0.0%
02:33:20 : 14/14  0.0%
02:33:24 : 15/15  0.0%
02:33:29 : 16/16  0.0%
02:33:33 : 17/17  0.0%
02:33:38 : 18/18  0.0%
02:33:42 : 19/19  0.0%
02:33:47 : 20/20  0.0%
02:33:49 : Test finished.

----------
Total: 20/20 (0.0%)


This test was very easy. The first file (MP3) has serious smearing (first three attacks - the rest is less disturbing). The second one sounds natural to my ears. I'm not sure that I can ABX it against the reference (I'm too tired now). This AAC encoding was very sharp, and I don't ear any other problems.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #8
guruboolez, the best way to compare your results with Cygnus would have been to use lame 3.96 not an alpha.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #9
Quote
Maybe is latest iTunes encoder broken somewhere? To be sure, I've quickly compared iTunes 4.6 AAC at 320 kbps to lame 3.97a3 at the same bitrate.


[a href="index.php?act=findpost&pid=230170"][{POST_SNAPBACK}][/a]


I think that's quite possible. You're using QT 6.5 to encode, right? Also, I used LAME 3.96 as opposed to 3.97a3. I will upload my samples for you...see if my 320kbps AAC file sounds different.

What I was hearing in the AAC file was a smearing/combining of a few "clicks" after the 2.5 second mark, along with a sort of flanging that went from L to R, whereas the original and MP3 file seemed to emanate from the "front" of the channel and the smearing was harder to recognize.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #10
I think I just solved the mystery, and the news is not good.

I was just playing around with samples, using QT instead of iTunes. When played through QT itself, the AAC file exhibits NO artifacts whatsoever. However, for the ABX test, I decoded the AAC back to wav in iTunes. The wav file decoded by iTunes has artifacts that the AAC file does not!!

Worse still, I have AAC files encoded with PsyTel --extreme that play back as though half the frames are missing! They play back fine in Real. Let me run a quick ABX test to make sure that the decoded file and the AAC file really do sound different. If so, this means that iTunes has some problems with AAC decoding! 

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #11
Sure enough, when decoded using iTunes 4.6, the 320kbps AAC file has artifacts that give it away every time, and the one decoded through QT sounds perfect.

Sample A: castanets aac.wav
Sample B: castanets aac iTunes.wav

1 of 1, p = 0.5
2 of 2, p = 0.25
3 of 3, p = 0.125
4 of 4, p = 0.062
5 of 5, p = 0.031
6 of 6, p = 0.015
7 of 7, p = 0.0070
8 of 8, p = 0.0030
9 of 9, p = 0.0010
10 of 10, p = <.001
11 of 11, p = <.001
12 of 12, p = <.001
13 of 13, p = <.001
14 of 14, p = <.001
15 of 15, p = <.001
16 of 16, p = <.001

Looks like I just found a bug in iTunes 

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #12
Can't sleep 
I could confirm the artifact (something electric/metallic in the middle of the castenet's bunch: very obvious). More strangely, the first four attacks (and not three as I said previously) are also unsharp compared to faad2 decoding (foobar2000). Something like pre-echo/hollow sound. It's not very pronounced, but not very hard to hear. I'll ABX it tomorrow to be really sure (I'm between reality and dream on the moment ).

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #13
Quote
I'll ABX it tomorrow to be really sure (I'm between reality and dream on the moment ).
[a href="index.php?act=findpost&pid=230194"][{POST_SNAPBACK}][/a]


Hopefully, you'll not hear castanets in your dreams  I myself have to be especially careful about what I listen to before bedtime!

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #14
It wasn't a hallucination. iTunes decoding has less sharpness than faad2 on the first four attacks of castanets.wav sample. Difference is in my opinon realy obvious (90 seconds only for a 21/21 score - listening range was 0.0 - 1.5). It's the first time I can hear pre-echo introduced by a DEcoder.
Did someone (Menno ) have an explanation? TNS decoding?


Code: [Select]
foo_abx v1.2 report
foobar2000 v0.8.3
2004/07/30 13:32:49

File A: file://C:\ABX\castanets-iTunes320.m4a
File B: file://C:\ABX\castanets-iTunes_decoding.wav

13:32:51 : Test started.
13:32:56 : 01/01  50.0%
13:33:00 : 02/02  25.0%
13:33:04 : 03/03  12.5%
13:33:08 : 04/04  6.3%
13:33:11 : 05/05  3.1%
13:33:16 : 06/06  1.6%
13:33:20 : 07/07  0.8%
13:33:24 : 08/08  0.4%
13:33:27 : 09/09  0.2%
13:33:32 : 10/10  0.1%
13:33:36 : 11/11  0.0%
13:33:39 : 12/12  0.0%
13:33:43 : 13/13  0.0%
13:33:46 : 14/14  0.0%
13:33:50 : 15/15  0.0%
13:33:54 : 16/16  0.0%
13:33:57 : 17/17  0.0%
13:34:01 : 18/18  0.0%
13:34:05 : 19/19  0.0%
13:34:09 : 20/20  0.0%
13:34:14 : 21/21  0.0%
13:34:15 : Test finished.

----------
Total: 21/21 (0.0%)

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #15
People might be interested to note that iTunes is not alone to suffer from this problem. I've just tried on a fresh installed Winamp 5.04 (no additionnal component like faad2), and the castanet.mp4 file exhibits the same problems (lack of sharpness, artifact ["clicks"] in the middle).
More strangely, if the same file is demuxed in a raw .aac form, Winamp 5.04 could play it flawlessly.

Can just say that: many thanks to menno for his great job on faad2.1

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #16
Apple's encoder is, IIRC, originally based on Dolby's. Likely, their decoder is also. The Dolby decoder has known bugs with TNS, and is used in Winamp too, when decoding .MP4.

When winamp decodes .AAC's, it uses the CodingTech decoder which does not have this bug. FAAD does not have this bug either.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #17
Confirmed that iTunes does not produce proper output. It doesn't look like the same bug as the Dolby decoder in Winamp has. The problem here seems to be in the frequencies below 1000 Hz.

Menno

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #18
So, it sounds like the pre-echo problems I thought were attributed to an encoder flaw turned out to be a decoder problem, NOT a problem with QT AAC at 320kbps. I wonder if this occurs at other bitrates as well?

Later today, I can conduct more ABX tests at different bitrates, comparing QT decoded AAC to that of iTunes. Let me know if you think this would be helpful.

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #19
It seems strange that QT would use a different decoder to iTunes...

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #20
Quote
Confirmed that iTunes does not produce proper output. It doesn't look like the same bug as the Dolby decoder in Winamp has. The problem here seems to be in the frequencies below 1000 Hz.

Menno
[a href="index.php?act=findpost&pid=230323"][{POST_SNAPBACK}][/a]



Have you try conducting conformance tests on the iTunes decoder?

wkwai

Pre-echo: Castanets, --api vs. iTunes AAC @ 320

Reply #21
I recently did a clean install of OS X 10.3 in order to go back to QT 6.4 and iTunes 4.2....the decoder problems are gone. Both iTunes encoded files and PsyTel VBR play back without any of the very obvious noise and glitches that I saw in 6.5.1/ iTunes 4.6. I am afraid of upgrading either iTunes or QT until Apple comes out with a version that corrects the decoding AND encoding glitches seen in the latest version.