Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Nero 48 kbps HE-AAC (+v2) test - FINISHED! (Read 19686 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Nero 48 kbps HE-AAC (+v2) test - FINISHED!

Okay, so results of the Nero Listening test are here:

Test1: HE-AAC v1 @48 kbps

VBR1  - 3.56
ABR 48  - 3.53
CBR 48 - 3.48
VBR2  - 3.29
MP3 128 - 4.41
MP3 48 - 1.19




Test2: HE-AAC v2 @48 kbps

VBR1  - 3.12
ABR 48  - 3.11
CBR 48 - 3.11
VBR2  - 3.08
MP3 128 - 4.08
MP3 48 - 1.14

Nero 48 kbps HE-AAC (+v2) test - FINISHED!

Reply #1 –
And... the test keys (attached with this post)

Codec IDs:

Quote
TEST 1 - HE TEST

Codec A - ABR 48 kbps
CODEC B - VBR1 - quality factor 0.15
CODEC C - VBR2 - quality factor 0.37
CODEC D - CBR 48 kbps
CODEC E - MP3 128 kbps
CODEC F - MP3 48 kbps

TEST 2 - HE v2 TEST

Codec A - ABR 48 kbps
CODEC B - VBR1 - quality factor 0.32
CODEC C - VBR2 - quality factor 0.54
CODEC D - CBR 48 kbps
CODEC E - MP3 128 kbps
CODEC F - MP3 48 kbps

Nero 48 kbps HE-AAC (+v2) test - FINISHED!

Reply #2 –
I tried only 2x3 samples (sorry, I didn't have more time) and got these results:

Code: [Select]
Test1    Sandman      Sympathy     Harp40_1     Average
A         2.00         1.70         2.70         2.13
B         2.40         1.90         2.70         2.33
C         2.20         1.90         2.00         2.03
D         2.00         1.50         2.30         1.93
E         4.30         4.50         1.40         3.40
F         1.00         1.00         1.00         1.00

Test2    Sandman      Sympathy     Harp40_1     Average
A         2.60         2.10         3.50         2.73
B         2.80         1.80         3.50         2.70
C         2.20         2.00         3.50         2.57
D         2.20         2.10         2.90         2.40
E         4.30         4.50         1.40         3.40
F         1.00         1.00         1.00         1.00


I would say that all contenders were pretty much tied with the Sandman and Sympathy sample.

The Harp40 _1 sample was quite interesting. Because the high anchor has a clear problem with this sample the AAC encoders were much better. In the Test 1, A and B settings didn't produce as ugly artifacts as C and D with this sample. This was actually the only obvious difference I heard in any of these 2x3 AAC samples.

Also, In general I would say that the encoder in Test 2 was slightly, but audibly better than the encoder in the Test 1 with all samples.

BTW, the LAME -b 128 / Harp40_1 problem is the worst I have ever heard!


[span style='font-size:7pt;line-height:100%']Edit: fixed the Test2 heading in the table.[/span]

Nero 48 kbps HE-AAC (+v2) test - FINISHED!

Reply #3 –
Quote
Also, In general I would say that the encoder in Test 2 was slightly, but audibly better than the encoder in the Test 1 with all samples.


Interesting, as it seems - although, outside of the scope of the tests, pn average PS was ranked significantly lower than HE - tests were indeed separated, but the anchors were the same.

It is also worth noting that LAME -b128 was ranked 4.41 on the first test, and only 4.08 at the second one.

Dunno what exactly caused this difference - but I think the tests cannot be directly compared (as it was planned), and the upcoming 48 kbps multivendor AAC test will finally give the most accurate answer of HE-AAC v1  vs. V2 ranking.

Nero 48 kbps HE-AAC (+v2) test - FINISHED!

Reply #4 –
Quote
Dunno what exactly caused this difference
[a href="index.php?act=findpost&pid=362313"][{POST_SNAPBACK}][/a]


IMHO It's almost certainly listener training which improves after performing test1.

Nero 48 kbps HE-AAC (+v2) test - FINISHED!

Reply #6 –
Quote
Interesting, as it seems - although, outside of the scope of the tests, pn average PS was ranked significantly lower than HE - tests were indeed separated, but the anchors were the same.

It is also worth noting that LAME -b128 was ranked 4.41 on the first test, and only 4.08 at the second one.

Dunno what exactly caused this difference - but I think the tests cannot be directly compared (as it was planned), and the upcoming 48 kbps multivendor AAC test will finally give the most accurate answer of HE-AAC v1  vs. V2 ranking.
[a href="index.php?act=findpost&pid=362313"][{POST_SNAPBACK}][/a]

I tested only 6 of the samples. How did the different samples score? Is there any pattern? Also, at one stage you had more results from the Test1. Did that change? If not, would it be possible that those who send also Test2 results were more experienced listeners, thus more critical in general?

Listener training is probably a big factor, if most of the testers started with Test1 and then continued with Test2.

Perhaps my test order eliminated the listener training effect in my case. It was this:

1. Sandman  Test2
2. Sandman  Test1
3. Sympathy Test2
4. Sympathy Test1
5. Harp40_1 Test1
6. Harp40_1 Test2

I did all testing in the same evening. It took about 5 hours, so I spent about 8 minutes with each codec/setting (including a couple of short breaks). Most of time I tried to put the AAC encoders in order and that was pretty tough. The AAC samples had more or less subtle differences, but it was not easy to decide which artifacts are less annoying. I listened the samples over and over again until I was pretty sure about the results. All 28 samples would have taken about 24 hours to test at this testing speed. I guess I am hopelessly slow and need more training...

I didn't know it was LAME -b 128 that failed badly with the harp40_1 sample and got 1.4. I spent a lot of time trying to find the high anchor from the samples that were audibly better. Though, I was a bit surprised that the 1.4 samples were quite good after the first two totally broken notes. As I said earlier, the only other obvious difference was with the harp40_1 test1 AAC samples. Two of the AAC settings didn't have one particularly bad artifact with one of the higher notes. I thought one of those better contenders was LAME@128. 

Nero 48 kbps HE-AAC (+v2) test - FINISHED!

Reply #7 –
On most samples, I honestly couldn't tell the difference between the different AAC samples, they tended to sound about the same (not that great, but then again, it's 48kbps...), but there were a few exceptions.
It was kind of weird to hear LAME -b 128 doing worse on harp40_1, I thought I'd made a mistake and had to recheck my results

There were more than a few PS samples that sounded slightly better to me than the non-PS counterparts, interestingly enough.

Nero 48 kbps HE-AAC (+v2) test - FINISHED!

Reply #8 –
Nero 7 package was updated ( 6th of feb). is  submited Nero codec for this test  in this package?
 

Nero 48 kbps HE-AAC (+v2) test - FINISHED!

Reply #9 –
Quote
Nero 7 package was updated ( 6th of feb). is submited Nero codec for this test in this package?


No - the codec will be published by Gabriel.

New AAC codec will be available as a completely new release - ETA is still not public, but we committed to Gabriel's condition that codecs will be integrated "within 3 months", so the release date is somewhere in that time frame