Abstract:This is a translation of an
old test, finished in
October 2011, and translated from Japanese to English in October 2013 for convenience.
Blind Comparison between libfaac(FAAC, abr), Apple AAC-LC(cvbr), NeroAACEnc(abr), and LAME, at 64kbps, 96kbps, 128kbps and 192kbps.
Encoders:libfaac (FAAC) 1.28
qaac 0.94
nero 1.5.4.0
LAME 3.99
Settings:64kbps ffmpeg18607 -y -i in.wav
-acodec libfaac -ab 64k out.mp4
qaac
--cvbr 64 -o out.mp4 in.wav
neroAacEnc -if in.wav
-lc -br 64000 -of out.mp4
lame399
-h --preset 64 in.wav out.mp3
96kbps ffmpeg18607 -y -i in.wav
-acodec libfaac -ab 96k out.mp4
qaac -
-cvbr 96 -o out.mp4 in.wav
neroAacEnc -if in.wav
-lc -br 96000 -of out.mp4
lame399
-h --preset 96 in.wav out.mp3
128kbps ffmpeg18607 -y -i in.wav
-acodec libfaac -aq 95 out.mp4
qaac_0.94
--cvbr 128 -o out.mp4 in.wav
neroAacEnc -if in.wav
-lc -br 128000 -of out.mp4
lame399
-h -V5.6 in.wav out.mp3
192kbps ffmpeg18607 -y -i in.wav
-acodec libfaac -aq 155 out.mp4
qaac_0.94
--cvbr 192 -o out.mp4 in.wav
neroAacEnc -if in.wav
-lc -br 192000 -of out.mp4
lame399
-h -V2 in.wav out.mp3
Samples:15 Sounds of various genres, including difficult samples.
http://www.hydrogenaudio.org/forums/index....showtopic=98003 (http://www.hydrogenaudio.org/forums/index.php?showtopic=98003)
Hardwares:ABC/HR for Java 0.53a + MHP-A1
Results(http://i42.tinypic.com/15zh0qu.png)
(http://i39.tinypic.com/21ahlau.png)
(http://i43.tinypic.com/35krgy8.png)
(http://i43.tinypic.com/2eye645.png)
(http://i43.tinypic.com/15z62kl.png)
Conclusions & Observations:qaac(AAC) > NeroAACEnc(AAC) > LAME(MP3) > libfaac(AAC)
Apple AAC had the best sound quality. libfaac was the worst among the 4 encoder tested.
Anova analysis:FRIEDMAN version 1.24 (Jan 17, 2002) [url=http://ff123.net/]http://ff123.net/[/url]
Blocked ANOVA analysis
Number of listeners: 15
Critical significance: 0.05
Significance of data: 0.00E+000 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings
Source of Degrees Sum of Mean
variation of Freedom squares Square F p
Total 119 36.04
Testers (blocks) 14 2.24
Codecs eval'd 7 29.71 4.24 101.63 0.00E+000
Error 98 4.09 0.04
---------------------------------------------------------------
Fisher's protected LSD for ANOVA: 0.148
Means:
qaac96 nero96 lame96 faac96 nero64 qaac64 lame64 faac64
3.61 2.95 2.78 2.43 2.40 2.37 2.30 1.83
---------------------------- p-value Matrix ---------------------------
nero96 lame96 faac96 nero64 qaac64 lame64 faac64
qaac96 0.000* 0.000* 0.000* 0.000* 0.000* 0.000* 0.000*
nero96 0.020* 0.000* 0.000* 0.000* 0.000* 0.000*
lame96 0.000* 0.000* 0.000* 0.000* 0.000*
faac96 0.755 0.449 0.085 0.000*
nero64 0.656 0.156 0.000*
qaac64 0.328 0.000*
lame64 0.000*
-----------------------------------------------------------------------
qaac96 is better than nero96, lame96, faac96, nero64, qaac64, lame64, faac64
nero96 is better than lame96, faac96, nero64, qaac64, lame64, faac64
lame96 is better than faac96, nero64, qaac64, lame64, faac64
faac96 is better than faac64
nero64 is better than faac64
qaac64 is better than faac64
lame64 is better than faac64
FRIEDMAN version 1.24 (Jan 17, 2002) [url=http://ff123.net/]http://ff123.net/[/url]
Blocked ANOVA analysis
Number of listeners: 15
Critical significance: 0.05
Significance of data: 0.00E+000 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings
Source of Degrees Sum of Mean
variation of Freedom squares Square F p
Total 119 49.21
Testers (blocks) 14 7.98
Codecs eval'd 7 33.08 4.73 56.76 0.00E+000
Error 98 8.16 0.08
---------------------------------------------------------------
Fisher's protected LSD for ANOVA: 0.209
Means:
qaac192 nero192 qaac128 lame192 faac192 nero128 faac128 lame128
4.82 4.76 4.15 4.14 4.03 3.79 3.37 3.30
---------------------------- p-value Matrix ---------------------------
nero192 qaac128 lame192 faac192 nero128 faac128 lame128
qaac192 0.614 0.000* 0.000* 0.000* 0.000* 0.000* 0.000*
nero192 0.000* 0.000* 0.000* 0.000* 0.000* 0.000*
qaac128 0.900 0.245 0.001* 0.000* 0.000*
lame192 0.299 0.002* 0.000* 0.000*
faac192 0.029* 0.000* 0.000*
nero128 0.000* 0.000*
faac128 0.508
-----------------------------------------------------------------------
qaac192 is better than qaac128, lame192, faac192, nero128, faac128, lame128
nero192 is better than qaac128, lame192, faac192, nero128, faac128, lame128
qaac128 is better than nero128, faac128, lame128
lame192 is better than nero128, faac128, lame128
faac192 is better than nero128, faac128, lame128
nero128 is better than faac128, lame128
Raw data:% AAC/MP3 64kbps+96kbps ABC/HR test score.
% This format is compatible with my graphmaker, as well as ff123's FRIEDMAN.
% [url=http://zak.s206.xrea.com/bitratetest/graphmaker3.htm]http://zak.s206.xrea.com/bitratetest/graphmaker3.htm[/url]
%64kbps 64kbps 64kbps 64kbps 96kbps 96kbps 96kbps 96kbps
%features 7 AAC-LC AAC-LC AAC-LC MP3 AAC-LC AAC-LC AAC-LC MP3
faac64 qaac64 nero64 lame64 faac96 qaac96 nero96 lame96
1.85 2.15 1.6 1.95 2.2 3.35 2.6 2.45
1.7 2.3 2.4 2.4 2.25 3.65 3.15 2.75
1.75 2.5 2.4 2.2 2.55 3.75 2.8 2.8
1.75 2.25 2.25 2.2 2.25 3.1 2.85 2.5
1.85 2.4 2.3 2.4 2.8 3.2 2.85 2.65
1.9 2.35 2.6 2.2 2.45 4.55 3.1 2.8
1.75 2.25 2.5 2.2 2.5 3.5 2.8 2.45
1.75 2.4 2.35 2.3 2.4 3.55 3.05 2.85
1.8 2.45 2.55 2.3 2.25 3.6 2.9 2.65
1.85 2.2 2.45 2.3 2.5 3.8 2.9 2.75
1.8 2.4 2.3 2.3 2.45 3.15 2.75 2.75
1.95 2.6 2.65 2.45 2.75 3.35 3 2.8
1.95 2.55 2.65 2.4 2.4 3.65 3.25 3.05
1.9 2.25 2.65 2.35 2.45 3.35 3.3 2.85
1.9 2.5 2.4 2.5 2.2 4.6 3 3.55
%samples 41_30sec hihats
%samples finalfantasy cemb
%samples ATrain Jazz
%samples BigYellow Pops
%samples FloorEssence Techno
%samples macabre orch
%samples mybloodrusts guitar
%samples Quizas Latin
%samples VelvetRealm Techno
%samples Amefuribana Pops
%samples Trust Gospel
%samples Waiting Rock
%samples Experiencia Latin
%samples Heart_to_Heart Pops
%samples Tom's_Diner Vocal
% AAC/MP3 128kbps+192kbps ABC/HR test score.
%features 6 128kbps 128kbps 128kbps 128kbps 192kbps 192kbps 192kbps 192kbps
%features 7 AAC-LC AAC-LC AAC-LC MP3 AAC-LC AAC-LC AAC-LC MP3
faac128 qaac128 nero128 lame128 faac192 qaac192 nero192 lame192
3 4.45 3.8 3.45 3.6 4.8 5 4.7
3.35 4.25 3.95 2.95 3.9 5 5 3.65
3.45 4.5 4.1 3.4 4 5 4.7 4.4
3.05 4.15 3.8 3.05 4.05 4.45 4.6 4.05
3 3.35 3.4 3.2 3.65 4.8 5 3.85
3.8 5 5 4.45 5 5 5 5
3.9 4.8 4.75 3.5 4.4 5 5 4.2
3.1 4 3.7 3.25 3.9 5 5 4.15
3.7 3.95 3.2 3.5 3.95 4.8 3.95 3.9
3.05 3.95 3.65 3.1 3.5 5 5 3.95
3.95 3.75 3.3 3.3 4.4 4.35 4 4.1
3.3 3.8 3.35 3.25 3.85 4.65 4.2 3.9
3.15 4.1 3.6 3.05 4 4.7 5 4.1
3.7 4.05 3.7 3.25 4.1 5 5 4.3
3.05 4.15 3.6 2.8 4.1 4.7 5 3.8
%samples 41_30sec hihats
%samples finalfantasy cemb
%samples ATrain Jazz
%samples BigYellow Pops
%samples FloorEssence Techno
%samples macabre orch
%samples mybloodrusts guitar
%samples Quizas Latin
%samples VelvetRealm Techno
%samples Amefuribana Pops
%samples Trust Gospel
%samples Waiting Rock
%samples Experiencia Latin
%samples Heart_to_Heart Pops
%samples Tom's_Diner Vocal
It's not strange that some scores get 0.05 scale, as I tested twice per each sample.
It's interesting to see that Nero and Apple are on par at 192k. However an average score describes only a part of a whole picture.
The minimum score (worst case) at 192k: Nero - 4.0, Apple - 4.4.
Have You tried Fraunhofer AAC encoder from the last Winamp?
Yes, sometimes it makes sense to look at the minimum score and a distribution.
Apple beats Nero at 128kbps, and it's natural to think Apple also beats Nero at 192kbps.
Insignificance doesn't mean there are no differences at all, and actual average can be anywhere within the CI bar.
(Although this claim is hard to verify, because both encoders are very close to the original and offer very good qualities.)
I haven't tested the Fraunhofer AAC. Those on the web are the only listening test results I have. I don't test anything and secretly withhold the results.
The results of FAAC, Nero, and LAME are still relevant today. FAAC and Nero haven't changed at all since 2010, and as for LAME, a tiny bitrate bloat issue was fixed but no other quality improvement.
Ohm, I wanted to ask You if You have ever tried FhG AAC winamp encoder or consider to .
While its performance is well known at 96k, it's not so at 128-192k.
Ohm, I wanted to ask You if You have ever tried FhG AAC winamp encoder or consider to .
While its performance is well known at 96k, it's not so at 128-192k.
No, I have never tried FhG. I don't have a winamp.
Oops, images deleted.
(https://listening-test.coresv.net/img2/mp3-aac-mos-rate-graph.png)
(https://listening-test.coresv.net/img2/mp3-aac-quality-64kbps-96kbps.png)
(https://listening-test.coresv.net/img2/mp3-aac-table-64kbps-96kbps.png)
(https://listening-test.coresv.net/img2/mp3-aac-quality-128kbps-192kbps.png)
(https://listening-test.coresv.net/img2/mp3-aac-table-128kbps-192kbps.png)