Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps (Read 13270 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Abstract:
Personal blind sound quality comparison of AAC-LC and xHE-AAC.
The targeted bitrate was 96kbps and 128kbps.

Encoders:
The latest versions as of 2020 August were used.

ffmpeg version N-98640-gfa7ad81dab Copyright (c) 2000-2020 the FFmpeg developers built with gcc 9.2.0 (Rev2, Built by MSYS2 project) FFmpeg's native AAC encoder (-c:a aac)
Freeware Advanced Audio Coder FAAC 1.30
FDK-AAC 2.0.1 (-c:a libfdk_aac)
exhale - ecodis extended high-efficiency and low-complexity encoder version 1.0.6 (x64, built on Jul 28 2020)

Settings:
FFmpeg's native AAC encoder, FAAC 1.30, and FDK-AAC 2.0.1 were used as the AAC-LC encoder.
exhale 1.0.6 was used as the xHE-AAC encoder.

ffmpeg98640 -i in.wav -c:a aac -b:a 96k out.mp4
faac-1.30 -b 96 in.wav -o out.mp4 --overwrite
ffmpeg98640 -i in.wav -c:a libfdk_aac -b:a 96k out.mp4
exhaleApp106 3 in.wav out.mp4

ffmpeg98640 -i in.wav -c:a aac -b:a 128k out.mp4
faac-1.30 -b 128 in.wav -o out.mp4 --overwrite
ffmpeg98640 -i in.wav -c:a libfdk_aac -b:a 128k out.mp4
exhaleApp106 5 in.wav out.mp4

Samples:
15 sound samples from Kamedo2's samples.
12 sound samples from IgorC's samples.
Total 27 sound samples.

Hardwares:
Sony PSP-3000 + RP-HT560.

Results:



Conclusions & Observations:
For 128kbps, exhale and fdk-aac were the winner, and the new FAAC 1.30 won the second place. There were no big difference between the FDK-AAC, one of the best AAC encoder, and the xHE-AAC encoder at 128kbps.
For 96kbps, the xHE-AAC encoder exhale won the decisive victory. For the AAC-LC encoders, the FDK-AAC seems to be the best.
The exhale at 96kbps has the comparable fidelity to the FDK-AAC encoder at 128kbps.

Anova analysis:
Code: [Select]
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 27
Critical significance:  0.05
Significance of data: 1.11E-016 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of         Degrees     Sum of    Mean
variation         of Freedom  squares   Square    F      p

Total              215          91.28
Testers (blocks)    26          19.78
Codecs eval'd        7          51.18    7.31   65.51  1.11E-016
Error              182          20.31    0.11
---------------------------------------------------------------
Fisher's protected LSD for ANOVA:   0.179

Means:

exh128   fdk128   exh096   faa128   fdk096   ffm128   faa096   ffm096  
  4.39     4.27     4.15     3.76     3.49     3.44     3.20     2.97  

---------------------------- p-value Matrix ---------------------------

         fdk128   exh096   faa128   fdk096   ffm128   faa096   ffm096  
exh128   0.208    0.011*   0.000*   0.000*   0.000*   0.000*   0.000*  
fdk128            0.194    0.000*   0.000*   0.000*   0.000*   0.000*  
exh096                     0.000*   0.000*   0.000*   0.000*   0.000*  
faa128                              0.004*   0.001*   0.000*   0.000*  
fdk096                                       0.597    0.002*   0.000*  
ffm128                                                0.009*   0.000*  
faa096                                                         0.012*  
-----------------------------------------------------------------------

exh128 is better than exh096, faa128, fdk096, ffm128, faa096, ffm096
fdk128 is better than faa128, fdk096, ffm128, faa096, ffm096
exh096 is better than faa128, fdk096, ffm128, faa096, ffm096
faa128 is better than fdk096, ffm128, faa096, ffm096
fdk096 is better than faa096, ffm096
ffm128 is better than faa096, ffm096
faa096 is better than ffm096


Raw data:
Code: [Select]
FFmpeg's native	FAAC	FDK-AAC	exhale	FFmpeg's native	FAAC	FDK-AAC	exhale
1.700 3.300 3.100 4.100 2.500 3.700 4.400 4.200
3.300 4.200 3.600 4.300 3.800 4.400 4.500 4.500
3.300 3.500 3.900 4.000 3.700 3.800 4.100 4.300
2.800 3.800 3.600 4.400 3.400 4.200 4.500 4.600
2.900 3.600 3.900 4.300 3.200 4.000 4.400 4.500
3.600 2.800 3.800 4.200 3.700 3.400 4.100 4.400
3.800 4.300 4.600 5.000 4.000 4.500 5.000 4.800
2.800 3.600 3.400 4.200 3.900 4.100 4.400 4.300
3.000 2.600 3.600 4.300 3.400 2.800 3.900 4.600
2.600 2.900 2.600 4.400 3.700 3.900 4.700 4.500
3.000 3.200 3.400 4.100 3.500 3.800 4.200 4.400
2.900 2.900 3.500 4.200 3.200 3.300 3.700 4.400
3.300 3.800 2.800 4.200 3.600 4.100 4.600 4.400
2.400 2.700 2.800 4.100 2.800 3.800 4.400 4.600
2.900 2.600 3.200 4.200 3.400 3.900 4.400 4.600
2.500 2.700 3.900 4.100 3.400 3.500 4.300 4.500
3.400 3.100 2.900 3.500 3.900 3.700 3.500 3.800
2.900 2.900 3.500 3.700 3.200 3.400 3.900 4.100
3.300 3.400 3.500 3.900 3.700 3.800 4.200 4.100
3.100 2.700 3.400 3.800 3.500 3.600 3.900 4.000
1.800 2.400 3.300 4.300 2.100 2.700 4.600 4.500
2.600 2.800 2.900 3.700 3.500 3.400 3.600 4.000
2.400 3.400 2.800 4.100 2.600 3.800 4.300 4.200
3.300 3.400 3.800 4.200 3.500 4.000 4.300 4.500
3.400 2.800 4.100 4.000 3.700 3.600 4.300 4.200
4.300 3.900 4.400 4.700 4.500 4.600 4.800 5.000
2.900 3.100 3.900 4.100 3.500 3.600 4.300 4.400


%samples 10xh_
%samples 11ff_
%samples 12at_
%samples 13by_
%samples 14fe_
%samples 15ma_
%samples 16mb_
%samples 17qu_
%samples 18vr_
%samples 19am_
%samples 20tt_
%samples 21wa_
%samples 22ex_
%samples 23ht_
%samples 24td_
%samples 01 castanets
%samples 02 fatboy_30sec
%samples 03 eig
%samples 04 Bachpsichord
%samples 05 Enola
%samples 06 trumpet
%samples 07 applaud
%samples 08 velvet
%samples 09 Linchpin
%samples 10 spill_the_blood
%samples 11 female_speech
%samples 12 French_Ad

%feature 6 96kbps 96kbps 96kbps 96kbps 128kbps 128kbps 128kbps 128kbps
%feature 7 AAC-LC AAC-LC AAC-LC xHE-AAC AAC-LC AAC-LC AAC-LC xHE-AAC
%feature 10 N-98640-gfa7ad81dab 1.30 2.0.1 1.0.6 N-98640-gfa7ad81dab 1.30 2.0.1 1.0.6
%feature 11 -c:a aac -b:a 96k -b 96 -c:a libfdk_aac -b:a 96k 3 -c:a aac -b:a 128k -b 128 -c:a libfdk_aac -b:a 128k 5



Bitrates:
Code: [Select]
%bitrate
ffm096 faa096 fdk096 exh096 ffm128 faa128 fdk128 exh128

98887 112366 97944 115669 131300 146159 129968 151578
98089 99166 97972 101497 130156 132743 130006 118268
98317 102915 98310 112582 130406 136782 130331 146921
98880 103008 98044 105946 131505 137363 130070 138027
98760 104367 98258 109011 130850 137397 130286 133642
98371 97415 98299 106918 130380 132568 130251 138900
98162 97374 98197 95103 130263 135925 130207 125482
98902 106935 97994 110720 131438 139649 129992 142797
99341 101023 99033 117131 131681 135181 131148 153120
98335 102025 98054 105617 130566 135417 130069 131421
98400 104228 97892 103077 130585 139330 129885 137104
98586 102282 98154 110226 130663 135832 130192 151458
98641 105235 98209 103057 130729 140772 130259 128035
98472 100875 98031 101595 130587 136719 130079 136095
97446 111618 98275 104741 129510 147383 130266 124974
100070 107761 99731 106715 133441 142437 131521 147149
98875 107698 97963 120819 131554 141888 129977 156670
98749 106207 98595 114971 131020 141940 130594 153683
98237 96608 98050 119109 130353 129754 130103 140047
98314 101868 98008 105957 130699 136644 130033 130707
99036 103395 99159 104689 131379 137940 131131 132266
99131 106791 99530 111672 131710 142118 131535 150644
103807 134514 98879 110170 142352 167044 130892 144596
99408 103523 98667 95729 130416 136598 130691 125506
98099 97316 97923 107519 130184 133041 129971 139827
98743 110765 98272 90216 131200 144657 130265 125079
98804 107369 98633 103539 131261 138269 130605 130585

%album bitrate
96458 99957 97435 99120 127797 133819 129435 128732

Re: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Reply #1
Excellent! looks supperiour than LC-AAC. How about versus OPUS?

Re: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Reply #2
I thought the built-in ffmpeg AAC encoder was supposed to be better than FAAC?

Re: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Reply #3
Great test, thanks!

exhale vs opus would be very interesting


Re: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Reply #5
Kamedo2,
Interesting test.  :)

There were no big difference between the FDK-AAC, one of the best AAC encoder...
I see You have included only open source encoders (hence popular ones) in your test. That makes sense.

Also now exhale should outperform even the best LC-AAC encoder (Apple)  and be on par with Opus at the rates You have tested.


Re: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Reply #6
After having seen this personal listening test (Thanks Kamedo2 for doing and sharing!!), I decided to install and managed to configure foobar2000 converter with Exhale encoder from Rarewares. I wanted to compare the quality at 128kbps with Apple AAC.

FLAC files get encoded at a slow speed and foobar2000 shows the average bitrates and tags of the Exhale file, but foobar2000 can't play the file. It seems I have to install a decoder able to play Exhale m4a files so I stopped here because I suspect playback of Exhale files could be a problem on most or all existing devices like my car jukebox.

Will someday Exhale files be playable with standard aac decoders or should decoders be made compatible?

Thanks

Re: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Reply #7
Will someday Exhale files be playable with standard aac decoders or should decoders be made compatible?
No, the files won't get decoded by most standard AAC decoders out there. They need to support USAC/xHE-AAC
To be able to play the files in Foobar2000 use this component: http://www.foobar2000.org/components/view/foo_pd_aac



 

Re: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Reply #8
I thought the built-in ffmpeg AAC encoder was supposed to be better than FAAC?

I tested the latest version of FAAC, extensively tuned by Robert Kausch, Krzysztof Nikiel, Takashi Yoshi, Fabian Greffrath, Michael Fink, and Eugène Filin.
This result is not from the currently widely circulated, ancient FAAC 1.28 or 1.29.9.2.
Their contributions could have helped the FAAC 1.30 winning over the built-in FFmpeg AAC encoder.

Re: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Reply #9
Bitrate-vs-quality scatter graph.

exhale has a cool UFO-like scatter.
Any ideal encoders should behave like this.



Re: Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps

Reply #10
Very interesting test kamedo.
It illustrates how quick the AAC's quality drops from 128 kbps to 100 kbps: at ~130 kbps sound is very satisfying (with a good encoder at least, like FDK) but is clearly less enjoying when the bitrate is reduced. Some other formats/encoders handles the bitrate starvation better (Vorbis, Opus, xHE-AAC).
The new FAAC and the new born FFMPEG are statistically tied but with more samples/listeners we could maybe see that FAAC sounds better with more confidence.

I've just finished my own AAC listening test. I'll expect to publish the results in the next days. Results are very similar :)