Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Personal Listening Test of FFmpeg AAC encoders (Read 18431 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Personal Listening Test of FFmpeg AAC encoders

Abstract:
Blind Comparison between FFmpeg native experimental AAC encoder with and without v4 patch, and FDK-AAC via FFmpeg, at 96kbps and 128kbps.

Encoders:
FFmpeg r55212(it was a current git head in August 4th 2013)
FFmpeg r55212 with aac-improvements-wip-v4-vbr.patch applied
https://ffmpeg.org/trac/ffmpeg/ticket/2686
fdk-aac-0.1.1 -acodec libfdk_aac via FFmpeg r55212
http://sourceforge.net/projects/opencore-amr/files/fdk-aac/

Settings:
ffmpeg55212 -y -i input.wav -c:a aac -strict experimental -b:a 96k output.mp4
ffmpeg55212_patchv4 -y -i input.wav -c:a aac -strict experimental -b:a 96k output.mp4
ffmpeg55212 -y -i input.wav -c:a libfdk_aac -b:a 96k -afterburner 1 output.mp4

ffmpeg55212 -y -i input.wav -c:a aac -strict experimental -b:a 128k output.mp4
ffmpeg55212_patchv4 -y -i input.wav -c:a aac -strict experimental -b:a 128k output.mp4
ffmpeg55212 -y -i input.wav -c:a libfdk_aac -b:a 128k -afterburner 1 output.mp4

Samples:
20 Sounds of various genres, including difficult samples.
http://www.hydrogenaudio.org/forums/index....showtopic=98003

Hardwares:
Sony PSP-3000 + RP-HT560.

Results



Conclusions & Observations:
The FDK-AAC had a significant and big quality advantage, compared to the FFmpeg experimental native AAC encoder.
Also, the positive effect of the v4 patch was confirmed in 128kbps. v4 patch certainly improves the audio quality.


Anova analysis:
Code: [Select]
FRIEDMAN version 1.24 (Jan 17, 2002) [url=http://ff123.net/]http://ff123.net/[/url]
Blocked ANOVA analysis

Number of listeners: 20
Critical significance:  0.05
Significance of data: 0.00E+000 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of        Degrees    Sum of    Mean
variation        of Freedom  squares  Square    F      p

Total              119          72.61
Testers (blocks)    19          10.68
Codecs eval'd        5          46.58    9.32  57.67  0.00E+000
Error              95          15.35    0.16
---------------------------------------------------------------
Fisher's protected LSD for ANOVA:  0.252

Means:

fdk_128k fv4_128k fdk_96k  fex_128k fv4_96k  fex_96k
  4.11    3.34    3.17    3.00    2.43    2.19

---------------------------- p-value Matrix ---------------------------

        fv4_128k fdk_96k  fex_128k fv4_96k  fex_96k
fdk_128k 0.000*  0.000*  0.000*  0.000*  0.000*
fv4_128k          0.184    0.009*  0.000*  0.000*
fdk_96k                    0.184    0.000*  0.000*
fex_128k                            0.000*  0.000*
fv4_96k                                      0.068
-----------------------------------------------------------------------

fdk_128k is better than fv4_128k, fdk_96k, fex_128k, fv4_96k, fex_96k
fv4_128k is better than fex_128k, fv4_96k, fex_96k
fdk_96k is better than fv4_96k, fex_96k
fex_128k is better than fv4_96k, fex_96k
Raw data:
Code: [Select]
% FFmpeg AAC 96kbps and 128kbps ABC/HR Score
% This format is compatible with my graphmaker, as well as ff123's FRIEDMAN.
% [url=http://zak.s206.xrea.com/bitratetest/graphmaker3.htm]http://zak.s206.xrea.com/bitratetest/graphmaker3.htm[/url]
fex_96k fv4_96k fdk_96k fex_128k fv4_128k fdk_128k
%feature 7 96kbps 96kbps 96kbps 128kbps 128kbps 128kbps
1.800 2.100 3.100 2.400 2.900 3.400
2.100 2.600 3.000 2.800 3.200 3.800
2.500 3.300 2.800 3.600 4.100 3.900
2.200 2.500 3.000 3.100 3.500 3.700
2.100 2.400 3.400 2.600 3.200 3.800
2.400 2.700 3.200 3.000 4.000 3.600
2.100 1.900 4.300 2.300 2.400 5.000
2.800 3.100 4.000 3.800 4.100 4.500
1.900 2.000 3.000 2.300 2.200 3.500
1.900 2.200 3.200 2.700 3.900 4.200
2.300 2.300 2.700 3.200 3.400 3.900
2.200 2.500 2.600 2.900 3.400 3.700
2.100 2.000 2.700 3.400 3.200 4.300
2.700 3.200 3.100 3.600 3.800 5.000
2.000 1.900 2.900 2.900 2.400 5.000
2.700 2.900 3.100 4.200 4.400 4.000
2.000 2.400 3.500 2.500 3.300 5.000
2.100 2.400 3.300 2.900 3.400 3.700
1.800 1.900 3.600 2.600 2.800 4.200
2.200 2.300 2.900 3.200 3.200 3.900
%samples 41_30sec hihats
%samples finalfantasy cemb
%samples ATrain Jazz
%samples BigYellow Pops
%samples FloorEssence Techno
%samples macabre orch
%samples mybloodrusts guitar
%samples Quizas Latin
%samples VelvetRealm Techno
%samples Amefuribana Pops
%samples Trust Gospel
%samples Waiting Rock
%samples Experiencia Latin
%samples Heart_to_Heart Pops
%samples Tom's_Diner Vocal
%samples Reunion_Blues Jazz
%samples French Speech
%samples undelete Pops
%samples Dimmu_Borgir Metal
%samples Run_up Pops
Bitrates:
Code: [Select]
%bitrates
98047 99633 97884 130027 132691 129902
97968 97493 97904 129893 128481 129932
98086 99534 98218 130174 131700 130228
98023 99672 97973 130025 132839 129987
98185 101107 98170 130210 134467 130176
98057 98083 98173 130099 130354 130189
97976 98413 98105 130058 130253 130097
98036 99669 97938 129986 132192 129933
98551 101322 98863 130614 134139 130954
98001 99409 97973 130011 131928 129996
97916 98599 97819 129968 130927 129819
98023 98929 98066 130070 131320 130101
98152 99640 98114 130198 132861 130146
98017 99379 97978 129990 131970 130014
98610 98271 98205 130611 129521 130166
98733 99587 97895 130899 132309 129893
98194 99502 97905 130235 128148 129938
97896 99076 97932 129981 131508 129981
98019 98110 98048 130155 130212 130136
98020 99044 98177 130087 131553 130171
v4 patch had the most bitrate fluctuation.

 

Personal Listening Test of FFmpeg AAC encoders

Reply #1
Thanks for sharing, this is certainly interesting, especially the fact that fdk-aac is "almost never" transparent at 128 kbits for you?
Did you test/plan to test quality based settings of fdk-aac as well?
PANIC: CPU 1: Cache Error (unrecoverable - dcache data) Eframe = 0x90000000208cf3b8
NOTICE - cpu 0 didn't dump TLB, may be hung

Personal Listening Test of FFmpeg AAC encoders

Reply #2
I wonder if the quality of the internal FFMPEG aac encoder has reached the quality of libfaac yet?

Based on past experiences the quality of the encoders is ranked as such:
libfdk_aac > libfaac > aac > libvo-aacenc

Personal Listening Test of FFmpeg AAC encoders

Reply #3
Thanks for sharing, this is certainly interesting, especially the fact that fdk-aac is "almost never" transparent at 128 kbits for you?
Did you test/plan to test quality based settings of fdk-aac as well?

Apple used to advertise AAC 128 kbps as "transparent", but according to my past test, Apple AAC consistently scores around 4.1. So I'd say that fdk-aac is an excellent encoder in 128kbps, the quality is even comparable to the mighty Apple, when encoding music at 128kbps. At 96kbps, Apple AAC typically gets 3.6, so fdk-aac may sound worse than the Apple. I may test the VBR fdk-aac in a distant future, but I don't expect too much from employing the VBR.

Personal Listening Test of FFmpeg AAC encoders

Reply #4
I wonder if the quality of the internal FFMPEG aac encoder has reached the quality of libfaac yet?

Based on past experiences the quality of the encoders is ranked as such:
libfdk_aac > libfaac > aac > libvo-aacenc

Based on this test, Claudio Freire is now writing a new patch that fixes the weak point exposed in this test. It may beat the outdated libfaac with margin, but it's too early to draw any conclusions. My recommendation is to wait for the new patch.

Why do people stick to the old libfaac? Libfaac is extremely outdated, exceptional low quality in 64kbps, unfriendly, and non-free. There should be all the reason to pick libfdk_aac over libfaac.

Personal Listening Test of FFmpeg AAC encoders

Reply #5
Why do people stick to the old libfaac? Libfaac is extremely outdated, exceptional low quality in 64kbps, unfriendly, and non-free. There should be all the reason to pick libfdk_aac over libfaac.

A recent discussion related to that: http://www.hydrogenaudio.org/forums/index....howtopic=102580

And thanks for taking the time to do this, Kamedo2.

Personal Listening Test of FFmpeg AAC encoders

Reply #6
Is there anybody who have color vision deficiency or who have difficulty reading the results for other reasons? Should I change how I plot?

Personal Listening Test of FFmpeg AAC encoders

Reply #7
Kamedo2,

The plot is ok.

Interesting test. You're all into open source AAC encoders. Do You know if there will be a long term development of ffmpeg AAC encoder?
There were some attempts to develop it during last years but it hasn't come any close to commercial AAC encoders in terms of quality.
Honestly I doubt there will be a big development of classic MDCT audio encoders. The viable options are Opus, USAC  or completely new  formats in future.       


Personal Listening Test of FFmpeg AAC encoders

Reply #8
Do You know if there will be a long term development of ffmpeg AAC encoder?

klaussfreire's new v5 patch is out for test now, and and it's the another major improvement of ABR.
He's now writing a v6 patch, which is I think a minor revision of v5, including optimization of VBR.