Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Personal test – MP3 vs AAC (high rates, VBR) (Read 16489 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Personal test – MP3 vs AAC (high rates, VBR)

Hi,

All versions of encoders were the newest and stable
LAME MP3 3.100 -V2/V0/320kbps (~192 kbps/~260kbps/320 kbps)
Apple LC-AAC 7.10.9.0 via QAAC TVBR q91  (~192 kbps)

Files
LINK

RESULTS

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #1
IgorC,

Great test! The Apple AAC seems to be a very stable codec at high rates.
Also, you have tested LAME CBR 320kbps and it is a record high rate in an ABC-HR listening test of MP3.
Interesting that V0 seems somewhat safer than the CBR320k.

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #2
Excellent. It shows that Lame V2 is not be all end setting. Quality
will go up @ V1 and higher.

Further more, Mp3 is still competitive @ 192k (V2). People that
don't want other formats can be happy using mp3 at 160.. 224k, and
up to 256..320k

 

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #3
A bit different visualization.




Code: [Select]
Apple AAC-LC 192k	LAME CBR320k	LAME V0	LAME V2
%feature 8 AAC MP3 MP3 MP3
5.0 4.7 4.9 3.8
4.6 3.5 4.5 4.4
5.0 4.3 4.3 3.4
5.0 4.9 4.9 4.8
5.0 4.8 4.9 4.5
4.8 4.7 4.9 4.4
5.0 5.0 4.9 4.9
5.0 5.0 5.0 4.8
4.8 4.9 5.0 4.9
5.0 4.9 4.9 4.9
5.0 4.9 4.9 4.8
4.9 5.0 4.8 4.3
%samples 01 castanets
%samples 13 fatboy_30sec
%samples 03 eig
%samples 04 Bachpsichord
%samples 05 Enola
%samples 06 trumpet
%samples 07 applaud
%samples 08 velvet
%samples 09 Linchpin
%samples 10 Angels_Fall_First_ringing
%samples 11 You look good to me
%samples Berling Drug

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #4
IgorC,

Great test! The Apple AAC seems to be a very stable codec at high rates.
:) 
Yes, Apple AAC@192kbps crashes everything that MP3 can offer at ~260-320 kbps.

That's +64-128kbps bitrate difference.

Interesting that V0 seems somewhat safer than the CBR320k.
CBR320k has smaller amount of short frames. This explains why it performs worse on transient signals.

A bit different visualization.
Much better.  :)  I was sleepy yesterday and wanted to finish it.

Further more, Mp3 is still competitive @ 192k (V2). People that
don't want other formats can be happy using mp3 at 160.. 224k, and
up to 256..320k
Yes, I will not hesitate to listen to V0. Though Apple AAC still provides a great bitrate reduction and actually handles critical samples better.

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #5
@IgorC
This test (and your other multicodec test at 192k) finally shows how much bitrate is needed to achieve very high quality at reasonable bitrate. Excellent work!

Are you familiar with this sample:
https://hydrogenaud.io/index.php?topic=120193.0

Do you think we have to worry about this sample and similar ones for 192-256k which is otherwise more than enough for quality listening which your tests confirm?
lame --abr 288 -f --lowpass 17 (+ mp3gain@92 dB)

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #6
I combined the result with the IgorC's past result.

In the results below, the xHE-AAC encoder, exhale, have undergone many improvements and updates since then.
The other codecs, LAME, Apple, Vorbis, Musepack, and Opus are the latest as of 2021 September.
As expected of modern codecs at 192kbps and higher, all of them offer very high fidelity.




Code: [Select]
MP3	MP3	MP3	AAC-LC	MP3	xHE-AAC	Vorbis	AAC-LC	Musepack	Opus
4.7 4.9 3.8 5.0 3.5 5.0 5.0 5.0 5.0 5.0
3.5 4.5 4.4 4.6 4.5 4.0 4.6 5.0 5.0 5.0
4.3 4.3 3.4 5.0 3.8 4.7 4.8 5.0 5.0 5.0
4.9 4.9 4.8 5.0 5.0 5.0 5.0 5.0 5.0 5.0
4.8 4.9 4.5 5.0 4.2 4.8 4.9 4.9 4.9 5.0
4.7 4.9 4.4 4.8 4.2 4.9 4.9 5.0 5.0 5.0
5.0 4.9 4.9 5.0 4.8 5.0 5.0 4.7 4.9 5.0
5.0 5.0 4.8 5.0 5.0 5.0 5.0 5.0 5.0 5.0
4.9 5.0 4.9 4.8 5.0 5.0 5.0 5.0 5.0 5.0
4.9 4.9 4.9 5.0 4.7 5.0 4.8 5.0 5.0 5.0
4.9 4.9 4.8 5.0 4.8 5.0 5.0 4.9 4.8 5.0
5.0 4.8 4.3 4.9 4.0 4.9 4.8 4.8 4.8 5.0
%feature 6 320kbps 260kbps ≒192kbps ≒192kbps ≒192kbps ≒192kbps ≒192kbps ≒192kbps ≒192kbps ≒192kbps
%feature 7 2021 Sep. 2021 Sep. 2021 Sep. 2021 Sep. 2020 Oct. 2020 Oct. 2020 Oct. 2020 Oct. 2020 Oct. 2020 Oct.
%feature 10 LAME 3.100 LAME 3.100 LAME 3.100 Apple CoreAudioToolbox.dll 7.10.9.0 LAME 3.100 exhale 1.0.7 Aotuv 6.03 Apple CoreAudioToolbox.dll 7.10.9.0 MPC Encoder 1.30.0 --stable Opus 1.3.1
%feature 11 -b 320 -V0 -V2 TVBR q91 -V2 -m 9 -q6 TVBR q91 Q5.5 -b 182
%feature 12 Average:4.72 4.82 4.49 4.92 4.46 4.86 4.90 4.94 4.95 5.00

%samples 01 castanets
%samples 13 fatboy_30sec
%samples 03 eig
%samples 04 Bachpsichord
%samples 05 Enola
%samples 06 trumpet
%samples 07 applaud
%samples 08 velvet
%samples 09 Linchpin
%samples 10 Angels_Fall_First_ringing
%samples 11 You look good to me
%samples Berling Drug

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #7
Great test Igor!

I always thought V0 would be slightly better than q91.
I was wrong.
gold plated toslink fan

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #8
So when you look at the combined results as Kamedo2 posted,

This recent 256 / 320k  'normal' isn't based on any rational.  You will find many posts on the net
saying the min 256k is needed for 'hifi'. Including Cnet and other 'credible' sources. Many posters say
just go with 320cbr as storage is cheap. Well its x 1.67 times larger than 192 , thats a LOT esp for portables.
Most of them can't tell apart -b320 vs  -b192 or even 160.  
MPC / Lame were tuned to 160..200 vbr.  Before that CBR excellence point was around 192. All this for good reasons.
You can do almost anything today around the same bitrates of 2002.  So it looks like nothing has been achieved since
then but gazillion of codecs, bitrates and settings. A lot of extra complexity , Yet its back to square one; 128 / 160 / 192  etc


Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #9
@IgorC
This test (and your other multicodec test at 192k) finally shows how much bitrate is needed to achieve very high quality at reasonable bitrate. Excellent work!
...

Great test Igor!

I always thought V0 would be slightly better than q91.
I was wrong.
Thank You, guys!

@IgorC
...
Are you familiar with this sample:
https://hydrogenaud.io/index.php?topic=120193.0
...
It's sample with sharp transients. Three first samples from my test are also transients. So this kind of signal is well presented in this test.
Btw interesting sample.  Saved. Thank You.

@IgorC
Do you think we have to worry ...
You problably will need to answer this for yourself as transparency level is personal.

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #10
Is VBR -V0 better than the CBR320k on average?

Somewhat likely, but since the p-value is 0.2411, we cannot be very certain.
If he tests another 12 tracks , 24 tracks, 48 tracks, and many many tracks, he may find that -V0 can be 0.30 better, or 0.08 worse, or anywhere in between.

Code: [Select]
> cbr = c(4.7, 3.5, 4.3, 4.9, 4.8, 4.7, 5, 5, 4.9, 4.9, 4.9, 5)
> vbr = c(4.9, 4.5, 4.3, 4.9, 4.9, 4.9, 4.9, 5, 5, 4.9, 4.9, 4.8)
> t.test(cbr, vbr, paired=TRUE)

        Paired t-test

data:  cbr and vbr
t = -1.239, df = 11, p-value = 0.2411
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.30078053  0.08411387
sample estimates:
mean of the differences
             -0.1083333

> R.version
               _                          
platform       x86_64-w64-mingw32         
arch           x86_64                     
os             mingw32                    
system         x86_64, mingw32            
status                                    
major          4                          
minor          0.4                        
year           2021                       
month          02                         
day            15                         
svn rev        80002                      
language       R                          
version.string R version 4.0.4 (2021-02-15)
nickname       Lost Library Book          
>

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #11
Kamedo2, I've read your twitter. There is some mention explaining why v0 could potentially do better than cbr320.
https://hydrogenaud.io/index.php?topic=85135.msg864674#msg864674

Translated from https://twitter.com/kamedo2/status/1442042506464481287
"The reason why V0 performs better than CBR320k is that it may be because VBR uses bit reservoir very aggressively after LAME 3.99 (hence, the local bitrate can greatly exceed 320kbps). I was taught at HA before."


Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #12
Kamedo2, I've read your twitter. There is some mention explaining why v0 could potentially do better than cbr320.
https://hydrogenaud.io/index.php?topic=85135.msg864674#msg864674

Translated from https://twitter.com/kamedo2/status/1442042506464481287
"The reason why V0 performs better than CBR320k is that it may be because VBR uses bit reservoir very aggressively after LAME 3.99 (hence, the local bitrate can greatly exceed 320kbps). I was taught at HA before."

Yeah, nu774 seems right. The improvement is 10 years old now.

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #13
Yeah, nu774 seems right. The improvement is 10 years old now.
Considering the age of MP3, the improvement is refreshingly recent and new.   :P  

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #14
Hi.
It's quite important to know what equipment was in these test. Funnily i noticed the same (mp3@320 is worse mp3@~240) about 15 years ago with lame 3.93 and 3.97.
And why aac is only 192, q96, to compete on filesize plane?

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #15
Hi.
It's quite important to know what equipment was in these test
Hello,
The equipment was the same as in this test https://hydrogenaud.io/index.php?topic=120007.0

And why aac is only 192, q96, to compete on filesize plane?
Given that AAC scored very high  (4.93) at 192 kbps, it's pretty clear what to expect at higher bitrates.

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #16
Very very interesting info, many thanks for sharing IgorC.
WavPack 5.6.0 -b384hx6cmv / qaac64 2.80 -V 100

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #17
Interesting how Apple's AAC is in a league of it's own. For me this settles it. No reason to use MP3 if you can use AAC. (Always thought at higher bitrates it doesn't matter.)

I wonder how FDK and QAAC would fair at higher bitrates like 192kbps.  :D


Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #19
Yep, I used it before.

I should've used QAAC both the times.

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #20
HI @IgorC
beautiful test !

but seems that lame V0 does perform better than CBR 320k , i have ripped lots of cds the past month ,and I was undecided about 320k and v0 , at the end I have encoded all at 320k

thank you so much the test

Re: Personal test – MP3 vs AAC (high rates, VBR)

Reply #21
i tried the cbr approach. all good except pre echo samples. It is worse out of abr / vbr.
I decided to go vbr . I remember one thing;  back in the lame 3.90 era;  the APS (v2) / APX
settings worked for almost everyone.  Quality was pretty good, that i failed abx most random regular samples.
Even at the V2 standard.  APS Fast standard I managed with an ac/dc track but not with fast extreme...
From lame 3.95 to present many people can abx random samples up to including V2. But looking at closely not
at V1 .  From my own limited tests I fail at -V1 or -V1.5.  So what has happened ?  The set and forget setting of
lame MP3 v3.90 was lost and gave way to very high bitrate setting like -b320 /V0 as the 'standard' of the scene..
So I cautiously assume that a v3.90 present day match is -V1.5. Bitrate is ~ 210k. Much leaner than 320k or V0.
 I think with that, MP3 ( around 200k or so VBR) shouldn't be feared .