Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3) (Read 20385 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Abstract:
Personal blind sound quality comparison of the Bluetooth codec, AAC and LC3, at 144 kbps, at 48 kHz.
As of 2021, Bluetooth earphones and speakers often support AAC codec.
LC3 is a new codec, which will be mandatory in the upcoming LE Audio in Bluetooth 5.2.
Two AAC encoders were tested; one engine is CoreAudio, used by the Apple, and the other engine, FDK-AAC is used by the Android.
One LC3 encoder was tested; the latest LC3plus encoder available from ESTI.

Encoders:
CoreAudioToolbox 7.10.9.0, via qaac 2.72 (x64 version).
FDK-AAC 2.0.2, via ffmpeg version N-102573-g9d4c018497.
LC3plus Floating Point Software V1.6.3ETSI, ETSI TS 103 634 V1.3.1.

Sample rate converter setting:
refalac64 in.44kHz.wav --rate 48000 -D -b 24 -o in.48kHz.wav

Settings:
qaac64 --cbr 144 -o out.mp4 in.48kHz.wav
ffmpeg -y -i in.48kHz.wav -c:a libfdk_aac -b:a 144k out.mp4
LC3plus -E -q -v in.48kHz.wav out.lc3 144000

Sample tracks:
15 sound samples from Kamedo2's samples.
12 sound samples from IgorC's samples.
Total 27 diverse music and speech sound samples.

Hardware:
Sony PSP-3000 + AKG K712.

Results:




Conclusions & Observations:
  • AAC offered very high fidelity at 144 kbps. Most scores were better than 4.0 (Perceptible, but not annoying).
  • LC3 encoder, provided by ESTI, did not offered fidelity comparable to the commonly used AAC codec at 144k.
  • The two AAC encoders were tied at 144 kbps CBR. It's not clear whether qaac was the better or the FDK-AAC was the better.

Anova analysis:
Code: [Select]
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 27
Critical significance:  0.05
Significance of data: 2.56E-012 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of         Degrees     Sum of    Mean
variation         of Freedom  squares   Square    F      p

Total               80          19.52
Testers (blocks)    26           7.06
Codecs eval'd        2           8.00    4.00   46.58  2.56E-012
Error               52           4.47    0.09
---------------------------------------------------------------
Fisher's protected LSD for ANOVA:   0.160

Means:

AndroidA AppleAAC LC3plus 
  4.41     4.40     3.74  

---------------------------- p-value Matrix ---------------------------

         AppleAAC LC3plus 
AndroidA 0.926    0.000*  
AppleAAC          0.000*  
-----------------------------------------------------------------------

AndroidAAC is better than LC3plus
AppleAAC is better than LC3plus

Raw data:
Code: [Select]
Apple AAC	Android AAC	LC3plus
%feature 5 AAC AAC LC3
%feature 10 CoreAudioToolbox 7.10.9.0, via qaac 2.72 FDK-AAC 2.0.2 V1.6.3 ETSI (floating point ver.)
%feature 11 --cbr 144 -c:a libfdk_aac -b:a 144k 144000
%genre Kamedo2's 15 sample
4.700 4.500 4.200
4.500 5.000 3.300
4.400 4.600 3.200
4.700 4.500 4.200
4.100 4.400 3.500
4.500 4.300 3.700
5.000 5.000 4.400
4.400 4.600 3.900
3.900 3.500 4.200
4.400 4.500 3.300
4.300 4.200 3.600
4.200 3.900 3.500
5.000 4.600 3.600
4.300 4.600 3.500
5.000 5.000 3.600
%genre IgorC's 12 sample
4.400 4.200 3.700
3.800 4.100 3.600
4.200 4.100 3.300
4.500 4.400 3.700
4.400 4.500 3.500
3.900 4.200 3.300
4.100 3.400 3.700
4.300 4.400 4.100
4.400 4.200 3.800
4.100 4.400 3.700
5.000 5.000 5.000
4.400 5.000 3.900

%samples 41_30sec Perc.
%samples finalfantasy Strings
%samples ATrain Jazz
%samples BigYellow Pops
%samples FloorEssence Techno
%samples macabre Classic
%samples mybloodrusts Guitar
%samples Quizas Latin
%samples VelvetRealm Techno
%samples Amefuribana Pops
%samples Trust Gospel
%samples Waiting Rock
%samples Experiencia Latin
%samples Heart to Heart Pops
%samples Tom's Diner Acappella

%samples 01 castanets inst.
%samples 02 fatboy_30sec Techno
%samples 03 eig Techno
%samples 04 Bachpsichord inst.
%samples 05 Enola Techno
%samples 06 trumpet inst.
%samples 07 applaud Live
%samples 08 velvet perc.
%samples 09 Linchpin Rock
%samples 10 spill_the_blood guitar
%samples 11 female_speech Speech
%samples 12 French_Ad Speech

Bitrates:

All encoders were tested with CBR settings.
All tracks were encoded within 145kbps - 150kbps range.


Code: [Select]
%bitrate
Apple AAC Android AAC LC3plus
146469 146146 145654
146500 146046 145613
146974 146453 145651
146778 146180 145642
146777 146356 145649
146921 146302 145626
147158 146361 145673
146681 146144 145653
148366 147429 145617
146607 146261 145618
146446 146024 145611
146748 146341 145645
146744 146334 145632
146687 146168 145627
146773 146411 145621
149514 147559 145751
146462 146076 145622
147719 146806 145700
146732 146184 145664
146422 146178 145644
148665 147225 145747

Other tests:


Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #1
Interesting test, Kamedo2

LC3plus doesn't shine at 144 kbps. I personally would be more interested in 192+ kbps area and compare it to SBC as here  :)
It's Bluetooth codec in the end, so most of the time it will operate at higher rates.

Accroding to https://www.bluetooth.com/blog/a-technical-overview-of-lc3/
Quote
Good quality music has a bit rate of 96 kbps or greater
Well, while it does so so at 144k dunno what to expect at 96 kbps.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #2
Which frame size (i.e. codec latency) does LC3 have in this test? Isn't it really low, like 10 ms? I also heard that the codec complexity is quite low so that it can run on low-power devices.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #3
Which frame size (i.e. codec latency) does LC3 have in this test? Isn't it really low, like 10 ms?

Yes, the default, 10ms.
You can set -frame_ms 2.5 or -frame_ms 5 to have even shorter latency.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #4
Very nice, interesting and usuful test, as usual.

Not bad but unimpressive performance of LC3plus at this bitrate.
I'm slightly surprised to see Apple AAC and FDK CBR at the same level. Good point for Android users (like me  ;) )

Thank you for the hard work Kamedo2

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #5
Thanks for confirming, Kamedo2! True, not that impressive compared with AAC-LC - having much higher latency - maybe, but note that in Kamedo2's previous Bluetooth test (https://hydrogenaud.io/index.php?topic=120182.0), SBC at 237 kbps scored only 3.55, so LC3's 3.74 at 144 kbps - almost 100 kbps less - actually are impressive IMHO.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #6
Indeed LC3 is superior to SBC.   Not only LC3 has lower latency (10ms) but also it supposedly has low complexity (how low not sure) wich is very important aspect for wireless devices.

I wonder whether Kamedo2 has included LC3 and AAC because the last one is supported by some wireless devices.


Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #7
I wonder whether Kamedo2 has included LC3 and AAC because the last one is supported by some wireless devices.

LC3 vs AAC had the most vote on twitter, beating LC3 vs SBC, and LC3 vs aptX.
Probably because high-end devices tend to support AAC, therefore the preferred codec in a precision listening.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #8
Did those six know the purpose of LC3 ... ?

(I had to google it.)

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #9
Did those six know the purpose of LC3 ... ?

(I had to google it.)

The choice is understandable.

The SBC is mandatory, universal and mediocre.
The AAC is optional but many people already have it and it is the preferred codec.
The LC3 will be a brand-new, and offer advantages AAC traditionally didn't serve - low latency and low battery drainage.
Understandably, people will be interested - should I migrate from AAC to LC3?

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #10
The LC3 will ... offer advantages AAC traditionally didn't serve - low latency and low battery drainage.
Are you sure that LC3 leads to noticeably lower battery usage than AAC-LC? Have you - or anyone else - measured the complexity and can report some results here? Since the AAC-LC technology is from the mid 1990s, I'd be very curious.

I think the key benefit of LC3 over AAC-LC or HE-AAC is its much lower codec latency, making it the much better candidate for e.g. communication applications.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #11
Have you - or anyone else - measured the complexity and can report some results here?

No, I did not personally confirm this.
I did not find any quantitative comparison of the decoder power consumption.
Obviously, developers have designed the LC3 codec with energy savings in mind, though. https://link.springer.com/book/10.1007/978-1-4842-6658-8

 

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #12
Android 13 developer preview is out, and it seems that it ships with LC3 related changes, e.g.:
https://developer.android.com/reference/android/bluetooth/BluetoothLeAudioCodecConfig#SOURCE_CODEC_TYPE_LC3
Did anybody try to extract and test the codec from the AOSP? Interesting how it compares to other AOSP Bluetooth codecs.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #13
Android 13 developer preview is out, and it seems that it ships with LC3 related changes, e.g.:
https://developer.android.com/reference/android/bluetooth/BluetoothLeAudioCodecConfig#SOURCE_CODEC_TYPE_LC3
Did anybody try to extract and test the codec from the AOSP? Interesting how it compares to other AOSP Bluetooth codecs.

The AOSP LC3 codec needs no extracting, since Google made https://github.com/google/liblc3 a separate thing on GitHub. PipeWire already uses it.