Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3) (Read 36331 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Abstract:
Personal blind sound quality comparison of the Bluetooth codec, AAC and LC3, at 144 kbps, at 48 kHz.
As of 2021, Bluetooth earphones and speakers often support AAC codec.
LC3 is a new codec, which will be mandatory in the upcoming LE Audio in Bluetooth 5.2.
Two AAC encoders were tested; one engine is CoreAudio, used by the Apple, and the other engine, FDK-AAC is used by the Android.
One LC3 encoder was tested; the latest LC3plus encoder available from ESTI.

Encoders:
CoreAudioToolbox 7.10.9.0, via qaac 2.72 (x64 version).
FDK-AAC 2.0.2, via ffmpeg version N-102573-g9d4c018497.
LC3plus Floating Point Software V1.6.3ETSI, ETSI TS 103 634 V1.3.1.

Sample rate converter setting:
refalac64 in.44kHz.wav --rate 48000 -D -b 24 -o in.48kHz.wav

Settings:
qaac64 --cbr 144 -o out.mp4 in.48kHz.wav
ffmpeg -y -i in.48kHz.wav -c:a libfdk_aac -b:a 144k out.mp4
LC3plus -E -q -v in.48kHz.wav out.lc3 144000

Sample tracks:
15 sound samples from Kamedo2's samples.
12 sound samples from IgorC's samples.
Total 27 diverse music and speech sound samples.

Hardware:
Sony PSP-3000 + AKG K712.

Results:




Conclusions & Observations:
  • AAC offered very high fidelity at 144 kbps. Most scores were better than 4.0 (Perceptible, but not annoying).
  • LC3 encoder, provided by ESTI, did not offered fidelity comparable to the commonly used AAC codec at 144k.
  • The two AAC encoders were tied at 144 kbps CBR. It's not clear whether qaac was the better or the FDK-AAC was the better.

Anova analysis:
Code: [Select]
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 27
Critical significance:  0.05
Significance of data: 2.56E-012 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of         Degrees     Sum of    Mean
variation         of Freedom  squares   Square    F      p

Total               80          19.52
Testers (blocks)    26           7.06
Codecs eval'd        2           8.00    4.00   46.58  2.56E-012
Error               52           4.47    0.09
---------------------------------------------------------------
Fisher's protected LSD for ANOVA:   0.160

Means:

AndroidA AppleAAC LC3plus  
  4.41     4.40     3.74   

---------------------------- p-value Matrix ---------------------------

         AppleAAC LC3plus  
AndroidA 0.926    0.000*   
AppleAAC          0.000*   
-----------------------------------------------------------------------

AndroidAAC is better than LC3plus
AppleAAC is better than LC3plus

Raw data:
Code: [Select]
Apple AAC	Android AAC	LC3plus
%feature	5	AAC	AAC	LC3
%feature	10	CoreAudioToolbox 7.10.9.0, via qaac 2.72	FDK-AAC 2.0.2	V1.6.3 ETSI (floating point ver.)
%feature	11	--cbr 144	-c:a libfdk_aac -b:a 144k	144000
%genre Kamedo2's 15 sample
4.700	4.500	4.200	
4.500	5.000	3.300	
4.400	4.600	3.200	
4.700	4.500	4.200	
4.100	4.400	3.500	
4.500	4.300	3.700	
5.000	5.000	4.400	
4.400	4.600	3.900	
3.900	3.500	4.200	
4.400	4.500	3.300	
4.300	4.200	3.600	
4.200	3.900	3.500	
5.000	4.600	3.600	
4.300	4.600	3.500	
5.000	5.000	3.600	
%genre IgorC's 12 sample
4.400	4.200	3.700	
3.800	4.100	3.600	
4.200	4.100	3.300	
4.500	4.400	3.700	
4.400	4.500	3.500	
3.900	4.200	3.300	
4.100	3.400	3.700	
4.300	4.400	4.100	
4.400	4.200	3.800	
4.100	4.400	3.700	
5.000	5.000	5.000	
4.400	5.000	3.900	

%samples	41_30sec	Perc.
%samples	finalfantasy	Strings
%samples	ATrain	Jazz
%samples	BigYellow	Pops
%samples	FloorEssence	Techno
%samples	macabre	Classic
%samples	mybloodrusts	Guitar
%samples	Quizas	Latin
%samples	VelvetRealm	Techno
%samples	Amefuribana	Pops
%samples	Trust	Gospel
%samples	Waiting	Rock
%samples	Experiencia	Latin
%samples	Heart to Heart	Pops
%samples	Tom's Diner	Acappella

%samples	01 castanets	inst.
%samples	02 fatboy_30sec	Techno
%samples	03 eig	Techno
%samples	04 Bachpsichord	inst.
%samples	05 Enola	Techno
%samples	06 trumpet	inst.
%samples	07 applaud	Live
%samples	08 velvet	perc.
%samples	09 Linchpin	Rock
%samples	10 spill_the_blood	guitar
%samples	11 female_speech	Speech
%samples	12 French_Ad	Speech

Bitrates:

All encoders were tested with CBR settings.
All tracks were encoded within 145kbps - 150kbps range.


Code: [Select]
%bitrate
Apple AAC	Android AAC	LC3plus	
146469	146146	145654
146500	146046	145613
146974	146453	145651
146778	146180	145642
146777	146356	145649
146921	146302	145626
147158	146361	145673
146681	146144	145653
148366	147429	145617
146607	146261	145618
146446	146024	145611
146748	146341	145645
146744	146334	145632
146687	146168	145627
146773	146411	145621
149514	147559	145751
146462	146076	145622
147719	146806	145700
146732	146184	145664
146422	146178	145644
148665	147225	145747

Other tests:

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #1 –
Interesting test, Kamedo2

LC3plus doesn't shine at 144 kbps. I personally would be more interested in 192+ kbps area and compare it to SBC as here  :)
It's Bluetooth codec in the end, so most of the time it will operate at higher rates.

Accroding to https://www.bluetooth.com/blog/a-technical-overview-of-lc3/
Quote
Good quality music has a bit rate of 96 kbps or greater
Well, while it does so so at 144k dunno what to expect at 96 kbps.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #2 –
Which frame size (i.e. codec latency) does LC3 have in this test? Isn't it really low, like 10 ms? I also heard that the codec complexity is quite low so that it can run on low-power devices.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #3 –
Quote from: C.R.Helmrich on
Which frame size (i.e. codec latency) does LC3 have in this test? Isn't it really low, like 10 ms?

Yes, the default, 10ms.
You can set -frame_ms 2.5 or -frame_ms 5 to have even shorter latency.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #4 –
Very nice, interesting and usuful test, as usual.

Not bad but unimpressive performance of LC3plus at this bitrate.
I'm slightly surprised to see Apple AAC and FDK CBR at the same level. Good point for Android users (like me  ;) )

Thank you for the hard work Kamedo2
Wavpack Hybrid -c4hx6

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #5 –
Thanks for confirming, Kamedo2! True, not that impressive compared with AAC-LC - having much higher latency - maybe, but note that in Kamedo2's previous Bluetooth test (https://hydrogenaud.io/index.php?topic=120182.0), SBC at 237 kbps scored only 3.55, so LC3's 3.74 at 144 kbps - almost 100 kbps less - actually are impressive IMHO.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #6 –
Indeed LC3 is superior to SBC.   Not only LC3 has lower latency (10ms) but also it supposedly has low complexity (how low not sure) wich is very important aspect for wireless devices.

I wonder whether Kamedo2 has included LC3 and AAC because the last one is supported by some wireless devices.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #7 –
Quote from: IgorC on
I wonder whether Kamedo2 has included LC3 and AAC because the last one is supported by some wireless devices.

LC3 vs AAC had the most vote on twitter, beating LC3 vs SBC, and LC3 vs aptX.
Probably because high-end devices tend to support AAC, therefore the preferred codec in a precision listening.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #8 –
Did those six know the purpose of LC3 ... ?

(I had to google it.)

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #9 –
Quote from: Porcus on
Did those six know the purpose of LC3 ... ?

(I had to google it.)

The choice is understandable.

The SBC is mandatory, universal and mediocre.
The AAC is optional but many people already have it and it is the preferred codec.
The LC3 will be a brand-new, and offer advantages AAC traditionally didn't serve - low latency and low battery drainage.
Understandably, people will be interested - should I migrate from AAC to LC3?

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #10 –
Quote from: Kamedo2 on
The LC3 will ... offer advantages AAC traditionally didn't serve - low latency and low battery drainage.
Are you sure that LC3 leads to noticeably lower battery usage than AAC-LC? Have you - or anyone else - measured the complexity and can report some results here? Since the AAC-LC technology is from the mid 1990s, I'd be very curious.

I think the key benefit of LC3 over AAC-LC or HE-AAC is its much lower codec latency, making it the much better candidate for e.g. communication applications.

Chris
If I don't reply to your reply, it means I agree with you.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #11 –
Quote from: C.R.Helmrich on
Have you - or anyone else - measured the complexity and can report some results here?

No, I did not personally confirm this.
I did not find any quantitative comparison of the decoder power consumption.
Obviously, developers have designed the LC3 codec with energy savings in mind, though. https://link.springer.com/book/10.1007/978-1-4842-6658-8

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #12 –
Android 13 developer preview is out, and it seems that it ships with LC3 related changes, e.g.:
https://developer.android.com/reference/android/bluetooth/BluetoothLeAudioCodecConfig#SOURCE_CODEC_TYPE_LC3
Did anybody try to extract and test the codec from the AOSP? Interesting how it compares to other AOSP Bluetooth codecs.

Re: Personal blind sound quality comparison of the Bluetooth codecs (AAC vs LC3)

Reply #13 –
Quote from: VEG on
Android 13 developer preview is out, and it seems that it ships with LC3 related changes, e.g.:
https://developer.android.com/reference/android/bluetooth/BluetoothLeAudioCodecConfig#SOURCE_CODEC_TYPE_LC3
Did anybody try to extract and test the codec from the AOSP? Interesting how it compares to other AOSP Bluetooth codecs.

The AOSP LC3 codec needs no extracting, since Google made https://github.com/google/liblc3 a separate thing on GitHub. PipeWire already uses it.