Abstract: Personal blind sound quality comparison of the Bluetooth codec, AAC and LC3, at 144 kbps, at 48 kHz.
As of 2021, Bluetooth earphones and speakers often support AAC codec.
LC3 is a new codec, which will be mandatory in the upcoming LE Audio in Bluetooth 5.2.
Two AAC encoders were tested; one engine is CoreAudio, used by the Apple, and the other engine, FDK-AAC is used by the Android.
One LC3 encoder was tested; the latest LC3plus encoder available from ESTI.
Encoders: CoreAudioToolbox 7.10.9.0, via qaac 2.72 (x64 version).
FDK-AAC 2.0.2, via ffmpeg version N-102573-g9d4c018497.
LC3plus Floating Point Software V1.6.3ETSI, ETSI TS 103 634 V1.3.1. (https://www.etsi.org/deliver/etsi_ts/103600_103699/103634/01.03.01_60/ts_103634v010301p0.zip)
Sample rate converter setting: refalac64 in.44kHz.wav --rate 48000 -D -b 24 -o in.48kHz.wav
Settings: qaac64
--cbr 144 -o out.mp4 in.48kHz.wav
ffmpeg -y -i in.48kHz.wav
-c:a libfdk_aac -b:a 144k out.mp4
LC3plus -E -q -v in.48kHz.wav out.lc3
144000Sample tracks: 15 sound samples from Kamedo2's samples (https://hydrogenaud.io/index.php/topic,98003.msg815018.html).
12 sound samples from IgorC's samples (https://drive.google.com/file/d/0ByvUr-pp6BuUSnlwUG1WNkZzeDA/view?usp=sharing).
Total
27 diverse music and speech sound samples.
Hardware: Sony PSP-3000 + AKG K712.
Results: (https://listening-test.coresv.net/img2/23lc3-en2x.png)
(https://listening-test.coresv.net/img2/23lc3-tracks-en2x.png)
(https://listening-test.coresv.net/img2/23lc3-bars-en2x.png)
Conclusions & Observations: - AAC offered very high fidelity at 144 kbps. Most scores were better than 4.0 (Perceptible, but not annoying).
- LC3 encoder, provided by ESTI, did not offered fidelity comparable to the commonly used AAC codec at 144k.
- The two AAC encoders were tied at 144 kbps CBR. It's not clear whether qaac was the better or the FDK-AAC was the better.
Anova analysis: FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis
Number of listeners: 27
Critical significance: 0.05
Significance of data: 2.56E-012 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings
Source of Degrees Sum of Mean
variation of Freedom squares Square F p
Total 80 19.52
Testers (blocks) 26 7.06
Codecs eval'd 2 8.00 4.00 46.58 2.56E-012
Error 52 4.47 0.09
---------------------------------------------------------------
Fisher's protected LSD for ANOVA: 0.160
Means:
AndroidA AppleAAC LC3plus
4.41 4.40 3.74
---------------------------- p-value Matrix ---------------------------
AppleAAC LC3plus
AndroidA 0.926 0.000*
AppleAAC 0.000*
-----------------------------------------------------------------------
AndroidAAC is better than LC3plus
AppleAAC is better than LC3plus
Raw data: Apple AAC Android AAC LC3plus
%feature 5 AAC AAC LC3
%feature 10 CoreAudioToolbox 7.10.9.0, via qaac 2.72 FDK-AAC 2.0.2 V1.6.3 ETSI (floating point ver.)
%feature 11 --cbr 144 -c:a libfdk_aac -b:a 144k 144000
%genre Kamedo2's 15 sample
4.700 4.500 4.200
4.500 5.000 3.300
4.400 4.600 3.200
4.700 4.500 4.200
4.100 4.400 3.500
4.500 4.300 3.700
5.000 5.000 4.400
4.400 4.600 3.900
3.900 3.500 4.200
4.400 4.500 3.300
4.300 4.200 3.600
4.200 3.900 3.500
5.000 4.600 3.600
4.300 4.600 3.500
5.000 5.000 3.600
%genre IgorC's 12 sample
4.400 4.200 3.700
3.800 4.100 3.600
4.200 4.100 3.300
4.500 4.400 3.700
4.400 4.500 3.500
3.900 4.200 3.300
4.100 3.400 3.700
4.300 4.400 4.100
4.400 4.200 3.800
4.100 4.400 3.700
5.000 5.000 5.000
4.400 5.000 3.900
%samples 41_30sec Perc.
%samples finalfantasy Strings
%samples ATrain Jazz
%samples BigYellow Pops
%samples FloorEssence Techno
%samples macabre Classic
%samples mybloodrusts Guitar
%samples Quizas Latin
%samples VelvetRealm Techno
%samples Amefuribana Pops
%samples Trust Gospel
%samples Waiting Rock
%samples Experiencia Latin
%samples Heart to Heart Pops
%samples Tom's Diner Acappella
%samples 01 castanets inst.
%samples 02 fatboy_30sec Techno
%samples 03 eig Techno
%samples 04 Bachpsichord inst.
%samples 05 Enola Techno
%samples 06 trumpet inst.
%samples 07 applaud Live
%samples 08 velvet perc.
%samples 09 Linchpin Rock
%samples 10 spill_the_blood guitar
%samples 11 female_speech Speech
%samples 12 French_Ad Speech
Bitrates: All encoders were tested with CBR settings.
All tracks were encoded within 145kbps - 150kbps range.
(https://listening-test.coresv.net/img2/23lc3-bitrates-en2x2.png)
%bitrate
Apple AAC Android AAC LC3plus
146469 146146 145654
146500 146046 145613
146974 146453 145651
146778 146180 145642
146777 146356 145649
146921 146302 145626
147158 146361 145673
146681 146144 145653
148366 147429 145617
146607 146261 145618
146446 146024 145611
146748 146341 145645
146744 146334 145632
146687 146168 145627
146773 146411 145621
149514 147559 145751
146462 146076 145622
147719 146806 145700
146732 146184 145664
146422 146178 145644
148665 147225 145747
Other tests: - Personal blind sound quality comparison of Opus hard-CBR with framesize options (https://hydrogenaud.io/index.php?topic=121552.0)
- Personal Blind Listening Test of the latest codecs at 40kbps, 48kHz (https://hydrogenaud.io/index.php?topic=121155.0)
- Personal Blind Listening Test of Bluetooth codecs (https://hydrogenaud.io/index.php?topic=120182.0)
- Personal Blind Listening Test of Opus and the exhale xHE-AAC encoder (https://hydrogenaud.io/index.php?topic=120936.0)
- Personal Listening Test of AAC-LC and xHE-AAC at 96kbps and 128kbps (https://hydrogenaud.io/index.php?topic=119861.0)
- Personal Listening Test of AAC and LAME encoders (old test, translated) (https://hydrogenaud.io/index.php?topic=102876.0)
- Personal Listening Test of AAC, WMA, and MP3 encoders (old test, translated) (https://hydrogenaud.io/index.php?topic=102875.0)
- Personal Listening Test of MP3 and Opus (https://hydrogenaud.io/index.php?topic=117489.0)
- Personal Listening Test of AAC encoders available from FFmpeg (https://hydrogenaud.io/index.php?topic=111085.0)
- Personal Listening Test of Experimental Modified Opus Encoders at 36, 48 kbps (https://hydrogenaud.io/index.php?topic=113985.0)
- Personal Listening Test of 2 Opus encoders (https://hydrogenaud.io/index.php?topic=111798.0)
- Personal Listening Test of LAME, iTunes and Helix MP3 encoders (https://hydrogenaud.io/index.php?topic=113324.0)
- Personal Listening Test of MP3/Opus/AAC at 96kbps (https://hydrogenaud.io/index.php?topic=109716.0)
Interesting test, Kamedo2
LC3plus doesn't shine at 144 kbps. I personally would be more interested in 192+ kbps area and compare it to SBC as here (https://www.bluetooth.com/wp-content/uploads/2019/12/2007_LC3_Chart_Update-01-1966x2048.png) :)
It's Bluetooth codec in the end, so most of the time it will operate at higher rates.
Accroding to https://www.bluetooth.com/blog/a-technical-overview-of-lc3/
Good quality music has a bit rate of 96 kbps or greater
Well, while it does so so at 144k dunno what to expect at 96 kbps.
Which frame size (i.e. codec latency) does LC3 have in this test? Isn't it really low, like 10 ms? I also heard that the codec complexity is quite low so that it can run on low-power devices.
Chris
Which frame size (i.e. codec latency) does LC3 have in this test? Isn't it really low, like 10 ms?
Yes, the default, 10ms.
You can set -frame_ms 2.5 or -frame_ms 5 to have even shorter latency.
Very nice, interesting and usuful test, as usual.
Not bad but unimpressive performance of LC3plus at this bitrate.
I'm slightly surprised to see Apple AAC and FDK CBR at the same level. Good point for Android users (like me ;) )
Thank you for the hard work Kamedo2
Thanks for confirming, Kamedo2! True, not that impressive compared with AAC-LC - having much higher latency - maybe, but note that in Kamedo2's previous Bluetooth test (https://hydrogenaud.io/index.php?topic=120182.0), SBC at 237 kbps scored only 3.55, so LC3's 3.74 at 144 kbps - almost 100 kbps less - actually are impressive IMHO.
Chris
Indeed LC3 is superior to SBC. Not only LC3 has lower latency (10ms) but also it supposedly has low complexity (how low not sure) wich is very important aspect for wireless devices.
I wonder whether Kamedo2 has included LC3 and AAC because the last one is supported by some wireless devices.
I wonder whether Kamedo2 has included LC3 and AAC because the last one is supported by some wireless devices.
LC3 vs AAC had the most vote on twitter (https://twitter.com/kamedo2/status/1421083641673773059), beating LC3 vs SBC, and LC3 vs aptX.
Probably because high-end devices tend to support AAC, therefore the preferred codec in a precision listening.
Did those six know the purpose of LC3 ... ?
(I had to google it.)
Did those six know the purpose of LC3 ... ?
(I had to google it.)
The choice is understandable.
The SBC is mandatory, universal and mediocre.
The AAC is optional but many people already have it and it is the preferred codec.
The LC3 will be a brand-new, and offer advantages AAC traditionally didn't serve - low latency and low battery drainage.
Understandably, people will be interested - should I migrate from AAC to LC3?
The LC3 will ... offer advantages AAC traditionally didn't serve - low latency and low battery drainage.
Are you sure that LC3 leads to noticeably lower battery usage than AAC-LC? Have you - or anyone else - measured the complexity and can report some results here? Since the AAC-LC technology is from the mid 1990s, I'd be very curious.
I think the key benefit of LC3 over AAC-LC or HE-AAC is its much lower codec latency, making it the much better candidate for e.g. communication applications.
Chris
Have you - or anyone else - measured the complexity and can report some results here?
No, I did not personally confirm this.
I did not find any quantitative comparison of the decoder power consumption.
Obviously, developers have designed the LC3 codec with energy savings in mind, though. https://link.springer.com/book/10.1007/978-1-4842-6658-8
Android 13 developer preview is out, and it seems that it ships with LC3 related changes, e.g.:
https://developer.android.com/reference/android/bluetooth/BluetoothLeAudioCodecConfig#SOURCE_CODEC_TYPE_LC3
Did anybody try to extract and test the codec from the AOSP? Interesting how it compares to other AOSP Bluetooth codecs.
Android 13 developer preview is out, and it seems that it ships with LC3 related changes, e.g.:
https://developer.android.com/reference/android/bluetooth/BluetoothLeAudioCodecConfig#SOURCE_CODEC_TYPE_LC3
Did anybody try to extract and test the codec from the AOSP? Interesting how it compares to other AOSP Bluetooth codecs.
The AOSP LC3 codec needs no extracting, since Google made https://github.com/google/liblc3 a separate thing on GitHub. PipeWire already uses it.