Abstract:
Personal blind sound quality comparison of the Bluetooth codec, AAC and LC3, at 192 kbps, and SBC at 345 kbps.
AAC is an optional codec in Bluetooth, but Bluetooth earphones and speakers usually support the AAC codec.
SBC is the mandatory codec in Bluetooth.
LC3 is a new codec, which became mandatory in the LE Audio in Bluetooth 5.2.
Lower bit rate settings were also tested.
The test was conducted on 48 kHz input on all codecs.
Encoders:
AAC: fdkaac-1.0.4 built against fdk-aac 2.0.2, x64, from rarewares, Feb 2023 version (not the latest version, as of May 2023).
SBC: FFmpeg N-109823-g385ec46424, Feb 2023 version (not the latest version, as of May 2023).
LC3: Google / liblc3 v1.0.1, Feb 2023 version (not the latest version, as of May 2023).
AAC Settings:
fdkaac -b 128000 -o out.mp4 in.48kHz.16bit2ch.wav
fdkaac -b 192000 -o out.mp4 in.48kHz.16bit2ch.wav
SBC Settings:
ffmpeg -y -i in.48kHz.16bit2ch.wav -c:a sbc -b:a 237k out.sbc
ffmpeg -y -i in.48kHz.16bit2ch.wav -c:a sbc -b:a 345k out.sbc
LC3 Settings:
elc3 -b 128000 in.48kHz.16bit2ch.wav out.lc3
elc3 -b 192000 in.48kHz.16bit2ch.wav out.lc3
Sample tracks:
15 sound samples from Kamedo2's samples.
12 sound samples from IgorC's samples.
Total 27 diverse music and speech sound samples.
Hardware:
Sony PSP-3000 + AKG K712.
Results (only the higher bitrate):
Results (including lower bitrates):
Conclusions & Observations:
- AAC offered very high fidelity at 192 kbps. Most scores were better than 4.0 (Perceptible, but not annoying).
- SBC also offered very high fidelity at 345 kbps. Most scores were better than 4.0 (Perceptible, but not annoying).
- LC3, also offered very high fidelity at 192 kbps, with majority of scores better than 4.0. Still not as good as AAC or SBC.
- It's not clear whether AAC at 192 kbps is better, or SBC at 345 kbps is better from this test alone.
- Reducing the bitrate to about two-thirds significantly impaired sound quality on all Bluetooth codecs tested.
Anova analysis:
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis
Number of listeners: 27
Critical significance: 0.05
Significance of data: 0.00E+000 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings
Source of Degrees Sum of Mean
variation of Freedom squares Square F p
Total 161 69.27
Testers (blocks) 26 9.59
Codecs eval'd 5 37.25 7.45 43.18 0.00E+000
Error 130 22.43 0.17
---------------------------------------------------------------
Fisher's protected LSD for ANOVA: 0.224
Means:
FDK-AAC1 SBC345kb liblc319 FDK-AAC1 SBC237kb liblc312
4.72 4.65 4.13 4.12 3.62 3.42
---------------------------- p-value Matrix ---------------------------
SBC345kb liblc319 FDK-AAC1 SBC237kb liblc312
FDK-AAC1 0.578 0.000* 0.000* 0.000* 0.000*
SBC345kb 0.000* 0.000* 0.000* 0.000*
liblc319 0.961 0.000* 0.000*
FDK-AAC1 0.000* 0.000*
SBC237kb 0.079
-----------------------------------------------------------------------
FDK-AAC192kbps is better than liblc3192kbps, FDK-AAC128kbps, SBC237kbps, liblc3128kbps
SBC345kbps is better than liblc3192kbps, FDK-AAC128kbps, SBC237kbps, liblc3128kbps
liblc3192kbps is better than SBC237kbps, liblc3128kbps
FDK-AAC128kbps is better than SBC237kbps, liblc3128kbps
Raw data:
By mistake, 04 Bachpsichord and 05 Enola were tested twice. The two scores were averaged in below.
FDK-AAC128kbps FDK-AAC192kbps liblc3128kbps liblc3192kbps SBC237kbps SBC345kbps
%genre Kamedo2's 15 sample
3.700 4.500 3.200 4.200 3.500 5.000
4.800 4.700 3.300 4.100 3.100 4.500
3.900 5.000 3.100 3.500 3.700 5.000
4.100 5.000 3.500 4.400 3.700 5.000
3.900 5.000 3.300 4.500 4.200 5.000
4.200 5.000 3.300 3.900 2.800 4.400
4.900 5.000 3.400 4.500 3.700 4.700
4.200 5.000 3.400 4.100 3.100 4.300
3.100 3.800 3.500 4.100 3.300 4.200
3.900 5.000 3.100 3.800 2.700 5.000
3.800 4.500 3.500 4.200 4.300 4.400
3.700 5.000 3.400 4.100 3.500 4.600
4.300 5.000 3.400 3.900 3.100 4.500
4.200 4.800 3.300 4.100 2.700 5.000
5.000 4.700 2.900 3.700 2.600 3.800
%genre IgorC's 12 sample
4.400 4.700 3.400 3.900 4.600 5.000
4.200 4.500 3.900 4.400 5.000 5.000
4.300 4.500 3.500 3.900 4.800 5.000
4.000 4.800 3.200 4.150 2.800 4.450
4.000 4.750 3.250 4.200 3.450 4.000
4.200 4.300 3.200 3.800 2.500 3.900
3.800 4.100 3.400 4.300 5.000 4.400
3.700 5.000 3.600 4.200 3.900 5.000
3.400 4.100 3.700 4.200 4.300 4.500
4.300 4.600 3.200 4.200 3.900 5.000
5.000 5.000 5.000 5.000 3.700 5.000
4.300 5.000 3.400 4.100 3.800 5.000
%samples 41_30sec Perc.
%samples finalfantasy Strings
%samples ATrain Jazz
%samples BigYellow Pops
%samples FloorEssence Techno
%samples macabre Classic
%samples mybloodrusts Guitar
%samples Quizas Latin
%samples VelvetRealm Techno
%samples Amefuribana Pops
%samples Trust Gospel
%samples Waiting Rock
%samples Experiencia Latin
%samples Heart to Heart Pops
%samples Tom's Diner Acappella
%samples 01 castanets inst.
%samples 02 fatboy_30sec Techno
%samples 03 eig Techno
%samples 04 Bachpsichord inst.
%samples 05 Enola Techno
%samples 06 trumpet inst.
%samples 07 applaud Live
%samples 08 velvet perc.
%samples 09 Linchpin Rock
%samples 10 spill_the_blood guitar
%samples 11 female_speech Speech
%samples 12 French_Ad Speech
Other tests: