Skip to main content
Topic: Personal Listening Test of LAME, iTunes and Helix MP3 encoders (Read 8763 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Abstract:
Blind sound quality comparison between MP3 encoders at 192 kbps.

Encoders:
Helix mp3 encoder v5.1 -V111 -HF2
LAME 3.99.5 -V2.3
LAME 3.100.alpha2 -V2.75
iTunes 12.4.2.4 high quality default, vbr enabled

Samples:
Total 15 samples from my corpus, tested twice per sample.
Total 12 samples from IgorC's test tracks, tested twice per sample.

Hardwares:
Sony PSP-3000 + RP-HT560(1st), RP-HJE150(2nd).

Results:

I will post the bitrate of those settings over diverse albums in 2016 Dec 31. The average bitrate of the album and sample bitrate is very close to 192kbps and equal.



Conclusions & Observations:
The superiority of the Helix mp3 encoder is effectively proved in this test. Helix encodes diverse music tracks at better audio quality using the same bitrate compared to the LAME.
The experimental version LAME 3.100.alpha2 does not offer significantly better average quality, compared to the current stable version of the LAME 3.99.5.
The iTunes has not caught up to the quality of LAME encoders in the mp3 encoding, as of August 2016.

Anova analysis:
Code: [Select]
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 27
Critical significance:  0.05
Significance of data: 1.42E-009 (highly significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of         Degrees     Sum of    Mean
variation         of Freedom  squares   Square    F      p

Total              107           8.62
Testers (blocks)    26           4.02
Codecs eval'd        3           1.98    0.66   19.62  1.42E-009
Error               78           2.62    0.03
---------------------------------------------------------------
Fisher's protected LSD for ANOVA:   0.099

Means:

helixv   l3995v   l100a2   itunes  
  4.46     4.35     4.32     4.09  

---------------------------- p-value Matrix ---------------------------

         l3995v   l100a2   itunes  
helixv   0.029*   0.005*   0.000*  
l3995v            0.530    0.000*  
l100a2                     0.000*  
-----------------------------------------------------------------------

helixv is better than l3995v, l100a2, itunes
l3995v is better than itunes
l100a2 is better than itunes

Raw data:
Code: [Select]
LAME3.100α2	LAME3.99.5	HelixV5.1	iTunes12.4.2.4	
%feature 10 -V2.75 -V2.3 -HF2 -V111 High Quality VBR
%feature 11 201k 201k 197k 194k
4.400 4.450 4.350 3.900
4.300 4.200 4.500 4.150
4.400 4.100 5.000 4.400
4.450 4.250 4.800 4.400
4.350 4.250 4.050 3.850
4.400 4.150 4.300 3.900
4.250 4.300 4.550 4.200
4.500 4.350 4.700 4.200
4.200 4.050 3.950 3.700
4.250 4.500 4.750 3.950
4.250 4.250 4.150 4.200
3.800 3.850 4.200 3.950
4.100 4.400 4.500 4.150
4.000 4.300 4.400 3.750
4.400 4.400 4.700 3.950
4.100 4.150 4.350 3.750
4.350 4.450 4.750 3.750
4.100 4.250 4.350 3.650
4.350 4.450 4.450 4.300
4.300 4.250 4.400 4.300
4.350 4.600 4.450 4.050
4.100 4.150 4.250 4.100
4.800 5.000 4.850 4.300
4.600 4.600 4.050 4.300
4.100 4.350 4.500 4.550
5.000 4.900 4.750 4.600
4.450 4.550 4.450 4.150
%samples 10.41_30sec hihats
%samples 11.finalfantasy cemb.
%samples 12.ATrain Jazz
%samples 13.BigYellow Pops
%samples 14.FloorEssence Techno
%samples 15.macabre orch
%samples 16.mybloodrusts guitar
%samples 17.Quizas Latin
%samples 18.VelvetRealm Techno
%samples 19.Amefuribana Pops
%samples 20.Trust Gospel
%samples 21.Waiting Rock
%samples 22.Experiencia Latin
%samples 23.Heart to Heart Pops
%samples 24.Tom's Diner Vocal
%samples 01 castanets inst.
%samples 02 fatboy_30sec Techno
%samples 03 eig Techno
%samples 04 Bachpsichord inst.
%samples 05 Enola Techno
%samples 06 trumpet inst.
%samples 07 applaud Live
%samples 08 velvet perc.
%samples 09 Linchpin Rock
%samples 10 spill_the_blood guitar
%samples 11 female_speech Speech
%samples 12 French_Ad Speech

Bitrates:
Code: [Select]
%bitrate
212892 213747 237940 192993
168438 155646 165261 192912
192156 192103 209717 193442
204127 205068 203986 193170
232176 233635 188200 193399
194632 186036 195050 193630
190862 202640 184752 193518
202882 200424 205121 193007
234691 231216 207721 194882
180358 180686 181454 193118
203168 206754 201816 192950
186724 189212 210432 193448
197553 195467 192937 193377
194530 194412 205535 193053
163321 166355 189490 193560
203531 209582 201625 196098
272590 277249 283337 193087
212430 217041 216048 194049
192217 176450 186486 193275
198941 190717 199853 193050
236340 228336 202575 194692
234272 230913 216725 195333
240073 247948 240665 194388
193818 198111 167236 193980
180981 180072 180092 192959
139441 145012 112302 193536
177329 180439 146029 194422

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #1
Thanks for the test, Kamedo!
I wonder what the authors of Helix did to make it so damn good.
How about encoding speed? As far as I remember, Helix was also significantly faster than Lame.

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #2
great Kamedo2.

Interesting test. Oldies never die (as of Helix)

I wonder how LAME and Helix scale up to  256 kbps VBR.

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #3
I added the album bitrates of all encoder settings to the graph image.

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #4
Hi Kamedo2,

thank you for testing these mature codecs. The results are still surprisingly - at least to me.
Could you please tell me why you have choosen the -HF2 switch in case of the Helix codec?
I just did a short test and found out that -HF2 emphasizes the coding of signals >16 kHz more than the alternative -HF switch?
Doesn't this automatically lead to a lower coding quality in the "important" region <16 kHz where other codecs often apply
their low-pass filters? Have you testet -HF2 vs. -HF switch?

... and a Happy New Year!

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #5
For anyone who would like to try Helix, I have found these parameters work fine in foobar2000 converter:
Code: [Select]
- %d -V150 -HF2 -X2 -U2
-Vn is the VBR quality: 0-150 (150 is about 235 kpbs)
-HF2 is optional; I used it to be consistent with what OP did
-X2 I found necessary otherwise you get a bad VBR header; foobar2000 reports wrong track length and kbps
-U2 tells the encoder to use SSE (Pentium II and above)
-h will give you the detailed help page including all available switches

You can find the encoder at RareWares: http://www.rarewares.org/mp3-others.php
Development appears to have stopped in 2005.

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #6
Could you please tell me why you have choosen the -HF2 switch in case of the Helix codec?
Actually, I tested the Helix without the -HF2 option on 224kbps, and it was not significantly better than the Lame.
https://hydrogenaud.io/index.php/topic,100896.0.html
So I needed something different to get some clearer results. Desteroid's comment sounds like using -HF1 or -HF2 on high bitrates is a good idea.
https://hydrogenaud.io/index.php/topic,100896.msg834601.html#msg834601

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #7
For those who need or prefer to use constant bit rate with the Helix mp3 encoder, these parameters work fine in foobar2000:
Code: [Select]
-X2 -U2 -B128 -HF2 -F18500 - %d

B128 corresponds to 256kbps CBR.  Use B160 if 320kbps is desired.  The -F command sets the lowpass filter.  Adjust or omit as preferred. 

The Helix encoder is lightning fast on my older laptop.  It will encode 500 songs from FLAC in about 10 minutes.  Perfect for a quick playlist switch on a micro SD card I use in an old flip phone with buggy software that doesn't like VBR and doesn't play gapless.  Lack of gapless encoding is the only drawback I know of in the Helix encoder.

Best regards.  LedHed8 

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #8
Lack of gapless encoding is the only drawback I know of in the Helix encoder.
I also noticed that Helix introduces sector boundary errors; LAME does not.

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #9
Lame 3.99 / 3.100 is very fast with -f and can be gapless with certain players. The quality of -f is fine and some problematic samples are  better strangely.  One could assume using --noreplaygain -f -V2 ..1.5..1.0  will bring it on-par.

wavpack 4.8 -b3x6c

Re: Personal Listening Test of LAME, iTunes and Helix MP3 encoders

Reply #10
The score improvement of each encoders, relative to the LAME 3.99.5 VBR.
Unfortunately, LAME 3.100 alpha2 is more likely to degrade quality than to improve quality.

 
SimplePortal 1.0.0 RC1 © 2008-2019