The purpose of this thread is to (finally) test LAME 3.96 throughly enough to make it the new recommended LAME version and enhance communication between LAME developers and the HA.org community. The recent confusion about the different LAME versions and compiles (-Qrcd trouble etc.) have led to the decision that moving along with LAME development has been postponed for long enough and LAME 3.90.X can hopefully be declared dead very soon.
Please test as many samples as possible according to the following guidelines and post any results here.
1. Use the following LAME compiles
(updated!):
lame3.90.3 (http://www.rarewares.org/files/mp3/lame-3.90.3.zip)
lame3.96b2 (http://www.rarewares.org/files/mp3/lame3.96b2.zip)
2. The focus of the test should be --alt-preset/--preset standard, since it will allow us to make conclusions regarding the overall performance of the 'code level tweaked' VBR presets. Other VBR/ABR/CBR presets are interesting too, but not as important. If problems with --alt preset standard are detected feel free to compare extreme and insane too. Test the following combinations please:
(alt)presets + VBR/ABR(320kbps) 3.96 --preset insane vs. 3.90.3 --alt-preset insane
(~256kbps) 3.96 --preset extreme vs. 3.90.3 --alt-preset extreme
(~210kbps)
[span style='font-size:11pt;line-height:100%']3.96 --preset standard vs. 3.90.3 --alt-preset standard[/span](~160kbps) 3.96 -V 4 vs. 3.96 --preset 160 vs. 3.90.3 --alt-preset 160
(~128kbps) 3.96 -V 5 vs. 3.96 --preset 128 vs. 3.90.3 --alt-preset 128
CBR3.96 --preset cbr <bitrate> vs. 3.90.3 --alt-preset cbr <bitrate>
If you want, you can additionally test VBR/ABR vs. CBR at comparable bitrates
3. You may use any sample you want, as long as you upload a losslessly compressed version of that (or provide a working link), so others can verify your results.
4. Your test results have to include the following:
- ABX results for
3.90.3 vs. Original
3.96b2 vs. Original
3.96b2 vs. 3.90.3 - ABC/HR results are appreciated especially at lower bitrates, but shouldn't be considered a requirement.
- (Short) descriptions of the artifacts/differences
Notes:
This thread is for results only, discuss
about the test here (http://www.hydrogenaudio.org/forums/index.php?showtopic=19876&), please.
The earlier 'regression examples" thread has been closed and we'll try to add all results from it to the collected test results below. The original thread is still available here (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&).
Please provide
direct links to the sample you tested with or upload it here (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&), even if it's a well-known one - it'll make things much easier for others.
[span style='font-size:7pt;line-height:100%']Edit by tigre, Mar 31 2004: Updated to 3.96b2 and modified/simplified medium bitrate ABR/VBR test recommendations according to 3.96 -V x mass encode average bitrates.[/span]
Test results so far:
320kbps
3.96b1 --p insane < 3.90.3 --ap insane :: Lazy_Jones (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191243) :: owowo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=196184) :: 0x verified so far
~ 256kbps
3.96b1 --p extreme < 3.90.3 --ap extreme :: Lazy_Jones (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191243) :: owowo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=196184) :: 0x verified so far
~ 210kbps
3.96b1 --p standard < 3.90.3 --ap standard :: myf_4sec (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195030) :: LoFiYo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195035) :: 2x verified by Wombat (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195198), harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&hl=)
3.96b2 : 1x verified by LoFiYo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=198120)
3.96b1 --p standard < 3.90.3 --ap standard :: Rebel (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=195375) :: Proxima (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195374) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=197239)
3.96b1 --p standard < 3.90.3 --ap standard :: (-) Ions (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191563) :: Westgroveg (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=192631) :: 1x verified so far by harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=193277)
3.96b1 --p standard < 3.90.3 --ap standard :: Hustlejet (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=187351) :: harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=193210) :: 0x verified so far
3.96b2 : 1x verified by harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=198402)
3.96b1 --p standard < 3.90.3 --ap standard :: trumpets1 (http://www.hydrogenaudio.org/forums/index.php?act=ST&f=16&t=3594) :: SometimesWarrior (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=193659) :: 0x verified so far
3.96b1 --p standard < 3.90.3 --ap standard :: Doesnair (http://members.cox.net/moitah/doesnair.flac) :: Moitah (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=191561) :: 1x verified by harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&hl=)
3.96b1 --p standard < 3.90.3 --ap standard :: cantwait (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195478) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195480) :: 2x verified by harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195512), Moitah (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195526)
3.96b1 --p standard < 3.90.3 --ap standard :: death2 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=195528):: SometimesWarrior (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=193714) :: 1x verified by harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&hl=)
3.96b1 --p standard < 3.90.3 --ap standard :: Destitute (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=196998) :: harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=197001) :: 0x verified so far
3.96b1 --p standard < 3.90.3 --ap standard :: Rosemary (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=196998) :: harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=197001) :: 0x verified so far
3.96b1 --p standard < 3.90.3 --ap standard :: drone_short (http://www.ff123.net/samples/drone_short.flac%5b/url) :: freakngoat (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=197017) :: 0x verified so far
3.96b1 --p standard < 3.90.3 --ap standard :: Chanchan1 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=197226) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=197228) :: 1x verified by harashin (http://www.hydrogenaudio.org/forums/index.php?act=ST&f=40&t=19813&st=0)
3.96b2 : 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=199021)
3.96b1 --p standard < 3.90.3 --ap standard :: Hosokawa___Atem_lied (http://www.hydrogenaudio.org/forums/index.php?showtopic=18360&view=findpost&p=188327) :: harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=197965) :: 0x verified so far
3.96b2 : 1x verified by harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=198402)
3.96b2 --p standard = 3.90.3 --ap standard :: 41_30sec (http://www.ff123.net/samples/41_30sec.flac) :: ViPER1313 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=198434)
3.96b1 --p standard > 3.90.3 --ap standard :: spahm (http://www.ff123.net/samples.html) :: Pio2001 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195221) :: 1x verified by Wombat (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195471)
3.96b1 --p standard > 3.90.3 --ap standard :: Birds (http://www.halke.net/files/Birds.flac) :: Wombat (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195226) :: 0x verified so far
3.96b1 --p standard > 3.90.3 --ap standard :: hokuscaredpiano (http://members.cox.net/moitah/hokuscaredpiano.flac) :: Moitah (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195477) :: 0x verified so far
3.96b1 --p standard > 3.90.3 --ap standard :: Lazy_Jones (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191243) :: owowo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195560) :: 1x verified by freakngoat (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=196983)
~ 192kbps
no results so far
~ 160kbps
3.96b2 --p 160 > 3.90.3 --ap 160 > 3.96b2 -V 4 :: myf_4sec (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195030) :: LoFiYo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=199397) :: 0x verified so far + missing ABX results*
~ 144kbps
no results so far
~ 128kbps
VBR/ABR
3.96b1 --p 128 < 3.90.3 --ap 128 :: Quizas (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=184033) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195434) :: 0x verified so far
3.96b1 --p 128 < 3.90.3 --ap 128 :: Doesnair (http://members.cox.net/moitah/doesnair.flac) :: Moitah (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=191561) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 < 3.90.3 --ap 128 :: (-) Ions (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191563) :: PVNC (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=191564) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 < 3.90.3 --ap 128 :: Applaud (http://www.mp3dev.org/mp3/gpsycho/quality.html) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195615) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 < 3.90.3 --ap 128 :: Bassdrum (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195611) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195615) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 < 3.90.3 --ap 128 :: Blackwater (http://www.ff123.net/samples.html) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195615) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 < 3.90.3 --ap 128 :: Campestre (http://xoomer.virgilio.it/fofobella/campestre.flac) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195615) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 < 3.90.3 --ap 128 :: Fall (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195618) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195615) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 < 3.90.3 --ap 128 :: Iron (http://www.mp3dev.org/mp3/gpsycho/quality.html) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195615) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 < 3.90.3 --ap 128 :: Thewayitis (http://www.ff123.net/samples.html) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195615) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 < 3.90.3 --ap 128 :: Tosca (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195614) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195615) :: 0x verified so far + missing ABX results*
3.96b1 --p 128 > 3.90.3 --ap 128 :: fatboy (http://www.mp3dev.org/mp3/gpsycho/quality.html) :: sony666 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=194932) :: 2x verified so far by Wombat (http://www.hydrogenaudio.org/forums/index.php?showuser=235), FatBoyFin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=198412)
3.96b1 --p 128 -q 0 < 3.90.3 --ap 128 :: Velvet (http://www.mp3dev.org/mp3/gpsycho/quality.html) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195615) :: 0x verified so far + missing ABX results*
3.96b1 -V 5 > 3.90.3 --ap 128 :: Quizas (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=184033) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195434) :: 0x verified so far
3.96b2 --V 5 > 3.96b1 --p 128 > 3.90.3 --ap 128 :: fatboy (http://www.mp3dev.org/mp3/gpsycho/quality.html) :: FatBoyFin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=199620) :: 0x verified so far
3.96b2 --V 5 > 3.90.3 --ap 128 > 3.96b1 --p 128 :: Its_me (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=201613) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=201617) :: 0x verified so far
3.96b2 --V 5 > 3.90.3 --ap 128 > 3.96b1 --p 128 :: entierren con rumba (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=201619) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=201622) :: 0x verified so far
CBR
3.96b1 --p cbr 128 < 3.90.3 --ap cbr 128 :: Quizas (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=184033) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195434) :: 0x verified so far
[span style='font-size:8pt;line-height:100%']* If a result doesn't match the minimum requirements (-> 4. in 1st post), the result is 'greyed out' until the missing data is provided or someone else confirms the results. If your result isn't included in this list at all, we're either too slow, or there's too much missing (ABX results, link to the sample, clear statment which version is better, information about lame version/setting used etc.). If you want to provide missing information, do it in a new post to ensure that we notice it.[/span]
3.96b1 shows great improvement with fatboy and --alt-preset 128 (ABR)
-wav vs 3.90.3: 8/8, horribly distorted "vocals", "knocking" artifact 1.5-3.0s
-wav vs 3.96b1: 8/8 badly distorted vocals
-3.90.3 vs 3.96b1: 8/8, clearly less distortion in 3.96b1
I tried aps, but failed miserably on both encoders this time.
I used this (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=195030) to test APS 3.90.3 vs APS 3.96b1.
3.90.3 APS was 138kbps; 3.96b1 APS was 116kbps. If you don't have a golden set of ears, I recommend you try this sample. It's very easy to ABX .
[span style='font-size:8pt;line-height:100%']ABC/HR Version 0.9b, 30 August 2002
Testname: My Funny Valentine - 3.90.3 vs 3.96b1
1R = C:\My Music\lab\MYF-4SEC\aps396b1.wav
2R = C:\My Music\lab\MYF-4SEC\aps3903.wav
---------------------------------------
General Comments:
#2 is the obvious winner.
---------------------------------------
1R File: C:\My Music\lab\MYF-4SEC\aps396b1.wav
1R Rating: 1.0
1R Comment: Distortion is very noticeable throughout the sample.
---------------------------------------
2R File: C:\My Music\lab\MYF-4SEC\aps3903.wav
2R Rating: 3.9
2R Comment: Much better than #1. Perceptible and almost not annoying.
---------------------------------------
ABX Results:
Original vs C:\My Music\lab\MYF-4SEC\aps396b1.wav
8 out of 8, pval = 0.004
Original vs C:\My Music\lab\MYF-4SEC\aps3903.wav
8 out of 8, pval = 0.004
C:\My Music\lab\MYF-4SEC\aps396b1.wav vs C:\My Music\lab\MYF-4SEC\aps3903.wav
8 out of 8, pval = 0.004[/span]
Just verified myf_4sec with aps
I could ABX 396b -> original 8/8
and 3903 -> original 5/8
As this sample is new to me i am surprised how obvious 396b suffers here with the third blow of the trumpet!
Wombat
I begin with the easiest samples for me. I included MPC out of curiosity, and WMA9 for reasons that are not necessary to explain here.
Lame 3.90.3 --alt-preset standard
Lame 3.96b1 --preset standard
Mppenc 1.14 --xlevel
WMA9 VBR 135-215 kbps
Sample : Badvilbel
Wav vs 3.90.3 ABX 11/11 Lots of pre echo, loss of transients, drop outs
Wav vs 3.96b1 ABX 8/8 Same as above
Wav vs MPC ABX 7/8 Gurgling on transients, ringing, drop outs, but better
Wav vs WMA9 ABX 8/8 Best, only transient problems, no drop outs at all
3.96.1 vs WMA9 : ABX 7/8
3.96.1 vs MPC : ABX 8/8
Conclusion of 3.90.3 vs 3.96b1 :
Same quality
Sample : Drone Short
3.90.3 : ABX 16/16 toooo easy ! Artifacts during the first half second, and time smearing during the second half.
3.96b1 : ABX 16/16 Somewhat louder artifacts, but not enough to talk about regression, just chance.
90 vs 96 : 11/14 (3% chance of guessing). The artifacts are not at the same place.
MPC : perfect. Maybe a tiny noise ? ABX 3/8, no.
WMA9 : 16/16, unacceptable quality
Conclusion of 3.90.3 vs 3.96b1 :
Same quality
Spahm
3.90.3 : Noise. 16/16
3.96.1 : perfect. Maybe a tiny noise ? 8/8 It seems so...
3.90.3 vs 3.96.1 : 16/16
MPC : Same effect as transients in badvilbel : grungy instead of hissy (mp3) 16/16
WMA : worse. 16/17 (I went too fast and made one mistake).
Conclusion of 3.90.3 vs 3.96b1 :
Big improvement Sample nearly completely solved.
For these three samples, compared to 3.90.3 :
3.96b1 is better one time, equal two times
MPC equal one time, slightly better one time, better one time.
WMA9 is unlistenable one time, worse one time, better one time.
Overall concusion : none for the time being. Results are sample dependant.
More tests to come, but with more difficult samples. It will take longer.
These samples are available on FF123's page : http://www.ff123.net/samples.html (http://www.ff123.net/samples.html)
I wrote befor in another thread about the birds sample and aps
3903 -> original 16/16 clear artifact when she sings the first "e" of become
396b only guessing
396b -> 3.903 8/8
396b is the first aps without a problem to me and birds.
The sample itself is now located here:
http://www.hydrogenaudio.org/forums/index....=0&#entry195385 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&st=0&#entry195385)
I did 16 this time, for trying how long i can clearly discern. I am not used to use abx tests much.
I have to admit it is not that easy, even when you know the problem in the test file.
Edit: location of the sample file
Wombat
Sample: rebel.wav
Artifact: Preecho.
ABX results (range 0:01-0:03 sec.):
Original vs. 3.90.2 (--aps -Z) 13/14
Original vs. 3.96b1 (--aps) 10/10
3.90.2 vs 3.96b1 10/10
Both non-transparent but 3.90.2 is slightly better.
As already reported in my previous test, the same sample also reveal ringing problem with 3.96b1 (--ap 128).
The sample is available in the apposite thread.
Sample: Quizas (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=184033)
Tested lame settings:
3.90.3 --alt-preset 128 (132kbps)
3.90.3 --alt-preset cbr 128 (128kbps)
3.96b1 --preset 128 (135kbps)
3.96b1 --preset cbr 128 (128kbps)
3.96b1 -V 5 (138kbps)
Decoded with fb2k with trackgain enabled
Ratings:
3.96b1 -V 5 >> 3.90.3 --alt-preset 128 > 3.90.3 --alt-preset cbr 128 >> 3.96b1 --preset 128 > 3.96b1 --preset cbr 128
ABC/HR results:
ABC/HR Version 0.9b, 30 August 2002
Testname: Quizas 128kbps
1L = quizas.wav_3903_ap128.wav
2R = quizas.wav_396b1_pcbr128.wav
3R = quizas.wav_396b1_V5.wav
4L = quizas.wav_396b1_p128.wav
5L = quizas.wav_3903_apcbr128.wav
---------------------------------------
General Comments:
Ratings bases on 1st half (0.0-14.0), biggest problems there.
---------------------------------------
1L File: quizas.wav_3903_ap128.wav
1L Rating: 3.0
1L Comment: pre-echo/smearing, easy to ABX, e.g. 8.6-10.9
---------------------------------------
2R File: quizas.wav_396b1_pcbr128.wav
2R Rating: 1.0
2R Comment: pre-echo + ringing/chirping, easy to ABX, e.g. 8.6-10.9
---------------------------------------
3R File: quizas.wav_396b1_V5.wav
3R Rating: 4.0
3R Comment: only very small smearing of percussions, ABXed 8.6-10.9
---------------------------------------
4L File: quizas.wav_396b1_p128.wav
4L Rating: 1.5
4L Comment: preecho and hf chirping/warbling, ABXed 5.1-8.6
---------------------------------------
5L File: quizas.wav_3903_apcbr128.wav
5L Rating: 2.5
5L Comment: preecho, easy to ABX, e.g. 8.6-10.9
---------------------------------------
ABX Results:
Original vs quizas.wav_3903_ap128.wav
10 out of 11, pval = 0.006
Original vs quizas.wav_396b1_pcbr128.wav
7 out of 7, pval = 0.008
Original vs quizas.wav_396b1_V5.wav
11 out of 12, pval = 0.003
Original vs quizas.wav_396b1_p128.wav
7 out of 7, pval = 0.008
Original vs quizas.wav_3903_apcbr128.wav
7 out of 7, pval = 0.008
quizas.wav_3903_ap128.wav vs quizas.wav_396b1_pcbr128.wav
11 out of 12, pval = 0.003
quizas.wav_3903_ap128.wav vs quizas.wav_396b1_V5.wav
7 out of 7, pval = 0.008
quizas.wav_3903_ap128.wav vs quizas.wav_396b1_p128.wav
7 out of 7, pval = 0.008
quizas.wav_3903_ap128.wav vs quizas.wav_3903_apcbr128.wav
11 out of 12, pval = 0.003
quizas.wav_396b1_pcbr128.wav vs quizas.wav_396b1_V5.wav
7 out of 7, pval = 0.008
quizas.wav_396b1_pcbr128.wav vs quizas.wav_396b1_p128.wav
8 out of 8, pval = 0.004
quizas.wav_396b1_pcbr128.wav vs quizas.wav_3903_apcbr128.wav
7 out of 7, pval = 0.008
quizas.wav_396b1_V5.wav vs quizas.wav_396b1_p128.wav
7 out of 7, pval = 0.008
quizas.wav_396b1_V5.wav vs quizas.wav_3903_apcbr128.wav
7 out of 7, pval = 0.008
quizas.wav_396b1_p128.wav vs quizas.wav_3903_apcbr128.wav
11 out of 12, pval = 0.003
3.96 --p standard < 3.90.3 --ap standard :: isitloveintro (http://members.cox.net/moitah/isitloveintro.flac) :: Moitah (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=191783) :: 0x verified so far + missing ABX results*
Could you remove this one? It isn't worse in 3.96, I was able to ABX for 3.90.3 as well (see further down in the original thread).
3.96 --p standard < 3.90.3 --ap standard :: Doesnair (http://members.cox.net/moitah/doesnair.flac) :: Moitah (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&view=findpost&p=191561) :: 0x verified so far + missing ABX results*
This sample is almost certainly a regression... I will do more thorough testing to make sure. Can someone else verify this (listen where the snair is hit, and its a good idea to start with --p 128 because the artifact is a lot more obvious)?
Sample: doesnair (http://members.cox.net/moitah/doesnair.flac)
3.90.3 --ap standard vs 3.96b1 --p standard
Original vs H:\lame\doesnair\1579_doesnair-390-s.wav
14 out of 16, pval = 0.002
Original vs H:\lame\doesnair\1583_doesnair-396-s.wav
16 out of 16, pval < 0.001
H:\lame\doesnair\1579_doesnair-390-s.wav vs H:\lame\doesnair\1583_doesnair-396-s.wav
13 out of 16, pval = 0.011
Both aren't transparent vs original, but 3.96b1 is worse.
spahm
This sample makes my ears vibrating somehow after too many tries!
3903 8/8 pretty easy i only listen the first beats and the added noise is pretty obvious
396b 7/8 the same noise but to a much lesser degree
hustlejet
just gave me helpless clicky, clicky and when i thought i heard the problem - Nada!
Not my cup of sample
btw. ff123 did a nice job with abchr!!
Wombat
Sample: hokuscaredpiano (http://members.cox.net/moitah/hokuscaredpiano.flac) (Hoku - I'm Scared)
3.96b1 --p standard vs 3.90.3 --ap standard
Original vs H:\lame\hokuscaredpiano\1586_hokuscaredpiano-396-s.wav
15 out of 16, pval < 0.001
Original vs H:\lame\hokuscaredpiano\1585_hokuscaredpiano-390-s.wav
14 out of 16, pval = 0.002
H:\lame\hokuscaredpiano\1586_hokuscaredpiano-396-s.wav vs H:\lame\hokuscaredpiano\1585_hokuscaredpiano-390-s.wav
13 out of 16, pval = 0.011
I'm hearing a problem on the piano (first chord change, 1.87 secs in), 3.96b1 sounds a bit better .
cantwait (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195478)
I wanted to use this CD for testing lower bitrates but at 128kbps the artifacts were so obvious that I decided to start with (alt)preset standard.
Both, 3.90.3 --alt-preset standard and 3.96b1 --preset standard make the singer's voice (or it's echo) sound metallic in many places. One easily noticable example is the "s" of the word "understand" at ~ 8 seconds. Other "s" sounds (but also other places) have similar problems. 3.96b1 is clearly worse.
ABXed with fb2k, original vs. mp3 9/9 each time, 3.90.3 vs. 3.96b1 11/12
EDIT:
Edited in Wombat's + Moitah's (and my) results.
P.S.: Can someone verify my findings, please - is it just me or can this be considered as lame problem/killer sample?
hustlejet
just gave me helpless clicky, clicky and when i thought i heard the problem - Nada!
Not my cup of sample
That's bad , it seems even ReplayGain didn't fix its clipping issue(I don't notice here though) on your system.
Anyway, I was able to confirm tigre's cantwait sample. Somehow I can't reach my hosting space at CyberQuébec today, I write these results here.
A file: I:\test\cantwait\cantwait.wav B file: I:\test\cantwait\3.96b1aps.wav
10/10 p< 0.1%
A file: I:\test\cantwait\cantwait.wav B file: I:\test\cantwait\3.90.3aps.wav
4/15 p=98.2%
A file: I:\test\cantwait\3.90.3aps.wav B file: I:\test\cantwait\3.96b1aps.wav
15/20 p=2.1%
EDIT: The server is back now. Results (http://cyberquebec.ca/harashin/cantwait_LAME_test.htm)
P.S.: Can someone verify my findings, please - is it just me or can this be considered as lame problem/killer sample?
Yeah, both are pretty easy to ABX against the original (--(alt-)preset standard):
Original vs H:\lame\1583_Max - Cant Wait Until Tonight (Dry Wurlitzer Mix)-396-s.wav
18 out of 20, pval < 0.001
Original vs H:\lame\1585_Max - Cant Wait Until Tonight (Dry Wurlitzer Mix)-390-s.wav
17 out of 20, pval = 0.001
Didn't have any luck ABXing them against eachother, but I wasn't concentrating as hard as I normally do... I'll try again later.
EDIT: Also, I only tried one part of the sample (the "s" in "understand").
EDIT 2: Took a break and switched to Grado SR225s (was using PortaPros before):
H:\lame\1585_Max - Cant Wait Until Tonight (Dry Wurlitzer Mix)-390-s.wav vs H:\lame\1583_Max - Cant Wait Until Tonight (Dry Wurlitzer Mix)-396-s.wav
20 out of 25, pval = 0.002
3.96b1 is worse.
Sample used: Lazy_Jones.flac (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191243)
3.96b1 --preset standard <268kbps>
vs. 3.90.3 --alt-preset standard <242kbps>
EDIT: Average bitrates according to Nero6
Encoded with lame.exe/LAMEDrop1.3 - Decoded with Nero6
-------------------------------------
WinABX v0.42 test report
03/21/2004 00:10:57
A file: C:\WINDOWS\Desktop\Lazy_Jones (wav).wav
B file: C:\WINDOWS\Desktop\Lazy_Jones 3.90.3 (--ap s).wav
00:11:59 1/1 p=50.0%
00:12:10 2/2 p=25.0%
00:12:19 3/3 p=12.5%
00:12:25 4/4 p=6.2%
00:12:29 5/5 p=3.1%
00:12:36 6/6 p=1.6%
00:12:40 7/7 p=0.8%
00:12:44 8/8 p=0.4%
00:12:55 9/9 p=0.2%
00:13:50 reset
00:14:18 test finished
-------------------------------------
WinABX v0.42 test report
03/21/2004 00:14:18
A file: C:\WINDOWS\Desktop\Lazy_Jones (wav).wav
B file: C:\WINDOWS\Desktop\Lazy_Jones 3.96b1 (--p s).wav
00:14:48 1/1 p=50.0%
00:14:58 1/2 p=75.0%
00:15:07 2/3 p=50.0%
00:15:16 3/4 p=31.2%
00:15:23 4/5 p=18.8%
00:15:31 5/6 p=10.9%
00:15:39 6/7 p=6.2%
00:15:49 7/8 p=3.5%
00:15:58 7/9 p=9.0%
00:16:28 reset
00:16:32 test finished
-------------------------------------
WinABX v0.42 test report
03/21/2004 00:17:47
A file: C:\WINDOWS\Desktop\Lazy_Jones 3.90.3 (--ap s).wav
B file: C:\WINDOWS\Desktop\Lazy_Jones 3.96b1 (--p s).wav
00:18:03 1/1 p=50.0%
00:18:12 2/2 p=25.0%
00:18:21 3/3 p=12.5%
00:18:30 4/4 p=6.2%
00:18:38 5/5 p=3.1%
00:18:47 6/6 p=1.6%
00:18:58 7/7 p=0.8%
00:19:09 8/8 p=0.4%
00:19:19 9/9 p=0.2%
00:19:22 reset
00:19:33 test finished
Comments:
3.90.3 --alt-preset standard; completely chokes on this one
3.96b1 --preset standard; near perfect accuracy
As requested by tigre i have to provide abx results for my listening test even if samples are very easy because there are few people that test a specific sample. I'm sorry if i'm too lazy to abx the two lame versions each other fot all the samples but you can always see ratings in the posted table (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&st=50&#) and comments. For the great majority of the samples 3.90.2 is better. Most samples should be well known to testers, nevertheless i uploaded some of them.
Original vs applaud396q0.wav
8 out of 8, pval = 0.004
Original vs applaud_3902.wav
11 out of 12, pval = 0.003
Original vs applaud_396b1.wav
10 out of 10, pval < 0.001
available here: http://www.mp3dev.org/mp3/gpsycho/quality.html (http://www.mp3dev.org/mp3/gpsycho/quality.html)
Original vs Bassdrum_3902.wav
10 out of 10, pval < 0.001
Original vs Bassdrum396q0.wav
8 out of 8, pval = 0.004
Original vs Bassdrum_396b1.wav
8 out of 8, pval = 0.004
sample uploaded
Original vs Blackwater_396b1.wav
8 out of 8, pval = 0.004
Original vs Blackwater_3902.wav
8 out of 8, pval = 0.004
Original vs Blackwater396q0.wav
8 out of 8, pval = 0.004
sample available here: http://www.ff123.net/samples.html (http://www.ff123.net/samples.html)
Original vs campestre396q0.wav
8 out of 8, pval = 0.004
Original vs campestre_396b1.wav
8 out of 8, pval = 0.004
Original vs campestre_3902.wav
8 out of 8, pval = 0.004
uploaded http://xoomer.virgilio.it/fofobella/campestre.flac (http://xoomer.virgilio.it/fofobella/campestre.flac)
Original vs emmtop_3902.wav
8 out of 8, pval = 0.004
Original vs emmtop_396b1.wav
8 out of 8, pval = 0.004
Original vs emmtop396q0.wav
14 out of 15, pval < 0.001
sorry, sample exceeds 30 sec.
Original vs fall_3902.wav
16 out of 28, pval = 0.286
Original vs fall_396b1.wav
8 out of 8, pval = 0.004
Original vs fall396q0.wav
18 out of 21, pval < 0.001
sample uploaded
Original vs iron396q0.wav
8 out of 8, pval = 0.004
Original vs iron_396b1.wav
8 out of 8, pval = 0.004
Original vs iron_3902.wav
8 out of 8, pval = 0.004
available here: http://www.mp3dev.org/mp3/gpsycho/quality.html (http://www.mp3dev.org/mp3/gpsycho/quality.html)
Original vs Rebel396q0.wav
8 out of 8, pval = 0.004
Original vs Rebel_3902.wav
8 out of 8, pval = 0.004
Original vs Rebel_396b1.wav
13 out of 14, pval < 0.001
sample uploaded
Original vs thewayitis_3902.wav
8 out of 8, pval = 0.004
Original vs thewayitis396q0.wav
8 out of 8, pval = 0.004
Original vs thewayitis_396b1.wav
8 out of 8, pval = 0.004
available here: http://www.ff123.net/samples.html (http://www.ff123.net/samples.html)
Original vs Tosca_396b1.wav
8 out of 8, pval = 0.004
Original vs Tosca_3902.wav
8 out of 8, pval = 0.004
Original vs Tosca396q0.wav
8 out of 8, pval = 0.004
sample uploaded
Original vs velvet_3902.wav
8 out of 8, pval = 0.004
Original vs velvet_396b1.wav
8 out of 8, pval = 0.004
Original vs velvet396q0.wav
8 out of 8, pval = 0.004
available here: http://www.mp3dev.org/mp3/gpsycho/quality.html (http://www.mp3dev.org/mp3/gpsycho/quality.html)
Ticket to Ride by The Beatles
I had to go down to 96kbps, because this one is easy to encode.
lame-3.90.3 --alt-preset 96
vs
lame-3.96b1 --preset 96
ABC/HR Version 0.9b, 30 August 2002
Testname: TICKET TO RIDE - ABR96 - 3.90.3 VS 3.96B1
1L = C:\My Music\lab\tkt\abr96-396b1.mp3.wav
2L = C:\My Music\lab\tkt\abr96-3903.mp3.wav
---------------------------------------
General Comments:
#2 was much harder to ABX than #1. For this sample, #2 is the clear winner.
---------------------------------------
1L File: C:\My Music\lab\tkt\abr96-396b1.mp3.wav
1L Rating: 2.5
1L Comment: The tambourine sounds distorted. The drum roll is also distroted.
---------------------------------------
2L File: C:\My Music\lab\tkt\abr96-3903.mp3.wav
2L Rating: 4.0
2L Comment: The tambourine sounds fine. The drum roll is distored.
---------------------------------------
ABX Results:
Original vs C:\My Music\lab\tkt\abr96-396b1.mp3.wav
10 out of 11, pval = 0.006
Original vs C:\My Music\lab\tkt\abr96-3903.mp3.wav
10 out of 13, pval = 0.046
C:\My Music\lab\tkt\abr96-396b1.mp3.wav vs C:\My Music\lab\tkt\abr96-3903.mp3.wav
5 out of 5, pval = 0.031
Sound Blaster Live! 5.1
foobar2000 ABX comparator
DSP: volume control, Resampler (SSRC) 48000Hz
all files replaygained on track basis
Harman/Kardon HK-395 speakers
(-) Ions (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191563) --alt-preset 128
wav vs 3.90.3 modified (494 KB)
8/8
0.4%
Comment: mp3 version sounds less sharp when volume of buzzing reaches peak level; mp3 sounds kind of smeared.
wav vs 3.96b1 (520 KB)
8/8
0.4%
Comment: distortions between volume peaks - this affects the lower-volume buzzing; (peak volume buzzing seems to sound like original wav).
3.90.3 modified (494 KB) vs 3.96b1 (520 KB)
8/8
0.4%
Comment: harder to ABX the two encoded versions. I found that 3.96's distortions were easier to differentiate the two than the peak buzzing volume. 3.90.3 is harder to pick out, at least with my ears.
Rock & Roll (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195724) --alt-preset 128
wav vs 3.90.3 modified (442 KB)
8/8
0.4%
Comment: cymbals and drum solo sound very "smeared." Also I noticed what seemed like an instant of stereo collapse.
wav vs 3.96b1 (449 KB)
8/8
0.4%
Comment: similar artifacts as 3.90.3, with the initial part of the drum solo seeming to suffer from a HF boost, but I did not notice the stereo problem.
3.90.3 modified (442 KB) vs 3.96b1 (449 KB)
17/21
(ABX comparator gives me N/A as a probability)
Comment: This was the toughest. The smearing of cymbals was of no use to differentiate the two, as it occurs in both, and I stopped getting positive results with the supposed HF boost in the drum solo (hence the 4 failed trials). Then I focused on the end of the last chorus and the final vocal, which sounded more distorted in 3.90.3 than in 3.96. That turned out to be the main difference.
I've done some tests to verify results of other members.
death2 (http://cyberquebec.ca/harashin/Death2_LAME_test.htm) myf_4sec (http://cyberquebec.ca/harashin/Myf_4sec_LAME_test.htm) doesnair (http://cyberquebec.ca/harashin/Doesnair_LAME_test.htm)
Sample used: Lazy_Jones.flac (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191243)
TEST: 3.96b1 (--preset extreme) <287.5 Kbps>
vs. 3.90.3 (--alt-preset extreme) <273.9 Kbps>
Average bitrates according to Nero6
Encoded with lame.exe/LAMEDrop1.3 (l3maniac) - Decoded with Nero6
-------------------------------------
WinABX v0.42 test report
03/22/2004 20:50:33
A file: C:\WINDOWS\Desktop\Lazy_Jones (wav).wav
B file: C:\WINDOWS\Desktop\Lazy_Jones 3.90.3 (--ap e).wav
20:50:55 1/1 p=50.0%
20:51:02 2/2 p=25.0%
20:51:11 3/3 p=12.5%
20:51:19 4/4 p=6.2%
20:51:34 5/5 p=3.1%
20:51:42 6/6 p=1.6%
20:51:50 7/7 p=0.8%
20:51:59 8/8 p=0.4%
20:52:09 9/9 p=0.2%
20:52:10 reset
20:52:19 test finished
-------------------------------------
WinABX v0.42 test report
03/22/2004 20:52:19
A file: C:\WINDOWS\Desktop\Lazy_Jones (wav).wav
B file: C:\WINDOWS\Desktop\Lazy_Jones 3.96b1 (--p e).wav
20:52:32 1/1 p=50.0%
20:52:45 2/2 p=25.0%
20:52:58 3/3 p=12.5%
20:53:09 4/4 p=6.2%
20:53:17 5/5 p=3.1%
20:53:26 6/6 p=1.6%
20:53:34 7/7 p=0.8%
20:53:44 8/8 p=0.4%
20:53:53 9/9 p=0.2%
20:53:54 reset
20:54:01 test finished
-------------------------------------
WinABX v0.42 test report
03/22/2004 20:54:01
A file: C:\WINDOWS\Desktop\Lazy_Jones 3.90.3 (--ap e).wav
B file: C:\WINDOWS\Desktop\Lazy_Jones 3.96b1 (--p e).wav
20:54:12 1/1 p=50.0%
20:54:20 2/2 p=25.0%
20:54:28 3/3 p=12.5%
20:54:36 4/4 p=6.2%
20:54:44 5/5 p=3.1%
20:54:52 6/6 p=1.6%
20:55:01 7/7 p=0.8%
20:55:09 8/8 p=0.4%
20:55:19 9/9 p=0.2%
20:55:21 reset
20:55:23 test finished
Comments:
3.90.3 (--alt-preset extreme); artifacts, sound like 'hi-hat close'
3.96b1 (--preset extreme); fatal flaw during initial attack during the first few milliseconds, sounds similar to a 'missing frame header' error, dead give-away not present in 3.90.3. Much clearer than 3.90.3, 'hi-hat close' artifacts not nearly as audible, but still present.
_____________________
Sample used: Lazy_Jones.flac (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191243)
TEST: 3.96b1 (--preset insane)
vs. 3.90.3 (--alt-preset insane)
Encoded with lame.exe/LAMEDrop1.3 (l3maniac) - Decoded with Nero6
-------------------------------------
WinABX v0.42 test report
03/22/2004 20:43:52
A file: C:\WINDOWS\Desktop\Lazy_Jones (wav).wav
B file: C:\WINDOWS\Desktop\Lazy_Jones 3.90.3 (--ap i).wav
20:44:34 1/1 p=50.0%
20:44:43 2/2 p=25.0%
20:44:54 3/3 p=12.5%
20:45:03 4/4 p=6.2%
20:45:12 5/5 p=3.1%
20:45:21 6/6 p=1.6%
20:45:29 7/7 p=0.8%
20:45:38 8/8 p=0.4%
20:45:46 9/9 p=0.2%
20:45:50 reset
20:45:59 test finished
-------------------------------------
WinABX v0.42 test report
03/22/2004 20:45:59
A file: C:\WINDOWS\Desktop\Lazy_Jones (wav).wav
B file: C:\WINDOWS\Desktop\Lazy_Jones 3.96b1 (--p i).wav
Start position 00:00.0, end position 00:06.3
20:46:30 1/1 p=50.0%
20:46:41 2/2 p=25.0%
20:46:51 3/3 p=12.5%
20:46:59 4/4 p=6.2%
20:47:08 5/5 p=3.1%
20:47:17 6/6 p=1.6%
20:47:26 7/7 p=0.8%
20:47:35 8/8 p=0.4%
20:47:45 9/9 p=0.2%
20:47:48 reset
20:48:00 test finished
-------------------------------------
WinABX v0.42 test report
03/22/2004 20:48:00
A file: C:\WINDOWS\Desktop\Lazy_Jones 3.90.3 (--ap i).wav
B file: C:\WINDOWS\Desktop\Lazy_Jones 3.96b1 (--p i).wav
20:48:17 1/1 p=50.0%
20:48:25 2/2 p=25.0%
20:48:34 3/3 p=12.5%
20:48:42 4/4 p=6.2%
20:48:51 5/5 p=3.1%
20:48:59 6/6 p=1.6%
20:49:09 7/7 p=0.8%
20:49:17 8/8 p=0.4%
20:49:27 9/9 p=0.2%
20:49:29 reset
20:49:31 test finished
Comments:
3.90.3 (--alt-preset insane); faint artifacts sound like "hi-hat close"
3.96b1 (--preset insane); fatal flaw during initial attack during the first few milliseconds, sounds similar to a 'missing frame header' error, dead give-away not present in 3.90.3. Much learer than 3.90.3
I am verifying Lazy Jones 3.90.3 aps vs. 3.96b1 ps. I focused on the last 1.5s of the sample:
3.90.3 aps vs. original
8/8 - popping artifacts in the last second of sample, very obvious
3.96b1 aps vs original
unable to ABX - sounds perfect
3.90.3 aps vs. 3.96b1 aps
8/8
edit: unable to ABX 3.96b1 or 3.90.3 at ape
I prefer 3.90.3 --alt-preset standard to 3.96b1 --preset standard on these samples.
Destitute (http://cyberquebec.ca/harashin/Destitute_LAME_test.htm) Rosemary (http://cyberquebec.ca/harashin/Rosemary_LAME_test.htm)
I tested drone_short (http://www.ff123.net/samples/drone_short.flac), 0-1 second:
3.90.3 aps vs. original 0-1s
8/8 - sounds like air being blown
3.96b1 aps vs. original 0-1s
8/8 - much easier to ABX, air sound louder and more noticable
3.96b1 aps vs. 3.90.3 aps
7/8 - missed last test, ears getting fatigued
3.90.3 sounds better
3.93.3 sounds better
I reckon you mean 3.90.3.
Oh thanks for catching that. I will go back and edit that...
It looks like no one is testing 3.96b1 vs 3.90.3 any more, but I did spend some time last night trying to ABX Rebel.wav (abr 128 & aps), but failed miserably...
I was able to distinguish between the original and the MP3s in ABC/HR, but couldn't prove it by ABXing.
@[proxima]: Was 3.90.3 the clear winner of the two on this sample, or was it only slightly better? I couldn't tell from your ABX results. If you think it was the clear winner, you must have pretty good ears . Or is it a matter of knowing which section of the sample to focus on?
@[proxima]: Was 3.90.3 the clear winner of the two on this sample, or was it only slightly better? I couldn't tell from your ABX results. If you think it was the clear winner, you must have pretty good ears . Or is it a matter of knowing which section of the sample to focus on?
@LoFiYo
Welcome to the club! I have always problems to ABX samples others simply describe as preecho problem samples. Must be an auditory canal defect of a kind
Wombat
Must be an auditory canal defect of a kind
By looking at your avatar, probably.
@[proxima]: Was 3.90.3 the clear winner of the two on this sample, or was it only slightly better? I couldn't tell from your ABX results. If you think it was the clear winner, you must have pretty good ears . Or is it a matter of knowing which section of the sample to focus on?
The problem is not only preecho, with --ap 128 the
very annoying artifact is chirping/ringing problem. I've redone the test, this time including 3.96b1 vs. 3.90.2 abx results
ABC/HR Version 0.9b, 30 August 2002
Testname:
1L = C:\Documents and Settings\proxima\Desktop\Batch\Nuova cartella\LAME_GUI\rebel_396b1_128.wav
2R = C:\Documents and Settings\proxima\Desktop\Batch\Nuova cartella\LAME_GUI\rebel_3902_128.wav
---------------------------------------
General Comments:
---------------------------------------
1L File: C:\Documents and Settings\proxima\Desktop\Batch\Nuova cartella\LAME_GUI\rebel_396b1_128.wav
1L Rating: 1.0
1L Comment: Chirping/ringing artifacts with high frequencies are very annoying !! The artifacted sound compromise the whole listening pleasure of the sample so that paying attention on other artifacts is very difficult. Preecho is present for sure too.
---------------------------------------
2R File: C:\Documents and Settings\proxima\Desktop\Batch\Nuova cartella\LAME_GUI\rebel_3902_128.wav
2R Rating: 2.7
2R Comment: here i can hear the classic preecho artifacts with noise spread before attacks. Chirping/ringing problems absent. This is far better than 1L.
---------------------------------------
ABX Results:
Original vs C:\Documents and Settings\proxima\Desktop\Batch\Nuova cartella\LAME_GUI\rebel_396b1_128.wav
12 out of 12, pval < 0.001
Original vs C:\Documents and Settings\proxima\Desktop\Batch\Nuova cartella\LAME_GUI\rebel_3902_128.wav
12 out of 12, pval < 0.001
C:\Documents and Settings\proxima\Desktop\Batch\Nuova cartella\LAME_GUI\rebel_396b1_128.wav vs C:\Documents and Settings\proxima\Desktop\Batch\Nuova cartella\LAME_GUI\rebel_3902_128.wav
12 out of 12, pval < 0.001
Similar problems are not isolated and can be heard in applaud and campestre sample i already uploaded. 3.90.2 does not suffers of this chirping/ringing issue. I encourage people to confirm this, i find ABXing even a waste of time for such problems. See spectral analisys for rebel.wav, there are droputs with high frequencies above 10 kHz.
Thanks for the hint Gabriel!
,Mar 25 2004, 09:19 AM]I've redone the test, this time including 3.96b1 vs. 3.90.2 abx results
Thanks for taking the time to redo the test
3.90.3 --aps wins over 3.96b1 --ps for this one: Chanchan1 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=197226):
The trumpets arround ~ 5.0 sound less clean, like surround/chorus effect added with both, 3.90.3 and 3.96, but 3.96 adds ringing additionally. ABX results:
[span style='font-size:7pt;line-height:100%']
foo_abx v1.2 report
foobar2000 v0.8
2004/03/25 21:38:17
File A: file://N:\lametest\Chanchan\ChanChan1.wav
File B: file://N:\lametest\Chanchan\chanchan1.wav_3903_aps.wav
21:40:30 : Trial reset.
21:41:17 : 01/01 50.0%
21:42:20 : 02/02 25.0%
21:42:27 : 03/03 12.5%
21:42:33 : 04/04 6.3%
21:42:41 : 05/05 3.1%
21:43:13 : 06/06 1.6%
21:43:24 : 07/07 0.8%
21:43:44 : 08/08 0.4%
----------
Total: 8/8 (0.4%)
--------------------------------------------------------
foo_abx v1.2 report
foobar2000 v0.8
2004/03/25 21:44:17
File A: file://N:\lametest\Chanchan\ChanChan1.wav
File B: file://N:\lametest\Chanchan\chanchan1.wav_396b1_ps.wav
21:45:06 : 01/01 50.0%
21:45:10 : 02/02 25.0%
21:45:14 : 03/03 12.5%
21:45:18 : 04/04 6.3%
21:45:25 : 05/05 3.1%
21:45:29 : 06/06 1.6%
21:45:32 : 07/07 0.8%
21:45:35 : 08/08 0.4%
----------
Total: 8/8 (0.4%)
---------------------------------------------------------
foo_abx v1.2 report
foobar2000 v0.8
2004/03/25 21:45:47
File A: file://N:\lametest\Chanchan\chanchan1.wav_3903_aps.wav
File B: file://N:\lametest\Chanchan\chanchan1.wav_396b1_ps.wav
21:47:09 : 01/01 50.0%
21:47:14 : 02/02 25.0%
21:47:18 : 03/03 12.5%
21:47:25 : 04/04 6.3%
21:47:42 : 04/05 18.8%
21:48:56 : 05/06 10.9%
21:49:12 : 06/07 6.3%
21:49:40 : 07/08 3.5%
21:49:57 : 08/09 2.0%
21:50:56 : 09/10 1.1%
21:51:02 : 10/11 0.6%
21:51:08 : 11/12 0.3%
----------
Total: 11/12 (0.3%)
[/span]
Rebel sample 3.90.3 --alt-preset standard vs. 3.96b1 --preset standard:
3.90.3 won
ABXed 3.0 - 6.0. Noticable pre-echo at drum hits, 3.96 is worse, slightly ringing/metallic sounding
ABX results: [span style='font-size:7pt;line-height:100%']
foo_abx v1.2 report
foobar2000 v0.8
2004/03/25 22:19:33
File A: file://N:\lametest\rebel.wav
File B: file://N:\lametest\rebel.wav_396b1_ps.mp3
22:20:30 : 01/01 50.0%
22:20:34 : 02/02 25.0%
22:20:38 : 03/03 12.5%
22:20:42 : 04/04 6.3%
22:20:45 : 05/05 3.1%
22:20:52 : 06/06 1.6%
22:20:55 : 07/07 0.8%
22:21:01 : 08/08 0.4%
----------
Total: 8/8 (0.4%)
------------------------------------------------------------
foo_abx v1.2 report
foobar2000 v0.8
2004/03/25 22:21:28
File A: file://N:\lametest\rebel.wav
File B: file://N:\lametest\rebel.wav_3903_aps.mp3
22:22:20 : 01/01 50.0%
22:22:24 : 02/02 25.0%
<snip>
22:25:39 : 19/25 0.7%
22:25:47 : 20/26 0.5%
22:25:55 : 21/27 0.3%
22:26:06 : 22/28 0.2%
----------
Total: 22/28 (0.2%)
------------------------------------------------------------
foo_abx v1.2 report
foobar2000 v0.8
2004/03/25 22:26:22
File A: file://N:\lametest\rebel.wav_3903_aps.mp3
File B: file://N:\lametest\rebel.wav_396b1_ps.mp3
22:26:50 : 01/01 50.0%
22:26:54 : 02/02 25.0%
22:27:00 : 03/03 12.5%
22:27:03 : 04/04 6.3%
22:27:06 : 05/05 3.1%
22:27:09 : 06/06 1.6%
22:27:15 : 07/07 0.8%
22:27:18 : 08/08 0.4%
----------
Total: 8/8 (0.4%)
[/span]
I prefer 3.90.3 --alt-preset standard to 3.96b1 --preset standard on these samples.
Destitute (http://cyberquebec.ca/harashin/Destitute_LAME_test.htm) Rosemary (http://cyberquebec.ca/harashin/Rosemary_LAME_test.htm)
no ABX results makes your claim entirely useless to us
I prefer 3.90.3 --alt-preset standard to 3.96b1 --preset standard on these samples.
Destitute Rosemary
no ABX results makes your claim entirely useless to us
Check his links, man.
I've had some more tests.
Hosokawa___Atem_lied (http://cyberquebec.ca/harashin/Hosokawa___Atem_lied_LAME_test.htm)
Original vs 3.96b1aps (10/10)
Original vs 3.90.3aps (13/14)
3.90.3aps vs 3.96b1aps (10/10)
Chanchan1 (http://cyberquebec.ca/harashin/Chanchan1_LAME_test.htm)
Original vs 3.96b1aps (10/10)
Original vs 3.90.3aps (15/18)
3.90.3aps vs 3.96b1aps (10/11)
Gabriel released 3.96b2 today. Download it here (http://www.rarewares.org/files/mp3/lame3.96b2.zip).
The Minimal bitrate for --preset standard has been 'adjusted' to 128kbps, so all samples where 3.96b1 did worse than 3.90.3 need retesting/validation.
I tested 3.96b2 vs 3.90.3 with MYF_4SEC.WAV. On this sample, 3.90.3 still sounds much better (closer to the original) to me. I did notice some improvement with 3.96b2 over 3.96b1, I think because it no longer drops down to 96kbps, but the distortion at the 3rd blow is still noticeable. 3.90.3 doesn't have that distortion and sounds much cleaner.
WinABX v0.22 (ABA) test report
03/28/2004 14:49:52
A file: C:\My Music\lab\MYF-4SEC\TEST.WAV
B file: C:\My Music\lab\MYF-4SEC\aps396b2.mp3.wav
14:50:37 1/1 p=33.3%
14:50:58 2/2 p=11.1%
14:51:23 3/3 p= 3.7%
14:51:54 4/4 p= 1.2%
14:52:13 5/5 p= 0.4%
14:53:33 6/6 p= 0.1%
14:53:41 7/7 p< 0.1%
14:57:03 test finished
-------------------------------------
WinABX v0.22 (ABA) test report
03/28/2004 14:57:49
A file: C:\My Music\lab\MYF-4SEC\TEST.WAV
B file: C:\My Music\lab\MYF-4SEC\aps3903.mp3.wav
14:59:12 0/1 p=100.0%
15:00:59 0/2 p=100.0%
15:01:10 1/3 p=70.4%
15:01:19 2/4 p=40.7%
15:01:54 2/5 p=53.9%
15:02:30 2/6 p=64.9%
15:02:36 2/7 p=73.7%
15:02:44 3/8 p=53.2%
15:02:55 4/9 p=35.0%
15:03:18 5/10 p=21.3%
15:03:30 6/11 p=12.2%
15:03:49 7/12 p= 6.7%
15:04:01 8/13 p= 3.5%
15:04:15 9/14 p= 1.8%
15:04:33 10/15 p= 0.9%
15:04:43 11/16 p= 0.4%
15:04:53 12/17 p= 0.2%
15:04:58 13/18 p= 0.1%
15:05:04 14/19 p< 0.1%
15:05:10 test finished
-------------------------------------
WinABX v0.22 (ABA) test report
03/28/2004 15:05:12
A file: C:\My Music\lab\MYF-4SEC\TEST.WAV
15:05:18 test finished
-------------------------------------
WinABX v0.22 (ABA) test report
03/28/2004 15:05:19
A file: C:\My Music\lab\MYF-4SEC\aps3903.mp3.wav
B file: C:\My Music\lab\MYF-4SEC\aps396b2.mp3.wav
15:05:49 1/1 p=33.3%
15:05:57 2/2 p=11.1%
15:06:03 3/3 p= 3.7%
15:06:08 4/4 p= 1.2%
15:06:14 5/5 p= 0.4%
15:06:22 6/6 p= 0.1%
15:06:32 7/7 p< 0.1%
15:06:35 test finished
Edit: for clarity & more detail
I still prefer 3.90.3 on these samples.
Hustlejet (http://cyberquebec.ca/harashin/Hustle_Jet_LAME_test2.htm)
Original vs 3.96b2aps (13/14)
Original vs 3.90.3aps (15/20) (http://cyberquebec.ca/harashin/Hustle_Jet_LAME_test.htm)
3.90.3aps vs 3.96b2aps (12/14)
Hosokawa___Atem_lied (http://cyberquebec.ca/harashin/Hosokawa___Atem_lied_LAME_test2.htm)
Original vs 3.96b2aps (10/10)
Original vs 3.90.3aps (13/14) (http://cyberquebec.ca/harashin/Hosokawa___Atem_lied_LAME_test.htm)
3.90.3aps vs 3.96b2aps (10/10)
Only for 3.96b1
Confirming Fatboy sample
3.90.3 --alt-preset 128 Vs WAV
8/8
Heavy Distortion
3.96b1 -preset 128 vs WAV
7/7
Lighter Distortion
3.90.3 --alt preset 128 vs 3.96b --preset 128
6/6
Conclusion: 3.96b1 has far less distortion
3.96b1 wins
v3.90.3 vs 3.96b2 using --preset standard on sample 41_30sec.flac (http://www.ff123.net/samples/41_30sec.flac)
I would call this test a tie between the two encoders.
ABC/HR Version 0.9b, 30 August 2002
Testname: Lame v3.90.3 vs v3.96b2 --preset standard
1R = Z:\Music\Test Samples\testTemp\Lame3.90.3\41_30sec.wav
2L = Z:\Music\Test Samples\testTemp\Lame3.96b2\41_30sec.wav
---------------------------------------
General Comments:
Both files suffer from pre-echo in the symbols from 0-2.3 seconds. I cant ABX the 2 files with any confidence.
---------------------------------------
1R File: Z:\Music\Test Samples\testTemp\Lame3.90.3\41_30sec.wav
1R Rating: 4.5
1R Comment:
---------------------------------------
2L File: Z:\Music\Test Samples\testTemp\Lame3.96b2\41_30sec.wav
2L Rating: 4.5
2L Comment:
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\testTemp\Lame3.90.3\41_30sec.wav
10 out of 10, pval < 0.001
Original vs Z:\Music\Test Samples\testTemp\Lame3.96b2\41_30sec.wav
10 out of 10, pval < 0.001
Z:\Music\Test Samples\testTemp\Lame3.90.3\41_30sec.wav vs Z:\Music\Test Samples\testTemp\Lame3.96b2\41_30sec.wav
25 out of 44, pval = 0.226
i tried my own (non-specifically chosen) samples but i couldn't do jack with them. i think i'll try some of the samples that came in the AAC-HE+PS test and maybe the one before that test, too to save time... (gotta hunt down the codec killer samples i downloaded somewhere on this computer, too)
I repeated (alt)preset standard test with cantwait sample and 3.90.3, 3.96b1 and b2. Result: If there's any change between 3.96b1 and b2, I didn't have enough patience to find and ABX it. ABXing original vs. 3.96b2 and 3.90.3 vs. 3.96b2 was as easy as with b1 before. ABC/HR log:
ABC/HR Version 0.9b, 30 August 2002
Testname: cantwait 3.96b2 / b1 / 3.90.3 standard
1L = N:\lametest\Cantwait\cantwait.wav_396b1_ps.wav
2L = N:\lametest\Cantwait\cantwait.wav_3903_aps.wav
3L = N:\lametest\Cantwait\cantwait_396b2_aps.wav
---------------------------------------
General Comments:
---------------------------------------
1L File: N:\lametest\Cantwait\cantwait.wav_396b1_ps.wav
1L Rating: 2.5
1L Comment:
---------------------------------------
2L File: N:\lametest\Cantwait\cantwait.wav_3903_aps.wav
2L Rating: 3.5
2L Comment:
---------------------------------------
3L File: N:\lametest\Cantwait\cantwait_396b2_aps.wav
3L Rating: 2.5
3L Comment:
---------------------------------------
ABX Results:
Original vs N:\lametest\Cantwait\cantwait.wav_396b1_ps.wav
7 out of 7, pval = 0.008
Original vs N:\lametest\Cantwait\cantwait.wav_3903_aps.wav
7 out of 7, pval = 0.008
Original vs N:\lametest\Cantwait\cantwait_396b2_aps.wav
7 out of 7, pval = 0.008
N:\lametest\Cantwait\cantwait.wav_396b1_ps.wav vs N:\lametest\Cantwait\cantwait.wav_3903_aps.wav
7 out of 7, pval = 0.008
N:\lametest\Cantwait\cantwait.wav_396b1_ps.wav vs N:\lametest\Cantwait\cantwait_396b2_aps.wav
6 out of 12, pval = 0.613
N:\lametest\Cantwait\cantwait.wav_3903_aps.wav vs N:\lametest\Cantwait\cantwait_396b2_aps.wav
7 out of 7, pval = 0.008
Result in short:
No improvement with 3.96b2
I dont know whether this is needed as it isnt an -aps test but ill post it anyway.
Testing Fatboy with 3.96b2 at 128 ABR.
Fatboy.wav vs 3.96b2 --preset 128
9/9
Some distortion
fatboy.wav vs 3.90.3 --alt-preset 128
9/9
Heavy distorion
3.96b2 vs 3.90.3
9/9
3.90.3 is far worse than 3.96b2
3.96b2 is still better than 3.90.3 for fatboy.wav
FatBoyFin, thanks for your effort, but ...
If I understand Gabriel here (http://www.hydrogenaudio.org/forums/index.php?showtopic=19876&view=findpost&p=197997) correctly, the quality related change between 3.96b1 and b2 is the minimum bitrate for -V2 (= preset standard) and -V1 (extreme). So the results for lower -V settings and abr/cbr presets should be identical. Because of this, 3.96b1 and b2 results for identical samples won't be added separately to the results post for -V 3 and lower and ABR/CBR presets.
Of course testing both can never do a harm, but since there aren't many results for < standard settings that match the test requirements (ABX results etc.) so far anyway, just testing 3.96b2 vs. 3.90.3 in ~ 128-160kbps range without 'looking back' should be enough. Testing -V 5 (and comparing with (alt)preset 128) additionally would be great too.
BTW: FatBoyFin, have you tried to ABX 3.96b1 vs. 3.96b2?
I tested 3.96b2 --preset 160 vs 3.90.3 --alt-preset 160 vs 3.96b2 -V4 using MYF_4SEC.wav. V4 was really bad on this sample, and
I think the 2 abr encodes were pretty much a tie, but I felt 3.96b2 was slightly better. I could not ABX 3.96b2 abr160 vs the original, mainly because I think my ears were really tired, but also because it was well encoded. Of course, I couldn't ABX the 2 abr encodes at all.
ABC/HR Version 0.9b, 30 August 2002
Testname: MyF_4sec - V4 vs abr160
1R = C:\My Music\lab\MYF-4SEC\396b2-abr160.mp3.wav
2R = C:\My Music\lab\MYF-4SEC\3903-abr160.mp3.wav
3L = C:\My Music\lab\MYF-4SEC\396b2-V4.mp3.wav
---------------------------------------
General Comments:
#1 and #2 are similar in quality. ABX failed between #1 & #2. #3 is really bad.
---------------------------------------
1R File: C:\My Music\lab\MYF-4SEC\396b2-abr160.mp3.wav
1R Rating: 4.0
1R Comment: ABX failed because of fatigue and also because encode is good.
---------------------------------------
2R File: C:\My Music\lab\MYF-4SEC\3903-abr160.mp3.wav
2R Rating: 3.5
2R Comment: Slight distortion on the first blow but not really bad.
---------------------------------------
3L File: C:\My Music\lab\MYF-4SEC\396b2-V4.mp3.wav
3L Rating: 1.5
3L Comment: Distortion is very noticeable throughout the file.
---------------------------------------
ABX Results:
Original vs C:\My Music\lab\MYF-4SEC\396b2-abr160.mp3.wav
18 out of 32, pval = 0.298
Original vs C:\My Music\lab\MYF-4SEC\3903-abr160.mp3.wav
7 out of 7, pval = 0.008
Original vs C:\My Music\lab\MYF-4SEC\396b2-V4.mp3.wav
13 out of 14, pval < 0.001
C:\My Music\lab\MYF-4SEC\396b2-abr160.mp3.wav vs C:\My Music\lab\MYF-4SEC\3903-abr160.mp3.wav
1 out of 2, pval = 0.750
C:\My Music\lab\MYF-4SEC\396b2-abr160.mp3.wav vs C:\My Music\lab\MYF-4SEC\396b2-V4.mp3.wav
13 out of 14, pval < 0.001
C:\My Music\lab\MYF-4SEC\3903-abr160.mp3.wav vs C:\My Music\lab\MYF-4SEC\396b2-V4.mp3.wav
10 out of 10, pval < 0.001
BTW: FatBoyFin, have you tried to ABX 3.96b1 vs. 3.96b2?
I havent ABX between these but the file sizes are identical so I presume they are exactly the same.
As there is a new 128Kbps test coming up I redid my Fatboy test to include the -V5 setting:
-V5 ,3.96b2 vs WAV
6/6
Hard to abx
ap128 ,3.90.3 vs WAV
8/8
Heavy Distortion
-V5 ,3.96b2 vs ap128 ,3.90.3
5/5
-preset128 ,3.96b2 vs WAV
7/7
Lighter Distortion
preset 128 vs -v 5
6/6
-V5 is alot better
Conclusion:
-V5 > --preset 128 > --alt-preset128 for FatBoy.wav
PS the bitrate for -V 5 was a huge 220 KBps could someone please check that I have encoded this correctly or should I not be using -V5?
Sample : Its me (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=201613) ~ 128kbps tested
Live recording, focussed on problems with applause, details see ABC/HR log
3.96b2 -V 5 won (3.0)
3.90.3 ap 128 was second (2.0)
3.96b2 p 128 was worst (1.5)
ABC/HR log (I tested vorbis codecs with HF reduction as well, to save space here I cut out the related parts in ABC/HR log.)
ABC/HR Version 0.9b, 30 August 2002
Testname:
1L = N:\lametest\Soul_Live\decode\017 Its me.wav_mtb3_4.wav
2L = N:\lametest\Soul_Live\decode\011 Its me.wav_23_4.wav
3L = N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav
4R = N:\lametest\Soul_Live\decode\015 Its me.wav_cqk32_4.25.wav
5R = N:\lametest\Soul_Live\decode\010 Its me.wav.wav
6R = N:\lametest\Soul_Live\decode\016 Its me.wav_hfr_4.25.wav
7R = N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav
8R = N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav
---------------------------------------
General Comments:
---------------------------------------
<snip>
---------------------------------------
3L File: N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav
3L Rating: 2.0
3L Comment: 0.0-3.4: surround-like, warbeling/metallic sound in applause, no hf noise boost -> should be lame.
---------------------------------------
<snip>
---------------------------------------
7R File: N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav
7R Rating: 3.0
7R Comment: should be lame; small ringing on right channel, otherwise quite good
---------------------------------------
8R File: N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav
8R Rating: 1.5
8R Comment: sounds like 3 with hf ringing problems of 7 added
---------------------------------------
ABX Results:
<snip>
Original vs N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav
7 out of 7, pval = 0.008
<snip>
Original vs N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav
7 out of 7, pval = 0.008
Original vs N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav
7 out of 7, pval = 0.008
<snip>
N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav vs N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav
11 out of 12, pval = 0.003
N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav vs N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav
11 out of 12, pval = 0.003
<snip>
N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav vs N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav
7 out of 7, pval = 0.008
Sample : entierren con rumba (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=201619) ~ 128kbps tested
Salsa with lots of percussion, details see ABC/HR log, results similar to the previous sample I tested
3.96b2 -V 5 won
3.90.3 ap 128 was second
3.96b2 p 128 was worst
ABC/HR log:
ABC/HR Version 0.9b, 30 August 2002
Testname: entierren con rumba lame 128
1L = N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav
2L = N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav
3L = N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav
---------------------------------------
General Comments:
Guiros (percussion) smeared, clearly noticable (and ABXed) at 2.8-6.0
1 is best, 2 and 3 are similar but 2 has ringing/metallic sound additionally.
After 6.0 it's similar with the additonal precussion instrument (shaker?).
---------------------------------------
1L File: N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav
1L Rating: 3.5
1L Comment:
---------------------------------------
2L File: N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav
2L Rating: 2.0
2L Comment:
---------------------------------------
3L File: N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav
3L Rating: 2.5
3L Comment:
---------------------------------------
ABX Results:
Original vs N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav
7 out of 7, pval = 0.008
Original vs N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav
7 out of 7, pval = 0.008
Original vs N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav
7 out of 7, pval = 0.008
N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav vs N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav
7 out of 7, pval = 0.008
N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav vs N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav
7 out of 7, pval = 0.008
N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav vs N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav
7 out of 7, pval = 0.008
____________________________________________________________________
[span style='font-size:12pt;line-height:100%']
BTW: IT WOULD BE GREAT IF SOME MORE PEOPLE COULD PROVIDE ~128KBPS RESULTS BEFORE RJAMORIM'S 128KBPS TEST STARTS.[/span]
Lame v3.96 final seems to have a serious problem w/ 41_30sec (http://www.ff123.net/samples/41_30sec.flac) in the 6-8 second range using --preset cbr 128. It sounds like a dropout / skip in the sound. This problem has been seen in Mitok's compile as well as ICL v8.0 and MSVC compiles made by myself. There is no skip using v3.90.3 . Could someone confirm this??
EDIT - Look at the waveforms - the volumes between the wav/3.90.3 and 3.96 are drastically different.
EDIT2 - Lame v3.96b2 does not suffer from this problem either.
Origional WAV:
(http://img82.imageshack.us/img82/6483/wavgy7.gif)
Lame v3.90.3 --preset cbr 128:
(http://img82.imageshack.us/img82/4973/3903xe5.gif)
Lame v3.96 --preset cbr 128:
(http://img147.imageshack.us/img147/1885/396vg2.gif)
Lame v3.96 final seems to have a serious problem w/ 41_30sec (http://www.ff123.net/samples/41_30sec.flac) in the 6-8 second range using --preset cbr 128. It sounds like a dropout / skip in the sound. This problem has been seen in Mitok's compile as well as ICL v8.0 and MSVC compiles made by myself. There is no skip using v3.90.3 . Could someone confirm this??
I can confirm it. I tested the 4.5-8.5 second range of 41_30sec, using Mitiok's 3.96 and john33's 3.90.3 and 3.96b2. The 3.96 final version definitely has some nasty dropouts not present the other encodes.
ABC/HR Version 0.9b, 30 August 2002
Testname:
1L = G:\Filetemp\diskwriter\41_30\41-396.wav
2L = G:\Filetemp\diskwriter\41_30\41-3903.wav
3R = G:\Filetemp\diskwriter\41_30\41-396b2.wav
---------------------------------------
General Comments:
---------------------------------------
1L File: G:\Filetemp\diskwriter\41_30\41-396.wav
1L Rating: 1.0
1L Comment: Pops or dropouts
---------------------------------------
2L File: G:\Filetemp\diskwriter\41_30\41-3903.wav
2L Rating: 2.5
2L Comment:
---------------------------------------
3R File: G:\Filetemp\diskwriter\41_30\41-396b2.wav
3R Rating: 2.2
3R Comment: A bit more swirling than sample 2
---------------------------------------
ABX Results:
G:\Filetemp\diskwriter\41_30\41-396.wav vs G:\Filetemp\diskwriter\41_30\41-3903.wav
8 out of 8, pval = 0.004
G:\Filetemp\diskwriter\41_30\41-396.wav vs G:\Filetemp\diskwriter\41_30\41-396b2.wav
8 out of 8, pval = 0.004
G:\Filetemp\diskwriter\41_30\41-3903.wav vs G:\Filetemp\diskwriter\41_30\41-396b2.wav
11 out of 12, pval = 0.003
Yes , i also can confirm it.
I test it too with 41_30sec file and at the 4.5 - 6 is definitely a Pop with Lame 3.96 Final.
No Problems with Lame 3.90.3 and 3.96b2.
I have created a gif animation that shows how the volume levels seem to change / drop out in the v3.96 final encoder when using 128kbps and to a lesser extent 160kbps - it can be found here. (http://www.angelfire.com/mt/viper1313/lame/compare.gif) Take a careful look at v3.90.3 vs. v3.96. More testing on this issue and bitrates compromised by it need to be performed.
This is just a random section (1.6-1.8sec) of the 41_30 clip, and this section is ABX-able (sounds like a click / popping).
I have created a gif animation that shows how the volume levels seem to change / drop out in the v3.96 final encoder when using 128kbps and to a lesser extent 160kbps - it can be found here. (http://www.angelfire.com/mt/viper1313/lame/compare.gif) Take a careful look at v3.90.3 vs. v3.96. More testing on this issue and bitrates compromised by it need to be performed.
This is just a random section (1.6-1.8sec) of the 41_30 clip, and this section is ABX-able (sounds like a click / popping).
Looks like a bug. Does this occur with abr as well (p 128)?
That could be a reason to eliminate 3.96final from consideration in Roberto's upcoming multiformat test.
ff123
From what I have seen and heard it does not seem to occur in ABR or VBR modes, although the encoder could just be throwing bits to where they are needed. The problems shown in 41_30sec seem to disappear using --preset 128, so IMO ABR seems fine.
OMG! At www.cdfreaks.com have published this new Lame version! Someone should told them to remove the news because of this buggy version.
thank you for the warning.....
i will try to contact an operator at cdfreaks
Cdfreaks has a disclaimer at the bottom of the article warning users of the bug
I would like people testing a little more than 1 sample before calling a version "buggy"
edit: additionnaly it could help if you could find other clearly problematic samples where you hear a similar behaviour (problem in cbr but not in abr)
I agree with Gabriel, who is to say that Lame 3.96 may break one sample, but fix 25 other samples? Unless it is tested more, you can't judge
I would like people testing a little more than 1 sample before calling a version "buggy"
edit: additionnaly it could help if you could find other clearly problematic samples where you hear a similar behaviour (problem in cbr but not in abr)
But, on the other hand, it would be good to know, why this sample is problematic with 3.96 and not with 3.96b2. Do you have any guess?
Tested "Waiting" (http://www.ff123.net/samples/Waiting.flac) sample at --preset cbr 128 using Lame v3.90.3, 3.96b2 and v3.96 final. Tested in the 0-2.2 second region of the song. While there are no clicks, there seems to be a pretty major quality regression between v3.96b2 and v3.96 final, while 3.90.3 still comes out as the winner.
ABC/HR Version 0.9b, 30 August 2002
Testname: Waiting --preset cbr 128
1L = Z:\Music\Temp\3.96 Final.wav
2R = Z:\Music\Temp\3.90.3.wav
3L = Z:\Music\Temp\v3.96b2.wav
---------------------------------------
General Comments:
---------------------------------------
1L File: Z:\Music\Temp\3.96 Final.wav
1L Rating: 3.7
1L Comment: Horrible warble / distortion in the 0-2 second region
---------------------------------------
2R File: Z:\Music\Temp\3.90.3.wav
2R Rating: 4.5
2R Comment: Best of the 3, very clean, no major distortions.
---------------------------------------
3L File: Z:\Music\Temp\v3.96b2.wav
3L Rating: 4.3
3L Comment: Slightly worse than 2, much better than 1, slight warble on the beginning pronunciation of "been."
---------------------------------------
ABX Results:
Original vs Z:\Music\Temp\3.96 Final.wav
8 out of 8, pval = 0.004
Original vs Z:\Music\Temp\3.90.3.wav
8 out of 8, pval = 0.004
Original vs Z:\Music\Temp\v3.96b2.wav
8 out of 8, pval = 0.004
Z:\Music\Temp\3.96 Final.wav vs Z:\Music\Temp\3.90.3.wav
13 out of 14, pval < 0.001
Z:\Music\Temp\3.96 Final.wav vs Z:\Music\Temp\v3.96b2.wav
16 out of 17, pval < 0.001
Z:\Music\Temp\3.90.3.wav vs Z:\Music\Temp\v3.96b2.wav
18 out of 20, pval < 0.001
Tested "Waiting" sample at --preset cbr 128 using Lame v3.90.3, 3.96b2 and v3.96 final.
What about abr?
Tested "Waiting" sample at --preset cbr 128 using Lame v3.90.3, 3.96b2 and v3.96 final.
What about abr?
Your wish is my command - "--preset 128" using v3.90.3, 3.96b2 and 3.96 final.
ABC/HR Version 0.9b, 30 August 2002
Testname: Waiting --preset 128
1R = Z:\Music\Temp\3.96 ABR.wav
2L = Z:\Music\Temp\3.90.3 ABR.wav
3L = Z:\Music\Temp\3.96b2 ABR.wav
---------------------------------------
General Comments:
All samples did better than in CBR, and no codec sounded horrible.
---------------------------------------
1R File: Z:\Music\Temp\3.96 ABR.wav
1R Rating: 4.1
1R Comment: Felt this had slightly more warble than 3 and is clearly worse than 2, although I am not 100% sure that I can ABX it against 3.
---------------------------------------
2L File: Z:\Music\Temp\3.90.3 ABR.wav
2L Rating: 4.6
2L Comment: Better than 1 or 3, sounded cleaner
---------------------------------------
3L File: Z:\Music\Temp\3.96b2 ABR.wav
3L Rating: 4.2
3L Comment: Slight warble, thought it was slightly better than 1 but could not ABX it with 100% confidence - got a 15 of 17, p = 0.001 but with later trials could not ABX - might just be listening fatigue.
---------------------------------------
ABX Results:
Original vs Z:\Music\Temp\3.96 ABR.wav
8 out of 8, pval = 0.004
Original vs Z:\Music\Temp\3.90.3 ABR.wav
8 out of 8, pval = 0.004
Original vs Z:\Music\Temp\3.96b2 ABR.wav
13 out of 14, pval < 0.001
Z:\Music\Temp\3.96 ABR.wav vs Z:\Music\Temp\3.90.3 ABR.wav
13 out of 14, pval < 0.001
Z:\Music\Temp\3.96 ABR.wav vs Z:\Music\Temp\3.96b2 ABR.wav
28 out of 42, pval = 0.022
Z:\Music\Temp\3.90.3 ABR.wav vs Z:\Music\Temp\3.96b2 ABR.wav
10 out of 10, pval < 0.001
At this point in time, I would sum up the whole situation by saying:
- I don't know if 41_30sec is an isolated case or not - I agree that more samples are needed before any judgment can be made.
- I have only seen the problem in --preset cbr 128 - other bitrates/modes don’t exhibit the problem in the same way or only to a degree that cannot be easily heard.
- I would not go so far as to call v3.96 buggy, but more testing needs to be done on the final version.
- v3.90.3 still seems to have higher quality in most samples at most bitrates (with the exception of fatboy and a few others)
- MORE TESTING OF v3.96 IS NEEDED
If other people could do some tests to prove or disprove what I have found, it would be a large help.
I don't know if 41_30sec is an isolated case or not - I agree that more samples are needed before any judgment can be made.
I can confirm that there is a failure on this sample on frame 272, 2nd granule (7.11s) when encoding in cbr 128. The output on mp3X is very strange.
OK. I've started a LAME 3.96 FINAL vs. 3.90.3 Test, Test/Result Thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&).
I'm a bit short of time right now - I'll add missing results to the related posts ASAP if noone's faster.
Gabriel:
I can confirm that there is a failure on this sample on frame 272, 2nd granule (7.11s) when encoding in cbr 128. The output on mp3X is very strange.
Is this something that is going to be fixed soon (so we can expect 3.96.1 or similar in a couple of days and should better wait with testing 3.96 final) or is it just the price that has to be paid for better performance elsewhere?
-----------------------------------------------------------------
DJED's post split to a separate thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=20724&).
Is this something that is going to be fixed soon (so we can expect 3.96.1 or similar in a couple of days and should better wait with testing 3.96 final) or is it just the price that has to be paid for better performance elsewhere?
Well, I think that this specific one should be fixed, but I do not know how soon this will be done.
My opinion is that you can test 3.96.
I've just edited all relevant 3.96beta results that were still missing in the results post (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=194779).
For further testing of 3.96 final as well as for continuing related discussion go here (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&) please.
If necessary I can start a separate thread for 3.96beta -> final regression cases and move related posts there. Opinions please?