HydrogenAudio

Hydrogenaudio Forum => Listening Tests => Topic started by: tigre on 2004-04-13 16:17:08

Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-13 16:17:08
[span style='font-size:7pt;line-height:100%']This is stolen from here (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=194481) mostly, since only few things have changed...
----------------------------------------------------------------------[/span]

The purpose of this thread is to (finally) test LAME 3.96 throughly enough to make it the new recommended LAME version and enhance communication between LAME developers and the HA.org community. The recent confusion about the different LAME versions and compiles (-Qrcd trouble etc.) have led to the decision that moving along with LAME development has been postponed for long enough and LAME 3.90.X can hopefully be declared dead very soon.
Please test as many samples as possible according to the following guidelines and post any results here.


1. Use the following LAME compiles (last updated on 2004/04/13):
lame3.90.3 (http://www.rarewares.org/files/mp3/lame-3.90.3.zip)
lame3.96 final (http://mitiok.free.fr/lame-3.96.zip)

2. The focus of the test should be --alt-preset/--preset standard, since it will allow us to make conclusions regarding the overall performance of the 'code level tweaked' VBR presets. Other VBR/ABR/CBR presets are interesting too, but not as important. If problems with --alt preset standard are detected feel free to compare extreme and insane too. Test the following combinations please:

VBR/ABR
(~256kbps) 3.96 --preset extreme vs. 3.90.3 --alt-preset extreme
(~210kbps) [span style='font-size:11pt;line-height:100%']3.96 --preset standard vs. 3.90.3 --alt-preset standard[/span]
(~160kbps) 3.96 -V 4 vs. 3.96 --preset 160 vs. 3.90.3 --alt-preset 160*
(~128kbps) 3.96 -V 5 vs. 3.96 --preset 128 vs. 3.90.3 --alt-preset 128*

CBR
(320kbps) 3.96 --preset insane vs. 3.90.3 --alt-preset insane
3.96 --preset cbr <bitrate> vs. 3.90.3 --alt-preset cbr <bitrate>

If you want, you can additionally test VBR/ABR vs. CBR at comparable bitrates

3. You may use any sample you want, as long as you upload a losslessly compressed version of that (or provide a working link), so others can verify your results. The upload thead is here (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&). Using samples that have been used for testing 3.96beta (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&) is a good idea to show progress/regressions.

4. Your test results have to include the following:Notes:

*
Since ABR presets decrease gain as workarround against audible clipping on decoding of loud tracks, please use replaygain (trackgain) for decoding to .wav / ABXing, ideally using foobar2000 (apply to the original as well to get matching volume, of course!). On quiet tracks --scale 1 can be added after --(alt-)preset <bitrate> on encoding alternatively. Details see here (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203062)


There won't be a separate discussion thread for now. To make it easy to find posts containing results, please start such posts (only!) with "RESULT". Results matching the requirements that have been posted elsewhere will be added to the "Results" post.

[span style='font-size:7pt;line-height:100%']-----------------------------------------------
Edit 1 [2004/04/13]: typo fixed (3.96b2 -> 3.96), thanks to Lyx
Edit 2 [2004[04[14]: Replaygain/--scale related comment added. (-> "*")[/span]
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-13 17:10:37
Test results so far:


VBR/ABR


~ 256kbps

----------------------- 3.96 --p extreme vs. 3.90.3 --ap exteme
3.96 --p extreme > 3.90.3 --ap extreme :: drone_short (http://ff123.net/samples/drone_short.flac) :: freakngoat (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204392) :: 0x verified so far
---------------------------------------------------
3.96b1 --p extreme < 3.90.3 --ap extreme :: Lazy_Jones (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191243) :: owowo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=196184) :: 0x verified so far

----------------------- 3.96 --p extreme vs. 3.96 -V 1
3.96 --p extreme < 3.96 -V 1 :: drone_short (http://ff123.net/samples/drone_short.flac) :: freakngoat (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204397) :: 0x verified so far + missing ABX results*



~ 210kbps
------------- 3.96 --p standard vs. 3.90.3 --ap standard
3.96b1** --p standard > 3.90.3 --ap standard :: spahm (http://www.ff123.net/samples.html) :: Pio2001 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195221) :: 1x verified by Wombat (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195471)
3.96b1** --p standard > 3.90.3 --ap standard :: Birds (http://www.halke.net/files/Birds.flac) :: Wombat (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195226) :: 0x verified so far
3.96b1** --p standard > 3.90.3 --ap standard :: hokuscaredpiano (http://members.cox.net/moitah/hokuscaredpiano.flac) :: Moitah (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195477) :: 0x verified so far
3.96b1** --p standard > 3.90.3 --ap standard :: Lazy_Jones (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191243) :: owowo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=195560) :: 1x verified by freakngoat (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=196983)
3.96 --p standard > 3.90.3 --ap standard :: fatboy (http://lame.sourceforge.net/download/samples/fatboy.wav) :: High Fidelity (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202791) :: 1x verified 2Bdecided (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204817)
3.96 --p standard > 3.90.3 --ap standard :: Drone_short (http://ff123.net/samples/drone_short.flac) :: freakngoat (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203857) :: 0x verified so far
3.96 --p standard > 3.90.3 --ap standard :: awe32_20sec (http://jaz.project-psy.com/files/awe32-20sec.flac) :: 2Bdecided (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204818) :: 0x verified so far
---------------------------------------------------
3.96b2 --p standard = 3.90.3 --ap standard :: 41_30sec (http://www.ff123.net/samples/41_30sec.flac) :: ViPER1313 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=198434) :: 0x verified so far
---------------------------------------------------
3.96b2 --p standard < 3.90.3 --ap standard :: myf_4sec (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195030) :: LoFiYo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=198120) :: 0x verified so far
3.96b2 --p standard < 3.90.3 --ap standard :: Hustlejet (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=187351) :: harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=198402) :: 0x verified so far
3.96b2 --p standard < 3.90.3 --ap standard :: Chanchan1 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=197226) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=199021) :: 0x verified so far
3.96b2 --p standard < 3.90.3 --ap standard :: Hosokawa___Atem_lied (http://www.hydrogenaudio.org/forums/index.php?showtopic=18360&view=findpost&p=188327) :: harashin (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=198402) :: 0x verified so far
3.96 --p standard < 3.90.3 --ap standard :: 41_30sec (http://www.ff123.net/samples/41_30sec.flac) :: Gecko (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203110) :: 0x verified so far

--------- 3.96 --p fast standard vs. 3.90.3 --ap fast standard
3.96 --p fast standard > 3.90.3 --ap fast standard :: myf_4sec (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195030) :: LoFiYo (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204717) :: 0x verified so far


~ 160kbps
---------------------------------------------------
3.96b2 -V 4 < 3.90.3 --ap 160 :: myf_4sec (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195030) :: LoFiYo (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=199397) :: 0x verified so far


~ 128kbps
-------------------- 3.96 -V 5 vs. 3.96 --p 128
3.96 -V 5 > 3.96 --p 128 :: Atrain (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 2x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203360), [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203960)
3.96 -V 5 > 3.96 --p 128 :: BachS1007 (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203360) :: 0x verified so far
3.96 -V 5 > 3.96 --p 128 :: FloorEssence (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 2x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203531), [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203960)
3.96 -V 5 > 3.96 --p 128 :: Layla (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 2x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203531), [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 -V 5 > 3.96 --p 128 :: MidnightVoyage (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203670)
3.96 -V 5 > 3.96 --p 128 :: Waiting (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 2x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703), tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203720)
3.96 -V 5 > 3.96 --p 128 :: Blackwater (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203531) :: 1x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 -V 5 > 3.96 --p 128 :: LifeShatters (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203670) :: 0x verified so far
---------------------------------------------------
3.96 -V 5 < 3.96 --p 128 :: BeautySlept (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203360) :: 0x verified so far
3.96 -V 5 < 3.96 --p 128 :: Blackwater (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 0x verified so far + missing ABX results*
3.96 -V 5 < 3.96 --p 128 :: TheSource (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203720)
3.96 -V 5 < 3.96 --p 128 :: LisztBMinor (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203670) :: 1x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 -V 5 < 3.96 --p 128 :: BachS1007 (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203960) :: 0x verified so far

-------------------- 3.96 -V 5 vs. 3.90.3 --ap 128
3.96 -V 5 > 3.90.3 --ap 128 :: fatboy (http://www.mp3dev.org/mp3/gpsycho/quality.html) :: FatBoyFin (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203662) :: 1x verified by High Fidelity (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202791)
3.96b2 -V 5 > 3.90.3 --ap 128 :: Its_me (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=201613) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=201617) :: 0x verified so far
3.96b2 -V 5 > 3.90.3 --ap 128 :: entierren con rumba (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=201619) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=201622) :: 0x verified so far
3.96 -V 5 > 3.90.3 --ap 128 :: Atrain (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203360)
3.96 -V 5 > 3.90.3 --ap 128 :: BeautySlept (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203360) :: 0x verified so far
3.96 -V 5 > 3.90.3 --ap 128 :: FloorEssence (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203531)
3.96 -V 5 > 3.90.3 --ap 128 :: Layla (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 2x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203531), [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 -V 5 > 3.90.3 --ap 128 :: LifeShatters (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203670)
3.96 -V 5 > 3.90.3 --ap 128 :: MidnightVoyage (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203670)
3.96 -V 5 > 3.90.3 --ap 128 :: Thear1 (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 0x verified so far + missing ABX results*
3.96 -V 5 > 3.90.3 --ap 128 :: Waiting (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 2x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703), tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203720)
3.96 -V 5 > 3.90.3 --ap 128 :: Blackwater (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203531) :: 0x verified so far
---------------------------------------------------
3.96 -V 5 < 3.90.3 --ap 128 :: BachS1007 (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203360) :: 1x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 -V 5 < 3.90.3 --ap 128 :: Blackwater (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 -V 5 < 3.90.3 --ap 128 :: TheSource (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203720)
3.96 -V 5 < 3.90.3 --ap 128 :: LisztBMinor (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203670) :: 1x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 -V 5 < 3.90.3 --ap 128 :: Atrain (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203960) :: 0x verified so far
3.96 -V 5 < 3.90.3 --ap 128 :: FloorEssence (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203960) :: 0x verified so far

-------------------- 3.96 --p 128 vs. 3.90.3 --ap 128
3.96 --p 128 > 3.90.3 --ap 128 :: Atrain (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203360)
3.96 --p 128 > 3.90.3 --ap 128 :: BeautySlept (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203360) :: 0x verified so far
3.96 --p 128 > 3.90.3 --ap 128 :: FloorEssence (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 0x verified so far + missing ABX results*
3.96 --p 128 > 3.90.3 --ap 128 :: Layla (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 0x verified so far + missing ABX results*
3.96 --p 128 > 3.90.3 --ap 128 :: LifeShatters (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 0x verified so far + missing ABX results*
3.96 --p 128 > 3.90.3 --ap 128 :: MidnightVoyage (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203670)
3.96 --p 128 > 3.90.3 --ap 128 :: Thear1 (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 0x verified so far + missing ABX results*
3.96 --p 128 > 3.90.3 --ap 128 :: Waiting (http://ff123.net/samples/) :: ff123 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=202952) :: 1x verified by tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203720)
3.96° --p 128 > 3.90.3 --ap 128 :: applaud (http://lame.sourceforge.net/download/samples) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20641&view=findpost&p=202640) :: 0x verified so far + missing ABX results*
3.96° --p 128 > 3.90.3 --ap 128 :: campestre (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=195617) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20641&view=findpost&p=202640) :: 0x verified so far + missing ABX results*
3.96° --p 128 > 3.90.3 --ap 128 :: fatboy (http://lame.sourceforge.net/download/samples) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20641&view=findpost&p=202640) :: 0x verified so far + missing ABX results*
3.96° --p 128 > 3.90.3 --ap 128 :: Preecho1 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=203851) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20641&view=findpost&p=202640) :: 0x verified so far + missing ABX results*
---------------------------------------------------
3.96 --p 128 < 3.90.3 --ap 128 :: Waiting (http://www.ff123.net/samples/Waiting.flac) :: ViPER1313 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=202700) :: 1x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 --p 128 < 3.90.3 --ap 128 :: BachS1007 (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203360) :: 1x verified by Atrain (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203960)
3.96 --p 128 < 3.90.3 --ap 128 :: FloorEssence (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203531) :: 1x verified by Atrain (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203960)
3.96 --p 128 < 3.90.3 --ap 128 :: Layla (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203531) :: 1x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 --p 128 < 3.90.3 --ap 128 :: LifeShatters (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203670) :: 0x verified so far
3.96 --p 128 < 3.90.3 --ap 128 :: LisztBMinor (http://ff123.net/samples/) :: tigre (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203670) :: 1x verified by [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703)
3.96 --p 128 < 3.90.3 --ap 128 :: Blackwater (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203703) :: 0x verified so far
3.96° --p 128 < 3.90.3 --ap 128 :: ct_reference (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=203852) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20641&view=findpost&p=202640) :: 0x verified so far + missing ABX results*
3.96° --p 128 < 3.90.3 --ap 128 :: fall (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=195618) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20641&view=findpost&p=202640) :: 0x verified so far + missing ABX results*
3.96° --p 128 < 3.90.3 --ap 128 :: rebel (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=195375) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20641&view=findpost&p=202640) :: 0x verified so far + missing ABX results*
3.96 --p 128 < 3.90.3 --ap 128 :: Atrain (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203960) :: 0x verified so far

---------- 3.96 -V 5 --athaa-sensitivity 1 vs. 3.96 -V 5
3.96 -V 5 --a-s 1 > 3.96 -V 5 :: Atrain (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204827) :: 0x verified so far + missing ABX results*
3.96 -V 5 --a-s 1 > 3.96 -V 5 :: Blackwater (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204827) :: 0x verified so far + missing ABX results*
3.96 -V 5 --a-s 1 > 3.96 -V 5 :: BachS1007 (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204827) :: 0x verified so far + missing ABX results*
3.96 -V 5 --a-s 1 > 3.96 -V 5 :: LisztBMinor (http://ff123.net/samples/) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204827) :: 0x verified so far + missing ABX results*
3.96 -V 5 --a-s 1 > 3.96 -V 5 :: Bayle___Jeita_or_Waters___Murmur___Etching (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=204880) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204877) :: 0x verified so far
3.96 -V 5 --a-s 1 > 3.96 -V 5 :: Track01__Avison___Concerto_Grosso_30_sec (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&view=findpost&p=204882) :: [proxima] (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204877) :: 0x verified so far



CBR


320kbps
no results so far


160kbps
no results so far


128kbps
---------------------------------------------------
3.96 --p cbr 128 < 3.90.3 --ap cbr 128 :: 41_30sec (http://www.ff123.net/samples/41_30sec.flac) :: ViPER1313 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=202211) :: 1x verified by SometimesWarrior (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=202243), Brainchild4010 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=202339)
3.96 --p cbr 128 < 3.90.3 --ap cbr 128 :: Waiting (http://www.ff123.net/samples/Waiting.flac) :: ViPER1313 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&view=findpost&p=202686) :: 0x verified so far
3.96 --p cbr 128 < 3.90.3 --ap cbr 128 :: Layla (http://www.ff123.net/samples/Layla.flac) :: ViPER1313 (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=203374) :: 0x verified so far


[span style='font-size:8pt;line-height:100%']* If a result doesn't match the minimum requirements (-> 4. in 1st post), the result is 'greyed out' until the missing data is provided or someone else confirms the results. If your result isn't included in this list at all, we're either too slow, or there's too much missing (ABX results, link to the sample, clear statment which version is better, information about lame version/setting used etc.). If you want to provide missing information, do it in a new post to ensure that we notice it.

** Results where 3.96 beta 1 --preset standard outperformed 3.90.3 --alt-preset standard were kept as the only difference between 3.96 beta1 and 3.96 final is the increased minimum bitrate, so no regression should be possible.

° Results by [proxima] with past-beta-2 internal version that should give results identical with 3.96 final[/span]


------------------------------------------------------------
Edit: All results from other threads I could find that belong here are added. - Tell me, if I've missed something, please.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Latexxx on 2004-04-13 17:39:39
Quote
small typo

Quote

4. Your test results have to include the following:
  • ABX results for
    3.90.3 vs. Original
    3.96b2 vs. Original  <--------
    3.96b2 vs. 3.90.3  <--------


  • ABC/HR results are appreciated especially at lower bitrates, but shouldn't be considered a requirement.


  • (Short) descriptions of the artifacts/differences

The latest version is 3.96, not 3.96b2.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: High Fidelity on 2004-04-13 20:18:31
RESULT


I did a short ABX with fatboy.wav (http://lame.sourceforge.net/download/samples/fatboy.wav)
for preset standard and --preset 128

My results:

3.96 --preset standard vs. 3.90.3 --alt-preset standard

3.90.3 vs. Original
Total: 10/10 (0.1%)
Comment: Details and voice sounded a bit brighter (?) or more open - the original (I guess at least) softer but a bit more full

3.96 vs. Original
Total: 7/10 (17.2%)
Comment: Only minimal details in the background seemed a bit softer.
Hardly to ABX! Very good!

3.96 vs. 3.90.3
Total: 10/10 (0.1%)
Comment: Soft details in the background a little bit different, but hard to ABX.


---------------------------

3.96 --preset 128 vs. 3.90.3 --alt-preset 128

3.90.3 vs. Original
Total: 10/10 (0.1%)
VERY easy to distinguish metallic and distorted sound

3.96 vs. Original
Total: 10/10 (0.1%)
Comment: still easy to distinguish from the original, but very much better than 3.90.3! less distorsion, sounds higher than original

3.96 vs. 3.90.3
Total: 10/10 (0.1%)
Comment: 3.96 is the winner!!!

---------------


At least for fatboy.wav 3.96 is my favourite.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-14 08:18:50
RESULT

My settings:

3.90.3 --alt-preset 128 --scale 1
3.96 --preset 128 --scale 1
3.96 -V 5

Using the 12 samples I used for the first 64 kbs test

Code: [Select]
                3.90.3 ap 128     3.96 p 128     3.96 V 5
Atrain          3.8               4.4            4.6
BachS1007       5                 5              5
BeautySlept     4.2               4.2            4.2
Blackwater      5                 5              4.5
FloorEssence    3.6               4.2            4.7
Layla           3.9               4.4            5
LifeShatters    4.1               4.4            4.4
liszt           4.8               4.8            4.8
MidnightVoyage  4.2               4.5            5
Thear1          4.5               5              5
TheSource       5                 5              4.5
waiting         3.9               4.3            4.8


abchr results files at http://ff123.net/export/lame3.96test.zip (http://ff123.net/export/lame3.96test.zip)
No ABX, though.

Code: [Select]
3.96V5   3.96p128 3.90.3ap 
 4.71     4.60     4.33  

---------------------------- p-value Matrix ---------------------------

        3.96p128 3.90.3ap
3.96V5   0.384    0.006*  
3.96p128          0.040*  
-----------------------------------------------------------------------

3.96V5 is better than 3.90.3ap128
3.96p128 is better than 3.90.3ap128


Looks like 3.90.3 ap 128 is the clear loser for me on these 12 samples

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-14 10:50:39
Thanks for the results so far.

I've performed a quick wave substraction test to find out for which settings the encoding results for beta1/beta2/final are identical. Results:
Quote
~128kbps
ABR:
final != beta1/beta2
CBR:
final != beta1/beta2
VBR (-V 5)
final == beta2 == beta1 (bit-identical)

~160kbps
ABR:
final != beta1/beta2
CBR:
final != beta1/beta2
VBR (-V 4)
final == beta2 == beta1 (bit-identical)

preset standard
final == beta2 != beta1

preset extreme
final == beta2 == beta1

preset insane
final != beta1/beta2


So the results from beta test thread are still valid for
3.90.3 --alt-preset 128 vs. 3.96 -V 5 (same for ap160 vs V4, but no results submitted)
3.90.3 --alt-preset standard vs. 3.96b2 --preset standard

edit: extreme/insane results added
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-14 14:39:45
The results post should be up-to-date now. Only the ABC/HR test by ff123 misses because I'm not sure if/how to add those results. I'd add them greyed out until they're verified with ABX results (by someone else) and only use the results with differences in ABC/HR ratings... Or should I add ABC/HR results like these as a completely separated section to results post?

Your opinion, ff123?
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-14 16:51:31
Quote
The results post should be up-to-date now. Only the ABC/HR test by ff123 misses because I'm not sure if/how to add those results. I'd add them greyed out until they're verified with ABX results (by someone else) and only use the results with differences in ABC/HR ratings... Or should I add ABC/HR results like these as a completely separated section to results post?

Your opinion, ff123?

Hmm, I don't know.  I guess it's up to you.  I'd like to see somebody else try all of those samples as well, though.

BTW, you'll notice that I added --scale 1 to the ABR command lines.  That's because they include some gain reduction in them that the VBR settings don't.

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-14 17:15:00
I am unable to reproduce the dropout problem in 41_30sec.wav in 3.96final using

--preset cbr 128

I have uploaded my encode to:  http://ff123.net/export/41_30sec_cbr128.mp3 (http://ff123.net/export/41_30sec_cbr128.mp3)

I can distinguish from the original, but it doesn't sound worse than plain artifacting to me.

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gabriel on 2004-04-14 17:18:53
Quote
I am unable to reproduce the dropout problem in 41_30sec.wav in 3.96final using

--preset cbr 128


There is a problem at 7.11s
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-14 17:26:22
Quote
Quote
I am unable to reproduce the dropout problem in 41_30sec.wav in 3.96final using

--preset cbr 128


There is a problem at 7.11s

Are you talking about the clicking sounds?  I hear them in the reference too.  Did you download my encode to see if it's the same?

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-14 17:32:04
Quote
BTW, you'll notice that I added --scale 1 to the ABR command lines.  That's because they include some gain reduction in them that the VBR settings don't.

I didn't notice the gain reduction until know, probably because I used foobar2000 with replaygain (track gain) enabled for decoding. Given that fb2k decodes gapless on replaygain scanning, this should be a decent sollution to avoid volume-related problems as well.

Related questions: I assume that the gain reduction of ABR presets is used to reduce problems related to clipping on decoding of loud tracks.
- Is there some other reason like a clipping problem with encoding comparable to musepack?
- Based on this: Is using --scale 1 for the test 100% save or can it cause problems?
I'd like to modify the test recommendations for ABR presets according to the answers to these questions: Recommend using replaygain for decoding (including original file) and/or using --scale 1 for ABR.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-14 17:48:50
Quote
I didn't notice the gain reduction until know, probably because I used foobar2000 with replaygain (track gain) enabled for decoding. Given that fb2k decodes gapless on replaygain scanning, this should be a decent sollution to avoid volume-related problems as well.

Related questions: I assume that the gain reduction of ABR presets is used to reduce problems related to clipping on decoding of loud tracks.
- Is there some other reason like a clipping problem with encoding comparable to musepack?
- Based on this: Is using --scale 1 for the test 100% save or can it cause problems?
I'd like to modify the test recommendations for ABR presets according to the answers to these questions: Recommend using replaygain for decoding (including original file) and/or using --scale 1 for ABR.

Back when Dibrom was tuning the cbr/abr presets, he noticed a clicking problem caused by clipping at low bitrates (I think the sample was badvilbel.wav).  So he put in a gain reduction which is greater at lower bitrates and progressively gets smaller as the bitrate increases.

I don't know if this problem still exists or not, but at 128 kbit/s, it was just barely audible, if I remember.

--scale 1 is better than using replaygain upon decoding because there is a resolution issue when using replaygain on mp3's.

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-14 18:07:42
Thanks for the explanation, ff123.

Quote
--scale 1 is better than using replaygain upon decoding because there is a resolution issue when using replaygain on mp3's.

Could you please explain? AFAIK foobar2000 decodes to 32bit float, applies DSPs (e.g. volume change) and truncates/dithers to fixedpoint (e.g. 16bit for use with ABC/HR). Maybe I've too many different things to focus on right now, but I don't see a resolution issue.

I've added your results to the post BTW. I hope this motivates people to repeat the test with these samples (-> ABX ... ).
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-14 18:18:43
Quote
Could you please explain? AFAIK foobar2000 decodes to 32bit float, applies DSPs (e.g. volume change) and truncates/dithers to fixedpoint (e.g. 16bit for use with ABC/HR). Maybe I've too many different things to focus on right now, but I don't see a resolution issue.

Oh, ok then.  I guess I was thinking of mp3gain, which can only adjust in 0.3 dB increments.  If fb2k applies replaygain subsequent to decoding, then everything is fine.

Edit:

Regarding testing of the 12 samples:  I think it's more important that 1 or 2 people verify the entire suite (even if ABX results are not supplied) than it is for several people to ABX several samples out of the 12.  So it would look more like the tests performed to find the best vorbis encoder.  However, beggars can't be choosers, so any verification at all is welcome.

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gabriel on 2004-04-14 18:36:17
Quote
Are you talking about the clicking sounds? I hear them in the reference too. Did you download my encode to see if it's the same?

Yes, the clicking sound.

I downloaded your encoded file, and it is the same as mine. However I have to admit that I did not checked it the click was also in the original one.
It was pointed that there was "a serious bug" in this section of the clip, with "awfull clicks". I searched for it in the encoded, but not in the original.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gecko on 2004-04-14 18:53:19
RESULT

Sample: 41_30sec.flac (Which I downloaded from here some time ago. There is the (slight?) chance it is not the same as the one others have tested here)

Time: Always from 7.9 to 9.0 seconds, focusing on the hi-hats.

Setting: --alt-preset standard

(Results are in the order of testing)

3.90.3 - 3.96 -- 11/12 + 4/4 = 15/16
3.96 more "wet" (pre-/post-echo?)

orig - 3.96 -- 15/16
3.96 less sharp. Slightly muffled perhaps

orig - 3.90.3 -- 8/8
3.90.3 stops too early or something. The original seems to reverbarate longer.

The artifact 3.90.3 produces sounds quite different than the one from 3.96. [wild speculation]Maybe aggressive anti-post-echo whatever is causing it.[/wild speculation] Personally I find 3.90.3 gives the better representation. I'd rather have something that doesn't remind me of the squishy pre-echo sound.

ABXYing was relatively hard, so I tried something new which I would like to call conscious self deception. You listen to A and B... try to pick a distinct sound of one of the samples (say, A sounds more squishy). Now play back both X/Y and try to mentally project that sound onto both samples. Try to make them sound like what you think A sounds like in your brain. More often than not, you will find this projection only works well on one of the samples. That is your hot candidate. If the projection doesn't work, you probably only picked up some placebo difference. Just try again!

Hope I'm making sense.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-14 21:56:05
Quote
The artifact 3.90.3 produces sounds quite different than the one from 3.96. [wild speculation]Maybe aggressive anti-post-echo whatever is causing it.[/wild speculation] Personally I find 3.90.3 gives the better representation. I'd rather have something that doesn't remind me of the squishy pre-echo sound.

None of this sounds like a bug, which is what the talk about dropouts made it sound like.  From the initial description, I was expecting something bad like the old Fraunhofer mp3enc 3.1 dropouts.

At this point, I think it's just normal artifacting which would naturally tend to sound worse in cbr than it would in abr or vbr.

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ViPER1313 on 2004-04-15 03:44:06
This whole thing has been blown out of proportion. Saying that there are bugs or major problems with v3.96 is most likely wrong. I stated

Quote
Lame v3.96 final seems to have a serious problem w/ 41_30sec in the 6-8 second range using --preset cbr 128. It sounds like a dropout / skip in the sound. This problem has been seen in Mitok's compile as well as ICL v8.0 and MSVC compiles made by myself. There is no skip using v3.90.3 . Could someone confirm this??

It's very possible it's just an artifact - nowhere did I ever state that v3.96 had a major drop-out problem with many clips. I still don't know why the clip seems to pop - I don't know why the clip would do it in one spot either and not over and over again. Has anyone checked the flac file for errors? Why would the psy model mess up on the one small section of the file (no the hardest to encode by any means.)

Quote
It was pointed that there was "a serious bug" in this section of the clip, with "awfull clicks". I searched for it in the encoded, but not in the original.

Nowhere did I ever state this. Also, the artifact is not in the origional from what I can hear.

Quote
I can confirm that there is a failure on this sample on frame 272, 2nd granule (7.11s) when encoding in cbr 128. The output on mp3X is very strange.

What do you mean by "failure?"

I agree that warnings of bugs and problems are not really necessary, but I feel that more testing is.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gabriel on 2004-04-15 08:41:59
Quote
What do you mean by "failure?"

I mean that the scalefactors are looking very suspicious on the second granule of frame 272. This seems strange to me.
If you are interested, you can check it with mp3x (available from Mitiok if I remember well)

I do not see a whole failure over the 41_30 clip, but at this specific point there is something strange to me.

Regarding the 41_30 clip itself, this one is unusual. It seems that there is a carrier or pilot signal around 15kHz.

Quote
This whole thing has been blown out of proportion. Saying that there are bugs or major problems with v3.96 is most likely wrong.

I totally agree.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: 2Bdecided on 2004-04-15 09:33:38
Quote
Regarding the 41_30 clip itself, this one is unusual. It seems that there is a carrier or pilot signal around 15kHz.

That's TV line whistle - you find that on lots of recordings.

Cheers,
David.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-15 14:42:02
RESULTS

Tested settings:
Lame 3.90.3 --alt-preset 128
Lame 3.96 --preset 128
Lame 3.96 -V 5

Tested samples (http://ff123.net/samples/) from ff123's 64kbps test:

Atrain
BachS1007
BeautySlept

All encoded files were ABXed successuflly (p < 0.01) against the original and against each other. For details (abx logs, descriptions of the audible differeces) see attatched .zip file.

Results (ABC/HR ratings):

Atrain
2.0 Lame 3.90.3 --alt-preset 128
3.0 Lame 3.96 --preset 128
3.5 Lame 3.96 -V 5

BachS1007
4.0 Lame 3.90.3 --alt-preset 128
3.0 Lame 3.96 --preset 128
3.3 Lame 3.96 -V 5

BeautySlept
2.0 Lame 3.90.3 --alt-preset 128
3.6 Lame 3.96 --preset 128
3.2 Lame 3.96 -V 5
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Lyx on 2004-04-15 15:28:21
I'm not sure if this is off-topic, so if it is feel free to move this post out of this thread.
I found an album where both 3.96b1 and 3.96final have severe problems to encode it gaplessly, while 3.90.3 does it just fine(or with to me inaudible glitches).

Its name is "Mathias Grassow - Himavat". Its an ambient album which seems to be hard to encode because it is a very synthetic- and pure-sounding album and mostly very quiet. Mostly only drones and often only two or one of them, so there aren't many layers of sounds(i think thats important for "masking"?).

I am sorry, i cannot provide samples right now, but i will try to do that soon - but it may take up to a week because i'm currently quite busy. If this is uninteresting, then just delete this post :-)

- Lyx
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ViPER1313 on 2004-04-15 15:48:18
RESULT

Layla (http://www.ff123.net/samples/Layla.flac) test sample using v3.90.3 and v3.96 final at --preset cbr 128 . I have a cold right now and everything I hear sounds like its lowpassed, so I do not know how this effects my ratings of the two files - I am certain that the beginning of Layla using v3.96 sounds worse than v3.90.3 though....

Quote
ABC/HR Version 0.9b, 30 August 2002
Testname: Layla --preset cbr 128

1R = Z:\Music\Temp\Layla v3.90.3 --preset cbr 128.wav
2L = Z:\Music\Temp\Layla v3.96 --preset cbr 128.wav

---------------------------------------
General Comments:
Focused on first two seconds (clapping) of clip
---------------------------------------
1R File: Z:\Music\Temp\Layla v3.90.3 --preset cbr 128.wav
1R Rating: 4.5
1R Comment: Very little distortion, very good overall
---------------------------------------
2L File: Z:\Music\Temp\Layla v3.96 --preset cbr 128.wav
2L Rating: 3.0
2L Comment: Heavy distortion in first two seconds of file, bad pre-echo
---------------------------------------
ABX Results:
Original vs Z:\Music\Temp\Layla v3.90.3 --preset cbr 128.wav
    8 out of 8, pval = 0.004
Original vs Z:\Music\Temp\Layla v3.96 --preset cbr 128.wav
    8 out of 8, pval = 0.004
Z:\Music\Temp\Layla v3.90.3 --preset cbr 128.wav vs Z:\Music\Temp\Layla v3.96 --preset cbr 128.wav
    13 out of 14, pval < 0.001
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-15 16:09:24
Lyx: Could you give some more details please?

- How do you notice that there are gap problems - Are you talking about audible glitches / clicks or have you checked file length and/or offset manually?

- What did you use for playback / decoding?
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Lyx on 2004-04-15 16:40:12
I didn't do the test with 3.90.3 myself yet. there are two people involved.

The following happened:
- i helped another person (zolder) how to setup EAC with LAME and preset standard.
- we did setup encoder and drive -offset calibration in EAC and used it
- then zolder ripped the CD to wav. Secure Mode, no normalizing, no change to the ripped audiodata.
- He listened to the WAVs with foobar, 16bit non-dithered or waveshaped output. No problems audible. Everything was fine.
- after that, he encoded them the LAME 3.96b1 --preset standard. And we both listened to it again on foobar (same fb2k-settings). At some trackchanges problems were audible.
- we did the same again with LAME 3.96final - same problems
- then, zolder encoded the album with LAME 3.90.3 --alt-preset standard and listened to it with foobar(same settings again), especially to the trackchanges which we learned are notorious for problems. According to him, everything sounded okay with 3.90.3. I have talked alot about gapless playback with him in the previous days, and he knows what to look for.

"How it sounds like": mostly clicks and pops.... like a sudden displacement of the waveform - as if the two waves(of the 2 tracks) aren't aligned correctly at the trackchange. I may be able to get the sample earlier, no guarantees, but i'll try my best.

- Lyx
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gecko on 2004-04-15 19:03:36
Quote
None of this sounds like a bug, which is what the talk about dropouts made it sound like.  From the initial description, I was expecting something bad like the old Fraunhofer mp3enc 3.1 dropouts.

At this point, I think it's just normal artifacting which would naturally tend to sound worse in cbr than it would in abr or vbr.

Well, I didn't test the range in which the dropout problem occurs and I only tested (alt) preset standard. Iirc, the problem has only been reported for one of the 128k-ish settings.

I don't think you can draw any conclusions about the dropout problem from my test.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ssamadhi97 on 2004-04-15 19:10:47
Quote
That's TV line whistle - you find that on lots of recordings.

How does this kind of noise make its way into studio tracks anyway? Just wondering..

(sorry 'bout the OT)
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-15 23:22:08
RESULTS

Tested settings:
Lame 3.90.3 --alt-preset 128
Lame 3.96 --preset 128
Lame 3.96 -V 5

Tested samples from ff123's 64kbps test:

Blackwater
FloorEssence
Layla

All encoded files were ABXed successuflly (p < 0.01) against the original and against each other (or got equal ratings). For details (abx logs, descriptions of the audible differeces) see attatched .zip file.

Results (ABC/HR ratings):

Blackwater
3.7 Lame 3.90.3 --alt-preset 128
3.7 Lame 3.96 --preset 128
3.9 Lame 3.96 -V 5

FloorEssence
2.5 Lame 3.90.3 --alt-preset 128
2.2 Lame 3.96 --preset 128
3.8 Lame 3.96 -V 5

Layla
3.4 Lame 3.90.3 --alt-preset 128
3.0 Lame 3.96 --preset 128
4.0 Lame 3.96 -V 5
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-16 01:03:35
Quote
RESULTS

Tested settings:
Lame 3.90.3 --alt-preset 128
Lame 3.96 --preset 128
Lame 3.96 -V 5

Tested samples from ff123's 64kbps test:

Blackwater
FloorEssence
Layla

It's looking like tigre's results may conflict with mine regarding the comparison between the 3.90.3 and 3.96 128 settings (although that could change with the rest of the samples).

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: FatBoyFin on 2004-04-16 12:29:43
RESULT

Quote
3.96b2 -V 5 > 3.90.3 --ap 128 :: fatboy :: FatBoyFin :: 1x verified by High Fidelity using 3.96 final


Ive also comfirmed this for 3.96 now with ABX results atleast 7/7.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-16 13:22:43
RESULTS

Tested settings:
Lame 3.90.3 --alt-preset 128
Lame 3.96 --preset 128
Lame 3.96 -V 5

Tested samples from ff123's 64kbps test:

LifeShatters
LisztBMinor
MidnightVoyage

All encoded files were ABXed successuflly (p < 0.01) against the original and against each other. For details (abx logs, descriptions of the audible differeces) see attatched .zip file.

Results (ABC/HR ratings):

LifeShatters
4.2 Lame 3.90.3 --alt-preset 128
4.0 Lame 3.96 --preset 128
4.5 Lame 3.96 -V 5

LisztBMinor
2.5 Lame 3.90.3 --alt-preset 128
1.5 Lame 3.96 --preset 128
1.0 Lame 3.96 -V 5

MidnightVoyage
3.0 Lame 3.90.3 --alt-preset 128
3.6 Lame 3.96 --preset 128
4.0 Lame 3.96 -V 5
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: [proxima] on 2004-04-16 15:48:33
RESULTS

Tested settings:
Lame 3.90.2 --alt-preset 128 --scale 1
Lame 3.96 --preset 128 --scale 1
Lame 3.96 -V 5

Layla is a strange case, 3.96 abr has a metallic artifact in the first two seconds. Preecho in all the three but (of course) well handled with VBR encoded sample.
3.90.2 : 3.0
3.96abr: 2.5
3.96_V5: 3.8

LisztBMinor is a low volume sample and VBR is not the best anymore. With all the three samples there are problems with background noise and ringing/chirping. VBR can offer better preecho handling but maybe with such samples is nothing special.
3.90.2 : 3.0
3.96abr: 2.1
3.96_V5: 2.0

Blackwater is very indicative for me. The well known chirping/ringing artifact, even if reduced with the new 3.96 are still present in the two 3.96 encodings, 3.90.2 doesn't ring at all. Again, preecho is better with -V 5.
3.90.2 : 3.2
3.96abr: 2.8
3.96_V5: 2.9

With Waiting critical sample VBR is advantaged. 3.96 --ap 128 manifested a bad ringing in some parts.
3.90.2 : 2.2
3.96abr: 1.5
3.96_V5: 3.3

ABC/HR and ABX results here: http://xoomer.virgilio.it/fofobella/396_1.zip (http://xoomer.virgilio.it/fofobella/396_1.zip)

However 3.96 is not always worse than 3.90.2: with some samples the progress is real. It's a pity for this slight ringing artifact that penalize 3.96 too much for me.
I plan to complete all the 12 samples during this weekend 

@tigre: In my previous post i linked a test done with lamex9 slightly in favour of 3.96. I cheched 3.96 vs lamex9 --p 128 --sfscale and they produce the same file, i think that this test could be added to the results even if greyed out.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-16 16:08:36
Quote
,Apr 16 2004, 06:48 AM] Blackwater is very indicative for me. The well known chirping/ringing artifact, even if reduced with the new 3.96 are still present in the two 3.96 encodings, 3.90.2 doesn't ring at all. Again, preecho is better with -V 5.
3.90.2 : 3.2
3.96abr: 2.8
3.96_V5: 2.9

Did lame switch back to gpsycho or something?

The ringing artifacting that people complained about in gpsycho was always something that I couldn't hear, because it occurs out of my high frequency range.  This was also something that nspsytune seemed to handle better.

I will have a tendency to hear lower frequency artifacting, such as preecho, warbling or fluttering.  So this might be a case where there could be an interesting split of opinions.

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-16 16:30:59
RESULTS

Tested settings:
Lame 3.90.3 --alt-preset 128
Lame 3.96 --preset 128
Lame 3.96 -V 5

Tested samples from ff123's 64kbps test:

Thear1
TheSource
Waiting

All encoded files were ABXed successuflly (p < 0.01) against the original and against each other (or got equal ratings). For details (abx logs, descriptions of the audible differeces) see attatched .zip file.

Results (ABC/HR ratings):

Thear1
4.5 Lame 3.90.3 --alt-preset 128
4.5 Lame 3.96 --preset 128
4.5 Lame 3.96 -V 5

TheSource
4.0 Lame 3.90.3 --alt-preset 128
4.0 Lame 3.96 --preset 128
3.8 Lame 3.96 -V 5

Waiting
2.0 Lame 3.90.3 --alt-preset 128
2.2 Lame 3.96 --preset 128
3.0 Lame 3.96 -V 5
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: [proxima] on 2004-04-16 16:37:25
Quote
Quote
,Apr 16 2004, 06:48 AM] Blackwater is very indicative for me. The well known chirping/ringing artifact, even if reduced with the new 3.96 are still present in the two 3.96 encodings, 3.90.2 doesn't ring at all. Again, preecho is better with -V 5.
3.90.2 : 3.2
3.96abr: 2.8
3.96_V5: 2.9

Did lame switch back to gpsycho or something?

I will have a tendency to hear lower frequency artifacting, such as preecho, warbling or fluttering.  So this might be a case where there could be an interesting split of opinions.

ff123

Ringing problems, at least for some samples are partially resolved with the changed noise shaping type for --preset 128. 3.96 that defaults to ns 2 is much more better with ringing problems (rebel sample is a very strong proof, even with spectral analisys there are less droputs above 10 kHz). Small problems still remain and i prefer 3.90.2 for some cases.
3.96 seems to be much more "aggresive" with background noise (see blackwater, LisztBMinor) and this could "unmask" artifacts with mid-high frequencies.
Ringing artifact noticeable with blackwater is not audible with 3.90.2 maybe because there is the original background noise.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-16 16:57:10
I haven't compared directly 3.96 beta vs. 3.96 final yet, but I share your observation that there are less ringing problems.

LisztBMinor is a very interesting sample indeed. I just did a quick test with 3.96 --preset standard. There ringing/warbeling added to background noise when the piano starts playing is quite obvious (of course much better then with --preset 128 or -V 5). This seems to be the next case where 3.90.3 outperforms 3.96 with --(alt-)preset standard. (and this is one of the few samples where the bitrate of 3.96 is higher).

I think this sample could be very useful to further improve 3.96 (mainly VBR modes).
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-16 17:00:53
For tigre, the results of the 12 samples do not indicate any preference:

Code: [Select]
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 12
Critical significance:  0.05
Significance of data: 1.69E-01 (not significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of         Degrees     Sum of    Mean
variation         of Freedom  squares   Square    F      p

Total               35          28.68
Testers (blocks)    11          21.23
Codecs eval'd        2           1.11    0.56    1.93  1.69E-01
Error               22           6.34    0.29
---------------------------------------------------------------


Also, when averaged with my results, there is no significant preference (although it is just on the cusp of doing so):

Code: [Select]
FRIEDMAN version 1.24 (Jan 17, 2002) http://ff123.net/
Blocked ANOVA analysis

Number of listeners: 12
Critical significance:  0.05
Significance of data: 5.45E-02 (not significant)
---------------------------------------------------------------
ANOVA Table for Randomized Block Designs Using Ratings

Source of         Degrees     Sum of    Mean
variation         of Freedom  squares   Square    F      p

Total               35          10.92
Testers (blocks)    11           7.08
Codecs eval'd        2           0.89    0.45    3.33  5.45E-02
Error               22           2.95    0.13
---------------------------------------------------------------


ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-16 17:01:41
Quote
,Apr 16 2004, 04:48 PM] @tigre: In my previous post i linked a test done with lamex9 slightly in favour of 3.96. I cheched 3.96 vs lamex9 --p 128 --sfscale and they produce the same file, i think that this test could be added to the results even if greyed out.

Edit: I found the post you're talking about here (http://www.hydrogenaudio.org/forums/index.php?showtopic=20641&view=findpost&p=202640)

Could you please provide links to the samples used (if I haven't found them already)? I'll add the results to the upload thread once I have links to all samples (editing the post is not very amusing so I'd like to do it in an efficient way). The samples are:

applaud :: from ff123's sample page? If yes, which one?
campestre :: Here (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&)?
ct_reference :: I have no idea where to find that one
fall :: Here (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&)?
fatboy :: ff123's version or another (longer) one?
preecho1 :: Tell me, please
rebel :: Here (http://www.hydrogenaudio.org/forums/index.php?showtopic=19882&)?
wating :: Probably the well know one from ff123's page...
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-16 17:10:00
Quote
For tigre, the results of the 12 samples do not indicate any preference:
<snip>
Also, when averaged with my results, there is no significant preference (although it is just on the cusp of doing so):
<snip>

Hehe - I've just tried the same. The only thing that gives a significant result is calculating a p-value from our total results for 3.96 -V 5 > 3.90.3 --alt-preset 128

(11+8)/(12+12) = 19/24 -> p = 0.0033

but I doubt that this calculation is correct since every sample is used twice.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Hoosierdaddy on 2004-04-16 19:19:46
"How it sounds like: mostly clicks and pops...like a sudden displacement of the waveform - as if the two waves(of the 2 tracks) aren't aligned correctly at the trackchange."

So this is a 'glitch' which the encoder encounters when trying to encode the interstill BETWEEN tracks on a CD? Hmmm. I thought the 'gap' on a cd was actually 'digital silence'. As I understand it, there isn't any actual audio data there to encode. Just two embedded 'flags' which  the signal end of one track and the beginning of another to a CD player. I wonder if the blooper occurs because there really isn't any audio data to encode?
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Shingetsu on 2004-04-16 20:08:40
and is there some quality loss betwen 3.90.3 and 3.96 which could be audibly in normal music ? this sounds all so complicated.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-16 20:13:58
Code: [Select]
Means:

3.96V5   3.96p128 3.90.3  
 4.03     3.74     3.65  

---------------------------- p-value Matrix ---------------------------

        3.96p128 3.90.3  
3.96V5   0.061    0.017*  
3.96p128          0.547    
-----------------------------------------------------------------------

3.96V5 is better than 3.90.3


Averaging in proxima's current results, it looks like 3.96V5 will be the winner out of these 3 competitors.

Quote
and is there some quality loss betwen 3.90.3 and 3.96 which could be audibly in normal music ? this sounds all so complicated.


For the abr mode, it looks like you win some and lose some, and it depends on your ears.  If you've got older ears like mine, you might prefer 3.96 abr at 128 kbs.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-16 20:18:10
BTW, have there been bitrate tests of 3.96 -V5?

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-16 20:54:26
Quote
BTW, have there been bitrate tests of 3.96 -V5?

ff123

Only with beta 1 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19876&view=findpost&p=198310) AFAIK.

Edit: for -V 5 3.96 beta 1 and final give bit-identical outputs. (Forget 3.90, was a typo.) So no further testing is necessary. (Well, doing some more bitrate tests can't do any harm of course  ).
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Jack Comics on 2004-04-16 21:24:23
Quote
Only with beta 1 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19876&view=findpost&p=198310) AFAIK.

So @ anyone who wants to help: Could you do some encodingings for 3.90 beta1 AND final with -V 5 and post the results here, please? Thanks in advance.

I'd imagine finding 3.90 beta 1 would be pretty difficult.  Why would you want to test it to begin with though?
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-16 22:17:32
Jack Comics - you're right. I've already found out and reported earlier in this thread:
Quote
VBR (-V 5)
final == beta2 == beta1 (bit-identical)

Hehe - thanks. I'll edit my previous post.

So ff123, -V 5 was tested and the results are "close enough" to 128kbps.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: detokaal on 2004-04-16 22:19:46
16 files 779,114 kb wav

Total Time is 75:22

Lame 3.96 -V 5 compresses to total 72,983 kb

Average bitrate from viewing the kbs for these 16 tracks in Winamp is 131.25

Lowest was 117 and highest was 144.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: [proxima] on 2004-04-17 00:51:45
Quote
Could you please provide links to the samples used (if I haven't found them already)?

Sure. Applaud and fatboy samples from here:
http://lame.sourceforge.net/download/samples (http://lame.sourceforge.net/download/samples)
ct_reference and preecho1 uploaded in the apposite thread. You've already found the others 
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-04-17 01:18:24
RESULTS

Drone_short (http://ff123.net/samples/drone_short.flac), aps, 0-1.2 sec

3.90.3 vs. original - 8/8
Easy to ABX.

3.96 vs. original - 8/8
Equally easy.

3.96 vs 3.90.3 - 8/8
3.96 sounds better than 3.90.3 now!

3.96 vs. 3.96b1 - 8/8
Air sounds are quieter in 3.96

Edit: added link to sample and encoding command line used
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-04-17 12:35:01
In drone_short, the use of alt-preset extreme greatly improves this sample in both encoders. At first glance, it seems 3.90.3 sounds slightly better; I'll try to do some ABX testing shortly. An interesting thing to note is that 3.96 alt-preset extreme is still smaller than 3.90.3 alt-preset standard, at 204kbps and 214kbps, respectively.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: eagleray on 2004-04-17 15:18:15
Is this size thing, 3.96 -pe being smaller than 390.3 -aps a common thing?  If that is the case, perhaps we should be looking at 3.96 -pe as the new standard.

A fair comparison between the two versions of lame does not require the same preset setting, it certainly is not being looked at that way for the 128 kbit tests.

If 3.96 -pe produces files that are constantly larger than 3.90.3 -aps, why not try 3.96 -V 1 to get the bit rates equal to 3.90.3 -aps?

I am a bit concerned that a judgment will be reached that 3.90.3 beats 3.96 by a nose, when the difference turns out to be due to a higher bit rate on the average, and not just on a few difficult samples.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: [proxima] on 2004-04-17 15:27:17
RESULTS

Tested settings:
Lame 3.90.2 --alt-preset 128 --scale 1
Lame 3.96 --preset 128 --scale 1
Lame 3.96 -V 5
Decoding: Foobar2000 dithering with strong ATH noise shape and DSP Advanced limiter.

With Atrain there are a lot of problems for all encodings: muffled cymbals, disorted trumpet, and... warbling background noise. 3.90.2 is better overall while 3.96 -V5 seems to suffer much than the others of warbling background.
3.90.2 : 3.1
3.96abr: 2.0
3.96_V5: 2.4

BackS1007 is very interesting because HF issues i have noticed are evident. This sample is another proof that 3.90.2 doesn't have this problem. Moreover again,  (Atrain) VBR is the worst with such artifact: there are even impulsive HF noise (clicks at beginning). VBR doesn't perform well with low volume samples like this.
3.90.2 : 4.2
3.96abr: 3.2
3.96_V5: 2.0

BeautySlept: although easy to abx against original, i can't differentiate bewteen different encodings. Equal rating for all.
3.90.2 : 3.0
3.96abr: 3.0
3.96_V5: 3.0

Flooressence: preecho is a bad problem here. There is a bad regression with 3.96abr. Obviously VBR handle preecho better but 3.90.2 is good.
3.90.2 : 2.7
3.96abr: 1.2
3.96_V5: 2.5

ABC/HR and ABX results here: http://xoomer.virgilio.it/fofobella/396_2.zip (http://xoomer.virgilio.it/fofobella/396_2.zip)
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: detokaal on 2004-04-17 16:02:07
Quick questions about ratings before I post results:  ABX is fairly easy most of the time with any mp3 vs. wav, but when you ABX mp3 vs mp3 how do you determine your numbers?  I don't mean the actual number like 2-3-4, I mean: 

What is most important to developers when assigning a number?

Do you give lowest numbers to (pardon my non-tech terms since I don't know any) pre-echo, warbling, stereo image, tone distortion, etc?  For example - sample A with heavy concert percussion handles pre-echo poorly but the brass section sounds accurate, but B gets the attacks however the brass sound tinny - which is most important to give consideration to?  Because, what bothers me may not be of interest to the developers or it may be more important to know when I give a rating, so:

What issues should be marked down more than others when rating samples?
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: eagleray on 2004-04-17 16:07:19
Forther to on file sizes:

These are the results for just one album, Solitude Standing (Suzanne Vega)
3.96 -ps    55.3 MB
3.96 -pe    70.3 MB
3.96 -V 1  62.7 MB
390.3 -aps 59.9 MB

What I suspect is that give enough tracks, 3.96 -pe is going to be too big to be comparable to either lame version at -ps or -aps, but output file sizes for 3.96 -V 1 are closer to 3.90.3 -aps than 3.90.3 -aps is to 3.96 -ps, suggesting that 3.96 -V 1 may be a fair comparison to 3.90.3 -aps.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-17 16:10:00
Quote
Is this size thing, 3.96 -pe being smaller than 390.3 -aps a common thing?  If that is the case, perhaps we should be looking at 3.96 -pe as the new standard.

A fair comparison between the two versions of lame does not require the same preset setting, it certainly is not being looked at that way for the 128 kbit tests.

If 3.96 -pe produces files that are constantly larger than 3.90.3 -aps, why not try 3.96 -V 1 to get the bit rates equal to 3.90.3 -aps?

I am a bit concerned that a judgment will be reached that 3.90.3 beats 3.96 by a nose, when the difference turns out to be due to a higher bit rate on the average, and not just on a few difficult samples.

It seems like focus is on testing ~128kbps settings right now so it would be good timing to do some mass-encoding test to find out... IMO it should include 3.90.3 --alt-preset standard & extreme and 3.96 final -V 2, -V 1 and -V 0.

BTW: Among the encodings I've done for testing I've already found some tracks/CDs where 3.96 --preset standard has a higher bitrate then 3.90.3 --alt-preset standard (up to 15kbps difference) - mainly classical and similar stuff (slow, accoustic instruments).

I've performed a test with 3.96 beta 1 for -V 2 (standard) ... -V 6. Results:

(http://www.noveo.net/tigre/lametest/bitrate01.png)
Since the minimum bitrate of -V 2 (standard) has been increased to 128kbps, the bitrates for 3.96 should be somewhat higher.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gecko on 2004-04-17 18:53:28
Quote
What issues should be marked down more than others when rating samples?

Whichever you find more annoying. We want to hear your personal subjective opinion in this case. The test procedure is to be done blindly to eliminate bias or placebo. It's just like that blindfolded lady Justitia.

[span style='font-size:8pt;line-height:100%']lol[/span]
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-17 19:02:13
Quote
What issues should be marked down more than others when rating samples?

whichever sample sounds more annoying to you overall should be given a lower score.  That's necessarily subjective; there's no way around it.  As for what would help the developers most is detailed descriptions of what you find wrong with the samples.

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gabriel on 2004-04-18 13:26:57
Quote
3.96° --p 128 > 3.90.3 --ap 128 :: fall :: [proxima] :: 0x verified so far + missing ABX results*

---------------------------------------------------

3.96° --p 128 < 3.90.3 --ap 128 :: fall :: [proxima] :: 0x verified so far + missing ABX results*


???
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-18 13:48:36
Quote
Quote
3.96° --p 128 > 3.90.3 --ap 128 :: fall :: [proxima] :: 0x verified so far + missing ABX results*
---------------------------------------------------
3.96° --p 128 < 3.90.3 --ap 128 :: fall :: [proxima] :: 0x verified so far + missing ABX results*

???

Thanks. Fixed (there was another mistake with [proxima]'s ct_reference sample as well).
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-04-19 02:59:43
RESULTS

Drone_short (http://ff123.net/samples/drone_short.flac), 0-1.2 sec

3.96pe vs. Original - 8/8
Sounds pretty good

3.90.3ape vs. Original - 8/8
Slightly more "rushing air" artifacts

3.96pe vs. 3.90.3ape - 8/8
3.96 wins.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-04-19 03:33:36
Though I think 3.96 beat 3.90.3 overall in both ps and pe, 3.96 at both settings suffers from a slight blip artifact at about .4-.5 seconds into the song (I'll admit calling drone_short a song is a bit of a stretch). In ps, this blip is fairly pronounced and  appears in the center (in stereo terms). In pe, this blip is more muted and seems to exist in the far left channel. 3.90.3 and of course the original do not contain this extra  blip. It sort of sounds like a soft frog sound or fart or something.

EDIT: Here's something weird. At -V 1, this artifact is more muted than even pe. Although I think it may still exist, I cannot localize it (in stereo terms) like in ps and pe. If it exists it blends in much more.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-04-19 05:18:28
RESULTS

Verifying fatboy with alt-preset standard.

3.90.3 vs. original - 8/8
Harshness of background static more pronounced--sound like additional noise is introduced.

3.96 vs. original - unable to ABX

3.96 vs. 3.90.3 - 8/8
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-19 08:45:13
I've repeated the bitrate test with 3.96 final and -V 2 - -V 0. Results in the attatched image. (Note that there's been a mistake in the last table I posted with the "Enjoy Baroque" sample (too low bitrate) which is corrected in the new table - the effect on the result is small enough not to worry about IMO.)

The difference for -V 2 (=preset standard) between 3.96b1 and 3.96 final is very small, in the 0.1% range.

My result suggests that 3.96 -V 2 comes closest to 3.90.3 --alt-preset standard bitrate-wise, but with another set of CDs tested (especially focussing on styles where 3.90.3 bitrate tends to be bloated), -V 1 might come closer. I've tried to cover a as broad range of styles as possible, but maybe the focus is too much on music with accoustic instruments...
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: LoFiYo on 2004-04-20 04:56:55
I'm not sure if this will be qualified as valid, because it's not aps against ps, but I will post it anyway.

Quote
Testname: MyFunnyValentine - 3.90.3aps vs 3.96 pfs

1L = C:\My Music\lab\MYF-4SEC\aps3903.mp3.wav
2R = C:\My Music\lab\MYF-4SEC\pfs396.mp3.wav

---------------------------------------
General Comments:

---------------------------------------
1L File: C:\My Music\lab\MYF-4SEC\aps3903.mp3.wav
1L Rating: 3.5
1L Comment: 1st blow is more distorted.
---------------------------------------
2R File: C:\My Music\lab\MYF-4SEC\pfs396.mp3.wav
2R Rating: 3.8
2R Comment: 1st blow is a little cleaner. A little harder to ABX.
---------------------------------------
ABX Results:
Original vs C:\My Music\lab\MYF-4SEC\aps3903.mp3.wav
    13 out of 14, pval < 0.001
Original vs C:\My Music\lab\MYF-4SEC\pfs396.mp3.wav
    13 out of 14, pval < 0.001
C:\My Music\lab\MYF-4SEC\aps3903.mp3.wav vs C:\My Music\lab\MYF-4SEC\pfs396.mp3.wav
    12 out of 16, pval = 0.038


Please note that I compared 3.90.3 APS vs 3.96 final preset FAST standard. The reason is that for this sample (MYF_4SEC.wav), 3.96 final produced much better results in fast standard preset than standard preset.  And it turned out that it's either a close tie or it beat 3.90.3 slightly. I felt that 3.96pfs was harder to ABX than 3.90.3aps. The bitrate was both 138kbps.

I am starting to wonder if 3.96PFS is better tuned for some samples than 3.96PS, and if we test 3.96PFS against 3.90.3APS where 3.96PS lost in the past, the results might be interesting/surprising...

PS I also tested 3.96b1 PFS vs 3.90.3APS. Although 3.90.3APS won because of 3.96b1's minimum bitrate being 96kbps, it was pretty close (3 for 3.96b1 vs 3.5 for 3.90.3 in ABC/HR).
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: 2Bdecided on 2004-04-20 13:39:16
Not very interesting, but...

fatboy_30sec.wav (uploaded the other week by me - see uploads forum).
3.90.3 --alt-preset standard
3.96 --preset standard


All 3 ABXes were 8/8 (using foobar)

The 3.96 encoding is better. There's an artefact (usual problem - the sound of paper being rubbed together) in 3.96 around 10 seconds, but there are more artefacts in more places in the 3.90.3 encoder, so 3.96 wins.

I'll say it again - if you're tuning with fatboy, use the longer sample!

Cheers,
David.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: 2Bdecided on 2004-04-20 13:45:53
More interesting...

awe32_20sec.wav (I'll upload it is anyone wants - it's an old favourite!)
3.90.3 --alt-preset standard
3.96 --preset standard


All 3 ABXes were 8/8 (using foobar)

The 3.96 encoding is much much better. The 3.90.3 encoding is covered with artefacts - funny swishes and whistles over the notes, and hissing over the drum beat. These are mainly gone in 3.96, but there is a new(? - probably hidden before!) harsh sounding artefact with the hi-hat sound in the latter part of the sample. It just sounds like extra noise.

So, 3.96 much better, but certainly not transparent with this artefact.

Cheers,
David.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: [proxima] on 2004-04-20 14:44:18
I think 3.96 with -V 5 quality is good and this is a great news for me but i find HF problems i previously described particularly annoying. Samples i've tested with VBR high frequencies problems are: Atrain, LisztBMinor, Blackwater, BachS1007. I'm trying to experiment a bit with settings trying to reduce this problem: with -V5 --athaa-sensitivity 1 all the four samples are better regarding ringing and warbling background while bitrate increase is small.

I've noticed a certain corrispondence between tigre's results and mine, i hope he or someone else can confirm my impression.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-20 15:10:46
With proxima's ratings from Apr 17, using the 12 samples again produces no significant results (-V5 was downgraded), when averaged between tigre, proxima and myself.

ff123

Edit:  mean scores are
3.90.3(2)    3.64
3.96p128    3.59
3.96V5      3.85
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: [proxima] on 2004-04-20 18:04:06
I just tested and uploaded two more samples provided by guruboolez in the past: it seems that --athaa-sensitivity 1 makes things better for ringing/high pitched noise with -V 5. Both samples should be very easy, with Bayle sample ringing artifatcs are very obvious and annoying.

Quote
ABC/HR Version 0.9b, 30 August 2002
Testname: Bayle

1L = C:\LAME_GUI\Bayle - Jeita or Waters' Murmur - Etching_V5.wav
2L = C:\LAME_GUI\Bayle - Jeita or Waters' Murmur - Etching_athaa.wav

---------------------------------------
General Comments:
Focising on 0:00 - 0:04 range.
---------------------------------------
1L File: C:\LAME_GUI\Bayle - Jeita or Waters' Murmur - Etching_V5.wav
1L Rating: 2.0
1L Comment: ringing, fluctuant HF noise
---------------------------------------
2L File: C:\LAME_GUI\Bayle - Jeita or Waters' Murmur - Etching_athaa.wav
2L Rating: 3.0
2L Comment: ringing, fluctuant HF noise is still present but reduced and less disturbing.
---------------------------------------
ABX Results:
Original vs C:\LAME_GUI\Bayle - Jeita or Waters' Murmur - Etching_V5.wav
    10 out of 10, pval < 0.001
Original vs C:\LAME_GUI\Bayle - Jeita or Waters' Murmur - Etching_athaa.wav
    10 out of 10, pval < 0.001
C:\LAME_GUI\Bayle - Jeita or Waters' Murmur - Etching_V5.wav vs C:\LAME_GUI\Bayle - Jeita or Waters' Murmur - Etching_athaa.wav
    11 out of 12, pval = 0.003

Quote
ABC/HR Version 0.9b, 30 August 2002
Testname: Avison

1L = C:\LAME_GUI\Track01 (Avison - Concerto Grosso 30 sec)_V5.wav
2R = C:\LAME_GUI\Track01 (Avison - Concerto Grosso 30 sec)_athaa.wav

---------------------------------------
General Comments:
ringing + chirping (high pitched notes)
---------------------------------------
1L File: C:\LAME_GUI\Track01 (Avison - Concerto Grosso 30 sec)_V5.wav
1L Rating: 2.5
1L Comment: ringing with crackles at ~ 2 sec. and after
chirping (high pitched notes) starting from 22 sec.
---------------------------------------
2R File: C:\LAME_GUI\Track01 (Avison - Concerto Grosso 30 sec)_athaa.wav
2R Rating: 3.5
2R Comment: ringing is still audible but reduced
no chirping at all
all HF artifacts are reduced
---------------------------------------
ABX Results:
Original vs C:\LAME_GUI\Track01 (Avison - Concerto Grosso 30 sec)_V5.wav
    10 out of 10, pval < 0.001
Original vs C:\LAME_GUI\Track01 (Avison - Concerto Grosso 30 sec)_athaa.wav
    10 out of 10, pval < 0.001
C:\LAME_GUI\Track01 (Avison - Concerto Grosso 30 sec)_V5.wav vs C:\LAME_GUI\Track01 (Avison - Concerto Grosso 30 sec)_athaa.wav
    10 out of 10, pval < 0.001

These two samples are not isolated cases, i choose them only because i can hear obvious problems with plain -V 5. According to my tastes -V5 with --athaa-sensitivity 1 is better with Atrain, LisztBMinor, Blackwater, BachS1007, Bayle, Avison samples.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-21 15:37:43
New results edited in.

[proxima], thanks for your attempt to improve -V 5. I think too that we both hear the same high frequency problems, but from some results it seems to me that either I can't hear them as good as you can or I listen to samples focused on trying to find them somewhat more than I do...

I haven't tested yet and probably I won't have enough time for testing before rjamorim's multiformat test will start, but no matter if someone can verify your results: Do you have some numbers about how much adding --athaa-sensitivity 1 increases bitrate - maybe it's so much that we would have to use -V 6 with it instead of -V 5 to get 128kbps on average... Besides this, testing for regressions is necessary, i.e choosing samples without obvious hf ringing problem to find out what happens with other problems.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: 2Bdecided on 2004-04-21 17:52:31
Another result...

vangelis1.wav
3.90.3 --alt-preset standard
3.96 --preset standard


3.90.3 vs orig: 7/8
3.96 vs orig: failed
3.96 vs 3.90.3: 8/8

3.96 is slightly better.

On 3.90.2 I previously heard a softening of the onset of the notes. I thought this was solved with 3.90.3, but never ABXed (not before, not today - didn't even try). However, listening today I found that there's a slight noise added behind the snare(?) hit at ~3 seconds on 3.90.3, and it's improved (solved?) with 3.96.

So, another good result for 3.96.

In case anyone has missed it, what I’m doing is going through 3.90.3 problem samples, seeing if 3.96 is an improvement. In the three cases so far, it is. That's cool! To balance it, we'll have to try to find some new problem samples for 3.96, but I guess that will take some time.

Cheers,
David.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: 2Bdecided on 2004-04-21 18:04:39
Maybe someone else can check guruboolez's sample Fugue (premières notes).wav with (a)ps?

I think 3.96 may be slightly worse - it's certainly not better, but I can't ABX 3.96 against 3.90.3, so they're probably equally bad (both 8/8 ABX against the original - the last note is especially noisy).

It would be cool if future lame development could solve the "harpsichord problem".

Cheers,
David.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Lyx on 2004-04-21 20:40:01
I just got the sample which triggered the gapless-playback issues yesterday. However, for some strange reason, when i encoded it with 3.90.3 and 3.96, the trackchange was just perfect. I DID have multiple problems with this album and 3.96 with gapless playback in the past, but either i did something wrong before (however, i cannot think of what that could be, since i just did the same as usual - the only difference is that with the previous tests, the mp3 was encoded via EAC, while this time, i encoded it from FLAC via foobar). Another explanation could be that the problem appears randomly and isn't reproducable. I currently dont have the time to investigate this more indeep, so i would like to apologise for blowing up unnecessary steam and causing false-alert - i'm sorry.

- Lyx
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: LoFiYo on 2004-04-21 21:52:55
Quote
--------- 3.96 --p fast standard vs. 3.90.3 --ap fast standard
3.96 --p fast standard > 3.90.3 --ap fast standard :: myf_4sec (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=195030) :: LoFiYo (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204717) :: 0x verified so far

Please change that to "3.96 --preset fast standard > 3.90.3 --alt-preset standard". Thanks 

Please read my post for detail. (http://www.hydrogenaudio.org/forums/index.php?showtopic=20715&view=findpost&p=204717)
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: [proxima] on 2004-04-21 23:30:39
Quote
but no matter if someone can verify your results: Do you have some numbers about how much adding --athaa-sensitivity 1 increases bitrate - maybe it's so much that we would have to use -V 6 with it instead of -V 5 to get 128kbps on average...

Apart the -V5 setting 128 Kbps target, i've posted my subjective impression about  HF problems. Adding --athaa-sensitivity 1, bitrate increase seems very small and some problems are noticeably reduced. Nevertheless, bitrate should not be the main problem because even plain -V4 (--preset medium) has more HF problems than -V5 --athaa-sensitivity 1 (see bayle sample)... maybe this switch could be useful for mid-low bitrate VBR tuning, not only for a setting around 128 kbps.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-04-22 01:02:07
CORRECTION:

For drone_short, 3.96 -V1 is not better overall than 3.96 pe--it just hides one particular artifact better. I did not mean to imply that it is better overall, as it suffers from louder "air rush" artifacts than pe...

Also, I have verified fatboy at --preset standard...
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Jebus on 2004-04-22 02:33:29
Quote
CORRECTION:

For drone_short, 3.96 -V1 is not better overall than 3.96 pe--it just hides one particular artifact better. I did not mean to imply that it is better overall, as it suffers from louder "air rush" artifacts than pe...

Also, I have verified fatboy at --preset standard...

uhm, --preset extreme and -V1 are exactly the same thing using 3.96. Am I missing something?

Did you ABX?
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-04-22 04:10:34
Quote
Quote
CORRECTION:

For drone_short, 3.96 -V1 is not better overall than 3.96 pe--it just hides one particular artifact better. I did not mean to imply that it is better overall, as it suffers from louder "air rush" artifacts than pe...

Also, I have verified fatboy at --preset standard...

uhm, --preset extreme and -V1 are exactly the same thing using 3.96. Am I missing something?

You are indeed.

--preset extreme is the same as -V0, --preset standard is the same as -V2.

I haven't ABXed it, but I suppose I could if somebody thinks it important.

Edit: wording
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Jebus on 2004-04-22 05:21:42
ah, forgot there was a -V0
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-22 15:27:58
[proxima]:

How does 3.96 -V5 --athaa-sensitivity 1 compare in a direct comparison with 3.90.2/3 ap 128 for the 12 samples?

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: [proxima] on 2004-04-22 22:33:29
Quote
How does 3.96 -V5 --athaa-sensitivity 1 compare in a direct comparison with 3.90.2/3 ap 128 for the 12 samples?

I never compared the two directly. Regarding -V5 vs. 3.90.2 --ap 128 tests i've done, i think that the VBR setting is very good for some samples (i.e. preecho) but quite worse in others (mainly low volume samples) because of HF problems. But the point is that i think "--athaa-sensitivity 1" could help with ringing related problems.

I have to do a direct comparison but i'm quite hopeful that the new VBR setting will perform closest or even better than --ap 128 with 3.90.2
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ff123 on 2004-04-22 23:20:55
Quote
,Apr 22 2004, 01:33 PM]
Quote
How does 3.96 -V5 --athaa-sensitivity 1 compare in a direct comparison with 3.90.2/3 ap 128 for the 12 samples?

I never compared the two directly. Regarding -V5 vs. 3.90.2 --ap 128 tests i've done, i think that the VBR setting is very good for some samples (i.e. preecho) but quite worse in others (mainly low volume samples) because of HF problems. But the point is that i think "--athaa-sensitivity 1" could help with ringing related problems.

I have to do a direct comparison but i'm quite hopeful that the new VBR setting will perform closest or even better than --ap 128 with 3.90.2

That's why I asked.  If it turns out that -V5 --athaa-sensitivity 1 beats out 3.90.2 --ap 128 in a side by side comparison over many samples, then that setting should be used in Roberto's test, assuming the bitrate is ok

ff123
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: tigre on 2004-04-23 13:33:44
Posts discussing technical details about --athaa-sensitivity 1 switch split to this thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=21012). Posts related to testing or mixed ones stay here.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gabriel on 2004-04-25 13:57:22
For those sensitive to high freq problems in preset cbr 128, there is a little test here:
http://www.hydrogenaudio.org/forums/index....howtopic=21052& (http://www.hydrogenaudio.org/forums/index.php?showtopic=21052&)
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gabriel on 2004-04-30 10:47:40
http://www.hydrogenaudio.org/forums/index....ndpost&p=207452 (http://www.hydrogenaudio.org/forums/index.php?showtopic=21201&view=findpost&p=207452)
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: cuan on 2004-05-07 17:49:09
so were there any conclusions drawn from this test.. Which one won 3.96 or good old 3.90.3. In particular with the -aps preset.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-05-11 22:43:54
Quote
so were there any conclusions drawn from this test.. Which one won 3.96 or good old 3.90.3. In particular with the -aps preset.

I don't think there were enough responses for a valid answer, either way. Personally, I would like to test some more, but I've been caught up with trying to graduate from school...

3.96 definitely improved some samples over 3.90.3, but also regressed in others. That's about all we know.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Jojo on 2004-05-12 13:09:47
Quote
3.96 definitely improved some samples over 3.90.3, but also regressed in others. That's about all we know.

and it does that at a much lower bitrate!
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Karlosak on 2004-05-12 15:00:28
Quote
Quote
3.96 definitely improved some samples over 3.90.3, but also regressed in others. That's about all we know.

and it does that at a much lower bitrate!

I have encoded a lot of albums (e.g. older Pink Floyd) where actually LAME 3.96 produces higher bitrate than LAME 3.90.3
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Jebus on 2004-05-12 15:23:47
I'm slowly reencoding my ~300CD collection right now in 3.96... once i get an acceptible sample size i'll let you people know what the average bitrate cost or savings is with the newer version.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: indybrett on 2004-05-12 15:55:04
I've encoded a few hundred CD's now with 3.96. Overall I would say the filesize is bigger on about 50% of the material, and smaller on about 50% of the material.

In other words, it's the same when doing a large collection that consists of varying types of music.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Sunhillow on 2004-05-12 16:02:00
Quote
I have encoded a lot of albums (e.g. older Pink Floyd) where actually LAME 3.96 produces higher bitrate than LAME 3.90.3

well, I noticed a bitrate increase on tracks with mainly ss frames, like Beatles or Pink Floyd.
Sometimes 3.96 --preset medium produces higher bitrates than 3.90 APS
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Matyas on 2004-05-14 12:25:50
So i won't loose anything by switching to 3.96, right? Only that the encoding will be considerable faster on my P233MMX!
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Jojo on 2004-05-14 13:03:36
Quote
So i won't loose anything by switching to 3.96, right? Only that the encoding will be considerable faster on my P233MMX!

that's right!
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: xmixahlx on 2004-05-14 17:17:04
right. faster
+better on some samples
+worse on others

(refer to "Recommended Encoder" sticky...)


later
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: LordofStars on 2004-05-15 02:02:33
The only think I have against 3.96. is that I don't have the same switch capability I had before. If you make it idiot proof your going to take away the options tweakers like. I'll quit bitching if I like to tweak so i'll stick with 3.90.3 until 4.0
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: bluddnok on 2004-05-15 03:13:31
Okay, I have a hard-drive full of .wav files just waiting to be compressed: which LAME version should I use?

I have read most of this thread, and I see that there are many arguments for both 3.90 and for 3.96, however most of the technical details have gone over my head, and I was hoping for a few generalised recommendations.

I am not too bothered with file size, and would like to compress these files at a quality which I will not decide is too low in a few years time (when I can afford a good-quality hi-fi system, for example)

I am presently using CDex to rip and RazorLAME to compress.

Thanks for any suggestions
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: LordofStars on 2004-05-15 05:44:28
Tried and true is all I have to say.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Lyx on 2004-05-15 13:34:23
Quote
The only think I have against 3.96. is that I don't have the same switch capability I had before. If you make it idiot proof your going to take away the options tweakers like. I'll quit bitching if I like to tweak so i'll stick with 3.90.3 until 4.0

actually, if i understood gabriel in another thread correctly, then 4.0 will have even more switches removed - for a good reason.

However, could be that i misunderstood him - but i think i didn't.

For further explanations on the "why" a longer commandline not necessarily improves quality, refer to the ha.org mp3 FAQ.
- Lyx
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Jojo on 2004-05-15 13:44:39
Quote
The only think I have against 3.96. is that I don't have the same switch capability I had before. If you make it idiot proof your going to take away the options tweakers like. I'll quit bitching if I like to tweak so i'll stick with 3.90.3 until 4.0

so what switches are you specifially talking about?
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Jojo on 2004-05-15 13:52:56
Quote
Okay, I have a hard-drive full of .wav files just waiting to be compressed: which LAME version should I use?

I have read most of this thread, and I see that there are many arguments for both 3.90 and for 3.96, however most of the technical details have gone over my head, and I was hoping for a few generalised recommendations.

I am not too bothered with file size, and would like to compress these files at a quality which I will not decide is too low in a few years time (when I can afford a good-quality hi-fi system, for example)

I am presently using CDex to rip and RazorLAME to compress.

Thanks for any suggestions

I'd use LAME 3.96...you probably won't hear the difference between the two anyway (even the samples that got better or worse). If you are not concerned about file size you could always go with lossless. It saves you ~30% of the original file and will sound identical (you can always restore the source).

Otherwise you could go with --preset extreme , which will result a higher average bitrate and therefore *may* sound better...on many samples it is an overkill, but if you not bothered by the higher filesize choose that setting.

Personally I use LAME 3.96 with --preset standard. Try both encoders and make some test samples and play around with either settings...and choose your own favorite
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Sebastian Mares on 2004-05-16 11:56:06
Here are the results for Preecho1:

Original vs. LAME 3.95 -V 5

Quote
ABX log
ABC/HR for Java Version 0.4b4
May 16, 2004 12:46:29 PM

Sample A: Original.wav
Sample B: 3.96 -V 5.wav
Playback Range: 00.000 to 22.982
    12:45:55 PM f 0/1 pval = 1.0
    12:45:59 PM p 1/2 pval = 0.75
    12:46:02 PM f 1/3 pval = 0.875
    12:46:05 PM p 2/4 pval = 0.687
    12:46:07 PM f 2/5 pval = 0.812
    12:46:10 PM p 3/6 pval = 0.656
    12:46:12 PM p 4/7 pval = 0.5
    12:46:15 PM p 5/8 pval = 0.363

---------
Total: 5 out of 8, p = 0.363


Original vs. LAME --alt-preset 128

Quote
ABX log
ABC/HR for Java Version 0.4b4
May 16, 2004 12:48:49 PM

Sample A: Original.wav
Sample B: 3.90.3 --alt-preset 128.wav
Playback Range: 00.000 to 22.982
    12:48:08 PM p 1/1 pval = 0.5
    12:48:10 PM p 2/2 pval = 0.25
    12:48:12 PM p 3/3 pval = 0.125
    12:48:13 PM p 4/4 pval = 0.062
    12:48:15 PM p 5/5 pval = 0.031
    12:48:17 PM p 6/6 pval = 0.015
    12:48:19 PM p 7/7 pval = 0.0070
    12:48:21 PM p 8/8 pval = 0.0030

---------
Total: 8 out of 8, p = 0.0030


LAME 3.96 -V 5 vs. LAME 3.90.3 --alt-preset 128

Quote
ABX log
ABC/HR for Java Version 0.4b4
May 16, 2004 12:54:04 PM

Sample A: 3.96 -V 5.wav
Sample B: 3.90.3 --alt-preset 128.wav
Playback Range: 00.000 to 23.014
    12:53:40 PM p 1/1 pval = 0.5
    12:53:45 PM p 2/2 pval = 0.25
    12:53:47 PM p 3/3 pval = 0.125
    12:53:49 PM p 4/4 pval = 0.062
    12:53:51 PM p 5/5 pval = 0.031
    12:53:53 PM p 6/6 pval = 0.015
    12:53:56 PM p 7/7 pval = 0.0070
    12:53:58 PM p 8/8 pval = 0.0030

---------
Total: 8 out of 8, p = 0.0030


Result: 3.96 wins!

The sound is much sharper in 3.96 compared to 3.90.3.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-05-24 20:16:13
Quote from: Jojo,May 15 2004, 12:52 PM
Quote from: bluddnok,May 14 2004, 06:13 PM

Otherwise you could go with --preset extreme , which will result a higher average bitrate and therefore *may* sound better...on many samples it is an overkill, but if you not bothered by the higher filesize choose that setting.

Personally I use LAME 3.96 with --preset standard. Try both encoders and make some test samples and play around with either settings...and choose your own favorite 

Don't forget that 3.96 also has V1 as well, which is in between preset standard and extreme. I'm thinking about using this setting... It shows improvements on certain ps problem samples.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: bluddnok on 2004-06-03 05:32:13
What is the advantage of V1? Isn't this just an untweaked VBR setting, and thus probably not as good as the other presets? And wasn't it available in previous versions? (or am I getting my switches mixed up)

Also, I have come across 3.96 downloads which appear to be different sizes. Are there different versions of 3.96 available, and if so, which version is recommended?

Thanks
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-06-03 06:50:13
Quote
What is the advantage of V1? Isn't this just an untweaked VBR setting, and thus probably not as good as the other presets? And wasn't it available in previous versions? (or am I getting my switches mixed up)

Also, I have come across 3.96 downloads which appear to be different sizes. Are there different versions of 3.96 available, and if so, which version is recommended?

In 3.96, preset standard corresponds to V2, and preset extreme corresponds to V0. V1 is in between, and should be just as "tweaked". This is a change from 3.90.3, where the alt presets use their own special tunings.

These different sized binaries of 3.96 are you have stumbled upon are probably compiled using different compilers. There probably won't be any audible difference between the different versions, but encoding speed may be affected depending on the compiler options used.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: bluddnok on 2004-06-03 08:16:31
Thanks for your reply. So  the switch --V1 will give better (eg higher quality) results than --preset standard?  Does this imply that all the --V(x) settings are as well optimised as the --preset settings? Sorry if i'm asking you to repeat yourself here, but I'm relatively new to this and want to be sure i'm getting it right.

Two other brief questions:
should i be thinking about using vbr-new, or is vbr-old still considered superior?
does the --V1 setting use joint or dual stereo?

thanks again for the advice
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: magic75 on 2004-06-03 08:36:50
- They haven't been tested as much as the presets.
- vbrold is slower but better I think.
- joint stereo always
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gabriel on 2004-06-03 09:39:10
Quote
So the switch --V1 will give better (eg higher quality) results than --preset standard?

In 3.96, yes.

Quote
Does this imply that all the --V(x) settings are as well optimised as the --preset settings?

In 3.96, yes.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-06-05 21:24:04
Quote
should i be thinking about using vbr-new, or is vbr-old still considered superior?


Not sure which 3.96 is using nowadays, but it shouldn't matter anyway. 3.96 is much faster than 3.90.3 now, so just use whatever is default for the VBR setting you're using.

Quote
- They haven't been tested as much as the presets.


The presets are V settings themselves, so this doesn't really make sense. But note that there has not been a large scale public testing of 3.96 V2 (preset standard) and above, other than what is in this thread.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ATHawk on 2004-06-12 14:23:35
Hello,
I bumped into certain "problem" that may be related with some ATH problems in Lame 3.96.
I am just a normal user and instead of doing demanding ABX tests, I use spectral analysis often.
(I know it's not the best thing, but it works for me as a good indicator, because I am sensitive in HF area).
I normally use, for quite some time, custom cut between preset extreme and insane, that is:
"lame -mj -q0 -v -V0 --nspsytune --nssafejoint --lowpass 20600"
I was happy with lame 3.90, 3.92 and 3.93 as all these versions encoded freqs upto the
lowpass freq. I thought it's the way it is meant to be.

I don't have the time to extensively test every new version, but I inclined to see new versions
as improvements. But recently there has been a few versions out, that were labeled as "not recommended",
having "serious issues" or "buggy" (3.95 on mitiok's site, 3.96 on doom9.org). For average user, who is not
interested in spending extravagant amount of time on sites like this, it is discouraging.

Recently I've made some encodings with 3.96b2 (with above settings) and I was unpleasantly surprised
with missing high freqs above 19kHz. And I was not able to find easy way to get them back. I tracked this
change of behavior back to 3.94b and It seems to be ATH-related (I tested --athonly vs. --noath).
There is probably reason for this, but I'd like to know the EXACT why (con's and pro's).
* Any suggestions? at least some link?
* Could it be related with the "3.96 aps  vs. 3.96 -V5 -athaa 1" issue? (the -athaa 1 don't work for me in this case)
* What is the best way to make the 3.96 to encode upto 20kHz? (I tried --athlower 10 with partial improvement,
but I don't know the negative side-effects of this)

THX
-ATHawk

P.S. I know that many ppl don't recognize anything above 18kHz, but my limit is around that 18kHz and
I want just some secure headroom. e.g. for better subconscious feelings or some virtual 3D decoding.
Musepack is HF-friendly too.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: dev0 on 2004-06-12 14:39:00
Quote
Hello, and welcome to Hydrogenaudio.

You are reading this because you violated forum rule number 8.
Don't worry - you probably didn't know about it, or didn't
understand the implications, and we understand that. The
Hydrogenaudio Terms Of Service are here:

http://www.hydrogenaudio.org/forums/index.php?showtopic=3974 (http://www.hydrogenaudio.org/forums/index.php?showtopic=3974)

The gist of rule #8 is that if you make a claim, you must have
proper supporting evidence for it. This rule is the very core
of Hydrogenaudio, so it is very important that you follow it.

This is a generic post, and not all what follows may be
applicable to this situation. Read through it nevertheless,
it contains essential information and will help you understand
what to do (or not to do).

Why should I bother with all of this, I just want to report
a problem? (link)

For audio quality matters, 'proper supporting evidence' is
a blind listening test result demonstrating that you can hear a
difference, together with a test sample.
Graphs, non-blind tests, subtracting two files and so on are
definetely not!

A proper blind test serves several purposes: it shows that you
are serious towards our community, it proves to yourself that
you can indeed hear a difference, it provides an indication of
the seriousness of the issue at hand, and it helps pinpointing
the problem for differnent listeners.

The easiest and most common way to do a blind test is an ABX
test. There are several free utilities to do one:

http://www.pcabx.com/ (http://www.pcabx.com/)
http://ff123.net/abchr/abchr.html (http://ff123.net/abchr/abchr.html)
http://www.kikeg.arrakis.es/winabx/winabx.zip (http://www.kikeg.arrakis.es/winabx/winabx.zip)
http://www.beryllium.net/~remco/linabx/ (http://www.beryllium.net/~remco/linabx/)

An ABX test requires you do identify an unknown (X) sample as
either the original (A) or the processed (B) sample. With some
statistics it can be figured out how likely it is that you were
actually hearing a difference instead of just guessing which was
which. Hydrogenaudio uses as a general guideline that < 5% change
of guessing is considered 'proof' that you are hearing a difference.
If you try the ABX test multiple tests, add up all attempts. You
can use http://www.ff123.net/abx/abx.html (http://www.ff123.net/abx/abx.html) to calculate the p-value
( < 5% = < 0.05), though most ABX programs have it built in.

If you managed to get a significant score, congratulations, it
seems that the problem is real.

If applicable, you'll need to upload the test clip you used so
other people can verify it and developers can tinker with it.

Uploading copyrighted music is generally illegal, but fair use laws
generally permit short clips (< 30 secs) to be used for purposes
such as this. Your ISP probably has allocated you some webspace for
a webpage. You can upload the clip to it. If you're low on space,
compress it with a lossless encoder (e.g. FLAC http://flac.sf.net) (http://flac.sf.net)),
which will approximately halve the required space. If you don't have
your own webspace, you might want to try the IRC channel, ask a
friend, or simply post on HA and ask if someone wants to help you out.

If you finally make your post to Hydrogenaudio, try to include
as much information as is relevant, and be sure to explain exactly
what and where (important but often forgotten) you hear the problem
best.

Audio is to a large extent a subjective matter, and as such,
quality matters are prone to a few problems. The first is listener
preferences. Something that applies to you may not apply to the
majority of people. Maybe the clip is an exception or problem
case and not representative of general performance. This is why
being able to verify a result is imporant, as well as giving the
developers something concrete to work with.

The second is the mind. The human mind is powerfull, but has some
weaknesses. It is very vulnerable to suggestion and subconscious
influences, even for people experienced in these tests. No matter
how how 'sure' you are  that a problem exists, verify that it's not
your mind playing tricks on you first, it'll save embarassement
later.

'Simply' reporting a problem generally doesn't tell us anything,
isn't indicative of anything, can be impossible to reproduce,
confuses people, and most importantly, wastes precious developer
time determining if the problem is real and serious or not.

You may have saved yourself 5 minutes, but you've cost other people
an hour. That's not very nice.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ATHawk on 2004-06-12 15:22:39
OK, I'll try...
Why the "Catch 22" is coming to my mind?
OK, ERRATA: *THERE IS NO PROBLEM IN LAME 3.96*

I want to encode music my way. I want to have freqs upto 20 KHz independently
of the fact, that I am able to ABX it or not. I cannot prove you, that lame 3.96 is
not encoding the freqs, because the output of spectral analysis is not a valid proof.

If you just imagine, that I want to encode ultrasonic whistle commands for my dog, then I see, I have no way to get any help here (short of faking some ABX tests with my dog .
Similarly, if I claim, that I want some headroom, it is not easy to proof with ABX tests, that a certain sample does not have that headroom. I have my reasons why not to completely trust ABX test - most importantly because the choice must be made consciously, while listening includes a non-conscious processing.

Supposing that lame 3.96 is not encoding freqs upto 20kHz, is there a way to get some educated advice on how to encode the freqs in lame 3.96 upto 20kHz the same or similar way as in 3.93? Or am I just in the wrong place?

Edit: making the question hypothetical due to a lack of proof
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: shadowking on 2004-06-12 15:59:53
You want to encode for your dog, want headroom...don't trust abx?

Too many hangups here - forget them or use lossless compression like FLAC, monkey's audio etc. That way you will get perfect output and a goodnight sleep.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: saratoga on 2004-06-12 21:35:30
Quote
OK, I'll try...
Why the "Catch 22" is coming to my mind?
OK, ERRATA: *THERE IS NO PROBLEM IN LAME 3.96*

I want to encode music my way. I want to have freqs upto 20 KHz independently
of the fact, that I am able to ABX it or not. I cannot prove you, that lame 3.96 is
not encoding the freqs, because the output of spectral analysis is not a valid proof.


Ignoreing issues of ABXing samples, how do you know 3.90 was encoding those frequencies if you didn't try and test ?  Remember a graph just shows you that theres energy at a frequency, not content.

Furthermore posting this sort of thing (x is better then y because of unsubstantiated reason z) is annoying to many people because it doesn't contribute anything but noise to discussions.  Particualarly in a thread about tuneing and testing (IE gathering objective and quantitative data).  If you still feel strongly about this issue, perhaps you could start a thread in MP3s or General Audio asking about this concern of yours.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ATHawk on 2004-06-13 04:23:16
Quote
Furthermore posting this sort of thing (x is better then y because of unsubstantiated reason z) is annoying to many people because it doesn't contribute anything but noise to discussions.

I really don't know if my English is so bad or you just don't read what I've written. I did not claim "x is better than y" and the reasons are substantiated for me, because listening is SUBJECTIVE. The reasons were not substantiated for you only because of the HA rules. I don't ignore ABX tests
Quote
why not to completely trust ABX test
, but I can't rely on them solely if I'm aiming for headroom. In my opinion, there is a relation between audio-quality and spectrum. I understand your emphasis on ABX tests in Lame development, but I didn't know, that simply mentioning spectrum analysis will upset you to the extent, that you are unable to understand or answer my questions.

I asked direct concrete questions but I got only evasive answers NOT answering my questions. (to shadowking: What is the whole HA for, if you just recommend lossless?)
Now I worked hard to get some ABX-tests, that could support my questions.
These are my 1st, so plz be patient If you think that there are errors.

The sample I used is rock song Painted On, from "Mother May I", album Splitsville 1995. I chose it, because I've done a lot of codec and listening tests with this before. Since the stress is on the high freqs, I used equalizer (http://spedia.webzdarma.cz/ATHawk/20kHz.feq) (fb2k) to accentuate the freqs around 20 kHz (in both the wav and mp3s). One of the reasons for equalizer was that my hifi equipment is not of high quality (Hama SL73 headphones, CMI8738 4.1 PCI soundcard).

I run 2 series of tests, used ABX function in Foobar2000
Sample 1 (http://spedia.webzdarma.cz/ATHawk/03-Painted_On-sample1-wav.ape): processed with equalizer AFTER mp3 encoding (linked is the original wav)
Sample 2 (http://spedia.webzdarma.cz/ATHawk/03-Painted_On-sample2-wav.ape): processed with equalizer BEFORE mp3 encoding

I was able to ABX lame 3.96 from original wav on sample1 (with eq), but I had to concentrate on high hat percussion impact in 1.3s. In mp3 it had slightly lower tone than original. ABX score 8/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s1-396.txt)
I was able to ABX lame 3.96 from 3.93, but the difference was much less noticable, so with maximum concentration I achieved 16/23 (4.7%) (http://spedia.webzdarma.cz/ATHawk/abx-log-s1-393vs396.txt)
(Didn't have time for 3.93 vs. original)

The sample 2 proved to be much more problematic. Both lame 3.93 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-393.txt) and 3.96 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-396.txt) had serious problems, both easy ABX 8/8 from original, but I include the ABX logs anyway. 
ABX 3.93 vs 3.96 was easy (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-393vs396.txt) too, but 3.93 sounded better to my ears, because the artifacts were not so continuous as in 3.96. (still both sounded pretty awful).
While this sample may be considered somewhat artificial, it does not sound too unreal to me and with Musepack I achieved much better (acceptable) results with comparable bitrate.

All encodings were done with these switches:
"-mj -q0 -v -V0 --nspsytune --nssafejoint --lowpass 20600"

The final question is same as in my previous post. Hopefully someone here can say st. reasonable without such strong feelings.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: phong on 2004-06-13 04:46:41
Have you tried using the presets?  They should offer the best performance.  It is also likely that adjusting the lowpass is preventing optimal encoding of audible frequencies.

Lossy audio encoders are entirely designed around preserving sounds that are audible to the human ear and throwing away other sounds to save space.  What you are trying to do is analogous to using a screwdriver to pound in a nail when what you really want is a hammer.  Adjusting the lowpass as you suggest is unlikely to really get what you want.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ATHawk on 2004-06-13 19:03:02
Nice try
AFAIK the presets are now just shortcuts to specific combination of switches. According to lame, Insane uses the lowpass at 20500kHz so there is hardly any difference. Extreme has lowpass at 19500kHz - no big difference either. But my point was, that lame ignores the high lowpasses anyway since 3.94b, at least in vbr.  I believe, that the main reason for at least 20kHz lowpass is to prevent aliasing. But I don't know of any reason why to go under 20kHz if you want maximum (but lossy) quality.
Moreover both pe and pi uses qval=3, if that's of any significance, than it should use faster/lower quality algorithms than qval=0 (according to lame --longhelp).

And definitely, if I can ABX lame 3.96 vs. original, there is certainly some deficiency in
Quote
preserving sounds that are audible to the human ear
.

Lame 3.96 -p. extreme (216k) vs. wav (sample 2 (http://spedia.webzdarma.cz/ATHawk/03-Painted_On-sample2-wav.ape))
ABX 8/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-396-pe.txt) (the same distortion as in my previous test, but now I'm realizing, that the main distortion is probably present in lower freqs than 19 kHz)

Lame 3.96 -p. insane (319k) vs. wav (the same sample 2)
much better job in HF on this sample. (probably due to cbr?)
ABX 3/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-396-pi.txt)

Lame 3.90.3  -p. extreme (236k) vs. wav
ABX 8/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-3903-pe.txt) needed some concentration

While this seems to be more issue of vbr vs. cbr, i think it's just one of the factors that has influence on the encoding of HF.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: PieterS on 2004-06-13 22:12:04
I have troubles with this sample (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=217884). It has artefacts like clicks or something. Version 3.90.2 produced this artefacts also and it was solved with the -Z option or 3.90.3. But 3.94, 3.95.1 and also 3.96 produces the artefacts again, however at a lower level.

The artefacts can be heard at 0:09 and 0:24 sec.

PieterS
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gabriel on 2004-06-14 08:29:08
Lame only encodes high frequencies if they are higher than the ATH level. So if they have enough energy, they will be preserved.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: magic75 on 2004-06-14 08:31:45
Quote
And definitely, if I can ABX lame 3.96 vs. original, there is certainly some deficiency in
Quote
preserving sounds that are audible to the human ear
.

The problem is that you are using an equalizer to accentuate high frequencies. Do you normally use this equalizer setting when you are listening to music? If not it is not really a fair test. As it has been said lossy codecs must throw away some information. By accentuating these high frequencies you are accentuating sounds that are normally masked by much louder, lower frequency sounds. If this is the equalizer you use normally then forget about this comment... If not, try to to see if you can ABX it under normal conditions as well.

EDIT: Ah, equalizer is only used after encoding on sample 1? OK, then sample 2 is interesting, at least theoretically.

The tricky part is to throw away information that is not audible. There are several ways of doing this, cutting high frequencies is only one of them. Even trickier is finding the right combination of all these methods (of throwing information away) to give you the best possible perceived quality at a certain bitrate. For this reason the presets were invented and tested very thoroughly. During this heavy testing it was clear that slightly lowering the low pass frequency to give more bits to encode lower frequencies was very benificial for perceived quality. Not very unexpteced as it is known that the human ear is most sensitive to lower (but not to low..) frequencies, and since high frequencies cost a lot of bits to encode. It is vital to understand that increasing the low pass frequency while not increasing bitrate, will always lead to decreased quality, since less bits are available for encoding contenty in the important areas (mid-frequencies). So if you want the best possible quality (regardless of bitrate) --preset insane would be the best choice.

What I am trying to say is that by increasing the headroom, by increasing the lowpass you are actually decreasing the headroom for lower frequencies. It is however a very good idea to have some extra headroom (more generally speaking), since a setting that is sufficient for one tune, may not be sufficient for another. Why not try ABX:ing some of the presets and then choose one preset higher than you were able to ABX?

Finally, however you have a point that with 3.95 and later lame overrides a lot of settings since the presets have been mapped to -b and -V. The reason for this is for newbies who only use -b and -V to get the best possible quality. You may be able to get around this by switching the order of the switches in your commandline. But as I have said above, I am not recommending it...
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: 2Bdecided on 2004-06-14 09:38:40
Quote
P.S. I know that many ppl don't recognize anything above 18kHz, but my limit is around that 18kHz and
I want just some secure headroom.

That's a totally meaningless statement.

Psychoacoustic codecs try to preserve something that sounds the same, not something that is the same.

You're thinking "OK, so I can hear up to ~18kHz, the codec seems to keep things up to ~18kHz, but I'll try and make it keep thing up to ~20kHz, so I've got a little headroom".

Quote
I want just some secure headroom. e.g. for better subconscious feelings or some virtual 3D decoding.
Musepack is HF-friendly too.


1. Musepack won't keep inaudible high frequency content unless you force it to.
2. Your high frequency hearing won't improve as you get older (though your equipment might) - if you can't hear above 18kHz on full range equipment now, you won't be able to in the future
3. Ultrasonic signals are irrelevant to virtual 3-D decoding
4. The idea that inaudible signals effect subconscious feelings is almost certainly rubbish (at least in terms of psychoacoustic based codecs - when people say "there's no audible difference, but it feels different man!", people here usually listen to the sample and find a huge ABXable audible difference!), but if you believe there's any truth in it, then "use lossless" is the only answer! Really!
5. If you really want "headroom", think about what mp3 encoding is doing in the audible range
6. Remember that mp3 and lame are not perfect. Pushing for a little "headroom" could make the imperfections go away, or it could add more - e.g. forcing it to use bits in places where it doesn't need to at the expense of areas where it does
7. If you're going to use graphs and ignore your ears, you've landed at the wrong forum!


btw, if you're going to try wild 3-D surround decoding, joint stereo is probably the last thing you should be using. However, if you're aiming for transparent encoding of the greatest number of samples for normal stereo listening (no graphs involved) then the most tested settings are the ones to go with.

There are some good graphs vs ears threads in the FAQ, which explain very carefully how you can have two signals that look identical on a spectral analysis, whilst sounding quite different, yet have two signals that look very different on a spectral analysis, whilst sounding identical. You have to under stand the tools you're using, and human ears.

Finally, it's quite funny if you've been using that command line since 3.90.3, because (I suspect) it'll be audibly worse that --alt-preset standard for certain types of sounds. Without the internal hacks of --alt-preset standard, the nspsy model has some weaknesses in 3.90.3. Forcing the bitrate ever higher may hide these for all but the most extreme samples, but then you might as well use 320kbps CBR - that'll give you all the headroom that mp3 has to offer!

Hope this helps.

Cheers,
David.

(btw, you did spot the genuine answer to your question from Gabriel, didn't you?)
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Martel on 2004-06-14 09:49:05
>ATHawk

I don't think that shifting the lowpass filter up to 21 kHz will improve quality (that means overall quality, not just high frequencies) with LAME, quite the opposite.

I went into some tests with LAME and black/doom metal songs and found out that it was actually better to lower the transition frequency to about 18.5 kHz at 192 kbits (default is, i think, 19.3 kHz at this bitrate).
If the lowpass frequency was too high, a hat (and other hi-freq percussions) could easilly eat up a reasonable portion of the bitstream, leaving snare, toms and guitars choking... Which was a disappointment.

You simply can't have everything, LAME ain't Musepack. 

MP3 (LAME) is nearly perfect for portables (thanks to great MP3 support), but i am rather skeptic about using LAME (even at high bitrates) for archival purposes...
If you want an archive copy of a CD, go Musepack/braindead or go lossless.

>added: Well, i see i ain't very fast writer, someone wrote the same while i was writing this... 
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Lyx on 2004-06-14 09:57:33
what does this have to do with testing 3.90.3 vs. 3.96 ? Thread-Split?

- Lyx
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Martel on 2004-06-14 10:00:51
Actually, i think there IS a problem with 3.96.

See:
http://www.hydrogenaudio.org/forums/index....ndpost&p=215791 (http://www.hydrogenaudio.org/forums/index.php?showtopic=22105&view=findpost&p=215791)
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: phong on 2004-06-14 16:23:34
Quote
Lame 3.96 -p. extreme (216k) vs. wav (sample 2 (http://spedia.webzdarma.cz/ATHawk/03-Painted_On-sample2-wav.ape))
ABX 8/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-396-pe.txt) (the same distortion as in my previous test, but now I'm realizing, that the main distortion is probably present in lower freqs than 19 kHz)

...

Lame 3.90.3  -p. extreme (236k) vs. wav
ABX 8/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-3903-pe.txt) needed some concentration

Can you ABX 3.96 from 3.90.3 at the extreme preset?  If so, it could be quite interesting because it would be a specific sample where 3.96 degrades performance over 3.90.3.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: freakngoat on 2004-06-17 03:49:00
Edit: Sorry, this post was a mistake... was doing some tests with the different versions, got them mixed up, and thought I found something profound...
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ATHawk on 2004-06-17 19:21:48
THX to Gabriel, magic75 and 2Bdecided for some factual answers.

I have many ideas, but for now I'd like to focus on the important
byproduct of my testing i.e. Sample 2 (http://spedia.webzdarma.cz/ATHawk/03-Painted_On-sample2-wav.ape).

It is processed before encoding, so I consider it fair to ABX even though it is a "bad scenario" case.
I've already posted ABX tests, that only support my impression, that there is irritating distortion in high freqs (guessing somewhere between 14-19kHz) in Lame 3.96, Lame 3.93 and Lame 3.90.3. (everything -preset extreme)
I made the ABX 3.96 -pe (236kbps) vs. 3.90.3 -pe (216kbps) for phong and for topic (but to me 3.93 sounds the same as 3.90.3). It is 8/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-3903vs396.txt) with some concentration (relatively easy). 3.90.3 (and 3.93) sounds better to me only because the distortion is not so continuous as in 3.96.

The main deficiency will be probably in the VBR (old) code, because this sample is transparent with -p. insane and even "-b 256" sounds better then -p. extreme, but is ABXable with concentration (7/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-396-pi-vs256cbr.txt)). The perceived difference in
"256k cbr" is not in the distortion but in the absence of some of the HF, so it sounds better then the VBR.
My guess: It seems that the VBR code is misguided by the presence of HF and certainly reluctant to allocate more bits, when the 320k bitrate limit is far from reach.
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: Gabriel on 2004-06-18 08:52:23
Quote
"256k cbr" is not in the distortion but in the absence of some of the HF, so it sounds better then the VBR.

By curiosity, could you try "--preset extreme -Y"
Title: LAME 3.96 FINAL vs. 3.90.3 Test
Post by: ATHawk on 2004-06-21 02:24:09
Greetings Gabriel!
I tried the "--preset extreme -Y" but It doesn't seem to change anything listen-wise vs. plain -pe.

Lame 3.96 -pe -Y vs. wav 8/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-396-pe-Y.txt) ...the same old distortion

Lame 3.96 -pe -Y vs. -pe 5/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-396-pe-vs-Y.txt) i.e. sounds the same

I tried a lot of switches, but no one improved the quality within the restraints of the old-vbr. BTW I found out, that the "--substep n" is no more recognized despite its presence in "--longhelp".
Moreover, the "-X n" switch behaves strangely, anything else then the default "-X 3" sounds the same - worse.

Lame 3.96 -pe vs. -pe -X 1 7/8 (http://spedia.webzdarma.cz/ATHawk/abx-log-s2-396-pe-vs-X1.txt)

Hope this will help with the curiosity