Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: LAME 3.96 vs. 3.90.3 Test (Read 127083 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

LAME 3.96 vs. 3.90.3 Test

Reply #50 –
Sample : Its me ~ 128kbps tested

Live recording, focussed on problems with applause, details see ABC/HR log
3.96b2 -V 5 won (3.0)
3.90.3 ap 128 was second (2.0)
3.96b2 p 128 was worst (1.5)

ABC/HR log (I tested vorbis codecs with HF reduction as well, to save space here I cut out the related parts in ABC/HR log.)
Quote
ABC/HR Version 0.9b, 30 August 2002
Testname:

1L = N:\lametest\Soul_Live\decode\017 Its me.wav_mtb3_4.wav
2L = N:\lametest\Soul_Live\decode\011 Its me.wav_23_4.wav
3L = N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav
4R = N:\lametest\Soul_Live\decode\015 Its me.wav_cqk32_4.25.wav
5R = N:\lametest\Soul_Live\decode\010 Its me.wav.wav
6R = N:\lametest\Soul_Live\decode\016 Its me.wav_hfr_4.25.wav
7R = N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav
8R = N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav

---------------------------------------
General Comments:

---------------------------------------
<snip>
---------------------------------------
3L File: N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav
3L Rating: 2.0
3L Comment: 0.0-3.4: surround-like, warbeling/metallic sound in applause, no hf noise boost -> should be lame.
---------------------------------------
<snip>
---------------------------------------
7R File: N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav
7R Rating: 3.0
7R Comment: should be lame; small ringing on right channel, otherwise quite good
---------------------------------------
8R File: N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav
8R Rating: 1.5
8R Comment: sounds like 3 with hf ringing problems of 7 added
---------------------------------------
ABX Results:
<snip>
Original vs N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav
    7 out of 7, pval = 0.008
<snip>
Original vs N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav
    7 out of 7, pval = 0.008
Original vs N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav
    7 out of 7, pval = 0.008
<snip>
N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav vs N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav
    11 out of 12, pval = 0.003
N:\lametest\Soul_Live\decode\012 Its me.wav_3903_ap128.wav vs N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav
    11 out of 12, pval = 0.003
<snip>
N:\lametest\Soul_Live\decode\014 Its me.wav_396b2_V5.wav vs N:\lametest\Soul_Live\decode\013 Its me.wav_396b2_p128.wav
    7 out of 7, pval = 0.008
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello
 

LAME 3.96 vs. 3.90.3 Test

Reply #51 –
Sample : entierren con rumba ~ 128kbps tested

Salsa with lots of percussion, details see ABC/HR log, results similar to the previous sample I tested
3.96b2 -V 5 won
3.90.3 ap 128 was second
3.96b2 p 128 was worst

ABC/HR log:
Quote
ABC/HR Version 0.9b, 30 August 2002
Testname: entierren con rumba lame 128

1L = N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav
2L = N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav
3L = N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav

---------------------------------------
General Comments:
Guiros (percussion) smeared, clearly noticable (and ABXed) at 2.8-6.0
1 is best, 2 and 3 are similar but 2 has ringing/metallic sound additionally.
After 6.0 it's similar with the additonal precussion instrument (shaker?).
---------------------------------------
1L File: N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav
1L Rating: 3.5
1L Comment:
---------------------------------------
2L File: N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav
2L Rating: 2.0
2L Comment:
---------------------------------------
3L File: N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav
3L Rating: 2.5
3L Comment:
---------------------------------------
ABX Results:
Original vs N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav
    7 out of 7, pval = 0.008
Original vs N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav
    7 out of 7, pval = 0.008
Original vs N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav
    7 out of 7, pval = 0.008
N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav vs N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav
    7 out of 7, pval = 0.008
N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_V5_.wav vs N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav
    7 out of 7, pval = 0.008
N:\lametest\Salsa\decode\entierren con rumba.wav_396b2_p128_.wav vs N:\lametest\Salsa\decode\entierren con rumba.wav_3903_ap128_.wav
    7 out of 7, pval = 0.008



____________________________________________________________________

[span style='font-size:12pt;line-height:100%']
BTW: IT WOULD BE GREAT IF SOME MORE PEOPLE COULD PROVIDE ~128KBPS RESULTS BEFORE RJAMORIM'S 128KBPS TEST STARTS.[/span]
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

LAME 3.96 vs. 3.90.3 Test

Reply #52 –
Lame v3.96 final seems to have a serious problem w/ 41_30sec in the 6-8 second range using --preset cbr 128. It sounds like a dropout / skip in the sound. This problem has been seen in Mitok's compile as well as ICL v8.0 and MSVC compiles made by myself. There is no skip using v3.90.3 . Could someone confirm this??

EDIT - Look at the waveforms - the volumes between the wav/3.90.3 and 3.96 are drastically different.

EDIT2 - Lame v3.96b2 does not suffer from this problem either.

Origional WAV:



Lame v3.90.3 --preset cbr 128:



Lame v3.96 --preset cbr 128:

LAME 3.96 vs. 3.90.3 Test

Reply #53 –
Quote
Lame v3.96 final seems to have a serious problem w/ 41_30sec in the 6-8 second range using --preset cbr 128. It sounds like a dropout / skip in the sound. This problem has been seen in Mitok's compile as well as ICL v8.0 and MSVC compiles made by myself. There is no skip using v3.90.3 . Could someone confirm this??

I can confirm it. I tested the 4.5-8.5 second range of 41_30sec, using Mitiok's 3.96 and john33's 3.90.3 and 3.96b2. The 3.96 final version definitely has some nasty dropouts not present the other encodes.

Code: [Select]
ABC/HR Version 0.9b, 30 August 2002
Testname: 

1L = G:\Filetemp\diskwriter\41_30\41-396.wav
2L = G:\Filetemp\diskwriter\41_30\41-3903.wav
3R = G:\Filetemp\diskwriter\41_30\41-396b2.wav

---------------------------------------
General Comments:

---------------------------------------
1L File: G:\Filetemp\diskwriter\41_30\41-396.wav
1L Rating: 1.0
1L Comment: Pops or dropouts
---------------------------------------
2L File: G:\Filetemp\diskwriter\41_30\41-3903.wav
2L Rating: 2.5
2L Comment: 
---------------------------------------
3R File: G:\Filetemp\diskwriter\41_30\41-396b2.wav
3R Rating: 2.2
3R Comment: A bit more swirling than sample 2
---------------------------------------
ABX Results:
G:\Filetemp\diskwriter\41_30\41-396.wav vs G:\Filetemp\diskwriter\41_30\41-3903.wav
    8 out of 8, pval = 0.004
G:\Filetemp\diskwriter\41_30\41-396.wav vs G:\Filetemp\diskwriter\41_30\41-396b2.wav
    8 out of 8, pval = 0.004
G:\Filetemp\diskwriter\41_30\41-3903.wav vs G:\Filetemp\diskwriter\41_30\41-396b2.wav
    11 out of 12, pval = 0.003

LAME 3.96 vs. 3.90.3 Test

Reply #54 –
 Yes , i also can confirm it.

I test it too with 41_30sec file and at the 4.5 - 6 is definitely a Pop with Lame 3.96 Final.

No Problems with Lame 3.90.3 and 3.96b2.

LAME 3.96 vs. 3.90.3 Test

Reply #55 –
I have created a gif animation that shows how the volume levels seem to change / drop out in the v3.96 final encoder when using 128kbps and to a lesser extent 160kbps - it can be found  here. Take a careful look at v3.90.3 vs. v3.96. More testing on this issue and bitrates compromised by it need to be performed.

This is just a random section (1.6-1.8sec) of the 41_30 clip, and this section is ABX-able (sounds like a click / popping).

LAME 3.96 vs. 3.90.3 Test

Reply #56 –
Quote
I have created a gif animation that shows how the volume levels seem to change / drop out in the v3.96 final encoder when using 128kbps and to a lesser extent 160kbps - it can be found  here. Take a careful look at v3.90.3 vs. v3.96. More testing on this issue and bitrates compromised by it need to be performed.

This is just a random section (1.6-1.8sec) of the 41_30 clip, and this section is ABX-able (sounds like a click / popping).

Looks like a bug.  Does this occur with abr as well (p 128)?

That could be a reason to eliminate 3.96final from consideration in Roberto's upcoming multiformat test.

ff123

LAME 3.96 vs. 3.90.3 Test

Reply #57 –
From what I have seen and heard it does not seem to occur in ABR or VBR modes, although the encoder could just be throwing bits to where they are needed. The problems shown in 41_30sec seem to disappear using --preset 128, so IMO ABR seems fine.

LAME 3.96 vs. 3.90.3 Test

Reply #58 –
OMG! At www.cdfreaks.com have published this new Lame version! Someone should told them to remove the news because of this buggy version.

LAME 3.96 vs. 3.90.3 Test

Reply #59 –
thank you for the warning.....

i will try to contact an operator at cdfreaks

LAME 3.96 vs. 3.90.3 Test

Reply #60 –
Cdfreaks has a disclaimer at the bottom of the article warning users of the bug

LAME 3.96 vs. 3.90.3 Test

Reply #61 –
I would like people testing a little more than 1 sample before calling a version "buggy"

edit: additionnaly it could help if you could find other clearly problematic samples where you hear a similar behaviour (problem in cbr but not in abr)

LAME 3.96 vs. 3.90.3 Test

Reply #62 –
I agree with Gabriel, who is to say that Lame 3.96 may break one sample, but fix 25 other samples? Unless it is tested more, you can't judge

LAME 3.96 vs. 3.90.3 Test

Reply #63 –
Quote
I would like people testing a little more than 1 sample before calling a version "buggy"

edit: additionnaly it could help if you could find other clearly problematic samples where you hear a similar behaviour (problem in cbr but not in abr)

But, on the other hand, it would be good to know, why this sample is problematic with 3.96 and not with 3.96b2. Do you have any guess?

LAME 3.96 vs. 3.90.3 Test

Reply #64 –
Tested "Waiting" sample at --preset cbr 128 using Lame v3.90.3, 3.96b2 and v3.96 final. Tested in the 0-2.2 second region of the song. While there are no clicks, there seems to be a pretty major quality regression between v3.96b2 and v3.96 final, while 3.90.3 still comes out as the winner.

Quote
ABC/HR Version 0.9b, 30 August 2002
Testname: Waiting --preset cbr 128

1L = Z:\Music\Temp\3.96 Final.wav
2R = Z:\Music\Temp\3.90.3.wav
3L = Z:\Music\Temp\v3.96b2.wav

---------------------------------------
General Comments:

---------------------------------------
1L File: Z:\Music\Temp\3.96 Final.wav
1L Rating: 3.7
1L Comment: Horrible warble / distortion in the 0-2 second region
---------------------------------------
2R File: Z:\Music\Temp\3.90.3.wav
2R Rating: 4.5
2R Comment: Best of the 3, very clean, no major distortions.
---------------------------------------
3L File: Z:\Music\Temp\v3.96b2.wav
3L Rating: 4.3
3L Comment: Slightly worse than 2, much better than 1, slight warble on the beginning pronunciation of "been."
---------------------------------------
ABX Results:
Original vs Z:\Music\Temp\3.96 Final.wav
    8 out of 8, pval = 0.004
Original vs Z:\Music\Temp\3.90.3.wav
    8 out of 8, pval = 0.004
Original vs Z:\Music\Temp\v3.96b2.wav
    8 out of 8, pval = 0.004
Z:\Music\Temp\3.96 Final.wav vs Z:\Music\Temp\3.90.3.wav
    13 out of 14, pval < 0.001
Z:\Music\Temp\3.96 Final.wav vs Z:\Music\Temp\v3.96b2.wav
    16 out of 17, pval < 0.001
Z:\Music\Temp\3.90.3.wav vs Z:\Music\Temp\v3.96b2.wav
    18 out of 20, pval < 0.001

LAME 3.96 vs. 3.90.3 Test

Reply #65 –
Quote
Tested "Waiting" sample at --preset cbr 128 using Lame v3.90.3, 3.96b2 and v3.96 final.

What about abr?

LAME 3.96 vs. 3.90.3 Test

Reply #66 –
Quote
Quote
Tested "Waiting" sample at --preset cbr 128 using Lame v3.90.3, 3.96b2 and v3.96 final.

What about abr?

Your wish is my command  - "--preset 128" using v3.90.3, 3.96b2 and 3.96 final.

Quote
ABC/HR Version 0.9b, 30 August 2002
Testname: Waiting --preset 128

1R = Z:\Music\Temp\3.96 ABR.wav
2L = Z:\Music\Temp\3.90.3 ABR.wav
3L = Z:\Music\Temp\3.96b2 ABR.wav

---------------------------------------
General Comments:
All samples did better than in CBR, and no codec sounded horrible.
---------------------------------------
1R File: Z:\Music\Temp\3.96 ABR.wav
1R Rating: 4.1
1R Comment: Felt this had slightly more warble than 3 and is clearly worse than 2, although  I am not 100% sure that I can ABX it against 3.
---------------------------------------
2L File: Z:\Music\Temp\3.90.3 ABR.wav
2L Rating: 4.6
2L Comment: Better than 1 or 3, sounded cleaner
---------------------------------------
3L File: Z:\Music\Temp\3.96b2 ABR.wav
3L Rating: 4.2
3L Comment: Slight warble, thought it was slightly better than 1 but could not ABX it with 100% confidence - got a 15 of  17, p = 0.001 but with later trials could not ABX - might just be listening fatigue.
---------------------------------------
ABX Results:
Original vs Z:\Music\Temp\3.96 ABR.wav
    8 out of 8, pval = 0.004
Original vs Z:\Music\Temp\3.90.3 ABR.wav
    8 out of 8, pval = 0.004
Original vs Z:\Music\Temp\3.96b2 ABR.wav
    13 out of 14, pval < 0.001
Z:\Music\Temp\3.96 ABR.wav vs Z:\Music\Temp\3.90.3 ABR.wav
    13 out of 14, pval < 0.001
Z:\Music\Temp\3.96 ABR.wav vs Z:\Music\Temp\3.96b2 ABR.wav
    28 out of 42, pval = 0.022
Z:\Music\Temp\3.90.3 ABR.wav vs Z:\Music\Temp\3.96b2 ABR.wav
    10 out of 10, pval < 0.001

LAME 3.96 vs. 3.90.3 Test

Reply #67 –
At this point in time, I would sum up the whole situation by saying:
  • I don't know if 41_30sec is an isolated case or not - I agree that more samples are needed before any judgment can be made.
  • I have only seen the problem in --preset cbr 128 - other bitrates/modes don’t exhibit the problem in the same way or only to a degree that cannot be easily heard.
  • I would not go so far as to call v3.96 buggy, but more testing needs to be done on the final version.
  • v3.90.3 still seems to have higher quality in most samples at most bitrates (with the exception of fatboy and a few others)
  • MORE TESTING OF v3.96 IS NEEDED
If other people could do some tests to prove or disprove what I have found, it would be a large help.

LAME 3.96 vs. 3.90.3 Test

Reply #68 –
Quote
I don't know if 41_30sec is an isolated case or not - I agree that more samples are needed before any judgment can be made.


I can confirm that there is a failure on this sample on frame 272, 2nd granule (7.11s) when encoding in cbr 128. The output on mp3X is very strange.

LAME 3.96 vs. 3.90.3 Test

Reply #69 –
OK. I've started a LAME 3.96 FINAL vs. 3.90.3 Test, Test/Result Thread.

I'm a bit short of time right now - I'll add missing results to the related posts ASAP if noone's faster.


Gabriel:
Quote
I can confirm that there is a failure on this sample on frame 272, 2nd granule (7.11s) when encoding in cbr 128. The output on mp3X is very strange.

Is this something that is going to be fixed soon (so we can expect 3.96.1 or similar in a couple of days and should better wait with testing 3.96 final) or is it just the price that has to be paid for better performance elsewhere?

-----------------------------------------------------------------
DJED's post split to a separate thread.
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

LAME 3.96 vs. 3.90.3 Test

Reply #70 –
Quote
Is this something that is going to be fixed soon (so we can expect 3.96.1 or similar in a couple of days and should better wait with testing 3.96 final) or is it just the price that has to be paid for better performance elsewhere?


Well, I think that this specific one should be fixed, but I do not know how soon this will be done.
My opinion is that you can test 3.96.

LAME 3.96 vs. 3.90.3 Test

Reply #71 –
I've just edited all relevant 3.96beta results that were still missing in the results post.

For further testing of 3.96 final as well as for continuing related discussion go here please.

If necessary I can start a separate thread for 3.96beta -> final regression cases and move related posts there. Opinions please?
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello