HydrogenAudio

Hydrogenaudio Forum => Listening Tests => Topic started by: AstralStorm on 2004-01-12 22:52:28

Title: Lame 3.95 vs 3.90.3
Post by: AstralStorm on 2004-01-12 22:52:28
Starting with mine:

ABC/HR for Java Version 0.4b, 12 styczeñ 2004
Testname: velvet (aps)

1R = C:\Music\LAMEsamples\velvet.3-90-3.wav
2L = C:\Music\LAMEsamples\velvet.3-95.wav

---------------------------------------
General Comments: Original vs Sample 1 - warm up, but still managed to ABX it at 32 tries. (I use techinque of min. 16x, 24x, 32x) Most noticeable artifacti 8.11-9.30
---------------------------------------
1R File: C:\Music\LAMEsamples\velvet.3-90-3.wav
1R Rating: 4.0
1R Comment: Slightly smeared percussion, no ATH problem
---------------------------------------
2L File: C:\Music\LAMEsamples\velvet.3-95.wav
2L Rating: 3.0
2L Comment: Smeared percussion, ATH problems
---------------------------------------

ABX Results:
Original vs C:\Music\LAMEsamples\velvet.3-95.wav
    16 out of 16, pval < 0.001
Original vs C:\Music\LAMEsamples\velvet.3-90-3.wav
    23 out of 32, pval = 0.01
--

Bad news. 3.95 still worse, noticeably.
I didn't even try to ABX these against themselves, the difference is glaring.

EDIT: Yes, glaring. Glaring PLACEBO. Failed ABX, pval ~0.4 at 32 tries.
Maybe I've got tired of this percussion, I'll redo this tomorrow.

EDIT2: Failed again, similar result.
Title: Lame 3.95 vs 3.90.3
Post by: rjamorim on 2004-01-12 22:53:51
Nice initiative. Thanks
Title: Lame 3.95 vs 3.90.3
Post by: AstralStorm on 2004-01-12 23:10:38
Oh, I see that there's 3.95.1 out now.
I'll retest with this later. (although the chance is slim the result may be different)
Next sample tomorrow.

Bug report - Lame 3.95.x can't decode at all. (jon33 compile)
Even can't decode its own files!
Title: Lame 3.95 vs 3.90.3
Post by: PlazzTT on 2004-01-13 00:15:59
Quote
Bug report - Lame 3.95.x can't decode at all. (jon33 compile)
Even can't decode its own files!

Verified.
Title: Lame 3.95 vs 3.90.3
Post by: sony666 on 2004-01-13 01:20:25
foobar 0.77a ABX, replaygain, no DSP
Sony MDR-P70 headphones, somewhat noisy PC, Terratec PCI soundcard 5 years old

the candidate: fatboy

1) --preset cbr 128
-wav vs. 3.90.3
outrageous, mp3 sounds very "scratchy".. 8/8, 0.4% guess prob.

-wav vs. 3.95.1
much better than 3.90.3, still easy to pick from wav. 8/8, 0.4%

-3.90.3 vs. 3.95.1:
3.90.3 very severe artifacts, 3.95.1 much better.  8/8, 0.4%

clear winner 3.95.1
---

2) --preset 160 (ABR)
wav vs. 3.90.3 (191kbit):
still very scratchy vocals, slightly better than 128CBR.  8/8 0,4%

wav vs. 3.95.1 (204kbit):
needs utmost concentration, scratching is gone  10/16, 22.7% guess prob.

3.90.3 vs. 3.95.1:
easy. 8/8

clear winner 3.95.1
---

3)--preset medium
wav vs. 3.90.3 (247kbit):
scratchy vocals, big letdown for that bitrate.  8/8

wav vs. 3.95.1 (237kbit):
4/16, 98,9% of guessing. transparent for me

3.90.3 (247kbit) vs. 3.95.1 (237kbit):
scratchy vocals in 3.90.3. 8/8

clear winner 3.95.1
---

4) --preset standard
wav vs. 3.90.3 (265kbit):
impossible
wav vs. 3.95.1 (268kbit):
impossible

both transparent to me
---

3.90.3 is very disapointing for this sample at medium bitrates, 3.95 does good
thanks, need a pause now
edit: added bitrates for ABR 160
Title: Lame 3.95 vs 3.90.3
Post by: amano on 2004-01-13 02:35:44
huh, astralstorm, please edit your post until you came up with a decision.

it is a bit ... biased now.
my feelings are too.

Sony666's results are very promising though. I think, that if there are any major differences, they will be rather in the low bitrate area.
testing preset medium with 3.90.3 is a bit unfair, because that wasn't tuned for 3.90.x.
it was added afterwards and tuned for 3.93, I think.

EDIT: typos
Title: Lame 3.95 vs 3.90.3
Post by: sony666 on 2004-01-13 03:51:39
more samples from http://lame.sourceforge.net/gpsycho/quality.html (http://lame.sourceforge.net/gpsycho/quality.html)

Metallica - Fade To Black (ftb_samp.wav)

1) --preset cbr 128
-wav vs. 3.90.3:
very small "warbling" effect at ~0.4s, only listenable with headphones, but 8/8

-wav vs. 3.95.1:
11/16 (10.5% of guess.) hmmm.... very hard to diff, if possible at all. need direct comparison vs. 3.90.1

3.90.3 vs. 3.95.1:
8/16 (59.8% guess).. bleh, I could have sworn the guitar attack at very start was more distorted with 3.90.3, but...

Conclusion: both do excellent on this real world sample. I was very surprised with 128k CBR quality  higher bitrates useless to abx here..
---


the "infamous" velvet

1) --preset cbr 128
-wav vs. 3.90.3:
obvious.. can't handle the sharp attacks, hissing...  8/8

-wav vs. 3.95.1:
better handling of sharp attacks, still hissing  8/8

3.90.3 vs. 3.95.1:
14/16 (0.2% guess) 3.90.3 has a slight error in first base drum attack

3.95.1 ahead
---

2) --preset 160 (ABR)
-wav vs. 3.90.3 (163kbit):
base drum has obvious errors  8/8

-wav vs. 3.95.1 (171kbit):
base drum errors are gone, trying to diff via hissing... yes. 8/8. hi-hat(?) sounds softer in original wav, but encoding is good here

3.90.3 vs. 3.95.1:
easy, 3.90.3 has erros in base drum that sound a little like clipping  8/8

3.95.1 clear winner for me
---

3) --preset standard
--wav vs. 3.90.3 (231kbit):
took long, long time to figure this out... slight unnatural hissing in encode. 13/16 (1.1% guess)

-wav vs. 3.95.1 (188kbit):
not hard to pick with the training from 3.90.3, slight unnatural sharpness. same as above  7/8 (3.5% guess)

-3.90.3 vs. 3.95.1:
7/8 (3.5%)... 3.95.1 sharp hissing seems a tiny little more obvious. I will dream of this sample tonight.

3.95 better at medium bitrates, --aps.. very hard.. numbers say 3.90.3 but wait for other opinions on that
Title: Lame 3.95 vs 3.90.3
Post by: ViPER1313 on 2004-01-13 04:35:06
I used the Waiting sample using the --preset standard command line on both v3.90.3 and 3.95.1 - I found v3.95.1 to be slightly better than 3.90.3 - hard to ABX the two - had to rest, then I got it

ABC/HR Version 0.9b, 30 August 2002
Testname: Waiting --preset standard

1L = Z:\Music\Test Samples\3.90.3\Wavs\001 Waiting.wav
2R = Z:\Music\Test Samples\3.95\Wavs\001 Waiting.wav

---------------------------------------
General Comments:
Tested 3.5 to 5 sec
---------------------------------------
1L File: Z:\Music\Test Samples\3.90.3\Wavs\001 Waiting.wav
1L Rating: 4.4
1L Comment: Smearing, seems worse than sample 2, very close
---------------------------------------
2R File: Z:\Music\Test Samples\3.95\Wavs\001 Waiting.wav
2R Rating: 4.7
2R Comment: Smearing as well, not as bad as 1, not anoying
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\3.90.3\Wavs\001 Waiting.wav
    13 out of 14, pval < 0.001
Original vs Z:\Music\Test Samples\3.95\Wavs\001 Waiting.wav
    10 out of 10, pval < 0.001
Z:\Music\Test Samples\3.90.3\Wavs\001 Waiting.wav vs Z:\Music\Test Samples\3.95\Wavs\001 Waiting.wav
    34 out of 45, pval < 0.001
Title: Lame 3.95 vs 3.90.3
Post by: ViPER1313 on 2004-01-13 04:55:15
ABC/HR Version 0.9b, 30 August 2002
Testname: Fatboy --preset standard

1L = Z:\Music\Test Samples\3.90.3\Wavs\Fatboy Audio Quality Test File.wav
2L = Z:\Music\Test Samples\3.95\Wavs\Fatboy Audio Quality Test File.wav

---------------------------------------
General Comments:
Very hard to call which encoder is really better at this sample - 1 seams to have problems with a sort of crackle, but 2 seems slightly distorted throughout - actually had trouble w/ 2 for a sec cause of room noise - i would call 2 the winner cause it doesnt seem as harsh as 1 does in places
---------------------------------------
1L File: Z:\Music\Test Samples\3.90.3\Wavs\Fatboy Audio Quality Test File.wav
1L Rating: 3.9
1L Comment: Slight crackle sound, not bad, close to origional
---------------------------------------
2L File: Z:\Music\Test Samples\3.95\Wavs\Fatboy Audio Quality Test File.wav
2L Rating: 4.3
2L Comment: Less distortion then 1, pre-echo, but no crackles
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\3.90.3\Wavs\Fatboy Audio Quality Test File.wav
    10 out of 10, pval < 0.001
Original vs Z:\Music\Test Samples\3.95\Wavs\Fatboy Audio Quality Test File.wav
    16 out of 18, pval < 0.001
Z:\Music\Test Samples\3.90.3\Wavs\Fatboy Audio Quality Test File.wav vs Z:\Music\Test Samples\3.95\Wavs\Fatboy Audio Quality Test File.wav
    10 out of 10, pval < 0.001
Title: Lame 3.95 vs 3.90.3
Post by: Gabriel on 2004-01-13 12:54:10
Don't forget that there are 2 vbr modes
Title: Lame 3.95 vs 3.90.3
Post by: sony666 on 2004-01-13 13:39:05
from http://sound.media.mit.edu/mpeg4/audio/sqam/ (http://sound.media.mit.edu/mpeg4/audio/sqam/)
harp40_1.wav (harpsichord)

1) --preset cbr 128
-wav vs. 3.90.3: obvious distortions, 8/8
-wav vs. 3.95.1: obvious distortions, 8/8
-3.90.3 vs. 3.95.1: unable to find diff, would be pure guess

2) --preset 160 (ABR)
-wav vs. 3.90.3 (144kbit): less annoying than 128k, still 8/8
-wav vs. 3.95.1 (145kbit): less annoying than 128k, still 8/8
-3.90.3 vs. 3.95.1: 3/8, pure guess

3) --preset medium (VBR)
-wav vs. 3.90.3 (134kbit): 8/8, obvious distortions like 128k
-wav vs. 3.95.1 (111kbit): 8/8, obvious distortions
-3.90.3 vs. 3.95.1: 2/8, both deliver equally bad results for that VBR preset

4) --preset standard
-wav vs. 3.90.3 (159kbit): 10/16, slight distortion at very first note, hard to find
-wav vs. 3.95.1 (144kbit): 16/16 yes..same as above, training from 3.90 helped finding it
-3.90.3 vs. 3.95.1: 10/10, 3.95.1 has slightly more annoying distortion

conclusion: If I was a Harpsichord fan I'd stay away from mp3 . But I dont even know what they look like.

edit:
5) --preset fast standard
-wav vs. 3.90.3 (159kbit): very easily noticeable background noise added during first sec, 8/8
-wav vs. 3.95.1 (165kbit): 15/20 something is not right about background noise during 1st second, hard to find
-3.90.3 vs. 3.95.1: 8/8, 3.90.3 sounds "dirty" during first second, 3.95 clearer
Title: Lame 3.95 vs 3.90.3
Post by: ViPER1313 on 2004-01-13 15:09:24
v3.90.3 seems to edge out v3.95.1 in Waiting using --preset fast standard - read the comments below...

ABC/HR Version 0.9b, 30 August 2002
Testname: Waiting --preset fast standard

1R = Z:\Music\Test Samples\3.95.1 Fast.wav
2L = Z:\Music\Test Samples\3.90.3 Fast.wav

---------------------------------------
General Comments:
Neither clip sounds great, comparing the encoded files to the origional was easy with or without boosted trebble, 2 seems to have a slight edge
---------------------------------------
1R File: Z:\Music\Test Samples\3.95.1 Fast.wav
1R Rating: 4.3
1R Comment: Slightly worse than sample 2, suffers from a loss of high end - seems like background hiss is removed, smearing, sounds harsh in places, such as the during the "at all"
---------------------------------------
2L File: Z:\Music\Test Samples\3.90.3 Fast.wav
2L Rating: 4.6
2L Comment: Very slight loss of clarity, loss of "air" in the file - easy to tell w/ boosted trebble, the "for.....this....moment" part of the song sounds smeared
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\3.95.1 Fast.wav
    18 out of 21, pval < 0.001
Original vs Z:\Music\Test Samples\3.90.3 Fast.wav
    15 out of 18, pval = 0.004
Z:\Music\Test Samples\3.95.1 Fast.wav vs Z:\Music\Test Samples\3.90.3 Fast.wav
    13 out of 14, pval < 0.001
Title: Lame 3.95 vs 3.90.3
Post by: ViPER1313 on 2004-01-13 15:21:45
LARGE difference in quality here - v3.90.3 has the clear advantage in this instance.

ABC/HR Version 0.9b, 30 August 2002
Testname: DaFunk --preset cbr 128

1L = Z:\Music\Test Samples\3.95.1 CBR 128.wav
2R = Z:\Music\Test Samples\3.90.3 CBR 128.wav

---------------------------------------
General Comments:
Very supprised between the large difference in qual between 1 and 2 here - 2 very close to the origional while 1 sounds distorted.
---------------------------------------
1L File: Z:\Music\Test Samples\3.95.1 CBR 128.wav
1L Rating: 3.2
1L Comment: Kind of ringing / warble in the background - anoying, much worse than sample 2
---------------------------------------
2R File: Z:\Music\Test Samples\3.90.3 CBR 128.wav
2R Rating: 4.2
2R Comment: Even at 128kbps this sounds decent - the warble that is present in 1 is almost completely absent - actually had to concentrate for a sec to ABX from the origional - crisper than 1
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\3.95.1 CBR 128.wav
    10 out of 10, pval < 0.001
Original vs Z:\Music\Test Samples\3.90.3 CBR 128.wav
    12 out of 12, pval < 0.001
Z:\Music\Test Samples\3.95.1 CBR 128.wav vs Z:\Music\Test Samples\3.90.3 CBR 128.wav
    13 out of 14, pval < 0.001

EDIT - Added encspot info - it seems that v3.95.1 uses many more short blocks than 3.90.3 (On this clip, 3.90.3 uses 7.9% short blocks while 3.95.1 uses 25.3%) - I have seen this in many other clips as well.


v3.90.3 --preset cbr 128[/u]

Bitrates:
----------------------------------------------------
128    ||||||||||||||||||||||||||||||||||||||||      99.9%
----------------------------------------------------

Type                : mpeg 1 layer III
Bitrate            : 127
Mode                : joint stereo
Frequency          : 44100 Hz
Frames              : 791
ID3v2 Size          : 0
First Frame Pos    : 0
Length              : 00:00:20
Max. Reservoir      : 486
Av. Reservoir      : 281
Emphasis            : none
Scalefac            : 37.4%
Bad Last Frame      : no
Encoder            : Lame 3.90

Lame Header:

Quality                : 58
Version String        : Lame 3.90
Tag Revision          : 0
VBR Method            : cbr
Lowpass Filter        : 17600
Psycho-acoustic Model  : nspsytune
Safe Joint Stereo      : no
nogap (continued)      : no
nogap (continuation)  : no
ATH Type              : 2
ABR Bitrate            : 128
Noise Shaping          : 2
Stereo Mode            : Joint Stereo
Unwise Settings Used  : no
Input Frequency        : 44.1kHz

v3.95.1 --preset cbr 128

Bitrates:
----------------------------------------------------
128    ||||||||||||||||||||||||||||||||||||||||      99.9%
----------------------------------------------------

Type                : mpeg 1 layer III
Bitrate            : 127
Mode                : joint stereo
Frequency          : 44100 Hz
Frames              : 791
ID3v2 Size          : 0
First Frame Pos    : 0
Length              : 00:00:20
Max. Reservoir      : 482
Av. Reservoir      : 175
Emphasis            : none
Scalefac            : 1.1%
Bad Last Frame      : no
Encoder            : Lame 3.95

Lame Header:

Quality                : 57
Version String        : Lame 3.95
Tag Revision          : 0
VBR Method            : cbr
Lowpass Filter        : 17500
Psycho-acoustic Model  : nspsytune
Safe Joint Stereo      : no
nogap (continued)      : no
nogap (continuation)  : no
ATH Type              : 4
ABR Bitrate            : 128
Noise Shaping          : 1
Stereo Mode            : Joint Stereo
Unwise Settings Used  : no
Input Frequency        : 44.1kHz
Title: Lame 3.95 vs 3.90.3
Post by: indybrett on 2004-01-13 15:23:37
If I'm reading these right, it would seem that 3.90.3 does better on some samples, with 3.95.1 doing better on others (when using APS).
Title: Lame 3.95 vs 3.90.3
Post by: Gabriel on 2004-01-13 15:34:51
Where can I download the DaFunk sample?
Title: Lame 3.95 vs 3.90.3
Post by: ViPER1313 on 2004-01-13 15:44:39
http://www.angelfire.com/mt/viper1313/dafunk.flac (http://www.angelfire.com/mt/viper1313/dafunk.flac) - you might have to right click the file and select "save-as" - I forget where I dl'ed this sample.....
Title: Lame 3.95 vs 3.90.3
Post by: ViPER1313 on 2004-01-14 00:31:31
Just tested the Layla sample at --preset cbr 128 :: v3.90.3 seems to be the distinct winner of this one as well - might try at --preset standard in a few min..... EDIT - tried and failed 

ABC/HR Version 0.9b, 30 August 2002
Testname: Layla --preset cbr 128

1R = Z:\Music\Test Samples\Layla 3.90.3 cbr 128.wav
2L = Z:\Music\Test Samples\Layla 3.95.1 cbr 128.wav

---------------------------------------
General Comments:
Started the ABX test focusing in the 3.5 - 5 sec area of the song, used it to compare the 2 files to the original (easy at 128kbps  Could not distinguish between the 2 mp3s in that area, focused on the 0-2 sec range and found a large difference.
---------------------------------------
1R File: Z:\Music\Test Samples\Layla 3.90.3 cbr 128.wav
1R Rating: 3.9
1R Comment: Does not exhibit the obvious under water sound of sample 2 in the 0-2 sec range, decent overall, rest of file sounds similar to 2
---------------------------------------
2L File: Z:\Music\Test Samples\Layla 3.95.1 cbr 128.wav
2L Rating: 3.0
2L Comment: Very bad under water sound from 0-2 sec, rest of file sounds less crisp than origional
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\Layla 3.90.3 cbr 128.wav
    10 out of 10, pval < 0.001
Original vs Z:\Music\Test Samples\Layla 3.95.1 cbr 128.wav
    13 out of 14, pval < 0.001
Z:\Music\Test Samples\Layla 3.90.3 cbr 128.wav vs Z:\Music\Test Samples\Layla 3.95.1 cbr 128.wav
    13 out of 14, pval < 0.001
Title: Lame 3.95 vs 3.90.3
Post by: amano on 2004-01-14 00:44:50
Hmm, is 3.95.1 generally worse at 128 kbps CBR? Viper, can you give 128 kbps ABR a try?
Title: Lame 3.95 vs 3.90.3
Post by: ViPER1313 on 2004-01-14 01:37:27
Quote
Hmm, is 3.95.1 generally worse at 128 kbps CBR? Viper, can you give 128 kbps ABR a try?

Sure.... results are pretty much the same as CBR, except both clips sound better overall

ABC/HR Version 0.9b, 30 August 2002
Testname: Layla --preset 128 ABR

1R = Z:\Music\Test Samples\Layla 3.90.3 128kbps ABR.wav
2L = Z:\Music\Test Samples\Layla 3.95.1 128kbps ABR.wav

---------------------------------------
General Comments:
Sample 2 sounds distinctively worse in the 0-2 sec range, both samples sound better than CBR - would bet sample 2 is v3.95.1 (sounds similar to CBR)
---------------------------------------
1R File: Z:\Music\Test Samples\Layla 3.90.3 128kbps ABR.wav
1R Rating: 4.1
1R Comment: Not bad sounding, better than sample 2
---------------------------------------
2L File: Z:\Music\Test Samples\Layla 3.95.1 128kbps ABR.wav
2L Rating: 3.6
2L Comment: Warble in the 0-2 sec range, worse than sample 1
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\Layla 3.90.3 128kbps ABR.wav
    16 out of 18, pval < 0.001
Original vs Z:\Music\Test Samples\Layla 3.95.1 128kbps ABR.wav
    14 out of 15, pval < 0.001
Z:\Music\Test Samples\Layla 3.90.3 128kbps ABR.wav vs Z:\Music\Test Samples\Layla 3.95.1 128kbps ABR.wav
    12 out of 12, pval < 0.001
Title: Lame 3.95 vs 3.90.3
Post by: jkml on 2004-01-14 01:41:03
Quote
conclusion: If I was a Harpsichord fan I'd stay away from mp3 . But I dont even know what they look like.


They look pretty elegant: 

http://www.henrylim.org/Harpsichord.html (http://www.henrylim.org/Harpsichord.html)
Title: Lame 3.95 vs 3.90.3
Post by: ViPER1313 on 2004-01-14 17:22:34
Tested the 41_30 sample with both encoders at --preset standard - I'll have to call this one a tie.

ABC/HR Version 0.9b, 30 August 2002
Testname: 41_30 --preset standard

1R = Z:\Music\Test Samples\3.95.1 41_30 --preset standard.wav
2R = Z:\Music\Test Samples\3.90.3 41_30 --preset standard.wav

---------------------------------------
General Comments:
Focused on the 0 - 2.3 sec range of this clip - both encoders suffer from smearing, but I can not ABX the difference between the 2 - I thought I could hear a difference at first, but couldnt ABX it - I'll have to call this one a tie
---------------------------------------
1R File: Z:\Music\Test Samples\3.95.1 41_30 --preset standard.wav
1R Rating: 4.5
1R Comment:
---------------------------------------
2R File: Z:\Music\Test Samples\3.90.3 41_30 --preset standard.wav
2R Rating: 4.5
2R Comment:
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\3.95.1 41_30 --preset standard.wav
    10 out of 10, pval < 0.001
Original vs Z:\Music\Test Samples\3.90.3 41_30 --preset standard.wav
    14 out of 15, pval < 0.001
Z:\Music\Test Samples\3.95.1 41_30 --preset standard.wav vs Z:\Music\Test Samples\3.90.3 41_30 --preset standard.wav
    15 out of 24, pval = 0.154
Title: Lame 3.95 vs 3.90.3
Post by: AstralStorm on 2004-01-14 22:25:18
Fresh new.
Heh, I like Pink Floyd - this one sound like a fragment of a piece from album "Saucerful of Secrets".
Highly psychedelic.

ABC/HR for Java Version 0.4b, 14 styczeń 2004
Testname: main_theme (preset standard)

1R = C:\Music\LAMEsamples\main_theme.3-90-3.mp3.wav
2R = C:\Music\LAMEsamples\main_theme.3-95-1.mp3.wav

---------------------------------------
General Comments: Well well, it seems that without Sensaura on (resamples -> 48 kHz) it is much easier to ABC/HR anything. I didn't even bother to ABX with it on.
---------------------------------------
1R File: C:\Music\LAMEsamples\main_theme.3-90-3.mp3.wav
1R Rating: 4.0
1R Comment: 5-7s: Sharper transition center->right, ABX vs orig is based sollely on this. Failure ABXing with 2 on that, but detected warbling in plates 3-5s
---------------------------------------

ABX Results:
Original vs C:\Music\LAMEsamples\main_theme.3-95-1.mp3.wav
    8 out of 17, pval = 0.685
Original vs C:\Music\LAMEsamples\main_theme.3-90-3.mp3.wav
    12 out of 16, pval = 0.038
C:\Music\LAMEsamples\main_theme.3-90-3.mp3.wav vs C:\Music\LAMEsamples\main_theme.3-95-1.mp3.wav
    18 out of 24, pval = 0.011
Title: Lame 3.95 vs 3.90.3
Post by: robert on 2004-01-15 13:55:15
Quote
Fresh new.
Heh, I like Pink Floyd - this one sound like a fragment of a piece from album "Saucerful of Secrets".
Highly psychedelic.

ABC/HR for Java Version 0.4b, 14 styczeń 2004
Testname: main_theme (preset standard)

1R = C:\Music\LAMEsamples\main_theme.3-90-3.mp3.wav
2R = C:\Music\LAMEsamples\main_theme.3-95-1.mp3.wav

I took that sample from "More"
Title: Lame 3.95 vs 3.90.3
Post by: adlai on 2004-01-15 20:15:39
so, from skimming over this thread, I'm guessing that 3.95 is a "meh" release since quality is more or less the same while file sizes have increased
Title: Lame 3.95 vs 3.90.3
Post by: AtaqueEG on 2004-01-15 20:28:57
Quote
Quote

conclusion: If I was a Harpsichord fan I'd stay away from mp3 . But I dont even know what they look like.


They look pretty elegant: 

http://www.henrylim.org/Harpsichord.html (http://www.henrylim.org/Harpsichord.html)

OT:
That Harpsichord is fake! Is made of LEGO pieces!
Although it does look like the real thing (http://images.google.com/images?q=harpsichord&ie=UTF-8&oe=UTF-8&hl=en&btnG=Google+Search)


On Topic:
I think that this version of LAME is not going to do very good on Roberto's test 
Title: Lame 3.95 vs 3.90.3
Post by: indybrett on 2004-01-15 21:11:03
Quote
so, from skimming over this thread, I'm guessing that 3.95 is a "meh" release since quality is more or less the same while file sizes have increased

On my music, filesizes decreased when using APS.
Title: Lame 3.95 vs 3.90.3
Post by: Digga on 2004-01-15 21:28:44
Quote
Quote
so, from skimming over this thread, I'm guessing that 3.95 is a "meh" release since quality is more or less the same while file sizes have increased

On my music, filesizes decreased when using APS.

on the samples I tried (mostly hip-hop), the size gets smaller too. no wonder though I guess, with bitrates on aps going now down to 96 kbps.
but I think we should respect the title of this trhead, shouldn't we?!
"ABC/HR test results here only", and no meh-stuff.
Title: Lame 3.95 vs 3.90.3
Post by: fairyliquidizer on 2004-01-15 21:43:58
Quote
Quote
Bug report - Lame 3.95.x can't decode at all. (jon33 compile)
Even can't decode its own files!

Verified.

The decoder is only there for the development frontend to allow analysis during testing.  It's not intended for wider use.

Cheers,
Fairy
Title: Lame 3.95 vs 3.90.3
Post by: phwip on 2004-01-15 22:58:46
Quote
Quote
Quote
so, from skimming over this thread, I'm guessing that 3.95 is a "meh" release since quality is more or less the same while file sizes have increased

On my music, filesizes decreased when using APS.

on the samples I tried (mostly hip-hop), the size gets smaller too. no wonder though I guess, with bitrates on aps going now down to 96 kbps.

I have just encoded a random selection of 30 tracks from my hard drive all from different albums with both 3.90.3 and 3.95.1, both --preset standard.  The songs are in the main rock and pop but there are a couple of classical and a couple of jazz tracks in there and the rest is a whole mixture of styles.

The total size of the wav files is 1.17 GB.  With 3.90.3 this compresses to 171 MB and with 3.95.1 to 164 MB.  21 of the 30 tracks were smaller when using 3.95.1.

Quote
but I think we should respect the title of this trhead, shouldn't we?!
"ABC/HR test results here only", and no meh-stuff.

You are right.  Having done the test I thought I might as well post the result.  But perhaps a moderator could move this whole offtopic section to a different thread?
Title: Lame 3.95 vs 3.90.3
Post by: indybrett on 2004-01-16 01:44:32
It seems that in the lower bitrates that 3.90.3 is winning most of the ABX tests. If the two are found to be of equal sound quality with APS, then the resulting filesize would probably be the determining factor.

There is probably no point in discussing filesize any further until there is a clear winner (or tie) with APS.
Title: Lame 3.95 vs 3.90.3
Post by: Vietwoojagig on 2004-01-16 09:58:40
Hi,

some people mentioned this before, but:

Don't you think that comparing 3.90.3 and 3.95.1 needs a developer-moderated and coordinated testing? I don't know how many tests they have made on which samples, but a little hint, which of the test-cases in the test-case-archive have been checked and which not, would make it more easy for the people around not to waste their time on already proved samples.

And: Only with a moderated and coordinated tesiting, everybody in this forum will accespt the results of the tests and (in the end) will accept, that 3.95.1 is better or worser than 3.90.3.

Thanks for your attention.
Title: Lame 3.95 vs 3.90.3
Post by: sld on 2004-01-16 10:38:50
Quote
OT:
That Harpsichord is fake! Is made of LEGO pieces!

It is real, and playable.
OT, but had to clarify to all.
Title: Lame 3.95 vs 3.90.3
Post by: AstralStorm on 2004-01-16 16:10:06
Quote
so, from skimming over this thread, I'm guessing that 3.95 is a "meh" release since quality is more or less the same while file sizes have increased

Not necessarily true, the filesizes of the samples I've tested (bytes, preset standard):

275 565 main_theme.3-90-3.mp3
286 648 main_theme.3-95-1.mp3

345 709 velvet.3-90-3.mp3
281 130 velvet.3-95-1.mp3

And the quality isn't same.

Quote
Don't you think that comparing 3.90.3 and 3.95.1 needs a developer-moderated and coordinated testing? I don't know how many tests they have made on which samples, but a little hint, which of the test-cases in the test-case-archive have been checked and which not, would make it more easy for the people around not to waste their time on already proved samples.


It isn't a waste of time, different people have different hearing...

EDIT: I'll edit the subject to say "(and filesizes too)"
EDIT2: Blah, it isn't possible, now you can flame Invision Board devs.
Title: Lame 3.95 vs 3.90.3
Post by: dev0 on 2004-01-16 16:23:16
Quote
Don't you think that comparing 3.90.3 and 3.95.1 needs a developer-moderated and coordinated testing? I don't know how many tests they have made on which samples, but a little hint, which of the test-cases in the test-case-archive have been checked and which not, would make it more easy for the people around not to waste their time on already proved samples.

I asked about this on lame-dev and tried to contact Gabriel, but have always been given some kind of "Well... whatever... test it." answer.
Of course a developer-moderated and coordinated testing would be more effective, but it seems like the lame-dev team in not really interested in quality and obviously nobody can force them to be.
Title: Lame 3.95 vs 3.90.3
Post by: Gabriel on 2004-01-16 17:56:24
Quote
but it seems like the lame-dev team in not really interested in quality

On this board you have to back up your claims with facts. Can you provide facts?

Quote
It isn't a waste of time, different people have different hearing...

This is very true. If I say "this and this samples are better when using those settings", will you believe me? I am not sure about it.
Title: Lame 3.95 vs 3.90.3
Post by: kennedyb4 on 2004-01-16 18:07:04
Quote
I asked about this on lame-dev and tried to contact Gabriel, but have always been given some kind of "Well... whatever... test it." answer.
Of course a developer-moderated and coordinated testing would be more effective, but it seems like the lame-dev team in not really interested in quality and obviously nobody can force them to be.

As an avid observer over the last several years, I have to say that the opposite appears to be true, at least to me.
Title: Lame 3.95 vs 3.90.3
Post by: indybrett on 2004-01-16 18:08:52
Time to split the thread off...maybe?
Title: Lame 3.95 vs 3.90.3
Post by: Atlantis on 2004-01-16 19:07:03
Quote
If I say "this and this samples are better when using those settings", will you believe me? I am not sure about it.

Hi Gabriel,
I agree with dev0 that "a developer-moderated and coordinated testing would be more effective".

I know that your time (and other developer's one) dedicated to developing LAME can be limited (everybody needs a life  ) but you (LAME developers) should really consider to coordinate the testing process.
I think that many people (me included) will gladly help you if asked for (let's say) a test about a sample using two different lame version or something like that.
I follow the lame-dev mailing list, but still I don't exactly know what is the impact of the changes added by 3.95x encoder: so maybe it could be good to have more "inverse feedback" (please smile at this definition) by you on test done by HA users.



Last but not least, thank-you for working on LAME: it is of great help for me.
Regards
Title: Lame 3.95 vs 3.90.3
Post by: Gabriel on 2004-01-16 19:16:09
Quote
I think that many people (me included) will gladly help you if asked for (let's say) a test about a sample using two different lame version or something like that.

When there is a specific change that needs to be tested, I agree. But for 3.95 against 3.90, there are many changes, and so nearly everything could be impacted.
Title: Lame 3.95 vs 3.90.3
Post by: [proxima] on 2004-01-16 19:55:56
Quote
Of course a developer-moderated and coordinated testing would be more effective

I perfectly agree with you and i've pointed to this point many times in my posts. I like doing listening tests but sometimes, i feel something like i'm wasting my time because lack of organization.
In this community there are for sure a lot of people interested in the future of LAME but what miss is a good organization: listening tests should be "guided" by someone with artifacts trained ears and with a little coding skills so that he can directly tweak the code. This role was covered very well by Dibrom in the past but now he lost interest and has very little time. Dibrom himself stated many times that the alt-presets are a sort of "hack" of LAME 3.90 and there is always the danger of modified behaviour with newer versions, even a different compilation switches seem to have altered slightly the behaviour of alt-presets !! So we have "our stable version" that is 3.90.2/3.90.3 but (even if very good in quality) now is quite old and we need to unblock this situation if we want to get advantages from future LAME progresses.
Alt-presets as they are now seem to have a too high sensitivity to slight encoder changes, IMHO what we need is to "stabilize" the alt-presets in the sense that they must be considered no more as "hacks" but as an integral part of the LAME code.
Of course, at this time, i don't see any people that could do this and i don't want to accuse developers, i perfectly understand the priorities and the limited time.
Quote
but it seems like the lame-dev team in not really interested in quality

Sure, there was some hurried releases marked as "stable" (3.93 for example). Maybe the alt-presets behavior of some "post 3.90" stable releases is not perfect but i think that what you stated is not literally true.
Title: Lame 3.95 vs 3.90.3
Post by: Atlantis on 2004-01-16 20:03:02
Quote
But for 3.95 against 3.90, there are many changes, and so nearly everything could be impacted.

Agreed.

In my first post I was just talking about (like you correctly understood) the "usual" development process (alpha => beta => release).

Regards,
Alberto
Title: Lame 3.95 vs 3.90.3
Post by: dev0 on 2004-01-16 20:47:16
Quote
,Jan 16 2004, 08:55 PM] Alt-presets as they are now seem to have a too high sensitivity to slight encoder changes, IMHO what we need is to "stabilize" the alt-presets in the sense that they must be considered no more as "hacks" but as an integral part of the LAME code.
Of course, at this time, i don't see any people that could do this and i don't want to accuse developers, i perfectly understand the priorities and the limited time.

From what I've gathered from the changelogs that's exactly what Gabriel is trying to do, there's only some more testing needed to verify that his changes and the new version provide an overall higher quality than the currently recommended version with Dibrom's original tunings.

A lot of fine tuning will have to be redone when LAME4 is approaching completition and will hopefully introduce us to a whole new level of MP3 quality.
Title: Lame 3.95 vs 3.90.3
Post by: Wombat on 2004-01-16 21:25:28
At first, i don´t abx much samples at this time.

One of the old samples i always hear a problem with aps is sophia2.
As i already mentioned in another thread the sophia2 sample has an artifact plop with 3.95.1 and has a sandpaper noise with 3.90.3 but no plop.
I prefer the sandpaper over the artifact but the best out of the mix would even be perfect
So someone beeing able to change the preset directly to test some circumstances will be the key like proxima already wrote.
The potential of lame curing this sample is there obviously.

Atm Gabriel seems to be the only one being able to this.
So i have to thank Gabriel, the developer, for his great work till here and wish him good luck finding the final tweak!


Wombat
Title: Lame 3.95 vs 3.90.3
Post by: Wombat on 2004-01-17 00:56:41
One thing to think about:
Even if lame 3.95.1 is behaving equal in the end with the testsamples it wouldn´t tell much!
It took about 2 years now to find all these problem music. It would take another 2 years to
find problems with 3.95 introduced with the modifications. There is no proove at all to set it
equal to 3.90.3.

There is no sense for using aps with any newer build than 3.90.3 until many testsamples "are really cured"

Isn´t it?

Wombat
Title: Lame 3.95 vs 3.90.3
Post by: AstralStorm on 2004-01-17 02:14:57
Actually, there is a reason: it is about 2x faster with similar quality as 3.90.3 most of the time.
It also doesn't produce larger files on (my) average, but it really depends on the type of music.
It's only up to the admins to determine if it's good enough to replace 3.90.3 as recommended version,
but to do this they need test results, hence this thread.
It seems like the recomended version needs a refresh - but when is it going to happen, we shall see.
Maybe just not now.

Anyway, there's no perfect psychoacoustic compression yet and there won't be any method soon.
MusePack is much closer to this than LAME. For True Quality ™, use the lossless compression.
We're trying here to just save some space with acceptable (possibly artifact-free) quality.
It's just the difference in tradeoff (perceived quality vs space vs speed).
One may even find 128kbps MP3s produced by l3enc good enough.
(shame on him, but we don't need to listen to that)

Enough discussion, please test further.
Title: Lame 3.95 vs 3.90.3
Post by: LoFiYo on 2004-01-17 03:52:10
Here is my test result using my plastic (as opposed to golden) ears  .

Code: [Select]
ABC/HR Version 0.9b, 30 August 2002
Testname: Polonaise 3.90.3 v 3.95.1 - ABR128

1L = C:\My Music\lab\Polonaise\Polonaise-ABR128-3.95.1.wav
2L = C:\My Music\lab\Polonaise\Polonaise-ABR128-3.90.3.wav

---------------------------------------
General Comments:
3.90.3 vs 3.95.1 (ABR 128kbps mode = alt-preset 128)
---------------------------------------
1L File: C:\My Music\lab\Polonaise\Polonaise-ABR128-3.95.1.wav
1L Rating: 3.7
1L Comment: After around 15.5 sec, noticeable distortion is detected.
---------------------------------------
2L File: C:\My Music\lab\Polonaise\Polonaise-ABR128-3.90.3.wav
2L Rating: 4.0
2L Comment: Distortion is still noticeable, but clearly better than the other encode.
---------------------------------------
ABX Results:
Original vs C:\My Music\lab\Polonaise\Polonaise-ABR128-3.95.1.wav
   15 out of 17, pval = 0.001
Original vs C:\My Music\lab\Polonaise\Polonaise-ABR128-3.90.3.wav
   13 out of 14, pval < 0.001
Title: Lame 3.95 vs 3.90.3
Post by: Continuum on 2004-01-17 08:16:37
LoFiYo, try an ABX-test between the two encoded samples as well.
Title: Lame 3.95 vs 3.90.3
Post by: LoFiYo on 2004-01-17 18:22:23
Quote
LoFiYo, try an ABX-test between the two encoded samples as well.

I gave it a try just now, but failed miserably. Actually I started out really well (up to the 7th or 8th trial), but ended up something like 9/13. It seemed that toward the end, my ears (and brain) got so tired that they stopped caring about the difference. I guess to me with this sample, the difference wasn't that big after all...
Title: Lame 3.95 vs 3.90.3
Post by: LoFiYo on 2004-01-17 19:57:00
I tried a personal sample file (the first 30 seconds of It's a Sin by Pet Shop Boys). I remember reading somewhere that newer versions of Lame should be a little better at lower bitrates than 3.90.3, so I tried a very low bitrate this time (ABR80).

Code: [Select]
ABC/HR Version 0.9b, 30 August 2002
Testname: ABR 80 - 3.90.3 v 3.95.1

1L = C:\My Music\lab\sin\ABR80-3.90.3.wav
2L = C:\My Music\lab\sin\ABR80-3.95.1.wav

---------------------------------------
General Comments:
The fist thiry seconds of "It's a Sin" by Pet Shop Boys
---------------------------------------
1L File: C:\My Music\lab\sin\ABR80-3.90.3.wav
1L Rating: 3.3
1L Comment: Though the distortion is still noticeable, overall this sounds much better than the other encode.
---------------------------------------
2L File: C:\My Music\lab\sin\ABR80-3.95.1.wav
2L Rating: 2.3
2L Comment: Distortion/chirpiness is a little bit more noticeable throughout the sample.
---------------------------------------
ABX Results:
Original vs C:\My Music\lab\sin\ABR80-3.90.3.wav
   8 out of 8, pval = 0.004
Original vs C:\My Music\lab\sin\ABR80-3.95.1.wav
   8 out of 8, pval = 0.004


I couldn't ABX between the two encodes, but when ABXing the original vs 3.95.1, it was much easier to notice the difference than the original vs 3.90.3.
Title: Lame 3.95 vs 3.90.3
Post by: ViPER1313 on 2004-01-17 20:49:04
ABC/HR Version 0.9b, 30 August 2002
Testname: BeautySlept --preset standard

1R = Z:\Music\Test Samples\testTemp\3.95.1 BeautySlept --preset standard.wav
2R = Z:\Music\Test Samples\testTemp\3.90.3 BeautySlept --preset standard.wav

---------------------------------------
General Comments:
I cant ABX this sample with either codec..... I think that sample 1 has somthing wrong with it but i cant ABX it with 100% certainty
---------------------------------------
ABX Results:
Original vs Z:\Music\Test Samples\testTemp\3.95.1 BeautySlept --preset standard.wav
    17 out of 26, pval = 0.084
Original vs Z:\Music\Test Samples\testTemp\3.90.3 BeautySlept --preset standard.wav
    4 out of 10, pval = 0.828
Z:\Music\Test Samples\testTemp\3.95.1 BeautySlept --preset standard.wav vs Z:\Music\Test Samples\testTemp\3.90.3 BeautySlept --preset standard.wav
    4 out of 7, pval = 0.500
Title: Lame 3.95 vs 3.90.3
Post by: Gecko on 2004-01-17 21:25:10
Tried one of my own samples at preset standard: "Cirillo - Cristallo" from the "Eye Trance 04" sampler. It has a relatively quiet section with sharp attacks which are slightly reverbed.

I blindly ranked 3951 lower but I later tried abxing it against 3903 without getting any significant results (I also felt like I was guessing). I stopped at 3/8.
3903.mp3 ... 635079 bytes
3951.mp3 ... 626224 bytes

Code: [Select]
ABC/HR Version 0.9b, 30 August 2002
Testname: cirillo lame aps test

1R = G:\NewMP3s\lame-aps\cirillo\3951_dec.wav
2L = G:\NewMP3s\lame-aps\cirillo\3903_dec.wav

---------------------------------------
General Comments:

---------------------------------------
1R File: G:\NewMP3s\lame-aps\cirillo\3951_dec.wav
1R Rating: 3.0
1R Comment: pre-echo worse than the other sample
---------------------------------------
2L File: G:\NewMP3s\lame-aps\cirillo\3903_dec.wav
2L Rating: 4.0
2L Comment: pre-echo
---------------------------------------
ABX Results:
Original vs G:\NewMP3s\lame-aps\cirillo\3951_dec.wav
   8 out of 8, pval = 0.004
Original vs G:\NewMP3s\lame-aps\cirillo\3903_dec.wav
   8 out of 8, pval = 0.004

[span style='font-size:8pt;line-height:100%']edit: was missing an "r"[/span]
Title: Lame 3.95 vs 3.90.3
Post by: AstralStorm on 2004-01-18 23:19:55
Quote
Testname: BeautySlept --preset standard

Where can I get the sample?
Title: Lame 3.95 vs 3.90.3
Post by: guruboolez on 2004-01-18 23:39:24
http://ff123.net/samples.html (http://ff123.net/samples.html)
http://ff123.net/samples/BeautySlept.flac (http://ff123.net/samples/BeautySlept.flac)