Skip to main content

Topic: LAME 3.96b1 vs. 3.90.3 Discussion (Read 16240 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • tigre
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
This thread has been split from a lame 3.90.3 vs. 3.96b1 test thread. It's for discussion about LAME 3.96b1 vs. 3.90.3, A new 'recommended version' for HA.org?! only. This thread is containing discussion related to the test too, so before posting here, you might want to read it.
  • Last Edit: 20 March, 2004, 08:45:13 AM by tigre
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

  • emtee
  • [*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #1
Shouldn't we wait for the developers to improve some of those regression examples? That topic had a lot of replies, and i'm sure the feedback will be useful to improve the codec. Maybe we should wait for a 3.96 beta 2 with some bugfixes before comparing it with the mighty 3.90.3, otherwise the regression thread was in vain.

  • john33
  • [*][*][*][*][*]
  • Developer
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #2
I believe I'm correct in saying that 3.96 is scheduled to go final at the end of this month and that any further commits will be bugfixes only and not any further tuning, etc., at this stage. Gabriel, correct me if I'm wrong.
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • dev0
  • [*][*][*][*][*]
  • Developer
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #3
Testing has to be done anyway and if another version is released meanwhile another testing thread will be opened, it's as simple as that.
"To understand me, you'll have to swallow a world." Or maybe your words.

  • amano
  • [*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #4
One problem is that with 3.96b bitrates aren't comparable with 3.90.3 anymore. With --preset 128 the difference between 116 and 138 kbps is quite significant. With --preset standard the differences are much bigger, up to 50 kbps.

Isn't there any chance for gabriel and the other devs to tune the presets to be comparable in size (at least the target kbps presets).

Other way I forsee the result that all samples are significantly smaller, but tend to sound worse. Then the results don't tell anything because the quality/size ratio is what counts and can only be guessed that way.
  • Last Edit: 19 March, 2004, 11:09:13 AM by amano

  • dev0
  • [*][*][*][*][*]
  • Developer
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #5
I assumed that --preset standard is still ment to be transparent on 99% of all samples, just like 3.90.3 --alt-preset standard.
The bitrate is a rather uninteresting factor here.
  • Last Edit: 19 March, 2004, 11:23:50 AM by dev0
"To understand me, you'll have to swallow a world." Or maybe your words.

  • kwanbis
  • [*][*][*][*][*]
  • Developer (Donating)
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #6
Quote
I assumed that --preset standard is still ment to be transparent on 99% of all samples, just like 3.90.3 --alt-preset standard.
The bitrate is a rather uniniteresting factor here.

totally agree on that one

  • evereux
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #7
Quote
I assumed that --preset standard is still ment to be transparent on 99% of all samples, just like 3.90.3 --alt-preset standard.
The bitrate is a rather uninteresting factor here.

I whole-heartedly dissagree. I don't think it's a good idea to have a new recommended Lame version that acheives transparency at the cost of file size (if that is indeed what happens on average) in comparison to an encoder that acheives transparency with a smaller file. Especially when that Lame version has many samples where it has regressed from 3.90.3 even in it's wrongly compiled state?
daefeatures.co.uk

  • dev0
  • [*][*][*][*][*]
  • Developer
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #8
If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!
"To understand me, you'll have to swallow a world." Or maybe your words.

  • Fede
  • [*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #9
It's really progression to compress with the same sound quality at higher bitrate? It's progression to reach the same quality we can have @192kbps with the 3.90.3 at 256 kbps with another encoder???? If the answer is yes I think there's many other encoder working good.
We are talking about reach the best quality @ best size, if size isn't a priority go for lossless.

Just my 2 cents.

  • evereux
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #10
I see some samples are being tested with 3.90.2 (the user hasn't stated whether -Z was used) and not 3.90.3. Whilst this goes against the test requirements I actually think this is a better idea since we have little idea of what effect the compilation issues has had on 3.90.3 (I'm referring to this discussion).
daefeatures.co.uk

  • [proxima]
  • [*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #11
Quote
I see some samples are being tested with 3.90.2 (the user hasn't stated whether -Z was used) and not 3.90.3.

if you are referring to me, i tested 3.90.2 only with abr 128 presets, so -Z is not included in 3.90.3.
I've only tested rebel.wav with --aps -Z and 3.90.2 and i specified this in my post.
Quote
Whilst this goes against the test requirements I actually think this is a better idea since we have little idea of what effect the compilation issues has had on 3.90.3

I think compilation issues should be very small since john33 compile include the same switches used by Dibrom. Nevertheless i agree with you, this is the reason because i tested 3.90.2 (with -Z when needed)
  • Last Edit: 21 March, 2004, 08:38:13 AM by [proxima]
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1

  • tigre
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #12
Some clarifications about ABX tests in this test:

Especially at low bitrates like ~128kbps many artifacts are so obvious that ABXing appears as waste of time, but even if it doesn't give the tester any additional security about the result, ...
- it gives security to others that a result is valid AND
- you can be sure that artifacts in one encoded version are really different from the ones in another version if you ABX two encoded versions against each other

To prove that one encoded version is worse than the other in such obvious cases, it should be enough to ABX
- Original vs. worse sounding encoded version
- better sounding vs. worse sounding encoded version

ABC/HR results without ABXing (especially the encoded versions that seem to be different against each other) can't be used for public tests like this because we need at least a minimum security (=ABX results) that the individual results of the participants are reliable. It would be different, if a big number of participants submitted ABC/HR results of the same sample(s). In this case the results could be evaluated using a statistical analysis similar to rjamorim's tests.
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

  • Jojo
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #13
Quote
I see some samples are being tested with 3.90.2 (the user hasn't stated whether -Z was used) and not 3.90.3. Whilst this goes against the test requirements I actually think this is a better idea since we have little idea of what effect the compilation issues has had on 3.90.3 (I'm referring to this discussion).

I agree to that in some extend. If the goal was just to get the closest to transparent the recommended setting should be --preset 320
But then, --preset 128 offers the best ratio of quality and size...so it's a really tricky question. However, I use LAME 3.95.1 since it uses lower bitrates than LAME 3.92 or LAME 3.90.3 in most cases. If Lame 3.96 is found to be just as good as LAME 3.90.3 it should be recommended since it's faster and uses as lower bitrate.
--alt-presets are there for a reason! These other switches DO NOT work better than it, trust me on this.
LAME + Joint Stereo doesn't destroy 'Stereo'

  • Jojo
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #14
Quote
If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!

Why isn't --preset 320 recommended then 
--alt-presets are there for a reason! These other switches DO NOT work better than it, trust me on this.
LAME + Joint Stereo doesn't destroy 'Stereo'

LAME 3.96b1 vs. 3.90.3 Discussion
Reply #15
Quote
Quote
If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!

Why isn't --preset 320 recommended then 

Because "--preset-insane" wastes bitrate without much gain in quality at all. --preset-standard is designed to achieve transparency on a vast majority of samples at the lowest bitrate possible. Transparency doesn't mean "kinda close". It means "transparent".

  • evereux
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #16
Quote
Quote
Quote
If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!

Why isn't --preset 320 recommended then 

Because "--preset-insane" wastes bitrate without much gain in quality at all. --preset-standard is designed to achieve transparency on a vast majority of samples at the lowest bitrate possible. Transparency doesn't mean "kinda close". It means "transparent".

His point was, when do we draw the line and say that these bit-rates are becoming too inflated for a standard preset? Transparency in the true sense of the word will never be reached.

That being said, I'm in the process of encoding around 20GBs of wav files and the file size increase doesn't look alarming at all. I'll be more specific when the encoding is complete.
daefeatures.co.uk

  • evereux
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #17
Quote
,Mar 21 2004, 01:37 PM]
Quote
I see some samples are being tested with 3.90.2 (the user hasn't stated whether -Z was used) and not 3.90.3.

if you are referring to me, i tested 3.90.2 only with abr 128 presets, so -Z is not included in 3.90.3.
I've only tested rebel.wav with --aps -Z and 3.90.2 and i specified this in my post.

You are quite right, my apologies.
daefeatures.co.uk

  • evereux
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #18
I've encoded 22.4GB of wav files using the LAME versions 3.96b1 (--preset standard) and 3.90.3 (--alt-preset standard). The resultant directory sizes are as follows:

3.90.3 = 3.24GB
3.96b1 = 2.99GB

I can provide an album list if required (I first have some sorting to do to ease the compilation of it).
daefeatures.co.uk

  • Jebus
  • [*][*][*][*][*]
  • Developer
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #19
Quote
Quote
Quote
Quote
If it saves bitrate that's a nice bonus, but the new recommended version should be at least as transparent using --preset standard as 3.90.3 is now.
Sacrificing quality for bitrate is not an option!

Why isn't --preset 320 recommended then 

Because "--preset-insane" wastes bitrate without much gain in quality at all. --preset-standard is designed to achieve transparency on a vast majority of samples at the lowest bitrate possible. Transparency doesn't mean "kinda close". It means "transparent".

His point was, when do we draw the line and say that these bit-rates are becoming too inflated for a standard preset? Transparency in the true sense of the word will never be reached.

That being said, I'm in the process of encoding around 20GBs of wav files and the file size increase doesn't look alarming at all. I'll be more specific when the encoding is complete.

no, your definition of "transparency" is wrong. Transparency does not mean bitwise equivelency, it means the files are audibly indistinguishable. This is the case with --alt-preset standard, or at least the intention. This is ALSO the case with --alt-preset insane, but the extra bits don't make the files sound any better since transparency was already acheived using --alt-preset standard.

  • evereux
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #20
Quote
no, your definition of "transparency" is wrong. Transparency does not mean bitwise equivelency, it means the files are audibly indistinguishable. This is the case with --alt-preset standard, or at least the intention. This is ALSO the case with --alt-preset insane, but the extra bits don't make the files sound any better since transparency was already acheived using --alt-preset standard.

I didn't define transparency, I said in the true sense of the word. Using your analogy, what then is the point of the insane preset? I still think the whole point is being missed and for me this discussion is redundant now anyway.
daefeatures.co.uk

  • evereux
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #21
Quote
I've encoded 22.4GB of wav files using the LAME versions 3.96b1 (--preset standard) and 3.90.3 (--alt-preset standard). The resultant directory sizes are as follows:

3.90.3 = 3.24GB
3.96b1 = 2.99GB

I can provide an album list if required (I first have some sorting to do to ease the compilation of it).

Futher to this:

comparison

Be aware that the table is 412kb, I couldn't find a better way to trim the size down (html tables are crap, or at least my knowledge of them).

Generated from Encspot output original files here.
daefeatures.co.uk

  • tigre
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #22
Thanks evereux.

If possible, could you (or anyone else) please do a similar test with 3.96b1 VBR?

With 3.90.3 there are no recommended VBR settings for quality lower then --alt-preset standard -Y because ABR sounds better in most cases at comparable bitrate. AFAIK Gabriel has changed a lot to impove -V 3 and lower, so this could lead to 3.96b1 VBR replacing 3.90.3 ABR as recommended lame compile/settings combination for < 192kbps bitrates. To make ABR vs. VBR tests useful, we need some test similar to the one you did, evereux.

In theory V-settings should give those average bitrates:
-V 5 : 128kbps
-V 4 : 144kbps
-V 3 : 160kbps
-V 2 : 192kbps

With the samples I tried so far (mainly -V 5) the numbers have been 10-20kbps higher on average. So could you please do some mass-encoding at -V 2 - -V 6, maybe a (representative) part of the tracks you used for (preset)standard bitrate comparison is enough...
Let's suppose that rain washes out a picnic. Who is feeling negative? The rain? Or YOU? What's causing the negative feeling? The rain or your reaction? - Anthony De Mello

  • LoFiYo
  • [*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #23
I think there should be an official guideline for a recommended version of LAME since HA will be endorsing it as its official recommendation. In other words what do HA admins consider a superior version of LAME to the current recommended version (3.90.3)?

I would think:
Version X is superior to Version Y when the sound quality of the encoded file is found to be higher 9 times (or more) out of 10 in all preset modes (from insane down to abr/cbr
  • kbps).

    What would be the official HA stance on this? I am asking this, because we can keep testing and testing many many samples, but if there isn't a definition as to what constitutes a superior version, the test might never end...

  • evereux
  • [*][*][*][*][*]
LAME 3.96b1 vs. 3.90.3 Discussion
Reply #24
Quote
With the samples I tried so far (mainly -V 5) the numbers have been 10-20kbps higher on average. So could you please do some mass-encoding at -V 2 - -V 6, maybe a (representative) part of the tracks you used for (preset)standard bitrate comparison is enough...
Sure, I can do that. I'll start at V6, work my way down the numbers and post my results as they're completed.

If anyone else would like to start at V2 V3, please do so.

edit: V2 V3 was V2
  • Last Edit: 23 March, 2004, 11:22:22 AM by evereux
daefeatures.co.uk