Skip to main content

Topic: LAME 3.100 alpha testing (Read 52190 times) previous topic - next topic

0 Members and 2 Guests are viewing this topic.
  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
New build of LAME  3.100 alpha is available on http://www.rarewares.org/mp3-lame-bundle.php

Changelog
  • Last Edit: 08 November, 2012, 12:26:32 AM by kode54

  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #1
As of bitrate for current  alpha 2 it's (3.99.5 V5)<(3.100a2 V5 )<(3.99.5 V 4.999)

So I tried one sample with these settings.
[!--sizeo:2--][span style=\"font-size:10pt;line-height:100%\"][!--/sizeo--]
Code: [Select]
ABC/HR Version 1.1 beta 2, 18 June 2004
Testname:

1R = D:\Audio\Test samples\09 Fugue Prem. Notes\lame 3.100a2\sample09 3.99.5 V5.wav
2R = D:\Audio\Test samples\09 Fugue Prem. Notes\lame 3.100a2\sample09 3.99.5 V4.999.wav
3R = D:\Audio\Test samples\09 Fugue Prem. Notes\lame 3.100a2\sample09 3.100a2 V5.wav

---------------------------------------
General Comments:

---------------------------------------
1R File: D:\Audio\Test samples\09 Fugue Prem. Notes\lame 3.100a2\sample09 3.99.5 V5.wav
1R Rating: 2.5
1R Comment:
---------------------------------------
2R File: D:\Audio\Test samples\09 Fugue Prem. Notes\lame 3.100a2\sample09 3.99.5 V4.999.wav
2R Rating: 2.8
2R Comment:
---------------------------------------
3R File: D:\Audio\Test samples\09 Fugue Prem. Notes\lame 3.100a2\sample09 3.100a2 V5.wav
3R Rating: 2.8
3R Comment:
---------------------------------------
ABX Results:
D:\Audio\Test samples\09 Fugue Prem. Notes\lame 3.100a2\sample09 3.99.5 V5.wav vs D:\Audio\Test samples\09 Fugue Prem. Notes\lame 3.100a2\sample09 3.99.5 V4.999.wav
   5 out of 5, pval = 0.031


[/size]


  • halb27
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #2
I tested 3.100a2 at the -V2 quality level for lead-voice, trumpet_myPrince, harp40_1, and eig_essence (samples are here).

lead-voice which is real bad with 3.99.5 -V2 has improved so much that IMO the result is in accordance with what -V2 users expect: transparency usually and no real obvious issue even on rare occasion.

Also with trumpet_myPrince the 3.100a2 result is better than that of 3.99.5. I was able to ABX the difference.

Same goes for harp40_1.

The obvious spot around sec. 3.0 of eig_essence however got a bit worse with 3.100a2. I was able to ABX the difference.
  • Last Edit: 08 November, 2012, 05:42:13 AM by halb27
lame3995o -Q1

  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #3
halb27,

Before anything. Does 3.100 -V2 comparable bitrate to 3.99.5 V2 on large number of files?
  • Last Edit: 08 November, 2012, 09:34:13 AM by IgorC

  • lameboy
  • [*]
LAME 3.100 alpha testing
Reply #4
How is the LAME header written in 3.100? (I'm using a Mac so I can't test the new alpha binary)

There was some controversy when 3.99.1 used "L3.99r1" instead of the standard "LAME3.99" causing some programs not to correctly identify LAME encodes (this was fixed in 3.99.2).
Max number of characters is 9. My suggestion was "LAME31001" for an eventual 3.100.1 release.

LAME 3.99 discussion

XLD // ALAC // OGG VORBIS

  • halb27
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #5
... Does 3.100 -V2 comparable bitrate to 3.99.5 V2 on large number of files?

Average bitrate is comparable though a tiny bit higher: for my standard test set of various pop music 3.99.5 -V2 takes an average of 190 kbps, for 3.100a2 -V2 it's 195 kbps. For -V0 it's 260 kbps (3.99.5) vs. 264 kbps (3.100a2).
lame3995o -Q1

  • Dynamic
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #6
So I tried one sample with these settings.


I'll share my findings, which in relation to 3.100.alpha2 back up what IgorC ABXed.

I tried the same Fugue_Premikres_notes sample as IgorC and was able to ABX the 3.100.alpha2 -V5 version (5/5 ABX). I realised this is another case of the transient+tonal problem sample class (harpsichord is usually going to be like this, and unlike guitar, you'll never expect to hear pitch-bending in a harpsichord so it can't pass as part of the music).

I was tempted to try halb27's high-bitrate short-block method (-Vn+) for this sample to see if it helped, and I still have an old (not final) version of 3.99.5y that unintentionally allowed -V5+ to be used.

I used 3.99.5yFinal to encode -V5 and 3.99.5yNotFinal to encode -V5+ and both resulting MP3 files are attached in the Upload thread along with my fb2k ABX logs. (There's a 3.99.5z version now, but the approach is slightly different).

Neither was transparent (5/5 ABX), but I felt -V5+ was better so to be sure I heard a difference, I ABXed -V5 against -V5+ and while this was harder (somewhat noisy environment today didn't help) and both had some wavering in the pitch of the harpsichord notes (esp the last note was nice and clean on the original, and cleaner on the -V5+ version, so I focused there). I knew this was tougher, so went for 10 trials, and got 9/10 ABX.

To try to describe the difference, in essence after the plucking transient, the pitch of the original lossless harpsichord was immediately a strong consistent tone. In the -V5+ version the tone seemed to come in about right then waver slightly before becoming consistent in pitch. In the -V5 version it seemed come in less precisely on-pitch and to waver a little before the bit of wavering that I picked out in the -V5+ version, so it wavered overall a little longer, seeming like about two mild pitch-bends or tremolo oscillations rather than one.

It was easy to ABX a strong consistent tone in the lossless against any of the tones that had a wavering, poorly defined starting pitch, but more difficult to focus on the duration and nature of the wavering pitch between the -V5+ and -V5 versions.

I know this is only loosely about 3.100.alpha2, which is supposed to improve the transient-tonal problems in lead-voice for example but clearly isn't enough to cope entirely with harpsichord - which is also both highly tonal and high in sharp transients.

If I get time, I might try to compare 3.100.alpha2 to 3.99.5yNotFinal -V5.
Dynamic – the artist formerly known as DickD

  • halb27
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #7
For a fair comparison (in terms of average bitrate) 3.99.5yNotFinal -V5+ should be matched against a 3.100.a2 setting like -V3 IMO.
Moreover I'd prefer if you used a newer version than 3.995.yNotFinal, for instance 3.99.5z -V3+, or 3.995c or newer if you want to use -V5+ or similar. In contrast to these I have no information about the internal parameters available for 3.995.yNotFinal because this version erraneously provided -V5+.
  • Last Edit: 08 November, 2012, 10:31:14 AM by halb27
lame3995o -Q1

  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #8
Average bitrate is comparable though a tiny bit higher: for my standard test set of various pop music 3.99.5 -V2 takes an average of 190 kbps, for 3.100a2 -V2 it's 195 kbps. For -V0 it's 260 kbps (3.99.5) vs. 264 kbps (3.100a2).

Such difference in bitrate makes actually difference.
That's OK  if some samples end up with higher bitrate and other with lower, but 3.100a2 V5/V2 isn't comparable 3.99.5 V5/V2 because first one always ends up with higher bitrate.

Otherwise we're comparing oranges vs elephants.


  • Dynamic
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #9
Yes, I should have made it clear that I was really just using that not Final 3.99.5y version just to investigate whether the short-block high bitrate approach would be enough to fix or improve the frequency resolution problem or whether some of it might well be outside the short blocks themselves (I believe the more recent versions of halb27's LAME extension also modify long-block behaviour, so I chose the old version that leaves them alone).

It wasn't supposed to be a like-for-like comparison of comparable bitrates but an enquiry into what part of the problem was in the short blocks. I'm confident that -V5+ improves it, so if I have enough time, I may try to find out what's required to make it transparent, if transparency is even possible with harpsichord sounds like this (and from my past reading I think that transient smearing at least might be somewhere that MP3 can't quite provide transparency).

It may well be that also the marginal masking decisions made by the -V5 thresholds (as opposed to -V3 or -V2 where they tend to be tuned to ensure quite robust transparency, with just a few exceptions) are allowing too much inaccuracy in quantization during the long blocks as well, which might be why the wavering goes on a little after the short blocks for the transient should have finished, and that these issues could be solved by going to a higher quality setting (lower -V number), while the short blocks might require the + mode in addition to continue to settle quickly on the accurate tone frequency).

For general encoding with high robustness against transient+tonal problems, I'd certainly recommend people look into halb27's latest versions in the lame 3.99.5z functional extension thread.
Dynamic – the artist formerly known as DickD

  • Porcus
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #10
IgorC: 190 vs 195 is oranges vs elephants ... seriously? Or did you read hundredandninetysomething vs twohundredandsixtysomething?

If it only takes a few sample signals to find one where 3.100 -V2 is ABXably better than 3.99 -V2 at the cost of 2.6 percent increase, then isn't that at this level a (small and preliminary) indication of a pretty good job? After all, the V of VBR is about knowing where to spend more bits.
  • Last Edit: 08 November, 2012, 11:23:58 AM by Porcus

  • halb27
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #11
Average bitrate for my test set using -V5:
3.98.4: 138 kbps
3.99.5: 128 kbps
3.100a2: 133 kbps
lame3995o -Q1

  • halb27
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #12
I finished my -V2 listening test with herding_calls and trumpet.

I could ABX herding_calls easily using 3.99.5. I was not able to ABX the 3.100a2 result.

I could also ABX the 3.99.5 result of trumpet (though I don't know why because today I could not consciously spot a difference). I could not ABX the 3.100a2 result.


So judging from the samples I care about: this is a great progress concerning tonal issues compared to previous Lame versions.
  • Last Edit: 08 November, 2012, 11:57:29 AM by halb27
lame3995o -Q1

  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #13
Average bitrate for my test set using -V5:
3.98.4: 138 kbps
3.99.5: 128 kbps
3.100a2: 133 kbps

5 kbps make difference. See my abx log in this thread between v4.999 and v5.
  • Last Edit: 08 November, 2012, 11:59:48 AM by IgorC

  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #14
IgorC: 190 vs 195 is oranges vs elephants ... seriously? Or did you read hundredandninetysomething vs twohundredandsixtysomething?

If it only takes a few sample signals to find one where 3.100 -V2 is ABXably better than 3.99 -V2 at the cost of 2.6 percent increase, then isn't that at this level a (small and preliminary) indication of a pretty good job? After all, the V of VBR is about knowing where to spend more bits.

1.
It's not about a few kbps for some samples. It's for ALL samples.

2.Who guarantee that quality improvement come from smarter algorithms and not from pure bitrate increase?
  • Last Edit: 08 November, 2012, 12:08:50 PM by IgorC

  • halb27
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #15
Who guarantee that quality improvement come from smarter algorithms and not from pure bitrate increase?

A 5 kbps difference really doesn't change quality because of this small increase in bitrate. Please do think of bitrate difference in terms of average bitrate difference for a representable set of full length regular tracks. If bitrate is significantly higher with problem sample snippets this is very welcome! And look at 3.98.4's average bitrate in my post above.
Your ABX log was about a rather modest improvement in quality. Maybe 5 kbps difference is sufficient to make up for this quality increase.
With the samples I tested there is a severe quality improvement however, and cannot be contributed to a 5 kbps higher average bitrate. But we do know that robert significantly improved on the machinery, the small increase in average bitrate is just a side effect of this.
  • Last Edit: 08 November, 2012, 12:49:36 PM by halb27
lame3995o -Q1

  • halb27
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #16
I tested 3.100a2 -V0 for the samples that were not transparent using -V2:

eig_essence: The rather obvious issues around second 3.0 are easily ABXable, but at least 3.100a2's -V0 quality is on par to me with the quality of 3.99.5.

lead-voice: With the very good -V2 result I was a bit surprised that I could still ABX the -V0 result of 3.100a2 rather easily. It's a small issue at second ~1.8 that enables me to do so.

harp40_1: I had a pretty hard time ABXing but I succeeded. I'd call the result 'close to transparent' however, and we probably cannot expect to get a transparent mp3 result.

trumpet_myPrince: ABXing as with harp40_1, so a 'close to transparent' result.
  • Last Edit: 08 November, 2012, 12:47:16 PM by halb27
lame3995o -Q1

  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #17
A 5 kbps difference really doesn't change quality because of this small increase in bitrate.

Third time I'm mentioning my previos abx log.

For that particular sample  3.100a2 V5  ends up with +4.4% extra bitrate comparing to 3.99.5 V5.  Meh, +4.4% big deal?!
Now when I've tried 3.99.5 V4.999 that ends up with +1.9% extra bitrate comparing to 3.100a2. Meh, +1.9% big deal again?!
But the "improvement" of 3.100a2 has evaporated.

If You could just notice it was a matter of a few kbps during all this time.

  • halb27
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #18
OK, so the -V5 quality improvement of this sample may be explainable by the bitrate difference.

Nonetheless the important question is: is there a noticeable progress for several problematic tracks? If yes, I wouldn't care about a small average bitrate increase.
With respect to my experience with -V2 the answer for me is: yes (and at 190 kbps a 5 kbps increase is even more negligible).
  • Last Edit: 08 November, 2012, 01:42:26 PM by halb27
lame3995o -Q1

  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #19
if 5 kbps doesn't matter then try 3.99.5 V1.9 or V1.8 and 3.100a V2.
Probably 3.99.5 V1.8 will end up with extra +5 kbps.
And I can say for sure LAME 3.100 (at least a2) won't present a quality improvement anymore.
Wanna try?

  • halb27
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #20
3.99.5 -V1.85 yields an average bitrate of 195 kbps for my test set and so is a good match to 3.100a2 -V2.
I did a quick comparison.
lead-voice is clearly better with 3.100a2 (no ABXing necessary).
trumpet_myPrince is clearly better too, though not as obvious, but I could ABX the difference.
The improvements of 3.100 are targeting at tremolo/ringing issues. So a significant improvement in this field is reason enough IMO to appreciate 3.100.
  • Last Edit: 08 November, 2012, 03:17:29 PM by halb27
lame3995o -Q1

  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #21
Thank You for understanding, halb27.

MPEG tests mention <1% for difference between total average bitrates. http://mpeg.chiariglione.org/quality_tests.php
And it's not craziness. It's really for a good reason. Those guys know what they do.

I will try some tonal ... and transients samples as well. Will report it later.
  • Last Edit: 08 November, 2012, 04:07:52 PM by IgorC

  • shadowking
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #22
As long as problematic samples benefit then slight overall bitrate increase is justified IMO.  If the psymodel is more robust that should not turn people off that there is 6% increase. So far its clear that from v3.98 is more defensive and 3.99 / 3.100 even more so . V1 / V0 produce higher bitrate and i have tested the other day some problem samples of the past and the quality is near transparent and thats what matters to me .  In the past  there was little to zero tunings on anything other than V2 or --preset standard. Going to --preset extreme often did nothing. Now there is a proper VBR scale that works.
wavpack -b4x4s1c

  • IgorC
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #23
As long as problematic samples benefit then slight overall bitrate increase is justified IMO.  If the psymodel is more robust that should not turn people off that there is 6% increase.

Most of people will agree with this wiki info.  It's the idea of VBR. But that should not turn off people to try  two versions of encoder with different vbr settings simply because it doesn't end up with the same bitrate anymore as previously.
It creates an illusion of endless improvement untill you get 300 kbps for v2 (?)


Shortly it's not a problem if an encoder increase bitrate on its hard samples, the problem is when encoder just use more bitrate for any kind of samples and people confuse this situation with the first scenario, buying it as improvement.
  • Last Edit: 08 November, 2012, 09:17:23 PM by IgorC

  • shadowking
  • [*][*][*][*][*]
LAME 3.100 alpha testing
Reply #24
As i can tell the recent encoders are using mega-bitrates with the new vbr code on samples like fatboy / emese even at V5. But not on 'normal' music. This is desirable behaviour for VBR.  Obviously V5 / V4 users are sensitive to bloat . But if V3 becomes 180k I don't see that as an issue , like wise if V1 / V0 are 250..300 that's fine for 'overkill' settings.
  • Last Edit: 08 November, 2012, 10:14:02 PM by shadowking
wavpack -b4x4s1c