Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Plans For --alt-presets, Etc, In Lame 3.93 (Read 47462 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #25
Quote
Ok... here's the binary, as promised

Before starting the process the program freezes the computer for a few seconds and then outputs the following error message: LAME: Can't find termcap entry for terminal "cygwin". After that the encoding starts as usual.

I encoded three files (classical music). Most notably, in all three cases there was nearly no difference in file size between aps and medium (both with tak3lame), less than 1kbit/s. Medium was even larger by a few bits! I suppose this is caused by the lack of strong HF in the samples.

The first sample I listened to revealed a flaw in portable and radio:
3.92 ap 170 - 169 kbit
tak3 ap portable - 163 kbit
tak3 ap radio - 150 kbit
3.92 ap 150 - 148 kbit

While 170 is nearly transparent to me (maybe noise problems and slight pre-echo) and 150 has only minor noise problems, the tak3 presets produce a disturbing "rumble" in the background. Medium seems to fix this.

The sample is available in this thread, the problem is most noticable at sec 5-6, just before the tenor comes in.

 

Plans For --alt-presets, Etc, In Lame 3.93

Reply #26
Would it be also possible to test the "--preset medium" that is in the current 3.93 alphas for comparison?

Plans For --alt-presets, Etc, In Lame 3.93

Reply #27
For information, mail exchange between Alexander and me:


> Gabriel Bouvigne wrote:
> > I am discussing if we should release now or wait a little more with people
> > on the Hydrogenaudio forum.
> > I will give you a summary of things discussed soon.
>
> I will try to have a look at the forum today.
>
> > Basically Dibrom would like to wait a lot more in order to have new
> > features, and I am trying to reach a consensus focusing on a release date
> > around nov, 15th, with less features in order to comply with this date.
>
> I'm thinking about a release as soon as possible and about another
> (beta) release soon after 3.93. Takehiro has a lot of work in his branch
> and we may want to merge it (depending on what Takehiro thinks about it)
> and make a beta release... What do you think about this?

I was also initially thinking about a release as soon as possible.
But Dibrom and Takehiro have some valid points: The current fast presets are a little broken.
This point can be considered as a stopping factor. (Without it I would have no doubt about a release now)
We would need to fix it before releasing.

However, Takehiro's branch is hanlding fast presets a little better than the main branch. So this would be extra work to fix fast presets in the main branch, and this would be a "waste" if we merge Takehiro's branch just after.

So my suggestions is:
*merge Takehiro's branch now
*focus on fixing current presets
*focus on a release aroun nov, 15th (1 extra month due to the merge of branchs and the need for testing)

If there is enough time, perhaps introduce new presets, but this would not be a priority.


(I am cc'ing this mail to lame-dev and HA forum)

Regards,

----
Gabriel Bouvigne
www.mp3-tech.org
personal page: http://gabriel.mp3-tech.org

Plans For --alt-presets, Etc, In Lame 3.93

Reply #28
Hi,

this is my first post here, so please forgive me if I made some mistakes and it doesn't look as good as it could.

Why I want to make a new release:
1) we have some bugfixes in the code:
    This is beneficial for every user.
2) we have some bugfixes in the configure script for unix users
    This is beneficial for every user of gcc 3.x and for some users
    with commercial unix derivates.

I don't expect much users with a commercial unix here in the forum, and I don't know the Windows:Linux ratio here, but both doesn't matter. We have some users which may benefit from a new release so we should make a new release.

Why I don't want Takehiro's code in 3.93:
At the moment we have code which is proven to be "stable" (about the fast preset issue later), and we have some fixes, which make the previus release a little bit better. Even if I have some trust in the work Takehiro does, his code isn't as tested as the actual code is. It doesn't hurts if we make a mature release yet, and another release with Takehiro's code shortly after that (I'm thinking about making the release 3.93 release as soon as possible and making a 3.94beta release after Takehiro's code is merged).
I don't want to bitch on Windows here (even if I'm more of a Unix guy, I'm actually writting this from a W2k system), but I have the impression that "Windows-guys" think about new releases as of "Yeah! Features! Features!" and personally I don't like this. I don't need features which aren't usable because of bugs (either because they aren't really tested or because the management decided to make a release). I don't want to make a "Features"-release, I want to make a solid-release.

The code of 3.92 can't be that bad , it's in widespread use. So why not release a bugfixed version of 3.92 (namely: the actual code in the HEAD CVS branch as 3.93) and make a known good release even better.

So here's my proposal:
1) fix the preset fast issue in the actual code
2) make a 3.93 release
3) merge Takehiro's code
4) make a 3.94beta release (if we are able to merge the code in one day I absolutely have no problem to go public with 3.94beta one day after the 3.93 release)
5) fix bugs which may show up in 3.94beta
6) add other presets / improve existing presets
7) make a 3.94 release (or another beta)

This way we satisfy both kinds of users, those which like to have rock solid software, and those which don't mind a bug or two in bleeding edge software.

Quoting Dibrom:
Quote
I don't really see the point in making a release just for the sake of having a new release in X amount of time. Releases should be dictated by the merit of their improvements otherwise it's just arbitrary and could easily cause more problems than it fixes.

Yes, I want to make an improvement over 3.92. 3.92 is stable, so an imroved version has to be stable too. Therefore I don't think we should release 3.93 with Takehiro's code (it's not as tested as the actual one).

Quote
Making a release without proper testing would be very irresponsible IMO. LAME has traditionally suffered very poor QA, let's not continue the trend if possible.

Yes, please let's not continue... the actual code has a lot of (life) testing, there are mostly bugfixes compared to 3.92, whereas Takehiro's code has a lot of changes which don't have such a large testing as the actual code has, the actual code has a very large user base.

Quote
I don't understand what you mean about "when a program need to be able to run under a whole range of different operating systems." Does 3.92 have some major problems preventing it from running on most major operating systems?

I don't like the word "major" in your sentence. There's more than Windows (or Linux). 3.92 has some problems on some systems which are fixed in 3.93.

Quote
The only thing I see in the changelog so far are portability fixes for Tru64 Unix and SunOS 4.x. How many people are those fixes going to affect vs how many people poor quality testing could affect?

There are some changes which aren't in the official ChangeLog on the website, those changes are only listed in the CVS log (or in the file ChangeLog in the LAME tree).
There are some code fixes, which result in a problem with the preset fast setting, but thats only problem. Every new (not as much tested as the "old" code) code is in a new feature (e.g. substep noise shaping) and needs to get explicitely activated (I don't think about bad quality in those settings as a problem). So we have a very large portion of the code with a lot of field testing (because it's the same code as in 3.92). If there are quality problems in the new code, don't use it.

Quoting Gabriel:
Quote
However, Takehiro's branch is hanlding fast presets a little better than the main branch. So this would be extra work to fix fast presets in the main branch, and this would be a "waste" if we merge Takehiro's branch just after.

I don't think so. As already said several times, the actual code has a lot of field testing, whereas Takehiro's code is only tested by a small amount of people. I think it will be less work to do to fix the actual fast preset than to do quality testing on Takehiro's code. Yes, we would have to also do the same work in Takehiro's code, but as I understand Diprom he want's to use some of the new features in the settings, so he has to look at every preset regardless of the status (buggy/not buggy) of the preset. BTW.: IMHO the actual presets should only get bugfixes, and shouldn't use new features for 3.93. We can make an all dancing, all singing 3.94beta with every improvement active developers come up with, but I like to have the 3.93 release to be known as a rock solid release (same quality, less bugs -- not counting bugfixes which affect quality in a positive way).


Think about 3.93 as a bugfix release, not of a release with new features (even if we have them). And remember, I'm fine with making a 3.94beta with every feature/change you can come up with at the same day.

Bye,
Alexander.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #29
There are 2 kinds of QA with Lame: portability QA, and quality QA.

I think that now, the portability one is ok. If we need to release (as a stable release) now, I think that this is possible.
The quality of presets does not seems worse than in previous release, only bitrate is higher in some cases (although not as higher as it has been).

If you have any concern about the medium preset being included (although I do not plan to document it yet) in a release, I can disable it if you want, we'll put it back in 3.94.

If 3.93 needs to be a release, then of course I do not think we should merge Takehiro's branch before.

About releasing a 3.94beta resulting from merging of branchs soon after 3.93 release, I agree.


This way we will have something like:
3.93 Stable release
3.94 beta with Takehiro's branch
3.95 alpha for new tweaking, new presets, ...


ps: Dibrom, any comment?

Plans For --alt-presets, Etc, In Lame 3.93

Reply #30
Alexander:

I'm not an expert, but I'd agree with all of that. Release a "safe" 3.93 final (including the fixes for the fast presets), and then make the "all dancing, all singing 3.94beta with every improvement active developers come up with" for us guys to test.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #31
Quote
tak3lame.exe seems to use qval=3 as standard. Should I use presets with -h?

No, Tak said that in this build -q3 is identical to -q2 in 3.92.
Juha Laaksonheimo

Plans For --alt-presets, Etc, In Lame 3.93

Reply #32
I'm posting this here because I'm not sure which of the two lame-dev mailing lists I should be using (sourceforge or minnie.tuhs.org). I'll repost if someone tells me the right one ...

I submitted some changes to the LAME configure script for Sun Ultrasparc + GCC 3.2 that resulted in a pretty large performance improvement (about 30%) on those systems. These changes went into 3.93.

Since then, there has been some pretty bad performance degradation, the Ultrasparc-optimized 3.93 is more than twice as slow on the average as optimized 3.92. This is using --alt-preset standard and cbr modes.

I'm not sure what happened, since the x86 users are seeing an improvement. Is there anything that can be done? I can provide an UltraSparc test system if necessary.

Thanks
--jth

Plans For --alt-presets, Etc, In Lame 3.93

Reply #33
uum, seems I am too much delayed to notice this thread.

What I can say is .... my branch contains too much experimental things and it changes everyday. Still more I have many pending patches. It's almost "before alpha" stage and will take very long to be "beta" stage.

It's not time to QA my branch. We LAME development team suffer from short of resources. Testing my branch "NOW" is only wasting resources. We should test the head branch.

I don't recomment to merge my branch into main, before releasing 3.93. Some of you say 3.93 will be triial, but 3.93 contains some of speed up(at least for me) and bug fix. It is worse releasing new version IMHO.

btw, for the people interested in my pending patches: This is it.

- impulse like signal detection for long/short block switching improvement.
- better temporal masking.
- more faster VBR.
- intensity stereo.
- mixed block support.
- make nspsy faster/"always" better than gpsycho and remove gpsycho.
- improve "fast" preset quality and remove "normal" preset.
- re-remapping -q option with safen the substep noise shaping.
- new and faster MDCT code.
- SSE/3DNow! FFT/quantization code
- default to use float instead of double
- integrate ssrc resampling code

There's no reason about the order.
May the source be with you! // Takehiro TOMINAGA

Plans For --alt-presets, Etc, In Lame 3.93

Reply #34
Quote
If you have any concern about the medium preset being included (although I do not plan to document it yet) in a release, I can disable it if you want, we'll put it back in 3.94.

I don't care if you add your new preset or not. It's not critical code. Talk with Dibrom about this, he seems to have some kind of "fear" (sorry, I don't know how o describe it better in english) this pollutes his other high quality presets.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #35
Quote
I'm posting this here because I'm not sure which of the two lame-dev mailing lists I should be using (sourceforge or minnie.tuhs.org). I'll repost if someone tells me the right one ...


Use sourceforge if it is only interesting for LAME (e.g. bugfixes, portability fixes) and the other one for diskussions about MP3 algorithms and the like.

Quote
I submitted some changes to the LAME configure script for Sun Ultrasparc + GCC 3.2 that resulted in a pretty large performance improvement (about 30%) on those systems. These changes went into 3.93.

Since then, there has been some pretty bad performance degradation, the Ultrasparc-optimized 3.93 is more than twice as slow on the average as optimized 3.92. This is using --alt-preset standard and cbr modes.

I'm not sure what happened, since the x86 users are seeing an improvement. Is there anything that can be done? I can provide an UltraSparc test system if necessary.


a) check out the latest source and have a look at the ChangeLog file. Don't look at changes in Takehiro's branch. Try to find a logmessage which looks like it is the reason for this (it may be something about loop unrolling, but it hasn't to be such a change which causes this). Backout this particular change and test it.
B) make a binary search. Checkout a version in the middle of 3.92 and the actual code (you can specify dates for a checkout/update). If it is fast: advance further in time (add the half of the timespan between the date of the actual source and the date you just tested); repeat. If it is slow, checkout an earlier version (analogical to what you did previously). This way you should be able to find the commit which resulted in this behavior. Talk with the developer of this commit how to improve the speed.

Bye,
Alexander.

Note: the settings in this board regarding the use of emoticons isn't sticky for a particular message. It was surprising for me. I thought I should be able to disable it, preview the post, and don't have to disable it again.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #36
I was thinking more of a "concern" from Dibrom about this preset.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #37
Is there any other thread I should look at (at the moment only 3.93 related please)?

Has anyone out there looked at py-lame? I like to make the first release (0.x) together with lame 3.93. If somebody has some suggestions he should speak up soon.

Bye,
Alexander.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #38
Quote
We have some users which may benefit from a new release so we should make a new release.


By the same token, we have many users to which it may be their detriment to make a new release without proper quality testing.

Quote
Why I don't want Takehiro's code in 3.93:
At the moment we have code which is proven to be "stable" (about the fast preset issue later),


Proven by whom?  Who has done the extensive listening tests to prove that 3.93 has not regressed in quality even in areas not related to the fast presets?  I don't know of any such tests.  Please let me know if they exist.

Quote
It doesn't hurts if we make a mature release yet, and another release with Takehiro's code shortly after that (I'm thinking about making the release 3.93 release as soon as possible and making a 3.94beta release after Takehiro's code is merged).


Yes but only if we are certain that this code is actually "mature".  Without proper listening tests, we cannot be sure of this.  If we are going to spend the time doing the listening tests, then why don't we test Takehiro's branch instead?  I know for a fact from just a few short tests I performed that his branch fixes some of the remaining major bugs in the --alt-presets.

Quote
I don't want to bitch on Windows here (even if I'm more of a Unix guy, I'm actually writting this from a W2k system), but I have the impression that "Windows-guys" think about new releases as of "Yeah! Features! Features!" and personally I don't like this.


I'm sorry, but you are mistaken here.  This isn't about "Windows users" (to coin a negative phrase), it's about people concerned about release quality.  The LAME developers seem to be more concerned about portability bug fixes then quality improvements.  That is what people here are worried about.  I don't think anyone here considers Takehiro's improvements of the --alt-presets to be "new features" but to be "bug fixes" in terms of quality improvements.

People want to see tangible improvements from LAME and they want to make sure that successive releases do not regress in terms of quality.  They are more worried about this than portability bug fixes.

More on this...

Quote
I don't need features which aren't usable because of bugs (either because they aren't really tested or because the management decided to make a release). I don't want to make a "Features"-release, I want to make a solid-release.


And I want to see a truly improved release, not simply a "it works on more non-mainstream OS's but the quality might suffer across the board because it hasn't been properly tested" release.

I'm not making a fuss about features, I'm making a fuss over the fact that 3.93 was about to be released with known bugs in the --alt-presets (which few of the core LAME developers seemed to have cared about, and fewer still had even bothered to do any testing).  This is very irresponsible.  This is not how the process works for nearly every other encoder discussed on this board.  Releases are verified (at least as much as possible) to show improvements before they are released into the wild.  From what I can tell of LAME, there's almost zero quality testing and usually the only bugs which are caught in terms of quality are those which are absolutely show stoppers -- so obvious and horrible that it takes really no testing to notice.

Quote
The code of 3.92 can't be that bad , it's in widespread use. So why not release a bugfixed version of 3.92 (namely: the actual code in the HEAD CVS branch as 3.93) and make a known good release even better.


Yes, but 3.93 isn't the same as 3.92.  There have been many small changes.  Are you sure that nothing except for the fast presets has been degraded in quality?  I'm not.

Quote
This way we satisfy both kinds of users, those which like to have rock solid software, and those which don't mind a bug or two in bleeding edge software.


Without proper testing, we won't be satisfying either.

Quote
Quoting Dibrom:
Quote

I don't really see the point in making a release just for the sake of having a new release in X amount of time. Releases should be dictated by the merit of their improvements otherwise it's just arbitrary and could easily cause more problems than it fixes.

Yes, I want to make an improvement over 3.92. 3.92 is stable, so an imroved version has to be stable too. Therefore I don't think we should release 3.93 with Takehiro's code (it's not as tested as the actual one).


As far as I'm concerned, they are both relatively untested.  I say we start with Takehiro's branch then because I am at least certain that some major bugs in the alt-presets have been fixed in this branch.  Let's start testing with the best.

Quote
Quote

I don't understand what you mean about "when a program need to be able to run under a whole range of different operating systems." Does 3.92 have some major problems preventing it from running on most major operating systems?

I don't like the word "major" in your sentence. There's more than Windows (or Linux). 3.92 has some problems on some systems which are fixed in 3.93.


You misinterpreted my intent.  Unlike your statement earlier deriding Windows users, I have no problems with other OS's.  What I was trying to illustrate is that more people will be affected by poor quality testing than will be affected by portability fixes to OS's which are certain to have few users.

Quote
Quote

The only thing I see in the changelog so far are portability fixes for Tru64 Unix and SunOS 4.x. How many people are those fixes going to affect vs how many people poor quality testing could affect?

There are some changes which aren't in the official ChangeLog on the website, those changes are only listed in the CVS log (or in the file ChangeLog in the LAME tree).


This is a problem.  Like I was trying to say earlier, because of the general haphazardness of modifications to the LAME code, nobody is absolutely certain all that has been changed.  Therefore, we cannot be certain that quality has not regressed without proper listening tests.

Quote
There are some code fixes, which result in a problem with the preset fast setting, but thats only problem.


And you are certain of this?

Quote
Think about 3.93 as a bugfix release, not of a release with new features (even if we have them). And remember, I'm fine with making a 3.94beta with every feature/change you can come up with at the same day.


Sure, but I'm not convinced that 3.93 is as "stable" or as "clean" as you seem to believe.  I don't want to see a release made which is going to degrade the --alt-presets which so many people rely on to provide them with very solid quality.

If you can assure me somehow (I have no idea how) that 3.93 is not any worse than 3.92 except with the fast presets (and that these can be fixed), then go ahead with a release, and I'll work something out on my own for the newer stuff.

I would also in the future like to see that any "fixes" which are made to the code which break these presets (which should be seen as the "stable" and "proven" aspects of LAME quality) be reworked in a manner which either do not break the presets, or that the person who breaks the presets also makes every effort to unbreak them as well.  Otherwise we end up with this sort of situation...

Plans For --alt-presets, Etc, In Lame 3.93

Reply #39
Quote
ps: Dibrom, any comment?

If you can provide assurances that 3.93 is no worse than 3.92 (once the fast presets are fixed), then, as far as I'm concerned, go ahead and release whenever you like.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #40
Quote
Quote
If you have any concern about the medium preset being included (although I do not plan to document it yet) in a release, I can disable it if you want, we'll put it back in 3.94.

I don't care if you add your new preset or not. It's not critical code. Talk with Dibrom about this, he seems to have some kind of "fear" (sorry, I don't know how o describe it better in english) this pollutes his other high quality presets.

You could call it a fear if you like.  You could also call it a geniune concern for the users.  You could call it "looking out for them" in terms of quality.  People have grown to trust the alt-presets and the work which I and many others have put into them to provide solid quality.

I'm sorry that you don't seem to share that concern.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #41
Quote
What I can say is .... my branch contains too much experimental things and it changes everyday. Still more I have many pending patches. It's almost "before alpha" stage and will take very long to be "beta" stage.


Well this appears to be the nail in the coffin then.

However, if this is the case, I would like for us to collaborate closely on these modifications so that there can be more rigorous quality control.. so that when you are "finished" (at least for the moment) with your improvements, that we will know where they stand in terms in stability in quality.

Quote
It's not time to QA my branch. We LAME development team suffer from short of resources. Testing my branch "NOW" is only wasting resources. We should test the head branch.


I'd disagree with that somewhat.  I think it's always time for QA, otherwise you will never be certain where you are heading in terms of quality.  Also, the head branch certainly does not fix some of the long standing bugs in the --alt-presets...

Quote
I don't recomment to merge my branch into main, before releasing 3.93. Some of you say 3.93 will be triial, but 3.93 contains some of speed up(at least for me) and bug fix. It is worse releasing new version IMHO.


OK.  I still think that we should be focusing on these improvements though and that much attention should be paid to the changes you are making.

Quote
btw, for the people interested in my pending patches: This is it.

- impulse like signal detection for long/short block switching improvement.
- better temporal masking.
- more faster VBR.
- intensity stereo.
- mixed block support.
- make nspsy faster/"always" better than gpsycho and remove gpsycho.
- improve "fast" preset quality and remove "normal" preset.
- re-remapping -q option with safen the substep noise shaping.
- new and faster MDCT code.
- SSE/3DNow! FFT/quantization code
- default to use float instead of double
- integrate ssrc resampling code


I'm very interested in these changes.  The only thing I'm uncertain about is the bit about removing the normal presets in favor of the fast presets.  I'm not certain this will work very well.  I'd like to see them left in place while still benefiting from all of these other improvements, at least until we are absolutely certain that the fast presets are unquestionably better in every regard.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #42
Anyway, since it appears that Takehiro himself does not wish to merge his branch with the mainline, and he's probably the only one who can do it properly, then, as far as I'm concerned, you can go ahead and release 3.93 once the fast presets are fixed and once we are certain that there have been no regressions in quality anywhere else.

I'll continue to work on my own (once again unofficial) releases...

Plans For --alt-presets, Etc, In Lame 3.93

Reply #43
Quick results for only one tested song (I have not the time to do more with my slow PII)

Song : Radiohead - Paranoid Android 6:23 (ripped with EAC secure mode)

presets :

   --aps :
3.90.2 : 202 kbps | av. reservoir 74 | Joint Stereo 33.2 ss/66.8 ms | Block 94.7 long/5.3 short
3.92  : 197 kbps | av. reservoir 75 | Joint Stereo 33.6 ss/66.4 ms | Block 94.7 long/5.3 short
3.93a2 (mitiok's 11/10) : 203 kbps | av. reservoir 448 | Joint Stereo 33.6 ss/66.4 ms | Block 94.7 long/5.3 short
3.93a2 (dibrom's compile) : 194 kbps | av. reservoir 74 | Joint Stereo 34.4 ss/65.6 ms | Block 97.2 long/2.8 short

   --alt-preset dm-medium :
3.93a2 (dibrom's compile) : 166 kbps | av. reservoir 87 | Joint Stereo 34.4 ss/65.6 ms | Block 97.2 long/2.8 short


--preset medium doesn't appear in this test (sorry gabriel) since I never succeded to make this preset work with the latest alpha (I use razorlame)

I will give my results for the whole album later if I find the time

Plans For --alt-presets, Etc, In Lame 3.93

Reply #44
Thanks for the results Continuum and Manu.

Due to the way circumstances have unfolded though (and some thoughts I'm having on how best to handle this situation), I'm going to halt development on these new presets shortly.

I'm hoping that development will resume in a much more organized and significant manner.  If things go as I plan, I'll give more information on this shortly.

If people want to continue testing, that would still be welcome, I'm just not going to be putting out any new compiles until a couple issues are taken care of first...

Thanks to all who have shown interest.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #45
That's too bad!

I can't think of any valid reason to release a new LAME version unless it offers a smaller compression rate with the quality we have come to expect from the "presets"

I mean, aps is transparent already in 3.92!!! What does 3.93 do for aps??? Encode it a slight bit faster? Who cares?

99.9% of MP3 users desire smaller file sizes with the the optimal quality for that given size. Otherwise we would all just listen to WAV files. Its all about making the file size as small as possible to get as many songs as possible on a given media. A VBR medium setting was a GREAT step in this direction.

It's too bad to see it delayed.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #46
Doh! 

Well, I just finished encoding another album with 3 of the new presets so I'll just post the bitrates:

James Gang - Yer Album (Remastered) | 11 tracks | Classic Rock

--alt-preset dm-medium: 177 kbps
--alt-preset portable: 175 kbps
--alt-preset dm-radio: 162 kbps

This album was originally released in 1969 (I believe) and the instruments are pretty heavily panned to the sides with very good stereo drums. It's very clean and one of the best mastered CDs I have.

I think these two clips from a song on the album might be useful for pre-echo testing. Clip1 is a pretty clean sounding drum solo with heavy stereo mixing and the other has an overdriven guitar and bass with the drums:

clip1 - flac (2.2 MBs)

clip2 - flac (1.2 MBs)

After some listening all the presets sound very good to me, I haven't noticed any harsh artifacts. Though I think heard some flanging type artifacts in the Halford CD with the dm-radio setting.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #47
I agree very much with Dibrom and I wish that in a new release there were real quality improvements. I really appreciate he's very active in guiding attention to the "quality integrity" of alt-presets for the future versions.

I want to submit my humble contribution:

Last week I've discovered a sample (le_monde.flac) that fails with LAME 3.90.2
Download le_monde.flac
MD5(le_monde.flac) = 1739177e0e82cfbf0a397bf21b905fd1

------------------------------------------------------
-- Original vs 3.90.2 encoded --

Settings: --aps
ABX: 16 out of 16, pval < 0.001

Settings: --ape
ABX: 15 out of 16, pval < 0.001

Settings: --ape -Z
ABX: 15 out of 16, pval < 0.001

Settings: --api
ABX: I think i can't ABX
------------------------------------------------------

This sample, taken from my favorite italian band,has a bad pre-echo expecially at 2.2 - 3.4 seconds. I was thinking that there wasn't any possibility for future quality improvements in the meantime and i was very disappointed about the suspended development of --nspsytune2, but now i'm appreciating Takehiro's new work and the intention of adopting his new tunings to the future presets.

I was wondering if Takehiro's new alpha would do better with the above sample, so i downloaded "takehiro-lame.exe" and i started encoding, so here are my results:

--------------------------------------------------------------------------
-- Original vs takehiro-lame.exe encoded --

Settings: --aps
ABX: 29 out of 45, pval  0.036
Comment: I need more tries to reach a good pval

Settings: --ape
ABX: 20 out of 25, pval  0.002
Comment: I need more tries to reach a good pval.
Worse than --aps ?!!
--------------------------------------------------------------------------

The results show a more difficult to abx sample.

-----------------------------------------------------------------
-- takehiro-lame.exe encoded vs 3.90.2 encoded --

Settings: --aps
ABX: 16 out of 16, pval < 0.001
Comments: easy, takehiro's one is better.
-----------------------------------------------------------------

I'm happy because i clearly noticed a higher quality, there are really "significant fixes to pre-echo handling". The sample is still not "perfect" and i think (and hope) that it could be useful for future tunings in this direction.

Just my 2 EURO cents
WavPack 4.3 -mfx5
LAME 3.97 -V5 --vbr-new --athaa-sensitivity 1

Plans For --alt-presets, Etc, In Lame 3.93

Reply #48
Quote from: Dibrom,Oct 11 2002 - 10:01 PM
Quote
We have some users which may benefit from a new release so we should make a new release.


By the same token, we have many users to which it may be their detriment to make a new release without proper quality testing.


The changes between 3.92 and the actual code are bug fixes, portability fixes, speed improvements for "non-mainstreem OS's" (as you like them to call) and also new code (substep noiseshaping and some VBR changes). The portability fixes and speed improvements for other OS's are done in a way to not affect the quality. The substep noiseshaping has to be explicitely activated. So these changes can't change the quality per definition. Can we agree on this?

Now for the bugfixes and for the VBR changes:
The VBR changes are from Robert, and Takehiro had some problems with them. Robert and Takehiro talked about the changes and fixed the problems. Reading the public parts of their discussion and reading the commit log of the changes I had the impression they also did listening tests and made sure the quality at least stayed the same. Takehiro can tell you more about this.

For the bugfixes: bugfixes are per definition improvements, even if they affect the quality in a negative way. If they affect quality in a negative way, the problem is the algorithm, not the bugfix. So if a bug results in better quality, this isn't a deterministic behavior, it may affect other input data in a negative way (this now depends on the definition of "bug", but at least the bugfixes after 3.92 are real bugs you want to have fixed). If I remember correctly, there are also fixes which result in a better portability of the generated MP3s (and these may affect the bitrate and may therefore result in a worser sounding MP3s, but even if the quality degrades, the resulting MP3 is an improvement because the bitstream is more correct). And I think this may be the reason for the higher bitrate for the fast preset.

I followed the lame-cvs list (every commit results in a mail to this list and the mail contains the files which got changed and the commit message for these files). For some of the commits I talked with the commiter about the change (mostly with Takehiro). The only problems I see at the moment is the preset fast issue.

Quote

Quote
Why I don't want Takehiro's code in 3.93:
At the moment we have code which is proven to be "stable" (about the fast preset issue later),


Proven by whom?  Who has done the extensive listening tests to prove that 3.93 has not regressed in quality even in areas not related to the fast presets?  I don't know of any such tests.  Please let me know if they exist.


I don't have a formal prove. But sometimes you can tell it by looking at the changes. I haven't looked at every change, but see above.

Quote

Quote
It doesn't hurts if we make a mature release yet, and another release with Takehiro's code shortly after that (I'm thinking about making the release 3.93 release as soon as possible and making a 3.94beta release after Takehiro's code is merged).


Yes but only if we are certain that this code is actually "mature".  Without proper listening tests, we cannot be sure of this.  If we are going to spend the time doing the listening tests, then why don't we test Takehiro's branch instead?  I know for a fact from just a few short tests I performed that his branch fixes some of the remaining major bugs in the --alt-presets.


Takehiro already answered, that the actual code is more mature than the code in his branch. And if we assume that the 3.92 code is mature, you only have to point out flaws in the changes. That's not that hard as it may sound, just take the file ChangeLog and read the commit messages. You just have to look at changes, where the commit message tells you the change is in a critical section but tells you nothing about listening tests.

E.g. we have the fast log changes. Theses changes increase the speed of lame by reducing the precission of some floating point calculations. Theses changes affect the output data, but they just result in minor rounding errors. And yes, some developers did listening tests with those changes.

Quote

Quote
I don't want to bitch on Windows here (even if I'm more of a Unix guy, I'm actually writting this from a W2k system), but I have the impression that "Windows-guys" think about new releases as of "Yeah! Features! Features!" and personally I don't like this.


I'm sorry, but you are mistaken here.  This isn't about "Windows users" (to coin a negative phrase),


It wasn't my intend to coind a negative phrase. It was just an observation that users of Windows are more forgiving to bugs. If there's a blue screen in Windows, most of the users sit there, watch how it reboots. Sometimes the say some bad words, but most of the time they just accept it. They don't seem to care. And if there's an update of a program or Windows itself, they don't ask how much bugs got fixed, they ask how many new features the new software has. They don't care how many security bugs the software has. Can you tell me how many published security holes -- which are unfixed -- still reside in your software? I'm able to do this for my software... at least for the Open Source one, I can't do it for my Windows software, because there are too much published holes. But at least I can make sure that those holes don't affect me (either by using programs on Windows, which don't have published holes (I don't use Windows much, so this isn't a problem for me), or by using my Unix software). There aren't many Windows users which know the actual security status of their system. Even most of the network administrators don't know it. And if there's some kind of new workm or virus, they bitch at the writters of those worms. I don't want to glorify worm or virus writters, I'm just talking about people which react in a wrong direction. Uhm... I'm getting of topic here, sorry.

I wanted only to make a distinction of two user groups. You can find both kinds of user groups on Windows and on Unix systems. But you can find one group more often on unix systems and the other group more often on Windows systems. As every generalization this one also doesn't work in every case. I apologize if I have insulted anyone, this wasn't intended.

Quote

it's about people concerned about release quality.  The LAME developers seem to be more concerned about portability bug fixes then quality improvements.  That is what people here are worried about.  I don't think anyone here considers Takehiro's improvements of the --alt-presets to be "new features" but to be "bug fixes" in terms of quality improvements.


Technically it's new code which isn't widely tested. Therefore it doesn't fit in our intended release. But we don't have to talk about it here, Takehiro has already spoken his last words on this issue.

Quote

People want to see tangible improvements from LAME and they want to make sure that successive releases do not regress in terms of quality.  They are more worried about this than portability bug fixes.


Feel free to come up with a formal and mostly automated setup to test for it. I'm not asking about a program which makes the decission "good" or "not good". Just come up with a set of samples, an automated procedure to generate MP3s out of those samples, compare them with the last official set of MP3s, and if they differ print out the files in question and a description about already found flaws in older versions of LAME, so the tester can make sure every one of those flaws doesn't exist in the new MP3 (we have already a framework for this in the CVS tree, you just have to provide samples, a description, and a set of lame command line switches for these samples!).

Quote

More on this...

Quote
I don't need features which aren't usable because of bugs (either because they aren't really tested or because the management decided to make a release). I don't want to make a "Features"-release, I want to make a solid-release.


And I want to see a truly improved release, not simply a "it works on more non-mainstream OS's but the quality might suffer across the board because it hasn't been properly tested" release.


The lame developers identified some deficits in the actual code (fast preset). And they feel comfortable with the rest of the code. Every developer makes listening tests before he commits changes which may affect the quality. We don't have a formal regression test suite, so we can't catch every possible flaw, but me make sure the quality is a s good as we are able to test it.
So please don't spread the FUD further and come up with examples where the actual code fails. I commited no code which affects quality, so I don't feel attacked by your sentence, but I don't think those developers which change the low level MP3 code deserve such sentences. Feel free to have a look at the archives and at the commit logs to see how hard they worked in their free time.

Quote

I'm not making a fuss about features, I'm making a fuss over the fact that 3.93 was about to be released with known bugs in the --alt-presets (which few of the core LAME developers seemed to


You are seeding FUD. I asked on lame-dev if there are some unresolved things to do before we can release 3.93. Gabriel answered he wants to do some updates to the html docs and perhaps wants to add an additional preset. And he said he is fine with a release over the weekend. I answered I don't want to rush the release and we should wait at least for a week (I know there are developers which don't have the time to read mails on lame-dev every day, so I wanted to give them some time to answer).

Quote

have cared about, and fewer still had even bothered to do any testing).  This is very irresponsible.  


See above (testing framework; how the developers test).

Quote

This is not how the process works for nearly every other encoder discussed on this board.  Releases are verified (at least as much as possible) to show improvements before they are released into the wild.  From what I can tell of LAME, there's almost zero quality testing and usually the only bugs which are caught in terms of quality are those which are absolutely show stoppers -- so obvious and horrible that it takes really no testing to notice.


Everyone here is invited to browse the archive of lame-dev and the ChangeLog file and decide if this is the truth. I don't comment further as I already have on this.

Quote

Quote
The code of 3.92 can't be that bad , it's in widespread use. So why not release a bugfixed version of 3.92 (namely: the actual code in the HEAD CVS branch as 3.93) and make a known good release even better.


Yes, but 3.93 isn't the same as 3.92.  There have been many small changes.  Are you sure that nothing except for the fast presets has been degraded in quality?  I'm not.


Feel free to prove it by providing examples where the actual code performs not as good as 3.92. If you decide to try to prove it, please drop me a note, I'm willing to delay the release until you find an example or until you give up.

Quote

Quote
Quote

The only thing I see in the changelog so far are portability fixes for Tru64 Unix and SunOS 4.x. How many people are those fixes going to affect vs how many people poor quality testing could affect?

There are some changes which aren't in the official ChangeLog on the website, those changes are only listed in the CVS log (or in the file ChangeLog in the LAME tree).


This is a problem.  Like I was trying to say earlier, because of the general haphazardness of modifications to the LAME code, nobody is absolutely certain all that has been changed.


Again: browse the ChangeLog file and the lame-dev archives.

Quote

Quote
Think about 3.93 as a bugfix release, not of a release with new features (even if we have them). And remember, I'm fine with making a 3.94beta with every feature/change you can come up with at the same day.


Sure, but I'm not convinced that 3.93 is as "stable" or as "clean" as you seem to believe.  I don't want to see a release made which is going to degrade the --alt-presets which so many people rely on to provide them with very solid quality.


I have been told the fast preset has only an increased bitrate, not degraded quality in the resulting MP3. Is this wrong?

Quote

I would also in the future like to see that any "fixes" which are made to the code which break these presets (which should be seen as the "stable" and "proven" aspects of LAME quality) be reworked in a manner which either do not break the presets, or that the person who breaks the presets also makes every effort to unbreak them as well.  Otherwise we end up with this sort of situation...


I'm fine with this. I'm anoxious to look at the regression test suite you can come up with. If you need some asistance in setting up the framework from the CVS repository (have a look at the test directory) or if you need some features in the framework, feel free to drop me a note.

Bye,
Alexander.

Plans For --alt-presets, Etc, In Lame 3.93

Reply #49
Quote
You could call it a fear if you like.  You could also call it a geniune concern for the users.  You could call it "looking out for them" in terms of quality.  People have grown to trust the alt-presets and the work which I and many others have put into them to provide solid quality.

I'm sorry that you don't seem to share that concern.

I have enough trust in Gabriel to be not concerned if he want's to add a new preset. And he said he may add a new preset, so the he hasn't made a final decission on it (at least not at the time he told it to me).

Bye,
Alexander.