HydrogenAudio

Basically, as Takahiro has noted that 4.0 is still a ways off, and 3.90 is several years old, should we work on testing 3.96 now? If this version tests out as well as 3.90.3, or does with some minor tweaking, then the speed advantages at the very least make for some incentive, don't you think?

Benefits of an upgrade now:

• Faster encodes
• Lower bitrates
• Built-in clipping removal
• Forced --alt-presets when using VBR

 What about: Wait for 3.96, and request developers to focus all further development towards FAAC? MP3 is as good as it gets anyway.

I vote for a test of the last version. Especially because the current recommended version has only been tested by Dibrom. I don't remember of any blind test results between alt-presets and custom command lines. Everytime someone wanted to challenge the presets, he disappeared without testing properly.
It would be a good thing to publicly test the presets.

I think we should test 3.96 and see about making it the recommended version.  I tested 3.95.1 and the encoding was noticably faster and the quality seemed quite good with lower bit rates than 3.90.3.  If we all put an effort in to test 3.96 I think it could become the recommended version and if we find flaws it would help the LAME developers to correct those issues so eventually we will get a new recommended version.

Tight preset tunning would need to be done for 3.96 as was done for 3.90.x by Dibrom.

I'm all for a 3.96 --preset listening test vs. the 3.90.3 --alt-presets (and an anchor, obviously).

I have a feeling this will be incredibly harder to differenciate than even the recent AAC @ 128Kbps test, though.

I'm all for a 3.96 --preset listening test vs. the 3.90.3 --alt-presets (and an anchor, obviously).

I have a feeling this will be incredibly harder to differenciate than even the recent AAC @ 128Kbps test, though.

It seems to me that if that were the case, it would be a good argument for 3.96 being able to take over as the new recommended build

Bring on the Testing  Would be great to have a new recommended version
Cheers

I second that. People tend to have more interest in AAC/Ogg nowadays, but hey - MP3 IS the defacto standard, and IS the king of the hill, and absolutely nothing will change that atleast in the next 5 years.

And yes, a new recommended version would be wonderful. I refuse to use 3.90.3 even now, i'd rather use 3.95.1.

Bring on the testing!

There's an old saying that says if it ain't broke, don't fix.  Unless LAME 3.96 offers the possibility of the same quality of aps at a reduced bitrate (which it may do, I'm not a LAME dev so I don't know) then I don't see why we should test it for now.  However, as I understand it LAME 4.0 will have many new features and I think it will be worth testing.

There's an old saying that says if it ain't broke, don't fix.  Unless LAME 3.96 offers the possibility of the same quality of aps at a reduced bitrate (which it may do, I'm not a LAME dev so I don't know) then I don't see why we should test it for now.  However, as I understand it LAME 4.0 will have many new features and I think it will be worth testing.

With that thinking there would never be any progress, comparing the very well tunned 3.90.3 & 3.96 will help improve later versions of lame. If everyone just stopped testing until 4.0 was released I think we will find it to be a very disappointing release.

I wasn't trying to discourage testing of new releases, just to discourage the massive amount of effort required to replace 3.90.3 as the recommended version.

Evolve use 3.96 , the developers keep working on the project, I see no reason to settle for less.

There's an old saying that says if it ain't broke, don't fix.  Unless LAME 3.96 offers the possibility of the same quality of aps at a reduced bitrate (which it may do, I'm not a LAME dev so I don't know) then I don't see why we should test it for now.  However, as I understand it LAME 4.0 will have many new features and I think it will be worth testing.
(....)

I wasn't trying to discourage testing of new releases, just to discourage the massive amount of effort required to replace 3.90.3 as the recommended version.

Interesting. But what about other encoders, like musepack, vorbis, or Nero AAC? Are there massive or collective tests? Or are people systematically adopting the latest version as reference encoder? This last behaviour is what happened and will happen with all encoders, except for lame, protected by very strange principles on this board.
Lame 3.96 is not an attempt to "fix" some problems, but a lot of work to improve quality and speed. Exactly what people like Frank Klemm did with musepack or Ivan Dimkovic with Psytel->Nero. I never saw the HA community discouraging people to use these new mpc or aac encoders; the whole community always trust their developers, and believe that new encoders are better, or at least flawless. Can't we trust lame developers? Why?

EDIT: spelling

I agree...test 3.96 now, then, once 4.0 is ready in a year or two, undergo testing with that. It can't hurt to have a new recommended version in the interim. Personally, I can't wait to see what the addition of IS in 4.0 does. Seeing as I can't successfully ABX anything above 80kbps @ 44.1khz from the original...it will be a good move for me.

But until then, 3.96...7...8...9 will have to do.

Gotta love the Sony MDR-201 headphones

I would like to see the latest version tested, and adopted if it is good enough.  However, on the few samples I've tried to ABX 3.90.3 --alt-preset standard against the source wav file I have failed completely, and these have been supposedly difficult to encode samples.

This doesn't really surprise me as I don't have headphones and as I understand it --alt-preset standard is supposed to be transparent where possible with lame.  But it does mean that I can't see that I have much to offer for the testing.  And there is the problem: testing so close to transparency is really only possible for those with very sensitive hearing.  So the rest of us can vote for testing but if those people are not interested in mp3 and lame then it is unlikely to ever happen.

I mean if a test was setup like AAC 128K we might have a successful test with a clearer view...I'm sure there would be really hard samples out there that would be good for alot of people to test...It would be nice to just be able to recommend the latest version of lame with --present standard.... 
Cheers

gday..


well. am noy that huge mp3 fan.. after i have bee introduced to HE-AAC
and MPC. the only reason i can think of.. using the mp3 format..
would be for compabilety.

the only build i have tested out.. (slightly) is 3.95.1
and i liked what i heard.

if the 3.96 is tested out propperly.. i would say it`s time to upgrade.


btw. is there a changelog aviable for 96..

The changelog is available in the archive file on rarewares.
You can also take a look here (http://forum.hardware.fr/forum2.php3?cat=3&post=61233&config=hardwarefr.inc&cache=&sondage=0&p=1&trash=0&subcat=0&owntopic=1&page=5#t600176).

The last option in every poll should always be "I Don't know / Can't decide", depending on the subject.

If a test was to take place (and assuming it's conducted in a similar manner to roberto's tests). What kind of a confidence margin would the newer Lame version (if it was to win) require in order to replace 3.90.3 as recommended Lame version? I doubt the tests would show one to be clearly better than the other, since I bet they're both pretty transparent at these bitrates for most music.

If a test was to take place (and assuming it's conducted in a similar manner to roberto's tests). What kind of a confidence margin would the newer Lame version (if it was to win) require in order to replace 3.90.3 as recommended Lame version? I doubt the tests would show one to be clearly better than the other, since I bet they're both pretty transparent at these bitrates for most music.

well, since LAME 3.95.1 is MUCH faster and produces a smaller file size in most cases (I replaced almost my entire CD-Collection with LAME 3.95.1 and it was always smaller than LAME 3.90.3 or even LAME 3.92; but some people reported a similar or a slightly higher bitrate on some samples), it would be enough if LAME 3.95.1 or LAME 3.96 would perform as good as LAME 3.90.3 . I'd say, if at least 50% of all tested samples sound better than 3.90.3 it should be updated...

Also, I think it kinda sucks for Gabriel and all the other LAME developers who have put so much time into the new releases and tuning them...since LAME 3.90 is still recommended it looks like if all there work from then was useless

IMO this can't be done in a way that is comparable to rjamorims listening tests, because:

- Something like 12 samples is not enough
- We need reliable results (= ABX testing required)
- We need to test several different presets / quality settings
- ...

Here's my suggestion how this should be done:

• Start a thread for reporting results only. The 1st post in this thread can be edited to collect the results.
• To make sure that comparable bitrates are tested (for abr/vbr presets that is), some mass-encoding and comparison has to be performed first (-> separate thread). The results are used to decide what to compare (e.g. 3.90.3 --alt-preset 128 vs. 3.96 ...)
• Everyone can use his own music or known problem samples to compare 3.90.3 vs. 3.96 at the quality setting he wants.
• Posting results in the thread requires:
  • Upload or link to sample
     
  • ABX results Original<->3.90.3, Original<->3.96, 3.90.3<->3.96, with detailed description of the difference(s)
     
  • Report about software/hardware used: Soundcard (resampling?), Player/ABXtool, DSPs (shouldn't be allowed, besides resampling to 48kHz and volume reduction/replaygain to prevent clipping <- both a 'must'), Amplifier, Speakers/Headphones
• Results must be confirmed by someone else before they are included in 'official' statistic, p-values must be < 0.05 for at least 2 people.
• I'm not sure if/how ABC/HR-like ratings should be used for this

The 1st post could look like this (everything is made up as example):
____________________________________________

Results for recommended lame version tests



1. 128kbps CBR, commandlines used: 3.90.3 --alt-preset CBR 128, 3.96 --preset CBR 128
___ samples where 3.90.3 is better,
___ samples where 3.96 is better so far.
Links to related posts: 1, 2, 3
Remarks:
3.90.3 has more problems with warbeling/flanging on cymbal-like sounds
3.96 has more pre-echo/smearing problems
...


2. 128kbps VBR/ABR, commandlines used: 3.90.3 --alt-preset 128, 3.96 ...
...
...


6. 160kbps VBR/ABR, commandlines used: 3.90.3 --alt-preset 160, 3.96: --preset medium
...
...
_____________________________________________

Hopefully there's some way to perform statistical analysis on these results that is able to tell after enough tests have been performed with e.g. > 95% reliability which one is better for a given quality/bitrate setting (ff123?)

Oh - and about ABR/VBR: We would need to know what to test, i.e. what setting is supposed to give best quality at a given average bitrate. IIRC the abr presets like --preset 128 still work with lame 3.95/3.96, but the -V settings have been changed by Gabriel and are supposed to work similar to Musepack/Vorbis -q ... settings. Probably it would be a good idea to test both while we're at it (e.g. --preset 128 and -V ... (whatever gives 128kbps on average)) - unless Gabriel (or someone else who should know) contradicts.

Interesting. But what about other encoders, like musepack, vorbis, or Nero AAC? Are there massive or collective tests? Or are people systematically adopting the latest version as reference encoder? This last behaviour is what happened and will happen with all encoders, except for lame, protected by very strange principles on this board.
Lame 3.96 is not an attempt to "fix" some problems, but a lot of work to improve quality and speed. Exactly what people like Frank Klemm did with musepack or Ivan Dimkovic with Psytel->Nero. I never saw the HA community discouraging people to use these new mpc or aac encoders; the whole community always trust their developers, and believe that new encoders are better, or at least flawless. Can't we trust lame developers? Why?

You bring up some very good points that I hadn't thought of, however...

Firstly I can't speak about MusePack, because I wasn't on this forum when the major development was happening.  Vorbis has seen little official devlopment since 1.0 (again I wasn't here for much pre-1.0 testing or what not) although people like Garf and QuantumKnot have been implementing there own tunnings to the codec.  As far as I know (but correct me if I'm wrong, I'm not really sure) GT3b1 had quite a bit of testing before people decided to use it over pre 1.0-to-post1.0 versions of Vorbis.  Futhermore, I was under the impression that Ahead had a QA department that made sure that at least nothing got worse in the codec compared to previous versions.

Again, I'm not trying to discourage some testing.  If an extensive test is organised, I'll probably try to even be apart of it.  I'm just trying to let people know that this sort of testing will be much more difficult then the 128kbps AAC test and only people with very good hearing will be able to provide ABX results.  It will also be much longer, with many more samples and many weeks (probably months) of testing before LAME 3.96 (or any other version) can replace 3.90.3.

P.S. I agree that LAME is protected by strange principles.  However, that doesn't make it any less of a fact that 3.90.3 is the most tested version of LAME ever, and that at least an equal (almost definately more) amount of testing will be required to replace it.

Nice idea to count test samples that became better or worse with 396b!

I can add here sophia2 is clean the first time with aps!

390.3 -> sandpaper noise
395 -> added plop
396b -> clean

Wombat

Oh - and about ABR/VBR: We would need to know what to test, i.e. what setting is supposed to give best quality at a given average bitrate. IIRC the abr presets like --preset 128 still work with lame 3.95/3.96, but the -V settings have been changed by Gabriel and are supposed to work similar to Musepack/Vorbis -q ... settings. Probably it would be a good idea to test both while we're at it (e.g. --preset 128 and -V ... (whatever gives 128kbps on average)) - unless Gabriel (or someone else who should know) contradicts.

I always thought LAME 3.90.3 was recommended for --APS only

LAME 3.90.3 is recommended over other versions for many different settings.  APS is the recommended setting for perceptual transparency.  Sometimes you need to make a compromise on quality vs. filesize (which means you'll lose transparency).  This is why other bitrates also need to be tested.

Is there a special reason to be testing 3.96 rather than the upcomming 3.97?

I'm not trying to sound sarcastic. Maybe the developers consider 3.96 to be a very good version. Maybe they feel it is just an interim release before the real spanking 3.98 comes out.

I'm asking because with every new version people will come and ask if it is recommended over the current one. I just want to make sure that good effort wich should maybe be spent on a later (not so distant) version isn't wasted on this version.

Serious testing should be done, even if lame 3.96 isn't a major version. There are two possibilities:
- lame 3.96 is fine => recommanded version, and bye-bye old 3.90
- lame 3.96 have major problems => this will help developers to improve the next version.

Waiting unfortunately doesn't help. Lame developers need feedback, otherwise, we can't expect major improvements.

Is there a special reason to be testing 3.96 rather than the upcomming 3.97?

I'm not trying to sound sarcastic. Maybe the developers consider 3.96 to be a very good version. Maybe they feel it is just an interim release before the real spanking 3.98 comes out.

I'm asking because with every new version people will come and ask if it is recommended over the current one. I just want to make sure that good effort wich should maybe be spent on a later (not so distant) version isn't wasted on this version.

maybe we should give Gabriel the chance to state his opinion. However, if you take a look at the change history, there is quite a big time frame between LAME 3.93.1 and the newest stable release 3.95.1 (more than 1 year!).

So I think we have to ask Gabriel. He once told me that there will be of course always a newer release...so it wouldn't make much sense to wait and wait, especially if LAME 3.97 would take another year or so...

Personally, I wouldn't wait to long for test...because AAC is coming much faster than expected...probably because of Apple's: iPod, Music Store, Superior AAC Encoder and iTunes...it's easy to use, free and gives very good quality at low bitrates; no confusion about all the different settings...so what I really want to say is, that a test that takes place in a year or so might not be that interesting for many people, because of other formats that more and more find their way on people's computer's... I mean, iTunes AAC @ 160kbps = awesome

ACH!  my post disappeared.

well, anyways, i say wait for 4.0

in my opinion there's very little difference in the quality of --alt-presets from version 3.90 up.  it would make more sense to go to all that testing effort with 4.0 rather than a release that it's taken as a given that the difference will be tremendously difficult to spot.

Is there a special reason to be testing 3.96 rather than the upcomming 3.97?

I'm not trying to sound sarcastic. Maybe the developers consider 3.96 to be a very good version. Maybe they feel it is just an interim release before the real spanking 3.98 comes out.

3.96 will probably be promoted to stable.

3.90.X are based on a more than 2 years old release.

You have some choices:
*keep using 3.90.X, and do not bother with new versions. Of course, you fully understand that the current justification used to not test new versions will forever be true, so it means that you will always keep using the same version.

*Try new versions. If they are fine to you, then use them. If they are not fine to you, then report why so it could be corrected in next versions.

By choosing the second option, you will be able to know if you can replace the version you are using, and will help new better versions to appear.
By choosing the first option, you choose to ignore if you can replace your current version, and are also choosing to not help in development.

You also have to keep in mind that 3.97 will be based on 3.96, and 3.98 will be based on 3.97. They will not be based on 3.90.X.

So jump in, or stay where you are. But in this case, you will likely stay there forever.

it would make more sense to go to all that testing effort with 4.0 rather than a release that it's taken as a given that the difference will be tremendously difficult to spot.

I hope that you do not have preset standard in mind, because a preset targeted to transparency will always be close to a preset targeted to transparency, beeing in 3.90.3, 3.96, 4.0 or 9.0.

Well the birds sample is cured absolutely also with aps!

Sandpaper noise is gone!

Very promising so far, Gabriel

Wombat

We can't keep the 3.90 release "haunting" us forever...it might be a landmark in the Lame history, but we have to check if things have actually progressed since then. Hydrogenaudio is all about testing and suggesting accordingly. Keeping up-to-date is not always essential, but we have to check v3.96's quality before condemning it as inferior and sticking again with 2 years old code. I don't know if the latest version is better, but I sure want to find out.

.... Lame developers need feedback, otherwise, we can't expect major improvements.

I think that's the point.

The developers have invested their valuable time to improve the new versions.
So we, the users should give our feedback.
Keeping 3.90.x as the holy cow forever and ignoring what the developers offer might be disrespectful ...
Maybe it is not necessary to test every new release to the maximum extend, but  the ones the developers recommend as worth to be.
Maybe Gabriel or s.o. could give a clue if 3.96 is THE ONE or when the next relase comes out that has the potential to be the successor of the honourable 3.90
...and as Guruboolez stated, this is also a chance to detect the difference between new releases, which could result in further improvements. 

Thanks for your answers, Gabriel. The remaining question would be this:
Can the current 3.96 beta be considered stable quality-wise? From what I read from the change-log there haven't been any quality-related changes since 3.95.1 - is this correct? Of course there can always be bugs that affect quality, finding them is one reason for testing - I just mean starting tests with big efford would be pointless if quality-related changes are planned before 3.97 stable. Comments?

Quality related changes are always planned: we are developing in mp3 encoder, and so quality is one of the goals.
So yes, there will be quality related changes, but how is that preventing you from testing the current version?

I hope that you are not waiting for the end of quality related changes before trying a new version , or you will probably never try any new version before the final stop of the Lame development.

you also have to take into account, that even if we now test 3.96 and tune it(if necessary)... and then one day there'll be an 3.97 - then 3.97 will probably take advantage of at least some of the tunings and fixes of 3.96.

So, later versions will take advantage of 3.96 testing anyways. Of course with the implementation of new features, this advantage will slowly fade. Still... even if we dont get the absoluetely-total-optimum-version to do testing, future versions will take advantage of this. In other words - no effort gets lost.

- Lyx

Thanks for your answers, Gabriel. The remaining question would be this:
Can the current 3.96 beta be considered stable quality-wise? From what I read from the change-log there haven't been any quality-related changes since 3.95.1 - is this correct? Of course there can always be bugs that affect quality, finding them is one reason for testing - I just mean starting tests with big efford would be pointless if quality-related changes are planned before 3.97 stable. Comments?

3.96 has some bugs fixed that have been in LAME for over a year. those bugs surely have an effect on quality. we want 3.96 to be as stable as we can get to have some anchor for further development. if there are no big problems (*encoding* problems), end of march we hope to release 3.96 stable. For 3.97 we will probably focus more on the decoder. there is a plan replacing mpglib by hip finally.

Bitrate bloat seems under control now as well. I was using --lowpass 17600 on 3.90.3. This version gives lower bitrates than that setting without even using a lowpass.

You have some choices:
*keep using 3.90.X, and do not bother with new versions.[...]

*Try new versions.[...]

By choosing the second option, you will be able to know if you can replace the version you are using, and will help new better versions to appear.
By choosing the first option, you choose to ignore if you can replace your current version, and are also choosing to not help in development.

I think you misunderstood me. I wasn't saying that we should stick to 3.90.3 forever and testing new versions was pointless.

I believe that the amount of testing required to possibly find a replacement for the current recommendation is very large. A great effort like this will not be made often and the amount of (free) time people are willing to invest should be wisely spent.

So I'm mainly asking if this version is worth it or if maybe you have a not so distant version on your roadmap that would benefit more from such an effort. For example a developer has been working on theoretically significant quality improvements which are going to be merged with the official branch in version 3.97.

I have no insight in the development process, so I was asking about that, without implying that the status quo should remain unchanged forever.

Quality related changes are always planned: we are developing in mp3 encoder, and so quality is one of the goals.
So yes, there will be quality related changes, but how is that preventing you from testing the current version?

I hope that you are not waiting for the end of quality related changes before trying a new version , or you will probably never try any new version before the final stop of the Lame development.

You've said that before, but obviously I wasn't able to ask my question in an understandable way...

My point is: If many people start testing with the current beta, but the next version you're working on right now contains modifications that would change these test results completely, the modifications are already finished, there's just no new beta version released yet ... in this case it would make more sense to test the next version than starting tests right now. Of course - if you want a comparision between current beta and next one, you can ask for tests and people will help, as it has happened here before.

So, later versions will take advantage of 3.96 testing anyways. Of course with the implementation of new features, this advantage will slowly fade. Still... even if we dont get the absoluetely-total-optimum-version to do testing, future versions will take advantage of this. In other words - no effort gets lost.

Well, if - as in the example I described above - tests are outdated while they're done, the benefit from the effort is not optimal ...

Anyway, Robert's post answers my question - thanks.

I believe that the amount of testing required to possibly find a replacement for the current recommendation is very large. A great effort like this will not be made often and the amount of (free) time people are willing to invest should be wisely spent.

But why do we need a grand test before changing the official recommandation? Couldn't we simply use lame 3.96 for two or three weeks, and then decree 3.96 fully safe if nothing wrong was reported? It makes sense. I don't see anybody keeping Buschman 1.78c encoder, just because Klemm's encoders weren't collectively tested. Same for AAC, same for vorbis, etc... From where come this idea of a big, collective and complete check-up? And why is it so specific to lame? When a new Nero version is released, all people expect a new release of aacenc32.dll; they're not saying to Ivan "sorry, but PsyTEL 2.15 was more tested than Nero 2.6.2.0, I'll consider Nero AAC in two years if stable. Bye."

If people really care about a possible and serious regression, they have to prove it. It's not to other people to prove that the newest version is superior to the 27 months (!!) old 3.90.x.

Your position makes sense, guruboolez.
The burden of the proof lies on the person making the claim. LAME has undergone tweaking and has become faster at encoding. That alone is an improvement that warrants upgrading. If the upgrade hasn't been made, is because there is an implicit claim stating that 3.90.3 is still better than 3.96, despite the slower encoding and its age. Is it so? Prove it. Otherwise, the logical choice is to recommend the newer version, which in theory should be better.

To make it clear: there is no major quality improvment ready to be released just now.

There is no major quality improvement IN 3.96 beta over 3.95?

Or do you mean that there isn't any major quality improvement to be expected for some time AFTER 3.96 (beta)?

There is no major quality improvement IN 3.96 beta over 3.95?

Or do you mean that there isn't any major quality improvement to be expected for some time AFTER 3.96

there is no major quality improvment ready to be released just now.

3.96b has already been released, so its quality improvments are already released.

TNX for clarification.

I hope, that the HA recommendation can be updated now soon.

TNX for spending time with working on LAME, Takehiro, Gabriel, Robert and the others.

It seems to me people want to keep "waiting" to test the LAME because the Newest, Ureleased Code will be Bigger, Better and Faster (In a sense)....

I think why continue to wait and lets test 3.96...If you keep waiting for the bigger/better product you will be waiting forever 

There is no time like the present....

*EDIT* I just did some quick test of some songs. Lame 3.96 has reduced some 3.90.3 MP3's by 1.5meg...That is a huge improvement and it is faster too...

This new version 100% needs to be tested, considering how much faster and space it could save...

Howdy,
Here is a quick test I did of 4 songs that vary a little bit. Promising results considering how much space/bitrate was saved using 3.96....All were encoded using alt-preset standard

Lame 3.90.3  - Bill Medley And Jennifer Warnes - (I've Had) The Time Of My Life.mp3  205     00:04:50     7,464,328     
Lame 3.96b  - Bill Medley And Jennifer Warnes - (I've Had) The Time Of My Life.mp3  188        00:04:50    6,821,741

Lame 3.90.3 -  George Michael - Amazing.mp3                    206       00:04:27    6,891,347    
Lame 3.96b  -  George Michael - Amazing.mp3                    167                00:04:27    5,595,479    

Lame 3.90.3 - Guy Sebastian - Angels Brought Me Here.mp3               198       00:04:00    5,957,192    
Lame 3.96b  - Guy Sebastian - Angels Brought Me Here.mp3             170       00:04:00    5,132,260    

Lame 3.90.3  - Nick Skitz - Slave To The Music (Skitz Airplay Mixx).mp3          217       00:03:36    5,880,020    
Lame 3.96b  - Nick Skitz - Slave To The Music (Skitz Airplay Mixx.mp3          178       00:03:36    4,830,701


Cheers

Howdy,
Here is a quick test I did of 4 songs that vary a little bit. Promising results considering how much space/bitrate was saved using 3.96....

Lame 3.90.3  - Bill Medley And Jennifer Warnes - (I've Had) The Time Of My Life.mp3   205     00:04:50     7,464,328     
Lame 3.96b   - Bill Medley And Jennifer Warnes - (I've Had) The Time Of My Life.mp3   188    00:04:50  6,821,741

Lame 3.90.3 -  George Michael - Amazing.mp3           206     00:04:27  6,891,347 
Lame 3.96b  -  George Michael - Amazing.mp3           167              00:04:27  5,595,479 

Lame 3.90.3 - Guy Sebastian - Angels Brought Me Here.mp3           198     00:04:00  5,957,192 
Lame 3.96b  - Guy Sebastian - Angels Brought Me Here.mp3        170     00:04:00  5,132,260 

Lame 3.90.3  - Nick Skitz - Slave To The Music (Skitz Airplay Mixx).mp3       217     00:03:36  5,880,020 
Lame 3.96b   - Nick Skitz - Slave To The Music (Skitz Airplay Mixx.mp3       178     00:03:36  4,830,701


Cheers

you encoded all using APS?

*EDIT* Yes, All were encoded using alt-preset standard

I just put together some samples of my own, but I have no idea how good they are, as samples go.  I encoded the whole set to both 3.90.3 modified and 3.96b1 at --alt-preset 128.  3.96 seemed faster (but I did not get the exact speed), and produced slightly larger files in all cases except one (in which it was 1 KB smaller than 3.90.3) - on average, 2% larger.  I'm not sure how representative 30-second samples are, though.

I ABXed what I'm guessing is the biggest problem sample (a selection from (-) Ions by Tool), comparing wav to 3.90.3, wav to 3.96, and 3.90.3 to 3.96.  I won't post any results until I have completed ABX tests of the oher 9 samples (maybe in a week - my ears still hurt from this one).  I don't feel comfortable trying to ABX -aps, though. 

Basically, I'm all for testing out this newest release of LAME.  I agree that this far down the line, how fair can we be if we keep saying that 3.90.3 is the "best" version, if we really don't know much about how the more recent ones compare?  Maybe the test will show that 3.90.3 is still the best - I really don't know.  If that's the case, we've still done the LAME developers a service by thoroughly testing their latest offering, and giving a lot of feedback.  I think that they would appreciate it, no matter what the results are.  If people are willing to give some of their time, it's basically a win-win situation.  We either get a new (and possibly faster) recommended version, or we give the LAME team some much-needed feedback.

The poll results are overwhelmingly towards (B) 'Let's thoroughly test 3.96 now, and then possibly upgrade.' That's my position aswell. So despite the vocal minority most people by far (currently almost 20 to 1) *aren't* saying stick to 3.90.x.. hope this encourages the devs.

I voted for the 'test and possibly upgrade' option here. I never touch MP3 encoding anymore except for a few rare occassions now but it seems logical to me if the new release is good to use the refinements that have been done since 3.90.3
And as previously mentioned the wider the user/test base, the faster issues can be found and addressed.

Having started, then read through this whole thread, I am increasingly of the opinion that Guru is right, and that barring any ABX results suggesting a regression on certain samples, we should upgrade NOW. I mean, all things being equal, the new version is better already.

So, we should start a thread for people to post possible regressions in 3.96b. If no one posts anything important in the next month or two, why don't we just bump the version on the FAQs?

EDIT: I started the thread here. (http://www.hydrogenaudio.org/forums/index.php?showtopic=19441&)

But why do we need a grand test before changing the official recommandation? Couldn't we simply use lame 3.96 for two or three weeks, and then decree 3.96 fully safe if nothing wrong was reported? It makes sense. I don't see anybody keeping Buschman 1.78c encoder, just because Klemm's encoders weren't collectively tested. Same for AAC, same for vorbis, etc... From where come this idea of a big, collective and complete check-up? And why is it so specific to lame? When a new Nero version is released, all people expect a new release of aacenc32.dll; they're not saying to Ivan "sorry, but PsyTEL 2.15 was more tested than Nero 2.6.2.0, I'll consider Nero AAC in two years if stable. Bye."

If people really care about a possible and serious regression, they have to prove it. It's not to other people to prove that the newest version is superior to the 27 months (!!) old 3.90.x.

I guess it's all about quality. Many people come here and demand to know the utterly best quality mp3 machine. That is what HA.org stands for.

I dread the following scenario: a newer version is recommended after not so rigorous testing, lots of people adopt the new version and spend a lot of time encoding music, someone finds a serious flaw, fury is unleashed upon HA.org. OK, I'm exaggerating, but that is why I think serious testing is needed.

Outside of the HA.org community I think most people are allready using the latest and greatest version of LAME. Also note that the majority here uses LAME with aps for near transparent encoding and it is very close to flawless. As has been shown, the presets are very sensitive regarding changes to the encoder and have been broken in the past.

Most tunings happening around OGG Vorbis and AAC concern the "good enough" quality range and are more easily verified than near transparency settings. In a similar fashion the superior performance of Garfs' tuned Vorbis encoder (which is recommended on HA.org) can be verified because the version you are comparing it to has large room for improvement. The situation with LAME here is different, the recommended version is much closer to optimal and as such it is much harder to improve uppon, let alone confirm this by listening tests.

When work was being done on MPC, it was done by one very knowledgable person dedicated to high quality. There have been a few very critical listeners that have been running their own tests for each new version. The AAC team at Ahead has this "quality assurance" thing which should at least catch major hickups. Apart from the experimental tuning done by some dedicated people here on the forum, OGG Vorbis has only developed very little since the final release. The development process of LAME as a whole and especially the priority given to quality related issues as well as the acceptance of quality related user feedback have been subject to criticism in the past.

What will happen if it turns out that 3.90.3 beats 3.96 quality wise? If 27 month old code is superior to today's. Will we get the same response from the LAME team as in the past? "We only do this in our free time as a hobby so don't you dare expect anything from us. If you don't like it...". If feedback is unappreciated, then I won't give any. Seing that 3.95 now uses the presets as default, maybe this attitude has changed.

I would like a new version to be promoted to recommended mainly because of its speed and closer real-world-usage resemblance. (For example I've allready been usnig 3.95.1 and will now use 3.96b to transcode my mpcs for my portable.)

If 27 month old code is superior to today's. Will we get the same response from the LAME team as in the past? "We only do this in our free time as a hobby so don't you dare expect anything from us. If you don't like it...". If feedback is unappreciated, then I won't give any. Seing that 3.95 now uses the presets as default, maybe this attitude has changed.

I hope that you do not want to start a flamewar...In case you really want, please check and provide facts before.

First, for vorbis, the final 1.0 was immediately adopted. It was twice faster (at least) than RC3. But people used it without hesitations: they were just happy, and I never seen HA.org warning for this new release. On the other side, Nero AAC "quality team" seems to be something really new; therefore, it's not an argument, because people used to adopt each new version without deep verification (I'm maybe wrong, correct me if needed)

Second, we can't say to lame developers "we need to test thoroughly the new encoder before recommending it" and then never test anything.
HA.org testing model is not something optimal. It's even not something working at all. As far as I saw it in the past, the attempts to test alphas of lame 3.94 were completely anarchic, and never exceed a week. Lame 3.95 was released some times ago: no serious test.

Last, if there are serious flaws in lame 3.96, we don't need 6 months to find them. Two weeks of daily use of the encoder should be enough to find them. Again, if some people on HA.org or elsewhere fear that something wong might occur with a new encoder, they have to found it. That's why lame team released 3.96 as a BETA, not as a stable version, and give us three weeks for testing. If we can't test lame in three weeks, I suppose it's not responsible nor respectful to wait for a future and highly hypothetical collective investigation.
And don't forget that -Z switch is now something official. It means that the "deeply tested", widely used and strongly recommanded 3.90.1/.2 missed this important (?) switch. Therefore, I think we could accept that possible minor flaw might appear in the future with a new developping branch. No?

I hope that you do not want to start a flamewar...In case you really want, please check and provide facts before.

I don't want to start a flamewar so this will be my last post to this thread concerning this issue. I just recall some threads that were "weird" to say the least. The issue has been discussed to death allready and to me it seems that there is no common ground.

http://www.hydrogenaudio.org/forums/index.php?showtopic=3836 (http://www.hydrogenaudio.org/forums/index.php?showtopic=3836)
http://www.hydrogenaudio.org/forums/index.php?showtopic=4582 (http://www.hydrogenaudio.org/forums/index.php?showtopic=4582)
http://www.hydrogenaudio.org/forums/index.php?showtopic=4544 (http://www.hydrogenaudio.org/forums/index.php?showtopic=4544)

If you tell me that the test result, whatever it may be, will be put to good use by the developers and hopefully motivate them, I'll be happy to believe you until proven otherwise.

What you can see/read from these threads is that the 'problem' of lame development compared to Musepack, Vorbis or Nero's AAC encoder is that it is a lot more chaotic and  less controlled. Many people contributing/fixing stuff is definetly a good thing, but also carries dangers.
The other mentioned encoders have only one person activly working on the core for a good reason, the chance of breaking something by turning the wrong screw is just too high.

Developing an encoder is a difficult task and there's a German saying, which states that 'many cooks don't make a good soup'; in this aspect codecs are a lot like soup.

If you tell me that the test result, whatever it may be, will be put to good use by the developers and hopefully motivate them

Tests results are always usefull.
But in no way that means that results from tests will be immediately put in use to correct flaws/problems. The biggest problem is available time.

Tests results do not increase our available time, se we can not promise anything regarding when a specific problem will be corrected.
However, you can be sure that test results are helping to use our time more efficiently.

What you can see/read from these threads is that the 'problem' of lame development compared to Musepack, Vorbis or Nero's AAC encoder is that it is a lot more chaotic and less controlled.

You are right, and also gave the obvious explanation:

The other mentioned encoders have only one person activly working on the core

That's why 'problem' is in quotation marks.

I am all for change, but not for its own sake. I have to admire the optimism of some of the posters here. Surely the overriding issue with regards to the adoption of a new recommended version is is the quality similar, or better than 3.90.2/3? Whether, or not, it is faster is completely irrelevant unless the quality is maintained. If speed is your main consideration, go use Fhg, or Gogo.

There seems to be an automatic assumption that newer is better. This is just not necessarily the case. The whole point of 3.90.2/3 was that it took the tuning of LAME to a level that surpassed anything that was done before, or, in my opinion, since. Since then, the core of the encoder has changed, and so has the structure of the presets. This is not necessarily a bad thing, but without some comprehensive testing of the quality output by the latest encoder, something that has not been done since Dibrom's original work, this is no benchmark upon which to say that the the recommendation of HA should be changed from 3.90.3 to a later encoder; it simply flies in the face of everything that was done in the development of 3.90.2/3.

To adopt a newer version after a period of time simply on the basis that no one has objected to the quality seems to me to be completely contrary to the basic ethos of HA.

First, for vorbis, the final 1.0 was immediately adopted. It was twice faster (at least) than RC3. But people used it without hesitations: they were just happy, and I never seen HA.org warning for this new release. On the other side, Nero AAC "quality team" seems to be something really new; therefore, it's not an argument, because people used to adopt each new version without deep verification (I'm maybe wrong, correct me if needed)

I don't recall that Vorbis RC3 had undergone any rigorous testing and tuning which is the key difference here. So unlike LAME 3.90.3, it was not "tested and approved". Maybe the way how 1.0 was adopted wasn't exactly optimal, that is why we should avoid such mistakes here.

I don't know much about Ahead's QA. I also have no insight in the way that quality related changes were made to the original Psytel encoder.

I'd like to turn the argument around. In general it isn't good practice to blindly adopt new software versions without testing. In the industry you will see lots of Windows NT stations and old Linux Kernel versions because there people have to be anal about tried and tested reliability. In a similar fashion some people are just as anal about the quality of their mp3s. Other concerns, such as encoding speed, are so minor that they do not make a new version appealing enough, if it hasn't been "tested and approved".

Second, we can't say to lame developers "we need to test thoroughly the new encoder before recommending it" and then never test anything.

I agree.

HA.org testing model is not something optimal. It's even not something working at all. As far as I saw it in the past, the attempts to test alphas of lame 3.94 were completely anarchic, and never exceed a week. Lame 3.95 was released some times ago: no serious test.

Unfortunatley, yes. So we need to spend further effort on coordinating the testing process, if we want to get it right. Proposals have been made (http://www.hydrogenaudio.org/forums/index.php?showtopic=4336) but discussion circles more around LAME development organization and it has been questionable if the desired goal of an improved LAME version can be reached.

Last, if there are serious flaws in lame 3.96, we don't need 6 months to find them. Two weeks of daily use of the encoder should be enough to find them. Again, if some people on HA.org or elsewhere fear that something wong might occur with a new encoder, they have to found it. That's why lame team released 3.96 as a BETA, not as a stable version, and give us three weeks for testing. If we can't test lame in three weeks, I suppose it's not responsible nor respectful to wait for a future and highly hypothetical collective investigation.

Hm... but at least aps wasn't just made for daily use but also for critical use. Maybe we would be able to find most serious flaws, but we also want to rule out subtle flaws as much as possible. I don't think daily use is going to cover that.

And don't forget that -Z switch is now something official. It means that the "deeply tested", widely used and strongly recommanded 3.90.1/.2 missed this important (?) switch. Therefore, I think we could accept that possible minor flaw might appear in the future with a new developping branch. No?

At least this shows that HA.org is indeed capable of adopting changes to the recommended version if further testing reveals an improvement.

I would like to suggest adding at the top of the "recommended compile" thread something like "As a HA member, you are strongly encouraged to use LAME 3.96 b1 (or the latest alpha/beta version that is out) whenever possible, and report any problems to the forum." in bold type.

Would this work?

I am always using the latest version of LAME since 3.92. I never used 3.90.1!

I trust the people who are behind LAME and that they learned from the 3.93 desaster. I think, no one can test the entire set of music of the world, so no one will ever knew, if one version is always better than another. But I hope, that their set of critical material is big enough to ensure a statistical chance.

Let's make the latest version always the recommended version!

I'm going to start using 3.96 from here out.  My initial results have been promising.  Definitely faster and definitely smaller files.  So long as nobody else has big issues with it, I'm thinking about doing a re-encode of my entire ~250CD collection  .  It currently takes up about 26-27GB, and if I could get that down a couple gigs, it would be worth the trouble.

I would wait and see over the next few weeks...
It looks as if this test isn't going to happen THO...

The thing is... people are looking at different aspects of this debate separately, which isn't right.

On one hand most of us seem to come to the conclusion that unfortunately, and despite the hard work of a few people, listening tests around here lately are... not what you'd hope, for several different reasons already mentioned earlier in this thread.

...And then on the other hand a lot of people are happily saying "Hell yeah, let's test the latest version, it would be great to have a new compile!" It would, but a lot of people already agree with the above paragraph, which makes that solution unworkable... it's a bit like saying "OK the solution is to have LAME 4 ready and stable in a month or two." That would be fantastic but it's not feasible, just as having a successful extensive test of the latest compile probably isn't feasible either.

So to sum up those last two paragraphs, I think most people are not looking at the real problem - the testing. The first step to the final solution here is to sort that out, give it some real structure and very clear goals (after determining which goals should be achieved; i.e. which settings to test most, whether to ABX against 3.90.3 or the original wav, or both, etc.). Then we need to see whether that is a realistic aim... for example, I and many others would love to be involved in a test of ABXing --aps 3.96 against 3.90.3; but our hearing just isn't good enough and so taking part in the test would be useless at best and counter-productive at worst. On the other hand, I'm not too keen on testing new compiles at lower bitrates either... because I don't use them and won't use them in the future.

However if a structured test was arranged where many people with the skills to ABX --aps agreed to do things like that, under the understanding that people with lesser hearing such as me took some time to ABX bitrates ~128... that's a different story. But this is what I was talking about in my first paragraph, we need a planned structure for this mass test with clear goals otherwise it probably won't work. But if we do plan it correctly and everyone does their bit then it would result in us all having a shiny new tweaked compile to play with. If.

A follow-on point to all that is that it might still not be worth it - that's a massive amount of effort for a lot of people, with minimal gains - i.e. --aps 3.90.3 and 3.95.1 are transparent for most people. I think the desire for this "shiny new tweaked compile" might result slightly from the placebo effect we all talk about so regularly, albeit about something slightly different.

Weighing up all the views/opinions/conjecture/suggestions brings me no conclusions, there's just too much to bear in mind. So looking at it logically I would have to say that as 3.90.3 has never failed or disappointed me with either quality or filesize, and version 4 is being developed (no matter how slowly...), I'd rather concentrate on saving my efforts for testing that.

I think that a massive point here is that a mass-tested and highly-tweaked version 4 compile would make 3.90.3 pretty much obsolete - if not with regards to quality of sound for most people, then with regards to speed and filesize. On the other hand, a tweaked 3.95.1/3.96 will be a fairly minor step in comparison. Again, I say that whilst comparing it to a tweaked version 4 and pointing out the massive effort and organisation involved in arranging a suitable test.

Last of all I'd like to say that unfortunately (and I do mean that) I think the results of this poll are completely useless. I am convinced that most results are people chosing the "let's test 3.96 now and have a new recommended version!" option because they hope lots of other people will work hard, so that in a few weeks they can proudly download a new recommended compile from Rarewares and feel good about themselves. I'm sure I don't need to point out that when most people take that stance, it's not going to happen. I say that not to patronise people; I admit that I did exactly the above myself on my first read of this thread about 2 days ago. Hopefully this post can put my mistake right.

Well that is your opinion...Why shouldn't we test 3.96 tho? 3.90.3 was tested so long ago that IMO I think it would be nice to be able to use the latest version of Lame...Testing would also help the LAME developers...What is LAME V4 dosen't see the light of day for years and years? We will be waiting forever. I say test now and hopefull upgrade....

You know why I don't bother switching from LAME 3.90.3?

Quite simple, because no one has organized a listening test that is guarenteed to be listened to by the LAME developers.

There is many reasons why --alt-preset standard has been so successful. Tossing aside the obvious quality gains, Dibrom's internal tweaks, and so on, these what stood out to me when I watched development take place as a user (I hope I am remembering right, it has been so long and I wasn't an active poster at the time, just a reader):

1. Dibrom asked in a thread on the forums for people to test out an alpha/beta version against all known killer samples plus normal everyday music.
2. Everyone (especially those wanting to leave --r3mix behind) followed Dibrom's instructions pretty much to the letter. Tons of music and killer samples were tested using ABX.
3. The results of the tests were posted on the board. Dibrom and the rest of the community was able to verify or debunk results very easily with another ABX test, this time by themselves.
4. New compiles were posted, new listening tests were organized, and everything was pretty much repeated.
5. The entire time Dibrom and those who helped him out were talking practically in real-time with the community, asking for input, trying to isolate problems. Everyone could read these discussions and everyone knew what was going on practically to the minute.

At the end, we had the most amazing MP3 encoder ever released. Dibrom and those who worked with him did all of the hard code work, but we as the community were able to have communication enough with him that we could easily organize a strong effort to test his work. The results speaked for themselves.

Then Dibrom left LAME development.

Communication has broke down between the community and those who do LAME development. If the LAME developers even bothered to post on Hydrogenaudio, it was and still is briefly and not often.

My impression has been that users (including myself) feel the developers just don't care about us. No one that we could rally behind was or is giving instructions and guidance on where to focus our endless energy. No one was or is even telling us what is upcoming in the future.

All of us are very interested in development of our favorite MP3 encoder. Sure AAC, OGG, MPC, etc. maybe sexier right now, but last time I checked MP3 by far is the most universal and standard codec today. Any improvements, even small ones, can effect the MP3 world for the better in a big way. That is why we are all drawn to LAME. That is why --alt-preset standard was such a huge success. If you asked me years ago when I was used Napster if we would get --alt-preset standard quality from MP3's at a reasonable bitrate, I would laugh my butt off. But look, it truly happened! It wasn't some large corporation or business trying to sell us the latest Musicmatch or RealPlayer type software. It was a community of dedicated people from across the world, with diverse music tastes, languages, and even time differences that put this together.

It used to be that there would be a big announcement on when a new alpha or beta version of LAME was released. All of us who crash the FTP servers just to get our hands on the release to test it. Now, we jumped from version 3.95 to 3.96 stable without one public notice of test alpha/beta versions that I can remember seeing and I check Hydrogenaudio daily. In the meantime, companies/people that release software that uses LAME always upgrade to the latest version that could/probably have bugs in it that would of easily been caught had there been a proper test early on in the alpha/beta stages.

What happened to the organized tests? What happened to the developers showing everyone that they care? Dibrom back in the day (man I sound like we are talking about 50 years ago!) gained his huge following because he cared about those who would be using his work. I guarentee Dibrom gained more staisfaction (and still does to this day) about how many people enjoy LAME (and talking with everyone about how how he improved/fixed certain issues) because of --alt-preset standard and his work then the satisfiaction he got from coding it.

Developers, show us you care about our feedback and that you want us to use LAME and we will show endless passion in return to help out. You may lose a little coding time, but in the long term you will gain a ton more.

If the developers don't care about us and don't change, worst case we are stuck with 3.90.3 as the last truly stable and tested version. That isn't the end of the world (there are still millions of people who are missing out on these benefits from --alt-preset standard), but we can still do better.

This is your chance LAME community and the developers. This is all I have to say.

Your Posts speaks the truth so well...........

Communication has broke down between the community and those who do LAME development. If the LAME developers even bothered to post on Hydrogenaudio, it was and still is briefly and not often.

Are you telling that I am not posting here?
Are you telling that there was no tuning done during the long 3.94alpha stage?

so what is that:
http://www.hydrogenaudio.org/forums/index....2;.94,and,alpha (http://www.hydrogenaudio.org/forums/index.php?showtopic=14385&hl=3\.94,and,alpha)
http://www.hydrogenaudio.org/forums/index....2;.94,and,alpha (http://www.hydrogenaudio.org/forums/index.php?showtopic=9044&hl=3\.94,and,alpha)
http://www.hydrogenaudio.org/forums/index....2;.94,and,alpha (http://www.hydrogenaudio.org/forums/index.php?showtopic=8520&hl=3\.94,and,alpha)
http://www.hydrogenaudio.org/forums/index....2;.94,and,alpha (http://www.hydrogenaudio.org/forums/index.php?showtopic=6390&hl=3\.94,and,alpha)
http://www.hydrogenaudio.org/forums/index....2;.94,and,alpha (http://www.hydrogenaudio.org/forums/index.php?showtopic=7790&hl=3\.94,and,alpha)

Now, we jumped from version 3.95 to 3.96 stable without one public notice of test alpha/beta versions that I can remember seeing and I check Hydrogenaudio daily

So what is that:
http://www.hydrogenaudio.org/forums/index....showtopic=19387 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19387)

Please check your facts before posting.

guys, you'd better thank all lame dev team for their efforts instead of starting a flame... especially if you don't follow lame dev very closely... also, understand that HA is not the official lame forum...

i've been using presets since the very beginning but i must say that i'm using only 3.95. now simply cos i find it fast and cos it sounds as good (if not better) as 3.90.3.
testing new versions and making them "official" will only improve lame's quality encoding. this is up to the original creator(s) of --alt preset to decide wether or not 3.90.3 is outdated now...

encore merci gabriel (et les autres) pour vos efforts!

I must say I think chrisgeleven's comments are taken out of the blue and not really representative for what most people think. Personally I think the LAME progress has picked up a lot of speed lately and I see no signs of the opposite. Sure, the testings have been a bit unorganized maybe, but there are easier things than that!

Just my opinion...

Although I may not fully appreciate the history of this situation, it does seem to me that some of the criticism against the lame development team here is strongly unfounded.  In particular I am very grateful to Gabriel for the continued presence he has on these boards and his swift and helpful involvement in issues and questions that are raised by board members.

I do think it is unfortunate that there does not seem to be a member of the lame development team who has responsibility for tuning based on public listening tests, and who leads in this area, driving it forward.  Although I suppose most encoder development does not have this, I think the gap is noticed because Dibrom did this in the past.  However, this is not in any way a criticism of the existing members of the development team, as clearly this is not their area of expertise.  It is just a shame that nobody else suitable and interested has taken this role.

I believe that your work is appreciate by a huge silent mass that diminishes the size of the small group of negative voiced opinions on forums like these

This 100% true, this board is just a frunction of the actual users and not in any way a statistical point of view, and yes some people here are too damn negative about many things.

Some people here often write about the developers not doing testing and don´t react on feedback.
Men you are wrong. Gabriel, Robert and other developers once came to R3mix only for helping with
tuning lame! Dibrom as good as his work has been wouldn´t have been able to tune 3.90 without
the help of some developers! I can remember Naoki and Gabriel tuning the code together with
Dibrom and others very time consuming. When you read this Dibrom, correct me if i am wrong.

btw. if some of you had invented the same time in testing some standard problem samples instead
of wasting so much time writing here we may be further!


Wombat

Communication has broke down between the community and those who do LAME development. If the LAME developers even bothered to post on Hydrogenaudio, it was and still is briefly and not often.

Dibrom was responsive on Hydrogenaudio because it's his forum. I'd imagine that the LAME developers have better communication in their own mailing lists and boards. Gabriel is a big contributor here, and I think he acts as a go-between for HA and the other LAME devs. It's a bit pompous to demand that all the LAME developers abandon whatever communication system they're using and borg with Hydrogenaudio.

Okay, now I have a question about the state of the Presets in the new Lame versions. Dibrom claimed that his presets were tuned to very tight thresholds, and that any change to noise measurements or whatever would throw the Presets out of whack. In other words, he said the Presets were tied to the 3.90 codebase. To my understanding, this was because he cobbled together many different algorithms and switched between them when appropriate, and he didn't improve the underlying algorithms themselves.

Is this true? Have any of the 3.94 testing threads shown that all the Preset test samples have fallen apart? Or are the Presets more resilient than Dibrom suggested? Can all the carefully-determined thresholds be carried over, or should new thresholds be determined?

Or should we try to move away from the tweaks-and-hacks of the Presets, and concentrate on command lines only? That way, the results of the tests would be more future-proof, because they wouldn't be as fragile as the Presets.

Or should we try to move away from the tweaks-and-hacks of the Presets, and concentrate on command lines only? That way, the results of the tests would be more future-proof, because they wouldn't be as fragile as the Presets.

BTW is there a command line alternative of ..lets say -APS ?
and if not why ?

In any case, none of the bitrate Presets (e.g. --preset 128) in 3.90.3 received the magic tweaks programmed by Dibrom; they're just simple command line combinations. They were tested and tweaked, but not to the extent of --preset standard. We should certainly  consider replacing the "HA recommended" Lame compile for the bitrate Presets, if nothing else.

OT: how are new problem samples found? I'm not familiar enough with my music to notice new artifacts just from listening, unless the errors are really obvious. None of the MPC problem samples, for example, would have been caught by me simply by listening to the MPC sample, although I can ABX them. Does someone who finds a problem sample for an otherwise-transparent encoder simply listen to albums they know like the back of their hand?

Quote

Or should we try to move away from the tweaks-and-hacks of the Presets, and concentrate on command lines only? That way, the results of the tests would be more future-proof, because they wouldn't be as fragile as the Presets.

BTW is there a command line alternative of ..lets say -APS ?
and if not why ?

from the FAQ (http://www.hydrogenaudio.org/forums/index.php?showtopic=7516&#entry74068):

Quote

Originally posted by xmixahlx
1. what are the current switches in each of the presets [i.e. extreme, standard, and their "fast" counterparts].

The actual switches are irrelevant. Due to use of code level tweaks the switches used are not comparable to normal use of external switches.
Just some special tweaks in alt-preset vbr settings:
- tweaked block switching threshold (short blocks/long blocks)
-Adaptive noise measurement (uses X3 when needed)
-Tweaked noise shaping functions (uses -h/-q2 but the use of scalefac_scale is more intelligently controlled)
-short block tweaks (bitrate, nsmsfix values)
etc.etc.
These and many other things can't be done with external switches, and the meaning of the swithces is not the same anymore.

In any case, none of the bitrate Presets (e.g. --preset 128) in 3.90.3 received the magic tweaks programmed by Dibrom; they're just simple command line combinations. They were tested and tweaked, but not to the extent of --preset standard. We should certainly  consider replacing the "HA recommended" Lame compile for the bitrate Presets, if nothing else.

This (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191563) sample is handled better by 3.90.3  (don't know any of the opposite) using --alt-preset 128, I'm sure most people can ABX with ease. This raises the question was it only Dibrom's tunning that has kept 3.90x the best performing LAME mp3 encoder or is it that LAME development since then has become some what chaotic ?

This (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=191563) sample is handled better by 3.90.3  (don't know any of the opposite) using --alt-preset 128, I'm sure most people can ABX with ease. This raises the question was it only Dibrom's tunning that has kept 3.90x the best performing LAME mp3 encoder or is it that LAME development since then has become some what chaotic ?

Well, we've got a sample that regressed from 3.90.3 to 3.96, but I don't think that's any reason to accuse the Lame developers of being haphazard. From my brief dabbling in the 3.94 alpha testing threads, I've gathered that there are new command line options that aren't being used by the bitrate Presets that could give significant quality improvements, for example blocktype-specific noise measurements. It's quite possible that some other 128kbit/s 3.96 commandline would out-perform the 3.90.3 --alt-preset 128 on your sample and many others.

Good work spotting that problem sample, but try to see if you can get 3.96 sounding better by fiddling with switches! Of course, it would be nice to know which switches to fiddle... hopefully there's some documentation on HA about this, or perhaps Gabriel can issue a statement regarding what parameters we should be playing with.

Gabriel, I'm sorry if I offended you, I have noticed that you post on here the most of often of the LAME developers and are one of the exceptions to what I observed. I am really sorry!

That said, all of those posts you gave me show one page (maybe two) threads about a new alpha/beta version of LAME. I honestly don't remember ever seeing these threads and their relative unpopularity might be the reason why I (and probably many others) didn't even notice these threads.

Your posts that you showed me are a good start to what I was talking about. However, we can do better. Get a master 3.96 thread made a sticky at the top of the MP3 forums. Have announcements of testing for LAME 3.97 Alpha and Beta posted on the front page of Hydrogenaudio. Encourage feedback.

Maybe we could have a bi-weekly status update about the direction of LAME, what has been updated, etc. in the latest builds and what we can expect in the future? Every little bit of information helps.

This is the one case where I hope my comments on the last thread are proved wrong or are fixed. I do not want LAME to stop being improved the best it can be improved.

So is a public test going to be conducted that shows Lame 3.96b better then 3.90.3?I wish I knew how to start such a huge task but I dont...

Everytime a new LAME versino comes out, it's not tested and IMO it's wasted...

I am still hoping that a TEST will be conducted so we know really IF the new version of LAME is better....

My opinion: no stable releases until a release is highly tested by the community. Throw up Release Candidates or something. Anything but a final stable release like 3.96 until the community has enough time to really help out and identifiy problem areas.

We do this, combined with some better communication, and we can release a stable version that can right away become the recommended one.

Nice idea to count test samples that became better or worse with 396b!

I can add here sophia2 is clean the first time with aps!

390.3 -> sandpaper noise
395 -> added plop
396b -> clean

Wombat

There is a thread, the purpose of which is to look for all of the samples that became worse with 3.96.b1. It seems as though nobody is looking for samples that got BETTER with 3.96.b1.

So, lets say there are 3 known samples that got worse with 3.96b1, and that there are 10 samples that got better. If nobody knows about the 10 samples that got better, then everybody will say that 3.96b1 is not good enough.

Maybe you could help find the samples that got better. Unfortunately, I can not.  My ears are too old I think.

Call it the progression thread

There is a thread, the purpose of which is to look for all of the samples that became worse with 3.96.b1. It seems as though nobody is looking for samples that got BETTER with 3.96.b1.

So, lets say there are 3 known samples that got worse with 3.96b1, and that there are 10 samples that got better. If nobody knows about the 10 samples that got better, then everybody will say that 3.96b1 is not good enough.

Maybe you could help find the samples that got better. Unfortunately, I can not.  My ears are too old I think.

Call it the progression thread

I figure this isn't as important, since there are already tangible benefits to the newer version (faster, lower bitrates with presets), meaning that all things being equal quality-wise we should switch. An improvements thread would still be interesting information though I suppose.

I'm disappointed nobody has replied to my post on the last page, we're all still talking about our hopes and the theory of development "a new version would be great!"; "A mass test would be great!" Yeah of course they would but so would LAME 5 which produces transparency at ~160kbps... but it's not going to happen for ages in real terms yet, even though most people might agree that it is feasible with enough work and research.

Everyone needs to stop saying "IMHO a new version would be great!!!" and actually think about what is realistically achievable and what you as an individual are willing to/capable of doing to help. Otherwise this is useless to everyone and serves only to pass your time and make you (incorrectly) feel like you're forwarding the development of LAME. It reminds me of a load of 15 year-olds on a car forum discussing which Ferrari is best even though they don't know how one works and will never even sit in one. The theory of it is great and it's also great to feel like you're part of something but the most important thing to realise is that just stating opinions of how you personally would like a codec to develop is useless if what you really mean is "hopefully, as usual, others will do all the work so that in a couple of months I can feel good about myself whilst using a spanking new version of LAME", I'm convinced that's what a lot of people here are thinking anyway, as is the case with most things.

I think that when I said "I think that we should possibly wait for LAME 4 before a mass-test for these reasons..." in my last post a lot of people thought "Well, fair enough, that's his opinion, he thinks the 3.xx line is not worth the bother..." but that's not the case, I just think that an appropriate mass test and tuning won't happen. I hope I am proved wrong and I am willing to participate in ABX tests to the best of my ability but I fear I'm right, unfortunately.

I just think that an appropriate mass test and tuning won't happen. I hope I am proved wrong and I am willing to participate in ABX tests to the best of my ability but I fear I'm right, unfortunately.

If you don't believe an appropriate test will be given to 3.x.x versions, what makes you believe things will change when version 4 comes out?

There's far more time to prepare a suitable structure for the test, i.e. overall goals and exactly who will be doing what.

If we did that now then by the time it was all over and the tweaks had been completed then 3.97/8 would probably be released and there would be a "Keep 3.95.1 forever or test 3.98 now!?!?" thread with carbon copies of these messages in.

Most people only think in version numbers, they're used to updating their copies of Nero/MP3gain/whatever else every couple of weeks and happily looking at the version number progressing... when the same doesn't happen with LAME they start to get uncomofortable.

Of course that's not true of everyone and 3.90.3 is not by any means perfect. I am also aware that all of that is speculation; but that's most any of us can offer in this thread without going off-topic anyway.

Personally, I'm of the opinion that the newest version of LAME should always be the recommended version.  Regardless of regressions, progression is the ultimate end we are seeking here.  Having a recommended version that never changes tells me that everyone is happy with 3.90.3.  If every body is happy with that version, why does LAME development continue? 

Basically if the newest version of LAME was always the default recommended version, it would spur testing and further developmental progression.  Regardless of regressions in newer versions, the goal is progression.  I seriously doubt we will progress much with LAME development if we keep falling back upon 3.90.3.  We need to put that version behind us and start fixing what the newer versions regress.  Sometimes one needs to sacrifice something good for something better in the end.

Your opinion is interesting, ChagFest. After all, this is lossy encoding. We shouldn't be so afraid of a small regression, that would be negligible, after all, compared to the losses that MP3 introduces anyway. If everyone uses the last version, problems will be found faster.

I think some people are missing the real issue, that a much older lame code (over 1 year & many releases) is outperforming the latest, this means as far as quality lame is getting worse not better so how can there be any progress?

Testing LAME with all it's options is surely a huge task!

But I bet, the majority of people on this board and around the world are using preset standard for encoding most of their stuff.
A poll could make this point clear.
So I think testing aps only would be a good point to start - and most of the people confident that something is moving forward.
If 3.96 performs well or at least not worse soundwise against 3.90.3, someone could compile an "aps only" version and post it as recommended aps version under the "most tested and recommended version for all other purposes".
When this step has taken, other presets could be tested and implemented.

That could reduce the effort and make most of the LAME users happy. 

Ok haven't read the entire thread (needs to be split alot of O/T stuff in it), but these are my comments...

I am sure there are alot of users out there on this board who only upgrade their software if there is some dire need to do so (e.g. new feature/bugfixes e.t.c.). I personally am one of those people, if the software does what it says it will do then I am happy with it why upgrade and possibly ruin the status quo. But then again without upgrades there will be no innovation in the process, but thats where just making --aps the "standard" switch and letting users decide which version of LAME to use. As Pio2001 said "shouldn't be so afraid of a small regression, that would be negligible..." and most serious bugs are found within a week or so anyways. And when there is a "killer" sample that affects a version and everyone runs around shouting "OMG!!" just remember that it most probably effects just that sample cause if it affected more that just that I am sure it would of been found within that first week of a version coming out, hence it will be an isolated case. This solves both problems... those who are happy with the status quo and those who need the latest and greatest (personally I use 3.92 for all my encodings and its yet to fail me).

I will just say any stable release LAME can be used and make the recommended setting --aps, that way its standardised and if something goes wrong we know which version it is.

So to recap make --aps the RECOMMENDED setting which means only 3.90.3 LAME onwards have them.

Regards

AgentMil

I think some people are missing the real issue, that a much older lame code (over 1 year & many releases) is outperforming the latest, ...

Is this a fact? If you've done extensive tests and came to this conclusion, please publish details. Would be a great help for what this thread is about. 

I don't want to join the discussion if it is quite neccessary to retune the presets with LAME 3.96b1 or not, to stay with 3.90.3 or with 3.96b1.
We can fill a lot of pages about talking if there's any need or not (and a lot of pages are filled right now), one means 'yes' cause he likes to push things to its limit to get out the best, another one means 'no' as he is quite happy with the quality of his mp3s.
I like ping-pong but why not go to the next step and do the same thing like everytime - those ones do the work who like to do it.
How to retune those presets? I don't know, I guess some others also doesn't know, but someone in this room could know and has to give detailed explanations or instructions.
At that point we have an additional feature: we are able to do Retuning instead of only talking about it.
Next point would be some organisation stuff, rules of playing, ...whatever
At least I can tell you that I would be interested in Retuning the presets with LAME 3.96b1, but I don't know how (this thread (http://www.hydrogenaudio.org/show.php/showtopic/18133) didn't made me know how to,but well,the question was not ask directly), for that I stay tuned *bzzzt*
...And the echo sounded from far away: Sorry, I interrupt your discussion...

Is a test going to be organised and coodinated? I don't even know where to start..It would be great if  some great of HA could come up with a test, so we can finally prove 3.96 is better....

I agree with the idea to start by testing "preset standard" and tuning it first.

mp3!

I'm looking at an old HA thread that links to Lame problem samples. Here's the thread: http://www.hydrogenaudio.org/forums/index....=ST&f=16&t=791& (http://www.hydrogenaudio.org/forums/index.php?act=ST&f=16&t=791&). However, none of the links work anymore. Is there a way to get these links working again? Are they stored somewhere else on the HA site?

Also, I thought the Test Sample Archives (http://www.hydrogenaudio.org/forums/index.php?showtopic=4601&) sticky used to have a link to a few dozen HA-hosted samples, but perhaps I am mistaken.

I've been using the latest LAME versions for quite a while now (Since some of the later 3.94 alphas, IIRC) for my MPC-MP3 transcodes for my portable. It works great. Overall, I've had fewer problems in this transcode pathway with recent LAME versions than I have with LAME 3.90.2.

I think I voted for the "wait for LAME 4" option. Honestly, I think the HA recommendation could be upgraded to --preset standard with LAME 3.95.1 and noone would notice a difference. There would be some people who'd notice regressions, others that would notice progressions, but overall there wouldn't be much in the way of change.

Gabriel did well when he asked for testing of the 3.94 alphas. There was a lot of work put into the 3.95 tuning. Although it may not have been as extensive as Dibrom's 3.90 tuning, there are lots of other factors that have been improved, such as                                                            speed.

Remember this: When the 3.90 tunings were released, the other "high-fidelity" option was --r3mix. Is 3.95 better than --r3mix? I'd bet on it. Is 3.95 better than 3.90? Meh. I'll never hear the difference. There's enough work there anyhow that it shouldn't matter much. Let's just say that 3.96 is the "recommended LAME version in testing" or something ang make sure that it is.

Guruboolez is dead-on accurate with what he says, IMO.

we can't say to lame developers "we need to test thoroughly the new encoder before recommending it" and then never test anything.
HA.org testing model is not something optimal. It's even not something working at all. As far as I saw it in the past, the attempts to test alphas of lame 3.94 were completely anarchic, and never exceed a week. Lame 3.95 was released some times ago: no serious test.

Well, even though his prediction was not 100% accurate (I think this time the test is lasting longer than a week  ), he had a point, and his post had some truth to it, considering the number of people that actually tested the new version and submitted the results.

I think the reason that people don't participate in the test is one or more of the following reasons:

1) they "moved on" to another format (like MPC)
2) they don't have the time
3) they don't think they have good enough ears
4) they would rather surf the web than doing something for the community
5) they don't care
6) they don't want to

I think #1 is a biggie.

1) they "moved on" to another format (like MPC)
2) they don't have the time
3) they don't think they have good enough ears
4) they would rather surf the web than doing something for the community
5) they don't care
6) they don't want to

I think #1 is a biggie.

#1 is not the biggie, but all others are

How about this simple reason...

At transparent bitrates, there are already more samples where 3.96 is known to be worse than 3.90.3, than there are samples where it is known to be better.

http://www.hydrogenaudio.org/forums/index....opic=19813&st=1 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&st=1)

This shows that 3.96 can't possibly be recommended over 3.90.3, so surely there's no point in any more testing (of this specific quality region) until a new version comes out?

(Having said that, I'm tempted to ABX my previous listening to get the results added to that thread, but still...)

Cheers,
David.

#3 here...

Sergio

At transparent bitrates, there are already more samples where 3.96 is known to be worse than 3.90.3, than there are samples where it is known to be better.

According to my results there is a worsening with abr/cbr modalities too. Unfortunately i've not provided 3.90.2 vs 3.96b1 abx and the results are greyed. I'm 100% secure of the results because artifacts between the two versions are quite easy to ear (and to see with spectral analisys) and abx is not really necessary so (personally speaking) there is no need for a new test. If you would try to confirm chirping/ringing problems i noticed with --ap 128 there are some easy samples: rebel, campestre, applaud.
I will stay with 3.90.x for now.

Well, from the first tests promising enhancements the testing slowed down as it was clear it isn´t superior.
For me at least. Encoding speed is no plus at all for me.

I hope we will get as many or even more testers if the lame developers need feedback quality wise when they introduce enhancements.

Unfortunately it was like "We have a beta version soon to release, please test it"

So testing this time lead to nothing but knowing 3.96 is inferior.

Wombat

2Bdecided, most likely you're right. Even if 50 more problem samples appear, it's highly unlikely that 3.96b1 outperforms 3.90.3 in a significant way for (alt)preset standard that would justify making 3.96b1 the recommended version. I doubt that the raised minimum bitrate in beta2 will solve enough problem samples to bring 3.96 back in the race. Anyway, this must be tested to be sure.

@[proxima]: I inclued your results greyed out to the results page to encourage some other people to veryfy them. You have done enough already, thanks again. Unfortunately this didn't work so far. But this might be the chance for "#3-people" (smz?  ) to contribute...

There's one thing left that would be still worth testing IMO:
For lame 3.90.3, the ABR presets were created/chosen as recommended lame settings because they were superior over VBR settings with comparable bitrate. Since Gabriel has tried to create high-quality medium bitrate VBR settings (-V 3 - 6) it's interesting how these perform compared to ABR presets. Based on the tests mentioned in this thread (http://www.hydrogenaudio.org/forums/index.php?showtopic=19876&) (see recent posts by evereux and me), there are 2 bitrate ranges that should be worth testing:
(~160kbps) 3.96 -V 4 vs. 3.96 --preset 160 vs. 3.90.3 --alt-preset 160
(~128kbps) 3.96 -V 5 vs. 3.96 --preset 128 vs. 3.90.3 --alt-preset 128
This range is especially interesting for portable use.

While I have the ear of the lame developers - I'd like to ask a question you've probably already been asked many times.

Do you think there's still a lot of room for improvement in the mp3 framework or is a shift to a new format (aac, ogg, etc) necessary? 

I suppose what I'm asking is if you think it's possible for mp3 to achieve transparency on most samples at bitrates below 192kbps, given that it seems that most people agree that it currently does this above 224kbps or so for most samples.

MP3's transparency (or as close to it as it reasonably gets) is usually regarded to be achieved with Lame's 3.90.3 "alt-preset standard" setting that averages at around 200 kbps, which, as an average, is quite below 224 kbps. While it is still usually above the 192 kbps you say, I don't think it would be too farfetched to think that with proper development, testing and tuning, Lame could push this bitrate some 10 kbps down with the same quality and hence hit the "average 192 kbps mark" you want. This is pure speculation from my part, though, and it is clear that MP3 is an old and mature format that has probably got most of its juice squeezed out by now.

(~160kbps) 3.96 -V 4 vs. 3.96 --preset 160 vs. 3.90.3 --alt-preset 160
(~128kbps) 3.96 -V 5 vs. 3.96 --preset 128 vs. 3.90.3 --alt-preset 128

Those settings look really interesting. I will test those 
By the way, could this be something like what Dibrom was trying to do toward the very end of his contribution to LAME? He was also talking about more flexible ABR modes. Whatever happened to that idea?

edit: total revision

At transparent bitrates, there are already more samples where 3.96 is known to be worse than 3.90.3, than there are samples where it is known to be better.

http://www.hydrogenaudio.org/forums/index....opic=19813&st=1 (http://www.hydrogenaudio.org/forums/index.php?showtopic=19813&st=1)

This shows that 3.96 can't possibly be recommended over 3.90.3, so surely there's no point in any more testing (of this specific quality region) until a new version comes out?

Several of those samples are pathological cases where the psychoacoustic model is failing, and 3.90.3 is only saved by its minimal bitrate of 128kbps.

3.96b2 was released with the same minimal bitrate (128kbps).

OK, I'll try some.

Cheers,
David.

the --noreplaygain switch is not supported by John's compile..  is it still debug-only?

edit: hmm just pulled CVS, works here. outdated version on RW

Quote

Also, I think it kinda sucks for Gabriel and all the other LAME developers who have put so much time into the new releases and tuning them...since LAME 3.90 is still recommended it looks like if all there work from then was useless

I agree 100%. For me, I think it's time for an upgrade, 3.90.3m has been great, it still is, but it's just too slow. If I can use a faster encoder without sacrificing quality one bit, then definately I'm gonna go with the newer version.

lame 3.96 was cool for me, but right now i am using 3.97 alpha