Skip to main content

Topic: LAME compiles test: Dibrom's & Mitiok's (Read 8896 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Megaman
  • [*][*][*]
LAME compiles test: Dibrom's & Mitiok's
I've made a short test with two LAME binaries:  Dibrom's 3.90.2 ICL latest compile and Mitiok's 3.92 latest compile.

I've uploaded a report here: LAME test.

Read at your own risk (!) , I'm not much more than a newbie so my conclusions might be completely wrong , however I tried to do my best.

  • Dibrom
  • [*][*][*][*][*]
  • Administrator
LAME compiles test: Dibrom's & Mitiok's
Reply #1
The only thing I'd be a little wary about is the conclusion that they don't sound different.  I've mentioned this before, but one of the changes made  after 3.90.2 was that /qrcd was no longer used in the Makefile.  Granted, it is "bad form" (or whatever you want to refer to it as) because it changes the rounding behavior of the code slightly, but even so, I have noticed on at least one occassion that there can be slight degredations in quality without it (I noticed this on fatboy).  I don't believe the LAME developers did any significant testing after removing this command from the makefile to make sure it did not impact quality.  This is one of the reasons I don't recommend using 3.92 right now.. since there is more room for quality degredation because of this slight change in the compile time options.  It's probably true that this won't affect most situations significantly, and that most people won't hear the difference anyway, but I still wouldn't make the conclusion that there is no detectable difference at all.
  • Last Edit: 23 November, 2002, 04:59:14 AM by Dibrom

  • Megaman
  • [*][*][*]
LAME compiles test: Dibrom's & Mitiok's
Reply #2
Quote
The only thing I'd be a little wary about is the conclusion that they don't sound different.......It's probably true that this won't affect most situations significantly, and that most people won't hear the difference anyway, but I still wouldn't make the conclusion that there is no detectable difference at all.

Right....maybe the difference would be noticed only on very critical samples but of course if you want the best possible quality under any possible circumstances no matter if highly critical or not , it would be reasonable to choose your compile , assuming that you heard that slight quality degradation with Mitiok's compile.
I will check out the fatboy sample myself although I doubt that I have the adequate gear to do any quality ABX testing at all , so I guess in the end it will be a matter of trust .
  • Last Edit: 23 November, 2002, 05:21:28 AM by Megaman

  • robert
  • [*][*][*][*][*]
  • Developer
LAME compiles test: Dibrom's & Mitiok's
Reply #3
I cannot recommend the use of 3.90.2 if that binary was compiled with the /qrcd switch turned on. It simply uses the wrong rounding mode.

  • Dibrom
  • [*][*][*][*][*]
  • Administrator
LAME compiles test: Dibrom's & Mitiok's
Reply #4
Quote
I cannot recommend the use of 3.90.2 if that binary was compiled with the /qrcd switch turned on. It simply uses the wrong rounding mode.

This is what I was talking about.

Ironically, this statement is probably not made with any basis in quality testing.

It's just "wrong", so it doesn't matter if it sounds better or not.  From a development point of view, this might be interesting, but from an end user point of view (something which LAME does not address very well.. tons of unnecessary experimental switches are an example of this), it is not.  Most of the people on this site who are looking for recommended compiles and who looking at the --alt-presets are doing so because they want the best quality.  They want something which has been verified via listening tests.  I doubt they would really care whether or not it uses the "intended" rounding modes if the actual end result is better.

EDIT: I do understand the need for code to behave a certain way, but with the way LAME is right now (basically no quality testing or control),  I don't think we can take anything for granted without testing its effects on quality -- this includes rounding behavior (in that, the behavior of the ICL compiles was changed after 3.90.2, but I don't think any quality testing was done by the people making this change).  The 3.93 release should be a lesson in this...

So basically what I'm trying to say is that regardless of how 3.90.2 works at a code level, regardless of rounding modes or anything which should work a certain way, I'm only making a judgement on how it actually sounds in practice -- I'm not going to make any assumptions.  I think this is, at least right now, the only true way to determine the actual quality of the output.  3.90.2 has been verified to be good.. 3.92 hasn't, and 3.93 has been verified to not be good.
  • Last Edit: 23 November, 2002, 06:52:58 AM by Dibrom

  • ErikS
  • [*][*][*][*][*]
LAME compiles test: Dibrom's & Mitiok's
Reply #5
Quote
The only thing I'd be a little wary about is the conclusion that they don't sound different.  I've mentioned this before, but one of the changes made  after 3.90.2 was that /qrcd was no longer used in the Makefile.  Granted, it is "bad form" (or whatever you want to refer to it as) because it changes the rounding behavior of the code slightly, but even so, I have noticed on at least one occassion that there can be slight degredations in quality without it (I noticed this on fatboy).  I don't believe the LAME developers did any significant testing after removing this command from the makefile to make sure it did not impact quality.  This is one of the reasons I don't recommend using 3.92 right now.. since there is more room for quality degredation because of this slight change in the compile time options.  It's probably true that this won't affect most situations significantly, and that most people won't hear the difference anyway, but I still wouldn't make the conclusion that there is no detectable difference at all.

I assume you made an ABX test... I can't hear this myself, but are there other persons who can pass an ABX on this sample? Can you repeat the test yourself and pass it again? What I mean: both Takehiro and Robert have now discouraged the use of this switch and it seems you are the only one who claims the opposite with one(?) abx test made some time ago. It would be good if your results on the ABX test was repeatable.
  • Last Edit: 23 November, 2002, 07:21:28 AM by ErikS

  • JohnV
  • [*][*][*][*][*]
  • Developer
LAME compiles test: Dibrom's & Mitiok's
Reply #6
Quote
What I mean: both Takehiro and Robert have now discouraged the use of this switch and it seems you are the only one who claims the opposite with one(?) abx test made some time ago. It would be good if your results on the ABX test was repeatable.

Hmm, you seem to forget that the whole development of --aps was made using Dib's compiles. It was all based on listening tests. Granted, I think 3.92 is pretty much the same "quality level" though.

It's quite ironic that the Lame devs are so concerned about some rounding issue, and even recommend something based on it, but then the real problems seem to go unnoticed..
http://www.geocrawler.com/lists/3/SourceFo...323/0/10244994/
  • Last Edit: 23 November, 2002, 08:40:07 AM by JohnV
Juha Laaksonheimo

  • Megaman
  • [*][*][*]
LAME compiles test: Dibrom's & Mitiok's
Reply #7
I agree with Dibrom's statement: what the end user wants is the highest possible perceptual quality no matter the rounding mode.Of course , if the wrong rounding mode results in a binary which delivers files of perceptually superior quality (even if only on critical samples) is something that needs to be backed up with several , seriously made listening tests.

Edit: I'm absolutely paranoid about my english
  • Last Edit: 23 November, 2002, 11:48:25 PM by Megaman

  • ErikS
  • [*][*][*][*][*]
LAME compiles test: Dibrom's & Mitiok's
Reply #8
Quote
Hmm, you seem to forget that the whole development of --aps was made using Dib's compiles. It was all based on listening tests. Granted, I think 3.92 is pretty much the same "quality level" though.

So?

1. Listening tests don't mean anything if they are not reproducable....  Can you hear any improvement with this compiler switch?

2. If the switch was used throughout the development of the alt-preset:s then it is very possible that something else was tuned to compensate for that "wrong" rounding mode, and when you switch back to the "correct" rounding mode it overcompensates and throws something else off. If this is the case the right approach is to fix lame so it can use the correct rounding mode. Many people don't use icl compiles - why should they suffer?

  • Megaman
  • [*][*][*]
LAME compiles test: Dibrom's & Mitiok's
Reply #9
I think that 99% out there use ICL compiles.
Mitiok's official compile is compiled with ICL (am I right?).I guess at least 98% of non-HA members use Mitiok's compile.

  • ErikS
  • [*][*][*][*][*]
LAME compiles test: Dibrom's & Mitiok's
Reply #10
So 99% of the people use windows? I doubt it.

  • Dibrom
  • [*][*][*][*][*]
  • Administrator
LAME compiles test: Dibrom's & Mitiok's
Reply #11
Quote
Quote
Hmm, you seem to forget that the whole development of --aps was made using Dib's compiles. It was all based on listening tests. Granted, I think 3.92 is pretty much the same "quality level" though.

So?

1. Listening tests don't mean anything if they are not reproducable....  Can you hear any improvement with this compiler switch?

2. If the switch was used throughout the development of the alt-preset:s then it is very possible that something else was tuned to compensate for that "wrong" rounding mode, and when you switch back to the "correct" rounding mode it overcompensates and throws something else off. If this is the case the right approach is to fix lame so it can use the correct rounding mode. Many people don't use icl compiles - why should they suffer?

I'm not really going to spend much effort on this for the following reasons.

1.  I don't have time.
2.  I'm tired of arguing pointlessly about LAME.
3.  I simply don't care enough anymore.

To address your first point, I agree with you that listening tests are pointless if they are not reproducable.  I do not plan on reproducing this particular listening test anytime soon though.  You can ask the other LAME developers if they would like to do so though.

As to whether or not my statement means anything aside from that in regards to this matter, well, I've done more reproducable listening tests for LAME than any of the other developers have ever done.  Most of them were blind, and nearly all of my results have been verified by other parties.  The fruits of my labor in regard to past listening tests can be heard through the --alt-presets.

Additionally, Robert and Takehiro are not authorities on quality.  It was commits from both of them to CVS which have resulted in 3.93's broken quality.  Given that, it's understandable for me to question their statements about what is quality and what isn't.  Furthermore, I've never seen an effort from either of them to do comprehensive blind testing with results verified by any sort of community.  This is not a reflection on their character, so please don't take this out of context.  I get along with these guys fine on a personal level, I'm only attempting to show why I don't believe their comments on this matter hold a lot of weight.

As for your second point... if they feel that there is a problem with the rounding mode in /qrcd, and that the --alt-presets have been tuned with this in mind (which is probably true, except for the possibility of a "problem" with the rounding mode.. this is unclear.. especially if the rounding mode led to higher quality), then they are more than welcome to correct the problem themselves.  Unfortunately, I don't think this will be easy without some extensive listening tests, which I don't think either of them has the time or desire to do -- it's easier to just claim that it's "wrong" and be done with it.

Nobody is made to "suffer" with the --alt-presets.  I am not forcing anyone to use anything, I am only making a recommendation.  I tuned the --alt-presets according to the code that was already in LAME (the /qrcd switch was put there by the LAME developers, not I.  Only after the --alt-presets were implemented was it removed), so it is not my responsibility to make sure that they still work after someone else changes their functionality.  It should be the responsibility of those who have made the change to put forth the effort to verify that quality has not taken a hit.  If that means asking me to help, that's fine.. but I was not asked to help.  Some may say that it is my responsibility, because I'm a LAME developer.  I would disagree with this.  LAME is, accordingly, driven by consensus.  My input into this consensus has always been ignored.  Because of this, and because of that fact that I don't think the other developers take me seriously (and thus, don't really consider me a peer), I don't consider myself to be a LAME developer.  To be honest, I'm not even sure why the --alt-presets ever were included in the first place, other than it was just another thing that could be added to LAME.

Ultimately I agree with your assessment that if /qrcd improves quality, then changes in LAME itself should be made.  Who is going to do this though?  I don't think the people complaining about /qrcd are going to do it.  Complaining to me about it isn't going to accomplish anything, because I have no control over LAME.  Furthermore, there is very little initiative or motivation in the LAME project to improve quality at such fine levels.  In addition, even if we could prove that there was a beneficial change that could be incorporated into LAME, you'd likely meet with much resistance to implementing your idea (basically, it'd never happen).

I recently had a discussion with Takehiro about the future of LAME and about large scale changes that should be made to the code.  He basically told me that LAME is too mature and too much of the code is interdepedant for there to be any more large scale modifications at this point.  He mentioned that he didn't believe LAME would ever improve or change significantly at this point.  Although I don't agree with the ideas that change can't happen, I do agree with his assessment of this particular situation, given all that has happened (and that which hasn't) over the past year.

  • Dibrom
  • [*][*][*][*][*]
  • Administrator
LAME compiles test: Dibrom's & Mitiok's
Reply #12
Quote
So 99% of the people use windows? I doubt it.

I guess you haven't done much research.  ICL is available for free on Linux, so that thows the "only windows" bit out.  I don't know if it compiles LAME, but the possibility it certainly there.

And, FWIW, gcc is not a very good compiler speed wise.  I get much better results running the 3.90.2 compile under WINE then I do compiling it with gcc.
  • Last Edit: 24 November, 2002, 05:14:15 AM by Dibrom

  • dev0
  • [*][*][*][*][*]
  • Developer
LAME compiles test: Dibrom's & Mitiok's
Reply #13
Did you try GCC 3.2? From what I've heard it's almost as fast as ICL.
"To understand me, you'll have to swallow a world." Or maybe your words.

LAME compiles test: Dibrom's & Mitiok's
Reply #14
are there win32 builds ?

  • Dibrom
  • [*][*][*][*][*]
  • Administrator
LAME compiles test: Dibrom's & Mitiok's
Reply #15
Quote
Did you try GCC 3.2? From what I've heard it's almost as fast as ICL.

No, I've tried GCC 3 or 3.1, forget which.

I'd be very surprised if GCC 3.2 was as fast as ICL though.

  • john33
  • [*][*][*][*][*]
  • Developer
LAME compiles test: Dibrom's & Mitiok's
Reply #16
Quote
Quote
Did you try GCC 3.2? From what I've heard it's almost as fast as ICL.

No, I've tried GCC 3 or 3.1, forget which.

I'd be very surprised if GCC 3.2 was as fast as ICL though.

It seems to depend rather on the application. The GCC3.2 compiles I made of the OggVorbis apps ran as fast, if not marginally faster in some cases, than the ICL compiles. However, I did produce 3.2 compiles of LAME for my own testing and they were significantly slower than the ICL compiles. So I guess the only conclusion to be drawn is that it depends on how the code is written.
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • ErikS
  • [*][*][*][*][*]
LAME compiles test: Dibrom's & Mitiok's
Reply #17
Quote
I guess you haven't done much research.  ICL is available for free on Linux, so that thows the "only windows" bit out.  I don't know if it compiles LAME, but the possibility it certainly there.

My bad. I assumed you needed MSDev to use ICL, plus I never heard that neither MSDev nor ICL exitsted for any other OS. But that is because I didn't look.

  • Gabriel
  • [*][*][*][*][*]
  • Developer
LAME compiles test: Dibrom's & Mitiok's
Reply #18
There is ICL for linux, but I'm not sure that it's free. Under win32, at least, it's not free at all...

  • Dibrom
  • [*][*][*][*][*]
  • Administrator
LAME compiles test: Dibrom's & Mitiok's
Reply #19
Quote
There is ICL for linux, but I'm not sure that it's free. Under win32, at least, it's not free at all...

It's free and "unsupported", but I think you do have to register at their developer website to download it.  Gentoo, which is a source based distro, has been working on ICL support (to compile the entire distro) for quite some time.. and I'm pretty sure they wouldn't be doing this if it wasn't available freely.

And I'm aware that the win32 version is non-free.

  • Gabriel
  • [*][*][*][*][*]
  • Developer
LAME compiles test: Dibrom's & Mitiok's
Reply #20
Free under linux...That's interesting...

  • john33
  • [*][*][*][*][*]
  • Developer
LAME compiles test: Dibrom's & Mitiok's
Reply #21
Quote
Free under linux...That's interesting...

And they only want your name, email address and country to download!
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • joro_09
  • [*]
LAME compiles test: Dibrom's & Mitiok's
Reply #22
Hi,
I've been reading different forums and recommendations about quality MP3 encoding for about 2 months up to now, but I still hesitate what switches to use. To be clear, my intention is to encode all my music in MP3 archive with best possible quality, but to be able (if I decide for some reason to do so) to decompress it again and write it to Audio Discs, without furthermore loss of quality.
So here are some questions for you:
1. Some people say --alt-preset insane is the best quality you can get from MP3 format (at least for LAME 3.92). Other people say extreme is better because of its intelligence during coding simple and harder parts of a song (e.g. extreme is better than insane in passing on the stereo panorama). What is the truth?!
2. Is there any official place where developers classify, in order of their quality, the LATEST, TESTED and STABLE switches (presets) for command  line? It is very confusing to me to see different developers recommending their own switches and to choose between them… Which is best!?
3. These days I understood LAME 3.93 have been released. Is it safe to use it, or should I wait for further tuning? Or is the lame3.90.2 better?
Thanks in advance for helping me.

Sincerely, George

  • KikeG
  • [*][*][*][*][*]
  • Developer
LAME compiles test: Dibrom's & Mitiok's
Reply #23
- According to my experience and others, --alt-preset insane is the best quality you can achieve, better than with alt-preset extreme.

- 3.93 is not advised, better use 3.90.2

  • Chun-Yu
  • [*][*][*][*]
  • Developer
LAME compiles test: Dibrom's & Mitiok's
Reply #24
BTW: The Intel C++ Compiler 7.0 has recently been released (but I haven't had time to try it yet).  The best thing about it is that they've added a "student license" that's only $30 - "Student pricing is for non-commercial licenses only. The licenses are individual, unsupported user licenses without a timeout on the license."