Skip to main content

Topic: LAME: different compilers producing significantly different bitstreams (Read 29017 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • john33
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #50
Just for the record, Robert is looking at this issue and will be back with something for testing in the not too distant future.  In the meantime, the higher bitrate is not hurting quality but it may be spending unnecessary bits.
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • lvqcl
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #51
Just for the record, Robert is looking at this issue and will be back with something for testing in the not too distant future.  In the meantime, the higher bitrate is not hurting quality but it may be spending unnecessary bits.


It's somewhere in calc_xmin() (in quantize_pvt.c). Move this function to a new file and compile it with MSVC compiler, and the rest with ICC.
  • Last Edit: 02 February, 2012, 03:53:29 PM by lvqcl

  • john33
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #52
...
It's somewhere in calc_xmin() (in quantize_pvt.c)....

So I believe, but I'll leave that to Robert.
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • Destroid
  • [*][*][*][*][*]
LAME: different compilers producing significantly different bitstreams
Reply #53
From the sidelines, I found this discussion of the FP rounding and/or compiler-related differences fascinating. Although this was the discussion started regarding LAME 3.99 release I was glad to have read-through it (just like a recent topic about layman's FFT explanation) and wanted to express my appreciation to those involved on the member and developers for sleuthing and posting their information. This is why HA has always rocked for me =)
"Something bothering you, Mister Spock?"

  • antman
  • [*][*][*]
LAME: different compilers producing significantly different bitstreams
Reply #54
Agreed Destroid.  This has been an interesting read.  I jumped back to find the start of the latest development.

And with that being said, maybe it's time to split the thread?  Little of this thread has been dedicated to the actual quality improvements/unimprovements of Lame 3.99.  It's been more about compiler optimizing and compiler output.

  • JJZolx
  • [*][*][*][*]
LAME: different compilers producing significantly different bitstreams
Reply #55
I'm surprised this thread wasn't split a long time ago. I can't imagine that 99% of users who might see this release announcement and  consider updating to 3.99 have either the time or the energy to read through well over 300 posts. I would expect some discussion of new 3.99.x releases and updated Rarewares compiles as they come out, but the minutiae in this thread really belongs in a tech forum.

  • DARcode
  • [*][*][*][*][*]
  • Members (Donating)
LAME: different compilers producing significantly different bitstreams
Reply #56
The "minutiae" in this thread are actually useful information: the difference in bitrate produced by different compiles might have an impact on quality, still to be determined exactly, and the LAME header issues do have an impact on usability, I'm glad both discussions have been consolidated here, just learn to skip the parts not relevant to you, this isn't a forum for Joe Average anyway.

P.S.
Minutiae is plural, if you wanna show off your Latin at least do it properly.

EDIT: The P.S..
  • Last Edit: 07 February, 2012, 08:18:10 AM by DARcode
WavPack 4.80.0 -b384hx6cmv / qaac 2.59 -V 100

  • john33
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #57
Robert is currently working on a solution that would make LAME more, or less, "compiler proof", or at least the variations between compilers would become inconsequential. Obviously some testing will be required when he has a solution that is 'fit for purpose' and HA will be the first to know.
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • apodtele
  • [*]
LAME: different compilers producing significantly different bitstreams
Reply #58
Robert is currently working on a solution that would make LAME more, or less, "compiler proof", or at least the variations between compilers would become inconsequential. Obviously some testing will be required when he has a solution that is 'fit for purpose' and HA will be the first to know.


This implies that the exact piece of code in question has been identified. Please confirm that it is in quantize_pvt.c and maybe point to a more precise location.

  • john33
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #59
There is a test compile (32 bit) now available at: www.rarewares.org/files/mp3/lame3.99.5-Test.zip

While there is a small bit distribution difference between VC and ICL compiles, the final bitrate is the same on the files tested to date.
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • john33
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #60
This implies that the exact piece of code in question has been identified. Please confirm that it is in quantize_pvt.c and maybe point to a more precise location.

EDIT: See Robert's post, below.
  • Last Edit: 07 February, 2012, 11:41:04 AM by john33
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • robert
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #61
This implies that the exact piece of code in question has been identified. Please confirm that it is in quantize_pvt.c and maybe point to a more precise location.

The code was quickly identified last week in vbrquantize.c, which was affected by floating point model fast math.

http://lame.cvs.sourceforge.net/viewvc/lam...amp;sortby=date (line 436)

  • apodtele
  • [*]
LAME: different compilers producing significantly different bitstreams
Reply #62
The code was quickly identified last week in vbrquantize.c, which was affected by floating point model fast math.

http://lame.cvs.sourceforge.net/viewvc/lam...amp;sortby=date (line 436)


Just a couple of jazz albums to re-encode... It does not look like 3.98.4 had this problem, right?

  • JJZolx
  • [*][*][*][*]
LAME: different compilers producing significantly different bitstreams
Reply #63
Here's that set of files again, encoded using the RareWares 32-bit LAME 3.95.5-test compile. The differences are very small now. lvqcl's compile still produces slightly smaller files, even at the lowest quality levels, but the differences in file size are generally only a fraction of a percent.

Code: [Select]
Aerosmith - Sweet Emotion.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        269  228  190  168  149  129  113  100    83
3.99.5-test (RareWares)  264  223  189  168  149  129  112    99    83
3.99.4 (lvqcl)            264  223  188  168  149  129  112    99    83

Allman Brothers - Melissa.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        263  231  202  184  164  142  124  112    90
3.99.5-test (RareWares)  260  227  199  184  164  142  123  111    90
3.99.4 (lvqcl)            259  227  198  184  164  142  123  111    90

Bill Evans - Polka Dots And Moonbeams.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        243  207  178  164  144  124  107    97    80
3.99.5-test (RareWares)  243  205  177  164  144  124  105    96    80
3.99.4 (lvqcl)            243  205  176  164  144  124  105    96    80

Black Keys - Set You Free.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        259  222  186  165  146  128  115    98    79
3.99.5-test (RareWares)  258  219  185  165  146  128  115    98    79
3.99.4 (lvqcl)            258  219  185  165  146  128  115    98    79

Bob Marley - Redemption Song.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        241  211  175  151  129  107    93    82    72
3.99.5-test (RareWares)  228  197  167  151  128  103    86    78    72
3.99.4 (lvqcl)            227  197  167  151  128  103    86    78    72

Count Basie - Lester Leaps In.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        280  244  199  180  158  135  120  107    93
3.99.5-test (RareWares)  258  229  197  179  156  130  112  102    93
3.99.4 (lvqcl)            257  228  196  179  156  130  112  102    93

Cowboy Junkies - Mining For Gold.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        249  208  176  159  139  120  104    93    77
3.99.5-test (RareWares)  249  207  175  159  139  119  102    93    77
3.99.4 (lvqcl)            248  206  174  159  139  119  102    93    77

Del McCoury - Rambling Boy.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        275  238  204  177  155  131  113  104    90
3.99.5-test (RareWares)  266  226  193  177  155  130  111  102    89
3.99.4 (lvqcl)            266  225  192  177  155  130  110  102    90

Dizzy Gillespie - Manteca.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        266  229  198  181  162  139  121  109    94
3.99.5-test (RareWares)  263  224  196  181  162  139  119  108    94
3.99.4 (lvqcl)            262  223  195  181  162  139  119  108    94

Ella Fitzgerald - Anything Goes.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        187  162  142  125  112    97    86    75    63
3.99.5-test (RareWares)  187  159  137  123  109    92    79    71    60
3.99.4 (lvqcl)            187  159  137  123  109    92    79    71    60

George Baker Selection - Little Green Bag.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        268  236  206  190  171  150  133  120    98
3.99.5-test (RareWares)  268  235  205  190  171  150  133  120    98
3.99.4 (lvqcl)            267  235  205  190  171  150  133  120    98

George Jones - She Thinks I Still Care.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        258  209  179  164  146  127  111  102    87
3.99.5-test (RareWares)  259  207  177  164  146  126  109  102    87
3.99.4 (lvqcl)            258  206  177  164  146  126  109  102    87

Jack Johnson - Better Together.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        264  229  195  172  152  130  114  103    87
3.99.5-test (RareWares)  263  227  193  172  152  130  113  103    87
3.99.4 (lvqcl)            262  226  192  172  152  130  113  103    87

Jimmy Cliff - Ska All Over The World.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        258  228  195  179  160  134  115  106    93
3.99.5-test (RareWares)  249  218  191  179  160  133  113  105    93
3.99.4 (lvqcl)            248  217  190  179  160  133  113  105    93

John Coltrane - Giant Steps.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        291  241  205  187  167  145  127  121  102
3.99.5-test (RareWares)  292  238  201  187  167  145  126  121  102
3.99.4 (lvqcl)            291  236  199  187  167  145  126  120  102

Johnny Winter - Mojo Boogie.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        275  238  200  169  149  128  112    97    79
3.99.5-test (RareWares)  260  221  187  169  149  128  110    96    79
3.99.4 (lvqcl)            259  221  187  169  149  128  110    96    79

Kanye West - Gold Digger.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        249  214  182  160  144  127  112    97    77
3.99.5-test (RareWares)  249  213  178  159  143  125  109    96    77
3.99.4 (lvqcl)            247  211  176  159  143  125  108    96    77

Katia Lebeque & the Cleveland Orchestra - Rhapsody in Blue.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        267  234  196  161  140  116  102    91    86
3.99.5-test (RareWares)  247  210  178  159  135  108    91    83    82
3.99.4 (lvqcl)            247  210  178  159  135  108    91    83    82

Leo Kottke - Jesus Maria.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        240  210  185  165  145  121  103    96    87
3.99.5-test (RareWares)  236  204  178  165  144  118    98    94    86
3.99.4 (lvqcl)            236  204  177  165  144  118    98    94    86

Liszt.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        283  245  183  156  136  119  111  103  101
3.99.5-test (RareWares)  257  208  172  149  125    97    82    76    83
3.99.4 (lvqcl)            257  207  172  149  125    97    82    76    83

Lotus - Sea Of Tranquility.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        231  196  166  145  128  110    96    86    73
3.99.5-test (RareWares)  229  192  160  142  124  102    88    79    69
3.99.4 (lvqcl)            228  191  160  142  124  103    88    79    69

Marcos Valle - Para Os Filhos De Abraao.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        261  222  187  167  148  128  114  101    82
3.99.5-test (RareWares)  261  221  186  167  148  128  113  101    81
3.99.4 (lvqcl)            260  220  186  167  148  128  113  101    81

Nirvana - Smells Like Teen Spirit.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        260  227  193  169  148  128  114    97    79
3.99.5-test (RareWares)  259  224  191  169  148  127  113    96    79
3.99.4 (lvqcl)            258  223  190  169  148  127  113    96    79

Ottmar Liebert - August Moon.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        300  255  222  202  182  158  136  130  107
3.99.5-test (RareWares)  299  248  218  202  181  157  134  129  107
3.99.4 (lvqcl)            298  246  216  202  181  157  134  129  107

Peter Tosh - Get Up, Stand Up.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        281  251  220  200  178  152  134  124  107
3.99.5-test (RareWares)  277  247  217  201  178  151  131  124  107
3.99.4 (lvqcl)            276  246  216  200  177  151  131  123  106

Pointer Sisters - Fire.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        264  230  195  169  149  128  112  102    89
3.99.5-test (RareWares)  261  226  192  169  148  126  110  101    89
3.99.4 (lvqcl)            261  225  191  169  148  126  110  101    89

Psychedelic Furs - Love My Way.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        267  225  183  164  145  126  111    98    81
3.99.5-test (RareWares)  262  220  182  164  145  125  110    98    81
3.99.4 (lvqcl)            261  219  181  164  145  125  110    97    81

Rolling Stones - Street Fighting Man.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        270  235  206  192  172  148  123  112    90
3.99.5-test (RareWares)  259  232  205  192  171  148  119  112    90
3.99.4 (lvqcl)            259  231  204  192  171  147  119  112    90

Shirley Bassey - Moonraker.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        264  229  198  178  157  136  119  103    81
3.99.5-test (RareWares)  261  225  194  177  157  135  118  103    80
3.99.4 (lvqcl)            261  225  194  177  157  135  118  103    80

Smiths - Please, Please, Please, Let Me Get What I Want.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        271  246  217  195  173  150  133  105    78
3.99.5-test (RareWares)  271  246  217  195  173  150  132  105    78
3.99.4 (lvqcl)            271  246  217  195  173  150  132  105    78

Steely Dan - Black Friday.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        266  231  196  174  152  129  115  103    85
3.99.5-test (RareWares)  266  231  196  174  152  128  113  102    85
3.99.4 (lvqcl)            266  231  195  174  152  128  113  102    85

Stevie Ray Vaughan - The House Is Rockin'.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        262  231  201  170  150  130  115  101    83
3.99.5-test (RareWares)  261  230  200  170  150  130  115  100    83
3.99.4 (lvqcl)            261  229  199  170  150  130  115  100    83

Sublime - Badfish.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        280  242  204  182  161  139  123  113    94
3.99.5-test (RareWares)  276  237  202  182  161  139  122  113    94
3.99.4 (lvqcl)            275  237  201  182  161  139  122  113    94

Thelonious Monk - Ruby, My Dear.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        229  201  174  153  135  110    94    86    85
3.99.5-test (RareWares)  226  196  166  150  132  100    80    75    80
3.99.4 (lvqcl)            226  196  166  150  132  101    80    75    80

Tomita - Introduction And Dance Of The Firebird.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        250  225  197  165  141  113    97    86    81
3.99.5-test (RareWares)  237  209  182  164  139  107    88    80    78
3.99.4 (lvqcl)            236  209  182  164  139  107    88    79    78

U2 - Bullet The Blue Sky.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        256  224  192  168  148  127  113    99    79
3.99.5-test (RareWares)  255  222  190  168  148  127  113    99    79
3.99.4 (lvqcl)            255  222  190  168  148  127  113    99    79

Van Morrison - Moondance.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        259  225  193  174  155  133  116  105    87
3.99.5-test (RareWares)  255  218  189  174  155  133  114  105    87
3.99.4 (lvqcl)            254  218  188  174  155  133  114  105    87

Will Ackerman - Abide The Winter.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        255  230  200  168  143  113    95    87    80
3.99.5-test (RareWares)  244  216  186  167  142  108    88    82    78
3.99.4 (lvqcl)            244  215  186  167  141  108    88    82    78

Yngwie Malmsteen - Leviathan.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        271  228  189  167  146  127  112    96    81
3.99.5-test (RareWares)  268  222  185  166  146  126  111    96    81
3.99.4 (lvqcl)            268  222  185  166  146  126  111    96    81

Yo-Yo Ma - Boccherini - Cello Concerto No. 7 in G major, G480 I. Allegro.wav
                         -V0  -V1  -V2  -V3  -V4  -V5  -V6  -V7  -V8
3.99.4 (RareWares)        250  223  189  165  143  118  101    91    83
3.99.5-test (RareWares)  239  209  180  164  141  114    95    88    82
3.99.4 (lvqcl)            239  209  180  164  141  114    95    87    82

  • halb27
  • [*][*][*][*][*]
LAME: different compilers producing significantly different bitstreams
Reply #64
I checked Alex B's Liszt sample, and 3.99.5 behaves nearly exactly as lvql's MSV10 compile bitratewise (208 kbps for-V1, and 97 kbps for -V5).
I also checked the bitrate distribution for -V5, and the Encspot result looks pretty much the same Alex B gave us.

Thank you, robert.

Oops, sorry, I din't see JJZolx had the sample in his test.
  • Last Edit: 09 February, 2012, 05:27:44 AM by halb27
lame3995n -Q0.5

  • Kilu
  • [*]
LAME: different compilers producing significantly different bitstreams
Reply #65
This thread was an interesting read.

So just to be clear, this is now resolved? Should I grab the 3.99.5-test from RareWares, or wait for 3.99.5 proper?

Another thing I'm wondering, I saw some encoding speed differences earlier in the original thread this was split from.
Was there encoding speed differences with 3.99.5-test and 3.99.4 (lvqcl), how about 32bit vs 64bit builds?
Or was that stuff just completely unrelated?
made of radiation

  • john33
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #66
This thread was an interesting read.

So just to be clear, this is now resolved? Should I grab the 3.99.5-test from RareWares, or wait for 3.99.5 proper?

Another thing I'm wondering, I saw some encoding speed differences earlier in the original thread this was split from.
Was there encoding speed differences with 3.99.5-test and 3.99.4 (lvqcl), how about 32bit vs 64bit builds?
Or was that stuff just completely unrelated?

I would be inclined to wait for the full release, if I were you. There could be further changes until then, but if you feel inclined to test, please go ahead. 

The speed stuff was unrelated, IRRC.
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • antman
  • [*][*][*]
LAME: different compilers producing significantly different bitstreams
Reply #67
Kilu, in summary, the encoding speed differences were between 3.98 and 3.99.  3.99 encoded slower than 3.98.  Through compiler optimizing 3.99's speed was brought up to par with 3.98.

Sure there is going to be encoding speed differences between 3.99.5-test (ICL), 3.99.4 (MSVS2010), 32 and 64-bit builds, but that's not what's on trial right now.  What's on trial right now is the output between all.

  • Kilu
  • [*]
LAME: different compilers producing significantly different bitstreams
Reply #68
Alrighty, thanks guys.
I think I'll wait for the full release since I'm not in any kind of hurry, might as well wait a bit.
made of radiation

  • john33
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #69
3.99.5 test 2 on the link below fixes the decoding regression:

http://www.rarewares.org/files/mp3/lame3.99.5-Test2.zip

John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/

  • psycho
  • [*][*][*]
LAME: different compilers producing significantly different bitstreams
Reply #70
lame -V 0

  • lvqcl
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #71
There's still no mention of 3.99.5 at http://sourceforge.net/projects/lame/files/lame/3.99/ and in the changelog

  • john33
  • [*][*][*][*][*]
  • Developer
LAME: different compilers producing significantly different bitstreams
Reply #72
3.99.5 is out, compiles anyone?

That log is incomplete as the release has not yet been made. It's work in progress.
John
----------------------------------------------------------------
My compiles and utilities are at http://www.rarewares.org/