Skip to main content

Topic: lame 3.98 alpha 2 testing thread (Read 14158 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
lame 3.98 alpha 2 testing thread
I must be brief.
Here are the results of a blind listening tests performed on 150 samples, classical music only. I planned to use other samples, but my week-end is not unlimited.

• 3.98 alpha 2 -V5
• 3.98 alpha 2 -V5 --vbr-new
• 3.98 alpha 2 -V5 --vbr-new --athaa-sensitivity 1


Results
:









ABX logs:
http://audiotests.free.fr/tests/2005.10/LAME_398_V5.7z

Bitrate table:

http://audiotests.free.fr/tests/2005.10...Q%20Analysis.7z



I can comment it (I'm very sorry about, but there some troubles about some weird usage of internet here at my working place, and internet is currently "under control").


• --vbr-new is apparently same to use with -V5 profile
• --athaa-sensitivity really help to lower the level of annoyance (ringing mostly).
It's maybe not the right answer to current quality problems occuring with "pure" V5 preset, but the fix is very helpful and doesn't cost too much (~6 kbps on average).
  • Last Edit: 29 December, 2005, 04:36:47 PM by guruboolez

  • Synthetic Soul
  • [*][*][*][*][*]
  • Global Moderator
lame 3.98 alpha 2 testing thread
Reply #1
Wow, your result formatting just gets better and better.  Your tests have always been exemplary of course.

I thought --athaa-sensitivity had gone.  I know you were querying the decision.

Does this mean it's back in for 3.98 or was never out?

As a -V5 user, thank you very much for your time.

Edit: OK, discussions mentioned above are here.  A spinoff thread is here.  It is out in 3.97b1 but remains in alpha versions.

My re-reading of these previous threads leads me to believe that this test is to keep this discussion alive, and to attempt to find a resolve for the "requirement" for --athaa-sensitivity with -V5.

Good luck to guruboolez and Gabriel.
  • Last Edit: 17 October, 2005, 10:17:58 AM by Synthetic Soul
I'm on a horse.

  • shadowking
  • [*][*][*][*][*]
lame 3.98 alpha 2 testing thread
Reply #2
An Xcellent read as always. What exactly are the changes from 3.97b1 > 3.98b2 ?
wavpack -b4x4s1c

  • Gabriel
  • [*][*][*][*][*]
  • Developer
lame 3.98 alpha 2 testing thread
Reply #3
Quote
I thought --athaa-sensitivity had gone. I know you were querying the decision.

Does this mean it's back in for 3.98 or was never out?

As other experimental switches, it is available in alpha and debug compiles.

  • Synthetic Soul
  • [*][*][*][*][*]
  • Global Moderator
lame 3.98 alpha 2 testing thread
Reply #4
Thanks Gabriel.  I found that out myself as well.  I decided I should do as I wish others would do sometimes, and actaully test instead of just asking for the answer.

Is 0.59 a marked improvement when talking about ABX?  I remember guruboolez once stating that a 4.0 is "perceptible but not annoying".  What is 3.0 please?  "perceptible and annoying"?  NB: I have tried to find the answer to this one.

I seem to remember seeing before that, if the two lines do not overlap, it can be said that there is a definite difference.  They don't overlap in any of the subtotals.  I guess this is why guruboolez makes the brief comments he has.
I'm on a horse.

  • stephanV
  • [*][*][*][*]
lame 3.98 alpha 2 testing thread
Reply #5
5.0 = imperceptible
4.0 = perceptible but not annoying
3.0 = perceptible and slighly annoying
2.0 = annoying
1.0 = very annoying
"We cannot win against obsession. They care, we don't. They win."

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
lame 3.98 alpha 2 testing thread
Reply #6
Few words about my notation principles.

When a small difference was found= +/- 0.5 point
When a serious difference was heard = +1 point or more
When a very small or unsure difference was heard = < +/- 0.5 point.

Therefore, don't take too much attention on the absolute notation (i.e. it's not totally linked to ITU reference scale).


What matters is:
1/ that --athaa-sensitivity 1 bring more improvement than --vbr-new over defaulted VBR mode
2/ that --athaa-sensitivity could highly reduce severe audible flaws (high level of ringing/noise reduction with -V5/-V5 --vbr-new).

  • Synthetic Soul
  • [*][*][*][*][*]
  • Global Moderator
lame 3.98 alpha 2 testing thread
Reply #7
Thanks stephenV.

Argh.  Just as we get a new HA recommendation and I consider re-encoding all my MP3s (a shocking amount of which are Blade and Xing) up pops this development.

3.97b1 or 3.98a2 --athaa-sensitivity 1?  Or do I wait until this anomaly has been properly diagnosed and ironed out?

I think I'm going to have to wait and see how this one progresses...
I'm on a horse.

  • robert
  • [*][*][*][*][*]
  • Developer
lame 3.98 alpha 2 testing thread
Reply #8
@guruboolez
is it possible to get one or two of your test samples?

  • Gambit
  • [*][*][*][*][*]
  • Developer
lame 3.98 alpha 2 testing thread
Reply #9
Quote
@guruboolez
is it possible to get one or two of your test samples?
[{POST_SNAPBACK}][/a]

Not sure if those are the used ones, but some of guru's samples are here:
[a href="ftp://ftp2.foobar2000.net/foobar/]ftp://ftp2.foobar2000.net/foobar/[/url]
Burrrn - http://www.burrrn.net/
MPEG Audio Collection - http://mac.sourceforge.net/

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
lame 3.98 alpha 2 testing thread
Reply #10
Quote
@guruboolez
is it possible to get one or two of your test samples?
[a href="index.php?act=findpost&pid=335204"][{POST_SNAPBACK}][/a]

The 150 samples are all downloadable:
EDIT: links have changed
  • Last Edit: 29 December, 2005, 04:37:49 PM by guruboolez

  • guruboolez
  • [*][*][*][*][*]
  • Members (Donating)
lame 3.98 alpha 2 testing thread
Reply #11
Some stats:



Code: [Select]
-V5 --vbr-new with and without --athaa-sensitivity 1

when difference is from: -1.0 to 0.4 point    (70 samples)    +2,84 kbps  in average with --athaa..1
when difference is from:  0.5 to 0.9 point    (29 samples)    +4,17 kbps  in average with --athaa..1
when difference is from:  1.0 to 1.4 points    (23 samples)   +7,70 kbps  in average with --athaa..1
when difference is from:  1.5 to 1.9 points    (13 samples)   +8,69 kbps  in average with --athaa..1
when difference is from:  2.0 to 2.4 points    (09 samples)   +14,44 kbps in average with --athaa..1
when difference is from:  2.5 points and more    (05 samples)    +19,00 kbps in average with --athaa..1



CONCLUSION: the perceptual quality improvment of --ahtaa-sensitivity 1 has a direct relation with the amount of bit added by this command line. When --ahtaa-sensitivity 1 has no impact on bitrate it has no impact on quality; when --ahtaa-sensitivity 1 "bloats" the bitrate it also "bloats" the quality.

10% of the tested samples reveal a tendency to "bitrate bloat" (+15 kbps or more). For these 14 samples, the average notation is going from 2.3 points (without --ahtaa-sensitivity 1) to 4.3 points (with --ahtaa-sensitivity 1).

____
Appendix:
RATIO
Code: [Select]
[average notation diff in points] / [average bitrate diff in kbps]

-0.01 / 2.84  =-0.0020
0.53 / 4.17  = 0.1264
1.03 / 4.70  = 0.1333
1.56 / 8.69  = 0.1796
2.00 / 14.44 = 0.1384
2.70 / 19.00 = 0.142
1
  • Last Edit: 18 October, 2005, 04:59:00 AM by guruboolez

lame 3.98 alpha 2 testing thread
Reply #12
Well, it looks like there's very very few instances of regression and that any increase in bitrate is well justified.  Since I haven't done any testing myself, I can only say thank you very much for doing so much testing!
  • Last Edit: 19 October, 2005, 01:30:43 AM by Brigstocke

  • mad_arab
  • [*][*]
lame 3.98 alpha 2 testing thread
Reply #13
3.97 was recommended while still in alpha stage. Are there any significant improvements that would make 3.98 a good choice yet?

  • Cartoon
  • [*]
lame 3.98 alpha 2 testing thread
Reply #14
No, 3.97 was not recommended until it hit beta. Even that caused some noise, as in principle it should be GA code. But as the 3.x tree is pretty mature now, it is pretty safe.

Besides, it is just a recommendation, not the law