Skip to main content
Topic: lame 3.98 alpha 2 testing thread (Read 14908 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

lame 3.98 alpha 2 testing thread

I must be brief.
Here are the results of a blind listening tests performed on 150 samples, classical music only. I planned to use other samples, but my week-end is not unlimited.

• 3.98 alpha 2 -V5
• 3.98 alpha 2 -V5 --vbr-new
• 3.98 alpha 2 -V5 --vbr-new --athaa-sensitivity 1


Results
:









ABX logs:
http://audiotests.free.fr/tests/2005.10/LAME_398_V5.7z

Bitrate table:

http://audiotests.free.fr/tests/2005.10...Q%20Analysis.7z



I can comment it (I'm very sorry about, but there some troubles about some weird usage of internet here at my working place, and internet is currently "under control").


• --vbr-new is apparently same to use with -V5 profile
• --athaa-sensitivity really help to lower the level of annoyance (ringing mostly).
It's maybe not the right answer to current quality problems occuring with "pure" V5 preset, but the fix is very helpful and doesn't cost too much (~6 kbps on average).

lame 3.98 alpha 2 testing thread

Reply #1
Wow, your result formatting just gets better and better.  Your tests have always been exemplary of course.

I thought --athaa-sensitivity had gone.  I know you were querying the decision.

Does this mean it's back in for 3.98 or was never out?

As a -V5 user, thank you very much for your time.

Edit: OK, discussions mentioned above are here.  A spinoff thread is here.  It is out in 3.97b1 but remains in alpha versions.

My re-reading of these previous threads leads me to believe that this test is to keep this discussion alive, and to attempt to find a resolve for the "requirement" for --athaa-sensitivity with -V5.

Good luck to guruboolez and Gabriel.
I'm on a horse.

lame 3.98 alpha 2 testing thread

Reply #2
An Xcellent read as always. What exactly are the changes from 3.97b1 > 3.98b2 ?
wavpack 4.8 -b256hx6c

lame 3.98 alpha 2 testing thread

Reply #3
Quote
I thought --athaa-sensitivity had gone. I know you were querying the decision.

Does this mean it's back in for 3.98 or was never out?

As other experimental switches, it is available in alpha and debug compiles.

lame 3.98 alpha 2 testing thread

Reply #4
Thanks Gabriel.  I found that out myself as well.  I decided I should do as I wish others would do sometimes, and actaully test instead of just asking for the answer.

Is 0.59 a marked improvement when talking about ABX?  I remember guruboolez once stating that a 4.0 is "perceptible but not annoying".  What is 3.0 please?  "perceptible and annoying"?  NB: I have tried to find the answer to this one.

I seem to remember seeing before that, if the two lines do not overlap, it can be said that there is a definite difference.  They don't overlap in any of the subtotals.  I guess this is why guruboolez makes the brief comments he has.
I'm on a horse.

lame 3.98 alpha 2 testing thread

Reply #5
5.0 = imperceptible
4.0 = perceptible but not annoying
3.0 = perceptible and slighly annoying
2.0 = annoying
1.0 = very annoying
"We cannot win against obsession. They care, we don't. They win."

lame 3.98 alpha 2 testing thread

Reply #6
Few words about my notation principles.

When a small difference was found= +/- 0.5 point
When a serious difference was heard = +1 point or more
When a very small or unsure difference was heard = < +/- 0.5 point.

Therefore, don't take too much attention on the absolute notation (i.e. it's not totally linked to ITU reference scale).


What matters is:
1/ that --athaa-sensitivity 1 bring more improvement than --vbr-new over defaulted VBR mode
2/ that --athaa-sensitivity could highly reduce severe audible flaws (high level of ringing/noise reduction with -V5/-V5 --vbr-new).

lame 3.98 alpha 2 testing thread

Reply #7
Thanks stephenV.

Argh.  Just as we get a new HA recommendation and I consider re-encoding all my MP3s (a shocking amount of which are Blade and Xing) up pops this development.

3.97b1 or 3.98a2 --athaa-sensitivity 1?  Or do I wait until this anomaly has been properly diagnosed and ironed out?

I think I'm going to have to wait and see how this one progresses...
I'm on a horse.

lame 3.98 alpha 2 testing thread

Reply #8
@guruboolez
is it possible to get one or two of your test samples?

lame 3.98 alpha 2 testing thread

Reply #9
Quote
@guruboolez
is it possible to get one or two of your test samples?
[{POST_SNAPBACK}][/a]

Not sure if those are the used ones, but some of guru's samples are here:
[a href="ftp://ftp2.foobar2000.net/foobar/]ftp://ftp2.foobar2000.net/foobar/[/url]

lame 3.98 alpha 2 testing thread

Reply #10
Quote
@guruboolez
is it possible to get one or two of your test samples?
[a href="index.php?act=findpost&pid=335204"][{POST_SNAPBACK}][/a]

The 150 samples are all downloadable:
EDIT: links have changed

lame 3.98 alpha 2 testing thread

Reply #11
Some stats:



Code: [Select]
-V5 --vbr-new with and without --athaa-sensitivity 1

when difference is from: -1.0 to 0.4 point    (70 samples)    +2,84 kbps  in average with --athaa..1
when difference is from:  0.5 to 0.9 point    (29 samples)    +4,17 kbps  in average with --athaa..1
when difference is from:  1.0 to 1.4 points    (23 samples)   +7,70 kbps  in average with --athaa..1
when difference is from:  1.5 to 1.9 points    (13 samples)   +8,69 kbps  in average with --athaa..1
when difference is from:  2.0 to 2.4 points    (09 samples)   +14,44 kbps in average with --athaa..1
when difference is from:  2.5 points and more    (05 samples)    +19,00 kbps in average with --athaa..1



CONCLUSION: the perceptual quality improvment of --ahtaa-sensitivity 1 has a direct relation with the amount of bit added by this command line. When --ahtaa-sensitivity 1 has no impact on bitrate it has no impact on quality; when --ahtaa-sensitivity 1 "bloats" the bitrate it also "bloats" the quality.

10% of the tested samples reveal a tendency to "bitrate bloat" (+15 kbps or more). For these 14 samples, the average notation is going from 2.3 points (without --ahtaa-sensitivity 1) to 4.3 points (with --ahtaa-sensitivity 1).

____
Appendix:
RATIO
Code: [Select]
[average notation diff in points] / [average bitrate diff in kbps]

-0.01 / 2.84  =-0.0020
0.53 / 4.17  = 0.1264
1.03 / 4.70  = 0.1333
1.56 / 8.69  = 0.1796
2.00 / 14.44 = 0.1384
2.70 / 19.00 = 0.142
1

lame 3.98 alpha 2 testing thread

Reply #12
Well, it looks like there's very very few instances of regression and that any increase in bitrate is well justified.  Since I haven't done any testing myself, I can only say thank you very much for doing so much testing!

lame 3.98 alpha 2 testing thread

Reply #13
3.97 was recommended while still in alpha stage. Are there any significant improvements that would make 3.98 a good choice yet?

lame 3.98 alpha 2 testing thread

Reply #14
No, 3.97 was not recommended until it hit beta. Even that caused some noise, as in principle it should be GA code. But as the 3.x tree is pretty mature now, it is pretty safe.

Besides, it is just a recommendation, not the law

 
SimplePortal 1.0.0 RC1 © 2008-2019