Updated V3 and V2 presets (vbr-new is unchanged)
What I am interested into is mainly results of V3 and V2 (especially V2) against either 3.96.1 or 3.97a7.
You can test it against 3.90.3 if you want, but I'd like to know the evolution from previous version first.
Results from 3.97a7:
http://www.hydrogenaudio.org/forums/index....showtopic=31255 (http://www.hydrogenaudio.org/forums/index.php?showtopic=31255)
Here's a quick and dirty compile (untested) done in MSVC6, without NASM optimizations.
Please don't link directly to it from elsewhere, it'll just stay there until John XXXIII comes up with a properly done compile.
Edit: look down
ABX results on trumpets sample.
a8 -V3 vs a7 -V3: 10/10 (0.1%) //a8 sounded worse.
a8 -V2 vs a7 -V2: 10/10 (0.1%) //a8 sounded worse.
3.97a8 -V3 vs 3.96.1 -V3: 10/10 (0.1%) //a8 sounded worse.
3.97a8 -V2 vs 3.96.1 -V2: 10/10 (0.1%) //a8 sounded worse.
In all four tests the latest, 3.97a8 version, sounded more "hoarse" during some parts of the sample, than the version it was compared with.
Does anyone know why I get a 30% faster lame (with preset standard) when I compile it myself?
I've uploaded it to www.geoshock.com/lame.zip , maybe it doesn't work as it should.
bye,
Michael
Does anyone know why I get a 30% faster lame (with preset standard) when I compile it myself?
I've uploaded it to www.geoshock.com/lame.zip , maybe it doesn't work as it should.
bye,
Michael
[a href="index.php?act=findpost&pid=279834"][{POST_SNAPBACK}][/a]
If you're referring to Roberto's compile, it's because it's (a) MSVC which is slower than ICL, and (b) mostly because it doesn't include the nasm assembler routines in the compile.
Anyway, new build now at Rarewares.
I still can ABX this sample very easy:
aps_Killer_sample.wav; from here (http://www.hydrogenaudio.org/forums/index.php?showtopic=30825)
My ABX results:
Settings: Lame 3.97a8 -V 2: ( 10/10 ) Very easy to ABX.
Settings: Lame 3.97a8 -V 2 --vbr-new: ( 10/10 ) Very easy to ABX again.
Angelic sample 397a7 vs 397a8.
-V 2
ABX 6/10 --> no difference to me.
-V 3
1st run: ABX 9/10 (very hard, need longer pauses between trials)
But damnit, I pressed cancel when foobar asked me to save the log because I wanted to jump back and try something, but it quit the ABX dialog and now I don't know which file was which!
2nd run to find out which is which: ABX 5/5 and to my ears 397a8 sounded slightly worse than 397a7. There was more high pitched "sparkle".
Total: 14/15
My badvilbel.flac (http://www.hydrogenaudio.org/forums/index.php?showtopic=7813&hl=badvilbel.flac) abx results:
Settings: Lame 3.96.1 --preset standard: ( 8/8 ) Very easy to ABX.
foo_abx v1.2 report
foobar2000 v0.8.3
2005/03/08 18:33:34
File A: file://C:\My Downloads\Encoders\Problem_audio_samples\badvilbel.flac
File B: file://C:\My Downloads\Encoders\Problem_audio_samples\3.96.1--aps\badvilbel.mp3
18:33:35 : Test started.
18:35:36 : 01/01 50.0%
18:36:41 : 02/02 25.0%
18:37:54 : 03/03 12.5%
18:39:37 : 04/04 6.3%
18:40:49 : 05/05 3.1%
18:41:56 : 06/06 1.6%
18:42:30 : 07/07 0.8%
18:43:04 : 08/08 0.4%
18:43:06 : Test finished.
----------
Total: 8/8 (0.4%)
---------------------------------------
Settings: Lame 3.97a8 --preset standard: ( 8/8 ) Very easy to ABX - although seemed to be slightly better (results do not support this at all though).
foo_abx v1.2 report
foobar2000 v0.8.3
2005/03/08 18:45:59
File A: file://C:\My Downloads\Encoders\Problem_audio_samples\badvilbel.flac
File B: file://C:\My Downloads\Encoders\Problem_audio_samples\3.97a8--aps\badvilbel.mp3
18:46:00 : Test started.
18:47:09 : 01/01 50.0%
18:47:31 : 02/02 25.0%
18:47:52 : 03/03 12.5%
18:48:10 : 04/04 6.3%
18:48:27 : 05/05 3.1%
18:48:47 : 06/06 1.6%
18:49:07 : 07/07 0.8%
18:49:25 : 08/08 0.4%
18:49:27 : Test finished.
----------
Total: 8/8 (0.4%)
---------------------------------------
Settings: Lame 3.97a8 -V 2 --vbr-new: ( 8/8 ) Very easy to ABX again.
foo_abx v1.2 report
foobar2000 v0.8.3
2005/03/08 19:08:32
File A: file://C:\My Downloads\Encoders\Problem_audio_samples\badvilbel.flac
File B: file://C:\My Downloads\Encoders\Problem_audio_samples\3.97a8-V2-vbr-new\badvilbel.mp3
19:08:33 : Test started.
19:09:19 : 01/01 50.0%
19:09:43 : 02/02 25.0%
19:10:00 : 03/03 12.5%
19:10:17 : 04/04 6.3%
19:10:34 : 05/05 3.1%
19:10:53 : 06/06 1.6%
19:11:10 : 07/07 0.8%
19:11:26 : 08/08 0.4%
19:11:38 : Test finished.
----------
Total: 8/8 (0.4%)
ABX results on castanets2 sample:
a8 -V3 vs a7 -V3: 10/10 (0.1%) // 3.97a8 sounded better than a7. Less pre-echo.
I couldn't hear a difference on any of the following tests:
a8 -V2 vs a7 -V2 (6/10)
a8 -V2 vs 3.96.1 -V2 (6/10)
a8 -V3 vs 3.96.1 -V3 (5/10)
LAME ACM bug
This probably is an old bug:
Here's example,
when the MIN bitrate is 8,
MAX is 320,
if i adjust the "step" slider from 16 to 12,
the "LAME MP3 codec:about" window will stop responding.
[span style='font-size:14pt;line-height:100%']LAME 3.97 alpha 8 TEST #1: -V 3[/span]
For the following tests, I tried to obtain the most accurate results. That’s why I didn’t include extra elements (like comparison with --vbr-new, or with an ancient lame version): my full attention was focused on difference between alpha8 and the previous one. For that reason, I put all efforts on ABX direct comparison between both encodings. 12 trials for each sample, no more, no less (except on training). No ABX tests between reference and encodings: unnecessary waste of time.
I’ve used previously 54 samples to test V4 preset. It was a luxury I can’t afford anymore, because my time is limited and my motivation quite over. Therefore, I returned to the general samples’ set used for alpha 5 and alpha 6 (20 samples, melting ‘general’ music, ‘classical’ and two critical samples). I’ve just replaced the cello’s sample (BachS1007.wav, too transparent at high bitrate) by another cello sample (Kodály.wav), more detailed and consequently harder to encode.
Other change: I’m now using the new java version of abc/hr, which is remarkable. It has a precious feature: it automatically saves the playback range. Now, developers could easily link all written comments to the corresponding part of the sample. I’ll precise all tested range in my synoptic table of results.
Other good point for schnofler’s software: it systematically cleans all comments wrote by the tester after each fresh test. ff123’s abchr beta 2 is buggy here, and often keeps previous comments. It’s sometimes funny (e.g. “distorted trumpet” on a solo piano sample…), but it’s often confusing. Comments are now fully reliable. I’ll recap the audible problems in my tables.
RESULTS[/u]
(http://audiotests.free.fr/tests/2005.03/lame397a8V3results.PNG)
ABX log files are here (http://audiotests.free.fr/tests/2005.03/lame397a8V3.7Z).
COMMENTS
Gabriel’s tuning have the same impact on -V3 than on -V4: it lowers the bitrate, and increase the quality. It’s important to note that the progress are also very similar to the progress noticed by testing -V4: there are less distortions (noticeable on cymbals mostly), no more issues with low volume parts, and less pre-echo (at least on non critical samples: castanets2.wav is still heavily smeared). It’s probably not a coincidence… I suppose that -V3 alpha 8 benefits from similar tunings than -V4 alpha7.
I’ve only noticed two serious regressions (ATrain and MidnightVoyage.wav). But on average, the new alpha 8 appeared to me clearly better than the old one.
[span style='font-size:14pt;line-height:100%']LAME 3.97 alpha 8 TEST #2: -V 2, aka --preset standard[/span]
Contrary to tunings performed by Gabriel on V4 and V3, the modified V2 preset doesn’t lower the average bitrate. It’s now slightly higher than before, and with a great disparity between samples. The most problematic case is thear1.wav: bitrate jumps from 213 (alpha 7) to 247 kbps, and the most annoying thing is the chasm between -V3 (167 kbps) and -V2 (247 kbps): + 48%, for an average difference between two presets of ~15% only. Is it exceptional? People listening to metal should take care.
RESULTS[/u]
(http://audiotests.free.fr/tests/2005.03/lame397a8V2results.PNG)
ABX log files are here (http://audiotests.free.fr/tests/2005.03/lame397a8V2.7Z).
I was glad to discover that the modified V2 is also better than the previous one. The progresses are similar to those noticed with -V4 and -V3: less preecho, less distortion, some rare ATH (?) issues corrected (no more ringing, or instability in highest frequencies). Two regressions: MidnightVoyage.wav (again), and castanets2.wav (no additional pre-echo, but an additional colouration which isn’t pleasant to my ears).
[span style='font-size:14pt;line-height:100%']LAME 3.97 alpha 8 EXTRA: comparison between different listening tests[/span]
It might be interesting to sum up the results of recent tunings made by Gabriel on defaulted VBR mode.
1/ -V4 was significantly inferior to -V4 --vbr-new (alpha 5), but after tunings (alpha 7), both produces similar quality encodings (with maybe a slight advantage for --vbr-new).
2/ -V2 was significantly inferior to -V2 --vbr-new (alpha 6). I didn’t compared the newer -V2 to -V2 --vbr-new, but the progresses noticed for -V2 alpha 8 are very similar (same genre, same level of improvement) to those noticed between alpha5 V4 and alpha7 V4. Therefore I strongly believe that -V2 and -V2 --vbr-new are now very close, and that 3.97 alpha 8 should reaches the same level of quality than 3.90.3 (see the following table comparison).
OT:
@guruboolez
i appreciate your work for lame.
imho lame would not be the same without your listening tests!
best regards
pest
Once again, thank you very much for those precious results.
Thanks a lot Guruboolez.
If more people could provide similiar test results (you don't need to test as many samples as Guru), 3.97 could easily become the recommended version once it's released.
thear1.wav bitrate at V2 is caused by extreme sfb21 bloating:
-V2 --> 247.3 kbps
-V2 -Y --> 173.1 kbps
@guruboolez
Overwhelming again! You are something of a living treasure!
Maybe a bit too overwhelming for a ABX newbie ... I think, it would be nice to have a small but defined sample-set with specific instructions for starters ... maybe this could motivate some people!
You lame developers are living treasure too
Ah, now -V2 and -V3 use x mode 9 instead of 3. This allows the maskingadjust and ath floor values to be set much lower (i.e. less masking and more sensitive ATH). This is essentially a "trading of the bits" and it appears the tradeoff is much better this way if guru's results are any indication.
These ABX tests were all done with vbr-old, correct?
@guruboolez
Overwhelming again! You are something of a living treasure!
Maybe a bit too overwhelming for a ABX newbie ... I think, it would be nice to have a small but defined sample-set with specific instructions for starters ... maybe this could motivate some people!
You lame developers are living treasure too
[a href="index.php?act=findpost&pid=281527"][{POST_SNAPBACK}][/a]
I agree! I wish I could help with testing but I just don't know where to begin. The problem is getting someone who knows what they are doing to actually organize some sort of standard ABX test.
I think, it would be nice to have a small but defined sample-set with specific instructions for starters...
I agree! I wish I could help with testing but I just don't know where to begin.
Cosign.
[span style='font-size:10pt;line-height:100%']ABX results on awe32_20sec[/span]
3.97a8 -V3 vs 397a7 -V3: 10/10
a8 gave some artifacts that sounded higher in pitch than the artifacts on the a7 encoded sample.
3.97a8 -V2 vs 3.97a7 -V2: 9/10
I didn't hear the high pitched artifacts with a8 -V2. Overall less artifacting than a7, so a8 was better.
3.97a8 -V2 vs 3.96.1 -V2: 10/10
3.97a8 was better.
[span style='font-size:10pt;line-height:100%']ABX results on OrionII (2.1)[/span]
3.97a8 -V3 vs 3.97a7 -V3: 8/10
Less smearing with a8, a8 was better.
3.97a8 -V2 vs 3.97a7 -V2: 10/10
Less smearing with a8.
3.97a8 -V2 vs 3.96.1 -V2: 3/10
They sounded the same.
[span style='font-size:10pt;line-height:100%']ABX results on hihat[/span]
wav vs LAME 3.97a8 -V3: 9/10
Each hit on the hihat produced a less precise/more spread sound with LAME.
wav vs LAME 3.97a8 -V2: 10/10
The same alteration here.
3.97a8 -V3 vs 3.97a7 -V3: 9/10
a8 sounded less precise than a7, a7 was closer to the original.
3.97a8 -V2 vs 3.97a7 -V2: 7/10
I'm not too sure here. Sometimes a7 sounded better, sometimes they sounded the same.
These ABX tests were all done with vbr-old, correct?
[a href="index.php?act=findpost&pid=281547"][{POST_SNAPBACK}][/a]
I've only used -V2 & -V3 command line. The VBR mode isn't --vbr-new.
Gabriel,
I tried to encode a track with this alpha today in a command line, and found there's a "switch" display between "long" and "short". I've checked the outputed mp3 and couldn't find any mixed-block, so it doesn't mean mixed-block.
I am curious what does this exactly mean.
transition blocks: start or stop
Now, I’ll probably make a good break with listening tests. Hope they were helpful. Anyway, a big thanks to all lame developers
Please don't stop now!? You've gotta follow this one through It's probably the most exciting thing happening here at HA right now, checking out Guru's meticulously performed tests! Together with the progress Gabriel is making with the alphas of course!
edit: typo
No need for testing a9?
Well, I hope the positive direction shown in V2n and V3n are trickled down to V4n and V5n which are what I use most of the time.
Thanks to the great testers in this group. I'm listening much less critically but still appreciate the progress that's being made.
Long Live Lame!
Now, I’ll probably make a good break with listening tests. Hope they were helpful. Anyway, a big thanks to all lame developers
Please don't stop now!? You've gotta follow this one through It's probably the most exciting thing happening here at HA right now, checking out Guru's meticulously performed tests! Together with the progress Gabriel is making with the alphas of course!
edit: typo
[a href="index.php?act=findpost&pid=282246"][{POST_SNAPBACK}][/a]
Good idea - i'm axciously awaiting your results.
- Lyx
No need for testing a9?
[a href="index.php?act=findpost&pid=282973"][{POST_SNAPBACK}][/a]
My thoughts as well, since I pulled and compiled the 3.97 Alpha 9 source earlier today.
No need for testing a9?
[a href="index.php?act=findpost&pid=282973"][{POST_SNAPBACK}][/a]
My thoughts as well, since I pulled and compiled the 3.97 Alpha 9 source earlier today.
[a href="index.php?act=findpost&pid=283708"][{POST_SNAPBACK}][/a]
Anyone know what changed on Alpha 9?
Alpha 9 doesn't matter as much, because LAME is now at Alpha 10. For those who want to browse any changes, you can view the tree online:
http://cvs.sourceforge.net/viewcvs.py/lame/lame/?sortby=date (http://cvs.sourceforge.net/viewcvs.py/lame/lame/?sortby=date)
The interesting bits are in the libmp3lame directory.
Alpha 9 doesn't matter as much, because LAME is now at Alpha 10. For those who want to browse any changes, you can view the tree online:
http://cvs.sourceforge.net/viewcvs.py/lame/lame/?sortby=date (http://cvs.sourceforge.net/viewcvs.py/lame/lame/?sortby=date)
The interesting bits are in the libmp3lame directory.
[a href="index.php?act=findpost&pid=283759"][{POST_SNAPBACK}][/a]
I didn't understand a thing
Although I believe no major changes have been made? I would like to think that if that was the case, Gabriel would have asked us to test.
We'll have to keep waiting.
By the way, does anyone know if the --athaa-sensitivity 1 switch has been added to the standard V5 preset, or do you still have to type it on the command line?
Anyone know what changed on Alpha 9?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=283739")
from Gabriel:
*correction of -Y with vbr-new
*removal of minimal bitrate for vbr-new
[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=32357]http://www.hydrogenaudio.org/forums/index....showtopic=32357[/url]
For your information, in alpha 10 I changed V1 and V0 dand updated a little V3 and V2.
Testing is not critical now for alpha10, as V3 and V2 changes are small. I am also planning to change a little ABR/CBR soon.
Anyway, if someone wants to test a10, feel free to start a new thread to report your results.
Gabriel,
Thanks again for all your hard work on Lame, really appreciate it.
Just wondering when you were planning on push the positive changes higher -V's down the line to V4 and V5? Especially the improvements in vbr-new.
already done