Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728 (Read 21401 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

I was listening to Pet Shop Boys - In The Night from album  (1986)  Disco (The Remix Album) in mp3 -V4 and noticed the sound was not clean. So I figured it was time to compare Lame v3.98, Nero v1.1.34.2 and Ogg aotuv20080728 (supossedly the best version available) in the 100-190 kbps range. Every year or so I compare the most recent versions available.

I focussed on the 11.6-13 sec range but there are a lot of spots suitable for ABXing. Here go my ABX results.

Lame v3.98 produced 163 kbps output.
Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.5
2008/08/14 12:18:44

File A: F:\Temp\Pet Shop Boys  (1986)  Disco (The Remix Album) -V4\01 - In The Night.mp3
File B: F:\Temp\01 - Pet Shop Boys - In The Night.flac

12:18:44 : Test started.
12:19:09 : 01/01  50.0%
12:19:18 : 02/02  25.0%
12:19:33 : 03/03  12.5%
12:19:46 : 04/04  6.3%
12:20:04 : 05/05  3.1%
12:20:18 : 06/06  1.6%
12:20:31 : 07/07  0.8%
12:20:46 : 08/08  0.4%
12:21:04 : 09/09  0.2%
12:21:25 : 10/10  0.1%
12:21:31 : Test finished.

 ---------- 
Total: 10/10 (0.1%)


As Nero mostly often needs less bits to get a similar result to Lame I choosed to ABX Nero q0.40 which produced 138 kbps output.
Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.5
2008/08/14 12:34:54

File A: F:\Temp\Pet Shop Boys  (1986)  Disco (The Remix Album) Q040\01 - In The Night.mp4
File B: F:\Temp\01 - Pet Shop Boys - In The Night.flac

12:34:54 : Test started.
12:35:30 : 01/01  50.0%
12:35:36 : 01/02  75.0%
12:35:43 : 02/03  50.0%
12:35:49 : 03/04  31.3%
12:35:59 : 04/05  18.8%
12:36:12 : 05/06  10.9%
12:36:19 : 06/07  6.3%
12:36:26 : 07/08  3.5%
12:36:38 : 08/09  2.0%
12:36:51 : 09/10  1.1%
12:37:05 : 10/11  0.6%
12:37:13 : 11/12  0.3%
12:37:22 : 12/13  0.2%
12:37:54 : Test finished.

 ---------- 
Total: 12/13 (0.2%)


AS Nero q0.40 was also easy to ABX I ABXed Lame 3.98 (163 kbps) against Nero q0.45 (170 kbps) to see if I could match both lossy files and draw a conclusion about quality.
Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.5
2008/08/14 12:41:22

File A: F:\Temp\Pet Shop Boys  (1986)  Disco (The Remix Album) Q045\01 - In The Night.mp4
File B: F:\Temp\Pet Shop Boys  (1986)  Disco (The Remix Album) -V4\01 - In The Night.mp3

12:41:22 : Test started.
12:41:58 : 01/01  50.0%
12:42:06 : 02/02  25.0%
12:42:17 : 03/03  12.5%
12:42:28 : 03/04  31.3%
12:42:40 : 04/05  18.8%
12:42:49 : 05/06  10.9%
12:43:00 : 06/07  6.3%
12:43:09 : 07/08  3.5%
12:43:42 : 07/09  9.0%
12:44:00 : 08/10  5.5%
12:44:09 : 09/11  3.3%
12:44:17 : 10/12  1.9%
12:44:27 : 11/13  1.1%
12:44:37 : 12/14  0.6%
12:44:47 : 13/15  0.4%
12:44:54 : 14/16  0.2%
12:44:59 : 15/17  0.1%
12:45:15 : 16/18  0.1%
12:45:21 : Test finished.

 ---------- 
Total: 16/18 (0.1%)


On this sample and at these bitrates Nero clearly wins on quality as the distortions are much smaller than with lame3.98.

I got dissapointed about Ogg at q4 (133 kbps) and q5 (174 kbps) because it was too easy to ABX, the distortions were clearly worse than with Lame and Nero. I've tried some other songs (Prince, DJ Tiësto) and must conclude that Ogg Vorbis can't keep the pace of quality improvement of Lame and Nero.

Edit: Sample can be found here.
Edit2: Output bitrates mentioned in my posts are averages of the entire song, not of the 27 secs sample.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #1 –
Quote from: Alexxander on
... On this sample and at these bitrates Nero clearly wins on quality as the distortions are much smaller than with lame3.98. ...

The relation of Nero vs. Lame won't change, but do you mind trying a Lame 3.98 very high quality setting like -V0 or --abr 270?

I think mp3 users are aware of trading top playback compatibility against inferior quality per given bitrate compared to other codecs in critical situations, but with today's players' capacities using very high bitrate is an option for many users.
lame3995o -Q1.7 --lowpass 17

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #2 –
My ABX results focussing on range 11.7-12.8 secs:

Lame v3.98 abr 200 which resulted in a 194 kbps average output. This one wasn't difficult.
Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.5
2008/08/18 16:17:03

File A: F:\Temp\01 - Pet Shop Boys - In The Night.flac
File B: F:\Temp\01 - Pet Shop Boys - In The Night --398abr200.mp3

16:17:03 : Test started.
16:17:43 : 01/01  50.0%
16:17:50 : 02/02  25.0%
16:17:57 : 03/03  12.5%
16:18:04 : 04/04  6.3%
16:18:19 : 04/05  18.8%
16:18:28 : 05/06  10.9%
16:18:33 : 06/07  6.3%
16:18:38 : 07/08  3.5%
16:18:47 : 08/09  2.0%
16:19:04 : 09/10  1.1%
16:19:12 : 10/11  0.6%
16:19:22 : Test finished.

 ---------- 
Total: 10/11 (0.6%)


Lame v3.98 V0 which resulted in a 278 kbps average output. This one wasn't very difficult to ABX which surprised me.
Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.5
2008/08/18 16:45:56

File A: F:\Temp\01 - Pet Shop Boys - In The Night.flac
File B: F:\Temp\01 - Pet Shop Boys - In The Night --398V0.mp3

16:45:56 : Test started.
16:46:03 : 01/01  50.0%
16:46:11 : 02/02  25.0%
16:46:17 : 03/03  12.5%
16:46:23 : 04/04  6.3%
16:46:33 : 05/05  3.1%
16:46:42 : 06/06  1.6%
16:46:58 : 07/07  0.8%
16:47:07 : Test finished.

 ---------- 
Total: 7/7 (0.8%)


Since I had a post in mind that a specific sample encoded in V2 did better than V0 I tried to ABX V2 and got no clear results as well as with abr 270. I took a break because I noticed in previous ABX sessions that hearing gets tired as well as my brain. When I tried again several hours later I got this:

Lame v3.98 abr 270 which resulted in a 264 kbps average output. This one was hard, these results I got after a third trial reset.
Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.5
2008/08/18 22:03:31

File A: F:\Temp\01 - Pet Shop Boys - In The Night.flac
File B: F:\Temp\01 - Pet Shop Boys - In The Night --398abr270.mp3

22:03:31 : Test started.
22:04:05 : 01/01  50.0%
22:04:13 : 02/02  25.0%
22:04:23 : 02/03  50.0%
22:04:40 : 03/04  31.3%
22:04:54 : 04/05  18.8%
22:05:03 : 05/06  10.9%
22:05:20 : 06/07  6.3%
22:05:41 : 07/08  3.5%
22:06:16 : 08/09  2.0%
22:06:26 : 08/10  5.5%
22:06:34 : 09/11  3.3%
22:06:43 : 09/12  7.3%
22:06:53 : 10/13  4.6%
22:07:04 : 11/14  2.9%
22:07:23 : 12/15  1.8%
22:07:38 : 13/16  1.1%
22:07:56 : 14/17  0.6%
22:08:22 : 15/18  0.4%
22:08:30 : Test finished.

 ---------- 
Total: 15/18 (0.4%)

Then I tried again Lame V2 (222 kbps output) and after 5 trial resets I gave up. Maybe I'll try again when I get up with a fresh mind.

Finally I listened to both V0 (278 kbps) and abr 270 (264kbps) and abx-ed both lossy files 8/10  5.5% but my actual goal on this one was to compare distortion and artifacts (nearly not present). Clearly File A sounded better and that was the abr 270 sample.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #3 –
Quote from: Alexxander on
Lame v3.98 abr 270 which resulted in a 264 kbps average output. This one was hard, these results I got after a third trial reset.


Quote from: Alexxander on
Then I tried again Lame V2 (222 kbps output) and after 5 trial resets I gave up. Maybe I'll try again when I get up with a fresh mind.


When you say trial reset, do you mean re-performances of the tests?  You might consider posting the results of the additional trials that you performed - they are required for the results to be meaningful.

Why do the number of trials in your tests vary?  11, 7, 18, 10, 13, 18.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #4 –
Quote from: Tahnru on
When you say trial reset, do you mean re-performances of the tests?  You might consider posting the results of the additional trials that you performed - they are required for the results to be meaningful.
I exit the ABX tool and try again later. With hard samples at high bitrates normally I have to train/warm up my ears and spot the artifact which can differ with different encoding parameters.

Quote from: Tahnru on
Why do the number of trials in your tests vary?  11, 7, 18, 10, 13, 18.
I stop abx-ing of very tough samples when guessing probability is below 1% because of hearing fatigue risk when continuing. Where to put the limit? For the purpose of the ability to distinguish files I think it's better ending abx-test based on guessing probability than a fixed number of trials.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #5 –
Quote from: Alexxander on
I stop abx-ing of very tough samples when guessing probability is below 1% because of hearing fatigue risk when continuing. Where to put the limit? For the purpose of the ability to distinguish files I think it's better ending abx-test based on guessing probability than a fixed number of trials.


Pio wrote an excellent piece which explains ABX testing methodology.

Rule 3 (parts A and B) are particularly important here.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #6 –
Quote from: Tahnru on
Pio wrote an excellent piece which explains ABX testing methodology.

Rule 3 (parts A and B) are particularly important here.

Thanks for the link, I haven't seen that post before nor the linked ones inside it.

Rule 3 is nice but a lot can be discussed about it because every coin has two sides. Leaving a complete discussion for an other thread I just want to mention that the consistency and rigor of this rule break by allowing training sessions. In the absence of a notary there will always be doubts.

I usually do not hide the results to shorten the time spent but reading other posts linked I see the weakness of finishing after a low enough p value. When treating tough samples hiding results would just lead to spending a lot more of time and better results (higher success/failure ratio). The actual problem is one never knows if he/she is in top shape, not even Usain Bolt will do the 100 metres every day below 10 secs (and abx-ing is not a competition).

There's always a tradeoff.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #7 –
I have this album and will try to ABX your sample.
LAME 3.98.2

First Trial:

Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.6
2008/10/28 16:05:08

File A: D:\tmp\Disco\01 - In the Night (Extended).flac
File B: D:\tmp\Disco\01 - In the Night (Extended)-V4.mp3

16:05:08 : Test started.
16:07:36 : 00/01  100.0%
16:08:00 : 01/02  75.0%
16:08:06 : 01/03  87.5%
16:08:38 : 02/04  68.8%
16:08:43 : Test finished.

 ---------- 
Total: 2/4 (68.8%)


Second Trial:

Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.5.6
2008/10/28 16:09:38

File A: D:\tmp\Disco\01 - In the Night (Extended).flac
File B: D:\tmp\Disco\01 - In the Night (Extended)-V4.mp3

16:09:38 : Test started.
16:10:21 : 00/01  100.0%
16:10:30 : 01/02  75.0%
16:10:42 : 01/03  87.5%
16:10:56 : 01/04  93.8%
16:11:06 : 02/05  81.3%
16:11:16 : 03/06  65.6%
16:11:39 : 04/07  50.0%
16:11:55 : 05/08  36.3%
16:12:19 : 05/09  50.0%
16:12:40 : 06/10  37.7%
16:12:57 : Test finished.

 ---------- 
Total: 6/10 (37.7%)


I thought I heard something, but then, it's V4, pretty transparent to me.
I guess V0 keeps being my comfort zone since I don't encode tracks at their limit of transparency.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #8 –
Quote
I got dissapointed about Ogg at q4 (133 kbps) and q5 (174 kbps) because it was too easy to ABX, the distortions were clearly worse than with Lame and Nero. I've tried some other songs (Prince, DJ Tiësto) and must conclude that Ogg Vorbis can't keep the pace of quality improvement of Lame and Nero.

Have you tried with the official ogg/vorbis 1.2.0? I have listened to some of the samples suggested at http://www.hydrogenaudio.org/forums/index....=64673&st=0, and for some of the samples I can't hear a difference between the original and mp3 at 128 - but I think that the difference is very clear i the example with "Vangelis___Chariots_of_Fire__edit_.flac". With ogg/vorbis at 128 I can't hear a difference.

So I would like to know what I should listen for - especially when using ogg/vorbis.

Regards,
Morten

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #9 –
Quote
'...my actual goal on this one was to compare distortion and artifacts'
.

Alexxander, could you please describe the Aotuv q5 distortions which 'were clearly worse than with Lame and Nero'?

Also, can anybody with good hearing capabilities confirm Alexxander's conclusion about the quality inferiority of the latest OGG Aotuv at q4-q5 to Lame 3.98 and Nero Digital?

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #10 –
Quote from: M.A. on
Alexxander, could you please describe the Aotuv q5 distortions which 'were clearly worse than with Lame and Nero'?

Oh my, I give it a try. In this sample I ABX on the synthesizer that's making the dominant tune (is on foreground) in the first 17 secs of the song and also on the pacemaking cymbalsound (some acompanying high frequency artifact). Let me make clear that lame3.98 -V4 (163kbps) sounds horrible but aoTuVb5.6 q5.0 is close to transparant to me, as opposed to q4.0.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #11 –
Thank you, Alexander, I wish I had ears half as good as yours. All my attempts to ABX your Pet Shop Boys sample encoded even with Aotuv q2 ended complete fiasco.

Some time ago I switched from Lame MP3 V0 to OGG Aotuv q7 for my music collection which I play on my HTPC through a very decent Hi-Fi hardware. I chose the OGG format because many test results and people opinions on this forum had made me believe in its advantage over MP3 quality wise while being free from some problems such as gapless playback.

Regards,
Misha

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #12 –
Quote
I got dissapointed about Ogg at q4 (133 kbps) and q5 (174 kbps) because it was too easy to ABX, the distortions were clearly worse than with Lame and Nero. I've tried some other songs (Prince, DJ Tiësto) and must conclude that Ogg Vorbis can't keep the pace of quality improvement of Lame and Nero.


That's a bit of a streched based upon only a few samples. 

Quote
Also, can anybody with good hearing capabilities confirm Alexxander's conclusion about the quality inferiority of the latest OGG Aotuv at q4-q5 to Lame 3.98 and Nero Digital?


He would need to post some samples first. I don't know exactly what he is hearing, but in terms of using Vorbis which I encode with I have rarely heard some samples with audible distortion. Albeit they do exist, but should not be a problem especially if you are encoding with -q 6 or even 7 for that matter.
budding I.T professional

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #13 –
I'm sorry but I didn't see an ABX test for OGG...

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #14 –
Quote from: Neasden on
I'm sorry but I didn't see an ABX test for OGG...

A quick one from yesterday I saved, those from august I haven't anymore as from time to time I empty my Temp folder. But if any ABX of other specific setting is wanted I could give it a try.

Code: [Select]
foo_abx 1.3.3 report
foobar2000 v0.9.6 beta 4
2008/11/11 15:55:41

File A: F:\Temp\01 - Pet Shop Boys - In The Night.flac
File B: F:\Temp\01 - Pet Shop Boys - In The Night.ogg --ogenc2.85libvorbis1.2.0 -q4.0.ogg

15:55:41 : Test started.
15:58:43 : 01/01  50.0%
15:58:55 : 02/02  25.0%
15:59:12 : 03/03  12.5%
15:59:31 : 04/04  6.3%
15:59:39 : 05/05  3.1%
15:59:49 : 06/06  1.6%
15:59:59 : 07/07  0.8%
16:00:14 : 08/08  0.4%
16:00:31 : 09/09  0.2%
16:00:42 : 09/10  1.1%
16:01:03 : 10/11  0.6%
16:01:16 : 11/12  0.3%
16:01:30 : 12/13  0.2%
16:01:42 : 13/14  0.1%
16:02:00 : 14/15  0.0%
16:02:26 : Test finished.

 ---------- 
Total: 14/15 (0.0%)

Song average is 127kbps

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #15 –
Quote from: HotshotGG on
Quote
I got dissapointed about Ogg at q4 (133 kbps) and q5 (174 kbps) because it was too easy to ABX, the distortions were clearly worse than with Lame and Nero. I've tried some other songs (Prince, DJ Tiësto) and must conclude that Ogg Vorbis can't keep the pace of quality improvement of Lame and Nero.

That's a bit of a streched based upon only a few samples. 

Yes, could be true. This is why I recently asked in an other post how to setup and do myself an ABC/HR test. Once the results of the now running mp3 Listening Test are known I will get the best mp3 encoder and do some ABC/HR against ogg and nero aac. I'm thinking about using the same samples as the running Listening Test but I have a mixed feeling about testing problematic samples against less complex ones 

Quote
He would need to post some samples first
...

Flac is in this post.

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #16 –
On -V4 I abxed the intro tsk-tsk.. 8/8 - some typical imprecise sound like preecho.
Then at 19secs preecho on snare 8/8.

-V3 was too much effort: same spots 4/8 then 5/8

Lame v3.98 vs Nero v1.1.34.2 vs Ogg aotuv20080728

Reply #17 –
Quote
I've tried some other songs (Prince, DJ Tiësto) and must conclude that Ogg Vorbis can't keep the pace of quality improvement of Lame and Nero.


Seriously, statements like this would require an intensive multi-format listening test with many samples. The last one I've seen, Vorbis defeated LAME in nearly all samples. But that was for classical only. And the current encoders haven't been tested.