Lame 3.97b1 vs Original CD

Topic: Lame 3.97b1 vs Original CD (Read 17108 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.

Lame 3.97b1 vs Original CD

2005-09-29 02:00:24

When I got lame 3.97b1 it was my first time to try any 3.97 lame and tried encoding some songs with preset fast standard to see what all the fuss was about. To my amazement, I was very satisfied with the results. Now I can rip much faster than 3.96.1 preset extreme and the audio quality isn't compromised... plus smaller filesizes. I tried encoding various rock and jazz songs from my cds and thought hmm this is the format I'm switching to. It sounded identical to the original cd to my ears. After several days, I stopped listening critically and just enjoyed the music. Yesterday, I ripped Patricia Barber's A Fortnight live in France and felt like I was missing something from the original CD. Uh oh, now for som abx testing again.... The First song gotcha for example started of with a band introduction then the music starts. Slight difference in the beggining in the definition of the bass and sharpness of attacks. The original sounded more realistic because of this slight audible difference. Surprising thing that such a little difference in perceived audio quality gives such a big difference in the feeling of realism. Then I cued the song to a part somewhere in the middle with some fast attacking percussion/bells/cymbals... The CD was startling. It makes you lose your breath everytime you hear the high frequency attacks. The mp3 sounded very similar. I couldn't hear the old mp3 artifacts of smearing but something was different. No more startling effect. I thought this was psychological so I kept on switching back and forth between the cd and the mp3... Same results, The cd was just more startling. This time I said hmm, maybe there is some smearing of transients but just not like mp3s way back when, this time more subtle. This time the original sound signature is intact just without the jump factor of the original CD.
After all this listening I decided to try ripping just the first song "gotcha" with preset fast extreme to see if I can get closer to the original CD in terms of realism. I switched back between the 2 files and the CD to get impressions of their audio quality and I discovered that preset fast extreme did not bump up the realism with the percussions/cymbal/bell part compared to preset fast standard. I was thinking this is probably the limits of mp3 encoding today and that maybe fast standard was the best compromise between quality and size. Preset fast extreme to me had a slight advantage with the clearing up of the bass in the beggining of the song but this was either psychological or very very slight because I couldn't feel it everytime unlike the startling high frequency transients in the middle of the song.
After doing these tests I decided preset fast standard is right for me. None of them can match the realism of the original cd but with songs that are highly processed, the differences are insignificant. It is only when listening to realistic sounding recordings
that I feel a little shortchanged.

By the way, all this listening was done with my laptop computer a pa2v2 amp and sennheiser hd555 headphones. Not the highest end of sources but decent enough to consistently tell that the original cd was more realistic in its attack of transients. maybe with a higher end audio setup, I could be able to nitpick more between the extreme and standard presets but this was less of a difference compared to the original CD.

My Vote for portable use= preset fast standard. Is it archival quality? Yes if you don't plan to own high end audio equipment but are still nitpicky on sound quality. No, If you want an exact to CD sound. Maybe I should try vorbis but thats for another time

Lame 3.97b1 vs Original CD

Reply #1 – 2005-09-29 02:14:04

Could you please post a 30-second sample of this song, and perhaps do an ABX test with foobar or ABC/HR with 16 trials? This could be very useful for development - but only if it can be reproduced in blind testing.

(Otherwise you may be violating TOS#8.)

Lame 3.97b1 vs Original CD

Reply #2 – 2005-09-29 02:16:28

You mentioned ABX, but you didn't mention any figures.

Did you really do double-blind tests? If not, expectation bias ("the placebo effect") might come into play and ruin the results. If you did perform double-blind ABX tests, please show us the results.

Publishing non-blind test-results is not acceptable, according to item #8 of the Hydrogenaudio Terms of Service.

Lame 3.97b1 vs Original CD

Reply #3 – 2005-09-29 02:41:12

Quote

You mentioned ABX, but you didn't mention any figures.

Did you really do double-blind tests? If not, expectation bias ("the placebo effect") might come into play and ruin the results. If you did perform double-blind ABX tests, please show us the results.

Publishing non-blind test-results is not acceptable, according to item #8 of the Hydrogenaudio Terms of Service.
[a href="index.php?act=findpost&pid=330297"][{POST_SNAPBACK}][/a]

The middle of the song with the high frequency transients I tested blind and could easily pinpoint the original cd from the mp3 because of the perceived speed of the attack. But only if I do them within a few seconds of each other. Maybe If I were to listen to the mp3 today and the cd tomorrow I would not be able to tell the difference because they seem to be audibly the same. Its just that the cd always has the jump factor at that part of the song that the mp3 lacks. With the beginning of the song where I compared the song with the extreme preset... Ill try testing again if I can pinpoint the difference blind cause it was only a feeling and wasn't strong enough to give consistent results.
About the item #8 thing, I don't know if I really violated that cause I did state that it wasn't audible but was a feeling of more speed in the notes and it startled me more. The weird thing though is that this one is consistent. I just wanted to post this finding for people to see if they feel the same way. This thread is partly a question for you guys to try if you have the same findings with your music. I am just encouraging people to listen for themselves and am not really making any conclusions except for the mini summary I gave about preset fast standard being worth using portably and yadayada

Lame 3.97b1 vs Original CD

Reply #4 – 2005-09-29 02:46:44

Quote

Could you please post a 30-second sample of this song, and perhaps do an ABX test with foobar or ABC/HR with 16 trials? This could be very useful for development - but only if it can be reproduced in blind testing.

(Otherwise you may be violating TOS#8.)
[a href="index.php?act=findpost&pid=330296"][{POST_SNAPBACK}][/a]

I am a very new user of foobar. I just discovered after I came to hydrogen. I have not tried abc/hr. I'll look into it. I just double clicked between the songs and moved the seekbar to the exact same area to hear the same part of the song. As for posting a 30 second sample, how do i do that?

Lame 3.97b1 vs Original CD

Reply #5 – 2005-09-29 02:55:36

Quote

As for posting a 30 second sample, how do i do that?

cut the uncompressed file using your favorite audio editor. Audacity is free.
then use a lossless codec (e.g. FLAC) to compress the sample.

and please use punctuation in your posts.

Lame 3.97b1 vs Original CD

Reply #6 – 2005-09-29 03:23:25

Quote

Quote
As for posting a 30 second sample, how do i do that?
cut the uncompressed file using your favorite audio editor. Audacity is free.
then use a lossless codec (e.g. FLAC) to compress the sample.

and please use punctuation in your posts.
[a href="index.php?act=findpost&pid=330305"][{POST_SNAPBACK}][/a]

sorry bout that. english is my second language

Lame 3.97b1 vs Original CD

Reply #7 – 2005-09-29 03:58:42

How do I upload the sample? I have a 10 second wav file ripped with EAC and cut with audacity. At 4 seconds there is a cymbal being hit that just isn't as startling when converted to mp3. I have compared it to the original cd and havent listened to the edited wav file but it should be the same sound quality from the cd except for the different type of seeking noises i got with the cd.

Lame 3.97b1 vs Original CD

Reply #8 – 2005-09-29 03:59:23

Donunus,

What you have been doing is neither blind nor abx testing. You need to run a real abx (try the software mentioned above). Without ABX results, no one will take you seriously and will all point to placebo effect.

Lame 3.97b1 vs Original CD

Reply #9 – 2005-09-29 04:05:13

you really should take a look at foobar's abx feature. first you will need to rip the cd in a lossless format, like wavpack or flac, and then you can conduct a blind test of the sample. it will make switching between the lossless sample and the lossy sample alot easier, as you wont have to stop between samples to switch. in addition, it will eliminate any chance of a placebo effect. also, by posting your abx results, particularly if your hearing and sensitivity to mp3 artifacts are as good as it appears, you can greatly help the development of mp3. so please, do yourself and us a favor and look into foobar's abx features.

edit: typo

also, don't worry about your struggles with english. i have not had any trouble understanding your meanings, and you are in good company. a large percentage of this community are not native english speakers.

Lame 3.97b1 vs Original CD

Reply #10 – 2005-09-29 04:16:05

Hey, one thing at a time guys This sort of thing is very complicated the first time around. Heck, it's still complicated for me.

Donunus, you should do a spot check on the two 10-second clips to make sure they are from the same section of the file, then make a new topic in the Uploads forum with the WAVs added as attachments. You may want to compress them by downloading a lossless compressor, like FLAC or Monkey's Audio, and running them through it. You may also need to put the clips inside .zip files due to a bug in HydrogenAudio's forum interface.

It would be REALLY REALLY nice if you could be sure that the two clips you provide are both of the exact same length (in samples) and start at the exact same point in time (in samples). That makes the ABX test much more sensitive. LAME may offset the audio itself when encoding - you can try to fix that yourself if you want, but don't worry about it otherwise.

Lame 3.97b1 vs Original CD

Reply #11 – 2005-09-29 05:19:03

personally, i would conduct an abx w/ foobar before i went through all the trouble of cutting and uploading the samples here, just to make sure you've really got something... good luck either way.

Lame 3.97b1 vs Original CD

Reply #12 – 2005-09-29 05:29:54

Donunus:

Don't complicate a simple matter. Just encode the same passage with with -preset medium and then -preset 128. The artifact should become more apprent. Then encode in -preset insane and the artifact should be weaker or dissappear. If still in doubt do an abx test.

Lame 3.97b1 vs Original CD

Reply #13 – 2005-09-29 06:43:07

Quote

Donunus:

Don't complicate a simple matter. Just encode the same passage with with -preset medium and then -preset 128. The artifact should become more apprent. Then encode in -preset insane and the artifact should be weaker or dissappear. If still in doubt do an abx test.
[a href="index.php?act=findpost&pid=330324"][{POST_SNAPBACK}][/a]

Ill Do this, and ill also check foobars abx. With my listening tests for that sample. preset fast standard and fast extreme were hard to tell apart. only the cd was different. I'll double check again today with foobar abx encoding from my 10 second wav clip. from preset 128 to insane

Lame 3.97b1 vs Original CD

Reply #14 – 2005-09-29 07:52:16

I tested(abx) the file with foobar... preset fast standard vs wav and was able to distinguish them apart 100% out of 7 trials. I did not do anymore cause it was always obvious to me as to which file was which. It was like a shwook before the cymbal hit or something before the actual ping of the cymbal

Lame 3.97b1 vs Original CD

Reply #15 – 2005-09-29 08:06:28

My guesses were correct when I first wrote the comparison. I think the artifact is more of an mp3 thing than a bitrate thing because to my surprise, preset 128 was much more similar to insane for example compared to the original wav. Hmm, It might be a frequency and sound signature that does not require a big bitrate to reproduce for mp3 but it had what I might call a common lame sound signature that differs it from the wav file no matter what the bitrate

Lame 3.97b1 vs Original CD

Reply #16 – 2005-09-29 08:08:15

wow, you apparently have good hearing and sensitivity to mp3 artifacts. you should try the test again with preset standard (or -V2) as opposed to preset fast standard (or -V2 --vbr-new). i would be very curious to see if the old vbr algorithm does better than the new one. also, you may want to try -V1, -V0, and even -b 320 to see where you achieve transparency. also, if you could post the sample and your test results when you get a chance, it would me much appreciated.

Lame 3.97b1 vs Original CD

Reply #17 – 2005-09-29 08:22:53

Quote

wow, you apparently have good hearing and sensitivity to mp3 artifacts. you should try the test again with preset standard (or -V2) as opposed to preset fast standard (or -V2 --vbr-new). i would be very curious to see if the old vbr algorithm does better than the new one. also, you may want to try -V1, -V0, and even -b 320 to see where you achieve transparency. also, if you could post the sample and your test results when you get a chance, it would me much appreciated.
[a href="index.php?act=findpost&pid=330352"][{POST_SNAPBACK}][/a]

I posted the sample wav clip in the uploads section. Try it for yourself and see what I mean. I doubt I'll be able to tell much between preset standard and fast standard because this file didn't seem too preset dependent. As I said, Insane and 128 sounded closer to me than the wav vs fast standard. It seems that the deterioration happens when it is converted to mp3 no matter what preset it is...even 320(insane).

Ive done listening tests before with 3.90.3 and although i havent discovered foobar abx yet, I thought 3.90.3 sounded drier than the wav overall at alt preset standard with 3.96 being a vast improvement for general music listening in my impressions. This didn't have abx test backing though like I did just now.

Lame 3.97b1 vs Original CD

Reply #18 – 2005-09-29 09:35:35

For the development of mp3 it would be good, if you would do the foobar abx test with 3.90.3 vs. 3.97b1 at preset standard vs. --V2 (--vbr-new), also the comparison 3.97b1 -V2 vs- -V2 --vbr-new
Now that you have a good sample, and you discovred foobar, it is easy for you

Lame 3.97b1 vs Original CD

Reply #19 – 2005-09-29 23:50:29

Quote

As I said, Insane and 128 sounded closer to me than the wav vs fast standard. It seems that the deterioration happens when it is converted to mp3 no matter what preset it is...even 320(insane).
[a href="index.php?act=findpost&pid=330356"][{POST_SNAPBACK}][/a]

I downloaded the clip, the original sound clip would also be nice for a foobar2000 ABX test. Without doing a ABX test then all "claims" of quality are not justified. I listened to the sample, it didn't sound bad but I have nothing to compare it to. The same 12 second clip would be needed of the original file (lossless or directly ripped to wave). I also have a hard time beleiving that preset 128 would sound better than preset fast standard. Additionally, the encoding of a mp3 to a wav file should result in no loss of quality with the resulting wav file compared to the mp3.

Lame 3.97b1 vs Original CD

Reply #20 – 2005-09-30 02:38:37

Quote

Quote
As I said, Insane and 128 sounded closer to me than the wav vs fast standard. It seems that the deterioration happens when it is converted to mp3 no matter what preset it is...even 320(insane).
[a href="index.php?act=findpost&pid=330356"][{POST_SNAPBACK}][/a]

I downloaded the clip, the original sound clip would also be nice for a foobar2000 ABX test. Without doing a ABX test then all "claims" of quality are not justified. I listened to the sample, it didn't sound bad but I have nothing to compare it to. The same 12 second clip would be needed of the original file (lossless or directly ripped to wave). I also have a hard time beleiving that preset 128 would sound better than preset fast standard. Additionally, the encoding of a mp3 to a wav file should result in no loss of quality with the resulting wav file compared to the mp3.
[a href="index.php?act=findpost&pid=330497"][{POST_SNAPBACK}][/a]

I did do an abx test. wav vs preset fast standard. i detected the differences 100% of the time for 7 trials. I only uploaded the wav so that you could encode the file yourself for testing in whatever format you want. As for the mp3 comparisons, I was having a hard time detecting the differences in quality for this clip. They all had the subtle mp3 smearing(maybe in different degrees) but i couldnt detect which bitrate was what. The WAV was the only one that really stood out for me

Lame 3.97b1 vs Original CD

Reply #21 – 2005-09-30 17:56:55

Quote

I did do an abx test. wav vs preset fast standard. i detected the differences 100% of the time for 7 trials. I only uploaded the wav so that you could encode the file yourself for testing in whatever format you want. As for the mp3 comparisons, I was having a hard time detecting the differences in quality for this clip. They all had the subtle mp3 smearing(maybe in different degrees) but i couldnt detect which bitrate was what. The WAV was the only one that really stood out for me
[a href="index.php?act=findpost&pid=330526"][{POST_SNAPBACK}][/a]

I am sorry, for some reason I thought that the wav was of the compressed mp3, not the original source. I had my friend take a look at it. He blind tested the sample via the ABX method using his equipment, since mine is only slightly above average while he has a really good sound setup. He could not hear a difference between the -V 2 --vbr-new m3 encoded with Lame3.97b1 and the wav. He did a total of 15 trials and failed them all. I sat in the pilot seat for 15 trials and I could not hear a difference either. I even tested some other audio codecs for the hell of it and I failed them all.

Unfortunately someone else is going to have to add what they thought of in their ABX tests.

Lame 3.97b1 vs Original CD

Reply #22 – 2005-09-30 18:47:52

I'm oversensitive to high frequency transients but not lucky about it. Thats why i usually avoid taking part in codec-tests, because i dont want to *train* myself in noticing even "better" what annoys me already anyways. But after listening to this sample, i thought that this might be a quick'n easy job for me where i could help out.

I encoded the sample with LAME 3.97b1 to preset fast medium, used foobars ABX-comparator, and did hide the results, so that i wouldn't need to decide about the number of trials beforehand (so that i could stop when suffering from fatigue which i knew would happen soon with this kind of audiomaterial).

I stopped after 7 trials, because i felt that the difference was beginning to fade and that my ears began to imagine all kinds of things.... the results after i did unhide them were:

Code: [Select]

foo_abx 1.2 report
foobar2000 v0.9 beta 8
2005/09/30 19:29:54

File A: D:\clip.wav
File B: D:\2 clip.mp3

19:29:55 : Test started.
19:31:52 : 01/01  50.0%
19:32:34 : 02/02  25.0%
19:34:11 : 03/03  12.5%
19:35:05 : 04/04  6.3%
19:36:35 : 05/05  3.1%
19:37:55 : 05/06  10.9%
19:41:15 : 06/07  6.3%
19:49:27 : Test finished.

 ---------- 
Total: 6/7 (6.3%)

edit: i focussed on the short and sharp attacks. They have pre-echo issues in the LAME-version, which doesn't suprise me that much, because those drums are almost microattacks.

edit2: kbit-average for the APFM-mp3 is 172

Lame 3.97b1 vs Original CD

Reply #23 – 2005-10-01 08:37:01

Quote

I'm oversensitive to high frequency transients but not lucky about it. Thats why i usually avoid taking part in codec-tests, because i dont want to *train* myself in noticing even "better" what annoys me already anyways. But after listening to this sample, i thought that this might be a quick'n easy job for me where i could help out.

I encoded the sample with LAME 3.97b1 to preset fast medium, used foobars ABX-comparator, and did hide the results, so that i wouldn't need to decide about the number of trials beforehand (so that i could stop when suffering from fatigue which i knew would happen soon with this kind of audiomaterial).

I stopped after 7 trials, because i felt that the difference was beginning to fade and that my ears began to imagine all kinds of things.... the results after i did unhide them were:

Code: [Select]
foo_abx 1.2 report
foobar2000 v0.9 beta 8
2005/09/30 19:29:54

File A: D:\clip.wav
File B: D:\2 clip.mp3

19:29:55 : Test started.
19:31:52 : 01/01  50.0%
19:32:34 : 02/02  25.0%
19:34:11 : 03/03  12.5%
19:35:05 : 04/04  6.3%
19:36:35 : 05/05  3.1%
19:37:55 : 05/06  10.9%
19:41:15 : 06/07  6.3%
19:49:27 : Test finished.

 ---------- 
Total: 6/7 (6.3%)
edit: i focussed on the short and sharp attacks. They have pre-echo issues in the LAME-version, which doesn't suprise me that much, because those drums are almost microattacks.

edit2: kbit-average for the APFM-mp3 is 172
[a href="index.php?act=findpost&pid=330666"][{POST_SNAPBACK}][/a]

I did get tired after 7 results too. I got a perfect score but the swoosh before the cymbal hit becomes less obvious after fatigue. I got scared to go above 7. hehehe

foo_abx v1.2 report
foobar2000 v0.8.3
2005/09/29 14:27:24

File A: file://C:\My Documents\clip.wav
File B: file://C:\My Documents\clipstandard.mp3

14:27:26 : Test started.
14:27:55 : Trial reset.
14:32:19 : 01/01 50.0%
14:33:26 : 02/02 25.0%
14:33:52 : 03/03 12.5%
14:34:19 : 04/04 6.3%
14:35:11 : 05/05 3.1%
14:35:44 : 06/06 1.6%
14:36:12 : 07/07 0.8%
14:36:55 : Test finished.

----------
Total: 7/7 (0.8%)

Lame 3.97b1 vs Original CD

Reply #24 – 2005-10-01 08:42:15

Oh Lyx, Did you try an abx between different LAME presets? If you did what were your results? Can you try Insane vs fast standard? That would be more interesting cause after doing the wav to mp3 test, I couldn't tell the difference between insane and fast standard with this sample. Maybe I should try it again before any fatigue

Notice