HydrogenAudio

Lossy Audio Compression => MP3 => MP3 - General => Topic started by: Jojo on 2005-04-15 20:52:19

Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-15 20:52:19
It seems that mp3Gain sometimes doesn't perfectly undo the applied volume change. I noticed that the Music CRC of some files doesn't match anymore even though I used the 'undo' feature. I'm not sure yet why it happens, because not every file is affected. I was able to reproduce this several times using the same file - always starting off with a fresh copy of it with matching CRC's...

It's also interesting that the same file was gained to 90,6dB and later to 90,5dB.

Any ideas?

Edit: I'm talking about LAME mp3's and the Music-CRC that is stored inside the LAME-Header...
Title: mp3Gain not always able to undo TrackGain
Post by: timcupery on 2005-04-15 21:43:00
If you're just getting the file CRC, it will be different after change+undo because mp3gain writes info to files using APE tags. If you change the file's gain, then undo it, and then "remove tags from files", then the file's CRC should be exactly the same as before touching it with mp3gain.
In cases where APE tags were already written to the file when you first got the CRC, then made gain change and then undid the gain change, the CRC should not change.
The point here being that the info in a tag, as well as the audio data, affects the file CRC.
Does "music CRC" refer to a checksum based only on music data that should be unaffected by the tag?

One other way to test this is by converting the mp3 to wav (using foobar2000 set on no dither) before and after changing using mp3gain. The wav output files should be identical.

I wouldn't worry about the 90.6 vs. 90.5, etc. This happens when a file's gain info is estimated on the border (so the actual value would be 90.5498 or something like that, in between 90.5 and 90.6).

Edit: it's also possible that I misunderstand the problem and you're talking about something else, especially on the CRC point. I just figured that this is likely what's going on.
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-15 23:14:56
Quote
Edit: it's also possible that I misunderstand the problem and you're talking about something else, especially on the CRC point. I just figured that this is likely what's going on.
[a href="index.php?act=findpost&pid=290879"][{POST_SNAPBACK}][/a]

thanks for your answer, but indeed, you misunderstood something
I should have mentioned that, but for me mp3 = LAME mp3. Anyway, I was talking about the Music CRC that is stored inside the LAME-Header. It doesn't matter if you add/delete/alter ID3v1/ID3v2/APEv2 tags after the checksum has been written, it will still match, because the (Lame) CRC only depends on the music part...

I don't care if there's a little volume difference, I just want that the Music-CRC matches again after I reset the volume. This works in most cases, but sometimes it doesn't...I think to reproduce that on every file you'd have to change the volume to 50dB or something...but I have files were I changed the volume from 96dB to 91dB and the error still accured...
Title: mp3Gain not always able to undo TrackGain
Post by: westgroveg on 2005-04-16 00:31:52
Quote
It's also interesting that the same file was gained to 90,6dB and later to 90,5dB.

MP3Gain can only make changes in 1.5db steps so i don't see how this could be possible.
Title: mp3Gain not always able to undo TrackGain
Post by: timcupery on 2005-04-16 17:11:53
I still doubt that the actual music data is different after doing + undoing with mp3gain. To test this, as I noted above, you should use convert the file to wav (without dithering the output) before touching it with mp3gain, and then after undoing with mp3gain. I expect that the actual wav output will be exactly the same, whether Lame's music CRC has changed or not.

Btw: westgroveg - I answered about the 90.5 vs 90.6 above. I figure it means nothing.
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-16 23:53:13
Quote
Quote
It's also interesting that the same file was gained to 90,6dB and later to 90,5dB.

MP3Gain can only make changes in 1.5db steps so i don't see how this could be possible.
[a href="index.php?act=findpost&pid=290907"][{POST_SNAPBACK}][/a]

not really...sometimes a file goes from 90.5 to 91.1 and vice versa...must have something to do with the rounding...
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-16 23:57:46
Quote
I still doubt that the actual music data is different after doing + undoing with mp3gain. To test this, as I noted above, you should use convert the file to wav (without dithering the output) before touching it with mp3gain, and then after undoing with mp3gain. I expect that the actual wav output will be exactly the same, whether Lame's music CRC has changed or not.
[a href="index.php?act=findpost&pid=291072"][{POST_SNAPBACK}][/a]

ok, I'll do that...but anyway, the LAME-Tag CRC is not touched, no matter what you do to the file, because it's stored within the LAME-Tag...it simply doesn't match if you re-calculate the current CRC and compare it to the one stored in the LAME-Tag...and again, that's not the case for every file...there are just some rare exceptions...
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-17 00:59:02
ok, here is an update:

1) I decompressed the original mp3 to wav
2) I changed the volume -3dB
3) I used mp3Gain's undo feature (volume was reported the same as before I applied the gain)
4) I calculated the mp3's Music-CRC and compared it to the one stored in the mp3. They didn't match anymore
5) I recalculated the volume using mp3Gain (just to make sure that the undo was applied).
6) I decompressed the mp3 to wav
7) I calculated the md5 hash of both files and compared them. They weren't matching!

8) I did the same with some other mp3. The only difference was that the Music-CRC was matching again after I used the undo function and that the 2 resulting wav files had matching md5 and sha-1 checksums...
Title: mp3Gain not always able to undo TrackGain
Post by: timcupery on 2005-04-17 13:01:06
Okay, interesting. This is good to know... but I'm still not sure. Two worries, the second of which may be valid:
1) since you had other mp3's where the wav files were identical, I presume you're using a decoder that doesn't dither
2) I'd be interested for you to test the "before" and "after" wav files in the instance that the CRC is different, using ExactAudioCopy's "compare to wav files" function. This will tell you what and where the differences.
I'm interested to make sure what's going on here, as I use mp3gain a lot.
Thanks.
Title: mp3Gain not always able to undo TrackGain
Post by: Snelg on 2005-04-18 00:02:35
Can you send me a copy of the mp3 that's behaving strangely?
I'd like to figure out what's going on here, too.

-Glen
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-18 17:55:31
File 1): 2 files, both copied from the same mp3. One song with matching CRC's, other song without matching CRC's, but both are at the same volume. I removed all tags from the files to verify this (using mp3Tag) + I analysed both files again using the re-calculate function mp3Gain offers. I went back to the original file I made the copy from (different hard drive and folder) and used the undo feature...all of a sudden the CRC's matched!

File 2): a re-calculating of the volume revealed that the volume stored by mp3gain was wrong. It's interesting that the 'original' file volume (the one the undo function refers to) was actualy the volume the file should have been adjusted to. So instead of applying -6dB, mp3Gain applied -12dB, but when came to write the mp3gain-tags it acted as if -6dB were applied only (like I requested it)

File 3): the undo-tag volume was set to 96dB, but should have been 97,5dB

File 4): couldn't figure out how to get the CRC's matching, but it was interesting to see that mp3Gain changed the volume to -15,1dB -> see screenshot.

thanks
Title: mp3Gain not always able to undo TrackGain
Post by: 2Bdecided on 2005-04-19 10:46:49
Is your machine overclocked?

Just a thought! Probably not, but worth checking...

Cheers,
David.
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-19 17:08:04
here's another strange thing of file 4, besides the one already reported on the first screenshot...
[a href="http://www.imageshack.us" target="_blank"]
Title: mp3Gain not always able to undo TrackGain
Post by: timcupery on 2005-04-21 21:20:23
I'd still be interested to hear about this; it may give you more of a sense of what's going on:
Quote
2) I'd be interested for you to test the "before" and "after" wav files in the instance that the CRC is different, using ExactAudioCopy's "compare to wav files" function. This will tell you what and where the differences. [a href="index.php?act=findpost&pid=291280"][{POST_SNAPBACK}][/a]
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-22 00:28:08
Quote
I'd still be interested to hear about this; it may give you more of a sense of what's going on:
Quote
2) I'd be interested for you to test the "before" and "after" wav files in the instance that the CRC is different, using ExactAudioCopy's "compare to wav files" function. This will tell you what and where the differences. [a href="index.php?act=findpost&pid=291280"][{POST_SNAPBACK}][/a]

[a href="index.php?act=findpost&pid=292507"][{POST_SNAPBACK}][/a]

how about if you read my post? I posted a screenshot that contains 3 examples...1 song, 3 different samples, same volume, 2 different CRC's...all compared to the song with matching CRC's...
Title: mp3Gain not always able to undo TrackGain
Post by: timcupery on 2005-04-22 21:43:56
Quote
how about if you read my post? I posted a screenshot that contains 3 examples...1 song, 3 different samples, same volume, 2 different CRC's...all compared to the song with matching CRC's...
[a href="index.php?act=findpost&pid=292555"][{POST_SNAPBACK}][/a]

Yeah, I saw your post. I just don't know enough about music CRC's to trust them.  (Which isn't to say that I think they're untrustworthy - I don't know enough to think that, either.) I'm just always slightly skeptical about adjustments to mp3 and subsequent calculations. So I'm curious if the actual sample-by-sample wav data that comes out of the mp3 when decoded without dither is actually different.
That's all.
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-22 21:56:00
Quote
Quote
how about if you read my post? I posted a screenshot that contains 3 examples...1 song, 3 different samples, same volume, 2 different CRC's...all compared to the song with matching CRC's...
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=292555")

Yeah, I saw your post. I just don't know enough about music CRC's to trust them.  (Which isn't to say that I think they're untrustworthy - I don't know enough to think that, either.) I'm just always slightly skeptical about adjustments to mp3 and subsequent calculations. So I'm curious if the actual sample-by-sample wav data that comes out of the mp3 when decoded without dither is actually different.
That's all.
[a href="index.php?act=findpost&pid=292748"][{POST_SNAPBACK}][/a]

another hint: take a look at this: [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=33296&view=findpost&p=291900]http://www.hydrogenaudio.org/forums/index....ndpost&p=291900[/url] the second screenshot...you must be blind...
Title: mp3Gain not always able to undo TrackGain
Post by: timcupery on 2005-04-22 22:39:36
Oops, sorry. I just saw the mp3gain screenshot on your second post with screenshots, and then skipped below. I must have just assumed that was all you had in the post, and I know that mp3gain can't actually tell you if samples are different. So no, I didn't see the second screenshot you added with the edit.

So, does this leave us with no sense of how mp3gain messed up your files?
Title: mp3Gain not always able to undo TrackGain
Post by: Snelg on 2005-04-23 08:23:28
Quote
So, does this leave us with no sense of how mp3gain messed up your files?


I know I'm confused. Jojo, yeah, send me the four copies of "file 4". I wanna look at the raw data myself.

Use the address that's conveniently located on the "Help - About..." screen in mp3gain itself.

-Glen
Title: mp3Gain not always able to undo TrackGain
Post by: Snelg on 2005-04-26 22:22:07
I looked at some mp3 files that Jojo sent me.
The mp3 data differed by 1 to 3 bits. Not even full bytes. 3 bits total.
The flipped bits aren't even in the "gain" part of the data, which is the only thing that mp3gain touches.
There are tag differences between the files (some have ID3v2 tags, some don't, some have title/artist/etc. in the APE tags, some don't), so mp3gain is not the only program that has made changes to the files.
Jojo, are you able to re-create the problem with a fresh, clean mp3 (i.e. no tags of any kind) using only mp3gain functions?

-Glen
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-27 00:28:32
Quote
I looked at some mp3 files that Jojo sent me.
The mp3 data differed by 1 to 3 bits. Not even full bytes. 3 bits total.
The flipped bits aren't even in the "gain" part of the data, which is the only thing that mp3gain touches.
There are tag differences between the files (some have ID3v2 tags, some don't, some have title/artist/etc. in the APE tags, some don't), so mp3gain is not the only program that has made changes to the files.
Jojo, are you able to re-create the problem with a fresh, clean mp3 (i.e. no tags of any kind) using only mp3gain functions?

-Glen
[a href="index.php?act=findpost&pid=293557"][{POST_SNAPBACK}][/a]

well, the only program I used for tagging was mp3tag. So I don't think that's part of the problem...when I first encountered the problem I didn't even tag the files. I used the original file with matching CRC's and changed the volume and went back to it's original volume. I've tried that several times and 5 out of 7 times the CRC's were broken. I always started with a fresh copy. Since the file is too big I decided to look for some more samples in my archive and hoped that this would help you to find the bug...

In addition, only 1 of the 4 samples I've sent you have been tagged using mp3Tag. The original hasn't been changed at all. The other sample has been changed using mp3Tag, however, I made 2 extra copies of it - which are different as well. And after I made the copies of it and started to mess around with the files they have not been changed with any other program than mp3Gain; even though I might have removed/added tags, but that was only because I wanted to see if that changes the file's *actual* CRC and therefore could have caused the problem, which wasn't the case...so even if mp3Tag changed one sample, which I can't retrace anymore, the question remains why are it's 2 copies different then?

Also, 2 of the samples have identical *actual* Music CRC's (that don't match with the Music CRC stored in the LAME-Tag) and they both behave differently. How do you explain that? Check sample 2 & 3! The max no clip value differs!

Anyway, I wonder why some bits make a rather big difference in calculating the actual volume + it's max no clip value...
Title: mp3Gain not always able to undo TrackGain
Post by: Snelg on 2005-04-28 06:11:20
Quote
Also, 2 of the samples have identical *actual* Music CRC's (that don't match with the Music CRC stored in the LAME-Tag) and they both behave differently. How do you explain that? Check sample 2 & 3! The max no clip value differs!


Simple: the tags say they have different peak amplitudes. If you choose "Ignore tags" from the mp3gain menu (or remove tags) and then re-calculate the mp3gain info of those two files, they look exactly the same.
Of course, that leaves the question of how they ended up like that in the first place.

What I need from you is a precise, step-by-step recreation of the problem starting with an extra-fresh mp3 with absolutely no tags whatsoever. No mp3gain tags, no nothin'.

Save a copy of the tag-less mp3.

Then just do an analysis in mp3gain (with tag usage turned on as normal) and save a copy of that mp3.

Then try to re-create the error, keeping track of precisely what you do. Not just "adjust gain", but "1) set 'Normal Volume' to 91dB. 2) Press 'Track Gain' button", etc.

Then when the error rears its ugly head, save a copy of that version, and send me the three files along with the detailed steps on how you went from the analyzed version to the messed up version.

Quote
Anyway, I wonder why some bits make a rather big difference in calculating the actual volume + it's max no clip value...


I wouldn't say 0.03 dB is a big difference in the volume
That's the "precise" difference: 98.34 dB vs. 98.37 dB. They just round differently.

As for the max no-clip value, the flipped bits are in the middle of the mp3 data, so apparently it's throwing off the decoder and making some slightly-too-loud samples for a few milliseconds. A tiny spike like that can throw off the max no-clip value without having a very large effect on the "volume". (which is the whole point behind the Replay Gain algorithm in the first place)
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-28 21:54:36
Quote
Quote
Also, 2 of the samples have identical *actual* Music CRC's (that don't match with the Music CRC stored in the LAME-Tag) and they both behave differently. How do you explain that? Check sample 2 & 3! The max no clip value differs!


Simple: the tags say they have different peak amplitudes. If you choose "Ignore tags" from the mp3gain menu (or remove tags) and then re-calculate the mp3gain info of those two files, they look exactly the same.
Of course, that leaves the question of how they ended up like that in the first place.

I hoped that this would give you a clue what might be going on...that seems to be part of the problem. I suspect that the volume tag is sometimes not written correctly - or maybe rounded wrong...

Quote
What I need from you is a precise, step-by-step recreation of the problem starting with an extra-fresh mp3 with absolutely no tags whatsoever. No mp3gain tags, no nothin'.
[a href="index.php?act=findpost&pid=293828"][{POST_SNAPBACK}][/a]

that's gonna be pretty hard to achieve...anyway, what if the bug has something to do with certain tags including the mp3Gain tags?
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-28 23:56:50
holly cow! I've found files that have a volume of 113dB when I applied the 'undo'. The max-no-clip is -17dB...there's no way that this was the original volume! Besides that, mp3Gain only applies volumes upto 105dB...the other 2 I spoted have a volume of 107dB when I used the undo feature...the max-no-clip is similar to the one mentioned above...
Title: mp3Gain not always able to undo TrackGain
Post by: Snelg on 2005-04-29 02:05:28
Quote
Quote
Of course, that leaves the question of how they ended up like that in the first place.

I hoped that this would give you a clue what might be going on...that seems to be part of the problem.


The only clue that gives me is that maybe something's going on with the tags.

Quote
Quote
What I need from you is a precise, step-by-step recreation of the problem starting with an extra-fresh mp3 with absolutely no tags whatsoever. No mp3gain tags, no nothin'.

that's gonna be pretty hard to achieve...anyway, what if the bug has something to do with certain tags including the mp3Gain tags?


No, I just mean start with a no-tag mp3 file.

The point here is that I have no idea what you're doing to these files. I can't recreate the problem on my end. I tried fiddling around with the files in MP3Gain (changing volume, un-doing, etc.), and I didn't have any problems. If I know exactly what you're doing, precisely step-by-step, then I can try doing those exact same steps.

"Repeatable error" is the key phrase here. If I can't repeat the error myself, I can't fully analyze it.

-Glen
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-04-29 18:40:20
How about if you check some songs in your library? 5% of the songs I've tested were messed up; even though I applied the original volume the CRC still wasn't matching, so I had to re-rip.

Grab, let's say 100 songs (200 would be better; I've checked 400), which you've ripped and that contain an undo tag. Copy those files to a seperate folder and restore their original volume. After that, use "Audio Identifier" (AI) to see whether the CRC's are matching or not. The fastest way to do so is to select all files at once and view the LAME-Tag (it might take some seconds to gather the information).

After that, you copy everything that's in the Lame-Tag box to a program that allows you to search easily. I used Ultra Edit...search for "Actual Music CRC:" and all you have to do is to click 'find next' etc. Since the CRC consists of 4 chars only it's pretty easy to find out if the CRC matches.

It's a matter of minutes...I'd be interested in your results...because I think that this error is not related to certain files, tags or anything. I always follow the same procedure, and I can't see any pattern. Some Albums don't have a single dud while others have 1 or 2...most Albums aren't affected at all and I chose files from the last 6 month...it seems totally random...
Title: mp3Gain not always able to undo TrackGain
Post by: 2Bdecided on 2005-05-23 10:50:38
You say it's totally random - are those albums louder or quieter than others? Or have louder or quieter parts?

Or may have been, I don't know, played in a different program from the others?

As you probably know - these things are never random - it's just that sometimes it's difficult to find the real cause.

Cheers,
David.
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-05-24 17:26:23
Quote
You say it's totally random - are those albums louder or quieter than others? Or have louder or quieter parts?

Or may have been, I don't know, played in a different program from the others?

As you probably know - these things are never random - it's just that sometimes it's difficult to find the real cause.

Cheers,
David.
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=299704")

no, not that I know off...I don't see any pattern at all. The best would be if somoene could try what I've posted here: [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=33296&view=findpost&p=294147]http://www.hydrogenaudio.org/forums/index....ndpost&p=294147[/url]
Title: mp3Gain not always able to undo TrackGain
Post by: Snelg on 2005-05-25 03:43:12
Haven't been ignoring you, Jojo. My daughter was born just after I made my last post, so I've been otherwise occupied for a while.

I just ran tests on about 450 files from various sources. No problems.

Not sure what to tell you at this point.

-Glen
Title: mp3Gain not always able to undo TrackGain
Post by: Jojo on 2005-07-25 23:38:48
Quote
As you probably know - these things are never random - it's just that sometimes it's difficult to find the real cause.
[a href="index.php?act=findpost&pid=299704"][{POST_SNAPBACK}][/a]

that's what kept bothering me the entire time, because I just couldn't track down the cause of the problem. Do you wanna know why? Because it was random! One of my RAM bars was screwed up, that's what caused the randomness and it also explains why that rather large file I was talking about was often faulty.

It all makes sense now and I'd like to thank you for your patience