Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Looking for MP3 bitstream checking utility (Read 8451 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Looking for MP3 bitstream checking utility

For a long time ago i had some problems with my hdd. (LBA48 problem)
Now I am looking for a good tool to search my entire MP3 music collection for corrupted files.
Why "-p" (add CRCs) swich is not on by default in lame???  MP3Utility and MP3Check does their job, but they are using only sync/ headers/crc check. More deep (and small) errors, that are less in size than a MP3 frame and does not cross the frame boundaries, unfortunately could not be detected by this tools.

Althought the lame decoder prints "bitstream problem" "couldn't rewind the stream by 30 frames" warning messages on some MP3 files what are "OK" according to above tools. Some MP3s have ugly artifacts, some not. Hence it must be a way to detect even this type of errors, because it corrupts the frame structure in some way.

The tool could just blindly run "lame --decode" recursively and decode to null (time is not a concern) and create a log for broken files.

I have some coding skills, so i can try to modify sources of some tool that does similar job. But i havent any time for this right now...

Any suggestions would be very appreciated

Sorry for possibly bad english

Looking for MP3 bitstream checking utility

Reply #1
Quote
Why "-p" (add CRCs) swich is not on by default in lame???
[a href="index.php?act=findpost&pid=279418"][{POST_SNAPBACK}][/a]

because it degrades sound quality (CBR, ABR) / or leads to bigger files (VBR)...
--alt-presets are there for a reason! These other switches DO NOT work better than it, trust me on this.
LAME + Joint Stereo doesn't destroy 'Stereo'

 

Looking for MP3 bitstream checking utility

Reply #2
Quote
because it degrades sound quality (CBR, ABR) / or leads to bigger files (VBR)...[a href="index.php?act=findpost&pid=279556"][{POST_SNAPBACK}][/a]
...since it uses bits for checksum data that would otherwise have been used for audio data.

Just to make it clear.

Looking for MP3 bitstream checking utility

Reply #3
Quote
Quote
because it degrades sound quality (CBR, ABR) / or leads to bigger files (VBR)...[a href="index.php?act=findpost&pid=279556"][{POST_SNAPBACK}][/a]
...since it uses bits for checksum data that would otherwise have been used for audio data.

Just to make it clear.
[a href="index.php?act=findpost&pid=279615"][{POST_SNAPBACK}][/a]


thx, that's really clearing things, thought it was additional info...

Looking for MP3 bitstream checking utility

Reply #4
16bit CRC consumes 2bytes per frame
MP3 frame header consumes 4bytes

The loss of space to store CRCs for each frame is less than for mp3 headers, and comparable to slack space used by average mp3 file

for a 60GB music collection CRCs will consume only a part of CD!

So it should not hurt quality a lot.

--
Probably i'll use lame sources as starting point...

Looking for MP3 bitstream checking utility

Reply #5
Quote
Quote
Why "-p" (add CRCs) swich is not on by default in lame???[a href="index.php?act=findpost&pid=279418"][{POST_SNAPBACK}][/a]
because it degrades sound quality (CBR, ABR) / or leads to bigger files (VBR)...[a href="index.php?act=findpost&pid=279556"][{POST_SNAPBACK}][/a]

this is an interesting thought imho regarding the 'archival quality' idea(l)
if it doesn't degrade quality for vbr but makes files a tiny bit bigger only, of course

Looking for MP3 bitstream checking utility

Reply #6
Quote
Quote
Why "-p" (add CRCs) swich is not on by default in lame???
[a href="index.php?act=findpost&pid=279418"][{POST_SNAPBACK}][/a]

because it degrades sound quality (CBR, ABR) / or leads to bigger files (VBR)...
[a href="index.php?act=findpost&pid=279556"][{POST_SNAPBACK}][/a]


It technically degrades the quality of VBR files as well (whether its audible or not is probably not known).

Looking for MP3 bitstream checking utility

Reply #7
Quote
16bit CRC consumes 2bytes per frame
MP3 frame header consumes 4bytes
[a href="index.php?act=findpost&pid=279721"][{POST_SNAPBACK}][/a]


Interesting, if that is true the overhead for a 128kbs mp3 would be less than half a percent of the total data.

On the other hand, it might be quite easy to sanity check mp3s bitstream without any CRCs.

Maybe the attention to detail used to tune lames presets so finely, just didnt want to let those bytes go.
no conscience > no custom

Looking for MP3 bitstream checking utility

Reply #8
Quote
It technically degrades the quality of VBR files as well (whether its audible or not is probably not known).[a href="index.php?act=findpost&pid=280211"][{POST_SNAPBACK}][/a]
Quote
Quote
16bit CRC consumes 2bytes per frame
MP3 frame header consumes 4bytes[a href="index.php?act=findpost&pid=279721"][{POST_SNAPBACK}][/a]
Interesting, if that is true the overhead for a 128kbs mp3 would be less than half a percent of the total data. [...]
Maybe the attention to detail used to tune lames presets so finely, just didnt want to let those bytes go.[a href="index.php?act=findpost&pid=280215"][{POST_SNAPBACK}][/a]

Of what kind of possible quality degradement do we speak here: Is it in the area of --preset vs. --fast-preset ?
Could any of those codec gods enlighten us about the way its implemented into lame? Would ABXing make sense or would that need a new specially designed preset level? The latter sounds to me like a lot of work on this level of sophistication.

Whatever the result is  (wether a new tool (multiplatform please!  ) or a defacement of a myth in the best case - we want to get rid of them, right?) I'm dead sure it will help quality of compressed audio in case of reliability. External CRC32/MD5 is "effete" in the age of ID3(V2)/APE and iTunes/Windows Media Player/etc. which modify all files as soon as they get their filthy hands on it.

edit:typos,clarification
p.s. could any mod move this into mp3-general/-tech where it belongs? thank you

Looking for MP3 bitstream checking utility

Reply #9
* It theoretically could have some abx-able difference with CBR (because frame data size is lower) on hard-to-encode passages...
* I doubt that it could be abx-able with VBR, since encoder could just use higher bitrate. But on VERY HARD to encode (320kbps frames).
* I don't know if lame have bugs (or not optimized for) frame sizes 2bytes shorter. In that case differences may be audible.

Quote
External CRC32/MD5 is "effete" in the age of...

CRC16 is not as secure as MD5, and unfortunately could not replace it. But if the quality degradation from CRC16 is neglible (less than from two sequential lame versions) then why not to use it? Since it if frame-based, one can easilly know even in what frame an error occured.

And MD5 could perfectly expand the secureness.

Looking for MP3 bitstream checking utility

Reply #10
Quote
* It theoretically could have some abx-able difference with CBR (because frame data size is lower) on hard-to-encode passages...
* I doubt that it could be abx-able with VBR, since encoder could just use higher bitrate. But on VERY HARD to encode (320kbps frames).
[a href="index.php?act=findpost&pid=280383"][{POST_SNAPBACK}][/a]


I doubt it is ABXable with VBR or CBR, but with VBR, the encoder could use a higher bitrate, but often doesn't. Its not as smart as you might think. That is to say--using CRC seems to affect CBR and VBR modes equally... Try encoding some samples and comparing them (in something like EncSpot) to see what I mean...

Looking for MP3 bitstream checking utility

Reply #11
Quote
And MD5 could perfectly expand the secureness.[{POST_SNAPBACK}][/a]
By "could" you mean as a tag like Replaygain is implemented into the more recent builds?

Oh, please take this as a feature request - I hope some dev listens ...and likes the idea as much as we "noobs" do.

...
Actually, i remember there has been a related approach (CRC32 checksums which should be sufficient for files <64mb) in the past by some sCeNe dUdE but his/hers implementation was hackish - it used the lyrics tags for that matter - plus it had problems with id3v2 as far as i remember. Of course it was Win32 only, and the webpage went down since aswell.

I do think there are legit interests in file verification that have nothing to do with the leet bathtube... Well i do, and i think everyone else creating their personal archive has, too. Some native solution for this would, er, kick ass (imho).

edit:readability

edit2: <errata> obviously that (irony ahead) shiny and brand new feature is already there. it's just not supported that well. i'm a redunant bugger sometimes. see [a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=21150]http://www.hydrogenaudio.org/forums/index....showtopic=21150[/url] and http://www.hydrogenaudio.org/forums/index....showtopic=28853 for reference/hints. i'd absolutely welcome some multiplatform cli for that (and do wonder if the linux port of encspot does cope with it). Kudos to the devs. I wonder why nobody who has contributed to this thread has noted this yet... Too obvious? Or was it all about md5?

Looking for MP3 bitstream checking utility

Reply #12
Quote
Quote
Quote
Why "-p" (add CRCs) swich is not on by default in lame???
[a href="index.php?act=findpost&pid=279418"][{POST_SNAPBACK}][/a]

because it degrades sound quality (CBR, ABR) / or leads to bigger files (VBR)...
[a href="index.php?act=findpost&pid=279556"][{POST_SNAPBACK}][/a]


It technically degrades the quality of VBR files as well (whether its audible or not is probably not known).
[a href="index.php?act=findpost&pid=280211"][{POST_SNAPBACK}][/a]

yep, but only for everything below --preset standard
--alt-presets are there for a reason! These other switches DO NOT work better than it, trust me on this.
LAME + Joint Stereo doesn't destroy 'Stereo'

Looking for MP3 bitstream checking utility

Reply #13
Quote
Quote
Quote
Quote
Why "-p" (add CRCs) swich is not on by default in lame???
[a href="index.php?act=findpost&pid=279418"][{POST_SNAPBACK}][/a]

because it degrades sound quality (CBR, ABR) / or leads to bigger files (VBR)...
[a href="index.php?act=findpost&pid=279556"][{POST_SNAPBACK}][/a]


It technically degrades the quality of VBR files as well (whether its audible or not is probably not known).
[a href="index.php?act=findpost&pid=280211"][{POST_SNAPBACK}][/a]

yep, but only for everything below --preset standard
[a href="index.php?act=findpost&pid=280693"][{POST_SNAPBACK}][/a]


How do you figure?

Looking for MP3 bitstream checking utility

Reply #14
Quote
Quote
Quote
Quote
Quote
Why "-p" (add CRCs) swich is not on by default in lame???
[a href="index.php?act=findpost&pid=279418"][{POST_SNAPBACK}][/a]

because it degrades sound quality (CBR, ABR) / or leads to bigger files (VBR)...
[a href="index.php?act=findpost&pid=279556"][{POST_SNAPBACK}][/a]


It technically degrades the quality of VBR files as well (whether its audible or not is probably not known).
[a href="index.php?act=findpost&pid=280211"][{POST_SNAPBACK}][/a]

yep, but only for everything below --preset standard
[a href="index.php?act=findpost&pid=280693"][{POST_SNAPBACK}][/a]


How do you figure?
[a href="index.php?act=findpost&pid=280743"][{POST_SNAPBACK}][/a]


--preset standard's target is to be transparent and therefore shouldn't have any restriction in bitrate. Therefore, the file will just be bigger. It's like if you don't use Joint-Stereo, the file will be bigger, but the quality will be the same. However, I think there's some bitrate restriction for -V4 for instance...but I could be wrong...
--alt-presets are there for a reason! These other switches DO NOT work better than it, trust me on this.
LAME + Joint Stereo doesn't destroy 'Stereo'

Looking for MP3 bitstream checking utility

Reply #15
Quote
--preset standard's target is to be transparent and therefore shouldn't have any restriction in bitrate. Therefore, the file will just be bigger. It's like if you don't use Joint-Stereo, the file will be bigger, but the quality will be the same. However, I think there's some bitrate restriction for -V4 for instance...but I could be wrong...
[a href="index.php?act=findpost&pid=280812"][{POST_SNAPBACK}][/a]

Well, the mp3 format has a bitrate restriction - 320kbps. This is one of the reasons why not using joint-stereo may harm the quality. Using some of this limited storage for a CRC may also cause damage to the quality...

Looking for MP3 bitstream checking utility

Reply #16
Quote
Quote
Quote
Quote
Quote
Quote
Why "-p" (add CRCs) swich is not on by default in lame???
[a href="index.php?act=findpost&pid=279418"][{POST_SNAPBACK}][/a]

because it degrades sound quality (CBR, ABR) / or leads to bigger files (VBR)...
[a href="index.php?act=findpost&pid=279556"][{POST_SNAPBACK}][/a]


It technically degrades the quality of VBR files as well (whether its audible or not is probably not known).
[a href="index.php?act=findpost&pid=280211"][{POST_SNAPBACK}][/a]

yep, but only for everything below --preset standard
[a href="index.php?act=findpost&pid=280693"][{POST_SNAPBACK}][/a]


How do you figure?
[a href="index.php?act=findpost&pid=280743"][{POST_SNAPBACK}][/a]


--preset standard's target is to be transparent and therefore shouldn't have any restriction in bitrate. Therefore, the file will just be bigger. It's like if you don't use Joint-Stereo, the file will be bigger, but the quality will be the same. However, I think there's some bitrate restriction for -V4 for instance...but I could be wrong...
[a href="index.php?act=findpost&pid=280812"][{POST_SNAPBACK}][/a]

That doesn't make much sense from a programming point of view. You'll find that increases in file sizes using CRC are not exactly (2 bytes)*(number of frames), which would be true if what you're saying was true. Also, no joint-stereo actually reduces quality like ancl said...

EDIT: added a missing word