Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: foo_verifier inconsistencies with FLAC files (Read 1332 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

foo_verifier inconsistencies with FLAC files

Hello folks,

I am running into an issue with foo_verifier where it is telling me FLAC files are corrupted with an MD5 mismatch and reported lengths as inaccurate.

To dig deeper, I checked the files with FLAC itself (flac -t file.flac). The files returned ok.. weird.

Thinking perhaps that flac -t was only performing CRC and not checking the md5, I decoded the file out to wav with flac -d and checked the MD5 of the wav file. The MD5 matches the MD5 in STREAMINFO of the original flac (checked with metaflac --show-md5sum file.flac).

Knowing for sure now that the audio is fine (based on MD5 checking), I started to look at the other metadata blocks. The file contains STREAMINFO, SEEKTABLE, VORBIS_COMMENT and PICTURE blocks, with a PADDING block at the end. Nothing looked out of place there either.

I decided to try removing the PADDING block (metaflac --remove --block-type=PADDING --dont-use-padding file.flac). I checked it with foo_verifier.. STATUS OK, no warnings. File verifies all of a sudden now that PADDING is removed?

Now I tried with the original file again, this time trying "Optimize file layout + minimize file size" from foobar itself. The file still fails to verify in foo_verifier but now, even worse, the file fails to verify with flac itself (0:FLAC__STREAM_DECODER_ERROR_STATUS_LOST_SYNC). The file is now officially corrupted and can't be decoded,

I did a few more experiments with metaflac and foobar but I think it only adds more confusion.

Does anyone know what is going on here?

Versions:

foobar 1.5.4
foo_verifier 1.3
flac 1.3.2
metaflac 1.3.2

Edit:

If I simply add 1 byte of padding with metaflac --add-padding=1 the file verifies fine in foo_verifier. The file size only increases by 5 bytes, as expected due to 4 bytes for the metadata block header. Something seems strange with the way foobar or foo_verifier is handling the PADDING.

Re: foo_verifier inconsistencies with FLAC files

Reply #1 –
Can you also verify your file with Audiotester?

Re: foo_verifier inconsistencies with FLAC files

Reply #2 –
I'd sure like to get my hands on such sample. I'll share it with Peter if I can confirm it's a foobar decoder issue.

Re: foo_verifier inconsistencies with FLAC files

Reply #3 –
Over night I stripped padding from a lot of my library and I don't have a copy of the original file I was testing above (thought I had a backup but I don't). I have found a file that has similar issues.

Passes with flac -t. Matches MD5 in STREAMINFO when manually decoded to WAV. Fails foo_verifier.

Also tried in Audiotester with this file and I get random fails each time I run it:

(FRAME_CRC_MISMATCH @ 0m 12s)
(LOST_SYNC @ 0m 56s)
(LOST_SYNC @ 0m 11s)
(LOST_SYNC @ 0m 25s)

LOST_SYNC seems more common but certainly there is no instance where the Audiotester passed in my tests.

However, if we strip the PADDING or simply add 1 byte of PADDING using metaflac, the file passes both foo_verifier AND Audiotester.

Re: foo_verifier inconsistencies with FLAC files

Reply #4 –
I've uploaded a "corrupted" file to reproduce the above. The password on the archive is foo_verifier and the link expires in 7 days.

[link removed]

Let me know when the file is in the right hands so I can remove the link.

Re: foo_verifier inconsistencies with FLAC files

Reply #5 –
Thanks for the bug report.
I've got the file, will investigate shortly.
Microsoft Windows: We can't script here, this is bat country.

Re: foo_verifier inconsistencies with FLAC files

Reply #6 –
flac -t says that this file is corrupted too

Re: foo_verifier inconsistencies with FLAC files

Reply #7 –
Shame you lost the original troublemaker. But I can't replicate what you described with this file. The flac.exe -t test tells the file is corrupted, removing padding with "metaflac --remove --block-type=PADDING --dont-use-padding <file.flac>" doesn't fix it and forcing the track to be decoded to WAV does not produce PCM data with the same checksum as is stored in the header's md5 field.

Re: foo_verifier inconsistencies with FLAC files

Reply #8 –
Apologies for the mix up, I seem to have zipped the file after confirming that foobar corrupts it.

I have uploaded the file again and confirmed it does test OK with flac -t and does fail verify with foo_verifier.

This time I did not test anything that might modify the file before uploading it.

Hopefully you can reproduce the findings on your end.

Password is foo_verifier
https://we.tl/t-EE5c9gn7WW

Re: foo_verifier inconsistencies with FLAC files

Reply #9 –
FWIW, the file linked to above tested bad for me in flac.exe -t, "error code 0:FLAC__STREAM_DECODER_ERROR_STATUS_LOST_SYNC".  In Audiotester, it fails with "(LOST_SYNC @ 0m 19s)."

BTW I was not asked for any password during the download or for the 7-Zip extraction.

Re: foo_verifier inconsistencies with FLAC files

Reply #10 –
The second file is bit-identical to the first one.

Re: foo_verifier inconsistencies with FLAC files

Reply #11 –
Wow.. OK thought I was going crazy. I was able to verify this behavior by checking the file, getting a pass, zipping the file and then unzipping it and getting a fail.

Then I took the file which was passing, made it read-only, copied it to a different location (without zipping it) and the new copy was failing flac -t, while the original file still passes.

I believe what I may have found here is a file integrity bug with StableBit Drive Pool. I can't replicate this on my boot drive, which is not using StableBit.

Very sorry for the confusion and appreciate all the patience, I was seriously not expecting to run into a filesystem bug. I will follow up with the StableBit team.
 

Re: foo_verifier inconsistencies with FLAC files

Reply #12 –
In the very rare chance someone else runs into similar issues, the conditions for this are:

StableBit DrivePool + Pool File Duplication + Real-time duplication + Read striping = corrupt files under certain conditions. This is regardless of whether or not you have enabled the "Verify after copy" setting of file duplication. Definitely concerning, but seems nothing to do with FLAC/foobar and only happened to show up for me because my Music folder is one of the only folders I duplicate across multiple drives (you would think that would result in less chance for losing files, not more).

As to why editing the metadata with metaflac seems to "fix" the files, I can only assume some filesystem magic that is way over my head is happening here.

Edit:

I've created a topic over in the DrivePool forums:
https://community.covecube.com/index.php?/topic/5414-pool-file-duplication-causing-file-corruption-under-certain-circumstances/

Edit:

After additional digging, I found others experiencing similar problems with the same setup as me:
https://www.reddit.com/r/DataHoarder/comments/h0qfw3/psa_stablebit_drivepool_readstriping_affects/

Thanks again your patience, all.