Hello. I recently updated to foobar v2.0 beta 12, and now it is telling me that all of the FLACs I encoded around a certain time in 2007 are corrupt. Older versions didn't say this, and it is literally all of the files from this time and not just one, so I don't believe my files have suddenly become corrupt. All of the music plays correctly and the error only pops up at the very end of playback.
Here an example message:
File verification error: Corrupted FLAC stream
"F:\music\Anime\Tenchi Muyo!\Tenchi Muyo! English Soundtrack\10 - Tenchi Muyo! Ryo-Ohki's Theme.flac"
The file tag Tool says this:
reference libFLAC 1.2.0 20070715
All files were ripped with Exact Audio Copy - likely whichever version was newest in 2007.
If need be I can zip my foobar folder and an example FLAC, though I'd rather not send a FLAC since it's all copyrighted music.
Lastly, if it does turn out that these files have been corrupt this whole time, is there an easy way to re-encode them with a new version of FLAC while keeping them in the same place?
Thank you.
Thanks for the bug report.
It would make things a lot simpler for me if you could send a sample of this to me: peter [at] hydrogenaud.io
New FLAC has become more zealous about errors yes.
Hunch, since you used EAC: ID3 at the end? See the bottom part of this: https://hydrogenaud.io/index.php/topic,123219.0.html
Or if not ... the top part.
If it is ID3 at the end, you cannot extract a part of the audio to check for errors, because the issue would be after the audio. You can open the .flac in a text editor and check the very end of it, if you can make sense out of that. Although I use Notepad++ all the time, EditPad Lite (https://www.editpadlite.com/portable.html) is much faster at bigg files.
Or if you want to just re-encode and see what happens: Try the command-line
flac -F troublesomefile.flac -o newfile.flac
Then dump both files into foobar2000, bitcompare and see that they match, open properties and see that MD5 are the same.
Thanks for the bug report.
It would make things a lot simpler for me if you could send a sample of this to me: peter [at] hydrogenaud.io
I sent a different sample. It's a short but full track I ripped at the same time as all the others. It produces the same error message.
Thanks.
Yes, it is ID3 tags. Removal of last 128 bytes makes the error message go away. No actual corruption of FLAC payload.
I'll change foobar2000 behavior to just say what it really is instead of raising a false alarm.
@Cannonaire and everyone who read my "-F" suggestion, that is not needed and -F incurs an unnecessary risk:
Or if you want to just re-encode and see what happens: Try the command-line
Don't use -F. Not unless you absolutely have to use it, and here there is no need - it turns out that FLAC will not abort on this kind of error. 1.4 (and 1.3) will recompress them and remove the ID3.
I'll change foobar2000 behavior to just say what it really is instead of raising a false alarm.
If you can make foobar2000 identify it, then maybe remove them too - since this probably is going to happen to a lot of EAC rips.
Suggestion: For the "Rewrite file tags" action, possibly with a checkbox under Advanced, or with a popup "these files have ID3, remove that as well?" or whatever.
(Yeah, easy for me to say when it is someone else's workload.)
I was thinking about removal with 'rewrite file tags', but it's not THAT easy to be 100% sure that 'TAG' at EOF-128 is an ID3 tag not some part of the file. We've had that happen with some other format already.
Would have to figure out where the legitimate EOF of FLAC stream is and look for ID3 tag from there. Maybe for some future fb2k update, not right now.
After updating to Beta 13 I'm still getting the FLAC messages. Confirmed in 32-bit version with the sample I sent you.
It is perhaps pertinent that I did an in-place update from beta 12 using the portable install (all previous installs were also portable).
Tested with a fresh install, and there was no message. I have no idea what of the myriad possible things could be causing this.
I was thinking about removal with 'rewrite file tags', but it's not THAT easy to be 100% sure that 'TAG' at EOF-128 is an ID3 tag not some part of the file. We've had that happen with some other format already.
Would have to figure out where the legitimate EOF of FLAC stream is and look for ID3 tag from there. Maybe for some future fb2k update, not right now.
Good points, unfortunately ... luckily FLAC [usually] has MD5, so at least you got a pretty good indication if (1) "TAG" at appropriate position AND (2) MD5 of all-but-128 matches AND (3) MD5 including the 128 fails.[lobbyist's note]
... I suppose that you have full control over how to avoid passing these data to a re-encoding attempt that by way of --ignore-chunk-sizes could result in those being treated as audio? If no such checks, that would be much worse.
[note] That seems to be another "rewrite stream" akin to what you are doing for MP3s? In which case, lobbying for a pet of mine:
"Optimize file layout + minimize file size" without padding is not a good thing for front-tagged formats - better would be to reclaim padding space with an "Optimize file layout & size" where for front-tagged formats the padding [to be adjusted in Advanced] be rounded up to file size = integer multiple of the standard 4096 file cluster size. Space already wasted.
Please pardon my inexperience, but I don't know anything about how tags work really. Is there something I need to do to make sure my files are correct? And is there any way to do that in an automated way? I have probably over a thousand tracks that will have this error.
Additionally, I'm still getting the error on foobar 2.0 beta 13 with the in-place update. Is there a setting I need to change in order to make the messages stop? If this is unintended behavior, I can zip my install folder if you need it for finding the bug. I am using Portable install.
Thank you!
EDIT
In addition to the previous error message, sometimes I get this instead:
File verification warning: Garbage at the end of file (ID3 tag?)
What you can do:
* Make sure your backups are up-to-date. Seriously, make sure your backups are up-to-date.
* Recompress using the reference flac encoder/decoder.
* Dump backup and recompressed into foobar2000 and bitcompare.
* Keep the backup in case something weird happens later.
* Recompress using the reference flac encoder/decoder.
Is there an easy way to do this for all of them in-place? It would take me weeks to move/rename everything then move it all back where it belongs after recompression.
Yes, flac recompresses in-place using the "-f" switch. And you can traverse directories with a FOR loop.
I suggested a workflow in this topic from this week: https://hydrogenaud.io/index.php/topic,123362 , replies 2 and 6.
If you have already backed up to a different drive, and are willing to run a recompress-in-place-with-overwrite, then you can do something "simpler" than the FOR loop in #6.
Can you handle a command-line?
I haven't had a chance to try yet. I hurt my neck yesterday.
I've been comfortable with command line in the past, but I don't know how Windows syntax works. I'll look into it after I update my at-home backup. No chance I can update my offsite, unfortunately.
The FOR syntax in the good (?) old CMD prompt would like the following:
FOR /r %f IN (*.flac) DO flac -fV -8 "%f"
... under a few assumptions:
* You have to be at the parent folder. Say that is M:\music .
Open a command window, that could lead you to c:\windows\system32 . M: <ret> sends you to M. cd music sends you to music.
(If you don't have any other .flac under M:, you don't have to go to the music directory, you can call it from M:)
* Your default user has to have write access to all the files. If you have them owned by some other user to prevent overwriting ... it will prevent overwriting.
* You need to have the flac executable in your PATH variable. Sounds weird? Instead of messing with the system variable, copy to the same directory (say M: or M:\music) a flac.exe that doesn't need the .dll. Case's compile or Wombat's compile don't need the dll.
* Options used in the flac command: You need the "-f" to overwrite. I used "-8" here assuming you want to encode with -8. If you want heavier compression ... well, options are there. If you are happy with the default, leave out the "-8". The "V" (capital V!") verifies afterwards and is "kinda optional": yeah you want to verify, but flac cannot tell if you have e.g. drive caching, so you should rather bit-compare to your backup afterwards, and if you do that, you don't need the V.
And I assumed that your FLAC files end with .flac and not .fla , but the Windows shell isn't case-sensitive, so .FLAC is OK too.
* None of your filenames are "too long". That is, longer than Windows' usual path character limits. FLAC will omit those which are too long, and you can find them later.
So that command will traverse the subdirectories for files ending with .flac and apply the command. What FLAC then will do to a filename.flac, is to create a temporary filename.flac.tmp,fl-ac+en'c and when done, it will delete filename.flac and rename the temp file. If it fails at renaming the temp file, it will leave it with that ugly name. If your .flac file count (say as reported by foobar2000) is not the same afterwards, that is hopefully the reason. You can search for any such afterwards.
Warnings:
1) don't do this on a hard drive that is so full that it cannot handle a handful of extra files. Have some GB left. (Reason: if the temp files are left, then it will spend the rest, and behaviour at full drive is ... you might get too much to clean up afterwards.)
2) don't have fb2k running and monitoring the folder as you go, because that can prevent flac from deleting the old files and moving the temp files.
3) If you want a .bat file, you have to replace the single percents with %%. The mysterious Gate[way]s.
Geeh I thought I was going to write a nineliner here ...
Open a command window, that could lead you to c:\windows\system32 . M: <ret> sends you to M. cd music sends you to music.
...or, in Windows Explorer (My Computer), navigate to the parent folder then shift+right_click and select "open command prompt here".
Yes, but some Windows versions don't offer that. Or they offer only PowerShell.
Actually, PowerShell can be opened, and then give the command
cmd
Also, it is possible to select all folders and drag-and-drop them into FLAC frontend (https://flacfrontend.sourceforge.net/). All FLAC files from subfolders will also be added into frontend automatically.
When I try to re-encode the files, flac spits out this error:
ERROR: input file F:\music\Anime\Nadesico\Martian Successor Nadesico TV OST\10 - I Want To Be Your Number One.flac has an ID3v2 tag
What should I do?
What FLAC version? (Command: flac --version )
Care to share the file (or a small one with the same issue)? Filebin or gofile and PM me a link.
I used official FLAC versions for Windows from xiph.org. I tried both the newest version and the oldest version available with a windows executable. I'm guessing I should use an alternate compile? Any recommendations? I might be able to upload something later but I can't at the moment. Also, it seems when I ran the batch file I created it re-encoded every FLAC on my drive (as intended) except the ones that needed it. Big facepalm moment.
I used official FLAC versions for Windows from xiph.org.
Apologies if you already knew this, but the latest 1.4.x versions from Xiph require the presence of the libFLAC.dll (included in the download package) in the same directory as flac.exe for things to work. This is not a requirement for the flac.exe from Case distributed in his Free Encoder Pack.
I used official FLAC versions for Windows from xiph.org.
Apologies if you already knew this, but the latest 1.4.x versions from Xiph require the presence of the libFLAC.dll (included in the download package) in the same directory as flac.exe for things to work. This is not a requirement for the flac.exe from Case distributed in his Free Encoder Pack.
I was aware of the DLL requirement. I put the FLAC win64 folder in my PATH (I've had to do this previously for other encoders). It ran correctly, but it just gave me an error and wouldn't accept the files with ID3 tags. Thanks! Do you think the encoder from Case might accept the files with ID3 tags?
It seems when I ran the batch file I created it re-encoded every FLAC on my drive (as intended) except the ones that needed it
Didn't you forget -F (not to be confused with -f)?
It seems when I ran the batch file I created it re-encoded every FLAC on my drive (as intended) except the ones that needed it
Didn't you forget -F (not to be confused with -f)?
I don't really know the commands. I just copied the command line from Porcus and made sure to follow their instructions. I thought I wasn't supposed to use -F. What does -F do? I really don't know what I'm doing, which is why I have everything backed up on disconnected drives.
This is the batch file I made:
f:
FOR /r %%f IN (*.flac) DO flac -fV -5 "%%f"
pause
It properly re-encoded every FLAC file on the drive
except the ones with errors that actually needed to be re-encoded.
Dang, was I wrong the second time about -F?
Apologies. Big facepalm moment yes.
-f and -F in brief:
Small "f" forces overwrite, so you need to do that on re-encode.
Big "F" decodes through errors. Even if there is an error, it will take the audio it got.
"Together they are dangerous" in that the new file will look OK even if the old audio was corrupted. So only use -F when you have to.
(Looks like you have to here. I was wrong. Apologies again.)
But you are using foobar2000, so - because you have ran a re-encode - here is what you can do if there are only a few troublemakers ("a few" meaning, so few it pays off to single out only them and you have space enough on the drive):
(1) You can search up the ones that were not re-encoded. If you used 1.4 to reencode, the query
%__tool% HAS flac AND NOT %__tool% HAS 1.4.
will select those which remain. Those are precisely the ones that need re-encoding, and now you can see how many GB they are!
(2) Dump these into a playlist. Save that playlist.
(3) Copy them to a new folder (say, "f:\reencode" - this requires you to have enough space on the drive) using a pattern like e.g.
%directoryname%\%filename%
This presumes that you have them all at the same "directory depth", and it requires you to have enough space on the drive.
(The problem about using "%path%" as naming is that all files will be .flac.flac - I actually often use that still, and rename afterwards.)
(4) cd to f:\reencode and run the same command. Now it will traverse only that directory.
(5) You can use foo_bitcompare to make sure the audio is the same in the playlist in (2).
No worries! All I lost is a little bit of time, which was mainly spent having a good conversation with my wife, so nothing was lost!
Back to the topic at hand, I would estimate there are over 1000 files that were not re-encoded. I'm more inclined to just run the whole thing again with the big -F, since I have multiple backups anyway and plenty of spare time tonight.
Running it again with this command line.
f:
FOR /r %%f IN (*.flac) DO flac -F -5 -fV "%%f"
pause
We'll see what happens!
Even with -F it refuses the files.
I just tried this on a single dir and this is the error I get:
ERROR: input file F:\music\Anime\Nadesico\Martian Successor Nadesico TV OST\10 - I Want To Be Your Number One.flac has an ID3v2 tag
**EDIT**
It occurs to me that the -F was before the input file in the command line in a previous example, but I don't know how to do that in a FOR loop. Is that my problem? How would I write this? My apologies, I don't know the proper syntax for this.
It occurs to me that the -F was before the input file in the command line in a previous example, but I don't know how to do that in a FOR loop. Is that my problem? How would I write this?
I don't follow what you mean.
FOR /r %%f IN (*.flac) DO flac -F -5 -fV "%%f"
...looks OK to me, the "-F" comes after "flac" and before "%%f" (which is the token representing the input file), so what do you think the problem is?
My apologies, I don't know the proper syntax for this.
- FOR <parameters> IN (<search>) DO <operation> initiates the looping structure, performing the <operation> on all items which match <search>;
- /r causes the search to recurse subdirectories;
- %% is the prefix to a token declaration (within batch files, use only single % on the command line), so in this specific case %%f is declared to represent each item which matches the search – the choice of letter must not conflict with other tokens in use;
- (*.flac) searches for all files with the .flac extension;
- flac <parameters> is the operation to be performed on each item found;
- "%%f%" is the token representing the search item (as previously declared), but is in quotes or double-quotes (it doesn't matter which, but if the %%f string might contain one use the other) to prevent any spaces in the %%f string causing a misinterpretation (because an unquoted space is the end-of-token marker).
Found it!? Maybe. If I did, then good news.
Solution first: Mp3tag - no re-encoding needed. (Edit: found a discussion here as well: https://community.mp3tag.de/t/flac-version-1-4-x-and-tags/59228/1 )
(1) Dump the offending files into Mp3tag (you can do that by querying up in fb2k, mark all and drag them over).
With the "Tag" column activated, you will see FLAC (FLAC ID3etc, like this:
(https://i.imgur.com/OUCDCRa.png)
(2) The following will cut away tags from them all, and then "undo" but in the right way. Cross fingres that you don't get a power outage (well you have backup) while it is doing the work:
* Mark all,
* Ctrl-x
* Ctrl-z
(3) Now the "Tag" column should show "FLAC (FLAC)". And flac -t will report them all just fine.
The obligatory follow-up questions appear to have the following answers:
* Why does Mp3tag work after an "undo"?
Mp3tag does not "undo" by writing back the old files, only by writing back the old tags - and in the appropriate format.
* Why did the -f, -F, -fF not work?
Those should work if there is only ID3v1 at the end. But here the file starts with ID3.
And we should have noticed from your error messages.
Thank you Porcus, and everyone else! MP3Tag did the trick! Turns out there were 1480 tracks with bad tags. Definitely not something I would have been able to do manually.
Porcus, if you still want a sample I can send one to you, but I think the screenshot will probably do.
Here's the MP3Tag screenshot:
Nah, I just needed to realize that EAC also writes ID3v2 tags at the beginning ... I don't need more files now :-)
It also seems to me that fb2kv2 beta 16 removes the leading ID3v2 tags if you ask it to rewrite/update tags.
But it leaves the trailing ID3v1 in, for reasons that Peter outlined in Reply #6.
For the hell of it, I tried to paste an APEv2 tag after the audio as well. Nothing seems to recognize that, but Mp3tag can remove that as well. Here you can see "how it looks" although your browser will probably not show you all the control characters, for that you need more of a text editing application: https://pastebin.com/raw/pssfKdbn
Contents, in order:
* The ID3v2 tag of one track (Bruce Soord, this is copied from what EAC wrote to a rip due to ticking the ID3 check box)
** "fLaC" is supposed to be the beginning of an actual FLAC file. After "reference libFLAC 1.3.2 20221022" (the vendor string, what I used to encode the signal),
** there follows a COMMENT which is the only proper FLAC tag in the file.
** Then some gibberish that is the actual audio (only 44 samples)
* later on the line, APETAGEX starts the APEv2 tag (not the same album as the ID3v2 tag I pasted in earlier - this is from the track you can 'hear' a millisecond of!). The APEv2 tag ends in a second APETAGEX and a few control characters
* Starting with TAG: An ID3v1 tag which again is copied from the EAC ripped FLAC file.
For a compliant FLAC file, only the "**" are supposed to be there!
I was thinking about removal with 'rewrite file tags', but it's not THAT easy to be 100% sure that 'TAG' at EOF-128 is an ID3 tag not some part of the file. We've had that happen with some other format already.
Would have to figure out where the legitimate EOF of FLAC stream is and look for ID3 tag from there. Maybe for some future fb2k update, not right now.
"not right now" is noted, but 1.6.14 and 2.0 beta 17 (x64) also differ in what they read and how.
In the previous reply #29 I let EAC tag a rip to FLAC. With that checkbox thing ticked, it writes a file with ID3v2 first, then stars the fLaC with Vorbis comments, then the audio and then the TAG starts the ID3v1 chunk. This far, not all well but all "as known".
* Dump it into two fb2k instances, one 1.6.14 and one beta 17, x64
Both look identical, also in Properties.
* In 1.6.14, Remove tags from file(s). In both, Reload info from file(s).
What actually happens: Vorbis comments removed, both ID3 tagsets remain.
1.6.14 shows: tag types id3v2.3|id3v1 and under properties, displays the ID3v2.3 tagset
2.0 beta 17 shows: no tag type, and consistently empty under properties.
* In 1.6.14 on this now-altered file, Rewrite file tags.
What actually happens: ID3 tags are still there, but the ID3v2 tags are copied into Vorbis comments.
Both now show it as if it had no ID3v2.3, but it still has.
* In 2.0 beta 17 on this now-altered-again file, Remove tags from file(s).
What happens: It successfully nukes both ID3v2.3 and Vorbis comments. It leaves in ID3v1.
1.6.14 shows: the ID3v1 tags.
2.0 beta 17 shows: nothing, ignoring the ID3.
Trying a bit back and forth it seems that both versions can remove some ID3v2.3 tagsets but not all, and not the same ones.
Recently, I downloaded a file from the Internet, which, when passed through the file integrity verifier in foobar 1.6.14, throws a garbage at the end of file error. Unfortunately, after deleting the tags, it's still the same.
MP3tag doesn't help either.
FLAC 1.3.3 decodes this file without errors. I don't know what's wrong with this file, I guess something other than id3 tags since neither mp3tag nor foobar remove this information and file integrity verifier reports the same error.
Below is a link to the file
https://bit.ly/2lM1tPC
You should try converting the file and then verifying that. Also do a bitcompare against the original to make sure the audio portion is the same.
A quick check reveals that it is encoded with 1.3.1 at -6. Re-encoding with that executable (for Windows x64) and that setting, I get precisely the same file except that the final few bytes are cut off.
Edit:
Using a hex editor, those bytes appear to be
F0 00 FF 0F 44 44 40 48 46 3C 36 0E 55 FF F0
OK, i converted this and bit compare says, that all tracks decoded the same, no differences found. I was just curious what makes this file suspicious by flac higher than 1.4.
What makes flac 1.4.0 raise the flag is that it implemented a become more zealously checking after what-should-be EOF, than 1.3.x.
What those bytes actually are ... does anyone recognize it?