Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: How does flac -f work concerning re-encoding and tags transfer? (Read 4884 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

How does flac -f work concerning re-encoding and tags transfer?

Thanks to an other thread I discovered the option -f for flac.exe

I did some testing by putting a few flac files in a test folder and executed on Windows
Code: [Select]
C:\test\flac.exe -f -8 *.flac
and it worked. Even all the populated tags were copied, or should I say tags maintained? I'm not sure how the re-encoding works (new file or overwriting source file?). Are always all tags transferred to the new flac file?

Thanks

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #1
When using flac -f FILE.flac, flac decodes the file you provided and encodes it to FILE.tmp,fl-ac+en'c, copying metadata. When flac is finished writing to FILE.tmp,fl-ac+en'c without encountering any errors, it deletes FILE.flac and renames FILE.tmp,fl-ac+en'c to FILE.flac.

Note that if you care for your audio data you should keep a backup.  As an extra assurance you can use the verify option -V to protect against encoder errors, but this does not catch all possible errors. There is a slim chance the new FLAC file isn't playable, for example because the file was written to a bad sector of your hard disk.
Music: sounds arranged such that they construct feelings.

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #2
Thank you for the excellent explanation.

I have several backups  ;) I guess the risk of writing to bad sectors of an hard disk sector is the same as for all kind of files.

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #3
I guess the risk of writing to bad sectors of an hard disk sector is the same as for all kind of files.
WavPack checks against that, FLAC does not: https://hydrogenaud.io/index.php?topic=121714.25
Still there are scenarios WavPack cannot protect against, for example if Windows' write caching (that is a setting you can avoid using) or https://en.wikipedia.org/wiki/Disk_buffer go wrong: the application will "see" a file written to disk, while in reality it hasn't made it it past buffer purgatory (or, a segment of it) and is still vulnerable to outages.

Easiest to verify - that doesn't need you to connect your backup - is if you use foobar2000 with https://foobar.hyv.fi/?view=foo_audiomd5 . Make a test first so you are sure you have stored the PCM checksum rather than the encoded audio.  Run it on the files to be updated; it will then write an AUDIOMD5 tag. Run the update, (reboot if you are paranoid), force foobar2000 to re-index the files, and check.
(Why the tag over the built-in MD5? Because the latter is written anew, the tag is transferred.)

Or if that is too late: Easiest for CD rips that are in AccurateRip is likely to reboot and then run a CUETools verification afterwards. It your files verify against someone else's rip, then ah-marvellous.

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #4
I always thought there should be some form of CTDB/ARDB, but for official WEB releases.
When there is a download we have no reference of what the MD5s or CRCs “should” be for a release. So how do we protect ourselves from data corruption or loss in the future? We can’t verify or repair if we don’t know what the data “should” be in the first place.

Would be nice to have something to compare our downloaded files to like we can currently do for anything ripped from CD

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #5
When there is a download we have no reference of what the MD5s or CRCs “should” be for a release.
When you purchase a FLAC download, it has an MD5. Not necessarily, but I would be surprised if any online store sells FLAC without MD5.
Run a verify when you have downloaded it.

Also, fine detail coming up: If flac -f detects actual corruption it will not overwrite - but it does not consider MD5 mismatch to be actual corruption, so if that is the only issue, it will overwrite (and with the freshly calculated MD5).
(Why this, you may ask? Speculation: once upon a time, some encoder would skip MD5 calculation in the interest of time (fair enough) and write bogus MD5 sum (err...).)

Secure ripping, AccurateRip and the CUETools database are for CDs because they are nowhere as safe as checksummed files.

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #6
Speculation: once upon a time, some encoder would skip MD5 calculation in the interest of time (fair enough) and write bogus MD5 sum (err...).)
There indeed was bug like this in ffmpeg's FLAC encoder - https://hydrogenaud.io/index.php?topic=111537.0 , https://trac.ffmpeg.org/ticket/4628

But, if FLAC stream itself is actually corrupted, this can be detected even if file has no md5.

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #7
When there is a download we have no reference of what the MD5s or CRCs “should” be for a release.
When you purchase a FLAC download, it has an MD5. Not necessarily, but I would be surprised if any online store sells FLAC without MD5.
Run a verify when you have downloaded it.

Also, fine detail coming up: If flac -f detects actual corruption it will not overwrite - but it does not consider MD5 mismatch to be actual corruption, so if that is the only issue, it will overwrite (and with the freshly calculated MD5).
(Why this, you may ask? Speculation: once upon a time, some encoder would skip MD5 calculation in the interest of time (fair enough) and write bogus MD5 sum (err...).)

Secure ripping, AccurateRip and the CUETools database are for CDs because they are nowhere as safe as checksummed files.


I believe Qobuz is the only provider that doesn’t have MD5 set in the STREAMINFO block

To me the issue is what happens over time if anyones data is to become corrupt/damage and what means do they have to repair it.

With CTDB/ARDB the correct values for CD sourced rips are known and if damaged, can (usually) be repaired.

With WEB releases, there is no known value or database to reference. You can detect errors but do little to accurately repair them or check that the files you have match what everyone else has. And as releases are often routinely removed from services and unable to be repurchased/redownload, one is left with no choice but to seek out alternative sources to get another copy… and then who knows if THOSE files are any good!

I certainly don’t have all the answers or the knowledge/resources to put such a thing together, but I have long thought a centralized database for WEB releases similar to CTDB would be a very welcomed idea… but if I’m the only one who thinks so, perhaps the problem is only with my thinking :)

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #8
I believe Qobuz is the only provider that doesn’t have MD5 set in the STREAMINFO block
Thx for the update. Made one attempt at Qobuz once, and they didn't want my credit card. (Wrong country.)
For Qobuz purchases, you can then
* use foo_audiomd5 to tag it with PCM MD5 sum,
and/or
* re-encode at once, and then bit-compare to the purchase.

Also note that quite some download purchases in 44.1/16 are the same as the CD and might verify with AccurateRip - at some offset or another.

To me the issue is what happens over time if anyones data is to become corrupt/damage and what means do they have to repair it.
"Silent data rot" and "errors upon re-writing" are overlapping topics but not completely the same ...

Actually, if you are worried about errors upon re-writing, then there is an issue about FLAC (and MP4 including ALAC, and MP3 with ID3v2.3): they have tags at the beginning of the files, and a too large tag update will force a rewrite of the entire file. APEv2 tags are at the end, so a tag update affects only the tags section. WavPack is arguably the best supported lossless format using APEv2.  APEv2 can also be used in MP3, but then you are in compatibility issues quite soon. (AFAIUnderstand, ID3v2.4 can be at either end?!)

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #9
Speaking of data rot, I somehow almost lost an album rip track to whatever happened during a random data migration. The error was not on the disk, or caused by random RAM bit flip. It was a whole block or more of audio broken, and caused a consistent 9215 or so sample difference against CTDB. I was, however, able to use CUETools to repair the track.

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #10
When I was a complete noob I saw my Windows95 PC showed RAM error during POST, which I did not really understand the severity and ignored it. Then I defragged my C drive on that PC, after this Windows was unable to boot.

There was also a time I saw Windows95 nagging me with kernel32.dll errors, then I deleted kernel32.dll in DOS mode and thought doing so could fix the error. :P

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #11
I bought bad RAM some fifteen years ago and within a couple days I noticed errors appearing and disappearing in checksummed files. A complete RAM test takes time, and I've not owned a computer that did it on every boot. A RAM error is pretty much the only way that data corruption can go unreported. Any kind of disk or network transfer would detect an error and lose at least an entire sector. There is a reissue of a game called Transport Giant, where the data file contains contains a few corrupted bits, leading to red buildings showing up.

WEB releases from CD more often than not have the pauses between tracks trimmed off, and wouldn't verify against databases because of different track lengths.

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #12
WEB releases from CD more often than not have the pauses between tracks trimmed off
Sounds strange why they would bother. I guess that the releases are not literally "from CD", but rather from the files used to create the CD. (Those files need not be sector-aligned - or, will a typical DAW default to doing so when exporting to a "final" format?)

As for data loss ... some time, Windows XP (or was it 2K?) had the idea that if a disk was NTFS, then it would have to be internal - meaning, write-caching enabled. Disaster on a mobo that would sometimes drop-and-reconnect its USB connection.

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #13
I assume j7n there meant the lack of pregap pauses. I don't know if that's a common problem, but I can imagine it happening (either from a bad rip or from original files).
The lack of 75fps alignment can also result in different files, yes. A DAW will not align tracks that way, unless you explicitly select CD/DDP as the format.

 

Re: How does flac -f work concerning re-encoding and tags transfer?

Reply #14
The lack of 75fps alignment can also result in different files, yes. A DAW will not align tracks that way, unless you explicitly select CD/DDP as the format.
Maybe even if you do. I own at least one "gapless" CD that has some small gaps at track boundaries that were clearly introduced by splitting the audio with the wrong alignment. (I haven't tried splicing it back together to see if the gaps are audible, but they're at least not noticeable - I only spotted it when analyzing the audio for other reasons.)