Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: re-encoding flac files with a new encoder (Read 2774 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

re-encoding flac files with a new encoder


If I have a library of nicely tagged flac files and I need or want to encode them into that format, say apple to play on my iphone, i can easily do that, but if I want to encode to flac again using a 'better' or newer flac encoder can I do that?

Thanks
dpr

Re: re-encoding flac files with a new encoder

Reply #1
There is really no gain in re-encoding it to FLAC again since the files is already lossless. What you can maybe gain is a little smaller file size but that's it imo.

Re: re-encoding flac files with a new encoder

Reply #2
Perhaps, but my question is simple. Can it re-encode?

Re: re-encoding flac files with a new encoder

Reply #3
Not without decoding first. Which is lossless.

Re: re-encoding flac files with a new encoder

Reply #4
Thanks. So use of flac and its tagging system is future proof.

Re: re-encoding flac files with a new encoder

Reply #5
As safe as it goes, as long as you start with a standard 16-bit or 24-bit PCM file - like CD or lossless digital store download. (Unless it is DSD or floating-point that the FLAC format doesn't support - if you have that, use WavPack.)

If you use the reference encoder, the command
flac --force <whatever_options_and_compression_level_you_have_patience_for> -- flacfilename.flac
will do the decoding and re-encoding and transfer tags.
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: re-encoding flac files with a new encoder

Reply #6
It's a great practice. Provided one has compatible GPU, they should always (re-)encode flac with latest version of FLACCL, which is the flac encoder with highest compression ratio at the moment (correct me if there's a better one).

Some time ago I re-encoded my flac library with FLACCL set to highest compression parameter via foobar2000. Given that many files were encoded with flac 1.2.1 probably set to avg mode, this has cut down on several GBs worth of space. Not only that, it exposed which files were corrupted so I could re-download and replace them. Now I automatically re-encode all flac I download with FLACCL for these two reasons. I just wish foobar had convert-and-replace option such that xrecode has, right now it's a little finicky because you have to delete original files and rename the converted ones. No biggie though, just a few keystrokes if you know how to set them up.

I often think that if music download stores and streaming services that offer flac (beatport, junodownload, bandcamp, tidal, deezer, qobuz etc.) caught up to this, it would probably allow them to save thousands of dollars in bandwidth costs monthly, if not more.

Re: re-encoding flac files with a new encoder

Reply #7
Corrupted FLACs ... you can find them by foo_verifier. I would in any case do that before trying to re-compress, in case I set parameters to overwrite corrupted files without warning - although the reference encoder yells at me if I try.

which is the flac encoder with highest compression ratio at the moment (correct me if there's a better one).
FLACCL obtains good results fast - I have no idea whether compression ratio depends on display card, does it?
I did some testing with a new beta and proposed a couple of settings here. Feel free to compare size to FLACCL and report, I am curious. On my not-very-new computer, FLACCL gets me similar compression levels at 3/4 of the time.
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: re-encoding flac files with a new encoder

Reply #8
Unless you don't use Windows.

Re: re-encoding flac files with a new encoder

Reply #9
Corrupted FLACs ... you can find them by foo_verifier.
Sure you can, but it's done during re-encoding anyway so why do it twice.

I did some testing with a new beta and proposed a couple of settings here. Feel free to compare size to FLACCL and report, I am curious.
So was I! I ran some tests and they were largely inconclusive: flaccl lost in compression but by a minuscule margin, and won in encoding speed significantly.

FLACCL obtains good results fast - I have no idea whether compression ratio depends on display card, does it? 
Beats me. But here's a file you can test and see if you get different results.

Unless you don't use Windows.
Virtual machines are pretty useful I hear.


Re: re-encoding flac files with a new encoder

Reply #10
You realise flacCL -11 does not produce standard subset files? You have to compare flac -8 results directly to flacCL -8. Even CUETools flake from the same guy that did flacCL compresses better at -8 then.
Is troll-adiposity coming from feederism?
With 24bit music you can listen to silence much louder!

Re: re-encoding flac files with a new encoder

Reply #11
You realise flacCL -11 does not produce standard subset files? You have to compare flac -8 results directly to flacCL -8. Even CUETools flake from the same guy that did flacCL compresses better at -8 then.
Technically I don't have to do anything. My personal curiosity lies in the best compression results for flac format.

Re: re-encoding flac files with a new encoder

Reply #12
Technically I don't have to do anything. My personal curiosity lies in the best compression results for flac format.
Subset is a stricter format that should be streamable, and the reference encoder sticks to that unless you deliberately instruct it to do otherwise with a special parameter that makes it more ... uh, fool proof.

 A "flac decoder" need not be able to stream a file generated by flaccl -11 file nor need it be able to stream a flac.exe --lax -m -b 32768 -r 15 -l 32 -A "partial_tukey(31)" -p -e generated file. In the reference encoder, non-subset is deliberately left to the geex who know what they are doing and are willing to give the commands themselves.

My personal curiosity lies in the best compression results for flac format.
Well here you got something to test your curiosity ... and your patience. This is worse than what in an older version of flac called --super-secret-totally-impractical-compression-level .
There is not really any limit to how slow compression can be - in the reference encoding, such a limit is just self-imposed.
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: re-encoding flac files with a new encoder

Reply #13
Unless you don't use Windows.
Virtual machines are pretty useful I hear.
Pretty swell when you want to limit a program to using half the host's CPU cores at most, and have the overhead of a second OS running on those cores. Unfortunately, a fine number of random encoders are broken even under Wine. But why not target exclusively Windows? Most people only use that anyway.

Re: re-encoding flac files with a new encoder

Reply #14
I recompress everything I download, as it helps to remove padding from album artwork, check for errors/corrupt files, unify filenaming schemes, and shrink file sizes even if only a little bit. My older personal CD rips were done at “only” -8, my newer cd rips are done with more technically advanced compression settings since my CPU can now handle it. I think I’d make some decent space savings if I recompressed all my old rips, and it shouldn’t take too long in theory, but I haven’t attempted it.

Re: re-encoding flac files with a new encoder

Reply #15
With current hard drive space re encoding flac isn't good. At worse, you
can even suffer corruption and data loss through sofware or HW bugs.
I suggest keep old flacs for a while just in case if it must be done.
wavpack -b3.63hhcs.5

Re: re-encoding flac files with a new encoder

Reply #16
With current hard drive space re encoding flac isn't good. At worse, you
can even suffer corruption and data loss through sofware or HW bugs.
I suggest keep old flacs for a while just in case if it must be done.

Thankfully with FLAC everything is secure. If there is ever an error it is immediately known. Unlike with ALAC

Re: re-encoding flac files with a new encoder

Reply #17
FLAC does not verify by default.  With a the verify option, FLAC (and WavPack) should be expected to be safe, as they write a temporary file and only in the end delete and rename. However I have had corruption recompressing FLAC using some front-end version: it would halt at some short fragment and nevertheless delete the old file.

I could reproduce trouble then. I tried again this year with a full hard drive and only the command-line FLAC, and I could not reproduce.
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: re-encoding flac files with a new encoder

Reply #18
FLAC does not verify by default.  With a the verify option, FLAC (and WavPack) should be expected to be safe, as they write a temporary file and only in the end delete and rename.
The FLAC verify option only verifies the encoding process, not the process of writing the file to disk. That would be incredibly hard to implement anyway, as it is rather hard to check for an user-space process whether a file is written to disk and when reading is directly from disk and not from cache. I guess even OS internals can't be sure whether something comes from disk or disk cache. Because of that, even with verify options, overwriting FLAC files to recompress them should, in my opinion, not happen without having a backup. I myself backed up my collection of FLAC files to Blu-ray disks (and let FLAC check them after burning!) before recompressing.

I presume the same holds for WavPack. If not, it must use some quite complicated process to make absolutely sure it is verifying from disk and not from cache, which I think isn't even possible.

If you don't want to make a backup, recompress to a different directory tree, shutdown and restart computer (to make sure caches are emptied) and then check the new files. If all checks out, remove the old files.
Music: sounds arranged such that they construct feelings.

Re: re-encoding flac files with a new encoder

Reply #19
I definitely have backup both on- and off site, but still it was an eye-opener. I mean, flac.exe does object when it cannot write to file. It is beyond me whether flac.exe will try to move-with-overwrite or rather try to (1) delete the old and (2) rename the tmp file - or if that choice even matters.

I am not so much into the Unix and Windows (sys)internals as to know how fool-proof https://docs.microsoft.com/en-us/sysinternals/downloads/sync is, but it is part of my daily soul & cache cleansing ritual.
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: re-encoding flac files with a new encoder

Reply #20
You can convert flac to alac (in m4a container) and back no problem. No loss no nothing. thats the beauty of lossless codecs.

Re: re-encoding flac files with a new encoder

Reply #21
The issue we're referring to here is the potential loss of the new file being written with data errors to disk, either due to bit flipping, or drive failure. And then you'd be deleting the original, leaving you with just the broken new file.

And there's no easy way to invalidate the system file cache to make sure that the new copy is 100% written correctly and reads back correctly as well.

Re: re-encoding flac files with a new encoder

Reply #22
(2) rename the tmp file - or if that choice even matters.
It does rename the tmp file but it doesn't really matter:  flac.exe writes a file disk and receives an OK (this travels from flac.exe -> OS -> disk firmware and back) but writes on hard disks are not verified, it is just assumed they are OK. I've had a few times already writing to an SD card silently fails this way.
Music: sounds arranged such that they construct feelings.

Re: re-encoding flac files with a new encoder

Reply #23
(Important thing here to anyone who thinks error-protected lossless formats are "safe" against everything:
* if you don't have a backup, go out buy another drive yesterday.
* and still, surely error-protected formats are much better than alternatives like ALAC-in-MP4 (and TTA, if that is even in use by anyone but ...?).
Don't let those essentials be disturbed by my digging into nitty-gritties!)

(2) rename the tmp file - or if that choice even matters.
It does rename the tmp file but it doesn't really matter:  flac.exe writes a file disk and receives an OK (this travels from flac.exe -> OS -> disk firmware and back) but writes on hard disks are not verified, it is just assumed they are OK. I've had a few times already writing to an SD card silently fails this way.
Doesn't it matter ... sometimes? It doesn't in the scenario where the file was written to a finalized corrupted file, but what if the write operation isn't completed, say, still in a write cache? Wouldn't the "move this file" fail if it isn't completed yet (and that is desirable, leaving the old one in)  - while "delete this other file" could happily be performed even if the new one is still in a write cache?

Still it is interesting to know whether codecs originally developed for the Windows platform does any more sanity-checking of the written file on (an NTFS) drive. @bryant, what about WavPack? And @TBeck , even if Tak/Takc cannot (AFAIK) re-compress .tak files in-place?
Last two months' worth of foobar2000.org ad revenue has been donated to support war refugees from Ukraine: https://www.foobar2000.org/

Re: re-encoding flac files with a new encoder

Reply #24
Wouldn't the "move this file" fail if it isn't completed yet (and that is desirable, leaving the old one in)  - while "delete this other file" could happily be performed even if the new one is still in a write cache?
On Linux platforms, depending on the filesystem used, one can rename or even move a file (given that it is moved on the same filesystem, not to another partition or disk) while it is still being written to. Heck, you can even delete a file that is still being read from or written to, full deletion will happen as soon as the process reading/writing from/to it closes the file.

I'd say the move-with-overwrite would be worse, because it involves two write operations, hence twice as many bits written and twice as many room for errors.
Music: sounds arranged such that they construct feelings.