If I have a library of nicely tagged flac files and I need or want to encode them into that format, say apple to play on my iphone, i can easily do that, but if I want to encode to flac again using a 'better' or newer flac encoder can I do that?
Thanks
dpr
There is really no gain in re-encoding it to FLAC again since the files is already lossless. What you can maybe gain is a little smaller file size but that's it imo.
Perhaps, but my question is simple. Can it re-encode?
Not without decoding first. Which is lossless.
Thanks. So use of flac and its tagging system is future proof.
As safe as it goes, as long as you start with a standard 16-bit or 24-bit PCM file - like CD or lossless digital store download. (Unless it is DSD or floating-point that the FLAC format doesn't support - if you have that, use WavPack.)
If you use the reference encoder, the command
flac --force <whatever_options_and_compression_level_you_have_patience_for> -- flacfilename.flac
will do the decoding and re-encoding and transfer tags.
It's a great practice. Provided one has compatible GPU (http://cue.tools/wiki/FLACCL#Supported_hardware), they should always (re-)encode flac with latest version of FLACCL (http://cue.tools/wiki/FLACCL), which is the flac encoder with highest compression ratio at the moment (correct me if there's a better one).
Some time ago I re-encoded my flac library with FLACCL set to highest compression parameter via foobar2000. Given that many files were encoded with flac 1.2.1 probably set to avg mode, this has cut down on several GBs worth of space. Not only that, it exposed which files were corrupted so I could re-download and replace them. Now I automatically re-encode all flac I download with FLACCL for these two reasons. I just wish foobar had convert-and-replace option such that xrecode has, right now it's a little finicky because you have to delete original files and rename the converted ones. No biggie though, just a few keystrokes if you know how to set them up.
I often think that if music download stores and streaming services that offer flac (beatport, junodownload, bandcamp, tidal, deezer, qobuz etc.) caught up to this, it would probably allow them to save thousands of dollars in bandwidth costs monthly, if not more.
Corrupted FLACs ... you can find them by foo_verifier. I would in any case do that
before trying to re-compress, in case I set parameters to overwrite corrupted files without warning - although the reference encoder yells at me if I try.
which is the flac encoder with highest compression ratio at the moment (correct me if there's a better one).
FLACCL obtains good results
fast - I have no idea whether compression ratio depends on display card, does it?
I did some testing with a new beta and proposed a couple of settings here (https://hydrogenaud.io/index.php?topic=120158.msg1004446#msg1004446). Feel free to compare size to FLACCL and report, I am curious. On my not-very-new computer, FLACCL gets me similar compression levels at 3/4 of the time.
Unless you don't use Windows.
Corrupted FLACs ... you can find them by foo_verifier.
Sure you can, but it's done during re-encoding anyway so why do it twice.
I did some testing with a new beta and proposed a couple of settings here (https://hydrogenaud.io/index.php?topic=120158.msg1004446#msg1004446). Feel free to compare size to FLACCL and report, I am curious.
So was I! I ran some tests (https://hydrogenaud.io/index.php?topic=120158.msg1004631#msg1004631) and they were largely inconclusive: flaccl lost in compression but by a minuscule margin, and won in encoding speed significantly.
FLACCL obtains good results fast - I have no idea whether compression ratio depends on display card, does it?
Beats me. But here's a file you can test and see if you get different results.
Unless you don't use Windows.
Virtual machines are pretty useful I hear.
You realise flacCL -11 does not produce standard subset files? You have to compare flac -8 results directly to flacCL -8. Even CUETools flake from the same guy that did flacCL compresses better at -8 then.
You realise flacCL -11 does not produce standard subset files? You have to compare flac -8 results directly to flacCL -8. Even CUETools flake from the same guy that did flacCL compresses better at -8 then.
Technically I don't have to do anything. My personal curiosity lies in the best compression results for flac format.
Technically I don't have to do anything. My personal curiosity lies in the best compression results for flac format.
Subset (https://xiph.org/flac/format.html#subset) is a stricter format that should be streamable, and the reference encoder sticks to that unless you deliberately instruct it to do otherwise with a special parameter that makes it more ... uh, fool proof.
A "flac decoder" need not be able to stream a file generated by flaccl -11 file nor need it be able to stream a
flac.exe --lax -m -b 32768 -r 15 -l 32 -A "partial_tukey(31)" -p -e generated file. In the reference encoder, non-subset is deliberately left to the geex who know what they are doing and are willing to give the commands themselves.
My personal curiosity lies in the best compression results for flac format.
Well here you got something to test your curiosity ... and your patience. This is worse than what in an older version of flac called --super-secret-totally-impractical-compression-level .
There is not really any limit to how slow compression can be - in the reference encoding, such a limit is just self-imposed.
Unless you don't use Windows.
Virtual machines are pretty useful I hear.
Pretty swell when you want to limit a program to using half the host's CPU cores at most, and have the overhead of a second OS running on those cores. Unfortunately, a fine number of random encoders are broken even under Wine. But why not target exclusively Windows? Most people only use that anyway.
I recompress everything I download, as it helps to remove padding from album artwork, check for errors/corrupt files, unify filenaming schemes, and shrink file sizes even if only a little bit. My older personal CD rips were done at “only” -8, my newer cd rips are done with more technically advanced compression settings since my CPU can now handle it. I think I’d make some decent space savings if I recompressed all my old rips, and it shouldn’t take too long in theory, but I haven’t attempted it.
With current hard drive space re encoding flac isn't good. At worse, you
can even suffer corruption and data loss through sofware or HW bugs.
I suggest keep old flacs for a while just in case if it must be done.
With current hard drive space re encoding flac isn't good. At worse, you
can even suffer corruption and data loss through sofware or HW bugs.
I suggest keep old flacs for a while just in case if it must be done.
Thankfully with FLAC everything is secure. If there is ever an error it is immediately known. Unlike with ALAC
FLAC does not verify by default. With a the verify option, FLAC (and WavPack) should be expected to be safe, as they write a temporary file and only in the end delete and rename. However I have had corruption recompressing FLAC using some front-end version: it would halt at some short fragment and nevertheless delete the old file. (https://hydrogenaud.io/index.php?topic=99803.msg921798#msg921798)
I could reproduce trouble then. I tried again this year with a full hard drive and only the command-line FLAC, and I could not reproduce.
FLAC does not verify by default. With a the verify option, FLAC (and WavPack) should be expected to be safe, as they write a temporary file and only in the end delete and rename.
The FLAC verify option only verifies the encoding process, not the process of writing the file to disk. That would be incredibly hard to implement anyway, as it is rather hard to check for an user-space process whether a file is written to disk and when reading is directly from disk and not from cache. I guess even OS internals can't be sure whether something comes from disk or disk cache. Because of that, even with verify options, overwriting FLAC files to recompress them should, in my opinion, not happen without having a backup. I myself backed up my collection of FLAC files to Blu-ray disks (and let FLAC check them after burning!) before recompressing.
I presume the same holds for WavPack. If not, it must use some quite complicated process to make absolutely sure it is verifying from disk and not from cache, which I think isn't even possible.
If you don't want to make a backup, recompress to a different directory tree, shutdown and restart computer (to make sure caches are emptied) and then check the new files. If all checks out, remove the old files.
I definitely have backup both on- and off site, but still it was an eye-opener. I mean, flac.exe does object when it cannot write to file. It is beyond me whether flac.exe will try to move-with-overwrite or rather try to (1) delete the old and (2) rename the tmp file - or if that choice even matters.
I am not so much into the Unix and Windows (sys)internals as to know how fool-proof https://docs.microsoft.com/en-us/sysinternals/downloads/sync is, but it is part of my daily soul & cache cleansing ritual.
You can convert flac to alac (in m4a container) and back no problem. No loss no nothing. thats the beauty of lossless codecs.
The issue we're referring to here is the potential loss of the new file being written with data errors to disk, either due to bit flipping, or drive failure. And then you'd be deleting the original, leaving you with just the broken new file.
And there's no easy way to invalidate the system file cache to make sure that the new copy is 100% written correctly and reads back correctly as well.
(2) rename the tmp file - or if that choice even matters.
It does rename the tmp file but it doesn't really matter: flac.exe writes a file disk and receives an OK (this travels from flac.exe -> OS -> disk firmware and back) but writes on hard disks are not verified, it is just assumed they are OK. I've had a few times already writing to an SD card silently fails this way.
(Important thing here to anyone who thinks error-protected lossless formats are "safe" against everything:
* if you don't have a backup, go out buy another drive yesterday.
* and still, surely error-protected formats are much better than alternatives like ALAC-in-MP4 (and TTA (https://hydrogenaud.io/index.php?topic=122094), if that is even in use by anyone but ...? (https://hydrogenaud.io/index.php?topic=122094.msg1007721#msg1007721)).
Don't let those essentials be disturbed by my digging into nitty-gritties!)
(2) rename the tmp file - or if that choice even matters.
It does rename the tmp file but it doesn't really matter: flac.exe writes a file disk and receives an OK (this travels from flac.exe -> OS -> disk firmware and back) but writes on hard disks are not verified, it is just assumed they are OK. I've had a few times already writing to an SD card silently fails this way.
Doesn't it matter ... sometimes? It doesn't in the scenario where the file was written to a finalized corrupted file, but what if the write operation isn't completed, say, still in a write cache? Wouldn't the "move this file" fail if it isn't completed yet (and that is desirable, leaving the old one in) - while "delete this other file" could happily be performed even if the new one is still in a write cache?
Still it is interesting to know whether codecs originally developed for the Windows platform does any more sanity-checking of the written file on (an NTFS) drive.
@bryant, what about WavPack? And
@TBeck , even if Tak/Takc cannot (AFAIK) re-compress .tak files in-place?
Wouldn't the "move this file" fail if it isn't completed yet (and that is desirable, leaving the old one in) - while "delete this other file" could happily be performed even if the new one is still in a write cache?
On Linux platforms, depending on the filesystem used, one can rename or even move a file (given that it is moved on the same filesystem, not to another partition or disk) while it is still being written to. Heck, you can even delete a file that is still being read from or written to, full deletion will happen as soon as the process reading/writing from/to it closes the file.
I'd say the move-with-overwrite would be worse, because it involves two write operations, hence twice as many bits written and twice as many room for errors.
Still it is interesting to know whether codecs originally developed for the Windows platform does any more sanity-checking of the written file on (an NTFS) drive. @bryant, what about WavPack? And @TBeck , even if Tak/Takc cannot (AFAIK) re-compress .tak files in-place?
For WavPack’s verify mode, the output file (which has a temporary name if the original is being overwritten) is closed, reopened, and reread in its entirety. Only if that passes the md5sum check (for lossless) is it renamed to the correct name (which deletes the original).
I think this is about as safe as you can easily get in a portable way. Yes, in some cases the entire file would be in cache (either in the OS or the drive), but there’s little chance that the file might not actually be being written at all (or be truncated).
On Linux this uses rename() which is reasonably safe (it does the rename and remove atomically and guarantees that if the operation fails, the old version will still be there). I wasn’t able to find anything equivalent on Windows that I could use, so it’s a separate remove and rename, which in theory leaves open the possibility of the original being deleted and the temp file not renamed (although it would still be there, and the name is obvious rather than gobbledygook).
As was mentioned, FLAC simply decodes the stream on its way out without even pretending to reread the file. This has the advantage that it works even when writing to pipes, and I seem to remember documentation somewhere stating that the purpose of this mode was intended more to verify the integrity of the algorithm on the given processor than the file integrity itself.
Coming back to this thread after I started looking into this again. Is there any way to re-compress a file in Foobar while retaining the original filename? (And only deleting the original after a verification of course.)
Coming back to this thread after I started looking into this again. Is there any way to re-compress a file in Foobar while retaining the original filename? (And only deleting the original after a verification of course.)
There is no way to do this in the source folder since both files would be named filename.flac . You'd need to do this in a seperate folder. %filename% in the name format will give you the source filename. If you are interested i have a foobar formatting script that will copy the whole relative source directory structure up to a specified root folder.
Coming back to this thread after I started looking into this again. Is there any way to re-compress a file in Foobar while retaining the original filename? (And only deleting the original after a verification of course.)
There is no way to do this in the source folder since both files would be named filename.flac . You'd need to do this in a seperate folder. %filename% in the name format will give you the source filename. If you are interested i have a foobar formatting script that will copy the whole relative source directory structure up to a specified root folder.
That could totally be a workable solution, I’d love to hear more about it! If the conversion creates new folders somewhere with the same directory structure, and I just batch delete the original FLAC files after conversion, and then merge the new folders with the old, the new files should theoretically end up where they are supposed to.
Coming back to this thread after I started looking into this again. Is there any way to re-compress a file in Foobar while retaining the original filename? (And only deleting the original after a verification of course.)
There is no way to do this in the source folder since both files would be named filename.flac . You'd need to do this in a seperate folder. %filename% in the name format will give you the source filename. If you are interested i have a foobar formatting script that will copy the whole relative source directory structure up to a specified root folder.
That could totally be a workable solution, I’d love to hear more about it! If the conversion creates new folders somewhere with the same directory structure, and I just batch delete the original FLAC files after conversion, and then merge the new folders with the old, the new files should theoretically end up where they are supposed to.
$puts(nameroot,Music) $puts(newpath,$directory(%path%,13)\$directory(%path%,12)\$directory(%path%,11)\$directory(%path%,10)\$directory(%path%,9)\$directory(%path%,8)\$directory(%path%,7)\$directory(%path%,6)\$directory(%path%,5)\$directory(%path%,4)\$directory(%path%,3)\$directory(%path%,2)\$directory(%path%,1)) $puts(posroot,$sub($strstr($get(newpath),$get(nameroot)),1)) $right($get(newpath),$sub($len($get(newpath)),$get(posroot)))\%filename%
Yea :| Dont look at it too long it ugly af. i did my best. In case your root folder is NOT "Music" you have to change that to what ever root you want to copy. You can watch how the resulting files will look like with the converter window.
Coming back to this thread after I started looking into this again. Is there any way to re-compress a file in Foobar while retaining the original filename? (And only deleting the original after a verification of course.)
What about the following:
* Keep a source files playlist.
* Naming pattern for the conversion:
directory_path(%path%)\%filename% (
@Kartoffelbrei : I think you are looking for $directory_path(%path%)?)
This will replace the ":" after the drive letter by "-". Say, you want to convert from a directory tree under L: to some new structure on the same drive: Then selecting "L:" as target folder and the above naming will copy things under L:\Music to L:\L-\Music. Make sure that there is no "L:\L-" already then, as that will be populated.
You need enough space for both source and target. Make sure that under "When done" you not only transfer everything you want tansferred, but also "Show full status report".
Upon done, you can select all the new files in the popup window, and get them into a new playlist.
* If you want to ensure that no "delayed write" or anything is fooling you, cf. ktf's reply 18, then reboot.
* Now you have a source playlist and a target playlist. Verify that they have the same number of elements (that takes a few seconds to do, could save you from having to wait a day for the obvious conclusion), dump them into the same same playlist, sort by path (get a path column if you don't already have that in your display!) - and run a foo_bitcompare. If everything matches, then highlight the source and File Operations -> Delete.
* Then you can move the first directory under "+" up to "L:\" and Windows will ask if you want to merge, and ... voilà - except, that you might get into trouble with say file/folder names that start or end with whitespace. Alternatively you can use foobar2000 and delete the appropriate characters from the $directory_path
This assumes that you never have simultaneously selected a filename.flac and a filename.wv from the same directory (with same filename for both) - that would make a conflict when both target files would be say filename.whateversuffix
Now, if you have only FLAC and/or WavPack files, and you want to recompress FLAC to FLAC and WavPack to WavPack, then you can do something else with foobar2000, assuming source and target is on same drive L:
* Instead of using foobar2000 to convert,
link to the L:\L- structure. (Either shift-rightclick File Operations -> Link, or select copy and choose the "Link" radio button). Linking will create an additional name for the same file. Deleting what appears to be a "new file" will only delete the new name-link; the file is lost only when the last filename is deleted.
* Now you have what looks to be a structure copied, but they are not copied - they don't take more space. Yet.
* As flac.exe and wavpack.exe can recompress in-place, you can traverse the "L-" hierarchy that way. The exe will create new files and delete the "newly generated filename" - taking more space!
* Dump into foobar2000, bitcompare and delete. But before doing that: check for temp files!
* Alternatively, after linking: (1) reboot, (2) make sure you have as many FLAC in "L-" as in source, (3) use flac.exe to recompress
source (leaving the "L-" as backup"), (4) check against temp files (actually, during (3) it might be a good idea to close fb2k or not have it monitor the folders, as that can prevent other applications from deleting and renaming files ...) (5) bitcompare, (6) delete the "L-".
You asked over at https://hydrogenaud.io/index.php/topic,122750.0.html too, but some more considerations that are not so foobar2000-specific:
* You have a backup, of course? You might run flac -f on the working set and foo_bitcompare against the backup afterwards.
* Before you do any recompression, verify your flacs. Recompressing will "write valid files with corrupted audio", and the information that there was ever an error will be lost from the .flac files. foo_verifier, or drag and drop into audiotester.exe found at http://www.vuplayer.com/other.php
* You might want to wait for a new FLAC version that seems to improve speed ... or compression for given speed.
* In that thread you were asking for maximum FLAC compression. Actually, you can have FLAC running for ages to squeeze out just a little bit more, and you don't want to do that, it isn't worth it. If you want heavier compression, then change codec. If you go to samefilenameasflac.tak, it is easy to run a loop that will copy each album.cue into a album.tak.cue and then you can batch edit all the album.tak.cue. (TAK isn't open source, but it has an open source decoder. If you want free software ... WavPack.)
* CUETools isn't that dependent upon filenames in .cue files being correct. But it does depend upon everything being CDDA.
* You might want to ask yourself if it all is worth the hassle (and the risk) - storage is not that expensive anymore. But if your drive is nearly full ... sure it might be a point.
Thanks all for the tips! I will continue experimenting with this a bit though I am starting to reconsider the immediacy with which I plan to carry this out across my entire library.
I tested recompression with a single FLAC file I have that I know is corrupt. Running the foobar verifier threw an error message as expected, but so did attempting to re-compress the file, and nothing was lost in the process.
Since the verification process seems to take nearly as long as just complete recompression, and gives the same error result when appropriate, I am not sure I see the utility in running a verification first.
I did a batch test by doing a set of files added in my library within a certain date range, and the total size of the selection prior to recompression was 238GB.
Recompressing this set of files took somewhere between 1-2 hours (I walked away from the PC and foobar doesn’t give a readout of the total elapsed time once complete, so I don’t know exactly how long it took for certain) and when it was done the new set of files was only 228GB.
I know hard drive space is cheap these days - and I’m not at a particular loss for space either - but 10GB is 10GB. That’s nothing to sneeze at! If nothing else, it feels better to me to not have drive space wasted on “hot air”, and managing this data in the future will itself be more efficient once compressed more efficiently. ie. Drive migrations, backups and other transfers of this data will not take as long now that it is 10GB smaller than it previously was.
Since this was also only a 238GB selection out of my music library, I estimate a full recompression could net me as much as 100GB of reclaimed space. Not insignificant by any means.
With my current hardware I think such a recompression effort could be carried out across the full library within a 24 hour timespan.
At some point in the future I may move everything to a BTRFS based NAS. I’ll probably wait to recompress everything as part of that migration process, otherwise I am in no particular rush but I would like to do this at some point going forward.
10 GB of 238 is in the 4 percent ballpark, and that is
quite a lot: In ktf's latest lossless test - CDDA at http://www.audiograaf.nl/losslesstest/revision%205/Average%20of%20all%20CDDA%20sources.pdf - it amounts to around the difference between FLAC and TAK. (Part of it could potentially be tags padding. If you have removed a big picture from tags, fb2k will by default not reclaim the "saved" space as that would trigger a full file rewrite.)
If you migrate to NAS, then the transfer speed is often throttling down so much that you can consider even slower FLAC options than -8.
-8p and -8e take about equal time, and it seems to me that the "p" gives more savings on CDDA material, the "e" gives more savings on high resolution material; -8ep is awfully slow.
But expect the next (?) official FLAC release to improve. ktf has teased improvements here.
I tested recompression with a single FLAC file I have that I know is corrupt. Running the foobar verifier threw an error message as expected, but so did attempting to re-compress the file, and nothing was lost in the process.
I think that depends on the tool. Be cautious.
Since the verification process seems to take nearly as long as just complete recompression
Ah, the joys of spinning drives :-)
and gives the same error result when appropriate, I am not sure I see the utility in running a verification first.
If you have a second set (commonly known as "backup"), then what about (1) recompress, (2) bitcompare to backup, (3) if there is anything lost (say information about corruption), copy from backup to working drive?
10 GB of 238 is in the 4 percent ballpark, and that is quite a lot: In ktf's latest lossless test - CDDA at http://www.audiograaf.nl/losslesstest/revision%205/Average%20of%20all%20CDDA%20sources.pdf - it amounts to around the difference between FLAC and TAK. (Part of it could potentially be tags padding. If you have removed a big picture from tags, fb2k will by default not reclaim the "saved" space as that would trigger a full file rewrite.)
If you migrate to NAS, then the transfer speed is often throttling down so much that you can consider even slower FLAC options than -8.
-8p and -8e take about equal time, and it seems to me that the "p" gives more savings on CDDA material, the "e" gives more savings on high resolution material; -8ep is awfully slow.
But expect the next (?) official FLAC release to improve. ktf has teased improvements here.
I tested recompression with a single FLAC file I have that I know is corrupt. Running the foobar verifier threw an error message as expected, but so did attempting to re-compress the file, and nothing was lost in the process.
I think that depends on the tool. Be cautious.
Since the verification process seems to take nearly as long as just complete recompression
Ah, the joys of spinning drives :-)
and gives the same error result when appropriate, I am not sure I see the utility in running a verification first.
If you have a second set (commonly known as "backup"), then what about (1) recompress, (2) bitcompare to backup, (3) if there is anything lost (say information about corruption), copy from backup to working drive?
My apologies for not being able to address some of these points in a more organized form by quoting it in parts (formatting when posting from mobile is not the easiest!) but I will try my best…
• I did strip any embedded album art tags before running the converter, knowing that recompressing would then remove the leftover padding… so that did account for SOME of that 10GB savings. But I’m not sure just how much of that can be attributed to the total 10GB savings as the image tags definitely weren’t present in every file (typically only attached to stuff from download providers) and they tend to be pretty small anyhow.
Even if I made a crazy assumption that half of that space was reclaimed just from removing image tags… 5GB is still quite a bit of shrinkage!
• in my string of commands, -8 -e -p were all separately present (I’m not sure if the -8ep you mentioned is a more “proper” way of using those 3 commands together or if that was just your way of referring to them being used as I did - my apologies if that’s a stupid question!) but as far as I am aware (again, please correct me if mistaken) using -8 -e -p should provide the highest compression, while still being fully compliant, at the expense of overall processing time/intensity.
• I did notice that in the process of running the standalone verification against the first set of files, the disks were pegged at 100% usage, which is what I attributed the overall slow speed to. However it seemed that running the actual compression - in which verification would be a part of that process as well - disk usage was very low, maybe hovering around 5-15%?
This is interesting to me since the compression involves reading AND writing to the same disk (or disks, as they were from a RAID array) so I figured there would be heavier disk activity there, not less.
Maybe it has something to do with the rate at which data is actually being read from the disk within those processes. I am thinking the compressor simply reads less data in at a time and that keeps things manageable, whereas the verifier tries to read the data all at once causing the disk usage to go haywire. Just a guess though.
I can experiment with the settings... I’m wondering if lowering the simultaneous thread count might put less stress on the disks when it’s reading the files? In theory, less threads might result in overall faster verification if the disks don’t get hit so hard. In which case, I’d be less hesitant to run extra verification checks!
-8 -e -p is synonymous to -8ep is synonymous to -p8e is synonymous to -ep8 etc. Also the "-f" for force overwrite can be merged into there.
It is possible to get smaller files by stacking up with windowing functions (FLAC will then run through them all).
Say, take a test file of a minute music or so, and give the command
flac -8f -A "partial_tukey(1);partial_tukey(2);punchout_tukey(3);welch;partial_tukey(2/0/999e-3);punchout_tukey(3/0/8e-2);partial_tukey(9);punchout_tukey(9)" testfile.flac
... which tests like 30 functions instead of standard -8's six. With either -e or -p (but not both!) it is still faster than -8ep. But if you throw both -e and -p into this ...
Read the output file sizes and see, it is hardly worth it.
Then you can try instead a git build (warning: should not be considered production ready, have backup!) posted at https://hydrogenaud.io/index.php/topic,122179.msg1013309.html#msg1013309 (post 119).
It is a bit slower than 1.3.4, but you will easily find files where this new -8 outdoes what that monster-long thing spends ... here it is 50 times as much time on.
How? This is (I think!) due to some parameters calculated with double precision, which makes for better accuracy (and smaller residual hence smaller file) but all those calculations take a bit longer time. Now the monster-long -8pef -A line with this build is not that much more time-consuming than with the old build.
You still got your 228 GB to experiment with? ;-)
-8 -e -p is synonymous to -8ep is synonymous to -p8e is synonymous to -ep8 etc. Also the "-f" for force overwrite can be merged into there.
It is possible to get smaller files by stacking up with windowing functions (FLAC will then run through them all).
Say, take a test file of a minute music or so, and give the command
flac -8f -A "partial_tukey(1);partial_tukey(2);punchout_tukey(3);welch;partial_tukey(2/0/999e-3);punchout_tukey(3/0/8e-2);partial_tukey(9);punchout_tukey(9)" testfile.flac
... which tests like 30 functions instead of standard -8's six. With either -e or -p (but not both!) it is still faster than -8ep. But if you throw both -e and -p into this ...
Read the output file sizes and see, it is hardly worth it.
Then you can try instead a git build (warning: should not be considered production ready, have backup!) posted at https://hydrogenaud.io/index.php/topic,122179.msg1013309.html#msg1013309 (post 119).
It is a bit slower than 1.3.4, but you will easily find files where this new -8 outdoes what that monster-long thing spends ... here it is 50 times as much time on.
How? This is (I think!) due to some parameters calculated with double precision, which makes for better accuracy (and smaller residual hence smaller file) but all those calculations take a bit longer time. Now the monster-long -8pef -A line with this build is not that much more time-consuming than with the old build.
You still got your 228 GB to experiment with? ;-)
I see, thank you for the clarification!
Still have those files of course, not to mention PLENTY more where they came from… total size of the FLAC library is about 2.5TB, and growing all the time…
I should mention I also commonly install the new git builds posted here by NetRanger (I like to live on the edge) so the version I just used in this short test is probably closer to an eventual 1.4.0 than it is to 1.3.4 and should include these new compression improvements
On my new PC build (AMD Ryzen 5950x) compressing a file with -8 -e -p takes about as long as it took my old PC to compress a file at a regular old -8. So right off the bat I already use -8 -e -p as a standard compression routine in EAC. It’s already done compressing by the time the CD is ripped, so the time spent is all the same either way.
I did occasionally use -8 -e -p on on the old PC, but it did take an ungodly amount of time… and I let it run anyway, not fun waiting though! Thankfully those days are behind me, but maybe I’ll try those new commands you posted and see what it’s like to re-live that experience…
If you use bleeding-edge git builds for recompressing your working set, and an older flac in foobar2000, then foo_bitcompare will verify that an official build decodes them to what they are supposed to be. (This is "better" than using -V, which will decode using the same executable, and if code is recycled between encoding and decoding, errors might be mirrored. I have never seen that happen though, but it is an easy fix that is kinda free.)
If you are really adamant about size, you can go for a different codec than FLAC. WavPack -hx4m ("m" for MD5) is exciting, but I don't know if it can still beat the new FLAC builds using -8pe -A tonsofthem. (You can specify up to 31 functions, but partial_tukey(n) and punchout_tukey(n) count as n each.) For high resolution material, ktf's most recent test (http://www.audiograaf.nl/losslesstest/revision%205/Average%20of%20hi-res%20sources%20up%20to%20192kHz.pdf) measures quite some difference (CDDA here (http://www.audiograaf.nl/losslesstest/revision%205/Average%20of%20all%20CDDA%20sources.pdf)). But it depends on the kind of music as well.
WavPack is FOSS and has third-party implementations. If you want to sacrifice the FOSS principle, ffmpeg has FOSS decoders for TAK (including multichannel) and Monkey's (stereo only). Beyond that, you are in frog territory.
For me it’s FLAC or bust. Extreme compression isn’t the ultimate priority enough to drive me towards another format, just more a side goal of trying to have a well maintained FLAC library in its most optimized form. To me, no more different than trying to make sure tags are correct, folders are well organized, etc.
I kinda married FLAC myself, and at a time when storage was more expensive. Slightly unfaithful when WavPack serves special purposes. I like WavPack (and I'm impressed at TAK) - but not enough to migrate.
Oh, and ...
I did occasionally use -8 -e -p on on the old PC, but it did take an ungodly amount of time…
I'll raise you ungodly amount of time: https://hydrogenaud.io/index.php/topic,120158.msg1001334.html#msg1001334
13.6 GB of (high resolution) PCM. Just for the hell of it to see FLAC kinda "maxing out", I gave a command where it took ... five days or so, to compress three hours and a half of 96 (mostly) / 24, while I was away from the computer.
I think it was done sequentially on one core, and the CPU isn't exactly new.