Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Mass FLAC re-encode with metadata intact (Read 1296 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Mass FLAC re-encode with metadata intact

Greetings,

I plan to re-encode ALL FLAC files (the latest version, max compression) in my collection (directory recursive), but preserving all metadata (incl. embedded pictures) and files' date/time. Preserving the metadata is the reason why I can't just uncompress everything to wav and compress again using standard tools.

Any clues please?

Re: Mass FLAC re-encode with metadata intact

Reply #1
What tools and what environment would you like to use?

If you are a *nix user like me, you would just use 

Code: [Select]
flac -8f *.flac

in some directory. Or, for recursive use,

Code: [Select]
find . -iname \*.flac -print0 | xargs -0 flac -8f

However, it is a good idea to have a backup if you do this.
Music: sounds arranged such that they construct feelings.

Re: Mass FLAC re-encode with metadata intact

Reply #2
The -f switch recompresses in-place and preserves metadata, yes.
Be sure to close any music player with a monitored library - e.g. foobar2000 (for Windows) may prevent the temporary file from being copied over the old one.

My workflow - on Windows with foobar2000 - would be to
1: make sure backup is updated, including running a binary compare on at least the audio (foobar2000 can do that)
2: file verifier (foobar2000 - until flac 1.4.0 I would also use Audiotester from VUPlayer / James Chapman, but flac 1.4.0 is more zealous about errors and AudioTester isn't updated with that, so ... well then I would use both)
3: take note how many files and samples there are in total,
4: make sure that the player is not running and the backup is disconnected
5: recompress the working set,
6: drive sync and reboot
7: re-index the working set in foobar2000
8: make sure that the file count and sample count matches,
9: same verification procedure
10: bitcompare the audioagainst backup

That kinda does triple work, but hell ... the risk is to have both drives connected at the same time.

If you don't have backup, then get one first. No, two: one ordinary backup set and one off-site.

Re: Mass FLAC re-encode with metadata intact

Reply #3
Preserving the metadata is the reason why I can't just uncompress everything to wav and compress again using standard tools.

Even this might work. Some programs like dBpoweramp, Foobar do write ID3 tags into a WAV so as long as you stick to them, you will be fine. https://www.thewelltemperedcomputer.com/KB/WAV_KB.htm

However, what I expect is the following. A program reads the tags, convert the audio to linear PCM, convert it to the new format, write the tags. Hence if you do the conversion inside Foobar or Musicbee or dBpoweramp, etc. I expect the tags to be preserved.

File creation time I expect to change as this is typically the information derived from the file system.
TheWellTemperedComputer.com

Re: Mass FLAC re-encode with metadata intact

Reply #4
If you are a *nix user like me, you would just use 

Code: [Select]
flac -8f *.flac

in some directory. Or, for recursive use,

Code: [Select]
find . -iname \*.flac -print0 | xargs -0 flac -8f

Note that this preserves filesystem mtime with limited resolution, only up to seconds.

If a 2-way file synchronisation software that looks at mtimes is being used it might replace the newly compressed FLACs with the older versions because their mtime would be appear newer due to the increased timestamp precision.

Re: Mass FLAC re-encode with metadata intact

Reply #5
Like porcus said, I will add;

- test the proceadure on a few albums before doing the lot (bitcompare etc)
- convert to a new destination (do not overwrite existing)
- Keep the old sources for a while; If your low on space move them to another drive.
- If its too time consuming or not sure; don't do it .
wavpack -b3.63hhcs.5

Re: Mass FLAC re-encode with metadata intact

Reply #6
- convert to a new destination (do not overwrite existing)
Not saying it is a bad idea, but then there is a different thing to check: If the operation aborts, it could leave an incomplete .flac file. (OTOH, with overwrite, it would only upon success delete the old flac and rename the temp.)

So again ... bitcompare in the end.

If you want to "create a new drive with the old as backup" this way:
In Windows, you would consider the following command to convert from source drive S to target drive T (given from source parent directory, possibly at S:\)
FOR /r %f IN (*.flac) DO flac "%f" -o "T:%~pnxf"
... except for a couple of things: flac won't create directories. So you need to create the directory structure first, either empty or first copy with .flac files excluded. If you copied empty directories, you can later "copy over but skip all overwrites" to get all non-flac files.

Re: Mass FLAC re-encode with metadata intact

Reply #7
As an alternative to the suggestions offered, fre:ac works pretty well, and the current "continuous" build even supports 32bit integer FLAC, if you are into that sort of thing...
Not sure how it treats embedded art, as I usually remove those in my collection and replace them with in-folder files instead.

Re: Mass FLAC re-encode with metadata intact

Reply #8
As an alternative to the suggestions offered, fre:ac works pretty well
When is used fre:ac, i noticed that it copy not all tags, only standard ones. Did i miss some option?

 

Re: Mass FLAC re-encode with metadata intact

Reply #9
When is used fre:ac, i noticed that it copy not all tags, only standard ones. Did i miss some option?

Oh! Sorry, I only use fre:ac in some circumstances and never noticed missing "non-standard" tags, as the files I encoded didn't have anything special in that regard. For most stuff I just use foobar2000.
What tags didn't fre:ac copy?

Re: Mass FLAC re-encode with metadata intact

Reply #10
What tools and what environment would you like to use?

If you are a *nix user like me, you would just use 

Code: [Select]
flac -8f *.flac

in some directory. Or, for recursive use,

Code: [Select]
find . -iname \*.flac -print0 | xargs -0 flac -8f

However, it is a good idea to have a backup if you do this.
I tried flac -8epf *.flac and it works. 0.005-0.030 gain over source FLAC files.

Problem is that flac encoder from RareWares is single-threaded while I have 12c/24t CPU and it is not recursive (I use Windows 10 Pro).
Best solution would be to set flac -8epf *.flac on a big archive for 24 threads and leave it for e.g. a weekend having all files re-encoded.

Re: Mass FLAC re-encode with metadata intact

Reply #11
-e is slow and typically not worth it. Except for particular signals, you can outcompress -e and in shorter time. In my testing,
-8p -A "subdivide_tukey(5)"
would outcompress -8pe in much shorter time. And since you got a quite powerful CPU, you could very well beef it up a bit extra into say
-8pfr7 -A "tukey(7e-1);subdivide_tukey(7);flattop"
(edit: the "f" for overwrite. Also, it wouldn't hurt compression - but it would cost time - to replace r7 by r8)


Problem is that flac encoder from RareWares is single-threaded while I have 12c/24t CPU and it is not recursive (I use Windows 10 Pro).
First-world problems ...  :))
I don't know how Windows allocates the processes to threads, but if you got a structure like say, M:\Music\ArtistYearAlbum\filename.flac try one .bat file per first letter.
recompress_A.bat traverses the subdirectories starting with "A" and covering artists like ABBA, ...
...
recompress_Z.bat recompresses ZZ Top.
You can generate the bat files by a FOR loop. You might have to do a few extras for first character not being a English letter (numbers blah blah).

Not unlikely, the Y will be done way before the M and the S (and the "V" in case you are have "Various Artists" there), but in the very least you should get a lot of threads running for a while. And if you have a spinning drive, a possible cause for a defragging.

Re: Mass FLAC re-encode with metadata intact

Reply #12
Does Windows have something similar to GNU Parallel?  You feed it a list of commands to execute and how many threads to use (all by default).  Each command can be unique too.

Re: Mass FLAC re-encode with metadata intact

Reply #13
If you have WSL you can probably use gnu parallel and flac with that.

Re: Mass FLAC re-encode with metadata intact

Reply #14
flac -8p *.flac is enough I think.

Maybe drag and drop on prepared bat file? Any idea what to write inside?

Re: Mass FLAC re-encode with metadata intact

Reply #15
bat file with "d:\flac.exe -8pf %*" works, but:
1. In case of feeding too many files (too many path/name characters) it wont start.
2. Sometimes error: "<file xyz.flac> has an ID3v2 tag" - what now?