Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: FLAC re-encoding (Read 165838 times) previous topic - next topic
0 Members and 3 Guests are viewing this topic.

FLAC re-encoding

Reply #100 –
Sounds like a good idea. Thanks

FLAC re-encoding

Reply #101 –
Following gotkaiser's suggestion, and a hassling PM (  ), I have made some amends to the script:

1. Added retainListOfSuccessfulFiles flag, to create flac-113-successful.txt, a list of successfully converted files.  Set it to 1 to keep, 0 to delete after processing.

REM ######################################################
SET pathToFLAC="FLAC.EXE"
SET flacOptions=-8 -A "tukey(0.5)"
SET retainListOfFailedFiles=1
SET retainListOfProcessedFiles=0
SET retainListOfSuccessfulFiles=0
SET flacVersion=113
REM ######################################################

2. Drag a txt file onto the batch file, instead of a FLAC file or folder, and the script will assume that it contains a list of files to process - using this list instead of creating its own from a folder.  Possibly useful to resubmit failed files, by dragging on flac-113-failed.txt - make sure it's not still in the same folder as flac-113.bat though, otherwise it wll be deleted before anything else*.

* flac-113-processed.txt, flac-113-successful.txt and flac-113-failed.txt are all deleted at the beginning of a new run, and started afresh.  If you want to keep these logs ensure that you move them after processing.
I'm on a horse.

FLAC re-encoding

Reply #102 –
@Synthetic Soul

Didn't want you to feel that I hustle things. Was just a question on the progress. 

Anyway, thanks 

FLAC re-encoding

Reply #103 –
S'OK. I was only joking.  Your PM didn't offend me in the slightest; it's good to know that someone is interested.

I had some spare time last night, so I thought I'd put it to some use.
I'm on a horse.

FLAC re-encoding

Reply #104 –
Ok, is clear. 

FLAC re-encoding

Reply #105 –
@Synthetic Soul

It's me --again-- 

I have one suggestion. (Feel free to work on it when ever you want to!) 

It would be helpfull to have a list to processed a long re-encoding process. For example if you have to abort, in the case you have to turn of your PC. And if you have a lot of flac files it's helpfull to have a list with the "not successfull" files. (an inverted SuccessfulFiles list) Then you could just drag this list an the batch file and proceed the process.
Hope you got the explanation.

I would do it myself but I have no clue how to do it.

So, just in the case you have some minutes left!


Thanks in advance

FLAC re-encoding

Reply #106 –
Yes, this is basically what I was discussing in a previous post:

Quote from: Synthetic Soul on
Secondly, but more difficultly, the script could somehow restart processing a folder and its subfolders, by using the newly suggested "flac-113-successful.txt" and "flac-113-processed.txt" to restart the list, skipping any files that exist in "flac-113-successful.txt".  Presumably this would be initiated by dragging "flac-113-processed.txt" onto the batch file, and having "flac-113-successful.txt" exist in the same folder as flac-113.bat.  Or something.

It would be difficult for me to create a list of files to process and then remove files from it as they were processed (leaving a list of files still needing processing).

If I, or anyone, can think of an easy/sensible way of doing this then I will try to implement it.

Now that we have flac-113-successful.txt and flac-113-processed.txt I still think that this may be the best way.  Perhaps dragging both files at the same time onto the script (to distinguish it from the new "drag a txt file" functionality).  The script would ensure that the files are called flac-113-successful.txt and flac-113-processed.txt, and then process flac-113-processed.txt, ensuring that files listed in flac-113-successful.txt are ignored... maybe...

It would be a good feature to have though.
I'm on a horse.

FLAC re-encoding

Reply #107 –
if you install the flacgetv.exe/libflac.dll/libmm.dll as mentioned earlier in the thread, the script will skip all files with the current version.

I drop my entire music folder on to a shortcut for the batch file and it scans thru the 600 gigs in a bout a minute or less, then starts checking/encoding any files which aren't current.

I don't really see the point to trying to process the log files when it doesn't take very long to re-scan everything anyway.

FLAC re-encoding

Reply #108 –
A very good point!  Thanks for the lateral thinking, maybe I should try some.
I'm on a horse.

FLAC re-encoding

Reply #109 –
SS,

Thanks for the batch file.  This made my entire process painless.  I just completed re-encoding 676GB (25,566 songs) from 1.1.2 to 1.1.3.  It took about five days to complete.  For those interested, I had my 1.1.2 files at -5 and used --best for the conversion and saved 6 GB.  Below are just a couple of things I noticed.

I actually encoded this batch of files twice in the last two weeks.  The first time it came back with 23 files were already the latest version.  However, I knew they were not as I have not started ripping with 1.1.3 yet.  It would have taken forever to try and figure out what 23 files it was talking about so I decided to run the batch file again without flacgetv in the folder.  This time it came back with 23 errors under the "list of failed files".  All of the 23 files failed the encoder test.  The list of failed files allowed me to see which one's had failed.  This did not seem like a coincidence that the first time flacgetv saw 23 already at 1.1.3 (when they weren't) and then 23 files failed the encoder test.  What did these 23 files have in common?  They all had "%" in them somewhere.  Two of the files had "%" in the name of the file.  The other 21 were in a subfolder that had a "%" in the name of the folder.  Here are some examples...

These are the two files that failed

E:\My Music\FLAC\Jane's Addiction\Jane's Addiction\04 - 1%.flac
E:\My Music\FLAC\Soul Asylum\Grave Dancers Union\11 - 99%.flac

This is one of the folders that failed all of the songs inside of it

E:\My Music\FLAC\Sweet, Matthew\100% Fun\all songs.flac

I tested this out by putting these files back through with flacgetv and sure enough it recognizes them as already the latest version (they are actually 1.1.2).  I then removed the "%" and ran them again and they encoded correctly.  I did the same without flacgetv in the folder and got the same results.  Is the "%" not good in 1.1.3?  No big deal, just something I noticed.

Another thing I noticed was that there is a 4kb difference in filesize from a file I ripped in EAC with 1.1.2 -5 and then converted with the batch file to 1.1.3 --best compared to a file ripped in EAC with 1.1.3 --best (4kb bigger).  I am sure there is some reason for this but I couldn't figure it out.  Any thoughts?

FLAC re-encoding

Reply #110 –
Quote from: Marino13 on
Another thing I noticed was that there is a 4kb difference in filesize from a file I ripped in EAC with 1.1.2 -5 and then converted with the batch file to 1.1.3 --best compared to a file ripped in EAC with 1.1.3 --best (4kb bigger).  I am sure there is some reason for this but I couldn't figure it out.  Any thoughts?

probably padding.  flac-1.1.2 adds 4k padding by default. 1.1.3 adds 8k, unless it >20min or so in which case it guesses it's a full album and adds 64k padding.  you can override that with the -P option.

Josh

FLAC re-encoding

Reply #111 –
Quote from: Marino13 on
I tested this out by putting these files back through with flacgetv and sure enough it recognizes them as already the latest version (they are actually 1.1.2).  I then removed the "%" and ran them again and they encoded correctly.  I did the same without flacgetv in the folder and got the same results.  Is the "%" not good in 1.1.3?  No big deal, just something I noticed.
Thanks for the info.  I have done some of my own testing this morning, and can confirm the issue.  Unfortunately I have not found a resolve as yet.

For some reason, when I pass a parameter with a % it gets removed.  Below are two ways to achieve the same thing (well, so I thought).  The first, which is adapted from flac-113.bat, results in the % being removed.  The second works fine.

Code: [Select]
FOR /R %1 %%G IN (*.flac) DO CALL :ListFiles "%%G"
PAUSE
GOTO:EOF
:ListFiles
ECHO %1
GOTO:EOF

Code: [Select]
FOR /R %1 %%G IN (*.flac) DO ECHO "%%G"
PAUSE

I will keep trying.
I'm on a horse.

FLAC re-encoding

Reply #112 –
Just wanted to say thanks for this script. I finally decided to switch my complete library from Monkey's Audio to FLAC now that compression is a little better. Mostly my decision to switch was due to the high level of hardware and software support for the format.

This script is a really great find, especially since I've never been completely sure that the -V switch works when using foobar2000 v0.9.4.2. It's also nice that the script will balk at FLAC files containing ID3 tags, particularly since fb2k refuses to say which type of tags are being used. If only fb2k would show the audio MD5 and codec profile (compression level) as well, like it does for WavPack and Monkey's Audio files, then I would be a little bit happier about having switched formats.

I ran into a few FLAC's recently that did indeed have ID3. I used MP3tag v2.37d to sniff them out which worked pretty well, and probably never would have known otherwise, or at least not until I got around to using this script (which was not a guarantee yet at that point). Not sure how fb2k handles the transfer of tags when converting from FLAC to FLAC, but it looks like the ID3 tags are properly converted to Vorbis Comments during conversion. I'm not 100% that the newly created files are 100% clean, especially since telling fb2k to rewrite the tags doesn't seem to fix them. Someone commented on converting to WavPack as an intermediary step which sounds like a good idea to me, so that is how I'm going to handle things from now on just to be safe.

Again, thanks for the great script, and thanks to all the others that have posted some of their own useful comments.

FLAC re-encoding

Reply #113 –
probably padding.  flac-1.1.2 adds 4k padding by default. 1.1.3 adds 8k, unless it >20min or so in which case it guesses it's a full album and adds 64k padding.  you can override that with the -P option.

Josh
[/quote]


Josh,

The -P function appears to remove all padding, is this correct?  I say that because the files are now 4kb less where before the were 4kb more.

FLAC re-encoding

Reply #114 –
-P has to be followed by the number of bytes of padding.  it can be 0 for a 0 zero-size padding block (which actually takes 4 bytes because of the block header) or you can say --no-padding for no padding block at all.

not sure exactly why it got exactly 4k smaller, I was just guessing.

FLAC re-encoding

Reply #115 –
Quote from: jcoalson on
-P has to be followed by the number of bytes of padding.  it can be 0 for a 0 zero-size padding block (which actually takes 4 bytes because of the block header) or you can say --no-padding for no padding block at all.

not sure exactly why it got exactly 4k smaller, I was just guessing.
I think you must be right.  The script certainly isn't doing anything related.  1.1.2 files has less padding by default, and I assume the conversion process does not add extra padding.
I'm on a horse.

FLAC re-encoding

Reply #116 –
Quote from: Digisurfer on
This script is a really great find, especially since I've never been completely sure that the -V switch works when using foobar2000 v0.9.4.2.

From some tests I ran on that very thing, it does seem to work.  Adding the -V switch to the commandline slowed the encoding speed down, as one would expect.

FLAC re-encoding

Reply #117 –
Quote from: jcoalson on
-P has to be followed by the number of bytes of padding.  it can be 0 for a 0 zero-size padding block (which actually takes 4 bytes because of the block header) or you can say --no-padding for no padding block at all.

not sure exactly why it got exactly 4k smaller, I was just guessing.



Like this, -P4 or like this -P 4 or is it some other way?

My EAC command line is currently...

--best -V -T "artist=%a" -T "title=%t" -T "album=%g" -T "tracknumber=%n" %s

FLAC re-encoding

Reply #118 –
@gib

I'm guessing that he was reffering to if fb2k would pop up it's console window and show an error message if a mismatch would ever occure i.e. flac.exe returning a non-zero return code.

@Marino13

The FLAC Documentation page for flac.exe shows it in this format "-P #", so that way defenetly works, but i haven't tested to see if the other way also would work. Also, remember that the value given with the "-P" switch of flac.exe is in bytes and not kilobytes, so if you e.g. want to have only 4KB padding added instead of 8KB(if under 20min.), then use "-P 4096".

FLAC re-encoding

Reply #119 –
Quote from: Martin H on
@gib

I'm guessing that he was reffering to if fb2k would pop up it's console window and show an error message if a mismatch would ever occure i.e. flac.exe returning a non-zero return code.
Ah!  Now that you mention it, I think you are exactly right.  Thank you. 
And that's a pretty good question, too.

FLAC re-encoding

Reply #120 –
Quote from: Martin H on
@Marino13

The FLAC Documentation page for flac.exe shows it in this format "-P #", so that way defenetly works, but i haven't tested to see if the other way also would work. Also, remember that the value given with the "-P" switch of flac.exe is in bytes and not kilobytes, so if you e.g. want to have only 4KB padding added instead of 8KB(if under 20min.), then use "-P 4096".



Yes, the -P 4096 did the trick.

FLAC re-encoding

Reply #121 –
First of all thanks for this script, very useful

I've been getting this message on some files tho?

Code: [Select]
Checking Vendor: "Z:\Music\FLAC\White Zombie\La Sexorcisto\14 - White Zombie - Warp Asylum.flac"
________________________________________________________________________
Error: Z:\Music\FLAC\White Zombie\La Sexorcisto\14 - White Zombie - Warp Asylum.flac not found or contains no FLAC vendor string.

File is already the latest version [113]


The files definitely aren't already 113 and haven't been encoded any differently to any of my others so not sure what it means. Also what are these errors?

Code: [Select]
  "Z:\Music\FLAC\Atari Teenage Riot\60 Second Wipe Out\10 - Atari Teenage Riot - Death Of A President D.I.Y.!.flac" [E]
  "Z:\Music\FLAC\Cypress Hill\Black Sunday\05 - Cypress Hill - Lick A Shot.flac" [T]
  "Z:\Music\FLAC\Muse\Hullabaloo (CD1)\01 - Muse - Forced In.flac" [E]

FLAC re-encoding

Reply #122 –
1. I'm afraid that's down to flacgetv.exe.  It's supposed to only report an errorlevel of 113 if the file is 1.1.3.  It appears it is not.  I would just reprocess these files without flacgetv.exe.

2. An error occurred while trying to process these files. "[T]" indicates that the file failed a test decode, suggesting that it is corrupt. "[E]" indicates that an error occurred when re-encoding - the file passed the initial test decode, but failed verification when re-encoding.  None of these files have been converted.  I don't know why a file would fail re-encoding afte passing the decode test, so it is very possible that these files have ID3v2 tags at the beginning, as per earlier reports in this thread.
I'm on a horse.

FLAC re-encoding

Reply #123 –
A few more ideas, should you be bored and/or looking for further improvements.

- note the file sizes before and after re-encoding, show the individual file gains and a summary at the end with the total original and re-encoded sizes.

- note the commandline parameters (in the summary)

- note the script and track encoding start/end date/times and calculate total time spent encoding and the total time the script to scan/test/encode. I understand that encode time is dependent on cpu/drive loads, however it would provide a ball park figure for differences in compression levels etc.

I suggest the above, so the script could be used to easily compare different flac versions and command lines (ie testing alpha1 etc) in a relatively consistent manner along with the results.

edit:

- date/time stamp the logs/summary file names so that they are not overwritten (optionally if you think its necessary)

FLAC re-encoding

Reply #124 –
Thanks Synthetic Soul, it really is a very useful script  Is there any way to get it to output the no vendor string files as this is only listed when it processes the file so when you have thousands of files being processed there's no realistic way of finding them?

Cheers again!