Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: AutoFLAC (Read 324706 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

AutoFLAC

Updated: 10/16/2006
Version 1.2 Released - This release includes support for variable naming schemes, much improved write-mode capabilities (including a new options GUI), ability to write from CUE sheets created with other applications, and a few important bug fixes (among other things).  Complete details below.

I just finished writing a new "frontend" (more of an automation script, really) that automates backing up and restoring CDs using EAC and FLAC.  I know there are at least a couple of similar applications out there (flacattack and REACT are the two that I'm aware of); however, despite being great apps themselves, neither supported all of the features that I was looking for.  So, I over the last week or so I wrote my own script, and I'm posting it here as I feel that some others may be able to benefit from it as well.

At this point I'm sure you're wondering, "What's so special about this script?"  The three main features, and primary reasons I wrote the script, are:
  • FLAC write support - The single greatest feature of EAC for me is it's ability to generate CUE sheets from an audio CD and reuse these CUE sheets to create a perfect duplicate of the CD of the original is ever lost, stolen, or damaged (anyone that has had CDs stolen will understand why this is important).  However, EAC only supports WAVE files when writing a CD, which means previously compressed tracks must first be converted to WAVE files.  AutoFLAC automates this process by decompressing all tracks, converting the CUE sheet to reference WAVE files rather than FLACs, and loading the CUE sheet in EAC's cd-writing interface.  After writing is complete, AutoFLAC will remove all temporary files.
  • Multi-disc support - This is probably more of a personal preference than anything else, but I really wanted an app to automate this for me.  Rather than ripping discs that are part of a single set to separate album directories, AutoFLAC can automatically renumber and retag discs so that they are saved to the same album directory.  If enabled in the AutoFLAC GUI, the disc track numbers will be renumbered Nxx, where N is the disc number and xx is the track number.  So, track three on disc two of a two-disc set will be numbered as track 203.
  • Data file support - AutoFLAC can check for the presence of a "data" track on an audio CD, which typically contain bonus material that can be viewed on a computer.  If enabled, AutoFLAC will copy all deta files to a Data subdirectory of the album after the ripping process is complete.
Full details and download links can be found on the AutoFLAC home page.  I always enjoy feedback, so if you download it and try it out (or just have any questions), let me know!

Direct downloads:

Installation Package - http://www.c1pher.com/autoflac12.exe
Binary Archive - http://www.c1pher.com/autoflac12_noinst.rar
UniExtract Source Code - http://www.c1pher.com/autoflac12_source.rar

Installation Package
Binary Archive
UniExtract Source Code
 
Changes in version 1.2
Code: [Select]
Added support for variable naming schemes
 Added GUI interface for write mode options
 Added support for writing CUE sheets that specify .wav files rather than .flac
 Added option for low priority encoding
 Added support for EAC's Test and Copy mode
 Added proper GUI for initial EAC binary selection (if needed)
 Added new AutoFLAC icon
 Fixed waiting for EAC to complete encoding of all tracks
 Fixed waiting for EAC to complete creating CUE sheet
 Fixed remembering location of EAC binary
 Fixed bug that may prevent converting FLACs to WAVs before writing
 Fixed names of some variables to prevent possible conflict
 Updated GUI button behavor to focus cursor on relevant field after selection
Thanks, and I hope you find this as useful as I have.

AutoFLAC

Reply #1
Just downloading the app for testing. Just wanted to inform you that the screenshots do not display in Opera. They display fine in IE and Firefox.

audiomars
Reason is immortal, all else mortal
- Pythagoras


AutoFLAC

Reply #3
Ok, I fixed it, though technically this was more Opera's fault than mine.  :-)  I use alpha-channel transparency in my images, which IE does not properly support.  In order to fix this I run a script that checks for IE running on win32, and if found it does some magic that fixes the transparency issue (I found the script online, so I'm not clear on the details).

Anyway, after playing around with Opera for a little while I realized that it was identifying itself as Internet Explorer.  Bad Opera!  Since it already supported the images correctly, the javascript hack was essentially hiding the images for some reason.  So, I figured out a way to make my script also differentiate between real IE and fake Opera IE user agent strings.  This should solve the problem, though the proper solution would be for Opera to identify itself correctly.

Anyway, thanks again for the heads up.  Everything should be working correctly now.

AutoFLAC

Reply #4
FYI, I just posted v1.0.1 to my site.  It's mainly just a bugfix release for a few issues I found during additional testing.

AutoFLAC

Reply #5
I always enjoy feedback, so if you download it and try it out (or just have any questions), let me know!

Thanks, and I hope you find this as useful as I have.


This is great!  Thanks, nitro.

A constructive criticism, if you don't mind? 

My own preference would be for full-disc FLAC image + cue, instead of multiple tracks plus cue.  Why?  The hidden track (Track 01 Index 00) issue, and secondarily the loss of metadata (e.g. TOC, etc.).  Without the full-disc image, the hidden tracks are stripped away, making AutoFLAC substantially less trustworthy for archiving CDs.

-brendan

AutoFLAC

Reply #6
This looks really promising. I'll try it later. The hidden-track issue is really important for full disc backups.
Question: Is there any checksum created? I use to verify my backups before restoring/re-burning them.

AutoFLAC

Reply #7
I haven't tested it out yet, but do you plan on supporting more lossess codecs? I'd be seriously interested if it supported WavPack as my entire collection is in that format.

AutoFLAC

Reply #8
Hi nitro, sorry for making you hack through code when it was Opera's fault  . Now the site loads properly in Opera. I tried it out and I find it very useful. However, as brendan suggested, maybe there could be an option to use a single full-length flac or use split flac files. That and support foe WavPack would make it just about perfect 

audiomars
Reason is immortal, all else mortal
- Pythagoras

 

AutoFLAC

Reply #9
I'm always up for some constructive criticism, so don't be shy. 

I explained on the AutoFLAC web page why I chose to use individual FLACs rather than a separate image.  My opinion on that subject still stands, for the reasons given.  However, the hidden track issue that several people have mentioned is new to me.  Can someone please explain that more thoroughly, or point me to a good reference?  I'm assuming that you're not simply referring to the"bonus" track that many CDs include at the end, right?

bhoar, you mentioned something about a metadata issue.  Could you expand on that as well?  What information would you get by creating a single disc image vs. separate tracks?  Currently AutoFLAC will tag each track with Artist, Album, Genre, Year, Title, and Number.  The .cue sheet saves most of this information as well as CD-TEXT, including the DISCID.  What's missing?

elmar3rd, no checksum is currently stored.  Honestly, this has never occured to me.  How do you create the checksum, and how do you verify it?  I'll certainly consider this for the next release.

ChangFest and audiomars, while I don't have anything against WavPack, the name of the program is AutoFLAC.  There's a reason for this.    I chose to use FLAC as my storage medium because it's extremely well-supported across all platforms, and given that I generally use Linux, this is very important to me.  I'll look into the possibility of supporting other formats, though.

One thing to keep in mind is that I originally wrote this to fulfill my own requirements.  I posted it here, as I said in my first post, because I think it's a very useful program and I felt it could help others as well.  I honestly would be more than happy to add support for additional features if it will be used by other people, but I didn't want to waste a lot of time up front coding features that I won't use if no one else was using it either. 

Since it looks like there are at least a few people interested in using it, though, I'll be happy to look into extending it.  I'll post an update once I've had a chance to test out some ideas.  In the meantime, I'd really appreciate it if you could answer my questions above.  Thanks.

AutoFLAC

Reply #10
Quote
elmar3rd, no checksum is currently stored. Honestly, this has never occured to me. How do you create the checksum, and how do you verify it? I'll certainly consider this for the next release.

I use to add MD5-checksums to all I back up on CD-R/DVD-R. This is an easy way to verify it later. There are a lot of MD5-Tools.
This could be a nice feature for AutoFLAC. But as you already mentioned, it is not that important to waste precious time for it.

AutoFLAC

Reply #11
I'm always up for some constructive criticism, so don't be shy. 

I explained on the AutoFLAC web page why I chose to use individual FLACs rather than a separate image.  My opinion on that subject still stands, for the reasons given.  However, the hidden track issue that several people have mentioned is new to me.  Can someone please explain that more thoroughly, or point me to a good reference?  I'm assuming that you're not simply referring to the"bonus" track that many CDs include at the end, right?

bhoar, you mentioned something about a metadata issue.  Could you expand on that as well?  What information would you get by creating a single disc image vs. separate tracks?  Currently AutoFLAC will tag each track with Artist, Album, Genre, Year, Title, and Number.  The .cue sheet saves most of this information as well as CD-TEXT, including the DISCID.  What's missing?

Since it looks like there are at least a few people interested in using it, though, I'll be happy to look into extending it.  I'll post an update once I've had a chance to test out some ideas.  In the meantime, I'd really appreciate it if you could answer my questions above.  Thanks.


Ok, I won't be shy! 

And nope, it's not the bonus track at the end of another track.  It's a bonus track found *before* the first track.  A technical explanation of the hidden track issue is this:  some CDs have an additional song stored in at the Track 01 Index 00 mark.  However, most CD rippers start ripping at Track 01 Index 01 when in track-extraction mode, losing the audio between index 00 and index 01.

CD Players also start playing at Index 01, so in order to hear the song on a CD player, you have to rewind "back into" index 00.

The daefeatures site catalogs drive features, including the "Hidden Track One Audio" feature:
http://www.daefeatures.co.uk/htoa.php

You can see in the example CUE file accompanying the test image how it works.

Regarding the metadata, I am a bit more fuzzy on that: I'd just seen it discussed in the forums as one reason to create a single FLAC for the disc.  Basically, I'd want to make sure that if I burned a copy of my FLAC-archived disc that freedb lookups would stillwork on the CD-R (important) and any normal special tracks would also be part of the archive (less important).

-brendan

PS - I'm planning to use AutoFLAC, myself, for archiving my own (and some other folks') collection using a robot.  I have already written a CLI tool to control the robot and an external (independent) script to use the tool to tell the robot to unload a finished disc and load a new disc every time a disc ejection is detected.  I may make some changes to the AutoFLAC script once I start using it, and I'll be sure to feed them back to you when I do so.

AutoFLAC

Reply #12
[quote name='elmar3rd' date='Jun 15 2006, 14:21' post='403288']I use to add MD5-checksums to all I back up on CD-R/DVD-R. This is an easy way to verify it later. There are a lot of MD5-Tools.[/quote]

So you run md5 on the .flac file after it's already been extracted and converted?  In that case it sounds like you're only verifying the integrity of the .flac file itself (if, for example, you copy your collection from one drive to another) rather than somehow verifying it based on the original CD track.  Is this correct?

Just trying to get a better understanding.

[quote name='bhoar' date='Jun 15 2006, 15:29' post='403305']And nope, it's not the bonus track at the end of another track.  It's a bonus track found *before* the first track.  A technical explanation of the hidden track issue is this:  some CDs have an additional song stored in at the Track 01 Index 00 mark.  However, most CD rippers start ripping at Track 01 Index 01 when in track-extraction mode, losing the audio between index 00 and index 01.[/quote]

Ok, I get the concept, but not sure I understand why the current method fails.  Whether I use EAC to rip tracks to a single WAVE file (image) or individual WAVE files, would it not start reading at the beginning of the audio information?  Is it a confirmed issue that EAC disregards any leading information when ripping to individual tracks?  It seems odd that it'd read the disc differently based on how the data is output.

[quote name='bhoar' date='Jun 15 2006, 15:29' post='403305']Regarding the metadata, I am a bit more fuzzy on that: I'd just seen it discussed in the forums as one reason to create a single FLAC for the disc.  Basically, I'd want to make sure that if I burned a copy of my FLAC-archived disc that freedb lookups would stillwork on the CD-R (important) and any normal special tracks would also be part of the archive (less important).[/quote]

I've tested CDDB (actually, freedb) lookups using CDs restored by AutoFLAC, and it was recognized correctly.  Like I said, the individual flacs and associated cue sheet should contain all meta information that the flac images contain.  If anyone can proove me wrong, though, feed free.  [/quote]

[quote name='bhoar' date='Jun 15 2006, 15:29' post='403305']PS - I'm planning to use AutoFLAC, myself, for archiving my own (and some other folks') collection using a robot.  I have already written a CLI tool to control the robot and an external (independent) script to use the tool to tell the robot to unload a finished disc and load a new disc every time a disc ejection is detected.  I may make some changes to the AutoFLAC script once I start using it, and I'll be sure to feed them back to you when I do so.[/quote]

Sounds interesting.  You're using an actual, physical robot to swap out the discs?  Sweet!


Edit:  Hmm...  does anyone know why my quotes aren't displaying properly in this post?

AutoFLAC

Reply #13
I believe that is the case, yes.

When in track output mode, EAC allows you to configure your preferences to save the Index-00->Index-01 audio, typically called the "gap"m either at the end of the previous track or at the beginning of the next track. 

1. Putting the gap onto the next track produces some pretty annoying side effects in general.

2. Putting the gap onto the previous track doesn't help with HTOA because there is no Track 00.

See:

  http://wiki.hydrogenaudio.org/index.php?title=Gap_settings

-brendan

AutoFLAC

Reply #14
The Red Book defines the pre-gap of track one to be between 2 and 3 seconds long(the time from INDEX 00 and INDEX 01 of track one), so the CDs that have hidden tracks in the pre-gap of track one isn't compatible with the CD-DA spec(Red Book). The INDEX 01 of track one identifies the start of the first track, and as bhoar previously said, then this is where the CD players begins playing from and EAC begins ripping from in track mode(without changing the gap treatment settings away from the default setting).

The discs burned from separate tracks with cuesheets will also be capable of doing freedb lookups even if the original CD had a hidden track in the pre-gap of track one. This is because EAC adds a PREGAP statement at the beginning of the first track in the cuesheet so that the writer during writing will generate a pre-gap with null samples of the same length as the hidden track, so that the discid will be computed identically on the copy as to the original, but just will have silence in the pre-gap instead of the hidden track.

There are no differences in the metadata from separate tracks with cuesheets compared to images with cuesheets.

AutoFLAC

Reply #15
Quote
So you run md5 on the .flac file after it's already been extracted and converted?  In that case it sounds like you're only verifying the integrity of the .flac file itself (if, for example, you copy your collection from one drive to another) rather than somehow verifying it based on the original CD track.  Is this correct?

Yes, exactly.

AutoFLAC

Reply #16
elmar3rd, I've implemented MD5 verification in the development version of AutoFLAC, but based on my initial testing so far it doesn't seem very useful.  For example, simply changing the track's comment tags (whether fixing an error in the track title, or adding a new tag such as PERFORMER) yeilds a different MD5.  Therefore, if you make ANY updates or changes to the .flac file after it's initially ripped and tagged, it'll fail verification.

Like I said, this doesn't seem very useful.  Perhaps there's a better way of implementing such verification that's a little more robust and can handle tag updates.  Any ideas?

bhoar and Martin H, thanks for the excellent explanations of HTOA.  I did some testing using the files at the link provided, but it turns out that my drive is apparently incapable of ripping audio from track 01 index 00.  It seems to recognize that there is indeed audio there, as I get the appropriate amount of leading data, but it's just silence.  After doing some additional searching through the forums it seems that if I get silence on the test CD then I'm pretty much SOL.  Sigh, figures. 

To everyone else, I'm currently looking into supporting both flac images and WavePack.  These will both take some time to implement, though, so please be patient.

AutoFLAC

Reply #17
bhoar and Martin H, thanks for the excellent explanations of HTOA.  I did some testing using the files at the link provided, but it turns out that my drive is apparently incapable of ripping audio from track 01 index 00.  It seems to recognize that there is indeed audio there, as I get the appropriate amount of leading data, but it's just silence.  After doing some additional searching through the forums it seems that if I get silence on the test CD then I'm pretty much SOL.  Sigh, figures. 

To everyone else, I'm currently looking into supporting both flac images and WavePack.  These will both take some time to implement, though, so please be patient.


Ouch.  I've got on the order of 25 CD/DVD(-ROM,-R,-RW,+R,+RW,-RAM) drives here I need to compile a DAE features list with in the next couple of weeks.  If you can't fine one nearby to test on that does HTOA, I'll try to send one your way.  No promises  other tasks are taking priority: just stopped myself halfway done setting up a test station for this, and decided I'd better stick with my current (growing) TODO list.  But it's there, item 17 at this point. 

-brendan

AutoFLAC

Reply #18
Perhaps there's a better way of implementing such verification that's a little more robust and can handle tag updates.  Any ideas?

If the FLACs are encoded with the -V switch, then there isn't any reason for verification IMHO, but to test the integrity of the file without concern for tag updates etc, then you can use the -t switch of flac.exe.

"flac will exit with an exit code of 1 (and print a message, even in silent mode) if there were any errors during decoding, including when the MD5 checksum does not match the decoded output. Otherwise the exit code will be 0"

AutoFLAC

Reply #19
If the FLACs are encoded with the -V switch, then there isn't any reason for verification IMHO, but to test the integrity of the file without concern for tag updates etc, then you can use the -t switch of flac.exe.

Wow, I never realized that flac files embed an MD5 signature in the file by default.  This seems like it should do the trick.

However, I'm not exactly sure how this is supposed to work.  For example, look at the following output:

Code: [Select]
jbreland@showdog ~/test $ metaflac --show-md5sum 01-Intro.flac
46e296291cfdb19f2ae8c6a36d6611a4
jbreland@showdog ~/test $ flac -ds 01-Intro.flac
jbreland@showdog ~/test $ md5sum 01-Intro.wav
b655d4ba6eaddec49b0cb517304cac0f  01-Intro.wav


The stored MD5 hash does not the md5sum of the decoded WAVE file.  How is the stored hash generated, and how exactly can it be used for verification?

Or here's another idea: EAC calculates and stores the track's CRC value in the log file.  Can I use this for verification?

Unfortunately, I can't figure out how this is supposed to match up to anything.  Even if I decode the flac and then open it in EAC's WAVE editer, it gives me a different CRC than is in the log.

AutoFLAC

Reply #20
Hi nitro322

During encoding, then flac.exe calculates a md5 signature of the raw audio data(without the WAV header) which then gets stored in the FLAC file. When decoding the FLAC file back to WAV format, then flac.exe checks for errors in the stream while also checking that the stored MD5 signature matches with the decoded raw output data. The -t switch of flac.exe does exactly the same as when using the -d switch to decode to WAV, but the only difference is that no output file is written.

EAC also calculates it's CRC values on the raw audio data without the WAV header.

AutoFLAC

Reply #21
To everyone else, I'm currently looking into supporting both flac images and WavePack.  These will both take some time to implement, though, so please be patient.

Thank you. I'll be patient 

AutoFLAC

Reply #22
I'm working on adding support for flac images, but I have a few questions about how exactly it should be done.  Since I don't use this method myself, I'd appreciate some input from a more knowledgeable person so that I make sure I implement it correctly.

Output location
How are flac images normally saved?  Normally I use the following structure for individual tracks:
$output\$genre\$artist\$album\
Should I do the same for the flac images?  Or since since each filename should be distinct, would it make more sense to just save everything directly to $output\?

File names
By default EAC names the images "Artist - Album.ext".  Should I stick with that format, or is there a better naming convention I should follow?

Embedding cue sheets
I see that I can embed the cue sheet using either flac (during initial encoding) or metflac.  Is there any particular advantage to using one over the other?  Is EAC able to natively embed the cue sheet, or should I always rip to an uncompressed wave and the encode and embed the cue sheet myself?

ReplayGain
How do I apply ReplayGain to FLAC images?  Does it automatically know how to split up the tracks, or do I have to instruct it somehow?

I think that should cover it.  I'd really appreciate any feedback you could provide.  Thanks.

AutoFLAC

Reply #23
Re: Output location and filename,  I am of multiple minds on this but the central idea is that these will be long term archives, so...
a) make the path and filename user modifiable
or
b) if not or if you just stick to $output/, make sure the filename is unique, just in case you have two editions of the same CD (e.g. a UK vs. US pressing and/or a rerelease) that differ slightly.  For flac, I'd be OK with "Artist - Album (freedbid).ext".

The other items, I will leave to people more versed in the arts.

-brendan

AutoFLAC

Reply #24
@nitro322

Output location:
EAC asks for the location to store images.  It would be *nice* to have it create directories based on tag variables.

File names:
The default of "Artist - Album.ext" is cool, but if you could allow one to use something additional, like "Year", that'd be cool.  FYI: I use "Artist ~ Year ~ Album.ext".  Notice the (~) tilde?  I've never run across any names, etc... that use it, so for a delimiter it's great.

Embedding cue sheets: Personally, I don't embed because it's difficult to update after the fact.

ReplayGain: Forget messing with it for images unless you want to get involved with WaveGain.  Let Foobar handle that one.

Here's how I have my music library setup:
\Music\Lossless\Artist ~ Year ~ Album\Artist ~ Year ~ Album.ext (Images)
\Music\Lossy\Artist ~ Year ~ Album\Artist ~ Album ~ Track ~ Title.ext (Single files)