HydrogenAudio

Lossless Audio Compression => Lossless / Other Codecs => Topic started by: urak on 2008-01-07 16:37:30

Title: Ape image (+cuesheet) to single ape tracks
Post by: urak on 2008-01-07 16:37:30
Hi to all.

Is there a program to cut a big ape file (image+cuesheet) into individual ape (with tag) DIRECTLY and not into wave which need to compress again?

Maybe shntool (but to me it's a bit difficult to use)?

many thanks.
Title: Ape image (+cuesheet) to single ape tracks
Post by: Keykey on 2008-01-07 16:42:09
Foobar.
Title: Ape image (+cuesheet) to single ape tracks
Post by: urak on 2008-01-07 16:51:41
Foobar.

Ape tracks directly (and not into wave)? Are you sure? Do you know how to set up the foobar's Converter section? (i downloaded mac.exe with stdin support from shntool site).

Thanks
Title: Ape image (+cuesheet) to single ape tracks
Post by: Synthetic Soul on 2008-01-07 17:01:44
http://www.hydrogenaudio.org/forums/index....showtopic=57563 (http://www.hydrogenaudio.org/forums/index.php?showtopic=57563)

Edit: If you are using the shntool version then I guess you could (IIRC) replace "%s" with "-", i.e.:

Code: [Select]
- %d -c2000
Title: Ape image (+cuesheet) to single ape tracks
Post by: SamHain86 on 2008-01-07 17:02:53
As far as I understand it, everything has to decompress the APE into PCM WAV, regardless if you see the temporary file or not.

We cover APE to * it seems every month.
Title: Ape image (+cuesheet) to single ape tracks
Post by: xmixahlx on 2008-01-07 22:16:29
well, it has to decode APE to WAV, yes - but it may not use a temporary file and pipe stdout to stdin (directly to the new encoder).

shntool IS a good tool for this (shnconv)


later
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-01-07 22:21:28
well, it has to decode APE to WAV, yes - but it may not use a temporary file and pipe stdout to stdin (directly to the new encoder).

shntool IS a good tool for this (shnconv)

...as is foobar2000.
Title: Ape image (+cuesheet) to single ape tracks
Post by: urak on 2008-01-07 23:27:38
Edit: If you are using the shntool version then I guess you could (IIRC) replace "%s" with "-", i.e.:

Code: [Select]
- %d -c2000

It doesn't works (- %d -c2000): foobar 0.9.5 writes the single ape file (converter with mac.exe, version 3.99u4 from shntool website) but it's not playable.

%s %d -c2000 works fine.

thanks
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-01-07 23:51:21
Oh, I didn't see that you're going from .ape to .ape.

So that's right, the special build of Monkey's Audio works with shntool but does not work with foobar2000 when encoding from stdin. foo_input_monkey.dll, OTOH does allow you to decode to other formats without writing an interim wave file.

Sorry about that.


Did you try using the Monkey's Audio GUI to create individual tracks using .apl files?
Title: Ape image (+cuesheet) to single ape tracks
Post by: urak on 2008-01-08 00:58:47
Did you try using the Monkey's Audio GUI to create individual tracks using .apl files?

No, 'cause i don't want to install anything (portable apps only, i hate apps that write to registry, to sys folders like %userprofile%, %appdata%, etc.).
What is the makeapl.exe latest version? Is it possible to use it without to install the entire monkey's audio package?
Anyway, foobar works fine, thanks 
Title: Ape image (+cuesheet) to single ape tracks
Post by: eevan on 2008-01-08 16:14:47
You can make APL files from the foobar. (http://www.hydrogenaudio.org/forums/index.php?act=ST&f=29&t=60142&hl=&view=findpost&p=538578)
Title: Ape image (+cuesheet) to single ape tracks
Post by: urak on 2008-01-10 22:38:02
You can make APL files from the foobar. (http://www.hydrogenaudio.org/forums/index.php?act=ST&f=29&t=60142&hl=&view=findpost&p=538578)

    No, I haven't the "Write APL link files" option (foobar 0.9.5 and foo_input_monkey 2.1.2 ).

thanks
Title: Ape image (+cuesheet) to single ape tracks
Post by: Zarggg on 2008-01-16 17:30:37
If you have a cuesheet, you can also use the program CueSplitter; that's what I've used for a while, and I've never had it fail me yet.
Title: Ape image (+cuesheet) to single ape tracks
Post by: shnutils on 2008-03-10 04:05:05
You should be able to pipe data directly to mac.exe now (i.e. "- %d -c2000" should work).  See this post (http://www.hydrogenaudio.org/forums/index.php?showtopic=61634&st=0&gopid=551904) for details.
Title: Ape image (+cuesheet) to single ape tracks
Post by: urak on 2008-03-10 12:00:21
You should be able to pipe data directly to mac.exe now (i.e. "- %d -c2000" should work).

Hi.

I tested with foobar 0.9.5.1 and your new mac.exe build.

Loaded (into foobar) cuesheet+ape image:

1- converted/extracted track 2 into a single ape file (File A) with these settings: %s %d -c3000

2- converted/extracted track 2 into a single ape file (File B) with these settings: - %d -c3000

It worked but File A and File B have different Foobar's Audio MD5 (hash of the audio stream only). Is it normal?

Thanks
Title: Ape image (+cuesheet) to single ape tracks
Post by: psycho on 2008-03-10 12:06:34
If you decide, you will install something anyway, Monkey's Audio has it's own tool for making APLs, typically the exe resides here: "C:\Program Files\Monkey's Audio\Tools\MakeAPL.exe"
Title: Ape image (+cuesheet) to single ape tracks
Post by: shnutils on 2008-03-10 17:57:37

You should be able to pipe data directly to mac.exe now (i.e. "- %d -c2000" should work).

I tested with foobar 0.9.5.1 and your new mac.exe build.

Loaded (into foobar) cuesheet+ape image:

1- converted/extracted track 2 into a single ape file (File A) with these settings: %s %d -c3000

2- converted/extracted track 2 into a single ape file (File B) with these settings: - %d -c3000

It worked but File A and File B have different Foobar's Audio MD5 (hash of the audio stream only). Is it normal?

It's probably a bug, especially if the audio hashes reported by "shntool hash" differ as well.  (If the shntool hashes match, I would be very interested to see your sample cuesheet)

I will play around with this a bit more to see what I missed.      In the meantime, I'll put the old mac.exe back on the website and mark this new build as experimental.  Thanks!
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-10 17:57:52
It's not a bug.

I don't know how it gets this hash, but foobar's "Audio MD5" is not the md5 for decompressed audio data.  I think its what Monkey's Audio uses for its quick verify routine but I could never confirm this.

Just as a quick test I took a wave file converted it using -c2000 and -c3000 and got different checksums.

You can always bit-compare decompressed wave files to be sure.
Title: Ape image (+cuesheet) to single ape tracks
Post by: shnutils on 2008-03-10 18:46:39
Hmm.  Well, then perhaps it's just a bug in mac.exe's MD5 calculation?  I assume your files checked out with a bit-comparison test.

Is this bug unique to my test version of mac.exe, or does it occur in other versions?  If the former, I'll try to fix it; if the latter, I can relax. 
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-10 19:11:29
You can relax (I haven't downloaded your version yet ).

I'm certain it's not a bug with MAC.  If it's the hash I think it is then it's for the compressed data.
Title: Ape image (+cuesheet) to single ape tracks
Post by: skamp on 2008-03-10 19:44:56
I'm certain it's not a bug with MAC.  If it's the hash I think it is then it's for the compressed data.

MAC's internal digest is that of the compressed data indeed, I recently took a look at the source code. That's also how I figured out why the verify option of the command-line application involves decoding of the audio, while I had read about a quick verify feature: for some reason that feature is disabled. It can be enabled by changing "bQuickVerifyIfPossible = FALSE" to "bQuickVerifyIfPossible = TRUE" in src/MACLib/MACLib.h, line 440.
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-10 20:03:42
Thanks for the info.  I've always wondered about this.

If anything the user should have the option (eg. -v for full verification, -vq for quick verification).
Title: Ape image (+cuesheet) to single ape tracks
Post by: shnutils on 2008-03-11 17:43:23
Thanks for the info.  I've always wondered about this.

If anything the user should have the option (eg. -v for full verification, -vq for quick verification).

The user now has the option. 

Version 3.99-u4-b5-s3 (http://www.etree.org/shnutils/shntool/) (that's a mouthful) re-enables quick verification, via a new '-q' option ('-v' will still do full verification).

I'm certain it's not a bug with MAC.  If it's the hash I think it is then it's for the compressed data.

MAC's internal digest is that of the compressed data indeed, I recently took a look at the source code. That's also how I figured out why the verify option of the command-line application involves decoding of the audio, while I had read about a quick verify feature: for some reason that feature is disabled. It can be enabled by changing "bQuickVerifyIfPossible = FALSE" to "bQuickVerifyIfPossible = TRUE" in src/MACLib/MACLib.h, line 440.

Actually it turns out that the MAC internal digest is basically comprised of everything BUT the compressed audio data (i.e. APE metainfo, seek tables, WAVE header, and trailing data).  That's what allows it to be quick.

Good catch on the bQuickVerifyIfPossible code... I modified MAC based on your info.  Thanks!
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-11 19:05:13
This seems to contradict my findings.  If I alter any data in the middle of a file the GUI's quick verify will fail (as it should).  If I alter metadata the quick verify will still pass.  Maybe there's something I'm not fully understanding which wouldn't be the first time, nor will it be the last.

Looking forward to checking out the new version. 

Thanks!
Title: Ape image (+cuesheet) to single ape tracks
Post by: skamp on 2008-03-11 20:16:42
This seems to contradict my findings.  If I alter any data in the middle of a file the GUI's quick verify will fail (as it should).  If I alter metadata the quick verify will still pass.

Same here.
Title: Ape image (+cuesheet) to single ape tracks
Post by: shnutils on 2008-03-11 20:56:27
This seems to contradict my findings.  If I alter any data in the middle of a file the GUI's quick verify will fail (as it should).  If I alter metadata the quick verify will still pass.

Same here.

As a point of reference, are you altering data with an audio editor, or by flipping bits with a hex editor?  And by metadata, I wasn't referring to any tags... just the internal APE data used by the decoder.

That said, I'm also seeing the quick verification failures -- even on files that have not been modified since their creation.  My tests were on files of pure digital silence (via shntool gen), which apparently work up until 4:24.11 -- beyond that, quick verification fails.  Time to do some more digging...

Thanks again for testing these new versions.  Hopefully these issues will be cleared up soon.
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-11 21:01:50
Flipping bits with a hex editor.

My interpretation of metadata was tag information, though I can alter bytes 5, 6 and 7 of a sample MAC file without affecting either type of verification.  I wasn't so lucky with bytes 0-4, 8, 9 or changes at random points much farther into the file.
Title: Ape image (+cuesheet) to single ape tracks
Post by: skamp on 2008-03-11 21:33:23
Note that execution times also suggest the compressed data get hashed (i.e. the quick verify takes about as much time as running md5sum on the whole file, and that time is proportional to the filesize).
Title: Ape image (+cuesheet) to single ape tracks
Post by: shnutils on 2008-03-11 23:22:45
Note that execution times also suggest the compressed data get hashed (i.e. the quick verify takes about as much time as running md5sum on the whole file, and that time is proportional to the filesize).

Upon further reflection, you guys are absolutely right.  In fact, MAC calculates the md5sum of the following, in order:

The only bytes not included are the first 52 bytes (APEDescriptor).  Also, the order in which these appear in the APE file is: 3, 4, 1, 2, 5; so it's not possible to just md5sum all but the first 52 bytes in sequence.   

Since I had only tested with files containing pure digital silence, I didn't see how quick verification may not be so quick, because MAC was compressing them so well...   

At any rate, I have found the issue with quick verification and am pondering solutions.
Title: Ape image (+cuesheet) to single ape tracks
Post by: shnutils on 2008-03-12 01:49:50
At any rate, I have found the issue with quick verification and am pondering solutions.

Okay, because the APE checksum depends initially on the WAVE header, and the WAVE header is not completely known until piped encoding is finished, there are only a few options here:

My own inclination is to default to option 1, but allow the user to specify option 3 via a command-line parameter (this parameter will essentially mean "ignore header size").  Option 2 is not attractive because it would lead to confusion in the APE user base.

Thoughts?
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-12 02:35:38
> Thoughts?

Give the user the option to create a proper (backwards-compatible) quick verification checksum during the encoding process but calculate nothing(*) by default.

(*) unless there is no noticeable speed penalty in calculating something knowing that it breaks compatibility (something is better than nothing).

I hope this provides the best of all worlds.  Maybe an option to be backwards compatible, calculate something that isn't and an option to calculate nothing if it gives a noticeable improvement in speed?
Title: Ape image (+cuesheet) to single ape tracks
Post by: skamp on 2008-03-12 07:02:56
IMO, you have to work with the format's quirks, unless you want to fork it. That means option #3. The encoder didn't have piping support to begin with, it's not like you're *losing* functionality.
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-12 07:50:02
Because MAC decoding speed is so slow, I've really grown to appreciate the quick verify function to test my archives to DVD.  My concern is that say a year down the road when I go back to test something I may not remember which version was used to create it.
Title: Ape image (+cuesheet) to single ape tracks
Post by: skamp on 2008-03-12 08:41:37
Well, the quick verify option is less useful on read-only media. You may as well run md5sum on the files themselves and store both the files and the MD5 sums list on the CD/DVD-R.
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-12 16:16:28
Why bother with an md5sum if it's already built-in?
Title: Ape image (+cuesheet) to single ape tracks
Post by: shnutils on 2008-03-12 16:23:36
IMO, you have to work with the format's quirks, unless you want to fork it. That means option #3. The encoder didn't have piping support to begin with, it's not like you're *losing* functionality.

Thanks everyone for the input.  I agree with the above statement, so that's what I went with.  The checksum is only recalculated if the WAVE header is updated for any reason (e.g. unknown size piped from foobar, truncated WAVE file on disk, etc.).  It's a one-time cost that is worth it, IMHO.

You can find the latest and greatest here (http://www.etree.org/shnutils/shntool/).
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-12 16:44:40
Great job!

"Audio MD5" in foobar's properties is still different for files created with foobar, but this makes sense since the seek table is different.  Correct?
Title: Ape image (+cuesheet) to single ape tracks
Post by: skamp on 2008-03-12 17:51:10
Why bother with an md5sum if it's already built-in?

That was in response to your worry about different encoder (and hashing) versions. In case of fixed media, external md5sum is just as good as internal.
Title: Ape image (+cuesheet) to single ape tracks
Post by: greynol on 2008-03-12 17:57:08
Ok.  The good news is this won't be an issue.

Should we start a new thread to give shnutils's excellent work better visibility?
Title: Ape image (+cuesheet) to single ape tracks
Post by: shnutils on 2008-03-12 18:16:51
"Audio MD5" in foobar's properties is still different for files created with foobar, but this makes sense since the seek table is different.  Correct?

That's correct.  The seek table is created by MAC before encoding even begins, and its size is based on the data size it has at that time.  For files that are truncated (i.e. shorter than the WAVE header initially reports), the resulting seek table will be larger than it needs to be.  Fortunately, that doesn't hurt anything.