HydrogenAudio

Lossless Audio Compression => Lossless / Other Codecs => Topic started by: Noble Tiger on 2010-10-06 06:52:47

Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Noble Tiger on 2010-10-06 06:52:47
Hi everyone,

I'm using iTunes 10.0.1.22 on Windows Vista Home Premium SP1 64-bit.

I noticed a potential bug in the Apple Lossless Codec in iTunes 10 where if you convert a WAV or AIFF file to Apple Lossless format in iTunes, then convert it back to it's original file type (WAV or AIFF respectively) in iTunes, the resulting files are not the same size in bytes or in time length as the original WAV or AIFF files. The files are usually 5KB or so smaller and can be about 1 second shorter than the original files. I did extensive testing with this using various WAV and AIFF files both generated by iTunes (e.g. ripping a CD track) and by other programs and every time the Apple Lossless codec in iTunes does not convert back to the original file's length or file size. Apple Lossless should convert back exactly, but it isn't. In previous versions of iTunes it did, but not in 10.0.1.22.  This may also effect the accuracy Apple Lossess (ALAC) file playback in iTunes and conversion to other formats within iTunes as well.

I think it's a bug, but can anyone else reproduce this on their systems? or is it just on my system?

To test in iTunes:

Convert a WAV to Apple Lossess then back to WAV and compare the two WAV's
Convert an AIFF to Apple Lossless then back to AIFF and compare the two AIFF's
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Engelsstaub on 2010-10-06 07:44:43
I just tried this and the file sizes are exactly the same (Win7 64.)

I didn't actually rip the track with iTunes (same version as yours,) but I decompressed a FLAC file to WAV with dBpoweramp. From thence I copied it into the iTunes library and created an Apple Lossless version from within. I decompressed that (ALAC) file within Explorer to WAV with dBpa (I'm not aware of any way to do this from within iTunes.)

My file-sizes were exactly the same. Ditto for the track-lengths. I don't know what to tell you. Maybe I should try ripping to WAV in iTunes? I never use the iTunes ripper anymore nor do I do conversions within the app. I use dBpoweramp and EAC for all of that.

IDK... I'll try it with a CD in the iTunes ripper to WAV and get back to you.

(EDIT) I was able to reproduce your error once I figured out how to switch the settings between "Create Apple Lossless version" and "Create WAV version." (I never saw the option to convert to WAV on left-click because I never had it set to rip directly to WAV.)

The original file/song was 41.48 Mb 4 min. 6 sec. for the original wav and 41.47 Mb 4 min. 6 sec. for the one converted to ALAC and back to WAV within iTunes. That IS strange...

...so it's not just your system.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: greynol on 2010-10-06 08:36:39
The files are usually 5KB or so smaller and can be about 1 second shorter than the original files.

5KB is less than three hundredths of a second of CD-quality audio stored as raw PCM data (read: WAV or AIFF).

Convert a WAV to Apple Lossess then back to WAV and compare the two WAV's
Convert an AIFF to Apple Lossless then back to AIFF and compare the two AIFF's

Please provide two 30 second (or less) clips of a wave file: one before and one after conversion.  Post them to our uploads sub-forum:
http://www.hydrogenaudio.org/forums/index.php?showforum=35 (http://www.hydrogenaudio.org/forums/index.php?showforum=35)
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: greynol on 2010-10-06 08:45:47
The original file/song was 41.48 Mb 4 min. 6 sec. for the original wav and 41.47 Mb 4 min. 6 sec. for the one converted to ALAC and back to WAV within iTunes. That IS strange...

These methods are far too crude to determine whether a conversion is lossless.  Because a wave file is nothing more than a container there can be differences in file size which will not affect the audio data in any way whatsoever.  Seeing that you are using Windows, I know of a few options to bit-compare the raw PCM data such as foobar2000 or EAC.  Similarly, you can generate md5 hashes of the PCM data using shntool at the command line and compare them.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Engelsstaub on 2010-10-06 09:15:17
I didn't know that. I just assumed that the file size would be the same if the data was the same within the container. I'm as terrified of the command line as I am of editing the registry; I'll only do it if I follow a tutorial verbatim. (I've only had a personal computer outside of work for about four years.)

I do have EAC and a portable Foobar 2000. I can give it a try just for the sake of learning something  It's just "Compare WAVs" in EAC, right? I'll have a go at it tomorrow and get back to this...
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Robertina on 2010-10-06 09:19:19
I do have EAC and a portable Foobar 2000. I can give it a try just for the sake of learning something  

For foobar2000 you would need Peter's Binary Comparator (http://www.foobar2000.org/components/view/foo_bitcompare). After installation select the two files in question and choose from foobar's context menu -> Utilities / Bit-Compare tracks.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Engelsstaub on 2010-10-06 10:36:06
This is what I got with F2K (and now I know how to do something new...so thanks for that, Robertina.)

Code: [Select]
Differences found in 1 out of 1 track pairs.

Comparing:
"...\Desktop\02 Cross My Heart And Hope To Die.wav"
"...\Desktop\original.wav"
Length mismatch : 4:06.503039 vs 4:06.573333, 10870784 vs 10873884 samples


(I modified the log to exclude the full directory and changed the file name of the original WAV ripped with iTunes so that I could differentiate between the two.)


EAC:

(http://img24.imageshack.us/img24/7535/captureyed.jpg) (http://img24.imageshack.us/i/captureyed.jpg/)

Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: andy o on 2010-10-06 14:29:17
This is what I got with F2K (and now I know how to do something new...so thanks for that, Robertina.)

So what about the first WAV conversion you made without iTunes? Seems to suggest the "problem" is in iTunes putting the audio in a wav container, not of the ALAC encoder.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: mixminus1 on 2010-10-06 15:18:55
Confirmed under Windows XP SP3: a 52 second WAV converted to ALAC in iTunes 10.0.1.22 and then converted back to *either* WAV or AAC and then WAV in iTunes is 2,570 samples shorter (WAVs checked in foobar2000).

I also tried the AAC->WAV conversion in fb2k and the result was identical.

Since Englesstaub doesn't experience this issue when converting from iTunes-compressed ALAC to WAV using dBPA, it would indeed appear to be an issue with the ALAC *de*coder in iTunes 10.0.1.22.

I won't have access to my Mac until later this evening - anybody on a Mac before then that can try to replicate this under OS X?
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: twostar on 2010-10-06 15:36:35
Same thing here using iTunes version 10.0.0.68 and Win7 Starter.

Here's the log from foobar2000's bit comparator.

Code: [Select]
Differences found in 1 out of 1 track pairs.

Comparing:
"C:\Users\ac25\Desktop\Foo Fighters - Greatest Hits\itunes.wav"
"C:\Users\ac25\Desktop\Foo Fighters - Greatest Hits\orig.wav"
Length mismatch : 2:14.304218 vs 2:14.306667, 5922816 vs 5922924 samples
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: mixminus1 on 2010-10-06 15:58:59
It's also interesting that there doesn't seem to be any consistency with regards to the length of the truncation - files of different lengths are producing different percentages of truncated samples.

Just for the helluvit, I created four WAVs in Cool Edit (1 kHz tone) of exactly 30, 60, 90, and 120 seconds in length - here are the number of truncated samples for each WAV:

- 30 sec: 4088
- 60 sec: 4080
- 90 sec: 4072
- 120 sec: 4064

Interesting, no?  A WAV cut randomly at around 1:48 lost 1,400 samples.

Very odd bug...

edit: grammar
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: tuffy on 2010-10-06 16:05:13
It's worth noting that the default block size for ALAC is 4096 samples.  So assuming you're using 44100Hz:

(44100 * 30) % 4096 = 4088
(44100 * 60) % 4096 = 4080
(44100 * 90) % 4096 = 4072
(44100 * 120) % 4096 = 4064

So it looks like you're losing exactly 1 ALAC frame each time for whatever reason.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: twostar on 2010-10-06 16:14:46
I wonder if other pre-iTunes 10 versions are affected as well.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: mixminus1 on 2010-10-06 16:39:16
It's worth noting that the default block size for ALAC is 4096 samples.  So assuming you're using 44100Hz:

(44100 * 30) % 4096 = 4088
(44100 * 60) % 4096 = 4080
(44100 * 90) % 4096 = 4072
(44100 * 120) % 4096 = 4064

So it looks like you're losing exactly 1 ALAC frame each time for whatever reason.

Ah, that's right, 4096, just like FLAC (@ 44.1 kHz).

Yep, looks like you nailed it:  doing a modulo 4096 on all the file lengths posted thus far checks out perfectly, so it looks like the ALAC decoder is indeed truncating one full frame.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Corby on 2010-10-06 17:14:21
Same on Mac version of iTunes.
Foobar (running on Wine): "Length mismatch : 4:15.626667 vs 4:15.605261, 11273136 vs 11272192 samples"
No differences when comparing original WAV and ALAC M4A.

Edit: no differences when decoding ALAC M4A to WAV using XLD. So it must be something somewhere inside iTunes WAV handling...
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Noble Tiger on 2010-10-06 19:19:20
It looks like it is a bug in iTunes 10.  I didn't notice this in iTunes 9 but I'm also not 100% sure that I tested it thoroughly.

I also can convert the ALAC file back to WAV accurately using dBpoweramp Music Converter, so it must be the iTunes ALAC decoder that's the problem.  This problem also occurs with AIFF files converted to ALAC and back, and I would theorize that this would also affect conversion of ALAC to AAC or ALAC to MP3 as well within iTunes.  Which is good to be aware of.

You'll notice that sometimes when you convert the WAV or AIFF to ALAC in iTunes 10, if you take a look at the time length for the files, the ALAC file will often be listed as 1 second shorter than the WAV or AIFF (but not always), so that could mean that even playback of ALAC in iTunes 10 could be truncated by a full frame at the end (If this bug is related to the ALAC decoder).

Most likely this problem is related to Quicktime I would guess, as Quicktime does the background encoding/decoding for iTunes doesn't it?  Can anyone duplicate this problem using Quicktime only? (I don't have Quicktime Pro myself or I'd try it).

I've reported the bug to Apple and I hope they release an update to either iTunes or Quicktime soon to address this.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: C.R.Helmrich on 2010-10-06 20:09:27
Just for the helluvit, I created four WAVs in Cool Edit (1 kHz tone) of exactly 30, 60, 90, and 120 seconds in length - ...

Interesting indeed. Just for the helluvit  , would you mind trying the same thing with an 11.61 second long 44.1-kHz file? That's 125*4096+1, so there should be 1 sample missing in the decoded WAV.

Chris
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: mixminus1 on 2010-10-06 20:28:54
Et viola:

Code: [Select]
Differences found in 1 out of 1 track pairs.

Comparing:
"C:\...\tone.1kHz.512001_samples.wav"
"C:\...\tone.1kHz.512001_samples_ALAC.wav"
Length mismatch : 0:11.610000 vs 0:11.609977, 512001 vs 512000 samples
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Engelsstaub on 2010-10-07 05:50:44
This is what I got with F2K (and now I know how to do something new...so thanks for that, Robertina.)

So what about the first WAV conversion you made without iTunes? Seems to suggest the "problem" is in iTunes putting the audio in a wav container, not of the ALAC encoder.


I would have to agree.

(...but my knowledge is limited on much of this and I guess I've been making a few assumptions until now. I would have never came across this "problem" had the OP not brought it up. That's the thing about not being a programmer or even working with a CLI once in a while: everything is GUI and I just trust the software I come to like. I always thought of iTunes (however "Apple" or bloated) as a decent ripper and such.

It would never occur to me to go testing the integrity of a dBpoweramp conversion. I hold Spoon's software in the highest esteem.)

...at this point I've become more a "follower" of the thread as you guys appear a bit more technically adept on its subject matter. 


Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: greynol on 2010-10-07 05:54:43
The problem with 3rd party encoders/decoders is that ALAC is proprietary and has to be reverse-engineered.

That Apple has botched their very own decoder is quite disconcerting!
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Engelsstaub on 2010-10-07 06:34:12
The problem with 3rd party encoders/decoders is that ALAC is proprietary and has to be reverse-engineered.

That Apple has botched their very own decoder is quite disconcerting!


Agreed. I was thinking the exact same thing. My last post was getting long-winded enough though.

I don't know why they don't just open the source-code. They could stipulate that it must carry the name Apple Lossless or an acronym like ALAC to make a proper attribution to it's original developers. They don't make a dime off of the format that I'm aware of. And as you've said, it seems now the reverse-engineers know more about what they're doing with it then they do.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: greynol on 2010-10-07 06:42:10
Well, no, I'm not ruling out the possibility that there aren't problems with 3rd party implementations of Apple lossless; rather, I simply steer clear of it.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Corby on 2010-10-07 06:46:34
The problem with 3rd party encoders/decoders is that ALAC is proprietary and has to be reverse-engineered.

That Apple has botched their very own decoder is quite disconcerting!


XLD is also using Apple's decoder and writes the WAVs that are identical to original WAV. That probably means that decoder is just fine, and that bug is in iTunes WAV handling routines.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Engelsstaub on 2010-10-07 06:52:51
If I didn't currently have an iPod I'd likely steer clear of it as well. It has no real practicality for anything other than Apple software and the iPod.

For a while I thought I was being silly about archiving CDs in FLAC and converting some to ALAC for iTunes/iPod. It appears I had a good reason for retaining the FLACs anyway, although I didn't exactly know it at the time. 

We always hear people say things like "lossless is lossless." This may be a good lesson in why we should consider one format "better" than another for reasons that were demonstrated here. For me, it just makes me an even bigger fan of FLAC and it's developers.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: greynol on 2010-10-07 06:57:59
I'm part of the camp that thinks lossless on portables is silly.

@Corby, apologies for prompting you to repeat yourself.  I wasn't following along carefully enough.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Engelsstaub on 2010-10-07 07:26:11
Yeah, I'm sorry too, Corby. (I think I was typing my stuff when you posted. I should have quoted greynol in my post. I wasn't blowing you off.) You're probably right: I've actually "tested" ALAC files with the original EAC/FLAC cue sheets after I converted them (with dBpoweramp) and put them in the iTunes library. When decompressed to WAV the CD-R I burned matched my previous rips.

@greynol: I've read you before opining that using lossless on portables was silly. I meant to pick your brain on that but never got around to it. I guess I just do it because I can (160 Gb iPod.) Not because I claim  there's some perceptible difference or anything like that. It also makes just another redundant backup of some CDs IMO. ...but I'm interested in your ideas on this as well.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: greynol on 2010-10-07 07:46:04
Briefly and not to belabor the point or change the topic I have no objection to those who can demonstrate that they can tell the difference through double-blind testing and are not willing to compromise this difference for increased space.  I see your point as well; 160GB can store a lot of music, even as lossless, and backups are important.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: vitos on 2010-10-08 15:02:33
I reported this bug to Apple.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: higherlove on 2010-10-08 21:27:35
I tried ALAC -> AAC(iTunes Plus).  Sample omission occurs.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: dv1989 on 2010-10-30 10:40:14
Noble Tiger, I don’t suppose this bug has been fixed in the latest update (that you posted about in the iTunes 10 topic)?
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Noble Tiger on 2010-11-02 04:46:15
No, unfortunately Apple hasn't released a new update to iTunes since I started this thread.  Version 10.0.1.22 came out in late September.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Noble Tiger on 2010-11-13 21:32:17
BUG FIXED!

It seems this bug has been fixed with the recent release of iTunes 10.1.0.54.  I just did some tests with WAV to Apple Lossless and back to WAV and  the files are identical in size and binary file comparisons (Using FC.EXE that comes with Windows).  I'm on a Vista 64-bit PC.  Can others confirm this is fixed on other platforms?  Also, can others check if this is fixed with Apple Lossless to AAC as well? (That it doesn't lose a frame)?

Hopefullly all will be well again now in iTunes Losslessland 
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: mixminus1 on 2010-11-14 02:37:02
Fix confirmed under OS X 10.6.5, iTunes 10.1.54 - ALAC encoded and decoded in iTunes byte-identical to original WAV (using cmp command in Terminal).
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Engelsstaub on 2010-11-14 03:30:52
I'm glad they fixed this issue. It was making them look pretty bad, IMO.

(It's too bad they can't "fix" the 32Mb buffer/cache in my 160 Gb iPod Classic too. Then I could actually use ALAC (or WAV/AIFF) without the annoying buffer-pause in nearly every song.)
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: kawaiigardiner on 2010-12-01 14:46:31
One of the very reasons I just use iTunes as a media manager - I tend to take an approach that a tool should do a couple of things really well than trying to be everything to everyone. iTunes has turned into this giant monster which is collapsing under its own weight so instead I'll use iTunes to manage my music and sync with my iPod with XLD being used for ripping and encoding.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: RandyOsbourne on 2011-01-15 01:43:36
The problem I see in iTunes is a 96khz-24 bit wav file converted to apple lossles (.m4a) ends up being a 44.khz-16 bit file!
No, no no!
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: mixminus1 on 2011-01-15 02:13:15
Not reproducible using iTunes 10.1.1 under OS X 10.6.6 - 24/96 WAV in (generated in Audacity) equals 24/96 ALAC out.

It is worth noting, however, that iTunes will not convert that ALAC back to a 24-bit WAV, only 16-bit (it does keep the 96 kHz sample rate, however).

Converting the 24/96 ALAC back to a 24/96 WAV using XLD produces a file that's bit-for-bit identical to the original.
Title: Potential serious bug in Apple Lossless codec in iTunes 10
Post by: Engelsstaub on 2011-01-15 05:49:45
The problem I see in iTunes is a 96khz-24 bit wav file converted to apple lossles (.m4a) ends up being a 44.khz-16 bit file!
No, no no!


I just converted a 24/96 FLAC file to ALAC with dBpowerAMP and added it to iTunes Library. It's still 24/96. If iTunes is doing that with WAV/PCM perhaps you should try doing it as I described?