HydrogenAudio

Hydrogenaudio Forum => Validated News => Topic started by: Bonzi on 2005-03-05 19:28:58

Title: Open Source ALAC Decoder
Post by: Bonzi on 2005-03-05 19:28:58
Hi, David Hammerton from the ffmpeg mailing list wrote a reverse engineered ALAC decoder. (http://craz.net/programs/itunes/alac.html)

Quote
Located here is a basic decoder for Apple Lossless Audio Codec files (ALAC). ALAC is a proprietary lossless audio compression scheme. Apple never released any documents on the format.
What I provide here is a C implementation of a decoder, written from reverse engineering the file format. It turns out that most of the algorithms in the codec are fairly well known. ALAC uses an adaptive FIR prediction algorithm and stores the error values using a modified rice or golumb algorithm. Further details are in alac.c.

Although an encoder is not provided, by using the decoder as a sort of specification it should be fairly trivial to write an encoder. By exploiting other lossless audio encoders, such as FLAC, the task will be much easier. Although one wouldn't be able to copy the compression algorithms verbatim, as adaptive compression is used in ALAC and not in FLAC. There are, however, a bunch of academic papers on the issue.

The program located here will not be able to handle all ALAC files, it can only handle mono or stereo files. ALAC allows up to 8 channels. It should be trivial to finish the implementation once I find files that I can test it with. Likewise the decoder only supports 16bit sample sizes. Again, it should be trivial to fix.

The decoder is fairly self explanatory, it can read an ALAC stream from either a file or from stdin, and write it as raw PCM data or as a WAV file to either stdout or a file. In theory one should be able to stream data to the decoder.


I uploaded a binary here. (http://www.hydrogenaudio.org/forums/index.php?act=Attach&type=post&id=1380)  Not sure about the legality of this, I will remove if necessary.
Title: Open Source ALAC Decoder
Post by: jcoalson on 2005-03-05 20:26:29
very cool.  I was about half-way done reverse engineering it last year, but I stopped when it seemed unlikely that ALAC could be converted to FLAC without reencoding.  but they are very close.

apple would be crazy to go after this though, as this will help adoption of the format.

Josh
Title: Open Source ALAC Decoder
Post by: guruboolez on 2005-03-05 21:34:47
Very nice new
I suppose that foobar2000, winamp or DirectShow filters are now possible. Am I right?
Title: Open Source ALAC Decoder
Post by: Bonzi on 2005-03-05 21:50:11
I should say that the binary that I uploaded is just for testing purposes *only*.  I am not sure if the output is indeed lossless.  I really didn't test it much.
Title: Open Source ALAC Decoder
Post by: guruboolez on 2005-03-05 23:09:14
Quote
I should say that the binary that I uploaded is just for testing purposes *only*.  I am not sure if the output is indeed lossless.  I really didn't test it much.
[a href="index.php?act=findpost&pid=279582"][{POST_SNAPBACK}][/a]


I've tested with one big file, encoded in m4a ALAC, and decoded with the 'hacked' decoder. Sound is great, but compared to iTunes decoder bass are not as powerfull, some details are slightly less precise and trebles are also a bit...  no, I'm joking of course

I did a bit-to-bit comparison, and foobar2000's conclusions are:
Code: [Select]
INFO (foo_bitcompare) : Comparing:
INFO (foo_bitcompare) : location: "file://C:\2005.02.12\Murail, Tristan (1947-)\Serendib - L'esprit des dunes - Désintégrations\02. L'esprit des dunes • 1993-1994 - pour ensemble et électronique.wv" (0)
INFO (foo_bitcompare) : location: "file://C:\cmd\dunes.wav" (0)
INFO (foo_bitcompare) : No differences in decoded data found.
INFO (foo_bitcompare) : Finished successfully.


Seems to work perfectly, at least on this full track.
Title: Open Source ALAC Decoder
Post by: rjamorim on 2005-03-05 23:27:21
  Amazing shit.

/me goes to change the lossless comparision table.
Title: Open Source ALAC Decoder
Post by: guest0101 on 2005-03-05 23:35:15
Quote
  Amazing shit.

/me goes to change the lossless comparision table.
[a href="index.php?act=findpost&pid=279600"][{POST_SNAPBACK}][/a]


Anybody have a MAC OS X binary of this yet for testing? I would but don't to know to make one.
Title: Open Source ALAC Decoder
Post by: saratoga on 2005-03-06 01:37:33
I think under MacOS you could just unzip it, browse to the folder from the command line and type 'make'.  Though my unix skills are subpar.
Title: Open Source ALAC Decoder
Post by: svkelley on 2005-03-06 02:26:37
It would be great to also extract the tags and then be able to convert a library of ALAC into FLAC, without having to type in all the album information again.

Sean
Title: Open Source ALAC Decoder
Post by: guest0101 on 2005-03-06 02:42:01
Quote
I think under MacOS you could just unzip it, browse to the folder from the command line and type 'make'.  Though my unix skills are subpar.
[a href="index.php?act=findpost&pid=279613"][{POST_SNAPBACK}][/a]


I tried that and it couldn't find any program named 'make' in Mac OS X. Anybody else know how to compile this for Mac?
Title: Open Source ALAC Decoder
Post by: Dibrom on 2005-03-06 03:00:18
Quote
Quote
I think under MacOS you could just unzip it, browse to the folder from the command line and type 'make'.  Though my unix skills are subpar.
[a href="index.php?act=findpost&pid=279613"][{POST_SNAPBACK}][/a]


I tried that and it couldn't find any program named 'make' in Mac OS X. Anybody else know how to compile this for Mac?
[a href="index.php?act=findpost&pid=279626"][{POST_SNAPBACK}][/a]


You need to have the development tools installed.  If you install XCode you'll get gcc, make, and all the other stuff necessary.

I just tried compiling this on OS X (just type "make" as said) and it at least compiles with no problem.  I haven't tried the program on any input files though.
Title: Open Source ALAC Decoder
Post by: guest0101 on 2005-03-06 04:28:16
Quote
You need to have the development tools installed.  If you install XCode you'll get gcc, make, and all the other stuff necessary.

I just tried compiling this on OS X (just type "make" as said) and it at least compiles with no problem.  I haven't tried the program on any input files though.
[a href="index.php?act=findpost&pid=279633"][{POST_SNAPBACK}][/a]


Thanks for the info Dibrom. Would it be possible to post the Mac OS X binary on HA or else have it posted on Rarewares? Thanks.
Title: Open Source ALAC Decoder
Post by: JEN on 2005-03-06 08:55:29
Quote
  Amazing shit.

/me goes to change the lossless comparision table.
[a href="index.php?act=findpost&pid=279600"][{POST_SNAPBACK}][/a]


Hi

When will this comparision table be ready?  Will you let us know when its been modifies. 

Thanks
Title: Open Source ALAC Decoder
Post by: ffooky on 2005-03-06 11:56:05
On my 1.25 gHz/ 768MB  eMac it seems significantly slower than iTunes:

    WAV -------- M4A ------ iTunes ---- alac

  59.1 MB --- 32.3 MB --- 10 sec --- 16 sec 
  73.3 MB --- 41.1 MB --- 14 sec --- 21 sec 
  67.9 MB --- 41.2 MB --- 15 sec --- 30 sec
146.0 MB --- 76.3 MB --- 23 sec --- 41 sec
244.2 MB --- 140.5 MB --- 46 sec --- 90 sec
Title: Open Source ALAC Decoder
Post by: Latexxx on 2005-03-06 12:01:26
This hasn't been optimised in any way. So it isn't any kind of a surprise.
Title: Open Source ALAC Decoder
Post by: rjamorim on 2005-03-06 15:09:49
Quote
When will this comparision table be ready?  Will you let us know when its been modifies.  [{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=279679")


It's been ready for months

[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=29655]http://www.hydrogenaudio.org/forums/index....showtopic=29655[/url]

And I post at that thread whenever there is a modification to the first post.


@Bonzi: can I host your compile at RareWares?
Title: Open Source ALAC Decoder
Post by: JEN on 2005-03-06 15:35:03
Quote
Quote
When will this comparision table be ready?  Will you let us know when its been modifies.  [{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=279679")


It's been ready for months

[a href="http://www.hydrogenaudio.org/forums/index.php?showtopic=29655]http://www.hydrogenaudio.org/forums/index....showtopic=29655[/url]

And I post at that thread whenever there is a modification to the first post.


@Bonzi: can I host your compile at RareWares?
[a href="index.php?act=findpost&pid=279729"][{POST_SNAPBACK}][/a]


Oh, that means I've been overlooking it! 
Thanks for setting things straight.
Title: Open Source ALAC Decoder
Post by: krmathis on 2005-03-06 16:28:56
Quote
Anybody have a MAC OS X binary of this yet for testing? I would but don't to know to make one.

I've uploaded a binary here (http://www.hydrogenaudio.org/forums/index.php?showtopic=32131).
Title: Open Source ALAC Decoder
Post by: rjamorim on 2005-03-06 16:34:24
Quote
I've uploaded a binary here (http://www.hydrogenaudio.org/forums/index.php?showtopic=32131).
[a href="index.php?act=findpost&pid=279755"][{POST_SNAPBACK}][/a]


Sweet. Can I host it a RareWares?
Title: Open Source ALAC Decoder
Post by: krmathis on 2005-03-06 17:08:09
Quote
Sweet. Can I host it a RareWares?
[a href="index.php?act=findpost&pid=279762"][{POST_SNAPBACK}][/a]
Off course! 
Title: Open Source ALAC Decoder
Post by: jcoalson on 2005-03-06 20:24:38
just got finished looking over the decoder source... first, that is one fantastic feat of reverse engineering.

second, ALAC does not resemble MPEG4-ALS, at least not the MPEG4-ALS I have read about.  some differences:

- ALAC transmits FIR coeffs directly w/linear quantization, ALS transmits arcsin transform of reflection coeffs
- ALAC adapts the FIR coeffs while ALS appears not to
- ALAC has more complicated stereo decorrelation
- ALAC adapts the rice parameter while coding the residual, ALS uses 1 or 4 fixed parameters per block.  ALS also has a mode which mixes rice and block codes
- ALAC has only 8 channels vs ALS' 256.

the ALAC design is similar to FLAC:

- first stage: optional interchannel decorrelation
- second stage: FIR interchannel decorrelation
- - (both have optional verbatim (http://flac.sourceforge.net/format.html#prediction) coding)
- - (both have optional fixed low-order predictors)
- third stage: residue coding with rice codes
- also, before second stage, FLAC has a 'wasted-bits' step which might have an ALAC analogue, it is hard to tell from the source

some differences:

1. interchannel decorrelation can use a linear combination of mid and side channels whereas FLAC computes only mid and side channels
2. ALAC FIR decorrelation adapts based on the sign of some measurement (have to look into that more)
3. rice parameter adapts, where FLAC uses precomputed parameters that are also transmitted

from those differences we can infer some things.  first, the decode complexity is higher that FLAC: there is an extra multiply per sample because of 1), 2) causes a few extra adds per sample times predictor order at least, and 3) is also more complex by an amount I haven't really quantified yet.  it is now clear also that the high decode speed on apple hardware is due to significant PPC optimization.

so it looks like apple did not make ALAC because FLAC's decode complexity was too high.

also, even with all these "improvements" the compression ratio of ALAC is similar to (but seems to average slightly lower) than FLAC.  so it wasn't made to blow FLAC out of the water on compression.

what's left?  either they just wanted something proprietary (well that didn't last long) or it has some unknown advantage on hi-res audio that isn't supported by the encoder yet.

Josh
Title: Open Source ALAC Decoder
Post by: AtaqueEG on 2005-03-06 20:53:06
Do you think we will see software implementation (ie: foobar2000) anytime soon?

Is there a brave a kind soul here that could take care of that?
Title: Open Source ALAC Decoder
Post by: Tec9SD on 2005-03-07 01:34:33
Quote
Very nice new
I suppose that foobar2000, winamp or DirectShow filters are now possible. Am I right?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=279579")
From [a href="http://ffmpeg.sourceforge.net/index.php]FFmpeg[/url]
Quote
(Mar 6, 2005) FFmpeg now includes a native decoder for Apple's Lossless Audio Codec (a.k.a. 'alac'). As usual, check out CVS.
Given the fact that FFDShow uses libavcodec, I would be surprised if a version of FFDShow supporting it were not available soon. Perhaps in FFDShow - Daily builds (http://ffdshow.sourceforge.net/tikiwiki/tiki-index.php?page=Getting+ffdshow).

tec

p.s. guru: hilarious about it sounding differently.
Title: Open Source ALAC Decoder
Post by: yong on 2005-03-07 10:42:02
Quote
Quote
Very nice new
I suppose that foobar2000, winamp or DirectShow filters are now possible. Am I right?
[{POST_SNAPBACK}][/a] (http://index.php?act=findpost&pid=279579")
From [a href="http://ffmpeg.sourceforge.net/index.php]FFmpeg[/url]
Quote
(Mar 6, 2005) FFmpeg now includes a native decoder for Apple's Lossless Audio Codec (a.k.a. 'alac'). As usual, check out CVS.
Given the fact that FFDShow uses libavcodec, I would be surprised if a version of FFDShow supporting it were not available soon. Perhaps in FFDShow - Daily builds (http://ffdshow.sourceforge.net/tikiwiki/tiki-index.php?page=Getting+ffdshow).

tec

p.s. guru: hilarious about it sounding differently.
[a href="index.php?act=findpost&pid=279924"][{POST_SNAPBACK}][/a]

It would be nice if ffdshow can support it, but you need a mp4 parser that can "recognize" the ALAC.
Title: Open Source ALAC Decoder
Post by: Tang on 2005-03-07 10:50:05
Quote
just got finished looking over the decoder source... first, that is one fantastic feat of reverse engineering.

(...)

so it looks like apple did not make ALAC because FLAC's decode complexity was too high.

also, even with all these "improvements" the compression ratio of ALAC is similar to (but seems to average slightly lower) than FLAC.  so it wasn't made to blow FLAC out of the water on compression.

what's left?  either they just wanted something proprietary (well that didn't last long) or it has some unknown advantage on hi-res audio that isn't supported by the encoder yet.

Josh
[a href="index.php?act=findpost&pid=279843"][{POST_SNAPBACK}][/a]
Very interesting feedback josh thanks...
Anyway the "unmotived" proprietary choice didn't surprised me too much from Apple...
Title: Open Source ALAC Decoder
Post by: Busemann on 2005-03-07 11:43:54
Quote
what's left?  either they just wanted something proprietary (well that didn't last long) or it has some unknown advantage on hi-res audio that isn't supported by the encoder yet.


Well, we already know that it will support 24bit multichannel encoding in QT 7..
Title: Open Source ALAC Decoder
Post by: guest0101 on 2005-03-07 14:39:07
Well it appears that news sources are quoting/metioning this particular thread here on HA and the Slashdot article about this new open source ALAC decoder. Very interesting to see this getting picked up in the computer news sites:

MacObserver.com article:
http://www.macobserver.com/article/2005/03/07.2.shtml (http://www.macobserver.com/article/2005/03/07.2.shtml)
Title: Open Source ALAC Decoder
Post by: jcoalson on 2005-03-07 15:45:22
Quote
Well, we already know that it will support 24bit multichannel encoding in QT 7..[a href="index.php?act=findpost&pid=280054"][{POST_SNAPBACK}][/a]

FLAC has 24bit multichannel too, but the question is whether ALAC's compression will be significantly better.  have to wait and see.

Josh
Title: Open Source ALAC Decoder
Post by: guest0101 on 2005-03-07 17:09:09
Uh, oh, now the news outlets just released a story portraying this Apple Lossless decoder as "Cracking iTunes". I can't believe it. You'd think he was breaking .m4p DRM or something.

Article:  Developer cracks iTunes encoding format

http://www.pcpro.co.uk/news/70103/develope...ing-format.html (http://www.pcpro.co.uk/news/70103/developer-cracks-itunes-encoding-format.html)

also

a story on MacDailyNews.com:
http://macdailynews.com/index.php/weblog/comments/5171/ (http://macdailynews.com/index.php/weblog/comments/5171/)
Title: Open Source ALAC Decoder
Post by: xmixahlx on 2005-03-07 20:17:11
this wasn't not mentioned here, and i thought it should be...

the developer does amazing work, also writing libopendaap + tunesbrowser:
http://craz.net/programs/itunes/ (http://craz.net/programs/itunes/)


later
Title: Open Source ALAC Decoder
Post by: yong on 2005-04-02 10:38:23
Did someone know?
Mplayer(Daily CVS version) now is able to playback ALAC, 
but still have bug(audio distorsion) haven't fix yet

EDIT:Sorry, the mplayer i used is outdated...(20050322),
May be this bug already fixed at new daily cvs version.
Title: Open Source ALAC Decoder
Post by: keytotime on 2006-10-26 01:13:29
Could somebody please compile the latest version found at http://craz.net/programs/itunes/alac.html (http://craz.net/programs/itunes/alac.html).
Title: Open Source ALAC Decoder
Post by: kurtnoise on 2006-10-26 09:19:47
For windows I guess ?

http://kurtnoise.free.fr/index.php?dir=mp4...coder-0.1.3.zip (http://kurtnoise.free.fr/index.php?dir=mp4tools/&file=alac_decoder-0.1.3.zip) or here (http://kurtnoise.free.fr/mp4tools/alac_decoder-0.1.3.zip).
Title: Open Source ALAC Decoder
Post by: john33 on 2006-10-26 09:48:55
Updated Win32 compiles at Rarewares Lossless page now.
Title: Open Source ALAC Decoder
Post by: keytotime on 2006-10-26 12:19:52
Wow That was Fast. Thank You.