Skip to main content

Topic: Apple Lossless Audio Codec is now open source (Apache license) (Read 63228 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • tuffy
  • [*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #75

I've found out that Apple's ALAC encoder is extremely fast encoder. I compiled Apple ALAC with Intel C++ max optimized. I'll do better ALAC vs. FLAC comparison later.

The Apple's reference ALAC encoder reads/writes the files in too short blocks and it slows down dramatically its performance.

Apple´s default compression must be pretty lousy likle between -3 and -4 for flac

FLAC has a lot of tunable parameters in each frame.  At the higher compression presets, it tries every combination of parameters per frame and writes out the smallest frame it can find.  This is why it parallelizes really well.  An encoder can pass off all those searches to their own threads and run them all simultaneously.

ALAC, on the other hand, has very few tunable parameters.  Files compatible with iTunes support only a handful of channel correlation possibilities, and LPC parameter counts of 4 or 8.  Everything else is fixed, relying on its adaptiveness to take up the slack I suppose.  So because it's not trying out so many options, it compresses relatively quickly.  But that also means it can't compress as well as FLAC.  And that same adaptiveness means its can't decompress as quickly as FLAC.

  • Destroid
  • [*][*][*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #76
So now the capsule summary of ALAC appears to be faster but weaker compression and not as easily decoded compared to FLAC (I guess from a -5 perspective).

Maybe some more tests can show its how well ALAC deals with mono vs. stereo as well as the single instruments vs. CD albums and that I usually try, but the question is what kind of binaries are there to try?
"Something bothering you, Mister Spock?"

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Apple Lossless Audio Codec is now open source (Apache license)
Reply #77
Some time ago I tested several lossless encoders:

FLAC 1.2.1: 187x realtime at -4 compression option, 147x at -5.
iTunes x64 10.5: ~100x realtime
refalac 0.04: 77x realtime (and 130x in fast mode)
FFmpeg ALAC: 124x realtime

So current ALAC encoders are slower than FLAC.

  • Wombat
  • [*][*][*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #78
So now the capsule summary of ALAC appears to be faster but weaker compression and not as easily decoded compared to FLAC (I guess from a -5 perspective).

Maybe some more tests can show its how well ALAC deals with mono vs. stereo as well as the single instruments vs. CD albums and that I usually try, but the question is what kind of binaries are there to try?

ALAC isn´t faster as flac -4 here and compresses worse but i didn´t do exact tests and i won´t because i don´t have much interest in ALAC. Decoding of ALAC is always much slower.
On my system the foobar "Decoding Speed Test" plugin, decoding in RAM is ~150x for ALAC and ~600x for flac -8, maybe someone can check against.

Now Gregory Chudov´s compressors are completely different beasts. His libalac can compress to around the same size or slightly better as flac 1.21 but at an immense speed penalty. 15x on a C2@3400.
Since libalac has the ability to use a compression value from 1 to 10 someone may find a sweet spot.
Modern graphics cards and OS´s should all be capable of using OpenCL and here FlacCL from Gregory can encode ~500x speed with higher compression as flac 1.21. For non-OpenCL systems his libflake archieves the same compression at ~50x speed here.
I think with both compressors Mr. Chudov maxed pretty much out the possible.

Is troll-adiposity coming from feederism?

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Apple Lossless Audio Codec is now open source (Apache license)
Reply #79
Results of my lossless encoding tests:
Encoding speed (x realtime) vs. compression ratio (encoded file size / original file size):


  • Wombat
  • [*][*][*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #80
Nice graph, thanks. To be really complete and fair it should contain Cuetools libflake or FlacCL imho since it has Cueetools ALAC.

Edit: with fair i mean this graph suggests ALAC can compress better as flac but in reality it only does because of the effort of the programmer. It should only be fair to use the corresponding flac encoder coming from the same synapses.
  • Last Edit: 10 November, 2011, 01:44:36 PM by Wombat
Is troll-adiposity coming from feederism?

  • googlebot
  • [*][*][*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #81
So about how much difference in storage costs are we talking here, for a 10000 track collection, with the presented encoding efficiencies? $0.50? The playback ecosystem is much more important.
  • Last Edit: 10 November, 2011, 04:29:06 PM by googlebot

  • Wombat
  • [*][*][*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #82
So about how much difference in storage costs are we talking here, for a 10000 track collection, with the presented encoding efficiencies? $0.50? The playback ecosystem is much more important.

Oh, sorry. i thought if we talk about ALAC we can go into detail. I wonder what hydrogenaudio stands for after all. It isn´t about storage cost i hope.
Is troll-adiposity coming from feederism?

  • bennie
  • [*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #83
Hi all,

First post after lurking for a while  Thanks for this lovely and very helpful community!

Does anyone know the correct value of the ALAC coefficients in order to produce bit-identical ALAC/CAF files with this official tool compared with the ones produced with iTunes? This seems to have a little  impact on filesize as well (i.e. "max coded framesize" is smaller with the official tool). I searched the internet but have not found any documentation about this. Hope someone who is more knowledgeable has some clues?

  • Destroid
  • [*][*][*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #84
@lvqcl - I saw your post in the FB2K 1.1.10 beta thread
foo_input_alac 1.0.7: 121x realtime ALAC decoding speed
fb2k 1.1.10b1: 221x realtime.

and ffmpeg x64 (libavcodec 53.28.0): 167x realtime.

I am surprised at the native ALAC performance being better than originally expected. Any guess to why native ALAC did so well (other than the brilliant FB2K programmers' prowess  )?

edit: found answer to one of my own questions, my bad.
  • Last Edit: 21 November, 2011, 04:26:02 AM by Destroid
"Something bothering you, Mister Spock?"

  • lvqcl
  • [*][*][*][*][*]
  • Developer
Apple Lossless Audio Codec is now open source (Apache license)
Reply #85
BTW, refalac 0.12 decoding speed: 198x realtime.

  • Peter
  • [*][*][*][*][*]
  • Administrator
Apple Lossless Audio Codec is now open source (Apache license)
Reply #86
Sadly this new decoder has multiple security issues:
- Missing input range checks lead to various out of range reads (no more harmful than just crashing the application), some of these might be triggerable on legit undamaged files, since functions such as dyn_get_32bit() will read more bytes from the input buffer than they really care about.
- "Partial frame" feature will happily cause arbitrary number of samples to be read from the file, possibly greater than the caller expects, writing past the buffer allocated by the caller. This one may be exploitable.

At least I see that iTunes itself cannot be too easily crashed with these, I guess they outsource decoding to external processes where these behaviors are acceptable-enough.
  • Last Edit: 22 November, 2011, 03:21:29 AM by Peter

Apple Lossless Audio Codec is now open source (Apache license)
Reply #87
Concerning what Apple had to gain from open sourcing their ALAC code... people finding bugs like this their software is ultimately a benefit to Apple. All software has bugs. Many pieces of software have some potential security vulnerabilities. Getting the public to find these for you for free saves money.
  • Last Edit: 22 November, 2011, 08:09:28 AM by db1989

  • neovibe
  • [*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #88
Hi all,

I am a big user of CUETools + libalac (to convert from FLAC to my apple ecosystem) and although I can't quite keep up to your technical knowledge, I'd like to know if it is foreseen that programs using libalac would move to the code made available by apple.

I am not aware of the differences (regarding the resulting file) but for the sake of future compatibility and general... harmonization, would you all agree that the apple code will be the only one used from here on?

Or is no one even looking into implementing this one instead of the reverse engineered options?

And again, no matter the future, thank you all that have taken the time to make something that makes the life of thousands of people easier without asking nothing in return.
(lame thanks I know, but it's due)

cheers everyone.

  • tuffy
  • [*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #89
I am not aware of the differences (regarding the resulting file) but for the sake of future compatibility and general... harmonization, would you all agree that the apple code will be the only one used from here on?

Or is no one even looking into implementing this one instead of the reverse engineered options?

I doubt anyone would use this code as-is.  It's not like libflac or libwavpack, with lots of documentation and easy to hook into other programs.  Even Apple is probably using a slightly different implementation in iTunes.  But it is useful as a reference to ensure that other encoders and decoders are handling files in a compatible way.  And in lieu of any formal standard, that will have to do.

  • Wombat
  • [*][*][*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #90
Hi all,

I am a big user of CUETools + libalac (to convert from FLAC to my apple ecosystem) and although I can't quite keep up to your technical knowledge, I'd like to know if it is foreseen that programs using libalac would move to the code made available by apple.

I am not aware of the differences (regarding the resulting file) but for the sake of future compatibility and general... harmonization, would you all agree that the apple code will be the only one used from here on?

Or is no one even looking into implementing this one instead of the reverse engineered options?


If i was the coder of libalac and did it with better compression in mind as the apple default i wouldn´t change anything. I am not aware of any non-compliant files it creates.
I may even ask you, as user to find any file that doesn´t work like it should and correct the code. Since this didn´t happen afaik nothing needs to be changed imho.

Help with your usage to improve the existing code with finding problem samples but don´t ask the coder to throw away his code.
Is troll-adiposity coming from feederism?

  • neovibe
  • [*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #91
I may even ask you, as user to find any file that doesn´t work like it should and correct the code. Since this didn´t happen afaik nothing needs to be changed imho.

Help with your usage to improve the existing code with finding problem samples but don´t ask the coder to throw away his code.


I am not proposing anyone to throw away anything. Thats exactly what I'm asking: is there any real technical advantage in using the apple code at all?
  • Last Edit: 09 December, 2011, 09:59:28 AM by neovibe

  • Wombat
  • [*][*][*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #92
I may even ask you, as user to find any file that doesn´t work like it should and correct the code. Since this didn´t happen afaik nothing needs to be changed imho.

Help with your usage to improve the existing code with finding problem samples but don´t ask the coder to throw away his code.


I am not proposing anyone to throw away anything. Thats exactly what I'm asking: is there any real technical advantage in using the apple code at all?

In another thread i already asked Gregory Chudov, the Cuetools developer if there are new insights from this apple code. There aren´t for compressing.
On the decoding side there seems to be a faster implementation used in foobar already.
Is troll-adiposity coming from feederism?

  • Peter
  • [*][*][*][*][*]
  • Administrator
Apple Lossless Audio Codec is now open source (Apache license)
Reply #93
I've published my bug-fixed version of Apple's library:
http://perkele.cc/software/ALAC

  • nu774
  • [*][*][*][*][*]
  • Developer
Apple Lossless Audio Codec is now open source (Apache license)
Reply #94
I've published my bug-fixed version of Apple's library:
http://perkele.cc/software/ALAC

That's nice, thank you for opening it...
Quote
* MSVC/Windows compatibility fixes (MSVC does not like "static inline").

I was just using ugly -D inline=_inline to shut them up (C99 "inline" is not usable in MSVC, but it does support MSVC specific "_inline" keyword for C) .
It seems compiling as C++ (with -Tp) works, too.

By the way, mp4aac.[ch] (not in the original distribution) is included intentionally? Seems like spoontypes.h is missing.

  • Peter
  • [*][*][*][*][*]
  • Administrator
Apple Lossless Audio Codec is now open source (Apache license)
Reply #95
By the way, mp4aac.[ch] (not in the original distribution) is included intentionally? Seems like spoontypes.h is missing.

Oops, it wasn't meant to go in there. Thanks for pointing, fixed now.

  • kode54
  • [*][*][*][*][*]
  • Administrator
Apple Lossless Audio Codec is now open source (Apache license)
Reply #96
To save 5 seconds of Googling, here's a reference article that may be useful for applying that compiler intrinsic change for GCC as well:

How to use MSVC intrinsics to get the equivalent of this gcc code, only in reverse. Handy bit mangling C version included as well, probably faster than doing a for loop over the entire number, especially considering all the branches that would involve.

  • nu774
  • [*][*][*][*][*]
  • Developer
Apple Lossless Audio Codec is now open source (Apache license)
Reply #97
Have found another bug in Apple implementation.
http://alac.macosforge.org/trac/ticket/8

Apparently they are not using this implementation themselves, since encoder of CoreAudioToolbox doesn't suffer from these channel layout bugs, and returns proper magic cookies.

  • jukkap
  • [*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #98
I've published my bug-fixed version of Apple's library:
http://perkele.cc/software/ALAC


That's great work ! Thanks.

(I am now complete-testing the new fixed version)
  • Last Edit: 21 December, 2011, 05:41:59 AM by jukkap

  • k.eight.a
  • [*][*][*][*]
Apple Lossless Audio Codec is now open source (Apache license)
Reply #99
I've read the whole thread. I thought I'd be able to find a command line decoder that benefits from the officialy open-sourced Apple Lossless Audio Codec.
If I understand correctly, so far there is none.
Sorry for my poor English, I'm trying to get better... ;)
"The greatest trick the Devil ever pulled, was convincing the world he didn't exist."