Skip to main content

Topic: Optimized MPEG ALS encoders (Read 32359 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Optimized MPEG ALS encoders
I spent some time with the MPEG ALS reference code, and applied several code and algorithmic optimizations, as well as builds with different compilers. The result seems to be a speedup between 0 and 200%, depending on the exact settings used.

I did not use the included lpc_adapt.o, but instead wrote my own algorithms for adaptive determination of the LPC order. It seems to work well, but it is slightly inferior in ratio compared to the shipped one, but it compensates that a bit by being faster.

If you are playing with ALS the following settings seem interesting from a speed/ratio point of view:

"Fast": -o5
"Default":
"High": -a -o30
"Extra High": -a -b -o50

Settings like -t, -p and -z1, -z2 and z3 don't really provide a seizable gain for large increases in encoding and decoding time. However, for maximal compression -z3 seems to be best.

Binaries can be downloaded at

http://sjeng.org/ftp/lossless/

[Edit: there's a single binary per arechitecture now]
For Intel machines mp4als_intel should be best. I did not benchmark on AMD's yet.

Please note that patents may apply to MPEG ALS, and that you may need a patent license for legal use of this program.
  • Last Edit: 05 March, 2006, 04:48:51 AM by Garf

  • tedgo
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #1
Good news.
So mpeg4als could be usable one day 

I tested your otimized mp4als_intel.exe on my old machine and compared it with the "original" mp4alsRM16 (made the test with the same track as the flac-test, so don't wonder about the "bad" overall compresion ratio )
-p doesn't work on my machine. The *.exe refuses to work with that switch.

Code: [Select]
Testsystem:

CPU:      1.5GHz P4 "Williamette"
RAM:      512 MB SD-RAM PC133
HDD:      WD 80GB 7.200upm
OS:      WinXP SP2

=================================================================================


MODE        SPEED x
   ENCODE      DECODE    RATIO

MPEG-4 ALS RM16
-o5             14,60  42,25   63,29%
Default         10,05  31,91   62,94%
-p               2,14  24,79   62,80%
-a -o30         11,98  22,81   62,85%
-a -b -o50       7,52  12,77   62,29%
-7               0,31   4,73   61,83%
-7 -p            0,17   4,65   61,82%
-z1              0,43   0,43   62,05%
-z1 -p           0,40   0,40   61,99%
-z2              0,23   0,23   61,53%
-z2 -p           0,22   0,22   61,51%
-z3              0,14   0,14   61,31%
-z3 -p           0,14   0,14   61,29%

MPEG-4 ALS optimized by Garf (mp4als_intel.exe)
-o5             33,33  48,00   63,29%
Default         24,49  36,81   62,93%
-p              CRASH  -----   ------
-a -o30         16,35  25,21   62,85%
-a -b -o50      10,33  15,31   62,29%
-7               0,69  10,42   61,96%
-7 -p           CRASH  -----   ------
-z1              0,46   0,45   62,05%
-z1 -p          CRAsH  -----   ------
-z2              0,23   0,23   61,53%
-z2 -p          CRASH  -----   ------
-z3              0,14   0,14   61,31%
-z3 -p          CRASH  -----   ------
  • Last Edit: 09 January, 2006, 02:43:32 PM by tedgo

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Optimized MPEG ALS encoders
Reply #2
Do the other binaries exhibit the same behaviour?

I didn't touch the -z and -p stuff.

  • tedgo
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #3
I only tested the mp4als_intel until now.
Wait a minute...

EDIT:
No, only mp4als_intel refuses to work with -p
mp4als_sse2 and mp4als_gen are working.
Will make a test with this two versions later at night.

EDIT2: (added new tests with mp4als_gen and mp4als_sse2, but without -z-switches, because they are too slow for realtime decoding, at least on my system...)
Code: [Select]
Testsystem:

CPU:   1.5GHz P4 "Williamette"
RAM:   512 MB SD-RAM PC133
HDD:   WD 80GB 7.200upm
OS:    WinXP SP2

=================================================================================


MODE               SPEED x
              ENCODE   DECODE   RATIO

MPEG-4 ALS RM16
-o5            14,60     42,25   63,29%
Default        10,05     31,91   62,94%
-p              2,14     24,79   62,80%
-a -o30        11,98     22,81   62,85%
-a -b -o50      7,52     12,77   62,29%
-7              0,31      4,73   61,83%

MPEG-4 ALS optimized by Garf (mp4als_intel.exe)
-o5            33,33     48,00   63,29%
Default        24,49     36,81   62,93%
-p             CRASH     -----   ------
-a -o30        16,35     25,21   62,85%
-a -b -o50     10,33     15,31   62,29%
-7              0,69     10,42   61,96%

MPEG-4 ALS optimized by Garf (mp4als_sse2.exe)
-o5            33,90     45,80   63,29%
Default        25,64     35,93   62,93%
-p              2,55     27,78   62,80%
-a -o30        16,71     25,75   62,85%
-a -b -o50     11,09     16,13   62,28%
-7              0,73     10,79   61,96%

MPEG-4 ALS optimized by Garf (mp4als_gen.exe)
-o5            33,71     45,80   63,29%
Default        25,21     35,50   62,93%
-p              2,54     27,65   62,80%
-a -o30        16,53     25,21   62,85%
-a -b -o50     10,89     15,92   62,29%
-7              0,72     10,75   61,96%

Very effective speed-up with all settings compared to the "original" mp4alsRM16.
Now i could die for a directshow-decoder and/or a foobar2000-plugin, because compression ratio and speed is near by my so far prefered wavpack 

Very excellent work!!!
Its a real speed-up of the codec.
Is it still specs-compliant or have you changed anything that could break the specs?
  • Last Edit: 09 January, 2006, 06:32:06 PM by tedgo

  • xmixahlx
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #4
i haven't been following the MPEG4-ALS stuff, but anyways - is the source available? and are your changes available?

i.e. the possibility of a *nix binary...


later

p.s. - how is the european comma-instead-of-a-period not confusing?

  • Shade[ST]
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #5
Is it me, or from the benchmarks is Garf's version slower than the ref?

  • sh1leshk4
  • [*][*][*][*]
Optimized MPEG ALS encoders
Reply #6
The numbers probably indicate encode speed time against realtime.
Like 14x, 33x, so on and so forth; not in seconds or minutes.


~probably...

  • Speek
  • [*][*][*][*]
Optimized MPEG ALS encoders
Reply #7
Quote
i haven't been following the MPEG4-ALS stuff, but anyways - is the source available?


Yes, the reference codec is available from this page:
http://www.nue.tu-berlin.de/forschung/proj...ess/mp4als.html

  • skamp
  • [*][*][*][*][*]
  • Developer
Optimized MPEG ALS encoders
Reply #8
Would you mind posting the modified source code as well? I'd like to try to build it under linux...
See my profile for measurements, tools and recommendations.

  • tedgo
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #9
@Shade[ST] and sh1leshk4
The numbers on speed means "encode speed time against realtime" not "seconds".
See the header of the table (Sorry if its been mistakable )
I used this way because a one minute song and a ten minute song would be both encoded with Garf's optimization at about 33 times realtime with -o5, while it took about 1,8 seconds for a one minute song but about 18 seconds for a ten minute song.

So "-o5" or "Default" with Garf's version is about twice as fast as the reference in encoding and about 10% faster in decoding.

Btw. would be great if mpeg4als would make the way into nero to make it usable in mp4. With hardware-support it could be THE lossless audio standard in the future.
I'm awaiting mpeg4sls now to close the gap between aac and als
  • Last Edit: 10 January, 2006, 04:53:05 AM by tedgo

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Optimized MPEG ALS encoders
Reply #10
Quote
Very effective speed-up with all settings compared to the "original" mp4alsRM16.
Now i could die for a directshow-decoder and/or a foobar2000-plugin, because compression ratio and speed is near by my so far prefered wavpack 

Very excellent work!!!
Its a real speed-up of the codec.
Is it still specs-compliant or have you changed anything that could break the specs?
[a href="index.php?act=findpost&pid=355821"][{POST_SNAPBACK}][/a]


It's fully spec-compliant.

I guess the real wait is for MP4 embedding to be finalized, so you can have ALS album images + chapters in MP4 files.

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Optimized MPEG ALS encoders
Reply #11
Quote
i haven't been following the MPEG4-ALS stuff, but anyways - is the source available?


Yes. Source for the reference encoder is available, minus adaptive LPC (which isn't needed for a working encoder, but is needed for making a good one).

Quote
and are your changes available?


Not yet. I will have to see what I do with this.
  • Last Edit: 10 January, 2006, 05:02:32 AM by Garf

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Optimized MPEG ALS encoders
Reply #12
Quote
Btw. would be great if mpeg4als would make the way into nero to make it usable in mp4. With hardware-support it could be THE lossless audio standard in the future.
I'm awaiting mpeg4sls now to close the gap between aac and als
[a href="index.php?act=findpost&pid=355973"][{POST_SNAPBACK}][/a]


Well I did this to learn some things about lossless codecs, don't read anything into this related to what my employer may or may not do.

ALS is nice, but it's also patented, and there are alternatives that are not. We will have to see how things pan out in the industry.

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Optimized MPEG ALS encoders
Reply #13
By the way, quite a bit of encode and  decode speedup should be possible with some assembler coding of the critical loops. The compiler doesn't seem do be doing a very good job there.

Also, ALS should gain massively in speed on 64 bit machines. I think I can compile a 64 bit version later this week.
  • Last Edit: 10 January, 2006, 05:11:53 AM by Garf

  • tedgo
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #14
Quote
Well I did this to learn some things about lossless codecs, don't read anything into this related to what my employer may or may not do.

ALS is nice, but it's also patented, and there are alternatives that are not. We will have to see how things pan out in the industry.
[a href="index.php?act=findpost&pid=355984"][{POST_SNAPBACK}][/a]

Would be too bad if mpeg4als wouldn't find the way into an application like nero, only because its patented.
Aac and mpeg4 part 10 are also patented (correct me if i'm wrong) and did the way into nero. And mpeg4als is developed as the lossless audio standard for the mp4-specs. So it offers itself as the lossless audio solution for mp4. A very interesting solution for audio storage or video soundtrack in mp4 on the forthcoming blue ray disc or hd-dvd.
Nero has made the first step, now its time to go the next one   
  • Last Edit: 10 January, 2006, 06:40:15 AM by tedgo

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Optimized MPEG ALS encoders
Reply #15
Yes, but HE-AAC and AVC don't really have free alternatives that are as good, and they are already accepted by the industry.

The industry currently seems to have adopted FLAC as far as lossless storage goes. Whether they are willing to switch to ALS, for, well, marginal benefits, but extra fees, remains to be seen.

There's also the issue of MPEG SLS being up-and-coming, too.

Encoding a video soundtrack in ALS seems very silly to me. What's wrong with AAC? It's not like you store the video losslessy
  • Last Edit: 10 January, 2006, 06:50:36 AM by Garf

  • tedgo
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #16
I think flac is only adopted by the industry because it is very popular, but flac has still one of the worst compression ratios around, even with -8.
With a lossless codec like mpeg4als, specified for the widely used mp4-container and its compatibility "guaranteed" through the specs, preferences of industry could change it in the future.
And what about drm? Its hated by the users, but loved by the industry...
You'll see, it will not last long until the first online musicstore will sell drm'ed lossless files (for far too much money...)
Maybe the real musicstore? After all real networks are one of the partners improved the old lpac-codec to develop mpeg4als...

Btw. video soundtrack in a lossless format does make sense!
Think of own recordings, not only dvd-rips. Or music-videos.
I used wavpack and xvid quant1 or x264 lossless in mkv so far, but would immediately change to nero avc and mpeg4als in mp4.
  • Last Edit: 10 January, 2006, 07:28:04 AM by tedgo

  • skamp
  • [*][*][*][*][*]
  • Developer
Optimized MPEG ALS encoders
Reply #17
Quote
You'll see, it will not last long until the first online musicstore will sell drm'ed lossless files (for far too much money...) [{POST_SNAPBACK}][/a]

It's [a href="http://musicgiants.com/]already[/url] the case.
See my profile for measurements, tools and recommendations.

  • SebastianG
  • [*][*][*][*][*]
  • Developer
Optimized MPEG ALS encoders
Reply #18
Quote
ALS is nice, but it's also patented [...]
[a href="index.php?act=findpost&pid=355984"][{POST_SNAPBACK}][/a]

Is it ? What parts ?
It's basically linear prediction + residual coding only, isn't it ? Some small differences compared to FLAC are:
- coding of LPC filter via non-uniformly quantized reflection coefficients instead of direct lpc coeffs (already done on LPC10 I suppose)
- entropy coding (is arithmetic coding mandatory??)
I don't know anything about ALS' interchannel prediction. Is it smart / patented ?


Sebi
  • Last Edit: 10 January, 2006, 07:39:40 AM by SebastianG

  • Lyx
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #19
Quote
I think flac is only adopted by the industry because it is very popular, but flac has still one of the worst compression ratios around, even with -8.
[a href="index.php?act=findpost&pid=356009"][{POST_SNAPBACK}][/a]

On average, this "worst" ratio is just 4% behind monkey's audio. You and many ha.org users may care about such small difference - but in the real world, 4% is insignificant.
I am arrogant and I can afford it because I deliver.

  • tedgo
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #20
@skamp
Yes, but only in the unpopular wma lsl format and only available in the U.S.
I think mpeg4als in mp4 would be more accepted, only because its not from microsoft... 
Although i don't understand the reservations against wma lsl! It often has a better compression ratio as the popular flac (but is much slower at least on my machine).
But i would still prefer mpeg4als, because it is muxable (or will be) in mp4-videos.

@Lyx
4% may be significant if you have a music library with thousands of lossless songs.
  • Last Edit: 10 January, 2006, 07:53:11 AM by tedgo

  • Lyx
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #21
You didn't really read what i wrote, right? The industry is not about you - it cares about normal users, not geeks. All the major lossless formats are in a marging of 4% difference in compression ratio: for normal users, this means that all major lossless compressors are *equal* in regards to ratio.

Now, if you are a big company, and you only have two criterias to base your choice on - compression-ratio and marketshare(userbase) - and asuming compression is almost equal, but marketshare differs greatly.... when what are you going to base your decision on, hmm?
I am arrogant and I can afford it because I deliver.

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Optimized MPEG ALS encoders
Reply #22
Quote
Quote
ALS is nice, but it's also patented [...]
[a href="index.php?act=findpost&pid=355984"][{POST_SNAPBACK}][/a]

Is it ? What parts ?
It's basically linear prediction + residual coding only, isn't it ? Some small differences compared to FLAC are:
- coding of LPC filter via non-uniformly quantized reflection coefficients instead of direct lpc coeffs (already done on LPC10 I suppose)
- entropy coding (is arithmetic coding mandatory??)
I don't know anything about ALS' interchannel prediction. Is it smart / patented ?

Sebi
[a href="index.php?act=findpost&pid=356012"][{POST_SNAPBACK}][/a]


Ha, I thought about this too! The thing is, there's not much to patent in the default mode,  but all the extra's (LTP, MCC, BMGC, the exact backwards prediction method used, progressive prediction) probably are patented to some extent.

So, while you could perhaps make a good encoder that is patentfree, a compliant decoder would still have to support the above.
  • Last Edit: 10 January, 2006, 08:07:44 AM by Garf

  • tedgo
  • [*][*][*][*][*]
Optimized MPEG ALS encoders
Reply #23
@Lyx
What should i have read over in your short post?
Quote
On average, this "worst" ratio is just 4% behind monkey's audio. You and many ha.org users may care about such small difference - but in the real world, 4% is insignificant.

The real world is big. If you opined the industry with "the real world" you should write it...

This discussion would be unnecessary if you'd really read what i wrote:
Quote
I think flac is only adopted by the industry because it is very popular, but flac has still one of the worst compression ratios around, even with -8.
With a lossless codec like mpeg4als, specified for the widely used mp4-container and its compatibility "guaranteed" through the specs, preferences of industry could change it in the future.
And what about drm? Its hated by the users, but loved by the industry...
You'll see, it will not last long until the first online musicstore will sell drm'ed lossless files (for far too much money...)
Maybe the real musicstore? After all real networks are one of the partners improved the old lpac-codec to develop mpeg4als...

The compression ratio was just one point which may be (with the other points) of interest for users and therefore the industry in the future.
So, why you get excited? 

But its enough off-topic, back to the topic's "roots" 
  • Last Edit: 10 January, 2006, 08:49:15 AM by tedgo

  • Garf
  • [*][*][*][*][*]
  • Developer (Donating)
Optimized MPEG ALS encoders
Reply #24
64 bit build: http://sjeng.org/ftp/lossless/mp4als_x64.exe

64 bit speedups on AMD X2 4400

Encoding

Default: 85x --> 110x
-7: 2.2x --> 2.5x
-z1: 1.4x --> 3.6x(!!)

Decoding

Default: 105x --> 160x
-a -o50: 70x --> 96x
-7: 43x --> 52x
-z1: 1.4x --> 3.6x
  • Last Edit: 10 January, 2006, 09:58:52 AM by Garf