Skip to main content

Topic: Yet another lossless audio compressor... (Read 84256 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • TBeck
  • [*][*][*][*][*]
  • Developer
Yet another lossless audio compressor...
Hi,

(sorry, my english isn't very good...)

i have been working (for fun) on lossless audio compression since about 1997. Finally i would like to bring this thing (especially the never-ending-search for just a tenth of a percent more compression...) to an end. In the light of the big bunch of existing Compressors, i am not quite sure, if it would be of any use to add one more Compressor to the public. The preparations for a useful release would be much further work, and i wouldn't like to waste my time for something not needed.

My Compressor uses similar techniques like FLAC, but far more elaborated. Compression ratios lie between Monkey's Audios High- and Extra-High-Mode (Can be better than Extra High at the expense of a considerable increase of encoding time).  Encoding Speed is a bit slower than Monkey's, Decoding Speed is much higher on most Files. Seek-Times should also be better cause of the maximum (independent) frame length of 250 ms.

I would like to read some opinions. Would it make any sense to release it?

Thanks

  Thomas

Yet another lossless audio compressor...
Reply #1
In the beginning, I really didn't understand lossless (this was back in 2003).  I didn't really see the 10% compression ratio as being significantly smaller than the original wav and, to me, a much smaller mp3 had the same sound.

Now I understand the importance of lossless encoders in that they can retain tags (something wav files can't do), some offer better than 10% compression ratios, and many programs now support them.

I would welcome another lossless encoder.  After all, competition is good.  Your encoder may not use newer technology when compared with Apple lossless for Monkey's Audio but still, it would be nice to see.

I am currently in the process of ripping my entire library to Apple lossless (I live in a iPod world).

Then again, my interest is purely for curiousity.  Sadly, I am afraid that many people are set with their lossless encoders with most people going with FLAC.  Unless your lossless encoder would obtain a high compatibility with current software and hardware, then I really don't see it going anywhere beyond curiosity and testing.  Please, don't take this in the wrong way what so ever.  I would love to see a lossless format developed by a HA poster.  I just don't know if it would have any practical use or not.

  • William
  • [*][*][*][*]
Yet another lossless audio compressor...
Reply #2
In my opinion, competition is always welcome. Take a look at the Linux world. More than 100 distros, but all distros are nearly the same at the heart (linux kernel + a lot of packages).

If the compressor has some "selling points" (such as a better-than-average compression ratio, or high performance, or both), then I believe it certainly deserves some mention, especially on a site that is dedicated to audio compression.

So please, go ahead.
  • Last Edit: 31 March, 2006, 09:49:18 PM by William

  • Enig123
  • [*][*][*]
Yet another lossless audio compressor...
Reply #3
TBeck,

Sounds interesting. I'm really happy to see another good lossless audio compressor available. Will you release your compressor under GPL license or something like?

  • TBeck
  • [*][*][*][*][*]
  • Developer
Yet another lossless audio compressor...
Reply #4
Quote
Unless your lossless encoder would obtain a high compatibility with current software and hardware, then I really don't see it going anywhere beyond curiosity and testing.  Please, don't take this in the wrong way what so ever.  I would love to see a lossless format developed by a HA poster.  I just don't know if it would have any practical use or not.
[a href="index.php?act=findpost&pid=377693"][{POST_SNAPBACK}][/a]


Thanks.

My Thinking goes into the same direction. Sigh...

Building of the compression engine has been very much work. The creation and promotion of a new (free) format, which seems to be necessary to make the technology useful, would be even more work. My biggest respect for Josh Coalson, who has made FLAC some standard.

I'm in doubt that i myself would be able to establish some new standard. And i'm not sure, if it would make sense. It's a pity, that i am too late. Some years ago my work possibly would have had a chance to enrich the development of FLAC.

  Thomas

  • TBeck
  • [*][*][*][*][*]
  • Developer
Yet another lossless audio compressor...
Reply #5
Quote
If the compressor has some "selling points" (such as a better-than-average compression ratio, or high performance, or both), then I believe it certainly deserves some mention, especially on a site that is dedicated to audio compression.

So please, go ahead.
[a href="index.php?act=findpost&pid=377694"][{POST_SNAPBACK}][/a]


Many thanks for the encouragement!

The selling points could be speed, low hardware requirements if someone wanted to implement the algorithms on a DSP (Digital Signal Processor, 16 Bit would be enough) and patent free algorithms (I hope so).

Compression efficiency should be on par or better than the most common audio compressors i know, that use forward prediction (FLAC, LPAC, MPEG4 Lossless if you use Mode -7).

It will never achieve the same Compression ratios as the compressors with forward prediction e.g. Optimfrog (possibly Optimfrog is some hybrid one, that uses both methods).

On the other hand it provides asymmetric Speeds: Decoding is much faster then encoding.


  Thomas

  • TBeck
  • [*][*][*][*][*]
  • Developer
Yet another lossless audio compressor...
Reply #6
Quote
TBeck,

Sounds interesting. I'm really happy to see another good lossless audio compressor available. Will you release your compressor under GPL license or something like?
[a href="index.php?act=findpost&pid=377710"][{POST_SNAPBACK}][/a]


I would like to do so, but possibly somewhat later. If I should go on with this project, the next step could be a (closed source) release of some simple compression tool, which would allow some public testing of the efficiency. If it should prove to be useful, it would make sense to invest more time into the development. Although my source code is quite clean, it would be much work to bring it into a form, that others could use. It's mainly written in Pascal (Borland Delphi) with some assembler. I have to translate it to C, to make it handy for other developers.

  Thomas

  • neomoe
  • [*][*][*][*]
Yet another lossless audio compressor...
Reply #7
Quote
Quote
Unless your lossless encoder would obtain a high compatibility with current software and hardware, then I really don't see it going anywhere beyond curiosity and testing.  Please, don't take this in the wrong way what so ever.  I would love to see a lossless format developed by a HA poster.  I just don't know if it would have any practical use or not.
[a href="index.php?act=findpost&pid=377693"][{POST_SNAPBACK}][/a]


Thanks.

My Thinking goes into the same direction. Sigh...

Building of the compression engine has been very much work. The creation and promotion of a new (free) format, which seems to be necessary to make the technology useful, would be even more work. My biggest respect for Josh Coalson, who has made FLAC some standard.

I'm in doubt that i myself would be able to establish some new standard. And i'm not sure, if it would make sense. It's a pity, that i am too late. Some years ago my work possibly would have had a chance to enrich the development of FLAC.

  Thomas
[a href="index.php?act=findpost&pid=377718"][{POST_SNAPBACK}][/a]



TBeck, you won't have to promote this new thing when it's good. the community will do this for you
en/decoding speed aren't the only points which make a format useable, what about tagging (RG-tags and the like) multi-channel support and stuff?
don't be afraid to announce your codec - if you din't your work would be lost!

regards, j~
  • Last Edit: 01 April, 2006, 03:23:22 AM by neomoe

  • MusicLover
  • [*][*][*][*]
Yet another lossless audio compressor...
Reply #8
Quote
Hi,

(sorry, my english isn't very good...)

i have been working (for fun) on lossless audio compression since about 1997. Finally i would like to bring this thing (especially the never-ending-search for just a tenth of a percent more compression...) to an end. In the light of the big bunch of existing Compressors, i am not quite sure, if it would be of any use to add one more Compressor to the public. The preparations for a useful release would be much further work, and i wouldn't like to waste my time for something not needed.

My Compressor uses similar techniques like FLAC, but far more elaborated. Compression ratios lie between Monkey's Audios High- and Extra-High-Mode (Can be better than Extra High at the expense of a considerable increase of encoding time).  Encoding Speed is a bit slower than Monkey's, Decoding Speed is much higher on most Files. Seek-Times should also be better cause of the maximum (independent) frame length of 250 ms.

I would like to read some opinions. Would it make any sense to release it?

Thanks

  Thomas
[a href="index.php?act=findpost&pid=377674"][{POST_SNAPBACK}][/a]


Hey, the 1st of April is today! It's not quite fair!
  • Last Edit: 01 April, 2006, 04:39:07 AM by MusicLover

  • Skymmer
  • [*][*][*]
Yet another lossless audio compressor...
Reply #9
Quote
I would like to read some opinions. Would it make any sense to release it?


No, until it will provide some extraordinary features.
See here
Gabber, Jazz and IDM

  • kwanbis
  • [*][*][*][*][*]
  • Developer (Donating)
Yet another lossless audio compressor...
Reply #10
Quote
In my opinion, competition is always welcome. Take a look at the Linux world. More than 100 distros, but all distros are nearly the same at the heart (linux kernel + a lot of packages).

So please, go ahead.
[a href="index.php?act=findpost&pid=377694"][{POST_SNAPBACK}][/a]

yes go ahead.

(off topic: i personally think that one of linux problems is just that, 100 distributions, a lot of problems, a lot of wasted resources)

  • TBeck
  • [*][*][*][*][*]
  • Developer
Yet another lossless audio compressor...
Reply #11
Quote
Hey, the 1st of April is today! It's not quite fair!
[a href="index.php?act=findpost&pid=377779"][{POST_SNAPBACK}][/a]


Huch... Did i miss something? 

  • TBeck
  • [*][*][*][*][*]
  • Developer
Yet another lossless audio compressor...
Reply #12
Quote
Quote
I would like to read some opinions. Would it make any sense to release it?


No, until it will provide some extraordinary features.
See here
[a href="index.php?act=findpost&pid=377800"][{POST_SNAPBACK}][/a]


Would much higher speed be relevant?

  • pest
  • [*][*][*]
Yet another lossless audio compressor...
Reply #13
I've worked on something similiar the last years.
my encoder uses an adaptive wavelet filterbank on the prediction
residuals. 
it has id3v1 and apev2 support but i stopped developing because
i think nobody needs yet another lossless compressor.

best regards

  • Triza
  • [*][*][*][*]
Yet another lossless audio compressor...
Reply #14
Nobody needs another one that is only 1-2% better. Personally I will stick with FLAC until someone matches its features and at least 15% better.

Triza

  • rutra80
  • [*][*][*][*][*]
  • Members (Donating)
Yet another lossless audio compressor...
Reply #15
An encoder with compression ratio as high as Monkey's Audio but faster at decoding would be certainly a nice thing. Count me as a first user if additionally it has these features:
1. Piping support (without requiring input's accurate lenght)
2. APEv2 tags support
3. Always up-to-date & not-hackish plugin for fb2k
4. Simple (no more than 2 switches for altering compression ratio & speed)

  • Duble0Syx
  • [*][*][*][*]
Yet another lossless audio compressor...
Reply #16
Quote
An encoder with compression ratio as high as Monkey's Audio but faster at decoding would be certainly a nice thing. Count me as a first user if additionally it has these features:
1. Piping support (without requiring input's accurate lenght)
2. APEv2 tags support
3. Always up-to-date & not-hackish plugin for fb2k
4. Simple (no more than 2 switches for altering compression ratio & speed)
[a href="index.php?act=findpost&pid=377878"][{POST_SNAPBACK}][/a]

Things like that are why I personally use WavPack.  Encodes fast even on high mode, smaller than flac, and decodes only a bit slower.  IMHO Monkey's Audio is a poor codec simply because it is slow as hell if you want decent compression.  If you are going to try to compete it would be more reasonable to try and get compression ratios like OptimFrog, MAC or LA, but with WavPack or FLAC's encoding/decoding speeds along with APEv2 tags, proper seeking and other nice features.  Then you'd have something worthy of competition.

  • xmixahlx
  • [*][*][*][*][*]
Yet another lossless audio compressor...
Reply #17
you really don't need an official release right now - why don't you just pack up the source and have some people test it.

the bigger question is if it is something you want to continue maintaining it...


later

  • boombaard
  • [*][*][*][*]
Yet another lossless audio compressor...
Reply #18
Quote
IMHO Monkey's Audio is a poor codec simply because it is slow as hell if you want decent compression. If you are going to try to compete it would be more reasonable to try and get compression ratios like OptimFrog, MAC or LA, but with WavPack or FLAC's encoding/decoding speeds along with APEv2 tags, proper seeking and other nice features. Then you'd have something worthy of competition.


..define slow as hell, if you would.. this is just a near-troll..
MAC extra high decodes @18~22x on my pc (athlon 2800+), and is probably the best choice for decoding speed vs. compression ratios there out there..
sure, OFR gets better compression ratios, but that's decoding at 3x playback or slower.. flac -8 is faster at decoding, but the compression ratios are significantly lower.. wavpack is better at compressing than flac, but still worse than MAC extra high (in that you can save GBs of space on modern sized HD), which i consider the benchmark here (since imo 20x playback is more than fast enough)..
LA i haven't tested, but the benchmarks seem to suggest it's impractically slow at comparable compression levels (<5x playback)

regardless.. you want: awesome compression, combined with high decoding speeds, and preferably encoding too.. obviously well-taggable, without bugs in it..
so basically you ask the world of someone who hasn't even released a .01 version of his en/decoder yet..
how encouraging this must be

anyway, TSer.. feel free to do with your free time what you want, i'm sure there'll be people here that are interested without saying you should invent cold fusion before you're allowed to post about it here

  • Skymmer
  • [*][*][*]
Yet another lossless audio compressor...
Reply #19
Quote
An encoder with compression ratio as high as Monkey's Audio but faster at decoding would be certainly a nice thing.
[a href="index.php?act=findpost&pid=377878"][{POST_SNAPBACK}][/a]


Agree here, but personaly I think that it will be quite hard to achieve high compression ratios while making format non-symmetric.
Gabber, Jazz and IDM

  • TBeck
  • [*][*][*][*][*]
  • Developer
Yet another lossless audio compressor...
Reply #20
Quote
Agree here, but personaly I think that it will be quite hard to achieve high compression ratios while making format non-symmetric.
[a href="index.php?act=findpost&pid=377941"][{POST_SNAPBACK}][/a]


Yeah, i like asymmetrical Compression. Possibly time for some Data:

Participiants:
  TAK        (Toms Audio K©ompression) Alpha, unreleased
  FLAC      V 1.1.2 Win
  Monkey    V 3.99
  OptimFrog  V 4.520b [2006.03.02] (beta)

System:
  Pentium III / 866 MHz

Code: [Select]
            TAK                       FLAC      Monkey 3.99       OptimFrog
Mode:       High    Extra   Insane |  -8     |  High    Extra  |  Best   |
-----------------------------------+---------+-----------------+---------+
Song_02     48.41   47.87   47.73  |  51.03  |  48.00   47.28  |  47.88  |
Song_04     33.15   32.59   32.56  |  37.27  |  33.58   32.35  |  32.76  |
Song_06     33.74   33.34   33.20  |  37.04  |  33.77   33.09  |  33.01  |
Song_08     44.97   44.56   44.45  |  49.74  |  44.81   43.59  |  43.82  |
Song_10     56.41   56.00   55.94  |  59.10  |  55.95   54.97  |  55.03  |
Song_12     53.86   53.33   53.27  |  57.62  |  53.04   51.99  |  51.57  |
Song_14     48.97   48.51   48.44  |  51.87  |  48.65   47.76  |  47.69  |
Song_16     74.16   73.82   73.79  |  75.95  |  73.70   73.44  |  73.27  |
-----------------------------------+---------+-----------------+---------+
Sum:        47.86   47.41   47.33  |  51.35  |  47.62   46.70  |  46.81  |
EncoTime:   53.01  270.94  595.41  |  --.--  |  57.15  109.02  |  --.--  |
DecoTime:   13.50   14.90   15.19  |  --.--  |  63.92  114.71  |  --.--  |
-----------------------------------+---------+-----------------+---------+
Bach_01     58.29   54.40   53.20  |  64.67  |  60.05   55.42  |  59.27  |
Bartok_01   48.69   48.19   48.13  |  53.30  |  49.17   47.74  |  48.40  |
Debussy_01  26.17   26.04   25.96  |  28.55  |  26.48   26.12  |  25.59  |
Mahler_01   48.32   47.69   47.42  |  49.87  |  47.73   47.09  |  47.51  |
Speech_01   30.96   30.46   30.17  |  34.15  |  31.81   31.78  |  29.10  |
-----------------------------------+---------+-----------------+---------+
Sum:        42.50   41.40   41.02  |  46.00  |  43.02   41.62  |  42.04  |
EncoTime:    8.87   46.97  104.87  |  --.--  |   9.23   18.12  |  --.--  |
DecoTime:    2.20    2.56    2.65  |  --.--  |  11.74   20.52  |  --.--  |
-----------------------------------+---------+-----------------+---------+
S_24_44_01  59.56   59.41   59.38  |  61.69  |  59.23   58.97  |  --.--  |
S_24_44_02  62.27   62.08   62.05  |  65.87  |  61.85   61.70  |  --.--  |
S_24_44_03  52.42   52.18   52.15  |  54.65  |  52.32   51.88  |  --.--  |
S_24_48_01  62.86   62.58   62.51  |  65.16  |  62.83   62.45  |  --.--  |
S_24_48_02  54.27   54.13   54.09  |  55.89  |  53.99   53.92  |  --.--  |
S_24_48_03  78.72   78.57   78.52  |  97.93  |  79.26   79.30  |  --.--  |
-----------------------------------+---------+-----------------+---------+
Sum:        58.13   57.94   57.90  |  60.88  |  57.89   57.65  |  --.--  |
EncoTime:   23.37  113.12  247.91  |  --.--  |  28.01   49.20  |  --.--  |
DecoTime:    5.79    6.21    6.28  |  --.--  |  30.83   53.37  |  --.--  |
-----------------------------------+---------+-----------------+---------+


I don't know, how long a post can be. So to be continued in the next post.

  Thomas

  • TBeck
  • [*][*][*][*][*]
  • Developer
Yet another lossless audio compressor...
Reply #21
The Table in the previous post contains results from my main test corpus,
which is divided into three parts:

Song_02..Song_16
CD-Ripps (Rock, Pop, Songwriter), 44 KHz, 16 Bit, about 90 Seconds per file.

Bach_01..Speech_01
Classic music from a compression test side, 44 KHz, 16 Bit, about 30 Seconds
per file.

S_24_44_01..S_24_48_03
Some Samples from a ADC/DAC (Analog-Digital-Converter)-Test-Side. 44/48 KHz, 24 Bit.

I have tested far more files in the past, but these ones have proven to be
the most representative ones.

You can see the relative compression ratio for each single file. Sum is the
mean Compression for each part, Enco and DecoTime is Duration of Encoding and
Decoding in seconds.

You can easily see the assymmetry in encoding vs. decoding time.


  Thomas

  • Skymmer
  • [*][*][*]
Yet another lossless audio compressor...
Reply #22
Truly speaking I'm little bit impressed. The ratios are very close to MA while providing fast decoding! Thats realy cool ! Can we get alpha version of your TAK to play with?
By the way I advise you to change the name to something else cause the whole name is little bit complicated and further more there were some lossy project called TAC from K+K Research so some people can be confused.
Gabber, Jazz and IDM

  • rjamorim
  • [*][*][*][*][*]
Yet another lossless audio compressor...
Reply #23
Quote
Code: [Select]
-----------------------------------+---------+-----------------+---------+
Sum:        58.13   57.94   57.90  |  60.88  |  57.89   57.65  |  --.--  |
EncoTime:   23.37  113.12  247.91  |  --.--  |  28.01   49.20  |  --.--  |
DecoTime:    5.79    6.21    6.28  |  --.--  |  30.83   53.37  |  --.--  |
-----------------------------------+---------+-----------------+---------+
[a href="index.php?act=findpost&pid=377983"][{POST_SNAPBACK}][/a]


That's good. Too good. It's indeed truly remarkable!

What day is today again?
Get up-to-date binaries of Lame, AAC, Vorbis and much more at RareWares:
http://www.rarewares.org

  • TBeck
  • [*][*][*][*][*]
  • Developer
Yet another lossless audio compressor...
Reply #24
Quote
Truly speaking I'm little bit impressed. The ratios are very close to MA while providing fast decoding! Thats realy cool ! Can we get alpha version of your TAK to play with?
By the way I advise you to change the name to something else cause the whole name is little bit complicated and further more there were some lossy project called TAC from K+K Research so some people can be confused.
[a href="index.php?act=findpost&pid=378000"][{POST_SNAPBACK}][/a]


Nice. That's very motivating!

Unfortunately this Alpha version definitely needs some further work, before i want to give it away. It still generates some single bit errors on some files. It's not a conceptual problem, so compression efficiency will not be reduced by the necessary corrections. Furthermore my code for reading the source (wave) files is very rudimentary and could possibly fail on some files. I don't think, that the actual release would make a really good impression.

But i will try to provide a working evaluation version as soon as possible. If nothing unexpected will happen, this should be done in about one month.

TAK was the first thing that came to my mind. And it did fit into the table, so i used it. But confusion with other projects would be bad, so i will look for another name.

Thanks

  Thomas