Skip to main content

Topic: TAK 2.2.0 (Read 85484 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • TBeck
  • [*][*][*][*][*]
  • Developer
TAK 2.2.0
Final release of TAK 2.2.0 ((T)om's lossless (A)udio (K)ompressor)

This release brings support for multi-channel audio and speed optimizations for encoder and decoder.

It consists of:
  • TAK Applications 2.2.0.
  • TAK Winamp plugin 2.2.0.
  • TAK SDK 1.1.1.
  • TAK Decoding library 2.2.0.

Download

[ Specified attachment is not available ]

  • TBeck
  • [*][*][*][*][*]
  • Developer
TAK 2.2.0
Reply #1
What's new

New features:
  • Support for multi-channel audio. While the stream format supports up to 16 channels, the codec currently is restricted to a maximum of 6 channels.
  • Support for the "Wave Format Extensible" file format.

Improvements:
  • Encoding speed improvements of up to 10 percent for my primary file set. Most of it is achieved by a modification of the algorithm which selects the optimal predictor order for each subframe. It will now often use less predictors than before, what may on average result in about 0.01 percent worse compression. You will only notice an advantage, if your files benefit from high predictor orders.
  • Decoding speed improvements of up to 18 percent for my primary file set. Some of it is attributed to the above-noted modification of the encoder's predictor order selection algorithm. Therefore it will only take effect when decoding files encoded with this version and only, if they benefit from high predictor orders. Additionally SSSE3-instructions can be used for predictor counts of 32 and more. This affects the presets p3, p4 and sometimes p2, but only, if a particular file benefits from high predictor orders.

Fixes:
  • The wave file reader reported an error if a file contained additional (meta) data following the audio data.
  • The wave file writer didn't add an optional zero byte to make the audio data chunk size a multiple of 2. This was only relevant when decoding mono audio with 8- or 24-bit samples without restoring the wave meta data (-wm0 applied on encoding and/or decoding).

Known issues:
  • If you use pipe decoding and the application reading the pipe is beeing terminated before the whole file has been read, TAKC may get into an endless loop and has to be manually killed with the task manager. I don't think this is a big issue but i will try to fix it in one of the next versions. BTW: Big thanks to shnutils for testing the pipe decoding!
  • There seem to be some compatibility issues with pipe decoding to some other applications ("crc1632.exe" has been reported). I will try to fix it in the next release.

More information

You can find some useful information and comparisons in the Alpha thread.

Have fun...

Thomas

  • TBeck
  • [*][*][*][*][*]
  • Developer
TAK 2.2.0
Reply #2
I took the freedom to compile an excerpt of the extensive compression results which m² has posted in the Alpha thread.

His notes:
Code: [Select]
All samples come from DVD-A discs and are 5.1

Albums:
Sting       - Brand New Day           (samples d*, r*), 24/48
Nightwish   - Dark Passion Play       (a*),             24/48
Ton Koopman - Bach: Organ Spectacular (t*),             24/96
Foreigner   - 4                       (u*, v*, w*),     24/96

Codecs:
als rm23 (-7, other switches as indicated)
flake SVN r264 (-12)
wavpack 4.60.1
ttaenc 3.4.1
tak 2.2.0 (-p4m -wm0)

The 24/48 results:
Code: [Select]
                   a0     a1     a3     a4     d0     d2     d3     d4     d5     d6     r0     r1     r2     r3     
WavPack -hx4       58,88  66,86  60,53  67,12  33,08  39,28  48,57  48,81  47,97  52,46  41,06  49,60  52,68  52,42  
WavPack -hx6       58,84  66,86  60,49  67,12  32,98  39,07  48,18  47,98  47,40  51,61  40,84  48,79  51,63  51,52  
WavPack -hhx6      58,79  66,83  60,44  67,09  32,76  38,79  47,74  47,58  47,09  51,12  40,51  48,37  51,18  51,07  
Flake -12          58,33  66,01  60,25  66,25  32,82  37,46  47,63  47,59  46,96  50,97  40,77  48,45  51,11  51,16  
TTA                65,76  74,80  66,77  74,94  51,35  55,96  67,49  66,45  66,50  71,47  63,61  70,14  72,08  73,33  
TAK -p4m           56,78  64,59  58,36  64,93  30,79  34,68  44,71  43,74  44,20  47,94  37,92  45,99  47,80  49,42  
Als -7 -o160 -t6   56,61  64,45  58,23  64,78  30,52  34,24  44,44  44,02  43,84  47,85  38,30  46,02  47,89  48,87  
Als -7 -o1023 -t6  56,43  64,37  58,13  64,72  30,18  33,99  44,43  43,99  43,78  47,84  38,25  45,98  47,83  48,79  
Als -z3 -t6        57,55  65,68  59,62  65,89  42,89  47,52  58,93  57,99  57,91  61,81  50,45  59,03  61,46  63,11

Compression ratio expressed as percentage of the original size.

The 24/96 results:
Code: [Select]
                   t0     t1     t2     t3     u0     u1     u2     u3     v0     v1     w0     w1     w2     w3     w4
WavPack -hx4       47,91  48,00  46,94  48,16  58,05  59,77  59,43  60,70  47,34  55,27  50,10  57,34  59,08  60,37  61,71
WavPack -hx6       47,89  47,99  46,94  48,16  57,97  59,69  59,33  60,57  47,23  55,22  50,00  57,26  58,98  60,31  61,68
WavPack -hhx6      47,88  47,97  46,93  48,16  57,32  59,11  58,77  59,92  46,66  54,70  49,49  56,86  58,52  59,92  61,37
Flake -12          47,73  47,87  46,96  48,21  56,79  58,62  58,18  59,36  46,01  53,91  48,81  55,89  57,48  59,00  60,81
TTA                49,61  49,66  48,69  49,87  66,98  69,26  68,71  70,62  55,04  63,73  59,10  66,16  67,55  69,29  71,25
TAK -p4m           42,80  43,10  41,94  43,01  55,73  57,72  57,33  58,46  45,23  52,62  47,82  54,92  56,30  57,94  59,95
Als -7 -o160 -t6   45,60  46,03  45,01  45,98  54,97  56,91  56,34  57,43  44,97  52,18  47,63  54,40  55,65  57,22  59,07
Als -7 -o511  -t6  45,26  45,76  44,78  45,53  54,97  56,91  56,34  57,42  44,94  52,19  47,27  54,36  55,65  57,17  59,05
Als -z3 -t6        41,22  41,96  41,07  41,92  60,96  62,86  62,30  63,24  50,94  58,03  53,28  60,33  61,66  63,11  64,85


m²'s comment:
Quote
Generally, TAK looses to ALS, but is rocket fast.
I haven't decided which one do I want to use yet...Really, I expected TAK to do better.

I feel quite comfortable with your results.

For the 24/48 samples the performance of TAK -p4m  is quite similar to Als -7 -o160 -t6. TAK compresses the t-set of the 24/96 samples better than Als -7 -o160 -t6, but indeed looses on the Foreigner 4 samples u, v and w.

Would it be possible to send me one or two 30 second snippets of the Foreigner files, where ALS's advantage manifests? I would like to look for tuning opportunities.

BTW: Thank you very much for such an extensive evaluation!

  Thomas



  • _m²_
  • [*][*][*]
TAK 2.2.0
Reply #3
Would it be possible to send me one or two 30 second snippets of the Foreigner files, where ALS's advantage manifests? I would like to look for tuning opportunities.


No problem. Will do it later today.

  • _m²_
  • [*][*][*]
TAK 2.2.0
Reply #4
Would it be possible to send me one or two 30 second snippets of the Foreigner files, where ALS's advantage manifests? I would like to look for tuning opportunities.


No problem. Will do it later today.



BTW by 'ALS's advantage' do you mean o160 or o512? For my purposes I compare to the latter, but can check with either.
  • Last Edit: 13 July, 2011, 06:06:23 AM by _m²_

  • TBeck
  • [*][*][*][*][*]
  • Developer
TAK 2.2.0
Reply #5
BTW by 'ALS's advantage' do you mean o160 or o512? For my purposes I compare to the latter, but can check with either.

I don't think TAK's worse performance has to be attributed to ALS's higher predictor count, therefore please use the one you like to. I don't know if the factor which determines TAK's worse performance is present in any section of the files, hence you may have to check several parts.

I have created a new table, which probably is easier to evaluate. I took TAK's compression ratio as baseline and substracted it from the other results. This way positive values indicate other compressors performing worse, negative values represent better performance:

Code: [Select]
                   a0     a1     a3     a4     d0     d2     d3     d4     d5     d6     r0     r1     r2     r3     
WavPack -hx4        2.10   2.27   2.17   2.19   2.29   4.60   3.86   5.07   3.77   4.52   3.15   3.61   4.88   3.00  
WavPack -hx6        2.06   2.27   2.13   2.19   2.19   4.39   3.48   4.24   3.21   3.67   2.92   2.80   3.84   2.10  
WavPack -hhx6       2.02   2.23   2.08   2.16   1.97   4.11   3.03   3.84   2.90   3.18   2.59   2.38   3.39   1.66  
Flake -12           1.55   1.42   1.89   1.32   2.04   2.78   2.93   3.84   2.77   3.03   2.85   2.46   3.32   1.74  
TTA                 8.98  10.21   8.41  10.01  20.56  21.28  22.78  22.71  22.31  23.53  25.69  24.15  24.28  23.91  
TAK -p4m            0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00  
Als -7 -o160 -t6   -0.17  -0.15  -0.12  -0.15  -0.27  -0.43  -0.26   0.28  -0.36  -0.09   0.38   0.03   0.09  -0.54  
Als -7 -o1023 -t6  -0.35  -0.22  -0.22  -0.21  -0.61  -0.68  -0.27   0.24  -0.41  -0.10   0.33  -0.01   0.03  -0.63  
Als -z3 -t6         0.77   1.09   1.26   0.96  12.11  12.84  14.22  14.24  13.72  13.87  12.53  13.04  13.67  13.70  

                   t0     t1     t2     t3     u0     u1     u2     u3     v0     v1     w0     w1     w2     w3     w4
WavPack -hx4        5.11   4.90   5.00   5.16   2.32   2.05   2.10   2.23   2.10   2.65   2.27   2.43   2.78   2.43   1.76
WavPack -hx6        5.09   4.89   5.00   5.16   2.24   1.97   2.00   2.11   2.00   2.61   2.18   2.34   2.68   2.37   1.72
WavPack -hhx6       5.08   4.87   4.99   5.16   1.59   1.38   1.44   1.46   1.43   2.08   1.66   1.94   2.22   1.98   1.42
Flake -12           4.93   4.77   5.03   5.20   1.07   0.90   0.85   0.90   0.78   1.29   0.98   0.97   1.19   1.06   0.86
TTA                 6.81   6.56   6.75   6.86  11.26  11.54  11.39  12.16   9.81  11.11  11.27  11.25  11.25  11.35  11.30
TAK -p4m            0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00   0.00
Als -7 -o160 -t6    2.80   2.93   3.08   2.98  -0.76  -0.81  -0.99  -1.03  -0.26  -0.44  -0.20  -0.52  -0.65  -0.72  -0.88
Als -7 -o511 -t6    2.45   2.66   2.84   2.52  -0.75  -0.81  -0.98  -1.04  -0.29  -0.42  -0.55  -0.56  -0.65  -0.77  -0.90
Als -z3 -t6        -1.58  -1.14  -0.87  -1.09   5.23   5.13   4.97   4.78   5.71   5.42   5.46   5.42   5.37   5.17   4.90
  • Last Edit: 13 July, 2011, 08:33:30 PM by TBeck

  • _m²_
  • [*][*][*]
TAK 2.2.0
Reply #6
Got the samples. Took some time...I didn't take into account als slowness.
http://tiny.pl/hfzwq
BTW restrictions on where I can link are wrong. If I'm not supposed to use file lockers, give me enough space here.

  • TBeck
  • [*][*][*][*][*]
  • Developer
TAK 2.2.0
Reply #7
Got the samples. Took some time...I didn't take into account als slowness.

Thank you! The analysis will take some time and i can not guarantee, that i will find an (compatible) optimization.

TAK 2.2.0
Reply #8
This is really awesome. Thanks!

TAK 2.2.0
Reply #9
TBeck, do you plan to add a CUDA support for further versions of TAK encoder?
IMHO, such progressive codec as TAK must have a support of such useful feature as CUDA. Maybe it is even possible to increase the compression level this way, isn't it?

  • zerowalker
  • [*][*][*][*]
TAK 2.2.0
Reply #10
Is there plans to make it playable on any player?
Or atleast Zoom Player and the like;O?

Cause i love this kind of thing, lossless music now in less size, what more can you ask for:D?
Good work, keep on going as much as you want and can:)!
  • Last Edit: 07 August, 2011, 11:03:51 AM by db1989

  • CoRoNe
  • [*][*][*]
TAK 2.2.0
Reply #11
Unlike WinAMP and Foobar, Zoom Player is a DirectShow media player. You'll need a DirectShow source/decoder filter in that case.
Take a look at my website for Livio Cavallo's TAK Source Filter (packed within my DSFP).
DC-Bass Source Mod: http://reino.degeelebosch.nl

  • zerowalker
  • [*][*][*][*]
TAK 2.2.0
Reply #12
Unlike WinAMP and Foobar, Zoom Player is a DirectShow media player. You'll need a DirectShow source/decoder filter in that case.
Take a look at my website for Livio Cavallo's TAK Source Filter (packed within my DSFP).


Nice got it working thanks:D
But does it output lossless as it is?
As i wonder if i can convert many FLAC files to TAK, without anything changing, as well as it decodes them right and such:)

  • CoRoNe
  • [*][*][*]
TAK 2.2.0
Reply #13
Of course it does. That's the point of lossless audio codecs.
DC-Bass Source Mod: http://reino.degeelebosch.nl

  • zerowalker
  • [*][*][*][*]
TAK 2.2.0
Reply #14
Of course it does. That's the point of lossless audio codecs.

Yeah but you never know if the decoder does it´s job corretly, so it is as if it was Wave.
Or atleast i don´t know:D

  • TBeck
  • [*][*][*][*][*]
  • Developer
TAK 2.2.0
Reply #15
Got the samples. Took some time...I didn't take into account als slowness.

Thank you! The analysis will take some time and i can not guarantee, that i will find an (compatible) optimization.

Finally i tried it. I manually selected some of TAK's filters (statically) for each (whole) file and got at least about 0.40 percent better compression for your sample files. A bit more of improvement might be possible, if the filters would be conditionally selected per frame.

Currently i regard those files as special cases, although i have to take into account, that i dont know nearly as much about the compression relevant properties (and their frequency) of-multi channel audio as i possibly know about CD-audio. For now i have added the files to my special cases folder which affects future tunings. Thank you for the files!

Currently i have no idea how to tune the encoder for such files without loosing a lot of encoding speed. I could implement a brute force approach which simply would encode all files with the mentioned filters switched on or off and then would choose the best result. But that's not the way TAK achieves it's high encoding speed. If it would fully calculate all possible combinations of it's many filter variations, it could be easily 1000 times (or more!) slower. Sometimes i read, TAK's encoding is so fast because of it's assembler optimizations. No, that's misleading. It's speed-efficient design and the assembler optimizations may speed up encoding by a factor of about 2 to 3 (for -p4m), but the most important factor are it's heuristics, which base all the filter deceisions on relatively simple estimations. That's where most of the encoder development time has gone.

But sometimes those heuristics fail, as can be seen with m2's files.

A side note: Your (m2) files have wasted bits (low significant bits are zero), this may explain the really bad performance of TTA, which -to my knowledge- doesn't check for this case.

TBeck, do you plan to add a CUDA support for further versions of TAK encoder?
IMHO, such progressive codec as TAK must have a support of such useful feature as CUDA. Maybe it is even possible to increase the compression level this way, isn't it?

I don't want to loose decoding speed, because i still think about possible hardware implementations. Therefore i don't want to significantly increase the complexity of TAK's filters. Then only encoder optimizations are possible. With a lot more of processing power available i could possible replace some of TAK's fast heuristics with brute force approaches. This could for instance help special files like those m2 has sent me, but i doubt, it would significantly increase the average compression ratio for a large corpus of files. Anectdotally: i spend the last two days implementing a brute force approach for one case, where TAK versions earlier than 2.0 could achieve 0.05 percent better compression. This was the most significant advantage i ever found for the brute force way of a filter selection! But with the better design and heuristics of TAK 2.x, the advantage shrinked to about 0.01 percent.

I don't think, the investment of maybe 20 or even 50 times more processing power (which probably would require a quite potent GPU) would result in more than about 0.05 to 0.10 percent better compression on average. But it could help some special files.

And then there is also my personal fun factor: I really like to beat brute force approaches whith more or less clever algorithms... Well, indeed it's often less cleverness but a bit of intuition and a lot of trial and error and collecting and evaluating a lot of relevant data.

Conclusion: Too little to gain for me by CUDA yet, but may be, if i have some new ideas...

It's not that i don't think about opportunities for compression improvements. But it's getting really hard to find some within my efficiency/speed constraints.

TAK 2.2.0
Reply #16
Thanks for reply.

Seem like now takc works fine with HyperThreading.

Code: [Select]
HT on:
takc -e -p4m        34.22 * real time
takc -e -p4m -tn1   34.28 * real time
takc -e -p4m -tn2   50.21 * real time
takc -e -p4m -tn4   66.48 * real time

HT off:
takc -e -p4m -tn2   50.49 * real time

(Core i3 530)

So, there is a speed improvement with -tn4 compared to -tn2 and lower, -tn2 with HT off/on is equal, and by default encoder uses only one thread. Everything is quite good. Thank you, TBeck
  • Last Edit: 12 August, 2011, 07:12:07 AM by Steve Forte Rio

  • jaro1
  • [*][*]
TAK 2.2.0
Reply #17
request: Mr. Beckers own fb2k decoder, i'll be always updated in the case of decoding improvements, and provided together with main TAK encoder release, as WA decoder is.

  • lvqcl
  • [*][*][*][*][*]
  • Developer
TAK 2.2.0
Reply #18
But you can always update tak_deco_lib.dll yourself.

  • jaro1
  • [*][*]
TAK 2.2.0
Reply #19
But you can always update tak_deco_lib.dll yourself.

If the whole point of foosions plugin is only link to tak_deco_lib.dll to use all of its functions, then yes. Otherwise, i don't know if it makes sense, coz there could be e.g. some new decoding functions in the library, which will not be used with old plugin. Simply i thought Mr. Beckers plugin would be handier solution IMHO, with only one library (maybe). Don't want to discuss much about current plugin here, this was only my "small" (but maybe reasonless) request.
  • Last Edit: 24 September, 2011, 01:51:03 PM by jaro1

  • boombaard
  • [*][*][*][*]
TAK 2.2.0
Reply #20
Final release of TAK 2.2.0 ((T)om's lossless (A)udio (K)ompressor)

This release brings support for multi-channel audio and speed optimizations for encoder and decoder.

It consists of:
  • TAK Applications 2.2.0.
  • TAK Winamp plugin 2.2.0.
  • TAK SDK 1.1.1.
  • TAK Decoding library 2.2.0.

Download

(Attachment Link)

Impressive.. Thank you for your dedication.

  • Xire
  • [*]
TAK 2.2.0
Reply #21
Have you tried to compile it with 64bit Delphi or 64bit FreePascal? Would be interesting to see the results and a 64bit library would be nice as well

  • boombaard
  • [*][*][*][*]
TAK 2.2.0
Reply #22
It appears as though the encoder refuses to encode when it has to output a file with non-western unicode characters somewhere in the file path. (e.g. Прокофьев, Сергей Сергеевич). It works fine when the input filenames have such chars, but it bugs when it has to output to them.
  • Last Edit: 25 September, 2011, 03:35:28 PM by boombaard

  • boombaard
  • [*][*][*][*]
TAK 2.2.0
Reply #23
(PS. Is it normal that I get different RG numbers even when the file I converted from was also lossless?)

  • anishbenji
  • [*][*]
  • Members (Donating)
TAK 2.2.0
Reply #24
If you used a recent version of foobar2000 (v1.1.6 or later) to calculate the replaygain values, a new algorithm is used which results in slightly different values. You can double check this by recalculating the replaygain of the original files using the latest version of foobar2000.
Anish