Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: TAK 2.3.1 (Read 24450 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

TAK 2.3.1

Final release of TAK 2.3.1 ((T)om's lossless (A)udio (K)ompressor)

It consists of:

- TAK Applications 2.3.1
- TAK Winamp plugin 2.3.1
- TAK Decoding library 2.3.1
- TAK SDK 1.1.1

Download


Re: TAK 2.3.1

Reply #1
What's new

This release brings significant speed optimizations for the encoder and a lot of source code cleanups in preparation of a migration from Delphi to Lazarus and/or C. Practical goals are Linux binaries and open source releases. This will be done step by step depending on my spare time. The cleanup revealed several bugs which affected the compression efficiency, but never the data integrity.

Improvements:

  • Encoding speed improvements of up to 43 percent in my tests. The slower presets benefit most. Between 6 and 28 percent for an i5-4460 (Haswell), between 9 and 43 percent for an i3-8100 (Coffee Lake).
  • Decoding speed improvements of up to 10 percent in my tests. The slower presets benefit most. Between 2 and 7 percent for an i5-4460 (Haswell), between -1 and 10 percent for an i3-8100 (Coffee Lake).
  • Really tiny compression improvements because of some bug fixes. See below.
  • Better source code and smaller binaries.

New features:

  • The multi-threaded encoder now supports up to 8 instead of 4 threads.
  • The cpu optimization option None now really disables any assembler optimizations. Previous versions still used some i386-assembly that could only be disabled by compiler switches. The new option ASM enables this code and is equivalent to None of earlier versions.

Fixes:

  • Small bugs in the encoder decreased compression by usually not more than 0.01 percent. One of my file sets lost 0.06 percent. Some special files will show stronger effects.
  • A bug in the plain pascal code path significantly decreased compression of some presets: up to 1.19 percent for my primary file set. To encounter this bug, you had to use V2.2.0 or 2.3.0 and explicitly disable assembler optimizations (-cpuNone) or run Tak on a cpu without even the MMX-instruction set (e.g. Pentium 1).
  • Added new tests to my already extensive validation procedure to detect regressions of the plain pascal code path. Til now i only checked the data integrity.
  • None of these bugs affected the data integrity.

Cleanup:

  • Removed the assembler optimizations from the TAK 1.x decoder and made it a lot more compact.
  • Replaced MMX with SSE2 assembly.
  • Replaced FPU with SSE2 assembly.
  • Removed assembler optimizations which had little effect on the speed.
  • Removed any inline assembly code.
  • Removed a lot of partial redundancies which had been introduced to gain some speed.

Known issues:

  • If you use pipe decoding and the application reading the pipe is beeing terminated before the whole file has been read, TAKC may get into an endless loop and has to be manually killed with the task manager. I don't think this is a big issue but i will try to fix it in one of the next versions. BTW: Big thanks to shnutils for testing the pipe decoding!
  • There seem to be some compatibility issues with pipe decoding to some other applications ("crc1632.exe" has been reported). I will try to fix it in the next release.

More information

You can find some useful information and comparisons in the beta thread.

Have fun...

Thomas

Re: TAK 2.3.1

Reply #2
Results

Here the results for my primary file set.

Test system: Intel i3-8100 (3.6 GHz / 1 Thread), Windows 10.
Code: [Select]
Preset  Compression %           Enco-Speed                Deco-Speed
---------------------------------------------------------------------------------
        2.3.0   2.3.1    Win    2.3.0    2.3.1    Win %   2.3.0    2.3.1    Win %
---------------------------------------------------------------------------------
-p0     58.75   58.74    0.01   809.08   878.29    8.55   807.68   800.12   -0.94
-p0e    58.33   58.33    0.00   622.98   697.03   11.89   806.35   806.26   -0.01
-p0m    58.22   58.22    0.00   346.10   413.76   19.55   809.03   808.92   -0.01
-p1     57.84   57.84    0.00   651.21   731.41   12.32   780.98   782.47    0.19
-p1e    57.52   57.52    0.00   407.25   467.87   14.89   780.58   784.63    0.52
-p1m    57.42   57.42    0.00   253.37   307.77   21.47   781.59   787.19    0.72
-p2     56.90   56.90    0.00   465.33   581.92   25.06   671.37   710.83    5.88
-p2e    56.70   56.70    0.00   259.44   341.04   31.45   666.22   709.60    6.51
-p2m    56.59   56.59    0.00   143.59   198.09   37.96   666.78   710.65    6.58
-p3     56.36   56.36    0.00   231.98   300.15   29.39   645.18   697.12    8.05
-p3e    56.27   56.27    0.00   184.86   239.29   29.44   643.99   696.21    8.11
-p3m    56.20   56.20    0.00    92.80   131.10   41.27   643.44   697.58    8.41
-p4     56.03   56.03    0.00   144.21   182.79   26.75   595.68   652.26    9.50
-p4e    55.94   55.94    0.00   123.28   157.95   28.12   594.05   653.27    9.97
-p4m    55.89   55.89    0.00    57.47    82.42   43.41   593.92   652.78    9.91
---------------------------------------------------------------------------------
Compression in percent relative to the original file size. Speed as multiple of realtime playback.

Future

  • I am already working on the next release which will bring unicode support to the command line version. I will also switch from Delphi 6 to Delphi 2007.
  • I am thinking of the implementation of a quick verify function as in WavPack 5.4.0.
  • Unicode support for the gui version.
  • Further preparations for a switch to another development platform. I am still not sure which way to go.

Re: TAK 2.3.1

Reply #3
Long time, no see!

Re: TAK 2.3.1

Reply #4
"Sometimes they come back."

From 'My life in Stephen King quotes'.

Re: TAK 2.3.1

Reply #5
Terrific, thank you sir.

Re: TAK 2.3.1

Reply #6
A release ahead of schedule ;)

I am excited about the future developments too.
"Something bothering you, Mister Spock?"

Re: TAK 2.3.1

Reply #7
Tried to test albums of different genres (i5-3570K 3.4 GHz, Win 10, -p4m -tn4):  :)

Code: [Select]
PS > Get-ChildItem *.wav -File -Name | Foreach-Object { .\taktest 10 $_ }
Version 230, 10 passes ..........
Version 231, 10 passes ..........

carl_orff_carmina_burana.wav, 00:56:15, 2 ch, 24 bit, 88200 Hz

Version Compress Filesize_bytes Enc_dur_sec Encode_speed Decode_speed
------- -------- -------------- ----------- ------------ ------------
WAV          100     1786113572
230        62,06     1108390330       28,12       120,05       218,88
231        62,05     1108333112       21,24       158,88       238,42


Version 230, 10 passes ..........
Version 231, 10 passes ..........

fleshgod_apocalypse_kng.wav, 00:57:30, 2 ch, 24 bit, 48000 Hz

Version Compress Filesize_bytes Enc_dur_sec Encode_speed Decode_speed
------- -------- -------------- ----------- ------------ ------------
WAV          100      993600068
230        76,06      755692758       17,74       194,52       470,69
231        76,03      755481355       14,77       233,65       491,04


Version 230, 10 passes ..........
Version 231, 10 passes ..........

the_prodigy_the_fat_of_the_land.wav, 01:06:07, 2 ch, 16 bit, 44100 Hz

Version Compress Filesize_bytes Enc_dur_sec Encode_speed Decode_speed
------- -------- -------------- ----------- ------------ ------------
WAV          100      699800012
230        69,39      485577995       21,54       184,18       568,81
231        69,39      485569546       17,56       225,88       616,79
“We are not stuff that abides, but patterns that perpetuate themselves.” – Norbert Wiener

Re: TAK 2.3.1

Reply #8
Nice. Thnx :)

Re: TAK 2.3.1

Reply #9
Sorry, didn't realize this topic was in News Submissions, I'll bump it into the News feed.

Re: TAK 2.3.1

Reply #10
I ran a CD image WAV test on Sandy Bridge 2500k WinXP 32-bits and the results show incredible speed improvements (although using HDD is definitely hampering the decoding speeds):
Code: [Select]
mode       2.3.0                2.3.1
-tn4  %ratio  enc   dec    %ratio  enc   dec
----  ------  ----  ----   ------  ----  ----
-p0   70.75%  319*  281*   70.75%  352*  323*
-p0e  70.45%  669*  255*   70.45%  792*  301*
-p0m  70.34%  627*  253*   70.34%  837*  296*
-p1   68.89%  645*  265*   68.89%  820*  298*
-p1e  68.70%  677*  221*   68.70%  895*  287*
-p1m  68.59%  645*  225*   68.45%  812*  300*
-p2   68.45%  709*  141*   68.59%  685*  296*
-p2e  68.27%  711*  238*   68.27%  835*  304*
-p2m  68.16%  401*  270*   68.15%  498*  288*
-p3   68.04%  644*  270*   68.04%  724*  309*
-p3e  68.01%  551*  286*   68.00%  611*  311*
-p3m  67.95%  298*  302*   67.95%  350*  292*
-p4   67.93%  441*  308*   67.93%  504*  295*
-p4e  67.90%  371*  301*   67.90%  429*  292*
-p4m  67.87%  191*  198*   67.87%  228*  321*
Great work, Thomas! (I can PM you the CPU measurements if you want.)

P.S. My earlier comment was regarding the March 28th release was a few days early :)
"Something bothering you, Mister Spock?"

Re: TAK 2.3.1

Reply #11
A release ahead of schedule ;)
Hm... Beta released on April 2, Final on March 28. Huh? Time leap backwards?
No, just the longest beta testing phase ever...

Tried to test albums of different genres (i5-3570K 3.4 GHz, Win 10, -p4m -tn4):  :)
...
Thank you. Nice to see some albeit tiny compression imrovements caused by the bugfix.

I ran a CD image WAV test on Sandy Bridge 2500k WinXP 32-bits and the results show incredible speed improvements (although using HDD is definitely hampering the decoding speeds):
Thank you too. I am very glad to see another familar er... face. My english never was good but the long abstinence from hydrogen definitely has worsened it. I am struggling for words...

I am a bit ashamed about the long delay but i am happy to say that im am just working on the next version.

Re: TAK 2.3.1

Reply #12
Very welcoming news TBeck...TY!  Was also nice to find the new streamlined foobar2000 plugin for TAK.  Something good came out of having to reinstall windows from scratch.  Cheers!


Re: TAK 2.3.1

Reply #14
It would need to have its assembly portions rewritten for aarch64 and NEON intrinsics or opcodes. For now, Rosetta 2 can do a decent enough job converting up to SSE4.2 or SSSE3 to that.

I'm benchmarking FLAC decoding a -8 encoded album, 8 threads, 5 passes:

Total length: 1d 12:24:43.200
Opening time: 0:36.564
Decoding time: 9:51.934
1668.526x realtime

And the encode:

Encode to -p2 took 13 seconds.

And a decoder benchmark of the resulting TAKs, using latest foo_input_tak:

Total length: 1d 12:24:43.200
Opening time: 0:20.804
Decoding time: 9:45.604
1729.307x realtime

Re: TAK 2.3.1

Reply #15
Too late.
Please remove my account from this forum.

Re: TAK 2.3.1

Reply #16
Too late for what?

Re: TAK 2.3.1

Reply #17
Well ... for world domination I guess.

More than fifteen years ago, @bryant wrote some thoughts about why WavPack never really caught on. Arguably, 2005 was already too late to get big - unless being pushed by some major player who for all the stupid reasons insisted on having their own thing that is worse than what was already around. WMAL (2003) arguably was more ephemera than ALAC (2004).

But if the purpose of 2.3.1 was "just" to maintain something that impresses enthusiasts ... well who cares that it didn't arrive in 2020. Right now I see more reason to relegate WMAL to the "Other" section at https://wiki.hydrogenaud.io/index.php?title=Lossless_comparison , since it is abandonware without the raison d'être that TAK/WavPack can claim.


Re: TAK 2.3.1

Reply #19
Tested. Felt a bit bored during the week-end, played around with -p2 and -p0, and then put some FOR loops at nighttime work for -p4m. I know that gives "unfair" speeds when compared to FLAC that I usually use.

I deliberately chose something high-bitrate:
* All my eight 96/24 albums. Seven hours. NIN: The Slip; The Tea Party: Tx 20; Cult of Luna: The Raging River and A Dawn to Fear; Kayo Dot: Hubardo; Clouds Taste Satanic: The Satanic Singles Series; Cascades s/t; and finally the https://opengoldbergvariations.org
* One 96/16 single (6 minutes) snuck in and was treated as if it were a 96/24. My only 88.2/24 "album" to see how it fares with three minutes of grindcore.
* And for the hell of it: a Merzbow CD which has been the worst I could expose TAK to. Yes CDDA this one, 50 minutes.

In addition to putting a hi-rez I am using a fanless computer with a fairly slow CPU, that explains the speeds.

-p0 test encodes, the 96kHz part
Compression:     60.75 %     both 2.3.0 and 2.3.1
Duration:       172.20 sec     for 2.3.0, improves to      163.75      for 2.3.1
Speed:          148.91 * real time     for 2.3.0, improves to      156.60

-p4m encode to spinning SATA drive with verify, the 96kHz part
Compression:     58.38 %     for 2.3.0, improves to      58.37      for 2.3.1, that is 1/100 of a percentage point
Duration:      1025.68 sec    for 2.3.0, improves to      846.12 sec      for 2.3.1
Speed:           25.00 * real time, improves to 30.31.
For comparison, a 2.3.0 test encode took 927.76 sec, 27.64 * real time.

Decoding -p4m using -t (all files combined). Both the HDD and the SSD are SATA-ed to the motherboard. Speeds sorted from slowest to fastest:
211.55: encoded with .1 decoded with .0 on HDD
212.40: encoded with .0 decoded with .0 on HDD
219.58: encoded with .1 decoded with .0 on SSD
220.14: encoded with .0 decoded with .0 on SSD
222.42: encoded with .0 decoded with .1 on HDD
224.67: encoded with .1 decoded with .1 on HDD
225.761 encoded with .1 decoded with foo_input_tak using fb2k's "Decoding speed test" utility (not Takc.exe, but put here in the order)
230.19: encoded with .0 decoded with .1 on SSD
231.17: encoded with .1 decoded with .1 on SSD

It is unfair to compare the high-compression level TAK to FLAC, but for reference the corresponding FLAC -8 files decode at 395 using the fb2k speed test.


Re: TAK 2.3.1

Reply #21
Well ... for world domination I guess.
Support in web browsers could make a huge difference.

Did you hear about FLIF image codec? Its author is working on the JPEG XL now, and all the best ideas from FLIF were used in the new codec, with some cool additions which make the new image codec truly universal. The format itself is basically a mixture of FUIF (from the author of FLIF), PIK and Brunsli image codecs. Google is already testing it in Chromium. When it is supported by major browsers, the format will become much more popular.

A similar thing could happen with a new cool lossless audio codec.

Re: TAK 2.3.1

Reply #22
Additional test, -p0-encoding the 96k .wavs with verification to real .tak files on a real HDD (but SATA to mobo) - compared to FLAC. Nothing surprising here I think, just a confirmation that bloopers like this didn't happen to TAK.

Takc (version 2.3.1) -e -p0 -v 
329 seconds encoding time (read from the output). Compare to 604 seconds for flac (version 1.3.3) --best --verify (read from echo:|time)
2786 bitrate, compare to 2677 for -p4m. 2786 is slightly better than the 2797 for FLAC [note at the end].
fb2k decoding speed test: 244.8. Not much worse than the 225.7 for -p4m. Quite a bit slower than FLAC's 372.7.


I would not have expected anything strange here except if there were some bad assembler optimization. It is qualitatively as what we know from http://www.audiograaf.nl/losslesstest/Lossless%20audio%20codec%20comparison%20-%20revision%204.pdf  :
Best FLAC compression is on par with TAK weakest compression, and FLAC wins big at decoding time.
Fastest FLAC encoding time (--verify -0, but who wants to use that?) is actually on par with fastes TAK encoding time, and then TAK wins big at compression (compared to 2955) and FLAC wins big at decoding speed.


As for the compression ratio: For the eight albums,
* there are two for which TAK -p0 compresses better than FLAC --best on every track in the album
* there are two for which FLAC --best compresses better than TAK -p0 on every track in the album
* the remaining four has at least one track of each.

Re: TAK 2.3.1

Reply #23
First: Thanks to everyone for your interest and testing! It's very encouraging, and i am glad to tell you, that i am making good progress in the implementation of unicode support for the next release of the command line version Takc. Basically it's working, but i have to perform a lot of testing because i had to modify a lot of code. The core of the codec is unaffected.

A release ahead of schedule ;)

Dudes, it was an April 1st reference (in 2006).
Ahhh... You must be a veteran!

I deliberately chose something high-bitrate:
That's definitely interesting. Most of my test files are CD-Audio and i haven't really checked the processing speed of hires audio lately. I will catch up.

In addition to putting a hi-rez I am using a fanless computer with a fairly slow CPU, that explains the speeds.
The new assembly optimizations are using unaligned SSE2 read instructions which is faster on relatively new cpu architectures (Skylake and later) but often considerably slower (than the old code)  on older cpus. This penalty will eat up a lot of the other speed optimazions and the advantage of the new version will be quite small.

Oh, and the 88.2/24 Anal Trump actually compressed worse with 2.3.1.
Available for free here: https://analtrump.bandcamp.com/album/that-makes-me-smart
Strange! How big is the difference? I would have checked it myself but i seem to be too dumb to downlad it for free...

--------------------
Sorry for bad English...
I can do badder!

Re: TAK 2.3.1

Reply #24
Hi Thomas, nice to see progress
  • Test-Set 48 tracks (Classic+ModernTalking+PhilCollins)
  • AMD PhenomII X4 960T BlackEdition, 3.0 GHz, 6 threads (anno 2011), Win7 64bit, RamDisk
Code: [Select]
Enc with -p4m
2.3.0 Enc  131.72s / 98.09x  (52.56%)
2.3.1 Enc  74.01s / 174.39x  (52.56%)

2.3.0 Dec  35.11s / 368.02x
2.3.1 Dec  29.96s / 431.28x
Osna-rocket launched 8)