Final release of TAK 2.3.0 ((T)om's lossless (A)udio (K)ompressor)
This release brings significant speed optimizations for encoder and decoder.
It consists of:
- TAK Applications 2.3.0
- TAK Winamp plugin 2.3.0
- TAK Decoding library 2.3.0
- TAK SDK 1.1.1
Download[attachment=7541:TAK_2.3.0.zip]
What's newImprovements:
- Encoding speed improvements between 5 and 44 percent for my primary file set. The faster presets benefit most. Only the compression performance of preset -p0e has changed. It's now performing about 0.03 percent worse on my primary file set. I modified the preset to make it faster, because -p0e and -p0m got to close after the speed optimizations.
- Decoding speed improvements between 14 and 29 percent for my primary file set. The faster presets benefit most.
New features:
- Two new file info modes -fim4/5 which output one line of raw data per file.
- New mode switch -version to display the program version.
Fixes:
- Takc.exe now contains a proper version info record.
- Removed an obsolete reference to the dedicated LossyWav-codec, which was part of V2.1.0 Beta, from the command line description of Takc.exe.
Many thanks to
Justin Ruggles for reporting this bug! The old decoder would deliver a corrupted frame under these extremely rare conditions:
- The samples are signed and the bit depth isn't higher than 16.
- The frame contains at least 16 samples per channel.
- The frame only contains samples with the maximum negative value (-32768 for 16 bit samples) or value 0.
- The first sample in the frame has the maximum negative value.
Since a frame usually consists of thousands of samples and samples of maximum negative value usually mean clipping, it's extremely unlikely to encounter such data in real life. The probability is higher for the last frame, which can be quite small. But usually the very last samples of a song don't represent only negative clips.
Modifications:
- Added caudec, dsfTAKSource and ImgBurn to the list of software with TAK support in the readme file.
Known issues:
- If you use pipe decoding and the application reading the pipe is beeing terminated before the whole file has been read, TAKC may get into an endless loop and has to be manually killed with the task manager. I don't think this is a big issue but i will try to fix it in one of the next versions. BTW: Big thanks to shnutils for testing the pipe decoding!
- There seem to be some compatibility issues with pipe decoding to some other applications ("crc1632.exe" has been reported). I will try to fix it in the next release.
More informationYou can find some useful information and comparisons in the beta thread (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=101128&view=findpost&p=835794).
Have fun...
Thomas
Here the (single thread) results for my primary file set.
Test system: Pentium Dual Core E5200 (2.5 GHz), Windows 7 / 32 bit.
Preset Compression % Enco-Speed Deco-Speed
---------------------------------------------------------------------------------
2.2.0 2.3.0 Win 2.2.0 2.3.0 Win % 2.2.0 2.3.0 Win %
---------------------------------------------------------------------------------
-p0 58.75 58.75 0.00 390.13 470.54 20.61 389.82 505.05 29.56
-p0e 58.30 58.33 -0.03 239.84 347.38 44.84 390.32 504.16 29.17
-p0m 58.22 58.22 0.00 133.99 192.11 43.38 391.08 506.40 29.49
-p1 57.84 57.84 0.00 312.56 370.65 18.59 386.80 481.24 24.42
-p1e 57.52 57.52 0.00 198.91 218.80 10.00 387.12 481.74 24.44
-p1m 57.42 57.42 0.00 105.05 137.64 31.02 388.84 483.49 24.34
-p2 56.90 56.90 0.00 233.32 259.47 11.21 356.84 427.91 19.92
-p2e 56.70 56.70 0.00 132.38 141.04 6.54 354.78 425.05 19.81
-p2m 56.59 56.59 0.00 65.64 76.69 16.83 356.87 426.95 19.64
-p3 56.36 56.36 0.00 115.94 122.37 5.55 331.84 388.69 17.13
-p3e 56.27 56.27 0.00 80.43 97.20 20.85 332.10 388.82 17.08
-p3m 56.20 56.20 0.00 43.93 48.51 10.43 331.16 387.65 17.06
-p4 56.03 56.03 0.00 63.65 73.28 15.13 302.79 348.11 14.97
-p4e 55.94 55.94 0.00 55.21 62.29 12.82 302.37 347.78 15.02
-p4m 55.89 55.89 0.00 28.09 29.95 6.62 301.86 347.56 15.14
---------------------------------------------------------------------------------
Compression in percent relative to the original file size. Speed as multiple of realtime playback.
Added caudec, dsfTAKSource and ImgBurn to the list of software with TAK support in the readme file.
Thanks for the inclusion!
Encoding a double disc album (37 files for a total of 2 hours, 27 minutes, 1494 MiB) with caudec, 8 processes, 2.2GHz mobile Core i7 CPU:
- FLAC -8: 40.67 seconds, 218.3x, 1021.7 MiB, 965 kbps
- FLAC -5: 14.91 seconds, 595.6x, 1023.6 MiB, 967 kbps
- TAK -p2: 10.3 seconds, 862.3x, 998.6 MiB, 943 kbps
- TAK -p0: 7.57 seconds, 1173.3x, 1022.7 MiB, 966 kbps
Decoding:
- FLAC -8: 5.23 seconds, 1697.2x
- FLAC -5: 4.99 seconds, 1778.5x
- TAK -p2: 10.32 seconds, 860.7x
- TAK -p0: 5.96 seconds, 1489.2x
Excellent!
Another interesting use case: a 74 minutes album, single WAV file.
Encoding:
- FLAC --best: 73.56 seconds, 490 MiB
- TAK -p0 -tn4: 7.02 seconds (that's not a typo), 490 MiB
- TAK -p0 -tn1: 10.45 seconds, 490 MiB
Decoding:
- FLAC --best: 8.10 seconds
- TAK -p0: 8.24 seconds
Edit: it has been brought to my attention that such comparisons are not fair to FLAC, since faster implementations exist, such as FLACCL. I'm unable to run the latter, however. And even so, comparing a GPU implementation to a CPU implementation wouldn't be fair either. So it seems that in the current state of affairs, no TAK comparison will satisfy all parties.
Added caudec, dsfTAKSource and ImgBurn to the list of software with TAK support in the readme file.
Thanks for the inclusion!
Im happy if there is something good to add!
- TAK -p0 -tn4: 7.02 seconds (that's not a typo), 490 MiB
- TAK -p0 -tn1: 10.45 seconds, 490 MiB
If neither file io nor ram speed are the bottleneck, this might be another hint, that i have to improve the multi-thread-performance...
Edit: it has been brought to my attention that such comparisons are not fair to FLAC, since faster implementations exist, such as FLACCL. I'm unable to run the latter, however. And even so, comparing a GPU implementation to a CPU implementation wouldn't be fair either. So it seems that in the current state of affairs, no TAK comparison will satisfy all parties.
It must be unfair, if a codec has been designed in a way, that makes it very fast without using brute force...
And in the past some people argued the other way round: "Comparisons between TAK and FLAC are unfair, because TAK is using MMX (what FLAC does too) or even SSSE3."
Shall i predict, what comes next? "TAK will never play in FLAC's league, because it's cpu usage on decoding is 0.020 instead of 0.015 percent!"
Please don't take me wrong: I think, the FLAC-team is doing a great job. And FLACCL is a great piece of software too.
But sometimes i am really irritated, what strange arguments some people are using, if something does not fit into their world view (possibly the wrong term).
But sometimes i am really irritated, what strange arguments some people are using, if something does not fit into their world view (possibly the wrong term).
How about reserving judgment when all you have is a decontextualized misrepresentation of a conversation which happened somewhere else w/o you present.
Much as the "source code" debate there is really no need for taking this topic off the rails on silly issues.
But sometimes i am really irritated, what strange arguments some people are using, if something does not fit into their world view (possibly the wrong term).
How about reserving judgment when all you have is a decontextualized misrepresentation of a conversation which happened somewhere else w/o you present.
Good point.
It's not that i was out for confrontation. That's the last i am looking for when i release a new version. But the post triggered memories of bad experiences with earlier releases.
You are right. I did overreact.
It's not that i was out for confrontation.
I apologize if I made it seem as if I were calling you out as "looking for confrontation."
It's not that i was out for confrontation.
I apologize if I made it seem as if I were calling you out as "looking for confrontation."
Thank you. It was just my interpretation, not really backed up by your words.
And i apologize for having been too thin-skinned. Well, not the best start for a release thread...
No harm/no foul.
Since the issue was raised here (as well as in the beta thread), posts complaining about the availability of source code will be viewed as trolling and will be binned without regard to any other information they may contain (useful or otherwise).
It's a shame that I feel the need to repeat this for every new announcement about TAK.
If neither file io nor ram speed are the bottleneck, this might be another hint, that i have to improve the multi-thread-performance...
That equates to about 106MiB/s, and I did it all on a ramdisk, so there is likely no I/O bottleneck in my tests.
- -p0 -tn4 uses about 215% of my CPU (i.e. a little over 2 CPU cores)
- -p1 -tn4 uses about 250%
- -p2 -tn4 uses about 340%
- -p3 -tn4 uses about 450%
- -p4 -tn4 uses about 410%
Note that my CPU has 4 physical cores, with "Hyperthreading" (2 threads per core). Though I'm not sure if Wine has any effect on my results (I don't think so, but I can't be sure).
Really some food for thought...
I have to admit, that i don't have much experience with multi-threading. I can't estimate, how much faster the multi-thread-encoder will be, when i have corrected the design flaw i know of.
But i am optimistic.
Note that my CPU has 4 physical cores, with "Hyperthreading" (2 threads per core). Though I'm not sure if Wine has any effect on my results (I don't think so, but I can't be sure).
As long as Wine is prioritizing the physical cores for -tn4, it should be ok.
I just finished a TAK/FLAC encoding comparison test for three albums on a RAM disk. The machine is a Core 2 Quad Q9550 (2,83 GHz). The results may be a little shaky as the machine was under occasional slight load from a local server, a couple of file watchers and an IDE (all mostly idle).
Since FLAC doesn't seem to have an option to do a test encode only without writing anything to the disk, I tested the normal encode of TAK for the sake of comparability and TAK's because I wanted to see how fast it can get if it doesn't have to write a file anywhere.
Abbreviations:
- Comp: Compression ratio in %
- tEnc(1): Encoding time using 1 thread (-tn1) in seconds
- xRT(1): Realtime factor using 1 thread
- tEnc(4) Encoding time using 4 threads (-tn4) in seconds
- xRT(4): Realtime factor using 4 threads
- tEst(4): Test encoding time using 4 threads (-te -tn4) in seconds
- xRTtest(4): Realtime factor for test encoding using 4 threads
Test albums (as single WAV files each):
- Barrington Pheloung - Broken Sword Director's Cut OST (40:46 min, 411 MByte)
- Disasterpeace - FEZ OST (78:08 min, 788 MByte)
- OCRemix - Deus Ex: Sonic Augmentation (36:13 min, 365 MByte)
Results:
Broken Sword Director's Cut OST
Codec/preset Comp tE(1) xRT(1) tE(4) xRT(4) tEst(4) xRTtest(4)
TAK -p0 44,69 4,88 501,39 2,12 1151,21 1,52 1604,29
TAK -p0e 44,32 6,49 377,10 2,36 1036,75 2,00 1221,70
TAK -p0m 44,23 11,41 214,48 3,76 651,38 3,49 700,56
TAK -p1 44,09 5,91 414,04 2,19 1119,18 1,97 1239,26
TAK -p1e 43,78 9,88 247,65 3,39 721,17 3,01 811,62
TAK -p1m 43,71 15,46 158,25 5,37 455,85 4,77 513,09
TAK -p2 43,33 8,13 301,07 2,76 891,70 2,52 971,64
TAK -p2e 43,25 14,79 165,44 4,98 490,80 4,69 521,62
TAK -p2m 43,17 29,37 83,29 8,65 282,75 8,20 298,44
TAK -p3 42,52 16,57 147,65 5,55 440,86 5,07 482,52
TAK -p3e 42,46 21,10 115,94 6,69 365,51 6,41 381,86
TAK -p3m 42,43 40,12 60,97 12,73 192,19 12,27 199,40
TAK -p4 41,86 27,14 90,13 8,75 276,47 8,30 294,65
TAK -p4e 41,83 32,27 75,81 10,27 238,22 9,93 246,30
TAK -p4m 41,81 60,18 40,24 19,39 126,17 19,32 126,61
FLAC -0 49,7 8,78 278,51
FLAC -1 47,8 9,33 262,20
FLAC -2 47,3 10,97 223,04
FLAC -3 47,3 12,03 203,38
FLAC -4 45,3 14,00 174,61
FLAC -5 45,0 18,30 133,67
FLAC -6 45,0 18,46 132,54
FLAC -7 44,9 35,07 69,75
FLAC -8 44,8 51,87 47,15
FEZ OST
Codec/preset Comp tEnc(1) xRT(1) tEnc(4) xRT(4) tEst(4) xRTtest(4)
TAK -p0 59,75 10,13 462,66 4,39 1068,33 3,52 1332,33
TAK -p0e 59,44 13,19 355,44 4,89 959,13 4,00 1171,39
TAK -p0m 59,19 22,48 208,55 7,40 633,46 6,81 688,82
TAK -p1 58,99 11,99 390,97 4,56 1027,12 3,61 1298,82
TAK -p1e 58,92 19,66 238,41 6,66 704,13 5,94 789,51
TAK -p1m 58,71 30,14 155,53 9,94 471,79 9,15 512,09
TAK -p2 58,52 16,09 291,42 5,58 840,45 4,89 959,47
TAK -p2e 58,48 28,71 163,27 9,77 479,68 9,06 517,34
TAK -p2m 58,28 51,73 90,61 16,77 279,51 15,88 295,20
TAK -p3 57,84 31,95 146,69 10,28 456,05 9,73 481,89
TAK -p3e 57,70 40,44 115,93 13,07 358,78 12,28 381,64
TAK -p3m 57,67 75,82 61,83 23,67 198,01 22,98 204,01
TAK -p4 57,12 51,22 91,51 16,41 285,72 15,69 298,74
TAK -p4e 57,11 60,89 76,98 19,86 235,98 18,70 250,61
TAK -p4m 57,09 110,78 42,31 38,19 122,74 34,57 135,59
FLAC -0 62,4 18,23 257,05
FLAC -1 61,7 19,49 240,57
FLAC -2 61,5 22,85 205,11
FLAC -3 59,9 23,96 195,62
FLAC -4 59,0 27,75 168,90
FLAC -5 58,9 35,99 130,25
FLAC -6 58,9 36,44 128,63
FLAC -7 58,9 70,34 66,64
FLAC -8 58,8 102,52 45,72
Deus Ex: Sonic Augmentation
Codec/preset Comp tEnc(1) xRT(1) tEnc(4) xRT(4) tEst(4) xRTtest(4)
TAK -p0 68,09 4,83 450,11 2,21 983,19 1,63 1329,46
TAK -p0e 67,69 6,25 347,38 2,37 917,68 1,88 1154,36
TAK -p0m 67,59 10,74 202,35 3,77 575,57 3,26 666,80
TAK -p1 67,05 5,84 372,09 2,27 956,61 1,73 1255,01
TAK -p1e 66,81 9,55 227,40 3,49 623,28 2,87 757,20
TAK -p1m 66,71 14,75 147,30 5,30 410,11 4,55 477,30
TAK -p2 66,45 8,12 267,52 2,96 735,01 2,49 871,00
TAK -p2e 66,32 14,10 154,05 5,30 409,59 4,62 469,93
TAK -p2m 66,20 25,78 84,26 9,35 232,36 8,40 258,74
TAK -p3 66,06 16,27 133,53 5,93 366,20 5,29 410,53
TAK -p3e 66,00 20,17 107,74 7,48 290,57 6,66 326,17
TAK -p3m 65,94 40,22 54,02 14,57 149,14 13,63 159,43
TAK -p4 65,85 26,75 81,22 9,95 218,34 9,22 253,72
TAK -p4e 65,79 31,18 69,68 11,59 187,47 10,73 202,42
TAK -p4m 65,75 64,87 33,49 24,72 87,89 23,50 92,45
FLAC -0 71,9 9,92 218,99
FLAC -1 71,1 9,77 222,49
FLAC -2 70,8 11,17 194,50
FLAC -3 69,7 11,72 185,44
FLAC -4 68,5 13,37 162,51
FLAC -5 68,3 17,27 125,81
FLAC -6 68,3 17,50 124,13
FLAC -7 68,2 32,81 66,22
FLAC -8 68,0 47,88 45,37
I double-checked that there are no typos. This thing is ridiculously fast.
Great, thanks.
No streaming support? foo_input_tak failed and now Winamp Plugin too.
The SDK header appears to be missing a few things, like tak_SSD_GetStreamInfo_V22, TtakAudioFormat::HasExtension, TtakAudioFormat::HasSpeakerAssignment, TtakAudioFormat::ValidBitsPerSample, tak_GetWaveExtensibleSpeakerMask, TMD5State, and tak_SSD_GetMD5, all of which foo_input_tak uses. I could locate and pull tak_deco_lib.h from an older beta SDK, but it would be nice if I could get these with the latest SDK. Some or all of them do appear to be in the import library, and they are in the dll, as it works with the current binary version of the component.
As far as I understand, foo_input_tak was using non-official, hacked version of tak_deco_lib.h (by lvqcl) to make use of those functions/structures that exist in the official 2.2.0 lib binary but are not declared in the official C header.
I also love to see official tak_deco_lib.h updated. But since library ABI hasn't changed, I think foo_input_tak can keep using it's hacked tak_deco_lib.h until then.
Oh, I wasn't aware that he hacked the API header file.
As for the ever recurring question on opening the source: Have a look at
http://wiki.hydrogenaudio.org/index.php?ti...Asked_Questions (http://wiki.hydrogenaudio.org/index.php?title=Tak#Frequently_Asked_Questions)
This is a case where users who take an effort searching the forum and reading FAQs before asking questions, only will get knocked over it. Internet forums would normally approve upon new users reading FAQs rather than asking them, and when FAQreading will get you labeled a troll, then something is wrong somewhere – hopefully in the FAQ.
Would a phrasing like
[blockquote]“There is an independently implemented open source decoder available with ffmpeg. The official encoder/decoder is closed source; Thomas did express intention to open the source at some point in time, but as of June 2013 he thinks ‘a lot of (not very exciting) work is required’ until the code is ready to be published, and that may or may not happen in the foreseeable future. This is a question that generally generates more noise than fruitful discussion.”[/blockquote]be (i) useful to the reader, and (ii) appropriate (I took the quote from the beta thread)?
In addition the “What can I compress” question only refers to what v1.0.3 can do. Maybe update to, whatever the current status is?
I just finished a TAK/FLAC encoding comparison test for three albums on a RAM disk. The machine is a Core 2 Quad Q9550 (2,83 GHz). The results may be a little shaky as the machine was under occasional slight load from a local server, a couple of file watchers and an IDE (all mostly idle).
Thank you very much for those interesting results!
Some quick thoughts:
1. I wouldn't have expected a significant encoding speed difference between -e and -te if a ram disk is beeing used. Interesting.
2. Two of your albums show nice improvements for -p4m over -p3m. They seem to benefit from the higher predictor order (160 vs. 80). I already contemplated about the possibility to raise the maximum predictor count for -p4m to 256, what would be a backwards compatible modification.
No streaming support? foo_input_tak failed and now Winamp Plugin too.
To be honest, i never use it. I am not even sure, if i exactly know what you mean... Well, i am performing very little audio processing and my playback setup is extremely simple and unambitious.
What i can say: The TAK format is perepared to connect to a running stream (which can not be seeked) at any time, but neither the tak_deco_lib nor my WinAmp-plugin provide support for it.
Possibly more knowledgeable peope than me can help a bit?
As far as I understand, foo_input_tak was using non-official, hacked version of tak_deco_lib.h (by lvqcl) to make use of those functions/structures that exist in the official 2.2.0 lib binary but are not declared in the official C header.
I also love to see official tak_deco_lib.h updated. But since library ABI hasn't changed, I think foo_input_tak can keep using it's hacked tak_deco_lib.h until then.
You are right. And it is safe to use the undocumented functions the lib exports. They will not be removed or modified unless a fundamental format change takes place or a severe bug or design flaw has to be fixed.
And i really should update the SDK! But i had not enough time. V2.3.0 rather was a suprise-release, not only for the users but for me too. I had some ideas and suddenly some time to implement them. More wasn't possible this time.
Would a phrasing like
[blockquote]“There is an independently implemented open source decoder available with ffmpeg. The official encoder/decoder is closed source; Thomas did express intention to open the source at some point in time, but as of June 2013 he thinks ‘a lot of (not very exciting) work is required’ until the code is ready to be published, and that may or may not happen in the foreseeable future. This is a question that generally generates more noise than fruitful discussion.”[/blockquote]be (i) useful to the reader, and (ii) appropriate (I took the quote from the beta thread)?
That's not exact, but it has to be, because this issue is so delicate . I have expressed the intention to release an open source
decoder. That would be the first realistic step.
But otherwise a good idea!
I tried to combine the suggestion by Porcus with both your reply and what was previously on the page:
The official encoder and decoder are currently closed-source. Thomas has expressed an intention to open the source of the decoder at some point in time, stipulating preconditions of its first being further refined, ported to C or C++, and documented. This may or may not lead to releases of other code. However, as of June of 2013, he feels that “a lot of (not very exciting) work is required” until the decoding source would be ready to be published, and that may or may not happen in the foreseeable future. Such questions generally generate more noise than fruitful discussion, so it is best to wait and see what happens. In any case, there is an independently implemented open source decoder available, bundled with ffmpeg.
I hope this is along the right lines. Of course, just change anything that you want to.
That's not exact, but it has to be, because this issue is so delicate . I have expressed the intention to release an open source decoder. That would be the first realistic step.
Ah, I should have read better, sorry.
Revised suggestion:
[blockquote]Is TAK available as open source?
ffmpeg offers a free open source decoder. Thomas did express intention to release an official decoder as open source, but as of June 2013 he thinks
‘a lot of (not very exciting) work is required’ until the code is ready to be published. There are no plans to release the encoder source code.[/blockquote]
The warning over noise content can be included or not.
edit: db1989 beat me at it. Oh well, someone can just take the editor-in-chief role and pick the wording. I tend to think that the ffmpeg implementation could be mentioned first (so users can get that “yes” first without having to read the detail).
I tried to combine the suggestion by Porcus with both your reply and what was previously on the page:
Thank you! This seems to be as cautiously worded as possible. Fine.
I would like to put it in more concrete terms, but i can't.
I've tested the Broken Sword Director's Cut OST with the LossyWAV 1.3.0 treatment. I skipped the -tn1 encodings as their encoding speed would have been quite predictably slower than that of the -tn4 encodings and otherwise uninteresting. Bear in mind that I used a different machine for this test (Core i7-3740QM, 2,7-3,7 GHz), so the encoding times do not compare to the other times posted above.
The command line used was "takc -e -p[XX] -fsl512 -tn4 bs-[Y].lossy.wav"
Results:
Broken Sword Director's Cut OST (40:46 min, 411 MByte)
LossyWAV --quality insane LossyWAV --quality standard LossyWAV --quality extraportable
Mode Comp tEnc(4) Mode Comp tEnc(4) Mode Comp tEnc(4)
-p0 38,18 % 2,94 s -p0 29,16 % 2,75 s -p0 19,82 % 2,66 s
-p0e 37,71 % 2,93 s -p0e 28,70 % 2,78 s -p0e 19,61 % 2,67 s
-p0m 37,65 % 3,65 s -p0m 28,64 % 3,55 s -p0m 19,56 % 3,44 s
-p1 37,80 % 2,88 s -p1 28,82 % 2,76 s -p1 19,55 % 2,65 s
-p1e 37,33 % 3,31 s -p1e 28,37 % 3,28 s -p1e 19,33 % 3,24 s
-p1m 37,29 % 5,27 s -p1m 28,33 % 5,18 s -p1m 19,31 % 5,14 s
-p2 37,39 % 3,13 s -p2 28,45 % 3,20 s -p2 19,39 % 3,21 s
-p2e 37,29 % 6,53 s -p2e 28,35 % 6,52 s -p2e 19,30 % 6,72 s
-p2m 37,12 % 11,71 s -p2m 28,20 % 11,82 s -p2m 19,23 % 11,90 s
-p3 37,24 % 5,29 s -p3 28,30 % 5,25 s -p3 19,27 % 5,29 s
-p3e 37,08 % 8,09 s -p3e 28,17 % 8,05 s -p3e 19,21 % 8,00 s
-p3m 37,08 % 13,61 s -p3m 28,16 % 13,41 s -p3m 19,20 % 13,99 s
-p4 37,20 % 5,73 s -p4 28,27 % 5,79 s -p4 19,25 % 5,73 s
-p4e 37,08 % 8,09 s -p4e 28,16 % 8,31 s -p4e 19,21 % 8,36 s
-p4m 37,08 % 13,58 s -p4m 28,16 % 13,51 s -p4m 19,20 % 13,88 s
Interesting details:
- Basically, -p3e and -p4e yield the same results regarding compression AND encoding time, same with -p3m and -p4m. I had expected negligible compression differences, but I did not expect the complete lack of speed differences.
- Contrary to the non-LossyWAV test, -pXe -pXm compress better than -p(X+1) with the exception of -p0e/-p0m.
- Not in the results: When not using -fsl512, for every file the -p4/-p4e/-p4m encodings compress worse than even the -p1/-p1e/-p1m encodings.
Basically, -p3e and -p4e yield the same results regarding compression AND encoding time, same with -p3m and -p4m. I had expected negligible compression differences, but I did not expect the complete lack of speed differences.
No surprise. -p3x and -p4m support up to 80 respectively 160 predictors. If the frame size is limited to 512, the overhead for storing that much predictors is too high. Only the dedicated LossyWav-codec, which was available in TAK 2.1.0 Beta, could take advantage of such high predictor orders. And TAK will not even try higher predictor orders, if the frame is too small. Therefore only tiny speed differences.
Not in the results: When not using -fsl512, for every file the -p4/-p4e/-p4m encodings compress worse than even the -p1/-p1e/-p1m encodings.
Again no surprise. Larger frames have to use the lowest wasted bit count of the 512-samples LossyWav-sub frames the frame contains. Only such files, where LossyWav deceides to remove nearly not bits, would benefit from larger TAK-Frames. Well, unless you use the dedicated LossyWav-codec, which is quite immune to such effects. It will nearly always achieve the optimum result.
I noticed that the encoder fails to work for regular ansi filenames if path happens to contain unicode characters. For example open command prompt in dir called "Pośród kwiatów i cieni" and try to encode a simple "test.wav" there. Usually encoders, even if they don't support unicode, have no problems doing this.
Hey I just wanted to say that TAK now is faster than ever and I love it ! My music playlist which consists of 2300+ songs is all in TAK because it's much faster than FLAC both when encoding and decoding. (I use OGG on my DAP, which is a Sansa ClipZip).
So thanks TBeck ! I wish the best for this format's future.
This is not by any means a neutral test, it is the least-FLACable piece of music from my collection, and it is the
least flattering for TAK that I have ever come across (it is WavPack-friendly as long as time is no object ...). Consider it a worst-case test.
Music is Merzbow:
I lead you towards glorous times, track 3 from Venereology (1994). For a listen: http://www.youtube.com/watch?v=gTRZdFqAOGA (http://www.youtube.com/watch?v=gTRZdFqAOGA)
Hardware: an old p4 3.06 with spinning hard drive.
tak.exe (GUI) used. No tweaking of options used except standard presets. No firm procedure for repeating, this is not meant to be taken as hard science, but the decoding tests were repeated because they surprised me.
Encoding using “test” (as I guessed writing is the bottleneck, definitely looks to be the case) and “compress”, p3 p3e p3m omitted because I'm lazy, but they all fail to break the 100% threshold.
p0 : test=219x encode=131x, size=100.96%
p0e: test=153x encode=89x, size=100.94%
p0m: test=86x encode=69x, size=100.50%
p1 : test=159x encode=83x, size=100.51%
p1e: test=92x encode=69x, size=100.52%
p1m: test=60x encode=52x, size=100.27%
p2 : test=113x encode=79x, size=100.30%
p2e: test=62x encode=51x, size=100.29%
p2m: test=34x encode=31x, size=100.15%
p4: test=38x encode=36x size=99.92%
p4e: test=31x encode=29x, size=100.04%
p4m: test=17x encode=17x, size=100.04%
For comparison:
FLAC-8 encoding to file: about 20x, size=98.8%.
Decoding using “test”. This result was so surprising that I repeated the experiment a few times – these are representative figures.
Glorious Times-p0.tak 170* Ok
Glorious Times-p0e.tak 134* Ok
Glorious Times-p0m.tak 165* Ok
Glorious Times-p1.tak 92* Ok
Glorious Times-p1e.tak 147* Ok
Glorious Times-p1m.tak 168* Ok
Glorious Times-p2.tak 121* Ok
Glorious Times-p2e.tak 140* Ok
Glorious Times-p2m.tak 177* Ok
Glorious Times-p4.tak 180* Ok
Glorious Times-p4e.tak 173* Ok
Glorious Times-p4m.tak 141* Ok
Duration: 27.26 sec
Speed: 145.17 * real time
Surprise – the higher compressed decode faster!? Would have made sense if there were large differences in file size which had to be read from drive, but these are virtually identical in size. This experiment was repeated due to the surprising result.
Decoding using “decompress”, writing to file; repeated due to the previous result being such a surprise. Somehow I don't get very consistent results here, but this is not way off:
Glorious Times-p0.tak 83* Ok
Glorious Times-p0e.tak 69* Ok
Glorious Times-p0m.tak 70* Ok
Glorious Times-p1.tak 58* Ok
Glorious Times-p1e.tak 77* Ok
Glorious Times-p1m.tak 58* Ok
Glorious Times-p2.tak 64* Ok
Glorious Times-p2e.tak 73* Ok
Glorious Times-p2m.tak 58* Ok
Glorious Times-p4.tak 48* Ok
Glorious Times-p4e.tak 72* Ok
Glorious Times-p4m.tak 60* Ok
Duration: 61.42 sec
Speed: 64.44 * real time
Seems obvious that speed is way beyond spinning drives.
I noticed that the encoder fails to work for regular ansi filenames if path happens to contain unicode characters. For example open command prompt in dir called "Po?ród kwiatów i cieni" and try to encode a simple "test.wav" there. Usually encoders, even if they don't support unicode, have no problems doing this.
I suppose this fails, because TAK always generates a full path specification. In your case it will determine the current directory and add it to the file name. Unfortunately this fails because of TAK's lack of unicode support. Indeed, the lack of unicode support is annoying...
Hey I just wanted to say that TAK now is faster than ever and I love it ! My music playlist which consists of 2300+ songs is all in TAK because it's much faster than FLAC both when encoding and decoding. (I use OGG on my DAP, which is a Sansa ClipZip).
So thanks TBeck ! I wish the best for this format's future.
Thank you very much!
However i don't expect TAK to decode much faster than FLAC. Your system must be something special.
This is not by any means a neutral test, it is the least-FLACable piece of music from my collection, and it is the least flattering for TAK that I have ever come across (it is WavPack-friendly as long as time is no object ...). Consider it a worst-case test.
I am always interested in such special files. Could you send me a short snippet?
I just want to say thank you for TBeck for making such an awesome codec. I just started using tak yesterday and must say I'm really impressed by it.
Excellent compression and encoding time. In -p4m i can encode on my system at ~108x. I've been using FLAC for over 10 years now for all my music CD's as well as recorded vinyl and i'm seriously thinking converting all to TAK.
I must congratulate you also on the best command line information i've seen in a CLI tool. You really don't need a manual for it.
Continue the good work!
Thank you. That's very encouraging.
Continue the good work!
I will! Well, if time permits... I've just optimized a new compression technique, which was encoding far too slow: 0.1 * realtime on my PC. Now it's more than 250 * realtime and therefore practicable. But it's integration would require a format change and the compression improvement isn't big enough to justify it. So i will have to look for more tuning oppurtunities. At some point the sum of improvements may be sufficient.
It's simply getting more and more difficult to squeeze out some more compression without significantly affecting the decoding speed.
I'm trying to get takc to work with CUETools to runs some tests, but it always fails with
"Takc.exe has exited prematurely with code 2: The pipe has been ended."
This is the command line I'm using: takc -e -p%M -md5 -overwrite - %O
%M is replaced by the profiles, %O by the outfile by CUETools. If I try to run the command on a testfile in the command prompt, it works since I don't use a pipe in that case. Wavpack works fine with CUETools and a similar configuration, what am I missing?
I wish rockbox supported TAK, I'm interested in which battery times I would get with it.
I have no experience with CUETools, therefore there will be more knowledgeable people than me to answer.
But possibly the addition of the "-ihs" switch might help. It tells takc to ignore the files size defined in the wave header, which is often 0 if piping is beeing used.
Argh. It's because the outfile contains non-ansi characters, so it's the dreaded "unicode not supported" case again >.<
I'm not that knowledgeable about it, but why didn't you include support for unicode right from the start?
try foobar instead? that uses temporary filenames when encoding and then renames them when done. so it really doesn't matter whether the encoders support unicode or not.
try foobar instead? that uses temporary filenames when encoding and then renames them when done. so it really doesn't matter whether the encoders support unicode or not.
In many cases it works for non Unicode supporting CLI encoders because they don't care or use full path name of those temporary filenames.
However, as is reported by Case and confirmed by TBeck in this thread, it will fail for takc if the target directory name contains Unicode characters not present in your locale, since fb2k creates the temporary file in the target directory, and takc constructs full path name from it.
For some reason, I don't have that problem when running TAK with Wine. It accepts filenames with UTF-8 characters just fine. Or maybe I misunderstood the nature of the problem?
I will! Well, if time permits... I've just optimized a new compression technique, which was encoding far too slow: 0.1 * realtime on my PC. Now it's more than 250 * realtime and therefore practicable. But it's integration would require a format change and the compression improvement isn't big enough to justify it. So i will have to look for more tuning oppurtunities. At some point the sum of improvements may be sufficient.
It's simply getting more and more difficult to squeeze out some more compression without significantly affecting the decoding speed.
I want to venture that most TAK users do not have a major issue with a format change, as longs as: that it is accompanied by a major revision number (i.e. TAK 3.xx); and, backwards compatibility for
decoding previous versions exists.
Of course, I expect you (the developer) already knows this. I merely state in public that fears over 'format change' might be exaggerated (unless *somehow* the reverse-engineered TAK decoder gained a lot of traction :shrug:). Looking at all the other lossless codecs, changing the format seems to derive from necessity and evolution via accumulative enhancements.
Whatever may be decided, I will try to stay updated and active with TAK hopefully as contributor rather than encumbrance
But it does imply upgrading decoders for playing the new files.
I want to venture that most TAK users do not have a major issue with a format change, as longs as: that it is accompanied by a major revision number (i.e. TAK 3.xx); and, backwards compatibility for decoding previous versions exists.
I wholeheartedly agree with this.
I'm not that knowledgeable about it, but why didn't you include support for unicode right from the start?
Because my old Delphi 6 from 2001 provides very little (none for the GUI) unicode support and TAK uses some libraries i have written long ago which too don't support unicode. Even the implementation of unicode support for the command line version only would be a lot of work. Currently i am not sure, if i will implement unicode support before porting TAK to C++.
Because this definitely will take quite long and of course i understand how important unicode support is, i may use a top-down-approach for the first step of the port:
- Put the Delphi codec core into a library (DLL), which can be called by C++.
- Port the much less comprehensive application logic to C++ and add unicode support.
But which road i will take depends on many factors i can't foresee. Therefore i can't make a clear statement.
Of course, I expect you (the developer) already knows this. I merely state in public that fears over 'format change' might be exaggerated (unless *somehow* the reverse-engineered TAK decoder gained a lot of traction :shrug:). Looking at all the other lossless codecs, changing the format seems to derive from necessity and evolution via accumulative enhancements.
I want to venture that most TAK users do not have a major issue with a format change, as longs as: that it is accompanied by a major revision number (i.e. TAK 3.xx); and, backwards compatibility for decoding previous versions exists.
I wholeheartedly agree with this.
I definitely don't intend to remove decoding support for older codec versions. If i ever had thought about it, the latest release of the great Monkey's Audio would have taught me better... But at some point i will remove the assembler optimizations for old versions. This will simplify the work on an open source decoder release. It's quite possible that one of the next TAK releases will come without assembler optimizations for V1.x files. I don't think that's a big issue. Decoding will still be quite fast.
unless *somehow* the reverse-engineered TAK decoder gained a lot of traction
What i would like. Yes, i have changed my mind... I don't think i will alter the format without releasing an open source decoder. I would like to make it easy for the ffmpeg developers to implement the new format.
But it does imply upgrading decoders for playing the new files.
Of course.
try foobar instead? that uses temporary filenames when encoding and then renames them when done. so it really doesn't matter whether the encoders support unicode or not.
Just to note that i've converted from FLAC over 60 albums with Foobar till now and haven't experienced any problems with filenames using lots of (´ ` ~ ^ ç ...and so on) on the filename characters, it converts everything flawlessly. So maybe it really depends how the applications pass the original names to the *.tak destination file or how they use the pipe for that matter.
Filename is taken care by fb2k, so you should have no problem if path to the destination directory doesn't contain Unicode characters not present in your code page. Otherwise it will fail.
If you don't get it, try encoding to C:\❤\☀\ or something.
Filename is taken care by fb2k, so you should have no problem if path to the destination directory doesn't contain Unicode characters not present in your code page. Otherwise it will fail.
If you don't get it, try encoding to C:\?\?\ or something.
Indeed:
1 out of 1 tracks converted with major problems.
Source: "D:\WAV Audio\02 Éàçãô.wav"
An error occurred while writing to file (The encoder has terminated prematurely with code 2 (0x00000002); please re-check parameters) : "D:\?\02 Éàçãô.tak"
Additional information:
Encoder stream format: 44100Hz / 2ch / 16bps
Command line: "C:\Users\Main\AppData\Roaming\foobar2000\encoders\Takc.exe" -e -p4m -tn4 -ihs - "temp-A849E32B71D002A5CD61CFDEA5B46919.tak"
Working folder: D:\?\
Conversion failed: The encoder has terminated prematurely with code 2 (0x00000002); please re-check parameters
But by using this example I very much doubt I will encounter any problems whatsoever with my files, but nonetheless i understand the problem better now. Thanks.
Filename is taken care by fb2k, so you should have no problem if path to the destination directory doesn't contain Unicode characters not present in your code page. Otherwise it will fail.
If you don't get it, try encoding to C:\?\?\ or something.
Well I don't know, this is what I'm getting if I try to convert this album:
Source: "\\server\music\Sawano Hiroyuki\Shingeki no Kyojin OST\01. ???? - ?t?æk 0N t??tn.flac"
An error occurred while finalizing the encoding process (Object not found) : "R:\01. ???? - ?t?æk 0N t??tn.tak"
There are no unicode characters in the space and the conversion is running, but fails on rename.
It also fails on more normal filenames, for example:
1 out of 1 tracks converted with major problems.
Source: "\\server\music\Sawano Hiroyuki\Shingeki no Kyojin OST\07. Cyua - Vogel im Käfig.flac"
An error occurred while finalizing the encoding process (Object not found) : "R:\07. Cyua - Vogel im Käfig.tak"
Conversion failed: Object not found
Only for .tak though, every other format is working.
edit: R:\ is my RAMdisk, up to 5GB space available. Not that it seems to matter, converting to another directory ends with the same error
Yes, the example above is quite unnaturally made up and you might not meet this kind of problem.
However, in my environment (Japanese, CP932), quite many of artist / album name (which will be naturally used as a folder name) actually bring the problem, because latin accent letters are missing in our code page, and available only through Unicode.
It's not made up though, just irregular: click (http://shingeki.tv/music/soundtrack.php).
I'm running Japanese locale myself, and even using filenames that only contain kanji will lead to the same error:
1 out of 1 tracks converted with major problems.
Source: "\\server\music\01. 戦場ヶ原ひたぎ(斎藤千和) - 二言目.flac"
An error occurred while finalizing the encoding process (Object not found) : "R:\01. 戦場ヶ原ひたぎ(斎藤千和) - 二言目.tak"
Conversion failed: Object not found
ChronoSphere, I'm quite certain your problem is caused by incorrect command line parameters. If you use pipes with foobar2000 you need to add -ihs parameter for TAK, otherwise it will remove the encoded file in the assumption that something went wrong when length didn't match.
Adding -ihs fixes that, indeed. I did read the takc help, but it wasn't clear the -ihs parameter is mandatory when piping to me.
Why not make it set the parameter automatically? Or is it something specific to the way foobar is piping the file?
CUETools still doesn't work though, but only with non-ansi names.
One more thing, is the way TAK works suitable for a (future) GPU implementation? I remember bryant saying that wavpack, for example, isn't.
Adding -ihs fixes that, indeed. I did read the takc help, but it wasn't clear the -ihs parameter is mandatory when piping to me.
Why not make it set the parameter automatically? Or is it something specific to the way foobar is piping the file?
Because it's possible that a software writes a valid wave header with valid size data to the pipe. Then you would like to store it in the TAK file to be able to restore the original file with bit-identical meta data. With -ihs applied, TAK will save no header and create it's own one on decoding. Which can differ from the original.
One more thing, is the way TAK works suitable for a (future) GPU implementation? I remember bryant saying that wavpack, for example, isn't.
At least as well as FLAC. Basically TAK is using the same kind of prediction filter as FLAC, the asymmetric mode of Mpeg4Als and LPAC. Possibly it can be implemented more efficiently, because it does only require 16 * 16 bit integer multiplications with an 32-bit accumulator. But i don't know, if current GPUs provide appropriate instructions to take advantage of the simplier arithmetic.
But i wouldn't expect a similar compression advantage of a more extensive evaluation of compression parameters as FLAC achieves. Im most of my evaluations TAK's fast heuristics came very close to a brute force approach which tries most of the possible parameter combinations.
But i wouldn't expect a similar compression advantage of a more extensive evaluation of compression parameters as FLAC achieves. Im most of my evaluations TAK's fast heuristics came very close to a brute force approach which tries most of the possible parameter combinations.
Well IMHO, TAK already compresses more than enough, way better than FLAC on some albums (413 vs 479MB for example), so my main interest would be encoding speed gained by making use of the GPU. Your current multithreading is still inferior to foobar starting 4x encoders for my setup, tn4 is around 90x, 4x tn1 is 125x (FlacCL: 200x), so maybe using the GPU would boost that.
Also as Destroid said, increasing encoding speed by changing format specifications is fine as long as backwards compatibility is kept. Since you said the feature you could optimize would go from 0.1x to 250x, it should be a nice boost even if it doesn't translate to decoding speed directly.
I'll continue keeping an eye on TAK for now.
try foobar instead? that uses temporary filenames when encoding and then renames them when done. so it really doesn't matter whether the encoders support unicode or not.
Just to note that i've converted from FLAC over 60 albums with Foobar till now and haven't experienced any problems with filenames using lots of (´ ` ~ ^ ç ...and so on) on the filename characters, it converts everything flawlessly. So maybe it really depends how the applications pass the original names to the *.tak destination file or how they use the pipe for that matter.
Those are supported yes, but middle-european diacriticals (such as in e.g. Mata?i?) are not, and if you try to convert files that are in folders with e.g. Cyrillic characters in them, the encoder will immediately exit (in foobar), regardless of output directory name choice.
You guys should tell the author of CUETools to start using tak_deco_lib.dll instead of piping Takc.exe's decoded output; that way, at least its verification features will not be hindered by TAK's inability to handle Unicode characters (the decoding library should have no problems handling them so long as CUETools doesn't -- this can also be observed with foobar2000's playback component).
Can I ask why other people should tell the author when it is your idea? The thread in CD Hardware/Software is likely to get your request viewed by Gregory, so I see no reason why you cannot post it there yourself.
I don't use CUETools much -- I find it hard 'pushing' my ideas into a developer whose software I don't even 'endorse.'
You guys should tell the author of CUETools to start using tak_deco_lib.dll instead of piping Takc.exe's decoded output; that way, at least its verification features will not be hindered by TAK's inability to handle Unicode characters (the decoding library should have no problems handling them so long as CUETools doesn't -- this can also be observed with foobar2000's playback component).
It's a possible workaround, yes. But to be honest, I'd rather see TBeck implement unicode, especially since using tak_deco_lib.dll still won't solve the inability to (batch) encode unicode filenames.
One of the few reasons I'm not migrating to TAK is because I can't convert my ~300GB library in one go - even on fast presets it takes about 8 hours, and if I have to mess around with my filenames, it adds a lot of unnecessary work. So I'd rather wait. (Another is lack of rockbox support)
I apologize if this skirts the line, but (at least in my case) I'm less concerned with open-sourcing in the short term, and more with the fact that Wine is a *really* heavy dependency to run takc (since I only use Linux). My concern is convenience, rather than ideology.
Because of that, I'd like to ask if you've tried http://www.lazarus.freepascal.org/ (http://www.lazarus.freepascal.org/) - it claims to be "a Delphi compatible cross-platform IDE for Rapid Application Development," and can convert Delphi projects in what seems to be a mostly-automated manner: http://wiki.freepascal.org/Delphi_Converter_in_Lazarus (http://wiki.freepascal.org/Delphi_Converter_in_Lazarus)
One nice thing is that it has a conversion mode that attempts to support both Lazarus and Delphi in the same project, but what I find most interesting is that it can compile for Linux. Doing so requires a Linux system, but that can be done in VirtualBox or VMWare and isn't hugely difficult to set up.
Thanks for your time and the awesome codec, TBeck!
Wine is a *really* heavy dependency to run takc (since I only use Linux). My concern is convenience, rather than ideology.
Heavy, how? Sure, the Arch Linux package is about 33MiB, but my ADSL line is fast enough that it doesn't bother me. It runs fast too, and doesn't seem to use too much RAM. With caudec (http://caudec.outpost.fr/), convenience is one of the things that I was aiming at. It makes encoding and decoding TAK files very fast and easy.
Wine is a *really* heavy dependency to run takc (since I only use Linux). My concern is convenience, rather than ideology.
Heavy, how? Sure, the Arch Linux package is about 33MiB, but my ADSL line is fast enough that it doesn't bother me. It runs fast too, and doesn't seem to use too much RAM. With caudec (http://caudec.outpost.fr/), convenience is one of the things that I was aiming at. It makes encoding and decoding TAK files very fast and easy.
I run a source distro, and wine itself has some heavy deps. I don't necessarily want X on every system I want to be able to transcode files to TAK on. Also, 32/64 bit multilib (required for wine) means that I basically have *every* package in wine's entire dep tree installed twice because my system is 64-bit native. If not for wine, my system could be pure 64-bit.
Caudec does help a great deal, and it's how I've been using TAK, but there are other wrinkles as well. For instance, I need to run takc once, then when it crashes on startup (consistent across multiple wine versions here) kill all wine processes including winedbg. After that, takc will run without any problems until next reboot.
I want to thank the author, TBeck for his great work!
On the Linux question, I can suggest something, yes?
Has anyone tried if foobar with the the latest 2.3.0 tak decoding AND encoding work without error in Crossover 12 instead of Wine on Linux Mint or Ubuntu?
That may be a way to run Tak in 64-bit Linux with least amount of unnecessary dependencies.
Ive switched to Linux, too on almost all my computers but I haven't tried audacity or foobar on Linux in this way because I am still using an XP computer for audio. Perhaps somebody here has done this?
I want to thank the author, TBeck for his great work!
On the Linux question, I can suggest something, yes?
Has anyone tried if foobar with the the latest 2.3.0 tak decoding AND encoding work without error in Crossover 12 instead of Wine on Linux Mint or Ubuntu?
That may be a way to run Tak in 64-bit Linux with least amount of unnecessary dependencies.
Ive switched to Linux, too on almost all my computers but I haven't tried audacity or foobar on Linux in this way because I am still using an XP computer for audio. Perhaps somebody here has done this?
Sorry I took so long for this, but Crossover *is* Wine, with some additions. Codeweavers are a major contributor to Wine upstream. In order to run 32-bit Windows programs, you need 32-bit Wine/Crossover; that means you need a great deal of 32-bit libraries. On a native 64-bit system, that can be a hassle.
TBeck
Thomas, please make a 64-bit version of the library tak_deco.lib.
I love this programme, thanks you much.
I just installed on Windows7 latest 64 build and it only uses 4 cores and 8 threads where 6 cores and 12 threads are available. Is there an adjustment I can make, or a programme default?
TBeck
Thomas, please make a 64-bit version of the library tak_deco.lib.
Did you also send me an email i didn't reply to? Sorry...
I really want to go 64 bit. Because it's getting standard and i am also interested into possible performance improvements. Not necessarily because of the 64-bit-integer arithmetic. TAK has been designed to rely on 32 bit arithmetic. But the 8 additional registers for the SSEx-optimizations could help a bit.
But first i have to perform some preparations:
- Install an OS with 64 bit. I am a bit behind...
- Choose an development platform which supports 64-bit compiles.
Although it was my intention to switch from my good old Delphi 6 to C++, i am currently thinking about an upgrade to Delphi XE5. For a limited time Embarcadero is offering an affordable upgrade path from my old Delphi. I haven't deceided on it.
I love this programme, thanks you much.
Thank you!
I just installed on Windows7 latest 64 build and it only uses 4 cores and 8 threads where 6 cores and 12 threads are available. Is there an adjustment I can make, or a programme default?
Sorry, no. I tend to be a bit conservative reagarding features i can't test myself. But there is no technical reason why TAK's multi-threading-code shouldn't work with more than 4 cores. I will remove this restriction in the next version.
Thomas
Any estimate when we might see a new beta?
I just finished moving my audio library (~20k files, 254GB in flac + metadata) to tak. Using the maximum preset, TAK was able to shave off 12GB!
Some might say it's pretty insignificant if you look at HDD prices, but for me it's more than enough to justify moving to it as my archiving/pc listening format.
One major annoyance is still present though - lack of unicode support. foobar2000/TAC/CUETools did work around filenames with foreign characters by using temporary filenames, but they can't do anything about paths with foreign characters (I'm actually surprised playback works). Those albums had to be handled manually, of course. And will have to every time I attempt to i.e. verify their integrity against CTDB.
I'd be really, really, really grateful, if unicode support became a priority for the next release.
I just finished moving my audio library (~20k files, 254GB in flac + metadata) to tak. Using the maximum preset, TAK was able to shave off 12GB!
Some might say it's pretty insignificant if you look at HDD prices, but for me it's more than enough to justify moving to it as my archiving/pc listening format.
One major annoyance is still present though - lack of unicode support. foobar2000/TAC/CUETools did work around filenames with foreign characters by using temporary filenames, but they can't do anything about paths with foreign characters (I'm actually surprised playback works). Those albums had to be handled manually, of course. And will have to every time I attempt to i.e. verify their integrity against CTDB.
I'd be really, really, really grateful, if unicode support became a priority for the next release.
It is the command-line tool which cannot handle Unicode paths (
Takc.exe);
tak_deco_lib.dll (which foobar2000's decoding component uses) has no problems whatsoever.
I am aware of that. tak_deco_lib probably passes the tak file as a pipe or something, avoiding paths altogether, as tak's code itself is not unicode-capable.
tak_deco_lib provides a way of callback based I/O that is similar to fmemopen(3) on Linux or funopen(3) on BSD, where I/O is done on the caller/application side. This allows the caller to open the file and provides functions for reading/seeking the file, that enables application to take care of OS specific file handling when library itself does not.
Similar style API has also been available on libFLAC, libwavpack or something, and that is why you could play FLAC files having Unicode file name through libFLAC, when libFLAC and it's CLI frontend didn't support Unicode file name on MS Windows.
For users of dBpoweramp:
If you want to use TAK with that program, you'll have to use a modified command line with it's CLI encoder extension to get it to work. Here's an example:
-e -p4m -overwrite -ihs -md5 -tt "Artist=[Artist]" -tt "Title=[Title]" -tt "Album=[Album]" -tt "Year=[year]" -tt "Track=[Track]" -tt "Genre=[Genre]" - "[outfile]"
The explanation for this is here (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=77604&view=findpost&p=857983)
One major annoyance is still present though - lack of unicode support.
Same for me. Started moving to TAK some years ago, but I make it manually mostly. My collection is over 150.000 tracks. Support for Unicode is essential, because it is impossible to check TAK files in Cue Tools in AR/CTDB without first making safe titles - a lot of useless manipulation. It is of course possible to encode them, check by DBs only once, then add all those special characters and leave it alone forever. But I like to make fresh checks by databases, also because most rips are not there on first checks.
Just curious how the ripping behaviour changed these years comparing to 10 years ago. I just stupidly recheck albums by AccurateRip / CueTools DB and feel pleasure...
Also lack of 32-bit support makes me use WavPack for studio masters in this quality, but most others including FLAC doesn't support either. TAK also has very nice breakaway from others on high samplerate. May save up to 100-200MB on some material, not 20-30MB as on 44/16. I use TAK for studio masters archive and it would be nice if it could be used smoothly with Cockos REAPER without need to encode WAV or FLAC to work with.
Can we maybe have unicode file support as a Christmas present? Pretty please?
I'm running into unicode problems much more often nowadays thanks to some weird naming sense of the artists, and having a rip suddenly cancel itself in the middle of the disc because tak chokes on an unicode is rather aggravating.
Hi Thomas,
hope you're doing well. I was wondering if you would be willing to tell us how your work on the next version of TAK is progressing. I appreciate that you're busy, so I don't mind much either way if it's still a long ways out or not; but I'm curious as to how far you are from reaching the goals you set for the next release.
hope you're doing well.
Thank you. I really want to work on the next release, but my paid job is taking up a lot of time. While i intended to port TAK to C, this looks unrealistic at the moment. Therefore the next version still has to be written in Delphi. Since my Delphi 6 ist lacking a lot required for the planned features, i finally bought an upgrade to Delphi XE 7 (There was a time limited offer for users of very old versions).
Planned features in order of current priority:
1. Unicode support
2. An 64 bit version especially of the decoder library.
3. Optimizations for the AVX2 instruction set.
4. Optimizations for multithreading.
5. An encoder library.
But to be honest, i really can't tell you when i will have more time to work on TAK. In the worst case it could be June (then a demanding project is completed). I hope to start earlier.
And i still have to answer a couple of mails...
Thomas
Thank you Thomas!
I appreciate the update and the knowledge that a x64 TAK is coming.
Thanks, Thomas.
Good to hear you're doing well and plan to continue you work on TAK in the future.
It's is one of the most efficient lossless codecs and it would be really sad if it's development stuck completely for an indefinite time.
Great, the latest version of the SDK is missing the MD5 API from the C headers.
Sorry. I hope, this helps for now:
type
Ttak_str_MD5 = Array[0..15] of Byte;
function tak_SSD_GetMD5 ( ADecoder : TtakSeekableStreamDecoder;
var AMD5 : Ttak_str_MD5) : TtakResult;
cdecl; external 'tak_deco_lib.dll';
Great, the latest version of the SDK is missing the MD5 API from the C headers.
I remember that I saw similar question several years ago... found it after a quick search: http://www.hydrogenaud.io/forums/index.php...st&p=837678 (http://www.hydrogenaud.io/forums/index.php?s=&showtopic=101386&view=findpost&p=837678)
Unofficial tak_deco_lib.h and tak_deco_lib.lib are still available in foo_input_tak_0.4.6_beta_3b.rar (http://www.hydrogenaud.io/forums/index.php?s=&showtopic=54087&view=findpost&p=828153)
Thanks for this notice, I have retrieved the necessary files from an old copy of that archive, and added them to the repository.
Is there a linux version so I can try it? :-D
Maybe you noticed that I happened to reply to a 3 year old topic? The closest you'll get on Linux is FFmpeg, and that's decode-only.
Maybe you noticed that I happened to reply to a 3 year old topic?
No, I didn't. :D
wine tak.exe
seems to be working just fine, also mpv seems to have no problems with playback.
People still using this? Thought support died long time ago. Latest version is 4,5 years old.
Differences in file size are really close with lossless compression. I've had my entire collection at WavPack once, but stumbled upon some compatibility issues. Went back to Flac for the moment.
Is you really need smaller file (at the expense of (to me) inaudible noise), you can gain a lot of space by using LossyWav. Be sure to not mix them up with you lossless collection though. Unless you don't care about that.
I myself are a bit of a purist in that. I experimented with LosseWav and hear no difference, but don't have the guts to delete the originals.
People still using this? Thought support died long time ago. Latest version is 4,5 years old.
Specally regged to answer to. Yes, I do. No development but the codec still is working, isn't it? :)
Differences in file size are really close with lossless compression. I've had my entire collection at WavPack once, but stumbled upon some compatibility issues. Went back to Flac for the moment.
Playing Wavpack CPU load is too high and the navigation is too slow comparing to TAK. Fraction of seconds of course but these fractions are annoying while the TAK rewinding is smooth or at least very near to. Just FYI my CPU is Core 5 and I use the 4 max TAK profile. Also I personally like the Waveform Seekbar plugin and it counts the waveform on the first play and wavpacks make the foobar stuck while taks don't. So the differences are between these codecs.
Is you really need smaller file (at the expense of (to me) inaudible noise), you can gain a lot of space by using LossyWav. Be sure to not mix them up with you lossless collection though. Unless you don't care about that.
I myself are a bit of a purist in that. I experimented with LosseWav and hear no difference, but don't have the guts to delete the originals.
My personal lovely choise is LossyTAK i.e. LossyWAV + TAK. My whole library is LossyWAVed (mostly -q 8 ) and TAKed because I don't hear any difference between the original files and LossyWAV 8 and the TAK decoding is as fast as FLAC's or even faster but the files are less in size.
One notice: LossyWAV is not too good for 16bit files so I use the conversion to 24bit. And also the TAK or LossyWAV (can't recall) can not process the 32bit files. And, again, the 24bit conversion is the remedy.
Also I convert the DSD to LossyTAK. For this I use the FIR filter (the second version in the bottom of the page) from Nazar Shtybel: http://s-audio.systems/catalog/dsd-filter
The page is on Russian so use the google translate if you can't read. The filter itself is written on strong math so the google translator won't help :D
Though my DAC plays DSD natively but waveforms are preferrable in some aspects. At least the foobar DSPs work with them. Well, and the file size matters.
The reports of TAK's death have been greatly exaggerated...
Initially i didn't want to post before the next release, but while i am working on it i am also regularly reading the posts at hydrogen and somehow they made me feel it's a good time to make a (small) statement.
TAK 2.3.1 (I like to call it the "Back to work"-release...) will again be faster. Currently the strongest mode -p4m encodes more than 20 percent faster on my (Haswell-) system. And this with good old SSE2 and SSSE3 instruction sets. AVX2 would be even faster but currently i am not too satisfied with it's advantage. Possibly it's faster on newer CPUs.
If the release would only consist of speed improvements, i could publish it within the next 3 days.
But i am also preparing an open source release of the decoder. I would definitely prefer a release in the C-language, but this would take a lot of time, probably more than i can invest now. Therefore i will try the second best approach and migrate the Delphi-Code to Lazarus.
It's more work than one might expect. I am refactoring and tidying the code and i have to extract or sometimes replace small inline assembler sections. I may sacrifice some percent of speed for better readability and portability.
Hopefully most of this work will be done when i release 2.3.1.
The next version 2.4.0 should contain a decoder-library compiled with Lazarus. If this works well, i will release it's source code.
While the time i have to spend on Tak is a lot less than earlier (and also less calculable) i hope to release 2.3.1 in 2 to 4 weeks. With some luck 2.4.0 may take less than 3 additional months. No guarantee, but i am quite confident.
Next big step then could be the work on an open source Lazarus encoder library.
Interim releases to add other features are likely.
When the open source decoder has been published i may also reintroduce the dedicated LossyWav-Codec, which was part of an earlier beta release. It worked very well but seemed to be quite useless without an open source decoder implementation.
I know there is a lot more to do, but for now i will focus on the next 2 versions as outlined above.
Thank you for your interest - and patience
Thomas
Great news. Thanks, Thomas, for post. It is important to have any feedback from the developer.
Out of curiosity:
Planned features in order of current priority:
1. Unicode support
Nice to see Thomas is still there. Makes me curious to go and test and do some comparisons.
At this moment I have no idea how much software is compatible. I mainly use Foobar (which has support) but also Kodi and Volumio support would be great (haven't searched yet). Also curious to see which compression levels there are.
Thanks for the info/news Thomas. :)
Out of curiosity:
Planned features in order of current priority:
1. Unicode support
At the top of the source of your quote i had written:
"While i intended to port TAK to C, this looks unrealistic at the moment. Therefore the next version still has to be written in Delphi. Since my Delphi 6 ist lacking a lot required for the planned features, i finally bought an upgrade to Delphi XE 7 (There was a time limited offer for users of very old versions)."
Delphi XE 7 has built in unicode support. But unfortunately now (3 years later) it does not seem to be a good option for future TAK development. Embarcadero just wrote me, that they will no longer provide upgrade options for older versions. You have to subscribe to a permanent and costly update. For my purposes that's a killer... Last time i looked for a free version, it did not support 64 bit.
Therefore the planned step-wise switch to Lazarus. I wanted to start this as soon as possible to avoid the feeling of wasting time on an inadequate (because too costly) development tool. This obviously changed priorities.
But Unicode support still has a top priority.
BTW: My english skills did not benefit from the long absence. I hardly train them outside of hydrogen. When writing the text above, i had the feeling of beeing a bit unclear and omitting at least some nuances.
Open source decoder would be great (and encoder as well. :) )
Not sure whether it was discussed.
According to http://www.audiograaf.nl/losslesstest/Lossless%20audio%20codec%20comparison%20-%20revision%204.pdf
p1 is better or equal to p0e/m by all parameters: decompression/compression speed and compression ratio.
Does it make p0e/m pretty irrelevant?
No. Hidden in TAK's ReadMe:
"Each preset selects a set of internal encoder options. Some of them affect only the encoding speed, the others also the decoding speed.
Slower decoding usually means higher hardware requirements (memory size and/or cpu power) for a playback device. Some devices may be too weak to support the more demanding internal encoder options. But any option, which only affects the encoding speed and complexity, is always applicable.
Therefore each preset consists of two components:
* The profile selects internal encoder options which affect both encoding and decoding speed. Profiles are declared as a number from 0 to 4 (fastest to strongest).
* The evaluation level selects only internal encoder options, which have no effect on the decoding speed. The evaluation levels are named Standard, Extra and Max and abbreviated as s, e and m (fastest to strongest).
Presets are beeing declared as a combination of the profile and the abbreviated evaluation level (if not specified Standard is beeing used): 0 is the fastest, 4m the strongest setting.
A hardware manufacturer supporting TAK has only to specify the strongest profile it's hardware can decode, because the evaluation level does not affect the hardware requirements.
Hint: If you want higher compression and fast encoding, and are able to accept some decrease in decoding speed, it is usually preferable to select a higher profile instead of increasing the evaluation level."
At the top of the source of your quote i had written:
"While i intended to port TAK to C, this looks unrealistic at the moment. Therefore the next version still has to be written in Delphi.
Hi TBeck,
I am the author of ImgDrive which could mount Cue sheets files of APE/FLAC/M4A/TTA/WAV/WV/BIN as AUDIO CD.
http://yubsoft.com/imgdrive/index.htm
i try to add support for TAK, which need DecodeTakFrame function implement in C and run in Ring 0 as kernel device driver. I have good skill of C/C++, if has some documentation or source code, i will help to port to C.
sorry for my bad english.
Does it make p0e/m pretty irrelevant?
No. Hidden in TAK's ReadMe:
I hope you didn't mistake my brevity (caused by my bad english). It definitely wasn't meant as (R)ead (T)he (F)...ing (M)anual!
i try to add support for TAK, which need DecodeTakFrame function implement in C and run in Ring 0 as kernel device driver. I have good skill of C/C++, if has some documentation or source code, i will help to port to C.
sorry for my bad english.
You are very welcome! Even if you will not help me to improve my english...
TAK 2.3.1 (I like to call it the "Back to work"-release...) will again be faster. Currently the strongest mode -p4m encodes more than 20 percent faster on my (Haswell-) system. And this with good old SSE2 and SSSE3 instruction sets.
Well, now -p4m is about 30 percent faster. Time to prepare the beta release.
I hope you didn't mistake my brevity (caused by my bad english). It definitely wasn't meant as (R)ead (T)he (F)...ing (M)anual!
Nah, it's not that. :)
I've just assumed that there is [very old and/or slow] hardware where those presets (p0m/p0e) makes sense.
But still p1 is better (or approx. equal in worst case) than p0m/p0e in all aspects (decoding/encoding speed, compression) even for 5-7+ years old CPU.
Good luck with a new release!
I've just assumed that there is [very old and/or slow] hardware where those presets (p0m/p0e) makes sense.
But still p1 is better (or approx. equal in worst case) than p0m/p0e in all aspects (decoding/encoding speed, compression) even for 5-7+ years old CPU.
IMHO it's the fastest possible setting that does make sense. Anything faster would compress significantly worse. It's using 4 predictors per sample, the minimum TAK permits. P1 is using 12 predictors like FLAC -8. One reason for these choices was to show what TAK can do when beeing restricted to similar cpu complexity as FLAC. Historical reasons...
Good luck with a new release!
Thank you ::)
For other historical reasons i would have loved to publish my back-to-work release at April 1st. It was april fools day when i first posted about TAK (named YALAC then) and not many readers believed me. Nostalgia...
Well, if nothing unexpected happens i will perform the release one day too late.
Given that ffmpeg people already made an open source decoder, and if you don't plan to change bitstream in a backward incompatible way, maybe it'd be better to focus solely on the encoder?
Does this decoder support multi-channel-audio? I don't think so, but i may be wrong. And this would be sad, because TAK's multi-channel-encoder is really strong. I also put a lot of work into the decoders error robustness; hydrogenaudio's users helped a lot in testing this.
I would estimate the effort to prepare the encoder release as at least 3 times more than the decoder release (bad english..). Therefore omitting the decoder would not save so much time. And i prefer to start with something smaller.
And i also thought about the reintroduction of the dedicated LossyWav-codec which was part of an earlier beta release.
But maybe things will change when i start the work.
It does support multichannel audio, also decoder is multi-threaded and code is better and faster than reference one.
Only missing is TAK 1.X support.
IIRC ffmpeg is faster only because it's multithreaded. Single-threaded ffmpeg is about 2x slower than TAK
(about code quality... I thought that TAK is closed-cource?)
For other historical reasons i would have loved to publish my back-to-work release at April 1st. It was april fools day when i first posted about TAK (named YALAC then) and not many readers believed me. Nostalgia...
Got to pity the doubters. Not really... :)
TAK+CUE still used for all CD backups here, the best 16-bit 2-ch archival format (IMO). Yes, I'm admitting to collecting discs still. Maybe it's lossless audio paranoia that I don't bother with purchasing music online whether it's lossy or otherwise. Maybe someday but even then there's still too many indie/private CD's where finding a second copy is nearly impossible.
Good to see your posts, Thomas! Looking forward to running another speed comparison in the nest version. :)
IIRC ffmpeg is faster only because it's multithreaded. Single-threaded ffmpeg is about 2x slower than TAK
(about code quality... I thought that TAK is closed-cource?)
How you calculated that numbers, i think that -t parameter does not decode at all, it just check CRC of every frame.
No. -t does anything -d does except writing the result to the disk.
So, are there any news about development and new release?
any news
unbelievable ! (https://hydrogenaud.io/index.php/topic,115769.0.html)
unbelievable !
Yeah, very funny, haha. :)
I ask about these plans (https://hydrogenaud.io/index.php/topic,101386.msg952670.html#msg952670), not about beta.
Yeah, very funny, haha. :)
I ask about these plans (https://hydrogenaud.io/index.php/topic,101386.msg952670.html#msg952670), not about beta.
When Thomas is ready, he will post it here.
How about a bit of patience?