HydrogenAudio

Lossless Audio Compression => Lossless / Other Codecs => Topic started by: TBeck on 2008-12-19 01:44:56

Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-19 01:44:56
Beta release 4 of TAK 1.1.0 ((T)om's lossless (A)udio (K)ompressor)

It consists of:

- TAK Applications 1.1.0 Beta 4
- Winamp plugin 1.1.0 Beta 4
- Decoding library 1.1.0 Beta 4

The SDK will come with the final release.

Download:

Download link removed.  TAK 1.1.0 Final (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=68454&view=findpost&p=607825) has been released.

Fixes in Beta 2 / 08-12-20

- Takc.exe contained a bug: When using pipe encoding with the parameter combination -ihs -sts0, the encoder stopped with an error message.

Only Takc.exe and it's readme have been modified.

Fixes in Beta 3 / 08-12-20

- On very rare occasions (about 1 of 250,000 seeks in my tests) the decoder could fail when seeking to the end of a file with a quite large (i am testing up to 4 MB) meta data structure (APEv2-tag) at the end. This resulted in an "undecodable" error message and affected the decoding library and the winamp plugin.

All binaries and the readme have been modified.

Fixes in Beta 4 / 08-12-30

- Tiny speed improvements.
- If pipe encoding is used without the -ihs paramter (that is for instance required for foobar), TAKC will now reserve only 46 bytes in the header for the storage of wave file meta data. That's sufficient for a standard wave file header. Earlier version reserved 1 MB (!) in the header, making the compressed file accordingly bigger.

All binaries and the readme have been modified.

What's new

New Features:

- Support for 192 Khz Audio.
- Seeking without seek table.

Caution: Decoders prior to those released with V1.1.0 can't decode 192 Khz files and can't seek in files without seek table.

Improvements:

- Encoding and decoding speed improvements of about 3 percent for presets p0 and p1 on my system. Also some decoding speedup for p2.
- Fixed a bug in the encoder that resulted in suboptimal compression of some loud files and especially high resolution audio. Some files may gain about 0.05 percent of compression. Not much, but it comes without any speed penality.
- Further clean up of the Code.

Modifications:

- I hope you don't mind but i always had the feeling 5 presets are enough. Therefore i dropped the appropriately 'Insane' named preset -p5 and instead made presets 3 and 4 stronger. Okay, new -p4 will nevertheless be slightly weaker than old -p5, because i have reduced the maximum predictor count from 256 to 160. Before doing this i performed a detailed analysis of predictor count * compression * speed. There are not many files which benefit from such high predictor orders. Two of my file sets contain many of such files, but even they will only loose about 0.10 percent compression. Not a big loss if in exchange you get nearly half the decoding (cpu power) requirements.
- Removed option to modify the Prefilter sensitivity.

Known issues:

- If you use pipe decoding and the application reading the pipe is beeing terminated before the whole file has been read, TAKC may get into an endless loop and has to be manually killed with the task manager. I don't think this is a big issue but i will try to fix it in one of the next versions. BTW: Big thanks to shnutils for testing the pipe decoding!
- There seem to be some compatibility issues with pipe decoding to some other applications ("crc1632.exe" has been reported). I will try to fix it in the next release.

Some remarks:

The beta release has been delayed because my enhanced validation procedure found some bugs in the new seeking-without-seektable feature. After the correction of the 3 bugs i found, i lost about 2 percent of speed because of now suboptimal code alignment.

Beta testing

The beta version has already gone through extensive testing performed by my automatic scripts. But especially because of the many changes for 1.1.0 rare bugs are still possible (as always...). Please try the beta release and report any bugs in this thread.

It's important to test the new seeking-without-seek table feature:

- Encode a file (big files would be nice and APEv2 tags too) with no seek table: -sts0
- Play it back with Winamp or Foobar and perform some seeks.

I would also be happy about tests of compression efficiency and speed. Because the final release will have identical performance (there may be a speed variation of 1 to 2 percent because of different code alignment of another build), it does make sense to test the beta.

Thanks for testing and have fun

Thomas
Title: TAK 1.1.0 - Beta release
Post by: Remedial Sound on 2008-12-19 09:01:15
Congrats on the new beta! 
Title: TAK 1.1.0 - Beta release
Post by: CPKTV on 2008-12-19 09:34:36
thanks for new beta.
does winamp plug-in supports transcoding?
Title: TAK 1.1.0 - Beta release
Post by: ssjkakaroto on 2008-12-19 15:54:59
Thanks for the beta Thomas.
Single file albums with embedded cuesheets and no seektable are seeking very fast here
Title: TAK 1.1.0 - Beta release
Post by: melomaniac on 2008-12-19 17:17:54
First, thank you Thomas for your dedicated work. 
I've used your codec for 9 months now and have never had a single problem.

I've just encoded/decoded some files with foobar2000, presets from p2 to p4. There are indeed some speed/compression changes/optimizations. p4 compression efficiency is very close to old p5 with a slightly faster speed. Synthetic Soul will have to update his lossless codec comparison (http://www.synthetic-soul.co.uk/comparison/lossless/).

I've also tried some long files using the seeking-without-seek table feature. Seeking is really fast and I've encountered no issues so far.

Looking forward to future releases and optimizations. 
Title: TAK 1.1.0 - Beta release
Post by: Squeller on 2008-12-19 20:32:39
I wanted to test -sts0 with fb2k but couldn't make it work.

Command line arguments for takc.exe in fb2k: -e -sts0 - %d

Code: [Select]
Source: "M:\Musik\Klassik\_Various\Bainton - Symphony No. 3, Boughton - Symphony No. 1 (Handley - BBC CO)\CDImage.tak" / index: 3
  An error occurred while finalizing the encoding process (Object not found) : "C:\Users\Michel\Desktop\CDImage.tak"
  Conversion failed: Object not found


Hmm, yes, after the conversion is finished the file disappears and the fb2k error message appears. The conversion goes from a tak with embedded CUEs to another Tak.
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-19 20:51:25
I wanted to test -sts0 with fb2k but couldn't make it work.

Command line arguments for takc.exe in fb2k: -e -sts0 - %d

You have to add the -ihs switch: -e -ihs -sts0 - %d

From the Readme:
Quote
...A similar problem arises if the other application does not know the final sample count of the audio data. This can happen, if the audio data is simultaneously beeing ripped from a cd or if the data is beeing converted from a format which doesen't provide an accurate size information before the whole decoding is done. In both cases the other application can not write an accurate sample count into the wave header which is beeing sent to TAK.

You have to tell TAK to ignore this size entry by adding the ignore-header-size switch to the command line:

Code: [Select]
takc -e -ihs - Outfile
Title: TAK 1.1.0 - Beta release
Post by: Neasden on 2008-12-19 20:55:12
It would be nice a TAK website featuring the winning TAK logo!
Title: TAK 1.1.0 - Beta release
Post by: Squeller on 2008-12-19 21:44:52
You have to add the -ihs switch: -e -ihs -sts0 - %d
Hmm. But I've just removed it because it brought me more trouble. This didn't even create a tak file at all.

- I load a cuesheet into fb2k, the references work, double clicking starts playback. Select some tracks now.
- start takc in the converter with "-e -ihs -sts0 - %d"
- The converter runs through at 1000x+, then "Could not start conversion process (Object not found)."
- Vista

Fb2k console says:

Quote
Command line: "M:\Musik\foobar2000\takc.exe" -e -ihs -sts0 - "CDImage.tak"
Working folder: C:\Users\Michel\Desktop\name\anothername\
Conversion failed: The encoder has terminated prematurely with code 2; please re-check parameters
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-19 22:11:06
You have to add the -ihs switch: -e -ihs -sts0 - %d
Hmm. But I've just removed it because it brought me more trouble. This didn't even create a tak file at all.

Does it work without "-sts0"?
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-19 22:39:33

You have to add the -ihs switch: -e -ihs -sts0 - %d
Hmm. But I've just removed it because it brought me more trouble. This didn't even create a tak file at all.

Does it work without "-sts0"?

Plobably not...

I can reproduce the bug! I will try to fix it soon.

Thanks for the report!

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-19 22:50:24
Ok, i've got it.

It only can happen with pipe encoding and the parameter combination -ihs -sts0.

There is code in my encoder that checks valid paramter combinations and i forgot to update it after implementing -sts0. Now the encoder refuses to work because of invalid arguments.

It's not dangerous, it will simply create no file and show an error message.

I hope to release a fixed beta 2 within 1 or 2 hours.

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: Squeller on 2008-12-19 22:54:47
Thank you Thomas, I'll use it tomorrow then.
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-19 23:46:20
Beta 2 has been released

The download link is in the first post.
Title: TAK 1.1.0 - Beta release
Post by: Synthetic Soul on 2008-12-20 08:40:35
Synthetic Soul will have to update his lossless codec comparison (http://www.synthetic-soul.co.uk/comparison/lossless/).
The problem is that my machine has been upgraded to XP, altering the test platform.  I would like to get some tests done - I've tested TAK since the alphas and I feel compelled to continue - but it may not be part of that comparison.
Title: TAK 1.1.0 - Beta release
Post by: Squeller on 2008-12-20 08:51:55
I encoded to some album image files with embedded Cuesheet and apev2 tags, sts0. Jumping from track to track and inside tracks works instantly so that currently I don't see a reason why I should use a seektable. BTW I chose tak since the beginning of my lossless archive, never ever had a single problem.
Title: TAK 1.1.0 - Beta release
Post by: BenniP on 2008-12-20 10:00:36
Hello,

i'am trying TAK for the first time, great compression and fast encoding speed!

It works fine but, don't know if this is beta related, I can't get pipe encoding to work.

My parameters in foobar
Code: [Select]
-e -p4 -sts0 - %d

foobar always tells me "could not enumerate tracks (Object not found) on:" follow by the filename.

I'm using the %s switch instead now.
Title: TAK 1.1.0 - Beta release
Post by: melomaniac on 2008-12-20 10:04:29
Synthetic Soul will have to update his lossless codec comparison (http://www.synthetic-soul.co.uk/comparison/lossless/).
The problem is that my machine has been upgraded to XP, altering the test platform.  I would like to get some tests done - I've tested TAK since the alphas and I feel compelled to continue - but it may not be part of that comparison.

I get your point. Nevertheless, I run on XP and the speeds mentionned on your page are very close to what I got. New tests would be very helpful for the community so feel free to do it even regardless of your previous ones.
By the way, your comparison helped me a lot deciding which codec and preset to use so I must thank you for that.

Hello,

i'am trying TAK for the first time, great compression and fast encoding speed!

It works fine but, don't know if this is beta related, I can't get pipe encoding to work.

My parameters in foobar
Code: [Select]
-e -p4 -sts0 - %d

foobar always tells me "could not enumerate tracks (Object not found) on:" follow by the filename.

I'm using the %s switch instead now.

Squeller mentionned the problem. Thomas fixed it in Beta 2. I just tried it with a long 30-minute Bruckner movement and an album image with embedded cue sheet: it works fine now. Your command line missed -ihs so use this:
-e -p4 -ihs -sts0 - %d
Title: TAK 1.1.0 - Beta release
Post by: Squeller on 2008-12-20 10:54:37
TBeck, if -ihs is mandatory when using -sts0, you should internally handle the lack of that switch, otherwise many users would get confused.
Title: TAK 1.1.0 - Beta release
Post by: BenniP on 2008-12-20 10:56:32
Now that was easy! 

Thank you very much!

/EDit: Self-Notice: Oh and reading the complete thread would be quiet helpful. 
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-20 13:10:49
Beta 3 has been released

The download link is in the first post.

I am very confident that this will be the last beta.

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: Synthetic Soul on 2008-12-21 20:57:12
Synthetic Soul will have to update his lossless codec comparison (http://www.synthetic-soul.co.uk/comparison/lossless/).
The problem is that my machine has been upgraded to XP, altering the test platform.  I would like to get some tests done - I've tested TAK since the alphas and I feel compelled to continue - but it may not be part of that comparison.
I tested TAK 1.1.0b3 as well as FLAC 1.2.1 again.  The results for FLAC 1.2.1 were so similar that I thought it OK to add the TAK 1.1.0 results to my comparison (http://www.synthetic-soul.co.uk/comparison/lossless/) (i.e.: I don't think the upgrade to XP has made a noticeable difference to the test set-up).  As usual, use ?all=1 (http://www.synthetic-soul.co.uk/comparison/lossless/?all=1) to view previous versions' settings.

Here's the data for TAK 1.0.4 and 1.1.0:

Code: [Select]
      |  1.0.4                    |  1.1.0
============================================================
-p0   |  65.803%    130x    147x  |  65.802%    131x    141x
-p0e  |  65.573%    108x    146x  |  65.573%    109x    142x
-p0m  |  65.455%     59x    146x  |  65.455%     61x    142x
-p1   |  64.837%    107x    145x  |  64.836%    108x    141x
-p1e  |  64.748%     89x    145x  |  64.748%     91x    141x
-p1m  |  64.640%     50x    146x  |  64.640%     51x    142x
-p2   |  64.078%     65x    128x  |  64.077%     66x    126x
-p2e  |  63.947%     52x    128x  |  63.946%     53x    126x
-p2m  |  63.861%     32x    128x  |  63.860%     32x    125x
-p3   |  63.833%     41x    117x  |  63.763%     38x    113x
-p3e  |  63.766%     33x    117x  |  63.695%     31x    112x
-p3m  |  63.727%     23x    116x  |  63.650%     20x    112x
-p4   |  63.643%     28x    111x  |  63.585%     24x    103x
-p4e  |  63.623%     18x    111x  |  63.562%     15x    104x
-p4m  |  63.604%     16x    111x  |  63.544%     14x    104x
-p5   |  63.576%     20x     98x  |
-p5e  |  63.543%     11x    102x  |
-p5m  |  63.525%     11x    102x  |
Title: TAK 1.1.0 - Beta release
Post by: sauvage78 on 2008-12-21 21:14:10
Thks Synthetic Soul, the update of your comparison was usefull to me.
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-21 21:16:47
thanks for new beta.
does winamp plug-in supports transcoding?

No.

Single file albums with embedded cuesheets and no seektable are seeking very fast here

I've just encoded/decoded some files with foobar2000, presets from p2 to p4. There are indeed some speed/compression changes/optimizations. p4 compression efficiency is very close to old p5 with a slightly faster speed.
...
I've also tried some long files using the seeking-without-seek table feature. Seeking is really fast and I've encountered no issues so far.

Fine!

I encoded to some album image files with embedded Cuesheet and apev2 tags, sts0. Jumping from track to track and inside tracks works instantly so that currently I don't see a reason why I should use a seektable.

Indeed i possibly will drop the seek table in one of the next version, when hopefully most users have upgraded to the new decoders with seeking-without-seek table support.

TBeck, if -ihs is mandatory when using -sts0, you should internally handle the lack of that switch, otherwise many users would get confused.

It's mandatory for foobar, but not in any other situation. For instance it's not required when reading the wave data directly from a file: "-e - <SomeWaveFile.wav"

It would be nice a TAK website featuring the winning TAK logo!

Yeah. I wanted to release this version with thelogo, but to be honest, i am not perfectly happy with it. It looks not so nice on my desktop, what is possibly caused by it's transparency feature. I will have to contact the creator to try some modifications.

Thank you all for testing the beta!

I intend to realease the final version within the next days. Please keep on reporting bugs if there are any....

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-21 21:52:24
Here's the data for TAK 1.0.4 and 1.1.0:
...

Thank you so much! 

Although the results are obviously different from what i expected: Decoding got slower on your system!

Well, i am optimizing for my two systems: An Pentium III with 1 GHz and an Pentium Dual Core with 2 GHz. Both systems are encoding and decoding faster with V1.1.0.

Unfortunately i don't have an Athlon system to check. Probably your CPU doesen't like one of the new optimizations. I will try to compensate this in the next version.

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: melomaniac on 2008-12-21 22:11:15
I tested TAK 1.1.0b3 as well as FLAC 1.2.1 again.
..

Thank you. Very informative.
I did some tests on my own (XP too) and they were very similar to yours.
We can see that now TAK 1.1.0b3 -p4 gives nearly the same compression efficiency than TAK 1.0.4 -p5 with a 4x gain in speed!
Thomas, you're a genius.
Title: TAK 1.1.0 - Beta release
Post by: greynol on 2008-12-21 22:15:23
If the speed has gone up with the compression remaining the same then the efficiency has gone up as well.
Title: TAK 1.1.0 - Beta release
Post by: buktore on 2008-12-22 06:47:29
Thanks for this, Thomas 

Some remarks: (using -p4, was using -p5)

- With loud music, better compression & faster encode-decode time 
- With quiet music, lower compression.
- For the first time ever, I got 1 corrupted TAK file when I converted my APE album image to separated TAK files. but I can fix it when convert just that track and it's fine, need more testing on this when I have time.
- Overall, decoding speed improvement for old&new TAK files. (amd 1800xp)
Title: TAK 1.1.0 - Beta release
Post by: Synthetic Soul on 2008-12-22 11:31:28
Although the results are obviously different from what i expected: Decoding got slower on your system!
...
I will try to compensate this in the next version.
I am still slightly concerned about my test set-up.  Perhaps I will re-run both 1.1.0 and 1.0.4 again, to ensure that they both have an equal testing environment.  My biggest concern with my comparison is misinforming users, and developers of course!
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-22 12:17:24
- For the first time ever, I got 1 corrupted TAK file when I converted my APE album image to separated TAK files. but I can fix it when convert just that track and it's fine, need more testing on this when I have time.

Can you provide more information please?! 

Some questions:

- Which beta have you used?
- From another thread i seem to remember you are using foobar?
- What command line parameters are you using?
- What means corrupt?
- Any piece of information can help...

Please try to respond fast!

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: buktore on 2008-12-22 13:52:00
Hmm.. I can't reproduce it.    I don't have the corrupted file either since I already overwrite it.

What happened is, I converted APE album image file to 14 tak files using foobar. when I try to replay-gain them foobar said something like "track 7" is damaged or corrupted (it stop at around 30% of the file) I scan 2 time but it still happen. I was busy at the time so I didn't bother testing more.

the command line is -e -p4 -ihs -sts0 - %d using the latest beta (beta 3)

AMD 1800+XP, XP SP3, foobar 0.9.6.1 beta

I'm 100% sure that this is no user error.

It might be my system.. but this is, for any formats, the first time this kind of thing happen to me.
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-22 14:24:17
Hmm.. I can't reproduce it.    I don't have the corrupted file either since I already overwrite it.

Thank you for the fast reply!

Do you still have the APE image file? Surely not...

What happened is, I converted APE album image file to 14 tak files using foobar. when I try to replay-gain them foobar said something like "track 7" is damaged or corrupted (it stop at around 30% of the file) I scan 2 time but it still happen. I was busy at the time so I didn't bother testing more.

In your first post you wrote, it did work with a separate file: "but I can fix it when convert just that track and it's fine, need more testing on this when I have time." Can you please explain what exactly you did?

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: buktore on 2008-12-22 14:43:31
Well.. I still have the APE file (actually APE & separate CUE)

EDIT: what I mean was, I still have the ZIP that contain APE file. NOT the one I used at that time. I kinda doubt that it make any different since I can replay-gain the APE file I use to convert to TAK at the time just fine.

I tried to convert it 5 times now, using exactly same method, couldn't reproduced the corrupt file.

I don't really mean that I can "fix" the corrupted file, I just covert "track 7" from the same APE file to TAK again and it seem to be fine this time so it "fix" the problem for me.

And since I was busy at the time, I didn't check that the APE file is corrupted or not? (It's not, according to foo_verifier), If I convert the whole album again not just "track 7" will it happen again? (the answer is NO) because of this, I said "need more testing on this when I have time"

I hope that make sense.. 
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-22 16:48:04
Well.. I still have the APE file (actually APE & separate CUE)

Great!

And since I was busy at the time, I didn't check that the APE file is corrupted or not? (It's not, according to foo_verifier), If I convert the whole album again not just "track 7" will it happen again? (the answer is NO) because of this, I said "need more testing on this when I have time"

Now i am not sure if you have tested the whole album again?

Sorry, although i didn't want to put pressure on you it might have happend. I am a bit in panic, because such errors can be a killer for an archiver... 

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: buktore on 2008-12-22 17:29:15
Quote
Now i am not sure if you have tested the whole album again?


Yes, I did. 5 times, everything's fine, no corrupted file show up so far. want me to try more?

Quote
Sorry, although i didn't want to put pressure on you it might have happend


No sweat  feel free to tell me what you want me to do.

Quote
I am a bit in panic


You sure is.  first I thought it was my system (then again, this never happen before) now, it's really look like you knew this might happen..  have any clues?
Title: TAK 1.1.0 - Beta release
Post by: Alexxander on 2008-12-22 17:37:49
A quick conversion test with 2 albums compressed in FLAC 1.2.1 -8 using foobar2000 v0.9.6:

I did 3 times the same conversion and post only the best times (others discarded).

Default compression, fb2k parameters -e -ihs - %d
Tak1.0.4 took 44 secs and produced 816.771 kBytes
Tak1.1.0b3 took 47 secs and produced 816.719 kBytes

Maximum compression, fb2k parameters -e -p5m -ihs - %d and -e -p4m -ihs - %d
Tak1.0.4 took 3:28 min:sec and produced 811.294 kBytes
Tak1.1.0b3 took 2:41 min:sec and produced 811.324 kBytes

With these 2 albums Tak1.1.0b3 encodes slower, I tried it a 4th time half an hour later and still Tak1.0.4 default is faster. The tags are all preserved so compression is clearly similar at default and maximum compression.

Edit:
Just encoded twice 6 other albums FLAC 1.2.1 -8 encoded at default level:
Tak1.0.4 took 2:00 min:sec and produced 2.093.069 kBytes
Tak1.1.0b3 took 2:08 min:sec and produced 2.093.002 kBytes

Machine is Vista+Sp2beta with Intel Core2Duo at 2,4GHz and there were now no other CPU intensive tasks nor hard disk activity.
Title: TAK 1.1.0 - Beta release
Post by: Synthetic Soul on 2008-12-23 09:20:36
Quote
I am a bit in panic
You sure is.  first I thought it was my system (then again, this never happen before) now, it's really look like you knew this might happen..  have any clues?
No, I think Thomas is, quite rightly, concerned about a "report" about a corrupted TAK file.  However, it seems to me, from everything that you've said, that this can only be an I/O issue.  If the same process repeated five times did not result in the same corruption then it can hardly be a fault with the format itself.
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-24 00:41:57
Quote
I am a bit in panic
You sure is.  first I thought it was my system (then again, this never happen before) now, it's really look like you knew this might happen..  have any clues?
No, I think Thomas is, quite rightly, concerned about a "report" about a corrupted TAK file.  However, it seems to me, from everything that you've said, that this can only be an I/O issue.  If the same process repeated five times did not result in the same corruption then it can hardly be a fault with the format itself.

That's also my interpretation. Lightened...
Although the results are obviously different from what i expected: Decoding got slower on your system!
...
I will try to compensate this in the next version.
I am still slightly concerned about my test set-up.  Perhaps I will re-run both 1.1.0 and 1.0.4 again, to ensure that they both have an equal testing environment.  My biggest concern with my comparison is misinforming users, and developers of course!

Yeah, this attitude is definitely one important reason why i am really happy and thankful that you intend to continue TAK testing! 

I hope, it isn't to late, but i would like you to wait with another test.

Actuallay i wanted to release the final version before christmas, but since the release now anyhow has been delayed because of the uncertainties raised by buktore's report (and hey buktore, i am really thankful for your report!!! Even if it hopefully was a false alarm.), i may try to fix the speed issues and release another beta.

With these 2 albums Tak1.1.0b3 encodes slower, I tried it a 4th time half an hour later and still Tak1.0.4 default is faster. The tags are all preserved so compression is clearly similar at default and maximum compression.
...
Machine is Vista+Sp2beta with Intel Core2Duo at 2,4GHz and there were now other CPU intensive tasks nor hard disk activity.

Too bad. Well, things are getting really difficult if io-activity has to be incorporated. I could imagine, that even a speed up of the code could result in an worse interaction with the io-system and overall lead to a performance drop. Thanks for the report!

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: Synthetic Soul on 2008-12-27 21:39:00
I am still slightly concerned about my test set-up.  Perhaps I will re-run both 1.1.0 and 1.0.4 again, to ensure that they both have an equal testing environment.  My biggest concern with my comparison is misinforming users, and developers of course!
OK, I'm glad I did (although I have no idea where this leaves my comparison).  These results show 1.1.0 in a more favourable light:

Code: [Select]
     |  1.0.4         |  1.1.0
======================================
p0   |  127x    139x  |  130x    141x
p0e  |  105x          |  108x
p0m  |   59x          |   61x
p1   |  105x    138x  |  109x    140x
p1e  |   89x          |   90x
p1m  |   50x          |   51x
p2   |   65x    123x  |   66x    125x
p2e  |   52x          |   53x
p2m  |   32x          |   33x
p3   |   41x    112x  |   38x    111x
p3e  |   33x          |   31x
p3m  |   23x          |   20x
p4   |   28x    107x  |   24x    104x
p4e  |   18x          |   15x
p4m  |   16x          |   14x
p5   |   21x     99x  |
p5e  |   11x          |
p5m  |   11x          |

For the record, here's both the old (comparison figures) and new figures for both versions, to show the differences in test.  You will see that the 1.1.0 results (thankfully) compare very well; the results for 1.0.4 are unsatisfactorily different though.

Code: [Select]
      |  1.0.4 (new/old)       |  1.1.0 (new/old)
======================================================
-p0   |  127/130x    139/147x  |  130/131x    141/141x
-p0e  |  105/108x    138/146x  |  108/109x    142/142x
-p0m  |   59/ 59x    139/146x  |   61/ 61x    141/142x
-p1   |  105/107x    139/145x  |  109/108x    141/141x
-p1e  |   89/ 89x    138/145x  |   90/ 91x    140/141x
-p1m  |   50/ 50x    138/146x  |   51/ 51x    140/142x
-p2   |   65/ 65x    124/128x  |   66/ 66x    125/126x
-p2e  |   52/ 52x    123/128x  |   53/ 53x    125/126x
-p2m  |   32/ 32x    123/128x  |   33/ 32x    125/125x
-p3   |   41/ 41x    113/117x  |   38/ 38x    112/113x
-p3e  |   33/ 33x    113/117x  |   31/ 31x    111/112x
-p3m  |   23/ 23x    112/116x  |   20/ 20x    111/112x
-p4   |   28/ 28x    107/111x  |   24/ 24x    104/103x
-p4e  |   18/ 18x    107/111x  |   15/ 15x    104/104x
-p4m  |   16/ 16x    107/111x  |   14/ 14x    104/104x
-p5   |   21/ 20x     97/ 98x  |
-p5e  |   11/ 11x     99/102x  |
-p5m  |   11/ 11x     99/102x  |

NB: Just seen Thomas' response above. Well, I am always happy to test some more Thomas.

Damn, that means I should not have updated my comparison, and I cannot update it again until I can find the time to re-test all codecs (which could be some time).
Title: TAK 1.1.0 - Beta release
Post by: lostintime on 2008-12-28 00:30:11
I've created a 60 second 48 kHz (i.e. 2,880,000 samples) 16 bit mono WAV file of silence which I've named "60 seconds of silence.wav".  When I do the following command:

Code: [Select]
type "60 seconds of silence.wav" | takc -e -pmax - "60 seconds of silence.tak"

I get a 1,052,878 byte file.  But if I use TAKC directly on the file instead, i.e. without piping it to TAKC, with this command:

Code: [Select]
takc -e -pmax "60 seconds of silence.wav"

Then I only get a 4,342 byte file.  Both the 1,052,878 byte file and the 4,342 byte file decode correctly to the original WAV file.  I've tested both version 1.0.4 and version 1.1.0 beta 3 of TAKC with the same results.

Version 1.2.1 of FLAC doesn't do this, it produces exactly the same output in both situations:

Code: [Select]
type "60 seconds of silence.wav" | flac -e -8 - -o "60 seconds of silence.flac"
flac -e -8 "60 seconds of silence.wav" -o "60 seconds of silence_2.flac"

Produces an identical 16,714 byte file both times.  Why is there such a big difference in size with TAKC?
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-28 02:20:59
I am still slightly concerned about my test set-up.  Perhaps I will re-run both 1.1.0 and 1.0.4 again, to ensure that they both have an equal testing environment.  My biggest concern with my comparison is misinforming users, and developers of course!
OK, I'm glad I did (although I have no idea where this leaves my comparison).  These results show 1.1.0 in a more favourable light:
...
For the record, here's both the old (comparison figures) and new figures for both versions, to show the differences in test.  You will see that the 1.1.0 results (thankfully) compare very well; the results for 1.0.4 are unsatisfactorily different though.
...
NB: Just seen Thomas' response above. Well, I am always happy to test some more Thomas.

Thank you, that' extremely helpful and will save me a lot of work! 

Currently i have only two ideas what could be causing the different results:

1) The disk-io-system is behaving differently. Possibly smaller or larger io-sizes in TAK could make a difference.
2) The cpu-time is beeing calculated differently.

My intuition prefers 2).

NB: Just seen Thomas' response above. Well, I am always happy to test some more Thomas.

Great! For the testing of my possible modifications a pass with -p0 will be sufficient.

I've created a 60 second 48 kHz (i.e. 2,880,000 samples) 16 bit mono WAV file of silence which I've named "60 seconds of silence.wav".  When I do the following command:

Code: [Select]
type "60 seconds of silence.wav" | takc -e -pmax - "60 seconds of silence.tak"

I get a 1,052,878 byte file.  But if I use TAKC directly on the file instead, i.e. without piping it to TAKC, with this command:

Code: [Select]
takc -e -pmax "60 seconds of silence.wav"

Then I only get a 4,342 byte file.  Both the 1,052,878 byte file and the 4,342 byte file decode correctly to the original WAV file.  I've tested both version 1.0.4 and version 1.1.0 beta 3 of TAKC with the same results.

If pipe encoding is used without the -ihs paramter (that is for instance required for foobar), TAK will try to save the wave meta data to the file. By default it will reserve 1 MB in the header! That's the default maximum size for both pipe and file based encoding.

When performing file beased encoding TAK can determine the actual meta data size and will reserve only as much space as is required. But with pipe encoding this isn't possible because there can be a footer at the end of the wave data and it's size can not be calculated before the end of the pipe has been encountered.

You may limit the maximum wave meta data size with the wm-switch:

-wm46 reserves 46 bytes for a standard wave header without any additional info.

-wm0 to reserve no space for the wave header. The reconstructed header of the encoded file may differ from the original file.

By default also a bit space for the seek table is beeing reserved. You may disable it with the sts-switch:
-sts0
Title: TAK 1.1.0 - Beta release
Post by: Bugs.Bunny on 2008-12-28 13:29:53
Hi TBeck,
I'm quite impressed by TAK! I did some testing and everything worked flawlessly. But for the moment I will stick with mokey's audio due to the lack of support in other software (bass audio/ffdshow...) I'm really looking forward to TAK 2.0 when these restrictions should fall (due to opening of the source code - if I got that right).
I think TAK has a great potential so thumbs up for the future development!
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-29 05:11:58
Damn, that means I should not have updated my comparison, and I cannot update it again until I can find the time to re-test all codecs (which could be some time).

Now i am getting a bit egocentric...    If your primary purpose is TAK-testing, i would propose this approach:

1) Since i tend to describe TAK's performance in relation to FLAC and Monkey's Audio, i would be very happy to see a new comparison of them. Well, i will release another, slightly faster (and final!) beta soon.

2) You may keep your old comparison (which has become famous outside of TAK testing...) and update the new comparison step by step (codec by codec), when you have time and/or a new codec version has been released.

If pipe encoding is used without the -ihs paramter (that is for instance required for foobar), TAK will try to save the wave meta data to the file. By default it will reserve 1 MB in the header! That's the default maximum size for both pipe and file based encoding.

The next beta release will come with the default size for pipe encoding set to 46 Bytes instead of 1 MB. Thanks to "lostintime" for reporting the inappropriate default value!

I think TAK has a great potential so thumbs up for the future development!

Thank you very much for your encouragement!

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: halb27 on 2008-12-29 08:07:06
No doubt: looking at compression ratio and important decompression speed TAK is the best player in the lossless field. Congratulations!

Just 2 questions @ TBeck:
a) TAK yields the best combination together with lossyWAV right now. Some months ago you announced specific development for use with lossyWAV to achieve further improvement. What's the current state here?
b) The impressive decompression speed makes TAK highly desirable for use with mobile DAPs (like FLAC, but with a better compression ratio). Has there been any contacts with DAP producers allowing them to implement TAK and giving them the necessary information? This would be great, especially as the time for lossless codecs on mobile DAPs is yet to come, but we may be close to it right now. It would be great to have TAK be a major player in this field.
Title: TAK 1.1.0 - Beta release
Post by: lostintime on 2008-12-29 08:29:37
The next beta release will come with the default size for pipe encoding set to 46 Bytes instead of 1 MB. Thanks to "lostintime" for reporting the inappropriate default value!


Could I suggest that you also change the behaviour of TAKC so that if doesn't report an "invalid file extension" whenever you try to give an output filename an extension anything other than tak?  FLAC will let you call the output file anything you want, with no extension added if not present in the output filename.
Title: TAK 1.1.0 - Beta release
Post by: jcoalson on 2008-12-29 17:11:09
b) The impressive decompression speed makes TAK highly desirable for use with mobile DAPs (like FLAC, but with a better compression ratio).

http://www.hydrogenaudio.org/forums/index....mp;#entry531840 (http://www.hydrogenaudio.org/forums/index.php?showtopic=58850&st=0&p=531840&#entry531840)
Title: TAK 1.1.0 - Beta release
Post by: halb27 on 2008-12-29 17:43:14
b) The impressive decompression speed makes TAK highly desirable for use with mobile DAPs (like FLAC, but with a better compression ratio).

http://www.hydrogenaudio.org/forums/index....mp;#entry531840 (http://www.hydrogenaudio.org/forums/index.php?showtopic=58850&st=0&p=531840&#entry531840)

What do you mean? I'm quite aware that decompression speed depends on the format, implementation details and how they match the specific hardware.
Do you think FLAC is expected to be significantly faster than TAK on mobile DAPs?
Title: TAK 1.1.0 - Beta release
Post by: greynol on 2008-12-29 17:46:18
To put it more simply, I think the question should be whether TAK will be fast enough.

The fact that flac decompression speeds on a PC aren't compared on an equal basis really shouldn't have anything to do with it.  It isn't like you're suggesting TAK is out-performing flac with anything except for the compression ratio.

EDIT: Formatting, removed an extra "really" and some other stuff too.
Title: TAK 1.1.0 - Beta release
Post by: jcoalson on 2008-12-29 19:12:20
I mean just that the comparison of command-line decoding to file on a pc does not tell enough about feasibility in common embedded devices.  we know many devices are capable of flac decoding (all modes) with a c decoder that is not optimized for the target, with some headroom.  this comparison appears to show some tak modes on par with all flac modes, except:

- that comparison is not apples-apples because the flac time includes md5 computation which is significant and not present in playback situations (accounted for here (http://flac.sourceforge.net/comparison_all_cpudectime.html) and here (http://www.synthetic-soul.co.uk/comparison/lossless/index.asp?Sort=DecodeTime&Desc=0&All=1))
- we already see variability in tak timing due to optimization for particular i/o cases on particular hardware

what matters is the complexity of the decode process; my hunch based on implementation of ape modes on rockbox and what little we know about tak is that with an open source c decoder, rockbox can probably get most but not all modes of tak decoding on most hardware.  for other devices (native support) that's the first of many hurdles.


see also: http://www.rockbox.org/twiki/bin/view/Main...manceComparison (http://www.rockbox.org/twiki/bin/view/Main/CodecPerformanceComparison)
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-29 19:31:32
- that comparison is not apples-apples because the flac time includes md5 computation which is significant and not present in playback situations (accounted for
Well, in Synthetic Souls' previous comparison TAK -p0 and -p1 where both decoding faster than FLAC -8 without MD5...

(http://flac.sourceforge.net[/quote)
- we already see variability in tak timing due to optimization for particular i/o cases on particular hardware

I don't know, why you think TAK has been optimized for particular hardware?

- The io routines and buffers are always the same. Some time ago i deceided to use a very small buffer for the io-system, which possibly isn't adaequate. I may increase it's size a bit. Would you call a fixed size for an io-buffer, which is always the same indpendently of the os or harware, a specific optimization?

- TAK does contain only one set of dsp functions (i386 and MMX) that are beeing used for any cpu.

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: halb27 on 2008-12-29 22:14:21
Well, TAK's speed compared to FLAC's as taken in for instance SyntheticSoul's comparisons isn't necessarily the same thing on a mobile DAP.
I think so far we all agree. This doesn't say however that TAK can't be a very fast codec on a mobile player.
It's all theory as long as there is no such player. Until then hope is justified by experience with a PC.

@TBeck: Any plans to contact DAP manufacturers? (Time is tight: It will be few years only that we will see giant flash memory so that everybody can encode his collection the lossless [or lossyWAV] way. It would be great to have TAK then as a widespread codec in case the speed hope remains true).
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-30 15:34:24
No doubt: looking at compression ratio and important decompression speed TAK is the best player in the lossless field. Congratulations!

Thank you! 

a) TAK yields the best combination together with lossyWAV right now. Some months ago you announced specific development for use with lossyWAV to achieve further improvement. What's the current state here?

I will make those specific improvements part of the new codec which will be introduced with TAK 2.0. Currently i am expecting these advantages for LossyWav files:

- At least 10 kbps better compression.
- The compression of audio parts where LossyWav hasn't removed any bits will be about as good as for the standard codec. Currently the compression here is suffering from the small frame sizes used for LossyWav.

b) The impressive decompression speed makes TAK highly desirable for use with mobile DAPs (like FLAC, but with a better compression ratio). Has there been any contacts with DAP producers allowing them to implement TAK and giving them the necessary information? This would be great, especially as the time for lossless codecs on mobile DAPs is yet to come, but we may be close to it right now. It would be great to have TAK be a major player in this field.

@TBeck: Any plans to contact DAP manufacturers? (Time is tight: It will be few years only that we will see giant flash memory so that everybody can encode his collection the lossless [or lossyWAV] way. It would be great to have TAK then as a widespread codec in case the speed hope remains true).

There is contact with one Rockbox member. But i don't think i will release any code before the beginning of the TAK 2.0 line.

Could I suggest that you also change the behaviour of TAKC so that if doesn't report an "invalid file extension" whenever you try to give an output filename an extension anything other than tak?  FLAC will let you call the output file anything you want, with no extension added if not present in the output filename.

I will think about it. For this release it's too late, because i would have to perform a significant amount of testing.

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2008-12-30 23:35:43
Beta 4 has been released

The download link and the history are in the first post.

Probably i could have made this a final release, but i am a bit cautious these days...

Well, if there are no problems with this beta, i will release it as final version this weekend.

There may be some very tiny speed improvements because i have optimized the code alignment of some functions. This shouldn't really matter, but i want to make sure, that accidental misaligment isn't the cause for the slightly worse decoding speed of TAK 1.1.0 in Synthetic Soul's comparison.

I still think that subtle io issues are responsible for the speed drop, but i have deceided to investigate this for the next version, which hopefully will be released in about 3 months.

Thomas
Title: TAK 1.1.0 - Beta release
Post by: adamjk on 2008-12-31 12:56:03
You are the best!
Title: TAK 1.1.0 - Beta release
Post by: Synthetic Soul on 2008-12-31 19:45:09
Code: [Select]
     |  1.1.0b3       |  1.1.0b4
     |  Enc     Dec   |  Enc     Dec
=====================================
p0   |  130x    141x  |  132x    142x
p0e  |  108x          |  110x
p0m  |   61x          |   61x
p1   |  109x    140x  |  109x    141x
p1e  |   90x          |   91x
p1m  |   51x          |   50x
p2   |   66x    125x  |   67x    125x
p2e  |   53x          |   53x
p2m  |   33x          |   33x
p3   |   38x    111x  |   38x    112x
p3e  |   31x          |   31x
p3m  |   20x          |   20x
p4   |   24x    104x  |   24x    104x
p4e  |   15x          |   15x
p4m  |   14x          |   14x
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2009-01-03 13:32:55
You are the best!

No, mommy is the best! 

Code: [Select]
     |  1.1.0b3       |  1.1.0b4
     |  Enc     Dec   |  Enc     Dec
=====================================
p0   |  130x    141x  |  132x    142x
...

Thank you, that was fast!

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2009-01-03 14:08:02
Prepearing the final release

I want to release the final version within the next two days. Well, i am already working on V1.1.1...

Please report any bugs or misbehaviour you may have encountered with the latest beta 4.

  Thomas
Title: TAK 1.1.0 - Beta release
Post by: Synthetic Soul on 2009-01-03 16:54:45
Thank you, that was fast!
No worries, the holiday season has given me a little more time to let tests run.

I know that you had said previously that -p0 was only required, but I didn't know whether that was still the case, and I was keen to perform yet another test to correlate my figures further.  I was pleased to see very similar values.  Where values are different I believe it is only with the faster speeds - where hundredths of a second may make a difference - and only a variance of one unit.  Except -p0 and -p0m's encoding speed, which were out by two.  Is this relevant to your changes do you think?  Were you hoping to see a larger variation?
Title: TAK 1.1.0 - Beta release
Post by: TBeck on 2009-01-05 04:53:03
Thank you, that was fast!
No worries, the holiday season has given me a little more time to let tests run.

Good for me! 

I know that you had said previously that -p0 was only required, but I didn't know whether that was still the case, and I was keen to perform yet another test to correlate my figures further.  I was pleased to see very similar values.  Where values are different I believe it is only with the faster speeds - where hundredths of a second may make a difference - and only a variance of one unit.  Except -p0 and -p0m's encoding speed, which were out by two.  Is this relevant to your changes do you think?  Were you hoping to see a larger variation?

No. That's quite exactly what i expected. Thank you again!

  Thomas


TAK 1.1.0 final has been released

See this thread (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=68454&view=findpost&p=607825).