Skip to main content
Topic: TAK - Alpha testing (Read 52385 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

TAK - Alpha testing

Reply #25
Although i have to admit, that i am not familar with FLAC -7.

Well, I meant FLAC preset level 7 (the command line is "flac -7", which is why I worded it that way)

I edited my previous post to clear up any misunderstanding.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

TAK - Alpha testing

Reply #26
Hmm. I've got a file that compresses better with Tak Normal than Tak High /Extra High.

The song i tried was "drug me" by Dead Kennedys. Here's the result:

Original Size:      20369896 (bytes)
Tak Turbo:            8743793
Tak Fast:              8732490
Tak Normal:          8500839
Tak High:              8666097
Tak Extra High:    8598122


More thorough testing coming soon.

TAK - Alpha testing

Reply #27
[deleted]

TAK - Alpha testing

Reply #28
Hope that is useful, funny to see how many times the max mode of a lower setting outperforms the next standard setting (and still decodes faster).

Oh yes.

What differentiates the higher from the lower presets?

1) A better evaluation of some compression parameters. The higher presets are adding more and more of the evaluation options. Max can do the same for the lower presets. If the compression advantage of a higher presets is caused by the better evaluation, then a lower preset + max can achieve the same.

2) Higher presets can use more predictors. But the usefullnes of more predictors varies very much depending on the files. Some are happy with 4 - 8 predictors, most like up to 32 (seems to be the sweet spot) and quite rare files benefit enormously from up to 256 (or more, earlier Tak used up to 512) predictors.

3) High and Extra can use the Prefilter. Very efficient on some files, small effect (0.1 to 0.2) percent most of the time, even can hurt sometimes. Unfortunately it is often beeing applied even if it's advantage isn't significant. But if applied, it will definitely reduce the decoding speed. Well, that's the price we have to pay for the very fast encoding: It's based upon fast estimations which work perfectly most of the time but sometimes fail. A very very slow insane mode could cure this.

Also funny to see how different the results are here than on the previous test I ran.

Yes, this is always a bit strange. It's really difficult to create a representative test set to judge codec performance.

Your first set was a very Tak friendly one. Only in rare cases it will outperform for instance Monkey's Extra High.

Your second set is closer to my average impression. Well, possibly a bit worse.

The truth may lie somewhere in between.

Thank you again!

  Thomas

TAK - Alpha testing

Reply #29
[deleted]

TAK - Alpha testing

Reply #30
I've compressed Seabound - Double Crosser Limited Edition 2-CD:

Code: [Select]
     548,423,733  OptimFROG 4.600ex extra high.ape
     547,063,517  OptimFROG 4.600ex insane.ape

These two are a little confusing to me. I think I know what you mean, but it could be clearer... 

BTW, the biggest part of the new WavPack release is the work I put into the -x modes levels 1-3 (which are now completely new) and I see you're not using any of them for your tests. The plain -x option (no param) is now completely usable in fast and normal mode. It gives a nice compression boost with not much of an encoding speed hit and (like before) no decoding speed hit. At this point I think that the only time it makes sense to not use it is when you're really in a hurry to encode something, and I've thought of making it the standard operation.

Just a thought, and thanks for your testing... 

TAK - Alpha testing

Reply #31
BTW, the biggest part of the new WavPack release is the work I put into the -x modes levels 1-3 (which are now completely new) and I see you're not using any of them for your tests. The plain -x option (no param) is now completely usable in fast and normal mode. It gives a nice compression boost with not much of an encoding speed hit and (like before) no decoding speed hit. At this point I think that the only time it makes sense to not use it is when you're really in a hurry to encode something, and I've thought of making it the standard operation.

Just a thought, and thanks for your testing... 

Thanks!  I was just about to ask what to switches to recommend for the latest WavPack evaluation. I was just using: -f, [default] and -h. I think the faster-decoding modes are appropriate to use when comparing to TAK.
"Something bothering you, Mister Spock?"

TAK - Alpha testing

Reply #32
[deleted]

TAK - Alpha testing

Reply #33
Hmm. I've got a file that compresses better with Tak Normal than Tak High /Extra High.

The song i tried was "drug me" by Dead Kennedys. Here's the result:

Original Size:      20369896 (bytes)
Tak Turbo:            8743793
Tak Fast:              8732490
Tak Normal:          8500839
Tak High:              8666097
Tak Extra High:    8598122

Interesting. Thank you!

This can happen, because the encoder only estimates the effect of some compression methods and sometimes this fails and it makes the wrong choice. Most probably it is applying the Prefilter although it hurts or it is using more predictors than neccessary.

I have one file in my test corpus with some similar failure.

Would it be possible to send me about 20 to 30 seconds (if it's legal in your country!) of this file which are able to reproduce this effect? I would like to look for the reason of this misbehaviour.

  Thomas

TAK - Alpha testing

Reply #34
[deleted]

TAK - Alpha testing

Reply #35
I've now emcoded 29 albums and various "problem tracks" and I haven't found any anomalies whatsoever. This will be my codec of choice once a Foobar plugin is released. This is the biggest change in speed and compression I've seen in any lossless codec in years.
I'll do some 24/96 multichannel later.

TAK - Alpha testing

Reply #36
Important question

I am tempted to release a second alpha until wednesday.

It fixes the 2 GB bug. For this i had to switch to Int64 types in several places of the code. Unfortunately the code parts where the default Int32 types and the Int64 types are mixed have some chance to create errors (i allready found some by myself). It is really neccessary to test this new version before the beta release.

I think it would be ok to use the new version for our testing,

- because anything else looks ok and i wouldn't expect to encounter more severe failures of the current alpha (V0.14)
- and because i don't want to ask you to repeat your testing for a new alpha released after christmas.

Compression and speed should not be affected by the bug fixes.

What do you think?

TAK - Alpha testing

Reply #37
As for me, it's ok

TAK - Alpha testing

Reply #38
[deleted]

TAK - Alpha testing

Reply #39
I'll be interested to try this codec once it enters the beta/stable stage!

Quote
This will be my codec of choice once a Foobar plugin is released.

If I decide to use lossless compression again (i.e. weigh up the need for with a 279 GB disk and limited CD collection), I may be joining you!

TAK - Alpha testing

Reply #40
TAK vs The Rest of The World:
http://www.synthetic-soul.co.uk/comparison/lossless/

TAK 0.14 vs TAK 0.13:
http://www.synthetic-soul.co.uk/comparison/tak/

Sorry for the delay, I thought I better update to FLAC 1.1.3 and WavPack 4.40 at the same time, and those WavPack x modes take some time!  Unfortunately it looks like I screwed them up somehow, so I'll have to spend a couple of days running them again!  I have reverted back to 4.4a3 for now.
OK, comparison is showing WavPack 4.40 now.

I've run the -v tests today also, so hopefully tonight I'll get opportunity to collate and post figures.
I'm on a horse.

TAK - Alpha testing

Reply #41
Here's CPU-only results, showing the difference between using -v (verify) while encoding, and not.

Code: [Select]
Preset       No -v     -v    Diff
=================================
Turbo          98x    63x    156%
Turbo Max      39x    32x    123%
Fast           64x    45x    143%
Fast Max       28x    24x    117%
Normal         43x    34x    128%
Normal Max     24x    21x    115%
High           28x    23x    120%
High Max       15x    13x    110%
Extra          18x    16x    114%
Extra Max       7x     6x    105%

I may compile these for CPU+IO rates as well, but the obvious assumption is that my hard drive throttling will narrow those differences a fair bit.  The fastest I seem to be able to get writing to disk is ~80x, so even with Turbo it should drop to around 130-140%.
I'm on a horse.

TAK - Alpha testing

Reply #42

Hmm. I've got a file that compresses better with Tak Normal than Tak High /Extra High.
...

Interesting. Thank you!

This can happen, because the encoder only estimates the effect of some compression methods and sometimes this fails and it makes the wrong choice. Most probably it is applying the Prefilter although it hurts or it is using more predictors than neccessary.

I had the opportunity to test (a part of) the file.

First the compression ratio for any preset, each with default and maximum evaluation:

Code: [Select]
          Standard   Max  (Evaluation)
----------------------------------------
Turbo       42.93      41.99
Fast        42.87      41.55
Normal      41.73      41.37
High        42.54      42.34
Extra       42.21      42.06

There is something in Max evaluation which really helps Fast: 1.32 better compression. It is also present in Normal default, where the advantage of Max is much smaller.

Let's find out what in evaluation max is responsible. Try Fast with each single option that would be added by evaluation max:

Code: [Select]
Fast (default)                42.87
Channel decorr.: Check Both   42.87
Channel decorr.: Mid/Side     42.84
Bitcoder: Optimize choice     42.81
Partition resolution: 256     42.77
Partition Calc.: Validate     42.65
Optimize Quantization         41.98
Fast + Max                    41.55

Sorted by compression.

Obviously "Optimize quantization" is the winner. That's very rare and therefore very interesting for me.

Now for High. Let's use maximum evaluation and play a bit with the PreFilter sensitivity:

Code: [Select]
High + Max         42.34 (Prefilter medium)
-Prefilter low     42.27
-Prefilter off     41.22

Huh, better disable the PreFilter for this file...

Well, sometimes my fast parameter estimation fails. I will evaluate this later.

TAK - Alpha testing

Reply #43
V0.15 Alpha is done

Changes:

Improved:

- Support for wave files of up to 4 GB Size (2 GB before). To be exact: 4 GB - 15 Bytes...
- When test encoding/decoding files existing destination files are beeing ignored.
- GUI version responds faster when using slow presets.

And an update for the Damage tool (V1.02):

- Support for files of up to 32 GB Size (2 GB before).

Release:

I hope to send the new version to the alpha testers within the next hours.

What should be tested:

No changes. Keep on your good work!

Feedback of the two testers who found the 2 GB bug would be nice.

  Thomas

TAK - Alpha testing

Reply #44
Thanks for the update Thomas. No change on the my greater than 2gb WAVs. The program still throws "Wave file not supported" for both.

TAK - Alpha testing

Reply #45
Thanks for the update Thomas. No change on the my greater than 2gb WAVs. The program still throws "Wave file not supported" for both.

Too bad...

Is there any meta data embedded?

Would it be possible to send me the first 100 KByte of the file?

Probably there is something in the file header that my very simple wave file reader can not handle.

Sorry...

P.S. Currently Tak is not able to handle files with more than two channels!

edit: P.S. added.

TAK - Alpha testing

Reply #46
I've uploaded my test results for the Drug Me Sample.

Download Links:
HTM: http://www.freewebs.com/kanak/DrugMe.htm
PDF: http://www.freewebs.com/kanak/DrugMe.pdf
XLS: http://www.freewebs.com/kanak/DrugMe.xls

Song Details:
Song Tested: Drug Me by Dead Kennedys (Album: Fresh Fruit For Rotting Vegetables)
Song Length: 1:55
Uncompressed Size: 19.4 MB

Test Bed:
Dell e1505 Laptop: Intel Core 2 Duo T7200 @ 2 Ghz, 1 GB Ram, 120 GB HDD 5400 RPM

Purpose:
0. To see if I have configured Synthetic Soul's scripts properly by doing a "real world" test on a small file.
1. To test TAK
2. To see if the anomalous behavior of TAK on this file (compresses better with TAK Normal than TAK High or Tak Extra High (See Here, and here) is also shown by other Encoders
3. To test the maximum compression + experimental option in the experimental version of OptimFrog (version 4.600ex)
4. To test the new -x switches in latest wavpack (version 4.40.0)

Encoders Versions and Switches Tested
1. TAK 0.15 Alpha: All Switches + Max for each
2. FLAC 1.1.3: -0 to -8
3. Wavpack 4.40.0: All Switches + (x1 through x6 for each)
4. Monkey's Audio 4.01: Fast through Insane
5. Optimfrog 4.600 ex: Default (no switches) and (--maximumcompression --experimental)
6. ALS RM18: Default (no switches), -7 ("maximum compression"), -a -o32

Remarks
1. The "anomalous behavior" is also shown by Monkey's Audio (the compressed size is smaller with Extra High than with Insane), also by FLAC (-1, and -2  compress better than -3), and by Wavpack (-fx5 and -fx6)
2. Decoding speed figures are pretty "unreliable" because the song is so short. It definitely can't be used to establish trends across different switch combinations for the same codec. But it's still somewhat valid across different codecs. Nevertheless, the TAK's decoding speed, especially at higher compressions, is unmatched by any other codec. Kudos to Thomas.
3. Tak specific comments:
Amazing Performance. TAK Turbo encodes faster AND compresses better AND has better decoding speed than FLAC-8. Similarly, TAK Normal trounces Wavpack even at -hhx6. However, there's still room for improvement at the maximum compression settings (TAK can't match OFR even at default OR monkey's at insane)
4. Wavpack switches
The -x1 to -x3 switches do seem more usable (i.e. don't have a significant performance hit) than before, although the performance with the -x4 to -x6 switches are pretty horrid (as warned).

Future Plans
- Test on larger files (album images for different albums across genres)
- (Hopefully) Include WMA Lossless/ALAC
- TAK Specific: Use Damage tool to test error resilience


Edit: Corrected some typos

TAK - Alpha testing

Reply #47
I've uploaded my test results for the Drug Me Sample.

Thanks for having the patience for testing so many of the WavPack modes! 

BTW, that is an interesting sample! Would it be possible for me to get a clip of it also for my test corpus? I would appreciate it...

TAK - Alpha testing

Reply #48
I've uploaded my test results for the Drug Me Sample.
...
Wow, thanks for providing such a nice amount of extremely interesting data.
I'm a DK fan too (course talking about the Jello era), gonna fiddle with that track a bit myself.

Moderation: Edited quote.
WavPack 5.1.0 -b384hx6cmv / qaac 2.64 -V 100

TAK - Alpha testing

Reply #49
I've uploaded my test results for the Drug Me Sample.

Very nice! Synthetic Soul's scripts seem to be really useful!

3. Tak specific comments:
Amazing Performance. TAK Turbo encodes faster AND compresses better AND has better decoding speed than FLAC-8. Similarly, TAK Normal trounces Wavpack even at -hhx6. However, there's still room for improvement at the maximum compression settings (TAK can't match OFR even at default OR monkey's at insane)

I'm glad that you are testing Mpeg4Als. I am always very interested into it's results because it is using similar  methods as TAK (and FLAC).

From my experience Mpeg4Als -7 is on average only up to 0.3 percent better than TAK's strongest mode. But on this sample it really outperforms TAK by more than 4 percent!

I allready knew that such samples exists. I have an idea how i can tune TAK to perform much better on them. Unfortunately this would be incompatible with the current encoder.

But i anyhow want to add some more codecs to TAK in the future.

The current codec is an all in one solution. It has to cover the whole spectrum from highest speed to highest compression. Some compromises where necessary.

I will probably add two specialized codecs: One for an even higher (decoding) speed and one for higher compression. The latter will sacrifice some decoding speed for higher compression, but not too much. TAK has to be fast, otherwise it would be quite useless.

But now stop dreaming about the future and back to the work. There is still the bug in the handling of some large wave files...

Future Plans
- Test on larger files (album images for different albums across genres)
- (Hopefully) Include WMA Lossless/ALAC
- TAK Specific: Use Damage tool to test error resilience

Great!

Thomas

 
SimplePortal 1.0.0 RC1 © 2008-2019