Skip to main content ↑ ↓
Topic: TAK - Alpha testing (Read 52437 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

TAK - Alpha testing

Content

This thread should be used to discuss the first alpha release of my lossless audio compressor TAK (formerly known as YALAC):

1. Bug reports

That's the most important purpose of this alpha testing!

Please try to provide as many details as possible about the conditions which created the error. And please keep the affected files. I may ask you for them.

2. Compression results

They should no longer be posted in the "YALAC - Comparisons" - thread.

3. Ideas for improvements and feature requests

I am aware, that the first release is missing many important features, especially tagging and plugin support for media players. No need to tell me about this...

I will collect your ideas and requests and try to implement them in later releases.


Participiants

Primary the alpha testers, because only they will have the opportunity to test TAK...

This should be the last closed test, the beta will be a public release.

TAK - Alpha testing

Reply #1 –
Can you add a link to the binaries? That would come in handy at this place...

unEDIT: Didn't see your response in time.

TAK - Alpha testing

Reply #2 –
Request for testers

No, the alpha version isn't done yet, but nevertheless i wanted to start this thread, because

- i want to force myself to release the alpha. I am still not really happy with the design and code quality of some higher level parts of the compressor (especially streaming and file io, the encoder core itself looks quite good). But i know myself, if i try it longer, i tend to get more and more demanding.

- i need some more testers for the alpha.

I will sent the alpha to my core testers:

Destroid
JohanDeBock
Josef Pohm
Synthetic Soul

Other earlier testers may sent me a mail, if they want to try the alpha.

Then i would like to invite up to 10 new testers.

Formal qualification:

"Member of hydrogen since 3 months or more and at least 5 posts."

Take it as a simple "i am really interested into audio processing" indicator.

Please contact me via personal mail.

BTW: I hope to release the alpha until next monday (fingers crossed).

  Thomas


New testers list

1. Thundik81
2. Qest
3. Gnerma
4. Omion
5. Zergen
6. Kanak
7. TrNSZ
8. Night Rain
9. audiofreak

edit: new tester added

TAK - Alpha testing

Reply #3 –
V0.14 Alpha is done

Let's search for bugs!

The file format will no longer change after this release. Later versions will be backwards compatible.

Changes

Encoder:

- Frame partition resolution 256 is back. It still has a very bad speed/compression ratio, but i hope to improve it later.

Presets:

- Additional evaluation MAX now enables Frame partition resolution 256. A tiny bit better compression, but with a considerable speed penality for higher presets.

Functionality:

- Better error handling: The program should no longer be aborted, if a file io error occurs.
- Removed output of internal encoder diagnostics if "Protocol level" Results & Diagnostics is selected.

File format:

- Generally faster seeking, especially if the seek table can not be used.
- Stronger protection (CRC) of meta data structures. Depending on the frame size up to 0.01 percent compression penality (preset TURBO).

Command line:

- New switch to enable the Verify function for encoding.
- I finally managed to remove some modules which were only needed for the GUI version. The exe is much smaller now and it should take a bit less time to initialize the program. Possibly the processing of single small files will be a bit faster.

Other:

- Fixed the bug reported by Destroid for V0.13: Decoding of a big file (about 700 MB) gave the error "Meta data damaged". Nevertheless the audio data itself could be decoded without any errors.
- Some other bugs removed.
- Rework of the ReadmMe.

Release:

I hope to send the new version to the following testers within the next 24 hours:

Destroid
JohanDeBock
Josef Pohm
Synthetic Soul
Thundik81
Qest
Gnerma
Omion
Zergen
Kanak

Any of the earlier testers not listed here can sent me an email anytime, and i will send the current version.

P.S.: Possibly i will get a faster internet connection within the next 24 hours. If i should encounter technical difficulties, there could be some delay.

What should be tested:

This time i don't want to tell you in detail what to do... Try all the things i never thought of and find the bugs i could not find! That's the most important thing.

Less important:

- Comparisons with other compressors are always welcome...
- You could try some encoding with verify enabled, to check, if it throws false alarms.

Plans for V 0.15 Beta:

- Bug fixes, if neccessary.

I would be happy to release the public beta in early january.

  Thomas

TAK - Alpha testing

Reply #4 –
why don't you just join the FLAC team man.....you'd be a great asset helping them!!!

TAK - Alpha testing

Reply #5 –
Because this is his encoder and it is better.

TAK - Alpha testing

Reply #6 –
so instead of fighting an uphill battle it would be great for a merge! no need to add formats and such, FLAC is so popular anyway

TAK - Alpha testing

Reply #7 –
I don't think he's competing with anyone, he just wants to make his own lossless codec. Besides, merging two completely different codecs is not as easy as you may think, even if they're both lossless.
Allegari nihil et allegatum non probare, paria sunt.

TAK - Alpha testing

Reply #8 –
Quote from: tempnegro on
so instead of fighting an uphill battle it would be great for a merge! no need to add formats and such, FLAC is so popular anyway


Quote from: ssjkakaroto on
I don't think he's competing with anyone, he just wants to make his own lossless codec. Besides, merging two

I could not have said it better...

I always was primary interested into trying what i can achieve in lossless audio compression. For many years i did it just for my own fun without telling anyone about it! Now it's fun for me to make something usable of it.

Quote from: ssjkakaroto on
Besides, merging two completely different codecs is not as easy as you may think, even if they're both lossless.

I totally agree.

There has been some conversation with Josh Coalson about an integration of Tak technology into FLAC. And i could imagine, that this will happen sometime in the future (if Josh still wants to and if he has not improved FLAC so much, that an integration of Tak would make no more sense...).

But i would not like to do this with the current implementation of Tak. Reasons:

1) The current codec is quite complex. There are about 600 KB of source code (core of the codec only) which had to be translated to C. This would be very much work (i am no wiz in C). I really don't like to spend so much of my free time for something so little interesting. I am in it for fun and would prefer to spend this time for codec improvements.

2) The code complexity is also a bad thing for the integration into FLAC or other open source projects. More code = more chances for bugs and more trouble with adaptions to other systems.

3) Tak is very fast on average, but the peak processing requirements can be quite high. In the discussion of Flake Josh warned Justin, that more than 12 predictors could increase the cpu requirements so much, that some hardware decoders were no longer able to play the files without stuttering. Hey, Tak is using up to 256 predictors! And even with only 12 predictors it's cpu requirements would be considerably higher than with FLAC.

After the public release of Tak i want to evaluate the possibility to build a new codec, based upon Tak technology but with far less code complexity and less cpu requirements. This could be a candidate for a inclusion into FLAC. But i have to try it, before i can say, if it is possible...

First i did not want to answer to this questions in this tread, but probably it was time for an update on this matter.

TAK - Alpha testing

Reply #9 –
Quote from: TBeck on
First i did not want to answer to this questions in this tread, but probably it was time for an update on this matter.

There is a lot of frustration among certain users regarding the sheer amount of competing projects in certain fields, with no single one doing everything they need. One of those fields is audio players under Unix, for instance. I can understand that some users can feel the same way about lossless codecs, with FLAC being well supported, but not so fast and not so efficient, others being almost not supported at all and even slower but much more efficient (OptimFROG [...]), others being faster and more efficient but more or less well supported (WavPack), etc... So, your explanation, for once, makes sense, explains a lot, and is very welcome as far as I'm concerned.

TAK - Alpha testing

Reply #10 –
Gnerma checking in. I've just started toying with it and here are a few things:

* No hi-def support in this release? I tried a stereo 24/96 WAV and got an error. The file is 2.31 gb uncompressed.
* Also got an error trying to compress a 2.45 gb 16/44 WAV.
* UI is slow to respond when using slow compression levels.
* UI checks to see if file exists when "Test" is attempted.

I'm not exactly sure if things like this are what you're looking for but if I'll report everything I find to be even slightly off.
 

TAK - Alpha testing

Reply #11 –
Quote from: Gnerma on
* No hi-def support in this release? I tried a stereo 24/96 WAV and got an error. The file is 2.31 gb uncompressed.
* Also got an error trying to compress a 2.45 gb 16/44 WAV.

The current implementation supports up to 24/96. But some parts of my code (file and wave io) are limited to file sizes up to 2 GB or exactly 2.147.483.647 Bytes. It's not a restriction of the encoder itself or my Tak container (supports up to more then 100 GB).

I hope, you got a somewhat senseful error message ("Wave file not supported") or did it simply crash?

To be honest, i didn't know that wave files can be bigger than 2 GB. I thought, there was the same restriction as with avi files...

Quote from: Gnerma on
Gnerma checking in. I've just started toying with it and here are a few things:

* UI is slow to respond when using slow compression levels.

You mean if you want to stop the process?

Well, i wanted it to be as fast as possible and therefore it checks not too often for input. If it's annoying, it should be changed.

Quote from: Gnerma on
* UI checks to see if file exists when "Test" is attempted.

I wanted "Test" to be an exact simulation of the real encode/decode process with file io. Wouldn't i be irritating for the user, if the test succeeds but the real processing gives an error ("file exists")?

After a bit of thinking: No, this check should be disabled. It could happen frequently that the user wants to check the just encoded files with the test option. Then it's quite probable to have both the source and the encoded files in the same folder. It would be annoying to always have to enable the overwrite option for the test...

Quote from: Gnerma on
I'm not exactly sure if things like this are what you're looking for but if I'll report everything I find to be even slightly off.

Not really. I wanted to hear that Tak is the greatest! 

Seriously: Thanks! It's exactly what i wanted!

I really didn't know that the size limitation would become so soon relevant! This will be among the top items on my to do list!

Many thanks

  Thomas

TAK - Alpha testing

Reply #12 –
Quote from: TBeck on
Quote from: Gnerma on

* No hi-def support in this release? I tried a stereo 24/96 WAV and got an error. The file is 2.31 gb uncompressed.
* Also got an error trying to compress a 2.45 gb 16/44 WAV.

The current implementation supports up to 24/96. But some parts of my code (file and wave io) are limited to file sizes up to 2 GB or exactly 2.147.483.647 Bytes. It's not a restriction of the encoder itself or my Tak container (supports up to more then 100 GB).

I hope, you got a somewhat senseful error message ("Wave file not supported") or did it simply crash?

To be honest, i didn't know that wave files can be bigger than 2 GB. I thought, there was the same restriction as with avi files...

Yes, 2 GB limit if you are too dumb to use an unsigned long for the size values... Forgive me, but when i started my work on Tak, the Delphi 3 compiler i was using did not support unsigned longs...

It should be quite easy to fix this and increase the size limit to 4 GB. Would this be sufficent for now?

(Currently i don't want to deal with wave files containing more than one audio data chunk.)

Well, looks as if there will be a second alpha release after this test...

TAK - Alpha testing

Reply #13 –
[deleted]

TAK - Alpha testing

Reply #14 –
Encoded 7 albums so far and found no issues at all. The speed and compression levels are astounding. This is a wonderful codec. Using a E6600 @ 4 GHZ.

TAK - Alpha testing

Reply #15 –
I am simply dumbfounded. I expected good results, but this is outstanding. I've tested about 5 albums and everything has been consistantly impressive across the board.

No bugs to report so far.

It's beautiful, TBeck. Beautiful.

TAK - Alpha testing

Reply #16 –
Quote from: TrNSZ on
Full testing is coming soon, but I've not yet found any problems with any files; I compressed a few GB in different modes as well as my small standard test set here.  Only 16-bit 44.1khz input was tested up to this point.  I think a slower, higher compressing mode would be appropriate as well, considering the current performance.  More work tomorrow, but so far, this is very nice.

Quote from: Night Rain on
Encoded 7 albums so far and found no issues at all. The speed and compression levels are astounding. This is a wonderful codec. Using a E6600 @ 4 GHZ.

Quote from: Qest on
I am simply dumbfounded. I expected good results, but this is outstanding. I've tested about 5 albums and everything has been consistantly impressive across the board.

No bugs to report so far.

It's beautiful, TBeck. Beautiful.

I am really grateful for this fast feedback! Thank you!

It's a very nice compensation for the many lonely nights with so little sleep...

BTW: Looks as if i have fixed the 2 GB bug. Now for the next two items on the new todo list...

  Thomas

TAK - Alpha testing

Reply #17 –
Quote
The current implementation supports up to 24/96. But some parts of my code (file and wave io) are limited to file sizes up to 2 GB or exactly 2.147.483.647 Bytes. It's not a restriction of the encoder itself or my Tak container (supports up to more then 100 GB). I hope, you got a somewhat senseful error message ("Wave file not supported") or did it simply crash?

Yes, the error in both instances was wave file not supported. The greater than 2gb 16/44 file is actually a 4 disc album that I'd spliced together because I'm weird like that. Not something too uncommon for people on forums like this though I'd wager.

I'm sure 4gb would be fine for most everybody but ideally the compressor should support any valid WAV file. If that is 400 tb then so be it.


Quote from: Gnerma on
* UI is slow to respond when using slow compression levels.

Well what seems to be happening is the UI is only getting updated when its absolutely necessary, or when the compressor finishes 1% of the operation. So if you're switching between windows you could end up getting a square with the dead image from the last window instead of the contents of the UI, until it updates anyway. I personally don't like this kind of behavior I suppose because it gives the user the impression that the program is unpolished. This is just me of course..

Stopping the process isn't a problem.

TAK - Alpha testing

Reply #18 –
Quote from: Gnerma on
* UI checks to see if file exists when "Test" is attempted.

Modified. Both test modes (encoding/decoding) now ignore the presence of the destination files.

Quote from: Gnerma on
Quote
The current implementation supports up to 24/96. But some parts of my code (file and wave io) are limited to file sizes up to 2 GB or exactly 2.147.483.647 Bytes. It's not a restriction of the encoder itself or my Tak container (supports up to more then 100 GB). I hope, you got a somewhat senseful error message ("Wave file not supported") or did it simply crash?

Yes, the error in both instances was wave file not supported. The greater than 2gb 16/44 file is actually a 4 disc album that I'd spliced together because I'm weird like that. Not something too uncommon for people on forums like this though I'd wager.

I'm sure 4gb would be fine for most everybody but ideally the compressor should support any valid WAV file. If that is 400 tb then so be it.

Fixed. Now up to 4 GB should be supported.

Breaking this limit will need considerable more work. I don't want to do it for the first public release. Sorry...

Quote from: Gnerma on
Quote from: Gnerma on

* UI is slow to respond when using slow compression levels.

Well what seems to be happening is the UI is only getting updated when its absolutely necessary, or when the compressor finishes 1% of the operation. So if you're switching between windows you could end up getting a square with the dead image from the last window instead of the contents of the UI, until it updates anyway. I personally don't like this kind of behavior I suppose because it gives the user the impression that the program is unpolished. This is just me of course..

Stopping the process isn't a problem.

Improved. Window updates should now be performed at least once every second. Well, this is not really exact, because the update latency currently depends on the system speed. But if it's fast enough on my good old P3-800, it should be ok for now.

Later i will implement an option to let you control the task priority, but not for the first public release.

Thank you for all your findings!

Thomas

TAK - Alpha testing

Reply #19 –
Quote from: TrNSZ on
point.  I think a slower, higher compressing mode would be appropriate as well, considering the current performance.

I agree. There are some optimizations possible for the current codec, but i wouln't expect more than an average improvement of about 0.3 percent.

Possibly i will work on a new codec similar to the current one but with more methods to improve the compression. I allready worked on it but there were some really complicated optimization problems which needed much evaluation, too much to perform it before this release.

But the file format is prepared for the addition of new codecs...

TAK - Alpha testing

Reply #20 –
I'll just bump in with a few questions although I'm not an alpha tester...

Wouldn't it be better to use a seperate thread for the encoding so that the UI isn't affected by it in any way? I think there are many problems when both tasks are coupled.

It seems that so far there's only a GUI version, so is a CLI encoder/decoder planned?

TAK - Alpha testing

Reply #21 –
Quote from: Fandango on
Wouldn't it be better to use a seperate thread for the encoding so that the UI isn't affected by it in any way? I think there are many problems when both tasks are coupled.

I am always a bit hesitant to use multi threading, because i don't have a multi core system and therefore am not able to check for thread synchronisation issues. If this changes, i will care for it and also built a multicore encoder (should be easy with TAK's design).

Quote from: Fandango on
It seems that so far there's only a GUI version, so is a CLI encoder/decoder planned?

There are TAK, the GUI version and TAKC, the command line version. But you are right, in the beginning there was only a GUI version.

TAK - Alpha testing

Reply #22 –
TAK vs The Rest of The World:
http://www.synthetic-soul.co.uk/comparison/lossless/

TAK 0.14 vs TAK 0.13:
http://www.synthetic-soul.co.uk/comparison/tak/

Sorry for the delay, I thought I better update to FLAC 1.1.3 and WavPack 4.40 at the same time, and those WavPack x modes take some time!  Unfortunately it looks like I screwed them up somehow, so I'll have to spend a couple of days running them again!  I have reverted back to 4.4a3 for now.

Once I've got decent WavPack 4.40 results, I will do some more timing tests with TAK, when verifying while encoding, as I think it will be interesting to compare to encoding with no verification.  After that I'll try to find some time for damage.
I'm on a horse.

TAK - Alpha testing

Reply #23 –
VERY nice.

I just encoded 58GB of music, and even preset 0 easily beats FLAC level 7.

Here's some compression results:
Code: [Select]
Input: 61994308200
TAK 4: 31826185295  48.66% savings
TAK 0: 32927983729  46.89%
FLAC7: 33929274512  45.27%


I also tried to trip up the codec with strange signals like adding DC offset, extreme clipping, odd sample rates, etc. No problems whatsoever. This is looking really good!

I'll see if I can do some more tests soon.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

TAK - Alpha testing

Reply #24 –
Quote from: Synthetic Soul on
Sorry for the delay,

Ok, for this time... 

Thank you!

Sorry for the trouble.
Quote from: Synthetic Soul on
Once I've got decent WavPack 4.40 results, I will do some more timing tests with TAK, when verifying while encoding, as I think it will be interesting to compare to encoding with no verification.

That's a very nice idea! I never really evaluated it.


Quote from: Omion on
I just encoded 58GB of music, and even preset 0 easily beats FLAC -7.

Here's some compression results:
Code: [Select]
Input: 61994308200
TAK 4: 31826185295  48.66% savings
TAK 0: 32927983729  46.89%
FLAC7: 33929274512  45.27%

Fine!

Although i have to admit, that i am not familar with FLAC -7.

Quote from: Omion on
I also tried to trip up the codec with strange signals like adding DC offset, extreme clipping, odd sample rates, etc. No problems whatsoever. This is looking really good!

That's really great! Thank you for such an exhaustive and creative testing!

This all really diminishes my fear to release a public version...
 
SimplePortal 1.0.0 RC1 © 2008-2019