HydrogenAudio

Lossless Audio Compression => Lossless / Other Codecs => Topic started by: TBeck on 2011-04-08 23:40:07

Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-08 23:40:07
Initially i thought, the next release would be 2.1.1.

I implemented

  - up to 10 percent faster encoding
  - up to 20 percent faster decoding
  - support for wave format extensible.

The latter was a preparation for multi channel audio support which i intended to implement in a later TAK version. But somehow i coudn't stop at this point. I was too curious to find out, what i could achieve. Now that i have a first working implementation, i also have a lot of questions.

I myself don't use any multi channel audio and my knowledge about it's practical application is very limited. More precise: I don't know, what kind of multi channel files files TAK users would like to compress. I have to know this to be able to tune TAK for the most likely application. It's easy to store the channels independently with TAK's codec, but i also want to exploit channel correlations to apply an extended version of TAK's joint stereo algorithms.

My evaluations seem to show, that the results depend to a high degree on the source material. Since i don't have the time to perform tunings for any possible format at once, i want to start with the formats users would like to compress.

What would they be?

Currently i am only testing 5.1 formats. Currently i am using the following files for tuning:

- HiRes-sample-files from 2L (http://www.2l.no/hires/index.html). Mostly 96 KHz converted from a DXD source.
- A 96 KHz version of Hotel California/ Eagles, DVD-A Audio.
- Several albums of CD-DTS-Audio (44 KHz) converted with foobar to wave. (Feeling a bit strange when converting lossy to lossless...)

I really have no idea, if those samples refelect by any means the possible applications of TAK users.

And therefore i have opened this thread. Please tell, what kind of multi channel audio you would like to compress with TAK.

I could have made a poll, but currently i simply don't know enough to ask the right questions.
Title: TAK 2.2.0 development
Post by: krafty on 2011-04-09 00:10:52
Hi Thomas, congrats for your codec!
I still use FLAC and some WavPack, however I'm always quick peeping on TAK.

All my music is Stereo. I even don't have mono files.

I am yet to see the point of multi-channel audio in music consumer market and I myself never bought any SACD or DVD-A. The movie market uses it a lot. The only thing I could think of using it was if I would encode a Blu-Ray using h.264 and 5.1 TAK audio. That would be the only scenario really, in which I would encode 5.1

IMO, multi-channel audio should not be a priority now. I see that you have improved even more TAK performance and that makes me want more and more a Linux version. Sorry if I bring this up again, I just did it because you're feeling unsure about multi-channel. Of course there are other users who will want multi-channel, so let's hear them too.
Title: TAK 2.2.0 development
Post by: sauvage78 on 2011-04-09 08:14:21
Hi TBeck,
I think almost the same as krafty, I still use Flac but I always have a look at Tak's thread because it is so impressive. Like him, I only use stereo right now but as he said, IMHO, the main use for 5.1 is to encode DTS-HD Master Audio (http://en.wikipedia.org/wiki/DTS-HD_Master_Audio) & Dolby TrueHD (http://en.wikipedia.org/wiki/Dolby_TrueHD) from blu-ray with eac3to (http://forum.doom9.org/showthread.php?t=125966) in matroska container.
The use for multichannel with music is IMHO a niche market for highly experimented users doing unusual rips.

Unlike krafty, I think multi-channel should be moderately high on your TODO list, because it opens TAK to the the video world which is a good thing because many skilled developers from Doom9 could have a look at your code & criticize your work.

EDIT: ... forgive me, I forgot TAK was closed source. [Sorry, I couldn't resist teasing Greynol & his colored warnings (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=86037&view=findpost&p=743682)  Just Kidding]

The DTS Master sample from Dances With Wolves that I personnaly use when I test 5.1 is 24bits per sample & 48Khz, but I think blu-ray supports many scenari, have a look at the DTS-HD Master Audio & Dolby TrueHD links that I posted above to have a clue of the variety, I have no clue which bit-depth/sample rate is the most widely used as I don't rip blu-ray regulary (I still have a Sempron 3000+ so I don't even try).

Have a look at this Blu-Ray audio codecs table (http://en.wikipedia.org/wiki/Blu-ray_Disc#Audio) for a summary.
Title: TAK 2.2.0 development
Post by: Nick.C on 2011-04-09 11:40:51
Thomas,

During the development of lossyWAV, I looked to accommodate the upper capabilities of the first lossless codec to which the method was applicable, namely FLAC. This resulted in 8 channel support and up to 24 bit integer support. I limited the samplerate support to 384kHz due to memory considerations and availability of samples - although this can be changed relatively simply.

As there are 7.1 setups out in the wild, I would suggest 8 channel support.

Have fun in the development!

Nick.
Title: TAK 2.2.0 development
Post by: Destroid on 2011-04-09 21:34:34
I would guess a majority of interest in multichannel lossless would be in current and next-generation movie discs that use lossless formats. DTS could also be of interest in "old" discs like CD and DVD. I, too, feel the creepy weirdness of a lossy format handled by a lossless compressor, but then I think of it as "data preserved in its entirety and accuracy without a widely available lossless source," then it seems slightly less spooky. Or not?

Seems DVD-A is largely ignored and I personally don't know of that many collectors. Does this standard implement loudness standards similar to the standard set by the motion picture industry? If not, then IMO the format seems even more worthless. Myself (and probably many readers here) would be in favor of loudness standard in the next-gen music format. As it turned out, seems MP3/AAC became the masses' format of choice and not DVD-A. But I digress, I just have the opinion DVD-A may not be as high interest for this multichannel endeavor.

I'm out of my league though, most of the material I compress losslessly are mono tracks. Also I have no experience with extracting multichannel audio from disc. I imagine there are some music vendors to download from but are there enough to give TAK a full variety of test material?

Will be happy to see the speed-ups, though.
Title: TAK 2.2.0 development
Post by: kritip on 2011-04-09 21:42:19
I personally don't see much point for a multichannel lossless encoder. If I am using multichannel for film copies etc, I will either keep the AC3 or DTS track if I am going for quality. If size is my concern, It'll be compressed with a lossy encoder such as AAC tp get the BR right down. The middle ground seems of little use to me.

Each to their own though  but thats my view.
Title: TAK 2.2.0 development
Post by: stereotype on 2011-04-09 23:32:51
Hi Thomas

First of all, congratulations on your amazing work.
I am not a TAK user yet, but I do follow it's development here with a lot of interest...

Regarding your question, perhaps one interesting use for multichannel audio would be as an intermediate/storage codec in videography, seeing that many cameras capture 5.1 audio, and the correlation between the channels would be very high...

By the way, can I take this opportunity to ask you a couple of things about TAK?

1. How suitable is TAK to be implemented in hardware?

2. What error correction / checksum features are implemented in TAK's file format?
(FLAC, if I'm not mistaken, checksums every frame and does an MD5 for the whole stream. Does TAK do something similar?)
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-10 15:13:20
IMO, multi-channel audio should not be a priority now. I see that you have improved even more TAK performance and that makes me want more and more a Linux version. Sorry if I bring this up again, I just did it because you're feeling unsure about multi-channel. Of course there are other users who will want multi-channel, so let's hear them too.

To me it's perfectly ok if someone asks for linux suport. This is likely to have an influence on my priority list. 

What should be avoided are discussions about linux support and going open source, because such discussions got quite nasty in the past. Furthermore they would be fruitless because i have deceided not to talk about those topics unless i am already working on those features.

Unlike krafty, I think multi-channel should be moderately high on your TODO list, because it opens TAK to the the video world which is a good thing because many skilled developers from Doom9 could have a look at your code & criticize your work.

EDIT: ... forgive me, I forgot TAK was closed source. [Sorry, I couldn't resist teasing Greynol & his colored warnings (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=86037&view=findpost&p=743682)  Just Kidding]

Thank you for reminding me...  And also for the links.

During the development of lossyWAV, I looked to accommodate the upper capabilities of the first lossless codec to which the method was applicable, namely FLAC. This resulted in 8 channel support and up to 24 bit integer support. I limited the samplerate support to 384kHz due to memory considerations and availability of samples - although this can be changed relatively simply.

As there are 7.1 setups out in the wild, I would suggest 8 channel support.

Have fun in the development!

Thank you! 

TAK's stream format now supports up to 16 channels. Initially it was designed for 32 channels, but i think that would be overkill. Ok, while looking for info about multichannel audio i read over an article about an 22 channel format specification, but i doubt, many people would install so many speakers.

16 channels is nice, because it provides some headroom for moderate extensions of the existing formats (7.1).

While the stream supports up to 16 channels, the first encoder version may be limited to 6 channels because i tend to only unlock support for formats the codec already is tuned for.

I would guess a majority of interest in multichannel lossless would be in current and next-generation movie discs that use lossless formats. DTS could also be of interest in "old" discs like CD and DVD. I, too, feel the creepy weirdness of a lossy format handled by a lossless compressor, but then I think of it as "data preserved in its entirety and accuracy without a widely available lossless source," then it seems slightly less spooky. Or not?

Good point!

I'm out of my league though, most of the material I compress losslessly are mono tracks. Also I have no experience with extracting multichannel audio from disc. I imagine there are some music vendors to download from but are there enough to give TAK a full variety of test material?

That's the problem and possibly also part of the answer...

I personally don't see much point for a multichannel lossless encoder. If I am using multichannel for film copies etc, I will either keep the AC3 or DTS track if I am going for quality. If size is my concern, It'll be compressed with a lossy encoder such as AAC tp get the BR right down. The middle ground seems of little use to me.

Each to their own though  but thats my view.

Perfectly understandable.

1. How suitable is TAK to be implemented in hardware?

TAK's decoder complexity is basically quite similar to FLAC's, as long as it isn't using more predictors for the linear prediction filter. The predictor count depends on the preset:

-p0x: 4 predictors. This is less than FLAC -3 is using (6).
-p1x: 12 predictors. This is the same as for FLAC -8.
-p2x: 32 predictors. The maximum FLAC supports, but outside of it's restricted subset. To my knowledge the higher presets of FLAKE are using up to 32 predictors.
-p3x: 80 predictors.
-p4x: 160 predictors.

It doesn't make a difference, if you use TAK's higher evaluation levels like -p1e or -p1m. They will only affect the encoding speed and compression ratio, but not the decoding complexity.

Since TAK is using a reduced precision for it's predictor filter, it may be implemented more efficently than FLAC on some hardware platforms, if optimized code is beeing generated. That's especially true for 24 bit audio.

2. What error correction / checksum features are implemented in TAK's file format?
(FLAC, if I'm not mistaken, checksums every frame and does an MD5 for the whole stream. Does TAK do something similar?)

Frame header checksum:  TAK: 24 bits, FLAC: 8 bits.
Frame data checksum:  TAK: 24 bits, FLAC: 16 bits.
Meta data item checksum: TAK: 24 bits, FLAC: i don't know.

MD5 of the raw audio data: TAK: has to be enabled by -md5, FLAC: is enabled by default.
Title: TAK 2.2.0 development
Post by: stereotype on 2011-04-10 16:12:05
Thanks for the info
With all your constant optimizations, how x86-specific is the code, if I may ask...

About the -md5 switch:
Is the MD5 info always written on encode, but only checked with -md5 on decode?
Or is it only written on encode with -md5?
And if the MD5 info is present in the file, is it automatically checked, or does it need -md5?
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-04-10 16:28:38
Initially i thought, the next release would be 2.1.1.

I implemented

  - up to 10 percent faster encoding
  - up to 20 percent faster decoding
  - support for wave format extensible.

The  latter was a preparation for multi channel audio support which i  intended to implement in a later TAK version. But somehow i coudn't  stop at this point. I was too curious to find out, what i could  achieve. Now that i have a first working implementation, i also have a  lot of questions.

I myself don't use any multi channel audio and  my knowledge about it's practical application is very limited. More  precise: I don't know, what kind of multi channel files files TAK users  would like to compress. I have to know this to be able to tune TAK for  the most likely application. It's easy to store the channels  independently with TAK's codec, but i also want to exploit channel  correlations to apply an extended version of TAK's joint stereo  algorithms.

My evaluations seem to show, that the results depend  to a high degree on the source material. Since i don't have the time to  perform tunings for any possible format at once, i want to start with  the formats users would like to compress.

What would they be?

Currently i am only testing 5.1 formats. Currently i am using the following files for tuning:

- HiRes-sample-files from 2L (http://www.2l.no/hires/index.html). Mostly 96 KHz converted from a DXD source.
- A 96 KHz version of Hotel California/ Eagles, DVD-A Audio.
  - Several albums of CD-DTS-Audio (44 KHz) converted with foobar to  wave. (Feeling a bit strange when converting lossy to lossless...)

I really have no idea, if those samples refelect by any means the possible applications of TAK users.

And therefore i have opened this thread. Please tell, what kind of multi channel audio you would like to compress with TAK.

I could have made a poll, but currently i simply don't know enough to ask the right questions.


Sounds great.
I have a couple of DVD-As, mostly 24-96, but I remember there was at least one 24-48.
Also, I got some 24-96 Quad rips.
IMO, multi-channel  audio should not be a priority now. I see that you have improved even  more TAK performance and that makes me want more and more a Linux  version. Sorry if I bring this up again, I just did it because you're  feeling unsure about multi-channel. Of course there are other users who  will want multi-channel, so let's hear them too.

To me it's perfectly ok if someone asks for linux suport. This is likely to have an influence on my priority list.  (http://www.hydrogenaudio.org/forums/style_emoticons/default/smile.gif)

Good.
Because, while I have personal interest in multichannel compressor that's better than FLAC, I actually agree with krafty that it's more important for TAK right now.
Title: TAK 2.2.0 development
Post by: CoRoNe on 2011-04-10 17:07:23
Hello Thomas,

On the Hydrogenaudio KB (http://wiki.hydrogenaudio.org/index.php?title=TAK#Future_Features) I can see cue sheets and cover art are already on your to-do list, but how much of a priority are they?
Currently I'm using WavPack (-hhx -w "Cuesheet=@*.cue" --write-binary-tag "Cover Art (Front)=@*.png"). I've already been experimenting a little with TAK and of course I can add cue sheets and cover art afterwards with Mp3tag because of the APEtag, but once TAK supports it natively I'm really thinking of switching over.
Keep up the good work!
Title: TAK 2.2.0 development
Post by: stereotype on 2011-04-10 17:19:29
TAK's stream format now supports up to 16 channels. Initially it was designed for 32 channels, but i think that would be overkill. Ok, while looking for info about multichannel audio i read over an article about an 22 channel format specification, but i doubt, many people would install so many speakers.

16 channels is nice, because it provides some headroom for moderate extensions of the existing formats (7.1).

While the stream supports up to 16 channels, the first encoder version may be limited to 6 channels because i tend to only unlock support for formats the codec already is tuned for.

I don't think 32 channels is overkill.
Maybe you are focusing too much on end-user media consumption.
TAK could also be VERY useful in intermediate media production and EXTREMELY useful in final storage.

For example in music production, where all the instruments have their own separate channels.
In a stereo setup, every instrument would take up two channels.
With 16 channels, you'd be limited to 8 voices/instruments...

Or like the example i gave before about the 5.1 audio in videography.
Videography is one of my hobbies, and the subject of intermediate codecs is still a hot debated topic.
There is hardly a good defacto intermediate lossless codec for video that works accross all workflows, and for audio, everyone just uses WAV.
What I mean is, while there are solutions that work, there is still a lot of room for improvement, and the players are not yet set...

Perhaps you shouldn't limit yourself to say X channels, but apply compression to groups of channels?
Like in that music production example, you could apply compression to every pair of stereo channels - since I don't think there would be much point trying to find correlation between voice and drums for example
Or in the 5.1 example, you could compress the 5 channels as a group, and the .1 as another group.
That way you don't have to limit yourself to say X channels, and maybe you could even support 65536 channels as WAV does
65536 of course is overkill, but why not 128 or 1024?

I found the following quote from FLAC's website:

FLAC is designed as a consumer audio format. It trades ease of editing for a featureful, robust transport layer more suited for playback, and encoding speed for more compression and faster decompression.

Maybe TAK can differentiate itself as a more professional audio format? Just some food for thought

By the way, what's the maximum file size TAK supports?
Title: TAK 2.2.0 development
Post by: lvqcl on 2011-04-10 17:52:09
Professional audio format that no professional programs/devices support?
Title: TAK 2.2.0 development
Post by: GeSomeone on 2011-04-10 23:36:32
Thomas, if your looking voor common multi channel configuration (for consumer playback), I'll give it a try.
All combinations from 2 to 8 channels are possible.
Most common are 5.1 (surround sound like on DVD-A) and 4.0 (Quadraphonic).

Then there is 5.0 (no separate Low Frequenty channel) that is sometimes used for surround music (not movie related). Some programs/formats confuse this and the rarer 4.1 (quadraphonic with (derived) sub channel) as the number of channels is the same.
Then there is 7.1 that can be configured is multiple configurations (normally with sides and surround backs, but nowadays one of those pairs can be used for height or front wide alternatively).
There do exist some 3 channel SACDs, where the 3 original studio tracks each get their own channel (L-C-R).

bit depth and sample rate are the same range as used in stereo.

Maybe this helps to get a bit of the picture.
Title: TAK 2.2.0 development
Post by: stereotype on 2011-04-11 19:26:52
Professional audio format that no professional programs/devices support?

A journey of a thousand miles begins with a single step
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-25 02:04:28
I have made great progress in the last two weeks. Therefore i am now able to present some first compression results for multi channel audio.

It wasn't easy to obtain appropriate test files for the codec tuning. And since multichannel audio files are quite big, each test run took a lot of time. That's quite annoying if you know you have to repeat this procedure for many other codec variations you want to try. It remined me of my very first attempts in lossless audio compression when i owned a i486...

Here comes the comparison:

Code: [Select]
           TAK             FLAC    WavPack 
Mode       -p2m    -p4m    -8      -hhx
-------------------------------------------------------------------------------
DXD (mostly): 24 bit, 96 khz, 6 channels (5.1)          
-------------------------------------------------------------------------------
m6_96_a    41.69   41.40   43.96   44.11    Samples from 2L - the Nordic Sound    
-------------------------------------------------------------------------------
DVD-A (PCM): 24 bit, 96 khz, 6 channels (5.1)          
-------------------------------------------------------------------------------
m6_96_b    53.77   53.61   55.85   56.56    Eagles - Hotel California
-------------------------------------------------------------------------------          
DTS: 16 bit, 44 khz, 6 channels (5.1)
-------------------------------------------------------------------------------          
m6_44_a    30.22   29.38   33.17   32.42    Al Di Meola - Consequence Of Chaos
m6_44_b    41.69   41.41   45.04   44.35    Pink Floyd - The Dark Side Of The Moon
m6_44_c    41.28   41.06   42.69   42.18    Roxy Music - Avalon
m6_44_d    40.18   39.76   43.23   42.09    The Police - Every Breath You Take
-------------------------------------------------------------------------------
Vinyl-Rip: 24 bit, 96 khz, 4 channels (quadro), denoised and decklicked          
-------------------------------------------------------------------------------          
m4_a       65.46   65.36   67.19   66.84    Pink Floyd - Wish you were here
m4_b       62.30   62.22   63.94   63.42    Santana - Caravanserai

Compression ratio expressed as percentage of the original file size. Each data row represents a complete album or a collection of files.

I couldn't test Monkey's Audio or Optimfrog, because both seem to have no support for multichannel audio. I didn't try a stronger mode for WavPack, beacuse this would have taken too long (4-5 hours were enough...).

I have tested only TAK -p2m and -p4m, because all other presets still have to be constructed (from the many internal encoder options) for optimum performance, what will require a lot of further testing.

I have some more ideas to further improve the compression. Some of them already have been tried and seem to work well. But this is getting very complex and time consuming. I have deceided, to stop the evaluation for now and to start to work on a release. Ok, some minor tuning might take place...

I am quite happy with the current results.

Maybe this helps to get a bit of the picture.

This was very helpful! Thank you.
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-04-25 11:49:45
Could you test it against flake and (especially) als?
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-25 16:15:16
Could you test it against flake and (especially) als?

No, at least not now, because

a) The Mpeg4Als binary is not well documented. I would have to look at the source code to determine the optimum parameters for multi-channel-decorrelation. Well, i did, but it's not a matter of just 5 or 10 minutes and currently i don't have more time. Furthermore the encoder was really slow when i tested it in the past and the addition of multi-channel-decorrelation will make things even worse.

b) The  latest official windows binary of Flake seems to be 0.11. Given the later development reported here at hydrogen, i would call it rather outdated. And when i tested 0.11 in the past, the results were quite variable, not seldom worse than with FLAC. The results reported on it's home page seemed to indicate something else, but i couldn't reproduce them on my test corpus and some users at hydrogen reported the same. I think, 0.11 had been tuned with files from lossy sources. That because 1.) i have downloaded the test files which were avaiable in the past and they seemed to be low-passed, and 2.) because i could reproduce it's quite good results compared with TAK 1.0.4 only for lossy processed sources. This does make sense, because TAK 1.0.4 had a severe weakness when compressing such files. I am sure, later development of Flake has removed this tuning-bias against lossy sources.

c) If i am right, later FLAC versions have adopted some of Flake's improvements (the most prominent exception might be variable block size switching), therefore i wouldn't expect to see large differences.

Out of interest i additionally tested "FLAC -8 -L 32 --lax" on some of my test files. That means "-8', but with 32 (maximum possible) instead of 12 predictors. Because this is outside of FLAC's restricted subset, you have to add "--lax".

Code: [Select]
           TAK             FLAC    FLAC    WavPack 
Mode       -p2m    -p4m    -8      -8 -l32 -hhx
---------------------------------------------------------------------------------------
DXD (mostly): 24 bit, 96 khz, 6 channels (5.1)          
---------------------------------------------------------------------------------------
m6_96_a    41.69   41.40   43.96   --.--   44.11    Samples from 2L - the Nordic Sound    
---------------------------------------------------------------------------------------
DVD-A (PCM): 24 bit, 96 khz, 6 channels (5.1)          
---------------------------------------------------------------------------------------
m6_96_b    53.77   53.61   55.85   --.--   56.56    Eagles - Hotel California
---------------------------------------------------------------------------------------          
DTS: 16 bit, 44 khz, 6 channels (5.1)
---------------------------------------------------------------------------------------          
m6_44_a    30.22   29.38   33.17   33.01   32.42    Al Di Meola - Consequence Of Chaos
m6_44_b    41.69   41.41   45.04   45.00   44.35    Pink Floyd - The Dark Side Of The Moon
m6_44_c    41.28   41.06   42.69   42.66   42.18    Roxy Music - Avalon
m6_44_d    40.18   39.76   43.23   43.16   42.09    The Police - Every Breath You Take
---------------------------------------------------------------------------------------
Vinyl-Rip: 24 bit, 96 khz, 4 channels (quadro), denoised and decklicked          
---------------------------------------------------------------------------------------          
m4_a       65.46   65.36   67.19   67.19   66.84    Pink Floyd - Wish you were here
m4_b       62.30   62.22   63.94   63.93   63.42    Santana - Caravanserai
---------------------------------------------------------------------------------------


edit: Somehow messed up the table...
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-04-25 16:27:26
Could you test it against flake and (especially) als?

No, at least not now, because

a) The Mpeg4Als binary is not well documented. I would have to look at the source code to determine the optimum parameters for multi-channel-decorrelation. Well, i did, but it's not a matter of just 5 or 10 minutes and currently i don't have more time. Furthermore the encoder was really slow when i tested it in the past and the addition of multi-channel-decorrelation will make things even worse.

b) The  latest official windows binary of Flake seems to be 0.11. Given the later development reported here at hydrogen, i would call it rather outdated. And when i tested 0.11 in the past, the results were quite variable, not seldom worse than with FLAC. The results reported on it's home page seemed to indicate something else, but i couldn't reproduce them on my test corpus and some users at hydrogen reported the same. I think, 0.11 had been tuned with files from lossy sources. That because 1.) i have downloaded the test files which were avaiable in the past and they seemed to be low-passed, and 2.) because i could reproduce it's quite good results compared with TAK 1.0.4 only for lossy processed sources. This does make sense, because TAK 1.0.4 had a severe weakness when compressing such files. I am sure, later development of Flake has removed this tuning-bias against lossy sources.

c) If i am right, later FLAC versions have adopted some of Flake's improvements (the most prominent exception might be variable block size switching), therefore i wouldn't expect to see large differences.

Out of interest i additionally tested "FLAC -8 -L 32 --lax" on some of my test files. That means "-8', but with 32 (maximum possible) instead of 12 predictors. Because this is outside of FLAC's restricted subset, you have to add "--lax".

Code: [Select]
           TAK             FLAC    FLAC    WavPack 
Mode       -p2m    -p4m    -8      -8 -l32 -hhx
---------------------------------------------------------------------------------------
DXD (mostly): 24 bit, 96 khz, 6 channels (5.1)          
---------------------------------------------------------------------------------------
m6_96_a    41.69   41.40   43.96   --.--   44.11    Samples from 2L - the Nordic Sound    
---------------------------------------------------------------------------------------
DVD-A (PCM): 24 bit, 96 khz, 6 channels (5.1)          
---------------------------------------------------------------------------------------
m6_96_b    53.77   53.61   55.85   --.--   56.56    Eagles - Hotel California
---------------------------------------------------------------------------------------          
DTS: 16 bit, 44 khz, 6 channels (5.1)
---------------------------------------------------------------------------------------          
m6_44_a    30.22   29.38   33.17   33.01   32.42    Al Di Meola - Consequence Of Chaos
m6_44_b    41.69   41.41   45.04   45.00   44.35    Pink Floyd - The Dark Side Of The Moon
m6_44_c    41.28   41.06   42.69   42.66   42.18    Roxy Music - Avalon
m6_44_d    40.18   39.76   43.23   43.16   42.09    The Police - Every Breath You Take
---------------------------------------------------------------------------------------
Vinyl-Rip: 24 bit, 96 khz, 4 channels (quadro), denoised and decklicked          
---------------------------------------------------------------------------------------          
m4_a       65.46   65.36   67.19   67.19   66.84    Pink Floyd - Wish you were here
m4_b       62.30   62.22   63.94   63.93   63.42    Santana - Caravanserai
---------------------------------------------------------------------------------------


edit: Somehow messed up the table...


Some time ago I did multichannel tests of flake svn and it was significantly better than both flac and wavpack. Honestly, I can't tell how much was it though.
ADDED:
And AFAIK als is the multichannel leader. What's the point of comparing to laggards?
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-25 16:32:41
Some time ago I did multichannel tests of flake svn and it was significantly better than both flac and wavpack. Honestly, I can't tell how much was it though.

If you send me a binary, i will test it...
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-25 16:36:15
And AFAIK als is the multichannel leader. What's the point of comparing to laggards?

References? Suggestions for command line parameters? Time to help me out by testing it yourself with the sample files from 2L i used?
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-04-25 16:43:33
And AFAIK als is the multichannel leader. What's the point of comparing to laggards?

References? Suggestions for command line parameters? Time to help me out by testing it yourself with the sample files from 2L i used?

For now, I can only point you to this (http://forum.doom9.org/showthread.php?t=104744) old thread.
Maybe tomorrow, when I'm back home I'll do some quick tests.
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-25 17:02:48
For now, I can only point you to this (http://forum.doom9.org/showthread.php?t=104744) old thread.
Maybe tomorrow, when I'm back home I'll do some quick tests.

I know this thread. Unfortunately it doesn't help with the multi-channel-options. I think, MCC (Multi-Channel Coding) has been implemented later.

It would be helpful, if you would evaluate the best options for multi-channel-audio. Then i could use them for my own test. Well, as soon as i have some more time to work on it.

Just to be sure: You know, that mpeg4als includes both an asymmetric and a symmetric codec? The latter is selected by the -z option.

In the past i only tested the asymmetric codec, because the other was too slow.

BTW: This wasn't meant as a comprehensive comparison, which probably will have to wait until the release of V2.2.0.
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-04-25 17:29:29
For now, I can only point you to this (http://forum.doom9.org/showthread.php?t=104744) old thread.
Maybe tomorrow, when I'm back home I'll do some quick tests.

I know this thread. Unfortunately it doesn't help with the multi-channel-options. I think, MCC (Multi-Channel Coding) has been implemented later.

It would be helpful, if you would evaluate the best options for multi-channel-audio. Then i could use them for my own test. Well, as soon as i have some more time to work on it.

Just to be sure: You know, that mpeg4als includes both an asymmetric and a symmetric codec? The latter is selected by the -z option.

In the past i only tested the asymmetric codec, because the other was too slow.

BTW: This wasn't meant as a comprehensive comparison, which probably will have to wait until the release of V2.2.0.

I found some multichannel Floyds on HDD, I'm doing some quick and dirty testing. Sadly, from preliminary results I already know that the track is not a representative one. I may do another one today (same album though) and maybe another later.
I'd like to do a serious multichannel audio test, but probably not before Christmas, as I will be picking a multichannel encoder when I'm done with compressing my stereo audio to OptimFROG.

And as to multichannel, there's only one switch about it unless you want to rearrange channels -s#. From my memory I can tell that the optimum value is either 2 or 3 for 5.1 audio, varying from track to track, IIRC 2 wins most often.
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-04-25 18:41:30
So here it is, results on some arbitrary file:
Pigs on the wing, 5.1, said to come from a SQ tape, though the fact that it's 5.1 instead of 4.0 makes me suspicious.
Code: [Select]
uncompressed              147571268
flac -8                    65494985
wavpack -hhx6              54864294
als -a -e -p -g5 -s2 -o60  49472082
als -a -e -p -g5 -s1 -o100 49969480
als -a -e -p -g5 -s2 -o100 49311074
als -a -e -p -g5 -s3 -o100 49062431
als -a -e -p -g5 -s6 -o100 48774241
als -z3                    48744061

I chose o100 because I remembered that I liked it.
I'm trying a different file now.
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-04-25 19:51:48
I cut 2 samples from another file. Now I know why I liked o100.
Code: [Select]
uncompressed                    17280080
flac -8                          9544125
wavpack -hhx6                    7571878
als -a -e -p -g5 -s1 -o60        7059115
als -a -e -p -g5 -s2 -o60        7027939
als -a -e -p -g5 -s3 -o60        6958482
als -a -e -p -g5 -s6 -o60        6937060
als -a -e -p -g5 -s6 -o100       6933432
als -a -e -p -g5 -s6 -o200       6934788
als -a -e -p -g5 -s6 -o400       6934958
als -a -e -p -g5 -s6 -o800       6934920
als -b -a -e -p -g5 -s6 -o100    6868655
als -l -b -a -e -p -g5 -s6 -o100 6868655
als -7 -o100 -s6                 6865530
als -z3                          7118789

Code: [Select]
uncompressed                     34560080
flac -8                          16740526
wavpack -hhx6                    13449682
als -a -e -p -g5 -s1 -o60        11882158
als -a -e -p -g5 -s2 -o60        11814740
als -a -e -p -g5 -s3 -o60        11727356
als -a -e -p -g5 -s6 -o60        11672213
als -a -e -p -g5 -s6 -o100       11657250
als -a -e -p -g5 -s6 -o200       11669488
als -a -e -p -g5 -s6 -o400       11675959
als -a -e -p -g5 -s6 -o800       11675990
als -b -a -e -p -g5 -s6 -o100    11505858
als -l -b -a -e -p -g5 -s6 -o100 11505858
als -7 -o100 -s6                 11525750
als -z3                          11740206
 
EDIT1: updated with -7
EDIT2: 2nd and last update, these files are not worth digging more.
EDIT3: Typo.
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-25 19:53:22
als -a -e -p -g5 -s6 -o100

I don't think this is optimal, nevertheless i am just testing it. This may take some hours.

So far 4 files have been processed, each of them worse than with TAK -p4m.
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-04-25 20:00:33
als -a -e -p -g5 -s6 -o100

I don't think this is optimal, nevertheless i am just testing it. This may take some hours.

So far 4 files have been processed, each of them worse than with TAK -p4m.



I updated the topic before I saw your answer, it's indeed suboptimal.
And I suggest trying -s2 and -s3. I think -s6 is specific to this album, though I have only memory to back it up.
ADDED: But it's nice to see that you beat it. In general, I like TAK more than ALS and anyway - progress is good.
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-26 04:12:50
I updated the topic before I saw your answer, it's indeed suboptimal.
And I suggest trying -s2 and -s3. I think -s6 is specific to this album, though I have only memory to back it up.

The addition of -b is important. Currently i am testing -7 as base line. That's really slow! I started the test 8 hours ago... The strongest (and slowest mode) of TAK does the work in a couple of minutes.

Because of the slowness i can't test all parameter variations on all files. I have selected 3 files with good oppurtunities for channel decorrelation for an iterative test. Now my secondary PC is busy too...

First Results: -7 -s2 gave me significantly worse results than -7 alone. -7 -t2 is just running and it seems to be advantegous. Therefore i will evaluate -t a bit more.

When i have determined the optimum t-parameter for the 3 files, i will try it with the whole test corpus. Well, i really dislike tests with such limited file sets, but because of the slowness anything else isn't practicable. At least not now.

For the final test i probably will reduce the maximum predictor count to 160 (-o 160). The default for -7 is 1023 for 44 Khz samples! 160 is an interesting choice, because it's also the maximum TAK is using.

ADDED: But it's nice to see that you beat it. In general, I like TAK more than ALS and anyway - progress is good.

-7 compresses some file sets better than the current version of TAK. This get's interesting.
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-26 07:43:47
First Results: -7 -s2 gave me significantly worse results than -7 alone. -7 -t2 is just running and it seems to be advantegous. Therefore i will evaluate -t a bit more.

When i have determined the optimum t-parameter for the 3 files, i will try it with the whole test corpus. Well, i really dislike tests with such limited file sets, but because of the slowness anything else isn't practicable. At least not now.

Done. Well, with only 3 files from my DTS-file set m6_44_b:

Code: [Select]
              ALS -7
Add           none    -s2     -t2     -t3     -t6
---------------------------------------------------
DTS: 16 bit, 44 khz, 6 channels (5.1)
---------------------------------------------------          
m6_44_b_02    45.28   45.51   45.24   45.19   45.15
m6_44_b_03    41.33   41.87   41.32   41.23   40.81
m6_44_b_04    45.12   45.25   45.08   45.04   44.95
---------------------------------------------------

-7 -t6 was the best. I just started a respective test with my complete DTS-set. Well, this will take another 10 hours or more...

But the results of the first run of Mpeg4Als are now ready:

Code: [Select]
           TAK             FLAC    FLAC    WavPack ALS
Mode       -p2m    -p4m    -8      -8 -l32 -hhx    -7      -7 -t6
-----------------------------------------------------------------          
DTS: 16 bit, 44 khz, 6 channels (5.1)
-----------------------------------------------------------------          
m6_44_a    30.22   29.38   33.17   33.01   32.42   28.67    
m6_44_b    41.69   41.41   45.04   45.00   44.35   42.21
m6_44_c    41.28   41.06   42.69   42.66   42.18   40.60
m6_44_d    40.18   39.76   43.23   43.16   42.09   40.45
-----------------------------------------------------------------

TAK -p4m is better on set b and d, ALS on set a and c.

Sets b and d are those, where TAK's channel decorrelation is most efficient. The following tables show it's advantage for the 4 file sets:

Code: [Select]
-------------------------------------------
DTS: 16 bit, 44 khz, 6 channels (5.1)
-------------------------------------------
m6_44_a    0.34    
m6_44_b    1.96
m6_44_c    0.28
m6_44_d    1.48
-------------------------------------------

ALS's better performance on a and c may be attributed to it's high predictor count of 1023 vs. TAK's 160. A quick test of 2 files with -7 -o160 resulted in a significant decrease of ALS's advantage.

BTW: I used mp4alsRM23 for the tests.
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-27 01:02:25
-7 -t6 was the best. I just started a respective test with my complete DTS-set. Well, this will take another 10 hours or more...

No, it wasn't so fast...

The last column now contains the results for Mpeg4Als -7 -t6:

Code: [Select]
           TAK             FLAC    FLAC    WavPack ALS
Mode       -p2m    -p4m    -8      -8 -l32 -hhx    -7      -7 -t6
-----------------------------------------------------------------          
DTS: 16 bit, 44 khz, 6 channels (5.1)
-----------------------------------------------------------------          
m6_44_a    30.22   29.38   33.17   33.01   32.42   28.67   28.65
m6_44_b    41.69   41.41   45.04   45.00   44.35   42.21   41.98
m6_44_c    41.28   41.06   42.69   42.66   42.18   40.60   40.57
m6_44_d    40.18   39.76   43.23   43.16   42.09   40.45   40.13
-----------------------------------------------------------------
Title: TAK 2.2.0 development
Post by: Chaser on 2011-04-27 13:17:11
Your findings are very interesting. I wish you good look with your further tuning, though I am uncertain whether spending A LOT of time on 0.5% is worth the hassle before "constructing a sound basement". I hope you understand, what I intend to stay.

Nevertheless I am impressed how fast you implemented a multi-channel encoder that competes that well in such a short time.
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-04-27 15:57:54
Like I said, I did a more extensive test of various als settings. Took almost 2 days and 1000 file-settings pairs. It happened so fast, because I cut short samples with the intention of maximizing variance.
You can download full summary here (http://www.hydrogenaudio.org/forums/index.php?showtopic=88312).
In short, "-7 -t6" seems best, though I found a case where "-z3 -t6" destroyed it...it's very strange because on all other files -z3 is a clear looser.
Title: TAK 2.2.0 development
Post by: TBeck on 2011-04-27 19:26:26
Your findings are very interesting. I wish you good look with your further tuning, though I am uncertain whether spending A LOT of time on 0.5% is worth the hassle before "constructing a sound basement". I hope you understand, what I intend to stay.

I think i do.

Nevertheless I am impressed how fast you implemented a multi-channel encoder that competes that well in such a short time.

The multi-channel-codec adds nothing really new, instead it uses the existing channel decorrelation filters i have already heavily tuned for stereo audio.

I know there exist more complex decorrelation approaches for multi channel audio, but i doubt, they are as fast as TAK's filters. Furthermore i want to avoid potentially patented technology. And i like to find simple but efficient solutions for potentially complex problems.

Like I said, I did a more extensive test of various als settings. Took almost 2 days and 1000 file-settings pairs. It happened so fast, because I cut short samples with the intention of maximizing variance.
You can download full summary here (http://www.hydrogenaudio.org/forums/index.php?showtopic=88312).
In short, "-7 -t6" seems best, though I found a case where "-z3 -t6" destroyed it...it's very strange because on all other files -z3 is a clear looser.

Thank you! Then it's probably ok to limit my evaluation to -7 -t6.

I am just exploring some tunings. The first of them improves the compression by about 0.10 to 0.15 percent for some of my file sets.

In the meantime another ALS test is finished. Now i am using -7 -t4 (the maximum for 4 channels) and -o160. The latter limits the predictor count of the primary filter to 160 predictors as TAK is using. This does make sense for my evaluations, because i don't want to explore the effect of higher predictor orders but the efficiency of my ny channel decorrelation.

Code: [Select]
           TAK             FLAC    FLAC    WavPack ALS
Mode       -p2m    -p4m    -8      -8 -l32 -hhx    -7 -t4 -o160
------------------------------------------------------------------------------------------
Vinyl-Rip: 24 bit, 96 khz, 4 channels (quadro), denoised and decklicked          
------------------------------------------------------------------------------------------          
m4_a       65.46   65.36   67.19   67.19   66.84   65.43   Pink Floyd - Wish you were here
m4_b       62.30   62.22   63.94   63.93   63.42   62.10   Santana - Caravanserai
------------------------------------------------------------------------------------------
         
Title: TAK 2.2.0 development
Post by: TBeck on 2011-05-04 12:50:15
I have got some more multichannel albums. Since it isn't practicable to use a large corpus for the evaluation of the new codec, i had to select an adequate files subset. I performed a lot of tests on all files to determine file properties that are relevant for the efficiency of the various compression algorithms. Then i selected between 2 and 3 files from each album which were in this respect representative for the whole album. This took two days but it will pay off because it will save me a lot of time in the future (when improvingthe the codec i will have to test the files over and over again).

As a side effect i can now present a new comparison:

Code: [Select]
           TAK             FLAC    FLAC    WavPack ALS
Mode       -p2m    -p4m    -8      -8 -l32 -hhx    -7 -o160 -t4/6
-----------------------------------------------------------------          
mc_4_dts   52.95   52.03   56.42   55.89   55.30   52.11
mc_4_dva   64.06   63.98   65.93   65.88   65.10   63.92
mc_6_dts   39.46   39.04   42.28   42.21   41.41   39.24
mc_6_dva   54.52   54.41   56.74   56.72   57.40   54.74
-----------------------------------------------------------------

For the file sets:

mc_4_dts = DTS:  16 bit, 44 khz, 4 channels (quadrophonic), 15 files
mc_4_dva = DVD-A: 24 bit, 96 khz, 4 channels (quadrophonic), 15 files
mc_6_dts = DTS:  16 bit, 44 khz, 6 channels (5.1 surround), 12 files
mc_6_dva = DVD-A: 24 bit, 96 khz, 6 channels (5.1 surround),  3 files
Title: TAK 2.2.0 development
Post by: Destroid on 2011-05-05 00:33:17
Good gracious! ALS is such a dog encoding-speed-wise I never took it seriously, but it is a multi-channel contender. Thanks to the testers so for tying up their machines for a week

Lately I have been learning DVD audio extraction. I'm glad I did since I will be ready to test multichannel whenever the beta may appear. I have at least one DVD that has DTS of a live rock concert- any Zepplin fans out there?
Title: TAK 2.2.0 development
Post by: Destroid on 2011-05-05 03:21:29
(Can't edit above post)

Highly disappointing to discover these DTS tracks have slipped-through any MPAA loudness standards- quite a lot of brick-wall compression/limiting on Front Left/Right  Almost prefer the AC3 versions  Maybe it would be worth trying losslessly compressing decoded AC3 for as ridiculous as it sounds...
Title: TAK 2.2.0 development
Post by: TBeck on 2011-05-18 03:29:19
Good gracious! ALS is such a dog encoding-speed-wise I never took it seriously, but it is a multi-channel contender. Thanks to the testers so for tying up their machines for a week

How true... It's really slow but probably the strongest competitor for multi-channel audio.

I have made good progress. It took me several days to construct optimally balanced multi-channel presets from the codecs internal encoder options. Even with TAK's high speed i had to spend a lot of time waiting for the many tests to complete. This provided an opportunity to think more about algorithms to speed up the encoding. Some of them worked well and may even be applied to stereo audio. I still have to check this, but i am confident that i can make the standard codec a bit faster too.

Then i will perform a lot of testing and finally prepare an alpha release of TAK 2.2. I can't tell you, how long this will take. Maybe 2 weeks.


Title: TAK 2.2.0 development
Post by: _m²_ on 2011-05-18 09:23:26
How about going for 'stronger' instead of 'faster'?
TAK is a rocket already.
Title: TAK 2.2.0 development
Post by: Destroid on 2011-05-19 04:46:55
@_m²_ On impulse I wanted to disagree, but there might be something to that suggestion. The question I had was how much does disk IO affect these super-size uncompressed source files being converted to considerably-sized TAK files?

Seems mechanical drives would suffer the worst, so in that scenario consider: if, for example, the -p0 settings are 1% faster than -p1 settings (because of suffocated disk IO) the size reduction makes -p1 the obvious choice and -p0 *almost* totally obsolete in for multichannel use.

Guess I may as well add, I don't plan to get an SSD this year either, maybe next year- not sure if that puts me in the user category of the majority or the minority
Title: TAK 2.2.0 development
Post by: TBeck on 2011-05-20 03:58:53
How about going for 'stronger' instead of 'faster'?
TAK is a rocket already.

I see hardly any opportunity to make it significantly stronger without affecting the decoding speed. Furthermore most possible modifications would result in some format incompatibilities. Last but not least: I always try to avoid possibly patented technologies.

A compatible modification would consist in an increase of the maximum predictor count from 160 to 256 predictors. Unfortunately this would only improve the compression of those files, which already achieve relatively high ratios. To be more precise: Those files, which compress significantly better when going from p3 to p4. That's mostly classical music or more general music with a high dynamic range.

@_m²_ On impulse I wanted to disagree, but there might be something to that suggestion. The question I had was how much does disk IO affect these super-size uncompressed source files being converted to considerably-sized TAK files?

Well, when testing the speed, i always try to eliminate the effect of disk-io... But surely i am aware of the fact, that speed increases of TAK's fastest presets p0/p1 usually have little practical effect on todays average hardware because of the disk speed limitations.

Seems mechanical drives would suffer the worst, so in that scenario consider: if, for example, the -p0 settings are 1% faster than -p1 settings (because of suffocated disk IO) the size reduction makes -p1 the obvious choice and -p0 *almost* totally obsolete in for multichannel use.

There are also those additional evaluation levels like for instance -p0e, p0m and so on...

Ok, i am sure not many people are using them. And this does make sense, if you don't care about a small decrease of the decoding speed.
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-05-20 13:42:04
Quote
A compatible modification would consist in an increase of the maximum predictor count from 160 to 256 predictors. Unfortunately this would only improve the compression of those files, which already achieve relatively high ratios. To be more precise: Those files, which compress significantly better when going from p3 to p4. That's mostly classical music or more general music with a high dynamic range.

From my observation, while moving from TAK for OptimFROG, I save the most on classical music and the least on thrash metal.

If you can make a file 1% smaller, why does it matter whether the file is 1/2 or 1/3 of the original?
Title: TAK 2.2.0 development
Post by: TBeck on 2011-05-20 20:30:29
Quote
A compatible modification would consist in an increase of the maximum predictor count from 160 to 256 predictors. Unfortunately this would only improve the compression of those files, which already achieve relatively high ratios. To be more precise: Those files, which compress significantly better when going from p3 to p4. That's mostly classical music or more general music with a high dynamic range.

From my observation, while moving from TAK for OptimFROG, I save the most on classical music and the least on thrash metal.

If you can make a file 1% smaller, why does it matter whether the file is 1/2 or 1/3 of the original?

An increase of the maximum predictor count will decrease the encoding speed (a little) and the worst case (minimal) decoding speed by a significant amount. The first is ok, but the latter is relevant to me because i still have later hardware implementations in mind. Important: No, i don't want to (and will not) discuss going open source issues, but this is still a relevant option affecting my design decisions.

Nevertheless i could be tempted to accept some decoding speed decreases if it would help to compress those files (usually dynamically compressed, and they become more) better where TAK is less efficient than Monkey's Audio. For more dynamical files TAK is already performing very well compared to Monkey's, as can be seen in FLAC's comparison (which is quite outdated regarding TAK, new versions are both faster and stronger now. And V2.2. will be faster again.).

Ok, here i am a bit ambivalent, as you have pointed out.

I don't see OptimFrog as a competitor, this because of different goals. OptimFrog wants to achieve the maximum compression ratio regardless of encoding and decoding speed. TAK want's to bring you the maximum compression at a decoding speed comparable to FLAC. This definitely limits my design decisions.
Title: TAK 2.2.0 development
Post by: Destroid on 2011-05-21 20:07:03
Quote
Well, when testing the speed, i always try to eliminate the effect of disk-io... But surely i am aware of the fact, that speed increases of TAK's fastest presets p0/p1 usually have little practical effect on todays average hardware because of the disk speed limitations.
...
Ok, i am sure not many people are using them. And this does make sense, if you don't care about a small decrease of the decoding speed
I didn't mean to sound like "-p0 is useless," I think some devices will greatly benefit from plain -p0 setting. I guess I was hinting that any upcoming multi-channel encode speed tests (which I plan on doing) might take into account the disk IO bottleneck. In the past I used TIMER.EXE  "kernel time" to spot disk IO bottlenecks with decoding. I was curious if there was a better way to measure the 'performance offset' caused by disk IO saturation.

Quote
I see hardly any opportunity to make it significantly stronger without affecting the decoding speed. Furthermore most possible modifications would result in some format incompatibilities.
I think it is smart for TAK to keep decoding speed a priority. I was always impressed by the encoding speed and compression figures too. However, users that are archiving and seldom access those archived files might not see the advantage of TAK's decoding speed, which adds up greatly over time- advantages like quick decompression on workstations and better battery life on portable portable media players.

Quote
An increase of the maximum predictor count will decrease the encoding speed (a little) and the worst case (minimal) decoding speed by a significant amount. The first is ok, but the latter is relevant to me...
...
Nevertheless i could be tempted to accept some decoding speed decreases if it would help to compress those files (usually dynamically compressed...
The increase in predictors could be what the 'archival users' would like, and those users might not notice the decoding performance. A suggestion I can make is increased predictors would be an option only available to -p4 (and then pray someone doesn't complain that -p4x works but -p1x isn't smaller than -p1m).
As for dynamically compressed material- it appears, unfortunately, that it's here to stay. Further up in this thread was my disappointment that dynamic compression found its way into DTS too, which led to my half-joking comment of "compressing decoded AC3" just because the loudness standard is enforced with those files. Yes, I plan to do it anyway-  just because I want to compare how TAK handles the loud DTS and the normal, non-brickwalled audio.
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-05-22 21:37:42
Quote
However, users that are archiving and seldom access those archived files might not see the advantage of TAK's decoding speed, which adds up greatly over time- advantages like quick decompression on workstations and better battery life on portable portable media players.

I regularly access my music and for me the difference is small as long as I can achieve realtime decoding speed. For portable players I have ogg, it's files are small and decoding is efficient.
Title: TAK 2.2.0 development
Post by: TBeck on 2011-05-22 22:46:49
In the past I used TIMER.EXE  "kernel time" to spot disk IO bottlenecks with decoding. I was curious if there was a better way to measure the 'performance offset' caused by disk IO saturation.

I plan to perform an own comparison for TAK's homepage (i also plan to put a lot more info there, but somehow i never find the time to do it...) and then i will test it with a ramdisk. I have no clue if this is a good approach. I'll have to try it.

The increase in predictors could be what the 'archival users' would like, and those users might not notice the decoding performance. A suggestion I can make is increased predictors would be an option only available to -p4 (and then pray someone doesn't complain that -p4x works but -p1x isn't smaller than -p1m).

And then there are always users who judge the speed only by the strongest setting.

If i increase the predictor count, this will be done for p4.

Unfortunately i am not feeling very comfortable with such an increase, because

- The advantage will not be big.  My primary test corpus, which contains quite some files that benefit from more predictors, compresses only about 0.15 percent (relative to the original file size) better. It's up to about 1 percent for rare files only. This is surely not enough to attract users who prefer OptimFrog.

- Bigger frames would help a bit, and frames which depend on previous frames even more. But i prefer small independent frames, which limit the possible loss in case of a bit error to only a small amount of samples.

- Mpeg4Als -7 is doing the above. Furthermore it uses higher bit resolution multiplications for the filters. This may bring an advantage of about 0.3 to 0.4 percent, but at the cost of also much slower processing. I wouldn't want to sacrifice most of TAK's speed advantages for this little. Ok, i am sure i could write an ALS-encoder which is a lot faster than the reference implementation, but still significantly slower than TAK.

- I am quite sure, i would have to use similar technologies as the symmetrical codecs to be able to compete with them compression wise. But this would be a whole new project. I was not joking when i said i invested thousands of hours in TAK's development. Ok, it would be less for such a new codec, because i already know a lot which would be useful.
Title: TAK 2.2.0 development
Post by: Destroid on 2011-05-22 23:57:38
I realized my suggestion from my prior post was lacking forethought, but it was too late to edit. Perhaps it's safe to say TAK is not the "end all" for lossless codecs, and- most likely- there is no such thing as the one codec that is best at everything. But I also think TAK has tangible and enviable advantages.

Disregard my unenlightened suggestions, thanks!
Title: TAK 2.2.0 development
Post by: TBeck on 2011-05-23 00:06:13
I realized my suggestion from my prior post was lacking forethought, but it was too late to edit. Perhaps it's safe to say TAK is not the "end all" for lossless codecs, and- most likely- there is no such thing as the one codec that is best at everything. But I also think TAK has tangible and enviable advantages.

Disregard my unenlightened suggestions, thanks!

oh no!

Sometimes my statements may sound a bit short and therefore harsh, because my english still is not really fluid.

And i had to think about it before i could answer. I know, i thought about this issue in the past, but forgot it. I should write a FAQ, especially for myself...
Title: TAK 2.2.0 development
Post by: Ljubo44 on 2011-05-23 14:21:08
Where I can to download TAK 2.2.0 encoder 
Title: TAK 2.2.0 development
Post by: marc2003 on 2011-05-23 14:47:05
read the thread? 

Then i will perform a lot of testing and finally prepare an alpha release of TAK 2.2. I can't tell you, how long this will take. Maybe 2 weeks.


Title: TAK 2.2.0 development
Post by: TBeck on 2011-05-23 18:08:13
read the thread? 

Then i will perform a lot of testing and finally prepare an alpha release of TAK 2.2. I can't tell you, how long this will take. Maybe 2 weeks.


Indeed.

I just started the first test runs. Even without the new multi channel audio tests the whole validation process already took more than a day. The tests are running on my primary PC (it's my fastest one). Since i also need it for other purposes, i often have to interrupt the tests. There is also a chance that i find bugs in the new code.

Therefore i still can't tell you an exact release date.
Title: TAK 2.2.0 development
Post by: _m²_ on 2011-05-23 22:13:11
Quote
- Mpeg4Als -7 is doing the above. Furthermore it uses higher bit resolution multiplications for the filters. This may bring an advantage of about 0.3 to 0.4 percent, but at the cost of also much slower processing. I wouldn't want to sacrifice most of TAK's speed advantages for this little. Ok, i am sure i could write an ALS-encoder which is a lot faster than the reference implementation, but still significantly slower than TAK.

Out of curiosity, increased complexity of both encoding and decoding or only encoding?
Title: TAK 2.2.0 development
Post by: TBeck on 2011-05-23 23:05:33
Quote
- Mpeg4Als -7 is doing the above. Furthermore it uses higher bit resolution multiplications for the filters. This may bring an advantage of about 0.3 to 0.4 percent, but at the cost of also much slower processing. I wouldn't want to sacrifice most of TAK's speed advantages for this little. Ok, i am sure i could write an ALS-encoder which is a lot faster than the reference implementation, but still significantly slower than TAK.

Out of curiosity, increased complexity of both encoding and decoding or only encoding?

Decoding too. If it was only for encoding, i would be far less hesistant.
Title: TAK 2.2.0 development
Post by: TBeck on 2011-05-27 17:43:03
I just started the first test runs. Even without the new multi channel audio tests the whole validation process already took more than a day. The tests are running on my primary PC (it's my fastest one). Since i also need it for other purposes, i often have to interrupt the tests. There is also a chance that i find bugs in the new code.

No bugs found, but the tests indeed lasted until now. One of the tests took more than 24 hours! I just upgraded my RAM from 2 to 4 GB to have more space for file caching. This was definitely a bottleneck for the multi channel audio tests. Especially because my totally silent desktop PC has only a single 2.5 harddisk.

Ok, enough whining. I will now prepare an alpha release.