Hi
I'm planning to run some damage tests to check TAK's error resilience and see how it fares against existing codecs. I plan on using TBeck's Damage tool to damage the files. My questions are:
0. How was the error handling column of the Lossless Comparison Wiki made? What does "yes" in the error handling column- that the file is still playable?
1. How do i quantitatively check for damage?
I was planning to see if the files are playable with foobar2000, to see if they decode to wav without any problems (using the commandline decoder e.g. wavunpack for wv files). Is there a more "scientific" way to do it?
TBeck had this to say about TAK:
Tak's decoding error protocol will show you something like this:
Header frames: 321
Valid frames: 320
Of the 321 frames of the file only 320 frames could be restored, hence one frame is lost.
You could compare the count of errors you have applied to the file with the count of lost frames.
Additionally you could calculate a total count of lost samples by multiplying the lost frames count with the frame size (in samples) which can be taken from the "Skipped data blocks" table:
No Source pos Size Sample pos Count
1 1069674 30665 429936 5512
2 2014491 34040 804752 5512
"Count" usually equals the frame size. Exceptions: Two succeeding frames have been damaged (then it's twice the frame size) or the last frame has been damaged (most of the time the last frame is smaller). Well, i should have added a total count of lost samples to the table...
Important: If you get an "Undecodable"-error you have to disable the "Restore wave file meta data" option to decode the damaged file: -wm0
Is there a way to do something similar for other codecs?
2. Do i need to check all settings of a codec to get the representative picture?
I was planning on checking the "extremes" only: e.g. Flac 0 & Flac 8, Wv Fast and HHx6, Monkey at Fast and Extra High. Will i get significantly different results if i compare the entire set (Flac 0 through 8 etc)?
TBeck had this to say about TAK:
There are two important structures within a Tak file:
1) The container is the same for every preset. Here it's enough to test one preset.
2) The encoded audio frames. Here it could be important, that the preset used for damage testing is using any of the internal compression methods of Tak. This is true for presets 3 and 4.
I would try -p3 or p3m. This should be sufficient.
Similarly, is testing say Flac 0 and Flac 8 "sufficient" (i.e. is it enough to establish a trend)?
3. Could anyone share their experiences in damage testing/ give tips or suggestions or anything?
i did check rjamorim, guruboolez and greynol's discussion about monkey's audio's error handling in the discussion for "which lossless codec is the best?" But it seemed like the tests that were done there mainly checked if the files were playable after damage.
Thanks
Kanak