DAMAGE files [-e -f -s -w]files specify file or directory (Dir\*.ext) to be processed-e x1 x2... specify up to 5 errors xn of type i or r: i 3 = (i)nvert a sequence of 3 bits r 10010 = (r)eplace a bit sequence with bits 10010 (msb left, up to 40 bits) The default definition is: i 1 i 2 i 3 i 36 r 000000000000000000000000000000000000-f r x relative frequency of damage as errors per MByte. Maximum: 128 Default: 1-f a x absolute frequency of damage as errors per file. Up to 1024, but will also be limited to relative maximum.-s x no damage until file position x in bytes. Default: 4096-w wait for enter key when finished
D:\VocComp\Tools\ATrain.yaaNo Position BitOfs BitNum Original New value 1 142614 00022D16 0 36 B1 DA 8A 49 DD 00 00 00 00 D0 2 444685 0006C90D 0 36 6F 17 E4 F2 14 90 E8 1B 0D 1B 3 771488 000BC5A0 5 1 95 B5 4 1046975 000FF9BF 5 3 9A 7A 5 1264657 00134C11 1 2 78 7E 6 1454473 00163189 5 36 7A F5 C2 36 2A 36 1A 00 00 00 00 36
' date='Jul 5 2006, 22:43' post='409194']This program is interesting, but it would be nice if you could set a damage frequency and generate damage randomly. Maybe using the Mersenne Twister could help you; Also, it would be nice having random bits instead of just inversed ones, or replaced with known bits.
' date='Jul 5 2006, 22:43' post='409194']Also, you should be able to limit damage to a specific zone (in bytes?) in the file, or to limit the damage per zone (eg, 3 bits changed every 30 bytes, maximum)
' date='Jul 5 2006, 22:43' post='409194']You're my favourite developper
My first implemementation has used random bit patterns. But that was not optimal for my purposes. I like to have control over the conditions. Otherwise the results would not be easy to interpret. And it seems to make even more sense to use controled conditions for comparisons between compressor. You can evaluate, how resistent they for instance are to 1 bit or 2 bit errors, but if you apply random errors and get different results for two compressors, you will not know, if this is beeing caused by the difference of the compressors or by the diffence of the test data caused by the randomness.
this can mostly be solved by using a pseudo-random generator and exposing the seed as a command-line option.
I just want to say that real errors are usually "bursts". (a group of consecutive bytes that are totally wrong -- ie a whole sector).
This could be modeled via a two-state system. Depending on the state the current bit will either be kept or replaced by a randomly chosen one. After processing a bit you either stay in the same state or change to the other based on a randomly chosen number and a threshold. Obviously the model's parameter are the two thresholds (one for each state). The initial state should be the "keep-original-data-state".I just realized that this is actually a simulation of a Markov process.
While I think a complete error simulation would be overkill, support for both random bit errors and burst errors would be useful. This is because error correction schemes respond differently to these two different classes of errors and may perform very well on one and really badly on the other.
Quote from: cabbagerat on 06 July, 2006, 11:10:43 AMWhile I think a complete error simulation would be overkill, support for both random bit errors and burst errors would be useful. This is because error correction schemes respond differently to these two different classes of errors and may perform very well on one and really badly on the other.I don't see a need for random patterns. The default test set allready provides some randomness:- The test patterns are beeing applied to random positions within the file.- The inversion of existing bits at random positions creates different patterns according to the variations of the original bits. Obvious exception: if the data bits don't vary (for instance all zero), the inversion brings no variation. But this is very unlikely to happen with compressed data, that should be random to some degree.