HydrogenAudio

CD-R and Audio Hardware => CD Hardware/Software => Topic started by: tigre on 2003-05-03 14:41:25

Title: EAC secure mode test proposal
Post by: tigre on 2003-05-03 14:41:25
[span style='font-size:8pt;line-height:100%']Edit : this discussion is splitted from this thread : http://www.hydrogenaudio.org/forums/index....=ST&f=20&t=8849 (http://www.hydrogenaudio.org/forums/index.php?act=ST&f=20&t=8849)[/span]


Quote
Quote
Well, actually errors 'always' (99.99...%) return different data for every new read.


How do you know that? it might be 60%, or 10%, because it cannot detect when the data is the same you cannot put a % on it. When AccurateRip makes its way onto EAC then you could put a percentage on it.

A. I haven't read anything about a "validated" percentage like this anywhere (wrong_samples_noticed_by_EAC_due_to_non-matching_results_on_re-read / total_number_of_wrong_samples).


B. This could be tested (volunteers appreciated) like this:

1. You need a (set of) reference .wav file(s). For this either rip a CD in perfect condition, EAC secure mode, ideally compare the CRC to a identical disk's ripped by someone else (cheers to AccurateRip  ) or choose some .wav files, burn them to CD-R and make sure you keep them unchanged on your HDD.

2. Take the CD(-R), damage it (e.g. different kind of scratches, caused by screwdrivers, sandig paper, ...; dirty fingerprints; draw lines with a thin marker on the data side; ...). Start carefully and try what happens in the next steps. If the disk turns unreadable and causes timing problems/sync errors in burst mode this won't work ...

3. Extract the damaged disk two times in burst mode resulting in a.wav and b.wav.

4. Do wave substractions (Cool Edit: mix paste - overlap - invert checked) r-a.wav=reference.wav - a.wav and r-b.wav = reference.wav - b.wav and a-b.wav = a.wav - b.wav
using Cool Edit or another wave editor capable of that - make sure that dither is disabled in this and the next steps.

5. We want to know how many samples have been turned "wrong" by damaging the disk, so we create a Cool Edit boolean operation™ like this: if a sample has been read correctly in a.wav, the corresponding sample in r-a.wav will be 0, otherwise something =|= 0. To find samples that are wrong in a OR b, we add r-a.wav and r-b.wav. To prevent errors here we multiply both files with themselves (Cool Edit: mix paste -> modulate), so all values are 0 (correct) or >0 (wrong). To prevent errors here (Cool Edit devides by 32768 after multiplying, so values -182<y<182 will turn 0 due to rounding) we amplify both .wavs by 100dB before. This will lead to 0 values = correct and 32767 values = incorrect (-> amplification by 100 dB makes all samples  =|= 0 clip in 16 bit resolution). So the steps are:

5.1. Amplifying: r-a_100.wav = r-a + 100dB; r-b_100.wav = r-a + 100dB
5.2. Copy r-a_100.wav to clipboard and mix paste - modulate r-a_100.wav with data from clipboard, result: r-a_100m.wav; do the same with r-b_100.wav => r-b_100m.wav
5.3. Add (= Cool Edit mix paste - overlap) r-a_100m.wav + r-b_100m.wav = wrong_samples.wav


In the file wrong_samples.wav. we have now all 0 samples stand for correctly extracted samples in both EAC burst mode runs while =|= 0 samples (32767) stand for extraction errors.

6. Now first interesting thing to do IMO would be to have a look at the error postitions. Are there isolated wrong samples or do they occur as blocks / "bursts"?

7. Now we want to know how many of the wrong samples have gotten different values in the two burst mode runs so EAC secure mode would have detected them. For this we
7.1. apply steps 5.1 and 5.2 to a-b.wav resulting in a-b_100m.wav
Step 7.2.
7.2.1. Take a-b_100m.wav and apply Channel Mixer -> Full mix (L = L+R; R = L+R). -> detected_errors.wav
7.2.2.: multiply wrong_samples.wav with detected_errors.wav (Mix Paste Modulate) and substract the result from detected_errors.wav, resulting in the corrected undetected_errors.wav.


The result will contain 0 samples for correct samples and detected errors and 32767 samples for undetected errors.

8. Counting the Number of 32767 samples in wrong_samples.wav (W) and in undetected_errors.wav (O) will enable you to calculate a percentage of correctly detected errors: P=(W-O)/W

Have fun

edit:
Additionally we could rip 3 or 4 times in burst mode instead of  2 times to find out if and how often Test & Copy in secure mode without C2 detects addtional errors.

And we could rip in secure mode, C2 enabled (if drive capable) to find out the "P" value for the drive's C2 detection.

edit2: changed some "?" to "=|=" which means "un-eaqual"

edit3:
I missed one thing: If in the 1st 2 reads an error occurs, the sample is re-read in both channels, no matter if the error was detected in one or both channels. So "undetected" errors in one channel are in fact detected if there's a detected error at the same postion on the other channel.

To take this into account in the test, step 7 is modified (see above).
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-03 15:11:19
For this kind of tests, it would be much better to use the DAEquality package, or the EAC compare wavs routines, because they can resynchronize the wavs if some skipping occur in order to go on comparing. Your method stands only if no skipping occurs.
Edit : it would be interesting to run these tests.

All other issues are already covered in the EAC website and hydrogenAudio FAQ.
Title: EAC secure mode test proposal
Post by: mrosscook on 2003-05-03 15:49:44
tigre,

Your test outline is very interesting and ingenious, but I think there may be a potential problem with step 7.  There, you are assuming implicitly that the "reread" from comparing two separate burst mode rips is comparable to the reread that EAC does when it is in secure mode, no C2.

I don't know that this is true -- in burst mode, my drive runs like a bat out of hell, and in comparison, the EAC secure read creeps along;  so the drive speed is very different.  Isn't it possible that the drive speed affects the accuracy of the read?

EDIT:  DrDoogie, you take my advice to get out of Iraq, you run like hell, and then you stop in North Korea!?  I would really advise you to change travel agents.
Title: EAC secure mode test proposal
Post by: liekloo on 2003-05-04 12:00:06
Quote
Tigre, your test outline is very interesting and ingenious, but I think there may be a potential problem with step 7.  There, you are assuming implicitly that the "reread" from comparing two separate burst mode rips is comparable to the reread that EAC does when it is in secure mode, no C2.

That should be correct though (EDIT: I am talking about Tigre). If there is a 'weak spot' somewhere in the suggestion, I think it would be the starting point: starting from a CD in perfect condition and assuming EAC will do its job correctly. (not that I'd doubt about that  )

(EDIT = clarification:  when I said 'should be correct', I was talking about Tigre).
Title: EAC secure mode test proposal
Post by: evereux on 2003-05-04 12:04:13
Quote
There, you are assuming implicitly that the "reread" from comparing two separate burst mode rips is comparable to the reread that EAC does when it is in secure mode, no C2.

But isn't the end result (having a matching CRC) one and the same thing?
Title: EAC secure mode test proposal
Post by: ExUser on 2003-05-04 12:16:36
Quote
Quote
DrDoogie,
Your flag says you are based in Iraq, and ripping a perfect CD is your biggest problem?
DUCK!  For God's sake, DUCK!

Well, I have to use all the US grants for something, and there's a limit to how many chemical weapons they are willing to sell me, after all.

Well met, you two. That little repartee gave me a nice big grin way down in my soul. Thank you both. That made my night.
Title: EAC secure mode test proposal
Post by: liekloo on 2003-05-04 12:35:18
Quote
Quote
There, you are assuming implicitly that the "reread" from comparing two separate burst mode rips is comparable to the reread that EAC does when it is in secure mode, no C2.

But isn't the end result (having a matching CRC) one and the same thing?

Yes.

however, if you want to take the (small) CRC algorithm's failure chance ( = different WAVs give identical CRC sums ) into consideration, then there is a difference (between twice burst mode + matching CRCs, and secure mode) :
the more secure the read mode, the greater the chance WAVs are REALLY identical ( = the lesser the chance CRC algorithm failed )
Title: EAC secure mode test proposal
Post by: evereux on 2003-05-04 12:50:07
I have read a file can be manipulated to give the same CRC value as another file. But the chances of two wav files, being read from the same source one having a flaw and the other not has to be about as likely as me winning the lottery (and I don't play)? 
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-04 13:50:46
EAC's CRC are 32 bits, thus there is one chance out of 4,294,967,296 to get the same CRC on different files. Furthermore, AFAIK, CRCs algorithms are made so that if only one bit is changed, the CRC is different 100% of the times.
Title: EAC secure mode test proposal
Post by: mrosscook on 2003-05-04 15:05:29
No, the point I'm trying to make doesn't have anything to do with CRCs that match by accident.  I'm talking about samples identified as errors, not checksums.

In steps 4 and 5, tigre creates his files (r-a).OR.(r-b), representing all wrong samples, and (a-b), representing differences in the samples of the two burst mode (or fast-read) wav files.  Then he cleverly modulates these into files where only two values are possible for each sample, 0 (correct) or 32767 (incorrect); these are his files wrong_samples, (where one or both burst reads differ from the reference) and a-b_100m (where the two burst reads disagreed).

NOW in step 7, we take the difference of these two files (i.e., {wrong_samples  - (a-b)_100m} = overlooked_errors),  and assume that it shows us how many times a normal,  secure mode, no C2 reread would have failed to identify an error. As tigre says, the point is "to know how many of the wrong samples have gotten different values in the two burst mode runs so EAC secure mode would have detected them."  The file "overlooked_errors" is actually the number of wrong samples that got the same  value in the burst runs, so that EAC wouldn't  have detected them, but that's OK, you see what we're talking about.

This is where my question comes in.  If the burst mode reads are performed much faster than the normal EAC double-read extraction, how do we know that ANY of the three files a, or b, or (a-b) are the same as EAC secure would have given?  Maybe there would be many more errors in a burst  read since it goes so fast.  Checksums don't enter into it.

I hope I'm being clear here. The whole thing makes my head hurt a little. I certainly do think the test should be carried out, preferably by somebody who has more experience with Cool Edit than I do.  But I think it might only give an UPPER bound to the rate of undetected errors in EAC secure mode.
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-04 19:44:11
Yes, you're right.
From experience, I think that there can be the same difference from a rip to another with the same settings, as between secure mode no C2 and test and copy burst. The speed up/down can be chaotic, you can get same CRC easily, then it becomes impossible (the drive has spun down, the CD has heated...), you can easily get a right test and wrong copy everytime, or the opposite, because of a sync error at the end of the track that would always spin down the drive between the test and the copy.

Having now studied Tigre's method, I'm sure that it would give very bad results, I mean a lot of undetected errors, compared to EAC's true behaviour. EAC's secure mode compares bunches of 27 sectors between them, and rereads as soon as one sample is different, whatever the numbers of overlooked errors there is. Then 8 matching reads are needed.

Thus EAC fails if every error in 15876 samples are "overlooked", which probability of happening is orders of magnitude smaller than getting one error read the same way two times, or it fails if the same error occurs more than 8 times in 16 rows.
Andre said that the 27 sectors were not all re-read. EAC only re-reads smaller and smaller parts when it tries to correct errors it has detected. How does it choose them, I don't know.
In order to really test EAC's secure mode, we need either to know the complete error correction algorithm of EAC so as to emulate it, either Andre to provide us a tool that outputs a binary file with the positions of the errors detected by EAC, like the C2extract.exe for C2 errors.
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-04 19:48:20
I think it's sensible to assume that the first time, the 27 sectors are all re-read, then the starting and trailing parts that matched 8 times are overlooked in the further re-read sessions.
Title: EAC secure mode test proposal
Post by: liekloo on 2003-05-04 20:23:31
Quote
Yes, you're right.
From experience, I think that there can be the same difference from a rip to another with the same settings, as between secure mode no C2 and test and copy burst. The speed up/down can be chaotic, you can get same CRC easily, then it becomes impossible (the drive has spun down, the CD has heated...), you can easily get a right test and wrong copy everytime, or the opposite, because of a sync error at the end of the track that would always spin down the drive between the test and the copy.

Is the following conclusion correct, Pio ?

Twice burst mode is not exactly the same as a normal secure read: they are different processes, so can give different results, but average secureness should be the same though?

(no C2)
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-04 21:04:06
I don't think that their secureness is even similar. I consider test and copy to be more secure, because a drive can always cache and be detected as not caching (EAC and Feurio disagreeing).
As said in other threads, "test and copy burst mode" should be as secure as "secure mode, no C2, cache, accurate, 100 % quality".
Title: EAC secure mode test proposal
Post by: tigre on 2003-05-05 09:33:43
@All: Thanks for your replies, suggestions and attention.

@ mrosscook: IMO your "extraction speed at secure mode doubts" are valid. As it's impossible to simulate changing extraction speed that seems to occur when EACs error correction kicks in, the only way to improve the test in this point would be to repeat it at different burst mode extraction speeds.

Q: If you un-check "Allow speed reduction during extraction", is everyting read at constant speed, also the re-reads for error correction? (Pio2001?) If it is like this the test would at least give similar results to secure mode with this option.

@ Pio2001: Thanks for the 27-sectors info. I have an idea how to adapt the test to it, but for this I should have validated information about what is re-read when error correction is performed and what not. What I'd like to know:

1. E.g. if there's a single 6-samples burst with differences between 1st and 2nd read - Are:
A. the same 27 sectors re-read 8 times as in the 1st two reads or
B. are the 13 sectors before the sector containing the burst, the sector itself and the 13 sectors after re-read?

2. If there occur 2 single errors with < 27 sectors between them - Is always everything between them re-read 8 times? (If 1.B. is correct, it'll be like this.)

3. Is the reading spead in the 1st 8 re-reads already lowered?

4. Is there a way to assume where sector borders start/end from a extracted .wav file? (counting from the beginning taking into account the drive's offset; error patterns; ...)
Title: EAC secure mode test proposal
Post by: tigre on 2003-05-05 09:39:49
I started a test as proposed:

The CD: VA  -  Stella Musical - Highlights. It's a pressed promotion CD and I have it twice. One version is scratched now.

Before scratching it, I extracted/tested both CDs with both drives I have (Plextor PX-W 121032A, LG DVD DRD 8120B) with CRCs always matching. I guess I have a perfect extraction result of it on my HDD now.

So far I did the test on the 1st 2 Tracks, both test extractions at 32x burst on the Plextor drive.

To make the results easier to read I want to introduce error patterns I found first:

c=(always) correct sample; w=(always) wrong sample; s=sometimes wrong, sometims correct sample

pattern 1a:
Basic structure: scscsc, can occur simultanous on both channels.
The basic structure occurs multiple times (up to 6) in one burst, separated by 42 correct samples.

pattern 1b:
Basic structure: ...scscs... can occur simultanous on both channels
The basic structure occurs multiple times in one burst, one burst can be up to some 1000s of samples long. It seems as if there's a pattern in a way that the whole burst is separated in blocks of 6 samples that are either scscsc or cccccc. So 1a is an example for 1b

If this needs clarification, I could post some screen shots (or Hex editor dumps - beware of asking for it  ).

results

Track 1:

r-a_100m.wav: 1 Burst containing 15 wrong samples, pattern 1a, lenght from 1st-last wrong sample: 241 Samples

r-b_100m.wav: no errors
=>
a-b_100m.wav = r-a_100m.wav
=>
P=100%


Track 2:

r-a_100m.wav: 85 Bursts containing various numbers of wrong samples, pattern 1b, variying length, 4 of them only on 1 channel

r-b_100m.wav: 104 Bursts, 5 only 1 channel, same type as a-r

a-b_100m.wav: 139 Bursts, 6 only 1 channnel, same type as a-r

wrong_samples.wav: 139 Bursts, 6 only 1 channel, 6309 errors.

undetected_errors.wav: 35 Bursts (all on both channels), 895 samples =|= 0.
_______________
Total Errors:  6324 (Track 1+2)
Undetected:    895
P = 85.8%


As Pio2001 wrote above this would be an awful result if you don't take into account that EAC re-reads 27 sectors when an error is found in the 1st 2 reads. Without the information I've asked for in my previous post it's hard to interpret this result. One remarkable result is though, that in every of the 139 Bursts some errors were detected, so there is a great probability that even undetected erroreous positions would have been re-read 8 times by EAC...
Title: EAC secure mode test proposal
Post by: DrDoogie on 2003-05-05 14:58:49
Tigre:
5 sounds good, I'll have a look at it later. But the underlying assumption of your argument, that EAC can be used to test the correctness of a drive's error-reporting / accurate rip, is that EAC doesn't mess up the wav in any way. Which sounds reasonable, only I would like to add that I have seen cases where cdparanoia and plextools would return identical rips, and EAC would disagree. And in this case, that using all variations of ripping modes in EAC (and a number of different speeds for the drive) still gave the exact same disagreement between rips.

So.. perhaps one should test different rippers as well? 

Quote
Isn't it possible that the drive speed affects the accuracy of the read?


Yes, that is correct. CAV seems better than CLV from my tests with daequality - I would guess the constant velocity of the drive is easier on the servo'es and thus the ECC-handling (less jitter, for starts).
Try daequality's disc and instructions out at different speed settings for your drive.

Quote
EDIT:  DrDoogie, you take my advice to get out of Iraq, you run like hell, and then you stop in North Korea!?  I would really advise you to change travel agents.


Who's running?
I'm just stoking up on radioactives, before I go on my annual holiday to the land of the free and the brave burger-munchers. You'd be amazed at how much material one can bring into the country if one only dilutes it, shields it a bit, and is patient.
Now, which city should be brought to justice, I wonder? "Inee-meeny-meeny-moo, where is the dirty bomb gonna blow?" 
Title: EAC secure mode test proposal
Post by: tigre on 2003-05-05 15:36:35
Quote
Tigre:
5 sounds good, I'll have a look at it later. But the underlying assumption of your argument, that EAC can be used to test the correctness of a drive's error-reporting / accurate rip, is that EAC doesn't mess up the wav in any way. Which sounds reasonable, only I would like to add that I have seen cases where cdparanoia and plextools would return identical rips, and EAC would disagree. And in this case, that using all variations of ripping modes in EAC (and a number of different speeds for the drive) still gave the exact same disagreement between rips.

So.. perhaps one should test different rippers as well?


It'd be interesting what these differences were like. Have you done a wave compare/substraction?

Anyway, with this test I want to get an impression how reliable EAC's reading twice is for detecting errors. So ATM I don't care about testing other programs.
Title: EAC secure mode test proposal
Post by: tigre on 2003-05-06 11:44:26
Test part 2
trying to take into account the amount of data re-read if an error is detected


To simulate the fact that EAC not only re-reads detected wrong samples themselves, but also a range arround these samples, I did this:

1. Take detected_errors.wav, add (mix paste -> overlap) two copies of itself, one of them with 1 sample of silence added in the beginning, the other with one sample cut before. Now in the resulting file detected_errors_0003.wav every "wrong" sample is replaced by 3 wrong samples in a row, with the originally wrong sample in the middle.

2. Now Step 7.2.2 is done again, resulting in undetected_errors_0003.wav
Quote
7.2.2.: multiply wrong_samples.wav with detected_errors.wav (Mix Paste Modulate) and substract the result from detected_errors.wav, resulting in the corrected undetected_errors.wav.

3. This whole thing (1.-2.) is repeated multiple times, but everytime the number of samples added/cut in step one before  is multiplied by 3, resulting in detected_errors_0009.wav with 9 wrong samples in a row, detected_m_0027.wav with 27 wrong samples in a row ....

To illustrate this here an example (mono; _= correct, -=wrong)

___________-_-____-___-_-_-_____-___-_______ : wrong : 8 errors
___________-_-________-_-___________-_______ : detected : 5 errors
__________________-_______-_____-___________ : undetected : 3 undetected errors

__________-----______-----________---_______ : detected_0003
__________________-_______-_____-___________ : undetected_0003 : 3 undetected errors

_______----------------------__---------____ : detected_0009
____________________________________________ : undetected_0009 : 0 undetected errors

Results with Track 2:
Code: [Select]
range      additionally    Remaining
re-read    re-read         undetected
if an      samples be-     errors
error     fore/after a
occurs     detected error

1      ->          0 :      895 wrong
3      ->          1 :      848 wrong
9      ->          4 :      543 wrong
27     ->         13 :      299 wrong
81     ->         40 :      160 wrong
243    ->        121 :       70 wrong
729    ->        364 :        6 wrong; all in 1 "pattern"

[/font]
Having a look at this remaining 1 pattern manually shows this:
beginning: 6175422; end: 6175475
next detected errors
before: 6175056; after: 6192877
6175475-6175056 = 419

In this case it would be enough to re-read a range of 419 samples before and after every detected error would be enough to cover all undetected errors.

Assuming that EAC always reads whole blocks (588 samples) it would be enough to re-read 3 blocks: the block containing the detected error + one block before and after.

As this is based on guesswork, I won't continue testing unless I have facts what EAC really does - it'd be pointless somehow.
Title: EAC secure mode test proposal
Post by: DrDoogie on 2003-05-06 14:12:35
I think I lost track of the argument somewhere along the way, BUT...

...when you are introducing errors to a CD, some of those errors will be unreported by the drive, and some will be uncorrectable by the drive (and subsequently hidden from the user by muting, interpolation etc. ad nausam).

Your fundamental assumptions seem to revolve around trust.
Trust that the drive reports all errors, trust that errors are handled identically in repetetive reads of the CD, and trust that EAC detects a single frame with errors in it.

But perhaps I have misunderstood.
So... what assumptions are you making in your tests, and what specifically errors (of the drive, of EAC, of...) do they seek to detect?
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-06 15:49:50
An error is given by the difference between the reference file and the ripped file, however it was ripped or got. The drive's ability to detect or correct them can only interfere if the drive detects the error, interpolates it, and the interpolation gives, by chance, the original value. Is this test, it will appear as an error reported by the drive, but that doesn't exists in reality

Errors unreported by the drive appears as wrong samples in the reference - rip substraction, as well as uncorrected errors, since most of the time, the interpolation doesn't restore the exact original value.
Title: EAC secure mode test proposal
Post by: tigre on 2003-05-06 16:06:56
1st: You ask a question, so I'd appreciate if you answer mine:
Quote
Quote

Tigre:
5 sounds good, I'll have a look at it later. But the underlying assumption of your argument, that EAC can be used to test the correctness of a drive's error-reporting / accurate rip, is that EAC doesn't mess up the wav in any way. Which sounds reasonable, only I would like to add that I have seen cases where cdparanoia and plextools would return identical rips, and EAC would disagree. And in this case, that using all variations of ripping modes in EAC (and a number of different speeds for the drive) still gave the exact same disagreement between rips.

So.. perhaps one should test different rippers as well?

It'd be interested what these differences were like. Have you done a wave compare/substraction?


Quote
I think I lost track of the argument somewhere along the way, BUT...

...when you are introducing errors to a CD, some of those errors will be unreported by the drive, and some will be uncorrectable by the drive (and subsequently hidden from the user by muting, interpolation etc. ad nausam).

If the audio extraction program doesn't use C2, there's no way to "report" any errors AFAIK (maybe oversimplified). The drive just passes sample values to the program, which can be corrected/interpolated by the drive of course.

Quote
Your fundamental assumptions seem to revolve around trust.
Trust that the drive reports all errors, ...

Nope. In EAC secure mode without C2 that I'm trying to simulate, error detection is done by reading twice and comparing the results. So there's no need for the drive reporting errors.

Quote
... trust that errors are handled identically in repetetive reads of the CD ...

I'd rather say I measured (though it would take more measurements with different drives+CDs to validate my findings) that in > 80% of cases errors are not handled identically in repetetive reads. And this is *good* because it makes EAC detect errors, re-read and report errors (=suspicious positions) if < 8 of 80 re-reads are not identical. So far my test has been about error detection, not about correction.

Quote
..., and trust that EAC detects a single frame with errors in it.
I found out that EAC does not detect all errors in the 1st two re-reads, so I performed a 2nd part (again: more tests - and details about how EAC works - needed for validation). IMO so far from this no conclusios are possible, only assumptions. Talking about isolated errors: Given the 85% probability of detecting errors by reading twice, it can happen very well, that isolated errors of a few wrong samples are not detected by secure mode - no C2, but would be detected (and corrected) using C2.

Quote
But perhaps I have misunderstood.
So... what assumptions are you making in your tests, and what specifically errors (of the drive, of EAC, of...) do they seek to detect?

Your question (something like "How secure is secure mode") made me think about it. I've extracted a lot of CDs so far with EAC and never have been dissapointed, so I trust it, but I'm just curious I think.
So the first question was: EACs error detection (secure mode no C2) works by reading twice and comparing the results. EAC's assumption seemed to be (my assumption) that a sample value returned by a drive either is correct or changing on every re-read, otherwise a wrong value could be returned twice, so the error would be undetected. In my 1st test I wanted to find out if this is true. It is not. Only ~ 85% are detected.

Given EAC's good performance on error detection my next assumption (that has been verified by Pio2001 and posts of Andre Wiedhof on digital in EAC forum) was: EAC re-reads much more than just the detected samples AND (verified by my test:) on (pressed) CDs isolated errors occur very seldom, so the high percentage of detected/corrected errors (tests to find out exact numbers can't be performed until I have detailed information on how EAC works) is caused by detected errors near to undetected that cause the undetected to be included into the re-read+error correction process.

If someone tells me how exactly the re-read process works (see questions in previous post) I'll be able to  provide numbers like
"In my test there were ......... wrong samples returned by drive xy in two reads. ... of them would have been undetected taking EACs error detection algorithm into account. This is a percentage of ...".
This is the goal so far for me.
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-06 16:20:45
Tigre, your result is interesting. How does look like the difference between the reference and the rip ? It seems that errors come by bursts, which is not surprising, granted your CD is dirty and scratched.
In my calculus (http://perso.numericable.fr/~laguill2/dae/chipset/chipset.htm), I've found that an elementary burst error (in drives correcting E42 errors without the help of EFM pointers) spreads into a range of 543 samples after de-interleaving, thus reading a +/- 600 range would allow to correct an elementary burst error when one sample (out of 60) is detected in them.
Getting stronger errors doesn't increase the width very much : 6 samples each lost frame (the elementary burst error being 17 lost frames spread into 91 after de interleaving).

The first problem will be drives not using EFM pointers for error correction, with them, there is no such thing as an elementary error. You can have isolated wrong samples.
But the main problem will be non-burst errors, such as those that occur when, instead of a damaged CD, you try to read a decaying CDR, or with a poor drive that tries to read a CD faster than it should. The result of your test might then look completely different.

Here are, in pink, the distribution of burst errors I get on the Memorex DVDMaxx 1648 drive, on the black mark of the DAE quality test CD, in number of errors.

(http://perso.numericable.fr/~laguill2/dae/chipset/zoom.gif)

Each pink square stands for a burst error. Their height is the number of errors they are made of. They are just sorted horizontally by ascendant number of errors. Actually, each square is the number of errors recorded in a 44100 sample range, thus some of the intermediate points might just be 60 samples burst errors overlapping into two contiguous ranges. The blue dots are not to be taken into acount.

Here is the same thing, in blue dots instead of pink, but with a decaying CDR instead of the DAEquality test CD, thus givin random errors instead of burst errors :

(http://perso.numericable.fr/~laguill2/dae/chipset/evencurve.gif)

In this situation, the errors don't gather in clusters, and reading a range around them shouldn't help in any way. Not anymore than reading any other part chosen at random.
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-06 18:41:59
It would be interesting to know how your drive behaves facing burst errors. This analysis is the missing part of my webpage (http://perso.numericable.fr/~laguill2/dae/dae.htm)), but building the DAEquality test CD, with just the black triangle, and looking at the wav substraction, will do. You don't even have to use adhesive tape in order to draw the triangle : I found none that left no marks on CDs, so you can see on the picture that my triangle was drawn by hand 

In the substraction of the rip (burst mode) from the reference, you'll easily see, looking at the first error clusters, if the errors come in clusters of 44 (E22 errors corrected, no use of EFM), 52 (E32 errors corrected, no use of EFM), 60 samples, (E42 errors corrected, no use of EFM), or in clusters growing progressively in size (use of EFM), that would be more annoying for EAC's error detection efficiency.
In the latter case, samples always coming by 3 inside the clusters come from E42 capable chip. If they come by 2, it's an E22 capable chip. If they come alternatively by 2 and 3, it's an E32 capable chip (see the 6 Excel files called "error pattern" in  appendix 2 (http://perso.numericable.fr/~laguill2/dae/chipset/chipset.htm))

It would be interesting to know if it is related to the C2 accuracy. I think that the more errors a drive can correct, the worse must be its C2 accuracy. Spath, one of the only two people that I know who could confirm this has not be very accurate on this subject :
http://forum.cdfreaks.com/showthread.php?s...&threadid=67558 (http://forum.cdfreaks.com/showthread.php?s=&threadid=67558)

He might not have read my last post (may 3rd) yet.
Title: EAC secure mode test proposal
Post by: tigre on 2003-05-27 11:35:23
Thanks to Pio2001 who provided detailed information on the EACs error correction I was able to finish the test.  As I don't know if it's okay to tell details publicly I won?t do this right now, only this: As said before in this thread, EAC indeed re-reads several blocks when a difference in the first 2 re-reads occur (I'll call this extended re-read here). The method I used to take this into accout is already decribed in previous posts - only the numbers are different.

Results:

Total time: 20:32

Total Errors: 10641
Undetected without extended re-read: 1930
P = 81.9 %

Undetected with extended re-read: at least* 22 at 3 different positions.

*at least because I didn't know a way to find out where sectors start/end. This would be needed to get exact results here, as most of the undetected errors (except for the mentioned 22 at 3 positions) are at positions where they would be detected if sector borders would "fit".

If anyone is interested in error patterns etc., I could provide the files detected_errors.wav, wrong_samples.wav ...
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-27 16:30:04
Wow ! This gives (1-20/10000)*100 = 99.8 % accuracy. This is below the C2 accuracy of some drives  ...and above the C2 accuracy of others...

Thank you very much for your work, Tigre. Very useful.
Title: EAC secure mode test proposal
Post by: ExUser on 2003-05-28 05:42:36
Quote
Wow ! This gives (1-20/10000)*100 = 99.8 % accuracy. This is below the C2 accuracy of some drives  ...and above the C2 accuracy of others...

Thank you very much for your work, Tigre. Very useful.

So this means everything we know is wrong and on hard CDs, EAC doesn't give perfect rips?

Or am I reading this wrong?
Title: EAC secure mode test proposal
Post by: tigre on 2003-05-28 08:32:55
Quote
Quote
Wow ! This gives (1-20/10000)*100 = 99.8 % accuracy. This is below the C2 accuracy of some drives  ...and above the C2 accuracy of others...

Thank you very much for your work, Tigre. Very useful.

So this means everything we know is wrong and on hard CDs, EAC doesn't give perfect rips?

Or am I reading this wrong?

I'd rather say: We knew before that EAC doesn't give 100% security (-> "different Test & Copy CRCs on error-free reported Secure Mode rip (Cache disabled, C2 off)" reports). This is why some ppl use Test & Copy + Secure mode.

IMO it's dangerous do generalize ("EAC doesn't give perfect rips."), because ...


I only tested 1 (pressed) CD with ~ 20 minutes of music ... - and "artificial" scratches (I drew a line with a marker and noticed that it caused too much trouble, so I removed it with "brute force". If I try to read the CD in secure mode, it fails (= unrecoverable errors).

My conclusions:
- On this CD EAC secure mode would have reported (many) unrecoverable errors anyway, so a few unnoticed error positions (all of them near to the unrecoverable ones) won't do any harm.

- If the pattern of blocks re-read on an error found would be changed (but same ammount of re-reads) more errors (in my test all) would be covered by secure mode's multiple re-reads.

- The test can show how to do more similar tests, maybe with
-- CDs damaged due to RL usage,
-- CDs that are less damaged, so no unrecoverable errors are shown in secure mode's report and/or
-- dying CD-Rs,
- but the result shouldn't be taken too serious; I'd say it plays in the same league as "different Test & Copy CRCs on error-free reported Secure Mode rip (Cache disabled, C2 off)" reports ...

So I invite everyone interested to repeat this test.
Title: EAC secure mode test proposal
Post by: tigre on 2003-05-28 15:26:48
I tested the same scratched CD with another drive: LG DVD-ROM DRD 8120B

Results:

Total time: 20:32

Total Errors: 1369
Undetected without extended re-read[/b]: 86
P = 93.7 %

Undetected with extended re-read: 36 at 1 position.

This time the undetected faulty position was surrounded by > 200 correct blocks before and after, so no chance to be detected correctly.

BTW: The difference is inaudible at this position, its peak amplitude is -59dB.
Title: EAC secure mode test proposal
Post by: DrDoogie on 2003-05-28 16:00:44
Quote
I'd rather say: We knew before that EAC doesn't give 100% security (-> "different Test & Copy CRCs on error-free reported Secure Mode rip (Cache disabled, C2 off)" reports). This is why some ppl use Test & Copy + Secure mode.

IMO it's dangerous do generalize ("EAC doesn't give perfect rips."), because ...


...because you have never heard the story of the white swans?

Claim: There are only white swans in the world.
Proof: Find all swans, prove that you have found all swans, and show that they are all white and have always been white.
(In other words, exterminate the swans).
Counter-proof: Find one swan that is not white.

...Can you see the "not-white" swan?
Title: EAC secure mode test proposal
Post by: tigre on 2003-05-28 16:31:09
You're right about swans ...

I just havn't hacked EAC so I can see every bit it gets from the drive and what it does with it - I just tried to simulate/approximate what it does, so be careful ...

Example: Some minutes EAC + my LG drive finished extracting the scratched CD in secure mode ... no errors reported - and the rip was bit identical to the reference. At this very moment I let my LG drive extract the position not detected correctly in my "simulation" over and over again in secure mode - until now (~15 rips) either perfect result or unrecoverable errors reported ...

A guess: The position I'm talking about lies ~ 250 blocks after a position with many errors where error correction kicks in every time. So probably when reading on after finishing error correction, the drive's speed is still reduced when getting to the position undetected in the test, so the error will be corrected immediately by the drive - or noticed.

EDIT:
BTW: Noone ever claimed EAC gives perfect security. My test is to get an idea of how many black swans you have to expect - and I just say the results don't tell much about this yet.
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-05-28 18:44:32
Saying EAC is not perfect, and not using it because of this is like saying "I prefer calculate by hand, with a paper and a pencil, because there is a slight chance that an electronic pocket calculator returns a false result in 1000 years. Therefore a calculator won't always give me an exact result".
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-06-04 22:50:54
Tigre, it seems what you call extended reread is assuming that all errors in the rereading range are detected.
One should rip 16 different versions of the CD, and keep the samples that are 8 or 9 times the same as good, then see if they actually are.
In this case, your 99.8 and 97.4 % results are just upper bounds for the actual accuracy, because among the 1930 and 86 other errors, many will remain undetected after rereading, since they are consistent. The 80 % accuracy is then not unlikely.

These consistent errors would be just samples damaged beyond recovery. Each time they are read, an error occurs, and they are always interpolated, thus returning a consistent result.
Title: EAC secure mode test proposal
Post by: tigre on 2003-06-05 00:11:14
Quote
Tigre, it seems what you call extended reread is assuming that all errors in the rereading range are detected.


You're right. I just tried to find the errors that will remain undetected for sure ...

Quote
One should rip 16 different versions of the CD, and keep the samples that are 8 or 9 times the same as good, then see if they actually are.

... because I'm too lazy to rip 16 more times - and do 15+ more comparisons. You're right: One should ... - but this "one" won't be me.

Quote
In this case, your 99.8 and 97.4 % results are just upper bounds for the actual accuracy, because among the 1930 and 86 other errors, many will remain undetected after rereading, since they are consistent. The 80 % accuracy is then not unlikely.
Hard to tell without testing. I'd rather think that most of the errors are detected during 1st 16 re-reads (because of reduced speed and the finding I mentioned in the post before: in secure mode (C2 off, Drive caches checked) the same drive + same CD either give perfect results or report errors at bad positions - 10 trials, no undetected error).

Quote
These consistent errors would be just samples damaged beyond recovery. Each time they are read, an error occurs, and they are always interpolated, thus returning a consistent result.

The way I understand how audio CDs work, there's quite some damage necessary to get an unrecoverable error (C1 + C2 failing). So as several damaged = "unsecure" bits read in raw data are needed to cause an error (that makes the drive interpolate), it's rather unlikely that all of these bits are returned identically on several re-reads resulting in identical interpolation.

Whatever ... My conlusions so far:
- More tests needed
- Real life secure mode performs much better on error detection than my "simulation" for so far unknown reasons.
- It seems like Secure mode (C2 off, correct cache settings) is more secure in error detecting than Burst mode Test & Copy with CRC check.
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-06-05 11:13:55
I think I'll perform some little tests myself.

About the 16 files, it would be easier to write a program that loads the 16 wavs and performs an automated detection.
Title: EAC secure mode test proposal
Post by: tigre on 2003-06-05 11:26:47
Quote
About the 16 files, it would be easier to write a program that loads the 16 wavs and performs an automated detection.

This should work with SOX + a batch file. For my test I prefered to use CEP because there weren't so many operations that could be automated - and I wanted visual control to be sure that e.g. after multiplying a wave with itself all values are positive. (If dither is enabled in CEP accidentially you'll get a few negative values too.)

Quote
I think I'll perform some little tests myself.

Good luck.  Feel free to ask e.g. if you need details about something I did.
Title: EAC secure mode test proposal
Post by: DrDoogie on 2003-06-05 15:05:40
Quote
Saying EAC is not perfect, and not using it because of this is like saying ...

Oh no, that's not what I meant at all.
I'm just sayin' it's healthy to keep a realistic perspective on the limitations inherent in the reading process.

EAC isn't perfect => Don't trust EAC alone

One should keep in mind that the reason Andre has given for writing EAC, was the lack of accuracy of "some" drives in reporting C2 errors.
Meaning they went by unnoticed, in which case re-reading was (and still is) a good idea.

However, now that the C2 error-reading capability of "some" drives are rather good, the next aspect we can grasp, is the inability of any possible drive in correcting an uncorrectable error.

I certainly do not in any way, shape or form "blame" this on EAC - that would have been a logical fallacy along the line of "killing the messenger bringing bad news". 
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-06-05 19:50:13
I agree with you. But the reason for which EAC was written was not the lack of C2 acuracy, but the fact that all rippers just ignored the errors reported.
Title: EAC secure mode test proposal
Post by: kdo on 2003-06-07 21:44:28
Quote
- It seems like Secure mode (C2 off, correct cache settings) is more secure in error detecting than Burst mode Test & Copy with CRC check.

I'd rather re-phrase that conclusion, to make it more specific:
it is more secure in detecting individual errors.

Because the plain "test&copy CRC OK" will leave undetected errors only if ALL those errors are consistent ("repeatable", "bogus-corrected") AND there are NO inconsistent errors. The probability of this event can be "measured" (estimated).
My understanding is that so far it was reported to be very low.
Exception is the recent report by DrDoogie, but it has to be verified whether the result can really be trusted.

In this Tigre's test there were always inconsistent errors, so there would always be CRC mis-match - indication of the fact "there are some errors" (but their number and location is unknown).

(Well, except those rare cases when CRC-algorithm itslef would fail, but then a bit-to-bit comparison would reveal the difference.)

Hope my understanding is right.
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-06-08 00:06:48
Quote
(Well, except those rare cases when CRC-algorithm itslef would fail, but then a bit-to-bit comparison would reveal the difference.)

What do you mean ?
Title: EAC secure mode test proposal
Post by: kdo on 2003-06-08 14:21:13
Quote
Quote
(Well, except those rare cases when CRC-algorithm itslef would fail, but then a bit-to-bit comparison would reveal the difference.)

What do you mean ?

Nothing ground-breaking. Negligible theoretical chanse that two different files have the same CRC.
consistent error is a lot more serious trouble to worry about, no doubt.
Title: EAC secure mode test proposal
Post by: Pio2001 on 2003-06-08 23:39:26
No doubt, no doubt !