HydrogenAudio

CD-R and Audio Hardware => CUETools => Topic started by: Skybrowser on 2010-05-24 22:19:19

Title: CTDB question: Repair function
Post by: Skybrowser on 2010-05-24 22:19:19
First off, i'd like to say that CTDB is probably the most useful and amazing tool i've seen since i started doing DAE a few months ago. Thankyou so much for creating something that can repair bad rips or essentially .. damaged CDs.

I've started using the CTDB repair function and i'm absolutely loving it. I'm struggling with a certain feature of it right now though.

    Some of the albums I want to repair are showing up in the CTDB but when I attempt to verify them, it says for example " AR: offset 112, rip not accurate (0/116), CTDB: could not be verified ".

Now this seems to be happening pretty much only for albums where the accuraterip log shows matches for multiple offsets and the CTDB repair function won't activate and encode my files because it doesnt know which offset to use. I tried telling cuetools what offset to use in order to force the encode, but it wouldnt do it. I tried fixing the offset of the files just incase that was causing it, but that wouldnt do it either. Now I know the repair function can only repair a certain amount of errors, but for these albums i dont think the error count would be that high. I think the CTDB just gets stuck when it doesnt know which offset to use..... which is weird, because if you look at the bold text above, cuetools + AR is able to figure out the right offset. Is there a way of telling the CTDB to use the offset that AR is seeing and then repairing the damaged tracks? This situation doesnt happen in all cases with multiple offsets, but just mostly when there is a large number of offsets or the confidence for track matches are quite similar for each of the offsets
Again, this DB is awsome. If you have the time to populate it with your verified albums then please do so, because everyone benefits in the end.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-24 22:31:41
Plz post the complete cuetools log so that we can understand you with "your offset fixing thing" ... which is chineese to me right now

As far as I understund what you meant: If the rip doesn't get accurate result for all tracks within the same offset it cannot be AR2 hence it cannot be repaired no matter if the tracks are all accurate separately on several different offsets (which happens sometimes). In this case it can be repaired but only if it is really damaged not if the fact that it is not AR2 is due to holes in the AR database.

Without any additionnal clue that there is a real scratch (a bad EAC log) you rip could be perfect & yet not in the AR database ...

I dunno if what I said matches your case but all I say is that you didn't provided enougth information to conclude anything.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-24 22:44:14
AR2?
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-24 22:46:27
Short for Accurate with confidency confidence 2.

Myself:
Quote
If the rip doesn't get accurate result for all tracks within the same offset it cannot be AR2 hence it cannot be repaired no matter if the tracks are all accurate separately on several different offsets (which happens sometimes).

This whole sentence is not really worded correctly anyway ... I say it cannot be repaired & then I say it can  ...

Skybrowser:
Quote
CTDB repair function won't activate and encode my files because it doesnt know which offset to use.

As I tried to explained with bad words, CTDB knows that it knows nothing, so it does nothing.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-24 22:50:13
Some sort of shorthand you've made up, I suppose.  AR2 has already been taken for the next generation of AccurateRip, so I think you might want to come up with something else.

Regardless, we already know that a confidence of one is all you need provided it wasn't your previous submission, barring consistent errors due to manufacturing defect, software bug or firmware bug; in which case a confidence of two is still no guarantee.

EDIT:
The reason I asked is that you gave me the impression that spoon has somehow adopted CTDB as part of AR.  Knowing now that this is not the case, I think I should point out to the OP that AR and CTDB are two different databases, independent of one another.  Just because you might get a hit against the AR database does not mean you'll be able to fix the title with CTDB.  Perhaps this is already known to the OP; I'm not all that familiar with the new lingo since CTDB has not yet been able to help me with a bad rip and because I have very few titles that need to be fixed, I have very little experience with it.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-24 23:02:49
Greynol:
I know all this, but still I use AR2 for safety, because as far as I understund AR1 rip can vanish from the database due to the way it works ... I have noticed this several times ... AR1 seems to be pending rips ... starting at AR2 the rips never vanish from the database (well so far I haven't noticed any AR2+ vanishing from the database. Edit: Outside of purges indeed) ... also I use AR2 as a reference even more now as CTDB accept submission starting at AR2. If I ever find AR2 faulty be sure I'll complain to the boss !

So even if AR1 is enougth if you own the CD, in the end the fact that a rip can be AR1 in April & not in database in May makes you feel that AR1 is not enougth.

I am not sure but I think I already used this shorthand with Gregory & we did understund each other I think, maybe it would be harder with Spoon. But I thought the whole idea behind AR2 (the new database I mean here) was forgotten for now anyway.

Greynol:
Quote
independent of one another
Well CTDB is AR dependent.
There is not much to understand in CTDB. It's either the rip is in database or it's not. Once it is in database it's either it's fixable or not ...
Title: CTDB question: Repair function
Post by: greynol on 2010-05-24 23:17:03
A confidence of one that vanished is really no better than a confidence of one that did not vanish, or any better than a confidence of two provided it was not your previous submission.

Quote
If I ever find AR2 faulty be sure I'll complain to the boss !
You might want to say something other than AR2 or you might confuse him.

Quote
So even if AR1 is enougth if you own the CD, in the end the fact that a rip can be AR1 in April & not in database in May makes you feel that AR1 is not enougth.
Speak for yourself, it doesn't make me feel that way.

Quote
CTDB is AR dependent.
How so exactly?  Must there be a record in the AR database for something to be in the CT database?
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-24 23:24:25
Quote
How so exactly?

Just like you said Greg uses AR2 to accept submission, so no AR no CTDB simply.
Without AR2 there would be no way to dismiss noise (scratched submission) from CTDB.

I think it would technically be possible to create an independent CTDB, by comparing submissions, but it would be hard for Gregory for no gain as AR already does that. To be short if Greg would have made CTDB independent he would have needed to reinvent the wheel ... the wheel being AR.

Edit:
You were faster:
Quote
Must there be a record in the AR database for something to be in the CT database?

Yes AR2 in AR database before being accepted in CTDB ... I learned something to Greynol ! unbelievable

Quote
A confidence of one that vanished is really no better than a confidence of one that did not vanish, or any better than a confidence of two provided it was not your previous submission.

I am not sure that I get what you meant but a confidence of 2 no matter if one is your own "precious" submission is better than a confidence of 1 because a confidence of 1 is almost like no confidence at all for me ... as it can disapear. To trust AR1 you have to keep records of what you submitted to the database which is insane in the long time.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-24 23:27:52
Just like you said Greg uses AR2 to accept submission, so no AR no CTDB simply.

I either didn't say this or have given you the wrong impression as this is not true.  CUEripper will submit to the CT database without an AR entry.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-24 23:28:54
Yes AR2 in AR database before being accepted in CTDB

Actually, no, this is not right; the intention of my question was rhetorical. See my previous response.

I am not sure that I get what you meant but a confidence of 2 no matter if one is your own "precious" submission is better than a confidence of 1 because a confidence of 1 is almost like no confidence at all for me ... as it can disapear. To trust AR1 you have to keep records of what you submitted to the database which is insane in the long time.

It is just a snapshot at any particular moment.  In the long run, a confidence of one that disappeared will eventually re-appear if it was right.  If it was right it was no less right when it disappeared.  If it wasn't your previous submission then it definitely was right.

If you suffered from a drive failure or had some other reason to re-rip your collection after a submission, then I understand.  Otherwise if you know you haven't previously submitted a disc (which is a question that is probably very easy to answer for most people) then there is no reason to throw away a log with a confidence of one to later test it to make sure it got bigger.  In my case it's simple:  I don't believe I've ever submitted to AR.  Furthermore, if I have I can pretty well guarantee that my submissions never made it to the database because I've been banned for a long time now.

So what if you've submitted more than once to the database with a different AR user ID?  Can you still trust a confidence of two? This is a rhetorical question (the answer is no). It's also a legitimate concern as I know at least one person who has and I have to agree with you: keeping a record of what you've submitted and how many times is insanity.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-24 23:36:54
Yes you're right, but I think this is temporary (Edit: Should have been). CTDB only accept CUEripper because Spoon doesn't accept CUEripper results in AR database. Once CUEripper will be mature enougth to be accepted in AR database then AR2 should be the norm.

Maybe Greg will always accept CUEripper results because he trust his own works, but overall the truth is that accepting CUEripper is a sort of trick. An acceptable trick.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-24 23:40:16
I think Gregory would take exception to the idea that it's a trick.

Perhaps you can provide a link to him saying it's just a trick and that he'll make the CT database dependent on the AR database once CUERipper is allowed to submit to the AR database?
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 00:18:13
He never said that, it was just obvious to me.

In this post (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=79882&view=findpost&p=704954) I argue that the CTDB confidence is an artificial number that shouldn't exist if CUERipper results were accepted in the AR database.

Greg didn't reacted to this so I assumed my assumption was right.

I may be completely wrong & Greg may disagree, it wouldn't be the first time that I would be wrong about AR & CTDB.

From a human point of view it is just logic that CTDB accepts CUERipper results, CTDB & CUERipper are time greedy for Greg & the possibility that accepting CUERipper in CTDB might help him to find bugs or even just help him improve CUERipper is the least that we can accept as a sort of "reward" for his job.

If we trust CTDB then we trust Greg's work so trusting him for CUERipper too is the least that we can do, to show him our trust.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 00:39:47
So is the answer as simple as the OP's disc not being in the database, or is it likely something else?  Again, I'm not really all that familiar with all the new logs and whatnot.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 00:50:37
Yes, as far as I understund, it can be as simple as that  As long as he doesn't provide a full cuetools log, I dunno.

I think that he has a log where all tracks are accurate separately somewhere but not on the same offset, so he thinks he can fix it. (When I was an AR beginner I tried to fix manually a rip like that (by applying offset/splitting/joining) which is a strange & funny idea to think about now  Like what the hell was I thinking of !!! Indeed it didn't work & I created a monster) If that's it ... his rip is maybe perfect, but the pressing simply not in the database. Cuetools doesn't want to fix it, as there is no evidence that it is broken.

Maybe it is something else, I dunno.
Title: CTDB question: Repair function
Post by: Skybrowser on 2010-05-25 02:08:13
Ok.... well following that massive discussion between you two wasnt easy, but I think i caught a few things here and there and i'll try to respond.

 - So far i've repaired 7 CDs, and I have about 123 to go..... sooo you can see why I'm quite passionate about this.

 - The situation I am talking about is when the rip is identified in both AR and CTDB with more than 2 confidence.

 - Like Greynol said, your use of AR2 was a tad bit confusing. I am going to assume that every time you used it, you were referring to a rip with AR confidence >2

 - Yes Greynol, CTDB submissions require AR confidence >=2

 - I thought i saw one of you mention something about a bad log file or something. Could my original EAC log files be confusing CT somehow?

 - I will give you an example. I hope it helps. This is an example of one where I didnt fix the offset. But even if i fix the offset, it just says 'cannot be verified' in cuetools for CTDB. As you can see there are 337 CTDB confidence, and around 560 AR confidence.

Code: [Select]
[CUETools log; Date: 24/05/2010 5:27:41 PM; Version: 2.0.9]
[CTDB TOCID: be8vy8P3FbyY5zsvkRW5op85K_o-] found.
        [ CTDBID ] Status
        [4135cf03] (337/337) No match
[AccurateRip ID: 0017deff-00cf9b72-890dd20b] found.
Track  [ CRC    ] Status
 01    [8164b9a6] (000/562) No match
 02    [8d6e86c8] (000/563) No match
 03    [1ad0833b] (000/565) No match
 04    [458425a6] (000/565) No match
 05    [b2b0db89] (000/568) No match
 06    [b87e1bef] (000/562) No match
 07    [c4209825] (000/557) No match
 08    [de139d6c] (000/566) No match
 09    [ab4603ce] (000/563) No match
 10    [49f419f5] (000/558) No match
 11    [71e15541] (000/540) No match
Offsetted by 112:
 01    [4e5b16c2] (039/562) Accurately ripped
 02    [3cf51194] (039/563) Accurately ripped
 03    [c9ba3e97] (039/565) No match but offset
 04    [fc7c30b5] (040/565) Accurately ripped
 05    [a548029a] (040/568) Accurately ripped
 06    [e4ed989c] (038/562) No match but offset
 07    [3f9d7f5d] (039/557) No match but offset
 08    [c37a916d] (039/566) No match but offset
 09    [ac5ad610] (038/563) Accurately ripped
 10    [5ea5e171] (039/558) No match but offset
 11    [cc46d7c1] (039/540) No match but offset

Track Peak [ CRC32  ] [W/O NULL] [  LOG  ]
 --  97.7 [676A08B4] [F6C8BC1A]         
 01  97.7 [04B564BA] [6A33D5F9]  CRC32 
 02  97.7 [3DC55F78] [0296C6D0]  CRC32 
 03  97.7 [AA7422DC] [43358095]  CRC32 
 04  97.7 [0251F8EF] [A91411AA]  CRC32 
 05  97.7 [4FD3CD70] [24E1DABF]  CRC32 
 06  97.7 [43E1D1C6] [E45B33D8]  CRC32 
 07  97.1 [618BD826] [5C35B999]  CRC32 
 08  97.7 [5A9E0514] [23BB1B34]  CRC32 
 09  97.7 [645C41ED] [DF1D3E84]  CRC32 
 10  97.7 [DABC5DF3] [90290BF6]  CRC32 
 11  97.7 [C8636CF9] [8B7161C9]  CRC32
 
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 02:39:28
With your last cuetools log: this rip is not accurate with a very very high confidence so it is not surprising that CTDB cannot fix it ... there is two possibility:

1: Very likely: the rip is not accurate & is scratched beyond repair.
2: Very unlikely: you own a rare pressing that is not in AR & doesn't match CTDB too

The possibility N°2 is still opened because of "Offsetted by 112:", so if you didn't fix the drive offset while ripping & the drive offset is +112, then possibility N°2 is closed & your rip is damaged.

Quote
- I thought i saw one of you mention something about a bad log file or something. Could my original EAC log files be confusing CT somehow?

This is where your EAC log will maybe help you, the records of the drive offset will maybe close possibility N°2. Other than that the EAC log is only usefull when the CD is enhanced & only if the log incluses the TOC.

Also:
Quote
337 CTDB confidence
be warned that as far as I understund this doesn't mean that 337 users submitted to CTDB, but only that the one that was submitted matches with AR at up to 337 accurate rips.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 02:46:05
- Yes Greynol, CTDB submissions require AR confidence >=2

I knew this already, however CTDB submissions from CUERipper don't require any AR confidence, so it is possible for there to be CTDB entry that does not exist in AR (IOW, the two databases are independent). Either way, I'm glad to see this part of your problem has been addressed.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 02:50:58
With your last cuetools log: this rip is not accurate with a very very high confidence
Sorry to nitpick again, but this is misleading.  Confidence applies to something that is accurate, not to something that is not accurate.

2: Very unlikely: you own a rare pressing that is not in AR & doesn't match CTDB too
I don't see how you are able to assess that this is not likely, especially without a rip log.  Where was the CD produced, how many were produced from that run, was the data edited between production runs
(click removal/eq/etc.), where and when was it purchased, how long ago was it released, how many copies were sold?
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 02:53:16
I may be wrong but I think that CTDB still requires an accuraccy of 1 even for CUERipper, there is a AR column in the CTDB page & I don't recall having seen any AR0 ... for me it is logic as an AR1+a CD in CUERipper means that the submission in AR is not from the same guy ... so in fact AR1+CUERipper 1= AR2 ... that's how I understand it.
Title: CTDB question: Repair function
Post by: Skybrowser on 2010-05-25 03:00:31
- My drive offset in logfiles for ripping is 667.

- The reason it says 112 in CT is because its a CDR. Thats why I mentioned fixing the offset earlier, i tried fixing offsets and still had the problem. Now i know what you're thinking, 'the problem is probably that its a bad burn' BUT I have already used CTDB to repair rips from burns that dont look damaged.... bad burns are repairable too and thus are not the problem.

- So how do i know the difference between an album that is unrepairable by CTDB and any other error i'm having? Do you think I should suggest to the developer that he uses different messages? One that notifies you that your album has too many errors to repair and one that says your album is not being recognized by the database or something?
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 03:01:36
Well what is likely or not is only a question of probability, it's like poker if you have 2 aces in the hand you have a big probability to win ... but you can still lose.

Reading non accurate Cuetools log is like that ... with the information that he provided there is actually more chance that his rip is not accurate. If his EAC log confirm the drive offset then it's like drawing an additionnal ace  If you get what I mean

It's a question of habbit. You have to much non-scratched CD
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 03:03:50
Ok I completely forgot that it was a CDR, well for me the rip is not accurate & beyond repair or you have a different pressing I can't tell. The fact that it is a CDR blurs the verdict, still even without certainty this rip is very suspicious IMHO.

Quote
- So how do i know the difference between an album that is unrepairable by CTDB and any other error i'm having? Do you think I should suggest to the developer that he uses different messages? One that notifies you that your album has too many errors to repair and one that says your album is not being recognized by the database or something?

I think CTDB has no way to know if it doesn't match because of pressing difference or because of scratches, hence the message doesn't make any difference. So my guess is that asking Greg for this will not help. I suspect that if it was possible Greg would have been clever enougth to make this difference. ... well I hope
Title: CTDB question: Repair function
Post by: Skybrowser on 2010-05-25 03:41:38
I already have a few encountering this situation. Like is the CTDB only really powerful enough to repair about 1 track? should i base it on how many in-accurate tracks I have on a specific CDR?
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 03:48:58
The ability of CTDB to repair damaged rips is random, the only thing you can do is accepting it. Based on my small experience, so far I would say that if the rip is in CTDB you have 50% (maybe a little less) chance to be able to fix it (note that this estimation is likely very random too). It is (much) better than nothing, just be happy that CTDB exists.

I have already fixed rip with more than hundreds of difference (But it only happened once so far) so it's pure luck. You cannot draw any conclusion from the accuracy of tracks in cuetools log & extrapolate it to the ability of CTDB to fix it.

The ability of CTDB to repair damaged rips seems to rely much more on how many data Greg decide to store in the database. But the more data he stores the bigger & harder to manage the database becomes.
Title: CTDB question: Repair function
Post by: Skybrowser on 2010-05-25 04:18:27
Ah, at first i didnt really realize why you were putting emphasis on different pressings, but I think i understand now. Just because that specific pressing is in AR and the album is in CTDB doesnt mean that specific pressing of the album is in CTDB even though it is in AR. Ok so because CTDB is quite new, i guess i'll have to wait alot longer for it to become more populated. Maybe if someone made a tool that could tell you the number of errors on an album and we could get an error count on what CTDB is able to repair then we could know whether we can fix an album or not.

Alright thanks for all the info guys, i guess i'll just wait it out for now.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 04:37:45
No, as far as I understund how CTDB works, it is designed to be "cross-pressing" or "offset independent" if you prefer, so even if your rip/pressing is not in the AR database, the fact that it doesn't match with CTDB means that you will never be able to fix it with the actual amount of data/strenght of CTDB. In other word your rip can suddenly appear in AR database & it should still not be fixable in CTDB.

In other words, because CTDB doesn't recognize your rip, it is very unlikely that your rip is accurate because the fact that CTDB doesn't recognize your rip cannot be due to a simple offset difference.

This also answer the question of Greynol of how can I know that this rip is unlikely to be accurate.

Warning: All the above rely on the fact that my understanding of CTDB is right, actually I am not 100% sure that CTDB is offset independent. Read this post (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=79882&view=findpost&p=704627) & the whole discussion around (read down to the conclusion (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=79882&view=findpost&p=704636) post #55) to be sure that you understand it like me.

Gregory: (from the linked discussion)
Quote
CTDB stores only one record for all pressings.


As far as I understund CTDB ignore a "big" (by "big" read bigger than any offsets) part of the beginning & end (10 frames) of the CD in order to be offset independent by focusing on the "heart of the CD". That's how I understund it, if I am wrong then the above is wrong.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 08:45:31
This also answer the question of Greynol of how can I know that this rip is unlikely to be accurate.

The fact of the matter is that you don't know; pressings can differ by more than just an offset.

Besides what I explained in my previous post (click removal/EQ/level change/compression/etc.), data can change between pressings as the result of corrupted data.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 08:57:01
for me it is logic as an AR1+a CD in CUERipper means that the submission in AR is not from the same guy

Unless that guy previously ripped that disc with EAC or dBpoweramp and submitted that result to the database.

By that logic, the same goes with non-CUERipper extractions as well.  Your AR results aren't instantly sent to the database, so if you get a confidence of one and you aren't the same guy, your submission would make two, but only after it is incorporated into the database.  This is precisely why a confidence of one is all that matters unless it's your own previous submission.  Now even if it was your own previous submission, the fact that you got the same result a second time gives you the same level of security as if you had used test and copy and got matching checksums.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 11:05:47
Quote
pressings can differ by more than just an offset.

I know it, pressings that differs by more than just offset often have another ID.

But even if it has the same ID:
I don't know exactly how the CTDB TOCID is calculated compared to the AccurateRip ID but I know that
1: Greg stated several times that:
Quote
Different pressings of the same CD are treated as the same disc by the database, it doesn't care.
from CTDB's about.
He doesn't seems to make any difference between pressing that differs by just offset & others.

2: CTDB seems to be able to deal with "pressings that differs by more than just offset" by storing a 2nd checksum in case it detects a 2nd  "pressings that differs by more than just offset"

I may be completely wrong on 2, but I recall I already saw a rip with 2 CTDBID for 1 CTDB TOCID.
I don't recall exactly but I think I saw this on an enhanced CD.

So anyway it seems CTDB knows how to avoid clash & deal with "pressings that differs by more than just an offset" inside the same CTDB TOCID. As far as I understund CTDB uses two ID specially to deal with "pressings that differs by more than just an offset".

Quote
Unless that guy previously ripped that disc with EAC or dBpoweramp and submitted that result to the database.

If the scratch is real, the results are not supposed to match anyway. By accepting AR1 rips from CUERipper, Greg only exploits this. CTDB knows the results in AR cannot be the same as the actual CD in CUERipper as CUERipper cannot submit to AR. CTDB also knows that if there were a real scratch it shouldn't match with the AR1 results anyway. So it concludes the rip is AR2 even if not actually AR2 in the AR database. It's a trick but it's logic.

I never said that AR1 wasn't enought in some case, I said that personnaly I don't want to bother with keeping record of rip that were already AR1 when I ripped them. Specially as I don't check AR while ripping (I delete EAC logs) but afterward (by keeping cuetools logs).

Edit:
I searched my enhanced CD for an example of two CTDBID inside the same CTDB TOCID, but I noticed that the CTDBID is in fact same ... so I don't have any example that would proves my claim, but I suspect it works this way.

All I could find are examples like this. Which doesn't really illustrate what I suspect.
Code: [Select]
[Verification date: 11/05/2010 16:06:17]
[AccurateRip ID: 000e88eb-0066551d-6c10b009] found.
CD-Extra data track length 18:07:07.
[CTDB TOCID: zmFGQm66ZqbXROW4CQDK4zA7T0s-] found.
        [ CTDBID ] Status
        [8217dca2] (240/249) Accurately ripped
        [8217dca2] (009/249) , Accurately ripped


Code: [Select]
[Verification date: 11/05/2010 16:07:20]
[AccurateRip ID: 000f43d8-00696936-770f5a09] found.
CD-Extra data track length 18:56:32.
[CTDB TOCID: LUb_4tK_Z4HQ0HbzmnB2HcDeoIc-] found.
        [ CTDBID ] Status
        [0ea56824] (241/252) Accurately ripped
        [0ea56824] (011/252) , Accurately ripped


Still a mix of CTDBID+the numbers in parenthesis seems to be used as an additionnal ID, to keep record of both the enhanced & non enhanced pressings. I am only guessing that the same kind of trick is used in case of pressings that differs by more than just offset.

Actually I think that the CTDB is too small for me to find an exemple, but time will tell if my intuition is right. I don't deny that I can be wrong, CTDB is still very young & it's hard to get some reliable knowledge about it as it is spreaded among plenty of posts (& personnaly also as I am not a native english speaker.)
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 17:53:12
As I couldn't found any rip with 2 CTDBID for 1 CTDB TOCID in my own collection, I had the idea of searching within the most popular releases in the CTDB homepage by sorting CTDB TOCID, I quickly found this link (http://db.cuetools.net/top.php?tocid=y0QUep3Mm6JaatHNdQj5f68zZls-) & this link (http://db.cuetools.net/top.php?tocid=bIDb5Sr8E4SNN6zhjCdr6IaRWOs-) & many more, so unless someone disagree, it seems to back up what I think.

So to answer Skybrowser, the only possibility I see for his rip to be accurate & not match CTDB is that he owns a rare pressing that differs by more then just the offset & is not in AR database in the same time. It is possible but unlikely.

The only thing that remains to be analysed are the numbers in parenthesis, (337/337) in CTDB vs. (039/540) in AR ... I am not very familiar with the CTDB numbers but I will have a look.

Edit:
The explanation of these CTDB numbers is here (http://www.hydrogenaudio.org/forums/index.php?showtopic=79882&st=50&p=704950&#entry704950)

I am not sure how to interpret these numbers but from what I understand the huge difference between 337 & 39, seems to be a positive hint about the fact that you might have a different pressing that differs by more than the offset ... but again I am 100% unsure about how to read these numbers.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 18:20:51
So anyway it seems CTDB knows how to avoid clash & deal with "pressings that differs by more than just an offset" inside the same CTDB TOCID.
I think the point is escaping you.  This is about knowing for certain whether a rip with tracks that don't match are not accurate.

I know it, pressings that differs by more than just offset often have another ID.
I never said it didn't.  I just said that there can be differences between different pressings of the same title that differ by more than just an offset.  The differences can actually be so minor that the the database might fix one to match the other, despite the fact that both are accurate and it could be that what was being fixed might be the one that is more desirable.

2: CTDB seems to be able to deal with "pressings that differs by more than just offset" by storing a 2nd checksum in case it detects a 2nd  "pressings that differs by more than just offset"
Sure, and I see no reason why it can't or why it shouldn't.  AR had this ability already, in the case of CT, there's the the additional data for correction.

Quote
Unless that guy previously ripped that disc with EAC or dBpoweramp and submitted that result to the database.
If the scratch is real, the results are not supposed to match anyway.
Why even care about AR since test and copy would handle results that do not match because of a scratch?  The fact of the matter is that errors resulting from a scratch (or some other problem that might not be due to a scratch!) can be consistent.

CTDB knows the results in AR cannot be the same as the actual CD in CUERipper as CUERipper cannot submit to AR.
CUERipper has absolutely no way of knowing if that particular physical CD was previously ripped with EAC or dBpoweramp and the results were submitted to the AR database.  If it does, please tell me how.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 18:28:37
Quote
CUERipper has absolutely no way of knowing if that particular physical CD was previously ripped with EAC or dBpoweramp and the results were submitted to the AR database.

I agree with this but as CTDB knows that even in this case the result cannot be the same, it doesn't matter, if it matches with the AR1 result, then CTDB considers it as if it was AR2, if it doesn't match, then CTDB ignore it as noise.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 18:30:28
So to answer Skybrowser, the only possibility I see for his rip to be accurate & not match CTDB is that he owns a rare pressing that differs by more then just the offset & is not in AR database in the same time. It is possible but unlikely.
I want to see how you arrived at the unlikely part, sauvage78.  The pressing may only be "rare" in the context of AR, which as plenty of people tell you does not contain information about every disc ever produced.  When U2 releases a CD and sells millions of copies on the first day, would you call it rare because there are no results in the AR database the next day if not the next week?

I am not sure how to interpret these numbers but from what I understand the huge difference between 337 & 39, seems to be a positive hint about the fact that you might have a different pressing that differs by more than the offset ... but again I am 100% unsure about how to read these numbers.
I'm pretty sure this is not correct.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 18:32:52
I agree with this but as CTDB knows that even in this case the result cannot be the same, it doesn't matter, if it matches with the AR1 result, then CTDB considers it as if it was AR2, if it doesn't match, then CTDB ignore it as noise.

Ignoring that the result can actually be the same but yet still be in error, can you provide a link to where you learned this "fact" about what CTDB considers and what it ignores?
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-25 18:37:14
As I said you may disagree with the "unlikely" conclusion, here this is a feeling based on experience, I know this is irrationnal to you, but it is like poker: the more you play, the more you know the probability of your hand to win. I update a folder full of 140go of non accurate rips every months the numbers of rips which suddenly become accurate is very low.

Quote
I'm pretty sure this is not correct.

Me too but I specially posted this for people to teach me what to do with these numbers !

Quote
Ignoring that the result can actually be the same but yet still be in error

Unless the guy is stupid & rerip with cueripper a scratched CDR he submitted to AR previously with EAC or dbpoweramp, the data cannot match because as I explained scratched data cannot match between two different rips. No I don't have a link sorry. I don't know how to explain better.

When you rip a commercial CD you own with EAC & that you match with an AR1 submission that is not yours, you know that the CD is not scratched. I know you know this. I don't understand why you don't admit that the same happens with cueripper & AR1. It's the same.

Again it seems only Greg can save my soul
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 18:50:00
As I said you may disagree with the "unlikely" conclusion, here this is a feeling based on experience, I know this is irrationnal to you, but it is like poker: the more you play, the more you know the probability of your hand to win. I update a folder full of 140go of non accurate rips every months the numbers of rips which suddenly become accurate is very low.

This is not so much about rationality.  As I said to you before, you don't seem to be aware of the bigger picture when you're presenting something as if it were a fact.

Backing off from this, I would simply say if I were in the OP's situation, rips without a log, especially those from CD-R to me would be suspected bad unless proven otherwise.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-25 19:08:38
Unless the guy is stupid & rerip with cueripper a scratched CDR he submitted to AR previously with EAC or dbpoweramp
or the data used to burn the CD-R was in error.

Quote
the data cannot match because as I explained scratched data cannot match between two different rips.
Provided you mean rips from two different physical CDs, the odds of the same exact error from a unique scratch IMO cannot possibly be significant, I suspect they're even lower than the odds of an AR hash collision which, assuming the analysis that is often presented is correct, is on the order of one in a few billion.

Quote
When you rip a commercial CD you own with EAC & that you match with an AR1 submission that is not yours, you know that the CD is not scratched.
I know if a CD is scratched by looking at it, not by what the AR numbers are.  As I've tried to tell you there are more reasons for errors than just a scratch, but I digress.

Quote
I don't understand why you don't admit that the same happens with cueripper & AR1.
Actually what I'm asking of you is where you're getting this impression that CUERipper does something unique with results with an AR confidence of one.  I also want to revisit whether CUERipper submits information to CTDB when there is no AR record, there is an AR record but it does not provide a match unless an offset is applied within a given range, and when there is an AR record but it doesn't provide a match and cannot provide a match when an offset is applied within a given range.
Title: CTDB question: Repair function
Post by: Skybrowser on 2010-05-26 22:55:35
Ok, I think i've finally gotten lost in you guys' discussion. My original question was basically about why CTDB wouldnt repair certain CDs, and how do i know if a CD is beyond repair or if something else is wrong and CTDB is confused by my CDRs.

But I picked up on something one of you said in an earlier reply and just today i came across an example of it. I think one of you said that cueripper automatically adds albums to the CTDB or something to that effect. 
    - Well just today i decided to try using cueripper to rip an album and just play around with it. After ripping the album it said rip in-accurate AR 0/13. CTDB not in database. Now i've had this same result for this album using EAC. Track 1 simply won't verify. This IS NOT a CDR. The Test and Copy CRC's matched, and the tracked quality showed 100% ... which doesnt necessarily mean anything because cueripper was on paranoid mode and will do ALOT of retries.  Cueripper said it submitted my data to CTDB...... even though my rip was in-accurate according to AR confidence 13. I thought this odd so i went into cuetools and did a verify ... and low and behold there was now a CTDB confidence of 1 for my album. Now maybe CTDB is smarter than I am, and I really do have a different pressing of this album. I actually burned this album years ago, and I cant get the first track on the burn to verify either.... so that could be evidence for a different pressing.... unless the original was damaged before i burned it.

I'm thinking of trying the same thing with a couple of other albums i'm having problems with. I'll let you know what happens if i do.
Title: CTDB question: Repair function
Post by: Teknojnky on 2010-05-26 23:04:19
After ripping the album it said rip in-accurate AR 0/13. CTDB not in database.



what this means is very simple.

there are 13 entries in accurate rip
your rip does not match any them
there are no entries in the CTDB
since yours does not match AR, then it will not add to CTDB

that is all.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-26 23:14:08
since yours does not match AR, then it will not add to CTDB

low and behold there was now a CTDB confidence of 1 for my album.

Teknojnky, sauvage78:
Please post evidence for your conclusions about what gets added to CTDB.  It would appear that your assumptions are erroneous.
Title: CTDB question: Repair function
Post by: Teknojnky on 2010-05-26 23:18:19
cuetools ripper can add to CTDB without the aid of AR

in order for cuetools to add to CTDB, it requires accurate rip to have a confidence of 2, and your rip has to match AR

I thought all this was obvious, I can't fathom that it took this long in the thread for someone to point it out.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-26 23:20:14
There is a reasonnable chance that this new rip is ok. Just wait for it to appear in AR.

As for why this non-accuraterip was accepted in CTDB, I dunno. I thought this kind of rip would have been rejected so maybe Greynol is right.

In case CTDB accept non accurate rip (I never tried by myself), my guess is that Greg store the necessary info in order to re-test the accuracy of your rip later. My guess is that maybe Greg does that in order to help populate the database as long as it is small & easy to backup. Later when the database will become too big, Greg will re-check these rip against the AR database & purge those which are definitely not AR. In the process some CTDB submissions that would otherwise have been rejected might be saved. This is the only clever explanation I see.

If it doesn't work like that then Greg is polluting his own database. Ask Greg, I think he is the only one who can answer what he is doing with his non accurate CTDB entry.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-26 23:24:29
Since neither of you could be bothered to look up some facts and have instead decided to fabricate your own, I've done some legwork for you:
Discs don't have to pass AR before being added to the CTDB, AR is used only as a kind of proof that there is a physical CD with such content when adding with CUETools.
CD Rippers can add CDs to CTDB even if AR doesn't know them. There is already a number of CDs in database submitted by CUERipper (http://db.cuetools.net/?agent=CUERipper), some of them have confidence 1 - that means they didn't pass AR check or weren't found in AR.

In case it isn't obvious, I am quite annoyed by the misinformation!
Title: CTDB question: Repair function
Post by: Teknojnky on 2010-05-26 23:24:38

It the same situation as accurate rip confidence of 1, when it was your own submission.

Title: CTDB question: Repair function
Post by: greynol on 2010-05-26 23:25:39
I thought all this was obvious, I can't fathom that it took this long in the thread for someone to point it out.

Yes and it was I who pointed it out in the ninth post in this discussion.
Title: CTDB question: Repair function
Post by: Teknojnky on 2010-05-26 23:28:56
like I said,

cue ripper can add to CTDB without AR

cue tools will not add to CTDB unless a) the rip is in AR, b) there is more than 1 confidence, c) your rip matches AR
Title: CTDB question: Repair function
Post by: greynol on 2010-05-26 23:34:34
Yes, thank you.  Unfortunately you failed to provide the correct information in your first reply to this discussion.  Skybrowser was talking about CUERipper, not CUETools.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-26 23:38:19
Well then this is a flaw of CTDB, I thought Greg wouldn't have made this misstake.

Quote
It the same situation as accurate rip confidence of 1, when it was your own submission.


Yes, the problem is that an AR1 rip is in fact not accurate if it's your own submission. I keep a folder of 100go of AR1 rips, every months some of them become "not present in database". This seems to be due to the way AR works if the second submitter disagree with the first, then both disapear as long as a 3rd one come up & agree with one of the 2 first, then it becomes AR2. If it's your own AR1 submission then you know nothing ...

Keeping those AR1 rips which may not be accurate is not the behavior I would have expected from CTDB, it only creates noise. Some of this noise may be good but some will certainly be bad.

I'd like Greg to purge these entries from CTDB  just like Spoon purged virtual drive from AR.
Title: CTDB question: Repair function
Post by: Teknojnky on 2010-05-26 23:42:56
I suggest you reference them as AR (1) and AR (2) to avoid confusion between accuraterip version 1 and version 2.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-26 23:46:05
Yes, the problem is that an AR1 rip is in fact not accurate if it's your own submission.
The fact that someone submits something to the database and it comes back later as a match means the rip was not accurate?

Well then this is a flaw of CTDB, I thought Greg wouldn't have made this misstake.
No, the mistake was clearly yours.  Unfortunately you seem to be unable to admit it.

Keeping those AR1 rips which may not be accurate is not the behavior I would have expected from CTDB, it only creates noise.
I find this somewhat ironic since there has been a lot of noise in this discussion.
Title: CTDB question: Repair function
Post by: sauvage78 on 2010-05-26 23:46:40
I like the idea I will use AR(2) from now on.

Edit:
Quote
The fact that someone submits something to the database and it comes back later as a match means the rip was not accurate? wacko.gif

well you know nothing except that your submission matches your submission, you don't know if it's accurate.

Quote
Unfortunately you seem to be unable to admit it.

Well I never said I was right so why would I admit that I was wrong ? I used plenty "maybe" "it seems" " "I think" ... I even used "warning" ... I would rather say that you always try to be right & have the last word ... this is a very inquisitive & annoying behavior of yours ... but I get used to it because I usually learn from discussion with you. All I tried to do was to help the topic starter to the best of my knowledge, as imperfect as it might be.

Sorry Skybrowser,
I was asked in pm by Greynol to quit this discussion because I was spreading missinformation. Having already been almost banned by Greynol I know that this means that if I keep answering I will get warned.

I don't want to upset him so I quit.
Title: CTDB question: Repair function
Post by: Skybrowser on 2010-05-26 23:56:10
Greynol, I wasn't attempting to fabricate any facts. I was merely stating observations and asking questions so that I can learn. It was not my intent to annoy anyone, and I do my best to look up information on my own, but if i don't know where it is..... i don't know where it is.
Title: CTDB question: Repair function
Post by: greynol on 2010-05-26 23:58:51
My comment wasn't directed at you.  It was at sauvage78 and then Teknojnky (though it seems he's just guilty of not following along very closely).
Title: CTDB question: Repair function
Post by: greynol on 2010-05-27 00:05:35
well you know nothing except that your submission matches your submission, you don't know if it's accurate.
That's a whole lot different than saying it's not accurate, now isn't it?

Quote
I used plenty "maybe" "it seems" " "I think" ... I even used "warning"
Please don't make me count the number of times you failed to qualify your "facts" as being less than factual.  I would not appear to need to get the last word in if you would stop making stuff up and then presenting it as fact.  Once called out you turn around and blame the developer of making a mistake!

While this probably makes pretty good reading for those on the sidelines, we could have avoided this whole mess if you wouldn't make stuff up with what is practically every new post.
Title: CTDB question: Repair function
Post by: Gregory S. Chudov on 2010-05-27 21:22:48
Back to original topic:

Some of the albums I want to repair are showing up in the CTDB but when I attempt to verify them, it says for example " AR: offset 112, rip not accurate (0/116), CTDB: could not be verified ".


Your rip isn't accurate according to AR database. No matter what offset do you apply, it will still be inaccurate, so there's no point in it. It could be repairable with CTDB if there were less errors. Offset doesn't matter for CTDB either.

The log that you quoted shows errors in almost all tracks, that's the reason CTDB can't identify it. As explained in http://db.cuetools.net/about.php (http://db.cuetools.net/about.php), CTDB can fix only up to 40 damaged sectors, and sometimes can fail to fix even 4 damaged sectors.