Hydrogenaudio Forums

CD-R and Audio Hardware => CUETools => Topic started by: Eli on 2010-03-31 02:27:26

Title: CUETools DB
Post by: Eli on 2010-03-31 02:27:26
I have only now become aware of Gregory S. Chudov's effort to develop CTDB (CUETools DB). I am very excited about this as I have actually been suggesting this to spoon (dbpoweramp's developer for over 5 years)

http://db.cuetools.net/about.php (http://db.cuetools.net/about.php)

for others that have missed it:
Quote
What's it for?
You probably heard about AccurateRip, a wonderfull database of CD rip checksums, which helps you make sure your CD rip is an exact copy of original CD. What it can tell you is how many other people got the same data when copying this CD. CUETools Database is an extension of this idea.
What are the advantages?

    * The most important feature is the ability not only to detect, but also correct small amounts of errors that occured in the ripping process.
    * It's free of the offset problems. You don't even need to set up offset correction for your CD drive to be able to verify and what's more important, submit rips to the database. Different pressings of the same CD are treated as the same disc by the database, it doesn't care.
    * Verification results are easier to deal with. There are exactly three possible outcomes: rip is correct, rip contains correctable errors, rip is unknown (or contains errors beyond repair).
    * If there's a match, you can be certain it's really a match, because in addition to recovery record database uses a well-known CRC32 checksum of the whole CD image (except for 10*588 offset samples in the first and last seconds of the disc). This checksum is used as a rip ID in CTDB.

What are the downsides and limitations?

    * CUETools DB doesn't bother with tracks. Your rip as a whole is either good/correctable, or it isn't. If one of the tracks is damaged beyound repair, CTDB cannot tell which one.
    * If your rip contains errors, verification/correction process will involve downloading about 200kb of data, which is much more than it takes for AccurateRp.
    * Verification process is slower than with AR.
    * Database was just born and at the moment contains much less CDs than AR.

How many errors can a rip contain and still be repairable?

    * That depends. The best case scenario is when there's one continuous damaged area up to 30-40 sectors (about half a second) long.
    * The worst case scenario is 4 non-continuous damaged sectors in (very) unlucky positions.

What information does the database contain per each submission?

    * CD TOC (Table Of Contents), i.e. length of every track.
    * Offset-finding checksum, i.e. small (16 byte) recovery record for a set of samples throughout the CD, which allows to detect the offset difference between the rip in database and your rip, even if your rip contains some errors.
    * CRC32 of the whole disc (except for some leadin/leadout samples).
    * Submission date, artist, title.
    * 180kb recovery record, which is stored separately and accessed only when verifying a broken rip or repairing it.
Title: CUETools DB
Post by: Eli on 2010-03-31 02:32:07
For those that would like to see this added to dBpoweramp go here:

http://forum.dbpoweramp.com/showthread.php?p=98346#post98346 (http://forum.dbpoweramp.com/showthread.php?p=98346#post98346)

I am not sure what Chudov's policy will be on letting others access the database, but hopefully its as open as AR.
Title: CUETools DB
Post by: Eli on 2010-03-31 17:50:42
Chudov,
What type of error correction are you using?

How did you decide on 180kb? It would be interesting to gather data and figure how much an average damaged disc is missing and where the sweet spot would be for recovery.

Any thought of adding more info to the database? What about CRC of every 1vs10vs50 mbs of info as well as the whole disc CRC? This would allow better identification of where damage is.

Do discs have to pass AR before being added to the CTDB?
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-03-31 18:46:22
Of course CTDB is open, and the code required to use it is LGPLed as all CUETools libraries. The only problem is it's in C#, i wonder if i will have to provide a .dll with C interface at some point. The algorithm is not very simple, there's quite a lot of code.

The basic algorithm is Reed-Solomon (http://en.wikipedia.org/wiki/Reed-Solomon) code on 16-bit words. Unfortunately, 32-bit Reed-Solomon is extremely slow, and 16-bit Reed-Solomon can be used directly only on chunks of up to 64k words == 128kbytes. So i have to process the file as a matrix with rows of 10 sectors (5880 samples == 11760 words/columns). Such matrix has up to ~30000 rows for a 70 minute CD, so i can use 16-bit Reed-Solomon for each column independently. Using the same notation as in wikipedia it's a (65536,65528) code, which produces 8 words for each column. So the total size is 8*11760*16bit = 188160 bytes.

N-word recovery record can detect and correct up to N/2 erroneous words, so this 8-word recovery record can detect up to 4 errors in each column. N cannot be much smaller, but it also cannot be much larger, because processing time grows proportionally to N, so N=8 was chosen as the highest value which is still "fast enough" - close to FLAC decoding speed.

Row size doesn't have such impact on performance, so it can be easily extended in the future, so that popular CDs can have larger recovery records. Current size was chosen so that if database contained as many entries as AccurateRip, it would fit on a 1TB drive. I also took into account that making records larger only helps in best-case scenario when the damage is sequential (scratches etc). When damage occurs at random points, fixing it requires larger N, not larger row size, but this has a performance impact. So the current record size was chosen to be more or less balanced.

Is there a point in better identification of where the damage is, when the database is unable to fix it?

Discs don't have to pass AR before being added to the CTDB, AR is used only as a kind of proof that there is a physical CD with such content when adding with CUETools.
CD Rippers can add CDs to CTDB even if AR doesn't know them. There is already a number of CDs in database submitted by CUERipper (http://db.cuetools.net/?agent=CUERipper), some of them have confidence 1 - that means they didn't pass AR check or weren't found in AR.
Title: CUETools DB
Post by: Eli on 2010-03-31 19:33:33
Is there a point in better identification of where the damage is, when the database is unable to fix it?


Not for RS repair, however for the ripper, this would allow re-ripping of the part of the disc where CRCs do not match and therefore are the problem areas.

Quote
Discs don't have to pass AR before being added to the CTDB, AR is used only as a kind of proof that there is a physical CD with such content when adding with CUETools.
CD Rippers can add CDs to CTDB even if AR doesn't know them. There is already a number of CDs in database submitted by CUERipper (http://db.cuetools.net/?agent=CUERipper), some of them have confidence 1 - that means they didn't pass AR check or weren't found in AR.


My reason for suggesting that the DB should only include AR confirmed discs is to verify that the correction data will fix a disc to the correct state. Also, it may help limit the size of the database by only adding correct discs.

Quote
Row size doesn't have such impact on performance, so it can be easily extended in the future, so that popular CDs can have larger recovery records.


I would argue that less popular discs may warrant more data as they may be less replaceable...

How is meta-data handled in your database since this info is also saved?
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-04-05 12:15:22
Not for RS repair, however for the ripper, this would allow re-ripping of the part of the disc where CRCs do not match and therefore are the problem areas.

This information will only be useful for rippers specially designed for it. I'm hoping very much that such rippers as EAC will support CTDB at some point, but i doubt that this support will go beyond simple verification/submission. Besides, ripper usually knows where the problem areas are (using C2 error pointers and comparing results from several passes).

My reason for suggesting that the DB should only include AR confirmed discs is to verify that the correction data will fix a disc to the correct state. Also, it may help limit the size of the database by only adding correct discs.

You can never be sure that correction data will fix a disc to the correct state. As with AccurateRip, all you can be sure of is that a certain number of submissions have the same data. If you rip a CD with EAC and there were errors, your incorrect rip will appear in AccurateRip database at some point, after that you can submit your incorrect rip to CTDB (if it accepts rips with confidence 1). Besides, there are CDs that are absent in AR database, and i want CTDB to be able to handle them.
As for the impact on database size, we will have to see how it goes. Maybe at some point i will have to do periodic purges of old unconfirmed submissions (with confidence 1).

I would argue that less popular discs may warrant more data as they may be less replaceable...

There are a lot more unpopular CDs than popular ones, so we can double the amount of data stored for CDs with > 100 submissions and the database will only grow by 10%.

How is meta-data handled in your database since this info is also saved?

For now it only keeps artist/title information. CTDB server also has a replicated MusicBrainz database clone. For the moment all this is used only in web interface, which helps manage the database. I plan to improve integration with MusicBrainz when the next version of MusicBrainz database schema comes out. Would be nice to fetch all the necessary data in one request to a single server. CUERipper currently has to contact 4 different databases (AR, CTDB, MuscBrainz, FreeDB), which can sometimes take a lot of time.
Title: CUETools DB
Post by: Eli on 2010-04-05 18:35:53
Gregory,
Thanks for the reply.

Any thought of changing to a track based system? Or length of disc/track based system?

I would think other developers would be more likely to develop with this in mind if it could correct single tracks. This would allow for a burst rip, check against AR and if not accurate, a repair with CTDB.

With the current setup, I guess the best workflow would be:

- burst rip entire discs
- check all tracks against AR
- if not all accurate, check number of C2 errors
- if errors are small enough correct with CTDB
- if not, go into re-rip/secure rip mode and try to recover errors
- if some errors are recovered but not all, try to repair again with CTDB


Also, have you thought of keeping track of how many errors are in the "average" disc to get a better idea of how much error recover to keep in the DB.
Title: CUETools DB
Post by: krafty on 2010-04-05 19:09:05
In the future, WILL THERE BE something that may detect by disc ID, which catalogue number & country a pressing actually is.

Example:
Disc ID 9F0BDB0D - the following pressings are a match of each other:
Jean Michel Jarre, Téo & Téa, CAT# 2561699766 Country EU
Jean Michel Jarre, Téo & Téa, CAT# 4607173157591 Country Russia

Or is this kind of thing is way too distant?
Title: CUETools DB
Post by: greynol on 2010-04-05 19:32:25
As I said elsewhere, consolidation is a good thing.  If the audio data stream is identical, it shouldn't matter that the pressing is, even if the pressings have different TOCs.

There is a problem however.  Some pressings may differ by more than an offset, TOC or even different amounts of non-null samples at the edges of the disc; some pressings actually have transfer errors that exist on the glass master.  We certainly don't have CUETools "correcting" a track that was ripped correctly that has no audible glitch with data that came from a later generation pressing that has an audible glitch.  If I had a pressing ripped accurately with an audible glitch, I would personally prefer that it got corrected using data from a different pressing removing that glitch.  I don't suspect that there's an easy solution to this, but I think people should be aware that this is a real phenomenon.
Title: CUETools DB
Post by: krafty on 2010-04-06 00:15:29
Well said, and if there was a way to identify these defective pressings, by CAT# and country, it should be easier to avoid them or mark them as such. It would take one step of further work from a development point-of-view, nonetheless very useful and informative.
Title: CUETools DB
Post by: Eli on 2010-04-06 02:04:47
A tool to scan audio for clicks/pops characteristic of scratches or DRM would be a useful tool. Scanning rips, since most people can't/don't listen to a rip to check it, would be very helpful.
Title: CUETools DB
Post by: Eli on 2010-04-08 00:11:21
dBpoweramp forum CUETools DB thread (http://forum.dbpoweramp.com/showthread.php?t=20915)
Quote
The fact it is not track based is a real issue, to make it track based it would have to store 10x more correction data, which would make it un-practical.


Would it be reasonable to consider adding smaller track based correction files to the database? I'm not sure how large they would have to be to make any sense. Given the computational time required to do a correction, this decreases with file size, and therefor would be cut down significantly by repairing a track vs a disc. Having smaller tracked based correction files would increase the DB size, but not as much as if each correction file were as large as the entire disc correction file. This is where knowing how large an average un-recoverable error is would really help.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-04-16 22:03:24
Also, have you thought of keeping track of how many errors are in the "average" disc to get a better idea of how much error recover to keep in the DB.

In the "average" disc there are no errors  I don't see how to gather reliable statistics on this. It's possible to gather statistics of actual repairs done, but that will only tell us the number of errors when it's small enough to be correctable. There's no way to tell how many errors there were if rip cannot be repaired.

In the future, WILL THERE BE something that may detect by disc ID, which catalogue number & country a pressing actually is.

Next version of Musicbrainz will keep track of TOC <=> release relationships. CTDB could improve on that using not only TOC, but offset and actual audio data, however there are very few discs which have the same TOC with different audio data, and offset is not that important in my opinion. I hope Musicbrainz will do the job just fine.

We certainly don't have CUETools "correcting" a track that was ripped correctly that has no audible glitch with data that came from a later generation pressing that has an audible glitch.

Yes, this is a good example of why CTDB repair be used very carefully. If it ain't broken, don't fix it.

A tool to scan audio for clicks/pops characteristic of scratches or DRM would be a useful tool. Scanning rips, since most people can't/don't listen to a rip to check it, would be very helpful.

I'm afraid i don't see any realistic ways to do this.

Quote
The fact it is not track based is a real issue, to make it track based it would have to store 10x more correction data, which would make it un-practical.

Would it be reasonable to consider adding smaller track based correction files to the database?

Mr. Spoon is right. Average CD consists of 8-9 tracks, and each track requires the same amount of correction data as the whole disc. Making correction data 10 times smaller will make it useless, it won't be able to fix any significant glitch. And keeping the same amount of correction data for each track would make database too large. If we can allow for 10 times more database space, we could instead make larger correction records for the whole disc.

Besides, CTDB is mostly aimed at CD archiving, making sure you have the exact copy of your CDs on your HD. If you rip one track from a CD, it's much less important to have a bit-exact copy.
Title: CUETools DB
Post by: Eli on 2010-04-19 23:03:19
Quote
The fact it is not track based is a real issue, to make it track based it would have to store 10x more correction data, which would make it un-practical.

Would it be reasonable to consider adding smaller track based correction files to the database?

Mr. Spoon is right. Average CD consists of 8-9 tracks, and each track requires the same amount of correction data as the whole disc. Making correction data 10 times smaller will make it useless, it won't be able to fix any significant glitch. And keeping the same amount of correction data for each track would make database too large. If we can allow for 10 times more database space, we could instead make larger correction records for the whole disc.

Besides, CTDB is mostly aimed at CD archiving, making sure you have the exact copy of your CDs on your HD. If you rip one track from a CD, it's much less important to have a bit-exact copy.


Thanks for the replies. I understand that a track based approach would require a much larger DB.

I can't speak for spoon, but the way I read his comment was that

1. Most of his users rip tracks, not discs
2. If they can't rip just one track, the feature is worthless to them
3. If its not useful to his large customer base he may be less interested in supporting CTDB

I know you don't benefit from having CTDB support in dBpoweramp, but I would very much like to see this.

When I originally proposed the idea to spoon I had suggested/recommended a distributed p2p storage of reed-solomon recovery files. Would it be feasible to keep a central database and point to torrents for the R-S recovery files (not audio)? This would allow for track based files without a serious concern over the size of the recovery files. It would even be possible to allow for 5+ second worth of repair data per track

Title: CUETools DB
Post by: Gregory S. Chudov on 2010-04-20 11:06:25
We all benefit from CTDB support in as many applications as possible, because it's usefulness depends on how many people submit their results to it.

I have no statistics to prove it, but i assume that the vast majority of CD rips made with dbpoweramp or any other software are rips of the whole CD, even if ripped to separate files.

Distributed storage is definitely an interesting idea and it's worth to consider it, but it has many problems.
First, it would make CTDB software too complicated. Developers are much more likely to support something simple and straightforward which doesn't require that much code.
Second, recovery files for rare CDs that were submitted only by one or two people will be unavailable most of the time.
Third, how do we convince people not only to submit their data to the database, but to permanently allocate disk space and bandwidth for their submissions?
What happens when someone's HDD crashes? We loose his submissions?
Title: CUETools DB
Post by: Eli on 2010-04-20 14:34:37
spoon has said in the past that the majority of his users only rip a couple of tracks. We see a very biased user base here at HA. Even the active users on the dBpoweramp forums represent a biased user base.

Clearly a distributed storage solution is not ideal. Maybe a hybrid solution would be best. A central DB where every recovery file is uploaded would be the only solution to loosing disc space. Enforcing a certain ratio would be the only way to encourage uptime and shares. A system where users allocate disc space and the central system fills that space would more evenly distribute and preserve the data.

Of course storage continues to get cheaper and the projected size of the database will only occur after a matter of years, at which time storage will be even cheaper.

What about selling access to the repair files to pay for the cost of storage, bandwith and maintaining the database?
Title: CUETools DB
Post by: Eli on 2010-04-20 15:50:33
After my last post I thought about this some more. I really think the best think may be a paid solution, similar to the one dBpoweramp users for premium meta-data

1. non-commercial users pay a $3-5 annual fee to access repair files
2. commercial users (users using batch ripper) pay ~$0.05/repair file

This may make it cost effective to have a large, track based database without the distributed storage.
Title: CUETools DB
Post by: zfox on 2010-04-20 16:03:26
After my last post I thought about this some more.


No, you shouldn't. We only deal with FOSS in this case. At least until now.

Quote
License:
GNU General Public License (GPL), GNU Library or Lesser General Public License (LGPL)
Title: CUETools DB
Post by: Eli on 2010-04-20 17:25:13
Wouldn't change the software license at all. There would just be a cost to access the database. Besides, doesn't matter what I say. Just an idea/suggestion.

Could offer current full disc repair size for free and track based with a charge.
Title: CUETools DB
Post by: zfox on 2010-04-20 18:05:40
That can be feasible even with GPL (dependency through the network with a character oriented protocol), but a lot of users (count me in) may stop submitting recovery files to a proprietary base if there is no free and full access by 3rd party apps.  AccurateRip allows free access to 3rd party software.
Title: CUETools DB
Post by: Eli on 2010-04-20 18:55:37
NP, I am sure you, zfox, would not mind providing free hosting of 10+ TB of data storage and the bandwith as well. Then we can all enjoy the benefits for free. Its very kind of you to offer to do this.

BTW, AR's DB size is a fraction of what CUETools storage requirements and bandwith would be.
Title: CUETools DB
Post by: Teknojnky on 2010-04-20 20:09:22
methinks the record companies would have a little issue with the sampling and reconstruction of their copywritten audio
Title: CUETools DB
Post by: zfox on 2010-04-20 20:21:12
Current size was chosen so that if database contained as many entries as AccurateRip, it would fit on a 1TB drive.

@Eli
How much does such a HD drive cost? You answer that.
How many new CD releases per month? 1000? Recovery data submission bandwidth is under control.

There is also no need to have a track based DB (10x size) for archival purposes.
Title: CUETools DB
Post by: zfox on 2010-04-20 20:26:38
methinks the record companies would have a little issue with the sampling and reconstruction of their copywritten audio

This is indeed an issue. Even if they know they cannot fight that in courts, a C&D letter may arrive.
Title: CUETools DB
Post by: Eli on 2010-04-20 21:55:10
methinks the record companies would have a little issue with the sampling and reconstruction of their copywritten audio



There is no sampling. There is no audio storage. Without the nearly complete audio a repair file is worthless. It can't be played. There is no copyright issue. This database is live and already happening. It may be below the radar. There is no legal violation. Thats not to say big business couldn't bury it financially, but there would really be little to no incentive to attack it. This is the main reason that spoon gave for not going forward with this type of idea when I suggested it years ago though.


Quote
@Eli
How much does such a HD drive cost? You answer that.
How many new CD releases per month? 1000? Recovery data submission bandwidth is under control.

There is also no need to have a track based DB (10x size) for archival purposes.


Clearly the current DB is reasonable for 1 person to host. The idea was to pay for a larger, more robust DB, in order to cover those costs. As I suggested, maybe the best solution would be to offer free access to the current DB structure, and a for fee access to the larger track based DB.

Title: CUETools DB
Post by: Respwaned2 on 2010-05-07 17:01:21
Hey, this is a great initiative. I always wondered what "contacting CTDB" means. Now, I know, big brother. 

On a rare occasion I get what looks like a repairable error, e.g. "Differs in 28 samples @26:19:54". But how do I actually repair the rip in that case? I'm using CUETools 2.0.8.
Title: CUETools DB
Post by: sauvage78 on 2010-05-07 17:28:53
Use encode in repair mode.

It seems CTDB can detect differences without being necessarily errors, verify that the rip is not accurate before trying to repair a perfect rip.
Title: CUETools DB
Post by: Respwaned2 on 2010-05-07 19:11:19
Use encode in repair mode.

It seems CTDB can detect differences without being necessarily errors, verify that the rip is not accurate before trying to repair a perfect rip.


Ok, I tried that, but I get a message box with "Exception: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index". This appears after the encoding finishes. Bug? Edit: I disabled lookup in all databases except AccurateRip (rant: did not feel like waiting for FreeDB again on the 2nd attempt, why the heck does it take so long anyway, re-encoding to Monkey Audio High takes less on my i3), and it worked, repair included. I get "CUETools DB: corrected 28 errors." Great!

My EAC log said "Range quality 99.9 %", so I assume the error was real, although I could not hear anything unusual on the damaged track, even with my Senn 280 headphones. Is there some visual diff tool for PCM files, so I can narrow down the range?

Now I have another question: if I have rip that's reported accurate by AccurateRip, except for the whole disc offset, how do I submit the repair info to CTDB? I see only about 10K discs in the database, many of my classical CDs aren't in. (I generally don't bother with fixing the offset: too much hassle for the tiny benefit, but I understand that CTDB works regardless.)
Title: CUETools DB
Post by: sauvage78 on 2010-05-07 19:36:03
1: Well I never had "Index out of range" when trying to repair but I already had this message several time when trying to encode, IIRC it means that one of the index is reporting a timing located after the end of the whole CD length which is impossible. Usually this means the index (read cue sheet) is broken. This is not directly related to repairing.
2: Use verify in submit mode.
Title: CUETools DB
Post by: Respwaned2 on 2010-05-07 19:41:08
1: Well I never had "Index out of range" when trying to repair but I already had this message several time when trying to encode, IIRC it means that one of the index is reporting a timing located after the end of the whole CD length which is impossible. Usually this means the index (read cue sheet) is broken. This is not directly related to repairing.

It appears it was a fluke, possibly caused by info from FreeDB. It worked fine with the cue sheet I had from EAC.

Quote
2: Use verify in submit mode.


Edit Worked fine if the disc is in AccurateRip db, but not in CTDB. It said: "CUETools DB: 1vH1n8CUu3EZgEhNA_amJjkUM8c- has been uploaded." Now you know who did that one!

But if the disc is not in either database, then I just get a message box saying "AccurateRip: disk not present in database", and there's no other report afterward. So, it doesn't work for discs that aren't in AccurateRip's database? Shouldn't it submit to both databases in that case?

Another question: what does the  "AR" column mean on http://db.cuetools.net/ (http://db.cuetools.net/)? Is it supposed to be the offset? Cause if that's the case, the info displayed is wrong. I submitted 4 discs, all ripped with my ancient LITE-ON LTR-52327S (which died a glorious death it 2004 ripping a large swath of my pressed CDs), which has an offset of 6, but the info in the AR column is all over the map from 1vH1n8CUu3EZgEhNA_amJjkUM8c- to 1S1uNzRfWHCqQx5LU1YM65CvFhQ-.
Title: CUETools DB
Post by: sauvage78 on 2010-05-07 20:15:47
I never tried to submit myself (I wait to have a faster CPU) but IIRC I think I read that Greg requires the rip to be AR3 before accepting the submission ... for me AR2 is enougth but you know there are some magic stuff behind CTDB that I don't really understand so far (specially 10 frames left out instead of 5 like AR or even 4 which is already enougth & leaving out submissions to CTDB with AR3 instead of AR2)

Edit: The AR columm cannot be an offset, my guess is that it is the accuraterip confidency, but it seems to conflict with what I was saying that Greg requires an AR of 3 because there are AR1 submissions. It's either I am wrong or these are pending submissions waiting to be AR3 before being swallowed in the CTDB, I dunno.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-07 20:44:20
methinks the record companies would have a little issue with the sampling and reconstruction of their copywritten audio

I don't expect any problems on this front. There are many databases which store various information about CDs - freedb, musicbrainz, discogs and AccurateRip to name a few. And now CTDB. None of them ever had any legal problems, because none of them store any actual audio, they all store various meta-information -  CD TOCs, artist names, crcs, etc.  Recovery records obviously fall under the same category.

Clearly the current DB is reasonable for 1 person to host. The idea was to pay for a larger, more robust DB, in order to cover those costs. As I suggested, maybe the best solution would be to offer free access to the current DB structure, and a for fee access to the larger track based DB.

I'd like to keep this project absolutely free. Current database contents was submitted by the users and it belongs to the users. If i'm not mistaken, Musicbrainz uses Creative Commons license for it's content. CTDB will use the same, if there are no objections. If hosting costs will become significant at some point, i think we will work something out without compromising the database freedom.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-07 20:52:36
It appears it was a fluke, possibly caused by info from FreeDB.

I recently found a glitch like this, which happens when manually specifying Pregap. If this was it, it'll be fixed in 2.0.9. If it wasn't, i'd like to know more about it.

Quote
But if the disc is not in either database, then I just get a message box saying "AccurateRip: disk not present in database", and there's no other report afterward. So, it doesn't work for discs that aren't in AccurateRip's database? Shouldn't it submit to both databases in that case?

Nope. CUETools uses AR as a proof of existence of physical CD with such content. The only way to submit data that is not in AR database is to rip a CD using a ripper. Only CUERipper supports CTDB at this point. It also cannot submit to AR database, which only accepts submissions from EAC/dbPowerAmp.

Edit: The AR columm cannot be an offset, my guess is that it is the accuraterip confidency, but it seems to conflict with what I was saying that Greg requires an AR of 3 because there are AR1 submissions. It's either I am wrong or these are pending submissions waiting to be AR3 before being swallowed in the CTDB, I dunno.

It is indeed AccurateRip confidence number (+ number of rips done with CUERipper). AR1 submissions are from CUERipper.
Title: CUETools DB
Post by: Respwaned2 on 2010-05-07 22:05:10
I recently found a glitch like this, which happens when manually specifying Pregap. If this was it, it'll be fixed in 2.0.9. If it wasn't, i'd like to know more about it.


I didn't manually specify anything, I'm too lazy for that. One of the CDs I was experimenting with (and have uploaded) does have a pregap though; I'm not sure what the "transient filter" thing is about:

Code: [Select]
[Verification date: 5/7/2010 10:01:41 PM]
[AccurateRip ID: 0021b8d3-01693d52-de0fd90e] found.
Pregap length 00:00:32.
HDCD: peak extend: none, transient filter: some, gain: none
[CTDB TOCID: sYAiRuv0qD8JKDHv8BYqLw9Pr0Y-] disk not present in database.


I'll see if I can reproduce it.

Quote
Nope. CUETools uses AR as a proof of existence of physical CD with such content. The only way to submit data that is not in AR database is rip a CD using a ripper. Only CUERipper can do this at this point. It also cannot submit to AR database, which only accepts submissions from EAC/dbPowerAmp.


That seems a little excessive, especially if the EAC log is present. Do you really think people would spoof it? Someone determined to do that kind of nonsense could do it at the protocol or source code level as well. Besides, physical existence of a real pressed CD (as opposed to cd-r) is more easily proved by FreeDB/MusicBrainz. For instance:

Code: [Select]
[Verification date: 5/7/2010 11:37:23 PM]
[AccurateRip ID: 003554dd-030b2cea-12109f14] disk not present in database.
[CTDB TOCID: Zva2.w2WY9olyZ6gyCpXAq3BQr8-] disk not present in database.
Disk not present in database.


But this disk is in FreeDB (although not in MusicBrainz)

Code: [Select]
EAC extraction logfile from 22. March 2004, 17:00 for CD
Aznavour / 40 chansons d'or - disc 1

Used drive  : LITE-ON LTR-52327S   Adapter: 1  ID: 0
Read mode   : Secure with C2, accurate stream, disable cache
Combined read/write offset correction : 0
Overread into Lead-In and Lead-Out : No

Used output format : Internal WAV Routines
                     44.100 Hz; 16 Bit; Stereo

Other options      :
    Fill up missing offset samples with silence : Yes
    Delete leading and trailing silent blocks : No
    Native Win32 interface for Win NT & 2000


Range status and errors
Selected range
     Filename F:\ape\calme\azn\1\CDImage.wav

     Peak level 100.0 %
     Range quality 100.0 %
     CRC 31594526
     Copy OK

No errors occured

End of status report


The disc seems to be this one: http://www.freedb.org/freedb/blues/0f109f14 (http://www.freedb.org/freedb/blues/0f109f14) (edit: based on the typo in "alller" on the last track, LOL, I only saw it now). Cuetools doesn't display the freedb id, although it calculates it for sure. I'm not ripping my CDs again just to submit them to your database, so it's your loss...

By the way, it appears the FreeDB search in CUETools takes so long because it's only approximate. It also offeres to pick http://www.freedb.org/freedb/blues/1f10b314 (http://www.freedb.org/freedb/blues/1f10b314) for the above, for instance.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-07 23:51:37
I'm not sure what the "transient filter" thing is about:

This is irrelevant. It just says that HDCD encoding was detected.

That seems a little excessive, especially if the EAC log is present. Do you really think people would spoof it?

It's not really intended as spoof protection, we have to rely on users good will. It's just a way to reduce the amount of bad rips in the database. Many people tend to trust the database entries, no matter how many times they are reminded that no database can really assure the accuracy of the data, so i have to do my best to increase this accuracy. I also have to keep database size low. If every rip was accepted, very soon we would have database polluted with unlimited number of inaccurate rips, some of them ripped with itunes or other unreliable rippers, with lots of errors, or downloaded on the net, we'll have even a certain amount of mp3=>flac transcodes.

I never thought of EAC log as an alternative submission criteria, i will think about it. Thank you, that is a useful idea.

By the way, it appears the FreeDB search in CUETools takes so long because it's only approximate.

The speed depends mostly on freedb server latency. It is often overloaded. All freedb searches are 'approximate', many CDs can have the same discid. Musicbrainz is much more precise and has better data quality - almost no typos, none of this UPPERCASE nonsense etc, however it's database is not yet as large as freedb's.
Title: CUETools DB
Post by: Respwaned2 on 2010-05-07 23:51:38
I've also got a "Exception: The specified path, file name, or both are too long." (more blah, blah omitted) error while trying to submit one one disc. Nothing was printed in the lower area. Cannot tell why: the paths and file names seem well below the 260 limit. I can open them just fine in explorer or foobar2000.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-07 23:59:55
Maybe the output path (for the log) was too long - the default template places it somewhere like C:\Documents and settings\username\My Music\.... which can easily get too long. I'll think how to handle this case better, maybe by abbreviating artist/album names to something shorter when generating a path.
Title: CUETools DB
Post by: odyssey on 2010-05-08 13:13:10
How do I submit a rip to CTDB??? I tried Cueripper.
Title: CUETools DB
Post by: Fandango on 2010-05-08 16:10:39
I guess it is submitted when the rip is done...

@Gregory: have you thought about also adding the pregaps to your database? Since I get inconsistent pregap readouts with CUERipper and EAC on some CDs, it may be a helpful for spotting problematic discs in general. Does the CTDB also store drive models, btw?
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-08 17:19:43
I don't store pregaps exactly because they are too inconsistent. There'll be too many different gap detection results for each CD, and hard to tell which ones are correct.

Drive models are stored: http://db.cuetools.net/?agent=CUERipper%20...20RW%20AD-7203S (http://db.cuetools.net/?agent=CUERipper%202.0.8:%20Optiarc%20%20-%20DVD%20RW%20AD-7203S)
Title: CUETools DB
Post by: odyssey on 2010-05-08 23:03:46
I just ripped a few CD's that should not be in the DB. They don't show up at the home page.

And how do I correct bad rips with it?
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-08 23:41:09
You can easily check if CD is in database - when you load CD in the drive, CUERipper shows a confidence number in it's status bar next to CTDB icon.

It doesn't submit in Burst mode or when there were suspicious positions with many retries.

To correct a rip you'll need to use CUETools. Currently CUERipper can only verify & submit.
Title: CUETools DB
Post by: odyssey on 2010-05-09 00:16:13
It doesn't submit in Burst mode or when there were suspicious positions with many retries.

Oh that's why...

To correct a rip you'll need to use CUETools. Currently CUERipper can only verify & submit.

I know, but I still don't understand when it will correct. Do I need to use Encode mode to do that?

Feature request: Put in log, when an album is found in CTDB when it's verifying multiple albums.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-09 00:21:22
Select encode mode, and select 'repair' from the combobox below (where other scripts are, 'fix offset' etc).
Title: CUETools DB
Post by: odyssey on 2010-05-09 00:32:01
It doesn't submit in Burst mode or when there were suspicious positions with many retries.

Why don't you allow burst-mode rips to create CTDB entrys? If the disc is already in AR you can just accept to submit it if it rips correctly. I actually considered this, just to put all my discs in the DB, but ripping them in secure mode takes too long I think.

Another approach (similar to dbpa): Rip burst, if accurate submit, if not accurate (or not available in AR), rip secure.
Title: CUETools DB
Post by: sauvage78 on 2010-05-09 04:12:40
While I agree that burst rip should be accepted as long as those are AR2, I disagree with the idea that burst mode would always be faster than secure mode. This is only true if you compare burst vs. secure on a non-scratched CD. It also depends on the number of re-read you're using in your secure mode & it also depends on if you're ripping as CDImage or as separate tracks (because re-ripping a single scratched track is faster than re-riping a whole CD, which means that burst works even faster with separate tracks). As soon as there are a few scratched CD in your collection (and the average joe always have a few scratched CD in his collection), it is "on average" faster to use the secure mode with a lower number of re-reads than to completely drop the secure mode in favor of the burst mode (This is specially true if you rip as CDImage as you cannot only re-rip the scratched track). This is due to the fact that the burst mode will be faster on a single pass but you will need several pass to achieve an AR result. So if you want to avoid the headache of re-ripping ten time the same track (or whole CD if you use CDImage) you'd better use a secure mode with fewer re-reads than completely drop the secure mode. This is specially true with CDImage, but this is also true with separate tracks because even if re-riping a single track in burst mode is very fast. You will have to verify each burst rip separatly with cuetools, & as it will be burst, the chances that it will be AR2 on a really scratched CD will be low. Overall this means that the time you will gain for using burst will likely be lost checking dozen of burst rip against AR with cuetools. (Edit: Here I am a special case as I delete EAC log but keep .accurip which forces me to use cuetools after the ripping process, the cuetools part might be irrevelant to you if you directly check AR thru EAC)

When AR was created I was thinking like you that it would make the secure mode obsolete in favor of burst+AR2. This is not true, AR only allow you to reduce the number of re-reads. My experience tells me that it is faster to rip a CD twice in Secure+Low than ripping it ten time in burst & then checking ten time if it's AR. Burst mode+AR2 can be faster if you're lucky (lucky to only have non-scratched CD in your collection), but it's pure random. The only exception is maybe if you just bought the CD as it will be shinning new. As I don't want to use two settings for new & old CD (even with profiles), this exception is irrevelant to me.

In the end this means that even if AR2 burst rip should be accepted, it is not really a priority IMHO because in the end the burst mode is not always as fast as it seems. It depends on what you rip (new or old CD) & on how you rip (what setting you're comparing burst to).

Edit: The above applies to EAC, I never tried CUERipper so far.

The fact that CTDB ignores 10 frames instead of 5 like AR is a "much bigger" (I know some people people might not consider 5 frames "big" but I disagree) design flaw IMHO, I already have 3 rips which are not AR but are CTDB OK ... even if AR & CTDB doesn't serve the same purpose, it is a little weird to have them conflicting.

Exemple:
Code: [Select]
[Verification date: 05/05/2010 21:42:49]
[AccurateRip ID: 001555ed-00b500c2-990b410b] found.
[CTDB TOCID: 9ZJTSx3s7H3pnsu6h7eKNf9M8xE-] found.
        [ CTDBID ] Status
        [2b6d1a2c] (190/190) Accurately ripped
Track [ CRC    ] Status
 01 [ec93376d] (76/203) No match but offset
 02 [dbd08c29] (77/203) Accurately ripped
 03 [83406c8a] (76/201) Accurately ripped
 04 [983f8873] (77/203) Accurately ripped
 05 [eabd4d44] (77/203) Accurately ripped
 06 [10c7c833] (77/204) Accurately ripped
 07 [d4328e57] (77/204) Accurately ripped
 08 [d98b9da8] (76/203) Accurately ripped
 09 [984cf36c] (77/203) Accurately ripped
 10 [0994ce6b] (77/203) Accurately ripped
 11 [4e9e9b41] (76/198) Accurately ripped

Note: This log was shortened.
Title: CUETools DB
Post by: Eli on 2010-05-10 02:51:36
Clearly the current DB is reasonable for 1 person to host. The idea was to pay for a larger, more robust DB, in order to cover those costs. As I suggested, maybe the best solution would be to offer free access to the current DB structure, and a for fee access to the larger track based DB.

I'd like to keep this project absolutely free. Current database contents was submitted by the users and it belongs to the users. If i'm not mistaken, Musicbrainz uses Creative Commons license for it's content. CTDB will use the same, if there are no objections. If hosting costs will become significant at some point, i think we will work something out without compromising the database freedom.



My suggestion was to keep the current database and any future entries to the way it is currently configured free, but have a parallel database with track based repair info. Since this would be 10+x bigger, it would be reasonable to charge a small amount for this.

Of course I would prefer to have the larger database for free, but if the choice is between paying for the bigger database or not having it available, I would pay.

I would really like to see spoon use this database in dbpoweramp. He seemed to imply that in its current configuration he is less likely to. I think this is very unfortunate, but his choice.
Title: CUETools DB
Post by: sauvage78 on 2010-05-10 03:51:16
I only post an interesting log that I just found in my collection which illustrates the previous case but in an even more dramatical way:

First the rip seems not AR (scratched on last track) but repairable:

Code: [Select]
[Verification date: 09/05/2010 23:08:09]
[AccurateRip ID: 000c13cb-00528e2b-5d099708] found.
[CTDB TOCID: V7Pd0AEJx0Xa0Jx_3rtludlTqmc-] found.
        [ CTDBID ] Status
        [61c027c2] (3/3) Differs in 5809 samples @40:54:57-40:54:62
Track [ CRC    ] Status
 01 [c1859a35] (00/05) No match
 02 [407b9e51] (00/05) No match
 03 [f62fa730] (00/05) No match
 04 [a810ba68] (00/05) No match
 05 [e5c90916] (00/05) No match
 06 [7129814f] (00/05) No match
 07 [542c41ed] (00/05) No match
 08 [de7f4fc1] (00/05) No match
Offsetted by 1650:
 01 [b1e49860] (05/05) Accurately ripped
 02 [36696c23] (05/05) Accurately ripped
 03 [45702970] (05/05) Accurately ripped
 04 [cbc2af16] (05/05) Accurately ripped
 05 [a887219d] (05/05) Accurately ripped
 06 [577756e8] (05/05) Accurately ripped
 07 [daf1beb2] (05/05) Accurately ripped
 08 [46a0b81d] (05/05) No match but offset

Track Peak [ CRC32  ] [W/O NULL]
 --  87,6 [D97FEE4E] [07750B53]         
 01  87,6 [8D9D5270] [6A32175F]         
 02  47,9 [C4C45951] [F17B1FAF]         
 03  82,8 [476003BF] [C71C7807]         
 04  82,2 [90F608CE] [07A337C8]         
 05  80,9 [4807F605] [019ADA7D]         
 06  69,2 [F8907160] [4A9941DB]         
 07  57,4 [193B474F] [D43FCCE8]         
 08  81,3 [C991DD9C] [EAC43C72]     

But after trying to repair it finally seems that the scratches is located in the hole between the 5 frames ignored by AR & the 10 frames ignored by CTDB. I agree it's a rare case (4th case so far), but as the start/end of the rip is usually a critical point, it does happen in real life case.

Code: [Select]
[Verification date: 10/05/2010 03:34:28]
[AccurateRip ID: 000c13cb-00528e2b-5d099708] found.
CUETools DB: corrected 5809 errors.
Track [ CRC    ] Status
 01 [c1859a35] (00/05) No match
 02 [407b9e51] (00/05) No match
 03 [f62fa730] (00/05) No match
 04 [a810ba68] (00/05) No match
 05 [e5c90916] (00/05) No match
 06 [7129814f] (00/05) No match
 07 [542c41ed] (00/05) No match
 08 [33c3d834] (00/05) No match
Offsetted by 1650:
 01 [b1e49860] (05/05) Accurately ripped
 02 [36696c23] (05/05) Accurately ripped
 03 [45702970] (05/05) Accurately ripped
 04 [cbc2af16] (05/05) Accurately ripped
 05 [a887219d] (05/05) Accurately ripped
 06 [577756e8] (05/05) Accurately ripped
 07 [daf1beb2] (05/05) Accurately ripped
 08 [8e3f75b6] (05/05) No match but offset

Track Peak [ CRC32  ] [W/O NULL]
 --  87,6 [F55BD6EA] [A6E84737]         
 01  87,6 [8D9D5270] [6A32175F]         
 02  47,9 [C4C45951] [F17B1FAF]         
 03  82,8 [476003BF] [C71C7807]         
 04  82,2 [90F608CE] [07A337C8]         
 05  80,9 [4807F605] [019ADA7D]         
 06  69,2 [F8907160] [4A9941DB]         
 07  57,4 [193B474F] [D43FCCE8]         
 08  81,3 [E5B5E538] [8E0A8761]         

When I compare AR database vs CTDB, 9247 discs vs. 1.4 Millions, it's either you fix it now or we will have to live with this flaw. The AR database history shows that once the database is populated it becomes almost impossible to change/fix it, as deleting the "populated but not optimal" database is heartbreaking & managing two databases is a headache. So IMHO the more you wait the less likely to be fixed it becomes.

I am not criticizing the database, it works: I fixed around 25 rips & used it to found the exact data track length of around 20 rips so far. It's just that it could work better. 5 frames might seems very small but scratches often happens in the first & last tracks, so IMHO this 5 frames have a special value due to their location. Also I simply don't understand the logic of not doing like AR when CTDB is so tied to its fate, so I'd rather ask to be sure: is there a technical reason to ignore those 5 extra frames or was it a random (uninspired) choice ?

Edit: I forgot to say that indeed the rip becomes CTDB OK after the repairing, so it cannot be a case where the damage is beyond repair.

Code: [Select]
[Verification date: 10/05/2010 04:27:21]
[AccurateRip ID: 000c13cb-00528e2b-5d099708] found.
[CTDB TOCID: V7Pd0AEJx0Xa0Jx_3rtludlTqmc-] found.
        [ CTDBID ] Status
        [61c027c2] (3/3) Accurately ripped
Title: CUETools DB
Post by: greynol on 2010-05-10 06:43:44
The last time you claimed "damage due to a scratch" I proved (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=80403&view=findpost&p=701936) that it wasn't.

What makes you so sure the data is in error this time around?  Have you taken the time to manually compare the the "repair" to the original rip to see what the differences are?

Differences between pressings can vary by more than just the offset at the edges.  For this reason, extending the area of what is ignored can actually be a good thing.

I've also pointed out that EAC used to have a bug that caused the last two frames of non-null samples on the disc to be repeated on occasion in secure mode when overreading is disabled.  It was not corrected until V0.99pb5.  Has anyone besides me bothered to investigate this phenomena?

I guess it's just easier to ignore the possibility that things aren't always so cut and dry and just attribute it to a scratch (presumably resulting in an error that your ripping program failed to identify?).
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-10 07:38:04
Of course I would prefer to have the larger database for free, but if the choice is between paying for the bigger database or not having it available, I would pay.

I would really like to see spoon use this database in dbpoweramp. He seemed to imply that in its current configuration he is less likely to. I think this is very unfortunate, but his choice.

I'm just not ready to handle financial aspects. I'm not good at this. So it's better to just keep database costs to a minimum, and hope that storage will be cheap enough in the future.

If Mr. Spoon will decide to support CTDB on condition that it's track based, we'll discuss it. Also nothing stops him from creating such a database himself based on CTDB code.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-10 07:55:37
The fact that CTDB ignores 10 frames instead of 5 like AR is a "much bigger" (I know some people people might not consider 5 frames "big" but I disagree) design flaw IMHO, I already have 3 rips which are not AR but are CTDB OK ... even if AR & CTDB doesn't serve the same purpose, it is a little weird to have them conflicting.

The issue is actually quite complicated.

Let's imagine a simple scenario, a simple CD with one track, no pregaps, no datatracks, with total length of 100 sectors. Let's also assume we're submitting with CUETools, no ripping involved.

Suppose AR verification shows a total confidence of 10, offset 0;
That means that sectors 5..95 are verified and we can upload recovery record for them to CTDB with confidence 10.

But suppose AR verification shows offset = 4 sectors.
That means that sectors 9..99 are verified and we can upload recovery record for them to CTDB with confidence 10.

More complicated case: AR verification shows offset = 4 sectors, confidence = 10 and offset = -4 sectors, confidence = 5;
Then sectors 1..8 have confidence 5, sectors 9..91 have confidence 15 and sectors 92..99 have confidence 10.

What shall we upload? Sectors 9..91 with confidence 15 or sectors 1..99 with confidence 5?

Other part of the problem is that if the boundaries of processed segment depend on AR verification results, submitting will require two passes.
Title: CUETools DB
Post by: sauvage78 on 2010-05-10 08:06:35
Well the two problems cases are very different:

- in the first problem case the AR database detects nothing wrong while the full checksum for the first track are different within the same pressing, this means that the problem is located inside the 5 frames ignored by AR.
This case have nothing to do with CTDB, I haven't tried CTDB on it, but I have deleted the bad doublons so I can't test anymore. Within all logic CTDB would have been of no help here because the difference being located ouside of the reach of the AR checksum it can only be located outside of the reach of the CTDB checksum (As CTDB ignores 5 added frames compared to AR.)

In this first problem case, I have only losslessly splitted the 5 first frames (as you've noticed it was painfull) & compared the Flac MD5 checksum which was different (The EAC wav comparator also detected differences). I cannot hear any difference (5 frames too too short to ABX anyway) & when I zoom on the 5 frames with any audio editor all I see is flat.

My first idea was naturally to think about a scratch, but I wasn't sure so I asked for advice & you told me that this was possibly due to an  -472 offset problem due to overread. My skill with an audio editor to obtain the same result were too weak. But I think that the explanation is logic & I know how serious you can be so I tend to trust you.

But IMHO this second case is slightly different, maybe you're right & maybe this is an even bigger overread into the lead-in problem ... but as far as I understund the 5 frames ignored by AR are already there to avoid this kind of problem. So the problem is beyond the safety margin which means that this time it is more likely to be a simple scratch. I have already deleted the un-corrected file so I cannot verify (but even if I would still have it the first problem case showed that I didn't knew how to do)

Anyway all this doesn't really matter what I wanted to illustrate is that it is possible to have a scratch detected by AR by that cannot be corrected by CTDB because of the difference between the length of ignored frames.

Quote
Differences between pressings can vary by more than just the offset at the edges. For this reason, extending the area of what is ignored can actually be a good thing.


I know that pressings can vary by a lot more than just the offset, but I don't understand the second part of the sentence, I don't see why it would be a good thing to extend the ignored area when this area is already enougth. There is not offset bigger than 5 frames in the database & I even think that 4 frames is already enougth if you only judge by drive in the AR database. 1 frame is not a big deal & I know Spoon chosen 5 frames back when there was virtual drive with big offset in his database which have been purged. A 1 frame margin is also eventually good as an added margin. But Greg adds 5 more frames this sounds enormous to me, he simply doubled the margin of Spoon. I don't understand why so I just ask. Maybe I am wrong.

I admit I haven't investigated the EAC bug, but from what I understand of it, if it's just 2 reapeated frames at the end, it only pushes the margin down to 3 frames at the beginning, even in this case you really need an offset higher than 3 frames to get in trouble. Even with this bug I can understand a choice of 5+2=7 ... but going up to 10 is overkill. Even so, I don't think that bug should affect the technical choice behind CTDB.

I admit, it's easier to attribute it to a scratch , but in this case it has a big chance of being a scratch IMHO.
Title: CUETools DB
Post by: greynol on 2010-05-10 08:10:56
Offset differences between pressings can be far greater than just five frames.  EAC's bug is not limited to the last five frames either.

Until you can rule out other far more likely possibilities, I would avoid these hasty conclusions.  Like with the case involving a lack of overreading, it doesn't seem that you're fully grasping the whole picture.  Discarding your data without carefully inspecting it doesn't help.    When you see things like different fade-ins and fade-outs between pressings with your wave editor you will gain a greater appreciation of what Gregory is trying to do; if not a better understanding.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-10 08:46:23
but going up to 10 is overkill.

You are forgetting that CTDB stores only one record for all pressings. Pressing offset adds to drive offset, and pressing offsets can be quite large. Even if drive offsets are limited to 4 sectors, pressing offsets of 6 sectors are not uncommon.
Title: CUETools DB
Post by: sauvage78 on 2010-05-10 08:57:35
Yes, your exemple made me realize that the more there are existing pressings with different offset in the AR database, the bigger the margin of CTDB must be in order to still have a core/heart of sectors to calculate the correction checksum from (if you only want to do 1 pass & not rely on AR offsets) which would still work with all pressings.

So if I understund correctly CTDB is not really suited to correct scratches which happens at the beginning & at the end ... it's a pity.

I had to read the exemple 3 times, & spend 5 min to think about the consequences but it was usefull
I was completely wrong with how I thought CTDB would work & about the "flaw" of not behaving like the AR database. My way of thinking was too tied to AR.

It must have been a real engineering nightmare ...

Edit: Thks for the explanation, Greg & thks ... for your patience Greynol ... I do tell bullshits sometimes ...
Title: CUETools DB
Post by: greynol on 2010-05-10 17:53:05
So if I understund correctly CTDB is not really suited to correct scratches which happens at the beginning & at the end ... it's a pity.

While this is true, scratches that are so localized to cause problems only in these extremely small areas will be very very rare.  I doubt that you have any.
Title: CUETools DB
Post by: Fandango on 2010-05-10 18:37:42
Is CTDB able to identify null sample tracks? AR fails to store these tracks in the database, I guess because its CRC is giving the same value for all totally silent tracks no matter how long their duration is.

I have one CD with three silent tracks at the end which fails to AR-verify in CUE Tools because of that. Although this is only a cosmetic issue of the log, will CTDB circumvent this problem? I mean even if a silent track is not much to listen to, if there was to be a read error, it may become audible and I think therefore it should be detectable by a verification database.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-10 18:43:25
CTDB doesn't care about tracks and it doesn't care about null samples, so yes, it is able to repair noise to silence.
Title: CUETools DB
Post by: greynol on 2010-05-10 19:01:14
I mean even if a silent track is not much to listen to, if there was to be a read error, it may become audible

Doubtful since interpolation between silence gives silence.  Errors so bad that cannot be interpolated are generally replaced by silence.  Perhaps a held DC value from an errant sample, but this seems unlikely.
Title: CUETools DB
Post by: sauvage78 on 2010-05-12 03:50:23
I have found one more strange log:
An AR3 rip with CTDB differences but with a confidence of 0. Any idea of why such thing can happen ? Has there already been a purge in the young CTDB ? Even if it was a purge the thing I don't understand is that if there is a checksum stored (there is obviously 1 as a differences are calculated) It should necessarily means a CTDB confidence of 1 ... so how can there be an existing checksum without any confidence level ? Specially on an AR3 rip ? Is it a CTDB bug ?

Code: [Select]
[Verification date: 11/05/2010 14:44:48]
[AccurateRip ID: 000d34c3-006c99e1-8609110a] found.
[CTDB TOCID: DKLLUM1E6Nrp_tsCx8Bqg4iloPk-] found.
        [ CTDBID ] Status
        [52f7fda3] (0/0) Differs in 29399 samples @08:25:08-08:25:33
Track [ CRC    ] Status
 01 [ca629bd7] (03/03) Accurately ripped
 02 [7f084dd2] (03/03) Accurately ripped
 03 [9e1e1f5f] (03/03) Accurately ripped
 04 [fd880f87] (03/03) Accurately ripped
 05 [15949383] (03/03) Accurately ripped
 06 [db667438] (03/03) Accurately ripped
 07 [f1b1d28d] (03/03) Accurately ripped
 08 [e91e2258] (03/03) Accurately ripped
 09 [fb7981e3] (03/03) Accurately ripped
 10 [07ad5387] (03/03) Accurately ripped

Track Peak [ CRC32  ] [W/O NULL] [  LOG  ]
 --  100,0 [7E17A9CC] [704C81CF]  CRC32 
 01  100,0 [9C8BFCDE] [5B747295]         
 02  96,8 [C8C2FC04] [A024B381]         
 03  96,4 [E083B51F] [E98ADF35]         
 04  100,0 [3E2F0464] [6C1251DB]         
 05  100,0 [E9DEF500] [DECB891C]         
 06  98,1 [E5E4A5E6] [83DBB535]         
 07  80,0 [0A5A0CCF] [61ABEEC0]         
 08  99,8 [579624EE] [4D2FD443]         
 09  94,2 [61CC98E3] [FA1F8E92]         
 10  96,8 [0FE9DA58] [8242B8EA]         


Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-12 03:59:05
I was actually testing CUERipper on that disc  My CD is scratched, so i couldn't get an accurate rip, and i just manually purged this entry afterwards. Would be nice if you could submit it
Title: CUETools DB
Post by: sauvage78 on 2010-05-12 04:10:54
Ok, Thks

I have submitted this CD so that you can fix yours.
Quote
CDImage.cue: DKLLUM1E6Nrp_tsCx8Bqg4iloPk- has been updated.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-12 04:12:52
Thanks
Title: CUETools DB
Post by: sauvage78 on 2010-05-12 05:51:22
I just re-checked this CD, now the reverse is happening, it has jumped directly to CTDB confidence 3 ? I thought it would have become CTDB confidence 1+AR3 after my submission. I am understanding the meaning of CTDB (3/3) wrongly (is it related to AR3 maybe, I thought it wasn't? I thought it was just the number of match/the number of CTDB submissions) or is there any other explanation ?

Code: [Select]
[Verification date: 12/05/2010 06:29:45]
[AccurateRip ID: 000d34c3-006c99e1-8609110a] found.
[CTDB TOCID: DKLLUM1E6Nrp_tsCx8Bqg4iloPk-] found.
        [ CTDBID ] Status
        [cedd9f43] (3/3) Accurately ripped
Track [ CRC    ] Status
 01 [ca629bd7] (03/03) Accurately ripped
 02 [7f084dd2] (03/03) Accurately ripped
 03 [9e1e1f5f] (03/03) Accurately ripped
 04 [fd880f87] (03/03) Accurately ripped
 05 [15949383] (03/03) Accurately ripped
 06 [db667438] (03/03) Accurately ripped
 07 [f1b1d28d] (03/03) Accurately ripped
 08 [e91e2258] (03/03) Accurately ripped
 09 [fb7981e3] (03/03) Accurately ripped
 10 [07ad5387] (03/03) Accurately ripped

Track Peak [ CRC32  ] [W/O NULL]
 --  100,0 [7E17A9CC] [704C81CF]         
 01  100,0 [9C8BFCDE] [5B747295]         
 02  96,8 [C8C2FC04] [A024B381]         
 03  96,4 [E083B51F] [E98ADF35]         
 04  100,0 [3E2F0464] [6C1251DB]         
 05  100,0 [E9DEF500] [DECB891C]         
 06  98,1 [E5E4A5E6] [83DBB535]         
 07  80,0 [0A5A0CCF] [61ABEEC0]         
 08  99,8 [579624EE] [4D2FD443]         
 09  94,2 [61CC98E3] [FA1F8E92]         
 10  96,8 [0FE9DA58] [8242B8EA]         

Also why does CTDB reports (6/6) No match instead of (0/6) No match in this log

Code: [Select]
[Verification date: 28/04/2010 15:56:02]
[AccurateRip ID: 000b5e1b-00460ba6-560aaa07] found.
Pregap length 00:00:33.
[CTDB TOCID: SrqfChprJfQaVZ.K2a_V4AalfcE-] found.
        [ CTDBID ] Status
        [c6cfb022] (6/6) No match
Track [ CRC    ] Status
 01 [c8243388] (00/39) No match
 02 [62d8c057] (00/40) No match
 03 [9a178a8a] (00/40) No match
 04 [3c9ae5bc] (00/39) No match
 05 [160e691c] (00/38) No match
 06 [f391b6f2] (00/39) No match
 07 [5e0f5ea5] (00/40) No match
Offsetted by -579:
 01 [ca0690bf] (05/39) Accurately ripped
 02 [b23d8e96] (05/40) Accurately ripped
 03 [9fbd6b42] (05/40) Accurately ripped
 04 [2cf80293] (05/39) No match but offset
 05 [53fbbb74] (05/38) Accurately ripped
 06 [298d69df] (05/39) Accurately ripped
 07 [07de6a7b] (05/40) Accurately ripped
Offsetted by -99:
 01 [9368165f] (00/39) No match
 02 [0e442f36] (07/40) No match but offset
 03 [5cb73842] (00/40) No match
 04 [c534c433] (00/39) No match
 05 [1f518474] (06/38) No match but offset
 06 [de4aadbf] (07/39) No match but offset
 07 [f7c754bb] (00/40) No match
Offsetted by -97:
 01 [bdd87e65] (00/39) No match
 02 [a0c87a8c] (00/40) No match
 03 [b0b3fe72] (00/40) No match
 04 [ab2c7c79] (06/39) No match but offset
 05 [8690cee4] (00/38) No match
 06 [7f0b77a1] (00/39) No match
 07 [a482f7d7] (00/40) No match
Offsetted by -96:
 01 [d310b268] (00/39) No match
 02 [ea0aa037] (00/40) No match
 03 [dab2618a] (07/40) No match but offset
 04 [1e28589c] (00/39) No match
 05 [ba30741c] (00/38) No match
 06 [cf6bdc92] (00/39) No match
 07 [fae0c965] (00/40) No match
Offsetted by -95:
 01 [e848e66b] (00/39) No match
 02 [334cc5e2] (00/40) No match
 03 [04b0c4a2] (00/40) No match
 04 [912434bf] (00/39) No match
 05 [edd01954] (00/38) No match
 06 [1fcc4183] (00/39) No match
 07 [513e9af3] (07/40) No match but offset
Offsetted by -94:
 01 [fd811a6e] (06/39) No match but offset
 02 [7c8eeb8d] (00/40) No match
 03 [2eaf27ba] (00/40) No match
 04 [042010e2] (00/39) No match
 05 [216fbe8c] (00/38) No match
 06 [702ca674] (00/39) No match
 07 [a79c6c81] (00/40) No match
Offsetted by 84:
 01 [be954484] (07/39) Accurately ripped
 02 [6c8d1c73] (07/40) Accurately ripped
 03 [61900e6a] (07/40) Accurately ripped
 04 [f73f2138] (07/39) No match but offset
 05 [06709f7c] (07/38) Accurately ripped
 06 [5332d606] (07/39) Accurately ripped
 07 [b4d8213d] (07/40) Accurately ripped
Offsetted by 102:
 01 [3c88ecba] (17/39) Accurately ripped
 02 [9333c279] (17/40) Accurately ripped
 03 [5573061a] (17/40) Accurately ripped
 04 [0cf49bae] (17/39) No match but offset
 05 [a7aa3d6c] (17/38) Accurately ripped
 06 [f9f9eef8] (17/39) Accurately ripped
 07 [c770dd39] (17/40) Accurately ripped

Track Peak [ CRC32  ] [W/O NULL]
 --  97,7 [8147D7CD] [553022C2]         
 01  97,7 [A8AC93F2] [1E768B56]         
 02  97,7 [BB22369D] [00143B9D]         
 03  97,7 [E189EB05] [32995898]         
 04  97,7 [7D38D94A] [8B6086CE]         
 05  97,7 [454CAF56] [EF251649]         
 06  97,7 [2C6E28C5] [723271B1]         
 07  97,7 [6B0CCFA5] [1D31FE76]         

I begin to have doubt about my understanding about what CTDB is reporting.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-12 06:12:48
I purged the second entry completely. Before i did, it looked like this:
Code: [Select]
        [ CTDBID ] Status
        [cedd9f43] (3/3) Accurately ripped
        [52f7fda3] (0/3) Differs in 29399 samples @08:25:08-08:25:33


CTDB confidence comes from AR confidence + number of rips done using CUERipper. In this case AR confidence = 3, number of rips done using CUERipper = 0, so the total is 3.

As for "(6/6) No match", it might be a bit confusing because it's different from AR log, but in fact it's very simple.
The first number is just a confidence value of this CTDB record, regardless of whether or not it matches your copy.
The second number is a sum of confidence values for all CTDB records with this TOCID.
So those numbers only tell us how authoritative this record is, not how well it matches the data.

UPD: fixed typo
Title: CUETools DB
Post by: sauvage78 on 2010-05-12 06:58:33
Ok, once more I was completly wrong about CTDB

It's a little artificial to put "(3/3)+Accurately ripped" on the same line when "(3/3)" has nothing to do with the conclusion "Accurately ripped", specially compared to how the AR part of the log works but it has the advantage of being shorter ... I guess people will have to learn what it means because if it's short it is not intuitive.

Also I never tried CUERipper so far but I hope that at some point CUERipper will get swallowed by AR so that the CTDB confidence can become AR only & not AR+CUERipper.

Overall these (X/Y) numbers are not very interesting because X will sooner or later become the same as the AR confidency & Y is (actually) useless because there are so many more entries in the AR database than in the CTDB, that if you want an idea of how authoritative your pressing is you'd better have a look at AR database. These numbers seems only interesting when the AR log is not accurate, because when it's accurate it should be doublons (specially X) with the AR info. Y & the supposed authoritative information is actually rather random with so few entries.

Maybe it is usefull for you as an ID to store different checksums ? Also is there any link between those numbers & the conclusion that I wouldn't have seen?
Title: CUETools DB
Post by: Saxo on 2010-05-17 14:25:34
I don't store pregaps exactly because they are too inconsistent. There'll be too many different gap detection results for each CD, and hard to tell which ones are correct.

Drive models are stored: http://db.cuetools.net/?agent=CUERipper%20...20RW%20AD-7203S (http://db.cuetools.net/?agent=CUERipper%202.0.8:%20Optiarc%20%20-%20DVD%20RW%20AD-7203S)


Gregory, Hello!

First, I wanted to thank you for the initiative of creating the CTDB! This is the Holy Grail I was waiting for, since I wasn't happy enough with Spoon's AccurateRip DB.

- Check on the whole CD and not track based (I hate multiple files for CD archival...)
- Possibility to repair an inaccurate rip

That was enough to convince me. I completely understand you about the difficulty of implementing the pregap informations in your CTDB because of its random results. However, IMHO, the pregaps are as much important as the audio stream itself for an Audio CD archival purpose!

Moreover (I'm sure you aware of this, but not everybody are) TOC and pregaps are not enough informations neither! In a very few CDs (I've encountered only two so far) some of their tracks were divided into parts or "chapters" (I don't know the right term for this, I should check my Red Book copy ). On the oldest CD players, there was a "chapter" counter next to the track counter.

The result in the cue sheet file, is multiple indexes. I pasted you an example here:

Code: [Select]
TITLE "2112"
PERFORMER "Rush"
REM REPLAYGAIN_ALBUM_GAIN -7.24 dB
REM REPLAYGAIN_ALBUM_PEAK 0.977356
FILE "1976 • Rush - 04 - 2112 • Remaster Mercury P2B-34626.flac" WAVE
  TRACK 01 AUDIO
    TITLE "2112: I. Overture / II. The Temples of Syrinx / III. Discovery / IV. Presentation / V. Oracle: The Dream / VI. Soliloquy / VII. Grand Finale"
    REM REPLAYGAIN_TRACK_GAIN -7.43 dB
    REM REPLAYGAIN_TRACK_PEAK 0.977356
    INDEX 01 00:00:00
    INDEX 02 04:32:40
    INDEX 03 06:46:00
    INDEX 04 10:15:22
    INDEX 05 13:57:27
    INDEX 06 15:57:35
    INDEX 07 18:18:45
  TRACK 02 AUDIO
    TITLE "A Passage To Bangkok"
    REM REPLAYGAIN_TRACK_GAIN -6.87 dB
    REM REPLAYGAIN_TRACK_PEAK 0.977356
    INDEX 00 20:32:67
    INDEX 01 20:33:35
  TRACK 03 AUDIO
    TITLE "The Twilight Zone"
    REM REPLAYGAIN_TRACK_GAIN -5.87 dB
    REM REPLAYGAIN_TRACK_PEAK 0.977356
    INDEX 00 24:07:47
    INDEX 01 24:08:30
  TRACK 04 AUDIO
    TITLE "Lessons"
    REM REPLAYGAIN_TRACK_GAIN -7.62 dB
    REM REPLAYGAIN_TRACK_PEAK 0.977356
    INDEX 00 27:25:70
    INDEX 01 27:27:57
  TRACK 05 AUDIO
    TITLE "Tears"
    REM REPLAYGAIN_TRACK_GAIN -5.09 dB
    REM REPLAYGAIN_TRACK_PEAK 0.977325
    INDEX 00 31:18:05
    INDEX 01 31:20:20
  TRACK 06 AUDIO
    TITLE "Something For Nothing"
    REM REPLAYGAIN_TRACK_GAIN -7.35 dB
    REM REPLAYGAIN_TRACK_PEAK 0.977356
    INDEX 00 34:50:55
    INDEX 01 34:54:40


You will tell me that all the cue/CDimage players don't bother with pregaps and "chapters" and that is so far irrelevant to take them into account, but I have a project of creating a player that will simulate the display of the old-fashioned Hi-Fi CD players. It's just a project in this state, however

In my opinion, I think we have to find a solution to implement this in the future in your DB. Perhaps a separated DB where people can compare the different cue for each CD ID entry? I know that's not easy at all...
Title: CUETools DB
Post by: Fandango on 2010-05-17 17:10:23
I have a project of creating a player that will simulate the display of the old-fashioned Hi-Fi CD players. It's just a project in this state, however

Also as a default user interface component for foobar2000? That would be nice eye-candy!
Title: CUETools DB
Post by: Saxo on 2010-05-22 17:46:42
I have a project of creating a player that will simulate the display of the old-fashioned Hi-Fi CD players. It's just a project in this state, however

Also as a default user interface component for foobar2000? That would be nice eye-candy!


Well... I think I'm going to disappoint you. First, I want to simulate the behavior of the old CD players more than their appearance  Secondly, I'm writing it in Objective-C using the Cocoa framework. Mac OS X lacks of cue/CDimage player, when Windows have foobar2000.

But since I'm still learning and very lazy, don't expect anything anytime soon.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-22 18:07:34
In my opinion, I think we have to find a solution to implement this in the future in your DB. Perhaps a separated DB where people can compare the different cue for each CD ID entry? I know that's not easy at all...

A repository of cue sheets is not very difficult to set up, i'm sure someone will do this someday.
The problem with it is not that it's difficult, but it's boring  Much less challenging than CTDB's idea.
And much less useful. You can re-rip a cuesheet from a CD in a matter of seconds.
Title: CUETools DB
Post by: sauvage78 on 2010-05-22 18:27:27
Well maybe the biggest use would be for protected CD ... I think that there is a bunch of CDImage with broken  INDEX 00 in the wild, most often the INDEX 00 is frozen & sometimes INDEX 00 is after the end of the whole CD range too ... I don't know if these are really related to CD protection breaking the Red Book or if it's just related to bad drives/settings but I guess it would maybe help with these rips ... the problem is that where it may help the database will probably get very random results. (I am only guessing)

In the end such a database would be really oriented toward piracy because if you own the CD you obviously will have no use of it except in really rare cases IMHO.

The paradox is that in case of real life HDD crash, a cue sheet database might be more usefull than the actual CTDB ... here experience is speaking ... I already had a crash several year ago, after a full Getdatataback recovery, I had more broken cue sheet with random data inside than broken CDImage.flac, the explanation seems to be that it is easier for recovery algorithms to recover bigger files with lot of HDD links (clusters) than to recover very small files with only 1 HDD link. I am not saying that CTDB would be useless, I had several broken flac too, so back then I would have liked CTDB to exist, it's just that I had around twice more broken cuesheet then broken flac (rounded up) which was a surprise to me.
Title: CUETools DB
Post by: Saxo on 2010-05-22 20:46:19
In the end such a database would be really oriented toward piracy because if you own the CD you obviously will have no use of it except in really rare cases IMHO.

The first use of the cue sheet is to archive a CD on a HDD. The INDEX tags keeps informations that allow anyone to have the whole original CD experience without touching the physical media. Personally, I rip a CD and don't put my hand on it anymore. I don't use different files for each track neither and I like to play a CD by opening its cue sheet in a player. I suspect that other peoples have the same opinion on this than me. So we use cue sheets all the time and not only in rare occasions. I understand the piracy problem point of view, but since CTDB give access to the track-only TOC, that won't be any different.

A repository of cue sheets is not very difficult to set up, i'm sure someone will do this someday.
The problem with it is not that it's difficult, but it's boring  Much less challenging than CTDB's idea.
And much less useful. You can re-rip a cuesheet from a CD in a matter of seconds.

I know that I could try reproducing more accurate cue sheet files with CUERipper if the previous ripper I used wasn't very good. The trick is that we don't talk about checking 10 CDs... but hundreds of them and a batch process could be very helpful! IMHO, it could also be very interesting to have the results of different persons, using different drives, and different rippers.

Of course, I'm talking about a general database here, that won't be specifically linked to CUERipper. I completely understand that you don't want to bother with that. I imagine a database where EAC, CUERipper and other rippers could access with write permission and I could try to do something about it myself. Your CTDB is already a marvelous gift to us
Title: CUETools DB
Post by: sauvage78 on 2010-05-22 22:04:30
Saxo:
I am not arguing about the usefullness of cuesheets, I am arguing about the usefullness (& honesty) of a database created in order to fix broken cuesheets.
The case were cuesheets can be broken are so rare (except broken drives & maybe bad settings), that the most obvious use of such a database would not actually be to repair bad cuesheets (like CTDB is doing for damaged audio), but obviously to create cuesheets for people without cuesheets at all ... which means that this people wouldn't physically own the CD but only a cheap pirated separate tracks copy. (If they cannot just re-create the cuesheet in a few sec from the CD)

Don't get me wrong: as a user I would indeed enjoy it if such a database would exist (who wouldn't except RIAA & Co?) but it is obvious to that this database would be used 99% of time to create a cuesheets for pirated files & 1% of time to actually repair broken cuesheets ... so I am only sarcastic about the true utility of such a database. I am even more sarcatic as over the year I have meet many people which were teaching me that cuesheets were not usefull ... thks god I never listened to them.

You can argue that the AR & CTDB databases can be miss-used for piracy too, & with that I totally agree, but the proportion between legal use & abuse is much lesser with AR & CTDB than with a cuesheet database ... simply because scratches happens way more often than broken INDEX.

Maybe the existence of such a database would prove that I am completely wrong & that cue sheet variations happens all the time ... afterall no-one (even Spoon I think & at last not me) suspected that there were so many pressings with offset difference before the creation of the AR database. Personnaly I would be more curious about such a database in order to learn about the robustness/consistency of cuesheets than just in order to fix cuesheets.
Title: CUETools DB
Post by: sauvage78 on 2010-05-27 18:34:42
Hi Greg,
In a discussion (http://www.hydrogenaudio.org/forums/index.php?showtopic=81152&st=0) (that took a bad direction because I was making a lot of assumptions about CTDB which ended to be erroneous, sorry if it is painfull to read), it appeared that CTDB seems to accept non accurate submissions as long as it is done with cueripper (I didn't test it myself). The so-called AR(1) entries.

I am now worried about this, as from my un-educated point of view it seems that it means that you possibly admit entries to CDTB that might be damaged. The proportion of bad submissions should be low & even in this case it is up to the user to verify that once "fixed" the rip is more accurate than before trying to fix it. ... still even if I think that I know how to avoid "fixing" damaged rips with damaged CTDB submissions, I don't like idea that such non accurate CTDB entries will appear in my .accurip.

So, without resurecting this thread, I wanted to simply ask you: Why you are doing this & why you think that this is a non-issue ? I ask because most of the reason why the linked thread derailed is that I couldn't believe myself that CTDB was behaving like this.

I don't understand the logic of being AR(2) dependent if the rip comes from EAC or dbpoweramp & in the same time not dependent of the accuracy at all if the rip comes from cueripper.

Is there any rationnal explanation ? Is it just that you wanted to populate the database quickly ? (too quickly ...) Have you stored any additionnal informations that would allow you to re-check the accuracy of these suspicious submissions later ? (& purge them ...) Do you think that this is a minor issue ?

I don't understand why you store submissions that might be damaged & which does take some space on your database/hardrive when I think I recall that you said that you couldn't store a stronger correction file to keep the database small.

As I don't want to make assumptions about CTDB anymore (I get trouble with Greynol when I do so), I thought that I'd rather ask you directly, sorry ... Am i just completely wrong once again ?

Thks for any answers & all apologies if this is a non-issue.
Title: CUETools DB
Post by: greynol on 2010-05-27 18:48:08
You might want to go over this thread again since some of what you're asking has already been answered.
http://www.hydrogenaudio.org/forums/index....st&p=704446 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=79882&view=findpost&p=704446)
Title: CUETools DB
Post by: sauvage78 on 2010-05-27 19:03:33
Gregory:
Quote
It doesn't submit in Burst mode or when there were suspicious positions with many retries.


Thks for the link Greynol ... I missed it. This reduces even more the probability of bad submissions.
Sorry but as there are no real FAQ or tutorial for AR & CTDB, the information is splitted everywhere.
Even if I am faulty, it is hard to follow all AR & CTDB subtilities. Sorry if I made you waste some time, but it was once again usefull to me to have an "unfriendly" discussion with you
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-27 19:57:26
It seems that it means that you possibly admit entries to CDTB that might be damaged.

Yes, of course, and there's no way around it. Same goes for AccurateRip - when the first submission for a CD is made, there's nothing to compare it with, and no way to make sure it's really accurate.

I don't like idea that such non accurate CTDB entries will appear in my .accurip.

I don't like it either. If e.g. AccurateRip log would show all non-matching entries, it would be very long and unreadable. CTDB verification log is much shorter because it doesn't have separate entries for pressings with different offsets, but still at some point we will probably have to ignore entries that don't match and have low confidence (1?). For now it shows all the entries and it's up to user to interpret this information.

I don't understand the logic of being AR(2) dependent if the rip comes from EAC or dbpoweramp & in the same time not dependent of the accuracy at all if the rip comes from cueripper.

Not that it's entirely logical, but there are reasons.

In ideal world CTDB shouldn't depend on AR at all and only accept submissions from rippers. You asked if i consider CUERipper submissions a hack - well no, it's the other way around. I had to add another way to submit data because 1) Only one ripper supports CTDB so far, 2) It doesn't even work on some drives, 3) I needed to jump-start the database, because noone would have taken it seriously if it only had a dozen CDs. So it's CUETools' submit feature that is a hack, although a necessary one.

The absolute minimum requirement for CUETools submissions is some evidence that the submitted files really come from a CD and were not modified since they were ripped. CUETools could in theory submit any rip that passes AR check with confidence of 1, or even as someone suggested, a rip that doesn't pass AR check, but has a log and matches CRCs from that log. I still hesitate whether i should do this or not. On one hand, this would speed up database growth. On the other hand, this would reduce data quality.

CTDB submissions use much more disk space/traffic, which demands higher data quality than in AR. As i said, in the end there's no way around it - there will be bad rips in the database, but we must minimize the number of those bad rips. So CUERipper for example doesn't submit a rip to CTDB if it had problems reading some sectors. EAC submits to AR anyway.

Have you stored any additionnal informations that would allow you to re-check the accuracy of these suspicious submissions later ? (& purge them ...)

I store the drive model and ripper version so that i could purge entries from virtual drives or ripper versions with known bugs. By the way, another reason to trust CUETools submissions less than CUERipper submissions is that there's no way i can find out drive model and ripper version (at least if there's no EAC log). Spoon might purge submissions from that drive later, but the erroneous entry would already be in CTDB.
Title: CUETools DB
Post by: greynol on 2010-05-27 20:15:41
Sorry but as there are no real FAQ or tutorial for AR & CTDB, the information is splitted everywhere.
Even if I am faulty, it is hard to follow all AR & CTDB subtilities. Sorry if I made you waste some time, but it was once again usefull to me to have an "unfriendly" discussion with you

Search for information before making claims and you should be ok.  If you are unsure it is better to ask a question rather than speculate.  It will result in far fewer incidents of being told you are not correct.  This is advice I try to follow myself.

A while back I tried to split the discussions between CUERipper and CUETools.  Now with CTDB, I feel more justified than ever with that decision.  Even though they are linked to one another and are bundled together (which I think is a mistake), they each serve a specific purpose.  Discussions are easier to follow when they are primarily focused on only one specific purpose.  I don't think there are a lot of people who want to search through a discussion with hundreds of posts in order get information about something that doesn't show up until far into the discussion.  If it weren't for this thread a lot of people would be asking, "WTF is CTDB? Where did this come from?".  In fact this is exactly how this discussion got started.  Wading through tons of logs with requests for clarification, requests for assistance with batch processing, feature requests, etc. with CUETools in order to find technical information about a ripping program is a nightmare.  Anyhow, sorry for ranting.
Title: CUETools DB
Post by: greynol on 2010-05-27 20:32:17
By the way, another reason to trust CUETools submissions less than CUERipper submissions is that there's no way i can find out drive model and ripper version (at least if there's no EAC log).
Let's also accept the cold hard reality that many people use CUETools to check their illegal downloadz.  The initial rip could easily have been in error but still submitted to AR.  It could have easily been solidified in the database through a subsequent submission from CD-R burned from a CUE sheet and a drive with a write offset of zero or one that is compensated if not zero.  If it's the only entry for that particular disc ID, it doesn't even need solidification [AR (1)]. (EDIT: thankfully AR(1) results are not submitted.)

With CTDB, there is a very real risk that a good rip from an honest person ripping a factory-pressed CD may be altered to match an illegal upload with an error that has been propagated through file sharing and made exponentially worse by this new system.  I think the term viral is a fitting description.
Title: CUETools DB
Post by: Teknojnky on 2010-05-27 20:33:15
maybe cuetools / cueripper have grown, or about to grow, to the point of having a dedicated subforum rather than to try to keep questions etc relegated to ever growing threads.
Title: CUETools DB
Post by: sauvage78 on 2010-05-27 20:40:44
Thks for the answers. I am reassured that the issue is much smaller than I feared.

Greg:
Quote
even as someone suggested, a rip that doesn't pass AR check, but has a log and matches CRCs from that log. I still hesitate whether i should do this or not.


This is definitely not a good idea. Logs are too easy to edit to be reliable, this is the exact reason why I delete logs. There is even insane developpers around which codes log generators... The only logs you can trust are your own, & most of the time (if perfect) your own logs are useless to you (except for enhanced CD). If you do this, the database will instantly be spoiled by guys uploading to CTDB rips they grabbed from P2P & some of those will have fake/edited logs. You would be surprised by how some people are able to lie to themselves to make them look important to their P2P communities. (specially teenagers...)

Greg:
Quote
In ideal world CTDB shouldn't depend on AR at all


In my ideal world CTDB should rely entirely on AR(2)  I guess it's a question of point of view. I understand that you trust your own software & that you want to be independent from Spoon.
But I have seen so many non-burst EAC log without re-reads with "no error occured" that I hardly trust even ultra paranoid rip anymore as long I don't have an AR match. On the one hand, I think that relying on parameters no matter how good is not perfect & on the other hand I agree that the idea of having a "pure" database is insane, so I guess lowering the risk is, in a way, a fair deal.
I think I can live with the way it works now, even if I was at first suprised by your choice.
Title: CUETools DB
Post by: greynol on 2010-05-27 20:54:37
maybe cuetools / cueripper have grown, or about to grow, to the point of having a dedicated subforum rather than to try to keep questions etc relegated to ever growing threads.

Good idea, though I think REACT is in line ahead of CUETools/CUERipper/CTDB.  This is ignoring the countless posts spent exclusively on EAC.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-27 20:56:21
Greg:
Quote
In ideal world CTDB shouldn't depend on AR at all

I understand that you trust your own software & that you want to be independent from Spoon.

It's not that. I think that users would benefit from my cooperation with Mr. Spoon, and if he decides to work together on CTDB and add CTDB support to dbPoweramp that would be great. But databases should be separate, because they are too different to depend on each other.
Title: CUETools DB
Post by: greynol on 2010-05-27 21:06:54
Logs are too easy to edit to be reliable, this is the exact reason why I delete logs.
That sounds like an admission that you use the program to check material that is not distributed through legal legal means.

The only logs you can trust are your own, & most of the time (if perfect) your own logs are useless to you (except for enhanced CD).
Not useless if you can't tell which of your rips are error-free and which are not.  Hopefully you have them organized by some other means.

Quote
In ideal world CTDB shouldn't depend on AR at all
+1!
To add, in an ideal world CTDB will only be populated by factory-pressed CD and only once per physical CD.  In the meantime, I suppose AR (2) is currently the best filter.
Title: CUETools DB
Post by: Fuki on 2010-05-27 21:25:51
Hi!

I've been away for a while. Since then I see a lot of good things have happened.
The last time I was here CueTools were at 2.0.4a.
I've upgraded to 2.0.9 and I'm impressed with CTDB: Great idea!

I'm wondering what does this mean:
Code: [Select]
        [ CTDBID ] Status
        [9ee07e8b] (20/20) Has no data track, No match

Does it mean that disc is correctly ripped but with no data track or that it couldn't be verified because it does not contain data track?

Regards,
Fuki
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-27 21:38:06
It means that database only knows about a pressing without data track.

It tried to ignore the data track and compare the rip, that works in some cases, but your pressing (with data track) appears to be too different from the pressing in database (without data track).
Title: CUETools DB
Post by: Fuki on 2010-05-27 21:48:33
This rip has no data track...
BTW What would be the message if the rips are identical except for the data track?
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-27 21:50:58
Has no data track, Accurately Ripped
Title: CUETools DB
Post by: Fuki on 2010-05-27 21:58:43
CUETools report this:
Code: [Select]
CD-Extra data track length 02:13:73 - 02:14:72.

before CTDB TOCID.
I'm not sure what would that mean because the disc has only audio tracks.
Title: CUETools DB
Post by: sauvage78 on 2010-05-27 22:08:02
greynol:
Quote
Hopefully you have them organized by some other means.

I rely 100% on AR(2). My collection is organized from AR results.:
AR(2), Silent Track, Data Track, AR(1), Not in Database, not AR(2) ... I re-check non AR(2) every month, hence the reason why I care a lot about .accurip & not at all about .log.
I wouldn't be so vocal in AR/CTDB/Cuetools threads, if I wasn't so dependent on Spoon & Greg's work.
Even if very thanksfull to Andree, I am slowly leaving the EAC sect & their log fanatism.
Title: CUETools DB
Post by: sauvage78 on 2010-05-27 22:24:28
Fuki:
If I were Greynol  I would say that "this sound like an admission that you use the program to check material that is not distributed through legal legal means." because your TOC says it is an enhanced CD, which you seem unaware. But hopefully I am not Greynol ! lol

Edit:
More seriously if this is an old EAC rip of yours without TOC in the log, you can try to manually add the data length in order to find it in the AR database. Try all length between 02:13:73 & 02:14:72, good luck. CTDB is actually useless to you because there is non-enhanced entry that doesn't match your enhanced pressing. Once you find the exact data length if it happens to be not accurate, then you will have to wait for your enhanced pressing to be submitted to CTDB by someone else before you can try to fix yours.
Title: CUETools DB
Post by: Fuki on 2010-05-28 08:59:27
Tnx sauvage78!
Like a lot of the rips I have on my hard drive, unfortunately I don't have the original any more so I cannot be sure if it had had the CD-Extra data track or not, but as I can remember there wasn't any.
Title: CUETools DB
Post by: spoon on 2010-05-28 09:29:07
>I store the drive model and ripper version so that i could purge entries from virtual drives or ripper versions with known bugs.

Unfortunately a percentage (around 5% of virtual drive users) set the virtual drive name to a real drive name...
Title: CUETools DB
Post by: Fandango on 2010-05-28 13:46:06
Unfortunately a percentage (around 5% of virtual drive users) set the virtual drive name to a real drive name...

So I guess AR.dll analyses the device directly. How do you do it? Or is it security by obscurity... I mean I doubt any virtual drive software developer would want to re-write their software to beat AccurateRip's detection, would they? If at all they care more about game DRM systems than about audio rippers I guess, so I believe there's no loss for the safety of the AR database when you disclose how you do it. 

As you point out the way CUE Ripper does it is not enough, since anyone who's able to read and follow a tutorial is able to beat it. Not that it should be a concern for people who only rip or check rips of valid retail CDs they have physical access to...
Title: CUETools DB
Post by: Goratrix on 2010-05-28 14:30:10
More seriously if this is an old EAC rip of yours without TOC in the log, you can try to manually add the data length in order to find it in the AR database. Try all length between 02:13:73 & 02:14:72, good luck.


Heh, even at the risk of being accused to be from the "use the program to check material that is not distributed through legal legal means"-camp, there is something that I thought of recently. Would it be somehow possible for CueTools to try all the various data track lenghts to see if there is one that matches? Or would that be too time/processing-intensive?

EDIT: eh, scratch it, I just remembered Foobar2000 already does AR verification without data track, so it's possible.
Title: CUETools DB
Post by: Saxo on 2010-05-28 15:57:57
Even if very thanksfull to Andree, I am slowly leaving the EAC sect & their log fanatism.


I totally agree  ! Also, EAC has too much options that could f*** up your rip. I'm also VERY thankful to that German guy since we won't be at this point in accurate rip without him, but I think CUERipper is the future
Title: CUETools DB
Post by: sauvage78 on 2010-05-29 06:30:46
Goratrix:
I already asked Greg for such a feature before the creation of CTDB, but now with CTDB as time goes by it will become less interesting because when populated CTDB should store the majority of possible data lengths. What is more interesting IMHO is the support of .TOC to actually store this data somewhere on your HDD for old EAC rips with no TOC or rips with no log.

Edit:
I just noticed a misstake in my post #91 (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=79882&view=findpost&p=707149) that I cannot edit anymore:
Sauvage78
Quote
because your TOC says it is an enhanced CD

should be read
"because your cuesheet says it is an enhanced CD" you have no TOC as you don't know your data length.
Title: CUETools DB
Post by: sauvage78 on 2010-05-29 07:56:00
Saxo:
Actually I still use EAC, so what I am critisizing is more some users of EAC believing that there would be some magic in their logs because some biased EAC guide told them so. An AR(2) .accurip being IMHO more trustworthy than logs informations, it pretty much kills their use (As soon as you get an AR match). The people which over-trusted EAC logs may have some surprise when they'll check their AR accuracy. A small proportion (1-2%) of their rips will not be accurate despite their so-called "perfect logs" & they might discover that AR is right. The proportion is tied to the settings they used for their drive, but even with good settings a small chance of error seems to remains. I have already found faulty logs with settings that I supposed to be right (with listenable scratches in tracks pointed out by AR) but I have never found an AR(2) rip with listenable scratches. Hence my disenchantment for logs. I am not beating EAC or all use of logs, I am only pointing out the limit of EAC logs & the fact that AR can make people independent from EAC. EAC has ruled for so long that some people seems to think that only EAC can achieve accurate rips. This is not true, AR(2) proves it. Now maybe I trust AR(2) too much & maybe being dependent from AR is not much better than being dependent from EAC, only time will tell. My only fear is Spoon neglecting its database, then I would (maybe) regret having deleted my EAC logs. But even is this nightmare case, actually I don't see why my AR(2) .accurip would suddenly be less valuable. The right use of .log is not to keep them as an evidence that your rip is "perfect" (this is more the use of .accurip), the right use of .log is to keep record of the used settings in case something went wrong, as nothing can be wrong if your rip is AR(2), .log becomes useless. My conclusion is that .log are only usefull when something is wrong (& in special cases like keeping the TOC for enhanced CD). So the problem is not EAC & with or without logs, but how you use it.
Title: CUETools DB
Post by: Fuki on 2010-05-29 18:05:53
Even if very thanksfull to Andree, I am slowly leaving the EAC sect & their log fanatism.

I totally agree  ! Also, EAC has too much options that could f*** up your rip. I'm also VERY thankful to that German guy since we won't be at this point in accurate rip without him, but I think CUERipper is the future

I tried the CUERipper, and it looks nice & simple (which is good), but I'm missing the option for controlling the additional command line options when ripping to MP3. I miss this because I would like to encode MP3s with 0 quality (-q 0), and would like to include replygain (--replaygain-accurate). Is there a chance for implementing this?

Regards,
Fuki
Title: CUETools DB
Post by: greynol on 2010-05-29 18:13:17
This topic is about the CUEToolsDB, not about configuring CUERipper or about why you don't like EAC.
Title: CUETools DB
Post by: Eli on 2010-05-31 04:22:35
Greg:
Quote
In ideal world CTDB shouldn't depend on AR at all

I understand that you trust your own software & that you want to be independent from Spoon.

It's not that. I think that users would benefit from my cooperation with Mr. Spoon, and if he decides to work together on CTDB and add CTDB support to dbPoweramp that would be great. But databases should be separate, because they are too different to depend on each other.


Have you been in contact with spoon. Posts on his forum indicate this is not a priority. However, maybe he would be willing to work with you to implement it in dBpoweramp? Please contact him. I would LOVE to have CTDB in dB.
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-05-31 05:21:36
Mr. Spoon obviously reads this topic. I don't see the point in reminding him each day  Putting pressure on a developer usually doesn't help. Let him think about it.
Title: CUETools DB
Post by: Eli on 2010-05-31 14:27:52
Yes, its clear he reads it. My thought was if you were interested and he was willing, that you do the development of CTDB for integration with dBpoweramp. It does not seem that thats where he wants to dedicate his effort at this time, however, maybe if you were willing to do the development he would be more interested.
Title: CUETools DB
Post by: sauvage78 on 2010-06-01 05:43:03
Even without working together, now that we're in June, it would be nice if we could have both a new version of Cuetools & an update of the AR database ... say before the 13rd of the month ! (with the cuetools update coming slightly before the AR database update, it would be perfect ) ... not that I want to force you both with anything but as it seems that if write down my wish here, it will not fall in the ears of deaf developers, (even if once again off-topic), I thought that I would authorize myself to speak aloud so that I can, maybe, be fulfilled  It seems reasonnable no ? specially as there was no May AR update & even if the Cuetools update is minor. Nevermind if I ask too much (I just thought the timing was good).
Title: CUETools DB
Post by: sauvage78 on 2010-06-16 15:20:51
Thx to Spoon, the June update of AR is now online  if you plan to update Cuetools, Greg, plz do it spoon because the monthly update of my non-AR(2) rips will spoon begin

[Any update regarding silent tracks inconsistent handling & .toc support for enhanced CD would be specially welcomed !]

What does it have to do in the CTDB thread you ask ? ... well nothing except that I will check my damaged rips very spoon too.

Edit: Some accidentally added p typo have been striked through
Title: CUETools DB
Post by: Porcus on 2010-09-01 10:32:57
I'd like to help populating the database:

I have a few thousand CDs ripped (with dBpoweramp), stored as one-.flac-per-track, one-folder-per-disc. Is there a command line way to script "submit to CTDB" based on folder without running AR verification on each track, which would take a couple of weeks on my hardware?


If so: which ones should I exclude? I can identify the following:

- discs ripped with known errors -- "insecure" in dBpoweramp (or worse, getting an unrecoverable error in Secure mode and hence burst-ripped only to get a sound file out) will only pollute the database, right?

- discs Secure (Warning), i.e. re-ripped until 10 identical hits (where the max # of re-rips is sometimes set so high as 9999) -- if I include all these, the probability of at least one error would be fairly significant. Still worth submitting, or should they be left out?

- discs Secure but Inaccurate (i.e., different pressings). I could try to verify by CueTools first, is that any good for the purpose of submitting to CTDB? Should they then be offset-corrected? What about those which fail to verify -- different pressings sometimes do differ in bits?

- discs defective-by-design? Would be good if these deficiencies could be actually fixed in the database, but I cannot imagine there is any authorative way to tell which rip is "the right" for these.

- discs with HTOA, stored as track #0 (dBpoweramp's behaviour). Would I e.g. need to pre-generate a cuesheet?

- discs with data tracks.
Title: CUETools DB
Post by: radu on 2010-09-13 20:21:12
Hi,

I'm currently verifying my rips with CUETools.

I am getting some very strange results. Most of the times there are a lot of results in AR db and I'm lucky if there is any in the CTDB, and some other times there are even more results in the AR db and almost as many in the CTDB. Can this be explained?

Here are some examples: note how Aphex Twin CD has a fairly big number of rips in AR db and none in CTDB, and how Animal Collective CD has a lot of rips in AR db, matched by an equal number in CTDB

.\Allen Toussaint - 2009-04-20 - The Bright Mississippi\Allen Toussaint - 2009-04-20 - The Bright Mississippi.cue: unable to locate the audio files.
.\Animal Collective - 2005-10-18 - Feels\Animal Collective - 2005-10-18 - Feels.cue: AR: rip accurate (84/84), CTDB: verified OK, confidence 79.
.\Animal Collective - 2007-09-11 - Strawberry Jam\Animal Collective - 2007-09-11 - Strawberry Jam.cue: AR: rip accurate (97/97), CTDB: verified OK, confidence 90.
.\Animal Collective - 2009-01-20 - Merriweather Post Pavilion\Animal Collective - 2009-01-20 - Merriweather Post Pavilion.cue: AR: rip accurate (238/238), CTDB: verified OK, confidence 238.
.\Aphex Twin - 2001-10-23 - Drukqs (disc 1)\Aphex Twin - 2001-10-23 - Drukqs (disc 1).cue: AR: rip accurate (85/85), CTDB: disk not present in database.
.\Aphex Twin - 2001-10-23 - Drukqs (disc 2)\Aphex Twin - 2001-10-23 - Drukqs (disc 2).cue: AR: rip accurate (80/80), CTDB: disk not present in database.
.\Arcade Fire - 2004-09-14 - Funeral\Arcade Fire - 2004-09-14 - Funeral.cue: AR: rip accurate (598/600), CTDB: verified OK, confidence 585.
.\Artifacts - 1994-10-25 - Between a Rock and a Hard Place\Artifacts - 1994-10-25 - Between a Rock and a Hard Place.cue: AR: rip accurate (3/3), CTDB: disk not present in database.
.\Arts the Beat Doctor - 2007-07-12 - Transitions\Arts the Beat Doctor - 2007-07-12 - Transitions.cue: AR: rip accurate (1/1), CTDB: disk not present in database.
.\At the Drive-In - 2000-09-12 - Relationship of Command\At the Drive-In - 2000-09-12 - Relationship of Command.cue: AR: rip accurate (36/36), CTDB: disk not present in database.
.\Audioslave - 2002-11-19 - Audioslave\Audioslave - 2002-11-19 - Audioslave.cue: AR: rip accurate (545/803), CTDB: verified OK, confidence 513.

And those two CDs are the most strange to me:

.\Audioslave - 2005-05-24 - Out of Exile\Audioslave - 2005-05-24 - Out of Exile.cue: AR: rip accurate (581/589), CTDB: verified OK, confidence 513.
.\Audioslave - 2006-09 - Revelations\Audioslave - 2006-09 - Revelations.cue: AR: rip accurate (454/458), CTDB: disk not present in database.
Title: CUETools DB
Post by: Eli on 2010-11-20 19:33:04
I'd like to help populating the database:

I have a few thousand CDs ripped (with dBpoweramp), stored as one-.flac-per-track, one-folder-per-disc. Is there a command line way to script "submit to CTDB" based on folder without running AR verification on each track, which would take a couple of weeks on my hardware?



I also have a few thousand discs I have personally ripped w/ dBpoweramp. All of them are accurate/secure. Mine are ripped as single files and all stored as FLACs (no cues, TOCs are in FLACs as metadata). A utility to scan my library and submitt to CTDB would be nice.
Title: CUETools DB
Post by: Eli on 2010-11-26 00:54:50
Gregory, would you consider creating an exe that another ripper could use to access CTDB? In the case of dbpoweramp, it has the option to run a DSP called "run external" ( http://www.dbpoweramp.com/dbpoweramp-dsp.htm (http://www.dbpoweramp.com/dbpoweramp-dsp.htm) ). This would allow an external exe to scan the results of the rip as well as read the log file. If the rip is accurate it could be added to the CTDB. If it needs to be repaired and its in the CTDB this option could be given. It could also add results of confirmation for accuracy to the log and possibly to the meta-data of the ripped files. It doesn't sound like spoon will be adding support any time soon, but this may be a way to get it in there.
Title: CUETools DB
Post by: Cynic on 2010-11-26 11:27:47
 
A utility to scan my library and submitt to CTDB would be nice.

CUETools already does half of this (not the scanning part, but perhaps some clever bloke could write a script for that?), just verify your albums with Submit selected in the pulldown menu.

Title: CUETools DB
Post by: Eli on 2010-11-27 16:46:50
 
A utility to scan my library and submitt to CTDB would be nice.

CUETools already does half of this (not the scanning part, but perhaps some clever bloke could write a script for that?), just verify your albums with Submit selected in the pulldown menu.


As much as I would like to help the database I don't plan to manually go through thousands of albums. If I am missing a way to do this w/ CT please let me know.
Title: CUETools DB
Post by: Eli on 2010-11-27 17:10:24
Looks like CTDB could now be more fully integrated by an outside developer as spoon has opened the DSP architecture:

http://www.hydrogenaudio.org/forums/index....showtopic=85243 (http://www.hydrogenaudio.org/forums/index.php?showtopic=85243)
Title: CUETools DB
Post by: Gregory S. Chudov on 2010-11-27 19:40:20
As much as I would like to help the database I don't plan to manually go through thousands of albums. If I am missing a way to do this w/ CT please let me know.

Just select a folder with your rips, choose 'Verify' action and 'submit' from the list below.

Looks like CTDB could now be more fully integrated by an outside developer as spoon has opened the DSP architecture:

http://www.hydrogenaudio.org/forums/index....showtopic=85243 (http://www.hydrogenaudio.org/forums/index.php?showtopic=85243)

That would be very nice. I'll be glad to help.
Title: CUETools DB
Post by: fadsplat on 2010-11-30 01:01:14
When trying to validate a rip, I get this error:
The specified path, file name, or both are too long.

Maybe the output path (for the log) was too long - the default template places it somewhere like C:\Documents and settings\username\My Music\.... which can easily get too long. I'll think how to handle this case better, maybe by abbreviating artist/album names to something shorter when generating a path.

1. Was this fixed?

Even with the information about the output path, the total path still seems less than 260, so I can't figure this out.

2. Are the names for album and tracks taken from the filenames or from the tags? I ask because I shortened the filenames and folder names significantly, yet I still see the error.

Thanks.
Title: CUETools DB
Post by: Eli on 2010-11-30 02:40:19
Looks like CTDB could now be more fully integrated by an outside developer as spoon has opened the DSP architecture:

http://www.hydrogenaudio.org/forums/index....showtopic=85243 (http://www.hydrogenaudio.org/forums/index.php?showtopic=85243)

That would be very nice. I'll be glad to help.


Gregory, are you indicating your intention to work on/develop a DSP to bring CTDB to dBpoweramp or your willingness to help a developer do so. Its just unclear to my how to read your response. Hopefully you intend to develop the DSP as no one has the knowledge set you have re CTDB.
Title: CUETools DB
Post by: Eli on 2010-12-21 19:32:37
Gregory, there have been recent discussion regarding consistent errors from drives, rippers, or discs. What information does CTDB store about the source of its data? Is the submitting drive and/or ripper stored? What about the source (previously ripped, vs cd ripper, etc...)
Title: CUETools DB
Post by: A_Day_Without_Me on 2010-12-27 02:16:04
Do you have to rip the disc using CueTools Ripper for a disc to be submitted or is disc submitted when using verify in CueTools?
Title: CUETools DB
Post by: greynol on 2010-12-27 04:00:42
Here you go eboyer93:
* CTDB now accepts rips with AR confidence >= 2

Title: CUETools DB
Post by: Eli on 2011-01-05 23:56:24
Gregory, can CueTools read the CD TOC tags used by dbpoweramp? Many CDs I have are not recognized as being in the AccurateRip DB when they are. Likely, because there are no CUE files. However, the songs are tagged with the CD TOC tag and that should be enough to recognize the disc in the AR DB.
Title: CUETools DB
Post by: Porcus on 2011-01-06 10:17:49
Gregory, can CueTools read the CD TOC tags used by dbpoweramp? Many CDs I have are not recognized as being in the AccurateRip DB when they are. Likely, because there are no CUE files. However, the songs are tagged with the CD TOC tag and that should be enough to recognize the disc in the AR DB.


I am not sure if CDTOC contains sufficient information, but the ACCURATERIPDISCID tag should?
Title: CUETools DB
Post by: Eli on 2011-01-12 14:04:26
Gregory, another question for you. How does CTDB identify unique discs? Pressings of the same data with different offsets are, basically the same discs. But what if a disc is remastered but has the same number of tracks with same tracks lengths, but different audio because its remastered?
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-01-12 14:36:17
Gregory, are you indicating your intention to work on/develop a DSP to bring CTDB to dBpoweramp or your willingness to help a developer do so.

The latter.

Gregory, there have been recent discussion regarding consistent errors from drives, rippers, or discs. What information does CTDB store about the source of its data? Is the submitting drive and/or ripper stored? What about the source (previously ripped, vs cd ripper, etc...)

It does store the submitting drive and unique user id hash. So it's possible in theory to eliminate erroneous submissions if a certain drive is found to be malfunctioning. In practice this hasn't been used yet. CTDB's purpose is not so much to tell a good rip from a bad rip, but to provide user with means to fix his copy, when he is certain that his copy is damaged and there is an undamaged one in database. It's up to user to decide on those things.

Do you have to rip the disc using CueTools Ripper for a disc to be submitted or is disc submitted when using verify in CueTools?

Currently, apart from CUETools ripper you can submit a disc using verify in CueTools if you select 'submit' mode explicitly. By default CUETools doesn't submit anything. Also, as the number of ripper submissions grows (http://db.cuetools.net/graph3.php), i will disable CueTools submissions at a certain point.

Gregory, can CueTools read the CD TOC tags used by dbpoweramp? Many CDs I have are not recognized as being in the AccurateRip DB when they are. Likely, because there are no CUE files. However, the songs are tagged with the CD TOC tag and that should be enough to recognize the disc in the AR DB.

CUETools doesn't support this tag currently. If CD TOC tag contains enough information, e.g. length of each track in frames including the last one, this should be enough to identify a disc, so i will keep that in mind.

I am not sure if CDTOC contains sufficient information, but the ACCURATERIPDISCID tag should?

If i remember correctly, CUETools recognizes this tag.

Gregory, another question for you. How does CTDB identify unique discs? Pressings of the same data with different offsets are, basically the same discs. But what if a disc is remastered but has the same number of tracks with same tracks lengths, but different audio because its remastered?

So far, i've never seen remasters having the same TOC. If this happens, those discs will have the same Id, but database can handle several entries with same Id. When correcting a rip you will be presented with a list of entries with this id, but will be able to use correction only with one of them. The other entry will look like it has too many differences with your disc to be used.

There's a similar example currently in database. I seem to remember that there are two sets of pressings for original CD release of Pink Floyd's Dark Side of the Moon. Probably this: http://db.cuetools.net/index.php?tocid=Md6...IFPA3GiDVrsn5c- (http://db.cuetools.net/index.php?tocid=Md6h9WxfnN6p0IFPA3GiDVrsn5c-) Those are not remasters, but differ slightly, perhaps in several hundred samples. One variant has AR confidence of 233, the second one - 530. My guess is, one of the master discs for this CD contained inaudible errors, and about 30% of all produced CDs have exactly the same errors. Using CTDB you can switch back and forth between those two variants and try to find out which one is 'right'. I wasn't able to hear any difference in the sound at the locations indicated by CTDB, but perhaps you can find out by looking at the waveshapes using Audacity or something like that.
Title: CUETools DB
Post by: Eli on 2011-01-12 18:08:01
Gregory, are you indicating your intention to work on/develop a DSP to bring CTDB to dBpoweramp or your willingness to help a developer do so.

The latter.



Thanks for the answer. Maybe if you find the time you could lay the groundwork with an open source DSP plug-in for CTDB. Honestly, I don't think anyone but you has the expertise and interest to get the project off the ground. I am very thankful spoon opened up the DSP architecture, now hopefully some developers take advantage of this.
Title: CUETools DB
Post by: tunetyme on 2011-04-18 05:33:07
I'm new to this forum.  I have had a bug about good rips since I started about 10 years ago.  I have taken Spoon's advice on hardware and software with great success.  One of his recommendations was Plextor Drives.  I bought the Plextor Premium which has Plextools.  This program gives me the ability to analyze my discs and measure the C1, C2 and uncorrectable errors.  I have ripped over 3,000 CD's.  I use EAC to rip all my CD's.  I predate the accurate rip DB.  What I have found is about 10% of the new CD's I purchase will be a very poor pressing with High C1, C2 and uncorrectable errors.  Frankly, I believe that the record companies know that they are selling garbage.  I take the reports to the retailer and return defective discs all the time.  I've given up on exchanging them for the same disc because every time I have, the replacement has been defective also.  Some artists have even sued the record companies for putting out junk. 

In a nutshell, I find that it is the individual tracks that need to be replaced.  Most of the CD is acceptable except for one or two tracks.  With accurate rip I add (def) to the track name so I can identify it latter. 

I believe that if you have the ability to correct the errors on a given CD (to minimize size of the DB) it may be possible to address the individual tracks as a subset and correct only those tracks that need to be.  Ideally we would all have tracks that are comparable to the master itself.  The greatest challenge will be to have a bit perfect copy to use as the master.  I am looking forward to testing all my CD's and check the quality of my rips especially for those that I ripped prior to accurate rip.  BTW If you have a bad pressing even the accurate rip DB may be wrong.  I have over 40,000 tracks and I am sure I will be adding to my list of defective tracks.

I am enthusiastically in favor of these efforts and I will be happy to contribute any way I can.

Tunetyme
Title: CUETools DB
Post by: tunetyme on 2011-04-21 15:20:09
I ran verify and submit on my Rock Collection over 2,200 CD's.  I copied the info from the results pane.  It would be nice if there was a log file that organized the info and recommend a course of action to address each of the exceptions.  All have been ripped with EAC secure mode and about half predate accurate rip.

Here is the letter A from my Rock Collection I am assuming that this is a representative sample of my entire Rock Collection with 74 CD’s.  Some discs had multiple print outs with listing 2 different songs.  The key question is what to do with the info and how do I address the ones that could not be verified?

Letter A Test of Cue Tools verification

AccurateRip: disk not present in database, will not submit.

\ABBA\Gold, Greatest Hits
\AC DC\Back In Black
\AC DC\Dirty Deeds Done Dirt Cheap
\AC DC\Highway To Hell
\Aerosmith\Done With Mirrors
\Aerosmith\Get A Grip
\Aerosmith\Get Your Wings
\Aerosmith\Greatest Hits
\AC DC\High Voltage
\Aerosmith\Nine Lives
\Aerosmith\Permanent Vacation
\Aerosmith\Pump
\Aerosmith\Rocks
\Aerosmith\Toys In The Attic
\Aguilera, Christina\Stripped
\Air Supply\The Definitive Collection
\Allman Brothers Band, The\A Decade Of Hits
\Allman Brothers Band, The\Brothers And Sisters
\Allman Brothers Band, The\Dreams Disc 1
\Allman Brothers Band, The\Dreams Disc 2
\Allman Brothers Band, The\Dreams Disc 3
\Allman Brothers Band, The\Dreams Disc 4
\Allman Brothers Band, The\Eat A Peach
\Allman Brothers Band, The\Idlewild South
\Allman Brothers Band, The\Live at Ludlow Garage Disk 1
\Allman Brothers Band, The\The Legendary Hits
\America\America's Greatest Hits - History
\America\Encore- More Greatest Hits
\Animals, The\The Best of the Animals
\Animals, The\The Complete Animals Disc 2
\Anka, Paul\His All Time Greatest Hits (30th Anniversary Collecti
\Ant, Adam\Antics in the Forbidden Zone
\Archies, The\Sugar, Sugar
\Argent\Greatest Hits
\Asia\Asia
\Asia\The Very Best of Asia- Heat of the Moment (1982-1990)
\Association, The\Greatest Hits!
\Astley, Rick\Greatest Hits
\Atlanta Rhythm Section\A Rock And Roll Alternative
\Atlanta Rhythm Section\The Best of Atlanta Rhythm Section [20th Century        Masters- The Millennium Collection]
\Atlantic Starr\Ultimate Collection
\Avalon, Frankie\The Frankie Avalon Collection
\Average White Band, The\AWB
43

recently verified: AR: database access error: Unable to connect to the remote server, CTDB: BadRequest.

\AC DC\Let There Be Roc
\AC DC\Powerage
\Adams, Bryan\Bryan Adams (Remastered)
\Adams, Bryan\Cuts Like A Knife
\Adams, Bryan\Into The Fire
\Adams, Bryan\On A Day Like Today
\Adams, Bryan\Reckless
\Adams, Bryan\Room Service
\Adams, Bryan\You Want It You Got It
\Adams, Bryan\11 (Deluxe Edition)
\Adams, Bryan\Waking Up The Neighbours
11

synchronization was lost.

\Action, The\Rolled Gold
\Adams, Bryan\So Far So Good
\Amboy Dukes, The\Loaded For Bear-The Best of Ted Nugent Ad The Amboy Dukes
\Ashford & Simpson\The Best of Ashford & Simpson
4

AccurateRip: confidence too low, will not submit.

\Amboy Dukes, The\Migration
\Animals, The\Animal Tracks (Japanese Edition)
\Animals, The\The Singles Plus Disc 1
\Association, The\Songs That Made Them Famous (1984)
\Atlanta Rhythm Section\Third Annual Pipe Dream
5

AR: offset X, rip accurate (X/Y), CTDB: disk not present in database, XY- has been uploaded.

\Allman Brothers Band, The\Beginnings
\Allman Brothers Band, The\Seven Turns
\Alpert, Herb\Definitive Hits
\Alpert, Herb\Going Places
\Ambrosia\Anthology
\Animals, The\The Singles Plus Disc 2
\April Wine\Best Of
7

recently verified: AR: rip accurate (X/Y).

\AC DC\For Those About To Rock
\Adams, Bryan\18 'Til I Die
2

CUEToolsDB: verified OK, confidence 112.

\a-ha\Hunting High and Low
1

AR: rip not accurate (0/2), CTDB: could not be verified, will not submit.

\Animals, The\The Complete Animals Disc 1
1

Tunetyme
Title: CUETools DB
Post by: tunetyme on 2011-04-22 20:55:14
I initially thought this program screwed up some my tracks cover art.  I've discovered it has deleted my tags.  Not on all my files but mostly 1-3 tracks on an album and up to 10 on a given album.  Out of 29,000 plus tracks I have to repopulate over 3,500 tracks.  Thankfully, I use JRiver Media Center and all the info is stored in an independent database.  Because of the cover art problem I have to album by album.  This is not good.

USE THIS SOFTWARE WITH GREAT CAUTION!!!!

I would like to help with this software but I need to be sure that it isn't going to damage my tags or files before I run it again.  It would also be nice if there was a report or log generated after a run with everything organized by Category (see previous post) and artist.  At that point I can at least update my database files to indicate all those that are accurate.  This program has tremendous potential.  It might be nice if the tags themselves could be updated to indicate that they have been verified by creating a field within the tags. 

Tunetyme
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-04-23 20:57:59
I believe that if you have the ability to correct the errors on a given CD (to minimize size of the DB) it may be possible to address the individual tracks as a subset and correct only those tracks that need to be.


You mean without reencoding the rest of the tracks? In theory that could be done, but i don't think that it would be a very important feature.
But you will still need to read all the other tracks to be able to use the recovery record.

It was suggested several times to make per-track recovery records, but that's impractical, because the size of the database would be multiplied by the number of tracks. Smaller recovery records won't work. And if we could afford to have tens of terabytes of data in the database, i'd prefer to use this space for larger whole-disc recovery records.
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-04-23 21:11:34
The key question is what to do with the info and how do I address the ones that could not be verified?


AccurateRip: disk not present in database, will not submit.
There are two possibilities here:
1) When a popular CD that should be in AR isn't found, that means that CUETools wasn't able to determine AR disc id,
because of the missing pregap or changed order of tracks. Most likely that's because you didn't keep the cue sheet.
There's no general way to fix it. To avoid this problem in the future, you should preserve the cue sheet.
2) CD is ok but not yet in AR database.
You can either wait for it to appear there and resubmit with CUETools, or use a ripper which can submit
directly to CTDB without AR verification.

recently verified: AR: database access error: Unable to connect to the remote server, CTDB: BadRequest.
Internet connection was down, you'll have to resubmit.

synchronization was lost.
That usually means that flac file was truncated. You'll have to rerip it.

AccurateRip: confidence too low, will not submit.
You can either wait until it's confirmed in AR and resubmit with CUETools, or use a ripper which can submit
directly to CTDB without AR verification.

AR: offset X, rip accurate (X/Y), CTDB: disk not present in database, XY- has been uploaded.
You successfully submitted CD to CTDB.

recently verified: AR: rip accurate (X/Y).
That's probably a bit annoying new feature, which i shouldn't have made on by default.
Turn off the 'Skip recently verified' checkbox. (icon with a clock)

CUEToolsDB: verified OK, confidence 112.
CD is already in CTDB, you don't need to submit it.

AR: rip not accurate (0/2), CTDB: could not be verified, will not submit.
Your rip probably contains errors, but you can't be sure because confidence is low.
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-04-23 21:17:29
I would like to help with this software but I need to be sure that it isn't going to damage my tags or files before I run it again.


CUETools doesn't modify tags by default. And nothing like this have happened before.
You either have turned on the option "Write AccurateRip tags" on verify in advanced settings, or it wasn't CUETools that ate your tags.
If you did turn on this option, and it did screw up your tags, please send me a sample file for which this happens.
Title: CUETools DB
Post by: tunetyme on 2011-04-27 16:52:30
I have selected verify from the drop down menu.  Under action I selected verify and submit. Mode all three are checked.  I am using flac files.  I don't see an advanced or write accurate rip tags option in 2.1.1.

I have been repairing all my tags from JRiver Media Center where all the tag info is stored independently.  I assume I can send you info to the address listed on your website.  It will take me awhile to sort through my file to see what hasn't been fixed.  I think the only thing left is cover art problems.

I am willing to do some testing on a smaller scale.  When I first ran this I used a few artist and then ran it on my whole rock database of 2200+ CD's.  One thing that would be really helpful is to have a field in the tag that states if it has been verified or not.  There could be several different codes to define the status of the album.

Tunetyme
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-05-06 03:43:56
I don't see an advanced or write accurate rip tags option in 2.1.1.

Advanced settings are opened when you click a gray cogwheel button in the top-left corner. If you didn't change those settings, then by default tag writing should be off and CUETools just couldn't have eaten your tags.

One thing that would be really helpful is to have a field in the tag that states if it has been verified or not.  There could be several different codes to define the status of the album.

CUETools saves this info in it's local database, which is a better way IMHO than modifying the files.
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-05-06 03:47:52
Thanks to CTDB, i was able to gather some interesting statistics on the use of pregaps:

(http://db.cuetools.net/pregaps1.png)

(http://db.cuetools.net/pregaps2.png)
Title: CUETools DB
Post by: sauvage78 on 2011-05-06 05:11:43
Interesting Yes, Thanks!!!
Title: CUETools DB
Post by: sauvage78 on 2011-05-18 13:10:05
I guess it doesn't matter much but I noticed CTDB counter is blocked at 50006 since a few days.
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-05-22 00:34:23
Strangely enough, constantly updating this counter was taking too much of servers resources, so i had to partially disable it. It will update about once per day now.

By the way, congratulations everyone on reaching 50k CDs in database!

I moved CTDB to a new hosting to prepare for further database growth, and am working on providing musicbrainz NGS metadata via CTDB.
Here's an example: http://db.cuetools.net/lookup2.php?toc=32:...z=1&fuzzy=0 (http://db.cuetools.net/lookup2.php?toc=32:19720:37477:63232:81375:100780:129857:148577:167265:189560:210052:232170:250767:268662:285200:303867:323640:341842&musicbrainz=1&fuzzy=0)
Title: CUETools DB
Post by: radu on 2011-05-22 19:50:54
I think that in order for the CTDB to grow it should be supported by EAC, as AR is.

Are there any plans  for an EAC plugin?
If no plans, why?
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-05-22 21:07:14
I'm working on it.
Title: CUETools DB
Post by: HotShotFR on 2011-05-24 00:32:30
Strangely enough, constantly updating this counter was taking too much of servers resources, so i had to partially disable it. It will update about once per day now.

By the way, congratulations everyone on reaching 50k CDs in database!

I moved CTDB to a new hosting to prepare for further database growth, and am working on providing musicbrainz NGS metadata via CTDB.
Here's an example: http://db.cuetools.net/lookup2.php?toc=32:...z=1&fuzzy=0 (http://db.cuetools.net/lookup2.php?toc=32:19720:37477:63232:81375:100780:129857:148577:167265:189560:210052:232170:250767:268662:285200:303867:323640:341842&musicbrainz=1&fuzzy=0)


Speaking of resources, I recently discovered CTDB and decided to check all my rips against AR/CTDB then submit all [AR-reported] error-free ones to CTDB... (that was about ~1000 new discs in a row, ouch). I noticed afterwards that the uploading step failed for about ~5 rips (they did not appear online at db.cuetools.net). Had to do a new submission and it worked. Server overload?
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-05-24 07:03:53
Thanks. Most probably it was caused by me doing some maintainance and development in the background. But it could be server overload.
I guess i will have to set up a dedicated server at some point. The good thing about VDS is that it's cheap
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-05-26 19:41:43
Great news everyone: EAC V1.0 beta 2 is out, and it supports plugins. You can download it here: http://www.exactaudiocopy.de/en/index.php/...urces/download/ (http://www.exactaudiocopy.de/en/index.php/resources/download/)
And CTDB plugin here: CUETools.CTDB.EACPlugin.rar (http://s3.cuetools.net/CUETools.CTDB.EACPlugin.rar)
Title: CUETools DB
Post by: Porcus on 2011-05-27 01:13:08
Plugins. Mmmmm, plugins! 
Title: CUETools DB
Post by: HotShotFR on 2011-05-27 14:08:48
Great news everyone: EAC V1.0 beta 2 is out, and it supports plugins. You can download it here: http://www.exactaudiocopy.de/en/index.php/...urces/download/ (http://www.exactaudiocopy.de/en/index.php/resources/download/)
And CTDB plugin here: http://db.cuetools.net/downloads/CUETools.CTDB.EACPlugin.rar (http://db.cuetools.net/downloads/CUETools.CTDB.EACPlugin.rar)

Great news  But what are we supposed to do with this CTDB plugin?
For instance I extracted the archive contents to my main Cuetools directory (overwriting files from a fresh 2.1.1), yet when verifying any file I get the error message: Exception: Method not found : 'Void CUETools.CTDB.CUEToolsDB.ContactDB(System.String)'

edit: Nevermind. Just figured out there are online instructions at http://db.cuetools.net/plugin.php (http://db.cuetools.net/plugin.php)
Title: CUETools DB
Post by: zfox on 2011-05-27 17:41:22
Code: [Select]
Track 31  accurately ripped (confidence 6)  [1278B705]  (AR v2)
Track 32  accurately ripped (confidence 6)  [26837954]  (AR v2)
 
All tracks accurately ripped

End of status report

---- CUETools DB Plugin V2.1.2

[CTDB TOCID: 17ng1NaZY3cpgsNWrDCX5d2xFZM-] found.
[98d58d47] (3/3) Differs in 20 samples @37:51:39,58:44:55-58:44:56
You can use CUETools to repair this rip.


==== Log checksum CC0459D22AA62E8188551B97513046D1FE68D69AD04EEF36AA7B60991B2BB0BE ====

Hi Greg,
Does the Log Checksum value that was produced by EAC cover the text that was inserted by your plugin?
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-05-27 22:33:26
It should. Plugin itself doesn't modify the log file, it just returns the message to EAC, and EAC inserts it into log file.
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-06-03 14:52:56
If you had trouble accessing the database for the past hour, that's because i was moving it to a new hosting. CUETools DB is now hosted on Amazon Cloud Services. Database runs on EC2 instance, and parity files are stored in S3. This should guarantee fast access and practically zero downtime.
Title: CUETools DB
Post by: HotShotFR on 2011-06-03 14:57:01
Congrats for the new functionalities and redesign.
Any plan to implement a 'search by album' function?

(Well done for the new Cuetools website too!)
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-06-03 15:03:52
I have to figure out how to further integrate CTDB with Musicbrainz (and possibly Discogs), then i can reuse their search capabilities.
Title: CUETools DB
Post by: Studio 308 on 2011-06-13 23:50:38
I only hope that this new method of adding plugins won't ruin EAC trust on some paranoid trackers.
Thanks for addition of this plugin.
Title: CUETools DB
Post by: Eli on 2011-07-02 16:04:45
Gregory, would you consider adding support for the CDTOC in meta-data. To many of my discs are not recognized when I try to verify and submit. I have over 5000 securely ripped CDs that I would like to submit to the DB.

The TOC is stored in the meta-data per these specs:

http://forum.dbpoweramp.com/showthread.php?t=16705 (http://forum.dbpoweramp.com/showthread.php?t=16705)
Title: CUETools DB
Post by: Porcus on 2011-07-03 22:55:19
Gregory, would you consider adding support for the CDTOC in meta-data. To many of my discs are not recognized when I try to verify and submit. I have over 5000 securely ripped CDs that I would like to submit to the DB.

The TOC is stored in the meta-data per these specs:

http://forum.dbpoweramp.com/showthread.php?t=16705 (http://forum.dbpoweramp.com/showthread.php?t=16705)


... and maybe even dBpoweramp's ACCURATEDISCID tag too? Despite the name, that is the track ID.
Title: CUETools DB
Post by: greynol on 2011-07-04 00:50:00
[...] dBpoweramp's ACCURATEDISCID [...] Despite the name, that is the track ID.

<nitpick>This is correct, though I think it's a bit misleading since only an enumeration is appended in order to differentiate tracks within the record which itself is named from the disc ID.  That this enumeration is actually part of the disc ID is definitely debatable, IMO.</nitpick>
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-08-06 14:51:25
Gregory, would you consider adding support for the CDTOC in meta-data. To many of my discs are not recognized when I try to verify and submit. I have over 5000 securely ripped CDs that I would like to submit to the DB.

Thanks for the info, this would help with pregap/data track issues, and the tag format seems sensible enough.
Title: CUETools DB
Post by: Porcus on 2011-08-08 17:43:30
Gregory, would you consider adding support for the CDTOC in meta-data. To many of my discs are not recognized when I try to verify and submit. I have over 5000 securely ripped CDs that I would like to submit to the DB.

Thanks for the info, this would help with pregap/data track issues, and the tag format seems sensible enough.


I second Eli's wish here. Including the '5000' figure for the database

And to quote myself:

[quote author=Porcus link=msg=0 date=]... and maybe even dBpoweramp's ACCURATEDISCID tag too? Despite the name, that is the track ID.[/quote]

That would not aid the database, but would make possible single-file verification. (@Greynol: Yeah, but you missed my point: the tag identifies the track and not only the disc.)
Title: CUETools DB
Post by: zfox on 2011-10-01 12:50:11
I noticed that I can increase the confidence number of a disc (stored in CTDB) by ripping it again and again with CUERipper using the same OS instalation. Should there be a protection mechanism to avoid that case? Spoon scrutinizes submissions just to keep his DB clean and credible. I think this paradigm can be applied to CTDB as well.

I am not sure if this issue was discussed before for CTDB.
Title: CUETools DB
Post by: Gregory S. Chudov on 2011-10-06 20:32:38
As i mentioned in another thread, right now confidence levels are a bit of a mess, because they combine past measurements of AR confidence levels with the number of direct CTDB submissions since that point.

In the near future, i plan to reset CTDB confidence levels to the actual number of independent submissions.

For now, i suggest to use CTDB as a source of repair material, not as verification of rip accuracy in itself.
Title: CUETools DB
Post by: mjb2006 on 2011-12-04 07:21:41
For now, i suggest to use CTDB as a source of repair material, not as verification of rip accuracy in itself.

Hmm, but I've found that when CTDB has a record for an error-containing rip, CUETools won't let that rip be repaired against any other CTDB submission.

For example, I have a scratched disc that I ripped with CUERipper in burst mode. Despite containing errors on 2 tracks, it was apparently submitted to CTDB. I thought it would be no problem; eventually someone would make a CTDB submission for a clean rip, and then I could correct mine against it. It does seem that there's now such a submission, but CUETools won't repair my rip against it, because it's finding the bad rip in CTDB:

Results for an Encode/repair action:
Code: [Select]
.\Various Artists - I-Robots_ Italo Electro Disco Underground Classics.flac: verified OK, confidence 1.


Results for a Verify/default action (edited to show just the CTDB part):
Code: [Select]
[CTDB TOCID: Ta8fLh3t.nOoXGvbbGehaUIiH64-] found.
        [ CTDBID ] Status
        [3bd0e182] (1/7) Accurately ripped
        [cce5f891] (5/7) No match
        [90a84fca] (1/7) No match


I assume if I re-rip, the errors will be different, and I'll be given the choice of correcting to match one of the three CTDB submissions (cce5f891 seems to be the one I want). I really wish I could just use my broken rip, though. Isn't that the ideal circumstance—keeping your bad rips around until they're repairable?
Title: CUETools DB
Post by: sauvage78 on 2011-12-27 18:44:36
Before I forget, if you want you can delete any entry under [CTDB TOCID: VTlxZK5YWAOpfUBqTTA67jhHTTM-], it's me playing with a scratched french Christmas songs CD, the submissions are obviously bad (hearable glitch). It would be nice if the EAC plugin would not automatically upload Secure rip with errors, unless you can verify if those are AR(2) despite errors which is unlikely (but far from impossible with bad EAC settings). The CTDB size has exploded since the introduction of the EAC plugin but without this filter I fear that there is a lot of scratched data (like mine) uploaded ... even if you shouldn't fix stuff without listening to what you fix & even if you should verify that it's AR(2) after fixing ... it would be better to filter crap from start to avoid an uncontrolled inflation of the database.
Title: CUETools DB
Post by: Gregory S. Chudov on 2012-01-18 00:21:41
Hmm, but I've found that when CTDB has a record for an error-containing rip, CUETools won't let that rip be repaired against any other CTDB submission.

According to the log, the supposedly "good" rip is listed as "No match", that just means that your rip had too many errors to be correctable. If it was correctable, you would see the list of error locations instead of "No match", and CUETools would offer you to repair using this entry.

it would be better to filter crap from start to avoid an uncontrolled inflation of the database.

I can always clean it up later  Database isn't yet that large. But i'm still slowly improving filters, it just wasn't a priority.

Beta-testing CTDB 2.0 with new version of EAC plugin:
https://plus.google.com/b/10448316314704930...sts/HthUiJ7ZSwz (https://plus.google.com/b/104483163147049308670/104483163147049308670/posts/HthUiJ7ZSwz)
Title: CUETools DB
Post by: Pepzhez on 2012-01-25 03:36:16
Now that the CTDB is ignoring pregaps, I just discovered that if I rip a disc that has pregaps with CUERipper, the database will no longer accept the submission. Unfortunately there is no option in CUERipper to ignore HTOA, so six of the ten discs I ripped today will not be in the database, even though all of them were AR confirmed with confidence levels of 3 or higher.

When will an update of CUERipper that includes an option to rip without HTOA be coming?
Title: CUETools DB
Post by: Eli on 2012-06-02 01:10:21
Looks like spoon/dbpoweramp plans to implement a separate but similar system to CTDB:

http://forum.dbpoweramp.com/showthread.php...ll=1#post120964 (http://forum.dbpoweramp.com/showthread.php?10945-Par2Par-p2p-PAR2-sharing-for-CD-rip-repair&p=120964&viewfull=1#post120964)
Title: CUETools DB
Post by: Gregory S. Chudov on 2012-06-02 01:36:14
Thanks, i've heard. I rather hoped we could cooperate on this, because IMHO having two separate databases is bit counterproductive. But let's see how it works.

By the way, CTDB recently reached an important mark - 1M rips and 0.5M discs.
Title: CUETools DB
Post by: Eli on 2012-09-01 21:10:47
Gregory, would you consider adding support for the CDTOC in meta-data. To many of my discs are not recognized when I try to verify and submit. I have over 5000 securely ripped CDs that I would like to submit to the DB.

Thanks for the info, this would help with pregap/data track issues, and the tag format seems sensible enough.


Gregory,
Where does this stand?
Title: CUETools DB
Post by: drfsupercenter on 2012-09-03 23:06:39
Hey guys

I like using FLACCL with CUERipper, however, since switching from a NVIDIA to an AMD card, it no longer works.  If I use command-line FLACCL, I can tell it to use the CPU instead of the GPU... does CUERipper have any way to tell it to do the same?  It provides better compression than the other methods (as far as I can tell), so it's a bit disappointing that I can't get it to work.
Title: CUETools DB
Post by: korth on 2012-09-03 23:36:17
This might get better response in the [a href='index.php?showtopic=64628']FLACCL[/a] thread.
I know you can change GPU/CPU on the Encoder tab (http://cuetools.net/wiki/CUETools_Advanced_Settings:_Encoders) in CUETools Advanced Settings but without the hardware to test it I don't know if the settings carry over to CUERipper. I don't see anything in %appdata%\CUERipper\settings.txt to change it.
Title: CUETools DB
Post by: korth on 2012-09-05 00:55:30
I guess I could have also told you how to add the FLACCL exe to CUERipper (if needed).

Close CUERipper and make backup of %appdata%\CUERipper\settings.txt
Edit %appdata%\CUERipper\settings.txt
Search for ExternalEncoders=
Above that line, add the following (replacing ## with the next number in sequence and exclude all the Red notations.

ExternalEncoder##Name=FLACCL cmd
ExternalEncoder##Modes=0 1 2 3 4 5 6 7 8 9 10 11  (Mode selection values - shown on slider)
ExternalEncoder##Mode=8 (Default selected mode)
ExternalEncoder##Extension=flac
ExternalEncoder##Path=CUETools.FLACCL.cmd.exe (Full path not required if exe is in CUETools folder)
ExternalEncoder##Lossless=1
ExternalEncoder##Parameters=-%M - -o %O (Placeholders %M = mode, - = input, and %O = output are required. You'll need to add the other parameters as needed for your hardware)

You then need to add 1 to the number of external encoders (if it was 19, then it would become 20).
ExternalEncoders=20
and save file.
Title: CUETools DB
Post by: drfsupercenter on 2012-10-11 02:23:34
Got another question regarding CUERipper.

I notice, in the newest build (2.1.4), whenever I select CBR (libmp3lame) to do rips of a CD, it automatically inserts its own advertising in the "Comments" tag of the mp3.  Specifically, "CUERipper v2.1.4 Copyright © 2008-12 Grigory Chudov"

Can I disable that?  It's incredibly annoying and one reason I started using CueRipper INSTEAD of other software was because it didn't force advertisements in my music.  I'd happily go back to 2.1.2a just to get rid of the comment spam, but I thought I would ask on here.

I looked through all the options and couldn't find anything at all pertaining to how files are tagged...

--EDIT--

It does it for FLAC too.  Looks like any of my rips are tainted with the ad.
Title: CUETools DB
Post by: korth on 2012-10-11 04:16:38
The version info is only inserted when the Comment field is blank. There is currently no way in 2.1.4 to disable other than adding some text to the Comment field.
Title: CUETools DB
Post by: Porcus on 2012-10-11 08:06:01
This is hardly the right thread anyway, but: can't one just remove all such lines afterwards?
Title: CUETools DB
Post by: drfsupercenter on 2012-10-12 03:25:39
Well that's disappointing. I guess I'm going back to 2.12a, I don't want advertisements in my files...

@Porcus, what?
Title: CUETools DB
Post by: Porcus on 2012-10-12 09:19:56
@Porcus, what?


This thread is for the database that verifies your rips. The thread for CUETools is at http://www.hydrogenaudio.org/forums/index....showtopic=66233 (http://www.hydrogenaudio.org/forums/index.php?showtopic=66233) .

By the way, couldn't you just remove that line from every cuesheet (in batch I mean, not one manually one at the time)?
Title: CUETools DB
Post by: frozenspeed on 2012-10-12 15:49:19
Well that's disappointing. I guess I'm going back to 2.12a, I don't want advertisements in my files...


What do you do about all the label, distribution & production advertisements on your cds?
Title: CUETools DB
Post by: drfsupercenter on 2012-10-15 02:42:13
Well that's disappointing. I guess I'm going back to 2.12a, I don't want advertisements in my files...


What do you do about all the label, distribution & production advertisements on your cds?


I don't include those when ripping the files, obviously.

Quote
This thread is for the database that verifies your rips. The thread for CUETools is at http://www.hydrogenaudio.org/forums/index....showtopic=66233 (http://www.hydrogenaudio.org/forums/index....showtopic=66233) .

By the way, couldn't you just remove that line from every cuesheet (in batch I mean, not one manually one at the time)


Ah.  I didn't realize that, I saw Cuetools and didn't know the DB was something separate.  My bad.

Also, these tags are not in the cuesheet but in the actual files themselves, so that solution won't work :/
Title: CUETools DB
Post by: ebz777 on 2012-11-27 22:05:08
Pb with recent Musicbrainz releases + CD toc, not found in CUETools DB

I encounter the following problem : when I'm adding a new release in MusicBrainz, attach the CD TOC to the release and then try to rip it, CUEripper doesn't find the MusicBrainz release. When I check CUETools DB, I can see that it is not in the DB either. This use to work.

Ex 1:
http://db.cuetools.net/top.php?artist=Carmen%20Miranda (http://db.cuetools.net/top.php?artist=Carmen%20Miranda)
select Carmen Miranda - Brazilian Recordings
If you look at the bottom of the page, in the CUETools DB album list section, it has only a single freedb album, but no MusicBrainz one.
But if you follow the MusicBrainz CD lookup you on that page you can see that a release actually exist for this CD TOC in MusicBrainz.

Ex 2:
http://db.cuetools.net/top.php?artist=George%20Harisson (http://db.cuetools.net/top.php?artist=George%20Harisson)
select George Harrison - All Things Must Pass
you can see that there a only 2 MusicBrainz releases (US and GB) in the album section
If you follow the MusicBrainz CD lookup for that TOC you can see that there is a third one.
The latest one recently added (2001, country XE) is not there.

and so on ...

I know that CTDB replicates MusicBrainz database hourly,
but I've be waiting for several days now and nothing happened.

What is the problem ?
Thanks in advance for your help.
Title: CUETools DB
Post by: Gregory S. Chudov on 2012-11-27 22:06:55
Musicbrainz recently released a new schema which required an upgrade to postgresql 9. I'm working on it.
Title: CUETools DB
Post by: Eli on 2012-12-30 13:53:59
I know this disc is in the CTDB, but I can't seem to get CT to do a repair. I have ripped with dbpoweramp. CTDB wouldn't recognize, so I re-ripped with CueRipper, which can't rip the last track. Tried in burst mode, but its still doing tons of error correction and gets stuck. Tried with EAC and it can't rip the track either (can't seem to find burst mode in the latest EAC with CTDB support).

Created a CUE with Cueripper and transferred the CUE and 00 - HTOA track to the dbpoweramp directory. CT was then able to recognize the disc, but still wouldn't do a repair.

Code: [Select]
[CUETools log; Date: 12/29/2012 5:05:10 PM; Version: 2.1.4]
[CTDB TOCID: ujvjyuIIqdR61HbcM3huXrxTUi8-] found.
Track | CTDB Status
  1   | (5/6) Accurately ripped
  2   | (5/6) Accurately ripped
  3   | (5/6) Accurately ripped
  4   | (5/6) Accurately ripped
  5   | (5/6) Accurately ripped
  6   | (5/6) Accurately ripped
  7   | (5/6) Accurately ripped
  8   | (5/6) Accurately ripped
  9   | (5/6) Accurately ripped
10   | (5/6) Accurately ripped
11   | (5/6) Accurately ripped
12   | (0/6) No match
[AccurateRip ID: 0018635f-00e3861c-930dc30c] found.
Track   [  CRC   |   V2   ] Status
01     [60bf57fe|22355748] (0+0/2) No match
02     [d4f00ec3|d7e586cb] (0+0/2) No match
03     [27877b37|c51e2f72] (0+0/2) No match
04     [4b27f567|e978299e] (0+0/2) No match
05     [9b76c989|857c20a6] (0+0/2) No match
06     [86e676e1|7211ae70] (0+0/2) No match
07     [1afb75c3|e2016a78] (0+0/2) No match
08     [f3503b15|3312270a] (0+0/2) No match
09     [e685750c|91e482e0] (0+0/2) No match
10     [5ed89126|cb73314f] (0+0/2) No match
11     [ac39c742|ffd92e9d] (0+0/2) No match
12     [8126c2c7|919d503c] (0+0/2) No match
Offsetted by 6:
01     [dc5ca8d0] (2/2) Accurately ripped
02     [f7492949] (2/2) Accurately ripped
03     [65ca5a8f] (2/2) Accurately ripped
04     [0a47b8cb] (2/2) Accurately ripped
05     [6e6cf5e3] (2/2) Accurately ripped
06     [787a66a3] (2/2) Accurately ripped
07     [1f853ad7] (2/2) Accurately ripped
08     [7aa64579] (2/2) Accurately ripped
09     [22337ad0] (2/2) Accurately ripped
10     [a1f8324a] (2/2) Accurately ripped
11     [5d24e404] (2/2) Accurately ripped
12     [fc192947] (0/2) No match (V2 was not tested)

Track Peak [ CRC32  ] [W/O NULL]
--  100.0 [AA03330F] [90E26911]          
01  100.0 [617CDA7F] [5EA0A1F0]          
02  100.0 [EA14B6A1] [9671F6FD]          
03  100.0 [9663ACE1] [D481CF8D]          
04  100.0 [16939E77] [46897AC9]          
05  100.0 [7D6384A5] [968D241D]          
06  100.0 [1DFA3E02] [202A1D6A]          
07  100.0 [FBDB8CDB] [997114D6]          
08  100.0 [5985FAC9] [3283B7DB]          
09  100.0 [59421096] [477547CE]          
10  100.0 [99E1FE74] [8FEB880D]          
11  100.0 [ABF90345] [539B6177]          
12  100.0 [08FF968A] [BF12DEDD]
Title: CUETools DB
Post by: korth on 2012-12-30 14:27:27
CT was then able to recognize the disc, but still wouldn't do a repair.

The recovery record can only repair a limited amount of damage. Your rip likely exceeds that amount of damage.
Title: CUETools DB
Post by: Eli on 2012-12-30 15:36:35
CT was then able to recognize the disc, but still wouldn't do a repair.

The recovery record can only repair a limited amount of damage. Your rip likely exceeds that amount of damage.


Korth, you are probably right. I wish CT would tell you something to that effect...
Title: CUETools DB
Post by: lvqcl on 2012-12-30 16:44:09
http://www.cuetools.net/wiki/CUETools_log (http://www.cuetools.net/wiki/CUETools_log)

Accurately ripped -- Your rip matches database records for this track
Differs in ... -- Partial match this track, might be repairable
No match -- Too many samples differ
Title: CUETools DB
Post by: ebz777 on 2013-01-01 13:49:16
Add Chromaprint and AcoustID relationships within CUETools DB

Hello Gregory,
The CUETools DB is already cross-referencing at the CD TOC level the freedb and MusicBrainz databases.
Would you also consider doing this at the tracklevel with Chromaprint fingerprints and AcoustIDs (see http://acoustid.org/) (http://acoustid.org/)) ?

This would help the match of an individual track to a release, regardless of any later lossy re-encoding. This is something you cannot do with the current CUETools DB tracks CRCs alone : there is now way to calculate that proper CRC for an mp3 track for example. The only way to ensure a consistent fingerprint calculation for a CD release is at ripping time.

It would require to calculate the chromaprint fingerprint for every tracked ripped with CUERipper and add these references to CUETtools DB.
I'm not sure though that CUETools DB should also link the AcoustID; the fingerprint -> AcoustID assignment process is still very far from perfect (see http://forums.musicbrainz.org/viewtopic.php?id=3869) (http://forums.musicbrainz.org/viewtopic.php?id=3869)). But it could be considered safe, there is no plan to merge AcoustID I believe and it would be a nice bonus !

Maybe CUERipper/CUETools could then also publish the calculated Chromaprint fingerprints to the AcoustID DB (see http://acoustid.org/webservice#submit) (http://acoustid.org/webservice#submit))

Additionally CUETools could then also tag the tracks it generates with the AcoustID

Thanks
Title: CUETools DB
Post by: bilbo on 2013-01-01 16:13:37
Add Chromaprint and AcoustID relationships within CUETools DB
This would help the match of an individual track to a release, regardless of any later lossy re-encoding.


CUEToolsDB is meant to verify and/or repair rips from a CD. Why would anyone want to verify a "lossy" track? That is the opposite of what the DB was designed to accomplish.
Title: CUETools DB
Post by: ebz777 on 2013-01-01 23:11:29
CUEToolsDB is meant to verify and/or repair rips from a CD. Why would anyone want to verify a "lossy" track?


Yes, the data correction made ??during a rip is what motivates a large number of users to publish their information in the database in the first place. But once the information is out there it is metadata that can be used for purposes that were not originally intended. Eg to identify similar recordings from different releases, to find out what recordings exactly are included in a compilation. To identify whether a particular release includes the original or remastered version of a recording. Or in the presence of a piece of music lossy you want to tag, to know what CD this song was coming from.

This is something that you cannot do with the track CRC calculated today, but that you could do if you add a fingerprint calculation to the track.

It's of course best calculated at rip time since it's the surest way to ensure consistency between a CD and its track information.
Title: CUETools DB
Post by: Gregory S. Chudov on 2013-01-02 00:15:46
It doesn't seem very fair - people that would be wasting their CPU cycles to calculate those fingerprints would be helping a completely different category of users to identify random mp3s.
Title: CUETools DB
Post by: Eli on 2013-06-04 01:51:45
CueTools 2.1.5

I am trying to repair a disc
The Rolling Stones
Exile on Main St.
CTDB TOCID: kXCUmR9eH1zJRjVprT_Hvhp_Kzc-

I am able to verify the disc fine:
Code: [Select]
[CUETools log; Date: 6/3/2013 8:46:47 PM; Version: 2.1.5]
[CTDB TOCID: kXCUmR9eH1zJRjVprT_Hvhp_Kzc-] found.
Track | CTDB Status
  1  | (40/41) Accurately ripped
  2  | (41/41) Accurately ripped
  3  | (41/41) Accurately ripped
  4  | (41/41) Accurately ripped
  5  | (40/41) Accurately ripped
  6  | (40/41) Accurately ripped
  7  | (40/41) Accurately ripped
  8  | (39/41) Accurately ripped
  9  | (39/41) Accurately ripped
 10  | (40/41) Accurately ripped
 11  | (40/41) Accurately ripped
 12  | (38/41) Accurately ripped
 13  | (38/41) Accurately ripped
 14  | (39/41) Accurately ripped
 15  | (31/41) Differs in 35 samples @00:13:31-00:13:32
 16  | (37/41) Accurately ripped
 17  | (35/41) Accurately ripped
 18  | (36/41) Accurately ripped
[AccurateRip ID: 002ab05f-023b7e5b-f50fc512] found.
Track  [  CRC  |  V2  ] Status
 01    [3d93162a|cd2757b3] (074+003/349) Accurately ripped
 02    [7c634221|95c2e2e1] (073+003/349) Accurately ripped
 03    [23d503ed|98b85eef] (074+003/355) Accurately ripped
 04    [3f1e5129|d771c8fa] (074+003/349) Accurately ripped
 05    [c8227004|8192ee79] (075+003/355) Accurately ripped
 06    [7e1afc91|8565f83f] (074+003/349) Accurately ripped
 07    [895203b2|735c7815] (073+003/349) Accurately ripped
 08    [47598ba8|ead177ad] (073+003/351) Accurately ripped
 09    [479220a6|8ace56d1] (072+003/347) Accurately ripped
 10    [4db31259|7bf5c4c3] (072+003/347) Accurately ripped
 11    [3eaf488d|1039e7ef] (072+003/346) Accurately ripped
 12    [fb309295|5e3a4f6b] (072+003/347) Accurately ripped
 13    [1d9e8a7c|25010632] (072+003/338) Accurately ripped
 14    [6ef1796d|ea98ffaa] (072+003/336) Accurately ripped
 15    [d781ac36|6768463e] (000+000/338) No match
 16    [bd9a1da4|6ab7466d] (071+003/334) Accurately ripped
 17    [c2e9e1c4|275f2638] (073+003/330) Accurately ripped
 18    [0c2fb6e7|39781207] (070+003/321) Accurately ripped
Offsetted by 76:
 01    [009fb622] (002/349) Accurately ripped
 02    [d23367ae] (002/349) Accurately ripped
 03    [159aef17] (002/355) Accurately ripped
 04    [53d16027] (002/349) Accurately ripped
 05    [554d3116] (002/355) Accurately ripped
 06    [cb2c24b8] (000/349) No match (V2 was not tested)
 07    [bab54b8d] (000/349) No match (V2 was not tested)
 08    [b278672a] (002/351) Accurately ripped
 09    [2420d59e] (002/347) Accurately ripped
 10    [aef143b6] (002/347) Accurately ripped
 11    [021e8ecb] (002/346) Accurately ripped
 12    [bf03a663] (002/347) Accurately ripped
 13    [c032788c] (002/338) Accurately ripped
 14    [fe7c9eb5] (002/336) Accurately ripped
 15    [69ee3f21] (000/338) No match (V2 was not tested)
 16    [4b96ee3b] (002/334) Accurately ripped
 17    [67b38275] (002/330) Accurately ripped
 18    [d1be152e] (000/321) No match (V2 was not tested)
Offsetted by 664:
 01    [a26e64a2] (168/349) Accurately ripped
 02    [f17cd341] (169/349) Accurately ripped
 03    [9fb7d390] (172/355) Accurately ripped
 04    [4ed38139] (168/349) Accurately ripped
 05    [a89fd22b] (175/355) Accurately ripped
 06    [fd29d425] (170/349) Accurately ripped
 07    [6e12aace] (169/349) Accurately ripped
 08    [38ac53d6] (169/351) Accurately ripped
 09    [ddfd4d89] (165/347) Accurately ripped
 10    [6db7a9a5] (168/347) Accurately ripped
 11    [ac16c55d] (167/346) Accurately ripped
 12    [f6515282] (171/347) Accurately ripped
 13    [0c33b759] (163/338) Accurately ripped
 14    [0092fdb4] (164/336) Accurately ripped
 15    [b2b9c150] (000/338) No match (V2 was not tested)
 16    [df81134e] (161/334) Accurately ripped
 17    [aed29e8e] (161/330) Accurately ripped
 18    [8d45be26] (154/321) Accurately ripped
Offsetted by 2140:
 01    [83ed544a] (075/349) Accurately ripped
 02    [12537512] (075/349) Accurately ripped
 03    [14c5cc55] (077/355) Accurately ripped
 04    [7366a48e] (075/349) Accurately ripped
 05    [159973ef] (073/355) Accurately ripped
 06    [f3b1b8a4] (075/349) Accurately ripped
 07    [a17d3ef2] (077/349) Accurately ripped
 08    [c8d95ab6] (077/351) Accurately ripped
 09    [4e49411f] (078/347) Accurately ripped
 10    [8d8409be] (075/347) Accurately ripped
 11    [f7ada841] (075/346) Accurately ripped
 12    [da86fb24] (072/347) Accurately ripped
 13    [de6ab9b4] (071/338) Accurately ripped
 14    [dd314f57] (070/336) Accurately ripped
 15    [b6ea7a75] (000/338) No match (V2 was not tested)
 16    [365fdba9] (072/334) Accurately ripped
 17    [93a1fc6f] (068/330) Accurately ripped
 18    [f510276c] (065/321) Accurately ripped

Track Peak [ CRC32  ] [W/O NULL]
 --  97.7 [F215327F] [561E67E0]         
 01  97.7 [FEB5E1BA] [81AB5674]         
 02  97.7 [A42C7CAD] [ADBFE045]         
 03  84.7 [B2486164] [57A64BDD]         
 04  97.7 [D31DB1EE] [89BD4C76]         
 05  97.7 [72F66D51] [E62C37D4]         
 06  97.7 [43EFFACF] [06A347A2]         
 07  97.7 [46B24C6F] [AF099ACD]         
 08  97.7 [47EBA0D1] [805EC507]         
 09  97.7 [EDBA4CEE] [0AF13865]         
 10  97.7 [8E17B700] [E1C62A3A]         
 11  85.9 [A621CDE9] [98631D15]         
 12  97.7 [A49DD7DF] [1FAECAD8]         
 13  86.9 [B76795DB] [C446C48B]         
 14  97.7 [4FF15683] [D2788406]         
 15  97.7 [D0E72E22] [9BA021C4]         
 16  97.7 [97D1D4D9] [F0E89505]         
 17  97.7 [61BF4321] [3A1119E0]         
 18  97.7 [00E9D0D0] [B2DA7473]         

But when I try to repair, I get an error that says

"Unsupported Encoder Settings"

I have been able to repair other discs with 2.1.5, also encoded with FLAC by dBpoweramp. And since it can read and verify, I don't understand why it can't repair.
Title: CUETools DB
Post by: Gregory S. Chudov on 2013-06-04 03:15:45
Does it let you convert without repair? I only have one theory so far - maybe there's an oversized (above 1m) artwork in source folder? 2.1.5 is unintentionally picky about that.
Title: CUETools DB
Post by: Eli on 2013-06-04 04:00:07
Does it let you convert without repair? I only have one theory so far - maybe there's an oversized (above 1m) artwork in source folder? 2.1.5 is unintentionally picky about that.


Ding! Ding! Ding!

Yup, 1.1mb folder.jpg. Moved it and repaired NP!

I would never have guessed that one.

Hopefully that bug gets squashed.
Title: CUETools DB
Post by: Boiled Beans on 2013-06-10 15:41:51
When ripping a disc with short silent pregaps, AccurateRip says it's ok, but CUETools DB always say some samples differ.
So I try to repair it with CUETools, but nothing happens, and CUETools' output log determines it's accurate on CUETools DB.

Should I be concerned with this or is this normal behaviour? BTW, my CD drive can't read pregaps.

Here is the EAC log of Suzanne Vega's S/T. Note the CUETools DB section of the log says some samples differ.

Code: [Select]
Exact Audio Copy V1.0 beta 3 from 29. August 2011

EAC extraction logfile from 2. June 2013, 22:21

Suzanne Vega / Suzanne Vega

Used drive  : HL-DT-STDVD-ROM GDR8161B  Adapter: 1  ID: 0

Read mode : Burst

Read offset correction                      : 102
Overread into Lead-In and Lead-Out          : No
Fill up missing offset samples with silence : Yes
Delete leading and trailing silent blocks  : No
Null samples used in CRC calculations      : Yes
Used interface                              : Native Win32 interface for Win NT & 2000

Used output format              : User Defined Encoder
Selected bitrate                : 1024 kBit/s
Quality                        : High
Add ID3 tag                    : No
Command line compressor        : C:\Program Files\Exact Audio Copy\Flac\flac.exe
Additional command line options : -8 -V -T "ARTIST=%artist%" -T "ALBUM=%albumtitle%" -T "DATE=%year%" -T "GENRE=%genre%" -T "COMMENT=%comment%" -T "COMPOSER=%composer%" -T "TOTALTRACKS=%numtracks%" %source% -o %dest%


TOC of the extracted CD

    Track |  Start  |  Length  | Start sector | End sector
    ---------------------------------------------------------
        1  |  0:00.32 |  2:50.45 |        32    |    12826 
        2  |  2:51.02 |  2:36.25 |    12827    |    24551 
        3  |  5:27.27 |  3:40.23 |    24552    |    41074 
        4  |  9:07.50 |  3:55.45 |    41075    |    58744 
        5  | 13:03.20 |  3:48.72 |    58745    |    75916 
        6  | 16:52.17 |  3:28.03 |    75917    |    91519 
        7  | 20:20.20 |  3:41.00 |    91520    |  108094 
        8  | 24:01.20 |  4:51.15 |    108095    |  129934 
        9  | 28:52.35 |  3:36.70 |    129935    |  146204 
      10  | 32:29.30 |  3:20.55 |    146205    |  161259 


Range status and errors

Selected range

    Filename C:\EAC Temp\Suzanne Vega - Suzanne Vega.wav

    Peak level 91.2 %
    Extraction speed 22.2 X
    Copy CRC B55735BF
    Copy OK

No errors occurred

 
AccurateRip summary
 
Track  1  accurately ripped (confidence 6)  [F35EFB10]  (AR v1)
Track  2  accurately ripped (confidence 6)  [C8D7A41C]  (AR v1)
Track  3  accurately ripped (confidence 6)  [65350556]  (AR v1)
Track  4  accurately ripped (confidence 6)  [11E12428]  (AR v1)
Track  5  accurately ripped (confidence 6)  [2528CCFA]  (AR v1)
Track  6  accurately ripped (confidence 6)  [B952A494]  (AR v1)
Track  7  accurately ripped (confidence 6)  [E06BE4DF]  (AR v1)
Track  8  accurately ripped (confidence 6)  [B199FC80]  (AR v1)
Track  9  accurately ripped (confidence 6)  [899AD981]  (AR v1)
Track 10  accurately ripped (confidence 6)  [ADB6953C]  (AR v1)
 
All tracks accurately ripped

End of status report

---- CUETools DB Plugin V2.1.3

[CTDB TOCID: Qg0b_gWtEeWHdK7xH1oZi2f.i_Q-] found, Submit result: discs with pregaps not supported in this protocol version
[e7389015] (359/359) Differs in 10921 samples @00:00:35-00:00:50
You can use CUETools to repair this rip.


==== Log checksum CC54404B0A57187EB1927F0F834AC46826EB66252E7D0629CFAD2C0E56AAA904 ====

So I try to repair it and this is the CUETools log, where after verification, nothing happens (no repair done even though it was selected)
Code: [Select]
[CUETools log; Date: 6/6/2013 11:52:45 AM; Version: 2.1.5]
Pregap length 00:00:32.
[CTDB TOCID: Qg0b_gWtEeWHdK7xH1oZi2f.i_Q-] found.
Track | CTDB Status
  1  | (7/7) Accurately ripped
  2  | (7/7) Accurately ripped
  3  | (7/7) Accurately ripped
  4  | (7/7) Accurately ripped
  5  | (7/7) Accurately ripped
  6  | (7/7) Accurately ripped
  7  | (7/7) Accurately ripped
  8  | (7/7) Accurately ripped
  9  | (7/7) Accurately ripped
 10  | (7/7) Accurately ripped
[AccurateRip ID: 000cf8f3-0069a54a-7908660a] found.
Track  [  CRC  |  V2  ] Status
 01    [f35efb10|526aaef9] (006+000/509) Accurately ripped
 02    [c8d7a41c|707d0538] (006+000/510) Accurately ripped
 03    [65350556|0e71e2fe] (006+000/513) Accurately ripped
 04    [11e12428|93d775d4] (006+000/510) Accurately ripped
 05    [2528ccfa|57a1f977] (006+000/508) Accurately ripped
 06    [b952a494|8f5c4f83] (006+000/510) Accurately ripped
 07    [e06be4df|359e04ca] (006+000/508) Accurately ripped
 08    [b199fc80|10e4f764] (006+000/510) Accurately ripped
 09    [899ad981|2aca0bbf] (006+000/511) Accurately ripped
 10    [adb6953c|42fe7c91] (006+000/506) Accurately ripped
Offsetted by 664:
 01    [a65516ad] (005/509) Accurately ripped
 02    [f14b992c] (005/510) Accurately ripped
 03    [710cf9b7] (005/513) Accurately ripped
 04    [4912f162] (005/510) Accurately ripped
 05    [7810f2a6] (005/508) Accurately ripped
 06    [b47e2297] (005/510) Accurately ripped
 07    [c03f9724] (005/508) Accurately ripped
 08    [229b4257] (005/510) Accurately ripped
 09    [78ace0fb] (005/511) Accurately ripped
 10    [de733224] (005/506) Accurately ripped
Offsetted by 1328:
 01    [7d991df3] (053/509) Accurately ripped
 02    [6292d017] (053/510) Accurately ripped
 03    [9378df28] (054/513) Accurately ripped
 04    [3ba6ee76] (053/510) Accurately ripped
 05    [4a8c9c15] (052/508) Accurately ripped
 06    [7c708585] (053/510) Accurately ripped
 07    [d03dfc37] (053/508) Accurately ripped
 08    [196d23b0] (054/510) Accurately ripped
 09    [9480a3da] (053/511) Accurately ripped
 10    [d9632cc2] (052/506) Accurately ripped
Offsetted by 1982:
 01    [27787140] (043/509) Accurately ripped
 02    [cb71f1f9] (043/510) Accurately ripped
 03    [f5565a73] (043/513) Accurately ripped
 04    [55534188] (043/510) Accurately ripped
 05    [6a875136] (043/508) Accurately ripped
 06    [ececab74] (043/510) Accurately ripped
 07    [0af0849b] (043/508) Accurately ripped
 08    [83e17263] (043/510) Accurately ripped
 09    [d1fa8789] (043/511) Accurately ripped
 10    [04b48f17] (043/506) Accurately ripped

Track Peak [ CRC32  ] [W/O NULL] [  LOG  ]
 --  91.1 [B55735BF] [DEF7E937]  CRC32 
 01  70.0 [C5A70BBB] [D07C3176]         
 02  76.9 [6D8B2C76] [6C946E4B]         
 03  86.2 [1D5F1DA9] [9F0AACFB]         
 04  73.9 [7DBF081B] [2B646C08]         
 05  84.0 [FB88C3B1] [A15421C2]         
 06  73.1 [B36FEC9C] [5AA1459C]         
 07  83.1 [ED92E0DA] [BC371C4B]         
 08  84.6 [C92788E9] [65F4967B]         
 09  82.4 [30445AB9] [9DD4B33C]         
 10  91.1 [2BBFDF5D] [09642A81]         

Here is another example, a more modern CD, the latest from RHCP. The EAC and CUETools logs are shown below.

Code: [Select]
Exact Audio Copy V1.0 beta 3 from 29. August 2011

EAC extraction logfile from 3. June 2013, 12:30

Red Hot Chili Peppers / I'm with You

Used drive  : HL-DT-STDVD-ROM GDR8161B  Adapter: 1  ID: 0

Read mode              : Secure
Utilize accurate stream : Yes
Defeat audio cache      : Yes
Make use of C2 pointers : No

Read offset correction                      : 102
Overread into Lead-In and Lead-Out          : No
Fill up missing offset samples with silence : Yes
Delete leading and trailing silent blocks  : No
Null samples used in CRC calculations      : Yes
Used interface                              : Native Win32 interface for Win NT & 2000

Used output format              : User Defined Encoder
Selected bitrate                : 1024 kBit/s
Quality                        : High
Add ID3 tag                    : No
Command line compressor        : C:\Program Files\Exact Audio Copy\Flac\flac.exe
Additional command line options : -8 -V -T "ARTIST=%artist%" -T "ALBUM=%albumtitle%" -T "DATE=%year%" -T "GENRE=%genre%" -T "COMMENT=%comment%" -T "COMPOSER=%composer%" -T "TOTALTRACKS=%numtracks%" %source% -o %dest%


TOC of the extracted CD

    Track |  Start  |  Length  | Start sector | End sector
    ---------------------------------------------------------
        1  |  0:00.15 |  4:12.32 |        15    |    18946 
        2  |  4:12.47 |  4:21.46 |    18947    |    38567 
        3  |  8:34.18 |  5:39.61 |    38568    |    64053 
        4  | 14:14.04 |  3:51.00 |    64054    |    81378 
        5  | 18:05.04 |  3:40.61 |    81379    |    97939 
        6  | 21:45.65 |  3:28.00 |    97940    |  113539 
        7  | 25:13.65 |  4:43.03 |    113540    |  134767 
        8  | 29:56.68 |  4:21.65 |    134768    |  154407 
        9  | 34:18.58 |  3:52.67 |    154408    |  171874 
      10  | 38:11.50 |  3:33.09 |    171875    |  187858 
      11  | 41:44.59 |  5:35.67 |    187859    |  213050 
      12  | 47:20.51 |  4:01.09 |    213051    |  231134 
      13  | 51:21.60 |  4:22.17 |    231135    |  250801 
      14  | 55:44.02 |  3:45.40 |    250802    |  267716 


Range status and errors

Selected range

    Filename C:\EAC Temp\Red Hot Chili Peppers - I'm with You.wav

    Peak level 100.0 %
    Extraction speed 4.5 X
    Range quality 99.9 %
    Copy CRC E14194A9
    Copy OK

No errors occurred

 
AccurateRip summary
 
Track  1  accurately ripped (confidence 51)  [1E4C821A]  (AR v2)
Track  2  accurately ripped (confidence 51)  [DF54E998]  (AR v2)
Track  3  accurately ripped (confidence 50)  [DD6BA9B1]  (AR v2)
Track  4  accurately ripped (confidence 51)  [33BF41DB]  (AR v2)
Track  5  accurately ripped (confidence 51)  [6F3DA516]  (AR v2)
Track  6  accurately ripped (confidence 51)  [51B6CC56]  (AR v2)
Track  7  accurately ripped (confidence 51)  [4A5A1A27]  (AR v2)
Track  8  accurately ripped (confidence 50)  [0D03EF5A]  (AR v2)
Track  9  accurately ripped (confidence 51)  [6AE09ADB]  (AR v2)
Track 10  accurately ripped (confidence 51)  [C0E3384A]  (AR v2)
Track 11  accurately ripped (confidence 51)  [4BF260FF]  (AR v2)
Track 12  accurately ripped (confidence 51)  [71D84C9C]  (AR v2)
Track 13  accurately ripped (confidence 51)  [FB76FC8F]  (AR v2)
Track 14  accurately ripped (confidence 51)  [AB1AD69D]  (AR v2)
 
All tracks accurately ripped

End of status report

---- CUETools DB Plugin V2.1.3

[CTDB TOCID: Z05mxINrbJ8ZVnge2ceC6GJBBec-] found, Submit result: discs with pregaps not supported in this protocol version
[a8d49654] (103/104) Differs in 5708 samples @00:00:20-00:00:25
[5362c05c] (001/104) No match
You can use CUETools to repair this rip.


==== Log checksum 0E8BBE6B99745A38676DA3BD6D4AB014520D71F2D942AA70D452A7F5CF741685 ====

Code: [Select]
[CUETools log; Date: 6/6/2013 11:53:15 AM; Version: 2.1.5]
Pregap length 00:00:15.
[CTDB TOCID: Z05mxINrbJ8ZVnge2ceC6GJBBec-] found.
Track | CTDB Status
  1  | (16/17) Accurately ripped
  2  | (16/17) Accurately ripped
  3  | (16/17) Accurately ripped
  4  | (16/17) Accurately ripped
  5  | (16/17) Accurately ripped
  6  | (16/17) Accurately ripped
  7  | (16/17) Accurately ripped
  8  | (16/17) Accurately ripped
  9  | (16/17) Accurately ripped
 10  | (16/17) Accurately ripped
 11  | (15/17) Accurately ripped
 12  | (14/17) Accurately ripped
 13  | (15/17) Accurately ripped
 14  | (16/17) Accurately ripped
[AccurateRip ID: 001eea4a-0148ae9d-bf0df10e] found.
Track  [  CRC  |  V2  ] Status
 01    [b78679fc|1e4c821a] (071+051/242) Accurately ripped
 02    [40dac34c|df54e998] (072+051/244) Accurately ripped
 03    [d78d5829|dd6ba9b1] (072+050/242) Accurately ripped
 04    [ac33068f|33bf41db] (072+051/244) Accurately ripped
 05    [d81eed38|6f3da516] (072+051/244) Accurately ripped
 06    [96c419f8|51b6cc56] (072+051/244) Accurately ripped
 07    [ba674fef|4a5a1a27] (072+051/244) Accurately ripped
 08    [7507b370|0d03ef5a] (072+050/243) Accurately ripped
 09    [269855ff|6ae09adb] (072+051/244) Accurately ripped
 10    [7ad627c7|c0e3384a] (072+051/244) Accurately ripped
 11    [b6523423|4bf260ff] (072+051/243) Accurately ripped
 12    [4938a7d4|71d84c9c] (071+051/242) Accurately ripped
 13    [f986d470|fb76fc8f] (071+051/241) Accurately ripped
 14    [6a0c1386|ab1ad69d] (068+051/240) Accurately ripped
Offsetted by -68:
 01    [44bd3978] (002/242) Accurately ripped
 02    [18d70fdf] (002/244) Accurately ripped
 03    [2d3ab6d3] (002/242) Accurately ripped
 04    [f84df1c0] (002/244) Accurately ripped
 05    [fdbb92a4] (002/244) Accurately ripped
 06    [e6c440e7] (002/244) Accurately ripped
 07    [58aa65f4] (002/244) Accurately ripped
 08    [70d1303a] (002/243) Accurately ripped
 09    [3a01940d] (002/244) Accurately ripped
 10    [76c84be3] (002/244) Accurately ripped
 11    [6963dffd] (002/243) Accurately ripped
 12    [53b57709] (002/242) Accurately ripped
 13    [197df91b] (002/241) Accurately ripped
 14    [1548e44e] (002/240) Accurately ripped
Offsetted by 6:
 01    [67644587] (031/242) Accurately ripped
 02    [0677fa98] (031/244) Accurately ripped
 03    [32faa2b1] (031/242) Accurately ripped
 04    [bcad8a85] (031/244) Accurately ripped
 05    [d9b1e6d5] (031/244) Accurately ripped
 06    [5cf2184b] (031/244) Accurately ripped
 07    [5eb0f830] (031/244) Accurately ripped
 08    [cc16642a] (031/243) Accurately ripped
 09    [7e2be00f] (031/244) Accurately ripped
 10    [ac395338] (031/244) Accurately ripped
 11    [a84dd667] (031/243) Accurately ripped
 12    [9a985450] (031/242) Accurately ripped
 13    [14f63347] (031/241) Accurately ripped
 14    [108ff469] (031/240) Accurately ripped
Offsetted by 12:
 01    [bbca82d3] (039/242) Accurately ripped
 02    [e42252dc] (040/244) Accurately ripped
 03    [8f47975d] (040/242) Accurately ripped
 04    [afef0e7d] (040/244) Accurately ripped
 05    [21742c47] (040/244) Accurately ripped
 06    [642ec1fc] (040/244) Accurately ripped
 07    [dd42452c] (040/244) Accurately ripped
 08    [4c9037bc] (040/243) Accurately ripped
 09    [d619210d] (040/244) Accurately ripped
 10    [805ed9a6] (040/244) Accurately ripped
 11    [70a4dab2] (039/243) Accurately ripped
 12    [95affb69] (039/242) Accurately ripped
 13    [bdd5f004] (038/241) Accurately ripped
 14    [b723d551] (039/240) Accurately ripped

Track Peak [ CRC32  ] [W/O NULL] [  LOG  ]
 --  100.0 [E14194A9] [3845DFCA]  CRC32 
 01  100.0 [232AE82E] [6C108268]         
 02  100.0 [C41EDE35] [0C5A877F]         
 03  100.0 [8C7394E9] [F79D8FCF]         
 04  100.0 [A601C4D5] [F6E8AB22]         
 05  100.0 [C1834BBB] [B0902F80]         
 06  100.0 [BEDA7784] [E044784E]         
 07  100.0 [965B33F1] [61119B89]         
 08  100.0 [0D2283E4] [8C793781]         
 09  100.0 [BAA82B49] [31DE3286]         
 10  100.0 [FAEAF6F0] [D3A0C125]         
 11  100.0 [FA306F45] [6B7D36EF]         
 12  100.0 [5072D66A] [B3D87E5A]         
 13  100.0 [8B6B13AF] [A3960C93]         
 14  100.0 [198A6DFA] [ED81A1CC]         
Title: CUETools DB
Post by: Gregory S. Chudov on 2013-06-10 15:44:34
"discs with pregaps not supported in this protocol version"

Your rips are ok. You are just using an old version of CTDB EAC plugin.

You can download the new one here: http://www.cuetools.net/wiki/CTDB_EAC_Plugin (http://www.cuetools.net/wiki/CTDB_EAC_Plugin)
Title: CUETools DB
Post by: Boiled Beans on 2013-06-10 15:56:32
"discs with pregaps not supported in this protocol version"

Your rips are ok. You are just using an old version of CTDB EAC plugin.

You can download the new one here: http://www.cuetools.net/wiki/CTDB_EAC_Plugin (http://www.cuetools.net/wiki/CTDB_EAC_Plugin)


Thanks for the link, I didn't know it was available for download separately. I thought it just came with EAC.
Title: CUETools DB
Post by: Eli on 2013-07-03 15:56:45
Having an odd situation:

Code: [Select]
[CUETools log; Date: 7/3/2013 10:34:49 AM; Version: 2.1.5]
[CTDB TOCID: mzwdPKfBtGHL5aUjLD8FvCt7Lq8-] found.
Track | CTDB Status
  1  | (3/3) Accurately ripped
  2  | (3/3) Accurately ripped
  3  | (3/3) Accurately ripped
  4  | (3/3) Accurately ripped
  5  | (3/3) Accurately ripped
  6  | (3/3) Accurately ripped
  7  | (3/3) Accurately ripped
  8  | (3/3) Accurately ripped
  9  | (3/3) Accurately ripped
 10  | (3/3) Accurately ripped
 11  | (3/3) Accurately ripped
 12  | (3/3) Accurately ripped
 13  | (3/3) Differs in 1605 samples @06:04:27,06:14:04-06:14:05,06:16:36-06:16:37,06:17:02,06:17:22,06:18:08,06:18:68-06:18:69,06:19:34,06:19:74,06:21:25-06:21:26,06:21:45-06:21:46,06:22:11,06:23:17,06:23:57,06:24:43,06:24:63-06:24:64,06:25:08-06:25:09,06:26:14-06:26:15,06:27:20-06:27:21,06:27:41,06:29:12,06:30:18,06:32:10,06:32:50-06:32:51,06:34:22,06:35:08,06:35:68-06:35:69,06:37:20,06:37:40,06:37:60,06:38:06,06:38:26,06:38:66,06:39:32,06:40:38,06:40:58,06:41:03-06:41:04,06:41:24,06:41:44,06:41:64,06:42:10,06:42:30,06:42:70-06:42:71,06:44:62,06:45:48,06:47:40,06:47:60-06:47:61,06:49:32,06:51:04,06:52:71,06:53:16,06:53:57
[AccurateRip ID: 001add78-010f2411-a40efc0d] found.
Track  [  CRC  |  V2  ] Status
 01    [0bf217c9|067165db] (25+03/28) Accurately ripped
 02    [796a4f21|aaf4cc9b] (25+03/28) Accurately ripped
 03    [700eea07|9ca0f8c7] (25+03/28) Accurately ripped
 04    [5038bf10|569e6bdc] (25+03/28) Accurately ripped
 05    [17ee826f|046fec78] (25+03/28) Accurately ripped
 06    [099116a7|03cbd736] (25+03/28) Accurately ripped
 07    [eb01b984|1b60b060] (25+03/28) Accurately ripped
 08    [5f7d4d8e|2c1d0bc0] (25+03/28) Accurately ripped
 09    [be7c196b|0c80f699] (25+03/28) Accurately ripped
 10    [5a968f4c|85b58f1e] (24+03/27) Accurately ripped
 11    [d83fb6f7|e6389b96] (25+03/28) Accurately ripped
 12    [a95a221b|a63b20cb] (24+03/27) Accurately ripped
 13    [9a348df7|a727961a] (00+00/26) No match

Track Peak [ CRC32  ] [W/O NULL]
 --  100.0 [8AA47F2C] [33AFE4A4]         
 01  100.0 [8A76B48A] [07AF1899]         
 02  100.0 [CB4302D3] [A795B85D]         
 03  100.0 [4846CCC0] [CA291DE9]         
 04  99.9 [EAACE5C5] [6D145D55]         
 05  100.0 [800E9F13] [3DBCD818]         
 06  100.0 [62885B5C] [44FECE35]         
 07  100.0 [DE9C8A49] [3F149FE0]         
 08  100.0 [5E78D45D] [6DD6D8E7]         
 09  100.0 [2C14DAC6] [A1FAB7D8]         
 10  100.0 [F17B60E5] [6FB2ACF5]         
 11  100.0 [AF5734FC] [741D2828]         
 12  100.0 [86AE24B2] [67041A35]         
 13  100.0 [21C22BED] [92D9AC11]         

This disc looks like it should be repairable.
However, using 2.1.5, I go through the repair process and I get:

Code: [Select]
[CUETools log; Date: 7/3/2013 10:53:23 AM; Version: 2.1.5]
[AccurateRip ID: 001add78-010f2411-a40efc0d] found.
Track  [  CRC  |  V2  ] Status
 01    [0bf217c9|067165db] (25+03/28) Accurately ripped
 02    [796a4f21|aaf4cc9b] (25+03/28) Accurately ripped
 03    [700eea07|9ca0f8c7] (25+03/28) Accurately ripped
 04    [5038bf10|569e6bdc] (25+03/28) Accurately ripped
 05    [17ee826f|046fec78] (25+03/28) Accurately ripped
 06    [099116a7|03cbd736] (25+03/28) Accurately ripped
 07    [eb01b984|1b60b060] (25+03/28) Accurately ripped
 08    [5f7d4d8e|2c1d0bc0] (25+03/28) Accurately ripped
 09    [be7c196b|0c80f699] (25+03/28) Accurately ripped
 10    [5a968f4c|85b58f1e] (24+03/27) Accurately ripped
 11    [d83fb6f7|e6389b96] (25+03/28) Accurately ripped
 12    [a95a221b|a63b20cb] (24+03/27) Accurately ripped
 13    [9a348df7|a727961a] (00+00/26) No match

Track Peak [ CRC32  ] [W/O NULL]
 --  100.0 [8AA47F2C] [33AFE4A4]         
 01  100.0 [8A76B48A] [07AF1899]         
 02  100.0 [CB4302D3] [A795B85D]         
 03  100.0 [4846CCC0] [CA291DE9]         
 04  99.9 [EAACE5C5] [6D145D55]         
 05  100.0 [800E9F13] [3DBCD818]         
 06  100.0 [62885B5C] [44FECE35]         
 07  100.0 [DE9C8A49] [3F149FE0]         
 08  100.0 [5E78D45D] [6DD6D8E7]         
 09  100.0 [2C14DAC6] [A1FAB7D8]         
 10  100.0 [F17B60E5] [6FB2ACF5]         
 11  100.0 [AF5734FC] [741D2828]         
 12  100.0 [86AE24B2] [67041A35]         
 13  100.0 [21C22BED] [92D9AC11] 
Title: CUETools DB
Post by: Gregory S. Chudov on 2013-07-03 15:59:32
Just for this cd, or is repair generally broken in latest 2.1.5?
Title: CUETools DB
Post by: Eli on 2013-07-03 16:01:26
sorry, for some reason CT had switched from repair mode to encode mode...
Title: CUETools DB
Post by: drfr on 2013-09-24 12:52:26
Don´t know if I understand this log correctly.
CTDBID: what do the lines 4 and 5 with a comma mean? Rips with extra  data track, I suppose?
ARID: how to interprete these? All submissions without extra data track?


Code: [Select]
[CUETools log; Date: 24. 9. 2013 12:38:54; Version: 2.1.5]
CD-Extra data track length 03:53:62.
[CTDB TOCID: psc7oJ_dm56UJM94MfL1VjlYeDQ-] found.
        [ CTDBID ] Status
        [65ee1459] (20/65) No match
        [6a30aaf3] (06/65) Has no data track, Accurately ripped
        [ef7f65fe] (02/65) Has no data track, No match
        [07899421] (35/65) , Accurately ripped
        [18363f86] (01/65) , No match
        [a83d9063] (01/65) No match
Track | CTDB Status
  1   | (42/65) Accurately ripped
  2   | (42/65) Accurately ripped
  3   | (42/65) Accurately ripped
  4   | (42/65) Accurately ripped
  5   | (42/65) Accurately ripped
  6   | (42/65) Accurately ripped
  7   | (42/65) Accurately ripped
  8   | (42/65) Accurately ripped
  9   | (42/65) Accurately ripped
10   | (42/65) Accurately ripped
11   | (42/65) Accurately ripped
12   | (42/65) Accurately ripped
13   | (42/65) Accurately ripped
14   | (42/65) Accurately ripped
15   | (42/65) Accurately ripped
16   | (42/65) Accurately ripped
17   | (42/65) Accurately ripped
18   | (41/65) Accurately ripped
[AccurateRip ID: 002bbebc-0237a41b-1c0f8c13] found.
Track   [  CRC   |   V2   ] Status
01     [b1c3c382|04b613f2] (000+000/152) No match
02     [fa14ab2d|669b86fd] (000+000/156) No match
03     [406c3308|efa31aeb] (000+000/156) No match
04     [8f884ad0|1c78ae74] (000+000/157) No match
05     [e9911840|61072316] (000+000/155) No match
06     [e6889373|12424fea] (000+000/154) No match
07     [6247eb3e|6f865449] (000+000/156) No match
08     [8b0f1141|17893bfc] (000+000/154) No match
09     [81aa71e9|8c52f988] (000+000/152) No match
10     [857fd4f2|11f7d852] (000+000/153) No match
11     [3263472c|381d6854] (000+000/154) No match
12     [d69566f0|906bbf64] (000+000/153) No match
13     [15cafdcf|c65782ad] (000+000/154) No match
14     [5902383a|740322e7] (000+000/155) No match
15     [2809b5e6|a62f53b9] (000+000/153) No match
16     [24e09729|51974ee5] (000+000/150) No match
17     [04ca1f8d|0ed1e000] (000+000/150) No match
18     [9acfa964|7f3e285f] (000+000/149) No match

Track Peak [ CRC32  ] [W/O NULL]
--   98,7 [8BA73C6D] [2854A0A6]          
01   98,7 [326B9AD4] [AF5D23A1]          
02   98,7 [EDD4F0E7] [A723FB04]          
03   98,7 [08DF454F] [68014E4C]          
04   98,7 [B5E13BE3] [0ADE82AD]          
05   98,7 [D4F6C0EC] [C3562EB0]          
06   98,7 [CAD27194] [DBCA2364]          
07   98,7 [F40D7FA4] [132460B7]          
08   98,7 [AE5A321A] [42BD4153]          
09   98,7 [36DB26F7] [AD622910]          
10   98,7 [5B331A41] [60657964]          
11   83,0 [B2C34CBA] [0AB523FD]          
12   98,7 [7CC3BA94] [81733D93]          
13   98,7 [E65885C1] [22A6551F]          
14   58,2 [4BA11EAA] [E1DED7BE]          
15   98,7 [304B19E2] [9402F6D0]          
16   98,7 [F5B3A86F] [A0F39AD9]          
17   98,7 [674BF278] [FD5B7723]          
18   78,2 [43727891] [E86C78A6]
Title: CUETools DB
Post by: korth on 2013-09-24 14:14:42
Quote
CTDBID: what do the lines 4 and 5 with a comma mean? Rips with extra data track, I suppose?

It's a [a href='index.php?act=findpost&pid=837616']known bug[/a]. It should tell you CD-Extra data track length 03:54:17 before the comma.

Quote
ARID: how to interprete these? All submissions without extra data track?

The records under this AccurateRip ID included the data track and length. Some reading about AccurateRip ID calculation [[a href='index.php?showtopic=79082']link[/a]] [[a href='index.php?showtopic=53583']link[/a]].
A few reasons for the lack of matches could be you have
Title: CUETools DB
Post by: drfr on 2013-09-30 07:17:07
It should tell you CD-Extra data track length 03:54:17 before the comma.


Thanks, that is a correct track length. How did you know?
Title: CUETools DB
Post by: korth on 2013-09-30 13:29:28
Code: [Select]
[CUETools log; Date: 24. 9. 2013 12:38:54; Version: 2.1.5]
CD-Extra data track length 03:53:62.
[CTDB TOCID: [color=#FF0000][b]psc7oJ_dm56UJM94MfL1VjlYeDQ-[/b][/color]] found.
        [ CTDBID ] Status
        [65ee1459] (20/65) No match
        [6a30aaf3] (06/65) Has no data track, Accurately ripped
        [ef7f65fe] (02/65) Has no data track, No match
        [[color=#FF0000][b]07899421[/b][/color]] (35/65) , Accurately ripped
        [[color=#FF0000][b]18363f86[/b][/color]] (01/65) , No match
        [a83d9063] (01/65) No match
It was in the database
CUETools Database TOCID: psc7oJ_dm56UJM94MfL1VjlYeDQ- (http://db.cuetools.net/?tocid=psc7oJ_dm56UJM94MfL1VjlYeDQ-)
Click 07899421 or 18363f86 in the CTDB Id column. Look on 2nd page of the Track section.
Title: CUETools DB
Post by: drfr on 2013-09-30 18:04:22
CUETools Database TOCID: psc7oJ_dm56UJM94MfL1VjlYeDQ- (http://db.cuetools.net/?tocid=psc7oJ_dm56UJM94MfL1VjlYeDQ-)


Actually I was looking for a way how to look up a CTDB TOCID in the database, didn ´t know it was that easy. Is this documented somewhere? Not in the wiki AFAIK.
Title: CUETools DB
Post by: korth on 2013-09-30 20:12:51
The wiki has a link (http://www.cuetools.net/wiki/CUETools_Database#Links) to the database web page. I didn't plan to add documentation of this type to the wiki but I can. I'm so far behind updating pages already though.
Search by 'tocid' using db.cuetools.net/?tocid=
Search by 'artist' (most recent first) using db.cuetools.net/?artist=
Search by 'artist' (most popular first) using db.cuetools.net/top.php?artist=
of course you need to fill in missing data after the '=' such as db.cuetools.net/top.php?artist=adele
Title: CUETools DB
Post by: ebz777 on 2013-11-15 09:29:15
It doesn't seem very fair - people that would be wasting their CPU cycles to calculate those fingerprints would be helping a completely different category of users to identify random mp3s.

It is not for random mp3s actually. When I buy a compilation and when I rip it, I like to know to which original album releases the tracks correspond. Then I like also to be able to deconstruct that compilation and tag each individual tracks with the meta data of the original album releases, with a high certainty that these are the right albums.

Today I cannot do these if I don’t also own the albums, because it is often not possible to find out that information through the compilation liner notes and CUETools DB track CRCs are no help because they are two precise. If tomorrow CUETools DB contained the AcoustID of all tracks, we could determine which tracks we are actually buying and ripping.

It's quite fast to calculate an AcoustID, but I understand that some persons, sensitive to the time their rip takes or to the number of CPU cycles are opposed to calculate and publish them. However this concern could be overcome by providing this calculation as an option. That way, those persons could keep the option disabled. This should not be a problem for the database. It is sufficient after all if for each Disc Id just one person is motivated enough to calculate the AcoustID once in order to have that information available in the database attached to every track.
Title: CUETools DB
Post by: bilbo on 2013-11-15 14:27:11
@ebz777

There are already several programs out there that use the MusicBainz database to identify tracks. Picard is one of them. They put the acoustic id in the tags and give you all the tag info. IIRC, MP3Tag also uses MusicBainz for retrieving tags. Have you tried this route? If there are programs that do what you want, why reinvent the wheel here?
Title: CUETools DB
Post by: ebz777 on 2013-11-15 17:46:46
@ebz777

There are already several programs out there that use the MusicBainz database to identify tracks. Picard is one of them. They put the acoustic id in the tags and give you all the tag info. IIRC, MP3Tag also uses MusicBainz for retrieving tags. Have you tried this route? If there are programs that do what you want, why reinvent the wheel here?

I’m using MusicBrainz/AcoustID through Picard and Jaikoz. Unfortunately fingerprints in acoustid.org are linked to recordings in MusicBrainz not to Disc IDs. MusicBrainz/AcoustID rely on these current tagging softwares to calculate the fingerprints and associate them to recordings. Since these software work at the track level and not the disc level they have no better option.

And these tagging softwares can’t also force any particular workflow. So you end up having users tagging a bunch of mp3’s and polluting the database by associating them with the wrong recordings. In the end it’s a useless mess for the purpose I was describing.

Only a special ripping software like Cuetools could calculate a fingerprint for each track and ensure that they are associated together properly with a unique Disc ID.
Title: CUETools DB
Post by: greynol on 2013-11-15 18:27:15
I would hope that further development will go in the direction of calculating ARv2 hashes for alternate pressings using AR offset hashes when a match cannot be made against an ARv1 hash. This secondary step could be a user-configurable option.

New releases are likely only in the AR database with v2 hashes. AFAIK this means that pressings from new releases not yet in the AR database will only hit the CTDB if they are ripped with CUERipper or with EAC that is configured to use the CTDB plugin.

I'm not sure what kind of time Gregory has to spend on this, but I imagine it could be knocked-out in a couple of hours at most.  It's definitely more consistent with core thrust of his development than metadata.

I suggest ebz777 solicit spoon and someone to develop a third-party plugin for EAC, or Andre Wiethoff to incorporate it into his current project, assuming he isn't interested in doing so with EAC.  This would be more productive than necro-bumping this idea that was shot-down a year ago.

Anyone who wishes to discuss acoustic IDs further should create a new topic.  All future replies on the matter will be binned as off-topic.  If there is any confusion as to what is on-topic, please refer back to the post that started this discussion.
Title: CUETools DB
Post by: eleria on 2014-03-10 17:19:34
Hi, I got this error while verifying a rip to CTDB :
[CTDB TOCID: 2dFsnKg1uTph6lYwvtHK_pLgfkM-] database access error: Il existe une erreur dans le document XML (1, 1)..

While AR gave a normal response :
Code: [Select]
[AccurateRip ID: 00021a0a-000701e7-17039603] found.
Track   [  CRC   |   V2   ] Status
01     [b90d2469|b0d7967a] (11+06/17) Accurately ripped
02     [0135d72a|e733738e] (11+06/17) Accurately ripped
03     [f3e7fcf9|c8267af8] (13+06/19) Accurately ripped

Track Peak [ CRC32  ] [W/O NULL]
--  100,0 [B2CFA3FF] [725B254F]          
01  100,0 [D0B0E04E] [13D6A5BA]          
02   97,8 [C18821AC] [48F1C053]          
03  100,0 [E53C98FB] [1A21F364]
Title: CUETools DB
Post by: sivadselim on 2014-03-10 17:29:03
Hi, I got this error while verifying a rip to CTDB :
[CTDB TOCID: 2dFsnKg1uTph6lYwvtHK_pLgfkM-] database access error: Il existe une erreur dans le document XML (1, 1)..

Yep, same, here.

"database access error: There is an error in XML document (1, 1).."
Title: CUETools DB
Post by: Gregory S. Chudov on 2014-03-10 17:35:21
Thanks for letting me know, should be fixed now.
Title: CUETools DB
Post by: sivadselim on 2014-03-10 17:41:00
Thanks for letting me know, should be fixed now.



Thank you for CUETools!       
Title: CUETools DB
Post by: remenor on 2014-07-10 20:30:25
I copied a CD with EAC (AR confidence 1, CTDB 1) secure mode -without C2- but the last track did not match AR or with CTDB. The results were automatically sent to CTDB (even though there was error correction by EAC, in this track)
Then I tried to rip that track in 'Secure Mode + C2' :  EAC showed several suspicious positions, who obviously failed to correct. The results were not sent to CTDB.
I tried ripping with CueRipper (Secure Mode) which showed error correction on that track... Also, the results were sent to CTDB.
Finally, I tried dBpoweramp, but 'ultrasecure ripping' warned suspicious positions (10 of 11 ; 9 of 11; etc) in various sectors.

The rips with the 3 programs show error correction, but get the same CRC (same unit, same chipset: no guarantees of 'perfection') however the results are sent to CTDB. Can be configured (CueRipper or CTDB plugin) to send only manually or in case of rips secure (without rereading due to errors) ?
Now there are 3 submissions (in CTDB), 2 mine... probably no secure (1 of EAC, 1 of CueRipper)
Can be eliminate at least one of those submissions?
Thanks!

P.S: I tried to fix with CTDB (confidence 1) but that rip does not match AR (confidence 1). So, any of all or none will be fine
Sorry for my english
Title: CUETools DB
Post by: Gregory S. Chudov on 2014-07-10 20:35:24
It's ok. CTDB or AR entries with confidence 1 are never secure. You will only know which version is correct when a few other people will get the same result.

UPD: greynol pointed out that my statement is unclear, so let me explain what i mean by that.

Neither CTDB nor AR make any significant guarantees about the quality of a single submission. They cannot make those guarantees for a number of reasons: existence of undetectable ripping errors, drives that don't report errors with caches that aren't being defeated by the ripping program, CDr copies made with errors etc. We could introduce stricter submission rules, that would require people to use known-good CD drives, software that would protect against virtual CD drives, CD-R detection, etc, but that's not the path that was chosen.

Any single result in the database can be wrong. It's only useful to you when somebody elses result matches your own result. In that case, it doesn't really matter how those results were obtained and how many times the ripping software had to reread - the idea of AR/CTDB is that the fact of CRCs matching is assumed to be the best protection against ripping errors.
Title: CUETools DB
Post by: Wombat on 2014-07-11 01:43:52
Hi Grigory! I saw at your db page you offer a possibility to donate "to help cover hosting expenses"
I never saw you asking for help or money and want to ask here if this project is in danger!?

I still can't believe how well the magic works with repairing a broken cd rip! If you told me that 10 years ago...
I really want to miss it never more.
How much is it to get the "Don't write comment from CUE to tag" option?
One problem may be this bitcoin thing. I have no clue how to use it.
Keep up the great work!
Title: CUETools DB
Post by: Gregory S. Chudov on 2014-07-11 03:34:00
Surprisingly, Amazon's prices go down faster than the database grows, which is what i was hoping for when i chose AWS as a hosting platform. I used to pay about 70$ a month, now it's just barely above 30$.
I already forgot about this button. To be honest, i was just playing with bitcoin widget at the time  Don't worry about it.
I'm waiting till somebody (probably not me, because i'm out of ideas) will find a better solution to the comment tag problem. Because adding 999 options is not a sustainable way to develop a software product. EAC comments can be annoying too. "TOOL" tag would be better place for some of the stuff, but again ripping programs, encoders and cuetools would fight for the right to fill it. Tags are always such a mess i sometimes regret i support tags at all
Title: CUETools DB
Post by: Wombat on 2014-07-11 03:51:00
Thanks for answering and fine to hear it is here to stay  These Tag thing is indeed a reason for some to make a hobby out of it and it must be a pita for developers i understand. I think i only asked because it worked up to version 2.1.2 the way i liked it.
Title: CUETools DB
Post by: remenor on 2014-07-11 20:37:44
Thanks for your reply!
Title: CUETools DB
Post by: ChronoSphere on 2014-11-08 15:37:25
Is there anything in CTDB that stops someone with enough malicious intent from writing a submitter that purposely populates CTDB with manipulated entries, boosting their confidence?
Or is it purely reliant on the good will of the users? I'm guessing the latter.
Title: CUETools DB
Post by: krafty on 2014-11-24 18:15:53
Hi folks, I recently formatted and reinstalled my OSes.
Installed EAC with CTDB support, then after configuring AccurateRip, I copied the 2.1.4 new DLL's into EAC folder.
However, CTDB is not writing the log, nor showing up any where anymore.
Anyone know a trick no how to activate it?

Thank you a lot!
Title: CUETools DB
Post by: Surfi on 2014-11-24 19:10:14
Hi folks, I recently formatted and reinstalled my OSes.
::

Depending on installed OS you need to install CUETools Prerequisites (http://cuetools.net/wiki/CUETools_Download#Prerequisites).
I'd reinstall EAC afterwards.


Regards, ...

::
Title: CUETools DB
Post by: Gregory S. Chudov on 2014-11-24 19:13:00
I'm pretty sure EAC would have taken care of prerequisites. It must be something else. Make sure you copied all 6 dlls from CUETools/interop/EAC folder to the right place, over the old ones, and didn't copy any extra ones.
Title: CUETools DB
Post by: krafty on 2014-11-25 01:10:55
Turns out that Windows 8.1 does not enable .NET framework out of the box, whereas Windows 7 does.
Solved the issue enabling it via Control Panel and Windows Features.

Thanks for your kind help.
Title: CUETools DB
Post by: shaboo on 2014-11-25 11:17:52
The wiki has a link (http://www.cuetools.net/wiki/CUETools_Database#Links) to the database web page. I didn't plan to add documentation of this type to the wiki but I can. I'm so far behind updating pages already though.
Search by 'tocid' using db.cuetools.net/?tocid=
Search by 'artist' (most recent first) using db.cuetools.net/?artist=
Search by 'artist' (most popular first) using db.cuetools.net/top.php?artist=
of course you need to fill in missing data after the '=' such as db.cuetools.net/top.php?artist=adele

Thanks for this information! Very much appreciated
Title: CUETools DB
Post by: korth on 2014-11-27 15:12:18
I'm pretty sure EAC would have taken care of prerequisites. It must be something else. Make sure you copied all 6 dlls from CUETools/interop/EAC folder to the right place, over the old ones, and didn't copy any extra ones.

I've received a confirmation from Andre that the EAC V1.0 beta 3 installer does not play well with the detection of the .NET framework in Windows 8 & 8.1 and if not already enabled, .NET framework 3.5 has to be enabled in the Control Panel by the user. The CUETools wiki has been updated to note this. Andre says the installer in the next version (hopeful to be released in about a month or so) is fixed. I've reminded him that the CTDB plugin needs to updated in the next version as well.

edit: the -> that
Title: CUETools DB
Post by: Gregory S. Chudov on 2014-11-27 15:13:59
Thank you!
Title: CUETools DB
Post by: CreativeStructure on 2015-04-04 15:31:43
I'm looking for some info about CTDB Metadata Lookup.

When I go to download CD info I see two listings... for the same CD. One has a music note icon and the other is a record icon. What do each of these icons stand for? Both have the same track listings, album # etc...

Below is an example showing both the Music note icon and the record icon in the CTDB Metadata Lookup:

(http://www.cuetools.net/w/images/thumb/0/0e/EACCTDB.png/700px-EACCTDB.png)

thanX

Title: CUETools DB
Post by: Gregory S. Chudov on 2015-04-04 16:10:35
First is from Musicbrainz (http://musicbrainz.org/), second is from Discogs (http://www.discogs.com/).
Title: CUETools DB
Post by: CreativeStructure on 2015-04-04 17:40:32
First is from Musicbrainz (http://musicbrainz.org/), second is from Discogs (http://www.discogs.com/).


Ah, makes sense...

I love my music and have restrained myself to going digital but find it necessary. Although recently I have gone "over the hill" in age and my hearing too... I still find original recordings and records sound better. Now that storage cost is down significantly, I think converting all my CD's to digital is in order. I have gone with a dedicated 1TB HD to start using EAC going to uncompressed, FLAC, and MP3 (cringe).

Now it's Musicbrainz VS Discogs. 

It seems every part of going digital is one choice after another, sacrificing this or that, preference of one or another. I guess a virtual live performance area in the house is coming soon to see all these bands live...
Title: CUETools DB
Post by: Porcus on 2015-06-03 16:17:49
Here is a neat one, that the CUETools database just corrected for me.  A short track, the 4 seconds beginning of the Matthäuspassion, 168756 samples.
Was quite unlistenable.

Corrected: 20136 samples :-)


Title: CUETools DB
Post by: SF01 on 2015-11-24 14:39:02
I have a problem with it in EAC, I get the following error:
[CTDB TOCID: PEK2pEiNTbua5GnzdIT1nuatmwc-] database access error: The process cannot access the file 'C:\Documents and Settings\Nordic\Ustawienia lokalne\Temp\gkl27mzw.dll' because it is being used by another process.

The name of dll file is random, the fileitself is not visible in Temp.

This happens for some CDs and some not. Help?
Title: CUETools DB
Post by: korth on 2015-11-24 16:01:49
Original German
http://www.digital-inn.de/threads/cuetools-db-plugin-v2-1-6-database-access-error.47229/ (http://www.digital-inn.de/threads/cuetools-db-plugin-v2-1-6-database-access-error.47229/)

Google Translate
http://www.digital-inn.de/threads/cuetools-db-plugin-v2-1-6-database-access-error.47229/ (http://translate.google.com/translate?hl=en&sl=auto&tl=en&u=http%3A%2F%2Fwww.digital-inn.de%2Fthreads%2Fcuetools-db-plugin-v2-1-6-database-access-error.47229%2F)
Title: CUETools DB
Post by: SF01 on 2015-12-28 12:56:52
Original German
http://www.digital-inn.de/threads/cuetools-db-plugin-v2-1-6-database-access-error.47229/ (http://www.digital-inn.de/threads/cuetools-db-plugin-v2-1-6-database-access-error.47229/)

Google Translate
http://www.digital-inn.de/threads/cuetools-db-plugin-v2-1-6-database-access-error.47229/ (http://translate.google.com/translate?hl=en&sl=auto&tl=en&u=http%3A%2F%2Fwww.digital-inn.de%2Fthreads%2Fcuetools-db-plugin-v2-1-6-database-access-error.47229%2F)


I've run several virus/rootkit removal programs, it successfully deleted few tihngs.
But my problem remains.
Title: Re: CUETools DB
Post by: Porcus on 2017-01-13 09:28:24
Looking back at it, I would have wished for the CUETools DB to also hold enough cuesheet information to update my rips with index marks. I miss some of those in my collection, which was ripped in 2008 with dBpoweramp.

Not holding my breath though.
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2017-01-13 18:25:00
Index marks are messy. Their detection is optional and not 100% reliable. We could still try to collect that information, but i'd rather not claim to have a feature that doesn't actually work that good. One of the obstacles would probably be that EAC doesn't pass this information to plugins.
Title: Re: CUETools DB
Post by: Porcus on 2017-01-14 01:26:46
Sure - but after all, CTDB is all about user-generated data with lots of errors ;-)
(The obstacle you mention could be significant, if there is no way to load the cuesheet it generates.)
Title: Re: CUETools DB
Post by: ChronoSphere on 2017-01-15 16:56:58
I've been having intermittent connection problems with CTDB lately, for example, now. Getting either timeouts or "CTDB: database access error: There is an error in XML document (1, 1)"

Did anyone else notice anything?
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2017-01-15 17:54:11
Thanks for letting me know. The server was acting weird. I've rebooted it.
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2017-01-15 20:37:06
Actually i've just completed the long overdue server upgrade.  Old one wasn't able to keep up with the load anymore. Let me know if i've broken anything in the process.
Title: Re: CUETools DB
Post by: ChronoSphere on 2017-01-15 23:03:58
Seems to be working just fine for the moment - thanks for maintaining/upgrading it!
Title: Re: CUETools DB
Post by: Nongorilla on 2017-01-17 01:06:27
Yes, thanks so much for such a great service.  Currently working fine with EAC.
Title: Re: CUETools DB
Post by: Twangster on 2017-01-17 16:24:05
Gregory Chudov asked to be notified of any oddities following the server upgrade.  I've seen a CueRipper problem that started since the upgrade with some track titles which contain apostrophes or ampersands.  They are being displayed and stored (both in filenames and tag data) as "&apos;" and "&amp;" respectively.   It doesn't happen on every CD - only some.   The rips where I've seen the problem are as follows:
John Barry – The Best of John Barry  TOCID: 2fYUnO_Hzpp1IIUwe5HRV2X9fuo-  ripped 2002 Jan 15th
The Tornados – Telstar                      TOCID: oLWEIQ8dFwJgkCIb_tdl22Iv2.Y-       ripped 2240 Jan 15th
The Spotnicks – 18 Greatest Hits       TOCID: aXQPrKCgZ8NxjJOC9r8Rojf60Z8-    ripped 2301  Jan 15th
The Ventures - Eight Classic Albums  (4 CD set)
                                                           TOCID: _DeWrDqWlZucSUP2vC.1clOodq4-  ripped 1400  Jan 17th
                                                           TOCID: PsZefuprLlzBUsXuAXDCZTYi8Ls-      ripped 1410  Jan 17th
                                                           TOCID: 1r7.tzEWSTuNKp1CNDVvsNfbDiw-  ripped 1438  Jan 17th
                                                           TOCID: spTw0sQAgUKT1Im7Y_eU6iPOMVI- ripped 1448 Jan 17th

(All times UK time - GMT).

Discussed this issue with a friend who is an experienced CueToools/CueRipper user and he could not recall seeing any examples of the problem.  He took things a stage further and wrote a program to scan his music server and look for the issue in track titles in all the .cue files.  He found none.  He sent me his program and I scanned all my music and there were no examples of this problem on rips done before Sunday Jan 15th.
To check it was not a PC specific issue, I loaded the CueTools suite onto my wife's laptop and tried the same discs there - same result.
Not all rips performed since the upgrade have been effected.  The two discs in Eric Clapton's "Blues" 2-CD set both have titles containing apostrophes but they ripped OK.

Additional info:  I use CueRipper version 2.1.6 running under Windows 7 Pro on a Dell Optiplex.  Also on HP laptop with Windows 10.
 
Title: Re: CUETools DB
Post by: ChronoSphere on 2017-01-17 19:14:57
Hmm, confirmed, tag data from Musicbrainz(?) comes back with &amp; now. Occurred with TOCID: E2Y_RX.NHuc_k_yPjdtpYeiBgNo-
Title: Re: CUETools DB
Post by: plugs13amp on 2017-01-17 19:45:54
I also confirm Twangsters problem report. Just ripped Well Respected Kinks (image rip to flac). Track 8 is listed as "Don&apos;t You Fret" and thats how it appears in the .cue. I've also never seen this problem before in some 5 years of using CueTools.

TOCID: wUWpygSKsY0K8wyJGsER2tFFUTQ-
data came from MusicBrainz, the only match returned in the metadata dropdown

Querying the CT database directly with http://db.cuetools.net/?tocid=wUWpygSKsY0K8wyJGsER2tFFUTQ- shows the proper apostrophe in the listing.

Clicking on the CTDB entry downloads the xml file and that shows the &apos; as would be expected

Clicking on the MusicBrainz entry shows Track 8 with the proper apostrophe

CueTools v2.16, Windows 10 Pro

Title: Re: CUETools DB
Post by: Twangster on 2017-01-18 13:28:19
An update on my previous post:

I neglected to include the TOCIDS for the 2 CDs from the Eric Clapton "Blues" set which did not have the problem.  They are;
Disc 1      TOCID: yUXrhevhgFw9onbQ9SZwaoJH66A-
Disc 2      TOCID: uZ1uhW8ulqFneDJAwUst3WrxddY-

I've also discovered an interesting situation with another 2-CD set which might help with diagnosis of the problem:
Mark Knopfler – “The Ragpicker’s Dream”
Disc 1       TOCID: Z848KNXtvH8wfhBd33JMrj45v2w-
Disc 2       TOCID: 53dJ8rfqwb14RWrepMNIfvHvTW0-

With disc 1,  CueRipper offers three metadata options in the dropdown (all MusicBrainz) for US, British(GB) and German(DE) pressings.  Selecting the US or the GB options results in the &apos; problem (in this case in the album title as well as in one track title).  But selecting the DE option gives the correct display of the apostrophes.

Similar situation with disc 2:   two options offered - US and DE.   US option gives the problem, DE option is OK .

I used the disc 1 TOCID to access the cuetools database directly (as per plugs13amp's post) and, via that route, all three MusicBrainz options display the apostrophes correctly.

Just to be clear, the problem is not confined only to rips where MusicBrainz is the metadata source.  For the CDs listed in my previous posts, the Tornados disc showed the problem with both MusicBrainz and freedb, the Spotnicks with freedb (only option offered), the Ventures with both freedb and Discogs (no MusicBrainz option offered) and the John Barry with MusicBrainz (only option offered).
Title: Re: CUETools DB
Post by: ChronoSphere on 2017-01-18 13:52:41
This seems to be an encoding/codepage issue, which would explain why the German one works fine for you - it's not the same as your locale, so the apostrophe doesn't  get detected (and escaped) as such.
Title: Re: CUETools DB
Post by: Twangster on 2017-01-18 19:39:04
I don't see that it can be an encoding/codepage issue as, although the the metadata relates to a CD pressing manufactured in Germany (and sold in the UK), the metadata itself is not German.  Also, for another Mark Knopfler CD - "Kill To Get Crimson" - the metadata for the German pressing does have the &apos; issue.

I've just found another real oddball.  Joe Bonamassa - "You & Me",  TOCID: 6Xr7qE6SYV77A4mFl4d5o25gcKA-
The album title has the &amp; problem but apostrophes in the titles for tracks 5 and 6 display correctly.
Title: Re: CUETools DB
Post by: ChronoSphere on 2017-01-19 17:49:49
I don't see that it can be an encoding/codepage issue as, although the the metadata relates to a CD pressing manufactured in Germany (and sold in the UK), the metadata itself is not German.  Also, for another Mark Knopfler CD - "Kill To Get Crimson" - the metadata for the German pressing does have the &apos; issue.
How do you know they all use the same codepage? Submissions from a PC set to  English locale could happen in pure ANSI, while submissions from a PC set to German do not (because it has some characters not in basic ANSI codepage).

To a human, an apostrophe looks the same no matter which codepage it is encoded as, but internally the code might differ. So if the code matches an apostrophe that would have to be escaped to not break xml, we end up with &apos;, for the other, we just see '

Though this:
Quote
I've just found another real oddball.  Joe Bonamassa - "You & Me",  TOCID: 6Xr7qE6SYV77A4mFl4d5o25gcKA-
The album title has the &amp; problem but apostrophes in the titles for tracks 5 and 6 display correctly.
invalidates my theory, I guess.
Title: Re: CUETools DB
Post by: plugs13amp on 2017-01-19 23:15:05
The issue of different codepages doesn't apply as CueRipper doesn't query all the different CD databases. It queries the CueTools database (as defined in the options page) with the TOCID. The CT database returns a single XML file which contains all the metadata that fills CueRipper's metadata dropdown, and all the fields (tracks, title, year, barcode, etc) that change with your selection. You can see this by manually querying the database here
http://db.cuetools.net/?tocid=Z848KNXtvH8wfhBd33JMrj45v2w-
then clicking on the CTDB Id on the right. Then on the resulting page click the CTDB link on the left (beside the little CD image with the red cross). This XML document has no DTD and hence no "encoding=" statement, so no codepage or UTFn coding is specified. The XML document shows the &apos; comes from the CT database server and is visible because despite being a declared "predefined entity" along with &lt; &gt; &amp; and &quot; it is not recognised as such by most web browsers and xml parsers. This includes IE, Edge, Chrome, and the parser inside CueRipper. If you patch one of the XML files in Cuetools/MetadataCache to include any of the other entities, insert the appropriate CD and select the local data (green jigsaw piece) you can see your edits are translated properly. It is recommended that &apos; is not used because of this. It would be difficult to tell if the XML contains any of the other four entities as your web browser would translate them and you would never see the &nnnn; format. See section 3.1 of the XML Wiki at https://en.wikipedia.org/wiki/XML 

Incidentally, Twangster was mistaken in his observation that the Joe Bonnamassa album has both &amp; and real apostrophies (byte value x'27') - the apostrophies are actually "close single quotes", byte value x'92'. Thats also true of The Ragpicker's Dream in the above example and quite a few others. An easy mistake to make as in many fonts they look identical. It also appears that Chrome doesn't translate the &amp; either, I haven't checked other browsers.

Since this all started after Gregory Chudov upgraded the server, I suspect this is nothing more than he's missed a configuration setting in the webserver or possibly the underlying php server, to not translate the "predefined entities" into their &nnnn; format, and hopefully he can fix it quite quickly.

Title: Re: CUETools DB
Post by: lorcan on 2017-02-11 10:47:58
Hi there,

Metadata retrieved (if any) in EAC using CUETools DB Metadata Plugin (v2.1.6 in my case) are still not in sync with MusicBrainz DB :-(
MB informations retrieved using CTDB plugin seem to be taken from a very old copy (cache?) of MB database (or an older MB database?).
I only retrieve information of releases added to MusicBrainz (or modified ) more than 1 year ago.
Newer additions/edits are not retrieved.

Maybe a MusicBrainz software update…

I know that this issue was detected few months ago. What is the current CUETools project status for this point?

Thanks in advance.
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2017-02-11 19:58:51
I only retrieve information of releases added to MusicBrainz (or modified ) more than 1 year ago.
I fixed Musicbrainz sync last month, so i find it hard to believe.
Is there any particular CD you have problems with?
Title: Re: CUETools DB
Post by: lorcan on 2017-02-14 20:00:09
I fixed Musicbrainz sync last month, so i find it hard to believe.
Is there any particular CD you have problems with?

I'm pleased to read you made great efforts to fix the MB sync issue  ;)  Thank you very much.

I've just noticed that few of the CDs that I added few months ago are now correctly retrieved. Thank you for this, and sorry for "complaining".  :-[

However, I still have problems with some other CDs (that are not retrieved)  :(

I added this CD 3 days ago (it is correctly retrieved from MB by foobar2000, not using EAC 1.3 with CTDB EAC Plugin 2.1.6):
http://musicbrainz.org/release/170fa072-4d08-4667-a09c-6ccf67934b2e
https://musicbrainz.org/cdtoc/attach?toc=1%2021%20229876%20150%2011475%2022950%2027225%2043650%2054750%2063675%2076125%2088575%2094800%20107025%20126900%20132150%20149700%20159675%20166500%20172500%20185775%20193050%20198375%20210300

This one was added to MB on 2016-11-12 (not by myself):
https://musicbrainz.org/release/278f09e7-4ad0-4f24-b371-e0ba82e955dc
https://musicbrainz.org/cdtoc/attach?toc=1%2018%20344214%20150%205093%2022443%2040358%2064203%2081718%20101156%20120306%20156426%20170571%20189736%20211213%20231648%20251973%20269088%20290853%20309636%20334053

Another one (added 2016-07-31):
https://musicbrainz.org/release/34374725-2727-43e8-b665-fbc2e28abfa3

EDIT: I have the same problem using CTDB EAC Plugin v1.2.5 (with different drives). However, CUERipper correctly retrieves the metadata. Strange. An EAC issue? Did you also fix the CTDB plugin the same way?

Thanks  ;)
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2017-02-14 20:04:11
Thanks, i'll take a look.
Title: Re: CUETools DB
Post by: lorcan on 2017-02-14 20:15:04
Thanks, i'll take a look.

Thank you.
Another information, maybe useful: for the mentioned CDs, the CTDB results window closes directly itself in EAC. I can't see possible results returned.
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2017-02-15 05:30:07
Well, apparently the database produces correct results for those discs (for example, http://db.cue.tools/lookup2.php?version=3&ctdb=0&fuzzy=0&metadata=default&toc=0:4943:22293:40208:64053:81568:101006:120156:156276:170421:189586:211063:231498:251823:268938:290703:309486:333903:344064). If the plugin rejects those results, i can't figure out the reason. Will test it in more detail this weekend.
Title: Re: CUETools DB
Post by: lorcan on 2017-02-15 11:13:22
Ooow. I'm sorry but I was wrong. My apologies!

Everything is OK now. All MB metadata are indeed retrieved in EAC.
I wasn't intelligent enough to notice that the plugin automatically applies metadata and closes its windows when there's only 1 result. Since I disabled the cover search, I was not able guess that the metadata was not CD-TEXT but MB's data. Maybe keeping the result windows open for 1 results (and 0) would cause less confusion?

Sorry for all this confusion.

I confirm that I also get HTML codes instead of Unicode characters from MB:
& → &amp;
' → &apos;

…

Thanks again
Title: Re: CUETools DB
Post by: rontokyo on 2017-02-18 23:55:10
I confirm that I also get HTML codes instead of Unicode characters from MB:
& → &amp;
' → &apos;
The problem seems to be fixed [from where I sit, at least].
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2017-02-19 00:19:28
Actually it wasn't, but i just fixed it ;) Apparently it was a bug in the version of XMLSerializer package that got installed with server update. https://pear.php.net/bugs/bug.php?id=15602
Title: Re: CUETools DB
Post by: lorcan on 2017-02-19 17:09:52
Thank you very much!
Title: Re: CUETools DB
Post by: Twangster on 2017-02-20 01:59:01
Yes, ditto from me.    Thanks for fixing the &apos;/&amp;/&quot; problem.

However, one thing I've learned from this exercise is that I still have to watch out for the issue identified by plugs13amp where close-single-quote (x92) characters have been used in metadata instead of actual apostrophes (x27), as the Yamaha RN500 network receiver that I use to stream music from my NAS can't display x92 characters - they appear as underscores.  (But the associated Yamaha NP control app on my iPad does display them correctly.)   It would interesting to know if this is also a problem with other players.
Title: Re: CUETools DB
Post by: plugs13amp on 2017-02-28 01:01:02
Belated thanks for the &apos fix from me too (been on holiday)

In response to Twangster, my Squeezebox setup doesn't have a problem with the close-single-quote (x92) character, neither the web interface nor the hardware player's display (V3 Classic). My little Sansa Clip+ also displays it properly with the native software, but I have it set up to dual boot with RockBox, and that displays a dotted rectangle instead.
Title: Re: CUETools DB
Post by: grim_lokason on 2017-08-25 18:30:19
Hi !
It seems that the sync between CTDB and musicbrainz is down again.
 ( probably because of : https://blog.musicbrainz.org/2017/05/16/schema-change-release-2017-05-15-including-upgrade-instructions/ )

Is it possible to have a look ?

Best Regard, and thanks for your work :D
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2017-08-27 16:56:59
Thanks for letting me know. Running the upgrade now.
Title: Re: CUETools DB
Post by: old_liquid on 2017-11-16 18:26:55
Running the upgrade now.
Is there possible a manual database update with correct data?
I know that the CTDB uses the same data for multiple releases of the same disc, but the releases data might be obsolete or incorrect.
For example my dual set PIO-CD-5180-2 Pioneer's release of Armitage III OST barcode 013023518025 gets recognized as Geneon release 51942  barcode 013023519428 which is not. Just because there is no such of release (Pioneer's) in CTDB.
Correct me if I not understanding it properly.
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2017-11-16 19:30:52
Manual intervention was only needed once, back in August because the database schema changed in May, and i didn't notice it. Normal replication that syncs the database every hour was resumed in August, on the day when i made that comment.

Musicbrainz is probably just missing information about your release. Or more likely, it knows about your release, but doesn't have a CD TOC attached to it. The same CD TOC is attached to other release though (probably because those releases have the same TOC). You can manually attach your TOC to the correct release. When you click the musicbrainz button in the CUERipper status bar, it actually takes you to the musicbrainz website to do just that. If you do that, in about an hour the correct information will reach CTDB.
Title: Re: CUETools DB
Post by: old_liquid on 2017-11-17 02:53:47
Nah, I just used eac_log_to_musicbrainz_discid.py to verify/merge discids from Geneon release to Pioneer (appears to be they contain the same data)
For convenience maybe this function can be extended to CUETools: to use EAC logs/TOCs as database verify medium instead or in addition to cues, and for submitting to Musicbrainz. Of course it is not THAT reliable as original discs but still viable option.
Title: Re: CUETools DB
Post by: zegrelo on 2017-12-15 22:29:49
Can anyone clarify me about "Rel" column? I think "Rel" comes from "Relevance" but I dont understand the meaning of it.
Sometimes "Rel" column is empty but other times there are values between 0 and 100 I think.
From my understanding empty would be the best "value", followed by 100 until 0, right?
Title: Re: CUETools DB
Post by: korth on 2017-12-16 04:51:44
The column is blank when the TOC based lookup results in a match.
Not certain how the relevance is scored but the 100-0 is attached to results from a fuzzy search (https://en.wikipedia.org/wiki/Approximate_string_matching).
Title: Re: CUETools DB
Post by: zegrelo on 2017-12-19 21:31:41
The column is blank when the TOC based lookup results in a match.
Not certain how the relevance is scored but the 100-0 is attached to results from a fuzzy search (https://en.wikipedia.org/wiki/Approximate_string_matching).

Thank you for your explanation. Maybe the developer @Gregory S. Chudov can explain anything else about how the score is achieved. As I'm new at CTDB and I'm exploring the database, I'd really like to understand how important is the Relevance scores. From what I can see in the ctdb.php, and I'm not an PHP expert, looks like the score is achieved in a different way between different music databases.
Title: Re: CUETools DB
Post by: genericusername on 2018-01-13 20:25:44
Hello,
I am having an issue with my CTDB plugin in EAC. It is not pulling any metadata in. I ripped a CD using the MusicBrainz FreeDB proxy, and I'm getting a database access error when trying to submit the TOCID to CTDB. I'm on plugin version 2.1.6 and the latest version of EAC.
Thanks.
Title: Re: CUETools DB
Post by: marc1983 on 2018-01-30 17:22:27
Hello,

I ripped a CD from The Cranberries.
After the first rip I got this message:

Track  5  accurately ripped (confidence 200)  [B143B820]  (AR v2)
All tracks accurately ripped


End of status report

---- CUETools DB Plugin V2.1.6

[CTDB TOCID: cVfZRstXeJJVtLUMDmuuIcqRiIQ-] found
Submit result: cVfZRstXeJJVtLUMDmuuIcqRiIQ- has been confirmed
Track | CTDB Status
  1  | (1476/1486) Accurately ripped
  2  | (1473/1486) Accurately ripped
  3  | (1473/1486) Accurately ripped
  4  | (1473/1486) Accurately ripped
  5  | (1263/1486) Accurately ripped, or (205/1486) differs in 1362 samples @03:06:66-03:06:67
  6  | (1474/1486) Accurately ripped
  7  | (1474/1486) Accurately ripped
  8  | (1472/1486) Accurately ripped
  9  | (1475/1486) Accurately ripped
10  | (1465/1486) Accurately ripped
11  | (1474/1486) Accurately ripped
12  | (1461/1486) Accurately ripped
13  | (1471/1486) Accurately ripped

I retried to rerip the cd to see if my cd is bad:

Track  5  accurately ripped (confidence 200)  [B143B820]  (AR v2)
All tracks accurately ripped


End of status report

---- CUETools DB Plugin V2.1.6

[CTDB TOCID: cVfZRstXeJJVtLUMDmuuIcqRiIQ-] found
Submit result: already submitted
Track | CTDB Status
  1  | (1477/1487) Accurately ripped
  2  | (1474/1487) Accurately ripped
  3  | (1474/1487) Accurately ripped
  4  | (1474/1487) Accurately ripped
  5  | (1264/1487) Accurately ripped, or (205/1487) differs in 1362 samples @03:06:66-03:06:67
  6  | (1475/1487) Accurately ripped
  7  | (1475/1487) Accurately ripped
  8  | (1473/1487) Accurately ripped
  9  | (1476/1487) Accurately ripped
10  | (1466/1487) Accurately ripped
11  | (1475/1487) Accurately ripped
12  | (1462/1487) Accurately ripped
13  | (1472/1487) Accurately ripped

Does it means my (two) rips are accurate (+1 accurately ripped on CTDB tracks submitted ?) but someone tried to rerip 205 times to try to fix his cd?

Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2018-01-30 17:31:37
It means there are two versions of this disc. You got a perfect rip of the more popular version. About 13% of copies (205/1487) slightly differ in just one spot spanning just a few milliseconds. There's no telling which one is "better" other than listening carefully to this exact spot in both versions.
Title: Re: CUETools DB
Post by: marc1983 on 2018-01-30 17:59:00
Alright, so from the time I get the message "Accurately ripped" and highest number for example : 700/1000 it means my disc is perfectly ripped and is the same than 699 other people?

When you say there are two versions it is because 205 other people have 1362 samples different @03:06:66-03:06:67.
Is it possible than one person retried 205 times to submit result from different rip or via cuetools?

I listened to the song at 3 min 06 and there is nothing unusual than I can heard
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2018-01-30 18:09:24
it means my disc is perfectly ripped and is the same than 699 other people?
Yes.

Is it possible than one person retried 205 times to submit result from different rip or via cuetools?
No. One person can only submit once. Maybe a few times, if they have multiple computers and/or multiple CD drives. But not 205 times. Notice that when you copied your CD for the second time, the log said "Submit result: already submitted".

I listened to the song at 3 min 06 and there is nothing unusual than I can heard
Sometimes those things are barely audible, if at all. But most probably your version is OK.
Title: Re: CUETools DB
Post by: marc1983 on 2018-01-30 18:12:40
Ok thanks!

The warning : "or (205/1487) differs in 1362 samples @03:06:66-03:06:67" scared me but since then I get "accurately ripped" and a large number like 5/8, 46/60 or even 130/160 my rip is safe?

but if I get thing like :

"[8eb13402] (01/76) Differs in 78 samples @48:10:50,50:31:02
  [e9bcfc58] (01/76) No match
  [295eb06b] (01/76) No match
  [7af752c1] (01/76) Differs in 4879 samples @65:36:73-65:36:74"

then my disc has to many scratches or doesn't match the 75 other ones? am I correct?
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2018-01-30 18:16:55
"[8eb13402] (01/76) Differs in 78 samples @48:10:50,50:31:02
  [e9bcfc58] (01/76) No match
  [295eb06b] (01/76) No match
  [7af752c1] (01/76) Differs in 4879 samples @65:36:73-65:36:74"
That doesn't tell me anything. Out of 76 people who copied this, this is just 4 individual copies that probably went wrong. You can pretty much ignore anything that says (01/xx). Look for the big number.
Title: Re: CUETools DB
Post by: marc1983 on 2018-01-30 18:23:15
I thought it meant it was the only wrong different tracks ripped of the other 75 people

Then how can I know like my previous problem than my rip isn't the same as the others?
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2018-01-30 18:25:59
I didn't understand the question.
Title: Re: CUETools DB
Post by: Porcus on 2018-01-30 18:29:56
I thought it meant it was the only wrong different tracks ripped of the other 75 people
This looks like a CD with one long single track?

With one track, it might be that there is another release with different music exactly the same track length (although that happens more often with CD singles). If you are one out of two who have ripped the rare one, and 74 have ripped the different music ...
Title: Re: CUETools DB
Post by: marc1983 on 2018-01-30 18:33:26
I didn't understand the question.

All I need to pay attention to is that the mention "accurately ripped" and a large number initially like: 3/5, 40/52 180/220?
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2018-01-30 18:35:09
All I need to pay attention to is that the mention "accurately ripped" and a large number initially like: 3/5, 40/52 180/220?
Basically, yes.
Title: Re: CUETools DB
Post by: marc1983 on 2018-01-30 18:35:40
I thought it meant it was the only wrong different tracks ripped of the other 75 people
This looks like a CD with one long single track?

With one track, it might be that there is another release with different music exactly the same track length (although that happens more often with CD singles). If you are one out of two who have ripped the rare one, and 74 have ripped the different music ...


It was a example from http://cue.tools/wiki/CUETools_log : CTDB (CUETools Database) Section
Title: Re: CUETools DB
Post by: marc1983 on 2018-01-30 18:43:02
All I need to pay attention to is that the mention "accurately ripped" and a large number initially like: 3/5, 40/52 180/220?
Basically, yes.

Alright this is all need to know. In other term I can ignore message like I got "(205/1486) differs in 1362 samples @03:06:66-03:06:67" as long as this number is lower (205/1486) than (1263/1486) because the majority has ripped the same pressing (presumably?) like mine.

Plus I can consider than my track is OK because there is  an previous verification from Accuraterip "Track  5  accurately ripped (confidence 200)  [B143B820]  (AR v2)" (confidence 200 means at least 200 people got the same checksum?)
Title: Re: CUETools DB
Post by: Porcus on 2018-01-30 18:50:21
(Edit)
as long as this number is lower (205/1486) than (1263/1486) because the majority has ripped the same pressing (presumably?) like mine

The majority 1263 vs 205 means nothing to the rip itself: even the 205 would have been perfectly ripped. Most likely even if it had been 5 and not 205.
Now you could argue that an idiosyncratic error at some pressing plant is less likely to show up in larger numbers, but what if the defective was pressed for a different market? If you don't hear anything wrong, then yours is good enough, don't worry about the others.

(Sometimes a mispress is recalled and replaced - if serious, it is likely to be noticed early and get a lower number.)
Title: Re: CUETools DB
Post by: Porcus on 2018-01-30 18:52:51
It was a example from http://cue.tools/wiki/CUETools_log : CTDB (CUETools Database) Section
OK, a rip as one.  Look at this, track by track:
  9   | (74/76) Accurately ripped, or (1/76) differs in 78 samples @01:41:24,04:01:51
 10   | (74/76) Accurately ripped
 11   | (71/76) Accurately ripped, or (1/76) differs in 4879 samples @07:43:59-07:43:60, [...]


Such a result would mean that it matches 74 others. (Or "73" if you retro-verify later and your own is one of the 74.)
The (1/76) is likely a bad rip.
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2018-01-30 18:59:24
It was a example from http://cue.tools/wiki/CUETools_log : CTDB (CUETools Database) Section
That part is from an optional detailed log. It shows you everything the database knows about other people's rips and how they relate to you. It's hard to understand, that's why it's disabled by default.

Alright this is all need to know. In other term I can ignore message like I got "(205/1486) differs in 1362 samples @03:06:66-03:06:67" as long as this number is lower (205/1486) than (1263/1486) because the majority has ripped the same pressing (presumably?) like mine.
Like Porcus said, majority is not terribly important. As long as the number of people who have the same result as you is "high enough". For example, if your CD was the minority version, it would say 'Accurately ripped (205/1486) or differs ... (1263/1486)", and you could still be confident you copied the CD correctly - 205 is definitely "high enough". If there's a problem, it was not with your rip, but with your CD, so no point in copying it again.

Plus I can consider than my track is OK because there is  an previous verification from Accuraterip "Track  5  accurately ripped (confidence 200)  [B143B820]  (AR v2)" (confidence 200 means at least 200 people got the same checksum?)
Yes.
Title: Re: CUETools DB
Post by: marc1983 on 2018-01-30 19:29:08
Ok thank you guys
Title: Re: CUETools DB
Post by: dpr on 2018-06-09 22:19:20
Is it possible to get a list of fields aka tags that are stored in the database, but more specifically returned to EAC via the plug-in.
thanks
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2018-06-10 02:56:10
You can see the XML for yourself, for example here (http://db.cue.tools/lookup2.php?version=3&ctdb=1&metadata=extensive&fuzzy=1&toc=0:22162:38893:65040:86858:105265:123297:140204:161650:178680:199065:217942).
Title: Re: CUETools DB
Post by: dpr on 2018-06-10 20:33:16
You can see the XML for yourself, for example here (http://db.cue.tools/lookup2.php?version=3&ctdb=1&metadata=extensive&fuzzy=1&toc=0:22162:38893:65040:86858:105265:123297:140204:161650:178680:199065:217942).

Thanks. It looks like just the basic album / track fields.
- no Composer
- no Lyrics
- no ISRC

etc.

Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2018-06-10 20:38:50
Well... yeah. The metadata part is a simplified interface to freedb/discogs/musicbrainz, that loses some details in favor of unified fast interface.
Title: Re: CUETools DB
Post by: Brand on 2018-09-01 14:05:24
Are the CRCs in the web database (db.cuetools.net) supposed to match the CRCs from EAC and CUETools (program)?
I noticed that EAC and CUETools show the same CRCs for all tracks, but the web database shows different CRCs for the first and last track of a CD.
Title: Re: CUETools DB
Post by: korth on 2018-09-01 14:26:06

The database excludes the first 10 sectors of the first track and the last 10 sectors of the last track to allow for read offset and pressing offset differences.
http://cue.tools/wiki/CUETools_Database

but going up to 10 is overkill.
You are forgetting that CTDB stores only one record for all pressings. Pressing offset adds to drive offset, and pressing offsets can be quite large. Even if drive offsets are limited to 4 sectors, pressing offsets of 6 sectors are not uncommon.
Title: Re: CUETools DB
Post by: Porcus on 2018-09-21 15:27:28
Just curious: retro-verifying one of my CDs gives
CTDB: verified OK, confidence 158, or verified OK, confidence 124.

Why?
Title: Re: CUETools DB
Post by: korth on 2018-09-21 16:08:06
Verify log please or at least provide the CTDB TOCID so I can look it up.

(without the opportunity to have a look) Sometimes a submission has a pressing offset greater than 10 sectors (10×588 samples) from the original submission. The data is still the same but it is out-of-range. When this happens a new CTDBID is created. If you have a rip of a pressing that is within 5880 samples (10 sectors) of both records, you receive results from both.
Title: Re: CUETools DB
Post by: Porcus on 2018-09-22 00:26:22
Looks like that. One example: http://db.cuetools.net/?tocid=WVtAckpKyOMYNcgIVL0PcPgKIcs-

The two matches are each less than 5880 away from mine, but more than 5880 apart from each other:
Code: [Select]
        [03be4ad4] (159/313) Accurately ripped
        [fbdb1263] (007/313) No match
        [f1381a20] (125/313) Accurately ripped
        [cc70dfce] (001/313) No match
        [5e23b55e] (001/313) No match
        [47894b5a] (018/313) No match
        [f8313530] (001/313) No match
        [556da365] (001/313) No match

(AccurateRip hits are with offsets within +910 and -709 ...)
Title: Re: CUETools DB
Post by: Gregory S. Chudov on 2018-10-16 16:11:56
I recently moved the CTDB website code from sourceforge to github - https://github.com/gchudov/db.cue.tools
And repaired discogs monthly snapshot import that was broken for a while.
Title: Re: CUETools DB
Post by: Thundik81 on 2018-10-16 16:35:07
I recently moved the CTDB website code from sourceforge to github - https://github.com/gchudov/db.cue.tools
And repaired discogs monthly snapshot import that was broken for a while.

Thank you!
Title: SPLIT: CUETools on GitHub. May we see a new compile soon? [split from topic CUETools DB]
Post by: korth on 2018-10-24 19:53:31
One or more of the messages of this topic have been moved to CUETools (https://hydrogenaud.io/index.php?board=74.0) - https://hydrogenaud.io/index.php?topic=116785.0
SimplePortal 1.0.0 RC1 © 2008-2018