Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: CUETools DB (Read 330151 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

CUETools DB

Reply #25 –
Hey, this is a great initiative. I always wondered what "contacting CTDB" means. Now, I know, big brother. 

On a rare occasion I get what looks like a repairable error, e.g. "Differs in 28 samples @26:19:54". But how do I actually repair the rip in that case? I'm using CUETools 2.0.8.

CUETools DB

Reply #26 –
Use encode in repair mode.

It seems CTDB can detect differences without being necessarily errors, verify that the rip is not accurate before trying to repair a perfect rip.

CUETools DB

Reply #27 –
Quote from: sauvage78 on
Use encode in repair mode.

It seems CTDB can detect differences without being necessarily errors, verify that the rip is not accurate before trying to repair a perfect rip.


Ok, I tried that, but I get a message box with "Exception: Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index". This appears after the encoding finishes. Bug? Edit: I disabled lookup in all databases except AccurateRip (rant: did not feel like waiting for FreeDB again on the 2nd attempt, why the heck does it take so long anyway, re-encoding to Monkey Audio High takes less on my i3), and it worked, repair included. I get "CUETools DB: corrected 28 errors." Great!

My EAC log said "Range quality 99.9 %", so I assume the error was real, although I could not hear anything unusual on the damaged track, even with my Senn 280 headphones. Is there some visual diff tool for PCM files, so I can narrow down the range?

Now I have another question: if I have rip that's reported accurate by AccurateRip, except for the whole disc offset, how do I submit the repair info to CTDB? I see only about 10K discs in the database, many of my classical CDs aren't in. (I generally don't bother with fixing the offset: too much hassle for the tiny benefit, but I understand that CTDB works regardless.)

CUETools DB

Reply #28 –
1: Well I never had "Index out of range" when trying to repair but I already had this message several time when trying to encode, IIRC it means that one of the index is reporting a timing located after the end of the whole CD length which is impossible. Usually this means the index (read cue sheet) is broken. This is not directly related to repairing.
2: Use verify in submit mode.

CUETools DB

Reply #29 –
Quote from: sauvage78 on
1: Well I never had "Index out of range" when trying to repair but I already had this message several time when trying to encode, IIRC it means that one of the index is reporting a timing located after the end of the whole CD length which is impossible. Usually this means the index (read cue sheet) is broken. This is not directly related to repairing.

It appears it was a fluke, possibly caused by info from FreeDB. It worked fine with the cue sheet I had from EAC.

Quote
2: Use verify in submit mode.


Edit Worked fine if the disc is in AccurateRip db, but not in CTDB. It said: "CUETools DB: 1vH1n8CUu3EZgEhNA_amJjkUM8c- has been uploaded." Now you know who did that one!

But if the disc is not in either database, then I just get a message box saying "AccurateRip: disk not present in database", and there's no other report afterward. So, it doesn't work for discs that aren't in AccurateRip's database? Shouldn't it submit to both databases in that case?

Another question: what does the  "AR" column mean on http://db.cuetools.net/? Is it supposed to be the offset? Cause if that's the case, the info displayed is wrong. I submitted 4 discs, all ripped with my ancient LITE-ON LTR-52327S (which died a glorious death it 2004 ripping a large swath of my pressed CDs), which has an offset of 6, but the info in the AR column is all over the map from 1vH1n8CUu3EZgEhNA_amJjkUM8c- to 1S1uNzRfWHCqQx5LU1YM65CvFhQ-.

CUETools DB

Reply #30 –
I never tried to submit myself (I wait to have a faster CPU) but IIRC I think I read that Greg requires the rip to be AR3 before accepting the submission ... for me AR2 is enougth but you know there are some magic stuff behind CTDB that I don't really understand so far (specially 10 frames left out instead of 5 like AR or even 4 which is already enougth & leaving out submissions to CTDB with AR3 instead of AR2)

Edit: The AR columm cannot be an offset, my guess is that it is the accuraterip confidency, but it seems to conflict with what I was saying that Greg requires an AR of 3 because there are AR1 submissions. It's either I am wrong or these are pending submissions waiting to be AR3 before being swallowed in the CTDB, I dunno.

CUETools DB

Reply #31 –
Quote from: Teknojnky on
methinks the record companies would have a little issue with the sampling and reconstruction of their copywritten audio

I don't expect any problems on this front. There are many databases which store various information about CDs - freedb, musicbrainz, discogs and AccurateRip to name a few. And now CTDB. None of them ever had any legal problems, because none of them store any actual audio, they all store various meta-information -  CD TOCs, artist names, crcs, etc.  Recovery records obviously fall under the same category.

Quote from: Eli on
Clearly the current DB is reasonable for 1 person to host. The idea was to pay for a larger, more robust DB, in order to cover those costs. As I suggested, maybe the best solution would be to offer free access to the current DB structure, and a for fee access to the larger track based DB.

I'd like to keep this project absolutely free. Current database contents was submitted by the users and it belongs to the users. If i'm not mistaken, Musicbrainz uses Creative Commons license for it's content. CTDB will use the same, if there are no objections. If hosting costs will become significant at some point, i think we will work something out without compromising the database freedom.
CUETools 2.1.6

CUETools DB

Reply #32 –
Quote from: Respwaned2 on
It appears it was a fluke, possibly caused by info from FreeDB.

I recently found a glitch like this, which happens when manually specifying Pregap. If this was it, it'll be fixed in 2.0.9. If it wasn't, i'd like to know more about it.

Quote
But if the disc is not in either database, then I just get a message box saying "AccurateRip: disk not present in database", and there's no other report afterward. So, it doesn't work for discs that aren't in AccurateRip's database? Shouldn't it submit to both databases in that case?

Nope. CUETools uses AR as a proof of existence of physical CD with such content. The only way to submit data that is not in AR database is to rip a CD using a ripper. Only CUERipper supports CTDB at this point. It also cannot submit to AR database, which only accepts submissions from EAC/dbPowerAmp.

Quote from: sauvage78 on
Edit: The AR columm cannot be an offset, my guess is that it is the accuraterip confidency, but it seems to conflict with what I was saying that Greg requires an AR of 3 because there are AR1 submissions. It's either I am wrong or these are pending submissions waiting to be AR3 before being swallowed in the CTDB, I dunno.

It is indeed AccurateRip confidence number (+ number of rips done with CUERipper). AR1 submissions are from CUERipper.
CUETools 2.1.6

CUETools DB

Reply #33 –
Quote from: Gregory S. Chudov on
I recently found a glitch like this, which happens when manually specifying Pregap. If this was it, it'll be fixed in 2.0.9. If it wasn't, i'd like to know more about it.


I didn't manually specify anything, I'm too lazy for that. One of the CDs I was experimenting with (and have uploaded) does have a pregap though; I'm not sure what the "transient filter" thing is about:

Code: [Select]
[Verification date: 5/7/2010 10:01:41 PM]
[AccurateRip ID: 0021b8d3-01693d52-de0fd90e] found.
Pregap length 00:00:32.
HDCD: peak extend: none, transient filter: some, gain: none
[CTDB TOCID: sYAiRuv0qD8JKDHv8BYqLw9Pr0Y-] disk not present in database.


I'll see if I can reproduce it.

Quote
Nope. CUETools uses AR as a proof of existence of physical CD with such content. The only way to submit data that is not in AR database is rip a CD using a ripper. Only CUERipper can do this at this point. It also cannot submit to AR database, which only accepts submissions from EAC/dbPowerAmp.


That seems a little excessive, especially if the EAC log is present. Do you really think people would spoof it? Someone determined to do that kind of nonsense could do it at the protocol or source code level as well. Besides, physical existence of a real pressed CD (as opposed to cd-r) is more easily proved by FreeDB/MusicBrainz. For instance:

Code: [Select]
[Verification date: 5/7/2010 11:37:23 PM]
[AccurateRip ID: 003554dd-030b2cea-12109f14] disk not present in database.
[CTDB TOCID: Zva2.w2WY9olyZ6gyCpXAq3BQr8-] disk not present in database.
Disk not present in database.


But this disk is in FreeDB (although not in MusicBrainz)

Code: [Select]
EAC extraction logfile from 22. March 2004, 17:00 for CD
Aznavour / 40 chansons d'or - disc 1

Used drive  : LITE-ON LTR-52327S   Adapter: 1  ID: 0
Read mode   : Secure with C2, accurate stream, disable cache
Combined read/write offset correction : 0
Overread into Lead-In and Lead-Out : No

Used output format : Internal WAV Routines
                     44.100 Hz; 16 Bit; Stereo

Other options      : 
    Fill up missing offset samples with silence : Yes
    Delete leading and trailing silent blocks : No
    Native Win32 interface for Win NT & 2000


Range status and errors
Selected range
     Filename F:\ape\calme\azn\1\CDImage.wav

     Peak level 100.0 %
     Range quality 100.0 %
     CRC 31594526
     Copy OK

No errors occured

End of status report


The disc seems to be this one: http://www.freedb.org/freedb/blues/0f109f14 (edit: based on the typo in "alller" on the last track, LOL, I only saw it now). Cuetools doesn't display the freedb id, although it calculates it for sure. I'm not ripping my CDs again just to submit them to your database, so it's your loss...

By the way, it appears the FreeDB search in CUETools takes so long because it's only approximate. It also offeres to pick http://www.freedb.org/freedb/blues/1f10b314 for the above, for instance.

CUETools DB

Reply #34 –
Quote from: Respwaned2 on
I'm not sure what the "transient filter" thing is about:

This is irrelevant. It just says that HDCD encoding was detected.

Quote from: Respwaned2 on
That seems a little excessive, especially if the EAC log is present. Do you really think people would spoof it?

It's not really intended as spoof protection, we have to rely on users good will. It's just a way to reduce the amount of bad rips in the database. Many people tend to trust the database entries, no matter how many times they are reminded that no database can really assure the accuracy of the data, so i have to do my best to increase this accuracy. I also have to keep database size low. If every rip was accepted, very soon we would have database polluted with unlimited number of inaccurate rips, some of them ripped with itunes or other unreliable rippers, with lots of errors, or downloaded on the net, we'll have even a certain amount of mp3=>flac transcodes.

I never thought of EAC log as an alternative submission criteria, i will think about it. Thank you, that is a useful idea.

Quote from: Respwaned2 on
By the way, it appears the FreeDB search in CUETools takes so long because it's only approximate.

The speed depends mostly on freedb server latency. It is often overloaded. All freedb searches are 'approximate', many CDs can have the same discid. Musicbrainz is much more precise and has better data quality - almost no typos, none of this UPPERCASE nonsense etc, however it's database is not yet as large as freedb's.
CUETools 2.1.6

CUETools DB

Reply #35 –
I've also got a "Exception: The specified path, file name, or both are too long." (more blah, blah omitted) error while trying to submit one one disc. Nothing was printed in the lower area. Cannot tell why: the paths and file names seem well below the 260 limit. I can open them just fine in explorer or foobar2000.

CUETools DB

Reply #36 –
Maybe the output path (for the log) was too long - the default template places it somewhere like C:\Documents and settings\username\My Music\.... which can easily get too long. I'll think how to handle this case better, maybe by abbreviating artist/album names to something shorter when generating a path.
CUETools 2.1.6

CUETools DB

Reply #37 –
How do I submit a rip to CTDB??? I tried Cueripper.
Can't wait for a HD-AAC encoder :P

CUETools DB

Reply #38 –
I guess it is submitted when the rip is done...

@Gregory: have you thought about also adding the pregaps to your database? Since I get inconsistent pregap readouts with CUERipper and EAC on some CDs, it may be a helpful for spotting problematic discs in general. Does the CTDB also store drive models, btw?

CUETools DB

Reply #40 –
I just ripped a few CD's that should not be in the DB. They don't show up at the home page.

And how do I correct bad rips with it?
Can't wait for a HD-AAC encoder :P

CUETools DB

Reply #41 –
You can easily check if CD is in database - when you load CD in the drive, CUERipper shows a confidence number in it's status bar next to CTDB icon.

It doesn't submit in Burst mode or when there were suspicious positions with many retries.

To correct a rip you'll need to use CUETools. Currently CUERipper can only verify & submit.
CUETools 2.1.6

CUETools DB

Reply #42 –
Quote from: Gregory S. Chudov on
It doesn't submit in Burst mode or when there were suspicious positions with many retries.

Oh that's why...

Quote from: Gregory S. Chudov on
To correct a rip you'll need to use CUETools. Currently CUERipper can only verify & submit.

I know, but I still don't understand when it will correct. Do I need to use Encode mode to do that?

Feature request: Put in log, when an album is found in CTDB when it's verifying multiple albums.
Can't wait for a HD-AAC encoder :P

CUETools DB

Reply #43 –
Select encode mode, and select 'repair' from the combobox below (where other scripts are, 'fix offset' etc).
CUETools 2.1.6

CUETools DB

Reply #44 –
Quote from: Gregory S. Chudov on
It doesn't submit in Burst mode or when there were suspicious positions with many retries.

Why don't you allow burst-mode rips to create CTDB entrys? If the disc is already in AR you can just accept to submit it if it rips correctly. I actually considered this, just to put all my discs in the DB, but ripping them in secure mode takes too long I think.

Another approach (similar to dbpa): Rip burst, if accurate submit, if not accurate (or not available in AR), rip secure.
Can't wait for a HD-AAC encoder :P

CUETools DB

Reply #45 –
While I agree that burst rip should be accepted as long as those are AR2, I disagree with the idea that burst mode would always be faster than secure mode. This is only true if you compare burst vs. secure on a non-scratched CD. It also depends on the number of re-read you're using in your secure mode & it also depends on if you're ripping as CDImage or as separate tracks (because re-ripping a single scratched track is faster than re-riping a whole CD, which means that burst works even faster with separate tracks). As soon as there are a few scratched CD in your collection (and the average joe always have a few scratched CD in his collection), it is "on average" faster to use the secure mode with a lower number of re-reads than to completely drop the secure mode in favor of the burst mode (This is specially true if you rip as CDImage as you cannot only re-rip the scratched track). This is due to the fact that the burst mode will be faster on a single pass but you will need several pass to achieve an AR result. So if you want to avoid the headache of re-ripping ten time the same track (or whole CD if you use CDImage) you'd better use a secure mode with fewer re-reads than completely drop the secure mode. This is specially true with CDImage, but this is also true with separate tracks because even if re-riping a single track in burst mode is very fast. You will have to verify each burst rip separatly with cuetools, & as it will be burst, the chances that it will be AR2 on a really scratched CD will be low. Overall this means that the time you will gain for using burst will likely be lost checking dozen of burst rip against AR with cuetools. (Edit: Here I am a special case as I delete EAC log but keep .accurip which forces me to use cuetools after the ripping process, the cuetools part might be irrevelant to you if you directly check AR thru EAC)

When AR was created I was thinking like you that it would make the secure mode obsolete in favor of burst+AR2. This is not true, AR only allow you to reduce the number of re-reads. My experience tells me that it is faster to rip a CD twice in Secure+Low than ripping it ten time in burst & then checking ten time if it's AR. Burst mode+AR2 can be faster if you're lucky (lucky to only have non-scratched CD in your collection), but it's pure random. The only exception is maybe if you just bought the CD as it will be shinning new. As I don't want to use two settings for new & old CD (even with profiles), this exception is irrevelant to me.

In the end this means that even if AR2 burst rip should be accepted, it is not really a priority IMHO because in the end the burst mode is not always as fast as it seems. It depends on what you rip (new or old CD) & on how you rip (what setting you're comparing burst to).

Edit: The above applies to EAC, I never tried CUERipper so far.

The fact that CTDB ignores 10 frames instead of 5 like AR is a "much bigger" (I know some people people might not consider 5 frames "big" but I disagree) design flaw IMHO, I already have 3 rips which are not AR but are CTDB OK ... even if AR & CTDB doesn't serve the same purpose, it is a little weird to have them conflicting.

Exemple:
Code: [Select]
[Verification date: 05/05/2010 21:42:49]
[AccurateRip ID: 001555ed-00b500c2-990b410b] found.
[CTDB TOCID: 9ZJTSx3s7H3pnsu6h7eKNf9M8xE-] found.
        [ CTDBID ] Status
        [2b6d1a2c] (190/190) Accurately ripped
Track	[ CRC    ] Status
 01	[ec93376d] (76/203) No match but offset
 02	[dbd08c29] (77/203) Accurately ripped
 03	[83406c8a] (76/201) Accurately ripped
 04	[983f8873] (77/203) Accurately ripped
 05	[eabd4d44] (77/203) Accurately ripped
 06	[10c7c833] (77/204) Accurately ripped
 07	[d4328e57] (77/204) Accurately ripped
 08	[d98b9da8] (76/203) Accurately ripped
 09	[984cf36c] (77/203) Accurately ripped
 10	[0994ce6b] (77/203) Accurately ripped
 11	[4e9e9b41] (76/198) Accurately ripped

Note: This log was shortened.

CUETools DB

Reply #46 –
Quote from: Gregory S. Chudov on
Quote from: Eli on
Clearly the current DB is reasonable for 1 person to host. The idea was to pay for a larger, more robust DB, in order to cover those costs. As I suggested, maybe the best solution would be to offer free access to the current DB structure, and a for fee access to the larger track based DB.

I'd like to keep this project absolutely free. Current database contents was submitted by the users and it belongs to the users. If i'm not mistaken, Musicbrainz uses Creative Commons license for it's content. CTDB will use the same, if there are no objections. If hosting costs will become significant at some point, i think we will work something out without compromising the database freedom.



My suggestion was to keep the current database and any future entries to the way it is currently configured free, but have a parallel database with track based repair info. Since this would be 10+x bigger, it would be reasonable to charge a small amount for this.

Of course I would prefer to have the larger database for free, but if the choice is between paying for the bigger database or not having it available, I would pay.

I would really like to see spoon use this database in dbpoweramp. He seemed to imply that in its current configuration he is less likely to. I think this is very unfortunate, but his choice.

CUETools DB

Reply #47 –
I only post an interesting log that I just found in my collection which illustrates the previous case but in an even more dramatical way:

First the rip seems not AR (scratched on last track) but repairable:

Code: [Select]
[Verification date: 09/05/2010 23:08:09]
[AccurateRip ID: 000c13cb-00528e2b-5d099708] found.
[CTDB TOCID: V7Pd0AEJx0Xa0Jx_3rtludlTqmc-] found.
        [ CTDBID ] Status
        [61c027c2] (3/3) Differs in 5809 samples @40:54:57-40:54:62
Track	[ CRC    ] Status
 01	[c1859a35] (00/05) No match
 02	[407b9e51] (00/05) No match
 03	[f62fa730] (00/05) No match
 04	[a810ba68] (00/05) No match
 05	[e5c90916] (00/05) No match
 06	[7129814f] (00/05) No match
 07	[542c41ed] (00/05) No match
 08	[de7f4fc1] (00/05) No match
Offsetted by 1650:
 01	[b1e49860] (05/05) Accurately ripped
 02	[36696c23] (05/05) Accurately ripped
 03	[45702970] (05/05) Accurately ripped
 04	[cbc2af16] (05/05) Accurately ripped
 05	[a887219d] (05/05) Accurately ripped
 06	[577756e8] (05/05) Accurately ripped
 07	[daf1beb2] (05/05) Accurately ripped
 08	[46a0b81d] (05/05) No match but offset

Track Peak [ CRC32  ] [W/O NULL] 
 --   87,6 [D97FEE4E] [07750B53]           
 01   87,6 [8D9D5270] [6A32175F]           
 02   47,9 [C4C45951] [F17B1FAF]           
 03   82,8 [476003BF] [C71C7807]           
 04   82,2 [90F608CE] [07A337C8]           
 05   80,9 [4807F605] [019ADA7D]           
 06   69,2 [F8907160] [4A9941DB]           
 07   57,4 [193B474F] [D43FCCE8]           
 08   81,3 [C991DD9C] [EAC43C72]     

But after trying to repair it finally seems that the scratches is located in the hole between the 5 frames ignored by AR & the 10 frames ignored by CTDB. I agree it's a rare case (4th case so far), but as the start/end of the rip is usually a critical point, it does happen in real life case.

Code: [Select]
[Verification date: 10/05/2010 03:34:28]
[AccurateRip ID: 000c13cb-00528e2b-5d099708] found.
CUETools DB: corrected 5809 errors.
Track	[ CRC    ] Status
 01	[c1859a35] (00/05) No match
 02	[407b9e51] (00/05) No match
 03	[f62fa730] (00/05) No match
 04	[a810ba68] (00/05) No match
 05	[e5c90916] (00/05) No match
 06	[7129814f] (00/05) No match
 07	[542c41ed] (00/05) No match
 08	[33c3d834] (00/05) No match
Offsetted by 1650:
 01	[b1e49860] (05/05) Accurately ripped
 02	[36696c23] (05/05) Accurately ripped
 03	[45702970] (05/05) Accurately ripped
 04	[cbc2af16] (05/05) Accurately ripped
 05	[a887219d] (05/05) Accurately ripped
 06	[577756e8] (05/05) Accurately ripped
 07	[daf1beb2] (05/05) Accurately ripped
 08	[8e3f75b6] (05/05) No match but offset

Track Peak [ CRC32  ] [W/O NULL] 
 --   87,6 [F55BD6EA] [A6E84737]           
 01   87,6 [8D9D5270] [6A32175F]           
 02   47,9 [C4C45951] [F17B1FAF]           
 03   82,8 [476003BF] [C71C7807]           
 04   82,2 [90F608CE] [07A337C8]           
 05   80,9 [4807F605] [019ADA7D]           
 06   69,2 [F8907160] [4A9941DB]           
 07   57,4 [193B474F] [D43FCCE8]           
 08   81,3 [E5B5E538] [8E0A8761]         

When I compare AR database vs CTDB, 9247 discs vs. 1.4 Millions, it's either you fix it now or we will have to live with this flaw. The AR database history shows that once the database is populated it becomes almost impossible to change/fix it, as deleting the "populated but not optimal" database is heartbreaking & managing two databases is a headache. So IMHO the more you wait the less likely to be fixed it becomes.

I am not criticizing the database, it works: I fixed around 25 rips & used it to found the exact data track length of around 20 rips so far. It's just that it could work better. 5 frames might seems very small but scratches often happens in the first & last tracks, so IMHO this 5 frames have a special value due to their location. Also I simply don't understand the logic of not doing like AR when CTDB is so tied to its fate, so I'd rather ask to be sure: is there a technical reason to ignore those 5 extra frames or was it a random (uninspired) choice ?

Edit: I forgot to say that indeed the rip becomes CTDB OK after the repairing, so it cannot be a case where the damage is beyond repair.

Code: [Select]
[Verification date: 10/05/2010 04:27:21]
[AccurateRip ID: 000c13cb-00528e2b-5d099708] found.
[CTDB TOCID: V7Pd0AEJx0Xa0Jx_3rtludlTqmc-] found.
        [ CTDBID ] Status
        [61c027c2] (3/3) Accurately ripped

CUETools DB

Reply #48 –
The last time you claimed "damage due to a scratch" I proved that it wasn't.

What makes you so sure the data is in error this time around?  Have you taken the time to manually compare the the "repair" to the original rip to see what the differences are?

Differences between pressings can vary by more than just the offset at the edges.  For this reason, extending the area of what is ignored can actually be a good thing.

I've also pointed out that EAC used to have a bug that caused the last two frames of non-null samples on the disc to be repeated on occasion in secure mode when overreading is disabled.  It was not corrected until V0.99pb5.  Has anyone besides me bothered to investigate this phenomena?

I guess it's just easier to ignore the possibility that things aren't always so cut and dry and just attribute it to a scratch (presumably resulting in an error that your ripping program failed to identify?).

CUETools DB

Reply #49 –
Quote from: Eli on
Of course I would prefer to have the larger database for free, but if the choice is between paying for the bigger database or not having it available, I would pay.

I would really like to see spoon use this database in dbpoweramp. He seemed to imply that in its current configuration he is less likely to. I think this is very unfortunate, but his choice.

I'm just not ready to handle financial aspects. I'm not good at this. So it's better to just keep database costs to a minimum, and hope that storage will be cheap enough in the future.

If Mr. Spoon will decide to support CTDB on condition that it's track based, we'll discuss it. Also nothing stops him from creating such a database himself based on CTDB code.
CUETools 2.1.6