HydrogenAudio

CD-R and Audio Hardware => CD Hardware/Software => Topic started by: korndawg on 2015-08-29 05:48:05

Title: Different Checksums
Post by: korndawg on 2015-08-29 05:48:05
When I rip the same song from from different CD releases (i.e. the original release versus a rerelease) or even different pressings of the same CD release (i.e. the first pressing versus the second pressing) the checksums are different (I am using cdparanoia to rip CDs and sha1sum to generate checksums). What exactly is different between these digital files? Is some type of diagnostic information about the release and/or pressing embedded within each digital file? Would songs ripped from different CD releases and/or pressings sound exactly the same, or would they sound different?

On a similar note, back when Columbia House and BMG Music Service sold CDs in bulk for dirt cheap prices, would a song ripped from one of those CDs sound exactly the same as from the store-bought version, or would they sound different?

Thanks in advance! 
Title: Different Checksums
Post by: KozmoNaut on 2015-08-29 08:01:07
A checksum is sort of a digital fingerprint of a file or string of data, even the slightest change will generate a completely different fingerprint.

What usually happens with compilation albums is that all the tracks are normalized to play back at the same volume. This alters the digital signature of the file completely, even though the music itself is untouched except for a slight change in volume.

On re-releases, there is often some remastering done, unfortunately it was very popular for a time to compress and amplify the hell out of everything in the name of "digital remastering". Luckily, there are also good remasters out there, but no matter if it's a good or bad remaster, it still changes the signal, and thus the checksum.

You can generally only expect to get the exact same checksum between two copies of an album, if both of those albums have the same label ID.
Title: Different Checksums
Post by: Porcus on 2015-08-29 11:51:10
When I rip the same song from from different CD releases (i.e. the original release versus a rerelease) or even different pressings of the same CD release (i.e. the first pressing versus the second pressing) the checksums are different (I am using cdparanoia to rip CDs and sha1sum to generate checksums). What exactly is different between these digital files?


For different pressings, the difference is often merely left-shift or right-shift of the entire bitstream. That is why software like dBpoweramp and CUETools can test different offsets. You should definitely try a ripper with AccurateRip support - it will compare to what others have gotten. morituri, Rubyripper, (pird ?,) or Wine up a MS-Windows application (people have gotten CUETools to work under mono (using .wav only), but I do not know if the ripper works). I am not up-to-date on precisely which rippers can check across offsets though - http://wiki.hydrogenaud.io/index.php?title...n_of_CD_rippers (http://wiki.hydrogenaud.io/index.php?title=Comparison_of_CD_rippers) may not be comprehensive and updated.

For different releases, it could very well be different masterings, which indeed are different.
Title: Different Checksums
Post by: ChronoSphere on 2015-08-29 13:26:37
CUERipper works fine here (win32 wine prefix).
Title: Different Checksums
Post by: korndawg on 2015-08-29 15:12:44
KozmoNaut, thanks for your reply. I understand that compilation albums are often normalized and/or remastered, and I was asking specifically about checksum differences between rereleases and/or pressings of the same CD.

Porcus, when you say a simple left-shift or right-shift, you're talking about a few bits and/or samples that would not affect the sound in any way, correct? I previously calculated my CD drive's offset using Exact Audio Copy, but I am a Linux guy so I prefer to use cdparanoia with the "--sample-offset" parameter. Although cdparanoia does not support AccurateRip technology per se, it performs thorough jitter and error detection. Please let me know if I'm missing something.

Regarding Columbia House and BMG Music Service CDs, I found this dated yet extremely interesting article last night:

http://www.stereophile.com/features/55/index.html (http://www.stereophile.com/features/55/index.html)

Thanks in advance for any additional input! 
Title: Different Checksums
Post by: KozmoNaut on 2015-08-29 16:45:35
Regarding Columbia House and BMG Music Service CDs, I found this dated yet extremely interesting article last night:

http://www.stereophile.com/features/55/index.html (http://www.stereophile.com/features/55/index.html)


Please don't believe a single word of that article. The measurements he claims to do are sound enough, such as the cancellation test, but after that it completely veers into audiophile bullshit.

The measurements show no difference at all between the store-bought and record club editions, but he's so damned sure he hears a difference that he has to make up a reason. And it turns out to be the ever-nebulous jitter, currently the foil of any measurement-shunning audiophile. In a decade or two, it'll be something else that is blamed for any imagined differences in sound.
Title: Different Checksums
Post by: korndawg on 2015-08-29 19:46:31
Regarding Columbia House and BMG Music Service CDs, I found this dated yet extremely interesting article last night:

http://www.stereophile.com/features/55/index.html (http://www.stereophile.com/features/55/index.html)


Please don't believe a single word of that article. The measurements he claims to do are sound enough, such as the cancellation test, but after that it completely veers into audiophile bullshit.

The measurements show no difference at all between the store-bought and record club editions, but he's so damned sure he hears a difference that he has to make up a reason. And it turns out to be the ever-nebulous jitter, currently the foil of any measurement-shunning audiophile. In a decade or two, it'll be something else that is blamed for any imagined differences in sound.


So in other words the Columbia House and BMG Music Service CDs contain exactly the same audio content as the store-bought versions? If hope that is the case because I have several Columbia House and BMG Music Service CDs from back in the day...
Title: Different Checksums
Post by: Porcus on 2015-08-29 21:09:12
Porcus, when you say a simple left-shift or right-shift, you're talking about a few bits and/or samples that would not affect the sound in any way, correct?


Yes. 


I previously calculated my CD drive's offset using Exact Audio Copy, but I am a Linux guy so I prefer to use cdparanoia with the "--sample-offset" parameter.


The reason why they could still differ said way, is that the masters are also written using different offsets. Per-drive offset correction in ripping only fixes offset differences between different drives used for reading.

The applications I mentioned run on penguinware. Myself I actually started using Windows to get dBpoweramp/fb2k ... of course you can use CUETools retroactively to verify with AccurateRip, provided that either you have cuesheets or all index marks are standard.

One more thing: there are many reasons to use .flac rather than .wav/.aiff, and one of them is the built-in checksum - which works on the (decoded) audio, so it is unaffected by tagging.
Title: Different Checksums
Post by: Porcus on 2015-08-29 21:23:12
So in other words the Columbia House and BMG Music Service CDs contain exactly the same audio content as the store-bought versions?


Those are. And you can safely write off the jitterbug from Stereophile as nonsense (and even if it were true, it would not matter for the rips!).

But generally: because two CDs of the same title could be different masterings, so could a Columbia House and a retail version be - you do not know from the outset whether they used mastering X or mastering Y.
Title: Different Checksums
Post by: KozmoNaut on 2015-08-30 08:27:57
But it is unlikely that Columbia House etc. would spend the resources on remastering or otherwise changing the sound, as that costs money, and the retail mastering job is perfectly adequate already. These clubs sell albums cheaper than retail, so there's absolutely no sense in them wasting money on unnecessary changes.

The whole "maybe they make deliberately more compressed versions for the 'lower class' stereos of their customer base" line of thought in the first paragraph is baseless audiophile elitism at its worst.
Title: Different Checksums
Post by: Porcus on 2015-08-30 15:13:15
But it is unlikely that Columbia House etc. would spend the resources on remastering or otherwise changing the sound, as that costs money, and the retail mastering job is perfectly adequate already.


Sure. But nothing says you are  less likely to get a different master from Columbia House, than in the record store. If customer asks "Why does [title] by [artist] from Columbia House sound different than the CD I already own?", then one should first make sure the customer knows enough to expect that there are different masterings around.

What I do not know anything about, is whether the CD clubs replace their masterings as the record company releases remastered editions. If not - or if "maybe not always" ...




The whole "maybe they make deliberately more compressed versions for the 'lower class' stereos of their customer base" line of thought in the first paragraph is baseless audiophile elitism at its worst.


Well ... I wouldn't call it "baseless" as it is essentially true, only if you point out the truly guilty part: lots of CD club releases sound inferior compared to the original, because the CD club releases are the very same masters as the ones the record companies crippled in the loudness war.
Title: Different Checksums
Post by: korndawg on 2015-08-31 00:17:06
The reason why they could still differ said way, is that the masters are also written using different offsets. Per-drive offset correction in ripping only fixes offset differences between different drives used for reading.

I assume the "master" is the CD that is used as the source image for commercial pressing? Who generates the master? And they don't know how to correct for drive write offsets?

One more thing: there are many reasons to use .flac rather than .wav/.aiff, and one of them is the built-in checksum - which works on the (decoded) audio, so it is unaffected by tagging.

I'm saving the ripped WAV files and then also compressing to FLAC as well. Is there a central location where these built-in FLAC checksums are stored?

Thanks so much for sharing your wisdom!
Title: Different Checksums
Post by: korndawg on 2015-08-31 00:23:08
Well ... I wouldn't call it "baseless" as it is essentially true, only if you point out the truly guilty part: lots of CD club releases sound inferior compared to the original, because the CD club releases are the very same masters as the ones the record companies crippled in the loudness war.

As a neophyte audiophile I'm torn on remastering. Some remastered discs sound fantastic, while others are obviously brickwalled and compressed. Do you guys evaluate each remaster on a case-by-case basis, or do you always opt for the original recording? Are there any particular discs that stand out as the absolute worst remastering jobs? It would be nice if there was a database out there where enthusiasts could rate the quality of each remastering.

Thanks again for your help!
Title: Different Checksums
Post by: korndawg on 2015-08-31 03:06:32
For different pressings, the difference is often merely left-shift or right-shift of the entire bitstream. That is why software like dBpoweramp and CUETools can test different offsets. You should definitely try a ripper with AccurateRip support - it will compare to what others have gotten. morituri, Rubyripper, (pird ?,) or Wine up a MS-Windows application (people have gotten CUETools to work under mono (using .wav only), but I do not know if the ripper works). I am not up-to-date on precisely which rippers can check across offsets though - http://wiki.hydrogenaud.io/index.php?title...n_of_CD_rippers (http://wiki.hydrogenaud.io/index.php?title=Comparison_of_CD_rippers) may not be comprehensive and updated.

Would you recommend Exact Audio Copy? Exact Audio Copy supports drive offsets and AccurateRip, and supposedly runs flawlessly under Wine...
Title: Different Checksums
Post by: mjb2006 on 2015-08-31 06:12:37
It would be nice if there was a database out there where enthusiasts could rate the quality of each remastering.

Such a database would probably be depressing to look at. It would just reflect the fact that even among "enthusiasts", anything remastered simply sells. Anything new sells, too; people think newer is better, and will say they like the newer sound because they can't imagine the record companies would ever let them buy something that sounds worse. People also generally prefer louder, brighter sounds...they'll rate highly any remastering that boosts the upper midrange, boosts the volume (even if it means killing the dynamic range), and widens the stereo image. Even people who seek out high-DR recordings may well say, in a blind and level-matched comparison, that they like the squashed remaster better because they like the updated EQ.

From personal experience, what sounded fine and bright when I was a teen sounds dull and muddy in my 40s. This is partly due to hearing loss, and partly due to the fact that I'm used to hearing modern music, which is mixed & mastered to be competitive EQ-wise, not just loudness-wise, and the trend has been to give certain kinds of sounds more "presence" and brightness than they would've had before the 1990s. Original masterings of pre-'90s recordings just sound like they're from another era, whereas newer remasters of that exact same material have a modern sound that fits better alongside contemporary music. Sometimes I like the updated sound of these remasters, sometimes I don't. I do consider the older masterings to be more authentic. On the other hand, what good is authenticity if I'm messing with the bass and treble anyway, listening on different gear, playing digital files instead of cassette tape, etc.? I'm still giving it an updated sound, to some degree.

Speaking of things from a bygone era, EAC isn't dead yet, but it's really falling off my radar. dBpoweramp and CUERipper are much simpler, slicker alternatives for secure ripping on Windows. However if they don't run well in emulators, you may be better off using EAC.

By the way, the CD drive "frame jitter" that cdparanoia copes with is not the same kind of sample-timing, wow & flutter-type of "jitter" that audiophiles normally talk about. What cdparanoia calls frame jitter is just inconsistent offset from read to read. Modern drives use Accurate Stream technology to minimize the chances of this happening, so the offset should always be the same on every read, and no detection and correction is necessary. However, I did have it happen to me once with a modern drive, so never say never.
Title: Different Checksums
Post by: KozmoNaut on 2015-08-31 08:56:36
Well ... I wouldn't call it "baseless" as it is essentially true, only if you point out the truly guilty part: lots of CD club releases sound inferior compared to the original, because the CD club releases are the very same masters as the ones the record companies crippled in the loudness war.

As a neophyte audiophile I'm torn on remastering. Some remastered discs sound fantastic, while others are obviously brickwalled and compressed. Do you guys evaluate each remaster on a case-by-case basis, or do you always opt for the original recording? Are there any particular discs that stand out as the absolute worst remastering jobs? It would be nice if there was a database out there where enthusiasts could rate the quality of each remastering.


It really is a case-by-case thing, as it depends heavily on who's doing the remastering.

As a rule of thumb, I think that as long as the original release doesn't have any glaring faults, just go with that version if you can find it. A lot of modern remasters are overly compressed, and I've even seen a few driven into clipping for no reason at all.

But there are also some very good remasters out there, like the 25th anniversary edition of Machine Head, which is definitely my favorite. It's clean, well-balanced and makes everything slightly less muffled without destroying the original intent.

The best example I can find, which is kinda cheating, is "First Daze Here" by Pentagram. I've heard some samples of the original master tapes, and they are absolutely horrifically noisy. The remastering job done on them was an absolutely epic effort. Sure, the noise reduction is noticeable a lot of the time, but the snarl of the Rickenbacker bass and the bite of the snare drum is preserved so well that you cannot believe it came from those borderline-destroyed master tapes.

Done right, remastering can work wonders. Done wrong, it can destroy everything (I'm looking at you, Iggy Pop. The remaster of Raw Power was horrifyingly bad).
Title: Different Checksums
Post by: Porcus on 2015-08-31 11:33:37
Is there a central location where these built-in FLAC checksums are stored?


In the file, you mean? It is part of the format, and e.g. vuplayer's audiotester.exe or foobar2000's file integrity verifier will decode, compute a checksum and compare to the one stored.

It would be nice if there was a database out there where enthusiasts could rate the quality of each remastering.


http://dr.loudness-war.info/ (http://dr.loudness-war.info/) offers some partial answers - not subjective "quality", but at least it gives some info on what is destroyed in the loudness war. Do not use it for vinyl, the metric is simply not suitable to compare a rip with vinyl noise to one without.
Regarding old masters, beware of pre-emphasis on a few titles.


Would you recommend Exact Audio Copy? Exact Audio Copy supports drive offsets and AccurateRip, and supposedly runs flawlessly under Wine...


Has EAC implemented cross-pressing verification yet? If not, it is still a great piece of software, though I have to agree with mjb2006.  I settled with dBpoweramp seven years ago (and would have considered CUETools today).
Title: Different Checksums
Post by: korndawg on 2015-08-31 22:20:33
Would you recommend Exact Audio Copy? Exact Audio Copy supports drive offsets and AccurateRip, and supposedly runs flawlessly under Wine...

Has EAC implemented cross-pressing verification yet? If not, it is still a great piece of software, though I have to agree with mjb2006.  I settled with dBpoweramp seven years ago (and would have considered CUETools today).

Wine and Mono were misbehaving (shocking I know), so I took morituri for a test drive and it is FANTASTIC! It supports AccurateRip, and it automatically tags each file (previously I was manually tagging each file with Kid3). Is there any downside to morituri? I still have to carve out some time to fully RTFM, but I'm guessing that if the two fingerprints match (in this case they are both "9266803a"), then the rip successfully matches the AccurateRip database:

    Track  8: rip accurate    (confidence  75 of 200) [9266803a], DB [9266803a]

Also, I'm assuming that the more people verify the fingerprint, the higher the confidence rating. The only trouble I've encountered so far is that one disc was misidentified, specifically the Beatles 2009 Remastered Red Album Disc 1 was misidentified as the original 1993 version. During ripping three possible matches were identified:

    Matching releases:

    Artist  : The Beatles
    Title  : 1962–1966 (2009 remasters) (Disc 1 of 2)
    Duration: 00:31:03.772
    URL    : http://musicbrainz.org/release/1903f00b-51...77-ecf1835a34bb (http://musicbrainz.org/release/1903f00b-5131-48ee-9f77-ecf1835a34bb)
    Release : 1903f00b-5131-48ee-9f77-ecf1835a34bb
    Type    : Compilation

    Artist  : The Beatles
    Title  : 1962–1966 (Disc 1 of 2)
    Duration: 00:31:00.239
    URL    : http://musicbrainz.org/release/301aa0fc-75...e2-74c3e9a7ebf7 (http://musicbrainz.org/release/301aa0fc-75a7-3e55-82e2-74c3e9a7ebf7)
    Release : 301aa0fc-75a7-3e55-82e2-74c3e9a7ebf7
    Type    : Compilation

    Artist  : The Beatles
    Title  : 1962–1966 (Disc 1 of 2)
    Duration: 00:31:01.973
    URL    : http://musicbrainz.org/release/d1ef3171-67...ce-f89c61bb2870 (http://musicbrainz.org/release/d1ef3171-678e-338d-a9ce-f89c61bb2870)
    Release : d1ef3171-678e-338d-a9ce-f89c61bb2870
    Type    : Compilation


However, unfortunately morituri picked the wrong disc:

    ARTIST:    The Beatles
    TITLE:      Love Me Do
    ALBUM:      1962–1966 (Disc 1 of 2)
    TRACKNUMBER:        1
    DATE:      1993-10-05
    MUSICBRAINZ_TRACKID:        1f518811-7cf9-4bdc-a656-0958e130f312
    MUSICBRAINZ_ARTISTID:      b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d
    MUSICBRAINZ_ALBUMID:        301aa0fc-75a7-3e55-82e2-74c3e9a7ebf7
    MUSICBRAINZ_ALBUMARTISTID:  b10bbbfc-cf9e-42e0-be17-e2c3e1d2600d
    MUSICBRAINZ_DISCID: VTsA.WWKwyZD3AJSspUvKIRPKcE-


The fingerprints did match, so the rip was clearly successful, and I can just update the tags accordingly. But does this sort of disc misidentication thing happen often?

Thanks again for your help!
Title: Different Checksums
Post by: Porcus on 2015-09-01 11:50:51
Also, I'm assuming that the more people verify the fingerprint, the higher the confidence rating.


As long as you have not submitted rips before, 2 or above will be practically equivalent in most cases. (If a certain CD has a confidence of 90 on all tracks but one, and 2 on that - well, it could be a manufactoring defect, but then there is most likely not "one correct way" to read it. If the odd track is the last one, then I guess the "lead out" should be the issue ... or whatever, I have forgotten the explanations I got, but it is not abnormal that the last track stands out with lower score.)



The only trouble I've encountered so far is that one disc was misidentified


The metadata sources are not more reliable than what you see; freedb can be populated by anyone but corrected by no-one. In particular, if you disagree with whoever submitted the release and the year on whether the year should be of the original or of the reissue in question, you will have lots of retagging to do.

dBpoweramp can combine multiple metadata sources (and even uses Allmusic - which is prone to listing the year the release was first available in the US though...) AFAIK, EAC has started to get better sources as well, and CUETools is decent. Windows applications all of them.
Title: Different Checksums
Post by: pdq on 2015-09-01 12:11:14
Also, I'm assuming that the more people verify the fingerprint, the higher the confidence rating.


As long as you have not submitted rips before, 2 or above will be practically equivalent in most cases.

Make that 1 or above, as long as you are not matching your own rip.
Title: Different Checksums
Post by: greynol on 2015-09-01 18:16:56
If the odd track is the last one, then I guess the "lead out" should be the issue ... or whatever, I have forgotten the explanations I got, but it is not abnormal that the last track stands out with lower score.

It was due to a bug* in EAC which was fixed.  I think it was addressed before ARv2.  If so then there should only be discrepancies with the v1 results.

I tried to verify when Andre said that he fixed the issue, but couldn't.  I did find this:
http://www.hydrogenaud.io/forums/index.php...st&p=670259 (http://www.hydrogenaud.io/forums/index.php?s=&showtopic=76505&view=findpost&p=670259)

(*) If you go back, you will see two separate issues.  The bug I'm mentioning here occurred when overreading was disabled (regardless of the capabilities of the drive).  This is not the same thing as when a drive incapable of overreading into the lead-out is configured as being able to overread, which is also problematic.
Title: Different Checksums
Post by: Porcus on 2015-09-02 00:35:41
The bug I'm mentioning here occurred when overreading was disabled (regardless of the capabilities of the drive).  This is not the same thing as when a drive incapable of overreading into the lead-out is configured as being able to overread, which is also problematic.


Ah! Not the same, but close enough to confuse an old memory. Thanks.
Title: Different Checksums
Post by: korndawg on 2015-09-02 22:30:31
The metadata sources are not more reliable than what you see; freedb can be populated by anyone but corrected by no-one. In particular, if you disagree with whoever submitted the release and the year on whether the year should be of the original or of the reissue in question, you will have lots of retagging to do.

Just to clarify, the metadata sources are a completely different entity than the AccurateRip fingerprints, correct? In other words, as long as the AccurateRip fingerprints match with a confidence rating of at least 2, then the ripped audio files exactly match the CD? Correcting the occasional errant batch of metadata is no big deal, but I want to ensure the rips are accurate.

Thanks again for your help!
Title: Different Checksums
Post by: Porcus on 2015-09-02 22:42:55
Just to clarify, the metadata sources are a completely different entity than the AccurateRip fingerprints, correct? In other words, as long as the AccurateRip fingerprints match with a confidence rating of at least 2, then the ripped audio files exactly match the CD?

Correct. And as pointed out by others: as long as you have not previously submitted the rip, it normally suffices to find a match to one other person's rip of the same title. (If the CD has manufactoring defects, the situation is somewhat different, but it is unknown whether anything can be done about those really.)

Notice that the term "fingerprint" is usually used for something different, namely for identifying music, not as a checksum for rips.


Title: Different Checksums
Post by: korndawg on 2015-09-05 16:37:35
Correct. And as pointed out by others: as long as you have not previously submitted the rip, it normally suffices to find a match to one other person's rip of the same title. (If the CD has manufactoring defects, the situation is somewhat different, but it is unknown whether anything can be done about those really.)

If I verify AccurateRip checksums (by either ripping a CD or explicitly verifying checksums with morituri), does that automatically submit the rip? Or do users submit rips manually?

Also, is there any way to detect a manufacturing defect?

Thanks again!
Title: Different Checksums
Post by: greynol on 2015-09-05 16:53:49
Maybe the list has expanded but submissions have historically been limited to rips* from dBpoweramp and EAC.

(*) This is not to be confused with verification of rips done after the fact.  The information submitted to the database was generated at the time of extraction.
Title: Different Checksums
Post by: Porcus on 2015-09-06 19:06:13
Also, is there any way to detect a manufacturing defect?


If a pristine disc "behaves like a scratched one", no matter what drive - that is an indication. (Use software which reports so-called C2 errors, which is where the cross-interleaved error correction lies.)
(Some copy protection schemes work by intentionally tampering with C2 errors. That is, they give you a product so fragile that ripping software will find errors all the time.)

Now if you have two CDs of the same title, and both yield the same error, then you can be fairly sure it is from the press. Even more if you rip them on two different drive; if discI and discII both yield result X on drive A and both yield result Y on drive B, you can be fairly sure.
Of course, if it is a popular title and the issue is at one spot only, then you should get a high AR score on all but that track, and possibly still an AR verification on that track, but with lower score. Here is a possible one:
(http://i.imgur.com/YNiTs3A.gif)

Disc 2, track 5. I match 89 out of hundredandfortysomething. Obviously it isn't just my CD. Trying to verify it now, seven years later, CUETools yields something that starts out like this:
Code: [Select]
[CUETools log; Date: 06.09.2015 19:41:09; Version: 2.1.6]
[CTDB TOCID: s3Ytf3x_6L3t.odNauMuBZHcwtU-] found.
Track | CTDB Status
  1   | (1044/1048) Accurately ripped
  2   | (1044/1048) Accurately ripped
  3   | (1043/1048) Accurately ripped
  4   | (1045/1048) Accurately ripped
  5   | (798/1048) Accurately ripped, or (223/1048) differs in 6 samples @03:56:27, or (6/1048) differs in 6 samples @03:56:27
  6   | (1044/1048) Accurately ripped
  7   | (1035/1048) Accurately ripped, or (6/1048) differs in 130 samples @02:34:05,02:37:14,02:40:08,02:44:35,02:45:54,04:12:41,04:15:69,05:55:31,05:56:21,05:58:32,06:02:22,06:04:01,06:07:66,06:08:23-06:08:24,06:08:72,06:09:45-06:09:46,06:10:19,06:11:08,06:11:57,06:11:73,06:13:04,06:13:36,06:13:52,06:14:42,06:14:57-06:14:58,06:15:47,06:16:04-06:16:05,06:16:21,06:16:37,06:16:53,06:17:11,06:17:43,06:18:16,06:18:32,06:18:48-06:18:49,06:21:33-06:21:34,06:23:29,06:25:41
  8   | (1042/1048) Accurately ripped
  9   | (1043/1048) Accurately ripped
10   | (1040/1048) Accurately ripped
11   | (1041/1048) Accurately ripped
12   | (1030/1048) Accurately ripped, or (6/1048) differs in 1 samples @01:31:72
13   | (1021/1048) Accurately ripped
[AccurateRip ID: 001e606b-013a2eac-aa10d30d] found.
Track   [  CRC   |   V2   ] Status
01     [8315cfaf|ac0b4c26] (200+200/1184) Accurately ripped
02     [47867f94|f61a68cb] (200+200/1187) Accurately ripped
03     [5226ca61|d4b65351] (200+200/1183) Accurately ripped
04     [4719e9bc|494184a0] (200+200/1179) Accurately ripped
05     [6151788d|9b46414f] (200+200/1511) Accurately ripped


(AR reports anything > 200 as 200.) I can only guess that one of the pressings got some bits wrong. Notice this track has been submitted about 28 percent more times. (Which by itself only proves that it has been ripped more times, but - ruling out people inserting this disc because they only want this track - shows that users find their rips suspicious.)
Title: Different Checksums
Post by: greynol on 2015-09-06 19:17:12
Have you used CTDB to "correct" the track and then checked what the differences were?
Title: Different Checksums
Post by: Porcus on 2015-09-06 19:36:56
Have you used CTDB to "correct" the track and then checked what the differences were?


Tried the "repair" right now.  The 137-samples-different and the 6-samples-different rip agree on this particular track. Difference to mine:

Differences found: 6 values, starting at 3:56.361837, peak: 0.0031738 at 3:56.361859, 1ch
Detected offset as 0 samples.

What can one infer from this? -25 dB, is that what one could expect to get by interpolating? (Hm ... maybe I should open a wave editor?)



BTW, the other differences to the 137 are about the same.

Track 7:
Differences found: 130 values, starting at 2:34.078639, peak: 0.0232849 at 2:37.196054, 1ch
Detected offset as 0 samples.

Track 12:
Differences found: 1 value, starting at 1:31.961043, peak: 0.0046997 at 1:31.961043, 2ch
Detected offset as 0 samples.
Title: Different Checksums
Post by: greynol on 2015-09-06 19:42:17
Yes, with a wave editor.  I should have specified earlier, sorry.
Title: Different Checksums
Post by: Porcus on 2015-09-06 23:16:18
At least for the first sample where they disagree, then visually it looks like my (majority) version is interpolated as simple average while the other is slightly more sinusoidal. Files, 1024 samples each: https://www.sendspace.com/file/hpefg7 (https://www.sendspace.com/file/hpefg7)

1024 because that is what ffmpeg returns; I tried to export the first twelve (per channel) samples with Audacity - then all for sudden fb2k reports sixteen different samples, not six. Is this normal?
Title: Different Checksums
Post by: lvqcl on 2015-09-06 23:41:36
Disable dithering in Audacity.

It would be interesting to look at the spectrograms, but it requires several thousand of samples before and after this place.
Title: Different Checksums
Post by: Porcus on 2015-09-07 07:48:37
Disable dithering in Audacity.


*facepalm*  thx ...

It would be interesting to look at the spectrograms, but it requires several thousand of samples before and after this place.


Full second: https://www.sendspace.com/file/p5nod7 (https://www.sendspace.com/file/p5nod7)
Title: Different Checksums
Post by: lvqcl on 2015-09-07 16:49:13
So it's possible to see some arefact in 05TGGITS_000.wav_3_56_to_57.wav, but not in 05TGGITS_137.wav_3_56_to_57.wav.

Spectrograms can be created by SoX:
Code: [Select]
sox infile.wav --null spectrogram -w Kaiser -o outfile.png
Title: Different Checksums
Post by: Porcus on 2015-09-07 20:23:44
So it's possible to see some arefact in 05TGGITS_000.wav_3_56_to_57.wav, but not in 05TGGITS_137.wav_3_56_to_57.wav.


Yep, it was that clear. The 000 is my physical CD, and the majority vote in AR/CTDB (double checked now to be sure).
I tried the last difference on the CD, and it was the other way around. Could be that the "good" one was the secondmost in CTDB (its track 5 is bit-identical to the "137").

I found the (same physical) disc re-ripped with CUETools/EAC/dBpoweramp. No sign of any errors or interpolation. So the likely explanation is ... what? An error in the glass master should have triggered C1/C2? So have they burnt it to a CD and sent it off to a pressing plant who ripped in burst mode before making the glass master?


(Mod, should this branch be split off the thread?)
Title: Different Checksums
Post by: Porcus on 2015-09-07 22:12:48
I tried the last difference on the CD

Well the remaining differences were in one track, so:

* In this one, I see no spike artifacts:
Code: [Select]
Track | CTDB Status
  1  | (1049/1053) Accurately ripped
  2  | (1049/1053) Accurately ripped
  3  | (1048/1053) Accurately ripped
  4  | (1050/1053) Accurately ripped
  5  | (245/1053) Accurately ripped, or (778/1053) differs in 6 samples @03:56:27
  6  | (1049/1053) Accurately ripped
  7  | (1040/1053) Accurately ripped, or (6/1053) differs in 130 samples @02:34:05,02:37:14,02:40:08,02:44:35,02:45:54,04:12:41,04:15:69,05:55:31,05:56:21,05:58:32,06:02:22,
06:04:01,06:07:66,06:08:23-06:08:24,06:08:72,06:09:45-06:09:46,06:10:19,06:11:08,06:11:57,06:11:73,06:13:04,06:13:36,06:13:52,06:14:42,06:14:57-06:14:58,06:15:47,06:16:04-06:16:05,06:16:21,06:16:37,06:16:53,06:17:11,06:17:43,06:18:16,06:18:32,06:18:48-06:18:49,06:21:33-06:21:34,06:23:29,06:25:41
  8  | (1047/1053) Accurately ripped
  9  | (1048/1053) Accurately ripped
 10  | (1045/1053) Accurately ripped
 11  | (1046/1053) Accurately ripped
 12  | (1035/1053) Accurately ripped, or (6/1053) differs in 1 samples @01:31:72
 13  | (1024/1053) Accurately ripped
[AccurateRip ID: 001e606b-013a2eac-aa10d30d] found.
Track  [  CRC  |  V2  ] Status
 01    [8315cfaf|ac0b4c26] (200+200/1184) Accurately ripped
 02    [47867f94|f61a68cb] (200+200/1187) Accurately ripped
 03    [5226ca61|d4b65351] (200+200/1183) Accurately ripped
 04    [4719e9bc|494184a0] (200+200/1179) Accurately ripped
 05    [03bfbf7a|3db47f89] (179+158/1511) Accurately ripped
 06    [5580a89d|49310d1e] (200+200/1171) Accurately ripped
 07    [2e61b880|a5386ae5] (200+200/1172) Accurately ripped
 08    [0f03e4f2|1cd0d62f] (200+200/1181) Accurately ripped
 09    [4e088bd3|58934bac] (200+200/1178) Accurately ripped
 10    [2e348ea5|8c0a3aaf] (200+200/1172) Accurately ripped
 11    [d42de47d|8be60f10] (200+200/1171) Accurately ripped
 12    [31b0304b|f37e78bc] (200+200/1164) Accurately ripped
 13    [d72532e1|9cc8a770] (200+200/1136) Accurately ripped'

[cutting]


Track Peak [ CRC32  ] [W/O NULL]
 --  100,0 [35879A55] [C6CDF1FA]         
 01  56,4 [BA732B5A] [0465C5C3]         
 02  84,0 [D26B081B] [8750F6F4]         
 03  85,3 [8D8B617D] [89413F20]         
 04  98,3 [83B90991] [8D073175]         
 05  85,7 [5E80E84B] [DEB9DD79]         
 06  100,0 [846CD5B6] [2B2FE479]         
 07  89,0 [52A1D786] [FD5A6455]         
 08  84,8 [1F5FE3F8] [006AE727]         
 09  87,4 [9B1BCFA9] [BB732C66]         
 10  87,8 [44265A02] [6B5B48BB]         
 11  81,4 [9088CB85] [49C1417D]         
 12  90,5 [5F8A20E5] [324EF438]         
 13  100,0 [42FE8B0F] [9C1039E0]   


* Mine is the "778" which differs in six samples in track 5. That is the one I posted, where lvqcl points out artifacts.
* The last one agrees with the top one on track 5, but has 131 samples disagreeing elsewhere: 130 in track 7 (visible spikes) and 1 in track 12 (ditto).

Could a CDTB score of 6/1053 be, uh, polluted by downloaders?
Title: Different Checksums
Post by: greynol on 2015-09-07 22:35:15
I think Spoon would be able to tell you the make and model of the drive used for each submission in the AR database, which might help to clear things up.
Title: Different Checksums
Post by: spoon on 2015-09-08 00:37:59
I do not easily have that info to hand, drives do not make it into the final database.
Title: Different Checksums
Post by: greynol on 2015-09-08 00:58:39
Oh well.

So that Porcus knows where I was going with this, I wouldn't rule out the possibility that the data on the CD didn't exploit a weakness/bug with a specific drive, ripping program (possibly configuration related) or combination of both.  I've raised issues in the past about the possibility of the AR database becoming populated with downloads that were in error, though that would land dead last on my list of possible reasons.
Title: Different Checksums
Post by: Porcus on 2015-09-08 10:42:56
I wouldn't rule out the possibility that the data on the CD didn't exploit a weakness/bug with a specific drive, ripping program (possibly configuration related) or combination of both.


I agree with the general statement, and one can still not rule it out coumpletely after having tried quite a few combinations ['] though it appears less likely:
Back then I was armed with three drives and dBpoweramp (Reference) with EAC as backup. Everything would be ripped with dBpoweramp from a Sony VGP-XL1B2 carousel with a Matshita drive that dBpoweramp could unfortunately not get C2 information from over the firewire; oddballs would go into the laptop's HL-DT-ST drive (with C2) and the stubborn ones into a PX-230A. At this stage, EAC would also have been invoked.


CUETools was not an option in 2008, therefore I tried it yesterday - on two drives, the above HL-DT-ST and a new one (TSSTCorp). Also I tried dBpoweramp disconnected from the 'net (I might not have thought of back then that this way I could force it into secure mode) and for the hell of it, EAC on the old HL-DT-ST.

There is no sign that any of the applications had any suspicion whatsoever. (I do not know how to get CUETools to report C2 errors, but I sat staring at a burst to see whether it triggered any retry.)


['] Disclaimer: I do not have the logs for more than the initial rip (of the whole CD) as I never managed to produce a difference - but I did notice this curious result, so I must have invoked something like my usual workflow. I could possibly even have tried a fourth drive, which did generally suck and was usually not involved when I tried to get reasonable results out of troublemaking discs. But this time the issue was to provoke an error.
Title: Different Checksums
Post by: greynol on 2015-09-08 12:34:35
Sure, but who's to say there would have to be some indication of a problem?  I would expect it to happen without any obvious sign of trouble other than the AR numbers.  I also wouldn't assume there was any problem with the disc itself.

It could also simply be that there's a different pressing created with errant PCM data (EFM and CIRC are perfectly OK) which has no offset relative to the other version.

Finally, without comparing the two versions and identifying interpolatation, time shifts, bit-flips, dropped half samples and what ever other possible type of error, how is anyone to know which version is correct? Majority rule?  I think that's for suckers (pointing to EAC's old bug when over reading is disabled).
Title: Different Checksums
Post by: Porcus on 2015-09-08 15:45:21
I don't completely follow what you refer to at each statement ... :


Sure, but who's to say there would have to be some indication of a problem?  I would expect it to happen without any obvious sign of trouble other than the AR numbers.


Not sure what you mean.  If you mean that it must be expected from time to time irrespective of whether one has AR figures to spot it then ... well, obviously AR does not define what is on the master, so I suppose you had something else in mind?


I also wouldn't assume there was any problem with the disc itself.


Since I get no error information from the rippers? Or, generally? If given only the information that two rips of the same title differ in only a handful of samples, then I would indeed make the first guess that there is a problem with (at least) one of them, so again I suppose you had something else in mind?


It could also simply be that there's a different pressing created with errant PCM data (EFM and CIRC are perfectly OK) which has no offset relative to the other version.


That is what I put my two cents on in this case, but then: where does it originate?



Finally, without comparing the two versions and identifying interpolatation, time shifts, bit-flips, dropped half samples and what ever other possible type of error, how is anyone to know which version is correct?


Closing in on "if there is only one version, ...?"  But in this case we have two.


Majority rule?


If the spectrogram spikes are to be read as indication of error (whatever "error" means), then the majority is "wrong" on this particular title.
Title: Different Checksums
Post by: greynol on 2015-09-08 16:04:57
Sure, but who's to say there would have to be some indication of a problem?  I would expect it to happen without any obvious sign of trouble other than the AR numbers.

Not sure what you mean.  If you mean that it must be expected from time to time irrespective of whether one has AR figures to spot it then ... well, obviously AR does not define what is on the master, so I suppose you had something else in mind?

You mentioned ripping multiple ways in hopes of getting some kind of alert from the ripping program about a problem.  I'm telling you it might be that two different people with the same supposed pressing are getting consistently different results on their respective systems without any such warnings.

I also wouldn't assume there was any problem with the disc itself.

Since I get no error information from the rippers? Or, generally? If given only the information that two rips of the same title differ in only a handful of samples, then I would indeed make the first guess that there is a problem with (at least) one of them, so again I suppose you had something else in mind?

I'm simply trying to show you that there are many other scenarios that you haven't yet entertained.

It could also simply be that there's a different pressing created with errant PCM data (EFM and CIRC are perfectly OK) which has no offset relative to the other version.

That is what I put my two cents on in this case, but then: where does it originate?

With all the different possibilities I dished out (which can't possibly be an exhaustive list), why would you think I know the answer?

If the spectrogram spikes are to be read as indication of error (whatever "error" means), then the majority is "wrong" on this particular title.

Sure, but don't always expect to see spikes in a spectrogram.
Title: Different Checksums
Post by: korndawg on 2015-09-08 18:43:14
I have a few more questions after reading the last several posts, all of which were fascinating!

1) Will different pressings of the same CD generate the same AccurateRip checksum? If so, but one of the pressings contains a manufacturing defect within a specific track, will there be a second entry for that track within the AccurateRip database in order to account for the manufacturing defect?

3) While working with cdparanoia in the past I have noticed that some troublesome tracks (perhaps like "The Great Gig In The Sky") generate "jitter" errors and displays "-" signs within the cdparanoia progress bar. However, ripping the troublesome track using a different range (for example ripping the track separately with "cdparanoia --batch 5" instead of "cdparanoia --batch 1-13" or "cdparanoia --batch") eliminates the "jiter" errors and "-"signs within the progress bar and seems to produce a "pristine" file. Supposedly cdparanoia corrects jitter errors (not to mention streaming errors represented by "+" signs within the progress bar), but the corrected file always generates a different file checksum (using sha1sum) than the pristine file. Would the AccurateRip checksums of these files match? Does morituri account for cdparanoia "jitter" errors such as this?

2) Will directly editing FLAC files within Audacity maintain audio quality? Should I disable dithering as well? Typically I only fire up Audacity in order to fade in/out live tracks.

Thanks again for your help!

Title: Different Checksums
Post by: pdq on 2015-09-08 18:57:55
2) Will directly editing FLAC files within Audacity maintain audio quality? Should I disable dithering as well? Typically I only fire up Audacity in order to fade in/out live tracks.

You are not directly editing the FLAC file. Audacity first extracts the PCM data and operates on that. The final audio quality is a function of what you have Audacity do to the data.

If most of the track is not changed then I would disable dithering to avoid noise being added unnecessarily.
Title: Different Checksums
Post by: greynol on 2015-09-08 19:09:17
I have a few more questions after reading the last several posts, all of which were fascinating!

I suggest you read some posts on the same subject in the CD Hardware/Software forum, as little here hasn't been discussed before.

Will different pressings of the same CD generate the same AccurateRip checksum?

Generally speaking, no.  Different pressings usually differ by at least an offset.  Different pressings can also have slightly different track timings, though AccurateRip will treat these as entirely different discs.

If so, but one of the pressings contains a manufacturing defect within a specific track, will there be a second entry for that track within the AccurateRip database in order to account for the manufacturing defect?

When multiple checksums for a track are available then each one will have been submitted by more than one unique user ID.  If a manufacturing defect results in different but consistent checksums depending on the drive and/or ripping algorithm then it is likely that multiple checksums will appear in the database, provided the title is popular enough.

Supposedly cdparanoia corrects jitter errors (not to mention streaming errors represented by "+" signs within the progress bar), but the corrected file always generates a different file checksum (using sha1sum) than the pristine file. Would the AccurateRip checksums of these files match?

It should be assumed that tracks with different sha1 checksums will also have different AccurateRip checksums.  AR ignores some data at the beginning of the first track on the disc and at the end of the last track on the disc so that these tracks can still be checked with drives that have different offsets which cannot overread; otherwise they might generate different checksums.

Will directly editing FLAC files within Audacity maintain audio quality? Should I disable dithering as well? Typically I only fire up Audacity in order to fade in/out live tracks.

Please start a new thread for these questions as they are no longer on-topic.
Title: Different Checksums
Post by: greynol on 2015-09-08 19:13:30
2) Will directly editing FLAC files within Audacity maintain audio quality? Should I disable dithering as well? Typically I only fire up Audacity in order to fade in/out live tracks.

You are not directly editing the FLAC file. Audacity first extracts the PCM data and operates on that. The final audio quality is a function of what you have Audacity do to the data.

If most of the track is not changed then I would disable dithering to avoid noise being added unnecessarily.

Yeah and without performing any operations, Audacity will also change the entire contents of a file if you load it and then save it when using the default settings.

Again, this really needs its own topic (not that it hasn't also been discussed on multiple occasions).