HydrogenAudio

CD-R and Audio Hardware => CD Hardware/Software => Topic started by: JoyO on 2007-02-16 18:11:21

Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-16 18:11:21
Is there any software capable of checking CRC's of lossless data (mostly FLAC, but would be nice if wv would be supported also) with the checksums contained in EAC log?
Title: Software to verify EAC log CRC's
Post by: Fandango on 2007-02-16 18:22:34
I've been wondering about this for a while, too.

A simple command line tool that accepts stdin WAV data would be enough for my needs.
Title: Software to verify EAC log CRC's
Post by: Remedial Sound on 2007-02-16 18:32:33
Gl.tter's CheckWavpackFiles app does exactly what it says, though I believe it uses MD5.  Check out this (http://www.hydrogenaudio.org/forums/index.php?showtopic=51434) thread.  I don't use flac so I'm not sure if a similar tool exists for it.

AFAIK the CRCs in EAC log files are for the ripped wav (pre-compression), so the lossless file would have to first be decoded before comparing CRCs.
Title: Software to verify EAC log CRC's
Post by: Eli on 2007-02-16 18:50:50
Spoon will soon have a tool to check crc accuraterip values against the file and the accurterip db for files ripped to lossless formats w/ dbpoweramp. I know it doesnt help w/ EAC but I am not aware of a solution there.
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-17 09:02:19
So far I found only this tool:
http://www.cdtag.com/ (http://www.cdtag.com/)
However, there is some bug in this function so it doesnt work.

I tried also doing such a compare manually:
-decoding flac to wav
-calculating CRC value of wav
-comparing it with CRC sorted in log file
and they all the time were different! Why is that? How exactly EAC CRC is calculated?
Title: Software to verify EAC log CRC's
Post by: ak on 2007-02-17 13:21:34
-decoding flac to wav

You definetly should decode to raw pcm or cut wave header. That's how they usually calculate checksums for audio.
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-17 14:19:23
Thanks ak, now it worked.

Now I will play a little, maybe it will be possible to make a simple batch file which would decode each flac file to stdout, calculate crc and save to txt file.
Comparing could be done manually, not a big problem. However, parsing log file shouldnt be hard to do as well.
Title: Software to verify EAC log CRC's
Post by: Fandango on 2007-02-17 14:38:59

-decoding flac to wav

You definetly should decode to raw pcm or cut wave header. That's how they usually calculate checksums for audio.

Uh, yeah.  It works with every tool that calculates CRC32... thanks, that's all I need to know.

So for decoding a FLAC to RAW PCM one can simply use this "flac -d --force-raw-format --endian=little --sign=signed <infile.flac> <outfile.raw>" and then a tool like fsum to get the CRC. It should be the same as the one in EAC's logs, unless EAC uses some sort of derivate of CRC32 like eDonkey2000 did with MD4.

Only problem now is to optimise checking the CRCs with as little temp files as possible, ideally none and using pipes instead. And of course accepting all major lossless input formats.
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-17 14:45:18
To optimise using -c option in flac would be the best, but it looks like fsum cant calc crc from stdin. Will have to find some other tool.

So it would be like:
flac.exe -d -c --force-raw-format --endian=little --sign=signed <file.flac> | sum.exe > crc.txt


---
found unix tool cksum, but it outputs crc in decimal format, what a hell :>
Title: Software to verify EAC log CRC's
Post by: [proxima] on 2007-02-17 15:22:27
Triza already implemented the same CRC calculation of EAC to check files against the log.
You can pipe the wav file into standard input and get the EAC CRC at the standard output. See here (http://www.hydrogenaudio.org/forums/index.php?s=&showtopic=14998&view=findpost&p=167167).
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-17 15:38:39
The same can be easy done with flac returning raw data to stdio (already accomplished). We just need some crc calculation program returning data in hex and being able to operate on stdin.

Anyway, can someone compile it? I have only borland free command line tools
Title: Software to verify EAC log CRC's
Post by: Fandango on 2007-02-17 16:04:58
Oh wait, Triza's post mentions the fact that EAC knows two ways to calculate the CRC, it's either based on audio with silence or without silence.

I think a proper tool for checking the CRCs should take this into account. Luckily he already did most of the work for us. All you have to do is change his code, for instance so that both CRCs are written to stdout or implement a command line switch to choose which CRC to use.
Title: Software to verify EAC log CRC's
Post by: kez on 2007-02-17 16:26:05
This may be useful -> http://www.kvipu.com/CDCheck/download.php (http://www.kvipu.com/CDCheck/download.php), look in Options>When processing wav files ignore...
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-17 17:02:10
Tried to port it for borland:
Code: [Select]
#include <stdio.h>
#include <io.h>
#include <assert.h>


#define BUFFERSIZE 65536  /* (64k) buffer size for reading from the file */

static const unsigned long crctable[256] = {
 0x00000000, 0x77073096, 0xee0e612c, 0x990951ba,
 0x076dc419, 0x706af48f, 0xe963a535, 0x9e6495a3,
 0x0edb8832, 0x79dcb8a4, 0xe0d5e91e, 0x97d2d988,
 0x09b64c2b, 0x7eb17cbd, 0xe7b82d07, 0x90bf1d91,
 0x1db71064, 0x6ab020f2, 0xf3b97148, 0x84be41de,
 0x1adad47d, 0x6ddde4eb, 0xf4d4b551, 0x83d385c7,
 0x136c9856, 0x646ba8c0, 0xfd62f97a, 0x8a65c9ec,
 0x14015c4f, 0x63066cd9, 0xfa0f3d63, 0x8d080df5,
 0x3b6e20c8, 0x4c69105e, 0xd56041e4, 0xa2677172,
 0x3c03e4d1, 0x4b04d447, 0xd20d85fd, 0xa50ab56b,
 0x35b5a8fa, 0x42b2986c, 0xdbbbc9d6, 0xacbcf940,
 0x32d86ce3, 0x45df5c75, 0xdcd60dcf, 0xabd13d59,
 0x26d930ac, 0x51de003a, 0xc8d75180, 0xbfd06116,
 0x21b4f4b5, 0x56b3c423, 0xcfba9599, 0xb8bda50f,
 0x2802b89e, 0x5f058808, 0xc60cd9b2, 0xb10be924,
 0x2f6f7c87, 0x58684c11, 0xc1611dab, 0xb6662d3d,
 0x76dc4190, 0x01db7106, 0x98d220bc, 0xefd5102a,
 0x71b18589, 0x06b6b51f, 0x9fbfe4a5, 0xe8b8d433,
 0x7807c9a2, 0x0f00f934, 0x9609a88e, 0xe10e9818,
 0x7f6a0dbb, 0x086d3d2d, 0x91646c97, 0xe6635c01,
 0x6b6b51f4, 0x1c6c6162, 0x856530d8, 0xf262004e,
 0x6c0695ed, 0x1b01a57b, 0x8208f4c1, 0xf50fc457,
 0x65b0d9c6, 0x12b7e950, 0x8bbeb8ea, 0xfcb9887c,
 0x62dd1ddf, 0x15da2d49, 0x8cd37cf3, 0xfbd44c65,
 0x4db26158, 0x3ab551ce, 0xa3bc0074, 0xd4bb30e2,
 0x4adfa541, 0x3dd895d7, 0xa4d1c46d, 0xd3d6f4fb,
 0x4369e96a, 0x346ed9fc, 0xad678846, 0xda60b8d0,
 0x44042d73, 0x33031de5, 0xaa0a4c5f, 0xdd0d7cc9,
 0x5005713c, 0x270241aa, 0xbe0b1010, 0xc90c2086,
 0x5768b525, 0x206f85b3, 0xb966d409, 0xce61e49f,
 0x5edef90e, 0x29d9c998, 0xb0d09822, 0xc7d7a8b4,
 0x59b33d17, 0x2eb40d81, 0xb7bd5c3b, 0xc0ba6cad,
 0xedb88320, 0x9abfb3b6, 0x03b6e20c, 0x74b1d29a,
 0xead54739, 0x9dd277af, 0x04db2615, 0x73dc1683,
 0xe3630b12, 0x94643b84, 0x0d6d6a3e, 0x7a6a5aa8,
 0xe40ecf0b, 0x9309ff9d, 0x0a00ae27, 0x7d079eb1,
 0xf00f9344, 0x8708a3d2, 0x1e01f268, 0x6906c2fe,
 0xf762575d, 0x806567cb, 0x196c3671, 0x6e6b06e7,
 0xfed41b76, 0x89d32be0, 0x10da7a5a, 0x67dd4acc,
 0xf9b9df6f, 0x8ebeeff9, 0x17b7be43, 0x60b08ed5,
 0xd6d6a3e8, 0xa1d1937e, 0x38d8c2c4, 0x4fdff252,
 0xd1bb67f1, 0xa6bc5767, 0x3fb506dd, 0x48b2364b,
 0xd80d2bda, 0xaf0a1b4c, 0x36034af6, 0x41047a60,
 0xdf60efc3, 0xa867df55, 0x316e8eef, 0x4669be79,
 0xcb61b38c, 0xbc66831a, 0x256fd2a0, 0x5268e236,
 0xcc0c7795, 0xbb0b4703, 0x220216b9, 0x5505262f,
 0xc5ba3bbe, 0xb2bd0b28, 0x2bb45a92, 0x5cb36a04,
 0xc2d7ffa7, 0xb5d0cf31, 0x2cd99e8b, 0x5bdeae1d,
 0x9b64c2b0, 0xec63f226, 0x756aa39c, 0x026d930a,
 0x9c0906a9, 0xeb0e363f, 0x72076785, 0x05005713,
 0x95bf4a82, 0xe2b87a14, 0x7bb12bae, 0x0cb61b38,
 0x92d28e9b, 0xe5d5be0d, 0x7cdcefb7, 0x0bdbdf21,
 0x86d3d2d4, 0xf1d4e242, 0x68ddb3f8, 0x1fda836e,
 0x81be16cd, 0xf6b9265b, 0x6fb077e1, 0x18b74777,
 0x88085ae6, 0xff0f6a70, 0x66063bca, 0x11010b5c,
 0x8f659eff, 0xf862ae69, 0x616bffd3, 0x166ccf45,
 0xa00ae278, 0xd70dd2ee, 0x4e048354, 0x3903b3c2,
 0xa7672661, 0xd06016f7, 0x4969474d, 0x3e6e77db,
 0xaed16a4a, 0xd9d65adc, 0x40df0b66, 0x37d83bf0,
 0xa9bcae53, 0xdebb9ec5, 0x47b2cf7f, 0x30b5ffe9,
 0xbdbdf21c, 0xcabac28a, 0x53b39330, 0x24b4a3a6,
 0xbad03605, 0xcdd70693, 0x54de5729, 0x23d967bf,
 0xb3667a2e, 0xc4614ab8, 0x5d681b02, 0x2a6f2b94,
 0xb40bbe37, 0xc30c8ea1, 0x5a05df1b, 0x2d02ef8d,
};


void crc32(register int fd, unsigned long *main_val)
{
 char   buf[BUFFERSIZE], *p;
 int   len = 0, nr;
 unsigned long crc = ~0, crc32_total = ~0;

 while ((nr = read(fd, buf, sizeof(buf))) > 0){
  assert(nr % 2 == 0);
  for (len += nr, p = buf; nr--; ++p) {
// Skip silence when calculate CRC. If EAC is set so that it
// does not skip silence when CRC is calculated, remove this if
// statement, or change its condition to non-zero, so it is always
// true.
if(*((__int16*)p) != 0){
  crc = (crc >> 8) ^ crctable[(crc ^ *p) & 0xff];
  crc32_total = (crc >> 8) ^ crctable[(crc32_total ^ *p) & 0xff];

  nr--;
  ++p;

  crc = (crc >> 8) ^ crctable[(crc ^ *p) & 0xff];
  crc32_total = (crc >> 8) ^ crctable[(crc32_total ^ *p) & 0xff];
}
else{
  nr--;
  ++p;
}
  }
 }

 assert(nr >= 0);

 *main_val = ~crc;
}


int main()
  {
  unsigned long Crc;

  // Skip the WAV header.
  const int Ret = lseek(fileno(stdin), 44, SEEK_SET);
  assert(Ret >= 0);

  crc32(fileno(stdin), &Crc);

  printf("%0X\n", Crc);
  }

Compiles, but gives wrong crc when used like that:
flac.exe -d -c --totally-silent 1.flac | crc.exe

Thats probably because I suck at c programming  Just googled the errors and changed a little bit code. I have no idea how to do it in Cygwin, and now I dont really want to mess with it

[!--sizeo:1--][span style=\"font-size:8pt;line-height:100%\"][!--/sizeo--]moderation: CODE => CODEBOX[/size]
Title: Software to verify EAC log CRC's
Post by: Fandango on 2007-02-17 18:18:10
Compiles, but gives wrong crc when used like that


Code: [Select]
     // Skip silence when calculate CRC. If EAC is set so that it
     // does not skip silence when CRC is calculated, remove this if
     // statement, or change its condition to non-zero, so it is always
     // true.


As I said in my previous post, EAC knows two ways of calculating the CRC and unfortunately it's not mentioned in the EAC log which one was used, so you have to try both ways when you're not sure. That's what I was talking about in my previous post... so try this:


Code: [Select]
while ((nr = read(fd, buf, sizeof(buf))) > 0){
   assert(nr % 2 == 0);
   for (len += nr, p = buf; nr--; ++p) {
     crc = (crc >> 8) ^ crctable[(crc ^ *p) & 0xff];
     crc32_total = (crc >> 8) ^ crctable[(crc32_total ^ *p) & 0xff];

     nr--;
     ++p;

     crc = (crc >> 8) ^ crctable[(crc ^ *p) & 0xff];
     crc32_total = (crc >> 8) ^ crctable[(crc32_total ^ *p) & 0xff];
   }
}
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-17 18:26:48
Yeah, I am aware of that, been doing this by changing 0 to other digit in if statement. Same result as your code (tested to be 100% sure).
This program gives 4969EC84 or 23DCDBC3 (depends on that silence), while it should return 8875EB97 (value in log as well as when calculating crc of: flac.exe -d -c -f --totally-silent --force-raw-format --endian=little --sign=signed 1.flac >out.raw )
Title: Software to verify EAC log CRC's
Post by: Borisz on 2007-02-17 20:12:00
I always thought an Accuraterip lookup plugin for Foobar2000, similar to its freedb plugin, would be an absolute godsend.
Title: Software to verify EAC log CRC's
Post by: tempnegro on 2007-02-17 20:36:03
If someone tells me how to attach something I have a EAC log checker (including bad read mode) not made by me but a friend in xls

its an excel spreadsheet that I soon hope one day will be coded into C# and released as a standalone program that can evetually check the logs, verify CRC's, make sure its not a transcode etc etc.
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-17 21:08:11
Would be nice to see it
You can upload it here (http://www.speedyshare.com/).
Title: Software to verify EAC log CRC's
Post by: sn0wman on 2007-02-17 21:34:55
hmm good idea, i will definately make my app able to verify EAC logs too !!!
http://www.hydrogenaudio.org/forums/index....showtopic=43463 (http://www.hydrogenaudio.org/forums/index.php?showtopic=43463)
Title: Software to verify EAC log CRC's
Post by: tempnegro on 2007-02-17 21:43:37
http://www.speedyshare.com/690884837.html (http://www.speedyshare.com/690884837.html)
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-17 23:17:19
Finally found proper tool
http://www.mirkes.de/en/freeware/batch.php (http://www.mirkes.de/en/freeware/batch.php)

Code: [Select]
flac.exe -d -c -f --totally-silent --force-raw-format --endian=little --sign=signed 1.flac | crc1632 /m32


Doing batch tomorrow
Title: Software to verify EAC log CRC's
Post by: [proxima] on 2007-02-18 01:07:17
Thats probably because I suck at c programming  Just googled the errors and changed a little bit code. I have no idea how to do it in Cygwin, and now I dont really want to mess with it
moderation: CODE => CODEBOX

I tried about two years ago to compile it with MinGW but the binary provide a different CRC.
Cygwin compiled binaries (cygwin1.dll needed) work perfect here.
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-18 08:32:16
How do you compile it?

"gcc -o crc.exe crc.c"  with cygwin1.dll gives error when using pipe
flac.exe -d -c --totally-silent 1.flac | crc.exe
error:
Code: [Select]
assertion "Ret >= 0" failed: file "crc.c", line 120
   4530 [sig] a 3868 open_stackdumpfile: Dumping stack trace to a.exe.stackdump
  27749 [sig] a 3868 E:\TEST\a.exe: *** fatal error - called with threadlist_ix -1


and returns bad checksum when used like that:
crc.exe<1.wav

When changed source for win and compiled with "gcc -mno-cygwin crc3.c" I get the same results as with borland which is wrong :/
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-20 19:30:49
Here is temporary solution (works for most logs):

Code: [Select]
@echo off

rem crc1632 - [url=http://www.mirkes.de/en/freeware/batch.php]http://www.mirkes.de/en/freeware/batch.php[/url]
set flac="E:\TEST\flac.exe"
set crc1632="E:\TEST\crc1632.exe"

rem takes first parameter as path to log and flac files
cd "%1"
%~d1

for %%f in (*.flac) do (
echo %%f
%flac% -d -c -f --totally-silent --force-raw-format --endian=little --sign=signed "%%f" | %crc1632% /m32
)

IF errorlevel=1 GOTO fail

for %%f in (*.log) do (
rem opens .log in notepad, to open in default app for .log files chage next line to: "%%f"
notepad.exe "%%f"
)

IF errorlevel=1 GOTO fail
GOTO end
:fail
echo.
echo Sth is wrong.
echo.
pause
:end
rem uncomment next line to prevent cmd window from closing
rem pause

Save as .bat and use in foo_run, total commander, windows explorer or any other tool
Could be 100% effective if I would know some way to compile Triza code. Maybe I will make own tool for this in c, for now this must be enough

Edit: small fix - added %~d1 to change drive.
Title: Software to verify EAC log CRC's
Post by: Alexxander on 2007-02-21 09:12:05
Here is temporary solution (works for most logs):

Code: [Select]
@echo off

rem crc1632 - [url=http://www.mirkes.de/en/freeware/batch.php]http://www.mirkes.de/en/freeware/batch.php[/url]
set flac="E:\TEST\flac.exe"
set crc1632="E:\TEST\crc1632.exe"

rem takes first parameter as path to log and flac files
cd "%1"
%~d1

for %%f in (*.flac) do (
echo %%f
%flac% -d -c -f --totally-silent --force-raw-format --endian=little --sign=signed "%%f" | %crc1632% /m32
)

IF errorlevel=1 GOTO fail

for %%f in (*.log) do (
rem opens .log in notepad, to open in default app for .log files chage next line to: "%%f"
notepad.exe "%%f"
)

IF errorlevel=1 GOTO fail
GOTO end
:fail
echo.
echo Sth is wrong.
echo.
pause
:end
rem uncomment next line to prevent cmd window from closing
rem pause

Save as .bat and use in foo_run, total commander, windows explorer or any other tool
Could be 100% effective if I would know some way to compile Triza code. Maybe I will make own tool for this in c, for now this must be enough

Edit: small fix - added %~d1 to change drive.

Nice script. I tried it on flac ripped albums and it calculates nicely CRC's but the results aren't identical to the CRC's in the EAC logfile. I suppose the CRC's are different because in EAC I rip with the option "No use of null samples for CRC calculations" marked (EAC Options >> tab Extraction).
Title: Software to verify EAC log CRC's
Post by: JoyO on 2007-02-21 13:57:14
This issue I think could be solved quiet fast and easy. We need to add another pipe before passing stream to crc1632, like
-sign=signed "%%f" | CLEAN.EXE | %crc1632% /m32
where CLEAN.EXE would be simple program that reads data from stdin, removes 0000 (not sure here as I dont have any flac file created with checked eac "No use of null samples for CRC calculations" option.) and writes to stdout. This should be a few lines of code. However, I need some some flac file to test it.
Title: Software to verify EAC log CRC's
Post by: greynol on 2007-02-21 18:01:35
Assuming 0000 means left-channel sample and right-channel sample (LLRR), your cleaning concept isn't going to duplicate EAC's method of CRC generation.

Your method must also change 00XX or XX00 to XX.


Never mind.  You're correct, 0x0000 represents a 16-bit null sample in a single channel.
Title: Software to verify EAC log CRC's
Post by: dmvn on 2008-02-19 18:36:26
Wow! I see people try to solve the problem that was solved months ago...
Please have a look here... I think this helps, no?
http://www.hydrogenaudio.org/forums/index....showtopic=57306 (http://www.hydrogenaudio.org/forums/index.php?showtopic=57306)

greynol asked me to provide a kind of console tool that can easily calculate all variants of CRC, & i think i can help, i know all the truth and already have (maybe some fixes will be necessary to support stdin) some C code...

PS very sorry for my english.