HydrogenAudio

Lossless Audio Compression => Lossless / Other Codecs => Topic started by: Drago on 2003-11-26 23:12:24

Title: .ape files corruption
Post by: Drago on 2003-11-26 23:12:24
hi! i've readed this forum for a long time and finally i've decide to register. i'm sorry for my english, i'm italian and i'm doing my best

i have a question: now i'm switching to .flac format because a lot of .ape files generated by my computer were corrupted. i have extracted audio files with exact audio copy with the correct settings to have a perfect extraction of the audio track, i've compressed the resulting .wav files with monkey's audio (3.97) (with the verify option) and almost all files resulted correct. but, when i play the resulting .ape files in foobar 2000 (or winamp) there are some bitstream errors. so i have re-encoded the corrupted files -with verify, and played in winamp, it all seems to be correct but after some weeks other files seems to be corrupted.

now i'm going with flac and i don't have problems, but i would like to know the cause of the corruption of the .ape files.

again, i'm sorry for my english
Title: .ape files corruption
Post by: TwoJ on 2003-11-27 00:09:32
I could be wrong but it sounds like you might have data corruption with your hard drive - nothing much to do with what type of file you're storing.
If that is the problem then going to flac will not really resolve the problem either. I had a problem like that with my ASUS motherboard that the VIA chipset was introducing data error especially when transferring large files.

You should create a MD5 file when you create the ape file and then you can determine where/when/how the file is becoming corrupt.

You probably should verify your other ape files and make some MD5 files for them.

good luck

Edit: - PS - it might be a good idea to make some par files if you don't have backups!
Title: .ape files corruption
Post by: Drago on 2003-11-27 07:38:40
i have a via motherboard too (asus a7v 333-x), and i often transfer VERY LARGE group of files between partitions (in the same physical disk), it might be the cause of the problem? i have converted my entire music collecion in flac now and it don't seem to be a problem (for now).
i'll check my hard drives later..
what is md5? a sort of log file? how i can create it?

for the backup, i'm planning to buy a new 120GB hard disk to store my music files...
Title: .ape files corruption
Post by: Sebastian Mares on 2003-11-27 08:05:35
MD5 is a hash algorithm like CRC, SHA and others. It will produce an 128 bit string which will be the representation of the file.
Hash (checksum) algorithms are mostly used when transfering files over the internet. The person who sends the file will also send the checksum and the person who receives the file will compute (let a program compute) the checksum of the file he received. When his checksum is the same as the checksum which was sent to him by the "sender", the files are identical (no errors occured during the transfer). In your case, you could make a MD5 string for all your files and check your files once in a while by creating another checksum and comparing it with the old one. If they differ, your files were altered (changed).
Title: .ape files corruption
Post by: Sebastian Mares on 2003-11-27 08:10:34
Check out this program: http://www.hydrogenaudio.org/forums/index.php?sho...ndpost&p=157634

It seems that the moderators deleted that post.

[Edit]
You can download it here (http://www.md5summer.org/). Download is smaller and HA's space/bandwith isn't unnecessarily spent. - tigre
[/Edit]
Title: .ape files corruption
Post by: Drago on 2003-11-27 11:18:15
so i should 'hash' my files, like p2p programs do, to check later the file's integrity-

in this way i will have a safe 'verify' of all my files? does it work also on .flac files (remember? i have already converted all my .ape to .flac-)?

anyway with flac this is not a problem, files are still here and every song that i have converted is correct.
Title: .ape files corruption
Post by: Sebastian Mares on 2003-11-27 13:37:57
A file digest (hash, checksum) will only tell you if your files got altered or not. It will not help you reproduce the original files.

Anyway, the MD5 sum can be calculated for every file (MP3, FLAC, DOC, EXE, ZIP...).
Title: .ape files corruption
Post by: Drago on 2003-12-02 13:50:32
i've found the problem (maybe). the corruption happens AFTER the tagging of the files (with foobar 2000) and with .flac files too! it could be the tagging the cause of my problems? i will try to not tag some files to watch in they become corrupt or not.
Title: .ape files corruption
Post by: Drago on 2003-12-04 11:07:32
nothing. it's not for the tagging, i have compressed some files from original cd to flac and moved those files to the 'audio' partition of my hard disk, result: 2 files were corrupted. i guess i will decompress the audio files to .wav and store it to an external 160/200 gb new hard disk..for now it's the only solution that i see. maybe i'll check my actual hard drive, it could have some problems if the files become corrupt without doing apparently nothing.

ps-there is a way to verify all .flac files on my hard disk without using flac frontend?
Title: .ape files corruption
Post by: Moneo on 2003-12-04 11:34:12
Drago, that really sounds like a hardware problem.

I would start by checking your RAM - http://www.memtest86.com (http://www.memtest86.com)
Title: .ape files corruption
Post by: tigre on 2003-12-04 11:36:24
Similar things happened to me once:
- After burning files to CD-R some of the files weren't identical with the Originals (I used Drinkcompare then because I didn't know of MD5)
- After moving/copying files from one partition to another some where corrupted
- lame sometimes crashed during encoding
- after defragging C: windows became unstable and I needed to reinstall (was Win98, could be other reasons  )

The reason I found: Bad RAM. All RAM test programs I tried didn't show errors, only the "slow" RAM test on bootup gave an error message. After replacing RAM no problems like this anymore.

So maybe the reason for your problems is bad RAM too or a similar system stability issue (like overclocking).

Besides using MD5 checksums for checking integrity of files you could use PAR2 error recovery files as kind of workarround until you found the real problem: Create a PAR2 file (using e.g. Quickpar) for each lossless music folder (small blocksize like 2048 bytes) and use them to correct corruption if necessary.
Title: .ape files corruption
Post by: tigre on 2003-12-04 11:40:55
Quote
ps-there is a way to verify all .flac files on my hard disk without using flac frontend?

Use something like
sweep for %i in (*.flac) do flac.exe -t %i

(sweep is a tool to process subfolders, available here (http://www.saunalahti.fi/~cse/files/).)

edit: -t added to comandline. Thanks jcoalson
Title: .ape files corruption
Post by: Drago on 2003-12-04 15:53:58
Quote
Quote
ps-there is a way to verify all .flac files on my hard disk without using flac frontend?

Use something like
sweep for %i in (*.flac) do flac.exe %i

(sweep is a tool to process subfolders, available here (http://www.saunalahti.fi/~cse/files/).)

thank you! you saved a lot of my time

so it could be the ram the cause of my problem? how it's possible (i think that the data became corrupt when is written from the disk into the 'bad' ram or viceversa - it is possible?)? and how i can avoid it (maybe replacing the 'fail' module with a new one..)?

now i'm dowloading memtest and later i will check my memory.
Title: .ape files corruption
Post by: Sebastian Mares on 2003-12-04 16:05:12
You might want to download a S.M.A.R.T. status reporting tool (available at WebAttck in the freeware category) and check if your drive reports any S.M.A.R.T. errors.
Title: .ape files corruption
Post by: tigre on 2003-12-04 16:26:39
Quote
i think that the data became corrupt when is written from the disk into the 'bad' ram or viceversa - it is possible?

Yes. That's in short words what I was trying to say in my last. 

Quote
and how i can avoid it (maybe replacing the 'fail' module with a new one..)?
Yup. It's important here to track down the problem carefully e.g. using the test tools as Moneo or Sebastian Mares suggested: Try to get reproducable error reports and then try changing things (other RAM stick, change BIOS settings ...) to see where problems disappear.

BTW: I edited the sweep/flac command line in my last post.
Title: .ape files corruption
Post by: Drago on 2003-12-05 11:46:51
Quote
Quote
Quote
ps-there is a way to verify all .flac files on my hard disk without using flac frontend?

Use something like
sweep for %i in (*.flac) do flac.exe %i

(sweep is a tool to process subfolders, available here (http://www.saunalahti.fi/~cse/files/).)

thank you! you saved a lot of my time

so it could be the ram the cause of my problem? how it's possible (i think that the data became corrupt when is written from the disk into the 'bad' ram or viceversa - it is possible?)? and how i can avoid it (maybe replacing the 'fail' module with a new one..)?

now i'm dowloading memtest and later i will check my memory.

i'm trying to test all my flac files but i have a problem with syntax: i use this command

sweep for %i in (e:\musica\album\*.flac) do flac.exe -t %i

everything is correct but it doesn't work: the prompt returns me without nothing (apparently) happens...what's wrong?
Title: .ape files corruption
Post by: Drago on 2003-12-05 13:31:08
the cause of my problems seems to be a bad ram module. here are memtest's results:

TEST 2 - 6 ERRORS
TEST 7 - 16 ERRORS

errors are in this memory size: 267.3MB; 276.8MB; 320.5MB; 511.6MB

settings:

cached: 512MB
rsvdram: 76K
cache: ON
ecc: OFF
test: STD
pass: 2
errors: 22
ecc errors: 0


since the errors are in the memory space from 267MB to 511MB i think it's the second ram module (i have 2x 256MB ram DDR (333Mhz))...am i wrong?
Title: .ape files corruption
Post by: tigre on 2003-12-05 13:32:18
Put copies of flac.exe and sweep.exe to Windows system folder (e.g. C:\Winnt\System32) OR to e:\music\album\ OR use path command to point to their locations.
Open DOS window.
Type cd e:\musica\album\
Type sweep for %i in (*.flac) do flac.exe -t %i

You'll see all results in the DOS window. You can probably pipe the results to a .txt file instead using
sweep for %i in (*.flac) do flac.exe -t %i > e:\musica\album\flaclog.txt

edit:
I'd do it at least twice - because of the bad RAM module there can be wrong results.

To find out which module is bad, remove each one separately and repeat the test.

edit2: added "sweep for" to 2nd command line.
Title: .ape files corruption
Post by: Moneo on 2003-12-05 14:29:48
Quote
since the errors are in the memory space from 267MB to 511MB i think it's the second ram module (i have 2x 256MB ram DDR (333Mhz))...am i wrong?

Just test the modules individually.

Remove the second one, test the first one.

Then remove the first one and test the second one.

Edit: oh, tigre beat me to it.
Title: .ape files corruption
Post by: TwoJ on 2003-12-05 15:10:43
Just a note;

If you are doing any overclocking - set your bios back to default settings - one of the problems of overclocking is random data corruption.
Don't forget that this could be a cascade problem your RAM & something else.
I mentioned the motherboard since that was the most common occurance of the problem was when transferring large files.

Good luck
Title: .ape files corruption
Post by: Drago on 2003-12-05 17:45:47
i forgot - my system is not overclocked. here's my spec:

motherboard: asus a7v333-x with latest bios (AV333X04.AWD) (could it be the bios the cause of my problems?)

processor: amd athlon xp 1700+

memory: 256+256 mb ddr pc2700 (333 mhz)

hard disk: maxtor diamond plus 9 120GB (ata 133)

video board: geforce 4 TI 4200

audio board: sound blaster live! player 1024 (soon i will replace it with a fortissimo III or a terratec 7.1 aureon)

dvd rom: liteon 16x

cd writer: hp 9100 (8x4x32)


that's all..tomorrow i will chech the ram banks separatly-

ps - thank you for all the support!!!
Title: .ape files corruption
Post by: Drago on 2003-12-07 19:42:15
ok! today i have tested my ram banks and now i have here on my table the 'bastard' module    in the next days i will contact my vendor for replacing it. i would like to thank all the people that have contributed to this discussion! you are all great

see you next week!
Title: .ape files corruption
Post by: Drago on 2003-12-13 14:52:22
the 'sweep' command is right, but i can't verify .flac files for a flac error: when i try to verify single files, with the command "flac -t" it can't open files. the same with the exact file name, chech this dos screens:



E:\Musica\Album>flac -t e:\musica\album\muse - absolution\01 - Muse - Intro.f
lac

flac 1.1.0, Copyright © 2000,2001,2002,2003 Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.


muse: ERROR initializing decoder
      state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

01: ERROR initializing decoder
    state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

Muse: ERROR initializing decoder
      state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

Intro.flac: ERROR initializing decoder
            state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE





E:\Musica\Album>flac -t e:\musica\album\muse - absolution\*.flac

flac 1.1.0, Copyright © 2000,2001,2002,2003 Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.


muse: ERROR initializing decoder
      state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

*.flac: ERROR initializing decoder
        state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE





where is the error? it seems that flac command cant't handle complex names ( 01 - name - title )...why this? the flac frontend works, instead..
help!
Title: .ape files corruption
Post by: madah on 2003-12-13 15:32:02
Quote
where is the error? it seems that flac command cant't handle complex names ( 01 - name - title )...why this? the flac frontend works, instead..
help!


You need to enclose filenames with spaces in them with "

Try this:

E:\Musica\Album>flac -t "e:\musica\album\muse - absolution\01 - Muse - Intro.flac"
Title: .ape files corruption
Post by: Drago on 2003-12-14 13:54:02
now, the sweep command works because the flac command extend his verify to all the subfolders, but there is the same problem with the name of the files..here is a screen:

command:

E:\Musica\Album>sweep for %i in (*.flac) do e:\musica\album\flac -t %i


result:

flac 1.1.0, Copyright © 2000,2001,2002,2003 Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.


08: ERROR initializing decoder
    state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

Zero7: ERROR initializing decoder
      state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

In: ERROR initializing decoder
    state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

the: ERROR initializing decoder
    state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

waiting: ERROR initializing decoder
        state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

line.flac: ERROR initializing decoder
          state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

E:\Musica\Album\ZERO7-~1\SIMPLE~1>e:\musica\album\flac -t 09 - Zero7 - Out of to
wn.flac

flac 1.1.0, Copyright © 2000,2001,2002,2003 Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.


09: ERROR initializing decoder
    state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

Zero7: ERROR initializing decoder
      state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

Out: ERROR initializing decoder
    state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

of: ERROR initializing decoder
    state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

town.flac: ERROR initializing decoder
          state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE


the same for all the .flac files. now i guess i'm close to the solution..i think the error is in the name format, but i tried to correct the value (*.flac) with ("*.flac") and "*.flac" but it doesn't work..help!
Title: .ape files corruption
Post by: jcoalson on 2003-12-15 18:26:18
you forgot the quotes again:

E:\Musica\Album>sweep for %i in (*.flac) do e:\musica\album\flac -t "%i"

Josh
Title: .ape files corruption
Post by: Drago on 2003-12-17 08:33:38
uh..i putted the quotes in the (*.flac) part...thank you for your help, now i will try this format.
Title: .ape files corruption
Post by: Drago on 2003-12-19 11:40:07
it doesn't work, and i don't know why..the quotes are correct now but flac decoder can't open files.




E:\Musica\Album>sweep for %i in (*.flac) do e:\musica\album\flac -t "i%"


(...)


E:\Musica\Album\ZERO7-~1\SIMPLE~2>e:\musica\album\flac -t i%

flac 1.1.0, Copyright © 2000,2001,2002,2003 Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.


i%: ERROR initializing decoder
    state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE

E:\Musica\Album\ZERO7-~1\SIMPLE~2>e:\musica\album\flac -t i%

flac 1.1.0, Copyright © 2000,2001,2002,2003 Josh Coalson
flac comes with ABSOLUTELY NO WARRANTY.  This is free software, and you are
welcome to redistribute it under certain conditions.  Type `flac' for details.


i%: ERROR initializing decoder
    state = 2:FLAC__FILE_DECODER_ERROR_OPENING_FILE



my folder tree is like this:


e:
--musica
----------album
flac.exe
sweep.exe
-----------------album 001
01.flac
02.flac
-----------------album 002
01.flac
02.flac
-----------------album 002
01.flac
02.flac
-----------------album 004

and so on.
Title: .ape files corruption
Post by: smack on 2003-12-19 13:55:41
Quote
it doesn't work, and i don't know why..the quotes are correct now but flac decoder can't open files.

E:\Musica\Album>sweep for %i in (*.flac) do e:\musica\album\flac -t "i%"

Make sure that you use "%i" and not "i%"
Title: .ape files corruption
Post by: Drago on 2003-12-20 11:38:25
whoops  i'm going blind
Title: .ape files corruption
Post by: Phill77 on 2004-01-10 00:12:07
Looks like Drago has sorted his problem now, but I thought I would add my .ape file corruption experience to this thread.

My audio server is a Athlon 900 with 256mb ram, Hitachi 120Gb drive, not overclocked, but I do use a home made fanless power supply which does not always give the most stable of outputs. This stays on permanently and I control what I listen to through my network.

I started to encounter occasional bitstream errors shortly after ripping most of my cd's. To check whether it was a problem with my audio server, I played the track back through a different PC and also copied the file to the PC and played it straight from the hard disk. With one or two tracks it played fine, but generally I would still encounter errors, so I deduced the file was corrupt.

I had slowly been creating a list of all the files with errors so I could have a big session to re-rip them.

However, a couple of days ago I was tidying the wiring up in my audio server and swapped the hard disk cable from a round one back to an 80 conductor flat ribbon.
Since then I have not encountered any new errors, and the previously 'corrupt' files all seem ok.
Whether this is due to a re-boot or the cable swap I am unsure, but I would be interested to hear if anyone else has encountered something similar.

Phill