Skip to main content

Topic: The best way to backup lossless audio data? (Read 10159 times) previous topic - next topic

0 Members and 1 Guest are viewing this topic.
  • jeremija
  • [*][*][*][*]
The best way to backup lossless audio data?
I have about 50 gb of lossless audio that I want to backup from HDD. If my hard drive fails, all of my .flac files will be lost and re-ripping and re-tagging are both time-consuming processes which I would like to avoid.

So, which would be the best way to backup:
  • on multiple dvd discs
  • on one or two dual layer blu ray discs
  • on an external hard drive which would serves only for backup
  • something else?

The main pluses are that it lasts long (ie. I would want the medium not to self-destruct itself in 10 years due to the aging process) and that it's not too expensive.

The way I see it - I should iether buy a blu-ray recorder (which costs about $200 where I live) plus a few blu-ray discs, or buy a new external hard disk (around $80). But I don't know will blu-ray disc become unreadable in a few years, or will hard disk stop working... So what do you think, which lasts longer?

  • 2E7AH
  • [*][*][*][*][*]
The best way to backup lossless audio data?
Reply #1
My approach is backup on DVD with 1 level of correction files (stored also on DVD)
Keep them in vertical position
Something like annual check in this case is recommended

  • jeremija
  • [*][*][*][*]
The best way to backup lossless audio data?
Reply #2
what exactly are the correction files and how to generate them?
  • Last Edit: 25 January, 2010, 03:57:10 AM by jeremija

  • 2E7AH
  • [*][*][*][*][*]
The best way to backup lossless audio data?
Reply #3

  • jeremija
  • [*][*][*][*]
The best way to backup lossless audio data?
Reply #4
thank you!

  • flacflac
  • [*][*]
The best way to backup lossless audio data?
Reply #5
Hi Jeremija,

While I also endorse the use of optical disc backups I also strongly recommend the following:

1) Get a second harddrive and make daily to weekly backups of your main drive - once your collection size increases you'll get dizzy recovering data from 20-50 DVDs, perhaps only to find out that one of them has disc rot and ends up useless. 

2) Instead of using a rather proprietary tool such as DVDisaster I would recommend using Par2 Parity-files, which also use Reed Solomon algorithms to generate redundancy data.  At 15-20% this can be read by many tools and recovered quickly, not only from DVDs but also if you have a bad sector on your harddrive that damages an audio file.

Easiest interface can be found at http://www.quickpar.org.uk/ .

ff 

  • Ardax
  • [*][*][*]
The best way to backup lossless audio data?
Reply #6
Par2 is just as "proprietary" as dvdisaster.  Both use Reed-Solomon error correction, they just apply it differently.  For optical media, I'd go with augmented images from dvdisaster.

But I'd rather use an external HD for backup purposes, simply because it's going to be a LOT faster for backup, verification, and restoring than optical media.  What's more, you avoid having to deal with swapping discs.  Connect it only when doing backups.  Use a combination of Par2 (I like MultiPar) and hashes for verifying and recovering any bit errors.  MD5 or SHA1 hashes (HashCheck are much faster to generate and verify than Par2 files, IME.
  • Last Edit: 25 January, 2010, 06:41:01 AM by Ardax

  • jeremija
  • [*][*][*][*]
The best way to backup lossless audio data?
Reply #7
Thank you all for your replies. But from the medium's lifetime point of view, which has a longer lifetime (if treated properly): hard disk or DVD disc?

  • 2Bdecided
  • [*][*][*][*][*]
  • Developer
The best way to backup lossless audio data?
Reply #8
How long is a piece of string?

Which are better - cats or dogs?


You're asking an impossible question, but you'll still get 10 pages of replies - all of which have been given many times before!

Cheers,
David.

  • GHammer
  • [*][*][*]
The best way to backup lossless audio data?
Reply #9
1) Get a second harddrive and make daily to weekly backups of your main drive - once your collection size increases you'll get dizzy recovering data from 20-50 DVDs, perhaps only to find out that one of them has disc rot and ends up useless.

2) Instead of using a rather proprietary tool such as DVDisaster I would recommend using Par2 Parity-files, which also use Reed Solomon algorithms to generate redundancy data. At 15-20% this can be read by many tools and recovered quickly, not only from DVDs but also if you have a bad sector on your harddrive that damages an audio file.



DVDisaster has one advantage, it reads damaged CD/DVD.
With a PAR solution, you'll have to get something to read the damaged media.

For files on hard drives, I just use WinRAR with a recovery record.

  • Teknojnky
  • [*][*][*][*]
The best way to backup lossless audio data?
Reply #10
50 gigs is a pretty small amount, you can easily and quickly keep 1 or more backup copies on external drives very cheaply and much easier than any amount of optical storage.

The primary thing to rememeber is, if you only have 1 instance of your data and that media fails or is damaged, you have no recourse. The safest data is backed up to multiple physical copies both locally and to a remote location (ie to protect from theft/fire/etc).
Last.FM Nodes for your library @ http://build.last.fm/item/356

  • Fandango
  • [*][*][*][*][*]
The best way to backup lossless audio data?
Reply #11
I would rather use DVDisaster because it has all in one application. ISO augmentation, CD/DVD ripping without halt on errors, etc.

And therefore, because of its ease of use, I wouldn't use RS01 error data, but RS02. Because where do you want to store the ECC files? What if the medium you store the ECC files on fails, too? RS02 can deal with ECC data corruption as well, besides it's so much more convenient to use. The only drawback is that you cannot use a DVDs full capacity, which never was a big deal for me.

Regarding PAR2. The only GUI tool for Windows Quickpar has no multi-threading support and it also does not support sub-folders, AFAIK. No support for sub-folder is a show stopper in itself for music backups unless you have all your music rips in one single folder. Or you want to create a PAR2 file set for each rip.

But there's a MT-hacked version of the command line tool par2.exe (http://www.chuchusoft.com/par2_tbb/). I strongly recommend this one instead of Quickpar, since it's faster and supports sub-folders. In case you want to use PAR2 instead of DVDisaster...

So in case of DVD backup, I'd recommend DVDisaster. And in case of HD backup, I'd recommend the multithreaded version of par2cmdline.

And I would do both. Backup to another HD (thanks to eSATA, that's as easy as it can get), and backup to DVDs.

  • Ardax
  • [*][*][*]
The best way to backup lossless audio data?
Reply #12
Quote
Regarding PAR2. The only GUI tool for Windows Quickpar has no multi-threading support and it also does not support sub-folders, AFAIK.

See the MultiPar tool that I linked to in my previous post.  I /think/ it does multi-threading as well, but it definitely handles subfolders.  And it has a GUI, for those that prefer them.

Thanks for the pointer to a CLI par2 program that handles subfolders though.

  • Dirge
  • [*]
The best way to backup lossless audio data?
Reply #13
I got to thinking it would be best, at least for my needs, to add PAR2 files to each album and include a checksum file. I would then back up to two separate external drives for redundancy.

I spotted a link to the HashCheck Shell Extension in this thread and got to wondering if I should generate .md5 or SFV files for each album. Are these file formats standardised or does each file verification app follow a different standard?

  • SCOTU
  • [*][*][*]
The best way to backup lossless audio data?
Reply #14
Assuming you have a spare place for a hard drive and a DVD burner, the DVDs are definitely the cheaper option.  However, the HDD offers the benefit of allowing you to much quickerly back up and restore things, less space, less hassle, and easier to identify if your backups are broken.  If your data is on a Data drive (not the OS' hard drive) you can get another one of the same (internal) disks and (assuming you have the 50GB to back it up temporarily somewhere) make a RAID 1 set with your current drive and the new one (if your motherboard supports RAID 1).  This would give you automatic backup that takes 0 time plus the other benefits of the hard drive.

  • odyssey
  • [*][*][*][*][*]
The best way to backup lossless audio data?
Reply #15
Personally I've given up on the optical medias. They degrade rather quickly - Some faster than others though. I have much more faith in a secondary harddrive:

I have bought a NAS-solution which I'm locating somewhere else and use rsync to keep it updated as I extend my library. No extra work, no hassle, always updated!
Can't wait for a HD-AAC encoder :P

  • viktor
  • [*][*][*][*]
The best way to backup lossless audio data?
Reply #16
DVDs are much less reliable than HDDs, not to mention their worse price/capacity ratio. and of course you'll have to switch between disks all the time, while on HDD everything's in one place. and HDDs are faster than DVDs.

bluray is simply not an option, disks are way too expensive, writers are of bad quality, hard to find, and even more expensive.

i'd strongly encourage you to use 2 HDDs, one for storage, one for backup. you can put the backup disk in a mobile rack and make a sync weekly. for syncing rsync (linux) is the best option imho, i use its windows port called deltacopy with an hdd in a mobile rack.

or if you're really maximalist, you can set up a NAS with raid1 and 802.11n, for example.
  • Last Edit: 10 August, 2010, 07:09:37 AM by viktor

  • odyssey
  • [*][*][*][*][*]
The best way to backup lossless audio data?
Reply #17
I have bought a NAS-solution...

It might seem overkill, but consider your house/apartment burned down to ground or something like that. Backup in a remote location is always a good thing
Can't wait for a HD-AAC encoder :P

  • SCOTU
  • [*][*][*]
The best way to backup lossless audio data?
Reply #18
DVDs are much less reliable than HDDs, not to mention their worse price/capacity ratio.


DVDs are actually the cheapest modern (idk how much tape costs) storage medium in terms of cents/GB.  a 100pack of DVDs nets you 437GB for as low as $16.  That's 3.7 cents/GB.  Cheapest hard drive price ratio I could find was 1TB (actual 931GB) for $60, or  6.4 cets/GB, almost double the price of the DVDs.


bluray is simply not an option, disks are way too expensive, writers are of bad quality, hard to find, and even more expensive.


This is where I'll have to revise my previous post.  The cheapest solution for you may actually be blu ray (assuming you already have a blu ray writer, that is.  For 50GB, you'd need 2 BDs @ $3 ea for a total of $6.  For DVDs, you'd need a very inconvenient 11, which 15 Packs are sold @ $8.

or if you're really maximalist, you can set up a NAS with raid1 and 802.11n, for example.


For backing up data, I would highly recommend gigabit Ethernet over any sort of wireless.  Even using 802.11n and having a good connection will average 25.5 MB/s at best.  Using gigabit ethernet, however, you may actually be limited by the write speed of your hard drive, with speeds that can be up to 100 MB/s average.

  • Akkurat
  • [*][*][*][*][*]
  • Developer
The best way to backup lossless audio data?
Reply #19
RAID is not for backupping (a main storage), not any RAID. RAID is meant to increase data availability/reliability and/or I/O performance. Versioning is important in backupping, RAID doesn't offer that. RAID doesn't help if e.g. file system goes crazy, program/OS/user deletes a file, failing PSU/controller destroys both drives. I.e. Wikipedia - RAID - backups. EDIT: If one has a main storage and a separate backup of that, I personally don't see why using RAID in neither one would be beneficial (from backup POV).

I guess having an external off-site HD for backup would be ideal (for backupping purposes). Personally I've been _thinking_ about setting up a following system: main storage of big/"media" files, daily/constant versioning backup of "work files" (or all files) from my desktop PC, plus weekly/monthly incremental HD images in NAS. On-site sync of that to USB HD (easier than off-site) & an online (internet, encrypted) backup of "critical" files. Wow, that's a rather complicated looking plan now when written down.. backupping is tedious.
  • Last Edit: 10 August, 2010, 11:49:38 AM by Akkurat

  • shadowking
  • [*][*][*][*][*]
The best way to backup lossless audio data?
Reply #20
Keep it simple (KISS):

Its a backup like any other. Don't worry about bit rot or corruption too much, Avoid RAID / DVD . Just backup on a spare drive . Having a yearly snapshot on an additional drive can provide extra security. The drives can even be  80gb models from used PC's - the chances of 2 ~ 3 drives going bad at once is next to zero.

In short just backup your windows profile that should contain your flacs or in unix type OS your /home
wavpack -b4x4s1c

  • krabapple
  • [*][*][*][*][*]
The best way to backup lossless audio data?
Reply #21
Keep it simple (KISS):

Its a backup like any other. Don't worry about bit rot or corruption too much, Avoid RAID / DVD . Just backup on a spare drive . Having a yearly snapshot on an additional drive can provide extra security. The drives can even be  80gb models from used PC's - the chances of 2 ~ 3 drives going bad at once is next to zero.

In short just backup your windows profile that should contain your flacs or in unix type OS your /home



And keep one drive offsite in case there's a catastrophe at home.

  • SCOTU
  • [*][*][*]
The best way to backup lossless audio data?
Reply #22
If you're backing data up remotely over the internet, make sure to keep a close eye on how much you're backing up.  Some ISPs will deny you service if you use too much bandwidth.  This can happen extremely easily if you're not paying attention to how much you back up in a month.

  • Akkurat
  • [*][*][*][*][*]
  • Developer
The best way to backup lossless audio data?
Reply #23
If you're backing data up remotely over the internet, make sure to keep a close eye on how much you're backing up.  Some ISPs will deny you service if you use too much bandwidth.

That is a good point/advice.. for those unfortunate who live in a country where ISPs do that.

The best way to backup lossless audio data?
Reply #24
Personally I've given up on the optical medias. They degrade rather quickly - Some faster than others though. I have much more faith in a secondary harddrive...

I'd go for both decent optical media and a backup hard drive. Having lost 8 out of 10 hard drives over the past 18 months for no obviously apparent reason, and only 1 correctly stored DVD-R out of over 100 in the past decade, I wouldn't put my faith entirely in hard drives without another form of backup. YMMVC.

Lose a DVD and you've only lost just over 4GB of data. Lose a full 2TB hard drive and you've lost 2TB of data. The words "eggs" and "baskets" spring to mind.
  • Last Edit: 10 August, 2010, 07:31:11 PM by Slipstreem