Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Why we should go back to tape (Read 4332 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Why we should go back to tape

Optical media has gone a long way. There were some nice and amazing technologies in the way that never caught on, like MiniDisc, but overall, with Blu-Ray and such, optical media made a nice progress.

But seriously, when not using them for random access data, like programs or collections of files, what is linear media, like music or videos doing on them? We hardly ever jump to a certain point in the file, we rather fast forward or "rewind" (funny this word survived). This works quite bad with things like video, where you see a stuttery slideshow of interleaved images. Same thing applies for audio. Usually, the audio is chopped up, or not audible at all when "rewinding" or fast forwarding the audio "track" (another word that survived from analogue audio days). So even though we have random access, we still use it in a linear manner.
But why is that?
Well, this lies in the very nature of the data itself. A song only exists as long as it is played linearly. Even though the song is stored on whatever media, it only exists in the moment of it's being played. It applies to video as well, even when you look at a still frame of a movie, it's not the movie you're watching. It's the video being played back what makes the movie exist in that instant.

Now, since the data itself is linear and we use it in a linear manner as well, why is it such a bad idea to use linear media? Although there hasn't been too much digital media for things like music or movies, there were some somewhat successful inventions in the past. Digital Audio Tape, I bet most of us will remember that. Back then, the cassettes couldn't store too much data, 120min was the standard in the mid 90's, but then again, the audio was lossless and uncompressed. It featured quality on-par or above that of the CD, with a capacity of around 1.2GB it surely wasn't large in that respect, but the CD from those days wasn't any larger.

Tapes did make progress since we all have forgotten about them. Magnetic tape is still the de-facto standard when storing large amounts of computer data. Optical media is simply too fragile and unreliable. It is not re-recordable as many times as a tape cartridge. And they store much more amounts of data (although manufacturers like to use the 2:1 compression for marketing purposes). Saving data on tape is still the way to go, if large amounts of data have to be securely stored away, and if it has to be accessed quickly. Even DAT is still out there! Now dubbed DDS/DAT (the newer products are named "DAT 72", "DAT 320", etc.), it uses modernized DAT cassettes for computer data. With DAT 320, which came out in 2009, 160GB of uncompressed data can be stored on one cassette. Compare that to Blu-Ray. Cassettes of that size are much easier to handle, you don't have to watch out not to let them fall on the ground, or scratch their surface. Cassettes really have only one drawback compared to optical media: they're not random access. On every other aspect, tape cassettes win. Did I overlook something? Nope, they're even cheaper!
There's more than DAT of course. The top-of-the-line magnetic tape format is LTO, designed for huge amounts of data, secure storage, but not so much handling and transport (that's more the league of DDS/DAT). Other competing formats, like AIT focus on other aspects, like high data rates and high capacities. DLT was designed with high robustness in mind, both for storage in secure vaults and easy transport (It was designed by DEC in 1984 and is still being actively developed by Quantum. Latest upgrade came in 2007). So there is well developed magnetic tape technology used and developed every day, it's not like it has to be re-invented or something.

Now, lets take the contemporary DAT format, shall we? Why not store a movie on it? What fits on a Blu-Ray, will fit with no problems on a 160GB tape, and 12.8MB/s should suffice for video playback, shouldn't it? The next generation of the Blu-Ray - called "BDXL" is around the corner, with over 100GB on a three-layer disc. But what about reliability, cost effectiveness and ease of handling? Tape formats get upgraded constantly as well. Other formats than DDS/DAT (which I've chosen here for its low price) have capacities of 500GB to over 1TB per tape, and much higher transfer speeds. And to top that of, not only are tapes recordable many times over, but they also have a technology which is designed for one write and many reads (WORM: Write Once Read Many). This, by the way, is a byproduct of large data aggregating applications, used where large quantities of data are being collected, and have to be studied many times afterwards, facilities like CERN use that.

Now, lets talk about compression. Back in the day, when DAT was state-of-the-art, lossless compression wasn't something people got sweaty hands, when using or talking about it. This has changed, as we all know. There's plenty of nice lossless formats, but since this place is about audio, I'd like to keep it simple and stick to FLAC, my personal favorite lossless audio format. Now why lossless? Well, CD's and DAT's are/were lossless as well, however uncompressed. FLAC can be streamed and decoded quite quickly (encoded as well, but that's not the issue at this point), in fact it was designed to be that, so putting it on tape would be just fitting like a glove. With today's large capacities of tapes, the possibilities are amazing.

Let's do some simple math to illustrate:
Back in the day, DAT tapes were rated by their running length. In the mid 90's, 120min was the standard. This was assuming a 44.1kHz sampling rate, at 16 bits resolution per channel, and all that for stereo (two channels). So one second of uncompressed audio is:
[blockquote]44100Hz * 2B * 2 tracks = 176400B/s[/blockquote]
Now, in 120min, there is 7200 seconds, so we get:
[blockquote]176400B/s * 7200s = 1270080000B[/blockquote]
Which is about 1.18GB, and that's consistent with what I've wrote above.

Now, let's take a 160GB tape. Tape manufacturers (like all other manufacturers of storage hardware) use the 10^3 factor, and not the 2^10 factor when talking about capacity, so we can assume that 160GB is 160000000000B.
[blockquote]160000000000B / 176400B/s = 907029.47845805s[/blockquote]
Which is 251 hours, 57 minutes and 9.47845806 seconds. That's a very long recording time.

But we're not finished just yet, let's take FLAC into account. Now, FLAC has a compression ratio of 30%-50% for most music (compression rates for voice recordings are much higher, but let's stick to music), so let's use that:
[blockquote]160000000000B / 123480B/s = 1295756.3977972s (30% compression)
160000000000B / 88200B/s = 1814058.9569161s (50% compression)[/blockquote]
This is 359 hours, 55 minutes and 56.9569161 seconds at 30% compression, and 503 hours, 54 minutes and 18.95691612 seconds at 50% compression.

Those figures are of course idealized, but even if you take off 10% of all those recording times I described above for overhead, etc., you'll still end up with a very, very long recording time. And this is of course, per tape. For recording and editing, one would use hard disks or flash media these days, but once you want to store it away, for reproduction only, tapes are ideal - especially since since they're quite cheap. Also, those drives aren't mass media, prices would be even much lower if they'd be used widely. For video encoding those calculations are quite similar. In fact, video production studios use tape drives for storing their uncompressed footage for archiving, etc. Quick access is no problem with tapes whatsoever. And there is no fear of data loss.

As I mentioned in my opening paragraph, good ideas tend to go under the carpet, or get sucked into the corporate vacuum cleaner. We'll probably use optical media for many years to come. That is Blu-Ray and CD's, which is rather funny, as CD's are older than digital tape media. Selection what wins on the market of those kind of things don't always make sense. VHS and Beta-Max is a prime example. We used VHS tapes which had a far worse video quality than Beta-Max, but VHS tapes had a running length that was more suitable for video renting, so it won. Optical media is shiny and futuristic, it caught on instantly. Be it DVD or CD. The Internet changed a lot of things, so music isn't bought in stores on any kind of media that often anymore, online shops are taking over. The prime way to save music for casual listening nowadays, is flash media, which makes sense, of course. This however, is comparable to put movies on a USB flash drive. For transport, casual watching on stand-alone media players, that's fine, but storing those kind of files on flash based media makes no sense at all.

Personally, I'd much rather have a digital tape player for watching videos in the living room, or listening to my music collection in my car. The huge ammount of storage space, together with easy handling and robustness, makes it clearly my favorite. Linearity is a feature here, not a problem.

Why we should go back to tape

Reply #1 –
While that's quite an impressive essay, the trend for music today is to eliminate physical media for the listener.  Certainly physical media is required for the storage of audio, but the end goal is to ensure that the listener does not care (or have to care) what that media is.

Why we should go back to tape

Reply #2 –
Interesting points brought up, indeed.  I have used high bias cassette, 2 inch reel to reel, DAT, DTRS, MiniDisc, and of course CD's and DVD's and hard drive recording/playback.  Of course some of the benefit of non-linear data formats is the redundancy and feedforward for error recovery.  But yeah, you make some really good points. 

I am personally sick of CD's and DVD's.  I don't like how they get scratched and smudged and covered with fingerprints.  I take good care of my discs, but when sharing them with others, most other people *STILL* don't know how to properly hold and care for a compact disc or DVD.  And I'm sick of some brands having the backing flake off, losing data. 

Tape has it's problems too, but CD/DVD is not the answer in my opinion.  I just recently converted nearly all of my CD's to FLAC and WAV files and it's really paid off.  I can keep tons of tunes on flash and hard drives.  It's truly amazing.
opinion is not fact

Why we should go back to tape

Reply #3 –
Quote from: astroidmist on
Tape has it's problems too, but CD/DVD is not the answer in my opinion.  I just recently converted nearly all of my CD's to FLAC and WAV files and it's really paid off.  I can keep tons of tunes on flash and hard drives.  It's truly amazing.

I do that too, mostly. But then again, flash drives and hard disks aren't really a good way to store data securely. If they break, pretty much all data is gone in one instant, and most of the time, it's quite non-visible, that something is bound to go wrong. When you have a large collection of music (from what I read, you certainly do), I suggest you settle for a way to store them to another media. Be it digital tape (I'm working in IT and academia, we use those for backup, secure storage and sometimes transport, so I'm kinda biased) or something else. Optical media is OK for backups, as long as it is checked frequently and backups are recreated in certain intervals (about every two to three years is advisable).

With flash media, that is really horrible, you never know when you might lose your data. Especially flash cards, like SD are quite prone to this symptom. It just stops working from one minute to another, and the data is lost forever. I was actually considering using SD cards or CF as "general data storage", so shun optical media, shun any kind of mechanical data storage technology, move on to flash! And boy, I learned my lesson... What killed my data on the flash drive, was caused by non-stable ambient temperature and humidity. Cold winter, hot summer, a temperature variation of around 30°C and different humidity levels made the NAND flash lose its data partially. I've lost about 20% of all my stored data in one year, and as Murphy knows best, those were the most vital 20%...

Of course it didn't erase or scramble 20% of all my files or flash cards, it just destroyed tiny bits of it, but rendering whole data chunks unusable. Those little errors were scattered around the whole media collection, diffused enough to kill 20% of all data in total. No fun at all.

Since I don't have too much valuable data, I'm back to using tapes, and hard disks for quick access. No progress whatsoever.

Why we should go back to tape

Reply #4 –
I am familiar with tape only as am analogue storage media...and I had Revox and Braun as recorders.
I rather used a vinyl record than tape. Hiss was increased, print through, loss of data through the machines (through user error) mangling the tape...I do not know how that compares to tape as digital storage, but afaik - there is no safe media out there.
Just back up twice. I use a 8tB hardrive in raid 5, which gives some security, and backup to a portable 3 TB drive. That is above the internal PC storage. I have close to 1Tb in music files in flac,
and the ripped cd's in a cool storage place, plus about 2k vinyl records. So there is some physical storage as well.

Why we should go back to tape

Reply #5 –
Quote from: kraut on
Just back up twice. I use a 8tB hardrive in raid 5, which gives some security, and backup to a portable 3 TB drive. That is above the internal PC storage. I have close to 1Tb in music files in flac,
and the ripped cd's in a cool storage place, plus about 2k vinyl records. So there is some physical storage as well.


Picture this: You drop your external hard drive from approx. 1m onto ground. You should immediately check data integrity. If you drop a tape from 1m onto concrete, you just check for cracks, and if there are none, it is safe to assume the tape is perfectly OK.

Digital media has none of the problems you described above with analogue media. Take a look at some videos on youtube, it explains how digital tape works better than I could do it in text form.

Why we should go back to tape

Reply #6 –
Quote
The demise of tape won’t be a big bang that results in a black hole through some instantaneous event. Instead, tape will die because it can’t follow or match current market conditions and requirements. Problems that limit tape’s popularity include mislabeling, labels that fall off, media failures, transport failures, library failures, media server failures, slow performance, risk of loss or theft, and the high cost of replacing, adding, or re-mastering tape for use by new generations. For all those reasons, users will simply stop writing data to tape; they’ll opt for disk instead. Every stronghold of value that tape once held has been supplanted by disk. Protection, performance, reliability, energy conservation, management, and cost all favor disk...
Gartner estimates that 15 percent of all backups fail. Additionally, 10 to 50 percent of all subsequent restores from tape fail, depending on the elapsed time since the backup occurred. Restoring data from tape older than five years fails 40 to 50 percent of the time. Much of this goes unnoticed. Both Gartner and Storage Magazine report that some 34 percent of companies never test a restore from tape. Of those that do test, 77 percent experienced failures in their tape backups.

http://www.mainframezone.com/storage/backu...collapsing-star
Blows some claims made here out of the water.

Since my raid 5 units in a 1ru rack casing sits securely in a rack, and consists of four drives where one is used for the raid backup, the chance of it crashing to the floor is likely to happen only in an earthquake of magnitude 9 and above. I welded the rack from 2 x 1" steel tube, and it weighs roughly 150lbs....
 

Why we should go back to tape

Reply #7 –
Quote from: kraut on
http://www.mainframezone.com/storage/backu...collapsing-star
Blows some claims made here out of the water.

Assumptions made in there are on aging media: LTO-3 Was released in 2005, we're at LTO-5 now, and LTO-6 is announced. While tapes are used for backup, that's not the sole application for it. There is a reason, why tape libraries are used up until now. Like I said in my opening post, there is different kinds of tapes, which specialize in different kinds of applications. Personally, I have worked with tape media on a daily basis, and with different kinds of tape species. What this article doesn't state, is what environment we're talking about here. I've seen unreliable and reliable tape media.

Quote from: kraut on
Since my raid 5 units in a 1ru rack casing sits securely in a rack, and consists of four drives where one is used for the raid backup, the chance of it crashing to the floor is likely to happen only in an earthquake of magnitude 9 and above. I welded the rack from 2 x 1" steel tube, and it weighs roughly 150lbs....

Why, that's nice and all, but I'd like to see how you carry that case out, when your house is on fire.
That aside, some security scenarios require complete offline storage. In case of complete offline storage, (mechanical) hard drives is the worst thing one can use. While data degradation is predictable for tape media pretty accurately, it is completely unpredictable for mechanical hard drives. A mechanical drive can easily have a life span of fifteen years, or fife months.

That said, it seems there is a discrepancy, in what backup actually is. There's system backup, where quick, rotating backups, on things like RAID arrays makes perfect sense and is secure enough. And there's data backup, which means to store data securely. I worked in academia, where a lot of medical data is generated. The data needs to be backed up while processing it, and at least one copy must be stored in an offline data vault, which was off-site, in a vault 25 deep in the earth. The genome and personal data stored there is never to be touched again, except if the processes have to be repeated. This doesn't mean, that we never take a look at the tapes again, they're checked at least once every six months, and are recreated every three years. Even when part of the data is broken, it's not a big deal, since we keep parity data on some of them. This makes them effectively an offline version of RAID 6. But that's not all. What the article completely forgets, is that there is actually RAID for magnetic tapes, fittingly called ROT (RAID On Tape). A Tape doesn't have just one track, it has somewhere in the order of 8 to 32 tracks. ROT is implemented on those tracks. Additionally, the RAID parity is spread over several tapes, making it a cascading RAID. We use RAID for system backups all the time, we use NAS for that. The point is, we use it because it is overwritten quite often, so even when the disk fails, it doesn't matter much, since there's the RAID parity. Since the RAID system is accessed pretty much all the time, since it's an online storage, disk failures get noticed pretty quickly, and even if two disks fail at the same time, the data can be restored. However, this all is pretty costly, building the 120TB array back in 2008/2009 wasn't quite cheap. We still use tapes to to transfer data, when upgrading the array for larger disks, though. Also, to a reliable RAID, there's also the connections to the RAID array. We use FC, but we've been on Myrinet for a short while in 2005 or so. But even though management is quick and easy, due to the unpredictability of mechanical disks, we would never use them in anything else than online RAID.

All that doesn't have anything to do with music though. We're talking about huge amounts of data, most of them very vital and expensive. Tapes may vanish one day from use in computing, but the only way of storing data in a secure manner similar to that of tapes, is flash drives. Once SSDs and other flash media reach capacities that are large and fast enough to hold backup data, it will be used in RAID arrays before being used as a tape replacement. Once that happens, though, we wouldn't be using any other kind of data storage anymore. When using digital music, on an every day basis, all we need is small and very portable flash drives. We already have that, which is USB drives and SD cards (there's other types of flash media, like Sony Memory Stick, Compact Flash, and others, but SD is the most common, thanks to cell phone manufacturers, etc.). The only thing hat needs better standards, is the filesystems on those drives. Standard FS on SD is FAT32, which is quite problematic. SDXC uses exFAT, which solves part of the problems of FAT32, but introduces new problems, for one thing, not being an open standard.

We're all migrating towards flash based drives. Until that moment, though, we'll be using tapes for quite a while to come. It is very well possible, that digital tape media will survive longer than optical media.