Why Cloud Storage Hasn’t Killed LTO Tape

Google “tape is dead” and you’ll see just how long the industry has been predicting the demise of LTO tape storage—and how long tape has defied those predictions. Cloud storage was supposed to be the final stake in its heart, yet tape persists. Why is that? Why hasn’t the entire media and entertainment industry migrated to the cloud? It turns out, when you crunch the numbers, current generation cloud storage like Amazon S3 is still too expensive when compared to tape. And cold storage tiers like Amazon Glacier are too slow to make the migration worthwhile.

The Real Total Cost of Ownership of LTO Tape

If you search online for “Total Cost of Ownership of LTO Tape” almost every first-page result is from a manufacturer of tapes or tape libraries, or an independent research firm hired by one. Some of the silliest “reports” are those that simply recite the cost of the raw media and compare that with complete disk-based systems from manufacturers like Dell EMC and NetApp. It’s not a fair comparison, of course, because the raw tape is only one component of a complete storage system.

The most thoughtful analysis comes from Enterprise Strategy Group (ESG), a reputable research organization specializing in storage. Their white paper, entitled “Analyzing the Economic Value of LTO Tape for Long-term Data Retention” by Jason Buffington, was commissioned by the LTO Program, a tape industry association. Their analysis is based on the following assumptions:

• Ten years of storage
• 1PB starting data
• 30% annual data growth
• 10% of data retrieved each month
• Compression: 50%
• Migration to next generation of LTO after 5 years
• 80% utilization of media
• Based on LTO-6 media with average cost per cartridge of $145

The conclusion is that the ten-year cost of storage on LTO tape would be $241,110 per year, or approximately $.02 per GB per month. The only issue I have with this analysis is the 50% compression. Not all data can be compressed. Video and audio media files, JPG pictures, and most other large unstructured files are not compressible. Furthermore, if a file can be compressed 50%, that would apply whether you were storing the data on tape, on disk on-premises, or in a public cloud. In my view, it’s irrelevant and the real measure should be the cost of storage per raw byte, which in this analysis would really be $.04/GB/mo. Counterbalancing this, however, is the fact that since this report was issued in 2016, LTO-7 tapes have hit the market. The cost of an LTO-7 cartridge is now roughly what an LTO-6 used to cost, with a raw capacity of 6TB vs. 2.5TB for LTO-6. So that takes us back to roughly the same place: ~$.02/GB/mo. In other words, slightly cheaper than cloud storage like Amazon S3.

What about Glacier and other cold storage clouds?

I recently had a meeting with one of the major film studios in Hollywood. They have a much bigger tape library: tens of petabytes. They calculate their total cost of ownership at $.007/GB/mo.—significantly lower than the ESG analysis. They reported hating their tape library because it is prone to mechanical problems and takes a long time to retrieve files (tape isn’t “random access” like a disk-based system).

I asked them if they hated tape so much, why haven’t they migrated to cloud storage? Considering the amount of content they have, they said their storage costs would triple if they migrated to a cloud service like Amazon S3. Cold storage tiers such as Amazon Glacier would work from an economic standpoint, but it would be a wash financially, as there would be a big cost of migration, and it wouldn’t solve their requirement for instant access.

Immediate access to archived files is important to them because they make extensive use of artificial intelligence to quickly identify and find specific footage based on machine learning techniques such as facial identification, object and scene identification, and natural language processing and indexing. Not long ago, content like this had to be manually tagged with metadata for it to be searchable. Today, thanks to new video management systems like those sold by Axle.ai, content is tagged automatically and easily searchable by type of scene, image, actor, even by words spoken in dialogue. So, you could run a search for “all scenes with R2D2 and 3CPO,” for instance, and find the droids you were looking for in seconds instead of hours. You don’t need “the force,” just the right software and data storage that is instantly and randomly accessible.

Glacier is not designed for instant access, but Wasabi hot storage is. I discuss this and other trends driving the migration from tape to hot cloud storage in my recent post on De-archiving Hollywood.

Is tape really dead this time?

Tape is slow. Tape gets lost. Robotic tape libraries are finicky Rube Goldberg contraptions of the first order. The industry professionals that use tape will be the first to admit they don’t like it. But it does the job (most of the time). And until recently, there hasn’t been a cheaper alternative.

For mass migration to the cloud to occur, there would need to be a substantial economic benefit to go through the trouble and expense of migration. Given the TCO analysis above, it won’t happen with Amazon S3. And Glacier, while priced right, doesn’t provide the instant access the industry needs to make the move from tape.

I believe Wasabi hot storage is the game changer that may finally put LTO tape to rest for good. At 1/5th the cost and 6x the speed of Amazon S3, Wasabi offers both the economic and innovation incentives the industry needs to fully embrace cloud storage.