De-archiving Hollywood: Mining for Gold in Old Video Footage

Haven’t heard the term “de-archiving?” You will because it’s a trend that is gathering momentum in the entertainment industry, as well as in insurance, banking, manufacturing, and many other industries. It’s all about bringing old data back to life . . . for fun and profit.

I first heard the term when we were visiting with the head of digital media for a major studio. He described it as taking valuable or even never-distributed movies and TV shows out of the archives and licensing them to new outlets hungry for digital content, such as Apple TV, Netflix, DirectTV, and the hundreds of other channels available on Internet services like Roku, Nyrius, and Chromecast.

Too Expensive for Digital Storage

Until now, bringing that old or unseen content into the light of day was too expensive. Digital movie files are huge. These days, almost all movies are shot digitally. For every shot, there may be as many as 10 cameras rolling simultaneously from different angles. Resolutions have jumped from HD 1080P to 4K and now to 8k, vastly increasing the sharpness and clarity—but also the size—of footage shot from each camera. And not every frame makes it into the movie. For each minute of finished film, there could be an hour or more of raw footage. Then there’s all the post-production versions, added CGI, and so forth. So when you’re finished creating the film you’re left with as much as 10 petabytes (PB) of data representing the raw footage and the complete, processed video.

Storing all that digital data comes at a massive price. Storing a 10 PB movie on live media using storage servers like those made by Dell EMC or NetApp would cost upwards of $3.5 million per year, based on a total cost of ownership of around 35 cents per GB per year. (See The Future of On-premise Storage for my complete cost analysis.) If you stored those files in Amazon S3 at a cost of 2.1 cents per GB per month, that would cost you $2.52 million per year. To store one movie! Both options are too expensive for studios to even consider storing their entire movie catalog digitally.

Inaccessible on LTO Tapes

That’s why, for years now, the archival medium of choice in Hollywood has been the lowly magnetic LTO tape. The current generation of LTO tapes, called LTO-6, has a raw storage capacity of 6 TB. To store 10 PBs would take about 1,700 tapes. That’s a boatload of tapes.

To make matters worse, archivists insist that tapes with valuable and unique files be transferred to newer generations of tape every two or three years. This is probably the biggest hidden cost in tape libraries. Even though manufacturers claim that tapes will last 10 years or more, in practice nobody keeps them that long because the machines that play those tapes become obsolete or wear out and the newer ones won’t necessarily be able to play the older generations of tapes.

Imagine a scenario where someone in sales at the movie studio has an opportunity to license out a segment of a film for an advertisement or promotion, but the customer wants to use a piece of raw footage, not just a clip from the finished film. Unlike the vast commercial libraries of still photography like The Image Bank or Getty Photos, you can’t just open a browser and flip through a mountain of tapes. A real person has to look up which tape is likely to hold the content, find the physical tape in a box somewhere, load it onto a player, and search for the appropriate segments. It’s a very laborious and time-consuming job. In general, studios won’t bother unless it’s a very big opportunity.

Every studio in Hollywood, New York and elsewhere has warehouses full of such tapes. Hundreds of thousands of them. Companies like Iron Mountain make millions of dollars each year storing Hollywood’s tape archives and, hopefully, preventing them from getting lost or stolen.

There are tens of thousands of old sitcoms, cooking shows, newscasts of historical significance, unreleased feature films and TV shows, documentaries, live concerts, you name it. These days, you know somebody, somewhere, wants to watch these assets. In the new world of infinite choice, this material could be released in on-demand digital formats ranging from an Amazon rental to ad-supported distribution on YouTube or any of the other dozens of online video repositories.

The problem is, those tapes translate into petabytes upon petabytes of digital data, which is expensive to store in any way that makes quick access possible. So, content with an aggregate production cost in the tens or hundreds of billions of dollars sits idle. Offline. Inaccessible. Earning no returns. While there’s an audience out there for all of it, willing to pay if only they could get access to it.

Cloud 2.0 Services Breathe New Life into Old Content

The advent of on-demand viewing over the Internet has changed the economics for resurrecting all this old content. Before services like Netflix and YouTube, there was no economical way to distribute content of narrow or limited appeal. But if the distribution can be made cheap enough, you can unlock all that potential value. And that, of course, is exactly what is happening with content delivered over the Internet. If the cost of distribution is less than what people are willing to pay (in the form of rental, subscriptions, or advertising) to watch, then you have profits. That’s the case with a movie blockbuster that costs $200 million to produce and earns $300 million at the box office, or an old cooking show that only earns $50 but costs a few dollars to distribute.

Hence, de-archiving—the process of taking all that content out of mothballs and putting it online. And the technology that is making it possible, and even lucrative, to transfer that content from tape to live media is very low-cost cloud storage. That’s the whole idea behind Cloud Storage 2.0 products like Wasabi. Cheap, fast, reliable cloud storage. In fact, the total cost of ownership of tape is significantly higher than the cost of hot storage in Wasabi. I’ll present a more detailed cost analysis of tape vs. Wasabi in a future blog, but here’s a hint: it’s about half the cost.

The de-archiving movement that is now gaining momentum reminds me a little of what has happened in the gold industry from time to time. Outside of every gold mine is what’s called a slag heap. It’s a mountain of the rubble that’s left after the gold has been extracted from the rock. There’s actually still a little gold left in that slag heap, but it isn’t worth the effort to extract that last little bit. However, when the price of gold shoots up or somebody develops a cheap new way to get those last bits of the mineral, then suddenly what people considered worthless literally turns into gold. That’s exactly the effect that Internet distribution coupled with cheap, fast, cloud storage is having on the movie business. That old footage in the archive can mean big money for the studios and hours of novel entertainment for a new generation of consumers.