Artificial Intelligence is Just a Dead Man Walking Without This

Most conversations about AI tend to focus on the increase in computing power or improvements in algorithms. Both are a critical part of achieving AI, but what’s often missed in those conversations is that AI is only as intelligent as the volume of data it has to learn from.

All of the recent advances in AI could simply not have been possible without our significantly increased ability to capture and store data. This applies to everything from using AI to find patterns in and to learn from the over 3.5 billion searches done on Google every day (one trillion searches year!) to the 11 terabytes per day that a Waymo driverless vehicle generates.

In each case, where we see AI making inroads and holding out the promise of radically new approaches to solving problems, we also find this tight connection between data creation and consumption to be true.

How Does AI Learn?

An easy way to understand why this is the case is to think about the way AI learns, which requires sifting through enormous amounts of data to create the equivalent of an experience base.

For example, Google DeepMind’s AlphaGo Zero, which won against both the world’s human Go champion as well as against its predecessor AlphaGo, had to play 20 million games against itself to train its AI.

Of all the factors driving increases in data storage, none will be as pervasive as AI. The value of AI in countless applications and the cost savings it will deliver will drive adoption. Research by Deloitte projects labor savings due to AI in federal, state, and local government of over $40 billion. If we apply the same savings to private industry, savings could exceed five trillion dollars annually.

There’s More to (Artificial) Life than Cost Savings

As extraordinary as that may seem, it is still calculating the value of AI based on how it simple streamlines existing processes rather than looking at all of the ways in which it will create new value and businesses that we can't yet identify.

Estimating this with precision is far more difficult; however, during the last 140 years of the industrial era, every reduction in the labor force brought about by technology savings has created far more jobs, economic opportunity, and new businesses than it has destroyed.

There’s no doubt that AI will amplify this effect many times over any prior technology. But as we start to realize the value of AI a virtuous cycle will begin to accelerate demand for data well beyond the anticipated currently anticipated rates of growth.

Take for example autonomous vehicles.

Even at 11 TB a day, today’s driverless vehicles use AI in a very narrow sense. In many ways their progress is being inhibited by the lack of enough data. This leads to one of the most pressing issues in the field of driverless vehicles and autonomous devices (what are collectively called AVs). In order to conserve storage, most of the algorithms that operate AVs are trained to gather specific data about a subset of the AV’s context. However, the ideal way in which to train an AV is to allow it to learn the same way that human drivers do, by capturing all available data about context. This has a dramatic impact on the amount of data that needs to be stored, increasing data storage from 11 TB to up to 200 TB a day.

And this is where the promise of AI hits the equivalent of a brick wall.

200 TB per day multiplied by 365 days results in 73 PB of data, which would today cost upwards of 21 million dollars in cloud storage costs.

So, while we can demonstrate AI in small use cases with a few dozen or even a few hundred AVs there is simply no way we could scale up to millions of vehicles.

The only conclusion is that we need to radically alter the economics of storage if AI is to be more than a curiosity. That means focusing on storage as a separate and distinct part of the cloud, rather than as a bolt on for cloud computing solutions.

What AI uses is your business bracing for?

Are you prepared for the storage requirements that will make or break the potential value of AI?

The future of AI and the future of the foundation to AI, data, are tied together explicitly.

Are you ready?