Should the storage “tail” wag the application “dog”?

Two truisms: a) Storage needs are exploding. Private sector data is expected to grow to 27,000 petabytes (million GB each) by 2010. b) Storage costs are coming down nicely – you and I can buy 1TB retail for around $ 100 these days or 10c a GB. And get backup cloud storage from Carbonite for $ 50 a year. And Google gives everyone 6 gb of Gmail storage for free!

Well, one of those statements is not true for corporate America.

10c for you and me, but for corporations TCO of Tier 1 Storage can often be over $ 30 a GB a year. Not kidding. Not just frames and arrays and other hardware. You have storage management software license, software maintenance, extended hardware support, data center costs, labor plus plus. Tier 2 and 3 is much cheaper but still around $ 10 a year, still a very far cry from the one time 10c. Then you have storage needs caused by backups and in many companies, backups of backups. Storage hygiene is awful in many companies. Worsened by recent SOX and other compliance demands.

Time to move to the cloud with amazon or Nirvanix?

Not for Tier 1. Talk to many companies, and they feel clouds are not ready for Tier 2 either - though as economic pressures mount the objections are gradually dropping

Time to hammer down on EMC and IBM and HP and NetApp and outsourcers who run your data centers and your storage? Yes, but just as important is getting users to accept they do not need Tier 1 for many applications.

In other words, we need to make storage considerations a far greater priority in application and data design. And relentlessly use data de-duping tools and have users not attaching big files but pointing to locations on central servers. And thinking a few times before they keep building large business intelligence databases. And pushing application and database vendors to show how storage has featured in their design. But importantly, educate business users on the different tiers of storage and what they get and the chargebacks they could be avoiding.

As this article says “while SNIA conservatively estimated 80% of data did not change after 90 days a survey of 900 mid-sized companies in Europe and US indicated at least 60% of the data did not change after five minutes of life!”

Demand management of storage is becoming even more important than storage supplier management and related price and SLA negotiation.

Unless of course like Google, you have managed to find the Infinite Possibility Drive