“You can’t handle the truth”

Hurricane Earl was whistling by the East Coast the week prior. Oracle OpenWorld comes up next week where I am sure we will be bombarded with talk about Exadata and other analytical tools. I am thinking ahead to my keynote on data and analytics at Defrag.

Given all that, in the last few book events I have found myself talking about the National Hurricane Center case study in the book just a bit longer.

My voiceover about the NHC goes something like this:

“After trillions invested in analytic technology over the last few years, most businesses blew - badly – their forecast of our “economic hurricane” in 2007-2008. We did not even come close. In contrast, The National Hurricane Center has steadily improved its forecasts. The track forecast error in the 1980s, 48 hours out, was 225 nautical miles. Today, that error is a little less than 100 nautical miles.

Living in Florida, I am glad the NHC keeps improving its forecasts. But I am even more grateful for their reduction in“false positives” – they don’t make me evacuate unless absolutely necessary. Evacuations typically cause chaos – emergency services, panic shopping, and other community disruption. Over the years, the improvements in track forecasts have amounted to hundreds of miles of coastline not evacuated and billions of dollars saved in emergency services. Think of the ROI – how many enterprise analytic projects even attempt to show a quantifiable payback?

To deliver those forecasts, the NHC team takes an invasive approach to collecting a wide variety of external, real - time data.  They use satellites, dropsondes, Hurricane Hunters, ocean buoy based sensors to collect air pressure, humidity, temperature, wind, and plenty of other data points.

Then the information is processed using multiple forecast models – many contradictory -  both for redundancy and for validation. They want models to challenge each other. Errors are a statistical reality. And then each year they audit and publicly report their forecast accuracy.

Finally, it is impressive in how many different formats the NHC reports its data from the simple bulletins radio stations read out to large streams of data that websites like StormPulse use to generate awesome graphics of the hurricanes”

So, here comes the Jack Nicholson admonition. Most enterprises cannot handle the truth about their analytics, nor can vendors about their customer’s analytics about why they missed the “economic hurricane” so badly. Here are some things we need to face up to:

• Most companies don’t seek out primary, real-time data.

Too much business forecasting today is based on internal data or looks at Google for answers or external sources such as Bloomberg and Gartner for its “primary” data. That data has typically being rinsed of most of the original “nutritional value”

• Most enterprises are too impatient to deal with contradictory models

Most enterprises data strategies are designed around “the single version of the truth”. Paul Kedorsky is quoted in the book saying “Why does the drunk look for lost keys under the lamp post even though the keys were dropped far away? Because that’s where the light is. They are looking for confirmation when they should be looking for falsification”. How many enterprises balance contradictory models like the NHC?

• Most enterprises keep looking for the elusive “universal report writer”

Yes, the holy grail is a single tool which will represent structured, unstructured, transaction, balance, matrix, drill down, web and every other form of data increasingly available. How many like the NHC look to present 15-18-20 views of the data?

• Most analysts focus on slice and dice, not decisions. Most vendors sell slice and dice.

David Axson argues in my book we have been focused too much in the last few years on analytical tools, not decisions. Vendors talk about pivot tables and schemas. Most enterprise team are just as bad – they talk about doing a BusinessObjects implementation, or being an Eloqua shop. What does any of that to do with the end-decisions?

• Historical data keeps exploding and is mostly worthless

Howard Dresner is quoted in the book as saying “Nassim Nicholas Taleb of The Black Swan book fame talks about “ highly improbable ” events and how historical data simply cannot help us fully anticipate them.”  In a volatile world, “Black Swans” are increasingly common. Yet we keep archiving petabytes of historical data (often multiple copies, at exorbitant storage costs) then stretching the limits of technology to do in-memory analytics.

• Few enterprises have the right analytical talent

Sure, everyone has Excel analysts and DBAs. How many enterprise have the equivalent of the brave Hurricane Hunters that fly into ferocious storms to collect data points? How about statisticians? Hal Varian, chief economist at Google, has been quoted as saying, “ I keep saying that the sexy job in the next 10 years will be statisticians.” Paul Kedrosky is a bit more poetic “I see a new generation of artists — not just data jocks, but those with the appreciation of the aesthetics of data.”

It would be nice during OOW to hear from customers – and Oracle about its customers – about adjustments they have made to their analytical framework, data sources, talent etc since 2008, not just about more speeds and feeds.

PS: The book has other examples besides the NHC of other impressive analytic applications such as at BestBuy and its customer analytics, Google with Flu Trends. IBM's Many Eyes and several others.