Nate Silver, the election forecaster was on CNN this morning. Try as hard as he might, Fareed Zakaria could not get very specific answers about November from Nate. I found that segment part frustrating because of Nate’s evasiveness, part fascinating because of Nate’s modesty in spite of all the data and technology he has access to.
It reminded me of my interviews with James Franklin of the National Hurricane Center in 2009 as part of my research for The New Polymath. Here is an agency which has consistently improved storm forecasts. The track forecast error in the 1980s, 48 hours out, was 225 nautical miles. Today, that error is a little less than 100 nautical miles.
In contrast, every institution around the world had missed the 2008 “economic hurricane”. Yet James was amazingly modest and talked about countless data sources, multiple forecasting models and annual audits to keep improving forecasts.
Fittingly, in his upcoming book and his NYT blog, Nate says “weather forecasters are the role models”.
I think both Nate and James should be role models for vendors of analytic technology.
I would love to hear an IBM or SAP salesman say “Don’t just use our tools. Supplement them with FICO’s, SaS’s and Oracle’s to create multiple models ”
I would love all of them to say “Audit your results from our tools and allow us to share them with other customers so industry averages can improve”
Think of how much the NHC spends on satellites, sensors on ocean based buoys, the Hurricane hunters to gather all kinds of speed, temperature, humidity and other data points. If every business similarly invested more in primary data rather than just getting cleansed data from Bloomberg or Gartner, think how differently they would look at forecasting.
So I would also love analytic vendors to say “Spend most of your money on primary data sources. If that data is incomplete or inaccurate our tools cannot help you much”.
Chances of all that happening? Gartner probability of 0.2
