Big data. Big opportunities!

Back in February, I flew into Aspen, Colorado, to speak at the leadership meeting for a major consulting firm. My topic? An extremely customized keynote on the trends and opportunities that are unfolding with the new era of analytics and what has come to be known as “big data.”

There was a tremendous amount of customized research for the event; it’s a big topic, so to speak. After that, I ended up focusing my upcoming May CAMagazine column on some of the ideas and concepts that I covered.

Cashing in on Big Data
CAMagazine, May 2012 

Have you ever noticed how, all of a sudden, a new phrase enters our lexicon and becomes the next big thing? So it is with “big data.” If you

haven’t heard this term yet, you will. What is it?

By analyzing and tying together massive amounts of information, we can change the way we conduct busi- ness, manage healthcare, work in the world of agriculture or manage energy consumption.

As everything around us plugs into the Internet — thermostats, fridges, washers, dryers, industrial HVAC equipment — we are generating huge amounts of information. Companies can examine this information through powerful analytical software to look for unique patterns and insight, which might then help to drive key business decisions.

Consider the energy industry, for example. The NEST thermostat, created by one of the original iPad designers, can recognize you when you walk in the room and adjust the heat to your favourite temperature. It’s also plugged into the Internet — and that’s where the potential for big data comes in.

Imagine the possibilities for an energy company to easily poll how millions of customers are using energy through such thermostats — and to then link that data with a deep analysis of upcoming weather patterns. Suddenly, it can predict and manage upcoming spikes in energy consumption, which could have a direct impact on the purchase of natural gas or other sources of energy. Through this analysis, it can do a far better job of managing its costs.

IBM expects big data to be a large driver of future growth, noting that “there are upwards of a trillion inter- connected and intelligent objects and organisms .. all of this is generating vast stores of information. It is estimated that there will be 44 times as much data and content coming over the next decade …reaching 35 zettabytes in 2020. A zettabyte is a 1 followed by 21 zeros.”

In the field of agriculture, people have been talking for years about the concept of “precision agriculture.” A farmer would use a system which knows, for each particular square yard of ground, how much fertilizer and seed to apply, based on real-time data insight and GPS information.

That world is arriving pretty quickly. Kenna, a data analytics firm in Mississauga, Ont., has put together a system which cross references customer purchase data to weather patterns to real-time tractor position based on GPS to do just that, with the goal of helping a farmer get the best yield possible.

Las Vegas is said to be using the idea of deep analytical research to offer a free lunch to a slot payer who is marginally closer to a payout — thereby reducing it’s risk.

Of course, there are big opportunities in big data. The data and analytics market is already worth an estimated $64 billion according to global management consultants McKinsey & Co. The firm also predicts there will be a shortfall of 140,000 to 190,000 graduates with deep analytical talent by 2018. Similarly, a survey by data giant EMC suggests that 65% of data professionals expect a significant shortage in “big data” skills expertise in just five years.

Accountants are, of course, supposed to be the folks with deep analytical insight. Do we have the capability to step up to the big opportunity that is unfolding here?