Cutting the crud, The big data hype is nothing more than that – hype!
By Mervyn Mooi, Director at Knowledge Integration Dynamics. (KID)
[Johannesburg, 05 Apr 2012]
What do planks and big data have in common?
Besides the fact that some hard-headed chaps are touting big data as a very important thing, Max Planck, in 1906, coined the phrase 'relative theory', which later became 'theory of relativity', which encompasses two of Einstein's theories. Einstein's connection to big data may be tenuous, but relativity has everything to do with big data.
Big data is so old that it should be taken 'round back and mercifully bulleted, before a lovely eulogy, and a quick cremation.
Big data is so old that it should be taken 'round back and mercifully bulleted, before a lovely eulogy, and a quick cremation. In 1956, IBM made a hard drive capable of storing five million characters, and it weighed a ton. That's around 5Mb, but at a ton, nobody can argue that it's not big data. For the day, the storage capacity was immense. My first PC hard drive was 20Mb, which was simply massive data. Who could possibly want to store that much?
But an entire gang of vendors aren't about to let a little detail get in the way of turning about their flagging fortunes, or reinvigorating paltry profits. They're pepping up that one-eyed, deaf old dog with a new name and punting it to the uninformed or, at best, partially informed.
There are tools that promise simplicity, a means of allowing bankers without IT skills to roll up their sleeves and perform complex data modelling. There are others that add predictive analytics capabilities to big data for marketers. There is cloud-based analytics to analyse advertising data. There are big data integration tools. There are methodologies that promise improved sales predictions.
Nothing new
But none of that is new. A portion of what bankers model will always be complex for the day – it's a relative thing. Predictive analytics was with us long before the first computer. There is already cloud-based everything (patent pending), so why not cloud-based analytics? Extraction, transformation and loading (ETL) has been around in one form or another since two competing database vendors opened their doors. Methodologies exist for many, many things, so why shouldn't this old big data nag get shiny new shoes too?
All of these tools exist to resolve the issue of big data. But what is the “issue” with big data? Essentially, there's a lot of data to sift through and that can be hard. But, if the business can sift the gems from the dirt, then it can steal a march on the competition. Or can it? Is there really a return to be had on the investment in all that sifting?
If companies want to be pragmatic about this, and most executives appreciate pragmatism when margins are wafer thin, then they have to admit that most businesses are having the devil's own job pointing out who understands the core business, let alone those with the knowledge and skills to deal with big data and, God forbid, actually put a team together to develop some practical solutions.
It's all relative
It gets worse, because not only is big data a bit of a black art, but the goalposts are constantly shifting. The post is pegged based on the abilities of the currently available crop of off-the-shelf databases and mining tools to deal with large volumes of data. But what is a large volume? A petabyte is big. An exabyte is bigger. A zetabyte is even bigger than that, and a yottabyte is as far forward as my knowledge extends before it all starts to sound a little like Mongolian houses.
So, do large or medium South African businesses really need to start grappling with the intricacies of big data? Are many companies' databases bigger than an exabyte? The fact of the matter is that big data has always existed and it's not about to go anywhere.
Unless companies are running a couple of zetabyte databases then big data's a catch-phrase that can happily be ignored. Companies don't need to fork out for bleeding-edge technology and find the quacks to drive it. The short answer is: no.