Big Data Is Old News

By: Yves de Montcheuil, VP of marketing at Talend

One can wonder, why all this noise around big data? After all, we (as human beings, and as organizations we belong to) have been collecting massive amounts of data since the origin of times. Of course, the advent of the printing press, then (a few centuries later) computers, have made it easier to collect and store data and have redefined the term “massive”.

Recently, the total volume of data in the world has exceeded a zettabyte (that’s 10^21 bytes, the rough equivalent of 100 billion iPods). Data growth is not linear: it has taken us a few thousand years to get there, but the next step – the yottabyte – is probably much closer on the horizon.

So we’ve been amassing big data for all this time. Why did we do it?  Because of regulations, of audit requirements. Because we may need it one day. Because storage is cheap. In any case, having big data is old news for most organizations.

However, most organizations haven’t been using their big data. Notable exceptions exist, of course. For example:

–        Financial institutions have been crunching big data, trying to detect fraud, since the dawn of times. Computers helped a lot and global communication networks introduced real-time into the fight. But basically, it comes down to designing complex mathematical models and having lots of processing power – both being expensive. One can be sure the ROI is real though!

–        Retail chains have been tracking consumer habits to deliver cross promotions. It’s a quick win: process purchasing big data, print coupons at register to get people back in the stores. The algorithms used there are not rocket science, but it pays off.

–        For airlines & hotels chains, yield management is nothing new: ticket and room prices fluctuate based on historical trends and current bookings. The models used are hardened and well known, and these companies are used to having big historical data and powerful computers available anyway.

All of these – and more – are big data applications. And clearly, they are not new. For example, the “beer and diapers correlation” urban legend has been traced back to as early as 1992, and blockbuster movie Catch Me If You Can depicted check fraud tracking that was taking place in the 1960s.

But few companies can hire Ph.Ds. in statistical analysis, and spend millions on super computers or (more recently) data warehousing beasts.  Unless these companies had made the strategic decision that data was their primary asset, and that they needed to harvest it.

Enter Google, Yahoo, Facebook, eBay, Twitter and others. These companies invested heavily into big data technologies such as Hadoop. And what’s more, they did not just use this technology – they invented it.  Why?  More than likely, because no existing solution on the market was deemed suitable for their needs.  And the open source collaborative model let them create Hadoop – for their own benefit, but also for the benefit of all users.

All of a sudden, with a new technology stack, the big data that organizations have been amassing for years can be used. It can be harvested. It can be analyzed. It can be leveraged for the business.

And big data is no longer old news.

Yves de Montcheuil is the Vice President of Marketing at Talend, the recognized leader in open source integration. Yves holds a master’s degree in electrical engineering and computer science and has 20 years of experience in software product management, product marketing and corporate marketing. He is also a presenter, author, blogger, social media enthusiast, and can be followed on Twitter: @ydemontcheuil.