Filtering Profits

Businesses must find ways to transform ‘big data’ into big money, or risk falling behind their competitors.

– The Wall Street Journal, Technology Report

Big data is a term bandied around the technology sector with great ease, but little precision. While exact definitions are hard to come by, what is clear is that the ability to gather, process and store huge amounts of data is unparalleled. The sheer quantity of information and the speed at which it is consumed means conventional methods of handling it are no longer sufficient.

According to analysts IDC, the amount of digital information created and replicated rose by 62% in 2010 to nearly 800,000 petabytes, which, IDC says, would fill a stack of DVDs reaching from the earth to the moon and back. By 2020, that pile of DVDs would stretch half way to Mars.

In its recent report, “Big data: The next frontier for innovation, competition and productivity”, McKinsey Global Institute estimated that a retailer using big data could increase its operating margin by more than 60%. The U.S. could reduce its healthcare expenditure by 8% and government administrators in Europe could save more than €100 billion ($143 billion).

The definition MGI uses for big data is deliberately vague and not based on a specific number. It refers instead to sets of data which are too large for current conventional database tools to capture, store, manage and analyze. It also takes account of the differences between sectors in the type of data and software available. …. Report