Keeping an eye on big data

Fri, May 3, 2013, 05:00

Billions of shades of grey. That’s one way to think about the phenomenon called big data, our propensity to generate and store astonishingly large amounts of information in digital formats (a zettabyte, or a billion billion megabytes, last year), and our ever more refined and revealing ways of searching and interpreting it. Computers used to be limited to looking for answers and performing tasks that were essentially either black or white. But, thanks to ever-dropping costs, and ever increasing capabilities, that’s all changed. Sophisticated computers and software are no longer confined to responding in black or white, but explore and interpret billions of shades of grey. Within all that data, most of it raw and “unstructured”, coming from mobiles, computers and other devices, sensors, webpages and more, computers search and find patterns and anomalies that can bring hidden insights.

Big data creates fantastic opportunities in areas such as the sciences, social and medical research, healthcare, government services, financial markets and customer care. We can pinpoint genes associated with specific diseases. We can improve public transport and management of electricity grids. On a more prosaic level, retailers can offer us more of what we want, thanks to tracking us through loyalty cards or our online shopping habits, offering us coupons and other perquisites. And the Government hopes big data will have a big upside for Ireland, providing new jobs in a growing area.

But big data has its less benign, even sinister, aspects. Commercial harvesting of our personal data remains controversial, often done without our full awareness or complete understanding. Inadvertently, we are turning into valuable commodities for companies that want to either sell or buy the insights our personal information brings. Such commercial abuses pale against possible misuses in government, law enforcement and intelligence spheres, where privacy and basic human rights could be threatened.

Large stores of personal data make for interesting snooping, especially when numerous databases are blended together. This can facilitate nosy privacy infringements but also, revelations and misinterpretation that can devastate lives. Under the banner of fighting terrorism and crime, many states worldwide are enacting laws that enable its agencies to gain unprecedented access to citizen data -- stored for increasingly long periods of time -- with few restrictions, and no obligation to reveal what they are doing, or why. This is the bleaker side of big data, requiring responsible national and international regulation and control. Big data offers spectacular benefits for us collectively, but we cannot allow states or commercial organisations to forget that individually, we are far more than just an aggregation of data, to be bought, sold, interpreted and exploited.

Sign In

Forgot Password?

Sign Up

The name that will appear beside your comments.

Have an account? Sign In

Forgot Password?

Please enter your email address so we can send you a link to reset your password.

Sign In or Sign Up

Thank you

You should receive instructions for resetting your password. When you have reset your password, you can Sign In.

Hello, .

Please choose a screen name. This name will appear beside any comments you post. Your screen name should follow the standards set out in our community standards.

Thank you for registering. Please check your email to verify your account.

We reserve the right to remove any content at any time from this Community, including without limitation if it violates the Community Standards. We ask that you report content that you in good faith believe violates the above rules by clicking the Flag link next to the offending comment or by filling out this form. New comments are only accepted for 3 days from the date of publication.