Using the internet to harness the wisdom of the crowd
Collective intelligence is a growing trend that seeks to exploit the computational power of millions of users
You have probably done it but maybe you didn’t realise. Or maybe you did it on purpose, but it was a game. What is it? Collective intelligence, or “human computation”, is a growing trend that looks to harness the wisdom of the crowd to solve problems.
Today, enormous computational power is distributed among millions of users, and the internet offers a means to connect it, explains Prof Barry Smyth, professor of computer science at University College Dublin.
One early initiative that looked to harvest that power was “SETI at home”, where people could sign up to run code on their computers during downtime to analyse radio telescope data from space in the search for signals from alien intelligence.
“The hope was that your computer would be the one that would find signs of extraterrestrial life,” says Prof Smyth. “That network of home computers made SETI into one of the most powerful supercomputers on the planet at the time.”
Yet it wasn’t quite collective intelligence because the human didn’t have any input other than signing up. Enter the “CAPTCHA”, a puzzle that presents a series of distorted letters and numbers and asks you to type them into a box. You’ve probably encountered it while buying tickets, and it ensures you are a human rather than a “bot”, because computer algorithms aren’t good at working out the symbols.
“There are literally hundreds of millions being solved every day,” says Prof Smyth. “So people started thinking of ways in which we can harness that ability for other tasks, and that really started collective intelligence as we know it now.”
One initiative, reCAPTCHA, has been helping the likes of Google to improve the accuracy of projects to digitise books and periodicals. The software that converts images of the written word to text isn’t perfect, so reCAPTCHA presents unrecognised words (scanning errors) as a second word for us to decode in our CAPTCHAs.
“So now modern CAPTCHAs present two distorted words, one of which is known to the computer, and the other is a scanning error,” explains Prof Smyth.
“The human must attempt to decipher both to sign up to their free email account or register with their social network. As millions of users decipher these scanning errors, their guesses tend to converge on the correct version of the mis-scanned words.
“Collective intelligence helps to better predict what the word should have been. They were able to farm out tens of millions of scanning errors and were able to scan large amounts of text very quickly.”
See Prof Barry Smyth’s talk on collective intelligence at dublintalks.ie
Sharing results: Smart 'social' search
Prof Barry Smyth is himself involved in bringing more collective and “social” intelligence to search on the Internet. He’s co-founder and chief scientist at HeyStaks, a UCD spin-out that has developed technology to let friends share search results online and across mobile devices.
In practice that means if you are searching for information and your friends have already found something interesting, you can see and build on the fruits of their labour.
“We are trying to figure out a better way to rank search results,” says Smyth. “The results that we are promoting to the top of the results list are essentially the product of the collective intelligence of other people who searched before you.”