‘Secret sauce’ for web image searches
Searching for images or video could be about to get a lot easier
The first element – his “secret sauce”, he says – is that they have devised a way of swiftly analysing a picture at pixel level to obtain very detailed visual descriptors, and then have that data packaged up in very compact form so that it can be quickly understood by a computer.
They then use “a simple trick but it works amazingly well”. They use their visual analyser to look at the images that come back with the page returns of a normal image search on a text-based query to Google or Bing. It learns from the pictures associated with the top site returns, which images are most likely to be relevant.
Now, it can recognise with a high degree of accuracy, other relevant images. “Our system does this in the background so the user doesn’t even notice it – it takes say the top 100 images and it uses them as training examples to learn on the fly a visual model of how Ferrari Formula One cars look, and it does it super efficiently, in a 10th of a second.”
Then, their search system reranks the pages returned in a regular search on Google or Bing. “Rather than searching for all the text, all the images on the web for every single query, which would be prohibitive, we say, look, traditional web search engines are already quite accurate so let’s leverage on that.”
The final results are a blend of the accuracy of a Google or Bing text query, and his team’s system’s ability to pinpoint sites with the very best images.
“I am only looking at documents that are already ranked high by good, accurate document search engines, so I already have removed all the [potential] silly mistakes and now am also working with this image recognition system to boost accuracy.”
Torresani hopes the technique might become the basis for a company. He already has a track record as an entrepreneur – as a graduate student at Stanford University he and several other graduates set up Like.com, a website that also coupled image recognition with search to enable people to find high-end products such as jewellery, shoes and handbags online. The company was sold to Google in 2010.
He would like to stay involved for a couple of years but says he prefers having one foot in industry, and one in research and academics.
“I have a love and hate relationship with industry. I like the fact that gives you very nice concrete problems to work on, problems particularly where you can make a big impact. But at same time I have also felt a bit constrained by the timelines and the priorities a company has. So I have found the best for me is to work in research, but to spend time in industry.”