The darknet rises: unravelling the web’s underbelly

The moment when a team of FBI agents raided a San Francisco library and arrested an innocuous-looking 29-year-old man who had been working on his laptop might not sound like a key time in law-enforcement history, but with that arrest on Tuesday afternoon the so-called war on drugs took a decisive step into the 21st century.

The man was Ross William Ulbricht, a physics graduate and libertarian activist who had gained notoriety online as the Dread Pirate Roberts. Under that pseudonym, the FBI believes, he set up and ran an infamous online drug bazaar called the Silk Road, a sort of eBay for narcotics where users could order all manner of illegal substances and pay with the virtual crypto-currency, Bitcoin. The Silk Road was a huge business: the FBI claims it had facilitated 1.2 million transactions during its two and a half years in existence, with Ulbricht taking a commission on each sale, earning him $80 million in Bitcoins.

Profiting from drug-dealing was not his only alleged crime: according to claims in court documents, Ulbricht attempted to get hit men to kill two Silk Road associates.

The case of the Silk Road raises many questions: principally, how did Ulbricht apparently manage to act like the Jeff Bezos of the narcotics world? And in a world of increasingly pervasive online surveillance, how did customers even get to the Silk Road without being detected?

To answer those questions, one has to venture into the so-called darknet, the anonymous internet underground that can't be accessed by normal web browsers or mined by search engines. It's a hidden part of the internet that plays host to all sorts of people who value absolute privacy and anonymity, from political dissidents and investigative journalists on the one side to black-market traders, terrorists and child-pornographers on the other.

Ulbricht, the FBI claims, couldn't go to any regular service provider, set up silkroad.com and hope for Google searches for LSD or heroin to bring him business. Ulbricht, according to the bureau, created the Silk Road using an anonymising network service called Tor, which offers secrecy and security both for his servers and for Silk Road users.

Tor stands for the Onion Router, a reference to the layers of encryption and security it offers; it functions as an anonymity network that bounces its users' communications "around a distributed network of relays run by volunteers all around the world", thereby protecting their identity, location and data. It was originally designed by the US Naval Research Laboratory for protecting government communication; it is now developed by a US-based nonprofit; donors include the US state department.

Tor users can browse the regular internet – or “clearnet”, in privacy parlance – securely, but the network can also be used to set up “hidden services”: a server delivering web pages that can’t be accessed by regular browsers, such as Chrome or Internet Explorer, and can be found only by other Tor users.

There are other anonymous networks, such as Freenet or I2P, that use different approaches, but in recent years Tor became so big that it is virtually synonymous with “darknet”.

(As is the fashion in the techno-lexicon, there are some distinct concepts with similar names: a darknet is an anonymised network, such as Tor or Freenet; the “dark internet” refers to that portion of the internet that can no longer be accessed, usually because of technical problems or physical decay of internet infrastructure; and “deep web” refers to the vast amount of data on the internet that can’t be trawled by search engines.)

Networks such as Tor are at the centre of a struggle about the future of online privacy. The scale of it has become clear only in the wake of Edward Snowden’s revelations of vast US and UK internet surveillance. There is an outspoken techno-libertarian element whose distrust of government agencies has been vindicated by the revelations, and anonymising technologies are seen as a key countervailing force to the threat of surveillance.

Tor’s anonymising network is used for vital and legitimate activities, but, given their level of anonymity, darknets are also a natural home for unsavoury activities and other criminal behaviour that can’t be readily discussed in an ordinary forum.

That much was made abundantly clear with the arrest, last month, of Eric Eoin Marques, an Irish-American man who was described as the "largest facilitator of child porn on the planet" by the FBI, which is trying to extradite him to the US from Ireland. Marques ran an Irish-based web-server company called Host Ultra Limited, and allegedly also operated Freedom Hosting, a "hidden services" provider that by some estimates hosted nearly half of the content on the Tor network, including the Silk Road and numerous child pornography sites. After Marques's arrest, Freedom Hosting was shut down in August.

But there is no shortage of opportunists willing to take Marques’s alleged place, just as the Silk Road has any number of imitators on different networks ready to step into the void that might have been left by Ulbricht’s arrest. The darknet, it’s clear, is not going to conform to the rule of law any time soon.