Take a Plunge into the Deep: Web 3.0

Did you know that a normal Google request only searches 0.03% of the internet?

In fact, use any of the standard search engines such as Yahoo, Bing or Ask Jeeves and you’ll get the same response; around 0.03% of the web.

Michael K Bergman, an American academic and entrepreneur, is one of the foremost authorities on this other internet. In the late '90s he undertook research to try to work out how large it is. "I remember saying to my staff, 'It's probably two or three times bigger than the regular web,"' he remembers. "But the vastness of the deep web . . . completely took my breath away. We kept turning over rocks and discovering things."

"The deep web is currently 400 to 550 times larger than the commonly defined world wide web," he wrote. "The deep web is the fastest growing category of new information on the internet … The value of deep web content is immeasurable”.

The deep web contains around 590 billion documents; the surface web contains around 1 billion.

So what’s down there? And why haven’t we visited it before? In effect, it’s a private section of server used to share data off the record. Google works by tracking hyperlinks from publicly listed webpages.

These pages are carefully hyperlinked with no link to other, surface web pages. Therefore, unless you know exactly what you’re looking for by typing in the perfect URL (they’re usually several hundred characters long) you can’t access them from a normal internet browser.

The deep web is arranged into huge databases that you can search. But be careful, some of the stuff on there is not nice. Remember Wikileaks? That stuff had been hanging around on the deep web for two years before.

Do you often read in the news about that weird school teacher who was found with 18,000 images of naked children on his computer? You can be sure he didn’t Google search for that; the deep web is a haven for illegal activity.

Many websites are based around the deep web currency; bitcoins or BTC’s. Using BTC’s you can purchase a whole range of different items in a way similar to Amazon or eBay.

There are all the books, music, movies you can possibly imagine, even links to sports betting and trade information, links to international drug markets, prostitution rings, assassin markets, black market products.

Even underground fighting tournaments to the death with very real trained professional fighters. There are guys that train with the best and don’t join the UFC or any other fight league; they really enjoy fighting to the death. These fights happen and some millionaires pay significant money to see them.

Scared yet? The Buzz isn’t going to post any links to the kind of information you’ll need to access the deep web. Safe to say it’s not difficult; there is a very popular web browser that goes by the name of a metallic surface that will give you access with a simple plugin.

Furthermore, a simple Google search will pull up enough information to give you access very quickly. But as we said, be careful. The kinds of people who stalk the deep web are often odd, ept with computers and have the kind of friends you only think exist in James Bond.

So what happened when I took the plunge? First, it’s odd how many of the websites look like they’re from the 1990s; definitely no Flash down here. With the different currency and the plain text it all looks a bit apocalyptic.

Secondly, it became apparent that on a few databases there were clear links to explicit content. For example, beneath a search category of ‘education’ was one for ‘death’.

Within that category I was offered the chance to see a 4 men be mauled by one bear. I ignored it; unlike on Youtube I have a feeling it wasn’t a hoax to get advertising clicks.

That aside, I found myself on some academic databases (yes, they really had .edu on the end!) that were fascinating. There was a whole trove of documents from an educational institution that seemed to be based in Chile. My poor Spanish managed to discern a little about Augusto Pinochet and the Valech Report of 1973.

As an experiment I picked out as many key words as I could and searched for the same document in Google. It was formatted in HTML so Google should have been able to read it easily. After 30 minutes of searching nothing similar came up. Not a decisive survey but interesting nonetheless.

But that’s as far as I went. As exciting as it sounds I don’t particularly want to come into contact with Serbian contract killers or Russian mafia. Although a lot of the literature I read before said that viruses were not a problem, I couldn’t help feeling that standard anti-virus software would be slightly out of its depth down here. Plus I don’t have any anti-virus software. But perhaps what I found most unpleasant about the Deep Web was the sheer scale of it.

On a standard day I visit around 10 websites at most: Facebook, BBC and bloody Moodle being the most. When I stray out of these few I have the comfort of Google to shepherd me in the right direction. On the deep web you are absolutely swarmed with information. For every interesting piece on Pinochet, there’s probably 50,000 pieces on US abuses in Iraq. I saw numerous links to videos of Syria and Pakistan; I watched one and it was so astonishingly horrific that I had to go for a walk.

The deep web is astonishingly good at reminding you of your own insignificance, and of the sheer numbers of sick people and horrible things out there. And what was really depressing is that, among its archaic design, it looked like the future. I think I’ll stick to the shallow web.