Google Trends

Analysing what we look for on the web can offer a remarkable insight into our anxieties and enthusiasms.

Four years ago, the writer and internet entrepreneur John Battelle had a sudden epiphany - the kind of moment that leaves you giddy, teetering on a conceptual cliff, as you contemplate its full ramifications. Battelle had already been preaching the transformative power of the internet for some time. But now his thinking turned to the millions of web searches that people were conducting around the world each day, using Google and a handful of other sites.

As people searched, he realised, they were inadvertently leaving a trail - a gargantuan historical archive of whatever was on the world's mind at a particular time, which remained stored on the central computers of firms such as Google, Yahoo and Microsoft.

Battelle called it "the database of intentions". "This information represents, in aggregate form, a placeholder for the intentions of humankind," he wrote breathlessly on his blog. What had been created was "a massive database of desires, needs, wants, and likes that can [be] archived, tracked, and exploited to all sorts of ends. Such a beast has never before existed in the history of culture ... this artefact can tell us extraordinary things about who we are and what we want as a culture. And it has the potential to be abused in equally extraordinary fashion."

Since then, the database of intentions has grown dizzyingly: in one month alone during 2007, the number of searches conducted using the five leading sites reached 9.4bn. But until recently we could only glimpse at the secrets it contained - for example, when the internet company AOL mistakenly released information on what 658,000 of its members had been searching for. (No names were released, but individual users' search histories included eyebrow-raising anomalies: "replica louis vuitton bag ... how to secretly poison your ex ... how to colour hair with clairol professional ...") And in 2005 an obscure internet forum about video technology became deluged with messages after it became the top result for Google searches on the phrase "I am lonely", which thousands of people, it turned out, were typing every day.

In contrast to those brief glimpses, the graphs on these pages provide a radically broader and deeper view of what people are searching for online, and what that might mean. They were generated using Google Trends, an experimental service that uses aggregated data from Google search results to compare the numbers of people searching for different words and phrases over time, from 2004 to the present. This enables you to track, for example, how interest in Tony Blair was gradually superseded by interest in Gordon Brown over the course of 2007, or how Amy Winehouse overtook Lily Allen in the notoriety stakes, or how the awareness of the term global warming has grown down the years.

Searches are also broken down geographically. So you can discover, for example, that of all British towns and cities Luton has the highest proportion of searches for "sex", followed by Milton Keynes - although whether this means their inhabitants are unusually liberated, or desperate, or just bored, is a matter for speculation. (The phrase "I am bored", by the way, forms a larger proportion of Google searches in Sheffield than anywhere else.) Google Trends is free to use, at google.com/trends, and it is easy to waste far too much time playing with it.

Indications of scale

This form of measurement is far from perfect. The results are only approximate, and Google will not reveal the actual numbers of searches - presumably because that information is gold-dust for internet advertisers, and it intends to make them pay for it.

As a result, the graphs come with no indication of scale. They merely show the volume of searches for a particular term as a proportion of all searches on Google, which makes it impossible to tell whether a sudden surge in searches for, say, Paris Hilton represents a leap of several thousand or several million. So the graphs here are only impressionistic.

Even with those limitations, though, they point to the extraordinary amount of information that is waiting to be mined from internet search data: as Battelle rightly suspected, these charts help show the shifting concerns of an entire culture.

Sometimes, people's interests are driven fairly obviously by the news agenda: when the Spice Girls announce a reunion, there's an immediate rush to find out more about them. Other results are strikingly seasonal: not too surprisingly, people seem to go shopping online for coats in winter, and for sandals in summer.

But the most fascinating possibility is that search data might help to predict behaviour. After all, when we search online for a certain brand of stereo system rather than another, we are surely indicating that it's more likely we will buy that brand.

Perhaps we search for a political candidate's name when we are thinking about voting for him or her, and maybe we even search for "stock market crash" or "recession" just before we start pulling out of our investments. This information could clearly be useful to a savvy marketer - it's already how Google decides which ads to show on its search results pages - or to a political campaign manager.

Marissa Mayer, a Google vice-president, argues that Google Trends correctly "predicted" George Bush's victory over John Kerry in the 2004 election: our graph shows that Bush maintained his lead over his rival, in terms of search volumes, even when polls suggested the race was on a razor's edge.

The same approach leads to the prediction that Hillary Clinton will beat Barack Obama in the race for the Democratic presidential nomination in 2008; before too long, we'll be able to verify that. But then again, as the graph here shows, the conservative Republican Ron Paul ranks above them both. This is the result of an internet cult around Paul that has not been reflected in regular opinion polls, which may demonstrate the limits of treating web searches as if they were representative of an entire population's opinions - unless, of course, he ends up winning.

Unsettling

There is something very unsettling about all this. We do not like to think that other people can see inside our brains, even on a collective level, and many of us will have conducted hundreds of thousands of web searches in recent years without ever giving a thought to where all that data was going. But though so much seems ephemeral in the age of the web, nothing really is. It is all stored somewhere. The internet never forgets.