Search engine: leveller or tyrant?

In its last issue, dated November 17, the Economist discusses a recently published online paper by Santos Fortunato et al, which argues that search engines have an egalitarian effect on web traffic. This argument runs against a well-known thesis that search engines exploit the power-law nature of the web, which creates a winner-take-all effect, where high traffic sites attract more searches, which in turn generates higher traffic and so on…

Not surprisingly, one of the leading proponents of this thesis and author of a widely publicised paper about the “googlearchy,” criticised Fortunato’s argument for its methodological weaknesses. For some reason, I was unable to find the Hindman’s comments, so I cannot assess their pertinence.

My own view is similar to that of Jacob Nielsen, who argued that specialisation mitigates the power-law effect of search engines. In other words, search engines accentuate the traffic hierarchy in general sites but facilitate access and diversity in specialised sites.

Any one sided and general argument about the relationship between search engines and site traffic ignores the complex nature of the web. It is simultaneously a mass media and a set of specialised channels. In those channels, it is not only the use that it is interactive but also the content. Users construct the content of their sites dynamically through active and repeated search, some of its driven by search engines, some by random links and some by peer group referrals. Also, their use of search is becoming more sophisticated (multiple keywords, proximity and synonym search) and more purposeful. For instance, users of financial websites are no longer content with price quotes and agency press releases. They will look for less accessible information and research about their portfolios, relying on their peers on bulleting boards and blogs for addresses of little-known specialists and tipsters.

Because of this fundamentally segmented nature of the web, neither all the searches nor all the sites visits traffic can be considered as equal. Their value to the user (and to the site owners) differs considerably sometimes dramatically. This is reflected in the value of bids for keywords. While general keywords have become cheap commodities, worth few cents, specialised keywords are often highly valuable to the tune of tens of euros. Any serious research on the impact of search on the traffic should therefore also consider the paid search (in its various forms). Only then, research could be deemed useful and relevant for site designers and owners.