Peter Marklund's Home

Searchenginewatch is In Search Of The Relevancy Figure for web search engines. As suggested by their article no quality or relevancy metrics exists for search engines so far. Instead relevance of search results is judged by anecdotes or at best by most peoples subjective experience of it.

The public and media give attention primarily to the number of documents that search engines index. Inktomi (MSN) and Google are currently at about 3 billion, and Inktomi claims that their index is updated at least once every two weeks.

However, consider that just as important as the size of the document base is the relevancy of the documents actually returned during a search. Relevance should be measured relative to the goal of the person performing the search, i.e. how useful are those documents? Note that it is often dificult to know what that goal is, i.e. someone using the words "dvd player" might be looking to buy a dvd player or to learn more about dvd players.

Some of the ad hoc relevancy tests mentioned are searching for a person or company name and checking for the homepage of that person or company. Searchenginewatch performed a relevancy test where they selected a number of high quality sites in different areas and then performed various searches and checked whether those sites were easily found.

The argument here is that the industry needs accepted standards for performing relevancy tests. Also, the whole premise of SearchEngineWatch.com is that we need to monitor the behaviour of search engines and make sure that they behave. This may make sense given the importance and power of search engines today. Google sometimes chooses not to list certain sites, and as they say, if you don't show up on Google, you don't exist.