Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

The retrieval effectiveness of web search engines: considering results descriptions

The retrieval effectiveness of web search engines: considering results descriptions Purpose – The purpose of this paper is to compare five major web search engines (Google, Yahoo, MSN, Ask.com, and Seekport) for their retrieval effectiveness, taking into account not only the results, but also the results descriptions. Design/methodology/approach – The study uses real‐life queries. Results are made anonymous and are randomized. Results are judged by the persons posing the original queries. Findings – The two major search engines, Google and Yahoo, perform best, and there are no significant differences between them. Google delivers significantly more relevant result descriptions than any other search engine. This could be one reason for users perceiving this engine as superior. Research limitations/implications – The study is based on a user model where the user takes into account a certain amount of results rather systematically. This may not be the case in real life. Practical implications – The paper implies that search engines should focus on relevant descriptions. Searchers are advised to use other search engines in addition to Google. Originality/value – This is the first major study comparing results and descriptions systematically and proposes new retrieval measures to take into account results descriptions. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Documentation Emerald Publishing

The retrieval effectiveness of web search engines: considering results descriptions

Dirk Lewandowski
Journal of Documentation , Volume 64 (6): 23 – Oct 17, 2008
Download PDF
23 pages

Loading next page...
 
/lp/emerald-publishing/the-retrieval-effectiveness-of-web-search-engines-considering-results-5bFQwUZf4i
Publisher
Emerald Publishing
Copyright
Copyright © 2008 Emerald Group Publishing Limited. All rights reserved.
ISSN
0022-0418
DOI
10.1108/00220410810912451
Publisher site
See Article on Publisher Site

Abstract

Purpose – The purpose of this paper is to compare five major web search engines (Google, Yahoo, MSN, Ask.com, and Seekport) for their retrieval effectiveness, taking into account not only the results, but also the results descriptions. Design/methodology/approach – The study uses real‐life queries. Results are made anonymous and are randomized. Results are judged by the persons posing the original queries. Findings – The two major search engines, Google and Yahoo, perform best, and there are no significant differences between them. Google delivers significantly more relevant result descriptions than any other search engine. This could be one reason for users perceiving this engine as superior. Research limitations/implications – The study is based on a user model where the user takes into account a certain amount of results rather systematically. This may not be the case in real life. Practical implications – The paper implies that search engines should focus on relevant descriptions. Searchers are advised to use other search engines in addition to Google. Originality/value – This is the first major study comparing results and descriptions systematically and proposes new retrieval measures to take into account results descriptions.

Journal

Journal of Documentation – Emerald Publishing

Published: Oct 17, 2008

Keywords: Wordwide web; Search engines; Information retrieval

References

  • The concept of relevance in IR
    Borlund, P.
  • A taxonomy of web search
    Broder, A.
  • Evaluating evaluation measure stability
    Buckley, C.; Voorhees, E.M.
  • US search engine rankings, April 2007” Search Engine Watch
    Burns, E.
  • Search engines for the world wide web: a comparative study and evaluation methodology
    Chu, H.; Rosenthal, M.
  • Relevance criteria identified by health information users during web searches
    Crystal, A.; Greenberg, J.
  • Measuring retrieval effectiveness: a new proposal and a first experimental validation
    Della Mea, V.; Mizzaro, S.
  • A comparative study of web search service performance
    Ding, W.; Marchionini, G.
  • Evaluation deutscher Web‐Suchwerkzeuge
    Dresel, R.; Hörnig, D.; Kaluza, H.; Peter, A.; Roßmann, N.; Sieber, W.
  • Finding information on the world wide web: the retrieval effectiveness of search engines
    Gordon, M.; Pathak, P.
  • Eye‐tracking analysis of user behavior in WWW search
    Granka, L.A.; Joachims, T.; Gay, G.
  • Evaluation of three German search engines: Altavista.de, Google.de and Lycos.de
    Griesbaum, J.
  • Deutsche Suchmaschinen im Vergleich: AltaVista.de, Fireball.de und Lycos.de
    Griesbaum, J.; Rittberger, M.; Bekavac, B.
  • Tauglichkeit von Suchmaschinen für deutschsprachige Anfragen
    Guggenheim, E.; Bar‐Ilan, J.
  • The indexable web is more than 11.5 billion pages
    Gulli, A.; Signorini, A.
  • The very large collection and web tracks
    Hawking, D.; Craswell, N.
  • Measuring search engine quality
    Hawking, D.; Craswell, N.; Bailey, P.; Griffiths, K.
  • Options for presenting search results
    Jacsó, P.
  • Cumulated gain‐based evaluation of IR techniques
    Järvelin, K.; Kekäiläinen, J.
  • Web retrieval systems and the Greek language: do they have an understanding?
    Lazarinis, F.
  • First 20 precision among world wide web search services (search engines)
    Leighton, H.V.; Srivastava, J.
  • Query types and search topics of German web search engine users
    Lewandowski, D.
  • Mit welchen Kennzahlen lässt sich die Qualität von Suchmaschinen messen?
    Lewandowski, D.
  • Trefferpräsentation in Web‐Suchmaschine
    Lewandowski, D.
  • Web Searching: a quality measurement perspective
    Lewandowski, D.; Höchstötter, N.
  • The freshness of web search engine databases
    Lewandowski, D.; Wahlig, H.; Meyer‐Bautor, G.
  • Wegweiser im Netz: qualität und Nutzung von Suchmaschinen
    Machill, M.; Neuberger, C.; Schweiger, W.; Wirth, W.
  • Relevance: the whole history
    Mizzaro, S.
  • Understanding user goals in web search
    Rose, D.E.; Levinson, D.
  • A general classification of (search) queries and terms
    Schmidt‐Maenz, N.; Koch, M.
  • Searching the web: the public and their queries
    Spink, A.; Wolfram, D.; Jansen, M.; Saracevic, T.
  • Internet‐Suchwerkzeuge im Vergleich
    Stock, M.; Stock, W.G.
  • Hitwise search engine ratings
    Sullivan, D.
  • Major search engines and directories
    Sullivan, D.
  • The pragmatics of information retrieval experimentation, revisited
    Tague‐Sucliffe, J.
  • New measurements for search engine evaluation proposed and tested
    Vaughan, L.
  • Search engine coverage bias: evidence and possible causes
    Vaughan, L.; Thelwall, M.
  • A comparative study of six search engines
    Véronis, J.