Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Do natural language search engines really understand what users want? A comparative study on three natural language search engines and Google

Do natural language search engines really understand what users want? A comparative study on... Purpose – The main purpose of this research is to determine whether the performance of natural language (NL) search engines in retrieving exact answers to the NL queries differs from that of keyword searching search engines. Design/methodology/approach – A total of 40 natural language queries were posed to Google and three NL search engines: Ask.com, Hakia and Bing. The first results pages were compared in terms of retrieving exact answer documents and whether they were at the top of the retrieved results, and the precision of exact answer and relevant documents. Findings – Ask.com retrieved exact answer document descriptions at the top of the results list in 60 percent of searches, which was better than the other search engines, but the mean value of the number of exact answer top list documents for three NL search engines (20.67) was a little less than Google's (21). There was no significant difference between the precision for Google and three NL search engines in retrieving exact answer documents for NL queries. Practical implications – The results imply that all NL and keyword searching search engines studied in this research mostly employ similar techniques using keywords of the NL queries, which is far from semantic searching and understanding what the user wants in searching with NL queries. Originality/value – The results shed light into the claims of NL search engines regarding semantic searching of NL queries. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Online Information Review Emerald Publishing

Do natural language search engines really understand what users want? A comparative study on three natural language search engines and Google

Nadjla Hariri
Online Information Review , Volume 37 (2): 17 – Apr 12, 2013
Download PDF
17 pages

Loading next page...
 
/lp/emerald-publishing/do-natural-language-search-engines-really-understand-what-users-want-a-2aRHgqO5qc
Publisher
Emerald Publishing
Copyright
Copyright © 2013 Emerald Group Publishing Limited. All rights reserved.
ISSN
1468-4527
DOI
10.1108/OIR-12-2011-0210
Publisher site
See Article on Publisher Site

Abstract

Purpose – The main purpose of this research is to determine whether the performance of natural language (NL) search engines in retrieving exact answers to the NL queries differs from that of keyword searching search engines. Design/methodology/approach – A total of 40 natural language queries were posed to Google and three NL search engines: Ask.com, Hakia and Bing. The first results pages were compared in terms of retrieving exact answer documents and whether they were at the top of the retrieved results, and the precision of exact answer and relevant documents. Findings – Ask.com retrieved exact answer document descriptions at the top of the results list in 60 percent of searches, which was better than the other search engines, but the mean value of the number of exact answer top list documents for three NL search engines (20.67) was a little less than Google's (21). There was no significant difference between the precision for Google and three NL search engines in retrieving exact answer documents for NL queries. Practical implications – The results imply that all NL and keyword searching search engines studied in this research mostly employ similar techniques using keywords of the NL queries, which is far from semantic searching and understanding what the user wants in searching with NL queries. Originality/value – The results shed light into the claims of NL search engines regarding semantic searching of NL queries.

Journal

Online Information Review – Emerald Publishing

Published: Apr 12, 2013

Keywords: Natural language searching; Keyword searching; Ask.com; Bing; Hakia; Google; Searching; Language; Search engines

References

  • Biomedical question answering: a survey
    Athenikosa, S.J.; Han, H.
  • Web search engines for children: a comparative study and performance evaluation of Yahooligans!, Ask.com Jeeves for kids, and Super Snooper
    Bilal, D.
  • ComScore releases may 2011 US search engine rankings
    ComSCORE
  • Performance evaluation and comparison of the five most used search engines in retrieving web resources
    Deka, S.K.; Lahkar, N.
  • The promise of natural language search
    Fox, V.
  • Semantic search beyond RDF – SemTech 2009 video
    Franzon, E.
  • ComScore April 2011 search engine market share
    Goodwin, D.
  • Overview
    Hakia
  • Natural language question answering: the view from here
    Hirschman, L.; Gaizauskas, R.
  • Towards a medical question‐answering system: a feasibility study
    Jacquemart, P.; Zweigenbaum, P.
  • Comparative evaluation of web search engines in health information retrieval
    Lopes, C.T.; Ribeiro, C.
  • Natural language search not a priority for google
    Mccormick, J.
  • Bing aims to reinvent search
    Meredith, L.
  • Question‐answering systems as efficient sources of terminological information: an evaluation
    Olvera‐Lobo, M.D.; Gutiérrez‐Artacho, J.
  • Performance of question‐based vs keyword‐based search engines and effect of web user characteristics on search engine performance
    Ozmutlu, S.
  • Are people asking questions of general web search engines?
    Ozmutlu, S.; Ozmutlu, H.C.; Spink, A.
  • Getting answers to natural language questions on the web
    Radev, D.R.; Libner, K.; Fan, W.
  • Web searching across languages: preference and behavior of bilingual academic users in Korea
    Rieh, H.Y.; Rieh, S.Y.
  • Putting google scholar to the test: a preliminary study
    Robinson, M.L.; Wusteman, J.
  • Automatic performance evaluation of web search engines using judgments of metasearch engines
    Sadeghi, H.
  • Search and retrieval using natural language technology
    Schabes, Y.
  • Comparing Google to a digital reference service for answering factual and topical requests by keyword and question queries
    Vakkari, P.
  • Cognitive evaluation of four online search engines for answering denitional questions posed by physicians
    Yu, H.; Kaufman, D.A.