Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

The development of a method for the evaluation of interactive information retrieval systems

The development of a method for the evaluation of interactive information retrieval systems The paper describes the ideas and assumptions underlying the development of a new method for the evaluation and testing of interactive information retrieval (IR) systems, and reports on the initial tests of the proposed method. The method is designed to collect different types of empirical data, i.e. cognitive data as well as traditional systems performance data. The method is based on the novel concept of a ‘simulated work task situation’ or scenario and the involvement of real end users. The method is also based on a mixture of simulated and real information needs, and involves a group of test persons as well as assessments made by individual panel members. The relevance assessments are made with reference to the concepts of topical as well as situational relevance. The method takes into account the dynamic nature of information needs which are assumed to develop over time for the same user, a variability which is presumed to be strongly connected to the processes of relevance assessment. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Documentation Emerald Publishing

The development of a method for the evaluation of interactive information retrieval systems

Journal of Documentation , Volume 53 (3): 26 – Aug 1, 1997

Loading next page...
 
/lp/emerald-publishing/the-development-of-a-method-for-the-evaluation-of-interactive-ZdrccM7xX0
Publisher
Emerald Publishing
Copyright
Copyright © 1997 MCB UP Ltd. All rights reserved.
ISSN
0022-0418
DOI
10.1108/EUM0000000007198
Publisher site
See Article on Publisher Site

Abstract

The paper describes the ideas and assumptions underlying the development of a new method for the evaluation and testing of interactive information retrieval (IR) systems, and reports on the initial tests of the proposed method. The method is designed to collect different types of empirical data, i.e. cognitive data as well as traditional systems performance data. The method is based on the novel concept of a ‘simulated work task situation’ or scenario and the involvement of real end users. The method is also based on a mixture of simulated and real information needs, and involves a group of test persons as well as assessments made by individual panel members. The relevance assessments are made with reference to the concepts of topical as well as situational relevance. The method takes into account the dynamic nature of information needs which are assumed to develop over time for the same user, a variability which is presumed to be strongly connected to the processes of relevance assessment.

Journal

Journal of DocumentationEmerald Publishing

Published: Aug 1, 1997

Keywords: Interaction; Information retrieval; Performance levels

There are no references for this article.