Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Detecting and treating errors in tests and surveys

Detecting and treating errors in tests and surveys PurposeSurveys that include skill measures may suffer from additional sources of error compared to those containing questionnaires alone. Examples are distractions such as noise or interruptions of testing sessions, as well as fatigue or lack of motivation to succeed. This paper aims to provide a review of statistical tools based on latent variable modeling approaches extended by explanatory variables that allow detection of survey errors in skill surveys.Design/methodology/approachThis paper reviews psychometric methods for detecting sources of error in cognitive assessments and questionnaires. Aside from traditional item responses, new sources of data in computer-based assessment are available – timing data from the Programme for the International Assessment of Adult Competencies (PIAAC) and data from questionnaires – to help detect survey errors.FindingsSome unexpected results are reported. Respondents who tend to use response sets have lower expected values on PIAAC literacy scales, even after controlling for scores on the skill-use scale that was used to derive the response tendency.Originality/valueThe use of new sources of data, such as timing and log-file or process data information, provides new avenues to detect response errors. It demonstrates that large data collections need to better utilize available information and that integration of assessment, modeling and substantive theory needs to be taken more seriously. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Quality Assurance in Education Emerald Publishing

Detecting and treating errors in tests and surveys

von Davier, Matthias
Quality Assurance in Education , Volume 26 (2): 20 – Apr 3, 2018
Download PDF
20 pages

Loading next page...
 
/lp/emerald-publishing/detecting-and-treating-errors-in-tests-and-surveys-kXhgmWEZbA
Publisher
Emerald Publishing
Copyright
Copyright © Emerald Group Publishing Limited
ISSN
0968-4883
DOI
10.1108/QAE-07-2017-0036
Publisher site
See Article on Publisher Site

Abstract

PurposeSurveys that include skill measures may suffer from additional sources of error compared to those containing questionnaires alone. Examples are distractions such as noise or interruptions of testing sessions, as well as fatigue or lack of motivation to succeed. This paper aims to provide a review of statistical tools based on latent variable modeling approaches extended by explanatory variables that allow detection of survey errors in skill surveys.Design/methodology/approachThis paper reviews psychometric methods for detecting sources of error in cognitive assessments and questionnaires. Aside from traditional item responses, new sources of data in computer-based assessment are available – timing data from the Programme for the International Assessment of Adult Competencies (PIAAC) and data from questionnaires – to help detect survey errors.FindingsSome unexpected results are reported. Respondents who tend to use response sets have lower expected values on PIAAC literacy scales, even after controlling for scores on the skill-use scale that was used to derive the response tendency.Originality/valueThe use of new sources of data, such as timing and log-file or process data information, provides new avenues to detect response errors. It demonstrates that large data collections need to better utilize available information and that integration of assessment, modeling and substantive theory needs to be taken more seriously.

Journal

Quality Assurance in Education – Emerald Publishing

Published: Apr 3, 2018

There are no references for this article.