Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 7-Day Trial for You or Your Team.

Learn More →

Estimating the Consistency and Accuracy of Classifications Based on Test Scores

Estimating the Consistency and Accuracy of Classifications Based on Test Scores This article presents a method for estimating the accuracy and consistency of classifications based on test scores. The scores can be produced by any scoring method, including a weighted composite. The estimates use data from a single form. The reliability of the score is used to estimate effective test length in terms of discrete items. The true‐score distribution is estimated by fitting a 4‐parameter beta model. The conditional distribution of scores on an alternate form, given the true score, is estimated from a binomial distribution based on the estimated effective test length. Agreement between classifications on alternate forms is estimated by assuming conditional independence, given the true score. Evaluation of the method showed estimates to be within 1 percentage point of the actual values in most cases. Estimates of decision accuracy and decision consistency statistics were only slightly affected by changes in specified minimum and maximum possible scores. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of Educational Measurement Wiley

Estimating the Consistency and Accuracy of Classifications Based on Test Scores

Download PDF
19 pages

Loading next page...
 
/lp/wiley/estimating-the-consistency-and-accuracy-of-classifications-based-on-jFpOIATKad

References (8)

Publisher
Wiley
Copyright
Copyright © 1995 Wiley Subscription Services, Inc., A Wiley Company
ISSN
0022-0655
eISSN
1745-3984
DOI
10.1111/j.1745-3984.1995.tb00462.x
Publisher site
See Article on Publisher Site

Abstract

This article presents a method for estimating the accuracy and consistency of classifications based on test scores. The scores can be produced by any scoring method, including a weighted composite. The estimates use data from a single form. The reliability of the score is used to estimate effective test length in terms of discrete items. The true‐score distribution is estimated by fitting a 4‐parameter beta model. The conditional distribution of scores on an alternate form, given the true score, is estimated from a binomial distribution based on the estimated effective test length. Agreement between classifications on alternate forms is estimated by assuming conditional independence, given the true score. Evaluation of the method showed estimates to be within 1 percentage point of the actual values in most cases. Estimates of decision accuracy and decision consistency statistics were only slightly affected by changes in specified minimum and maximum possible scores.

Journal

Journal of Educational Measurement – Wiley

Published: Jun 1, 1995

There are no references for this article.