Reliability of Speeded Number-Right Multiple-Choice Tests
AbstractContrary to common belief, reliability estimates of number-right multiple-choice tests are not inflated by speededness. Because examinees guess on questions when they run out of time, the responses to these questions generally show less consistency with the responses of other questions, and the reliability of the test will be decreased. The surprising implication is that adding questions to a multiple-choice test may lower its reliability when the test is speeded. This article develops the mathematical derivations and shows the effects of speededness on reliability in simulations.