How to Meta-Analyze Coefficient-of-Stability Estimates:Some Recommendations Based on Monte Carlo Studies
Abstract
Reliability generalization studies have provided estimates of the mean reliability coefficients and examined factors that explain the variability in the reliability estimates across studies for many different tests and measures. Different authors have used different data analyses to do such meta-analyses, and little research has addressed whether some methods are more accurate than others. Three methods of meta-analysis for reliability data were compared using Monte Carlo techniques. The meta-analytic methods were those described by Hedges and Vevea, Hunter and Schmidt, and Vacha-Haase. The results suggested that a combination of methods worked best and that Hunter and Schmidt's method should be used to estimate the mean and random-effect variance component, but weighted regression should be used to model continuous moderators.