What Is the Interpretation of the Validity Generalization Estimate = − ?
Abstract
The estimate is difficult to interpret; it is not a simple variance estimate. Its expectation under the hypothesis that all population correlation coefficients are equal is about zero, but it can be negative and depends on the unknown constant population correlation coefficient. The probability distribution of can have large negative probability mass even if its expectation is positive. Observed estimates of , even if negative, cannot be taken as unqualified evidence supporting the validity generalization hypothesis because expectations under certain alternative hypotheses may have smaller expectations than under certain validity generalization hypotheses. It appears virtually impossible to provide a general hypothesis-testing framework that distinguishes between validity specificity and generalizability based on .