Quality & Quantity 33: 429–435, 1999.
© 1999 Kluwer Academic Publishers. Printed in the Netherlands.
On the Relationship Between Validity and Power
Department of Psychology, Fordham University, Bronx, NY 10458, U.S.A.
Abstract. This article examines the relationship between predictive validity and statistical power. It
is shown that higher validity may be associated with scales exhibiting lower power, whereas higher
power may be warranted for scales possessing lower validity. High validity is in general neither a
necessary nor sufﬁcient condition for high power with scales contemplated for use as dependent
variables in social science research. Conversely, high power is neither necessary nor sufﬁcient for
high validity. A numerical example illustrates this interrelation.
Key words: effect-size, F -distribution, validity, noncentrality parameter, power, prediction.
1. On the Relationship Between Validity Statistical Power
A question of major relevance in the social sciences is that of validity of measure-
ment. In broad terms, validity is deﬁned as the extent to which a behavior-related
scale, test, or instrument indeed assess what they purport to measure, or what their
developers claim they should be assessing (e.g., Crocker & Algina, 1986). A form
of scale validity often utilized in research practice is predictive validity. It can be
described as the degree to which observed or recorded scores with the utilized in-
strument can be used to make ‘correct’ decisions about (selection from) the people
to whom it has been administered, with regard to their performance on variables
or indicators assessed subsequently. Usually, predictive validity is indexed by the
correlation between the scale scores and an outcome or success indicator/measure,
e.g., a criterion variable.
High-quality measuring instruments are much desired in social science research.
Based on them, it is hoped that examined relationships between variables of interest
will be best captured. Within the frequently employed hypothesis testing frame-
work then, a main query that should be asked yet continues to be largely ignored in
social and behavioral practice, is that of associated statistical power (Cohen, 1992).
It represents the probability of rejecting an incorrect null hypothesis of ‘no effect’
(‘no group differences’) and thus retaining its alternative that the phenomenon
under investigation is present in the population(s) of interest.