journal article
LitStream Collection
Criterion Validity of the Social Cognition and Object Relations Scale With the Thematic Apperception Test: A Meta-Analytic Review
Cain, Lylli; Diener, Marc J.; Giannopoulos, Evangeline; Bornstein, Robert F.; Szymanski, Kate; Hilsenroth, Mark J.
doi: 10.1037/pap0000595pmid: N/A
This is the first meta-analytic review of the criterion validity of the Social Cognition and Object Relations Scale (SCORS) as applied to Thematic Apperception Test narratives, evaluating whether SCORS ratings significantly correlate with external indices of psychological and behavioral functioning. The primary hypothesis of the present study was that the overall criterion validity of the SCORS with Thematic Apperception Test narratives would yield a small to medium overall weighted effect size, with specific effects aligned with individual study hypotheses. In addition, we developed several secondary hypotheses. We hypothesized that higher SCORS ratings—indicative of more adaptive object relational functioning—would be significantly associated with a range of specific external criterion variables, including (a) membership in nonclinical versus clinical populations; (b) adaptive differences among diagnostic, clinical, and developmental groups; (c) positive psychotherapy process and outcome indicators; (d) lower levels of overall symptom severity or psychopathology; (e) higher ratings of intrapsychic functioning; (f) higher ratings of interpersonal functioning; and (g) greater performance on measures of cognitive functioning. A random effects meta-analysis was conducted on 37 articles representing 32 independent samples (N = 2,399). The primary hypothesis was supported as SCORS ratings demonstrated significant criterion validity, yielding an overall weighted effect size of r = .227 (p < .001). Heterogeneity testing showed no significant between-study variability, Q(31) = 33.210, p = .360, I2 = 7%, suggesting consistency of validity effects across samples. In addition, when effect sizes were classified separately based on their different sources of criterion variables, each of these subgroup meta-analyses yielded significant results (p < .02) consistent with secondary hypotheses. Publication bias analyses were conducted, and two of these suggested some potential risk of bias. However, analyses conducted to correct for any potential risk of bias yielded an overall weighted effect size that remained statistically significant. Interrater reliability of SCORS coding in the original studies included in this meta-analysis was excellent, with an average intraclass correlation coefficient of .847 across 1,676 ratings. Analysis of Thematic Apperception Test card usage indicated that Cards 1, 2, 3BM, 4, and 13MF were most frequently administered (> 70%), supporting their use as a core set for standardized clinical assessment and research protocols. Findings provide robust evidence supporting the SCORS as a psychometrically sound measure of relational and affective functioning, with practical implications for clinical diagnosis, treatment planning, and tracking therapeutic change.