TY - JOUR
AU - Levine, Timothy, R.
AB - Abstract Meta-analysis indicates moderate correlations between the Verbal Aggressiveness Scale (VAS) and other self-report measures but near-zero correlations with behavioral measures. Accurately interpreting correlations between the VAS and other variables, however, requires an examination of the untested error theory underlying the measurement model for the VAS. In two separate studies, the results of single-factor correlated uniqueness confirmatory factor analytic models revealed a pattern of significant error covariances indicating that VAS item scores are confounded by systematic error attributable to multiple unspecified latent effects. After pruning the item sets, we identified 4 items that were free of latent variable influences other than trait verbal aggressiveness. Implications for interpreting the verbal aggressiveness literature are discussed along with recommendations for revising the VAS. Exploratory factor analytic (EFA) investigations of the Verbal Aggressiveness Scale (VAS), beginning with Infante and Wigley's (1986) initial scale development study, consistently reveal two factors. One factor consists of items presenting verbally aggressive tactics. Infante and Wigley (1986) described the items of the other factor as measuring verbal benevolence. However, the structure of the instrument remains in doubt owing to the ambiguity of the second factor. Infante and Wigley (1986) interpreted the second factor as indicating that the VAS was “unidimensional with a latent variable being item wording” (p. 65). The VAS features a five-option response format for each item (almost never true; rarely true; occasionally true; often true; and almost always true). According to Infante and Wigley (1986), wording bias occurred because respondents were reluctant to agree to items that inquired about the use of socially undesirable (verbally aggressive) behaviors. Several researchers have offered competing interpretations of the meaning of the second factor (Beatty, Rudd, & Valencic, 1999; Hamilton & Hample, 2011; Hamilton & Tafoya, 2012; Kotowski, Levine, Baker, & Bolt, 2009; Levine et al., 2004; Levine, Kotowski, Beatty, & Van Kelegom, 2012), but Infante and colleagues (Infante, Rancer, & Wigley, 2011) have insisted that the second factor is merely an artifact due to a latent method effect. Although researchers have suggested a variety of labels for the two factors, we have chosen to use verbal aggressiveness and verbal benevolence to facilitate comparison of our analysis to Infante and Wigley's (1986) original work. By proposing a latent method factor, Infante and Wigley (1986) specified a measurement model associated with an error theory that deviates from the central assumption of classical test theory (CTT; Lord & Novick, 1968). Specifically, CTT assumes that item errors are uncorrelated, whereas latent method effects necessarily entail correlated error terms among the items affected by the latent factor. If the latent variables are specified as factors in the model, the correlations among error terms should be small and nonsignificant. Expecting method effects strong enough to produce separate factors in EFA, as Infante and Wigley (1986) did, raises the specter of common method error that might account for a substantial portion of the variance shared between the VAS and other self-report measures. The validity of measurement models, in general, and factor scores in particular, derived from EFA requires uncorrelated item errors. If, on the other hand, item errors are significantly correlated, factor scores include the influence of variables in addition to or other than the construct being measured. As a result, reliability estimates of the factor scores can be profoundly misleading and correlations with other variables can be confounded. Violating the independent error assumption would be far less troubling if the VAS accurately predicted verbally aggressive behavior. Ultimately, the value of trait perspectives depends on their usefulness in understanding human communication behavior. However, meta-analysis of the huge body of research using the VAS (Levine et al., 2012) indicated that, even though the VAS correlates moderately with other self-report measures, the correlations between the VAS and behavioral measures are near zero in magnitude. On one hand, the sample of studies examined by Levine et al. (2012) contained few studies focused on observable behavior. On the other hand, if the VAS does not predict behavior, publication bias alone would limit the number of behavior studies that appear in the literature. Regardless, the case for any measurement model of the VAS would be greatly strengthened if the resulting factors consistently predicted verbally aggressive behavior. Although numerous speculative explanations of the pattern of results could be proffered, accurate interpretation of these findings requires clarity regarding the nature of the VAS item error profile. Fortunately, however, the error theory underlying Infante and Wigley's (1986) proposed measurement model can be tested directly by modeling the implied error covariance pattern within a confirmatory factor analytic (CFA) framework. As Levine and Kotowski (2010) pointed out, theoretical advancement regarding verbal aggression depends on precise measurement of the relevant constructs. Indeed, testing and eliminating competing theories require that alternative models for measuring the relevant constructs can be rejected. Discovering what McCallum (1986) referred to as the true or correct factor structure of an item set (i.e., one that achieves a close fit, no indication of significant localized areas of strain, and parsimony) in the face of competing alternatives requires an evaluation of the error theories underlying proposed structures as well as a test of the proposed measurement models. A comprehensive research literature must include empirical tests of not only all relevant measurement models but their associated error theories as well (Brown, 2006). Although measurement models specify the number of proposed factors and delineate the items expected to load on each factor, error theories specify anticipated relationships, if any, among the unique variances or item errors. Measurement models are evaluated in terms of overall fit, modification indices, residuals, and factor loadings. Error theories are evaluated by inspecting the correlations among the error terms of the items. When no significant correlations among item errors are observed, one can reasonably conclude that item scores reflect only differences among participants with respect to the construct being measured and random error. Correlations among error terms, however, indicate that a latent variable or variables (e.g., a method effect) other than the factor the items were designed to measure has affected the item scores, thus making the assumptions about the interitem correlations spurious to some degree. In such cases, item errors contain both systematic and random error. An error theory positing significant correlations among the error terms for all 10 VAS items affected by wording bias but uncorrelated error terms for the other 10 VAS items is precisely the error theory corresponding to Infante and Wigley's (1986) initial conceptualization and Infante et al.'s (2011) more recent stance. Unfortunately, this error theory has never been explored in the literature. In the absence of an examination of the error theory associated with the VAS, factor analytic results must be interpreted cautiously because a variety of measurement models may show a reasonable fit, masking the influence of unmeasured and undetected method errors on the derived solution. This is particularly true when the data are exclusively self-report, numerous questionnaires containing both positive and negatively worded items are administered, derived factors consist exclusively of items that are either negatively or positively worded, and the criteria for interpreting CFA results and the methods employed to produce those results differ across studies (Brown, 2006). The aforementioned description accurately characterizes the extant literature focused on the factor structure of the VAS. Despite the fact that alternative factor structures to Infante and Wigley's (1986) single-factor with a method effect model, ranging from two-factor (e.g., Levine et al., 2004) to hierarchical models (Hamilton & Hample, 2011; Hamilton & Tafoya, 2012), have been advanced in the literature, the error theory underlying Infante and Wigley's (1986) original conceptualization remains untested. However, empirical verification of Infante and Wigley's (1986) error theory would resolve the controversy about the factor structure of the VAS, rendering alternative factor structures such as those proposed in the literature theoretically unnecessary. On the other hand, if the item error profile of the VAS fails to conform to Infante and Wigley's theoretical expectations, the single-factor model can be confidently discounted. Moreover, empirical failure of Infante and Wigley's (1986) error theory would warrant serious consideration of the alternative models presented in the VAS literature, the best-fitting of which currently is Hamilton and colleagues' hierarchical model. The fact that items measuring verbal aggressiveness and those depicting verbal benevolence loaded on separate factors derived from EFA does not constitute sufficient evidence of a method effect. Rather, evidence of an unmeasured latent factor, whether methodological or theoretically meaningful, is the presence of correlated error terms among all the item scores affected by the latent factor. An adequate test of Infante and Wigley's (1986) interpretation, therefore, requires a measurement model capable of estimating whether the measurement errors of the 10 items assumed to have been influenced by a latent method effect are correlated. Although error covariance (correlated measurement errors) constitutes the classic statistical signature of a latent variable, error covariance cannot be modeled within an EFA framework (Brown, 2006) because measurement error is treated as random within EFA. An adequate test of the error theory underlying the assertion that the VAS is a unidimensional measure with a single-method effect requires the evaluation of either a two-factor measurement model (one verbal aggressiveness factor and one latent method factor) or, alternatively, a mathematically equivalent single-factor correlated uniqueness measurement model, both of which can be estimated within a CFA framework. In the present study, we (a) explicate the error theory necessary to model Infante and Wigley's (1986) interpretation and (b) in two separate studies, test and evaluate the measurement model implied by the pattern of error covariances. Conceptual framework Specifying the error theory Error terms of any indicator affected by the latent variable should be correlated with the error term of every other indicator influenced by that latent variable. As Brown (2006) noted, “if it is believed that method effects exist for questionnaire items . . . correlated errors should be freely estimated for all such indicators, not just a subset of them” (p. 186). Infante and Wigley's (1986) verbal proposition that a separate 10-item factor emerged due to a single latent variable entails a corresponding statistical model with a saturated pattern of error covariances (i.e., one in which the error terms of all 10 items affected are correlated with each other). On the other hand, clusters of items with correlated error terms would indicate the effects of multiple rather than one latent method effect variable, an outcome inconsistent with Infante and colleagues' position. Infante and Wigley (1986) based their conclusions on the results of an EFA, which are insufficient to detect method effects such as those produced by item wording. At times, the presence of both reverse-coded items and items depicting the presence of a construct results in complex factor solutions when subjected to EFA. At other times, the second factor represents a theoretically meaningful construct rather than a method effect (Brown, 2003, 2006; Marsh, 1996). And at still other times, items depicting the presence of a construct and reverse-coded items load on a single factor differentiated only by the valence of their loadings. Detecting or confirming the existence of a proposed latent variable requires modeling the relationships among the error or unique variances of the items affected by the latent variable. Brown (2006) underscored the inadequacy of EFA in this respect, noting that, “although both EFA and CFA differentiate common and unique variances, within EFA the specification of relationships among unique variances is not made. . . . EFA factor models must be specified under the assumption that measurement error is random. In contrast, correlated measurement error can be modeled in a CFA solution” (p. 46). The VAS has been subjected to CFA a few times (Hamilton & Hample, 2011; Hamilton & Tafoya, 2012; Levine et al., 2004; Suzuki & Rancer, 1994), but no attempt was made to model item error covariances in those studies. In summary, therefore, the foundational assumption underlying Infante and colleagues' interpretation of the second VAS factor as representing one latent variable, although contested (Beatty et al., 1999; Hamilton & Hample, 2011; Hamilton & Tafoya, 2012; Kotowski et al., 2009; Levine et al., 2004, 2012), has never been tested. Specifying the measurement model(s) One approach to an adequate test of Infante and Wigley's (1986) interpretation is to specify and evaluate a single-factor CFA model, inspecting the degree of ill fit regarding the independence among unique variances as well as the overall fit. In general, as Brown (2006) noted, “A common error in applied CFA research is to evaluate models exclusively on the basis of overall goodness-of-fit; in some instances, overall goodness-of-fit indices suggest acceptable fit despite the fact that some relationships among indicators in the sample data have not been reproduced adequately” (pp. 113–114). Thus, as a general recommendation, Brown (2006) suggested that “after substantive justification of the model is established, the acceptability of the fitted CFA solution should be evaluated on the basis of three major aspects: (a) overall goodness-of-fit; (b) the presence or absence of localized areas of strain in the solution (i.e., specific points of ill fit); and (c) parameter estimates” (p. 113). Brown pointed out that “the size of factor loadings should be considered closely” (p. 130) because cross-loadings (i.e., factor loadings on factors other than the primary factor that the indicator is supposed to measure) are not specified within CFA. This means that the square of the completely standardized factor loadings equals the percentage of item or indicator variance attributable to the factor being measured. An item or indicator's error variance is equal to 1 minus the squared factor loading. Standardized factor loadings lower than .40, therefore, indicate that considerable item variance is error. For that reason, it is not uncommon to set the threshold values for minimally acceptable factor loadings in the .50 or even the .60 region for construct validation studies of composite measures (Brown, 2006, p. 130). Specifying these values for factor loadings is merely a rule of thumb or convention, however. Viewed from a broader theoretical perspective, there may be cases in which maintaining continuity of indicators used across studies to measure a central construct may well outweigh modest gains in model fit to a sample covariance matrix. This is especially the case if the model has a sound conceptual foundation, a history of replication, and the localized areas of strain are limited to aspects such as slightly misestimated covariance between two specified factors, residuals of correlations within factor indicators, or indications of nonzero cross-loadings (although cross-loadings are set at zero in CFA, modification indices can indicate that the assumption of nonsignificant cross-loadings is statistically unjustified). Detection of significant error covariances, on the other hand, is an entirely different matter. Even Kline (2005), who emphasized the critical importance of theory in the specification and evaluation of measurement models, has considered, as have others (e.g., Brown, 2006; Raykov, 2001), the indication of correlated error terms among indicators a substantive justification for model respecification. The problem is considered critical because correlated error terms among items on a questionnaire such as the VAS, for example, may indicate substantial effects of exogenous variables not included in the model on item scores. Accordingly, correlated error terms indicate that important explanatory constructs are probably missing from the model regardless of overall model fit statistics, casting doubt on the theoretical foundation for the measurement model. Observation of significant error covariances requires the specification and an empirical test of an error theory to account for the correlated error terms. Infante and Wigley (1986) responded to their suspicion that one factor was attributable to a single-method effect by collapsing the two factors into a single index. They did this by combining items with independent error terms and items with correlated error terms. As a consequence, the reliable variance in item scores on the VAS was confounded by systematic error due to the potential latent variable effects. This problem manifests itself in two ways. First, the reliability estimates of the VAS based on Cronbach's (1951) alpha that are reported in the literature are probably inaccurate. One assumption of Cronbach's formula is that the measure it is being applied to is congeneric (i.e., the conditions of CTT have been met). However, methodologists (Brown, 2006; Meyer, 2010; Raykov, 2001) writing about measurement have noted that CTT assumptions are often unrealistic. Importantly, one major limitation of alpha is that, when item errors are correlated, making the measure noncongeneric, alpha is a misestimator of reliability (Raykov, 2001).1 The extent of the misestimation of reliability depends on (a) the magnitude of the error correlations relative to the interitem correlations and (b) the number of item pairs affected. A comparison of alpha to a reliability estimate derived from an interitem correlation matrix after the systematic error due to latent factor effects have been removed would clearly reveal the magnitude of the misestimation. If alpha coefficients are relied on as estimates of internal consistency, which is the case in the verbal aggressiveness literature, the problem will go undetected. Because validity cannot exceed the square root of reliability (for a discussion of the mathematical relationship between reliability and validity, see Guilford, 1954), ignoring the potential impact of error covariance has potential methodological and theoretical implications. Second, the interpretations of correlations reported in the literature between the unidimensional, 20-item VAS and criterion variables may also be spurious to the extent that the latent variables—whether method effects or substantive variables—influencing VAS item scores are also common to the criterion variables. If Infante and Wigley (1986) were correct, then wording bias in criterion variables would require attention, but it would constitute the only potential source of method error. However, if multiple latent variables, indicated by clusters of correlated error terms, were responsible for failure to produce a clean unidimensional solution in EFA studies of the VAS, interpretation of the verbal aggression literature involving the VAS becomes far more complicated. As with the reliability issue, the extent to which assumptions about correlations between the VAS and other measures are spurious depends on the degree to which the VAS is noncongeneric. The problems just discussed, however, are not unique to the VAS. Researchers in psychology interested in measurement issues have long been attentive to the psychometric problems posed by wording effects in questionnaires, especially when the effects are strong enough to produce significant error covariances (Brown, 2003, 2006; Byrne, Fisher, Lambert, & Mitchell, 1974; Gaudry, Vagg, & Spielberger, 1975; Marsh, 1996; Marsh & Smith, 1982). A number of strategies for coping with error covariances due to a latent item-wording method effect have emerged, including parceling indicator scores if only a few items are affected, deleting the least theoretically important items from item clusters with correlated error terms until a congeneric set is obtained, or specifying a model with correlated residuals (again, if only a few are involved, although this approach has been severely criticized; see Cole, Ciesla, & Steiger, 2007). The specific approach depends on the precise pattern of the correlated error terms. Accordingly, to the extent that Infante and Wigley's (1986) decision to combine items affected by a method factor with items not affected produced significant correlation among error terms, (a) the reliability coefficients reported in the literature are untrustworthy, and (b) correlations between the VAS and criterion variables may be misleading if the unspecified latent factor(s) responsible for error covariance is(are) common to the VAS and the criterion measures. Testing the unidimensional VAS with one method effect theory By declaring that scores on responses to 10 of the VAS items are influenced by the same latent method variable, Infante and colleagues committed the VAS to a particular measurement model with a particular error theory. Specifically, if Infante and associates are correct, (a) a pattern of saturated (i.e., every item error is correlated with every other item error) significant error covariances among the 10 items affected by the proposed single latent method factor should be observed when a single-factor CFA with all 20 items as indicators is conducted, and (b) a two-factor measurement model specifying one verbal aggressiveness factor and one latent method effect factor should be sufficient to explain all latent effects. Any other significant deviations from this pattern would demonstrate that the single latent method factor error theory is untenable (e.g., clusters of items with correlated error terms, indicating multiple unspecified factors, rather than a saturated 10-item set are observed or if significant error covariance remains unaccounted for in the single-factor model with one method effect). Because specification searches (McCallum, 1986) were involved in our study, we conducted a second study to reduce the likelihood that the results of Study 1 capitalized on chance. Both studies were approved by the Institutional Review Board (IRB). Study 1 Method Participants and procedures The participants were 363 (male n = 121, female n = 240, missing n = 2) undergraduate students enrolled in communication courses at a private university in the Southeast. The students ranged in age from 18 to 33 (M = 20.36, Mdn = 20.00, SD = 1.79), and most (53%) were Caucasian. Participants were recruited following IRB guidelines regarding age, voluntary participation, and anonymity. All participants received extra credit in their class in exchange for participation. Following instructions and questions, participants responded to a questionnaire containing the demographic and VAS items during regular class time. After questionnaires were completed and gathered, respondents were debriefed. Measure Infante and Wigley's (1986) VAS was used to measure trait verbal aggressiveness. The 10 items representing verbal benevolence were reverse scored when computing unidimensional VAS scores. The complete measure is presented in the Appendix. In Study 1, the mean for the VAS was 49.18, the standard deviation was 10.54, and alpha was .84. Results and discussion of Study 1 The data analyses were conducted in a two-step sequence using Mplus Version 7. First, although the measurement model underlying Infante and Wigley's (1986) interpretation of EFA results for the VAS specified one verbal aggressiveness factor and one latent method factor, Infante and Wigley recommended collapsing all 20 items into a single index. This, however, ignores the specified method effect. Prior to testing the two-factor (one verbal aggressiveness factor and one latent method factor) model, we conducted a single-factor CFA on the VAS with the covariances among all error terms fixed at zero (i.e., the error terms were made independent of each other). This was done to examine the psychometric consequences of treating the 20 VAS items as a unidimensional instrument. Furthermore, error covariances among all 10 items (i.e., a saturated error covariance pattern) affected by the latent method factor proposed by Infante and Wigley (1986) should be apparent in the modification indices. If Infante and Wigley's (1986) error theory pertaining to a single latent method effect is correct, then significant reductions in the model chi-square should result from the free estimation of the covariance between error terms for each possible pair within the 10-item set. Second, the two-factor (one verbal aggressiveness factor and one latent method factor) model and the alternative single-factor with correlated uniqueness models were tested directly using CFA. Single-factor independent error model A single-factor CFA with all error terms specified as independent (i.e., all item error covariances were fixed at zero) was a poor fit to the data both in terms of overall fit, χ2(170, N = 363) = 543.32, p < .001, RMSEA = 0.078 (90% CI [0.07, 0.085]), SRMR = 0.072, CFI = 0.748, TLI = 0.718, and in terms of standardized factor loadings for the items (over half were below .50, with three below .40). Moreover, inspection of the modification indices for the covariances among error terms revealed that 55 of the 190 error covariances were significant, p < .05 (critical value = 3.84). Even at the p < .01 level (critical value 6.64), 44, or almost 25% of all the possible error covariances, were significant.2 Contrary to Infante and Wigley's (1986) error theory, (a) separate clusters and pairs of error covariances unrelated to other items, indicating effects of multiple latent variables (rather than a saturated pattern among the 10-item set required for a single latent method effect), were observed, (b) clusters of error covariances appeared within both 10-item sets, not just within the verbal aggressiveness items, (c) 27 item pairs consisting of a verbal aggressiveness item and a verbal benevolence item with significant error covariance were detected, which is inconsistent with a single latent method effect affecting items from one item set, and (d) not all items within either 10-item set displayed any covariances with the errors of other items. Although alpha was .84 for the VAS when treated as a unidimensional scale, the reliability coefficient dropped to .68 when Raykov's (2001) formula for measures containing items with correlated error terms was calculated.3 This analysis indicated that 32% of the variance in VAS scores was error, with only 16% attributable to random measurement error and the remaining 16% due to systematic error. As indicated by these results, responses to the VAS are markedly affected by latent variables other than trait verbal aggressiveness and fail to meet CTT assumptions. Two-factor (one verbal aggressiveness factor and one latent method factor) model As might be suspected based on the previous analysis, a two-factor CFA with a verbal aggressiveness factor and one latent method factor that directly tested Infante and Wigley's error theory underlying the two-factor EFA result failed to adequately reproduce the sample covariance matrix. Moreover, inspection of the modification indices for the covariances among error terms revealed that 55 were significant, p < .05 (critical value = 3.84). Even at the p < .01 level (critical value 6.64), 44, or almost 25% of the 190 possible error covariances, were significant. Because the effects of the latent method variable, presumed to be item wording, are accounted for by specifying a latent method factor in addition to the verbal aggressiveness factor, adequate fit of the data to the model should result in (a) no additional significant modification indices because the verbal aggressiveness factor and the method factor should account for all latent effects on indicators (VA items), (b) the standardized factor loadings should be acceptable (λ > .60 is typically set by convention as a minimum value to consider a factor loading as strong; Brown, 2006) for all indicators, and (c) the CFA solution should achieve a close fit to the extent that variance in indicators due to latent effects has been accounted for. Moreover, in an effort to conduct a lenient test of the error theory underlying a unidimensional model of the VAS, we set p < .01 as the level of significance (critical value = 6.64) for modification indices associated with error covariances. In this way, we minimized the risk of rejecting the measurement model on the basis of small error covariances that achieved only borderline significance (e.g., p < .05). In contrast to Infante and Wigley's (1986) error theory, there were nine significant modification indices, suggesting the presence of error covariances even after the effects of a single latent method factor were modeled. In fact, all nine modification indices were larger than the critical value (7.88) for p < .005, with the average correlation among the error terms, r = .19, only slightly smaller in magnitude than the average correlation among the VAS items, r = .22. Similarly, the lack of localized fit was apparent in the factor loadings. Only five loadings were .60 or above, seven were below .50, and two were below .40 (mean λ = .52). With all item error covariances fixed at zero and a correlation between factors equal to .57, the overall fit indices ranged from adequate to good, χ2(169, N = 363) = 317.64, p < .001, RMSEA = 0.05 (90% CI [0.04, 0.06]), SRMR = 0.05, CFI = 0.90, TLI = 0.89. Figure 1 shows the pattern of error covariances suggested by the modification indices for the two-factor (one verbal aggressiveness factor and one latent method factor) model if the error covariances between items were allowed to be freely estimated rather than fixed at zero. Figure 1 Open in new tabDownload slide Two-factor VAS model, where factor 2 represents verbal aggressiveness and factor 1 represents the latent method effect proposed by Infante and Wigley (1986), with all significant at (p < .01) error covariances freely estimated, for Study 1. Curved double-headed arrows to the right of residual arrows for indicators represent significant error covariances. Unique variance estimates have been omitted because their inclusion in the figure obscures the presentation of error covariances. Figure 1 Open in new tabDownload slide Two-factor VAS model, where factor 2 represents verbal aggressiveness and factor 1 represents the latent method effect proposed by Infante and Wigley (1986), with all significant at (p < .01) error covariances freely estimated, for Study 1. Curved double-headed arrows to the right of residual arrows for indicators represent significant error covariances. Unique variance estimates have been omitted because their inclusion in the figure obscures the presentation of error covariances. Single-factor correlated uniqueness model A single-factor correlated uniqueness model, estimating only the error covariances suggested by the modification indices associated with the single-factor independent error model and significant at the p < .05 level, made the inflationary effect of cumulative nonsignificant contributions to model fit apparent, χ2(162, N = 363) = 433.27, p < .001, RMSEA = 0.07 (90% CI [0.06, 0.08]), SRMR = 0.07, CFI = 0.82, TLI = 0.79. When only statistically significant error covariances are freely estimated, a model specifying the error theory that the VAS is a unidimensional measure with a latent method factor does not fit the data well. Figure 2 shows the error covariances required to reproduce the sample covariance matrix. As that figure shows, freely estimating error covariances between item pairs in addition to those supposed to be affected by the latent method effect hypothesized by Infante and Wigley (1986) was required. Figure 2 Open in new tabDownload slide Single-factor correlated uniqueness model of the VAS with all significant at (p < .01) error covariances freely estimated, for Study 1. Curved double-headed arrows to the right of residual arrows for indicators represent significant error covariances. Unique variance estimates have been omitted because their inclusion in the figure obscures the presentation of error covariances. Figure 2 Open in new tabDownload slide Single-factor correlated uniqueness model of the VAS with all significant at (p < .01) error covariances freely estimated, for Study 1. Curved double-headed arrows to the right of residual arrows for indicators represent significant error covariances. Unique variance estimates have been omitted because their inclusion in the figure obscures the presentation of error covariances. The error theory implied by the results of the present study is that VAS scores are affected by nine latent variables in addition to the latent method factor specified and trait verbal aggressiveness. The degree of latent effect was evident in the reliability analysis. Two of the latent factors contributed to verbal aggressiveness items and four latent factors affected the verbal benevolence items even after the effects of the single latent method variable were accounted for in a two-factor model. Thus, the error theory specifying multiple latent effects, rather than a unidimensional measure with a single latent method effect, most precisely reproduces the sample covariance matrix. Study 2 The results of Study 1 have implications for the structure of the VAS. Therefore, it was important to determine whether those findings were unique to the Study 1 sample. Study 2 was undertaken in an attempt to replicate Study 1. Method The procedures employed in Study 1 were followed in Study 2. The VAS was administered to 205 students (male n = 84; female n = 121) enrolled in undergraduate courses at a large public university in the Midwestern United States following the university's IRB guidelines. Students ranged in age from 17 to 28 (M = 19.91, Mdn = 20.00, SD = 1.62) and most were Caucasian (61%). In Study 2, the mean and standard deviation of the VAS were 51.36 and 10.37, respectively. Alpha was .82. Results and discussion of Study 2 As in Study 1, results contradicted Infante and Wigley's (1986) assertion that the VAS is a unidimensional measure with a single latent method effect. Rather, regardless of the model specified, inspection of the modification indices for error covariances indicated that participants' responses to many of the items were influenced by latent factors other than individual differences in verbal aggressiveness and a latent method effect. Single-factor independent error model Following the analytic steps in Study 1, we first tested a measurement model that represents the way in which Infante et al. (2011) recommend treating the VAS. Specifically, a single-factor CFA with all error terms specified as independent (that represents the measurement model underlying the summation of VAS items into a single index) was specified and evaluated. Consistent with the results of Study 1, the model showed a poor fit to the data both in terms of overall fit, χ2(170, N = 205) = 361.68, p < .001, RMSEA = 0.07 (90% CI [0.06, 0.09]), SRMR = 0.08, CFI = 0.78, TLI = 0.70, and in terms of the standardized factor loadings for the items (only two were .60 or larger, 15 were below .50, with six of those below .40 and three of which were less than .30). Furthermore, 17 of 190 possible modification indices for error covariances were significant, p < .01, critical value = 6.64. Also similar to the results of Study 1, the violation of CTT assumptions manifest in significant error covariance profoundly inflated the alpha coefficient. When Raykov's (2001) procedure for accounting for systematic error due to unmeasured latent factors was followed, the reliability coefficient dropped from .82 to .65. Thus, in Study 2, 34% of the variance in VAS scores was error, with 18% traceable to random measurement error and the remaining 17% contributed by the latent variable(s) other than individual differences in verbal aggressiveness. Two-factor (one verbal aggressiveness factor and one latent method factor) model As in Study 1, a two-factor CFA model with one factor representing verbal aggressiveness and the other factor representing the latent method effect was specified and evaluated. Specifying all 10 verbal benevolence items as loading on the latent method factor permits cumulative effects of nonsignificant error covariances contributed by ill-fitting indicators to inflate fit indices. As such, compared to results for a single-factor correlated uniqueness model in which only those item pairs with demonstrated significant error covariances are freely estimated, the overall fit indices associated with the latent method effect model provide a somewhat lenient test of the VAS as a unidimensional measure with one method effect. Even so, the model was not a good fit to the data. First, standardized factor loadings for the items were unimpressive (only four were equal to or larger than .60, 11 were below .50 with two of those below .30). Second, even though the overall fit was reasonable (but not close), χ2(169, N = 205) = 266.64, p < .001, RMSEA = 0.053 (90% CI [0.04, 0.07]), SRMR = 0.06, CFI = 0.86, and TLI = 0.85, as noted in the results section of Study 1, no significant error covariance should be detected if the model adequately reproduces the sample covariance matrix regardless of overall fit. In contrast to that requirement, however, 10 item pairs showed significant error covariances even when p < .01 was set as the criterion. This result indicates that latent factors other than verbal aggressiveness and the specified latent method effect affect VAS item scores. Moreover, as observed in Study 1, multiple clusters of items with correlated error terms rather than one or two saturated sets were apparent, even within the set of items specified as loading on the method factor. Thus, in contrast to Infante and Wigley's (1986) interpretation, latent factors in addition to a single-method effect influenced responses to the items depicting verbal aggressiveness. Also, of particular importance, some item pairs sharing correlated errors occurred within the verbal aggressiveness item set, a finding inconsistent with the notion that scores were affected only by reluctance to agree to items depicting socially undesirable behavior. Finally, the modification indices suggested significant error covariances between some verbal benevolence and some verbal aggressiveness items. Had the interfactor covariance been fixed at .00 (i.e., orthogonal), error covariances across sets of items could be erroneously attributed to a general verbal aggressiveness factor (Anderson & Gerbing, 1982). However, we allowed the error covariances between specified factors to be freely estimated (r = .60). As such, error covariances across item sets represent the effect of unspecified and unmeasured latent factors. Overall, this pattern of error covariances is inconsistent with the notion that the VAS should be viewed as a unidimensional measure with one latent method effect (i.e., wording bias). A model indicating the pattern of error covariances unexplained by the unidimensional measurement model for the VAS with one latent method factor is presented in Figure 3. Figure 3 Open in new tabDownload slide Two-factor VAS model, where factor 2 represents verbal aggressiveness and factor 1 represents the latent method effect proposed by Infante and Wigley (1986), with all significant at (p < .01) error covariances freely estimated, for Study 2. Curved double-headed arrows to the right of residual arrows for indicators represent significant error covariances. Unique variance estimates have been omitted because their inclusion in the figure obscures the presentation of error covariances. Figure 3 Open in new tabDownload slide Two-factor VAS model, where factor 2 represents verbal aggressiveness and factor 1 represents the latent method effect proposed by Infante and Wigley (1986), with all significant at (p < .01) error covariances freely estimated, for Study 2. Curved double-headed arrows to the right of residual arrows for indicators represent significant error covariances. Unique variance estimates have been omitted because their inclusion in the figure obscures the presentation of error covariances. Single-factor correlated uniqueness model A single-factor CFA model with correlated uniquenesses specifying correlated error terms only for item pairs indicating significant error covariances, p < .01, in the single-factor independent model also failed to achieve a good fit, χ2(166, N = 205) = 315.19, p < .001, RMSEA = 0.07 (90% CI [0.06, 0.08]), SRMR = 0.07, CFI = 0.79, TLI = 0.76. This model, which was an attempt to fit Infante and Wigley's (1986) general idea, also failed to produce acceptable factor loadings for most items (only two were .60 or larger, nine were lower than .40, two of which were less than .30) and failed to account for 12 additional item pairs showing significant, p < .01, error covariances, some of which were within item sets indicating verbal benevolence and some of which were across item sets depicting verbal benevolence and item sets depicting verbal aggressiveness. Figure 4 shows the pattern of freely estimated error covariances required to reproduce the sample covariance matrix. As that figure indicates, specification of freely estimated error covariances for items in addition to those implicated by the error theory underlying Infante and Wigley's (1986) position was required. Figure 4 Open in new tabDownload slide Single-factor correlated uniqueness model of the VAS with all significant (at p < .01) error covariances freely estimated, for Study 2. Curved double-headed arrows to the right of residual arrows for indicators represent significant error covariances. Unique variance estimates have been omitted because their inclusion in the figure obscures the presentation of error covariances. Figure 4 Open in new tabDownload slide Single-factor correlated uniqueness model of the VAS with all significant (at p < .01) error covariances freely estimated, for Study 2. Curved double-headed arrows to the right of residual arrows for indicators represent significant error covariances. Unique variance estimates have been omitted because their inclusion in the figure obscures the presentation of error covariances. General discussion Overview of results Our results that were based on two separate samples drawn from two different types of student populations (one from a small university in the Southeastern United States and the other from a large university in the Midwestern United States) clearly indicated that, regardless of whether the method effect was specified in the model or a correlated uniqueness approach was followed, measurement models representing the VAS as a unidimensional measure with one latent method effect failed to adequately reproduce the sample covariance matrix in either study. The pattern of error covariances constituted the most glaring inadequacy of the error theory underlying Infante and Wigley's (1986) position. Specifying a single latent method effect left several error covariances unaccounted for. Even when the criterion for significant error covariances was set at p < .01, and the path between factors was freely estimated, the model specifying one verbal aggressiveness factor and one latent method effect factor failed to account for several clusters of items with correlated error terms. Moreover, within single-factor models, clusters of item pairs with significant error covariances indicating multiple latent factors were observed rather than a single web of saturated paths among the error terms. The latter would be required for a common latent method factor (Brown, 2006) and expected for the 10 items Infante and Wigley (1986) claimed were affected by one latent method factor. Additionally, significant error covariances were detected for item pairs that should have been statistically independent according to the error theory underlying the VAS as a unidimensional measure with one latent method factor model.4 In both studies, three aspects of our results are inconsistent with the theory that the VAS is a unidimensional measure with one latent method factor: (a) evidence of multiple latent factors affect scores on items depicting verbally aggressive behaviors, (b) the latent factors affecting items depicting verbal aggressiveness appear to be different than the factors affecting verbal benevolence items, and (c) some latent factors are common to some verbal aggressiveness items and some items depicting verbal benevolence. Some item pairs, such as #9 and #18, #14 and #15, and #17 and #20, had significant error covariances in both studies. This indicates the presence of at least three latent variables replicated across samples in addition to the two factors proposed in the CFA model. Whether these latent variables represent method effects or substantive factors, this finding poses a serious challenge to the assumption that the VAS is unidimensional with one method factor. Other clusters, however, did not replicate across samples. Certainly, this could indicate that error covariances for some clusters were due to sampling error.5 On the other hand, given the similarity between the overall fit indices and the factor loadings for the items in the single-factor independent error models for the two samples, the slight differences between the samples regarding the composition of the latent variable clusters could possibly represent sample-specific latent influences. Verifying the source of differences between error covariance clusters, however, is beyond the scope of the present study. Although detecting error covariance patterns that replicate across samples is clearly theoretically interesting, we would note, however, that measurement experts emphasize that detecting error covariances among items is sufficient to warrant respecification of the model irrespective of whether those error covariance patterns replicate (Brown, 2006; Kline, 2005). Measurement models that fit the data should display no significant error covariances among items (Brown, 2006; Kline, 2005; Raykov, 2001). The detection of any significant error covariances should be sufficient to give researchers pause, whether or not the source(s) of the error covariances can be identified. In addition, although our sample sizes were not especially huge, Kline (2005, p. 15) has pointed out that sample sizes between 100 and 200 participants can be considered “medium,” and “sample sizes that exceed 200 cases could be considered ‘large.’” Although model respecification based purely on statistical rather than substantive grounds is generally discouraged (Brown, 2006; Kline, 2005), detection of statistically significant error covariances is acknowledged as substantive justification to reconsider a measurement model (Brown, 2006; Kline, 2005) precisely because substantial correlations among error terms signal that indicator scores reflect the effects of factors other than those researchers intend to measure. The degree of threat posed to construct validity depends on the magnitude of the correlation among error terms relative to correlations among the items. By setting the criterion for significance at p < .01, error covariances for offending item pairs translated into average correlations among error terms that were nearly as high as interitem correlations for VAS items in the two studies (Study 1, mean interitem r = .22; Study 2, mean interitem r = .19). The magnitude of error covariance effects on the VAS was most clearly illustrated by our comparative reliability analyses. The alpha coefficients for Study 1 (α = .85) and Study 2 (α = .82) were similar to those for the VAS reported in the trait verbal aggressiveness literature. However, when systematic error owing to the unspecified latent factors was accounted for, the corrected reliability coefficients dropped to .68 and .65 for Study 1 and Study 2, respectively. Implications for theory and research Infante et al. (2011) argued that treating the VAS as a unidimensional measure with one latent method factor was based on theory. Two points are important. First, we could find no compelling rationale for a unidimensional conceptualization of aggressiveness in the literature cited by Infante and Wigley in the development of the construct or measure (e.g., Bandura, 1973; Berkowitz, 1962). Indeed, contemporary theorists have proposed and verified multidimensional models of aggressiveness (Barratt, Stanford, Dowdy, Liebman, & Kent, 1999; Dodge & Coie, 1987; Kingsbury, Lambert, & Hendrickse, 1997; Mathias et al., 2007; Poulin & Boivin, 2000). We see no reason why verbal aggressiveness might not also be multidimensional. Indeed, informed by the work of Dodge and Coie (1987), Valencic, Beatty, Rudd, Dobos, and Heisel (1998) suggested that reactive and proactive verbal aggressiveness constitute distinct factors. This proposal was more recently echoed by Wigley (2008, 2010). Second, although the initial measurement models are often guided by theory, the adequacy of specified models must be judged in light of data as well as theory, especially when error covariances are detected (Brown, 2006; Kline, 2005). Just because a researcher has a theory does not mean it is a good one. As Kline (2005) put it, “a model is a given at the beginning of the analysis, and one of the main questions to be answered is whether it is supported by the data” (p .10). The data collected in two separate studies did not support the error theory underlying Infante and Wigley's (1986) model of the VAS. Rather, the error theory that most completely reproduces the sample covariance matrix and accounts for significant error covariance requires multiple unspecified latent factors in addition to trait verbal aggressiveness. One of the major general implications of our findings is that the error theory depicting the VAS as a unidimensional measure with one latent method factor can be discounted. Although further research using behavioral measures as criterion variables is required to clarify the nature of the multiple latent factors, the hierarchical structure of the VAS proposed by Hamilton and colleagues (Hamilton & Hample, 2011; Hamilton & Tafoya, 2012) presents the best-fitting measurement model of the alternatives appearing in the current literature. According to Hamilton and Hample (2011), the hierarchical model of verbal aggressiveness “proposes that the general trait of motivation to argue and verbal aggressiveness are second-order factors exerting top-down influence on subsidiary motives and attitudes” (p. 250). Further, the model suggests that the VAS measures two separate tendencies, verbal destructiveness and verbal collaborativeness. Whether behavioral studies validate Hamilton and associates' model is a matter for future research. However, our findings are clearly consistent with a multifactor solution, and the pattern of relationships we found among item errors is not inconsistent with a hierarchical model (for a discussion of the relationship between latent variables and hierarchical models see, Gerbing & Anderson, 1984) such as that proffered by Hamilton and associates. Regardless of the specific model that best explains VAS scores, our findings clearly indicate that the VAS is multidimensional. Our findings have implications for researchers interested in verbal aggressiveness as well as for the use of self-report measures in general. We deem it noteworthy that the problem of error covariances found in the VAS has been observed previously regarding self-report measures (Brown, 2003, 2006; Marsh, 1996; Podsakoff, MacKenzie, Lee, & Podsakoff, 2003; Podsakoff, MacKenzie, & Podsakoff, 2012; Raykov, 2001). Certainly, Infante and colleagues directed attention to the potential effects of verbal aggression in family, workplace, and interpersonal contexts. From a historical perspective, the trait verbal aggressiveness research preceded the broader societal attentiveness to forms of antisocial verbal and nonverbal behavior, including hostility in the workplace, harassment, hate speech, and verbal abuse. This focus, however, is clearly on antisocial acts and patterns of behavior, not merely on attitudes or predispositions. As a unidimensional measure, the VAS has failed to bridge the gap between self-reported trait-like verbal aggressiveness and verbally aggressive behavior (Levine et al., 2012). Within that context, the implications of our findings for researchers' common practice when investigating verbal aggressiveness fall into two categories. Interpreting research findings When reviewing the literature, researchers should remain mindful of the observation that the VAS is not free of significant error covariances. Although clusters of items affected by error covariances could possibly represent undeveloped item pools regarding substantive dimensions of verbal aggressiveness, error covariances most frequently indicate some form of method error (Brown, 2006; Podsakoff et al., 2003, 2012). As a consequence, researchers should be concerned that correlations between the VAS and other self-report measures are inflated due to shared method error. This issue also holds true for research literatures in general that are built on relationships among self-report measures. Multitrait-multimethod research into method effects (Podsakoff et al., 2003, 2012) indicates that in the general psychological literature, on average, common method bias accounts for 9% of the shared variance between a wide range of self-report measures. Double-barreled questions, common item wording, item ambiguity, mood induced by item content, reverse wording, social desirability, and personality-based reactions to item content are among the method biases contributing to inflated correlations between self-report measures. Recall that Infante and Wigley (1986) suspected that wording bias in the VAS items could be strong enough to produce a complex factor structure. If this were the case, researchers could expect the VAS to correlate with other self-report variables, also affected by wording bias, even if the construct purportedly measured by the self-report instrument and verbal aggressiveness were unrelated. To the extent that error covariances represent substantive constructs, researchers should suspect that correlations reported in the literature may be inflated to some degree. Regardless of the source(s) of error covariances, scholars reviewing research literature should question conclusions supported solely by correlations between self-report measures if independence among item errors has not been established for those measures. As a related matter, researchers should remain cognizant of the fact that the alpha coefficient (Cronbach, 1951) misestimates internal consistency of measures with nonzero error covariances among items. Accordingly, the reliability estimates of the VAS, which are often cited as evidence for the measure's value, are most likely inflated. This, however, is true of any measure that fails to meet the independent error assumption. Treatment and analysis of self-report measures Our findings have implications for researchers using self-report measures in their studies. With regard to verbal aggressiveness in particular, our findings clearly suggest that researchers use neither the 20-item unidimensional version of the VAS recommended by Infante and Wigley (1986) nor the 10-item version recommended by Beatty et al. (1999). Merely deleting the 10 verbal benevolence items, as Beatty et al. have suggested, does not eliminate the problem of error covariances within the verbal aggressiveness item set. Our findings taken in the context of other studies focused on method error (Podsakoff et al., 2003, 2012) suggest that in general researchers should examine the random or independent error assumption regarding self-report measures before committing to their use in hypothesis testing. Regarding the VAS, virtually all the errors of the 20 items covaried with the error of at least one other item. However, we were able to prune the measure using a procedure described by Brown (2006) in which the least theoretically important items are deleted until an item set meeting the assumptions of independent error terms remains. We were able to accomplish this within the data set for each study. First, we followed the recommendations in the contemporary literature regarding reverse-coded items (e.g., Brown, 2006) and limited our search to positively worded items (i.e., those indicating verbal aggressiveness). Second, we systematically deleted one item at a time, starting with those with the most complex web of correlated error terms. Ultimately, four out of 20 items common to both Study 1 and Study 2's data sets and free of significant correlations among error terms remained. The items were #2 (i.e., “When individuals are very stubborn, I use insults to soften their stubbornness”), #6 (i.e., “If individuals I am trying to influence really deserve it, I attack their character”), #16 (i.e., “When people do things that are mean or cruel, I attack their character in order to help correct their behavior”), and #18 (“When nothing seems to work in trying to influence others, I yell and scream in order to get some movement from them”). A single-factor CFA model with covariances among item errors fixed to .00 fit the data reasonably well when applied separately to the Study 1 and the Study 2 samples.6 Cronbach's alpha reliability for the four items was .62 when calculated for the combined data from the two studies (N = 568). By comparison, the reliability for the 10-item version of the VAS was .64 when Raykov's (2001) formula accounting for systematic error was calculated. Although we recommend developing additional items to bolster scale reliability, the reliability estimates for the 4-item set was comparable to the estimate obtained for the 10-item version of the VAS when systematic error is removed. However, these four items were free of latent factor influence, method error or otherwise, as indicated by the absence of significant correlations among error terms. Furthermore, item content regarding reactions (i.e., character attacks, insults, yelling, and screaming) when interpersonal goals are thwarted, appears to represent the central thrust of Infante and Wigley's (1986) initial conceptualization of trait verbal aggressiveness. Certainly, from a construct validity standpoint, these four items offer a superior alternative to either the 10 or 20-item unidimensional treatment of the VAS. In retrospect, Infante and Wigley's (1986) inclusion of both positively and reverse-coded items and their treatment of the VAS as a unidimensional measure was psychometrically problematic but consistent with practices for developing questionnaires at the time (for a discussion see Brown, 2006). We believe it is also important to put the performance of the VAS in our study in proper perspective. As Levine and Kotowski (2010) noted, of the measures that have been subjected to rigorous contemporary psychometric tests, few have fared very well. However, the pruning procedure used in our study to produce the four-item version of the VAS provides an illustration of one strategy researchers can employ when confronting self-report measures that exhibit significant error covariance. Theory testing and model building Finally, the issues related to our findings have implications for scholars attempting to construct and evaluate theory. Researchers are interested in the relationships between constructs that self-report measures represent rather than the relationships between the measures themselves. Statistically distinct factors, whether attributable to method error or substantive constructs, indicate separate clusters of items accounting for different components of the measure's total score. When the factors are collapsed into a single unidimensional measure, a researcher cannot determine which aspect of the measure is responsible for correlations with other variables. For instance, whether the correlation between the VAS and another self-report measure is due to a relationship between the constructs or merely due to wording bias in both measures will be unclear. Scholars interested in clear delineation of relationships between constructs rely on construct validity estimates to link measures to the underlying constructs. In light of our findings and the contemporary literature on measurement, the construct validity of the VAS is a matter of speculation. Ideally, researchers should submit measures to CFA, inspecting possible error covariance patterns. At the very least, researchers should select measures, or items from measures, for which the independence error assumption has been met in previous studies. Appendix Below are the Verbal Aggressiveness Scale (VAS) items (A = indicates verbal aggressiveness, B = indicates verbal benevolence) developed by Infante and Wigley (1986). The scale features a five-option response format for each item (almost never true; rarely true; occasionally true; often true; and almost always true). Items indicating verbal benevolence are reverse coded before scoring the unidimensional VAS. I am extremely careful to avoid attacking individuals' intelligence when I attack their ideas. (B) When individuals are very stubborn, I use insults to soften the stubbornness. (A) I try very hard to avoid having other people feel bad about themselves when I try to influence them. (B) When people refuse to do a task I know is important, without good reason, I tell them they are unreasonable. (A) When others do things I regard as stupid, I try to be extremely gentle with them. (B) If individuals I am trying to influence really deserve it, I attack their character. (A) When people behave in ways that are in very poor taste, I insult them in order to shock them into proper behavior. (A) I try to make people feel good about themselves even when their ideas are stupid. (B) When people simply will not budge on a matter of importance I lose my temper and say rather strong things to them. (A) When people criticize my shortcomings, I take it in good humor and do not try to get back at them. (B) When individuals insult me, I get a lot of pleasure out of really telling them off. (A) When I dislike individuals greatly, I try not to show it in what I say or how I say it. (B) I like poking fun at people who do things which are very stupid in order to stimulate their intelligence. (A) When I attack persons' ideas, I try not to damage their self-concepts. (B) When I try to influence people, I make a great effort not to offend them. (B) When people do things which are mean or cruel, I attack their character in order to help correct their behavior. (A) I refuse to participate in arguments when they involve personal attacks. (B) When nothing seems to work in trying to influence others, I yell and scream in order to get some movement from them. (A) When I am not able to refute others' positions, I try to make them feel defensive in order to weaken their positions. (A) When an argument shifts to personal attacks, I try very hard to change the subject. (B) Notes 1 " The term congeneric originated in the early writing about confirmatory factor analysis (Jöreskog, 1971) and should not be confused with multidimensionality. Multidimensionality refers to multiple theoretically meaningful dimensions or facets of a construct. Noncongenericity can occur due to methodologically related variables not theoretically related to the construct such as common method error. 2 " To put this pattern in context, even if the matrix of correlations among item errors were examined as if it were a matrix of correlations among item scores, fewer than two significant correlations should have been expected due to chance at the p < .01 level. Moreover, because the implications of significant correlations among error terms differ from those for correlations among items scores, the matrices must be interpreted differently. Within a regular correlation matrix among scores, if only one correlation out of 100 reaches the critical value for the p < .01 level of significance associated with a single hypothesis test, the possibility that it occurred due to chance must be entertained. As such, interpreting the correlation as theoretically meaningful risks capitalizing on chance. Faced with this ambiguous result, the prudent choice is to attribute the finding to chance unless it can be replicated. We are uncertain, however, whether it is appropriate to apply the principles for interpreting correlations among scores to a matrix of correlations among item errors. Within a matrix of correlations among item errors, significant correlations at the p < .01 level, even within a 100-correlation matrix, represent either chance occurrences or significant influences of latent variables that confound the measure. In this context, attributing the correlation to chance risks using items in research that are confounded by outside influences. Therefore, the prudent, conservative choice is to hold the item suspect until further investigation can be undertaken. It is for these reasons that Kline (2005) and Brown (2006) pointed out that detecting correlated error terms for even one pair of items can be grounds for model respecification. 3 " Application of Raykov's (2001) formula results in lower reliability estimates than does alpha only to the extent that the item set is confounded by latent effects. Latent effects reduce validity. Because correcting correlations for attenuation assumes that measures are valid, corrections should be carried out only after the items affected by latent variables have been removed and the reliability of the measure is recalculated. Otherwise, correction for attenuation based on the original measure containing confounded items will result in inflated corrected correlations by capitalizing on variance contributed by outside variables to the supposed bivariate correlations. 4 " Like all previous tests of the VAS (Beatty et al., 1999; Infante & Wigley, 1986; Levine et al., 2004; Suzuki & Rancer, 1994), the model testing procedures employed in the present study assume linear relationships between items and latent variables. Although it is possible that the items on the VAS form a Guttman simplex, our purpose was not to discover the true structure of the 20-item VAS. Rather, the purpose of our study was to test the specific error theory underlying Infante and Wigley's (1986) original model. 5 " In part, the lack of replication for item pairs linked by error covariance is attributable to the alpha level set for error covariance within each study. We set alpha at .01 to minimize detecting error covariance by chance within the 20 × 20 correlation matrix. Had we set alpha at .05 instead, a large number of items linked by error covariance would have been common to both samples. Many pairs attaining p < .01 level in one study were significant at the p < .05 level but not the p < .01 level in the other study. This magnifies the differences between the two samples. Within each study, the difference between clusters is not likely due to sampling error. Many of the correlations between the error terms for items in one cluster and those in another were zero or near zero. Confidence intervals based on Study 1 data, for instance, revealed that the upper bound region of the 95% CI (.10) for r = .00 was just outside the lower bound 95% CI for r = .21. Several of the clusters consisted on two or more items with correlations between the error terms equal to or greater than .21. Although the error covariances between some items within clusters may be attributable to sampling error, it is unlikely that the detection of clusters of item covariance in general were due to sampling error. 6 " The results for Study 1 were χ2(2, N = 363) = 0.50, p < .77, RMSEA = 0.01 (90% CI [0.00, 0.07]), SRMR = 0.01, CFI = 1.00, TLI = 1.03, and for Study 2, χ2(2, N = 205) = 4.62, p < .10, RMSEA = 0.08 (90% CI [0.00, 0.18]), SRMR = 0.03, CFI = 0.97, TLI = 0.98. Except for a marginal factor loading for item #16 in Study 2 (i.e., .30), the factor loadings were acceptable, ranging from .70 to .48 in the two studies. Although a significant error covariance was initially detected between items 9 and 18, item 18 was retained because no significant error covariances with any other item were observed after the pruning process. Frequently, the covariance between items is due to a cuing effect on an item caused by a preceding item (Brown, 2006). If this is the case with these two items, removing item 9 from the questionnaire would eliminate the latent effect on item 18. However, researchers should inspect the psychometric properties of all four items before using them. References Anderson , J. C. , & Gerbing , D. W. ( 1982 ). Some methods for respecifying measurement models to obtain unidimensional construct measurement . Journal of Marketing Research , 19 , 453 – 460 . doi:10.2307/3151719 Google Scholar Crossref Search ADS WorldCat Bandura , A. ( 1973 ). Aggression: A social learning analysis . Englewood Cliffs, NJ : Prentice-Hall . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Barratt , E. S. , Stanford , M. S., Dowdy , L., Liebman , M. J., & Kent , T. A. ( 1999 ). Impulsive and premeditated aggression: A factor analysis of self-reported acts . Psychiatry Research , 86 , 163 – 173 . doi:10.1016/S0165-1781(99)00024-4 Google Scholar Crossref Search ADS PubMed WorldCat Beatty , M. J. , Rudd , J. E., & Valencic , K. M. ( 1999 ). A re-examination of the verbal aggressiveness scale: One factor or two? Communication Research Reports , 16 , 10 – 17 . doi:10.1080/08824099909388696 Google Scholar Crossref Search ADS WorldCat Berkowitz , L. ( 1962 ). Aggression: A social psychological analysis . New York, NY : McGraw-Hill . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Brown , T. A. ( 2003 ). Confirmatory factor analysis of the Penn State Worry Questionnaire: Multiple factors or method effects? Behavior Research and Therapy , 41 , 1411 – 1426 . doi:10.1016/S0005-7967(03)00059-7 Google Scholar Crossref Search ADS WorldCat Brown , T. A. ( 2006 ). Confirmatory factor analysis for applied research . New York, NY : Guilford Press . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Byrne , D. , Fisher , D., Lambert , J., & Mitchell , H. E. ( 1974 ). Evaluations of erotica: Facts or feelings? Journal of Personality and Social Psychology , 29 , 111 – 116 . doi:10.1037/h0035731 Google Scholar Crossref Search ADS PubMed WorldCat Cole , D. A. , Ciesla , J. A., & Steiger , J. H. ( 2007 ). The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis . Psychological Methods , 12 , 381 – 398 . doi:10.1037/1082-989X.12.4 Google Scholar Crossref Search ADS PubMed WorldCat Cronbach , L. J. ( 1951 ). Coefficient alpha and the internal structure of test . Psychometrika , 16 , 297 – 334 . doi:10.1007/BF02310555 Google Scholar Crossref Search ADS WorldCat Dodge , K. A. , & Coie , J. D. ( 1987 ). Social information processing factors in reactive and proactive aggression in children's peer groups . Journal of Personality and Social Psychology , 53 , 1146 – 1158 . doi:10.1037/0022-3514.53.6.114 Google Scholar Crossref Search ADS PubMed WorldCat Gaudry , E. , Vagg , P., & Spielberger , C. D. ( 1975 ). Validation of the state-trait distinction in anxiety research . Multivariate Behavioral Research , 10 , 331 – 341 . doi:10.1207/s15327906mbr1003_6 Google Scholar Crossref Search ADS PubMed WorldCat Gerbing , D. W. , & Anderson , J. C. ( 1984 ). On the meaning of within-factor correlated measurement errors . Journal of Consumer Behavior , 11 , 572 – 580 . doi:10.1086/208993 Google Scholar Crossref Search ADS WorldCat Guilford , J. P. ( 1954 ). Psychometric methods (2nd ed.). New York, NY : McGraw-Hill . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Hamilton , M. , & Hample , D. ( 2011 ). Testing hierarchical models of argumentativeness and verbal aggressiveness . Communication Methods and Measures , 5 , 250 – 273 . doi:10.1080/19312458.2011.596991 Google Scholar Crossref Search ADS WorldCat Hamilton , M. , & Tafoya , M. A. ( 2012 ). Toward a collective framework on verbal aggression: Hierarchical and antagonistic processes . Journal of Language and Social Psychology , 31 , 112 – 130 . doi:10.1177/0261927X11425038 Google Scholar Crossref Search ADS WorldCat Infante , D. A. , Rancer , A. S., & Wigley , C. J. ( 2011 ). In defense of the argumentativeness and verbal aggressiveness scales . Communication Quarterly , 59 , 145 – 154 . doi:10.1080/01463373.2011.563439 Google Scholar Crossref Search ADS WorldCat Infante , D. A. , & Wigley , C. J. III ( 1986 ). Verbal aggressiveness: An interpersonal model and measure . Communication Monographs , 53 , 63 – 69 . doi:10.1080/03637758609376126 Google Scholar Crossref Search ADS WorldCat Jöreskog , K. G. ( 1971 ). Statistical analysis of sets of congeneric tests . Psychometrika , 36 , 109 – 133 . doi:10.1007/BF02291393 Google Scholar Crossref Search ADS WorldCat Kingsbury , S. J. , Lambert , M. P., & Hendrickse , W. ( 1997 ). A two-factor model of aggression. Psychiatry, 60, 224–232. Retrieved from http://search.proquest.com/docview/220691673?accountid=14585 Kline , R. B. ( 2005 ). Principles and practices of structural equation modeling (2nd ed.). New York, NY : Guilford Press . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Kotowski , M. R. , Levine , T. R., Baker , C. R., & Bolt , J. M. ( 2009 ). A multitrait-multimethod validity assessment of the verbal aggressiveness and argumentativeness scales . Communication Monographs , 76 , 443 – 462 . doi:10.1080/03637750903300247 Google Scholar Crossref Search ADS WorldCat Levine , T. R. , Beatty , M. J., Limon , S., Hamilton , M. A., Buck , R., & Chory-Assad , R. M. ( 2004 ). The dimensionality of the verbal aggressiveness scale . Communication Monographs , 71 , 245 – 268 . doi:10.1080/0363452042000299911 Google Scholar Crossref Search ADS WorldCat Levine , T. R. , & Kotowski , M. R. ( 2010 ). Measuring argumentativeness and verbal aggressiveness: Psychometric concerns and advances . In T. A. Avtgis & A. S. Rancer (Eds.), Arguments, aggression, and conflict: New directions in theory and research (pp. 67 – 81 ). New York, NY : Routledge . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Levine , T. R. , Kotowski , M. R., Beatty , M. J., & Van Kelegom , M. J. ( 2012 ). A meta-analysis of trait-behavior correlations in argumentativeness and verbal aggression . Journal of Language and Social Psychology , 31 , 95 – 111 . doi:10.1177/0261927X11425037 Google Scholar Crossref Search ADS WorldCat Lord , F. M. , & Novick , M. R. ( 1968 ). Statistical theories of mental scores . Reading, MA : Addison-Wesley . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC McCallum , R. C. ( 1986 ). Specification searches in covariance structure modeling . Psychological Bulletin , 100 , 107 – 120 . doi:10.1037/0033-2909.100.1.107 Google Scholar Crossref Search ADS WorldCat Marsh , H. W. ( 1996 ). Positive and negative self-esteem: A substantively meaningful distinction or artifactors? Journal of Personality and Social Psychology , 70 , 810 – 819 . doi:10.1037/0022-3514.70.4.810 Google Scholar Crossref Search ADS PubMed WorldCat Marsh , H. W. , & Smith , I. D. ( 1982 ). Multitrait-multimethod analyses of two self-concept instruments . Journal of Educational Psychology , 74 , 430 – 440 . doi:10.1037/0022-0663.74.3.430 Google Scholar Crossref Search ADS WorldCat Mathias , C. W. , Stanford , M. S., Marsh , D. M., Frick , T. J., Moeller , F. G., Swann , A. C., & Dougherty , D. M. ( 2007 ). Characterizing aggressive behavior with the impulsive/premeditated aggression scale among adolescents with conduct disorder . Psychiatry Research , 151 , 231 – 242 . doi:10.1016/j.psychres.2006.11.001 Google Scholar Crossref Search ADS PubMed WorldCat Meyer , J. P. ( 2010 ). Reliability . London, England : Oxford University Press . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Podsakoff , P. M. , MacKenzie , S. B., & Podsakoff , N. ( 2012 ). Sources of method bias in social science research and recommendations on how to control it . Annual Review of Psychology , 63 , 539 – 569 . doi:10.1146/annurev-psych-120710-100452 Google Scholar Crossref Search ADS PubMed WorldCat Podsakoff , P. M. , MacKenzie , S. B., Lee , J. Y., & Podsakoff , N. ( 2003 ). Common method biases in behavioral research: A critical review of the literature and recommended remedies . Journal of Applied Psychology , 88 , 879 – 903 . doi:10.1037/0021-9010.88.5.879 Google Scholar Crossref Search ADS PubMed WorldCat Poulin , F. , & Boivin , M. ( 2000 ). Reactive and proactive aggression: Evidence of a two-factor model . Psychological Assessment , 12 , 115 – 122 . doi:10.1037/1040-3590.12.2.115 Google Scholar Crossref Search ADS PubMed WorldCat Raykov , T. ( 2001 ). Bias of coefficient α for fixed congeneric measures with correlated errors . Applied Psychological Measurement , 25 , 69 – 76 . doi:10.1177/01466216010251005 Google Scholar Crossref Search ADS WorldCat Suzuki , S. , & Rancer , A. S. ( 1994 ). Argumentativeness and verbal aggressiveness: Testing for conceptual and measurement equivalence across cultures . Communication Monographs , 61 , 256 – 279 . doi:10.1080/03637759409376336 Google Scholar Crossref Search ADS WorldCat Valencic , K. M. , Beatty , M. J., Rudd , J. E., Dobos , J. A., & Heisel , A. D. ( 1998 ). An empirical test of a communibiological model of trait verbal aggressiveness . Communication Quarterly , 46 , 327 – 341 . doi:10.1080/01463379809370105 Google Scholar Crossref Search ADS WorldCat Wigley , C. J. III ( 2008 ). Verbal aggression interventions: What should be done? Communication Monographs , 75 , 339 – 350 . doi:10.1080/03637750802524269 OpenURL Placeholder Text WorldCat Wigley , C. J. III ( 2010 ). Verbal trigger events—other catalysts and precursors to aggression . In T. A. Avtgis & A. S. Rancer (Eds.), Arguments, aggression, and conflict: New directions in theory and research (pp. 26 – 43 ). New York, NY : Routledge . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC Author notes " Corrections made after online publication August 17, 2014: author names have been updated. © 2014 International Communication Association
TI - Two Studies Examining the Error Theory Underlying the Measurement Model of the Verbal Aggressiveness Scale
JO - Human Communication Research
DO - 10.1111/hcre.12039
DA - 2015-01-01
UR - https://www.deepdyve.com/lp/oxford-university-press/two-studies-examining-the-error-theory-underlying-the-measurement-nRMUhWeB9o
SP - 55
VL - 41
IS - 1
DP - DeepDyve
ER -