British Journal of Mathematical and Statistical Psychology

Brusco, Michael J.; Shireman, Emilie; Steinley, Douglas; Brudvig, Susan; Cradit, J. Dennis

doi: 10.1111/bmsp.12084pmid: 28130935

The emergence of Gaussian model‐based partitioning as a viable alternative to K‐means clustering fosters a need for discrete optimization methods that can be efficiently implemented using model‐based criteria. A variety of alternative partitioning criteria have been proposed for more general data conditions that permit elliptical clusters, different spatial orientations for the clusters, and unequal cluster sizes. Unfortunately, many of these partitioning criteria are computationally demanding, which makes the multiple‐restart (multistart) approach commonly used for K‐means partitioning less effective as a heuristic solution strategy. As an alternative, we propose an approach based on iterated local search (ILS), which has proved effective in previous combinatorial data analysis contexts. We compared multistart, ILS and hybrid multistart–ILS procedures for minimizing a very general model‐based criterion that assumes no restrictions on cluster size or within‐group covariance structure. This comparison, which used 23 data sets from the classification literature, revealed that the ILS and hybrid heuristics generally provided better criterion function values than the multistart approach when all three methods were constrained to the same 10‐min time limit. In many instances, these differences in criterion function values reflected profound differences in the partitions obtained.

Vera, José Fernando

doi: 10.1111/bmsp.12079pmid: 27996084

Stability or sensitivity analysis is an important topic in data analysis that has received little attention in the application of multidimensional scaling (MDS), for which the only available approaches are given in terms of a coordinate‐based analytical jackknife methodology. Although in MDS the prime interest is in assessing the stability of the points in the configuration, this methodology may be influenced by imprecisions resulting from the inherently necessary Procrustes method. This paper proposes an analytical distance‐based jackknife procedure to study stability and cross‐validation in MDS in terms of the jackknife distances, which is not influenced by the Procrustes method. For each object, the corresponding jackknife estimated points are considered as naturally clustered points, and stability and cross‐validation are analysed in terms of the MDS distances arising from the jackknife procedure, on the basis of a weighted cluster‐MDS algorithm. A jackknife‐relevant configuration is also proposed for cross‐validation in terms of coordinates, in a cluster‐MDS framework.

Holtmann, Jana; Koch, Tobias; Bohn, Johannes; Eid, Michael

doi: 10.1111/bmsp.12081pmid: 28116783

A new multilevel latent state graded response model for longitudinal multitrait–multimethod (MTMM) measurement designs combining structurally different and interchangeable methods is proposed. The model allows researchers to examine construct validity over time and to study the change and stability of constructs and method effects based on ordinal response variables. We show how Bayesian estimation techniques can address a number of important issues that typically arise in longitudinal multilevel MTMM studies and facilitates the estimation of the model presented. Estimation accuracy and the impact of between‐ and within‐level sample sizes as well as different prior specifications on parameter recovery were investigated in a Monte Carlo simulation study. Findings indicate that the parameters of the model presented can be accurately estimated with Bayesian estimation methods in the case of low convergent validity with as few as 250 clusters and more than two observations within each cluster. The model was applied to well‐being data from a longitudinal MTMM study, assessing the change and stability of life satisfaction and subjective happiness in young adults after high‐school graduation. Guidelines for empirical applications are provided and advantages and limitations of a Bayesian approach to estimating longitudinal multilevel MTMM models are discussed.

Chen, Ping; Wang, Chun; Xin, Tao; Chang, Hua‐Hua

doi: 10.1111/bmsp.12083pmid: 28130937

Multidimensional computerized adaptive testing (MCAT) has received increasing attention over the past few years in educational measurement. Like all other formats of CAT, item replenishment is an essential part of MCAT for its item bank maintenance and management, which governs retiring overexposed or obsolete items over time and replacing them with new ones. Moreover, calibration precision of the new items will directly affect the estimation accuracy of examinees’ ability vectors. In unidimensional CAT (UCAT) and cognitive diagnostic CAT, online calibration techniques have been developed to effectively calibrate new items. However, there has been very little discussion of online calibration in MCAT in the literature. Thus, this paper proposes new online calibration methods for MCAT based upon some popular methods used in UCAT. Three representative methods, Method A, the ‘one EM cycle’ method and the ‘multiple EM cycles’ method, are generalized to MCAT. Three simulation studies were conducted to compare the three new methods by manipulating three factors (test length, item bank design, and level of correlation between coordinate dimensions). The results showed that all the new methods were able to recover the item parameters accurately, and the adaptive online calibration designs showed some improvements compared to the random design under most conditions.

Li, Xinru; Dusseldorp, Elise; Meulman, Jacqueline J.

doi: 10.1111/bmsp.12088pmid: 28130936

In the framework of meta‐analysis, moderator analysis is usually performed only univariately. When several study characteristics are available that may account for treatment effect, standard meta‐regression has difficulties in identifying interactions between them. To overcome this problem, meta‐CART has been proposed: an approach that applies classification and regression trees (CART) to identify interactions, and then subgroup meta‐analysis to test the significance of moderator effects. The previous version of meta‐CART has its shortcomings: when applying CART, the sample sizes of studies are not taken into account, and the effect sizes are dichotomized around the median value. Therefore, this article proposes new meta‐CART extensions, weighting study effect sizes by their accuracy, and using a regression tree to avoid dichotomization. In addition, new pruning rules are proposed. The performance of all versions of meta‐CART was evaluated via a Monte Carlo simulation study. The simulation results revealed that meta‐regression trees with random‐effects weights and a 0.5‐standard‐error pruning rule perform best. The required sample size for meta‐CART to achieve satisfactory performance depends on the number of study characteristics, the magnitude of the interactions, and the residual heterogeneity.

Sijtsma, Klaas; Ark, L. Andries

doi: 10.1111/bmsp.12078pmid: 27958642

Over the past decade, Mokken scale analysis (MSA) has rapidly grown in popularity among researchers from many different research areas. This tutorial provides researchers with a set of techniques and a procedure for their application, such that the construction of scales that have superior measurement properties is further optimized, taking full advantage of the properties of MSA. First, we define the conceptual context of MSA, discuss the two item response theory (IRT) models that constitute the basis of MSA, and discuss how these models differ from other IRT models. Second, we discuss dos and don'ts for MSA; the don'ts include misunderstandings we have frequently encountered with researchers in our three decades of experience with real‐data MSA. Third, we discuss a methodology for MSA on real data that consist of a sample of persons who have provided scores on a set of items that, depending on the composition of the item set, constitute the basis for one or more scales, and we use the methodology to analyse an example real‐data set.

Böckenholt, Ulf; Meiser, Thorsten

doi: 10.1111/bmsp.12086pmid: 28130934

Two different item response theory model frameworks have been proposed for the assessment and control of response styles in rating data. According to one framework, response styles can be assessed by analysing threshold parameters in Rasch models for ordinal data and in mixture‐distribution extensions of such models. A different framework is provided by multi‐process item response tree models, which can be used to disentangle response processes that are related to the substantive traits and response tendencies elicited by the response scale. In this tutorial, the two approaches are reviewed, illustrated with an empirical data set of the two‐dimensional ‘Personal Need for Structure’ construct, and compared in terms of multiple criteria. Mplus is used as a software framework for (mixed) polytomous Rasch models and item response tree models as well as for demonstrating how parsimonious model variants can be specified to test assumptions on the structure of response styles and attitude strength. Although both frameworks are shown to account for response styles, they differ on the quantitative criteria of model selection, practical aspects of model estimation, and conceptual issues of representing response styles as continuous and multidimensional sources of individual differences in psychological assessment.

doi: 10.1111/bmsp.12093pmid: N/A