British Journal of Mathematical and Statistical Psychology

doi: 10.1111/bmsp.12247pmid: N/A

Welz, Thilo; Doebler, Philipp; Pauly, Markus

doi: 10.1111/bmsp.12242pmid: 33934346

Meta‐analyses of correlation coefficients are an important technique to integrate results from many cross‐sectional and longitudinal research designs. Uncertainty in pooled estimates is typically assessed with the help of confidence intervals, which can double as hypothesis tests for two‐sided hypotheses about the underlying correlation. A standard approach to construct confidence intervals for the main effect is the Hedges‐Olkin‐Vevea Fisher‐z (HOVz) approach, which is based on the Fisher‐z transformation. Results from previous studies (Field, 2005, Psychol. Meth., 10, 444; Hafdahl and Williams, 2009, Psychol. Meth., 14, 24), however, indicate that in random‐effects models the performance of the HOVz confidence interval can be unsatisfactory. To this end, we propose improvements of the HOVz approach, which are based on enhanced variance estimators for the main effect estimate. In order to study the coverage of the new confidence intervals in both fixed‐ and random‐effects meta‐analysis models, we perform an extensive simulation study, comparing them to established approaches. Data were generated via a truncated normal and beta distribution model. The results show that our newly proposed confidence intervals based on a Knapp‐Hartung‐type variance estimator or robust heteroscedasticity consistent sandwich estimators in combination with the integral z‐to‐r transformation (Hafdahl, 2009, Br. J. Math. Stat. Psychol., 62, 233) provide more accurate coverage than existing approaches in most scenarios, especially in the more appropriate beta distribution simulation model.

Katsikatsou, Myrsini; Moustaki, Irini; Jamil, Haziq

doi: 10.1111/bmsp.12243pmid: 33856692

Methods for the treatment of item non‐response in attitudinal scales and in large‐scale assessments under the pairwise likelihood (PL) estimation framework and under a missing at random (MAR) mechanism are proposed. Under a full information likelihood estimation framework and MAR, ignorability of the missing data mechanism does not lead to biased estimates. However, this is not the case for pseudo‐likelihood approaches such as the PL. We develop and study the performance of three strategies for incorporating missing values into confirmatory factor analysis under the PL framework, the complete‐pairs (CP), the available‐cases (AC) and the doubly robust (DR) approaches. The CP and AC require only a model for the observed data and standard errors are easy to compute. Doubly‐robust versions of the PL estimation require a predictive model for the missing responses given the observed ones and are computationally more demanding than the AC and CP. A simulation study is used to compare the proposed methods. The proposed methods are employed to analyze the UK data on numeracy and literacy collected as part of the OECD Survey of Adult Skills.

Wilcox, Rand R.

doi: 10.1111/bmsp.12244pmid: 33950536

Consider a two‐way ANOVA design. Generally, interactions are characterized by the difference between two measures of effect size. Typically the measure of effect size is based on the difference between measures of location, with the difference between means being the most common choice. This paper deals with extending extant results to two robust, heteroscedastic measures of effect size. The first is a robust, heteroscedastic analogue of Cohen's d. The second characterizes effect size in terms of the quantiles of the null distribution. Simulation results indicate that a percentile bootstrap method yields reasonably accurate confidence intervals. Data from an actual study are used to illustrate how these measures of effect size can add perspective when comparing groups.

Mair, Patrick; Gruber, Kathrin

doi: 10.1111/bmsp.12245pmid: 34089620

In this article we extend the framework of explanatory mixed IRT models to a more general class called explanatory additive IRT models. We do this by augmenting the linear predictors in terms of smooth functions. This development offers many new modeling options such as the inclusion of nonlinear covariate effects, the specification of various temporal and spatial dependency patterns, and parameter partitioning across covariates. We use integrated nested Laplace approximation (INLA) for accurate and computationally efficient estimation of the parameters. Uninformative, weakly informative, and informative prior settings for the hyperparameters are discussed. Running time experiments and Monte Carlo parameter recovery simulations are performed in order to study the accuracy and computational efficiency of INLA when applied to the proposed explanatory additive IRT model class. Using a real‐life dataset, a variety of application scenarios is explored, and the results are compared with classical maximum likelihood estimation when possible. R code is included in the supplemental materials to allow readers to fully reproduce the examples computed in the paper.

Gheondea‐Eladi, Alexandra; Gheondea, Aurelian

doi: 10.1111/bmsp.12246pmid: 34228357

In a previous paper, the evolution of certainty measured during a consensus‐based small‐group decision process was shown to oscillate to an equilibrium value for about two‐thirds of the participants in the experiment. Starting from the observation that experimental participants are split into two groups, those for whom the evolution of certainty oscillates and those for whom it does not, in this paper we perform an analysis of this bifurcation with a more accurate model and answer two main questions: what is the meaning of this bifurcation, and is this bifurcation amenable to the approximation method or numerical procedure? Firstly, we have to refine the mathematical model of the evolution of certainty to a function explicitly represented in terms of the model parameters and to verify its robustness to the variation of parameters, both analytically and by computer simulation. Then, using the previous group decision experimental data, and the model proposed in this paper, we run the curve‐fitting software on the experimental data. We also review a series of interpretations of the bifurcated behaviour. We obtain a refined mathematical model and show that the empirical results are not skewed by the initial conditions, when the proposed model is used. Thus, we reveal the analytical and empirical existence of the observed bifurcation. We then propose that sensitivity to the absolute value of certainty and to its rate of change are considered as potential interpretations of this split in behaviour, along with certainty/uncertainty orientation, uncertainty interpretation, and uncertainty/certainty‐related intuition and affect.

Lyu, Weicong; Bolt, Daniel M.

doi: 10.1111/bmsp.12251pmid: 34350978

Among the various forms of response bias that can emerge with self‐report rating scale assessments are those related to anchoring, the tendency for respondents to select categories in close proximity to the rating category used for the immediately preceding item. In this study we propose a psychometric model based on a multidimensional nominal model for response style that also simultaneously accommodates a respondent‐level anchoring tendency. The model is estimated using a fully Bayesian estimation procedure. By applying this model to a real test data set measuring extraversion, we explore a theory that both response styles and anchoring might be viewed as evidence of a lack of effortful responding. Empirical results show that there is a positive correlation between the strength of midpoint response style and the anchoring effect; further, responses indicative of either anchoring or response style both negatively correlate with response time, consistent with a theory that both phenomena reflect reduced respondent effort. The results support attending to both anchoring and midpoint response style as ways of assessing respondent engagement.

Kang, Hyeon‐Ah; Han, Suhwa; Betts, Joe; Muntean, William

doi: 10.1111/bmsp.12252pmid: 34462913

Increasing use of innovative items in operational assessments has shedded new light on the polytomous testlet models. In this study, we examine performance of several scoring models when polytomous items exhibit random testlet effects. Four models are considered for investigation: the partial credit model (PCM), testlet‐as‐a‐polytomous‐item model (TPIM), random‐effect testlet model (RTM), and fixed‐effect testlet model (FTM). The performance of the models was evaluated in two adaptive testings where testlets have nonzero random effects. The outcomes of the study suggest that, despite the manifest random testlet effects, PCM, FTM, and RTM perform comparably in trait recovery and examinee classification. The overall accuracy of PCM and FTM in trait inference was comparable to that of RTM. TPIM consistently underestimated population variance and led to significant overestimation of measurement precision, showing limited utility for operational use. The results of the study provide practical implications for using the polytomous testlet scoring models.

Liu, Siwei; Rhemtulla, Mijke

doi: 10.1111/bmsp.12253pmid: 34632565

Random effects in longitudinal multilevel models represent individuals’ deviations from population means and are indicators of individual differences. Researchers are often interested in examining how these random effects predict outcome variables that vary across individuals. This can be done via a two‐step approach in which empirical Bayes (EB) estimates of the random effects are extracted and then treated as observed predictor variables in follow‐up regression analyses. This approach ignores the unreliability of EB estimates, leading to underestimation of regression coefficients. As such, previous studies have recommended a multilevel structural equation modeling (ML‐SEM) approach that treats random effects as latent variables. The current study uses simulation and empirical data to show that a bias–variance tradeoff exists when selecting between the two approaches. ML‐SEM produces generally unbiased regression coefficient estimates but also larger standard errors, which can lead to lower power than the two‐step approach. Implications of the results for model selection and alternative solutions are discussed.

Verkuilen, Jay

doi: 10.1111/bmsp.12254pmid: 34687451

This article examines the Fisher information functions, I(θ), and explores implications for scoring of binary ideal point item response models. These models typically appear to have I(θ) that are bimodal and identically equal to 0 at the ideal point. The article shows that this is an inherent property of ideal point IRT models, which either have this property or are indeterminate and thus violate the likelihood regularity conditions. For some models, the indeterminacy can be resolved, generating an effectively unimodal I(θ), albeit with violated regularity conditions. In other cases, I(θ) diverges. All reasonable ideal point IRT models exhibit this behaviour. Users should exercise caution when relying on asymptotics, particularly for shorter assessments. Use of simulated plausible values or prediction from a fully Bayesian estimation is recommended for scoring.