Access the full text.
Sign up today, get DeepDyve free for 14 days.
Loading next page...
/lp/sage/a-note-on-the-relationship-of-the-shannon-entropy-procedure-and-the-cyG300FhQH
- Publisher
- SAGE
- Copyright
- Copyright © 2022 by SAGE Publications Inc, unless otherwise noted. Manuscript content on this site is licensed under Creative Commons Licenses.
- ISSN
- 2158-2440
- eISSN
- 2158-2440
- DOI
- 10.1177/2158244019899046
- Publisher site
- See Article on Publisher Site
Abstract
The purpose of this study is to investigate the relationship between the Shannon entropy procedure and the Jensen–Shannon divergence (JSD) that are used as item selection criteria in cognitive diagnostic computerized adaptive testing (CD-CAT). Because the JSD itself is defined by the Shannon entropy, we apply the well-known relationship between the JSD and Shannon entropy to establish a relationship between the item selection criteria that are based on these two measures. To understand the relationship between these two item selection criteria better, an alternative way is also provided. Theoretical derivations and empirical examples have shown that the Shannon entropy procedure and the JSD in CD-CAT have a linear relation under cognitive diagnostic models. Consistent with our theoretical conclusions, simulation results have shown that two item selection criteria behaved quite similarly in terms of attribute-level and pattern recovery rates under all conditions and they selected the same set of items for each examinee from an item bank with item parameters drawn from a uniform distribution U(0.1, 0.3) under post hoc simulations. We provide some suggestions for future studies and a discussion of relationship between the modified posterior-weighted Kullback–Leibler index and the G-DINA (generalized deterministic inputs, noisy “and” gate) discrimination index. Keywords cognitive diagnosis models, computerized adaptive testing, item selection methods, the Shannon entropy procedure, the Jensen–Shannon divergence Summative assessments are typically used for grading and mandated the selection and use of diagnostic assessments to accountability purposes, and formative assessments are often improve teaching and learning and the new federal grant pro- used for supporting student learning (Laveault & Allal, gram known as “Race to the Top” (RTTT) has led into a new 2016). Researchers and practitioners began to focus on for- era of K–12 assessments which emphasized both account- mative assessments for student learning, rather than focus ability and instructional improvement (Chang, 2012). solely on summative assessments because many evidences Computerized adaptive testing (CAT) has become a popu- showed that formative assessments produce significant and lar mode of many summative and formative assessments often substantial learning gains and improve student confi- (Quellmalz & Pellegrino, 2009). As a method of administer- dence and achievement (Black & Wiliam, 1998; Laveault & ing test items, CAT tailors the item difficulty to the ability Allal, 2016). Cognitive diagnosis assessment (CDA) can be level of the individual examinee (Chang & Ying, 2007). It is regarded as a kind of formative assessments because it is attractive to practitioners because it yields a high measure- intended to promote assessment for learning to modify ment precision with a short test. In the framework of CAT, instruction and learning in classrooms by providing the for- mative diagnostic information about students’ cognitive Jiangxi Normal University, Nanchang, China strengths and weaknesses (Jang, 2008; Leighton & Gierl, Corresponding Author: 2007). CDA has received increasing attention in recent years Lihong Song, Elementary Education College, Jiangxi Normal University, 99 (Leighton & Gierl, 2007; Rupp et al., 2010; K. K. Tatsuoka, Ziyang Avenue, Nanchang, Jiangxi 330022, China. Email: viviansong1981@163.com 2009), especially since the No Child Left Behind Act of 2001 Creative Commons CC BY: This article is distributed under the terms of the Creative Commons Attribution 4.0 License (https://creativecommons.org/licenses/by/4.0/) which permits any use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access pages (https://us.sagepub.com/en-us/nam/open-access-at-sage). 2 SAGE Open cognitive diagnostic computerized adaptive testing (CD-CAT) of general CDMs are described here. CDMs have been is also a popular mode of online testing for cognitive diagno- defined by Rupp and Templin (2008) as “probabilistic, con- sis, as it can help one make informed decisions about the next firmatory multidimensional latent variable models with a steps in instruction for each student and greatly facilitate indi- simple or complex loading structure” (p. 226). The loading vidualized learning (Chang, 2015) and provide many benefits structure for a CDM is represented by its Q-matrix (K. K. to support formative assessments (Gierl & Lai, 2018). Tatsuoka, 1983, 2009). The entries of a Q-matrix indicate 1 Particularly, the U.S. National Education Technology Plan or 0, in which q =1 when item j involves attribute k for jk 2017 with the title of “Reimagining the Role of Technology in answering item j correctly and otherwise. CDMs q = 0 jk Education” (U.S. Department of Education, 2017) empha- define an item response function of a Q-matrix, examinee’s sizes that technology can help us redefine assessment to meet discrete latent variables, and item parameters to predict the the needs of the learner in a variety of ways. For technology- probability of an observable categorical response to an item. based formative assessments or CAT, test items are adapted to This study only focuses on CDMs designed to handle dichot- learner’s ability and knowledge during the testing process. omous responses. For a dichotomous CDM, the form of an Thus, CAT can provide real-time reporting of results during item response function for a binary response variable is the instructional process, which is crucial for personalized denoted by PP () αα == (| Uu ,, q β ) , where u∈{, 01}, ju iiji jj learning (Chen & Chang, 2018). q is the j th row of a Q-matrix, and β is item parameters for j j A key ingredient in CD-CAT is the item selection index. item j. Note that this study will use P () α to discuss the ju i Researchers have attempted to investigate many item selec- theoretical relationship between the SHE procedure and the tion indices. The first type of index is based on the Kullback– JSD index. While the G-DINA model and other CDMs are Leibler (KL) information, such as the KL index (Cheng, described below only for showing details about different 2009; McGlohen & Chang, 2008; C. Tatsuoka & Ferguson, item response functions for dichotomous CDMs. 2003; Xu et al., 2003), the likelihood- or posterior-weighted Let Kq = denote the number of required attri- j ∑ jk k=1 KL (LWKL or PWKL) index and the hybrid KL index (Cheng, butes for item j, where K is the number of attributes of a test. 2009), the restrictive progressive or threshold PWKL index K For K attributes, there are 2 distinct attribute patterns in (Wang et al., 2011), the aggregate ranked information index the universal set of knowledge states, attribute patterns, or and the aggregate standardized information index (Wang latent classes. Let α denote an attribute pattern from the et al., 2014), the modified PWKL (MPWKL) index (Kaplan universal set of knowledge states. In the G-DINA model, et al., 2015), the KL expected discrimination index (W. Y. * K item j that measures K attributes partitions the 2 distinct Wang et al., 2015), the posterior-weighted cognitive diagnos- j attribute patterns into 2 latent groups. To simplify the tic model (CDM) discrimination index and the posterior- * notation, let α be the reduced attribute pattern of the full ij weighted attribute-level CDM discrimination index (Zheng & attribute pattern α with respect to the required attributes for Chang, 2016), and the information product index (Zheng item j. Let be the response of examinee i to item j. We ij et al., 2018). The second is based on the Shannon entropy, * define PP () αα == (| Uu ) to be the probability distri- ju iijij called the Shannon entropy (SHE) procedure (Cheng, 2009; bution of the binary random variable U , where P () α and ij ji 1 McGlohen & Chang, 2008; C. Tatsuoka, 2002; C. Tatsuoka & are the probabilities of getting a right PP () αα =− 1 () ji 01 ji Ferguson, 2003; Xu et al., 2003, 2016). The third is based on answer and wrong answer on item j by examinee i with full the mutual information, including the expected mutual infor- * attribute pattern α or reduced attribute pattern α . For the i ij mation index (Wang, 2013) and the Jensen–Shannon diver- G-DINA model, the probability of correctly answering item gence (JSD) index (Kang et al., 2017; Minchen & de la Torre, by examinee i is given by de la Torre (2011) and Ma et al. 2016; Yigit et al., 2018). There are other indices, such as the (2016). generalized deterministic inputs, noisy “and” gate (G-DINA; de la Torre, 2011) model discrimination index (GDI; Kaplan et al., 2015), the rate function approach (Liu et al., 2015), the gP () αδ =+ δα ji 10 jj ∑ 1 ik halving algorithm (C. Tatsuoka & Ferguson, 2003; W. Y. k =1 * * Wang et al., 2015; Zheng & Wang, 2017), and so on. Yigit K −1 1 K K j j j * ** et al. (2018) has proved that the mutual information index and + δα αδ ++ ... α , ∑ ∑ ∏ jkki ’ k ik ’ ik jK 12 k =1 kk ’=+1 k =1 the JSD index are equivalent. Although the previous simula- (1) tion studies showed that the SHE and the JSD or mutual infor- mation perform quite similarly, the main purpose of this study where δ is the intercept for item , δ is the main effect j0 j1 * * * is to describe the theoretical relationship between the SHE due to α , δ is the interaction effect due to α and α ik jkk ’ ik ik ’ ** procedure and the JSD index. and δ is the interaction effect due to αα ,..., . In * * i1 jK 12 iK j j addition, link function can be formulated using gP [(α )] ju i the logit, log, and identity links. The logit link results in a CDMs general model which is equivalent to the log-linear CDM Before introducing item selection indices for CD-CAT, the (Henson et al., 2009) and can be viewed as a special case of general concept of CDMs and the G-DINA model as a kind the general diagnostic model (GDM; von Davier, 2005, Wang et al. 3 2008). The resulting model from the log link function is Assuming that πα () is an updated prior probability it , c referred to as the log CDM (de la Torre, 2011). distribution and Uu = is an item response for candidate ij () t For the identity link, that is gP [(αα )] = P () , the j R item in , the posterior distribution it , +1 then becomes ju ijui “deterministic input; noisy ‘and’ gate” (DINA) model (de la Torre & Douglas, 2004; Haertel, 1989; Junker & Sijtsma, πα P α () () () t it , cjuc πα == πα |, u Uu = . () 2001), the “deterministic input; noisy ‘or’ gate” (DINO) it ,, ++ 11 ci tc () i ij K model (Templin & Henson, 2006), and the additive CDM πα P α () () ∑ it , c jju c c=1 (A-CDM) can be obtained from the G-DINA model when appropriate constraints are applied. For example, the item (4) response function of the DINA model is j * () t P () αδ =+ δα , by setting all lower-order From the last term, πα (| u ,) Uu = can be rewritten ji 10 j ∏ ik it , +1 ci ij jK 12 k =1 () t () t interaction terms to zero and by taking δ = g and as πα (| Uu = ) , where α has an updated prior dis- jj 0 it , +1 c ij c δ =− 1 sg − g . The parameter is the probability of * tribution π . jj j jK 12 j i,t correctly guessing the answer if an examinee lacks at least By considering the uncertainty of item response ij , the mar- one of the required attributes, and the parameter s refers to ginal probability distribution of item response on item given the probability of slipping and incorrectly answering the the probability distribution π can be computed as follows i,t item if an examinee has mastered all the required attributes. The DINA model is a parsimonious and interpretable model t t () () PU = uP |, uu == Uu α | () () ij i ∑ ij ci that requires only two parameters for each item regardless of c=1 the number of attributes being considered. (5) = πα P α , () () ∑ it , cjuc c=1 Overview of Two Item Selection Indices for CD-CAT where the second term follows directly from () t PU (| == uP u ,) αα (| Uu = ), ij i cijc which can be derived SHE Procedure from the assumption of local independence; as the current After an item bank has been calibrated with a CDM, one posterior distribution π can be viewed as a new prior for i,t () t () t t () must determine how to choose items for examinees from the α after having seen test data u , PU (| = u u ) can be c i ij i () t () t () t item bank. CD-CAT employs algorithms to select items simplified to PU () == uP πα () () α when α ij ∑ it , c ju c c c=1 sequentially on the basis of examinee’s responses, which is is substituted for α in the last term. designed to classify student’s attribute pattern accurately The next item to be selected for examinee i by the SHE () t with a short test. The SHE procedure (Cheng, 2009) and the is the one in R that minimizes the expected SHE: JSD index (Minchen & de la Torre, 2016; Yigit et al., 2018) are described below. () t () t SHEP == Uu || uu HU πα ,. = u (6) Suppose that the prior is chosen as πα () for attribute ij ∑ () ij i () it , +1 () ci ij 0 c K u=0 pattern α , where c = 12 ,,..., 2 . For examinee i , suppose that t items are selected, the vector of corresponding item () t () t () t () t As shown above, PU (| = u u ) and πα (| u ,) Uu = , responses is denoted as u , and a set R represents the set ij i it , +1 ci ij i i () t respectively, become PU () = u and πα (| Uu = ) . of available items at this stage. The posterior distribution ij it , +1 c ij () t () t () t Let HU (| α ) be the conditional entropy of α given πα (| u ) then becomes c ij c ci U . From the definition of conditional entropy (Cover & ij () t HU (| α ) Thomas, 2006), is defined as the weighted sum () t c ij πα () L u | α () () t 0 ci c () t of HU (| α= u) over each possible value of u taken by πα = πα | u = , () c ij () it , cc i K (2) 2 2 the random variable U , using PU () = u as the weights. () t ij ij πα L u | α () () ∑ 0 ci c SHE Thus, ij can be considered as the conditional entropy c=1 () t () t HU (| α ) of given U . c ij ij () t where is the likelihood function, and it is the L(| u α ) i c JSD Index product of each item response function if the assumptions of local independence are satisfied. The SHE of the posterior The JSD as a new class of information measures based on the distribution π can then be written as i,t SHE was introduced by Lin (1991) to measure the overall dif- ference of any finite number of distributions. Let PP () αα ,( ),..., P () α be 2 item response functions ju 12 ju ju H ππ =− απ log. α 2 () () () it ,, ∑ it c it , c (3) with weights of prior probabilities πα (),( πα ),..., c=1 it ,, 12 it 4 SAGE Open πα () , respectively. By the definition of the generalized CD-CAT are linearly related. Because the JSD itself is it , JSD in Equation 5.1 of the paper of Lin (1991), or from defined by the SHE, we apply the well-known relationship Equations A.3, A.4, and A.5 in online appendices of the paper between the mutual information (or JSD) and SHE to estab- of Yigit et al. (2018), the JSD for item can be written as lish a relationship between the item selection criteria that are developed using these two measures. The mutual informa- tion and SHE satisfy two well-known equations 2.43 and () JSDH == PU uH |, u − πα P α () () () () ij () ij i ∑ it , cjuc (7) 2.44 from Theorem 2.4.1 in Cover and Thomas (2006, p. 21); c=1 that is, and IX (;YH )( =− Y ) IX (;YH )( =− XH )(XY |) , where and are respectively HY (| X ) IX (;Y ) HX (|Y ) where mutual information and conditional entropy for two random () t variables X and Y . Let XU = and Y =α , with probabil- 1 2 ij c () t () t HP Uu = |l u =− πα () P () α og ity distributions PU () = u and π . As JSDI = (; U α ) () () ij i ∑ ∑ it , cjuc ij ij ij c i,t u =0 c=1 was proved in Yigit et al. (2018), we have K () t () t () t JSDI == (; UH αα )( )( − HU α |) , which follows (8) ij ij c c c ij π α αα P , () () ∑ it , cjuc directly from the second well-known equation. As shown in c=1 SHE the “SHE Procedure” section, can be written as the ij () t conditional entropy HU (| α ) . Thus, we have JSD = c ij ij and HS () π− HE it , ij Next, we will provide an alternative way to prove the above statement, which would be useful for a better under- πα HP α = () () () it , cjuc (9) standing of the relation. For simplicity, let the denominators c=1 or the normalizing constants of Equations 2 and 4 be 2 1 2 2 − πα () PP () αα log () . () t ∑ it , cj ∑ uc ju c CL = πα () (| u α ) and CP = πα () () α . 10 ∑ ci c 2 ∑ it , cjuc c=1 u =0 c=1 c=1 Note that the right-hand side of C is relevant to u . The detailed mathematical steps are described below. Substituting The next item to be administered for examinee i is the one () t Equations 4 and 5 into Equation 6, the can be written SHE in R that maximizes JSD . Yigit et al. (2018) have proved ij i ij in the following equivalent form that JSD can be considered as the mutual information ij IU (; αα )( = IU ;) between the two discrete random vari- iijiji ables of α and U . Thus, the JSD index is also a measure of 1 2 i ij πα () P () α it , cjuc SHEP = πα α H . the amount of information one random variable α contains () () ij ∑ ∑ it , cjuc i (10) u =0 c=1 2 about another U . ij Similar results have been observed by Kang et al. (2017) within the framework of dual-objective CD-CAT (Kang et al., By the definition of SHE, Equation 10 can be computed by 2017; McGlohen & Chang, 2008; Wang et al., 2014; Zheng et al., 2018). For simultaneously estimating examinees’ α 1 2 and general ability θ , the dual-objective CD-CAT is based on SHEP π = αα () () ij ∑ ∑ it , cjuc a CDM and an item response theory model. In other words, u =0 c =1 (11) item response has two Bernoulli distributions P () α and ij ju i πα P α πα P α () () () () it , cjuc it , cjuc . Take the two-parameter logistic model as an exam- P () θ − log ju i ∑ C C ple, the probability of responding correctly to item is c = =1 2 2 defined as PD () θθ =+ 11 /( exp(−− ab ( ))) , that is, ji 1 ji j Pu () θθ =− exp( Da () bD )/ (e 1+− xp( ab (θ ))). Here, ju ij ij ji j Recall from two basic logarithmic properties that the log is a constant, a is the discrimination parameter, and b is j j of a quotient is equal to the difference between the logs of the the difficulty parameter. The JSD of dual-objective CD-CAT numerator and denominator, and the log of a product is equal was defined as mutual information between the two discrete to the sum of the logs of the factors. Equation 11 can be writ- random variables and Z , where U has a mixture distri- ij ij ten as bution between P ()α and P ()θ , and Z is the binary indi- ju ju cator variable for each distribution. For detailed information about relationship between the JSD, KL information, and 1 2 SHEP = πα α () () ∑ ∑ Fisher information, please refer to Kang et al. (2017). ij it , cjuc u =0 c =1 πα () P () α Relationship Between the SHE and the JSD it , cjuc − logl πα + og PC α − log. () () () ∑ it , cjuc 2 C C c =1 The purpose of this section is to establish the statement that the SHE and the JSD as two item selection criteria in (12) Wang et al. 5 Notice C and logC can be factored out from the third JSD minimizing SHE is equivalent to maximizing 2 2 ij ij summation, as they remain constant over the summation because they select the same item for administration at the index from 1 to . Hence, Equation 12 has the form 2 () t+1 th stage of testing. So far, this completes the proof that the SHE and the JSD have a linear relationship under the 1 2 G-DINA model. Note that two proofs of the relationship SHEP =− πα () () απ logl () αα + og P () () () ij ∑ ∑ it ,, cjuc it cjuc between the SHE and JSD just rely on the form of an item u=0 c=1 response function for a binary response variable and do not + CC log, depend on any particular CDM, because the item response ∑ 22 u= function P () α , the prior distribution πα () , and the cur- ju i 0 c () t (13) rent item response vector u are all we need to calculate SHE and mutual information. since CP = πα α . () () Tables A1 and A2 in the appendix are presented for the 2 it , cjuc c=1 After changing the order of the summation and factoring illustration of computation of values of indices for the SHE πα () log( πα ) πα () two constant terms (i.e., and ) it ,, ci tc it , c and JSD. Here, the SHE and JSD are computed for two items out from the new second summation, the first term on the with different item response distributions or different item right-hand side of the Equation 13 can be written as in the parameters, where a discrete uniform prior distribution for following equivalent form: attribute patterns was used. From these two tables, the rela- tionship between the SHE and JSD for the two items satis- fied Equation 16 or Equation 17, and minimizing SHE is 2 1 ij − πα () log( πα ) P () α equivalent to maximizing JSD . Empirical examples show ∑ it ,, ci tc ∑ ju c ij c=1 u =0 that the two item selection criteria are expected to behave 2 1 (14) similarly in CD-CAT. − πα P α log P α () (() () ∑ ∑ it , cj uc ju c c=1 u=0 KK 2 2 Simulation Study =− πα log πα + πα HP α , () () () () () ∑∑ it ,, ci tc it , cjuc c== 1 c 1 Design A small-scale simulation study was conducted to compare the P () α = 1 ∑ ju c performance of the SHE and JSD. Following a design similar which follows from and Equation 9. u=0 to that in Cheng (2009) and Xu et al. (2016), the DINA model Based on Equations 3, 13, and 14, the SHE can be writ- ij and five independent attributes were considered in the simu- ten as lation study. For the generation of four item banks, a Q-matrix 2 1 for 300 items should be first simulated. The entries of the SHEH = ππ + αα HP + CC log. () () () () ∑∑ ij it ,, it cjuc 22 Q-matrix were generated item by item and attribute by attri- cu == 1 0 bute. Each item has 20% chance of measuring each attribute. (15) Four item banks were considered: (a) slipping and guessing parameters were fixed as one of the three levels, such as 0.05, From Equation 8, the third term on the right-hand side of 0.1, or 0.2, and (b) both slipping and guessing parameters () t −= HP ((Uu |) u ) Equation 15 is equal to , since ij i were randomly draw from a uniform distribution on the inter- val [0.1, 0.3]. Test length was either fixed at 5 or 10 items. CP = πα () α . Then based on Equations 7 and () 2 it , cjuc c=1 The sample size of examinees was set to 2,000. Attribute pat- SHE 15, the can be written as ij terns for all examinees were randomly drawn from all possi- ble attribute patterns with equal probability. Details of SHEH = π − JSD , (16) () ij it , ij simulation design are presented in Table A3 in the appendix. To consider the impact of the simulation of item responses in CD-CAT on the performance of the SHE and JSD, two which can be rewritten as types of CAT simulation were considered: full simulations or post hoc simulations (Magis et al., 2017). In case of a full JSDH = π − SHE . () (17) ij it , ij CAT simulation, an item response for examinee i on item was randomly drawn from a Bernoulli distribution, denoted The meaning of Equation 17 is consistent with the fact by Bernoulli() P () α . Full CAT simulations imply that item ji 1 that the JSD or mutual information is a special case of a more responses for examinee i on the same set of items may be general quantity called relative entropy. As H(π ) is not a different. Under the post hoc simulation scenario, a complete i,t function of both item parameters and item responses of the item response matrix was created first for all examinees on candidate item j, it is a constant for examinee i. Thus, each item bank before CD-CAT administrations and the 6 SAGE Open Table 1. Mean and Standard Deviation (in brackets) of Attribute and Pattern Recovery Rate for Slipping and Guessing Parameters of 0.05. Attribute Simulations Test length Method 1 2 3 4 5 Pattern Post hoc 5 SHE 0.950 (0.008) 0.950 (0.007) 0.950 (0.008) 0.950 (0.008) 0.950 (0.008) 0.774 (0.011) JSD 0.950 (0.008) 0.950 (0.007) 0.950 (0.008) 0.950 (0.008) 0.950 (0.008) 0.774 (0.011) 10 SHE 0.985 (0.005) 0.987 (0.006) 0.988 (0.005) 0.989 (0.006) 0.986 (0.007) 0.941 (0.009) JSD 0.984 (0.005) 0.990 (0.006) 0.987 (0.005) 0.989 (0.006) 0.986 (0.007) 0.942 (0.009) Full 5 SHE 0.950 (0.008) 0.951 (0.007) 0.950 (0.008) 0.951 (0.007) 0.951 (0.008) 0.775 (0.012) JSD 0.951 (0.007) 0.950 (0.007) 0.950 (0.008) 0.950 (0.007) 0.950 (0.009) 0.773 (0.009) 10 SHE 0.985 (0.005) 0.987 (0.006) 0.988 (0.006) 0.989 (0.006) 0.986 (0.007) 0.942 (0.009) JSD 0.984 (0.005) 0.989 (0.005) 0.987 (0.005) 0.989 (0.005) 0.986 (0.007) 0.943 (0.011) Note. SHE = Shannon entropy; JSD = Jensen–Shannon divergence. Table 2. Mean and Standard Deviation (in brackets) of Attribute and Pattern Recovery Rate for Slipping and Guessing Parameters of 0.1. Attribute Simulations Test length Method 1 2 3 4 5 Pattern Post hoc 5 SHE 0.901 (0.008) 0.900 (0.007) 0.898 (0.008) 0.902 (0.008) 0.898 (0.008) 0.589 (0.011) JSD 0.901 (0.008) 0.900 (0.007) 0.898 (0.008) 0.902 (0.008) 0.898 (0.008) 0.589 (0.011) 10 SHE 0.951 (0.005) 0.955 (0.006) 0.954 (0.005) 0.963 (0.006) 0.952 (0.007) 0.816 (0.009) JSD 0.949 (0.005) 0.955 (0.006) 0.952 (0.005) 0.962 (0.006) 0.955 (0.007) 0.814 (0.009) Full 5 SHE 0.897 (0.008) 0.901 (0.007) 0.898 (0.008) 0.902 (0.007) 0.900 (0.008) 0.591 (0.012) JSD 0.899 (0.007) 0.900 (0.007) 0.900 (0.008) 0.898 (0.007) 0.898 (0.009) 0.587 (0.009) 10 SHE 0.950 (0.005) 0.955 (0.006) 0.954 (0.006) 0.962 (0.006) 0.954 (0.007) 0.815 (0.009) JSD 0.947 (0.005) 0.958 (0.005) 0.952 (0.005) 0.961 (0.005) 0.957 (0.007) 0.816 (0.011) Note. SHE = Shannon entropy; JSD = Jensen–Shannon divergence. responses to the selected items for the SHE or JSD were Figure 1 presents pattern recovery rates for different test drawn from the complete item response matrix. First of all, lengths and simulation types under slipping and guessing post hoc simulations were considered to make use of exactly parameters of U(0.1, 0.3). From the two top panels of the same item responses for two item selection methods Figure 1, it can be observed that pattern recovery rates (SHE and JSD) under each item bank in CD-CAT. We repli- obtained by the SHE and JSD are the same for each replica- cated each type of simulation process 100 times under each tion under post hoc simulations. While for the full simula- condition and recorded final estimates of attribute patterns tions, pattern recovery rates for the SHE and JSD were for all examinees. different for each replication because different item responses had an impact on test item selection. When the test length was 5, 43% pattern recovery rates of the SHE Results were higher than the mean of pattern recovery rates of the The attribute-level recovery rate is defined as the proportion JSD, and 51% pattern recovery rates of the JSD were higher of each attribute that is correctly identified. The pattern than the mean of pattern recovery rates of the SHE. The recovery rate is defined as the proportion of entire attribute percentages became 49% and 60%, respectively, when the pattern that is correctly recovered. Mean and standard devia- test length was 10. This result is consistent with the previ- tion of attribute-level and pattern recovery rates for each ous finding: “The mutual information item selection algo- level of item parameters are shown in Tables 1–4. For the rithm generates nearly the most accurate attribute pattern SHE, our results are consistent with the results of Xu et al. recovery in more than half of the conditions” (Wang, 2013, (2016). Consistent with our theoretical conclusions, the SHE p. 1030). and JSD behaved quite similarly, because their attribute- We also checked whether two item selection algorithms level and pattern recovery rates were very close to each other selected the same set of items for each examinee under post under all conditions. hoc simulations. For the first three item banks, two item Wang et al. 7 Table 3. Mean and Standard Deviation (in brackets) of Attribute and Pattern Recovery Rate for Slipping and Guessing Parameters of 0.2. Attribute Simulations Test length Method 1 2 3 4 5 Pattern Post hoc 5 SHE 0.800 (0.008) 0.807 (0.007) 0.800 (0.008) 0.797 (0.008) 0.799 (0.008) 0.329 (0.011) JSD 0.800 (0.008) 0.807 (0.007) 0.800 (0.008) 0.797 (0.008) 0.799 (0.008) 0.329 (0.011) 10 SHE 0.845 (0.005) 0.870 (0.006) 0.854 (0.005) 0.871 (0.006) 0.853 (0.007) 0.508 (0.009) JSD 0.847 (0.005) 0.870 (0.006) 0.851 (0.005) 0.868 (0.006) 0.855 (0.007) 0.509 (0.009) Full 5 SHE 0.801 (0.008) 0.802 (0.007) 0.803 (0.008) 0.799 (0.007) 0.800 (0.008) 0.330 (0.012) JSD 0.801 (0.007) 0.801 (0.007) 0.797 (0.008) 0.798 (0.007) 0.803 (0.009) 0.329 (0.009) 10 SHE 0.844 (0.005) 0.865 (0.006) 0.850 (0.006) 0.873 (0.006) 0.855 (0.007) 0.505 (0.009) JSD 0.850 (0.005) 0.865 (0.005) 0.847 (0.005) 0.867 (0.005) 0.853 (0.007) 0.504 (0.011) Note. SHE = Shannon entropy; JSD = Jensen–Shannon divergence. Table 4. Mean and Standard Deviation (in brackets) of Attribute and Pattern Recovery Rate for Slipping and Guessing Parameters of U(0.1, 0.3). Attribute Simulations Test length Method 1 2 3 4 5 Pattern Post hoc 5 SHE 0.884 (0.008) 0.882 (0.007) 0.848 (0.008) 0.897 (0.008) 0.838 (0.008) 0.530 (0.011) JSD 0.884 (0.008) 0.882 (0.007) 0.848 (0.008) 0.897 (0.008) 0.838 (0.008) 0.530 (0.011) 10 SHE 0.953 (0.005) 0.946 (0.006) 0.939 (0.005) 0.937 (0.006) 0.913 (0.007) 0.747 (0.009) JSD 0.953 (0.005) 0.946 (0.006) 0.939 (0.005) 0.937 (0.006) 0.913 (0.007) 0.747 (0.009) Full 5 SHE 0.884 (0.008) 0.882 (0.007) 0.846 (0.008) 0.898 (0.007) 0.838 (0.008) 0.529 (0.012) JSD 0.884 (0.007) 0.880 (0.007) 0.846 (0.008) 0.898 (0.007) 0.838 (0.009) 0.529 (0.009) 10 SHE 0.953 (0.005) 0.946 (0.006) 0.939 (0.006) 0.938 (0.006) 0.914 (0.007) 0.747 (0.009) JSD 0.953 (0.005) 0.946 (0.005) 0.940 (0.005) 0.938 (0.005) 0.915 (0.007) 0.749 (0.011) Note. SHE = Shannon entropy; JSD = Jensen–Shannon divergence. selection algorithms based on the SHE and JSD indeed select one of test items with the same value of the SHE or selected the same set of items but with slightly different orders. JSD index for administration at the next stage of testing, Because all test items in these item banks have the same val- because H(π ) is a constant for examinee i and minimizing i,t ues of item parameters, some items presented in different posi- SHE is equivalent to maximizing JSD at the current stage ij ij tions have the same value of SHE or JSD. For example, two of testing. items with same item parameters but a single distinct attribute This study is not without limitations. Theoretically, SHE, may have the same value of SHE or JSD. For the fourth item KL information, and mutual information are three ways to bank, we found that two item selection algorithms based on measure the uncertainty, and they are related to each other. It the SHE and JSD selected the same set of items. would be interesting to further investigate relationships of item selection indices based on the KL information, the SHE, the JSD, and other indices under general dichotomous or Discussion polytomous CDMs. For example, the GDI and MPWKL In this study, we complete the proof that the SHE procedure might be related, because they perform similarly and better and the JSD are linearly related under CDMs. In other words, than the PWKL in terms of correct attribute classification we showed that minimizing JSD and maximizing SHE can rates or test lengths. We believe GDI is simply a weighted be used interchangeably because they will select the same variance of the probabilities of success of an item associated items in CD-CAT. The two measures are linearly related but with attribute patterns given an attribute pattern distribution, they are not equal, meaning that two measures have the form and therefore we can start with comparing the weighted KL JSDH =− () π SHE . Although they are not equal, item with the weighted variance to show a relationship. The GDI ij it , ij selection methods based on the SHE and JSD will randomly is defined as follows (Kaplan et al., 2015): 8 SAGE Open Figure 1. Pattern recovery rate for different test lengths and simulation types under slipping and guessing parameters of U(0.1, 0.3). 2 2 1 ζ = πα PP α − , = πα PP αα log () () () () () ∑ (18) ij it , cj1 cij ∑∑ it , djud ju d c=1 d =1 u=0 K K 2 1 2 − P απ απ α log P α () () () () ∑∑∑ ju di,, td it ccjuc where PP = πα () () α . The MPWKL is defined as ij it , cj1 c c=1 c=1 u=1 d =0 (22) follows (Kaplan et al., 2015): 2 1 MPWKL = ij = πα PP αα log () () () ∑∑ it , cjuc ju c (23) c=1 u =0 K K 2 2 1 P () α K K ju d 2 1 2 log P απ α πα . () () () ∑ ∑ ∑ ju di,tc it , d − πα log PP α απ α P α () () () () () ∑∑ it , cjuc ∑ ju ud it , d d =1 c=1 u =0 ju c c=1 u =0 d =1 (19) The following algebraic procedures will simplify the calcu- lations in Equation 19 above K K 2 1 2 = πα PP αα − πα log P α () () () () () ∑ it ,, cj ∑ ∑ uc ∑ ju di td ju c c=1 u =0 d =1 MPWKL = ij K K 2 2 1 (24) logl PP () αα − og () P () απ () απ (α )) () ∑∑∑ ju djuc ju di,, tc it d d =1 c=1 u=0 K = πα PP αα − logl PP −− og 1 α , () () () () (20) () ∑ ∑ it , cj11 cijj cj1 c c=1 K K 2 2 1 (25) = PP () απ () απ () αα log () ∑∑∑ ju di,, tc it djud d =1 c=1 u=0 where Equation 20 follows from the quotient rule of logarith- K K 2 2 1 (21) mic properties and Equation 25 follows directly from the − P α πα πα log P α ( ))( )( )( ) ∑∑∑ ju d it ,, ci td ju c complement rule in probability, as expressed by the equation d =1 c=1 u=0 PP () αα =− 1 () . An easy way to calculate the MPWKL jc 01 jc Wang et al. 9 is provided by Equation 25. From Equations 18 and 25, both the effectiveness of item selection algorithm in CD-CAT will the GDI and MPWKL are a function of πα ()(( PP α )) − , impact the quality of curriculum delivery and the outcomes of it , cj1 cij but with different weights log( PP αα )l −− og(1 ()) and learning. If individual diagnosis results with a high measure- jc 11 jc PP () α− . Thus, we know that the GDI is closely related ment precision can be provided by using an effective item ju cij to the MPWKL. selection algorithm of CD-CAT, then diverse instructional The findings of this study may contribute to the growing materials can cater to the diverse needs or specific knowledge literature on formative assessments. First, theoretical deriva- status of all learners (Lashley, 2019). Finally, information- tions and empirical examples have shown that both indices based indices are now not only widely applied in CD-CAT, but (SHE and JSD) are expected to select the same next item given also useful for any test construction stage where test items are item response pattern of the same set of previous test items in selected based on their statistical characteristics (e.g., Henson CD-CAT. Consistent with our theoretical conclusions, simula- & Douglas, 2005; Henson et al., 2008; Kuo et al., 2016). For tion results have shown that the SHE and JSD behaved quite example, the cognitive diagnostic index, the attribute-level similarly in terms of attribute-level and pattern recovery rates. discrimination index, and their modified indices as KL infor- This finding can possibly be useful to help practitioners to mation based measures have been used for the construction of choose an effective item selection algorithm (SHE or JSD) in diagnostic tests. Future research on automated test assembly the development and application of CD-CAT system in the for cognitive diagnosis will expand the scope of the applica- field of educational and psychological measurement. Second, tion of the current finding. Appendix Table A1. Example 1 for the Illustration of Computation of Values of Indices for the SHE and JSD. Function α = (, 00) α = (, 10) α = (, 01) α = (, 11) SUM 1 2 3 4 πα () 0.25 0.25 0.25 0.25 1.00 i,0 −π () απ log( () α ) 0.15 0.15 0.15 0.15 H() π = 06 . 0 ii ,, 00 i ,0 P ()α 0.12 0.95 0.33 0.95 2.35 j1 0.88 0.05 0.67 0.05 1.65 PP () αα =− 1 () jj 01 HP ((αα )) =− PP ()log( () α ) 0.16 0.09 0.28 0.09 0.61 jj . ∑ uju u=0 πα ()P () α 0.03 0.24 0.08 0.24 C = 05 . 9 ij ,01 21 πα ()P () α 0.22 0.01 0.17 0.01 C = 04 . 1 ij ,00 20 0.04 0.02 0.07 0.02 S = 01 . 5 πα ()HP ((α)) ij ,0. 1 πα (| UP == 1)( πα)(α)/ C ii ,, 10 ji j121 0.05 0.40 0.14 0.40 1.00 πα (| UP == 0)( πα)(α)/ C 0.53 0.03 0.41 0.03 1.00 ii ,, 10 ji j120 −= πα (| UU 11 )log(πα (| = )) 0.07 0.16 0.12 0.16 H = 05 . 0 ii ,, 11 ji ij −= πα (|UU 00 )log(πα (| = )) 0.15 0.05 0.16 0.05 H = 04 . 0 ii ,, 11 ii 11 0 SHEC =+ HC H = 04 . 6 ij 21 1200 JSDC =−(log(CC )l +− og(CS )) = 01 . 4 ij 21 21 20 20 1 JSDH =− () π SHE =−= 06 .. 00 46 01 . 4 ij ii ,0 j Note. The base of the logarithm is 10. 10 SAGE Open Table A2. Example 2 for the Illustration of Computation of Values of Indices for the SHE and JSD. α = (, 10) Function α = (, 00) α = (, 01) α = (, 11) SUM 1 3 4 πα () i ,0 0.25 0.25 0.25 0.25 1.00 H() π = 06 . 0 i ,0 −π () απ log( () α ) 0.15 0.15 0.15 0.15 ii ,, 00 P ()α 0.33 0.47 0.33 0.77 1.90 j1 PP () αα =− 1 () 0.67 0.53 0.67 0.23 2.10 jj 01 HP ((αα )) =− PP ()log( () α ) 0.28 0.30 0.28 0.23 1.09 jj . ∑ uju u=0 C = 04 . 8 πα ()P () α 0.08 0.12 0.08 0.19 ij ,01 C = 05 . 3 πα ()P () α 0.17 0.13 0.17 0.06 ij ,00 S = 02 . 7 πα ()HP ((α)) 0.07 0.08 0.07 0.06 ij ,0. πα (| UP == 1)( πα)(α)/ C 0.17 0.25 0.17 0.41 1.00 ii ,, 10 ji j121 πα (| UP == 0)( πα)(α)/ C 0.32 0.25 0.32 0.11 1.00 ii ,, 10 ji j120 H = 05 . 7 −= πα (| UU 11 )log(πα (| = )) 0.13 0.15 0.13 0.16 ii ,, 11 ji ij H = 05 . 7 −= πα (| UU 00 )log(πα (| = )) 0.16 0.15 0.16 0.11 ii ,, 11 ii 11 SHEC =+ HC H = 05 . 7 ij 21 1200 JSDC =−(log(CC )l +− og(CS )) = 00 . 3 ij 21 21 20 20 1 JSDH =− () π SHE =− 06 .. 00 57 = 00 . 3 ij ii ,0 j Note. The base of the logarithm is 10. Table A3. Details of Simulation Design. Factors Details Attribute structure Independent structure with five attributes CDM The DINA model Examinees Sample size is 2,000 Attribute patterns are generated by taking one of the 2 possible patterns with equal probability Item banks Each of four item banks consists of 300 items Each item has 20% chance of measuring each attribute Item parameters are set to s = g = 0.05, s = g = 0.1, s = g = 0.2, or s~U(0.1, 0.3) and g~U(0.1, 0.3) CD-CAT Test length is either fixed at 5 or 10 items Two item selection indices are the SHE and JSD with a prior uniform distribution MLE method is used to estimate attribute patterns of examinees Simulations Full simulations or post hoc simulations are used to generate item responses Note. CDM = cognitive diagnostic model; DINA = deterministic inputs, noisy “and” gate; CD-CAT = cognitive diagnostic computerized adaptive testing; SHE = Shannon entropy; JSD = Jensen–Shannon divergence; MLE = maximum likelihood estimation. Declaration of Conflicting Interests Chang, H.-H. (2012). Making computerized adaptive testing diag- nostic tools for schools. In R. W. Lissitz & H. Jiao (Eds.), The author(s) declared no potential conflicts of interest with respect Computers and their impact on state assessment: Recent his- to the research, authorship, and/or publication of this article. tory and predictions for the future (pp. 195–226). Information Age Publisher. Funding Chang, H.-H. (2015). Psychometrics behind computerized adaptive The author(s) disclosed receipt of the following financial support testing. Psychometrika, 80(1), 1–20. for the research, authorship, and/or publication of this article: This Chang, H.-H., & Ying, Z. (2007). Computerized adaptive testing. work was partially supported by the Key Project of National In N. J. Salkind & K. Rasmussen (Eds.), Encyclopedia of mea- Education Science “Twelfth Five Year Plan” of the Ministry of surement and statistics (Vol. 1, pp. 170–173). Sage. Education of China (Grant No. DHA150285). Chen, Y., & Chang, H.-H. (2018). Psychometrics help learn- ing: From assessment to learning. Applied Psychological ORCID iD Measurement, 42(1), 3–4. Cheng, Y. (2009). When cognitive diagnosis meets computerized Wenyi Wang https://orcid.org/0000-0002-2677-4405 adaptive testing: CD-CAT. Psychometrika, 74(4), 619–632. Cover, T. M., & Thomas, J. A. (2006). Elements of information References theory (2nd ed.). John Wiley. Black, P., & Wiliam, D. (1998). Inside the black box: Raising standards de la & Torre, J. (2011). The generalized DINA model framework. through classroom assessment. Phi Delta Kappa, 80(2), 1–13. Psychometrika, 76(2), 179–199. Wang et al. 11 de la Torre, J., & Douglas, J. (2004). Higher-order latent trait mod- McGlohen, M. K., & Chang, H.-H. (2008). Combining computer els for cognitive diagnosis. Psychometrika, 69(3), 333–353. adaptive testing technology with cognitively diagnostic assess- Gierl, M. J., & Lai, H. (2018). Using automatic item generation to ment. Behavior Research Methods, 40(3), 808–821. create solutions and rationales for computerized formative test- Minchen, N., & de la Torre, J. (2016, July). The continuous ing. Applied Psychological Measurement, 42(1), 42–57. G-DINA model and the Jensen–Shannon divergence [Paper Haertel, E. H. (1989). Using restricted latent class models to map presentation]. Paper presented at the International Meeting of the skill structure of achievement items. Journal of Educational the Psychometric Society, Asheville, NC, United States. Measurement, 26(4), 301–321. Quellmalz, E. S., & Pellegrino, J. W. (2009). Technology and test- Henson, R., & Douglas, J. (2005). Test construction for cognitive ing. Science, 323(5910), 75–79. diagnosis. Applied Psychological Measurement, 29(4), 262– Rupp, A. A., & Templin, J. L. (2008). Unique characteristics of 277. diagnostic classification models: A comprehensive review of Henson, R., Roussos, L., Douglas, J., & He, X. (2008). Cognitive the current state-of-the-art. Measurement, 6(4), 219–262. diagnostic attribute-level discrimination indices. Applied Rupp, A. A., Templin, J. L., & Henson, R. A. (2010). Diagnostic Psychological Measurement, 32(4), 275–288. measurement: Theory, methods, and applications. Guilford Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a Press. family of cognitive diagnosis models using log-linear models Tatsuoka, C. (2002). Data analytic methods for latent partially with latent variables. Psychometrika, 74(2), 191–210. ordered classification models. Journal of the Royal Statistical Jang, E. E. (2008). A framework for cognitive diagnostic assess- Society: Series C (Applied Statistics), 51(3), 337–350. ment. In C. A. Chapelle, Y. R. Chung, & J. Xu. (Eds.), Towards Tatsuoka, C., & Ferguson, T. (2003). Sequential classification on adaptive CALL: Natural language processing for diagnostic partially ordered sets. Journal of the Royal Statistical Society language assessment (pp. 117–131), Iowa State University. Series B (Statistical Methodology), 65(1), 143–157. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment mod- Tatsuoka, K. K. (1983). Rule space: An approach for dealing with els with few assumptions, and connections with nonparametric misconceptions based on item response theory. Journal of item response theory. Applied Psychological Measurement, Educational Measurement, 20(4), 345–354. 25(3), 258–272. Tatsuoka, K. K. (2009). Cognitive assessment: An introduction to Kang, H.-A., Zhang, S., & Chang, H.-H. (2017). Dual-objective the rule space method. Taylor & Francis. item selection criteria in cognitive diagnostic computerized Templin, J. L., & Henson, R. A. (2006). Measurement of psycholog- adaptive testing. Journal of Educational Measurement, 54(2), ical disorders using cognitive diagnosis models. Psychological 165–183. Methods, 11(3), 287–305. Kaplan, M., de la Torre, J., & Barrada, J. R. (2015). New item U.S. Department of Education. (2017). Reimagining the role selection methods for cognitive diagnosis computerized of technology in education: 2017 National Education adaptive testing. Applied Psychological Measurement, 39(3), Technology Plan update. https://tech.ed.gov/files/2017/01/ 167–188. NETP17.pdf Kuo, B.-C., Pai, H.-S., & de la Torre, J. (2016). Modified cogni- von Davier, M. (2005). A general diagnostic model applied to tive diagnostic index and modified attribute-level discrimi- language testing data (ETS Research Report RR-05-16). nation index for test construction. Applied Psychological Educational Testing Service. Measurement, 40(5), 315–330. von Davier, M. (2008). A general diagnostic model applied to Lashley, L. (2019). A reflective analysis of the selection and pro- language testing data. British Journal of Mathematical and duction of instructional material for curriculum delivery at Statistical Psychology, 61(2), 287–307. the primary level in postcolonial Guyana. SAGE Open, 9(2), Wang, C. (2013). Mutual information item selection method in cog- 1–15. nitive diagnostic computerized adaptive testing with short test Laveault, D., & Allal, L. (2016). Implementing assessment for length. Educational and Psychological Measurement, 73(6), learning: Theoretical and practical issues. In D. Laveault & L. 1017–1035. Allal (Eds.), Assessment for learning: Meeting the challenge of Wang, C., Chang, H.-H., & Huebner, A. (2011). Restrictive sto- implementation (pp. 1–20). Springer. chastic item selection methods in cognitive diagnostic comput- Leighton, J. P., & Gierl, M. J. (2007). Cognitive diagnostic assess- erized adaptive testing. Journal of Educational Measurement, ment for education: Theory and applications. Cambridge 48(3), 255–273. University Press. Wang, C., Zheng, C., & Chang, H.-H. (2014). An enhanced Lin, J. (1991). Divergence measures based on the Shannon entropy. approach to combine item response theory with cognitive diag- IEEE Transactions on Information Theory, 37(1), 145–151. nosis in adaptive testing. Journal of Educational Measurement, Liu, J., Ying, Z., & Zhang, S. (2015). A rate function approach 51(4), 358–380. to computerized adaptive testing for cognitive diagnosis. Wang, W. Y., Ding, S. L., & Song, L. H. (2015). New item-selection Psychometrika, 80(2), 468–490. methods for balancing test efficiency against item-bank usage Ma, W., Iaconangelo, C., & de la Torre, J. (2016). Model simi- efficiency in CD-CAT. Springer Proceedings in Mathematics larity, model selection, and attribute classification. Applied & Statistics, 89, 133–151. Psychological Measurement, 40(3), 200–217. Xu, G., Wang, C., & Shang, Z. (2016). On initial item selection Magis, D., Yan, D., & von Davier, A. A. (2017). Computerized in cognitive diagnostic computerized adaptive testing. British adaptive and multistage testing with R: Using packages catR Journal of Mathematical and Statistical Psychology, 69(3), and mstR. Springer. 291–315. 12 SAGE Open Xu, X. L., Chang, H. H., & Douglas, J. (2003). A simulation study to Author Biographies compare CAT strategies for cognitive diagnosis [Paper presen- Wenyi Wang is an associate professor at Jiangxi Normal University. tation]. Paper presented at the annual meeting of the American His primary research interests include cognitive diagnostic assess- Educational Research Association, Chicago, IL, United States. ment and computerized adaptive testing. Yigit, H. D., Sorrel, M. A., de la Torre, J. (2018). Computerized adaptive testing for cognitively based multiple-choice data. Lihong Song is an associate professor at Jiangxi Normal University. Applied Psychological Measurement, 43(5), 388–401. https:// Her primary research interests include cognitive diagnostic assess- doi.org/10.1177/0146621618798665 ment and application of statistical methods to education. Zheng, C., & Chang, H.-H. (2016). High-efficiency response Teng Wang is a postgraduate student at Jiangxi Normal University. distribution-based item selection algorithms for short-length His primary research interests include cognitive diagnostic assess- cognitive diagnostic computerized adaptive testing. Applied ment and computerized adaptive testing. Psychological Measurement, 40(8), 608–624. Zheng, C., He, G., & Gao, C. (2018). The information product meth- Peng Gao is a postgraduate student at Jiangxi Normal University. ods: A unified approach to dual-purpose computerized adaptive His primary research interests include cognitive diagnostic assess- testing. Applied Psychological Measurement, 42(4), 321–324. ment and computerized adaptive testing. Zheng, C., & Wang, C. (2017). Application of binary searching for Jian Xiong is a postgraduate student at Jiangxi Normal University. item exposure control in cognitive diagnostic computerized His primary research interests include item response theory and adaptive testing. Applied Psychological Measurement, 41(7), computerized adaptive testing. 561–576.
Journal
SAGE Open – SAGE
Published: Jan 10, 2020
Keywords: cognitive diagnosis models; computerized adaptive testing; item selection methods; the Shannon entropy procedure; the Jensen–Shannon divergence