Package 'TDCM'

Title: The Transition Diagnostic Classification Model Framework
Description: Estimate the transition diagnostic classification model (TDCM) described in Madison & Bradshaw (2018) <doi:10.1007/s11336-018-9638-5>, a longitudinal extension of the log-linear cognitive diagnosis model (LCDM) in Henson, Templin & Willse (2009) <doi:10.1007/s11336-008-9089-5>. As the LCDM subsumes many other diagnostic classification models (DCMs), many other DCMs can be estimated longitudinally via the TDCM. The 'TDCM' package includes functions to estimate the single-group and multigroup TDCM, summarize results of interest including item parameters, growth proportions, transition probabilities, transitional reliability, attribute correlations, model fit, and growth plots.
Authors: Matthew J. Madison [aut, cph] , Sergio Haab [aut], Minjeong Jeon [aut, cph], Michael E. Cotterell [aut, cre, cph] , University of Georgia [cph] (The content and opinions expressed in this material do not necessarily reflect the views of nor are they endorsed by the University of Georgia or the University System of Georgia.), Institute of Education Sciences [fnd] (This work is supported by the U.S. Department of Education Institute of Education Sciences under IES Award Number R305D220020.), National Science Foundation [fnd] (This work is supported by the National Science Foundation under NSF Award Number 1921373.)
Maintainer: Michael E. Cotterell <[email protected]>
License: MIT + file LICENSE
Version: 0.2.0.9000
Built: 2025-02-24 05:51:38 UTC
Source: https://github.com/cotterell/tdcm

Help Index


Several data sets for the TDCM package.

Description

Several data sets for the TDCM package.

Usage

data.tdcm01

data.tdcm02

data.tdcm03

data.tdcm04

data.tdcm05

Format

data.tdcm01 is a simulated dataset with two time points, four attributes, twenty items, one group of size 1000, and a single Q-matrix. The format is a list of two:

  • data: a data frame of binary item responses

  • q.matrix: a data frame specifying the Q-matrix

data.tdcm02 is a simulated dataset with three time points, two attributes, ten items, one group of size 2500, and a single Q-matrix. The format is a list of two:

  • data: a data frame of binary item responses

  • q.matrix: a data frame specifying the Q-matrix

data.tdcm03 is a simulated dataset with three time points, two attributes, one group of size 1500, and three different ten-item Q-matrices for each time point. Anchor items are specified as items 1/11/21 and items 14/24. The format is a list of five:

  • data: a data frame of binary item responses

  • q.matrix.1: a data frame specifying the Q-matrix for the first time point

  • q.matrix.2: a data frame specifying the Q-matrix for the second time point

  • q.matrix.3: a data frame specifying the Q-matrix for the third time point

  • q.matrix.stacked: data frame specifying the combined Q-matrix for all time points

data.tdcm04 is a simulated dataset with two time points, four attributes, twenty items, two group of size 800 and 900, respectively, and a single Q-matrix. The format is a list of three:

  • data: a data frame of binary item responses

  • q.matrix: a data frame specifying the Q-matrix

  • groups: a vector specifying the examinee group membership

data.tdcm05 is a simulated dataset with one time point, four attributes, and twenty items. For use with the 1-PLCDM. The format is a list of two:

  • data: a data frame of binary item responses

  • q.matrix: a data frame specifying the Q-matrix

Examples

#############################
## Example 1: T = 2, A = 4 ##
data(data.tdcm01, package = "TDCM")
data <- data.tdcm01$data
q.matrix <- data.tdcm01$q.matrix
model <- TDCM::tdcm(data, q.matrix, num.time.points = 2)
results <- TDCM::tdcm.summary(model)
results$item.parameters
results$growth.effects

#############################
## Example 3: T = 3, A = 2 ##
data <- data.tdcm03$data
q1 <- data.tdcm03$q.matrix.1
q2 <- data.tdcm03$q.matrix.2
q3 <- data.tdcm03$q.matrix.3
q <- data.tdcm03$q.matrix.stacked

#TDCM with anchor items constrained
m1 <- tdcm(data, q, num.time.points = 3, num.q.matrix = 3,
anchor = c(1,11,1,21,14,24), num.items = c(10,10,10))

#TDCM without anchor items
m2 <- tdcm(data, q, num.time.points = 3, num.q.matrix = 3, num.items = c(10,10,10))

#Compare models to assess measurement invariance
tdcm.compare(m1, m2)

#Summarize model 1
r1 = tdcm.summary(m1, transition.option = 3)
r1$item.parameters
r1$growth
r1$growth.effects

Estimating item influence measures.

Description

Function to estimate estimate item influence measures. Code adapted from (Jurich & Madison, 2023). This function is not available for longitudinal DCMs.

Usage

item.influence(model, data, fullcorrelation = FALSE, progress = TRUE)

Arguments

model

a previously calibrated model; an object of class gdina.

data

a required N×IN \times I matrix. Binary item responses are in the columns.

fullcorrelation

optional logical argument indicating a full or reduced response-classification correlation matrix.

progress

An optional logical indicating whether the function should print the progress of estimation.

Details

For DCMs, item influence quantifies how much an item impacts classifications. Given an estimated DCM and item response data, this function estimates five item influence measures, including item pull, item override, proportion of attribute information, response-classification correlation (corr1), and response-posterior correlation (corr2).

Value

A list containing several item influence measures.

Note

Currently, this function currently only runs on DCMs estimated at a single time point. It will not run properly for TDCM objects.

References

Jurich, D. & Madison, M. J. (2023). Measuring item influence for diagnostic classification models. Educational Assessment.

Examples

## Item influence illustration
#load data (simulated based on Jurich and Bradshaw (2014))
qmatrix <- CDM::data.sda6$q.matrix
responses <- CDM::data.sda6$data

#Estimate the full LCDM
model1 <- CDM::gdina(responses, qmatrix, linkfct = "logit", method = "ML")

#Estimate item influence measures
influence <- TDCM::item.influence(model1, responses)

#Summarize influence statistics
influence$Pull #item pull
influence$Override #item override
influence$Information #proportion of attribute information
influence$Correlation1 #correlation of responses and classifications
influence$Correlation2 #correlation of responses and posterior probabilities

Estimating the multigroup transition diagnostic classification model (TDCM)

Description

This function estimates the multigroup TDCM (Madison & Bradshaw, 2018).

Usage

mg.tdcm(
  data,
  q.matrix,
  num.time.points,
  rule = "LCDM",
  linkfct = "logit",
  groups,
  forget.att = c(),
  group.invariance = TRUE,
  time.invariance = TRUE,
  progress = TRUE
)

Arguments

data

A required N×T×IN \times T \times I matrix. For each time point, binary item responses are in the columns.

q.matrix

a required I×AI \times A matrix indicating which items measure which attributes.

num.time.points

The number of time points (i.e., measurement/testing occasions), integer 2\ge 2.

rule

A string or a vector of the specific DCM to be employed. Currently accepts the same values as rule in tdcm(): "LCDM", "DINA", "DINO", "CRUM", "RRUM", "LCDM1" for the LCDM with only main effects, "LCDM2" for the LCDM with two-way interactions, "LCDM3", and so on. If rule is supplied as a single string, then that DCM will be assumed for each item. If entered as a vector, a rule can be specified for each item.

linkfct

A string indicating the LCDM link function. Currently accepts "logit" (default) to estimate the LCDM. Can be specified "identity" to estimate the GDINA model. Also accepts a "log" link function.

groups

A required vector of integer group identifiers (e.g., 1, 2, 3) for multiple group estimation.

forget.att

An optional vector allowing for constraining of individual attribute proficiency loss, or forgetting. The default allows forgetting for each measured attribute (e.g., P(10)0P(1 \rightarrow 0) \neq 0). This vector is specified to indicate the attributes for which forgetting is not permitted.

group.invariance

logical. If TRUE (default), item parameters are assumed to be equal for all groups. If FALSE, item parameters are not assumed to be equal for all groups.

time.invariance

logical. If TRUE (default), item parameters are assumed to be equal for all time points. If FALSE, item parameters are not assumed to be equal for all time points.

progress

logical. If FALSE, the function will print the progress of estimation. If TRUE (default), no progress information is printed.

Value

An object of class gdina with entries as indicated in the CDM package. For the TDCM-specific results (e.g., growth, transitions), use TDCM::mg.tdcm.summary().

Note

Currently, the TDCM::mg.tdcm() function only accepts a single Q-matrix.

References

de la Torre, J. (2011). The Generalized DINA Model Framework. Psychometrika 76, 179–199. doi:10.1007/s11336-011-9207-7

George, A. C., Robitzsch, A., Kiefer, T., Gross, J., & Ünlü , A. (2016). The R package CDM for Cognitive Diagnosis Models. Journal of Statistical Software, 74(2), 1-24. doi:10.18637/jss.v074.i02

Henson, R., Templin, J., & Willse, J. (2009). Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables. Psychometrika, 74, 191-21. doi:10.1007/s11336-008-9089-5

Johnson, M. S., & Sinharay, S. (2020). The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models. Journal of Educational Measurement, 47(1), 5–31. doi:10.3102/1076998619864550

Kaya, Y., & Leite, W. (2017). Assessing Change in Latent Skills Across Time With Longitudinal Cognitive Diagnosis Modeling: An Evaluation of Model Performance. Educational and Psychological Measurement, 77(3), 369–388. doi:10.1177/0013164416659314

Li, F., Cohen, A., Bottge, B., & Templin, J. (2015). A Latent Transition Analysis Model for Assessing Change in Cognitive Skills. Educational and Psychological Measurement, 76(2), 181–204. doi:10.1177/0013164415588946

Madison, M. J. (2019). Reliably Assessing Growth with Longitudinal Diagnostic Classification Models. Educational Measurement: Issues and Practice, 38(2), 68-78. doi:10.1111/emip.12243

Madison, M. J., & Bradshaw, L. (2018a). Assessing Growth in a Diagnostic Classification Model Framework. Psychometrika, 83(4), 963-990. doi:10.1007/s11336-018-9638-5

Madison, M. J., & Bradshaw, L. (2018b). Evaluating Intervention Effects in a Diagnostic Classification Model Framework. Journal of Educational Measurement, 55(1), 32-51. doi:10.1111/jedm.12162

Madison, M.J., Chung, S., Kim, J., & Bradshaw, L.P. (2024) Approaches to estimating longitudinal diagnostic classification models. Behaviormetrika, 51(7), 7-19. doi:10.1007/s41237-023-00202-5

Rupp, A. A., Templin, J., & Henson, R. (2010). Diagnostic Measurement: Theory, Methods, and Applications. New York: Guilford. ISBN: 9781606235430.

Schellman, M., & Madison, M. J. (2024). Estimating the reliability of skill transition in longitudinal DCMs. Journal of Educational and Behavioral Statistics.

Templin, J., & Bradshaw, L. (2013). Measuring the Reliability of Diagnostic Classification Model Examinee Estimates. Journal of Classification, 30, 251-275. doi:10.1007/s00357-013-9129-4

Wang. S., Yang. Y., Culpepper, S. A., & Douglas, J. (2018). Tracking Skill Acquisition With Cognitive Diagnosis Models: A Higher-Order, Hidden Markov Model With Covariates. Journal of Educational and Behavioral Statistics, 43(1), 57-87. doi:10.3102/1076998617719727

Examples

## Example 4: G = 2, T = 2, A = 4
data(data.tdcm04, package = "TDCM")
data <- data.tdcm04$data
q.matrix <- data.tdcm04$q.matrix
groups <- data.tdcm04$groups

# Estimate full multigroup TDCM with invariance assumed.
mg.model <- TDCM::mg.tdcm(data, q.matrix, num.time.points = 2, groups = groups)

# summarize results
results <- TDCM::mg.tdcm.summary(mg.model)

# plot results
TDCM::tdcm.plot(results)

Multigroup TDCM results compiler and summarizer.

Description

A function to compile results from calibration of the multigroup TDCM (Madison & Bradshaw, 2018).

Usage

mg.tdcm.summary(
  model,
  transition.option = 1,
  classthreshold = 0.5,
  attribute.names = c(),
  group.names = c()
)

Arguments

model

a gdina object returned from the mg.tdcm function.

transition.option

option for reporting results. = 1 compares the first time point to the last. = 2 compares the first time point to every other time point. = 3 compares successive time points. Default = 1.

classthreshold

probability threshold for establishing proficiency from examinee posterior probabilities. Default is .50, which maximizes overall classification accuracy. It can be set to a lower value to minimize false negatives (i.e., misclassifying proficient examinees as non-proficient) or set to a higher value to minimize false positives (i.e., misclassifying non-proficient examinees as proficient).

attribute.names

optional vector of attribute names to include in plots.

group.names

optional vector of group names to include in plots. Enter in order corresponding to the integer labels in the groups vector specified in the mg.tdcm function.

Details

Provides a summary of multigroup TDCM results including item parameters, attribute posterior probabilities, transition posterior probabilities, classifications, group-wise growth, group-wise transition probabilities, attribute correlations, several transition reliability metrics, and model fit. Includes longitudinal versions of reliability metrics developed by Templin and Bradshaw (2013) and Johnson and Sinharay (2020).

Value

A list with the following items:

  • $item.parameters: item parameter estimates from the specified DCM.

  • $growth: proficiency proportions for each time point and each attribute

  • $growth.effects: growth effect size estimates for each attribute and specified transitions including growth in proficiency proportion, odds ratio = odds proficiency at later time point divided by odds of proficiency at earlier time point, and Cohen's h (arcsine-transformed difference in proportions; Cohen, 1988)

  • $transition.probabilities: conditional attribute proficiency transition probability matrices

  • $posterior.probabilities: examinee marginal attribute posterior probabilities of proficiency

  • $transition.posteriors: examinee marginal attribute transition posterior probabilities

  • $most.likely.transitions: examinee most likely transitions for each attribute and transition

  • $classifications: examinee classifications determined by the specified threshold applied to the posterior probabilities

  • $reliability: estimated transition reliability metrics for each attribute for the specified transitions. “pt bis” = longitudinal point biserial metric; “info gain” = longitudinal information gain metric; “polychor” = longitudinal tetrachoric metric; “ave max tr” = average maximum transition posterior metric; “P(t>k)” = proportion of examinee marginal attribute transition posteriors greater than k; “wt pt bis” = weighted longitudinal point biserial; “wt info gain” = weighted longitudinal information gain.

  • $att.corr: estimated attribute correlation matrix

  • $model.fit: Several model fit indices and tests are output including item root mean square error of approximation (RMSEA; von Davier, 2005), mean RMSEA, bivariate item fit statistics (Chen et al., 2013), and absolute fit statistics such as mean absolute deviation for observed and expected item correlations (MADcor; DiBello, Roussos, & Stout, 2007), and standardized root mean square root of squared residuals (SRMSR; Maydeu-Olivares, 2013)

References

Chen, J., de la Torre, J. ,& Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50, 123-140.

DiBello, L. V., Roussos, L. A., & Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao and S. Sinharay (Eds.), Handbook of Statistics, Vol. 26 (pp.979–1030). Amsterdam: Elsevier.

Johnson, M. S., & Sinharay, S. (2020). The reliability of the posterior probability of skill attainment in diagnostic classification models. Journal of Educational Measurement, 47(1), 5 – 31.

Madison, M. J. (2019). Reliably assessing growth with longitudinal diagnostic classification models. Educational Measurement: Issues and Practice, 38(2), 68-78.

Madison, M. J., & Bradshaw, L. (2018). Evaluating intervention effects in a diagnostic classification model framework. Journal of Educational Measurement, 55(1), 32-51.

Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models (with discussion). Measurement: Interdisciplinary Research and Perspectives, 11, 71-137.

Schellman, M., & Madison, M. J. (2024). Estimating the reliability of skill transition in longitudinal DCMs. Journal of Educational and Behavioral Statistics.

Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30, 251-275.

von Davier M. (2008). A general diagnostic model applied to language testing data. The British journal of mathematical and statistical psychology, 61(2), 287–307.

Examples

## Example 4: G = 2, T = 2, A = 4
data(data.tdcm04, package = "TDCM")
dat4 <- data.tdcm04$data
qmat4 <- data.tdcm04$q.matrix
group4 <- data.tdcm04$groups

# estimate mgTDCM with invariance assumed and full LCDM
mg1 <- TDCM::mg.tdcm(dat4, qmat4, rule = "LCDM", num.time.points = 2,
  group = group4, group.invariance = TRUE, time.invariance = TRUE)

# summarize results
results1 <- TDCM::mg.tdcm.summary(mg1)

# plot results
TDCM::tdcm.plot(results1)

# estimate mgTDCM without group invariance
mg2 <- TDCM::mg.tdcm(dat4, qmat4, rule = "LCDM",num.time.points = 2,
  group = group4, group.invariance = FALSE, time.invariance = TRUE)


# compare models to assess group invariance
TDCM::tdcm.compare(mg1, mg2)

One-parameter log-linear cognitive diagnosis model.

Description

Function to estimate the 1-PLCDM (Madison et al., 2023; Maas et al., 2023).

Usage

oneplcdm(data, q.matrix, progress = TRUE)

Arguments

data

a required N×IN \times I matrix. Binary item responses are in the columns.

q.matrix

a required I×AI \times A matrix indicating which items measure which attributes.

progress

An optional logical indicating whether the function should print the progress of estimation.

Details

Estimates the single-attribute and multi-attribute 1-PLCDM described in Madison et al. (2024). Example shows that attribute subscores are sufficient statistics for classifications.

Value

An object of class gdina with entries as indicated in the CDM package.

Note

Currently, this model cannot be embedded within the TDCM via the rule argument.

References

George, A. C., Robitzsch, A., Kiefer, T., Gross, J., & Ünlü , A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74(2), 1-24.

Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log linear models with latent variables. Psychometrika, 74, 191-21.

Madison, M.J., Wind, S., Maas, L., Yamaguchi, K. & Haab, S. (2024). A one-parameter diagnostic classification model with familiar measurement properties. Journal of Educational Measurement.

Maas, L., Madison, M. J., & Brinkhuis, M. J. (2024). Properties and performance of the one-parameter log-linear cognitive diagnosis model. Frontiers.

Examples

## Example 1: A = 4
data(data.tdcm05)
dat5 <- data.tdcm05$data
qmat5 <- data.tdcm05$q.matrix

# calibrate LCDM
m1 <- CDM::gdina(dat5, qmat5, linkfct = "logit", method = "ML")

# calibrate 1-PLCDM
m2 <- TDCM::oneplcdm(dat5, qmat5)
summary(m2)
#demonstrate 1-PLCDM sum score sufficiency for each attribute
subscores <- cbind(rowSums(dat5[, 1:5]), rowSums(dat5[, 6:10]),
rowSums(dat5[, 11:15]), rowSums(dat5[, 16:20]))
colnames(subscores) <- c("Att1", "Att2", "Att3", "Att4")
proficiency <- cbind(m2$pattern[, 6] > .50, m2$pattern[, 7] > .50,
m2$pattern[, 8] > .50, m2$pattern[, 9] > .5) * 1
table(subscores[, 1], proficiency[, 1])
table(subscores[, 2], proficiency[, 2])
table(subscores[, 3], proficiency[, 3])
table(subscores[, 4], proficiency[, 4])

#plot sum score sufficiency for each attribute
posterior1pl <- m2$pattern[, 6:9]
posteriorlcdm <- m1$pattern[, 6:9]
oldpar <- par(mfrow = c(2, 2))
for (i in 1:4) {
 plot(subscores[, i], posteriorlcdm[, i], pch = 19,las = 1, cex.lab = 1.5,
 xlab = "Sum Scores", ylab = "P(proficiency)",
 cex.main = 1.5, col = "grey", xaxt = "n", yaxt = "n", cex = 1.2,
 main = paste("Attribute ", i, sep = ""))
 graphics::axis(side = 1, at = c(0, 1, 2, 3, 4, 5), )
 graphics::axis(side = 2, at = c(0, .1, .2, .3, .4, .5, .6, .7, .8, .9, 1.0), las = 1)
 graphics::points(subscores[, i], posterior1pl[, i], col = "black", pch = 18, cex = 1.5)
 graphics::abline(a = .50, b = 0, col = "red")
 graphics::legend("bottomright", c("1-PLCDM", "LCDM"), col = c("black", "grey"),
 pch = c(18 ,19), box.lwd = 0, box.col = "white", bty = 'n')
}
par(oldpar)

Estimating the Transition Diagnostic Classification Model (TDCM)

Description

tdcm() is used to estimate the transition diagnostic classification model (TDCM; Madison & Bradshaw, 2018a), which is a longitudinal extension of the log-linear cognitive diagnosis model (LCDM; Henson, Templin, & Willse, 2009). For the multigroup TDCM, see mg.tdcm(). It allows for the specification of many specific DCMs via the rule option. The default DCM rule and link function specifies the LCDM. The rule can be changed to estimate the DINA model, DINO model, CRUM (i.e., ACDM, or main effects model), or reduced interaction versions of the LCDM. The link function can be changed to specify the GDINA model.

Usage

tdcm(
  data,
  q.matrix,
  num.time.points,
  invariance = TRUE,
  rule = "LCDM",
  linkfct = "logit",
  num.q.matrix = 1,
  num.items = c(),
  anchor = c(),
  forget.att = c(),
  progress = TRUE
)

Arguments

data

A required N×T×IN \times T \times I data matrix containing binary item responses. For each time point, the binary item responses are in the columns.

q.matrix

A required I×AI \times A matrix indicating which items measure which attributes. If there are multiple Q-matrices, then they must have the same number of attributes and must be stacked on top of each other for estimation (to specify multiple Q-matrices, see num.q.matrix, num.items, and anchor).

num.time.points

A required integer 2\ge 2 specifying the number of time points (i.e., measurement occasions).

invariance

logical. If TRUE (the default), then item parameters will be constrained to be equal at each time point. If FALSE, item parameters are not assumed to be equal over time.

rule

A string or a vector indicating the specific DCM to be employed. A vector of supported rule values is provided by tdcm.rules. If rule is supplied as a single string, then that DCM will be assumed for each item. If entered as a vector, a rule can be specified for each item.

linkfct

A string or a vector indicating the LCDM link function. Currently accepts "logit" (default) to estimate the LCDM. Can be specified "identity" to estimate the GDINA model. Also accepts a "log" link function.

num.q.matrix

An optional integer specifying the number of Q-matrices. For many applications, the same assessment is administered at each time point and this number is 1 (the default). If there are different Q-matrices for each time point, then this argument must be specified and should be equal to the number of time points. For example, if there are three time points, and the Q-matrix for each time point is different, then num.q.matrix = 3. If there are three time points, and the Q-matrix is different only for time point 3, then num.q.matrix is still specified as 3.

num.items

An integer specifying the number of items. When there are multiple Q-matrices, the number of items in each Q-matrix is specified as a vector. For example, if there are three time points, and the Q-matrices for each time point have 8, 10, and 12 items, respectively, then num.items = c(8, 10, 12).

anchor

When there are different tests at each time point, this optional argument is a vector of pairs of item numbers indicating which items are the same across time points and should be held invariant. For example, if there are three Q-matrices with 10 items each, and Items 1, 11, and 21 are the same, and Items 14 and 24 are the same, then anchor = c(1,11,1,21,14,24). Default is an empty vector to indicate absence of anchor items. Note: when anchor is specified, invariance is automatically set to false for non-anchor items.

forget.att

An optional vector allowing for constraining of individual attribute proficiency loss, or forgetting. The default allows forgetting for each measured attribute (e.g., P(10)0P(1 \rightarrow 0) \neq 0). This vector is specified to indicate the attributes for which forgetting is not permitted.

progress

logical. If FALSE, the function will print the progress of estimation. If TRUE (default), no progress information is printed.

Details

Estimation of the TDCM via the CDM package (George, et al., 2016), which is based on an EM algorithm as described in de la Torre (2011). The estimation approach is further detailed in Madison et al. (2023).

Value

An object of class gdina with entries as described in CDM::gdina(). To see a TDCM-specific summary of the object (e.g., growth, transitions), use tdcm.summary().

References

de la Torre, J. (2011). The Generalized DINA Model Framework. Psychometrika 76, 179–199. doi:10.1007/s11336-011-9207-7

George, A. C., Robitzsch, A., Kiefer, T., Gross, J., & Ünlü , A. (2016). The R package CDM for Cognitive Diagnosis Models. Journal of Statistical Software, 74(2), 1-24. doi:10.18637/jss.v074.i02

Henson, R., Templin, J., & Willse, J. (2009). Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables. Psychometrika, 74, 191-21. doi:10.1007/s11336-008-9089-5

Johnson, M. S., & Sinharay, S. (2020). The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models. Journal of Educational Measurement, 47(1), 5–31. doi:10.3102/1076998619864550

Kaya, Y., & Leite, W. (2017). Assessing Change in Latent Skills Across Time With Longitudinal Cognitive Diagnosis Modeling: An Evaluation of Model Performance. Educational and Psychological Measurement, 77(3), 369–388. doi:10.1177/0013164416659314

Li, F., Cohen, A., Bottge, B., & Templin, J. (2015). A Latent Transition Analysis Model for Assessing Change in Cognitive Skills. Educational and Psychological Measurement, 76(2), 181–204. doi:10.1177/0013164415588946

Madison, M. J. (2019). Reliably Assessing Growth with Longitudinal Diagnostic Classification Models. Educational Measurement: Issues and Practice, 38(2), 68-78. doi:10.1111/emip.12243

Madison, M. J., & Bradshaw, L. (2018a). Assessing Growth in a Diagnostic Classification Model Framework. Psychometrika, 83(4), 963-990. doi:10.1007/s11336-018-9638-5

Madison, M. J., & Bradshaw, L. (2018b). Evaluating Intervention Effects in a Diagnostic Classification Model Framework. Journal of Educational Measurement, 55(1), 32-51. doi:10.1111/jedm.12162

Madison, M.J., Chung, S., Kim, J., & Bradshaw, L.P. (2024) Approaches to estimating longitudinal diagnostic classification models. Behaviormetrika, 51(7), 7-19. doi:10.1007/s41237-023-00202-5

Rupp, A. A., Templin, J., & Henson, R. (2010). Diagnostic Measurement: Theory, Methods, and Applications. New York: Guilford. ISBN: 9781606235430.

Schellman, M., & Madison, M. J. (2024). Estimating the reliability of skill transition in longitudinal DCMs. Journal of Educational and Behavioral Statistics.

Templin, J., & Bradshaw, L. (2013). Measuring the Reliability of Diagnostic Classification Model Examinee Estimates. Journal of Classification, 30, 251-275. doi:10.1007/s00357-013-9129-4

Wang. S., Yang. Y., Culpepper, S. A., & Douglas, J. (2018). Tracking Skill Acquisition With Cognitive Diagnosis Models: A Higher-Order, Hidden Markov Model With Covariates. Journal of Educational and Behavioral Statistics, 43(1), 57-87. doi:10.3102/1076998617719727

Examples

## Example 1: T = 2, A = 4
data(data.tdcm01, package = "TDCM")
data <- data.tdcm01$data
q.matrix <- data.tdcm01$q.matrix

# Estimate full TDCM with invariance assumed.
model1 <- TDCM::tdcm(data, q.matrix, num.time.points = 2)

# Summarize results with tdcm.summary().
results <- TDCM::tdcm.summary(model1)
results$item.parameters
results$growth
results$transition.probabilities

Comparing the fit of two TDCMs

Description

Provides a comparison of two TDCMs. Can be used to compare different measurement models or assess measurement invariance over time or over groups in the multigroup TDCM case. Only accepts two models.

Usage

tdcm.compare(model1, model2)

Arguments

model1

a gdina object returned from the tdcm or mg.tdcm function.

model2

a second gdina object returned from the tdcm or mg.tdcm function

Value

This function returns a data frame with model fit statistics (AIC/BIC) and results from a likelihood ratio or deviance test.

Note

  • Currently, this function currently accepts two models for comparison.

  • Both models must be fit to the same item responses and Q-matrix.

  • The function will provide results for two non-nested models. Please ensure that models are nested before interpreting the likelihood ratio test for nested models.

  • The likelihood ratio test is not valid for some model comparisons (e.g., LCDM vs DINA) because of model constraints.

Examples

## Example 1: T = 2, A = 4
data(data.tdcm01, package = "TDCM")
dat1 <- data.tdcm01$data
qmat1 <- data.tdcm01$q.matrix

# estimate TDCM with invariance assumed and full LCDM
m1 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "LCDM")

# estimate TDCM with invariance not assumed
m2 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = FALSE, rule = "LCDM")

# compare models to assess measurement invariance.
TDCM::tdcm.compare(m1, m2)

Plotting TDCM Results

Description

tdcm.plot() visualizes the results from TDCM analyses.

Usage

tdcm.plot(results, attribute.names = c(), group.names = c(), type = "both")

Arguments

results

results from tdcm.summary or mg.tdcm.summary

attribute.names

an optional vector of attribute names to include in plots.

group.names

an optional vector of group names to include in plots.

type

an option to specify the type of plot in single group cases; "both" is default and will produce a line plot and a bar chart; "line" will produce a line plot; and "bar" will produce a bar chart.

Value

No return value, called for side effects.

Examples

## Example 1: T = 2, A = 4
data(data.tdcm01, package = "TDCM")
dat1 = data.tdcm01$data
qmat1 = data.tdcm01$q.matrix

#estimate TDCM with invariance assumed and full LCDM
m1 = TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "LCDM")

#summarize results with tdcm.summary function
results1 = TDCM::tdcm.summary(m1)

#plot results
TDCM::tdcm.plot(results1, attribute.names = c("Addition", "Subtraction",
"Multiplication", "Division"))

TDCM Condensation Rules

Description

A condensation rule is a formula that states how different attributes combine to form an observed or latent response (Rupp, Templin, & Henson, 2010). The TDCM package includes support for "LCDM", "DINA", "DINO", "CRUM", "RRUM", "LCDM1" for the LCDM with only main effects, "LCDM2" for the LCDM with two-way interactions, "LCDM3", and so on. Evaluate TDCM::tdcm.rules$TDCM for a complete list of condensation rules supported by the TDCM package.

Usage

tdcm.rules

Format

An object of class data.frame with 15 rows and 2 columns.

References

Rupp, A. A., Templin, J., & Henson, R. (2010). Diagnostic Measurement: Theory, Methods, and Applications. New York: Guilford. ISBN: 9781606235430.

Examples

TDCM::tdcm.rules$TDCM

DCM scoring function.

Description

Function to score responses with fixed item parameters from a previously calibrated LCDM.

Usage

tdcm.score(
  calibration.model,
  newdata,
  q.matrix,
  attr.prob.fixed = NULL,
  progress = TRUE
)

Arguments

calibration.model

the previously calibrated model; an object of class gdina.

newdata

a required N×IN \times I matrix. Binary item responses are in the columns.

q.matrix

a required I×AI \times A matrix indicating which items measure which attributes.

attr.prob.fixed

optional argument for attribute profile proportions. Default is uniform distribution of profiles.

progress

An optional logical indicating whether the function should print the progress of estimation.

Details

Obtain classifications for new responses to items that were previously calibrated. The calibrate-and-score approach is further detailed in Madison et al. (2023).

Value

An object of class gdina with entries as indicated in the CDM package.

References

George, A. C., Robitzsch, A., Kiefer, T., Gross, J., & Ünlü , A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74(2), 1-24.

Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log linear models with latent variables. Psychometrika, 74, 191-21.

Madison, M.J., Chung, S., Kim, J., & Bradshaw, L. (2023). Approaches to estimating longitudinal diagnostic classification models. Behaviormetrika.

Examples

## Example 1: T = 2, A = 4
data(data.tdcm01, package = "TDCM")
dat1 <- data.tdcm01$data
qmat1 <- data.tdcm01$q.matrix
pre <- dat1[, 1:20]
post <- dat1[, 21:40]

# calibrate LCDM with post-test data
m1 <- CDM::gdina(data = pre, q.matrix = qmat1, linkfct = "logit", method = "ML")

# score pre-test responses
m2 <- TDCM::tdcm.score(m1, newdata = post, q.matrix = qmat1)
summary(m2)
m2$pattern

TDCM results compiler and summarizer.

Description

Function to summarize results from TDCM analyses.

Usage

tdcm.summary(
  model,
  transition.option = 1,
  classthreshold = 0.5,
  attribute.names = c()
)

Arguments

model

a gdina object returned from the tdcm function.

transition.option

option for reporting results. = 1 compares the first time point to the last. = 2 compares the first time point to every other time point. = 3 compares successive time points. Default = 1.

classthreshold

probability threshold for establishing proficiency from examinee posterior probabilities. Default is .50, which maximizes overall classification accuracy. It can be set to a lower value to minimize false negatives (i.e., misclassifying proficient examinees as non-proficient) or set to a higher value to minimize false positives (i.e., misclassifying non-proficient examinees as proficient).

attribute.names

optional vector of attribute names to include in results output.

Details

Provides a summary of TDCM results including item parameters, attribute posterior probabilities, transition posterior probabilities, classifications, growth, growth effects, transition probabilities, attribute correlations, several transition reliability metrics, and model fit. Includes longitudinal DCM reliability metrics developed by Schellman and Madison (2024).

Value

A list with the following items:

  • $item.parameters: item parameter estimates from the specified DCM.

  • $growth: proficiency proportions for each time point and each attribute

  • $growth.effects: growth effect size estimates for each attribute and specified transitions including growth in proficiency proportion, odds ratio = odds proficiency at later time point divided by odds of proficiency at earlier time point, and Cohen's h (arcsine-transformed difference in proportions; Cohen, 1988)

  • $transition.probabilities: conditional attribute proficiency transition probability matrices

  • $posterior.probabilities: examinee marginal attribute posterior probabilities of proficiency

  • $transition.posteriors: examinee marginal attribute transition posterior probabilities

  • $most.likely.transitions: examinee most likely transitions for each attribute and transition

  • $classifications: examinee classifications determined by the specified threshold applied to the posterior probabilities

  • $reliability: estimated transition reliability metrics for each attribute for the specified transitions. “pt bis” = longitudinal point biserial metric; “info gain” = longitudinal information gain metric; “polychor” = longitudinal tetrachoric metric; “ave max tr” = average maximum transition posterior metric; “P(t>k)” = proportion of examinee marginal attribute transition posteriors greater than k; “wt pt bis” = weighted longitudinal point biserial; “wt info gain” = weighted longitudinal information gain.

  • $att.corr: estimated attribute correlation matrix

  • $model.fit: Several model fit indices and tests are output including item root mean square error of approximation (RMSEA; von Davier, 2005), mean RMSEA, bivariate item fit statistics (Chen et al., 2013), and absolute fit statistics such as mean absolute deviation for observed and expected item correlations (MADcor; DiBello, Roussos, & Stout, 2007), and standardized root mean square root of squared residuals (SRMSR; Maydeu-Olivares, 2013)

References

Chen, J., de la Torre, J. ,& Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50, 123-140.

Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.

DiBello, L. V., Roussos, L. A., & Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao and S. Sinharay (Eds.), Handbook of Statistics, Vol. 26 (pp.979–1030). Amsterdam: Elsevier.

Johnson, M. S., & Sinharay, S. (2020). The reliability of the posterior probability of skill attainment in diagnostic classification models. Journal of Educational Measurement, 47(1), 5 – 31.

Madison, M. J. (2019). Reliably assessing growth with longitudinal diagnostic classification models. Educational Measurement: Issues and Practice, 38(2), 68-78.

Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models (with discussion). Measurement: Interdisciplinary Research and Perspectives, 11, 71-137.

Schellman, M., & Madison, M. J. (2024). Estimating the reliability of skill transition in longitudinal DCMs. Journal of Educational and Behavioral Statistics.

Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30, 251-275.

von Davier M. (2008). A general diagnostic model applied to language testing data. The British journal of mathematical and statistical psychology, 61(2), 287–307.

Examples

## Example 1: T = 2, A = 4
data(data.tdcm01, package = "TDCM")
dat1 <- data.tdcm01$data
qmat1 <- data.tdcm01$q.matrix

# estimate TDCM with invariance assumed and full LCDM
m1 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "LCDM")

# summarize results with tdcm.summary function
results1 <- TDCM::tdcm.summary(m1, transition.option = 1)
results1$item.parameters
results1$growth
results1$growth.effects
results1$transition.probabilities
results1$reliability
head(results1$most.likely.transitions)
results1$model.fit$Item.RMSEA