Title: | The Transition Diagnostic Classification Model Framework |
---|---|
Description: | Estimate the transition diagnostic classification model (TDCM) described in Madison & Bradshaw (2018) <doi:10.1007/s11336-018-9638-5>, a longitudinal extension of the log-linear cognitive diagnosis model (LCDM) in Henson, Templin & Willse (2009) <doi:10.1007/s11336-008-9089-5>. As the LCDM subsumes many other diagnostic classification models (DCMs), many other DCMs can be estimated longitudinally via the TDCM. The 'TDCM' package includes functions to estimate the single-group and multigroup TDCM, summarize results of interest including item parameters, growth proportions, transition probabilities, transitional reliability, attribute correlations, model fit, and growth plots. |
Authors: | Matthew J. Madison [aut, cph] |
Maintainer: | Michael E. Cotterell <[email protected]> |
License: | MIT + file LICENSE |
Version: | 0.2.0.9000 |
Built: | 2025-02-24 05:51:38 UTC |
Source: | https://github.com/cotterell/tdcm |
Several data sets for the TDCM package.
data.tdcm01 data.tdcm02 data.tdcm03 data.tdcm04 data.tdcm05
data.tdcm01 data.tdcm02 data.tdcm03 data.tdcm04 data.tdcm05
data.tdcm01
is a simulated dataset with two time points, four
attributes, twenty items, one group of size 1000, and a single Q-matrix. The
format is a list of two:
data
: a data frame of binary item responses
q.matrix
: a data frame specifying the Q-matrix
data.tdcm02
is a simulated dataset with three time points, two
attributes, ten items, one group of size 2500, and a single Q-matrix. The
format is a list of two:
data
: a data frame of binary item responses
q.matrix
: a data frame specifying the Q-matrix
data.tdcm03
is a simulated dataset with three time points, two
attributes, one group of size 1500, and three different ten-item Q-matrices
for each time point. Anchor items are specified as items 1/11/21 and items
14/24. The format is a list of five:
data
: a data frame of binary item responses
q.matrix.1
: a data frame specifying the Q-matrix for the first time point
q.matrix.2
: a data frame specifying the Q-matrix for the second time point
q.matrix.3
: a data frame specifying the Q-matrix for the third time point
q.matrix.stacked
: data frame specifying the combined Q-matrix for all time
points
data.tdcm04
is a simulated dataset with two time points, four
attributes, twenty items, two group of size 800 and 900, respectively, and a
single Q-matrix. The format is a list of three:
data
: a data frame of binary item responses
q.matrix
: a data frame specifying the Q-matrix
groups
: a vector specifying the examinee group membership
data.tdcm05
is a simulated dataset with one time point, four
attributes, and twenty items. For use with the 1-PLCDM. The format is a list
of two:
data
: a data frame of binary item responses
q.matrix
: a data frame specifying the Q-matrix
############################# ## Example 1: T = 2, A = 4 ## data(data.tdcm01, package = "TDCM") data <- data.tdcm01$data q.matrix <- data.tdcm01$q.matrix model <- TDCM::tdcm(data, q.matrix, num.time.points = 2) results <- TDCM::tdcm.summary(model) results$item.parameters results$growth.effects ############################# ## Example 3: T = 3, A = 2 ## data <- data.tdcm03$data q1 <- data.tdcm03$q.matrix.1 q2 <- data.tdcm03$q.matrix.2 q3 <- data.tdcm03$q.matrix.3 q <- data.tdcm03$q.matrix.stacked #TDCM with anchor items constrained m1 <- tdcm(data, q, num.time.points = 3, num.q.matrix = 3, anchor = c(1,11,1,21,14,24), num.items = c(10,10,10)) #TDCM without anchor items m2 <- tdcm(data, q, num.time.points = 3, num.q.matrix = 3, num.items = c(10,10,10)) #Compare models to assess measurement invariance tdcm.compare(m1, m2) #Summarize model 1 r1 = tdcm.summary(m1, transition.option = 3) r1$item.parameters r1$growth r1$growth.effects
############################# ## Example 1: T = 2, A = 4 ## data(data.tdcm01, package = "TDCM") data <- data.tdcm01$data q.matrix <- data.tdcm01$q.matrix model <- TDCM::tdcm(data, q.matrix, num.time.points = 2) results <- TDCM::tdcm.summary(model) results$item.parameters results$growth.effects ############################# ## Example 3: T = 3, A = 2 ## data <- data.tdcm03$data q1 <- data.tdcm03$q.matrix.1 q2 <- data.tdcm03$q.matrix.2 q3 <- data.tdcm03$q.matrix.3 q <- data.tdcm03$q.matrix.stacked #TDCM with anchor items constrained m1 <- tdcm(data, q, num.time.points = 3, num.q.matrix = 3, anchor = c(1,11,1,21,14,24), num.items = c(10,10,10)) #TDCM without anchor items m2 <- tdcm(data, q, num.time.points = 3, num.q.matrix = 3, num.items = c(10,10,10)) #Compare models to assess measurement invariance tdcm.compare(m1, m2) #Summarize model 1 r1 = tdcm.summary(m1, transition.option = 3) r1$item.parameters r1$growth r1$growth.effects
Function to estimate estimate item influence measures. Code adapted from (Jurich & Madison, 2023). This function is not available for longitudinal DCMs.
item.influence(model, data, fullcorrelation = FALSE, progress = TRUE)
item.influence(model, data, fullcorrelation = FALSE, progress = TRUE)
model |
a previously calibrated model; an object of class |
data |
a required |
fullcorrelation |
optional logical argument indicating a full or reduced response-classification correlation matrix. |
progress |
An optional logical indicating whether the function should print the progress of estimation. |
For DCMs, item influence quantifies how much an item impacts classifications. Given an estimated DCM and item response data, this function estimates five item influence measures, including item pull, item override, proportion of attribute information, response-classification correlation (corr1), and response-posterior correlation (corr2).
A list containing several item influence measures.
Currently, this function currently only runs on DCMs estimated at a single time point. It will not run properly for TDCM objects.
Jurich, D. & Madison, M. J. (2023). Measuring item influence for diagnostic classification models. Educational Assessment.
## Item influence illustration #load data (simulated based on Jurich and Bradshaw (2014)) qmatrix <- CDM::data.sda6$q.matrix responses <- CDM::data.sda6$data #Estimate the full LCDM model1 <- CDM::gdina(responses, qmatrix, linkfct = "logit", method = "ML") #Estimate item influence measures influence <- TDCM::item.influence(model1, responses) #Summarize influence statistics influence$Pull #item pull influence$Override #item override influence$Information #proportion of attribute information influence$Correlation1 #correlation of responses and classifications influence$Correlation2 #correlation of responses and posterior probabilities
## Item influence illustration #load data (simulated based on Jurich and Bradshaw (2014)) qmatrix <- CDM::data.sda6$q.matrix responses <- CDM::data.sda6$data #Estimate the full LCDM model1 <- CDM::gdina(responses, qmatrix, linkfct = "logit", method = "ML") #Estimate item influence measures influence <- TDCM::item.influence(model1, responses) #Summarize influence statistics influence$Pull #item pull influence$Override #item override influence$Information #proportion of attribute information influence$Correlation1 #correlation of responses and classifications influence$Correlation2 #correlation of responses and posterior probabilities
This function estimates the multigroup TDCM (Madison & Bradshaw, 2018).
mg.tdcm( data, q.matrix, num.time.points, rule = "LCDM", linkfct = "logit", groups, forget.att = c(), group.invariance = TRUE, time.invariance = TRUE, progress = TRUE )
mg.tdcm( data, q.matrix, num.time.points, rule = "LCDM", linkfct = "logit", groups, forget.att = c(), group.invariance = TRUE, time.invariance = TRUE, progress = TRUE )
data |
A required |
q.matrix |
a required |
num.time.points |
The number of time points (i.e., measurement/testing occasions),
integer |
rule |
A string or a vector of the specific DCM to be employed. Currently accepts the
same values as |
linkfct |
A string indicating the LCDM link function. Currently accepts "logit" (default) to estimate the LCDM. Can be specified "identity" to estimate the GDINA model. Also accepts a "log" link function. |
groups |
A required vector of integer group identifiers (e.g., 1, 2, 3) for multiple group estimation. |
forget.att |
An optional vector allowing for constraining of individual attribute proficiency
loss, or forgetting. The default allows forgetting for each measured attribute
(e.g., |
group.invariance |
logical. If |
time.invariance |
logical. If |
progress |
logical. If |
An object of class gdina
with entries as indicated in the CDM package.
For the TDCM-specific results (e.g., growth, transitions), use TDCM::mg.tdcm.summary()
.
Currently, the TDCM::mg.tdcm()
function only accepts a single Q-matrix.
de la Torre, J. (2011). The Generalized DINA Model Framework. Psychometrika 76, 179–199. doi:10.1007/s11336-011-9207-7
George, A. C., Robitzsch, A., Kiefer, T., Gross, J., & Ünlü , A. (2016). The R package CDM for Cognitive Diagnosis Models. Journal of Statistical Software, 74(2), 1-24. doi:10.18637/jss.v074.i02
Henson, R., Templin, J., & Willse, J. (2009). Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables. Psychometrika, 74, 191-21. doi:10.1007/s11336-008-9089-5
Johnson, M. S., & Sinharay, S. (2020). The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models. Journal of Educational Measurement, 47(1), 5–31. doi:10.3102/1076998619864550
Kaya, Y., & Leite, W. (2017). Assessing Change in Latent Skills Across Time With Longitudinal Cognitive Diagnosis Modeling: An Evaluation of Model Performance. Educational and Psychological Measurement, 77(3), 369–388. doi:10.1177/0013164416659314
Li, F., Cohen, A., Bottge, B., & Templin, J. (2015). A Latent Transition Analysis Model for Assessing Change in Cognitive Skills. Educational and Psychological Measurement, 76(2), 181–204. doi:10.1177/0013164415588946
Madison, M. J. (2019). Reliably Assessing Growth with Longitudinal Diagnostic Classification Models. Educational Measurement: Issues and Practice, 38(2), 68-78. doi:10.1111/emip.12243
Madison, M. J., & Bradshaw, L. (2018a). Assessing Growth in a Diagnostic Classification Model Framework. Psychometrika, 83(4), 963-990. doi:10.1007/s11336-018-9638-5
Madison, M. J., & Bradshaw, L. (2018b). Evaluating Intervention Effects in a Diagnostic Classification Model Framework. Journal of Educational Measurement, 55(1), 32-51. doi:10.1111/jedm.12162
Madison, M.J., Chung, S., Kim, J., & Bradshaw, L.P. (2024) Approaches to estimating longitudinal diagnostic classification models. Behaviormetrika, 51(7), 7-19. doi:10.1007/s41237-023-00202-5
Rupp, A. A., Templin, J., & Henson, R. (2010). Diagnostic Measurement: Theory, Methods, and Applications. New York: Guilford. ISBN: 9781606235430.
Schellman, M., & Madison, M. J. (2024). Estimating the reliability of skill transition in longitudinal DCMs. Journal of Educational and Behavioral Statistics.
Templin, J., & Bradshaw, L. (2013). Measuring the Reliability of Diagnostic Classification Model Examinee Estimates. Journal of Classification, 30, 251-275. doi:10.1007/s00357-013-9129-4
Wang. S., Yang. Y., Culpepper, S. A., & Douglas, J. (2018). Tracking Skill Acquisition With Cognitive Diagnosis Models: A Higher-Order, Hidden Markov Model With Covariates. Journal of Educational and Behavioral Statistics, 43(1), 57-87. doi:10.3102/1076998617719727
## Example 4: G = 2, T = 2, A = 4 data(data.tdcm04, package = "TDCM") data <- data.tdcm04$data q.matrix <- data.tdcm04$q.matrix groups <- data.tdcm04$groups # Estimate full multigroup TDCM with invariance assumed. mg.model <- TDCM::mg.tdcm(data, q.matrix, num.time.points = 2, groups = groups) # summarize results results <- TDCM::mg.tdcm.summary(mg.model) # plot results TDCM::tdcm.plot(results)
## Example 4: G = 2, T = 2, A = 4 data(data.tdcm04, package = "TDCM") data <- data.tdcm04$data q.matrix <- data.tdcm04$q.matrix groups <- data.tdcm04$groups # Estimate full multigroup TDCM with invariance assumed. mg.model <- TDCM::mg.tdcm(data, q.matrix, num.time.points = 2, groups = groups) # summarize results results <- TDCM::mg.tdcm.summary(mg.model) # plot results TDCM::tdcm.plot(results)
A function to compile results from calibration of the multigroup TDCM (Madison & Bradshaw, 2018).
mg.tdcm.summary( model, transition.option = 1, classthreshold = 0.5, attribute.names = c(), group.names = c() )
mg.tdcm.summary( model, transition.option = 1, classthreshold = 0.5, attribute.names = c(), group.names = c() )
model |
a |
transition.option |
option for reporting results. |
classthreshold |
probability threshold for establishing proficiency from examinee posterior probabilities. Default is .50, which maximizes overall classification accuracy. It can be set to a lower value to minimize false negatives (i.e., misclassifying proficient examinees as non-proficient) or set to a higher value to minimize false positives (i.e., misclassifying non-proficient examinees as proficient). |
attribute.names |
optional vector of attribute names to include in plots. |
group.names |
optional vector of group names to include in plots. Enter in order corresponding to the
integer labels in the groups vector specified in the |
Provides a summary of multigroup TDCM results including item parameters, attribute posterior probabilities, transition posterior probabilities, classifications, group-wise growth, group-wise transition probabilities, attribute correlations, several transition reliability metrics, and model fit. Includes longitudinal versions of reliability metrics developed by Templin and Bradshaw (2013) and Johnson and Sinharay (2020).
A list with the following items:
$item.parameters
: item parameter estimates from the specified DCM.
$growth
: proficiency proportions for each time point and each attribute
$growth.effects
: growth effect size estimates for each attribute and specified transitions
including growth in proficiency proportion, odds ratio = odds proficiency at later time point divided by odds
of proficiency at earlier time point, and Cohen's h (arcsine-transformed difference in proportions;
Cohen, 1988)
$transition.probabilities
: conditional attribute proficiency transition probability matrices
$posterior.probabilities
: examinee marginal attribute posterior probabilities of proficiency
$transition.posteriors
: examinee marginal attribute transition posterior probabilities
$most.likely.transitions
: examinee most likely transitions for each attribute and transition
$classifications
: examinee classifications determined by the specified threshold applied to the
posterior probabilities
$reliability
: estimated transition reliability metrics for each attribute for the specified
transitions. “pt bis” = longitudinal point biserial metric; “info gain” = longitudinal information
gain metric; “polychor” = longitudinal tetrachoric metric; “ave max tr” = average maximum transition
posterior metric; “P(t>k)” = proportion of examinee marginal attribute transition posteriors greater than k;
“wt pt bis” = weighted longitudinal point biserial; “wt info gain” = weighted longitudinal information gain.
$att.corr
: estimated attribute correlation matrix
$model.fit
: Several model fit indices and tests are output including item root mean square
error of approximation (RMSEA; von Davier, 2005), mean RMSEA, bivariate item fit statistics (Chen et al.,
2013), and absolute fit statistics such as mean absolute deviation for observed and expected item
correlations (MADcor; DiBello, Roussos, & Stout, 2007), and standardized root mean square root of
squared residuals (SRMSR; Maydeu-Olivares, 2013)
Chen, J., de la Torre, J. ,& Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50, 123-140.
DiBello, L. V., Roussos, L. A., & Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao and S. Sinharay (Eds.), Handbook of Statistics, Vol. 26 (pp.979–1030). Amsterdam: Elsevier.
Johnson, M. S., & Sinharay, S. (2020). The reliability of the posterior probability of skill attainment in diagnostic classification models. Journal of Educational Measurement, 47(1), 5 – 31.
Madison, M. J. (2019). Reliably assessing growth with longitudinal diagnostic classification models. Educational Measurement: Issues and Practice, 38(2), 68-78.
Madison, M. J., & Bradshaw, L. (2018). Evaluating intervention effects in a diagnostic classification model framework. Journal of Educational Measurement, 55(1), 32-51.
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models (with discussion). Measurement: Interdisciplinary Research and Perspectives, 11, 71-137.
Schellman, M., & Madison, M. J. (2024). Estimating the reliability of skill transition in longitudinal DCMs. Journal of Educational and Behavioral Statistics.
Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30, 251-275.
von Davier M. (2008). A general diagnostic model applied to language testing data. The British journal of mathematical and statistical psychology, 61(2), 287–307.
## Example 4: G = 2, T = 2, A = 4 data(data.tdcm04, package = "TDCM") dat4 <- data.tdcm04$data qmat4 <- data.tdcm04$q.matrix group4 <- data.tdcm04$groups # estimate mgTDCM with invariance assumed and full LCDM mg1 <- TDCM::mg.tdcm(dat4, qmat4, rule = "LCDM", num.time.points = 2, group = group4, group.invariance = TRUE, time.invariance = TRUE) # summarize results results1 <- TDCM::mg.tdcm.summary(mg1) # plot results TDCM::tdcm.plot(results1) # estimate mgTDCM without group invariance mg2 <- TDCM::mg.tdcm(dat4, qmat4, rule = "LCDM",num.time.points = 2, group = group4, group.invariance = FALSE, time.invariance = TRUE) # compare models to assess group invariance TDCM::tdcm.compare(mg1, mg2)
## Example 4: G = 2, T = 2, A = 4 data(data.tdcm04, package = "TDCM") dat4 <- data.tdcm04$data qmat4 <- data.tdcm04$q.matrix group4 <- data.tdcm04$groups # estimate mgTDCM with invariance assumed and full LCDM mg1 <- TDCM::mg.tdcm(dat4, qmat4, rule = "LCDM", num.time.points = 2, group = group4, group.invariance = TRUE, time.invariance = TRUE) # summarize results results1 <- TDCM::mg.tdcm.summary(mg1) # plot results TDCM::tdcm.plot(results1) # estimate mgTDCM without group invariance mg2 <- TDCM::mg.tdcm(dat4, qmat4, rule = "LCDM",num.time.points = 2, group = group4, group.invariance = FALSE, time.invariance = TRUE) # compare models to assess group invariance TDCM::tdcm.compare(mg1, mg2)
Function to estimate the 1-PLCDM (Madison et al., 2023; Maas et al., 2023).
oneplcdm(data, q.matrix, progress = TRUE)
oneplcdm(data, q.matrix, progress = TRUE)
data |
a required |
q.matrix |
a required |
progress |
An optional logical indicating whether the function should print the progress of estimation. |
Estimates the single-attribute and multi-attribute 1-PLCDM described in Madison et al. (2024). Example shows that attribute subscores are sufficient statistics for classifications.
An object of class gdina
with entries as indicated in the CDM package.
Currently, this model cannot be embedded within the TDCM via the rule
argument.
George, A. C., Robitzsch, A., Kiefer, T., Gross, J., & Ünlü , A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74(2), 1-24.
Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log linear models with latent variables. Psychometrika, 74, 191-21.
Madison, M.J., Wind, S., Maas, L., Yamaguchi, K. & Haab, S. (2024). A one-parameter diagnostic classification model with familiar measurement properties. Journal of Educational Measurement.
Maas, L., Madison, M. J., & Brinkhuis, M. J. (2024). Properties and performance of the one-parameter log-linear cognitive diagnosis model. Frontiers.
## Example 1: A = 4 data(data.tdcm05) dat5 <- data.tdcm05$data qmat5 <- data.tdcm05$q.matrix # calibrate LCDM m1 <- CDM::gdina(dat5, qmat5, linkfct = "logit", method = "ML") # calibrate 1-PLCDM m2 <- TDCM::oneplcdm(dat5, qmat5) summary(m2) #demonstrate 1-PLCDM sum score sufficiency for each attribute subscores <- cbind(rowSums(dat5[, 1:5]), rowSums(dat5[, 6:10]), rowSums(dat5[, 11:15]), rowSums(dat5[, 16:20])) colnames(subscores) <- c("Att1", "Att2", "Att3", "Att4") proficiency <- cbind(m2$pattern[, 6] > .50, m2$pattern[, 7] > .50, m2$pattern[, 8] > .50, m2$pattern[, 9] > .5) * 1 table(subscores[, 1], proficiency[, 1]) table(subscores[, 2], proficiency[, 2]) table(subscores[, 3], proficiency[, 3]) table(subscores[, 4], proficiency[, 4]) #plot sum score sufficiency for each attribute posterior1pl <- m2$pattern[, 6:9] posteriorlcdm <- m1$pattern[, 6:9] oldpar <- par(mfrow = c(2, 2)) for (i in 1:4) { plot(subscores[, i], posteriorlcdm[, i], pch = 19,las = 1, cex.lab = 1.5, xlab = "Sum Scores", ylab = "P(proficiency)", cex.main = 1.5, col = "grey", xaxt = "n", yaxt = "n", cex = 1.2, main = paste("Attribute ", i, sep = "")) graphics::axis(side = 1, at = c(0, 1, 2, 3, 4, 5), ) graphics::axis(side = 2, at = c(0, .1, .2, .3, .4, .5, .6, .7, .8, .9, 1.0), las = 1) graphics::points(subscores[, i], posterior1pl[, i], col = "black", pch = 18, cex = 1.5) graphics::abline(a = .50, b = 0, col = "red") graphics::legend("bottomright", c("1-PLCDM", "LCDM"), col = c("black", "grey"), pch = c(18 ,19), box.lwd = 0, box.col = "white", bty = 'n') } par(oldpar)
## Example 1: A = 4 data(data.tdcm05) dat5 <- data.tdcm05$data qmat5 <- data.tdcm05$q.matrix # calibrate LCDM m1 <- CDM::gdina(dat5, qmat5, linkfct = "logit", method = "ML") # calibrate 1-PLCDM m2 <- TDCM::oneplcdm(dat5, qmat5) summary(m2) #demonstrate 1-PLCDM sum score sufficiency for each attribute subscores <- cbind(rowSums(dat5[, 1:5]), rowSums(dat5[, 6:10]), rowSums(dat5[, 11:15]), rowSums(dat5[, 16:20])) colnames(subscores) <- c("Att1", "Att2", "Att3", "Att4") proficiency <- cbind(m2$pattern[, 6] > .50, m2$pattern[, 7] > .50, m2$pattern[, 8] > .50, m2$pattern[, 9] > .5) * 1 table(subscores[, 1], proficiency[, 1]) table(subscores[, 2], proficiency[, 2]) table(subscores[, 3], proficiency[, 3]) table(subscores[, 4], proficiency[, 4]) #plot sum score sufficiency for each attribute posterior1pl <- m2$pattern[, 6:9] posteriorlcdm <- m1$pattern[, 6:9] oldpar <- par(mfrow = c(2, 2)) for (i in 1:4) { plot(subscores[, i], posteriorlcdm[, i], pch = 19,las = 1, cex.lab = 1.5, xlab = "Sum Scores", ylab = "P(proficiency)", cex.main = 1.5, col = "grey", xaxt = "n", yaxt = "n", cex = 1.2, main = paste("Attribute ", i, sep = "")) graphics::axis(side = 1, at = c(0, 1, 2, 3, 4, 5), ) graphics::axis(side = 2, at = c(0, .1, .2, .3, .4, .5, .6, .7, .8, .9, 1.0), las = 1) graphics::points(subscores[, i], posterior1pl[, i], col = "black", pch = 18, cex = 1.5) graphics::abline(a = .50, b = 0, col = "red") graphics::legend("bottomright", c("1-PLCDM", "LCDM"), col = c("black", "grey"), pch = c(18 ,19), box.lwd = 0, box.col = "white", bty = 'n') } par(oldpar)
tdcm()
is used to estimate the transition diagnostic classification model
(TDCM; Madison & Bradshaw, 2018a), which is a longitudinal extension of the
log-linear cognitive diagnosis model (LCDM; Henson, Templin, & Willse,
2009). For the multigroup TDCM, see mg.tdcm()
. It allows for the
specification of many specific DCMs via the rule
option. The default DCM
rule and link function specifies the LCDM. The rule can be changed to
estimate the DINA model, DINO model, CRUM (i.e., ACDM, or main effects
model), or reduced interaction versions of the LCDM. The link function can be
changed to specify the GDINA model.
tdcm( data, q.matrix, num.time.points, invariance = TRUE, rule = "LCDM", linkfct = "logit", num.q.matrix = 1, num.items = c(), anchor = c(), forget.att = c(), progress = TRUE )
tdcm( data, q.matrix, num.time.points, invariance = TRUE, rule = "LCDM", linkfct = "logit", num.q.matrix = 1, num.items = c(), anchor = c(), forget.att = c(), progress = TRUE )
data |
A required |
q.matrix |
A required |
num.time.points |
A required integer |
invariance |
logical. If |
rule |
A string or a vector indicating the specific DCM to be employed.
A vector of supported |
linkfct |
A string or a vector indicating the LCDM link function. Currently accepts "logit" (default) to estimate the LCDM. Can be specified "identity" to estimate the GDINA model. Also accepts a "log" link function. |
num.q.matrix |
An optional integer specifying the number of
Q-matrices. For many applications, the same assessment is administered at
each time point and this number is 1 (the default). If there are
different Q-matrices for each time point, then this argument must be
specified and should be equal to the number of time points. For example,
if there are three time points, and the Q-matrix for each time point is
different, then |
num.items |
An integer specifying the number of items. When there are
multiple Q-matrices, the number of items in each Q-matrix is specified as
a vector. For example, if there are three time points, and the Q-matrices
for each time point have 8, 10, and 12 items, respectively, then
|
anchor |
When there are different tests at each time point, this
optional argument is a vector of pairs of item numbers indicating which
items are the same across time points and should be held invariant. For
example, if there are three Q-matrices with 10 items each, and Items 1,
11, and 21 are the same, and Items 14 and 24 are the same, then |
forget.att |
An optional vector allowing for constraining of individual
attribute proficiency loss, or forgetting. The default allows forgetting
for each measured attribute (e.g., |
progress |
logical. If |
Estimation of the TDCM via the CDM package (George, et al., 2016), which is based on an EM algorithm as described in de la Torre (2011). The estimation approach is further detailed in Madison et al. (2023).
An object of class gdina
with entries as described in
CDM::gdina()
. To see a TDCM-specific summary of the object (e.g.,
growth, transitions), use tdcm.summary()
.
de la Torre, J. (2011). The Generalized DINA Model Framework. Psychometrika 76, 179–199. doi:10.1007/s11336-011-9207-7
George, A. C., Robitzsch, A., Kiefer, T., Gross, J., & Ünlü , A. (2016). The R package CDM for Cognitive Diagnosis Models. Journal of Statistical Software, 74(2), 1-24. doi:10.18637/jss.v074.i02
Henson, R., Templin, J., & Willse, J. (2009). Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables. Psychometrika, 74, 191-21. doi:10.1007/s11336-008-9089-5
Johnson, M. S., & Sinharay, S. (2020). The Reliability of the Posterior Probability of Skill Attainment in Diagnostic Classification Models. Journal of Educational Measurement, 47(1), 5–31. doi:10.3102/1076998619864550
Kaya, Y., & Leite, W. (2017). Assessing Change in Latent Skills Across Time With Longitudinal Cognitive Diagnosis Modeling: An Evaluation of Model Performance. Educational and Psychological Measurement, 77(3), 369–388. doi:10.1177/0013164416659314
Li, F., Cohen, A., Bottge, B., & Templin, J. (2015). A Latent Transition Analysis Model for Assessing Change in Cognitive Skills. Educational and Psychological Measurement, 76(2), 181–204. doi:10.1177/0013164415588946
Madison, M. J. (2019). Reliably Assessing Growth with Longitudinal Diagnostic Classification Models. Educational Measurement: Issues and Practice, 38(2), 68-78. doi:10.1111/emip.12243
Madison, M. J., & Bradshaw, L. (2018a). Assessing Growth in a Diagnostic Classification Model Framework. Psychometrika, 83(4), 963-990. doi:10.1007/s11336-018-9638-5
Madison, M. J., & Bradshaw, L. (2018b). Evaluating Intervention Effects in a Diagnostic Classification Model Framework. Journal of Educational Measurement, 55(1), 32-51. doi:10.1111/jedm.12162
Madison, M.J., Chung, S., Kim, J., & Bradshaw, L.P. (2024) Approaches to estimating longitudinal diagnostic classification models. Behaviormetrika, 51(7), 7-19. doi:10.1007/s41237-023-00202-5
Rupp, A. A., Templin, J., & Henson, R. (2010). Diagnostic Measurement: Theory, Methods, and Applications. New York: Guilford. ISBN: 9781606235430.
Schellman, M., & Madison, M. J. (2024). Estimating the reliability of skill transition in longitudinal DCMs. Journal of Educational and Behavioral Statistics.
Templin, J., & Bradshaw, L. (2013). Measuring the Reliability of Diagnostic Classification Model Examinee Estimates. Journal of Classification, 30, 251-275. doi:10.1007/s00357-013-9129-4
Wang. S., Yang. Y., Culpepper, S. A., & Douglas, J. (2018). Tracking Skill Acquisition With Cognitive Diagnosis Models: A Higher-Order, Hidden Markov Model With Covariates. Journal of Educational and Behavioral Statistics, 43(1), 57-87. doi:10.3102/1076998617719727
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") data <- data.tdcm01$data q.matrix <- data.tdcm01$q.matrix # Estimate full TDCM with invariance assumed. model1 <- TDCM::tdcm(data, q.matrix, num.time.points = 2) # Summarize results with tdcm.summary(). results <- TDCM::tdcm.summary(model1) results$item.parameters results$growth results$transition.probabilities
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") data <- data.tdcm01$data q.matrix <- data.tdcm01$q.matrix # Estimate full TDCM with invariance assumed. model1 <- TDCM::tdcm(data, q.matrix, num.time.points = 2) # Summarize results with tdcm.summary(). results <- TDCM::tdcm.summary(model1) results$item.parameters results$growth results$transition.probabilities
Provides a comparison of two TDCMs. Can be used to compare different measurement models or assess measurement invariance over time or over groups in the multigroup TDCM case. Only accepts two models.
tdcm.compare(model1, model2)
tdcm.compare(model1, model2)
model1 |
|
model2 |
a second |
This function returns a data frame with model fit statistics (AIC/BIC) and results from a likelihood ratio or deviance test.
Currently, this function currently accepts two models for comparison.
Both models must be fit to the same item responses and Q-matrix.
The function will provide results for two non-nested models. Please ensure that models are nested before interpreting the likelihood ratio test for nested models.
The likelihood ratio test is not valid for some model comparisons (e.g., LCDM vs DINA) because of model constraints.
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") dat1 <- data.tdcm01$data qmat1 <- data.tdcm01$q.matrix # estimate TDCM with invariance assumed and full LCDM m1 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "LCDM") # estimate TDCM with invariance not assumed m2 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = FALSE, rule = "LCDM") # compare models to assess measurement invariance. TDCM::tdcm.compare(m1, m2)
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") dat1 <- data.tdcm01$data qmat1 <- data.tdcm01$q.matrix # estimate TDCM with invariance assumed and full LCDM m1 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "LCDM") # estimate TDCM with invariance not assumed m2 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = FALSE, rule = "LCDM") # compare models to assess measurement invariance. TDCM::tdcm.compare(m1, m2)
tdcm.plot()
visualizes the results from TDCM analyses.
tdcm.plot(results, attribute.names = c(), group.names = c(), type = "both")
tdcm.plot(results, attribute.names = c(), group.names = c(), type = "both")
results |
results from |
attribute.names |
an optional vector of attribute names to include in plots. |
group.names |
an optional vector of group names to include in plots. |
type |
an option to specify the type of plot in single group cases; "both" is default and will produce a line plot and a bar chart; "line" will produce a line plot; and "bar" will produce a bar chart. |
No return value, called for side effects.
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") dat1 = data.tdcm01$data qmat1 = data.tdcm01$q.matrix #estimate TDCM with invariance assumed and full LCDM m1 = TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "LCDM") #summarize results with tdcm.summary function results1 = TDCM::tdcm.summary(m1) #plot results TDCM::tdcm.plot(results1, attribute.names = c("Addition", "Subtraction", "Multiplication", "Division"))
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") dat1 = data.tdcm01$data qmat1 = data.tdcm01$q.matrix #estimate TDCM with invariance assumed and full LCDM m1 = TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "LCDM") #summarize results with tdcm.summary function results1 = TDCM::tdcm.summary(m1) #plot results TDCM::tdcm.plot(results1, attribute.names = c("Addition", "Subtraction", "Multiplication", "Division"))
A condensation rule is a formula that states how different attributes
combine to form an observed or latent response (Rupp, Templin, & Henson,
2010). The TDCM package includes support for "LCDM"
, "DINA"
,
"DINO"
, "CRUM"
, "RRUM"
, "LCDM1"
for the LCDM with only main effects,
"LCDM2"
for the LCDM with two-way interactions, "LCDM3"
, and so on.
Evaluate TDCM::tdcm.rules$TDCM
for a complete list of condensation rules
supported by the TDCM package.
tdcm.rules
tdcm.rules
An object of class data.frame
with 15 rows and 2 columns.
Rupp, A. A., Templin, J., & Henson, R. (2010). Diagnostic Measurement: Theory, Methods, and Applications. New York: Guilford. ISBN: 9781606235430.
TDCM::tdcm.rules$TDCM
TDCM::tdcm.rules$TDCM
Function to score responses with fixed item parameters from a previously calibrated LCDM.
tdcm.score( calibration.model, newdata, q.matrix, attr.prob.fixed = NULL, progress = TRUE )
tdcm.score( calibration.model, newdata, q.matrix, attr.prob.fixed = NULL, progress = TRUE )
calibration.model |
the previously calibrated model; an object of class |
newdata |
a required |
q.matrix |
a required |
attr.prob.fixed |
optional argument for attribute profile proportions. Default is uniform distribution of profiles. |
progress |
An optional logical indicating whether the function should print the progress of estimation. |
Obtain classifications for new responses to items that were previously calibrated. The calibrate-and-score approach is further detailed in Madison et al. (2023).
An object of class gdina
with entries as indicated in the CDM package.
George, A. C., Robitzsch, A., Kiefer, T., Gross, J., & Ünlü , A. (2016). The R package CDM for cognitive diagnosis models. Journal of Statistical Software, 74(2), 1-24.
Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log linear models with latent variables. Psychometrika, 74, 191-21.
Madison, M.J., Chung, S., Kim, J., & Bradshaw, L. (2023). Approaches to estimating longitudinal diagnostic classification models. Behaviormetrika.
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") dat1 <- data.tdcm01$data qmat1 <- data.tdcm01$q.matrix pre <- dat1[, 1:20] post <- dat1[, 21:40] # calibrate LCDM with post-test data m1 <- CDM::gdina(data = pre, q.matrix = qmat1, linkfct = "logit", method = "ML") # score pre-test responses m2 <- TDCM::tdcm.score(m1, newdata = post, q.matrix = qmat1) summary(m2) m2$pattern
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") dat1 <- data.tdcm01$data qmat1 <- data.tdcm01$q.matrix pre <- dat1[, 1:20] post <- dat1[, 21:40] # calibrate LCDM with post-test data m1 <- CDM::gdina(data = pre, q.matrix = qmat1, linkfct = "logit", method = "ML") # score pre-test responses m2 <- TDCM::tdcm.score(m1, newdata = post, q.matrix = qmat1) summary(m2) m2$pattern
Function to summarize results from TDCM analyses.
tdcm.summary( model, transition.option = 1, classthreshold = 0.5, attribute.names = c() )
tdcm.summary( model, transition.option = 1, classthreshold = 0.5, attribute.names = c() )
model |
a |
transition.option |
option for reporting results. |
classthreshold |
probability threshold for establishing proficiency from examinee posterior probabilities. Default is .50, which maximizes overall classification accuracy. It can be set to a lower value to minimize false negatives (i.e., misclassifying proficient examinees as non-proficient) or set to a higher value to minimize false positives (i.e., misclassifying non-proficient examinees as proficient). |
attribute.names |
optional vector of attribute names to include in results output. |
Provides a summary of TDCM results including item parameters, attribute posterior probabilities, transition posterior probabilities, classifications, growth, growth effects, transition probabilities, attribute correlations, several transition reliability metrics, and model fit. Includes longitudinal DCM reliability metrics developed by Schellman and Madison (2024).
A list with the following items:
$item.parameters
: item parameter estimates from the specified DCM.
$growth
: proficiency proportions for each time point and each attribute
$growth.effects
: growth effect size estimates for each attribute and specified transitions
including growth in proficiency proportion, odds ratio = odds proficiency at later time point divided by odds
of proficiency at earlier time point, and Cohen's h (arcsine-transformed difference in proportions;
Cohen, 1988)
$transition.probabilities
: conditional attribute proficiency transition probability matrices
$posterior.probabilities
: examinee marginal attribute posterior probabilities of proficiency
$transition.posteriors
: examinee marginal attribute transition posterior probabilities
$most.likely.transitions
: examinee most likely transitions for each attribute and transition
$classifications
: examinee classifications determined by the specified threshold applied
to the posterior probabilities
$reliability
: estimated transition reliability metrics for each attribute for the specified
transitions. “pt bis” = longitudinal point biserial metric; “info gain” = longitudinal information gain
metric; “polychor” = longitudinal tetrachoric metric; “ave max tr” = average maximum transition posterior
metric; “P(t>k)” = proportion of examinee marginal attribute transition posteriors greater than k;
“wt pt bis” = weighted longitudinal point biserial; “wt info gain” = weighted longitudinal information gain.
$att.corr
: estimated attribute correlation matrix
$model.fit
: Several model fit indices and tests are output including item root mean square
error of approximation (RMSEA; von Davier, 2005), mean RMSEA, bivariate item fit statistics
(Chen et al., 2013), and absolute fit statistics such as mean absolute deviation for observed and
expected item correlations (MADcor; DiBello, Roussos, & Stout, 2007), and standardized root mean
square root of squared residuals (SRMSR; Maydeu-Olivares, 2013)
Chen, J., de la Torre, J. ,& Zhang, Z. (2013). Relative and absolute fit evaluation in cognitive diagnosis modeling. Journal of Educational Measurement, 50, 123-140.
Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Publishers.
DiBello, L. V., Roussos, L. A., & Stout, W. F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In C. R. Rao and S. Sinharay (Eds.), Handbook of Statistics, Vol. 26 (pp.979–1030). Amsterdam: Elsevier.
Johnson, M. S., & Sinharay, S. (2020). The reliability of the posterior probability of skill attainment in diagnostic classification models. Journal of Educational Measurement, 47(1), 5 – 31.
Madison, M. J. (2019). Reliably assessing growth with longitudinal diagnostic classification models. Educational Measurement: Issues and Practice, 38(2), 68-78.
Maydeu-Olivares, A. (2013). Goodness-of-fit assessment of item response theory models (with discussion). Measurement: Interdisciplinary Research and Perspectives, 11, 71-137.
Schellman, M., & Madison, M. J. (2024). Estimating the reliability of skill transition in longitudinal DCMs. Journal of Educational and Behavioral Statistics.
Templin, J., & Bradshaw, L. (2013). Measuring the reliability of diagnostic classification model examinee estimates. Journal of Classification, 30, 251-275.
von Davier M. (2008). A general diagnostic model applied to language testing data. The British journal of mathematical and statistical psychology, 61(2), 287–307.
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") dat1 <- data.tdcm01$data qmat1 <- data.tdcm01$q.matrix # estimate TDCM with invariance assumed and full LCDM m1 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "LCDM") # summarize results with tdcm.summary function results1 <- TDCM::tdcm.summary(m1, transition.option = 1) results1$item.parameters results1$growth results1$growth.effects results1$transition.probabilities results1$reliability head(results1$most.likely.transitions) results1$model.fit$Item.RMSEA
## Example 1: T = 2, A = 4 data(data.tdcm01, package = "TDCM") dat1 <- data.tdcm01$data qmat1 <- data.tdcm01$q.matrix # estimate TDCM with invariance assumed and full LCDM m1 <- TDCM::tdcm(dat1, qmat1, num.time.points = 2, invariance = TRUE, rule = "LCDM") # summarize results with tdcm.summary function results1 <- TDCM::tdcm.summary(m1, transition.option = 1) results1$item.parameters results1$growth results1$growth.effects results1$transition.probabilities results1$reliability head(results1$most.likely.transitions) results1$model.fit$Item.RMSEA