A comparison of methods for item analysis and DIF using classicaltest theory, item response theory, and generalized linear model

Using three different theoretical frameworks classical test theory, item response theory, and generalized linear model this study analyzed items and searched for DIF on a nationwide English exam for students in the third grade of middle school in South Korea. The results from the three approaches were compared. For item analysis, the estimates of item difficulty and item discrimination were examined, and the correlations between indices for item parameters were calculated. The item parameter indices from classical test theory, item response theory, and generalized linear model behaved very similarly, and the correlations were high ranging from .882 to 1.0. Thus, the results of this study indicated that indices for item parameters under three different approaches are very comparable and may be substitutable for each other. Unlike item analysis, there were large differences in detecting DIF under the different frameworks. The IRTLR procedure for item response theory and the logistic regression model under generalized linear model were more general and flexible than the Mantel-Haneszel statistic in detecting DIF.