This dissertation uses a simulation study to understand the model-data fit challenges in educational assessments, particularly ones under extreme base rate mastery conditions. The Log-linear Cognitive Diagnosis Model (LCDM) is the psychometric model used in this paper. The study investigates the impact of extreme data conditions on item parameter recovery and classification accuracy. Results from the simulation are analyzed using bias and RMSE between actual and estimated parameters. Hybrid, unsupervised (UML), and supervised machine learning (SML) methods for preprocessing data were used to mitigate potential estimation difficulties due to extreme base rate conditions. This involved leveraging multiclass classification and clustering algorithms. The evaluation included a comparative analysis of standard classification metrics by emphasizing the stability or enhancement in classification accuracy of Machine Learning and Artificial Intelligence algorithms under typical or extreme conditions found in educational evaluations. The algorithms that consistently demonstrated robust performance in classification accuracy under these conditions are further discussed as applied to the results of the LCDM in the scenarios studied here.
