Files
Abstract
Computerized adaptive testing (CAT) is a popular method for boosting efficiency, reducing costs, and improving examinee reactions in employee selection. Extant research has primarily established that CAT provides utility over static personality testing when the response model is monotonic (i.e., higher standing on the trait results in participant endorsement of a higher response option). Given the recent emergence of the use of ideal point item response theory (IRT) models for personality testing—which assume that higher response probability is inversely related to an individual’s distance from the item—it is important that research examine whether these models support effective CAT, and the test characteristics that may play a role. This is because ideal point models require more response data than monotonic models to accurately estimate θ, potentially hindering the utility of CAT. The present study used real-data simulations to examine the performance of different CAT conditions using a pool of conscientiousness items calibrated under the generalized graded unfolding model (GGUM) on a sample of 1,724 Amazon Mechanical Turk workers. General measurement accuracy/precision and the accuracy of dichotomous employee selection decisions based upon theta estimates were examined while manipulating the cut-score adopted for employee selection, total test length, number of pre-adaptive items presented (i.e., an initial testlet), and the use of a sequential versus multistage testing design. Results indicate that adaptive tests outperform ideal point static tests on general measures of accuracy but not on employee selection decision accuracy. The most critical test characteristic for successful adaptive testing is the presence of an initial testlet. Implications for testing theory, CAT design considerations, and future research directions are discussed.