Multiple-choice tests commonly administered in undergraduate biology courses often emphasize factual recall and fail to develop students' critical thinking skills. Since abandoning multiple-choice tests is not realistic for instructors with hundreds of students, this tripartite study was conducted to develop a method for writing and validating multiple-choice items that require critical thinking skills and to demonstrate the advantages of Item Response Theory (IRT) over Classical Test Theory (CTT) for analyzing exams. Two semesters, Spring 2008 and Spring 2010, of multiple-choice, final exam data from an undergraduate introductory biology course were analyzed using CTT and IRT. Both measurement paradigms generated estimates of item difficulty, student ability, and test reliability. However, the IRT analysis provided more information than the CTT analysis. The IRT analysis showed that the exams did not contain enough difficult questions to precisely measure the ability levels of high achieving students. The second phase of this study began with the development of 41 multiple-choice items for the undergraduate biology course that proposed to require critical thinking skills. To validate if the items require critical thinking skills, they were submitted to faculty reviewers who rated whether or not the items required critical thinking skills and they were tested in cognitive think-aloud sessions with undergraduate students. Data from the validation studies provided strong evidence that 32 of the 41 items required critical thinking skills and weak evidence for two items. Data on the remaining 7 items either showed that they did not require critical think skills or were inconclusive. This phase showed the need to investigate the validity of test items and demonstrated a method for doing so. Twenty-three of the validated critical thinking items were included on the final exam for undergraduate introductory biology course. An IRT analysis was conducted on the exam and again it was found that the test did not contain enough difficult questions to precisely measure the ability levels of high achieving students. The IRT analysis also provided insights into writing multiple-choice items for undergraduate biology that require critical thinking, sources of item difficulty, and areas of student difficulty.