Linear Models of Optimal Test Design

Over my nearly forty years of teaching and conducting research in the ?eld of psychometric methods, I have seen a number of major technical advances that respond to pressing educational and psychological measu- mentproblems. Thedevelopmentofcriterion-referencedassessmentwasthe ?rst, beginning in the late 1960s with the important work of Robert Glaser and Jim Popham, in response to the need for assessments that considered candidate performance in relation to a well-de?ned body of knowledge and skills rather than in relation to a norm group. The development of criterion-referenced testing methodology with a focus on decision-theoretic concepts and methods, content validity, standard-setting, and the recog- tionofthemeritsofbothcriterion-norm-referencedandcriterion-referenced assessments has tremendously in?uenced current test theory and testing . The second major advance was the introduction of item response-theory (IRT) and associated models and their applications to replace classical test theory (CTT) and related practices. Beginning slowly in the 1940s and 1950s with the pioneering work of Frederic Lord, Allan Birnbaum, and GeorgRasch,bythe1970sthemeasurementjournalswerefullofimportant research studies describing new IRT models, technical advances in model parameter estimation and model ?t, and research on applications of IRT models to equating, test development, the detection of potentially biased test items, and adaptive testing. The overall goal has been to improve and expand measurement practices by overcoming several shortcomings of cl- sicaltesttheory:dependenceoftest-itemstatisticsandreliabilityestimates on examinee samples, dependence of examinee true score estimates on the particular choices of test items, and the limitation in CTT of modeling ex-viii Foreword aminee performance at the test level rather than at the item level.

This book begins with a reflection on the history of test design--the core activity of all educational and psychological testing. It then presents a standard language for modeling test design problems as instances of multi-objective constrained optimization. The main portion of the book discusses test design models for a large variety of problems from the daily practice of testing, and illustrates their use with the help of numerous empirical examples. The presentation includes models for the assembly of tests to an absolute or relative target for their information functions, classical test assembly, test equating problems, item matching, test splitting, simultaneous assembly of multiple tests, tests with item sets, multidimensional tests, and adaptive test assembly. Two separate chapters are devoted to the questions of how to design item banks for optimal support of programs with fixed and adaptive tests. Linear Models for Optimal Test Design, which does not require any specific mathematical background, has been written to be a helpful resource on the desk of any test specialist.

Wim J. van der Linden is Professor of Measurement and Data Analysis, University of Twente, The Netherlands. His specialization is psychometric theory and methods, and he has been an active researcher of item response theory throughout his career. His current research is on test design, adaptive testing, test equating, and response-time modeling. Professor van der Linden is a past president of the Psychometric Society and a recipient of the NCME lifetime achievement award for his work on educational measurement.