Accounting for Unit of Scale in Standard Setting Methodologies

Substantial sums of money are invested by governments in school testing programs and in reporting student performance against standards. Standard-setting exercises are commonly used to determine the point at which it can be said that students have demonstrated achievement of the standard. These exercises invariably take place in situations where the reporting of educational standards has a high profile and is of political importance. Often a wide number of judges are invited to participate and two or three different standard setting methodologies are used. Inevitably, however, variability between the judges or the methodologies causes concern. Can the public have confidence in the standard set if different cut-scores are produced? Two different standard-setting methodologies have been employed in this study to translate descriptions of the standards into cut-scores. The study provides the opportunity to examine the impact that differences in the unit of scale have on the location of the cut-score.

Dr Sandy Heldsinger teaches in educational assessment and measurement at the Graduate School of Education, University of Western Australia. Her central research interest is in examining ways to help teachers obtain good information about growth in student learning.