Methods of Evaluating Educational Technology

This volume gathers some of the methods being developed by evaluators from university settings and the private sector. While providing models and methods, these authors also raise larger questions, such as: "How can schools meet the challenge of educating all children without being limited by the educational legacy of a 'one size fits all' curriculum and normative testing?" More than documenting an "apprenticeship to gadgetry," evaluators are seeking to measure meaningful learning and changes in teaching - investigating approaches that are not possible or that are less accessible when students are in traditional classrooms without technology.

In this first volume of the series Research Methods for Educational Technology (RMET) the contributing authors draw upon examples of their work evaluating the implementation and development of educational technology as well as the impact of policies and programs in this field. Within this volume several authors have written about the implementation and evaluation of technology across cultures and national boundaries, pointing to an area of research that will rapidly expand in this decade. The concern for meeting the needs of policymakers is also apparent in several of these chapters, but there is tension between providing them with positive results to support their efforts and reexamining the questions they are asking and how these questions are developed. We know that evaluation is not the extended arm of public relations, and yet it becomes clear that evaluators are often asked to demonstrate a project "is successful" on the threat that the funding will be cut. While this decision-making process fits the timetable of the fiscal year, it does not acknowledge that evaluation can be formative and strengthen programs. This timetable also ignores the investment of time that is needed when implementing innovations like the Internet into teaching and learning.

Many of the authors included in this volume write from the context of evaluating federally-funded programs, and they provide valuable insight for future projects which are created and evaluated at the state-level. As technology initiatives are developed and funded outside of the federal arena, more evaluators will be called upon. From approaches developed from federally-funded projects, we can build upon these methods and models for evaluation within regional projects to answer questions related to budgets and accountability. As we answer these immediate questions, we can move forward to examine the long-term impact of technology, and the possibility that exercises in conformity will replace the adventure of human enlightenment for our children.

Walter F. Heineke is assistant professor of Educational Policy and Evaluation Studies at the Curry School of Education, University of Virginia. He serves as a member of the Center for Technology and Teacher Education. He conducts research on evaluation of educational technology, educational technology policy and practice. He is coeditor of the annual Research Methods for Educational Technology.

Laura Blasi is a doctoral candidate at the University of Virginia in the Curry School of Education. While serving as a fellow at the Center for Technology and Teacher Education (CTTE), she has also been evaluating educational technology in K-12 and university settings, with emphasis on equity issues. She is currently co-editor of the annual Research Methods for Educational Technology.

Prior to her work in Charlottesville, she developed equity programs for the Association of Colleges and Universities (AAC&U) including Diversity Web. She can be reached at blasi@mindspring.com.