Evaluation Techniques for Difficult to Measure Programs: For Human Services, Nonprofit, Grant Funded, and Education Programs

Evaluation Techniques for Difficult to Measure Programs demonstrates the weaknesses of poorly crafted outcome measures and provides the reader with techniques to strengthen programs and provide clients with the quality services they deserve. Programs with difficult to measure outcomes provide inviting environments for weak evaluations, and this book illustrates why typical evaluation methods result in less than stellar results. Examples of difficult to measure programs present techniques that can make any evaluation more rigorous. This book will guide the reader in overcoming inappropriate measures, false perceptions, and misconceptions that plague many evaluations.

The real work of evaluations is using limited resources to provide the best services possible for their clients. This job becomes very difficult when the results of an evaluation indicate that clients are not served well by the program. Old data collection methods might not have been adequate, methods of analyzing the data might not bring issues to light, assumptions of benefits might be false, or the program never questioned a myriad of improper uses of data. We discover the ugly side of program evaluations when programs use data because it is easy to collect, use the data that always provides glowing reports, or find a way to “make data say what they want it to say.”

The correct data to answer the research question may not be available because there has never been any effort to collect it. Data may be available, but no one has taken the time to analyze it properly. There are far too many reasons to grab data, even if it has been manipulated, and present it as valid findings. The program may have conducted an analysis, but it only makes the program look like it wasted its resources and they did not achieve its outcomes. This book will help anyone involved in programs to stop pretending they have valid data and move toward evaluations that will protect scarce resources and provide the services their clients deserve.