Making Monitoring and Evaluation Systems Work: A Capacity Development Toolkit (World Bank Training Series)

Written for those who said to the authors (and for others in the same situation): “We know we need M and E, but we don’t know how to set up an M and E system, or how to make ours work well and provide the information we need.” This detailed, practical manual explains the skills and steps for making a monitoring and evaluation (M and E) system that functions well. The goal is an M and E system -- people, processes and partnerships -- that collects, verifies and analyzes good quality information that is useful and used by decision makers, managers, implementers, funders, and other stakeholders. The manual begins by introducing the concept of results-based management, and the internationally accepted “12 components of functional M and E Systems” framework. It describes the purpose and functions of each component, and builds skills by taking users through the steps for setting up, or improving what already exists. (The 12 modules – one for each component – are listed overleaf.) Each module has clear learning objectives, detailed systematic explanations and useful examples, and exercises for learners to work through that apply what is being taught (model answers are available on line). Use: It can be used as a reference manual, a training text, or for self-study. It is likely to be especially useful for trainers and trainees trying to meet the need for skilled professionals, capacity and technical advice in building sustainable M and E systems. Using the 12 components of any working M and E system as a benchmark, users can easily assess their own system for sustainability, identify where improvements are needed, and follow the detailed, systematic steps and “how to” guidance. Potential Users: The broad audience for this manual includes all those responsible for M and E, across all sectors. The primary audience is people who set up, operate, or oversee M and E functions, trying to ensure that data are collected, compiled, and provided in a useful way to decision makers. The book was written mostly for staff in government agencies at national and sub-national levels, but is fully relevant to NGOs, the private sector, and funding agencies. All implementing or policy-making organization with programs that aim for results and change need monitoring and evaluation, will find this guide useful, practical, systematic and thorough. Development: The manual shares the authors’ extensive practical, hands-on experience helping clients to build and use M and E systems, and advising on managing for results. The materials were developed and tested “in the field”, and benefited from detailed discussions with other M and E practitioner-experts to get consensus on the approach. The draft was refined after extensive peer reviews, and pilot use in training courses inTanzania and Turkey. The manual complements and extends the globally recognized best-selling book co-written by one of the authors, widely used by academic institutions, governments, and developing partners worldwide to better understand the principles and practices of results-based M and E. “Making Monitoring and Evaluation Systems Work” goes beyond the broad consensus on the need to measure, monitor, and manage to results, and the investments in monitoring and evaluation systems, to go step-by-step through the details and practicalities of making sure M and E systems are functional and sustainable. Contents Introductory Module: Introduces the Training Course, the concept of results-based management and the 12 Components of Functional M and E Systems 1. The Enabling Environment: People, organization, partnerships and planning Module 1: Organisational Structures for M and E: Explains the importance of including M and E in organisational structures, and how to plan for human resources to meet the needs of an M and E system. Module 2: Human Capacity Building for M and E: Describes the three levels of human capacity development that are an integral part of an M and E system. The module focuses on the ‘individual’ level -- on building human resource skills needed for the M and E system to function well. Module 3: M and E Partnerships: Strong M and E partnerships bring people from different organisations together to work within similar objectives and goals. M and E partnerships are especially important given that many of the people involved do not work in the same institutions or even in the same sectors. Module 4: M and E plans: Describes how to develop or review your organisation’s M and E plan. This plan, together with a costed M and E work plan, is at the heart of an M and E system -- it describes the purpose of the system, the data the system will collect, and how the system will operate Module 5: Costed M and E work plans: Explains how to develop, cost, and prioritise an M and E work plan and mobilise resources for it. The M and E work plan is an action plan that includes activities, time frames, responsibilities, and costs to make all 12 components of an M and E system work. Module 6: Advocacy and communication to create positive M and E cultures in organizations: Covers how to plan, develop and manage an advocacy and communication strategy for your organisation or country’s M and E system. Explains that the purpose of an advocacy and communication strategy is to help ensure knowledge of, and commitment to, M and E and the M and E system among policy-makers, program managers, program staff and other stakeholders. 2.Basic Activities: Data collection, capturing and verification Module 7: Routine monitoring: Explains how to identify and manage the different types of routine monitoring data that an organisation needs to collect, report on and use. Module 8: Surveys and surveillance: Explains the factors to consider in deciding whether a survey is needed for collecting needed data and how to design and implement a high quality survey. Module 9: M and E databases: The basics of how to develop an electronic database that can hold all of an organisation’s monitoring and evaluation data. Module 10: Supervision and data auditing: Describes the steps for improving and assuring the quality of data and building the capacity of staff involved in M and E through routine supportive supervision and data auditing processes. Module 11: Evaluation and research: How doing better, more targeted and well-coordinated research and evaluations can improve a program. 3.The whole point: Using information to improve results Module 12: Data analysis, information dissemination and use: Most important of all – how to plan to use the data and information generated by an M and E system to learn and gain knowledge about a program, to be able to improve it and achieve stronger results.

In Trinidad, we used the 12-component model as a diagnostic framework to assess capacity building needs to improve our national health monitoring and evaluation system. We evaluated our own system against each of the 12 components to determine where we are today, and where we need to be to better be able to monitor and manage HIV/AIDS and other health issues in our country. We appreciate this simple, yet comprehensive model and plan to continue to use it in our M&E capacity building plans. --Dr. Violet Duke , Government Official, Ministry of Health, Trinidad and Tobago

On its way toward development, it is imperative for Turkey to pull different systems together in a logical way: to make systems talk to each other. The 12 components provided different public sectors-from transport to agriculture, health to education with a way of commonly communicating about their M&E systems, understand linkages, and even plan joint activities. It enables someone who wants to make a career out of M&E to learn about one sectors 12-component M&E system and to apply this learning in another sector. The 12 components made me see how the pieces of our M&E system fit together. --Mehmet Uzunkaya, Planning Specialist, Prime Ministry of Turkey, State Planning Organization

The training I undertook on the 12 components has come in very useful. I am developing a course on Results-based M&E in Ireland, and am also using the framework to develop M&E plans and frameworks for a number of organizations, including Barnardos, UCC, and the Ministry of Resources and Rural Development in Malta and the Hope Foundation in Calcutta. This is the most practical program I have ever been exposed to on monitoring and evaluation and I intend to incorporate its principles in my work going forward. --Niamh Kenny, M&E Consultant, EXODEA Europe Consulting, Cork, Ireland