Research, Monitoring and Evaluation, in Public Health

This is a presentation on the overview of the role of monitoring and evaluation in public health. It describes the various components and how a robust M&E system can possitively impact the results or effectiveness of a public health intervention.

1. WEST AFRICAN INSTITUTE OF PUBLIC HEALTH (WAIPH) Health Literacy and Leadership Programme (HLLP).

2. MONITORING AND EVALUATION IN PUBLIC HEALTH Ojore Godday Aghedo, MPH, BLS, AMIIM HRMA, June 13, 2024

3. OUTLINE Objectives of M&E Introduction What is Monitoring and Evaluation (M&E)? Importance of M&E The M&E Process M&E Plans Steps to building an M&E Plan Challenges and Solutions M&E Best Practices Tools and Resources Conclusion Q&A

4. Objectives of M&E Assessing program performance and progress. Identifying strengths and weaknesses. Informing decision-making and program improvements. Ensuring accountability to stakeholders.

5. INTRODUCTION What is Monitoring and evaluation (M&E)? It is a process used to assess the effectiveness of public health programs and interventions. It involves collecting data, analyzing it, and making informed decisions based on the findings. It helps to assess the effectiveness of these programs, identify areas for improvement, and ensure that resources are being used efficiently.

6. WHAT IS “MONITORING”? Monitoring  is the systematic process of collecting, analyzing and using information to track a programme’s progress toward reaching its objectives and to guide management decisions. Monitoring is sometimes referred to as process evaluation because it focuses on the implementation process and asks key questions: How well has the program been implemented? How much does implementation vary from site to site? Did the program benefit the intended people? At what cost?

7. EXAMPLES OF MONITORING Monitoring usually pertains to counting , tracking, and collecting data. For example:

8. Examples of program elements that can be monitored

9. ILLUSTRATION OF PROGRAMME MONITORING

10. WHAT IS “EVALUATION”? Evaluation  is the systematic assessment of an activity, project, programme , strategy, policy, topic, theme, sector, operational area or institution’s performance. Assessing the outcomes and impact of public health interventions. Determines the extent to which program goals are achieved. Utilizes various methods such as surveys, interviews, and data analysis.

11. REQUIREMENTS FOR AN EVALUATION Data collection at the start of a program (to provide a baseline) and again at the end, rather than at repeated intervals during program implementation A control or comparison group in order to measure whether the changes in outcomes can be attributed to the program A well-planned study design

12. Types of Evaluation Formative Evaluation Conducted during program development or implementation. Focuses on identifying areas for improvement and refining program strategies. Summative Evaluation: Conducted at the end of a program or project. Assesses overall effectiveness and impact. Provides insights for future planning and replication.

13. OECD-DAC EVALUATION CRITERIA Relevance Effectiveness Efficiency Impact Sustainability

14. ILLUSTRATION OF PROGRAMME EVALUATION

15. EXERCISE…1 What situation does each of the following call for, “Monitoring” or “Evaluation”? The National Council of Population and Development is interested in learning whether the initiatives being implemented in province “A” are lowering teenage pregnancy rates there. USAID wants to know the number of sex workers that have been reached by your program this year. A country director is curious to know whether the post-abortion treatment offered in public clinics satisfies national criteria of quality. A 2 year HIV/AIDS project is being implemented and will close-out in one year time. The funder wants to know if the services will be sustained after eventual close-out.

16. IMPORTANCE OF M&E It allows us to track progress Identify areas for improvement

17. THE M&E PROCESS

18. M&E PLANS M&E Plans: This is the fundamental document that details a program's objectives, the interventions developed to achieve these objectives, and describes the procedures that will be implemented to determine whether or not the objectives are met.

19. SIX STEPS TO DEVELOPING AN M&E PLAN

20. CHALLENGES AND SOLUTIONS Data Quality: Ensuring accuracy, reliability, and completeness of data. Resource Constraints: Secure multiple sources of funding and build capacity of staff. Complexity: Public health interventions often involve multiple stakeholders and factors. Identfy stakeholders and pay advocacy visits to them – involve them. Sustainability: Ensuring M&E systems are maintained over the long term.

21. M&E BEST PRACTICES Stakeholder Engagement: Involving all relevant stakeholders in the M&E process. Capacity Building: Training staff in M&E methods and techniques. Use of Technology: Leveraging digital tools for data collection, analysis, and reporting. Adaptability: Being flexible and responsive to changing needs and circumstances. Clearly define the goals and objectives of the monitoring and evaluation process. Ensure that data collection methods are appropriate for the specific context and population being studied. Use standardized tools and protocols to ensure consistency and comparability of data. Regularly analyze and interpret data to identify trends, patterns, and areas for improvement. Involve stakeholders and community members in all stages of the monitoring and evaluation process. Disseminate findings in a timely and accessible manner to all relevant stakeholders. Continuously monitor and evaluate the monitoring and evaluation process itself to identify areas for improvement.

22. TOOLS AND RESOURSES The Framework: Contextual Framework, Results Framework and Logic Model M&E Plan Training manuals Supervisory Checklist Data Quality Assessment Checklist Databases and HMIS Forms and Registers Mobile technology for data collection and analysis (Excel, KoboCollect , SPSS, R, Stata, Epi-Info, Python, CSPro , PowerPoint, Google docs, etc …) use of big data and artificial intelligence (AI)

23. EXERCISE…2 When Should M&E Take Place? As a rule of thumb, what proportion of the budget should be allocated to M&E?

24. CONCLUSION M&E is an integral part of every public health programme that is planned to succeed, no matter how small or big. Without proper M&E systems, all public health programmes will either fail or make little impacts. M&E activities should be planned for and budgeted for at the planning stage of the projects. M&E unit should be properly staffed and equipped to run smoothly.

25. THANK YOU!