The importance of quality management in development evaluations has become increasingly recognized, particularly in the context of efforts to improve development effectiveness. International organizations and donor agencies have established evaluation systems and processes to ensure quality of evaluations. In Korea, evaluation was made mandatory for ODA implementing agencies in 2014, resulting in a surge in the number of agencies conducting evaluations as well as evaluation reports they produce. However, there remains limited awareness among many institutions regarding the issue of quality in evaluations, with the lack of a system to assess the quality of evaluation reports, causing a particular concern.

This study aims to explore the evaluation quality management systems and practices in place at international organizations and development agencies. The goal is to provide implications and recommendations for establishing quality standards for evaluation and introducing a quality management system to enhance the evaluation quality of Korea’s ODA. The study focuses on the quality assessment systems and methods leading up to the evaluation report, which is the final outcome of the evaluation process. 

Chapter 2 highlights the current management practices for evaluation quality at multilateral organizations, with a particular emphasis on the United Nations funds and programs. The chapter analyzes the cases of UNICEF and UNFPA in detail. Initially introduced to enhance the quality of decentralized evaluations at regional and country office levels, the quality management system has since evolved into a broader axis of evaluation management. Both UNICEF and UNFPA have implemented organization-wide evaluation quality management systems (GEROS and EQAA, respectively) and regularly conduct quality assurance and assessment of the entire evaluation process and reports based on relevant quality standards. Their integrated quality assessment includes central evaluations carried out by the evaluation departments of the organization headquarters. Similarly, other UN agencies have adopted comparable quality assessment systems, engaging external evaluators for the task.

Chapter 3 examines the evaluation quality management practices of major bilateral donors and analyzes the quality assessment system in use at Norad and KOICA. In bilateral donor agencies, quality assessment is often conducted as part of internal evaluation management, unlike the standardized approach adopted by many UN agencies. Most bilateral donors, including Norad, undertake internal quality reviews or external meta-evaluations. KOICA has established the Independent Evaluation Panel to perform regular quality assessment which involves both internal and external reviews.

The key characteristics of evaluation quality management identified from the four case studies can be summarized as follows: first, quality management is integrated into institutional evaluation systems based on widely adopted evaluation principles in development cooperation, such as the evaluation norms of the United Nations Evaluation Group or the evaluation quality standards of the OECD DAC; second, quality assessments are conducted on a rolling basis by external experts to ensure objectivity, using detailed checklists as a guide; and third, these assessments serve multiple purposes, including assessing the usability of evaluation results, accumulating evaluation data, and identifying best practices. To optimize the utilization of evaluation quality assessment system, it is essential to secure procedural legitimacy and consistency across the quality standards among those who conduct the assessment.

To ensure the long-term effectiveness of evaluations undertaken by Korea’s ODA implementing agencies, it is crucial to develop guidelines for managing their overall evaluation quality based on an assessment of evaluation procedures as well as organizational and human capacity. A comprehensive analysis of the merits and challenges of quality assurance systems being used by other donors would yield insights into how evaluation quality management could be institutionalized in Korean agencies. Additionally, a follow-up research is required to identify the role of the Committee for International Development Cooperation in managing the quality of evaluations conducted by ODA implementing agencies and supporting them in improving evaluation quality management practices.