Policy Reference
PUBLISH
Policy Reference
Exploring Unintended Effects in Development Evaluations: A Case Study of the Republic of Korea
ODA,
ODA Evaluation
Author Jisun Jeong, Aila Yoo, and Chami Park Series 24-03 Language Korean Date 2024.12.31
This study investigates the consideration of unintended effects within South Korea’s burgeoning Official Development Assistance (ODA) programs. Despite Korea’s growing role as an emerging donor, research on the unintended consequences of its aid interventions remains limited. This analysis addresses this gap by examining how key Korean ODA implementing agencies, the Korea International Cooperation Agency (KOICA), the Export-Import Bank of Korea (KEXIM) and the Korea Foundation for International Healthcare (KOFIH), address unintended consequences in their evaluation reports.
Employing a mixed-methods approach – literature review, text analysis, and qualitative content analysis – the study explores the extent to which these agencies analyze and report on unintended effects in evaluations conducted over the past decade. Text analysis of agency reports (extracted from KOICA, KEXIM and KOFIH websites) identifies the frequency and nature of unintended effects discussed, utilizing software (CAQDAS) and keyword searches (“unintended,” “unplanned,” etc.). Thematic analysis then categorizes identified effects based on established typologies(Koch 2024; Jabeen 2018).
Findings reveal that Korea’s major aid agencies analyzed unintended effects in approximately 8% of evaluations, with KOICA featuring a balanced view of both positive and negative effects, while KEXIM and KOFIH focused predominantly on positive effects. The most frequently observed positive unintended effect was the spill-over effects, spread of benefits to non-beneficiary groups and area, fostering community solidarity, economic gains, and empowerment for women. A microcredit program targeting women for increased household income inadvertently led to partners’ awareness of women’s economic and social capacity, leading to enhanced understanding of human rights. Additionally, a cross-sectoral effect was observed in a school feeding program, where children’s educational outcomes improved as a result of better nutrition.
In contrast, negative unintended effects, such as elite capture and regressive targeting—wherein benefits unintentionally excluded vulnerable populations—highlighted gaps in project design and stakeholder engagement. Notably, rural development programs like the Samaeul initiative were found to unintentionally benefit wealthier farmers, excluding poorer ones (regressive targeting, Koch, 2024). Similarly, vocational training for internally displaced persons (IDPs) in post-conflict settings risked unintended discrimination within IDP communities, potentially undermining social cohesion. In microcredit program targeting former refugee villagers provided loans for agricultural activity, unintentionally marginalizing female-led/elderly-led household as they tend to have insufficient labor to increase agricultural productivity.
Conversely, macro-level effects like anti-aid sentiments, governance issues, and environmental impacts were rarely mentioned. These findings may be attributed to factors like East Asian cultural tendencies that discourage open discussions about failure, the relatively recent and smaller scale of Korean interventions, and the absence of a colonial past unlike traditional donors.
Additional semi-structured interviews with domestic and international evaluation experts, reaffirmed the study’s conclusions and contributed to refining its recommendations. The analysis underscores the importance of developing more complexity theory and systems thinking based Theory of Changes that could help predict and indentify not only intended but also potential unintended consequences of the evaluated intervention. Strategies include assessing risks and designing safeguards for negative unintended effects during project planning, fostering mechanisms to enhance positive spillover, and strengthening human rights and gender considerations. In performance management, recommendations emphasize monitoring not only outputs and outcomes but also underlying assumptions and risks, and refining ToRs to mandate or recommend UIE evaluation where relevant. Additionally, evaluators could adopt open-ended, exploratory interview technique and evaluation methods like outcome harvesting, and participatory narrative inquiry (PNI) which are arguably better suited to identify the unintended effects in development interventions.
The study suggests institutional measures to conduct more strategic evaluations with long-term perspectives. Establishing a culture that distinguishes evaluation from auditing is also recommended as a precondition for further discussion and evaluation of unintended effects of international development cooperation as donor agencies are reluctant to openly discuss the unintended effects of their intervention, which are often perceived as negative results of mismanagement of taxpayer’s money.
This study marks the first in-depth examination of unintended effects in Korean ODA evaluations. By comparing practices across donor countries and defining unintended effects in the Korean context, the study provides critical insights for enhancing developmental effectiveness, helping policymakers, evaluation managers, and researchers consider new, often-overlooked aspects of development cooperation. Importantly, this research underlines the need to address both planned and unintended effects in Korea’s ODA, setting a foundation for resilient, inclusive, and sustainable aid interventions.
Employing a mixed-methods approach – literature review, text analysis, and qualitative content analysis – the study explores the extent to which these agencies analyze and report on unintended effects in evaluations conducted over the past decade. Text analysis of agency reports (extracted from KOICA, KEXIM and KOFIH websites) identifies the frequency and nature of unintended effects discussed, utilizing software (CAQDAS) and keyword searches (“unintended,” “unplanned,” etc.). Thematic analysis then categorizes identified effects based on established typologies(Koch 2024; Jabeen 2018).
Findings reveal that Korea’s major aid agencies analyzed unintended effects in approximately 8% of evaluations, with KOICA featuring a balanced view of both positive and negative effects, while KEXIM and KOFIH focused predominantly on positive effects. The most frequently observed positive unintended effect was the spill-over effects, spread of benefits to non-beneficiary groups and area, fostering community solidarity, economic gains, and empowerment for women. A microcredit program targeting women for increased household income inadvertently led to partners’ awareness of women’s economic and social capacity, leading to enhanced understanding of human rights. Additionally, a cross-sectoral effect was observed in a school feeding program, where children’s educational outcomes improved as a result of better nutrition.
In contrast, negative unintended effects, such as elite capture and regressive targeting—wherein benefits unintentionally excluded vulnerable populations—highlighted gaps in project design and stakeholder engagement. Notably, rural development programs like the Samaeul initiative were found to unintentionally benefit wealthier farmers, excluding poorer ones (regressive targeting, Koch, 2024). Similarly, vocational training for internally displaced persons (IDPs) in post-conflict settings risked unintended discrimination within IDP communities, potentially undermining social cohesion. In microcredit program targeting former refugee villagers provided loans for agricultural activity, unintentionally marginalizing female-led/elderly-led household as they tend to have insufficient labor to increase agricultural productivity.
Conversely, macro-level effects like anti-aid sentiments, governance issues, and environmental impacts were rarely mentioned. These findings may be attributed to factors like East Asian cultural tendencies that discourage open discussions about failure, the relatively recent and smaller scale of Korean interventions, and the absence of a colonial past unlike traditional donors.
Additional semi-structured interviews with domestic and international evaluation experts, reaffirmed the study’s conclusions and contributed to refining its recommendations. The analysis underscores the importance of developing more complexity theory and systems thinking based Theory of Changes that could help predict and indentify not only intended but also potential unintended consequences of the evaluated intervention. Strategies include assessing risks and designing safeguards for negative unintended effects during project planning, fostering mechanisms to enhance positive spillover, and strengthening human rights and gender considerations. In performance management, recommendations emphasize monitoring not only outputs and outcomes but also underlying assumptions and risks, and refining ToRs to mandate or recommend UIE evaluation where relevant. Additionally, evaluators could adopt open-ended, exploratory interview technique and evaluation methods like outcome harvesting, and participatory narrative inquiry (PNI) which are arguably better suited to identify the unintended effects in development interventions.
The study suggests institutional measures to conduct more strategic evaluations with long-term perspectives. Establishing a culture that distinguishes evaluation from auditing is also recommended as a precondition for further discussion and evaluation of unintended effects of international development cooperation as donor agencies are reluctant to openly discuss the unintended effects of their intervention, which are often perceived as negative results of mismanagement of taxpayer’s money.
This study marks the first in-depth examination of unintended effects in Korean ODA evaluations. By comparing practices across donor countries and defining unintended effects in the Korean context, the study provides critical insights for enhancing developmental effectiveness, helping policymakers, evaluation managers, and researchers consider new, often-overlooked aspects of development cooperation. Importantly, this research underlines the need to address both planned and unintended effects in Korea’s ODA, setting a foundation for resilient, inclusive, and sustainable aid interventions.
Sales Info
| Quantity/Size | 130 |
|---|---|
| Sale Price | 7 $ |
