Highlights from Impact Evaluation Conference in Cairo
By Julia Coffman

Cairo, Egypt, March 2009: Seven hundred participants from around the world met for the Perspectives on Impact Evaluation Conference. Conference organizers used a broad definition of impact evaluation; the term was not limited to any specific methodology.

A main conference topic was “Impact Evaluation: Approaches and Methods.” Experimental or quasi-experimental designs can be impractical or inappropriate for advocacy efforts. More suitable techniques for advocacy were presented, including:

Contribution Analysis
John Mayne
In advocacy work, it can be impossible to show cause-and-effect between programs and outcomes. In such cases, “Contribution Analysis” is a useful approach. The method focuses on contribution, rather than attribution. It shows whether a credible case can be made that an intervention (or advocacy effort) contributed to certain outcomes (such as policy changes). The six-step process includes familiar theory of change construction, but adds less-conventional steps to test that theory. For more information, see Mayne’s 2008 brief, Contribution analysis: An approach to exploring cause and effect.

RealWorld Evaluation
Michael Bamberger, Jim Rugh, and Linda Mabry
Evaluators often work under less-than-ideal conditions. We almost always have limited budgets and timeframes. We often lack baseline data. We can face political pressures during the evaluation process. RealWorld Evaluation offers practical advice for handling such constraints while maintaining methodological rigor. The approach is detailed in a book from Sage Publications. Read free excerpts on the RealWorld Evaluation website.

Most Significant Change Technique
Rick Davies and Jess Dart
This technique is useful for complex efforts, like advocacy, that produce diverse and emergent outcomes. Participants offer stories about the changes an effort has achieved; an evaluator verifies the stories and selects the most significant. The approach produces compelling data and fosters a culture of reflection and learning. See Clear Horizon’s summary  or the comprehensive MSC user guide for more.

Participatory Performance Story Reporting
Jess Dart
Performance stories are short reports about how an effort contributed to its intended outcomes. The technique has two main elements:

• A five-step process for generating the performance story, and
• A five-part structure for reporting it.

A unique feature of this process is its “outcomes panel.”  The panel consists of people with relevant scientific, technical, or substantive knowledge. The panel determines whether the stories build a credible case that the effort contributed to its outcomes. Read more about this approach on Clear Horizon’s website.

The Perspectives on Impact Evaluation conference was sponsored by the African Evaluation Association (AfrEA), International Initiative for Impact Evaluation (3ie), and the Network of Networks for Impact Evaluation (NONIE).

« Back to AEU #6