|
Session Title: A Seven Year External Evaluation of an International Aid Program in 25 Countries
|
|
Panel Session 872 to be held in El Capitan B on Saturday, Nov 5, 9:50 AM to 11:20 AM
|
|
Sponsored by the International and Cross-cultural Evaluation TIG
|
| Chair(s): |
| Michael Scriven, Claremont Graduate University, mjscriv1@gmail.com
|
| Abstract:
Four of the key participants will describe their experiences, approaches, and lessons learned, in the course of a external impact evaluation that went on for seven years and involved putting evaluation teams into villages and homes in 25 countries from all the non-polar continents. Some issues and achievements that may be of general interest include: selecting and supervising interpreters and local researchers; facilitating open communications; developing a complex model that was both program specific and readily adaptable to other programs; finding a way to establish causation beyond reasonable doubt without using control groups; retaining reasonable independence despite a long and amiable relationship with the client (including contracts that were always limited to one year at a time); getting at values and attitude changes as well as the directly observable housing, nutrition, and economic changes.
|
|
The Role of the Evaluation Manager
|
| Thomaz Chianca, COMEA Evaluation Ltd, thomaz.chianca@gmail.com
|
|
“The Role of the Evaluation Manager” will review some of the statistics on number of interviews at different levels, and the complex logistics involved, as well as lessons learned about handling resistance to evaluation, or to having the occasional all-female teams, and some of the interesting issues about how to help teams when the country they were in ran into typhoons or revolutions.
|
|
|
The Client’s Point of View
|
| Rienzzie Kern, Heifer International, rkern@heifer.org
|
|
“The Client’s Point of View” will cover his take on the Heifer management support for an external impact evaluation; his own hopes and conclusions about it; and internal acceptance of the reports and recommendations.
| |
|
Design and Methodology Issues
|
| Michael Scriven, Claremont Graduate University, mjscriv1@gmail.com
|
|
“Design and Methodology Issues” will look in particular at (i) the management of the 20+ values that Heifer wanted impact on, and how to avoid a merely goal-based approach; (ii) the development of a systematic alternative to the RCT approach (namely, the GEM model, standing for General Elimination Methodology) and its defense against the usual criticisms plus the more serious but rarely mentioned ‘upswing terminator’ problem; and (iii) the interesting case of a Heifer middle manager who decided he would use threats of non-cooperation in order to get changes made in our design.
| |
|
|
| Paul Clements, Western Michigan University, paul.clements@wmu.edu
|
|
Refining the Quantitative Approach to Cost Analysis
| |