| Session Title: Assessing Impacts in Real World Evaluations: Alternatives to the Conventional Statistical Counterfactual |
| Think Tank Session 236 to be held in CROCKETT C on Thursday, Nov 11, 9:15 AM to 10:45 AM |
| Sponsored by the International and Cross-cultural Evaluation TIG |
| Presenter(s): |
| Michael Bamberger, Independent Consultant, jmichaelbamberger@gmail.com |
| Discussant(s): |
| Jim Rugh, Independent Consultant, jimrugh@mindspring.com |
| Megan Steinke, Save the Children, msteinke@savechildren.org |
| J Bradley Cousins, University of Ottawa, bcousins@uottawa.ca |
| Abstract: Only a small fraction of program evaluations can estimate impacts using a statistically defined counterfactual. However, it is widely recognized that the absence of a methodology for defining and testing alternative possible explanations (rival hypotheses) of the observed changes in the project population increases the risk of biased or unreliable estimates of project effects. So what advice can we offer to evaluators on alternatives to the conventional statistical? The proposed session is a follow-up to a 2009 AEA think tank attended by over 50 participants in which a range of quantitative, mixed methods and theory based approaches to defining alternative counterfactuals based on participants own experience in the field were identified. There has been active follow-up resulting in the documentation of these alternative approaches. The 2010 think tank will build on the approaches and challenges identified in Orlando and will explore methodological questions relating to these innovative approaches. |