| Session Title: Alternatives to the Conventional Statistical Counterfactual: Assessing Effects and Impacts in Real-world Evaluations |
| Think Tank Session 713 to be held in Sebastian Section I2 on Saturday, Nov 14, 9:15 AM to 10:45 AM |
| Sponsored by the Quantitative Methods: Theory and Design TIG |
| Presenter(s): |
| Michael Bamberger, Independent Consultant, michaelbamberger@gmail.com |
| Discussant(s): |
| Jim Rugh, Independent Consultant, jimrugh@mindspring.com |
| Fred Carden, International Development Research Centre, fcarden@idrc.ca |
| Michael Bamberger, Independent Consultant, michaelbamberger@gmail.com |
| Abstract: Probably no more than ten percent of project or program evaluations use strong quantitative designs where impacts are estimated using a statistically defined counterfactual. Yet clients and other stakeholders of the remaining ninety percent (or more) of programs are equally concerned to know the extent to which their interventions have achieved demonstrable impact. Most evaluators believe that estimating program effects and impacts requires a methodology to define and eliminate alternative explanations (rival hypotheses) for the observed changes in the program population. So what advice can we offer to evaluators on alternatives to the conventional statistical counterfactual (e.g. randomized control trials and quasi-experimental designs)? The think tank will review some of the alternative approaches mentioned in the literature including: logic models, concept mapping, theories of change, most significant changes, participatory rapid appraisal and other participatory group methods, pipeline evaluation designs and creative uses of secondary data. |