| In a 90 minute Roundtable session, the first
rotation uses the first 45 minutes and the second rotation uses the last 45 minutes.
|
| Roundtable Rotation I:
Evaluation in the Wild: The Challenges of Effectiveness Evaluations |
|
Roundtable Presentation 209 to be held in the Limestone Boardroom on Thursday, Nov 6, 9:15 AM to 10:45 AM
|
|
Sponsored by the Theories of Evaluation TIG
|
| Presenter(s):
|
| Andrea Beesley,
Mid-Continent Research for Education and Learning,
abeesley@mcrel.org
|
| Sheila A Arens,
Mid-Continent Research for Education and Learning,
sarens@mcrel.org
|
| Abstract:
This session addresses the difference between tightly controlled pilot studies or trials (efficacy), as compared to evaluations of programs in a more realistic, less controlled setting (effectiveness). Efficacy evaluations emphasize internal validity, while effectiveness evaluations take place (usually) after some evidence of efficacy has been obtained, and the emphasis is on external validity. Because effectiveness evaluations take place “in the wild,” that is, in the real-world setting with all of its variations, diversity, and external influences, the evaluator cannot necessarily expect that the program will be implemented exactly as it was in pilot studies or efficacy trials. The presenters will discuss the differences between efficacy and effectiveness evaluations, and give examples from their work. They will also raise issues of client/developer relations, recruiting, implementation fidelity, and data collection and analysis in an effectiveness evaluation. Participants will be encouraged to discuss these issues and contribute knowledge from their own experiences.
|
| Roundtable Rotation II:
What Evaluation Theory Doesn’t Tell Us: A Conversation Between Evaluator and Evaluand |
|
Roundtable Presentation 209 to be held in the Limestone Boardroom on Thursday, Nov 6, 9:15 AM to 10:45 AM
|
|
Sponsored by the Theories of Evaluation TIG
|
| Presenter(s):
|
| Kyoung Jin Kim,
University of Illinois Urbana-Champaign,
kkim37@uiuc.edu
|
| Tania Rempert,
University of Illinois Urbana-Champaign,
trempert@uiuc.edu
|
| Abstract:
Using actual quotations and comments as shared in relation to two recent evaluations, the authors consider what an evaluator and evaluand might discuss if they were to consider evaluation theory within the context of their evaluation in practice; uncovering the effect that contextual factors have on evaluation theory when translated into evaluation practice. As a dialogue between an evaluator and evaluand demonstrates, this paper addresses 1) What role does evaluation theory play in helping determine how evaluations will be implemented?, 2) Does evaluation theory offer enough practical advice to evaluation practitioners regarding what strategies, methods, and tools are appropriate for a particular context?, and 3) Does evaluation theory help guide the evaluator’s thinking about how to interact with stakeholders, enact value commitments, and attend to key characteristics of different contexts?
|