| Session Title: Evaluation in the Face of Uncertainty: Resolving the Tension Between Need for Design Integrity and Need to Adopt Evaluation to Shifting Outcomes |
| Think Tank Session 466 to be held in Avila B on Thursday, Nov 3, 4:30 PM to 6:00 PM |
| Sponsored by the Systems in Evaluation TIG |
| Presenter(s): |
| Jonathan Morell, Fulcrum Corporation, jamorell@jamorell.com |
| Discussant(s): |
| Sanjeev Sridharan, University of Toronto, sridharans@smh.ca |
| Abstract: Good evaluation usually require maintaining the integrity of a design over time. As examples, it may be necessary to conduct interviews just before treatment, to maintain control groups, to assure the usefulness of a validated scale, or to nurture relationships with gatekeepers to particular data sets. In these examples, good design means inflexible design. So what to do when a design optimized for one set of outcomes confronts an unexpected set of outcomes? How can a design be made robust in the face of such change, or agile to adapt to new circumstances? Solutions exist, but each can induce its own set of problems. We will explore these issues via: 1) A short overview of possible solutions. 2) Breakouts for groups to deal with unexpected change in a scenario common to all the groups. 3) Report back discussions to explore the relative merits of each group's solution. |