|
Session Title: Rapid Response Methods for Real-time Evaluation
|
|
Panel Session 855 to be held in Suwannee 16 on Saturday, Nov 14, 1:40 PM to 3:10 PM
|
|
Sponsored by the Non-profit and Foundations Evaluation TIG
|
| Chair(s): |
| Gale Berkowitz, David and Lucile Packard Foundation, gberkowitz@packard.org
|
| Abstract:
In evaluation practice, timing can be everything. To ensure their work gets used, evaluators that aim to support real-time learning and decision making must deliver the right data at the right time. While this kind of real-time evaluation sounds good in theory, it can be difficult to achieve successfully in practice. When quick decisions need to be made, methods must both allow for quick design, implementation, and analysis, and provide useful and trustworthy strategy-level data. Unfortunately, many conventional evaluation methods are neither responsive nor quick. This session will introduce a set of "rapid response methods" that emphasize timing, flexibility, and responsiveness. They have quick turnaround times and bring evaluation data, in accessible formats, to the table for reflection and use in decision making. The session's presenters will describe the rapid-response approaches they have developed, discovered, and used.
|
|
With These Methods, Time is On Our Side. Yes It Is!
|
| Julia Coffman, Harvard Family Research Project, jcoffman@evaluationexchange.org
|
|
This presentation will feature the results from recent research on rapid-response methods. It will start by grounding the discussion about rapid-response approaches in evaluation history that draws on the tenets of responsive, utilization-focused, and developmental evaluation, and recognizes the contributions of a family of approaches called rapid evaluation and assessment methods (REAM). The presentation will then describe new methods uncovered during this research that can be used to quickly gather trustworthy data that decision makers can use at critical points in time. These methods either can form the basis for an evaluation design, or can be used in combination with other more conventional evaluation methods.
|
|
|
What Do We Want? Data! When Do We Want It? Now!
|
| Ehren Reed, Innovation Network Inc, ereed@innonet.org
|
|
Rapid response methods and real-time approaches are particularly useful for advocacy efforts in which strategy constantly evolves without a predictable script. To make informed decisions, advocates need timely answers to the strategic questions they regularly face and evaluation can help fill that role. This presentation will present a suite of rapid-response methods being used in the advocacy evaluation field. Methods discussed will include media scorecards, system mapping, intense-period debriefs, and policymaker ratings.
| |
|
Time is the Enemy of Utilization
|
| Andy Rowe, ARCeconomics, andy.rowe@earthlink.net
|
|
Evaluators need techniques to quickly return information and useful insights; particularly early in an evaluation when this can significantly increase the evaluator's social capital in the eyes of those who influence evaluation use. This presentation will discuss two rapid response techniques: 1) WURT (While U R There), a technique developed to obtain data on the organizational development elements of a major governance and urban environmental services program in India, and 2) web-based surveys that cut to two weeks the time from design to reporting. The presentation will discuss how rapid response techniques change the social contracts and power in an evaluation, encourage collaboration, and promote use. It also will discuss the challenges these techniques bring, such as why some methods (web surveys) flourish, and others (WURT) founder, and how to deal with the precipitous modification of programs or strategies before evaluators are fully comfortable with their own advice.
| |