|
Session Title: Design and Implementation of Large-scale Evaluations: Lessons Relearned
|
|
Panel Session 279 to be held in Laguna A on Thursday, Nov 3, 10:45 AM to 11:30 AM
|
|
Sponsored by the Quantitative Methods: Theory and Design TIG
|
| Chair(s): |
| George Julnes, University of Baltimore, gjulnes@ubalt.edu
|
| Discussant(s):
|
| Frederick L Newman, Florida International University, newmanf@fiu.edu
|
| George Julnes, University of Baltimore, gjulnes@ubalt.edu
|
| Abstract:
Amidst the controversies over methodology across evaluation paradigms, there remains the imperative of conducting good evaluations. The presenters in this session, Len Bickman and Debra Rog, offer lessons learned and relearned in designing and implementing complex, large-scale evaluations.
|
|
Implementing a Complex Systems Evaluation
|
| Debra Rog, Westat, debrarog@westat.com
|
|
This presentation will describe the evaluation of a comprehensive initiative intended to reform the housing and service delivery systems for homeless families in three counties. Designed as a highly formative evaluation with developmental features, the evaluation of the Gates Foundation Washington Families Fund Systems Initiative is intended to both provide ongoing guidance to the Foundation and assess the Initiative's outcomes at multiple levels. The presentation will review the design and its implementation, including a qualitative longitudinal assessment of each of the three target communities and two comparison communities, case studies of selected organizations, and a family impact study involving a baseline cohort, an intervention cohort, and a constructed comparison sample from state data at both baseline and intervention time frames. Opportunities that have been seized will be highlighted as well as challenges and strategies that have been used to deal with them.
|
|
|
Designing and Implementing a Complex, 28-Site Randomized Cluster Evaluation
|
| Leonard Bickman, Vanderbilt University, leonard.bickman@vanderbilt.edu
|
|
Designing a complex multi-site evaluation is difficult enough; implementing it is even more so. Lessons learned will be presented and discussed. The presentation will deal with implementation lessons learned from this and other studies conducted by the author and will describe "salvage" strategies when the inevitable expected but unpredictable problems occur. Discussion will also include approaches that can be taken in grant proposals to ameliorate these predictably unpredictable problems.
| |