|
Session Title: Policy and Practice Issues for Evaluators, Project Directors and the Community: Lessons Learned From the Intersection of Local and National Multi-site Evaluations
|
|
Panel Session 631 to be held in Room 107 in the Convention Center on Friday, Nov 7, 1:35 PM to 3:05 PM
|
|
Sponsored by the Cluster, Multi-site and Multi-level Evaluation TIG
|
| Chair(s): |
| Sandra Ortega,
National Data Evaluation Center,
ortegas@ndec.us
|
| Abstract:
The panel focuses on how lessons learned from multi-level evaluations can impact policy and practice. Panel members have worked on numerous national projects either as local evaluators, national evaluators, or local project directors. The panel will identify the main challenges that evaluators face during multi-level projects and propose solutions to overcome them. They will also review strategies that have not worked for them in the past and examine why they believe these were unsuccessful. The panel will discuss the unique challenges presented by multi-level and multi-site evaluation projects the importance of collaborative work between the multiple levels of stakeholders, how to make national data useful for local communities, how local evaluators can build an effective relationship with local community members, how national stakeholders can facilitate the work of local evaluators, and whether some evaluation practice models are more fitting than others for national demonstration projects.
|
|
The Relationship Between Project Directors and Evaluators: Evidence from a National Demonstration Project
|
| Rusti Berent,
University of Rochester,
rberent@childrensinstitute.net
|
| Bill Goddard,
Beta Social Research,
wegoddard2000@yahoo.com
|
| Antoine Beauchemin,
Kent State University,
abeauche@kent.edu
|
|
This paper presents the results of a study about the relationship between project directors and evaluators with the ultimate aim of improving collaboration and communication between the two. Data from evaluators and project director colleagues engaged in a large, national demonstration project were collected using an on-line survey adapted from The Readiness for Organizational Learning and Evaluation Instrument (Preskill and Torres 2000) and a follow-up telephone interview. The results highlight the tension that exists between project staff and evaluators. For example, evaluators were more likely than project directors to be optimistic about evaluation results having a positive impact on the program. Project directors were more likely than evaluators to minimize the influence of the evaluation on their own work. The panel will discuss these tensions and offer lessons learned from the respondents on maximizing collaboration between evaluation teams and stakeholders.
|
|
|
Policy and Practice Issues of Multi-site Evaluations: Lessons Learned From a Local Evaluator's Perspective
|
| Joy Kaufman,
Yale University,
joy.kaufman@yale.edu
|
| Cindy Crusto,
Yale University,
cindy.crusto@yale.edu
|
|
Dr. Kaufman will describe her experience of working as a local evaluator within national demonstration site and her experience as lead evaluators on multi-site evaluations. Her comments will focus on the structures, processes and technical assistance that is needed from the lead evaluation team and how local site evaluators can utilize the unique strengths of their site to gain buy-in for the evaluation.
Dr. Crusto will discuss the role of the local evaluation team in carrying out the evaluation plan of nation initiatives at the local level. The local evaluator is charged with managing and balancing the evaluation requirements of national initiatives with the capacity of the local community to implement the requirements. The local evaluator is in a unique position to facilitate reciprocal knowledge transfer between local communities and Federal initiatives and evaluation capacity building at the local and national levels.
| |
|
Policy and Practice Issues of Multi-Site Evaluations: Lessons Learned From a Local Project Director's Perspective
|
| Judith Simpson,
Techno-Communications Corporation,
tccorp@tampabay.rr.com
|
|
Ms. Simpson will describe her experiences and the lessons learned from working on multi-site projects from a local project director's perspective. Her insights will ground recommendations for aligning evaluation goals with local program goals and requirements. She will provide participants with a firm discussion on methodological concerns for cross-site designs that influence learning communities as well as strategies for gaining community approval.
| |
|
Policy and Practice Issues of Multi-site Evaluations: Lessons Learned From a National Evaluator's Perspective
|
| Brigitte Manteuffel,
Macro International Inc,
brigitte.a.manteuffel@macrointernational.com
|
|
Brigitte Manteuffel will contribute her experience with implementing the national protocol for evaluating the Comprehensive Community Mental Health Services for Children and Their Families program in over 100 communities that include all 50 States, Guam, Puerto Rico, and tribal entities. Implementing this national evaluation requires partnership and collaboration with communities. How concepts of participatory and empowerment evaluation can be applied to a national evaluation will be address.
| |