|
Session Title: Serving Two Masters: Local Evaluators Trying to Maintain Evaluation Quality and Use While Participating in a National Multi-site Evaluation
|
|
Panel Session 207 to be held in Lone Star F on Thursday, Nov 11, 9:15 AM to 10:45 AM
|
|
Sponsored by the Cluster, Multi-site and Multi-level Evaluation TIG
|
| Chair(s): |
| Tom Kelly, Annie E Casey Foundation, tkelly@aecf.org
|
| Discussant(s):
|
| Mary Achatz, Westat, maryachatz@westat.com
|
| Abstract:
Multi-site initiatives are complex programs to implement and evaluating them can be an even more complex undertaking. Making Connections is a community change initiative (CCI) in 10 urban neighborhoods across the U.S. A key component of its national cross-site evaluation has been the implementation of local evaluations that are relevant to and integrated in the neighborhood work on the ground. These local evaluations have been responsible not only for collecting data on the implementation and outcomes of the initiative but also for the building of community capacity to understand and use data for learning and accountability, while also contributing to the cross-site evaluation. The local evaluators have had to navigate the multiple challenges, demands, and differential capacity local evaluation clients and the national funder and cross-site evaluators. This panel will identify lessons learned in strengthening local evaluation quality and relevance.
|
|
Two Bosses, Two Clients, Two Timetables: Local eValuators’ Challenges in Meeting Expectations of Community Partners and National Funder and Evaluators
|
| Sue Tripathi, Making Connections, Denver, sue.tripathi@unitedwaydenver.org
|
|
Throughout the implementation of Making Connections, there has been a tension between the needs and theories of the local community and organizational partners and the interests of the national funder. In addition, the local evaluation teams have had to deal with and address these tensions in order to develop working partnerships with community and national evaluators, build local evaluation capacity concurrent with the needs of both local and national evaluations, and participate in cross-site evaluation data collection and analyses. This has required much skill and art in local evaluators across the 10 cities. And the local relevance and utility and cross-site uniformity and comparability have been a constant balancing act. The experiences and lessons learned together by the local and national evaluators contribute much insight into establishing and maintaining evaluation quality, rigor, and relevancy.
|
|
|
Building Local Evaluation Capacity of Coalition Partners
|
| Sebastian Schreiner, City of San Antonio, sebastian.schreiner@sanantonio.gov
|
|
Creating and maintaining evaluation quality in coalition and community initiative settings is linked to its own specific set of necessary upfront considerations, preparatory work, and execution challenges. These challenges will be explored based on the evaluation capacity building efforts and experiences of Making Connections—both from the perspective of individual sites as well as the cross-site evaluation. This local evaluation capacity cannot be built only within a few individuals or organizations but across the community and coalition as a whole. Building self-evaluation, learning, and accountability capacity and infrastructure among and inside organizations, systems, and the community is a necessary task of the evaluation. But it faces challenges of data quality in multi-partner/multi-site settings, data access and confidentiality, continuously evolving local goals, differences in community partners in levels of sophistication and tolerance levels for data topics, and resolving differences in evaluation philosophy among partners.
| |
|
Defining and Measuring Community Engagement at Local and National Levels
|
| Tanja Kubas-Meyer, Independent Consultant, tkubasmeyer@cox.net
|
|
Community and family engagement within social service programs and community change initiatives is a critical issue, including community participation in evaluation. The definition and vision of engagement varies widely across a spectrum, however, from residents as program participants and/or informants to residents mobilized for political action. The Providence Making Connections initiative is working with multiple partners and at least three evaluative frames, including the national evaluation, to identify a shared vision and set of common or complementary metrics for family engagement in its work to have "children healthy and prepared to succeed in school." The presentation will focus on the learning and challenges in this work and how the evaluation dealt with different definitions and frames of resident and family engagement used by local and national evaluations.
| |