|
Session Title: Multiple Sites, Multiple Layers, Multiple Players: Lessons From the Field on Keeping Quality High and Frustration Low
|
|
Panel Session 665 to be held in Lone Star D on Friday, Nov 12, 4:30 PM to 6:00 PM
|
|
Sponsored by the Non-profit and Foundations Evaluation TIG
|
| Chair(s): |
| Jacqueline WIlliams Kaye, Atlantic Philanthropies, j.williamskaye@atlanticphilanthropies.org
|
| Abstract:
Multi-site initiative evaluations face multiple layers of complexity rooted in factors such as variation in local cultures and contexts, sheer numbers of entities involved, the initiative design itself, as well as other factors. This session’s panel of experts will discuss strategies to address issues such as creating appropriate communication and coordination mechanisms across stakeholders; “right-sizing” the evaluation relative to the scope of the initiative and capacity of the players; balancing the desire for comparable, cross-site indicators with flexibility for local customization; when and how to make technical assistance available to local sites; and maintaining quality and integrity in the data collection, analysis and reporting processes. The perspectives of evaluation staff from two large funding organizations as well as evaluation professionals representing both national and local evaluator roles will provide hard-won insights into strategies for ensuring high quality multi-site evaluations that also respect the interests and constraints of stakeholders involved.
|
|
Leveraging the Opportunity in Having Multiple Evaluation Teams
|
| Jacqueline Williams Kaye, Atlantic Philanthropies, j.williamskaye@atlanticphilanthropies.org
|
| Steven LaFrance, Learning for Action Group, steven@lfagroup.com
|
| Stephen Baker, Learning for Action Group, steven@lfagroup.com
|
| Malu Gonzalez, University of Texas, El Paso, mlgonzalez6@utep.edu
|
|
The ability to connect and convene is an important supplement to grant dollars. Providing opportunities for grantees to network often enhances a funder’s initiative. It’s less common for funders to foster relationships among organizations providing support services or technical assistance to grantees. Using experience from a four-site initiative with local evaluation teams this presentation explores how relationships among the evaluators can be established and maintained. Then we’ll discuss how evaluation peers – in the same role but in different places – can enhance the quality of their individual efforts as well as the quality of the overall initiative evaluation. How can individual evaluation team strengths be used to raise all boats? When is having a group of peers to provide input useful and when is it “too many cooks”? What happens when evaluation process works better in some sites than others? These and other questions will be explored among the panelists, and with the audience.
|
|
|
Aligning Interests, Needs and Methods: Managing Cross-site Evaluation
|
| Meridith Polin, Public/Private Ventures, mpolin@ppv.org
|
|
Integrated service initiatives are highly complex; they include multiple service providers who have their ‘own’ missions and data systems, but also must adopt the ‘shared’ mission of the initiative including integrated data collection and management practices. Unique opportunities and challenges abound in the design and implementation of an evaluation in this integrated framework. This presentation describes how Public/Private Ventures, one of Elev8’s evaluation partners, developed and manages ongoing data collection for the Initiative from two perspectives: the site level and across four sites nationally. The presentation will discuss (1) the need for consistency as well as flexibility within a data collection system, (2) the process for ongoing feedback and technical assistance, (3) practices for establishing and fostering collaboration among the partners, and (4) features of the system that ensure its long-term sustainability and use among service providers.
| |
|
Evaluator Capacity and Evaluation Quality in a Multi-site National Initiative
|
| Scott Hebert, Sustained Impact, shebert@sustainedimpact.com
|
| Thomas Kelly, Annie E Casey Foundation, tkelly@aecf.org
|
|
Making Connections is a ten-year community change initiative in ten urban neighborhoods across the US. Its cross-site evaluation has relied on the ability of local evaluators to implement effective evaluations of process and outcomes and the funder invested time and resources in building local site capacity to participate in and use evaluations. Even with adequate resources and a clear demand for an evaluation, maintaining communication and coordination across ten local teams with the national cross-site team was a constant struggle throughout the decade. Changes in staffing and local circumstances and limited time and attention of busy implementers limited the ability to achieve and maintain a shared national framework for evaluation, including minimum standards and expectations for evaluation comprehensiveness and use at local and national levels. This presentation will summarize key strategies for addressing evaluation capacity and quality with multiple evaluation partners and clients at local and cross-site levels.
| |