|
Session Title: Collaborative Development and Implementation of a Cross-site Evaluation of a National Systems and Policy Change Initiative Across Diverse Sites and Approaches
|
|
Panel Session 988 to be held in San Simeon A on Saturday, Nov 5, 2:20 PM to 3:50 PM
|
|
Sponsored by the Collaborative, Participatory & Empowerment Evaluation TIG
|
| Chair(s): |
| Laurie Carpenter, University of Michigan, lauriemc@umich.edu
|
| Discussant(s):
|
| Martha Quinn, University of Michigan, marthaq@umich,edu
|
| Abstract:
In 2007, the W.K. Kellogg Foundation funded nine collaboratives in diverse communities around the country, collaboratives now part of the Food & Community program, to develop action plans to work toward systems and policy change to increase access to healthy, locally grown food and safe and inviting options for physical activity for vulnerable children and their families. The evaluators and leaders from the nine collaboratives worked together with evaluators from the University of Michigan to develop a cross-site evaluation to measure the impact of the initiative on children and families living in each community, and also across all communities. The panelists will provide participants with an intimate look into the work undertaken by the evaluators to integrate community members and other stakeholders into the implementation of the evaluation, using the collaboratively developed cross-site evaluation tools, and to build local capacity toward the evaluation of systems and policy change efforts.
|
|
Using Evaluation Findings to Prioritize and Further Community-based Systems and Policy Change Work
|
| Rickie Brawer, Thomas Jefferson University and Hospital, rickie.brawer@jeffersonhospital.org
|
|
Evaluators working with the Philadelphia Urban Food & Fitness Alliance (PUFFA) took an active role in working with PUFFA leadership to move the work of the collaborative forward, in changing systems and policies in the food and active living environments in targeted neighborhoods in Philadelphia. This presentation will describe the various methods utilized by the evaluators in helping the leadership prioritize and move the planning process forward, including a number of diverse data collection methods and the presentation of evaluation findings to collaborative committees and the community.
|
|
|
An Overview of the Development of the Food and Fitness Cross-site Evaluation
|
| Laurie Carpenter, University of Michigan, lauriemc@umich.edu
|
| Laurie Lachance, University of Michigan, lauriel@umich.edu
|
| Martha Quinn, University of Michigan, marthaq@umich.edu
|
| Maggie Wilkin, University of Michigan, mwilkin@umich.edu
|
| Noreen Clark, University of Michigan, nmclark@umich.edu
|
|
The collaborative process used to develop the cross-site evaluation of the Food & Fitness initiative involved the evaluators and leaders from the nine collaboratives and evaluators from the Center for Managing Chronic Disease at the University of Michigan. During the planning phase of the initiative, evaluators, guided by an evaluation advisory group, decided upon the most important elements to consider in the evaluation, and worked to integrate those elements into relevant and useful tools to utilize across the nine sites. The main components of the cross-site evaluation include information related to collaborative partners, resources leveraged, and details regarding the major systems and policy change efforts of the collaboratives. Once the initial drafts of tools were developed and pilot tested by the local evaluation teams in their communities, additional refinements were made to arrive at the final set of cross-site tools, with which the evaluators collected data in 2009 and 2010.
| |
|
Cross-site Evaluation as a Way to Create Reflection Opportunities for and Build the Capacity and Sustainability of Initiatives
|
| Mary Emery, South Dakota State University, mary.emery@sdstate.edu
|
|
Evaluators working in a cross-site evaluation context are challenged to address the data collection requirements related to the cross-site data collection efforts and at the same time attend to the capacity building aspects inherent in formative or developmental evaluatory approaches. Success in broadening and deepening the Initiative work necessarily also means additional time and effort in data collection. At this stage in community initiative work, questions about the sustainability of the effort and the hoped for increase in capacity to sustain the effort into the future also require time and effort. This presentation will discuss our experience with successful and not so successful strategies to assist in the Northeast Iowa Food & Fitness Initiative to engage in reflection leading to improvements in project operation. The presentation will also identify key challenges encountered in tackling evaluation issues related to capacity building and sustainability within a cross-site context.
| |
|
Participatory Assessment and Evaluation with Community Youth: To Understand the Food and Active Living Environments in New York City
|
| Heewon Lee, Columbia University, hl2001@tc.columbia.edu
|
| Pam Koch, Columbia University, pak14@tc.columbia.edu
|
| Isobel Contento, Columbia University, irc6@tc.columbia.edu
|
|
Youth have been active members of the New York City Food & Fitness Partnership evaluation team, involved in both assessment and evaluation activities. For our assessment of foods available in retail outlets in Central Brooklyn, we utilized youth in the process of interviewing store managers and completing a modified version of the Nutrition Environment Measures Survey (NEMS). The youth learned how food availability and pricing varied in different neighborhoods, and the findings were presented to a city council member interested in improving food availability in the community. Another group of youth advocates assessed the activities available at block parties in their community, and these data helped the youth be able to make recommendations for how to make the closed streets into useful spaces for active recreation. Youth involvement expanded our ability to collect data for both assessment and evaluation and created a valuable learning experience for the youth involved.
| |
|
Using Cross-Site Evaluation Tools as an Entry Point for Participatory Evaluation
|
| Mia Luluquisen, Alameda County Public Health Department, mia.luluquisen@acgov.org
|
| Anaa Reese, Alameda County Public Health Department, anaa.reese@acgov.org
|
| Lauren Pettis, Alameda County Public Health Department, lauren.pettis@acgov.org
|
|
Oakland's HOPE (Health for Oakland's People and Environment) Collaborative used a participatory evaluation approach for piloting and utilizing the cross-site evaluation Collaborative Partners Form. During the initial 2-year planning grant, the HOPE Collaborative formed an evaluation team that included members of four Community Action Teams that worked closely with the project evaluators to develop the data collection process, data analysis and interpretation of results. The evaluation team conducted the data collection and came together to analyze the data and provide preliminary results. The team formulated a summary and interpretation of the data that they presented to the full HOPE Collaborative. The findings from the Collaborative Partners tool helped to 1) inform the implementation strategies of the HOPE Collaborative, 2) shape the final versions of the policy and systems change evaluation instruments and 3) model a participatory evaluation process for other W.K. Kellogg Food & Fitness sites.
| |
Integrating Cross-site and Site-specific Evaluations to Build Capacity Toward the Evaluation of Systems and Policy Change Efforts
| Chris Navin, Navin Associates, navin@navinassociates.com
|
| Marian Milbauer, Navin Associates, milbauer@navinassociates.com
|
|
The Evaluation Committee of the Boston Collaborative for Food & Fitness (BCFF) draws its members from each of the other content area subcommittees, and is overseen by a team of evaluators working with the collaborative as independent consultants. The Evaluation Committee is tasked with overseeing the implementation of both the cross-site and site-specific evaluations, and as the initiative has moved from planning to implementation, the role of the committee has evolved. This presentation will describe the nature of BCFF and the role of its Evaluation Committee, design processes for the evaluation of the work of the collaborative, resource constraints under which the committee must operate to implement its responsibilities, and the successes of and challenges to implementation of the evaluation.
|
Using Cross-site Evaluation Tools With Collaborative Leadership and Community as a Means of Furthering the Work of the Collaborative
| Catherine Sands, Partnership in Practice, chsands@fertilegroundschools.org
|
|
The participatory methods of using cross-site evaluation tools with collaborative leadership and community as a means of furthering the work of the Holyoke Food & Fitness Policy Council. Through the cross-site and process evaluation, we have created an evaluation learning environment, in which an intergenerational, racially diverse collaborative membership share their different perspectives in dialogue and think together as a group. Holyoke's cross-site implementation is enhanced through multiple, participatory, flexible approaches, designed to address real issues. The cross-site evaluation is used:
- To focus the development of leadership, community knowledge and evaluation skills
- To provide guidance in developing strategy and readiness to bring to bear when windows of opportunity arise for systems and policy change opportunities.
|
| |