| Session Title: Three's Company: Results and Lessons Learned Through a Collaboration Among Funder, Grantee, and Evaluator to Establish Targets and Measure Child Progress and Parent Engagement |
| Multipaper Session 247 to be held in Sunset on Thursday, Nov 3, 8:00 AM to 9:30 AM |
| Sponsored by the Human Services Evaluation TIG and the Pre-K - 12 Educational Evaluation TIG |
| Chair(s): |
| Emily Moiduddin, Mathematica Policy Research, emoiduddin@mathematica-mpr.com |
| Discussant(s): |
| Emily Moiduddin, Mathematica Policy Research, emoiduddin@mathematica-mpr.com |
| Abstract: Through a collaborative process, a funder, a grantee and a team of evaluators developed a set of targets for child progress in multiple domains of school readiness and parent engagement. These performance targets are part of an accountability framework between the funder (First 5 Los Angeles-F5LA) and grantee (Los Angeles Universal Preschool-LAUP). LAUP maintains a network of over 300 preschools serving more than 10,000 4-year-olds throughout Los Angeles County. In the years before targets were set, F5LA commissioned Mathematica Policy Research (Mathematica) to conduct the Universal Preschool Child Outcomes Study (UPCOS). The session papers will describe how data from UPCOS were used to identify metrics and establish targets, how outcomes were measured, and how LAUP performed in the first year. Papers also will describe how the results are being used for program improvement and provide reflections on the collaborative dynamic. Throughout the presentation lessons learned will be highlighted. |
| Elements of the Collaborative Dynamic |
| Sharon Murphy, First 5 LA, smurphy@first5la.org |
| This paper will provide an overview of the collaborative process among the partners and identify those elements which proved essential for a successful collaboration. The grantee, funder, and evaluation team were tasked with establishing a set of targets for child progress and parent engagement. Given that mandate, the team worked together over a period of several months and utilized a data-driven, decision-making process that took advantage of previous years' data. The thoughtful, efficient design as well as the quality and characteristics of the partners contributed to the success of the collaborative effort. |
| Using Evaluation to Inform the Process of Setting and Meeting Shared Goals |
| Yange Xue, Mathematica Policy Research, yxue@mathematica-mpr.com |
| Emily Moiduddin, Mathematica Policy Research, emoiduddin@mathematica-mpr.com |
| Sally Atkins-Burnett, Mathematica Policy Research, satkins-burnett@mathematica-mpr.com |
| Elisha Smith, Mathematica Policy Research, esmith@mathematica-mpr.com |
| Cay Bradley, Mathematica Policy Research, cbradley@mathematica-mpr.com |
| Ama Atiedu, Los Angeles Universal Preschool, aatiedu@laup.net |
| As part of the Universal Preschool Child Outcomes Study, Mathematica has been working with First 5 LA and Los Angeles Universal Preschool to conduct outcomes evaluations in the areas of child progress (since 2007) and family engagement (since 2009). Data from these studies are being used for the purpose of setting targets in the context of the performance-based contract between the two organizations. In this paper we describe the data that were collected (direct child assessments, self-administered questionnaires to parents and providers), report how the findings for those studies were used to develop targets for child progress and engagement, and discuss how data are being used to determine whether the targets have been met (including key findings). This paper illustrates how data from descriptive evaluations can be used to both inform program improvement efforts and enable a funder and program to work together to determine whether shared goals are met. |
| Using Evaluation Findings and Lessons Learned for Program Improvement |
| Kimberly Hall, Los Angeles Universal Preschool, kimberly.m.hall@gmail.com |
| Schellee Rocher, Los Angeles Universal Preschool, srocher@laup.net |
| This presentation will describe how a collaborative process involving a funder, a grantee and a research entity led to policy and program changes in a network of over 350 pre-kindergarten programs in Los Angeles County. With representation from the three entities, the collaborative worked together to address study design issues, set performance targets, interpret results and identify programmatic implications. The purpose of this presentation is to highlight key programmatic changes that were made once targets were set and those that were initiated later based on the study findings. The presenter will also share lessons learned by all three entities throughout the collaborative process. |