| Session Title: The Communities in Schools National Evaluation: Using a Comprehensive Evaluation Strategy to Understand the Value-Added of Integrated Student Supports |
| Multipaper Session 549 to be held in Sebastian Section I3 on Friday, Nov 13, 1:40 PM to 3:10 PM |
| Sponsored by the Cluster, Multi-site and Multi-level Evaluation TIG |
| Chair(s): |
| Yvette Lamb, ICF International, ylamb@icfi.com |
| Discussant(s): |
| Allan Porowski, ICF International, aporowski@icfi.com |
| Abstract: The Communities In Schools National Evaluation is a multi-level, multi-method study that was designed to identify the most successful strategies for preventing students from dropping out of school. This five-year study includes secondary data analyses, a quasi-experimental study, eight case studies, a "natural variation" study, an external comparison study, and three randomized controlled trials. At the conclusion of the evaluation, all of the findings will be compiled so that the overall impact of the CIS model of integrated student services can be analyzed and replicated. In this presentation, we present both methods and findings from the first four years of the evaluation, and demonstrate how a multiple components from comprehensive evaluation design can be brought together to inform both policy and practice. |
| Communities In Schools National Evaluation: Year 2 Results for Student-Level Randomized Controlled Trials |
| Christine Leicht, ICF International, cleicht@icfi.com |
| Felix Fernandez, ICF International, ffernandez@icfi.com |
| Heather Clawson, ICF International, hclawson@icfi.com |
| Susan Siegel, Communities In Schools, siegels@cisnet.org |
| Communities In Schools, Inc. (CIS) is a nationwide initiative to connect community resources with schools to help at-risk students successfully learn, stay in school, and prepare for life. CIS is currently in the midst of a comprehensive, rigorous five-year national evaluation, culminating in a multi-site randomized controlled trial (RCT) to ascertain program effectiveness. In this presentation, we will draw from our experience working with Austin, TX, Jacksonville, FL, and Wichita, KS public schools, present our overall study design, and the process involved in conducting a student-level RCT. Preliminary results from Year Two will also be discussed. |
| Communities In Schools: Elementary, Middle, and High School Models and Their Implications for Integrated Student Supports |
| Kelle Basta, ICF International, kbasta@icfi.com |
| Sarah Decker, ICF International, sdecker@icfi.com |
| Jessica DeFeyter, ICF International, jjohnson@icfi.com |
| Dan Linton, Communities In Schools, lintond@cisnet.org |
| Communities In Schools (CIS) is a nationwide initiative to provide community-based integrated student supports, interventions that improve achievement by connecting community resources with the academic and social needs of students. As part of a comprehensive national evaluation, we completed a quasi-experimental study that compared 602 CIS schools with 602 matched comparison schools on a number of outcomes, including academic achievement, attendance, dropout, and graduation rates. Through propensity score matching, we were able to obtain a precisely-matched comparison group and achieve a highly rigorous study design. This presentation will report findings from the evaluation that specifically address the effect of integrated student supports on various school levels. It will include an assessment of how CIS models differ at the elementary, middle, and high school levels, and whether these disparities in service delivery contribute to differential outcomes for students. |
| The Communities In Schools Natural Variation Study: Providing Context for Performance |
| Jing Sun, ICF International, jsun@icfi.com |
| Julie Gdula, ICF International, jgdula@icfi.com |
| Aikaterini Passa, ICF International, apassa@icfi.com |
| Dan Linton, Communities In Schools, lintond@cisnet.org |
| Communities In Schools, Inc. (CIS) is a nationwide initiative to connect community resources with schools to help young people successfully learn, stay in school, and prepare for life. To measure the services delivered and coordinated by CIS and the variation in local program operations, ICF International, the National Evaluation Team, has developed a comprehensive, multi-level, multi-phased evaluation model. One evaluation component, the Natural Variation Study, examines the degree to which program models differ between high performing sites and lower performing sites (as defined by outcomes such as dropout and graduation). The Natural Variation analyses allow the evaluation team to exploit the natural variation between sites to determine specific contexts that are particularly conducive to the achievement of positive outcomes. Findings indicate that the degree and intensity of CIS services varies across different subgroups and outcomes. |
| The CIS External Comparison Study: Organizational Benchmarking to Improve Operations |
| Yvette Lamb, ICF International, ylamb@icfi.com |
| Susan Siegel, Communities In Schools, siegels@cisnet.org |
| In July 2008, a two-year study of five national youth-serving federations was completed as part of the CIS National Evaluation. The goal of the study was to improve and strengthen the CIS federation, as well as use the information gathered to contribute to the field of youth development. The organizations interviewed for this study include America's Promise Alliance, Boys & Girls Clubs of America, The Children's Aid Society, City Year, and YouthBuild USA. Five elements considered to be integral to successful implementation of a high-impact federation were included: (1) branding, (2) public policy, (3) quality management of service delivery, (4) innovation, and (5) brokering of services. The information resulting from this study provides insight into the complexity of federation organizational operations and practical advice regarding activities that can be utilized to create impact in the five areas explored in this study. |