|
Aligning Evaluation Tools in Curriculum and Non-curriculum-based After School Programs
|
| Presenter(s):
|
| Kelly Murphy, Claremont Graduate University, kelly.murphy@cgu.edu
|
| Tiffany Berry, Claremont Graduate University, tiffany.berry@cgu.edu
|
| Krista Collins, Claremont Graduate University, krista.collins@cgu.edu
|
| Abstract:
Durlak and Weissberg (2007) suggest that afterschool programs positively promote youth development if activities align with SAFE program features (i.e., Sequenced, Active, Focused, and Explicit). However, SAFE features were established within curriculum-based afterschool programs that taught a particular skill (e.g., self-confidence, etc.). It is unclear whether SAFE features predict positive outcomes in non-curriculum based afterschool enrichment programs. Thus, in this session, we discuss (1) the definition and how to measure each of the SAFE features, (2) if SAFE features are valid in non-curriculum based, enrichment type afterschool programs, (3) the consistency in reporting methods of SAFE features, and (4) if SAFE features should be incorporated into evaluation frameworks of afterschool programs. Implications of this paper apply to educational evaluators who grapple with measuring setting-level variables within both curriculum-based and non-curriculum school-linked programs. Our discussion will include data from an on-going external evaluation of After-School All-Stars, Los Angeles.
|
|
Implementing Bullying Prevention
|
| Presenter(s):
|
| Benjamin Cohen, Center for Schools and Communities, bcohen@csc.csiu.org
|
| Heather Cecil, Center for Schools and Communities, hcecil@csc.csiu.org
|
| Abstract:
We evaluate a large scale implementation of the Olweus Bullying Prevention Program [OBPP] (Olweus, 1994) in over 40 schools in Pennsylvania to assess how change in bullying climate is related to program implementation quality. To evaluate the program, we linked over 1,100 teacher records to student (N=2,018) bullying victimization data. Teachers’ reports of classroom level program activities, and their perceptions and attitudes about the program serve as independent variables. We use ordinal regression analysis to relate different levels of teacher attitudes and activities to the probability of different bullying outcomes. Analyses revealed that certain program activities are associated with higher levels of teacher interventions in bullying incidents. We also describe a method for evaluating the quality of a school violence prevention program using a large scale evaluation system.
|
|
The Playworks Evaluation: A Randomized Study of a School-based Intervention to Promote Physical Activity and Play
|
| Presenter(s):
|
| Daniel Finkelstein, Mathematica Policy Research, dfinkelstein@mathematica-mpr.com
|
| Susanne James-Burdumy, Mathematica Policy Research, sjames-burdumy@mathematica-mpr.com
|
| Martha Bleeker, Mathematica Policy Research, mbleeker@mathematica-mpr.com
|
| Abstract:
Playworks is a physical activity program for elementary schools that places full-time coaches in low-income schools to organize activities during recess and other periods throughout the school day. These activities are designed to engage students in physical activity, foster social skills related to cooperation and conflict resolution, and reduce behavioral problems during recess that spillover into the classroom and interrupt instruction. Mathematica Policy Research is conducting a randomized study of the Playworks program at 20 schools in five cities to assess the program’s impact on student outcomes over the course of the 2010-2011 academic year. The evaluation will assess outcomes through teacher and student surveys, administrative records, and direct observation of students during recess periods. This presentation will provide an overview of the evaluation design and discuss the methodological challenges associated with measuring the impact of this school-based intervention.
|
|
Quality Assessment of Woodcraft Rangers After-School Program: Challenges and Solutions
|
| Presenter(s):
|
| Lisa Garbrecht, EVALCORP, lgarbrecht@evalcorp.com
|
| Cathie Mostovoy, Woodcraft Rangers, cmostovoy@woodcraftrangers.org
|
| Julie Slay, EVALCORP, jslay@evalcorp.com
|
| Kristen Donovan, EVALCORP, kdonovan@evalcorp.com
|
| Mona Desai, EVALCORP, mdesai@evalcorp.com
|
| Abstract:
Research shows an association between participation in high quality after-school programs and positive developmental outcomes for youth. However, few research studies or evaluations examine which components of program quality specifically contribute to positive outcomes. As part of a comprehensive quality assessment, evaluators measured a range of variables within and across 62 after-school program sites to determine the extent to which program quality was associated with student outcomes. A “quality model” was designed in collaboration with program staff, and factor analysis was utilized to identify categories of quality. Due to limitations related to the use of factor analysis with small samples, several other methodological approaches were used to create a unique approach to assessing quality. Setting benchmarks and testing “model” sites provided insight into how to evaluate quality and its relationship to youth outcomes. Recommendations for measurement designs and linking program implementation quality to student outcomes will be discussed.
|
| | | |