| Session Title: Evaluation Strategies Designed to Measure Program Context: Implications for Promoting Retention in After School Programs |
| Multipaper Session 853 to be held in Suwannee 14 on Saturday, Nov 14, 1:40 PM to 3:10 PM |
| Sponsored by the Pre-K - 12 Educational Evaluation TIG |
| Chair(s): |
| Tiffany Berry, Claremont Graduate University, tiffany.berry@cgu.edu |
| Discussant(s): |
| Sae Lee, Harder + Company, slee@harderco.com |
| Abstract: Retaining youth in after school programs is a challenge, especially for programs serving middle school students. Recent national evaluations of afterschool programs suggest that students participate less than one day per week in middle school (Kane, 2004). For students to achieve social and academic benefits from after school programs, students must demonstrate sustained participation. The purpose of this session is to discuss how evaluation could be used to promote and sustain youth participation in after school programs. Specifically, we will discuss how to (1) measure the way the program context interacts with students' characteristics, (2) identify and target students at-risk for dropping out of programs before they leave; and (3) measure more sensitively after school attendance through tracking activity level attendance as well as global attendance levels. Our discussion will include evaluation data from an on-going external evaluation of After-School All-Stars, Los Angeles. |
| Comparing Measures of Dosage as a Method for Understanding Program Context |
| Tiffany Berry, Claremont Graduate University, tiffany.berry@cgu.edu |
| Kelly Murphy, Claremont Graduate University, kelly.murphy@cgu.edu |
| This paper will focus on measuring student dosage afterschool, often considered one of the most important measures of program context. We will compare global indicators for attendance (number of days attended overall) to activity level attendance (the number of days attended by type of activity, such as enrichment or academic) to determine whether different measures of students' attendance differentially predicts students' social and academic outcomes. It is possible that participation in similarly themed enrichment programs (e.g., sports, arts, academics, dance, etc.) might relate more closely to students social and academic outcomes than global indicators of program attendance. If so, then evaluators should consider incorporating activity level attendance, rather than just daily counts of program participation, into evaluation practices. We will also couch our analyses within traditional measures of dosage, such as duration, intensity, and breadth, similar to the classifications discussed by Chaput, Little, and Weiss (2004). |
| Promoting Retention by Matching Activity Type to Students' Individual Characteristics |
| Krista Collins, Claremont Graduate University, krista.collins@cgu.edu |
| Research has suggested that high attrition in afterschool programs may stem from the programs' inability to captivate students enough for them to return (Lauver, Little, & Weiss, 2004). Thus, it is important to examine whether increased retention results when program activities match students' personality characteristics (such as introversion or extroversion). Using the concept of "flow" (Csikszentmihályi,1990) as a measure of engagement, we will examine the relationship between personality trait and activity type, as well as how these factors influence attendance rates within the After-School All-Stars program. Although this type of evaluation technique is not feasible or desired in all evaluation contexts, it provides a good example of how to measure youth participation more intensively than just through retrospective self-reports. |
| Self-Joined Versus Other-Joined Students: Identifying Students At-Risk for Dropping Out of After School Programs |
| Katherine Byrd, Claremont Graduate University, katherine.byrd@cgu.edu |
| Tiffany Berry, Claremont Graduate University, tiffany.berry@cgu.eu |
| As practitioners struggle to recruit and retain youth, evaluators can assist by identifying students who may be at-risk for dropping out of programs. This paper will describe an evaluation technique useful for identifying students who may be particularly at-risk for dropping out of the program. In a recent evaluation of After-School All-Stars, Los Angeles, we measured whether students joined because they wanted to (self-joined) or if they joined because of their parents, teachers or friends (other-joined). This paper will describe social outcomes (e.g., self-efficacy, feelings of autonomy, prosocial behavior) as well as the attendance patterns of these two groups of students. Based on evaluation results in 07/08, self-joiners reported higher levels of program satisfaction, self-efficacy, and prosocial behavior than other-joiners (Berry et al., 2009). Incorporating students' motivations for initially participating may help programs identify students who may drop out prior to realizing the benefits of their participation. |