|
Assessing the Effectiveness of Online Supplemental Education Services
|
| Presenter(s):
|
| Joel Shapiro,
Rockman Et Al,
joel@rockman.com
|
| Abstract:
This paper examines the effectiveness of online supplemental education services (SES) and factors that can encourage and inhibit its success. NCLB requires that underperforming districts provide SES, typically via after-school tutoring by external vendors. Face-to-face SES suffers from low participation rates, difficulty in providing instruction to students in remote geographic areas, and other problems. SES that is delivered through remote technologies has the potential to remedy some of these problems, due largely to students' ability to participate from home. This study uses a randomized design of approximately 300 middle school students in six schools in three states to isolate the effect of a large SES provider's math program on students' achievement, motivation, and attitudes towards school and learning. Finally, the study reports on the results of school site visits to identify those district, school and classroom teacher-level factors that can encourage and/or inhibit the success of online SES provision.
|
|
Considering Dosage in After-School Programs: Linking Activity Types to Outcomes
|
| Presenter(s):
|
| Laurie Van Egeren,
Michigan State University,
vanegere@msu.edu
|
| Celeste Sturdevant Reed,
Michigan State University,
csreed@msu.edu
|
| Heng-Chieh Wu,
Michigan State University,
wuhengch@msu.edu
|
| Nai-Kuan Yang,
Michigan State University,
yangnaik@msu.edu
|
| Abstract:
Despite advances in conceptualizations of participation in after-school programs, measures of dosage often remain global and difficult to link to specific outcomes. Using data from the state evaluation of the Michigan 21st Century Community Learning Centers, we demonstrate methods for: (a) examining links between students' dosage of specific types of activities and hypothesized outcomes; (b) characterizing dosage in multiple ways that have differential relationships with outcomes; and (c) controlling for differences in students' dosage that result from constraints in what programs offer and policies about activity choice. Multi-level analyses of 187 sites serving over 25,000 students will be used to demonstrate ways to distinguish between students' individual dosage (collected through a web-based tracking system) and site-level factors that affect dose-response rates in student-, parent-, and teacher-reported outcomes and student achievement.
|
|
Building Evaluation Capacity in 21st Century Community Learning Center After-school Programs
|
| Presenter(s):
|
| Tara Donahue,
Learning Point Associates,
tara.donahue@learningpt.org
|
| Abstract:
Quality evaluations are critical for after school programs to gauge the effectiveness of programs and to help programs engage in the process of continuous program improvement. This paper examines how a 21st Century Community Learning Center (21st CCLC) after school program in northern Michigan worked with its local evaluator through both utilization and empowerment evaluation frameworks to build evaluation capacity within the program. This paper offers lessons learned about how to effectively integrate multiple methodologies including quantitative analysis through surveys and outcome measures, qualitative analysis through interviews and document review, and self-assessment analysis through the use of a validated instrument in after school evaluations. This evaluation informed the program staff and helped them make decisions for continuous program improvement leading to positive outcomes. Additionally, the evaluation work done in this program helped the evaluator gain insight into how to apply these concepts, strategies, and tools to the larger field of after school evaluation.
|
|
Keeping Youth Engaged in Out-of-school Time Programs: Results From a Five-year Evaluation in Denver Public Schools.
|
| Presenter(s):
|
| Robin Leake,
JVA Consulting LLC,
robin@jvaconsulting.com
|
| Veronica Gardner,
JVA Consulting LLC,
v@veronicagardner.com
|
| Sheridan Green,
JVA Consulting LLC,
sheridan@jvaconsulting.com
|
| Abstract:
This is the fifth year of an evaluation of the Denver Public Schools out-of-school (OST) program to examine whether participation in OST improves students' academic achievement, school-day attendance and behavior. The purpose of this evaluation is to investigate the impact of attendance, intensity, duration and breadth of attendance in OST programming on student outcomes compared to a control group of students from the same school who did not participate in OST activities. Student achievement was measured using the state-mandated Colorado Student Assessment Program reading, writing and math test scores. Results across eleven elementary and middle school sites demonstrated that involvement in OST activities for three or more years improves students' school-day attendance and achievement scores in reading and math compared to the control group students. A formative evaluation component was added in Year-5 to identify programmatic characteristics associated with greater retention of students, particularly from elementary to middle school.
|
| | | |