|
Session Title: More Bang for the Buck: Getting the Most From a Statewide Evaluation of Programs for At-Risk Students
|
|
Panel Session 407 to be held in Mineral Hall Section D on Thursday, Nov 6, 4:30 PM to 6:00 PM
|
|
Sponsored by the Cluster, Multi-site and Multi-level Evaluation TIG
|
| Chair(s): |
| Ken Seeley,
National Center for School Engagement,
kseeley@coloradofoundation.org
|
| Abstract:
Increasingly educational researchers are demanding comprehensive and often costly evaluation designs. How can evaluators do justice to multi-program initiatives with limited resources? Panelists will talk about the evaluation of a Colorado categorical program designed to educate expelled and at risk students. Each year more than 50 school districts, charter schools, private and parochial schools are funded to provide academic and support services for expelled and at risk students. Recently state lawmakers legislated a 1% allocation of the total monies to evaluation. Through online data collection, selective in-depth inquiry, and analysis of data collected at the state level, the evaluation of this program has provided enough evidence to not only continue but also increase the dollars allocated by policy makers. Come to this panel to learn how to satisfy policy makers with enough data despite limited resources.
|
|
More Mileage from One Tank: Limiting Program Staff Burden but Reporting 'Enough' to Key Decision Makers
|
| Cindy Wakefield,
Colorado Department of Education,
wakefield_c@cde.state.co.us
|
|
Cindy Wakefield, from the Colorado Department of Education will provide a context for this panel including the history of the Expelled and At Risk Student Services program (EARSS), which sprang from an amendment to the Colorado Constitution. Cindy will also explain how the evaluation has been funded since early 2000 and the recent legislation, which allocates 1% of the program category to evaluation. Most importantly, the evolution of the evaluation will be discussed. Key aspects of this presentation will include the process of moving from a paper based to online system, how we determined key data elements for collection, challenges of getting accurate data and attempts to limit the burden on school staff. Finally, Cindy will give details about how the evaluation results have been reported to the state legislature and joint budget committee as well as how these legislative bodies have used these data.
|
|
|
Taking a Bird's Eye View: Using State Student Identifiers in Combination with Standardized Test Scores to Evaluate Legislated Programs
|
| Heather MacGillivary,
University of Colorado Boulder,
heather.macgillivary@colorado.edu
|
|
Legislators and other policy makers want to know how programs impact student success, however limited evaluation resources often prohibit in-depth academic assessments or long-term follow-up. To address this predicament, evaluators of the Expelled and At Risk Student Services (EARSS) program, took advantage of statewide data systems to assess student outcomes. The EARSS Leadership Team selected five programs to participate in this pilot longitudinal study. Student state ID's and corresponding educational information were obtained for each student who participated in five programs. Program staff also provided student participation and program exit data. These data were combined with data from the student state identification system and the student assessment data for multiple years proceeding and following program enrollment. Heather MacGillivary the principal evaluator will discuss the challenges of conducting this research, including issues of confidentiality, data access, analysis and interpretation of the results.
| |
|
Adding Depth to a Statewide Evaluation: A Qualitative Study of Family School Partnerships for At Risk Students
|
| Judy Martinez,
Colorado Department of Education,
martinez_j@cde.state.co.us
|
|
Large statewide evaluations can typically only provide an overview of aggregate results, however this is often not enough to make local programmatic improvements. This was the challenge facing the evaluators of the Colorado's Expelled and At-Risk Student Services initiative. In particular, we knew from program feedback and previous evaluation results that involving families of at risk students was challenging. A qualitative study of family involvement was undertaken to uncover promising practices. Over thirty telephone interviews were conducted with program staff. Six parent focus groups were completed and multiple parent involvement activities were observed in communities across Colorado. Judy Martinez from the Colorado Department of Education will discuss the findings as well as how these findings are being used to improve practice at the local level. Judy will also discuss other strategies for adding depth such as collecting success stories, funding a local evaluator, and investigating clusters of similar programs.
| |