|
Session Title: Evaluating Literacy Curricula for Adolescents: Results From Three Years of Striving Readers
|
|
Panel Session 526 to be held in BONHAM D on Friday, Nov 12, 9:15 AM to 10:45 AM
|
|
Sponsored by the Pre-K - 12 Educational Evaluation TIG
|
| Chair(s): |
| Stefanie Schmidt, United States Department of Education, stefanie.schmidt@ed.gov
|
| Discussant(s):
|
| David Francis, University of Houston, david.francis@times.uh.edu
|
| Abstract:
This panel will discuss the findings from three years of Striving Readers program evaluations from 5 sites across the country. Panel members will report findings based on experimental research designs that provide the most rigorous evaluations to date on a number of adolescent reading interventions. These independent evaluations provide a wealth of detailed information to policymakers and school administrators on the important, but under-researched area of adolescent literacy education.
The papers being presented add significantly to our understanding of middle school literacy education and the potential for several intervention strategies to be effective. They also provide insights into the challenges of maintaining a high-quality experimental research design in the field. Despite substantial obstacles to conducting rigorous experiments in school settings, evaluators have been able to negotiate compromises that do not diminish the quality of their evaluation designs. The results advance our understanding of both adolescent literacy and practical research methodology.
|
|
Striving Readers: Results From Ohio
|
| William Loadman, Ohio State University, loadman.1@osu.edu
|
|
One of the eight Striving Reader awards was made to the Ohio Department of Youth Services (DYS) to investigate enhancing the reading ability of incarcerated youth in the state of Ohio. DYS operates high schools in 7 correctional facilities serving approximately 1300 youth, with an average stay of 10.5 months. Approximately 50% of the youths are reading significantly below grade level at intake.
Using a randomized controlled trials design, the evaluation has found that students in Read 180 for four consecutive terms have gained approximately 75 lexile points more than a randomly equivalent comparison group on the Scholastic Reading Inventory (significantly beyond the rate of growth historically achieved by these youths). The analysis consisted of using the CAT as a covariate and then using an HLM longitudinal analysis to examine the growth over the nine data points.
|
|
|
Striving Readers: Results From Newark
|
| Jennifer Hamilton, Westat, jenniferhamilton@westat.com
|
| Matthew Carr, Westat, matthewcarr@westat.com
|
|
Newark’s Striving Readers program aims to improve the reading skills of struggling students by implementing Scholastic’s Read 180 curriculum, which utilizes adaptive and instructional software, high-interest literature, and direct instruction. The evaluation used a cluster randomized design that assigned 19 middle schools to either the READ 180 curriculum or to continue using the district’s standard literacy curriculum. Using student-level data, a 2-level HLM regression analysis was employed to examine if exposure to READ 180 had an impact on students’ vocabulary, comprehension, and language arts scores on the SAT10. Students were divided into 6 analytic groups based on their grade level and whether they had received one, two, and three years of treatment. Preliminary analyses show significant effects in all 3 literacy subtests after a minimum of two years of treatment. Significant effects were also found for certain subgroups of students, including special education students, males, and African-American students.
| |
|
Striving Readers: Results From the Mid-South
|
| Debra Coffey, Research for Better Schools, coffey@rbs.org
|
|
In 2005 the Mid-South City Schools system, which serves about 120,000 students, was awarded one of eight Striving Readers grants. Eight MCS middle schools were selected for participation in this Striving Readers project, and all students who scored in the bottom quartile on the state reading assessment were selected to participate and randomly assigned to the control or experimental group. All students who participated received regular English/language arts instruction, and students in the experimental (treatment) group also received READ 180.
The purpose of this presentation will be to describe Year 3 impacts of READ 180 on struggling readers’ reading achievement, as measured by the Iowa Tests of Basic Skills, and on their achievement in four core content areas, as measured by the state NCLB-related assessment. In addition to describing the experimental impacts, presenters will describe the results of treatment-on-the-treated (TOT) analyses completed using Bloom’s adjustment.
| |
|
Striving Readers: Results From Portland
|
| Bonnie Faddis, RMC Research Corporation, bfaddis@rmccorp.com
|
| Margaret Beam, RMC Research Corporation, mbeam@rmccorp.com
|
|
The Oregon Striving Readers grant serves 4 high schools and 6 middle schools using the Content Literacy Continuum developed by the University of Kansas. The targeted intervention is the Xtreme Reading curriculum and the whole school intervention uses Content Enhancement Routines designed to help all students understand key content.
Students were randomly assigned to treatment and control conditions within each school and grade. This presentation focuses on the impact of the targeted and whole school interventions over the past 3 years. A multilevel model was used to estimate the impact of the targeted intervention on spring GRADE scores, revealing significant effects for middle school but not high school students. The presenters will discuss implementation and its relationship to student outcomes. An Interrupted Time Series analysis was used to evaluate the effect of the whole school intervention, beginning 3 years prior to the Striving Readers grant through Year 3.
| |
|
Results from Springfield/Chicopee
|
| Kimberly Sprague, Brown University, kimberly_sprague@brown.edu
|
|
The Education Alliance at Brown University is conducting a randomized control trial (RCT) evaluating the effectiveness of two adolescent literacy interventions on the reading achievement of low performing students, as implemented by two school districts in western Massachusetts. Both teachers and students in five participating high schools were randomly assigned to one of three conditions: Read180, Xtreme, or the control condition. Year 3 results indicated positive effects on reading achievement for one of the two targeted interventions as compared to the control group. Despite challenges in the specification for implementation and monitoring, patterns emerged in the treatment-only or Treatment of the Treated (TOT) group and implementation levels. Results are presented in the context of the implementation study results. The final five-year study results will provide administrators and educators with the information necessary to make informed choices about which program to select and how best to implement it within their schools.
| |