|
Evaluating Impact: Use of Yin's Partial Comparisons Case Study Approach
|
| Presenter(s):
|
| Michael Trevisan, Washington State University, trevisan@wsu.edu
|
| Jennifer LeBeau, Washington State University, jlebeau@wsu.edu
|
| Abstract:
Since the passing of the No Child Left Behind Act in 2002, randomized controlled trials (RCTs) have maintained a prominent place in the debate over valid assessment of program impact. While proponents of RCTs argue that this methodology is the most valid means of assessing impact of educational programs, the idea that RCTs can be applied to all projects and programs seems unsatisfying and inadequate. One strategy proposed to assess impact of programs in which RCTs may not be useful is a case study approach that employs multiple partial comparisons rather than one overall design (Yin, 1995). The purpose of this paper is to describe the technique, to illustrate its use in the assessment of actual projects, and to discuss its strengths and limitations in light of the national push for documentation of impact and the determination of what works in social policy and programs.
|
|
The Evaluation of A Multilevel Analysis of Teacher-Student Racial and Ethnic Congruence on Student Mathematics Learning in the Context of a Randomized, Controlled Experiment
|
| Presenter(s):
|
| Antionette Stroter, University of Iowa, a-stroter@uiowa.edu
|
| Abstract:
There has been increased attention given to the underlying concerns of educational inequalities related to matters of race (Jost, Whitefield, & Jost, 2005). We evaluated a corpus, using Hierarchical Linear Modeling (HLM), centered on student learning gains gathered in the context of a successful focused, randomized, controlled experiment of middle school mathematics learning that contrasted a 3-week curricular unit using SimCalc MathworldsG™ curriculum and a TexTeams control. Performance data from the 92 7th grade mathematics teachers in several regions of Texas and 1342 of their students were examined to investigate the effects of (1) teacher-student racial/ethnic congruence for this Hispanic and White sample, (2) Hispanic-Hispanic versus White-White racial/ethnic congruence, and (3) teacher race above and beyond student ethnicity (matched or unmatched). The study revealed evidence suggestive that embedded complex differences in racial and ethnic groups have serious implications for comparisons and generalizability between and within these minority groups.
|
|
Evaluating Intervention Effect of a Reading Program for Low-Achieving Incarcerated Youth With Multi-level Growth Modeling
|
| Presenter(s):
|
| Jing Zhu, The Ohio State University, zhujingosu@gmail.com
|
| William Loadman, The Ohio State University, loadman.1@osu.edu
|
| Richard Lomax, The Ohio State University, lomax.24@osu.edu
|
| Raeal Moore, The Ohio State University, moore.1219@osu.edu
|
| Abstract:
Although there has been a substantial increase in rigorous evaluations of curricula, literacy instruction for adolescent struggling readers is one neglected area of investigation. This longitudinal study evaluated if the Scholastic READ 180 program had a meaningful impact on reading proficiency of low-achieving incarcerated youth in a large mid-western state, when salient covariates were controlled for their influences. The study was based on an experimental design in which eligible youth were randomly assigned to either the READ 180 program or a comparison group being instructed by traditional English class. The course of investigation lasted for two school years (2006-2008) during which the subjects were measured by the Scholastic Reading Inventory before treatment and at the end of each of the eight terms. Multilevel growth modeling was applied to the longitudinal assessment data for program evaluation. Results indicated that subjects receiving READ 180 demonstrated more rapid reading growth over time.
|
| | |