|
Comparison Schools: How Many, If Any, Are Enough?
|
| Presenter(s):
|
| Rita O'Sullivan, University of North Carolina, ritao@email.unc.edu
|
| Abstract:
Currently much emphasis is placed on the use of experimental or quasi-experimental designs in educational evaluation and social science research, which require the use of equivalent control or comparison groups. When the school is the unit of analysis, typically researchers 'match' schools based on a variety of criteria and then randomly assign one of the matched pairs into the treatment and the other becomes part of the control groups. The effectiveness of this procedure has been questioned in the past (O'Sullivan et al., 2003; O'Sullivan et al., 2005; O'Sullivan et al., 2008). The purpose of this study is to further expand the discussion around the usefulness of different matching procedures to form equivalent groups, using data from the North Carolina Schools Report Card reports from 2001-06, Correlational analyses and group mean differences are used to address the research question with multiple school districts.
|
|
Evaluating Outcomes and Implementation of a Summer Learning Program: Challenges of Using a Nonequivalent Comparison Group Design and Disaggregating Data by Student Subgroups
|
| Presenter(s):
|
| Helen Wang, Montgomery County Public Schools, helen_wang@mcpsmd.org
|
| Abstract:
This paper explores the effectiveness of a Title I summer program in alleviating summer academic loss and promoting continued learning at a large school district.
It discusses the methodological advance and strategy to use a nonrandomized comparison group pre- and posttest quasi-experimental design within a school-based implementation setting. The focus of the outcome evaluation is to examine whether academic benefits from the program vary by student subgroups with respect to demographic and intervening variables. Program-relevant and timely assessments are used for the outcome measures. The analysis of covariance (ANCOVA) procedure is employed and both statistical significance and effect size are reported for the observed between-group difference in student performance. This paper also shares stakeholders' perceptions of the summer program and fidelity of curriculum implementation with data collected through online surveys and structured classroom observations.
|
|
Context and Quasi-Experiments in Schools: The Case for Cohort Comparison Groups
|
| Presenter(s):
|
| Tamara M Walser, University of North Carolina at Wilmington, walsert@uncw.edu
|
| Abstract:
The current No Child Left Behind climate favoring experimental and quasi-experimental studies has left educational evaluators with the task of implementing quality comparison group studies in school settings where the appropriateness of such designs is often in question. The purpose of this presentation is to describe the use of a historical cohort comparison group as a viable option; that is, the selection of a cohort comparison group from archival data matched to a group of students currently receiving an educational intervention. The presentation will include a brief description of the current federal context of educational evaluation, as well as the context of school-based evaluations and related questions of the appropriateness of comparison group studies; a description of the historical cohort comparison group design as a viable quasi-experimental option; and an example of a historical cohort comparison group evaluation design being implemented in nine elementary schools.
|
|
The Evolution of Academic Case Management in Multicultural Urban Schools
|
| Presenter(s):
|
| Nancy Van Kannel-Ray, Western Michigan University, nancy.vankannel-ray@wmich.edu
|
| Pamela Zeller, Western Michigan University, pamela.zeller@wmich.edu
|
| Warren Lacefield, Western Michigan University, warren.lacefield@wmich.edu
|
| Abstract:
This paper is the third in a series of longitudinal studies on academic case management. The paper documents the evolution of the academic case management intervention over three years in two low socio-economic, multicultural, urban, middle schools. Students in these schools have a history of struggling both academically and behaviorally. Initial findings documented that this intervention helped students learn to cope with their own personal challenges within the system of public school education with the overall goal of improving students' academic performance. Specifically, academic case management strategies made a significant difference in suspending the rapid decline in academic performance for the case managed student group. Additionally, this paper demonstrates to evaluators the use of one technique for using participants as their own controls while examining differences in performance patterns over time, testing and controlling for differences between comparison groups within schools and between schools and districts.
|
| | | |