|
Findings and Methodological Lessons From an Evaluation of a Project to Integrate the Arts Into Elementary School Reading and Mathematics Instruction
|
| Presenter(s):
|
| Paul Brandon,
University of Hawaii,
brandon@hawaii.edu
|
| Brian Lawton,
University of Hawaii,
blawton@hawaii.edu
|
| Abstract:
This paper discusses the results of a quasi-experimental evaluation of an elementary-school project (with three project schools and three control schools) to improve students' achievement and attitudes toward school by integrating arts strategies into reading and mathematics instruction, and it comments on problems with conducting studies of this sort. The results of a fixed-effects analysis of differences in project outcomes among treatment and control groups show some positive results, but the results on project implementation show that the teachers did not use arts strategies frequently, that the average level of quality of implementation was middling, and that implementation showed no correlation with outcomes. We briefly describe the results in the paper, discuss their contradictions, describe the threats to the validity of the findings, and present an overview of some of the difficulties, which we believe are common, in conducting small experimental studies of this sort.
|
|
Selecting Matched Samples Using Propensity Scoring Analysis
|
| Presenter(s):
|
| Manya Walton,
Windwalker Corporation,
manya.walton@windwalker.com
|
| Abstract:
Windwalker Corporation provides technical assistance to 14 Magnet Schools Assistance Program (MSAP) grantees conducting rigorous evaluations. Most rigorous evaluation grantees are implementing quasi-experimental designs with non-equivalent control groups to evaluate the impact of their interventions on student achievement. Propensity scoring analysis was used by some grantees to select treatment and comparison students for their study sample. This presentation discusses the uses of propensity scoring in education evaluation; the propensity scoring methods applied by MSAP grantees to match treatment and comparison groups; and the result of the student matching as it relates to sampling techniques for quasi-experimental evaluations.
|
|
Applying an Experimental Design to Evaluate the Effects on Student Achievement and Attitudes of a Materials Science Program Versus a Traditional Science Curriculum
|
| Presenter(s):
|
| Kris Juffer,
Action Research & Associates Inc,
drkjuffer@comcast.net
|
| Abstract:
Using a multiple method, experimental design, Action Research & Associates, Inc. evaluated the effects of an innovative science program developed with NSF funding, Materials World Modules (MWM). Action Research designed a blind, experimental study with two-tiered-randomized sampling with students randomly selected from a pool developed across Maryland and then randomly assigned in matched pairs to Treatment and Control Groups. Data was triangulated to ensure reliable data. Action Research evaluated a the Materials World Modules Program vs. traditional science study in a 24-hour, 7-days-a-week, 4-week long immersion residential Summer Institute for 96 middle school and high school students, controlling intervening variables. MWM is a multi-disciplinary materials science program for secondary classrooms developed on the principles of hands-on, inquiry-and- design-based learning incorporating scientific problem-solving and iterative design projects and labs, including student presentations and defense of project process and outcomes. The effects of learning science via MWM compared with traditional science textbooks was explored by looking at relative changes in students' science knowledge and attitudes towards studying science. Student and educator collected information about changes in the students' interests, attitudes and experiences in science and other mediating variables, before and after experiencing Institute instruction. Student outcome variables were assessed with a science knowledge test including items assessing their abilities to problem-solve. To triangulate the data, students and teachers were administered surveys about their own experiences, their perceptions of each other, and independent researchers made classroom observations.
|
| | |