| Session Title: Weights and Measures: Resolving and Reporting on Differences Between Qualitative and Quantitative Findings |
| Multipaper Session 267 to be held in Suwannee 13 on Thursday, Nov 12, 10:55 AM to 12:25 PM |
| Sponsored by the Pre-K - 12 Educational Evaluation TIG |
| Chair(s): |
| Beverly Farr, MPR Associates, bfarr@mprinc.com |
| Abstract: This presentation reports on the study of a multi-site implementation of demonstration sites implementing a multiple pathways approach that integrates academic and technical skill and knowledge in a range of high schools. The variation in contextual features at the 16 sites provided the first challenge in conducting cross-site analyses, but establishing a balance between reporting on the quantitative and qualitative results, related to a notable high school reform strategy, proved to be the biggest challenge. Interest in the achievement results, which were mixed but indicative of positive results, seemed to overshadow the strong positive findings from the qualitative study of implementation. |
| Celebrating Success: Cross-site Analysis of Qualitative Findings |
| Beverly Farr, MPR Associates, bfarr@mprinc.com |
| This paper describes the qualitative methods used to study implementation of the multiple pathways approach in 16 high schools in California. Selected as demonstration sites, the schools reflect a range in format and contex, 'from autonomous high schools, to career academies, to course sequences. Since the designers had developed a rubric reflecting quality indicators, it was possible to use it to develop classroom observation and other instruments to document implementation quality. While some of the qualitative outcomes, effects on student behaviors, attitudes, and perceived learning, as well as on teacher practices, were very strong, the political realities related to school reform and funding for such reforms led to an over-emphasis on the achievement results. |
| Responding to the Over-emphasis on Quantitative Results |
| Denise Bradby, MPR Associates, dbradby@mprinc.com |
| This paper describes the quantitative methods used to study the student achievement outcomes of the multiple pathways approach in 16 high schools in California. The achievement indicators included the state high school exit exam and the California Standards Tests as well as grade-grade promotion, attendance, and graduation rates. The data were first analyzed for the network as a whole, against statewide results. These results were controlled for race/ethnicity which led to improved results for program participants. The researchers also made appropriate comparisons for each site, either to the rest of the school in which the program was found or to similar schools in the district. The results of the foregoing analyses was a mixed picture. The most significant challenge was the examination of ratings on the implementation rubric to achievement results. |