|
Session Title: Classroom Observations: Lessons Learned About Five Protocols From Five Multi-site Education Evaluations
|
|
Panel Session 725 to be held in BONHAM D on Saturday, Nov 13, 8:00 AM to 9:30 AM
|
|
Sponsored by the Pre-K - 12 Educational Evaluation TIG
|
| Chair(s): |
| Elizabeth Autio, Education Northwest, elizabeth.autio@educationnorthwest.org
|
| Abstract:
the same time, classroom observations are incredibly time- and budget-intensive. It is therefore important that evaluators use these resources of time and budget wisely. Getting the most from classroom observations requires the selection and/or development of a protocol that is aligned with the content, goals, and intended outcomes of the program being studied. To help guide decisions about observation protocols, in this session, evaluators will share what they have learned about five different classroom observation protocols – three that are commercially available, and two that were developed for a specific evaluation – from five different multi-site education evaluations. The session will result in tangible lessons learned from each protocol that participants can “take away” after the session and apply to their own work.
|
|
Using the Classroom Assessment Scoring System (CLASS) to Measure Classroom Organization, Interactions, and Support
|
| Jason Greenberg Motamedi, Education Northwest, jason.greenberg.motamedi@educationnorthwest.org
|
|
The Classroom Assessment Scoring System (CLASS) is a rubric of ten items that assess classroom organization, the quality of instructional and social interactions and support observed between teachers and student within a classroom (Pianta, La Paro & Hamre, 2008). The CLASS is available at pre-K, elementary, and secondary levels, however this presentation will concentrate upon the Pre-K and early elementary levels, discussing and describing the CLASS, detailing its intended uses, the training and certification procedure, as well as a description of why the CLASS was selected for an evaluation of Early Reading First classrooms in Montana, and how it was employed and reported. Finally, this presentation will discuss the lessons learned from using the CLASS as an observational tool alongside the ELLCO, as well as a discussion of other factors that evaluators may want to take into account when considering this protocol.
|
|
|
Looking at Literacy With the Early Language and Literacy Classroom Observation Tool (ELLCO) and the Sheltered Instruction Observation Protocol (SIOP)
|
| Elizabeth Autio, Education Northwest, elizabeth.autio@educationnorthwest.org
|
|
The Early Language and Literacy Classroom Observation Tool (ELLCO) (Smith, Brady & Anastasopoulos,2008) is a rubric of 19 items that assess quality of classroom environment and teacher practices in literacy. The Sheltered Instruction Observation Protocol (Echevarria & Short, 2001; Echevarria, Vogt, & Short, 2007) is a research-based model of instruction for English language learners, which also includes an observation checklist of 30 items. Both of these protocols are used for feedback loops among educators, but are increasingly employed for evaluation data collection purposes. This session will discuss what the protocols are, why they were selected by the evaluators, cost, ease of use, and lessons learned, including feedback on the appropriateness of developer-created rubrics in the evaluation context. This information will be helpful particularly to others evaluating literacy programs and programs addressing the needs of English language learners.
| |
|
Development of an Observation Protocol for a Reading Intervention
|
| Caitlin Scott, Education Northwest, caitlin.scott@educationnorthwest.org
|
|
Developed in 1991 by Dee Tadlock, Read Right is a reading intervention program designed to improve the reading of struggling students of all ages. Education Northwest was hired by the Sherwood Foundation in 2009 to evaluate Read Right in Omaha Public middle and high schools. As part of this evaluation, Education Northwest created an observation protocol. Because Read Right is a relatively scripted program delivered in a setting with one adult tutor per five students, the observation protocol is primarily a count of activities. For example, the observer counts and categorized each statement the tutor makes that corrects the student’s reading. The description of the development of this protocol—from first draft to field tested product—will help the audience understand how protocol can be developed based on the content of the observed program. This information will be helpful particularly to others evaluating scripted educational programs.
| |
|
Development of an Observation Protocol for Adolescent Literacy Interventions
|
| Kari Nelsestuen, Education Northwest, kari.nelsestuen@educationnorthwest.org
|
|
The federal Striving Readers program requires evaluators to measure fidelity of implementation of selected reading interventions. Since valid and reliable observation protocols for adolescent literacy do not exist, the presenter will describe the protocols developed to measure fidelity of implementation of two adolescent literacy interventions– Read to Achieve (Marchand-Martella & Martella, 2010) and Phonics Blitz (Farrell and Hunter, 2007). The presentation will focus on the development process, including how evaluators worked closely with the publishers to identify and define the critical components of the program model, quantify these components, and field test multiple versions of the instruments (Mowbray et al, 2003). The final protocols are a combination of activity counts and ratings of the identified critical components such as teacher support. Presenters will also describe how the protocol is used to calculate fidelity ratings of high, medium, and low implementation.
| |