|
Session Title: Evaluating the Reading First Program: Best Practices and Lessons Learned
|
|
Panel Session 123 to be held in Federal Hill Suite on Wednesday, November 7, 4:30 PM to 6:00 PM
|
|
Sponsored by the Pre-K - 12 Educational Evaluation TIG
|
| Chair(s): |
| Michael Long,
Macro International Inc,
michael.c.long@orcmacro.com
|
| Discussant(s):
|
| Michael Long,
Macro International Inc,
michael.c.long@orcmacro.com
|
| Abstract:
The Reading First program is a federal literacy grant that has had a tremendous impact on how reading is taught in elementary schools across the country. This panel will consist of four evaluators of state Reading First programs in Delaware, Indiana, Maryland, and Ohio. Each evaluator will discuss their work in their state, including the types of evaluation questions they are answering, the variables that they are measuring, and the instruments and methods they have found most effective. They will also address the methodological, practical, and political obstacles that they have encountered, and how they have addressed these challenges.
|
|
Evaluation of Policy and Practice in Delaware's Reading First Initiative
|
| Linda Grusenmeyer,
University of Delaware,
lgrusen@udel.edu
|
|
No Child Left Behind has the Federal Reading First program as its model for early literacy instruction. Within this national context, Delaware's Reading First (DERF) program proposes to effect local change through teacher training in Scientifically Based Reading Research practices and effective use of data to inform instruction. As external providers of DERF's program evaluation, the Delaware Education Research and Development Center examines the implementation and impact of the five-year project at three levels: student level (attitudes and achievement), classroom/teacher level (teacher knowledge, beliefs, opinions and practices), and system level (alignment and efficacy of curriculum, professional development, state and district support, and teacher preparation programs.) With its rich and varied evaluation design, Reading First presents a range of challenges to Delaware's evaluation team. Among those issues that will be discussed will be maintaining balance between formative and summative measures and providing guidance in collecting and using data.
|
|
|
Evaluation of the Maryland Reading First Program: Successes and Challenges
|
| Michael Long,
Macro International Inc,
michael.c.long@orcmacro.com
|
|
In this presentation Mr. Long will discuss Macro International's evaluation of the Maryland Reading First program. He will begin my describing the original design of the evaluation, as well as some of the factors and considerations that led to that design. He will then describe how this design has changed over time, and what has prompted these changes. Some of the issues that he will focus on are working with the state to develop clear research questions, instrument selection and development, and the challenges of managing a statewide evaluation in a state that has strong local control of schools. He will conclude by providing some strategies that have worked well, as well as recommendations for other evaluators based on his experience with this project.
| |
|
Evaluation of the Indiana Reading First Program
|
| Amy Kemp,
Indiana University,
amkemp@indiana.edu
|
|
Dr. Kemp will discuss the Center for Evaluation and Education Policy's evaluation of the Indiana Reading First (RF) program, which provides summative and formative feedback as well as capacity building and technical assistance. Guiding questions of this evaluation include: What is the role of the coach in the classroom? To what extent can the successful implementation of RF be attributed to Leadership Teams? To what extent are classroom instructional strategies guided by assessment results? Data are collected through 18 annual visits to schools; web-based surveys of teachers, principals, coaches, and interventionists; and phone interviews with district representatives. In addition, ISTEP+, Terra Nova, and DIBELS data are analyzed to determine growth over time; analyses compare the growth of at-risk students to overall student growth. In addition to describing her work, Dr. Kemp will provide advice and suggestions for evaluators involved in similar projects.
| |
|
It Can't Be Either/Or: Using Internal Evaluation Methods and Procedures to Complement and Strengthen External Evaluation
|
| James Salzman,
Cleveland State University,
j.salzman@csuohio.edu
|
|
In his presentation, Dr. Salzman will present a brief overview of the purposes, limitations and usefulness of internal evaluation from his perspective as Co-Director of the Reading First Ohio Center and supervisor of internal evaluation. He will describe why both internal and external evaluation cycles are necessary, valuable, and informative pieces of project implementation. He will then argue for a strong internal evaluation plan, including the use of traditionally summative tools and methods used for formative purposes, as central to making appropriate programmatic adjustments-especially in the early stages of implementation. Finally, he will discuss the need to build in ongoing conversations among both external evaluators and program deliverers for the purpose of clarifying the implementation and being able to use the results of the different types of evaluation to strengthen the communication to internal and external stakeholders.
| |