| Session Title: Learning From the Consequences of No Child Left Behind on Evaluation (Part 2 of 2) |
| Multipaper Session 770 to be held in International Ballroom B on Saturday, November 10, 12:10 PM to 1:40 PM |
| Sponsored by the Pre-K - 12 Educational Evaluation TIG |
| Chair(s): |
| Rebecca Eddy, Claremont Graduate University, rebecca.eddy@cgu.edu |
| Abstract: The primary purpose of this panel (Part 2 of 2) is to explore what the field of evaluation has learned since the inception of No Child Left Behind (NCLB), and is a continuation of Part 1. Specifically, we will discuss how the practice of evaluation within the context of NCLB has itself been changed, and as a result, influences the practice of educational evaluation. In light of the reauthorization of NCLB set to occur in 2007, it is clear that this legislation has substantively impacted programs, schools, and districts (Part 1). The current panel extends this discussion in which we will explore challenges related to topic-specific foci, including both curriculum and assessment. In addition, we have included a culminating discussion of changes to evaluation purpose, design, and practice as well as a broader discussion of the challenges and opportunities for the field of evaluation in light of NCLB. |
| Measuring Student Progress: Changes and Challenges Under No Child Left Behind |
| Mariam Azin, Planning, Research and Evaluation Services Associates Inc, mazin@presassociates.com |
| Miriam Resendez, Planning, Research and Evaluation Services Associates Inc, mresendez@presassociates.com |
| The focus of this presentation will be to examine important issues in the assessment of student progress as a result of the No Child Left Behind Act (NCLB). In particular, NCLB has produced substantial changes on states' accountability systems and facilitated a plethora of assessment data that is potentially available to evaluators to use as 'outcome measures.' Thus, we will provide a brief overview of the current status of state accountability systems, including a description of the variability across state assessment systems in terms of what is being measured and how, and the intended and unintended consequences of such systems. In addition, we will discuss how evaluators can navigate the challenges we face in: a) using state assessment data to measure student progress adequately; and b) conducting educational evaluations in schools that are operating under the accountability umbrella of NCLB. |
| Using Research to Inform Educational Curricula |
| Marcy Baughman, Pearson Educational Measurement, marcy.baughman@pearsoned.com |
| The No Child Left Behind Act (NCLB) changed the development and publishing process for K - 12 educational materials. Prior to NCLB, scientific research to support product development or to demonstrate product efficacy was not required, and some viewed it as a 'value-added' component to product development. However, NCLB mandated that, 'educational materials purchased with public funds must be proven by scientific research to improve student achievement in the classroom (NCLB, 2002).' Research has been added into the product development cycle with little to no impact on school districts. However, the integration of research into the post-publication product cycle has proven challenging. Publishers need to test their products in the classroom using rigorous research methods to test product efficacy, best practice implementation guidelines, teacher satisfaction, and ease of use. This panel will provide a discussion of the intended and unintended consequences of research on educational curricula and the classroom. |
| Consequences of No Child Left Behind on Evaluation Purpose, Design, and Practice |
| Linda Mabry, Washington State University, Vancouver, mabryl@vancouver.wsu.edu |
| In 2003, the U.S. Department of Education proposed to prioritize for funding experimental, quasi-experimental, and other causality-focused designs in evaluation. Despite vigorous protest from some in the evaluation community, the priority was established as part of the No Child Left Behind Act of 2001 (NCLB). The priority threatened federal funding for the most frequently reported evaluation designs in Education, innovation in evaluation approach, contextualized formative feedback to program personnel, and ethical practice. It also reignited the "paradigm wars" over the relative merits of quantitative and qualitative methodology. The consequences will be discussed with reference to the analogous restrictions established by the World Bank and their consequences for evaluation design and practice and for societal consequences. In conclusion, an odd instance of contradictory federal initiatives related to NCLB will be discussed as to its implications for recognizing and understanding evaluation more broadly than the federal priority suggests. |
| No Child Left Behind and the Discipline of Evaluation: Challenges and Opportunities |
| Rebecca Eddy, Claremont Graduate University, rebecca.eddy@cgu.edu |
| Tiffany Berry, Claremont Graduate University, tiffany.berry@cgu.edu |
| This paper explores the challenges faced and opportunities afforded to the discipline of evaluation within the context of No Child Left Behind (NCLB). Specifically, this paper will detail such challenges as (1) reconciling what evaluation means under NCLB with what it means outside of education; (2) emphasizing outcomes at the expense of process assessment; and (3) prioritizing compliance needs ahead of other stakeholder needs. Opportunity costs associated with these challenges will also be explored. In response to NCLB, the evaluation community has been afforded several opportunities, many of which we should build upon as a growing discipline. For example, specific opportunities include (1) increased funding for evaluation; (2) capitalizing on evaluation as a transdiscipline across educational contexts (e.g., within schools, programs, curricula development, etc.); (3) increased need for evaluator training; and (4) ample opportunities for research on evaluation to examine empirically how NCLB has impacted the discipline of evaluation. |