|
Programmed for Trouble: Evaluation of Federally Funded Education Programs
|
| Presenter(s):
|
| Keith Murray,
M A Henry Consulting LLC,
keithsmurray@mahenryconsulting.com
|
| Martha Henry,
M A Henry Consulting LLC,
mahenry@mahenryconsulting.com
|
| Abstract:
Federally funded education programs often mandate external evaluation as part of their grant projects. U.S. Department of Education and National Science Foundation projects consume considerable resources and reach into thousands of classrooms. The external evaluation requirement may seem to reflect the evaluation field's incorporation as a crucial partner in successful program implementation in this critical area. In practice, however, the challenges of managing evaluation given the programs' structure and constraints are substantial. Ambiguous chains of authority and communication, ambivalent funder support and oversight, inconsistent definitions of evaluators' responsibilities, and grantee control over evaluator contracts are among the challenges experienced. This paper examines the role of evaluation in federal education programs, informed by input from grantor program staff, grantees, and evaluators. The authors conclude with recommendations for a reconsideration of evaluator responsibilities, grantee education, and realignment of the grantor-evaluator relationship to optimize evaluation's service to education projects and their many stakeholders.
|
|
Evaluating No Child Left Behind Test-based Educational Accountability
|
| Presenter(s):
|
| Katherine Ryan,
University of Illinois at Urbana-Champaign,
k-ryan6@uiuc.edu
|
| Abstract:
Audit cultures continue to intensify nationally and globally--reflections of New Public Management which rely on performance-based management (Power, 1997). The No Child Left Behind Act (NCLB) of 2001 displays this performance-based management approach by dramatically increasing the impacts of test-based educational accountability for managing and improving education. This is essentially a performance measurement system (Ryan, 2007).
Importantly, while there are significant impacts (e.g., school closings) associated with NCLB accountability, provisions for evaluating NCLB high stakes assessment and accountability are notably absent. This paper presents a framework for evaluating state- wide NCLB test-based educational accountability to examine whether the performance measurement system (e.g., high stakes assessments) is being implemented as planned. In addition, the framework will incorporate intended and unintended consequences of performance measurement uses in relationship to students, instruction, and educational outcomes. I review several literatures in developing this framework including the learning organization, implementation theory, standards for accountability system and others.
|
|
Assessing the Feasibility of Incorporating Geocoding and Census Data Into the National Assessment of Educational Progress (NAEP) Framework
|
| Presenter(s):
|
| Stacey Merola,
American Institutes for Research,
smerola@air.org
|
| Steve Davis,
American Institutes for Research,
sdavis@air.org
|
| Abstract:
As part of our ongoing work to develop a new composite measure of socioeconomic status (SES) for NAEP, we are investigating using census data in conjunction with NAEP. Incorporating census data into the NAEP framework presents some logistical and methodological challenges that must be addressed. In this paper we will present the results of a geocoding pilot study conducted in March 2007 with 167 schools across the United States. The users were asked to collect student addresses and then geocode those addresses using a geocoding software that has been developed specifically for NCES. The users will be asked via a survey and debriefings about their experiences. The data collection is currently underway, and to date the results are positive.
|
|
Conducting a Robust Performance Needs Assessment: An No Child Left Behind (NCLB) Case Study
|
| Presenter(s):
|
| Willis Thomas,
Western Michigan University,
willis.h.thomas@wmich.edu
|
| Abstract:
This paper will begin by discussing the recently completed year-long, two-part, No Child Left Behind Parental Involvement evaluation, and how that project led to a performance needs assessment of the related Parent Corps Program.
The design of the performance needs assessment was quite unique, integrating good practices from a variety of methodologies and approaches. This performance needs assessment looked at performance needs, treatment needs, and solutions. It investigated the core-common-critical needs as well as developmental needs of program participants. It resulted in an excellent model, and is worth considering when engaging in program analysis or action research.
|
| | | |