|
School Reform Without Evaluation: A Policy That Undermines Evaluation Quality- A Guide for Schools to Respond
|
| Presenter(s):
|
| Louis Cicchinelli, Mid-continent Research for Education and Learning, lcicchinelli@mcrel.org
|
| Zoe Barley, Mid-continent Research for Education and Learning, zbarley@mcrel.org
|
| Abstract:
Quality in evaluation must originate with program developers and adequately support program implementers. This presentation will feature an evaluation guide based on the recent school improvement grant requirements for low-performing schools/districts whether or not they have received school improvement funding. The grant requirements do not support a local evaluation –only limited data collection. The lack of evaluation guidance is a serious quality issue both for the program and for evaluation as a field. The guide presented begins with a feasibility analysis and then assists the school with developing a logic model, a local school evaluation plan and assigning a team to carry it out (with an external evaluator to assist). Worksheets are included as well as examples of completed worksheets. With audience recommendations for improving the guide, we intend to make it widely available.
|
|
Effective Policy in Saudi Arabia' Gifted Organizations
|
| Presenter(s):
|
| Fayez Shafloot, Western Michigan University, eval.p@hotmail.com
|
| Abstract:
This paper will describe how the evaluation policy in gifted and talent organization in the Kingdom of Saudi Arabia (KSA) affects the program input, processes, and outcomes' quality. KSA has a new gifted organization, King Abdulaziz and his Companions Foundation for the Gifted, which is supported by the government and privet sectors, serves as a leader organization for supporting and conducting all required events and activities for gifted students. The evaluation policy is being used is imported from western policies that fit the western culture and is consistent with their school systems and families supports. The evaluation policy will affect the program reputation on the longtime if that policy would not be reevaluated and updated to meet the KSA culture.
|
|
Blueprint for Quality Evaluation and Quality Services in Human Service Organizations: Why Evaluation Policy Matters
|
| Presenter(s):
|
| Kristin Kaylor Richardson, Western Michigan University, kkayrich@comcast.net
|
| Abstract:
The quality of evaluation, either as a single study, a series of studies, a cumulative process of knowledge building, or as “evaluation capacity” mainstreamed or built into the very fabric of an organization will in large measure depend upon the nature, scope and influence of evaluation policies. Human service agencies must be responsive to accountability demands, yet many also strive to build evaluation capacity, developing communities of practice rich in knowledge and well-positioned to act with purpose and wisdom. How can evaluation models and methods be strategically adopted and integrated to support accountability, learning and intelligent action? What is the connection between evaluation policy and how evaluation is conceptualized and practiced in agency settings? Why does evaluation policy matter? This paper addresses these questions through review and critique of existing agency-centered evaluation frameworks, and then proposes an integrated approach to strengthen the quality of evaluation in human service organizations.
|
|
Evaluation Policy Inventory
|
| Presenter(s):
|
| Margaret Johnson, Cornell Univeristy, maj35@cornell.edu
|
| William M Trochim, Cornell University, wmt1@cornell.edu
|
| Abstract:
Many organizations lack clear policies about how, when, what to evaluate and how evaluation will be supported and sustained. In the absence of a conscious and comprehensive approach to evaluation policy, some policies of an organization may support evaluation, others undermine it. Policies at different levels of an organizational system may overlap or contradict one another. This presentation uses an empirically derived taxonomy of evaluation policies built from the survey responses of AEA members in 2009 to develop an Evaluation Policy Inventory instrument (EPI). The EPI is intended for use by any organization wishing to take stock of existing evaluation-related policies, pinpoint conflicting evaluation policies and show gaps in evaluation policy.
|
|
Exploring Dueling Federal Funder Preferences and Program Evaluation Needs: Challenges With Conflicting Interests and Evaluator Roles
|
| Presenter(s):
|
| Holly Downs, University of Illinois at Urbana-Champaign, hadowns@illinois.edu
|
| Abstract:
A growing tension exists between federal funder preferences for an “external” evaluator and the evaluation needs of a program. This paper will explore this dilemma encountered when a federally-funded undergraduate science and mathematics program at the University of Illinois shifted from an external to an internal evaluation team (i.e., a team at the university but outside the program departments). While the funder encouraged an external evaluator in the RFP, a shift by the program coordinators, who wanted better onsite data collection and Institutional Review Board coordination, trumped this funder preference post-award. These dueling dynamics can cause a chasm between what is valued and encouraged by funding agencies and the needs of the actual program. The goal of this paper is twofold: to discuss the impact of federal funders’ preferences of external evaluators on this program and to consider the implications of these issues on the field of evaluation.
|
| | | | |