|
Session Title: Evaluation Policy and Evaluation Practice: Where To Next?
|
|
Panel Session 892 to be held in Sebastian Section L4 on Saturday, Nov 14, 3:30 PM to 5:00 PM
|
|
Sponsored by the Theories of Evaluation TIG
|
| Chair(s): |
| William Trochim, Cornell University, wmt1@cornell.edu
|
| Abstract:
An evaluation policy is any rule or principle that a group or organization uses to guide its decisions and actions when doing evaluation. Every group and organization that engages in evaluation - including government agencies, private businesses, and nonprofit organizations - has evaluation policies. Sometimes these are formal, explicit and written; at other times they are implicit and ad hoc principles or norms that have simply evolved over time. Evaluation policies profoundly affect the day-to-day work of all evaluators. Many recent and current controversies or conflicts in the field of evaluation can be viewed, at least in part, as struggles around evaluation policy. Because evaluation policies typically apply across multiple evaluations, influencing policies directly may have systemic and far-reaching effects for practice. This panel discusses current thinking on the topic of evaluation policy, especially how it is informed by and affects evaluation practice, and suggests directions for future work in this area.
|
|
Introduction to Evaluation Policy
|
| Melvin Mark, Pennsylvania State University, m5m@psu.edu
|
| Leslie J Cooksy, University of Delaware, ljcooksy@udel.edu
|
| William Trochim, Cornell University, wmt1@cornell.edu
|
|
This introductory presentation will address the following questions in general terms and provide a foundation for the panel:
-What is evaluation policy? What questions or issues should a comprehensive organizational evaluation policy address?
-How does evaluation policy influence evaluation practice?
-When does systematic evaluation get deployed? What programs, policies, or practices are chosen as the subject of evaluation, when, and why?
-What policies should guide the identification and selection of evaluators? What credentials should evaluators have? What kind of relationship should evaluators have to the program or entity being evaluated?
-What policies should guide the timing, planning, budgeting and funding, contracting, implementation, methods and approaches, reporting, use and dissemination of evaluations?
-What policies should guide how evaluation participants and respondents are engaged and protected?
-How can existing (e.g., the Guiding Principles for Evaluators) or prospective professional standards inform evaluation policy?
|
|
|
Evaluation Policy and Evaluation Practice: Taxonomy and Methodology
|
| William Trochim, Cornell University, wmt1@cornell.edu
|
|
This presentation significantly extends Trochim's 2008 Presidential Address and begins by describing an evaluation policy as "any rule or principle that a group or organization uses to guide its decisions and actions when doing evaluation." Examples of evaluation policies are provided to illustrate the form they might take. The paper offers a tentative taxonomy of evaluation policies, dividing them into eight broad topical areas: goals, participation, capacity building, management, roles, process and methods, use and meta-evaluation. The idea of evaluation policy methodology is introduced and a general method based on the notion of the policy wheel is described. Key principles that guide evaluation policy formulation are discussed, including: specificity; inheritance; encapsulation; exhaustiveness; continuity; delegation and accountability. The methodology is illustrated in the context of U.S. federal evaluation. Critical issues and challenges for the field of evaluation policy, and the implications for policy development methodology and for future research are considered.
| |
|
Evaluation Policy and Evaluation Practice: Where Do We Go From Here?
|
| Leslie J Cooksy, University of Delaware, ljcooksy@udel.edu
|
| Melvin Mark, Pennsylvania State University, m5m@psu.edu
|
| William Trochim, Cornell University, wmt1@cornell.edu
|
|
This presentation synthesizes key themes and issues regarding evaluation policy, including those raised by conference presentations of 2008 on this theme and by the other presentations in this panel. The paper describes how key issues regarding evaluation policy encompass different settings (e.g., domestic and international; academic and public sector) and foci (e.g., the theory of evaluation policy, overarching national policies, and policies specific to the organizational location of the evaluation function). Common and complementary themes are identified. The discussion includes suggestions for future directions in evaluation policy including: the need for an empirically-derived taxonomy of evaluation policy categories and how such might be created; the development of evaluation policy audits (including checklists and measures that might be used in accomplishing them); the need for evaluation policy development processes and methods (including further development of methods outlined in Trochim's 2008 Presidential Address); the need for evaluation policy archives and how they might be structured; and the need for organizational structures that support evaluation policy.
| |