|
Session Title: Promoting Context-Appropriate Valuation of Public Programs and Policies
|
|
Panel Session 839 to be held in Panzacola Section H2 on Saturday, Nov 14, 1:40 PM to 3:10 PM
|
|
Sponsored by the Quantitative Methods: Theory and Design TIG
|
| Chair(s): |
| George Julnes, University of Baltimore, gjulnes@ubalt.edu
|
| Discussant(s):
|
| Leslie J Cooksy, University of Delaware, ljcooksy@udel.edu
|
| Ernest House, University of Colorado at Boulder, ernie.house@colorado.edu
|
| Michael Scriven, Claremont Graduate University, mjscriv@gmail.com
|
| Abstract:
The recent controversy over a federal agency giving priority to random assignment experiments for establishing the causal impacts of interventions (see Julnes & Rog, 2007; Donaldson, Christie, & Mark, 2008) has raised questions about the role that the evaluation community should play in guiding federal policies on evaluation methods. A core element in the eventual position advocated by AEA was that method-choice should be driven by the needs of specific contexts. A similar controversy may be developing around the methods favored by federal agencies for reaching judgments of the value of programs and policies. This session will not seek to resolve controversies over preferred approaches to assigning values in public decision-making, but it will develop the dialogue on context-appropriate ways to understand stakeholder values and to represent those values in aggregate judgments based on the multiple criteria used in federal policymaking.
|
|
Distinguishing Approaches to Public-sector Valuation: Developing Contextual Frameworks That Promote Productive Dialogue
|
| George Julnes, University of Baltimore, gjulnes@ubalt.edu
|
|
There are three major families of conclusions relevant to most evaluations: representation of needs, activities, and outcomes; causal attribution regarding program mechanisms and impacts; and valuation as it relates to claims about the value of one or more alternatives. Although there has been considerable controversy over preferred approaches to supporting causal attribution, one could argue that valuation is the area with the least consensus over the preferred approaches to use in different contexts. This presentation will address the many approaches that are used to establish the relative or absolute value of public programs and policies. Organizing these approaches in terms of some of their major features can help us identify ways that methods of valuing can be understood in terms of their usefulness in specific contexts. This, in turn, can help promote a more productive dialogue on matching methodological approaches to valuation with their most effective contexts.
|
|
|
Program Value, Program Ethics, and Program Evaluators: How Do We Do the "Right" Thing?
|
| Michael Morris, University of New Haven, mmorris@newhaven.edu
|
|
Most people would agree that ethical considerations should be taken into account when rendering judgments concerning the value of a program or policy. What role, if any, should evaluators play in the formulation and presentation of those considerations? My comments will focus on the opportunities and challenges associated with evaluators providing input into stakeholder discussions of program/policy ethics.
| |
|
Different Criteria Federal Stakeholders Use for Valuing Federal Programs
|
| Stephanie L Shipman, United States Government Accountability Office, shipmans@gao.gov
|
|
How we "value" a program reflects our preferences and priorities, i.e., the importance we place on competing criteria. Recent federal performance-based management initiatives spotlighted program assessment-and made the choice of assessment criteria more transparent. Federal agencies, Congress, and the Office of Management and Budget often disagreed on "comprehensive" assessments of programs over multiple dimensions, because they differed on which dimensions or criteria they considered most important. Evaluators tend to focus on the extent to which a program achieves its objectives; others may focus on its overlap with other programs. The criteria that federal stakeholders use to value a program's merit or worth reflect where they stand vis-a-vis the program, the decisions they face, as well as their "politics," i.e., their policy values and priorities. I will discuss the different criteria these parties used to "value" programs from my perspective at Congress' primary oversight agency.
| |