|
Session Title: Risks and Rewards of Conducting Politically Sensitive and Highly Visible Evaluations
|
|
Panel Session 331 to be held in Room 110 in the Convention Center on Thursday, Nov 6, 1:40 PM to 3:10 PM
|
|
Sponsored by the Government Evaluation TIG
|
| Chair(s): |
| Rakesh Mohan,
Idaho Legislature,
rmohan@ope.idaho.gov
|
| Discussant(s):
|
| Rakesh Mohan,
Idaho Legislature,
rmohan@ope.idaho.gov
|
| Abstract:
Politically sensitive and highly visible evaluations are often associated with risks and rewards. Evaluations associated with high risks do not automatically suggest that such evaluations be avoided. Evaluation organizations and evaluators can take certain steps to mitigate those risks and increase the likelihood of achieving higher level of rewards.
This panel will discuss examples of politically sensitive and highly visible evaluations on various subjects, including education, health, and social services. The discussion will focus on the types of risks evaluators face, how those risks affect evaluation policies and practices, how to mitigate those risks in order to conduct successful evaluations, and whether rewards outweigh risks.
|
|
The At-Risk Evaluation Office: Walking a Tightrope in the State Legislature
|
| Nancy Zajano,
Learning Point Associates,
nancy.zajano@learningpt.org
|
|
In conducting evaluations for a state legislature, the risks of being a messenger with unwelcome news are plentiful. There is always someone or some group who views the evaluation findings as detrimental to a particular interest, no matter how sound the methodology. On the other hand, evaluations done at the behest of a state legislature have the potential to directly influence policy. In some instances the report’s recommendations are translated almost verbatim and immediately into a bill directing the future implementation of a state-funded program. Examples of both the risks and rewards from an education oversight office in Ohio will be presented – including the story of the demise of the office itself when the findings from a series of reports on charter schools were considered unwelcome by some officials.
|
|
|
Anticipating and Preparing for Attacks on Your Evaluation Methodology
|
| Tedd McDonald,
Boise State University,
tmcdonal@boisestate.edu
|
|
For a politically sensitive and highly visible evaluation, the likelihood that your methodology will be attacked, or at least questioned, is high. Evaluators and their organizations must anticipate this and planned accordingly – they need to be prepared to answer tough questions about their methods in a public setting. This presentation will discuss a study that evaluated the largest government health and welfare agency in a state. The stakes were high, because legislators who directed the study questioned the performance of the agency management. The evaluators anticipated the tough road ahead and planned their study accordingly. The amount of efforts spent on planning and scoping the study paid off – the study was a success in terms of having both policy and programmatic impact.
| |
|
Biting the Hand that Feeds You: A Dilemma for Evaluators in the Legislative Environment
|
| Bob Thomas,
King County Auditor's Office,
rthomasconsulting@msn.com
|
|
Evaluations conducted as part of legislative oversight may carry certain risks for the evaluation teams. When the outcomes of an evaluation are politically sensitive, the process and the competence of the evaluation itself can come under intense scrutiny. Although such risks “go with the territory” and usually can be mitigated by developing a sound methodology and following standards, sometimes the political stakes are so high that the normal process breaks down and the risks to the evaluators become severe. This can happen when the legislature itself has contributed to the problems under review. This presentation will draw on examples of evaluations that have faced and overcome such risks, and will identify the strengths of the evaluation process that can make success possible.
| |