| Session Title: Elephants in the Evaluation Room: Managing Evaluations Amid Clashing Values of Program Staff and Professional Evaluators |
| Think Tank Session 921 to be held in Huntington B on Saturday, Nov 5, 12:35 PM to 2:05 PM |
| Sponsored by the Evaluation Managers and Supervisors TIG |
| Presenter(s): |
| Michelle Mandolia, United States Environmental Protection Agency, mandolia.michelle@epa.gov |
| Discussant(s): |
| Yvonne Watson, United States Environmental Protection Agency, watson.yvonne@epa.gov |
| Michelle Mandolia, United States Environmental Protection Agency, mandolia.michelle@epa.gov |
| Matt Keene, United States Environmental Protection Agency, keene.matt@epa.gov |
| Abstract: Evaluation results have the greatest potential to effect program or policy change when certain criteria have been met: 1) objectivity (quality control against subjective bias); 2) active stakeholder involvement in the evaluation; 3) attention to rigorous design issues appropriate for evaluation context; and 4) granting the evaluator license to provide nuanced interpretation of complex results. Attention to these non-exhaustive characteristics of good evaluation illustrates that sometimes these values conflict, particularly when program staff members become heavily invested in shaping the final evaluation product. Those with evaluation management oversight must manage this conflict. Evaluation managers will discuss composite case examples that illustrate the situation whereby program staff's vested interests in the evaluation interfere with the objectivity and rigor of the final evaluation. The presenters will solicit feedback and concrete strategies for negotiating this dilemma with breakout groups focusing on managing evaluation process, evaluation design development, evaluation personnel, and evaluation results. |