|
Session Title: From Compliance to Reliance: Critical Moments in Integrating Evaluation Into an Organization’s Work
|
|
Panel Session 677 to be held in TRAVIS C on Friday, Nov 12, 4:30 PM to 6:00 PM
|
|
Sponsored by the Evaluation Use TIG
and the Organizational Learning and Evaluation Capacity Building TIG
|
| Chair(s): |
| Jennifer Iriti, University of Pittsburgh, iriti@pitt.edu
|
| Abstract:
What are the critical moments in an organization’s journey from “evaluation for compliance” to relying on evaluation for core program design and decision-making? This panel will introduce participants to a success case in which evaluators worked with an education non-profit over 6 years to integrate evaluation and evaluative thinking into the work of the organization. First, context about the case and how the organization’s orientation toward evaluation has changed is provided. Then the following issues are examined from the perspectives of evaluator, organizational leader, program implementer, and funder: a) value-added of the evaluation work; b) critical moments in the work; c) generalizable lessons about evaluation integration and use; and, d) challenges to the organization’s shift toward reliance on evaluation. The goal of the session is to draw on a real-world longitudinal example to bring to life the actions, conditions, and tools that supported use and ownership of evaluation.
|
|
Evaluator Perspective
|
| Jennifer Iriti, University of Pittsburgh, iriti@pitt.edu
|
| Catherine A Nelson, Independent Consultant, catawsumb@yahoo.com
|
|
The evaluator perspective will be shared by Jennifer Iriti. Dr. Iriti has been co-evaluator with Catherine Nelson on this evaluation since its inception. She will both facilitate the session overall and contribute as a panelist. She will summarize the nature and general components of the evaluation activity and provide contextual information about the organization. As a panelist, she will offer insights about the case from the evaluator perspective. Key issues she will discuss include: specific ways that program staff’s participation in the evaluation design process affected on its own program design work (process use), how the evaluation design and refinement kept the user at the center of the work, how the collaborative development of an assessment tool was a turning point in the organization’s valuing of evaluation, and the ways that implementation evaluation early on shifted the perceived utility of evaluation more generally.
|
|
|
Organization Leader Perspective
|
| Mary Hicks, Boundless Readers, mhicks@boundlessreaders.org
|
|
Mary A. Hicks is the Executive Director of Boundless Readers (the focus of the evaluation) and has been the key program decision-maker with regard to evaluation since 2003. As the leader of the organization, she offers a perspective that incorporates the strategic, political, human resources, and financial issues of the organization overall in relation to the evaluation work. She has chosen to sustain the evaluation activities of the organization well beyond the time and scope demands that initially sparked the evaluation. Key issues that Ms. Hicks will share include: how the collaborative work with evaluators led to a re-visioning and standardization of the organization’s measurable goals across programs, how the context of “new program development” served as a natural entrée for evaluation use, the ways in which internal program planning was permanently altered by participating in the evaluation, and the challenges of human and financial resources required to sustain evaluation.
| |
|
Program Implementer Perspective
|
| Nancy Plaskett, Chicago Children's Museum, nancyp@chicagochildrensmuseum.org
|
|
Nancy Plaskett was the Program Director at Boundless Readers from July 2005 through September 2007 during which time intensive evaluation was conducted for several programs. She will offer insights about evaluation from the program implementer’s perspective and will discuss the benefits of evaluation to produce strong formative information that enhances program philosophy, structure, activities, and targeted outcomes. She will share examples of specific program and process improvements that resulted from the evaluation and efforts to build capacity in staff to implement rigorous programming that resulted from evaluation-related revisions to the design. Challenges she will discuss include program participant resistance to incorporate evaluative elements in to the work of the program as well as challenges in maintaining a high fidelity of implementation of the program evaluation itself.
| |
|
Program Funder Perspective
|
| Regina Dixon-Reeves, Lloyd A Fry Foundation, rdixonreeves@fryfoundation.org
|
|
Regina Dixon-Reeves is a program officer with the Lloyd A. Fry Foundation, which has been a core funder of Boundless Readers since 1990. This funder played a particularly important role in this case in that it specifically funded the evaluation work for the Rochelle Lee Teacher Awards, the organization’s 22 year old flagship program. Dr. Dixon-Reeves will share her perspective on why the foundation supported an evaluation and how she has seen the organization change as a result. Specifically, she will discuss how the foundation viewed the changes that took place in Boundless Readers’ decision-making and programming as a result of the evaluation. In addition, Dr. Dixon-Reeves will talk about the role of evaluation in the funding relationship with Boundless Readers over time.
| |