|
Session Title: How Practitioners and Researchers Can Use Models to Build and Sustain Evaluation Capacity
|
|
Panel Session 817 to be held in Capistrano A on Saturday, Nov 5, 8:00 AM to 9:30 AM
|
|
Sponsored by the Organizational Learning and Evaluation Capacity Building
|
| Chair(s): |
| Steffen Bohni, Ramboll Management Consulting, sni@r-m.com
|
| Discussant(s):
|
| Hallie Preskill, FSG Social Impact Consultants, hallie.preskill@fsg.org
|
| Abstract:
In the past two decades evaluation capacity building (ECB) has become an increasingly widespread practice among evaluation practitioners. Since the 2000 AEA conference theme on evaluation capacity building, research on what constitutes evaluation capacity and capacity building has increased markedly. Yet, there has been little effort to consolidate approaches and models. In this panel the aim is twofold. First, contributors to this debate will convene and discuss the similarities and differences of their various evaluation capacity models and value the various contributions. Second, they will discuss how evaluation capacity models can and should inform the practice of evaluation capacity building. - And ultimately create value.
|
|
Evaluation Capacity Conceptualizing and Measuring an Ambiguous Construct
|
| Steffen Bohni, Ramboll Management Consulting, sni@r-m.com
|
| Sebastian Lemire, Ramboll Management Consulting, setl@r-m.com
|
|
The purpose of this presentation is to outline a model of evaluation capacity and tool to measure evaluation capacity using the evaluation capacity index (ECI) and discuss its potential use for evaluation. The point of departure is a recently published article in the American Journal of Evaluation (Nielsen, Lemire & Skov, 2011). First, the presenters will outline model of evaluation capacity. Second, the processes and findings in validating the ECI will be described. Third, the presenters will engage in a discussion with the panel about diagnosing, building and sustaining evaluation capacity.
|
|
|
Using the Evaluation Capacity Index (ECAI) to Conceptualize, Measure and Build Evaluation Capacity
|
| Tina Taylor-Ritzler, Dominican University, tritzler@dom.edu
|
| Yolanda Suarez-Balcazar, University of Illinois at Chicago, ysuarez@uic.edu
|
|
The purpose of this presentation is to describe the components and validation results of the Suarez-Balcazar, Taylor-Ritzler and Garcia-Iriarte (2010) evaluation capacity building model, compare and contrast it with other ECB models, and discuss the uses of the Evaluation Capacity Assessment Instrument (ECAI). First, the presenters will describe the model components: individual factors (awareness of the benefits of evaluation, motivation, and competence), organizational cultural and contextual factors (leadership, learning climate and resources), and evaluation capacity outcomes (use of evaluation processes and findings). Second, the presenters will describe the processes used to validate the ECAI, including mixed methods and confirmatory factor analysis and structural equation modeling. Third, the presenters will engage in discussion with other panel members to identify commonalities among and differences between current ECB models. Finally, the presenters will talk about uses of the ECAI in practice, including to diagnose, build and sustain evaluation capacity.
| |
|
The Evaluation Capacity Building Checklist: A Tool to Build Evaluation Capacity
|
| Jean A King, University of Minnesota, kingx004@umn.edu
|
|
The ECB checklist (Volkov & King, 2005) resulted from an empirical study of evaluation capacity building in three organizations (a museum, a large non-profit, and a school district). The resulting model has three components: (1) organizational context (including internal and external power hierarchies, administrative culture, and decision-making processes); (2) ECB structures, i.e., the mechanisms in the organization that enable the development of evaluation capacity (an explicit ECB plan, related infrastructure, socialization into the evaluation process, and peer learning structures); and (3) resources to support the development of evaluation capacity. King will explicate these components, including an update based on recent literature and an ongoing study of capacity building in the school district that was part of the original research.
| |