|
A Systematic Map of the Empirical Literature on Participatory Evaluation in Relation to Evaluation Use: Toward Reality Testing
|
| Presenter(s):
|
| Pierre-Marc Daigneault, University of Laval, pierre-marc.daigneault.1@ulaval.ca
|
| Abstract:
This presentation studies the relationship between participation and evaluation use through a scoping study of the empirical literature. A scoping study is “a technique to ‘map’ relevant literature in the field of interest” (Arksey & O’Malley, 2005, p. 20) that stands somewhere between traditional literature reviews and systematic reviews. It uses systematic and transparent techniques to search for studies, determine their relevance and extract data, but it does not assess study quality or synthesize their findings. The results will be addressed in terms of relevant dimensions such as the number of studies identified, research questions and methodology, conceptualization and measurement of variables, context (country, policy sector, level of analysis) and findings. The objectives are twofold: (1) to catch a glimpse of the main findings on the relationship between PE and evaluation use, and (2) to identify gaps in the empirical literature in order to guide future inquiry on evaluation.
|
|
An Overview of the Empirical Studies of Stakeholder Involvement in Program Evaluation
|
| Presenter(s):
|
| Landry Fukunaga, University of Hawaii, lfukunag@hawaii.edu
|
| Paul Brandon, University of Hawaii, brandon@hawaii.edu
|
| Abstract:
The interaction of evaluators with program stakeholders for the purpose of improving evaluations has a long history in program evaluation. Evaluators have reported involving stakeholders to enhance the use of evaluation findings, enhance evaluation validity, improve stakeholders’ evaluation capacity, help ensure social justice, and empower stakeholders. Broad summaries of this participation have been reported, but to our knowledge, no comprehensive overview of published studies that report stakeholder involvement has been published. We address this deficit in this paper in a review of 181 studies that we identified in a comprehensive search for studies about stakeholders in evaluation published since 1985. We summarize the purposes, methods, and effects of stakeholder involvement in the studies, the professions or disciplines targeted, and the methods for collecting data about stakeholder involvement. Finally, we arrive at conclusions about the strength of the evidence about the breadth and depth of stakeholder involvement.
|
|
An Investigation of the Relationship Between Participatory Evaluation and Use of Evaluation in Three Multi-site Evaluations
|
| Presenter(s):
|
| Denise Roseland, University of Minnesota, rose0613@umn.edu
|
| Abstract:
This dissertation research describes an investigation that explores the nature of the relationship between participation in evaluation and the use of evaluation findings and processes within three large-scale multisite evaluations. The purpose of this study is to test whether assumptions and theories about participation translate into evaluation use in the same ways as seen in single evaluation sites. Using canonical correlation analysis and a collection of 20 interviews, this paper describes and tests the relationship between these two critical conceptual powerhouses in evaluation. Using data that were collected as a part of the NSF-funded research “Beyond Evaluation Use” (Lawrenz & King, 2009), this study found that some theories and beliefs about participatory evaluation contribute to use and influence in similar ways as single-site evaluations. The differences identified in this research highlight potential planning and implementation considerations that might allow multi-site evaluators to enhance use and influence of multi-site evaluations.
|
|
Boldly Going Inward, Outward, and Forward: Studying (How to Study) the Intersection of Theory and Practice in Evaluation
|
| Presenter(s):
|
| Jeehae Ahn, University of Illinois at Urbana-Champaign, jahn1@illinois.edu
|
| Abstract:
As a set of principles, values, and assumptions about what evaluation is, what it can and should accomplish, as well as what it means to be an evaluator, evaluation theory can serve as a powerful guiding tool that can usefully inform important dimensions and dynamics of one's evaluation practice--from conceptualizing particular evaluation purposes to selecting certain methodological procedures of data collection, analyses, interpretation and reporting. This paper explores ways to systematically “study” this organic conjunction between evaluation theory and practice in “real time” and on the ground as integral part of one’s evaluation work, as well as in connection to evaluation quality. Interweaving stories and examples along the way, the paper discusses specific built-in processes of examining the structure, process and character of how one’s theory converses with one’s practice (and vice versa) with the aspired end result being enriched evaluation theory and improved evaluation practice.
|
| | | |