Return to search form  

Session Title: Building and Assessing Capacity for Evaluation: Creating Communities of Learners Among Service Providers

Panel Session 867 to be held in Hanover Suite B on Saturday, November 10, 3:30 PM to 5:00 PM

Sponsored by the Collaborative, Participatory & Empowerment Evaluation TIG

Chair(s):

Tina payday loan  Taylor-Ritzler payday loan ,  University of Illinois, Chicago,  tritzler@uic.edu

Discussant(s):

David payday loan  Fetterman payday loan ,  Stanford University,  profdavidf@yahoo.com

Abstract: Community-based organizations are currently experiencing pressure to learn about evaluation and conduct their own evaluations. Some are able to meet these demands through partnerships with academic institutions designed to build capacity for evaluation utilizing empowerment and participatory approaches. Although there is literature available on evaluation capacity building, much is needed in terms of understanding its conceptualization, the process of building it and how to measure it. In this session, several researchers will present their work with a variety of community-based organizations in creating capacity for evaluation. First, we will present an ecological, contextual and interactive framework of evaluation capacity building that integrates models from the evaluation literature. Second, we will describe methods and strategies in measuring capacity building and we will discuss how to create learning communities with agency staff. Third, we will provide exemplars and discuss challenges encounter when doing this work and implications for the field of evaluation. Fourth, we will discuss evaluation capacity building strategies (ECB) used and how they have been evaluated based on a review of research on ECB. Finally, we will hear commentaries from a prominent researcher in the field: David Fetterman.

Building Capacity for Evaluation Among Service Providers: Conceptual Framework and Exemplar Yolanda payday loan  Suarez-Balcazar payday loan ,  University of Illinois, Chicago,  ysuarez@uic.edu Based on the work we have been doing at the Center for Capacity Building for Minorities with Disabilities Research, we propose a contextual framework of capacity for evaluation. A contextual and interactive model suggests a dynamic interplay between person factors and organizational factors. The person or group factors are exemplified by agency staff and/or program implementers while organizational factors speak for organizational policies, organizational culture and support systems that create an environment that facilitates capacity for evaluation. The framework assumes interplay between personal factors and organizational factors. As such, a CBO staff member may be willing and ready to learn how to evaluate a program he/she implements but lacks organizational support to do so in the form of lack of allocated time and resources. Capacity for evaluation can be created and facilitated at the individual level. Here, we are referring to the staff member or members who implement agency programs, are in direct contact with participants, and are experiencing tremendous pressure to document what they do and to produce tangible outcomes. We will discuss individual factors such as personal readiness, level of competence and experience, and individual leadership. The environment, policies, procedures and culture of the organization may be more or less facilitative of building capacity for evaluation in individual staff and the organization as a whole. The presenters will also discuss several factors at the organization level that can facilitate the process of building capacity: including organizational readiness, organizational resources and support allocated to evaluation, organizational leadership, organizational culture, organizational capacity to mainstream evaluation practices, organizational capacity to utilize findings and develop practices that sustain evaluation capacity, and organizational capacity to seek funding for their programs. We will also discuss implications for the art and the science of evaluation.

Measuring Evaluation Capacity: Methodologies and Instruments Tina payday loan  Taylor-Ritzler payday loan ,  University of Illinois, Chicago,  tritzler@uic.edu Although there is a large literature on evaluation capacity building, it lacks specificity on issues of measurement and assessment of evaluation capacity. Most have looked only at evaluation products agencies generate (reports to funders) and satisfaction with training. We will present our multiple method system for assessing and measuring evaluation capacity. In this session, we will present the work being conducted nationally by the Center for Capacity Building on Minorities with Disabilities Research. We will describe in detail the instruments and procedures we use and challenges we encounter when measuring evaluation capacity building with organizations serving ethnic minorities with disabilities. We will share data drawn from multiple case study examples. Finally, our discussants will share their perspective on the contribution of our work to scholarship on evaluation capacity building.

Using Collaborative Evaluation as a Strategy for Evaluation Capacity Building: First 5 Los Angeles' Quality Care Initiative Rita payday loan  O'Sullivan payday loan ,  University of North Carolina, Chapel Hill,  ritao@email.unc.edu Collaborative Evaluation (O'sullivan, 2004) uses an approach to evaluation, which results in enhanced evaluation capacity building among key stakeholders. Evaluation Assessment and Policy Connections (EvAP) at the University of North Carolina at Chapel Hill worked collaboratively with First 5 Los Angeles staff and its 53 Childcare grantees to design a 30-month evaluation that would provide process, outcome, and policy information about the initiative. The evaluation activities also addressed enhancing the evaluation capacity of First 5 grantees, staff, partners, and Commissioners. Collaborative evaluation engages key program stakeholders actively in the evaluation process. Unlike distanced evaluation, where evaluators have little or no contact with program staff, collaborative evaluation deliberately seeks involvement from all program stakeholders during all stages of the evaluation. A collaborative stance can strengthen evaluation results and increase utilization of evaluation findings. Additionally, programs participating in collaborative evaluations develop an enhanced capacity to consume and conduct evaluations, while evaluators gain a better understanding of the program. The collaborative evaluation approach assumes that evaluation expertise within programs is developmental; and thus, the degree of collaboration must vary by the nature and readiness of the program. Evaluations completed with this collaborative approach have yielded improved evaluation capacity as measured by data quality, report writing, and evaluation use with program in the areas of education, social services, and health; the presenter also has found that collaborative evaluation may increase the resources available to the evaluation. This presentation will report how the evaluation contributed to the capacity building of the 53 grantees the majority of which were community based organizations.

A Review of Research on Evaluation Capacity Building Strategies Jennifer payday loan  Duffy payday loan ,  University of South Carolina,  jenduffy@sc.edu The growing literature on evaluation capacity building is one resource for learning more about what evaluation capacity building looks like in the field and what evidence there is for the success of these strategies. We will present findings from a review of empirical research on evaluation capacity building. The strategies for building evaluation capacity that are identified in this research will be described, and the methods used to evaluate these strategies will be discussed. We will highlight the evidence for successful strategies and limitations of the existing research. Questions for future research will be identified, with a focus on identifying successful strategies for building evaluation capacity.