|
Session Title: Measuring Evaluation Capacity to Enhance Quality of Evaluation Capacity Building (ECB) Activities in Federal Public Health and Systemic Change Programs
|
|
Panel Session 720 to be held in PRESIDIO B on Saturday, Nov 13, 8:00 AM to 9:30 AM
|
|
Sponsored by the Organizational Learning and Evaluation Capacity Building TIG
and the Health Evaluation TIG
|
| Chair(s): |
| Maureen Wilce, Centers for Disease Control and Prevention, muw9@cdc.gov
|
| Discussant(s):
|
| Hallie Preskill, FSG Social Impact Advisors, hallie.preskill@fsg-impact.org
|
| Abstract:
Measuring evaluation capacity among program evaluation staff can inform the development of high quality evaluation capacity building (ECB) activities. In this panel, speakers will describe the development, adaptation and application of a tool to measure evaluation capacity among program staff in three public health / social service programs. Decentralized programs must build capacity for internal evaluation at all organizational levels, including national, state, and local, as applicable. Panelists will share details of the development of a tool to measure individual and organizational capacity for evaluation among state grantees and the subsequent adaptation of the instrument to assess evaluation capacity among evaluation staff at federal and local levels. Finally, panelists will discuss how they are using evaluation capacity assessment information at multiple organizational levels to tailor ECB activities to better meet needs.
|
|
Measuring Evaluation Capacity: Development of a Tool
|
| Carlyn Orians, Battelle Memorial Institute, orians@battelle.org
|
|
This presentation describes the development of a tool to measure evaluation capacity among local evaluators to inform the development of ECB activities. First, we review the definition and conceptual basis for ECB. Then we discuss the specific capacities that ECB activities may target and the specific constructs that can usefully be measured at baseline to inform these activities. Then we describe the development and content of an evaluation capacity measurement tool designed to meet these needs. The tool addresses individual knowledge, experience, skills, and attitudes related to evaluation, as well as organizational practices and readiness that support evaluation. Initial pretest results are also presented.
|
|
|
Building Evaluation Capacity for Asthma Control Programs: Assessing What You Don’t Know
|
| Kari Cruz, Centers for Disease Control and Prevention, hgv3@cdc.gov
|
| Maureen Wilce, Centers for Disease Control and Prevention, muw9@cdc.gov
|
|
Building evaluation capacity in an organization is a complex, multi-faceted process. CDC’s National Asthma Control Program has embarked on a process to build evaluation capacity among its national program and funded partners to enable them to conduct useful and feasible program evaluations. This presentation will share experiences in adapting and using a new self-assessment tool to strategically assess CDC’s capacity to provide high quality technical assistance and identify programmatic evaluation needs and organizational needs for evaluation capacity development. The self-assessment tool is designed to support the professional development of evaluators practicing at the state, territory and national levels by identifying strengths and needs and by stimulating reflection, both individually and with fellow evaluators. Through this process CDC can tailor its ECB activities, identify opportunities for evaluators to advance their evaluation skills, and recommend additional evaluation resources that enhance evaluation practice.
| |
|
Building Evaluation Capacity for State Oral Health Programs: Using Data to Plan and Focus Initiatives
|
| Cassandra Martin, Centers for Disease Control and Prevention, bkx9@cdc.gov
|
|
Developing and enhancing the evaluation capacity of state oral health programs is an essential and multifaceted component of the CDC Division of Oral Health (DOH) Infrastructure and Capacity Development program. To support grantee evaluation efforts, the DOH evaluation team undertakes evaluation capacity building (ECB) activities utilizing workshops, trainings, technical assistance and resources. Yet, the lack of systematic and ongoing measurement of evaluation capacity at the state and federal level proves to be challenging in understanding the growth in evaluation competency. Thus, an evaluation needs assessment was conducted among the evaluation team and grantees. Results from the assessment pinpointed the staff development needs for evaluation as well as identified the critical components to build a pathway for implementing ECB activities. The presenters will illustrate the experience with conducting the evaluation needs assessment, and unveil the evaluation capacity building plan for state oral health programs.
| |
|
Targeting Evaluation Capacity Building (ECB) Needs in State and Tribal Child Welfare Systemic Change Projects
|
| Julie Morales, University of Denver, julie.morales@du.edu
|
|
In 2008, the Administration of Children and Families expanded their Training and Technical Assistance Network by creating five regional Child Welfare Implementation Centers charged with working with State and Tribal child welfare systems to implement in-depth, long-term systemic change projects aimed at improving outcomes for children and families. As evaluation lead for the Mountains and Plains Child Welfare Implementation Center (MPCWIC), which serves regions 6 (New Mexico, Texas, Oklahoma, Arizona and Louisiana) and 8 (Montana, North Dakota, Wyoming, South Dakota, Utah and Colorado), Julie Morales works with State/Tribal leadership to develop their project evaluation plans to assess the overall effectiveness of their system change projects. In this role, the MPCWIC evaluation team assesses evaluation capacity and provides technical assistance on needed ECB. Data on the first cohort of MPCWIC project sites will be presented and how this information is used to inform targeted evaluation capacity building TA will be discussed.
| |