|
Enhancing Evaluation Quality and Use for Dynamic Organizations: Lessons Learned from a Developmental Evaluation of a Competency-based Medical Education Innovation
|
| Presenter(s):
|
| Cheryl Poth, University of Alberta, cpoth@ualberta.ca
|
| Shelley Ross, University of Alberta, shelleyross@med.ualberta.ca
|
| Rebecca Georgis, University of Alberta, georgis@ualberta.ca
|
| Mike Donoff, University of Alberta, mike.donoff@ualberta.ca
|
| Paul Humphries, University of Alberta, phumphries@ualberta.ca
|
| Ivan Steiner, University of Alberta, ivan.steiner@ualberta.ca
|
| Abstract:
This paper reports the lessons learned from a two-year developmental evaluation of a medical education innovation, Competency Based Achievement System (CBAS). Developmental evaluation has been described as useful to support innovation implementation and organizational decision-making. However, few empirical studies have documented the influence of a developmental approach on evaluation quality and use during the pilot implementation of an innovation. The study used a series of four focus groups with the organizational team to capture the perspectives of the five medical educators and the evaluator during the evaluation. Findings from the analysis using diffusion theory revealed that each evaluation team member played a unique role that led to the identification of seven guiding CBAS principles and that building confidence in the validity and reliability of the findings was crucial for informing implementation decisions. Implications for developmental evaluation practice include effective strategies for building organizational relationships and mitigating emerging challenges.
|
|
Sustainable Quality Evaluation: Evaluating Continuous Quality Improvement Processes
|
| Presenter(s):
|
| Matthew Galen, Claremont Graduate University, matthew.galen@cgu.edu
|
| Deborah Grodzicki, University of California, Los Angeles, dgrodzicki@gmail.com
|
| Abstract:
This paper will explore a theoretically-grounded, practical framework for implementing an evaluation of a Continuous Quality Improvement (CQI) program. CQI is an intra-organizational, customer-oriented approach to evaluating systems of production and service delivery. A successful evaluation of CQI programs requires a broad set of evaluation tools drawing from program evaluation theories as well as the conceptual thinking surrounding meta-evaluations. The paper will include resources for relevant CQI evaluation design and methodology, as well as techniques for involving program stakeholders in a collaborative effort to optimize their CQI process. A proposed evaluation of county-wide Emergency Medical Services CQI process will be presented to illustrate this process.This paper will explore a theoretically-grounded, practical framework for implementing an evaluation of a Continuous Quality Improvement (CQI) program. CQI is an intra-organizational, customer-oriented approach to evaluating systems of production and service delivery. A successful evaluation of CQI programs requires a broad set of evaluation tools drawing from program evaluation theories as well as the conceptual thinking surrounding meta-evaluations. The paper will include resources for relevant CQI evaluation design and methodology, as well as techniques for involving program stakeholders in a collaborative effort to optimize their CQI process. A proposed evaluation of county-wide Emergency Medical Services CQI process will be presented to illustrate this process.
|
|
Building Evaluation Culture and Evaluation Quality in Brazil
|
| Presenter(s):
|
| Daniel Brandão, Fonte Institute, daniel@fonte.org.br
|
| Martina Otero, Fonte Institute, martina@fonte.org.br
|
| Abstract:
The relevance of program evaluation is recognized by all players engaged in social initiatives in Brazil. Despite this recognition the country has a fragile structure to promote evaluation quality, such as lack of knowledge about the evaluation field, rare courses targeted to social leaders and evaluators or concrete spaces to promote the exchange of experiences among people interested in this area.
This paper aims to present the efforts and strategies developed in a partnership between a private foundation and a civil society organization to strengthen the evaluation culture and quality in the country. Project strategies, results, and challenges will be discussed and the relationship between evaluation culture and evaluation quality will be addressed.
|
| | |