|
Session Title: Values and Perspectives on Evaluating Clinical and Translational Science
|
|
Panel Session 473 to be held in El Capitan A on Thursday, Nov 3, 4:30 PM to 6:00 PM
|
|
Sponsored by the Health Evaluation TIG
|
| Chair(s): |
| Cathleen Kane, Weill Cornell Clinical Translational Science Center, cmk42@cornell.edu
|
| Discussant(s):
|
| Donald Yarbrough, University of Iowa, d-yarbrough@uiowa.edu
|
| Abstract:
Clinical and translational science is at the forefront of biomedical research and practice in the 21st century. The NIH-funded Clinical and Translational Science Awards (CTSAs) are the largest initiative at NIH and the 55 center grant evaluation teams constitute a unique national field laboratory in the evaluation of biomedical research and practice. The four presentations in this panel address: the value of evaluation to key stakeholders; the values that shape qualitative and mixed methods work in assessing biomedical research and practice; whose values determine which outcomes are assessed and how this plays out in practice; and how different values shape the cross-center national evaluation. The panel will present four different evaluation studies and discuss their implications both practically in the context of the CTSAs and theoretically in terms of the values that influence this work.
|
|
Assessing the Perceived Value of Evaluation at a Clinical and Translational Science Award (CTSA) Institution
|
| Christine Weston, Johns Hopkins University, cweston@jhsph.edu
|
|
One of the major goals of internal evaluation is to increase the evaluation capacity of the organization. In order to increase capacity, evaluation first needs to be appreciated and valued. Unfortunately, evaluation is often undervalued, misunderstood, or dismissed. As a result, the efforts of internal evaluators are often met with resistance. The purpose of our study is to investigate the degree to which Program Evaluation is valued at our institution. We aim to address the following questions: 1) To what extent do our evaluation stakeholders understand the a) role of evaluation in the organization, b) the benefits of evaluation to the organization, and c) value of evaluation to organizational learning? 2) What are the misconceptions about evaluation, and how can they be corrected? What are the misconceptions about evaluation, and how can they be corrected? As a result of our assessment we aim to develop a targeted intervention to increase the perceived value of program evaluation in our organization.
|
|
|
Beyond Telling Their Stories: The Added Value of Qualitative Research to CTSA Evaluations
|
| Nancy Bates, University of Illinois, Chicago, nbates@uic.edu
|
| Jessica Hyink, University of Illinois, Chicago, jessicah@srl.uic.edu
|
| Timothy Johnson, University of Illinois, Chicago, tjohnson@srl.uic.edu
|
| Mary Feeney, University of Illinois, Chicago, mkfeeney@uic.edu
|
| Megan Haller, University of Illinois, Chicago, mhalle1@uic.edu
|
| Priyanka Nasa, University of Illinois, Chicago, pnasa2@uic.edu
|
| Linda Owens, University of Illinois, Chicago, lindao@srl.uic.edu
|
| Eric Welch, University of Illinois, Chicago, ewwelch@uic.edu
|
|
In CTSA evaluations, it is important to tell the success stories of researchers who have transformed their research to clinical and translational work, using CTSA resources. Beyond these stories, however, qualitative methods can bring added value to CTSA evaluations by yielding important findings that cannot be learned through quantitative methods alone. This presentation will introduce qualitative data collection and analysis methods used to understand the process, implementation and outcomes of each CTSA's core group's Specific Aims and Logic Models. Integration with quantitative data will be shown. Examples and lessons learned from the University of Illinois at Chicago Center for Clinical and Translational Science will be reported.
| |
|
Values and the Selection of Outcomes for Evaluating CTSAs
|
| D Paul Moberg, University of Wisconsin, Madison, dpmoberg@wisc.edu
|
| Janice Hogle, University of Wisconsin, Madison, jhogle@wisc.edu
|
| Christina Hower, University of Wisconsin, Madison, cjhower@wisc.edu
|
|
Large, expensive and complex programs, such as the CTSAs, have numerous constituents and stakeholders, resulting in a multiplicity of potential outcomes that could be measured quantitatively or assessed qualitatively in an evaluation. Selection of outcomes to be given priority reflects both the expressed intent of the funders and the values of the stakeholders. This presentation will explore those valued outcomes for CTSA's, and their implications for what we should be measuring, assessing and documenting. Data from qualitative interviews of key stakeholders in the University of Wisconsin CTSA, informed by interaction with other CTSA evaluators and written documentation, will be used to explicate the local and national range of valued outcomes. Theoretical analysis will conceptually situate valued outcomes within cultural, professional, political and community contexts of the stakeholders in the CTSA enterprise
| |
|
The National Evaluation of the Clinical and Translational Science Awards Initiative: Challenges and Strategies for Addressing Them
|
| Joy Frechtling, Westat, joyfrechtling@westat.com
|
| Meryl Sufian, National Institutes of Health, sufianm@mail.nih.gov
|
|
The National Evaluation of the Clinical and Translational Science Awards (CTSA) Initiative is designed to provide initial information on the progress of the program, examining both the accomplishments of the first four CTSA cohorts of academic medical centers and the Consortium overall that they comprise. Using a mixture of surveys, interviews, field visits, bibliometrics, and expert review, the evaluation is designed to gather preliminary data on the impacts of the program on the clinical and translational workforce, the development of collaborations and collaborative research, and the quality of clinical and translational science. The presentation will include an overview of the national evaluation and some of the strategies that the evaluation team and NCRR are using to build an accurate picture of this multi-faceted, multi-level initiative. Lessons for both future evaluations of the CTSA and of other similarly complex programs will be offered.
| |