|
Session Title: Translational Health Research: Implications for Evaluation Theory From a Practice Imperative
|
|
Panel Session 619 to be held in the Granite Room Section B on Friday, Nov 7, 1:35 PM to 3:05 PM
|
|
Sponsored by the Health Evaluation TIG
|
| Chair(s): |
| James Dearing,
Kaiser Permanente,
james.w.dearing@kp.org
|
| Abstract:
In theorizing about the evaluation of social and health programs, evaluation theorists have tended to focus on issues of internal validity (effect of program components on outcomes) more so than on external validity (replication of program effects across sites) or program diffusion (broad spread of a program across many practice sites). Current Federal emphasis on late-stage translation of research results to affect behavior in practice settings is an opportunity for evaluation researchers to prioritize the study of program external validity as advocated by evaluation theorists (Cronbach, Cook, and Shadish) and program diffusion as epitomized by Everett Rogers. Here, we focus on challenges of an external validity or diffusion design perspective, key variables of interest to evaluators and their clients, and introduce tools that can be used to formatively assess program potential for external validity and diffusion. We provide examples of translational research from the nation's largest nonprofit integrated healthcare system.
|
|
New Foci for Evaluation Theory and Practice: External Validity and Diffusion Variables and Measures
|
| James Dearing,
Kaiser Permanente,
james.w.dearing@kp.org
|
|
The translation of evidence-based practices and programs from research-based interventions to implementation as practice improvements in organizations has been long-sought but rarely studied by evaluators. Typically, research interventions are evaluated for internal validity - the extent to which they achieve intended effects - but not external validity or diffusion. And beyond research interventions, what practitioners do often bears little resemblance to what was validated by researchers, even when practitioners believe that they have implemented an evidence-based program. This presentation focuses on variables and measures of key concepts that can be operationalized to represent external validity (including setting heterogeneity, fidelity, adaptation, implementation support, sustainability, and institutionalization) and diffusion (including innovation attributes, social influence, reach, rate, and need).
|
|
|
Using Theory to Guide Evaluation: Evaluating a Practice Based Childhood Obesity Program Through a Randomized Control Trial
|
| Jo Ann Shoup,
Kaiser Permanente,
jo.ann.shoup@kp.org
|
| Paul A Estabrooks,
Virginia Polytechnic Institute and State University,
estabrkp@vt.edu
|
|
Given the significantly increasing rates of obesity in children, evaluation of existing practice-based child obesity programs and enhanced program innovations is precedence.
We evaluated the relative effectiveness of Family Connections (FC) Workbook, FC-Group, or FC Interactive Voice Response (IVR) counseling interventions to support parents of overweight children in changing the home environment. Parent and child (between 8 and 12 years old) dyads (n = 220) were randomly assigned to one of the three FC interventions. Child BMI and other variables were assessed at baseline, 6, and 12 months post randomization. Children whose parents completed at least 6 of the 10 FC-IVR counseling calls decreased BMI z-scores to a significantly greater extent than children in the FC-Workbook or Group conditions at both 6 (p <.05) and 12 months (p <.01). This trial demonstrated that automated telephone counseling can support parents of overweight children to reduce the extent to which children were overweight.
| |
|
Using Theory to Design and Evaluate a Practical and Generalizable Smoking Reduction Study
|
| Bridget Gaglio,
Kaiser Permanente,
bridget.gaglio@kp.org
|
| Russ Glasgow,
Kaiser Permanente,
russ@re-aim.net@internet
|
|
Our research group has developed a smoking reduction intervention that was designed to be broadly applicable and is integrated into other smoking modification options in a large managed care organization. The focus of the program was on behavioral approaches to reduce the number of cigarettes smoked, and not on the use of alternative tobacco products. A social-ecological theoretical approach, including risk perceptions, self-efficacy, problem solving, and environmental support, was used for intervention development. We used the RE-AIM framework to evaluate this intervention. Results will include the reach of this smoking reduction program offered in conjunction with other smoking services of a large HMO, to determine the effectiveness of the program (short-term and long-term outcomes) relative to an enhanced usual care condition in a practical randomized controlled trial, and overall implementation of the intervention components. While results of this study are encouraging additional research is indicated to evaluate public health impact.
| |