|
Session Title: Environmental Strategy Implementation Fidelity Assessment: Exploring the Continuum between Standardized Measures and Case-specific Methodology
|
|
Panel Session 300 to be held in Capitol Ballroom Section 7 on Thursday, Nov 6, 1:40 PM to 3:10 PM
|
|
Sponsored by the Alcohol, Drug Abuse, and Mental Health TIG
|
| Chair(s): |
| Ann Landy,
Westat,
annlandy@westat.com
|
| Abstract:
Methods for assessing implementation fidelity of environmental strategies for substance abuse prevention are not fully developed. A major challenge is the tension between the need for consistent measurement across communities and the need for flexible measures that reflects community-specific adaptations and conditions. Panelists--members of a national cross-site evaluation team and state evaluators--will first discuss the development of a standardized tool to assess 21 evidence-based environmental prevention strategies, potential adaptations to account for specific community circumstances, and how fidelity rating data may augment current knowledge about the impact of environmental strategies. They will then discuss challenges in implementing uniform assessment procedures that adequately capture community-specific variation. Panel members will not recommend one method but will provide an opportunity for sharing different perspectives about the need for, challenges with, and experiences or lessons learned in attempting to measure or monitor implementation fidelity of environmental strategies across diverse multi-site 'systems.'
|
|
Developing a Standardized Tool to Assess Implementation Fidelity of Environmental Substance Abuse Strategies
|
| Kristianna Pettibone,
MayaTech Corporation,
kpettibone@mayatech.com
|
|
In developing a method for assessing implementation fidelity of environmental strategies for a cross-site evaluation that included over 400 communities, we had to take a standardized approach to ensure that comparable measures were used across all sites. Development of the tool was based on literature search to identify evidence-based environmental strategies. Core activities were identified from the literature and a rating scale was assigned to assess levels of fidelity. These measures were shared with external reviewers with experience implementing environmental strategies and their feedback was incorporated. Data from the fidelity measures will also be used to assess the relative importance of key implementation activities within an environmental strategy.
|
|
|
Measuring Fidelity to Best Practice Standards Across a Statewide Substance Abuse Prevention System
|
| Beth Welbes,
University of Illinois,
echamb@uiuc.edu
|
|
Since 2002, Illinois has contracted with U of I/CPRD to review, rate and comment on the extent to which its system of 125 community-based grantees implement prevention programs, policies and practices in line with clearly communicated standards of practice. Diverse types of environmental programs are implemented across the state from mentoring to public policy and enforcement efforts. CPRD has created a mechanism to annually rate fidelity to the standards of practice and provide customized feedback to each grantee regarding areas of strength and deficiency. This is one of the primary mechanisms used to monitor individual and system level performance. Challenges and lessons learned will be shared.
| |
|
The Effects of Multiple Scoring Approaches on Fidelity Measurement of Environmental Strategies
|
| Richard Cervantes,
Behavioral Assessment Inc,
bassessment@aol.com
|
|
Measuring environmental strategies and approaches, as well as outcomes related to these strategies can be complex and challenging. A variation in specific strategies, activities, and programs further complicates standardized measures of implementation fidelity. This presentation will highlight psychometric considerations in measuring fidelity across different forms and types of environmental approaches. Further, recommendations will be made related to the new SAMHSA/CSAP Users Guide Measuring SPF Strategies and Programs to improve reliability and validity of these measures.
| |
|
Reconciling Competing Needs: Options for Assessing Environmental Strategy Implementation Fidelity to Address Needs at Federal, State, and Local Levels
|
| Ann Landy,
Westat,
annlandy@westat.com
|
|
One of the primary challenges associated with a standardized approach for assessing implementation fidelity within a diverse, multi-site initiative, is facilitating the adaptation of the measures to meet the needs of individual communities. The degree to which the application of a standardized approach, using core activity ratings, can accommodate practitioner needs and suggestions for tailoring the process to individual communities while collecting comparable data across communities will be discussed.
| |