|
Session Title: Assessing Implementation Fidelity of Substance Abuse Prevention Environmental Change Strategies: Lessons Learned From the Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Prevention (CSAP), Strategic Prevention Framework State Incentive Grant (SPF-SIG), and National Cross-site Evaluation
|
|
Panel Session 602 to be held in PRESIDIO C on Friday, Nov 12, 1:40 PM to 3:10 PM
|
|
Sponsored by the Alcohol, Drug Abuse, and Mental Health TIG
|
| Chair(s): |
| Kristi Pettibone, The MayaTech Corporation, kpettibone@mayatech.com
|
| Abstract:
States and jurisdictions,funded through the Strategic Prevention Framework State Incentive Grant (SPF SIG) initiative, were required to fund communities to implement a range of substance abuse prevention interventions, including interventions designed to change the environment in which substance abuse occurs. The SPF SIG cross-site evaluation team developed implementation fidelity (IF) measures for environmental change strategies to assist grantees in implementing these strategies and evaluating their effectiveness. Three presentations comprise this panel. The first presentation describes the need for, and development of, measures for assessing IF of environmental change strategies, the data collection process, and the characteristics of data submitted by SPF SIG grantees. The second presentation discusses the preliminary assessment of the connection between the IF data and the outcomes or accomplishments associated with environmental change strategies. The third presentation describes a grantee’s adaptation of the assessment process to accommodate state-level and community-level conditions.
|
|
A Process for Assessing Implementaion Fidelity for Substance Abuse Prevention Environmental Change Strategies
|
| Ann Landy, Westat, annlandy@westat.com
|
| Elisabeth Cook, Westat, elisabethcook@westat.com
|
|
Communities funded through state SPF SIG grants were required to implement participant-based prevention interventions and environmental change strategies, in addition to the five SPF steps, to address key consumption and consequence problems identified through analyses of epidemiology data. To help states assess community-level performance, the SPF SIG national cross-site evaluation team developed a series of implementation fidelity (IF) rating scales for environmental strategies, participant-based interventions, and the SPF steps. Sixteen of 26 Cohort 1 and Cohort 2 SPF SIG grantees used the IF assessment tool, or an adaptation of the tool, on a voluntary basis, to collect and submit implementation fidelity data. This presentation specifically describes the process for identifying core activities for 21 environmental change strategies, construction of the environmental change strategy IF rating scales, and state data collection procedures. Presenters will report a summary of the environmental change strategy IF data submitted by the 16 SPF SIG grantees.
|
|
|
Exploring the Relationship Between Implementation Fidelity and Process and Outcome Data to Evaluate the Effectiveness of Environmental Change Strategies
|
| Kristi Pettibone, The MayaTech Corporation, kpettibone@mayatech.com
|
|
Sixteen of 26 SPF SIG states participated in a voluntary data collection activity designed to assess the relationship between implementation fidelity of substance abuse prevention environmental change strategies and process and outcome measures. Process measures include counts of the number of environmental change strategies implemented and the number of activities conducted within these strategies obtained through a standardized, web-based tool that each state's funded community partners completed twice a year. Outcome data include measures of substance use/abuse behaviors and consequences such as, motor vehicle crashes, crime and alcohol-, tobacco- or drug-related mortality. This presentation will report summaries of IF rating data for environmental change strategies, submitted by the 16 grantees, and the process data as well as results of an initial assessment of the relationships among the IF data, process data, and outcome data.
| |
|
Adaptation of the Strategic Prevention Framework State Incentive Grant (SPF SIG) Implementation Fidelity Assessment User's Guide in Texas
|
| Adrian Reyes, Behavioral Assessment Inc, aistrategies@aol.com
|
|
The SPF SIG Implementation Fidelity (IF) Assessment Guide—developed by the national cross-site evaluation team--was distributed to each of the 26 Cohort 1 and Cohort 2 grantee evaluation teams. The Texas SPF SIG evaluator, Behavioral Assessment, Inc., reviewed the Guide and determined that the recommended approach for collecting rating data should be adapted to accommodate the needs of the Texas funded communities. This presentation describes the conditions that prompted the need for adaptation, the on-line tool that was developed for collecting the IF ratings, and the use of the tool for rating environmental change strategy implementation. The presenter also will describe the training and technical assistance provided to the local evaluators, through webinars and site visits, for using the on-line tool and maintaining inter-rater reliability. Environmental strategy IF data obtained from six communities will be presented and advantages and disadvantages of using the system will be discussed.
| |