| Session Title: Small Wins are Winsome: Aggregating Learning From Small Evaluations Into Systems Change |
| Multipaper Session 401 to be held in Royale Conference Foyer on Thursday, November 8, 1:55 PM to 3:25 PM |
| Sponsored by the Alcohol, Drug Abuse, and Mental Health TIG |
| Chair(s): |
| Paul Florin, University of Rhode Island, pflorin@mail.uri.edu |
| Discussant(s): |
| John Stevenson, University of Rhode Island, jsteve@uri.edu |
| Abstract: What consequences can small and focused evaluation studies produce within discrete elements of a state substance abuse prevention system? Do consequences from each evaluation study stand alone or can they together aggregate and amplify into broader systems change? Each paper in this multi-paper session describes a small and focused evaluation whose results were fed back to stakeholders in a state prevention system. Each paper describes how results produced initial "deviations" from normal operations at a particular ecological level (e.g. program, community or state level). The discussant, drawing on more than twenty years of evaluation practice within the state system under consideration, reflects on both the potential and limits of aggregating evaluations across projects over time to produce "tipping points" for systems change. The conclusions from the papers and the discussant's reflections can inform the practice (and perhaps the patience) of those who seek to use evaluation results to change systems. |
| Insights Into Implementation of Evidence-based Programs: Lessons Learned Through Focus Groups |
| Thomas Sawyer, University of Rhode Island, tsaw5413@postoffice.uri.edu |
| Evidence-based programs for substance abuse prevention present special opportunities and challenges. In our role as the statewide evaluation team for a three-year evidence-based demonstration project in Rhode Island, we held monthly meetings with staff liaisons from each of 8 local sites. Valuable learning took place at these meetings that led us to conduct three specific focus groups to gain further insight into start-up and implementation issues with respect to delivering such programs. This paper reports on these one-hour focus groups with agency staff and directors at the final liaison meeting of the second year of the project. The results from the focus groups were fed back to state decision makers who were continuing to promote local implementation of evidence-based programs and were disseminated to local agencies. The 'small win' in this paper was change in implementation practice at the program level. |
| Lessons Learned from the Evaluation of Environmental Strategies in Community Interventions |
| Jessica Nargiso, University of Rhode Island, jnargiso@mail.uri.edu |
| Historically, substance abuse prevention efforts have targeted individual factors. More recently, the importance of addressing the social and environmental conditions that facilitate substance use has been recognized. The use of environmental strategies, such as media campaigns, policy change, and enforcement enhancement, introduce new challenges to evaluating the effectiveness of these strategies within communities. As part of the evaluation of the State Incentive Grant (SIG) in Rhode Island, a monthly interview is conducted with a representative from each SIG community to monitor actions taken and outcomes produced by environmental strategies. This paper will focus on what was learned from the evaluation, including methodological challenges and the importance of training and technical assistance. The small win in this paper is how these lessons informed changes implemented at the state level delivered through a training and technical assistance system serving fourteen funded community coalitions. |
| Lessons Learned from the Evaluation of Prevention Training |
| Crystelle Egan, University of Rhode Island, crystelleann@yahoo.com |
| Formal training is an important method of disseminating evidence-based practices to practitioners. However, research indicates that trainees often fail to transfer learned material to their jobs. In order to understand how trainees applied principles from a 21-hour course in the 5-step Getting Prevention Results (GPR) prevention planning and implementation framework, a precursor to CSAP's Strategic Prevention Framework, a qualitative evaluation was conducted with trainees one year post-training. Factors facilitating and impeding the application of GPR principles were explored. Key findings of the study will be addressed in this paper. Results from the study were reported to the developers of a newly-established statewide training and technical assistance system. The 'small win' in this paper was the incorporation of recommendations for enhancing training transfer into statewide prevention training. |