|
Session Title: National Evaluation of the Addiction Technology Transfer Center Network (ATTC): Findings and Observations From a Contextually Rich, Mixed Method Evaluation Study
|
|
Panel Session 682 to be held in PRESIDIO C on Friday, Nov 12, 4:30 PM to 6:00 PM
|
|
Sponsored by the Alcohol, Drug Abuse, and Mental Health TIG
|
| Chair(s): |
| Roy Gabriel, RMC Research Corporation, rgabriel@rmccorp.com
|
| Abstract:
The Substance Abuse and Mental Health Services Administration contracted with MANILA Consulting Group, Abt Associates, and RMC Research to conduct the first independent national evaluation of SAMHSA/Center for Substance Abuse Treatment’s Addiction Technology Transfer Center (ATTC) program since it was funded in 1993. The ATTC program supports the workforce that provides addictions treatment services to 23 million Americans each year through training, consultation and product development. The 3-year evaluation was preceded by a two-year evaluation design contract to develop an appropriate evaluation approach for this long standing, contextually rich program. The final design comprised 3 studies aiming to identify and build upon the successes of technology transfer and disseminate effective strategies. This session will include (1) an overview of the ATTC program and the importance of the evaluation; (2) presentations on each of three studies; and (3) a discussion of key findings, decisions and challenges related to this evaluation effort.
|
|
Substance Abuse and Mental Health Services Administration (SAMHSA), Center for Substance Abuse Treatment (CSAT) Overview of the ATTC Network and the Importance of Evaluation
|
| Deepa Avula, United States Department of Health and Human Services, deepa.avula@samhsa.hhs.gov
|
|
In complement with the SAMHSA/CSAT mission of promoting the quality and availability of community-based substance abuse treatment services, the ATTC Network was established to enhance clinical practice and improve the provision of addictions treatment. SAMHSA/CSAT leadership considers the ATTC Network the cornerstone of the national effort to infuse evidence-based practices in the publicly funded substance abuse treatment system. In 2008, SAMHSA/CSAT funded the first external evaluation of the ATTC Network to identify the successes of technology transfer efforts and build upon them in the future, share lessons learned across regions for the enhancement of all regions’ activities, and distinguish between region-specific and more cross-regional processes and outcomes. The evaluation was designed using a formative, highly participatory approach, which yielded significant buy-in from key SAMHSA/CSAT and ATTC staff regarding the process and the foci of the evaluation. This buy-in and participation was critical to the success of the evaluation.
|
|
|
Planning and Partnering Study: Overview and Key Findings
|
| Margaret Gwaltney, Abt Associates Inc, meg_gwaltney@abtassoc.com
|
| Cori Sheedy, Abt Associates Inc, cori_sheedy@abtassoc.com
|
|
The Planning and Partnering Study was designed to determine the processes and partnerships the ATTCs undertake to meet the needs of the addictions treatment workforce. The study utilized a mixed-method approach, including semi-structured individual and small-group interviews with staff from the 15 ATTCs, telephone interviews with more than 150 stakeholders, a survey of ATTCs’ Advisory Board members, and ATTC-reported data on event and activity characteristics. The presentation will describe how ATTCs are organizationally structured and the services they provide. Study findings demonstrate that ATTCs have leveraged their relationships with state agencies, educational institutions, provider organizations and other partners to provide training, technical assistance and other support to enhance the skills of the addiction workforce. Best practices for building and sustaining these relationships will be presented. The session will also discuss stakeholder-identified priorities for the treatment field and the extent to which ATTCs have been able to meet these needs.
| |
|
Customer Satisfaction and Benefit Study: Overview and Key Findings
|
| Andrea Muse, MANILA Consulting Group Inc, amuse@manilaconsulting.net
|
| Megan Cummings, MANILA Consulting Group Inc, mcummings@manilaconsulting.net
|
| Cliff Chamberlin, MANILA Consulting Group Inc, cchamberlin@manilaconsulting.net
|
| Richard Finkbiner, MANILA Consulting Group Inc, rfinkbiner@manilaconsulting.net
|
|
The Customer Satisfaction and Benefit Study was designed to assess the degree of satisfaction and benefit from ATTC activities and products among the range of stakeholders important to the ATTC Network. Drawing from respondent lists provided by each region, sampling within each region was employed with six types of stakeholders: Single State Agency personnel, senior leadership of State treatment provider associations, region-specific stakeholders (e.g., cultural leaders), addiction educators, treatment agency staff, and activity participants (e.g., clinical staff). The presentation will describe the perceived quality and benefits of ATTC services among the various stakeholders. Study findings indicate a significant variation in the level of satisfaction among individuals’ experiences with the ATTCs, including individuals’ motivations to seek services from the ATTCs, the services that were most useful, and how information was used by the ATTCs’ customers.
| |
|
Change in Practice Study: Overview and Key Findings
|
| Jeffrey Knudsen, RMC Research Corporation, jknudsen@rmccorp.com
|
|
The examination of change in practice is central to the identification of successes of technology transfer efforts. The Change in Practice (CIP) Study was the sole component of the national evaluation where behavioral changes in an ATTC target population were examined. The CIP Study addressed the outcomes of technology transfer efforts in three topical areas: clinical supervision, motivational interviewing, and assessment-based treatment planning. In particular, the CIP Study assessed the impact of technology transfer efforts on the implementation of discrete behavioral activities (i.e., critical actions) core to each practice. Study methodology utilized the Success Case Method approach, and all participants were surveyed, and those reporting the highest and lowest levels of post implementation success were interviewed to provide detailed accounts of their experiences. This session will review overall implementation levels of critical actions, illustrative examples of change, impacts of change, and facilitators and barriers to change.
| |
|
Key Findings, Decisions and Challenges Accompanying the National ATTC Evaluation
|
| Roy Gabriel, RMC Research Corporation, rgabriel@rmccorp.com
|
|
The national evaluation of the ATTC Network was aided enormously by the preceding two-year design phase, in which the evaluators worked closely with both the evaluand and the funding agency in formulating the most rigorous and responsive evaluation approach possible. During the evaluation itself, it became a challenge to maintain this participatory evaluation methodology, while assuring other stakeholders of the objectivity of the evaluation. Key findings emerging from the three interrelated studies comprising this evaluation included the incredible diversity of partnerships and collaborators established by the regional ATTCs; the major focus on specific cultural groups in some of the regions; the high degree of satisfaction with their products and services expressed by a wide range of ATTC customers; the characteristics associated with successful implementation of evidence-based practices following extensive training by the ATTCs; and the specific barriers reported by those who were not successful.
| |