|
Analysis of Strategic Plans to Assess Fidelity to a Model-based Planning Process
|
| Presenter(s):
|
| Melissa Gutierrez Barrett,
Westat,
melissagutierrez@westat.com
|
| Ann Landy,
Westat,
annlandy@westat.com
|
| Robert Orwin,
Westat,
robertorwin@westat.com
|
| Abstract:
In this presentation, we describe the analysis of 26 strategic plans to assess fidelity to SAMHSA's Strategic Prevention Framework (SPF). The SPF is a five-step model that requires grantees to engage in data-driven planning. Specifically, grantees must demonstrate that needs assessment results are linked to selection of substance use-related priorities and community funding allocation plans. Three-point scales were used to assess the strength of these linkages. The assessment procedure involved four-steps: 1) plans were reviewed by multiple raters, 2) descriptive information was extracted for 10 topic areas, 3) thematic codes were defined and scale anchors were constructed, and 4) fidelity scores were assigned using a consensus process. The scores will be used to predict outcomes. The development of this procedure serves not only the SPF SIG cross-site evaluation but is potentially applicable for evaluating prevention initiatives of the future.
|
|
Assessing School “Community Readiness” to Take Action: Comparison of Structured Versus Semi-structured Interview Methods
|
| Presenter(s):
|
| Kristen Ogilvie,
Pacific Institute for Research and Evaluation,
kogilvie@pire.org
|
| Knowlton Johnson,
Pacific Institute for Research and Evaluation,
kwjohnson@pire.org
|
| Abstract:
Community readiness is increasingly recognized as important in achieving substance abuse prevention program success. The Community Readiness Model (CRM), an established and respected model developed at Colorado State, involves a lengthy ethnographic in-person or telephone interview for assessment. Copious notes or transcripts are laboriously coded to score readiness. While this method provides invaluable information on both readiness and community dynamics, it is time-consuming, especially when involving multiple communities. As an alternative, this paper describes the conversion of the respected CRM assessment instrument to a structured telephone survey for the purpose of assessing the readiness of 15 rural Alaskan school districts to participate in a school-based prevention research study to address youth substance abuse sponsored by the National Institute on Drug Abuse. Drawing on the researchers' experience, the paper explores the benefits and liabilities of the different interview methods in assessing readiness and their implications for program evaluation.
|
|
Evaluating the Fidelity of Implementation of Research-based Substance Abuse/School Crime Prevention Programs in Schools
|
| Presenter(s):
|
| Scott Crosse,
Westat,
scottcrosse@westat.com
|
| Michele Harmon,
Westat,
micheleharmon@westat.com
|
| Barbara Williams,
Westat,
barbarawilliams@westat.com
|
| Carol Hagen,
Westat,
carolhagen@westat.com
|
| Liam Ristow,
Westat,
liamristow@westat.com
|
| Abstract:
To increase the effectiveness of school-based prevention programs, funding sources have embraced the idea that such programs should be research-based and be implemented in ways that maximize the likelihood of their producing desired outcomes. However, very little is known about either the prevalence of such programs or the quality of their implementation. This paper describes how the Study of Research-based Programs developed national estimates of the fidelity of implementation of research-based prevention programs in schools. Doing so required creating fidelity criteria that were specific to each of several research-based programs, by reviewing implementation materials for those programs. For a given program, the Study compared the fidelity criteria against survey data gathered on corresponding measures of program implementation. In addition, the paper reports findings on fidelity, as well as on factors associated with it, such as training on program implementation and monitoring of program providers.
|
|
26 States, 300 Communities, and One Fidelity Instrument: Fabulous Feat or Futile Folly in the Strategic Prevention Framework Cross-site
|
| Presenter(s):
|
| Robert Orwin,
Westat,
robertorwin@westat.com
|
| Roy Gabriel,
RMC Research Corporation,
rgabriel@rmccorp.com
|
| Ann Landy,
Westat,
annlandy@westat.com
|
| Janis Wolford,
Westat,
janiswolford@westat.com
|
| Abstract:
All Strategic Prevention Framework (SPF) states are required to implement the 5-step framework: 1) Needs/Readiness Assessment, 2) Capacity Building, 3) Strategic Planning, 4) Intervention Implementation, and 5) Monitoring/Evaluation. The communities each state funds are also required to implement the steps. Recent work has highlighted the strong relationship between implementation fidelity and outcomes in community prevention programming. Will that relationship hold when the program is a “model” and the model will be implemented differently in up to 300 communities across 26 states? This paper describes the collaborative development of a generalizable metric of community-level SPF fidelity whose intent is to determine exactly that. It covers how we 1) developed rating scales to assess implementation quality of each of the 5 steps, 2) developed scoring rubrics for a composite rating for each step, and 3) provided guidance to grantees for using these tools. Preliminary data will also be presented
|
| | | |