|
Using Evaluation Findings to Enhance Learning: From Theory to Practice
|
| Presenter(s):
|
| Paul Stiles,
University of South Florida,
stiles@fmhi.usf.edu
|
| Catherine Batsche,
University of South Florida,
cbatsche@fmhi.usf.edu
|
| Amber Gum,
University of South Florida,
agum@fmhi.usf.edu
|
| Roger Boothroyd,
University of South Florida,
boothroy@fmhi.usf.edu
|
| Abstract:
For the past three years, faculty at the Louis de la Parte Florida Mental Health Institute have conducted a 9-week intensive Research Experience for Undergraduates (REU), funded by the National Science Foundation. The REU is focused on mental health services research and evaluation and includes a series of seminars on research methods as well as an experiential component in which students design and conduct a research project. A program supplement provides an enhanced emphasis on research ethics. As part of the program, an extensive course evaluation has been developed and implemented. Modeled after Kirkpatrick's (1959) longstanding approach to training evaluation, the results have been used extensively each year to modify and improve the REU. The presentation will provide an overview of the REU, provide a detailed description of the evaluation approach, and detail how the evaluation findings has directly resulted in change in the scope and format of the program.
|
|
Challenges for Evaluating Model Programs to Reduce Alcohol Related Problems Among Youth: Communities Mobilizing for Change on Alcohol
|
| Presenter(s):
|
| Linda Bosma,
Bosma Consulting LLC,
linda@bosmaconsulting.com
|
| Abstract:
Communities Mobilizing for Change on Alcohol (CMCA) is a SAMHSA Model Program that uses community organizing as a method of reducing problems related to underage drinking by reducing youth access to alcohol. Community organizing is a new paradigm for many public health agencies more used to school based educational models. Traditional evaluation approaches such as student surveys do not adequately assess this program, which aims to make changes at the community level which will then reduce youth drinking rates. This research examined three groups that were implementing CMCA. This information was used to create an evaluation framework for the evaluation of CMCA. This paper gives an overview of the research, describes the evaluation framework that emerged from this research, and discusses the use of this model in other settings where CMCA is being implemented.
|
|
The Relationship Between Large Scale Student Surveys and Program Evaluation: Risks and Rewards
|
| Presenter(s):
|
| Gregory Rainone,
New York State Office of Alcoholism and Substance Abuse Services,
gregoryrainone@oasas.state.ny.us
|
| Michael Maranda,
Independent Consultant,
mic_maranda@yahoo.com
|
| Abstract:
Legal challenges at the local level and policy changes at the state and federal level (e.g. amendment of the Protection of Pupil Rights Act by No Child Left Behind) have made it increasingly difficult to conduct student surveys of illegal and socially undesirable behaviors such as substance abuse. Consequently, school administrators are increasingly reluctant to participate in these surveys; and, thus, researchers are forced to discover new strategies to encourage participation. One such strategy is to implement school-based surveys that could serve the purposes of both evaluation and epidemiology. This strategy can include collaboration with evaluators through the provision of baseline data and general population norms for outcome measures. This strategy may also include producing reports tailored to the interests of school administrators. The advantages and limitations of variations of this strategy are discussed drawing upon the experience of a survey unit in a single state agency.
|
|
The Non-use of a Longer-term Follow-up Evaluation in Drug and Alcohol Treatment Programs
|
| Presenter(s):
|
| Sharon Ross,
Founder's Trust,
sross@founderstrust.org
|
| Abstract:
While discussions of the use or misuse of evaluation and evaluation results is common in the literature, less common is the discussion of the non-use of evaluation. This paper examines the case of a group of drug and alcohol treatment programs who requested the development of a survey to measure client outcomes at six-months and one-year post-discharge. Despite the strong expression of need by the programs to track and report this data, none of the programs had begun to collect data after two months of having the instrument. Interviews with key program staff reveal how the complexities of implementing evaluation plans and staff beliefs toward evaluation impact whether and how evaluation tools are used. Specific attention is given to the issues that arise from choosing longer-term follow-up surveys to evaluate drug and alcohol programs.
|
| | | |