|
Session Title: Place Randomized Trials: Design, Implementation and First Results From Evaluating Ambulatory Psychotherapy
|
|
Panel Session 801 to be held in International Room on Saturday, November 10, 12:10 PM to 1:40 PM
|
|
Sponsored by the Quantitative Methods: Theory and Design TIG
|
| Chair(s): |
| Lee Sechrest,
University of Arizona,
sechrest@u.arizona.edu
|
| Discussant(s):
|
| Fred Newman,
Florida International University,
newmanf@fiu.edu
|
| Abstract:
The suitability of randomized clinical trials or randomized control group designs as an assumed gold standard has been heavily debated in evaluating educational programs and interventions. Recently we also experienced a recommendation split and some controversy within our organization. It is our opinion that evaluators should tailor the best design, best data analysis and best evaluation/decision tools to an evaluation project, considering the circumstances and constructs, which cannot be easily altered.
Place randomized trials are a type of design with potential of high internal(causal)validity and high external validity (generalizability)in many applied setting where randomization at the individual level is not possible. This symposium will report about our strategies, namely the five data-box conceptualization we use as a comprehensive framework, our experiences and first results of a large scale program evaluation focusing on the impact of ambulatory psychotherapy in Germany.
|
|
Designing a Place Randomized Trial in a Field Setting
|
| Werner Wittmann,
University of Mannheim,
wittmann@tnt.psychologie.uni-mannheim.de
|
|
Evidence based research to improve and evaluate the quality and impact of interventions in different areas, e.g. the health system has reached Germany. We have been asked by a leading health insurance company, the 'Techniker Krankenkasse, Hamburg' (Technicians health insurance company,Hamburg) to do the evaluation of a computerized assessment system, giving feedback to the therapist about the progress of clients seeking help in ambulatory psychotherapy. We chose to randomize at the level of the therapist, because randomization at the client level was not feasible. The control group therapists had no access to the computerized feedback system. The whole evaluation design was conceptualized according to the Northwestern path of the five data box conceptualization, developed as a comprehensive evaluation system by the author. The different steps in implementing design, assessment and the necessary power to satisfy different stakeholder interests will be described in detail.
|
|
|
Tailoring Assessment Tools to Different Stakeholder Groups, Randomization and Treatment Integrity Checks
|
| Andros Steffanowski,
University of Mannheim,
steffanowski@tnt.psychologie.uni-mannheim.de
|
| David Kriz,
University of Mannheim,
krizdavid@yahoo.com
|
|
An overview about the various assessment tools recommended and implemented to map the different data boxes is given. Three different treatment schools, namely behavior therapy, analytical therapy and depth psychology participate in the study. The common, the school specific and the disease specific assessment instruments are described along with the randomization checks, the number of treatment sessions and their integrity. Multiple act criteria as the fairest approach to map stakeholder interest and symmetry between treatment and outcome criteria in this project are discussed.
| |
|
Evaluating Ambulatory Psychotherapy in a Field Setting With a Place Randomized Trial, First Results
|
| Manuel Voelkle,
University of Mannheim,
voelkle@rumms.uni-mannheim.de
|
|
A priori power analysis in detecting effect sizes just above the break-even point forced us to assess more than 2000 clients and the main stakeholder complied to this number. So far we have more than 800 clients assessed. There are at least five assessments of the clients progress during therapy and a one year follow-up. It will be demonstrated how to analyze such a repeated measurement or time series design with classical and modern data analytic tools. First results in terms of pre-post effect sizes mapping progress are reported. First results are very promising, medium effect sizes right after the first sessions bolstering late Ken Howard's research and medium to large effect sizes later on.
| |