|
Session Title: Structural Equation Modeling and Longitudinal Perspectives in Evaluation Research: Promises and Pitfalls
|
|
Panel Session 447 to be held in Santa Monica on Thursday, Nov 3, 2:50 PM to 4:20 PM
|
|
Sponsored by the Quantitative Methods: Theory and Design TIG
|
| Chair(s): |
| Georg Matt, San Diego State University, gmatt@sciences.sdsu.edu
|
| Abstract:
Path coefficients from longitudinal designs, variance accounted for as r-square, validity coefficients, effect sizes among many others are quantitative indicators and measures many evaluative judgments and decisions are based on. Can we trust these indicators are there any alternatives?
Structural equation modeling computes these coefficients on the assumption of analyzing measurement error free latent variables. Analyzing our program evaluation studies with these tools hold the promises of coming close to the truth of the problems? The first presentation focuses on various alternative in analyzing longitudinal data, the second challenges whether path coefficients and effect sizes for structural equation modeling are an overestimate or not.
The third one describes the evaluative conclusions to be drawn by applying multiple act criteria analyzed as reflective or formative constructs and compares them to strategies used in meta-analysis. Examples used are based on longitudinal ambulatory psychotherapy outcome research all the presenters had been involved.
|
|
Time Matters!: The Analysis of Dynamic Models in Continuous Time - An Example From Evaluating Outpatient Psychotherapy
|
| Manuel Voelkle, Max Planck Institute for Human Development, voelkle@mpib-berlin.mpg.de
|
|
This presentation has three goals: First, we will discuss problems with several popular approaches to analyze change over time, such as autoregressive models, cross-lagged models, or dynamic factor models for single or multiple subjects from a structural equation perspective. We will argue that most real-world processes happen in continuous time, and that ignoring the time interval between measurement occasions - as done in these models - may lead to a false representation of the world and thus wrong conclusions. Second, we will propose a better alternative to these approaches by using stochastic differential equations and demonstrate how to formulate these models within an SEM framework. Finally, we will use data from a recently completed evaluation study on outpatient psychotherapy to illustrate our arguments. Special emphasis will be put on the theoretical basis of continuous time modeling, but we will also discuss software issues and practical implications.
|
|
|
Are There any Biases in Using Effect Sizes From Latent Variables in Structural Equation Modeling?
|
| Werner Wittmann, University of Mannheim, wittmann@tnt.psychologie.uni-mannheim.de
|
|
Effect sizes and correlation coefficients between latent constructs derived with structural equation modeling (SEM) are often much larger than what we are familiar with using classical statistical tools. This is no wonder because these coefficients are based on measurement error free latent constructs. We discuss the differences between the effects of corrections for attenuation and other artifacts from classical psychometric theory and these communality based true score effects in SEM. Selected examples from our program evaluation studies will be used to demonstrate the consequences for evaluation and whether the promises of SEM will stand up under the scrutiny of a Feynman test, namely the problem of cargo cult science, i.e. will the air crafts land or not?
| |
|
Should we Analyze Outcome Criteria Either as Formative or Reflective Latent Constructs or use Something Else? The Case of Single and Multiple Act Criteria in Evaluating Outcomes
|
| Andres Steffanowski, University of Mannheim, andres@steffanowski.de
|
| Werner Wittmann, University of Mannheim, wittmann@tnt.psychologie.uni-mannheim.de
|
|
Fishbein and Aizen proposed the use of multiple act criteria in many applied research settings. In our longitudinal psychotherapy outcome studies we continuously capitalize on their ideas. From a methodological stance using a structural equation modeling perspective these criteria can be either analyzed as formative or as reflective ones. We report what conclusions can be drawn from both variants and demonstrate implications for using effect sizes from formative or reflective multiple act outcome criteria. We also illuminate the relationship to strategies used in meta-analysis.
| |