|
Research Validity: State-of-the-Art and Barriers
|
| Presenter(s):
|
| Tony Lam, University of Toronto, tonycm.utoronto.ca
|
| Flanny Alamparambil, University of Toronto, flannya@hotmail.com
|
| Abstract:
The concept of validity in quantitative research methodology and program evaluation has evolved over several decades, and its most current structure and content has been developed and presented by Shadish, Cook and Campbell (SCC) in their 2002 publication Experimental and Quasi-Experimental Designs for Generalized Causal Inference. However, we have found quantitative method books published as recently as 2010 have ignored SCC's conceptualization of validity. We have also noticed that some authors of published articles do not adhere to the modern framework in discussing validity. It appears that researchers continue to embrace obsolete validity types and their associated threats. Our research assesses the extent to which the field of quantitative method espouses the SCC's validity typology, and the misconceptions and sources of confusion about this validity framework. We also offer elaborations about validity and validity threats and explain how measurement validity is being addressed within SCC's validity framework.
|
|
Minimizing Pregroup Differences with Matching and Adjustment
|
| Presenter(s):
|
| William Holmes, University of Massachusetts, Boston, william.holmes@umb.edu
|
| Abstract:
This presentation examines the combined use of matching and regression adjustment to produce results superior to matching or adjustment alone. It will discuss strengths and weaknesses of each procedure and the circumstances in which one strategy performs better than the other. It will explain why the combined use of matching and adjustment produces superior results in reducing pregroup differences. This presentation provides an example of its use and diagnostic evidence as to whether the results are reliable. The combined use improves estimates of treatment effects and reduces bias from pregroup differences. The example uses data from a dose response evaluation of a family services program providing intensive case management to substance abusing families that have been substantiated as having abused or neglected their children. The findings show the program has positive effects even after removing pregroup differences.
|
| |