|
Session Title: Innovative Applications of Propensity Scores and Propensity Score Methodology Adjustments to Address Data Constraints
|
|
Panel Session 120 to be held in PRESIDIO A on Wednesday, Nov 10, 4:30 PM to 6:00 PM
|
|
Sponsored by the Quantitative Methods: Theory and Design TIG
|
| Chair(s): |
| Vajeera Dorabawila, New York State Office of Children and Family Services, vajeera.dorabawila@ocfs.state.ny.us
|
| Discussant(s):
|
| MH Clark, Southern Illinois University, mhclark@siu.edu
|
| Abstract:
The objective of this presentation is to illustrate innovative applications of propensity scores and methodology adjustments that can be made to address data constraints. This session is of particular interest to the Quantitative Methods Topical Interest Group as it covers propensity scores in a way that will appeal to both novices and experts. It will be of interest and accesible to novices as the first presentation will discuss various propensity score matching techniques and share computer programs. Both experts and novices will find it of interest as the applications outline how data constraints and evaluation needs can be addressed through the use of propensity scores. In doing so, the presentors will describe novel applications and methods of addressing data issues.
|
|
A Step-by-Step Application Of Propensity Score Matching With Act Data
|
| Oksana Wasilik, University of Wyoming, oksana@uwyo.edu
|
| Anita Drever, University of Wyoming, adrever@uwyo.edu
|
|
The researchers will present a case study that uses propensity scores to control for background characteristics so that ACT scores before and the after the implementation of a state scholarship program can be compared. The focus of the paper will be not on the results, however, but rather on the process of applying propensity score matching. The presentation will appeal to novices learning how to use propensity scores as well as to experts who will have an opportunity to share their insights and recommendations concerning the presented case study. We will discuss the advantages and disadvantages associated with different matching methods (nearest-neighbor, kernel and stratification), as well as how to handle datasets that do not include optimal covariates. The presentation will involve sharing copies of the annotated STATA programs detailing the process the authors used to generate the results.
|
|
|
Multiple Imputation and Propensity Scores Matching to Address Data Constraints in Constructing a Control Group to Evaluate an Intervention to Reduce Recidivism of Juvenile Delinquents Released From Residential Facilities
|
| Vajeera Dorabawila, New York State Office of Children and Family Services, vajeera.dorabawila@ocfs.state.ny.us
|
| Leigh Bates, New York State Office of Children and Family Services,
|
| Susan Mitchell-Herzfeld, New York State Office of Children and Family Services,
|
| Therese Shady, New York State Office of Children and Family Services,
|
| Do Han Kim, New York State Office of Children and Family Services,
|
|
The objective of this presentation is to demonstrate how propensity scores were utilized to address administrative data constraints when constructing a control group in a quasi-experimental design. In this study, the treatment group consisted of juvenile delinquents that received a short-term community based aftercare program upon release from New York State residential facilities. The purpose of the intervention program was to reduce rates of recidivism. In constructing the control group, the limited number of variables available and sometimes missing values in administrative databases was a data constraint. This constraint was addressed through the multiple imputation technique which maximized the utility of available variables. The next and related constraint was in identifying a match for each treatment individual. This constraint was addressed through a phased approach to nearest neighbor matching within calipers defined by the propensity score. This approach, an adjustment of that recommended by Rosenbaum and Rubin (1985), was able to balance the treatment and control groups.
| |
|
An Innovative Use of Propensity Score Matching to Evaluate Alcoholics Anonymous’ Effect on Drinking
|
| Stephen Magura, Western Michigan University, stephen.magura@wmich.edu
|
|
Evaluation studies consistently report correlations between Alcoholics Anonymous (AA) participation and less drinking or abstinence. Randomization of alcoholics to AA or non-AA is impractical and difficult. Unfortunately, non-randomization studies are susceptible to artifacts due to endogeneity bias, where variables assumed to be exogenous (“independent variables”) may actually be endogenous (“dependent variables”). A common such artifact is selection bias, where different types of people choose to participate or not participate in AA and also have different drinking outcomes. The paper will present a secondary analysis of a national alcoholism treatment data set, Project MATCH, that controls for selection into AA participation using propensity score matching (PSM). The PSM analysis is unique in statistically exploiting the random treatment assignment feature of Project MATCH, since that resulted in a correlation between treatment assignment and AA participation. The presentation will be accessible to evaluators without advanced statistical training. Supported by R21 AA017906.
| |