|
Session Title: Learning From Experience With Random Assignment Experiments: Lessons From Social Security Benefit-Offset Pilot Projects
|
|
Panel Session 439 to be held in International Room on Thursday, November 8, 3:35 PM to 5:05 PM
|
|
Sponsored by the Quantitative Methods: Theory and Design TIG
|
| Chair(s): |
| George Julnes,
Utah State University,
gjulnes@cc.usu.edu
|
| Discussant(s):
|
| Mark Green,
United States Social Security Administration,
mark.green@ssa.gov
|
| Joe Entwisle,
National Consortium for Health Systems Development,
jentwisle@hdadvocates.org
|
| Abstract:
Given the recent emphasis on evidence-based practice and the use of rigorous methods to yield the required evidence, it is important for evaluators to appreciate the issues involved in implementing randomized policy experiments. This session reports on the challenges and tentative outcomes of a 4-state pilot demonstration project that used random assignment to study a U.S. Social Security Administration (SSA) policy innovation designed to encourage persons with disabilities receiving Social Security Disability Insurance (SSDI) support to increase their work efforts and move towards financial independence. In each of the four states, individuals receiving SSDI assistance were recruited and randomly assigned to either “experimental rules” (gradual reduction of cash assistance as earnings increase beyond the SSDI threshold) or “current rules” (abrupt end of cash assistance when earnings cross the threshold) conditions.
|
|
The Vermont Social Security Disability Insurance (SSDI) $1 for $2 Benefit Offset Pilot: Experiences From the Field
|
| Alydia Payette,
Vermont Division of Vocational Rehabilitation,
alydia.payette@dail.state.vt.us
|
| James Smith,
Vermont Division of Vocational Rehabilitation,
james.smith@dail.state.vt.us
|
| Alice Porter,
Vermont Division of Vocational Rehabilitation,
alice.porter@dail.state.vt.us
|
| Peter Burt,
Vermont Division of Vocational Rehabilitation,
peter.burt@dail.state.vt.us
|
|
Vermont's participation in the SSDI Benefit Offset Pilot offered unique opportunities for insight into the implementation issues of a random assignment project involving SSA beneficiaries. Vermont enrolled 577 individuals, 285 in the intervention group, 292 in the control group. Vermont's successful recruitment effort and management of benefits for pilot enrollees required intensive effort and close monitoring. Lessons learned from the process evaluation have been instructive for future pilot design and policy development. Vermont's outcomes evaluation relies on administrative data-including Unemployment Insurance quarterly wage reports, public benefits participation and claims records, and disability benefits data supplied by SSA. While these data are not yet reliable, preliminary data suggest that pilot enrollees are working at a higher level than control group members.
|
|
|
Understanding the Challenges and Opportunities of Randomized Policy Experiments: Lessons From the Utah Social Security Administration Benefit Offset Study
|
| Anne Reither,
Utah State University,
areither@gmail.com
|
| Cathy Chambless,
University of Utah,
cathy.chambless@cppa.utah.edu
|
| Sara McCormick,
University of Utah,
sara.mccormick@cppa.utah.edu
|
| George Julnes,
Utah State University,
gjulnes@cc.usu.edu
|
|
In an effort to understand the employment impacts of a proposed policy change, the Social Security Administration contracted with four states, including Utah, to conduct pilot policy experiments that were to inform a subsequent national experimental study. The Utah project enrolled 502 participants from September 2005 to October 2006 and assigned them randomly to either experimental rules (gradual reduction of cash assistance as earnings increase beyond the SSDI threshold) or current rules (abrupt end of cash assistance when earnings cross the threshold) conditions. This presentation will report on the lessons learned in implementing this policy experiment and offer insights into the dynamics related to employment.
| |
|
The Benefits Offset Pilot Project: The Connecticut Experience
|
| Julie Robison,
University of Connecticut,
jrobison@uchc.edu
|
| Cynthia Gruman,
University of Connecticut,
gruman@uchc.edu
|
| Amy Porter,
Connecticut Bureau of Rehabilitation Services,
amy.porter@ct.gov
|
|
Connecticut's Benefits Offset Pilot enrolled 265 participants: 128 test group and 137 control group. Recruitment challenges included a large number of ineligible applicants due to expired Extended Period of Eligibility and delays in applying the Offset. Connecticut used a mixed-methods approach which included telephone interviews with all participants at multiple time points: baseline, 3, 6, 9 and 12 months. Participants are 56% female, 80% white and 96% have at least a high school diploma. Most reported a physical disability (63%) and/or a mental illness (53%). 73% were employed at enrollment, with 20% working full time. At 3 months after enrollment, test participants were significantly more likely than controls to have: increased their income by $100 or more, told their employer they could work more hours or earn more, increased their work hours, started a job (if unemployed at baseline) or added a second job.
| |
|
Challenges of Conducting Randomized Field Trials: The Experience of the Wisconsin Social Security Disability Insurance (SSDI) Employment Pilot
|
| Barry S Delin,
University of Wisconsin, Stout,
delinb@uwstout.edu
|
| Christopher Sell,
University of Wisconsin,
sellcw@dhfs.state.wi.us
|
| Anne Reither,
Utah State University,
areither@gmail.com
|
|
This paper describes challenges faced in conducting a randomized field trial involving decentralized enrollment and data collection processes and a risk adverse population. The project is one of four state based pilots of a benefit offset provision for the Social Security Disability Insurance (SSDI) program. Wisconsin’s pilot is substantially more decentralized than the others with recruitment, enrollment, and service provision accomplished through state contracted agencies. These agencies are responsible for providing encounter data for research purposes. Thus, persons not trained researchers are responsible for implementing tasks critical to the evaluation’s quality. This paper is organized as a case study using multiple sources of qualitative data from process evaluation activities. It also utilizes information on data consistency and accuracy to collaborate findings based on the qualitative data. Finally, the presentation will include some initial outcomes data.
| |