|
The Long Road of Longitudinal Studies: Learning What to Do and Not to Do Along the Way
|
| Presenter(s):
|
| Bruce Yelton,
Praxis Research Inc,
praxis1@att.net
|
| Paula Plonski,
Praxis Research Inc,
pmplonski@carolina.rr.com
|
| Grant Morgan,
Praxis Research Inc,
praxisgm@aol.com
|
| Mary Beth Gilbert,
Praxis Research Inc,
marybethgilbert@bellsouth.net
|
| Abstract:
The use of longitudinal studies to demonstrate program effects over time presents evaluators with challenges that are immediately obvious and frustratingly difficult to anticipate. Evaluators considering the use of longitudinal studies should consider both the theoretical implications of implementing a longitudinal study and the more frequently mentioned methodological barriers to this type of investigation. These and other issues are discussed in relation to four longitudinal studies of North Carolina's “SMART START” initiative to prepare young children for school. These studies were conducted between 1998 and the present and involved following young children from their childcare/preschool years into public school.
Themes for discussion in the presentation include: 1) Stakeholder Expectations, 2) Theory Driven Evaluation, 3) Data, Technology, and Access, and 4) Attrition. Results, examples, and practical solutions will be offered.
|
|
Using Cox Regression Modeling to Predict Recidivism for Youth Departing Out-of-home Care: Implications for Program Evaluation and Treatment of At-risk Youth
|
| Presenter(s):
|
| Jay Ringle,
Girls and Boys Town,
ringlej@girlsandboystown.org
|
| David Kingsley,
University of Kansas,
gridave@sunflower.com
|
| Stephanie Ingram,
Girls and Boys Town,
ingrams@girlsandboystown.org
|
| Beth Chmelka,
Girls and Boys Town,
chmelkab@girlsandboystown.org
|
| Ron Thompson,
Girls and Boys Town,
thompsonr@girlsandboystown.org
|
| Abstract:
Although high-quality treatment programs for troubled youth reduce the risk of serious life-problems, some youth will nevertheless experience post-treatment difficulties. Arrest for illegal activity is one common risk factor addressed in most youth treatment programs. This study examined risk factors associated with youth being arrested up to five years post-discharge from an out-of-home residential treatment setting. Using a Cox Regression Proportional Hazards modeling procedure, results indicate two protective factors against future recidivism: an absence of a criminal background at admission and a positive departure from care. Using this modeling procedure, organizations serving at-risk youth can use data they are already collecting to evaluate outcomes and identify risk factors associated with those youth most likely to re-offend in order to modify and improve their treatment appropriately.
|
|
Evaluating Impacts of Place-based Initiatives: An Application of a Spatially Improved Interrupted Time Series Design
|
| Presenter(s):
|
| Julia Koschinsky,
University of Illinois at Urbana-Champaign,
koschins@uiuc.edu
|
| Abstract:
Evaluators frequently apply the interrupted time series (ITS) design with an added nonequivalent, no treatment control group time series to assess whether targeted neighborhood revitalization investments generate spillover effects to surrounding areas (Accordino et al. 2005; Galster et al. 2004). Although this literature begins to incorporate advances in spatial analysis/econometrics, no comprehensive applications exist to date.
To address this research gap, the purpose of this paper is to increase the “spatial intelligence” of the best current adjusted ITS models. Using spillover effects from low-income rental housing in Seattle, WA as an example, this is done by 1) applying exploratory spatial data analysis techniques, including geographically weighted regression (with aerial images); 2) by modeling the spatial segmentation of housing markets; 3) by testing for the presence of spatial autocorrelation; and 4) by comparing the performance of the adjusted ITS models to state-of-the-art spatial regression models (Anselin 1988).
|
|
Evaluating Value-added Methodology for Standards-based Accountability Assessment
|
| Presenter(s):
|
| Lihshing Wang,
University of Cincinnati,
leigh.wang@uc.edu
|
| Kent Seidel,
University of Cincinnati,
kent.seidel@uc.edu
|
| Suzane Franco,
Wright State University,
suzanne.franco@wright.edu
|
| Abstract:
As stakeholders continue to push for longitudinal assessment of learning growth under the No Child Left Behind Act, value-added methodology (VAM) emerges as a promising tool for evaluating teacher quality and school effectiveness. This study critically examines the epidemiological basis of value-added scores and their psychometric sensitivity to demographic confounding and scaling operation. Using recent statewide assessment data to substantiate our claims, we found that VAM is both epidemiologically inconsistent with the standards-based accountability ideology and psychometrically unstable for assessing teacher/school effects. Furthermore, the value-added scores were found to be divergent from the Adequate Yearly Progress index currently in place for measuring school performance. We conclude by issuing cautions on applying and interpreting VAM for standards-based accountability assessment and proposing an alternative conceptual framework for measuring student growth that combines the merits of VAM and other longitudinal models.
|
| | | |