|
Changing Program Policy and Practice Using Data from Real-Time Methods: A Case Example
|
| Presenter(s):
|
| Brian Pagkos,
Community Connections of New York,
pagkos@hotmail.com
|
| Heidi Milch,
Community Connections of New York,
hmilch@gateway-longview.org
|
| Mansoor Kazi,
University at Buffalo - State University of New York,
mkazi@buffalo.edu
|
| Dawn M Skowronski,
Mid-Erie Counseling and Treatment Services,
|
| Abstract:
A recent request for proposals in Erie County, NY led to the formulation of a quality management organization (QMO) responsible for monitoring all vendor and care coordination agencies delivering wraparound services in the county, and analyze functioning of the system as a whole. The QMO uses a blended paradigm approach (utilitarianism and realist) to accomplish this task, resulting in products that are responsive to agency needs. Working collaboratively with primary users, data are used to develop both agency and system level policy as well as direct practice.
Presenters will describe an agency case example where this evaluation intervention was put into place. Although findings were positive overall, the evaluation team worked closely with program supervisors and staff to troubleshoot areas needing improvement. This resulted in policy development surrounding formalized group supervision practices, worker training, outcome measure inter-rater reliability tests, and standardized methods to complete client assessments.
|
|
Evaluation Practice of Investigating What Works and in What Circumstances
|
| Presenter(s):
|
| Mansoor Kazi,
University at Buffalo - State University of New York,
mkazi@buffalo.edu
|
| Brian Pagkos,
University at Buffalo - State University of New York,
pagkos@gmail.com
|
| Abstract:
This paper presents examples of real-time evaluation practice to investigate the patterns between the client contexts, the intervention, and outcomes. The binary logistic regression method (Kazi, 2003) is used where multiple factors are influencing the outcome, potentially with a prediction of the odds of achieving a given outcome in particular circumstances. Hierarchical Linear Modeling (HLM) is used with hierarchical structure of data, such as measures within persons or repeated measures designs (Raudenbush & Byrk, 2002). The regression discontinuity design can be used as a comparison group design in which the participants are assigned to program or comparison groups solely on the basis of a cutoff score on a pre-program measure (Shadish, Cook & Campbell, 2001). All of these three approaches to data analysis can be used at regular intervals whenever an outcome measure is repeated and the findings can inform practice prospectively in each three month period, in real-time.
|
| |