| Session Title: Measuring Fidelity and Assessing Impact of Service Interventions in Ohio's Title IV-E Waiver Evaluation |
| Multipaper Session 686 to be held in International Room on Friday, November 9, 4:30 PM to 6:00 PM |
| Sponsored by the Human Services Evaluation TIG |
| Chair(s): |
| Madeleine Kimmich, Human Services Research Institute, kimmich@hsri.org |
| Discussant(s): |
| Andrea Sedlak, Westat, andreasedlak@westat.com |
| Abstract: Federal waivers to Title IV-E of the Social Security Act enable state child welfare programs to redirect federal funds from foster care to alternative services for children suffering abuse or neglect. Ohio's Title IV-E waiver demonstration project operates in thirteen of Ohio's county-administered public child welfare agencies. The county agencies are experimenting with three promising interventions: family team meetings, supervised visitation, and supports to kinship caregivers. Key to evaluating the impact of targeted services on child outcomes is assessing whether the services as implemented conform to the original model. If fidelity varies across or even within sites, can one expect a measurable outcome effect? What can be learned from varied applications of a single model intervention? Three papers discuss fidelity assessment in a multi-year evaluation of Ohio's Title IV-E waiver demonstration. We describe fidelity measures and offer initial findings, highlighting challenges and limitations to fidelity assessment. |
| Measuring the Fidelity of Protect Ohio Family Team Meetings |
| Madeleine Kimmich, Human Services Research Institute, kimmich@hsri.org |
| Amy Stuczynski, Human Services Research Institute, astuczynski@hsri.org |
| Family Team Meetings is generally seen as a 'best practice'. Regular meetings, facilitated by a trained professional and bringing together family, friends, service providers and advocates, can lead to creative and effective solutions to case challenges, ultimately reducing the need for foster care placement and improving permanency outcomes. This paper describes the model adopted by 13 demonstration sites, defines the fidelity measures used, presents fidelity findings, and discusses evaluation challenges. Fidelity is measured using case-level and county-level variables. Key issues encountered include: how rigorously to define the model when making judgments about fidelity, how to choose measures that balance the need for specific data with the need to ensure that data is not too onerous to collect, and how best to provide fidelity information to practitioners for the purpose of ensuring that there is a model to evaluate. |
| Supervised Visitation as a Model Intervention |
| Adrienne Zell, Human Services Research Institute, azell@hsri.org |
| Julie Murphy, Human Services Research Institute, murphy@hsri.org |
| One service delivery model selected by Ohio counties participating in the Title IV-E Waiver is Supervised Visitation, an enhanced visitation program for children in out-of-home care and their parents. This visitation model provides increased consistency and structure, and is expected to improve parent-child interactions and maximize the chance for reunification. Five programmatic elements define this particular model. Challenges involved in determining model fidelity include: providing clear definitions of the model components, uncovering factors which influence how the model is implemented, and differentiating among counties with consistently high fidelity. As required by law, all child welfare agencies offer supervised visitation of some sort; therefore a unique challenge to fidelity evaluation of this intervention is determining how the model fidelity of study counties compares to non-intervention counties with similar elements in place. Along with this discussion, we also pose the methodological question of examining fidelity-dosage at an individual client level. |
| Supporting Kinship Caregivers |
| Julie Murphy, Human Services Research Institute, murphy@hsri.org |
| Madeleine Kimmich, Human Services Research Institute, kimmich@hsri.org |
| Six Ohio counties have focused on identifying and supporting kinship caregivers more consistently. They believe that placing children with relatives or friends is less disruptive than formal foster care and ultimately decreases the time children spend in paid placements. Following the kinship model is expected to lead to increased use of kinship settings and more support to these placements (i.e. offering a variety of services and/or subsidies, having designated kinship staff). Measuring adherence to the model is difficult: not all children placed with kin are identifiable in existing data systems, services provided to kinship caregivers are poorly documented, and the Waiver model is not a truly unique approach -- it simply enhances what other counties were already doing to support kin. The paper describes how we have addressed these issues and how, over time, we have adjusted the evaluation plan. |