|
Evaluating a Child Welfare Demonstration Program: Evolution, Considerations, and Lessons Learned
|
| Presenter(s):
|
| Heather L Scholz, Action Consulting and Evaluation Team (ACET) Inc, heather@acetinc.com
|
| Stella SiWan Zimmerman, Action Consulting and Evaluation Team (ACET) Inc, stella@acetinc.com
|
| Kirsten L Rewey, Action Consulting and Evaluation Team (ACET) Inc, kirsten@acetinc.com
|
| Ellie Skelton, The Wayside House Inc, ellies@waysidehouse.org
|
| Abstract:
Wayside House is implementing the “Incarnation Family Connections” (IFC), a comprehensive substance abuse program where chemically dependent mothers receive treatment while maintaining custody of their children. The goals of IFC are to improve children’s safety, permanency, and well-being by treating the entire family using a comprehensive, systems approach. Mothers receive a wide range of services, including parenting support, while children receive early interventions (e.g., speech, psychological). The evaluation will utilize a mixed method quasi-experimental approach with a comparison group added mid-Year 1. Families, staff, and community partners will complete focus groups and individual interviews and families will complete additional assessments for family functioning and children’s development. The presentation will summarize findings from the first six months of the program, the evolution of the evaluation design, considerations made in selecting the final design, and lessons learned, especially in the context of evaluating this demonstration program while collaborating with multiple external agencies.
|
|
A Multi-faceted Implementation Assessment: Comparing Ratings From Observers, Supervisors, Staff, and Clients to Examine Program Implementation
|
| Presenter(s):
|
| Kristin Duppong Hurley, University of Nebraska, Lincoln, kdupponghurley2@unl.edu
|
| Nikki Wheaton, University of Nebraska, Lincoln, nikkiwheaton@gmail.com
|
| Abstract:
The objective of this presentation is to summarize ongoing efforts of a grant project to (1) develop multifaceted measures to assess the program context, adherence, and competence of implementation of a manualized treatment intervention for youth in residential care and (2) examine the psychometrics of these implementation assessment measures. These multifaceted implementation measures assess the key components of the program from a variety of perspectives (observers, supervisors, youth, and direct-care staff). A key area of interest is if there is agreement among the assessment methods to identify low, adequate. and high levels of implementation. Audience members will learn about how the implementation measures were constructed, their preliminary psychometric information, and how the different implementation measures correlate with each other.
|
|
Evaluating a Child Care Quality Rating and Improvement System: Lessons Learned
|
| Presenter(s):
|
| Michel Lahti, University of Southern Maine, mlahti@usm.maine.edu
|
| Allyson Dean, University of Southern Maine, adean@usm.maine.edu
|
| Sarah Rawlings, University of Southern Maine, srawlings@usm.maine.edu
|
| Abstract:
This paper will present findings from three methods of evaluating a state level quality rating and improvement system for licensed child care programming. The Quality for ME program is Maine's initiative to improve the quality of state licensed child care programs. The presentation will describe the process and current findings from three related methods; on site observations of child care, parent report on child care programming, and staff reports on child care programming. While evaluation findings will be presented, the paper will focus on the challenges associated with maintaining quality of the evaluation in the following areas; maintaining the reliability of data collection activities, reporting results and effect on relationships with evaluation participants, and presenting information from three related and different methodological approaches.
|
|
Synthesis of a Multi-component Five-Year National Evaluation: Results and Lessons Learned
|
| Presenter(s):
|
| Allan Porowski, ICF International, aporowski@icfi.com
|
| Aikaterini Passa, ICF International, apassa@icfi.com
|
| Kelle Basta, ICF International, kbasta@icfi.com
|
| Susan Siegel, Communities In Schools, siegels@cisnet.org
|
| Abstract:
Communities In Schools, Inc. (CIS) is a nationwide initiative to connect community resources with schools to help at-risk students successfully learn, stay in school, and prepare for life. Five years ago, CIS commissioned a third-party evaluation of its entire network. This multi-level evaluation has ten sub-studies (components) that build together to result in a comprehensive national evaluation of the entire federation model, which includes a CIS National Office, state offices, local affiliates, and local sites. As the National Evaluation neared completion, our challenge was to synthesize the components of the ten studies and build a comprehensive set of findings that will be of value for various stakeholders. Members of the national evaluation team will discuss principles of cross-design synthesis, and demonstrate how results from a large, multi-component national evaluation can be wrapped up into a coherent set of evaluation findings.
|
| | | |