| Session Title: It's About the Relationship! Crosscutting Organizational policies That support Program Evaluation and Institutional Learning in Social Services. |
| Multipaper Session 525 to be held in Mineral Hall Section F on Friday, Nov 7, 9:15 AM to 10:45 AM |
| Sponsored by the Social Work TIG |
| Chair(s): |
| Benjamin Kerman, Annie E Casey Foundation, bkerman@caseyfamilyservices.org |
| Discussant(s): |
| Diane Purvin, Annie E Casey Foundation, dpurvin@caseyfamilyservices.org |
| Abstract: Social Services evaluation policy can be conceptualized as a series of formal and informal contracts developed among members of an organization to learn from its collective experience. The contract metaphor points to the intrinsic role of relationship in using evaluation effectively. Shared principles and goals underscore the contract parameters, such as the direct connection between evaluation and shared mission, valuing the commitment and capacity to apply feedback, the delineation of roles, confidentiality and the protection of participant rights. This session features three papers describing diverse evaluation activities with social services staff and managers illustrating how evaluation policies and activities manifest explicit and implicit contracts. The three projects involve the routine review of performance indicators, an implementation evaluation of a program redesign, and an outcome evaluation using a longitudinal follow-up. Evaluation policies need to adhere to common principles while remaining flexible enough to allow for adaptation to questions and context. |
| The Role of Evaluation Policies in Integrating Agency-wide Performance Measurement |
| Ellen Ross, Annie E Casey Foundation, eross@caseyfamilyservices.org |
| This paper examines how evaluation policy can facilitate agency wide performance measurement. In 2002, Casey Family Services adopted results-based accountability across the agency's core goals. This framework provides a simple, disciplined rubric, common language, and common ground for implementing new ideas. Operational tasks included how integrating evaluation into practice, recruiting and engaging social workers from the onset, and institutionalizing the framework. By involving all organizational levels in routine indicator review, leadership and staff were alerted to areas of concern and accomplishment. Critical in this process was the involvement and buy-in from senior management, as well as collaboration across research, field operations, information technology, and direct care staff. Challenges included: timing, funding, buy-in from stakeholders, and organizational culture. Strategies used to address these challenges included building departmental relationships, development of policies and infrastructure to embed the data collection and review in ongoing activities and structures, and attending to diverse stakeholder's needs. |
| Policies, Pressures, and Expectations: Managing Relationships During a Multifaceted Implementation Evaluation |
| Judy Lee, Casey Family Services, judymlee@msn.com |
| Benjamin Kerman, Annie E Casey Foundation, bkerman@caseyfamilyservices.org |
| Casey's Move to Greater Permanency initiative reflects an intensified focus on providing permanency planning and foster care services to youth in effort to increase legal and relational permanence. While some program refinements were similar across service sites (e.g., use of Permanency Teaming), additional refinements were site specific. As expectations for a swift and complete integration of Permanency Teaming practice increased among agency administrators, pressures also mounted among field leaders and staff who were experiencing increased uncertainty given the major shifts in practice and procedures. Amidst these changing demands and social worker anxiety, the evaluators used an array of existing policies to conduct a multifaceted implementation and short-term outcome evaluation. Close coordination between internal evaluators and the reliance on external consultants helped ensure that the relationships between evaluator and evaluand were appropriate and corresponded optimally to the particular evaluation question and component. |
| Follow-up Evaluation and the Long View: Preparation for the Casey Longitudinal Study of Foster Youth Development |
| Gretta Cushing, Casey Family Services, gcushing@caseyfamilyservices.org |
| Diane Purvin, Annie E Casey Foundation, dpurvin@caseyfamilyservices.org |
| Longitudinal and follow-up projects may require additional policies and structures that go beyond those of either performance measurement or concurrent program evaluation designs. This paper will discuss how researchers and social workers collaborated in the development of a longitudinal study aiming to provide the agency leadership with an unbiased view of outcomes, as well as to examine applied developmental questions concerning vulnerable youth and their families. All youth who were placed in a foster home after December 31 1993 and spent one year with Casey are included. Data collection includes record reviews and young adult interviews at ages 19, 22 and 25. A variety of agency policies and structures are essential to the conduct of longitudinal service outcome research, including location of former clients and response to service requests, internal communication and confidentiality, and handling reports of maltreatment, as well as the time investment inherent in longitudinal research. |