|
Session Title: Improving Evaluation Policy by Focusing State, County, and Community Social Service Providers on Results-Oriented Services
|
|
Panel Session 621 to be held in the Quartz Room Section A on Friday, Nov 7, 1:35 PM to 3:05 PM
|
|
Sponsored by the Collaborative, Participatory & Empowerment Evaluation TIG
|
| Chair(s): |
| Gordon Hannah,
Finger Lakes Law and Social Policy Center Inc,
gordonjhannah@gmail.com
|
| Abstract:
New York State passed a law in 2007 requiring all services provided by state agencies to include outcome or performance provisions. This multi-paper presentation will describe an intervention designed to help nine county social service departments meet the requirements of this new law. The intervention attempted to achieve this goal by promoting the systematic use of evaluation and continuous quality improvement processes to achieve desired outcomes. Such systematic use was encouraged through changes to policies and practices regarding contracting and monitoring third-party providers. The papers in this presentation will (1) describe the evaluation policies in place at both the state and county level prior to the intervention; (2) describe the goals and design of the intervention, and how it played out; (3) describe the evaluation policies that changed as a result of our intervention; and (4) discuss factors that impacted the effectiveness of the intervention.
|
|
Child Welfare Evaluation Policy in New York State: The Interplay Between State, County, and Service Provider Rules, Regulations, and Norms
|
| Larry Pasti,
New York State Office of Children and Family Services,
larry.pasti@ocfs.state.ny.us
|
|
Good evaluation policy can improve the quality of program evaluations which subsequently can improve the implementation of programs and lead to better outcomes for consumers. Child welfare services seek better outcomes for children and are influenced by evaluation policies stemming from state, county, agency, and service provider organization rules, regulations, and norms. This presentation will describe the evaluation policies present across nine counties in New York State in regards to child welfare services prior to the implementation of a new law requiring all services to include outcome or performance provisions. The interaction between these various organizations in the creation and implementation of these evaluation policies and the impact of these policies on child welfare services will be discussed. Plans to meet the requirements of the new state law will be briefly described and elaborated on in subsequent presentations.
|
|
|
A Design to Influence Evaluation Policy: Goals, Capacity, and a Support System
|
| Marilyn Ray,
Finger Lakes Law and Social Policy Center Inc,
mlr17@cornell.edu
|
|
This paper describes how we planned, designed, and implemented Getting To Outcomes (GTO) to assist nine counties in NYS develop evaluation policies to include accountability for outcomes, a new state policy. The framework developed to guide the project is a systems model which shows the project goals, initial assessments of county capacities, the project design (including tools, training, TA, and quality improvement/quality assurance), and how this process leads to the outcomes achieved by the counties. Prior to the new state policy, county evaluation policies focused on reporting on units of services. By encouraging a more specific and rigorous planning process, GTO enhances policies to evaluate quality and fidelity of implementation more accurately. The state requires that counties focus on outcomes, and GTO helps formulate new county outcome evaluation policy to measure achievements. Our systems model also includes our use of quality improvement/quality assurance to adjust our process to individual county needs while maintaining fidelity to our systems model.
| |
|
Changes in Evaluation Policy as a Result of a Getting to Outcomes System Intervention with Departments of Social Services
|
| Gordon Hannah,
Finger Lakes Law and Social Policy Center Inc,
gordonjhannah@gmail.com
|
|
The New York State Office of Child and Family Services contracted with PDP and Finger Lakes Law and Social Policy Center to provide training and technical assistance to nine counties in the use of the Getting to Outcomes Results-Based Accountability System in order to help counties change evaluation policies to meet the requirements of a new state law requiring that all services include performance or outcome provisions. This presentation will discuss the changes that occurred in evaluation policies in these counties as a result of this intervention and the implications of these policy changes on evaluation practice both within county agencies and within the service providers with whom they contract. Such policy changes will include changes to contracts, contract review processes, reporting requirements, and program monitoring processes. The implications of these evaluation policy changes on service quality will also be discussed.
| |
|
Contextual and Organizational Factors Impacting the Success of an Intervention to Enhance Evaluation Policy within Departments of Social Service
|
| Abraham Wandersman,
University of South Carolina,
wandersman@sc.edu
|
|
In this presentation we explore how the nine counties in our intervention responded to the project and explore explanations for the varying success of the intervention across the counties using different theories of systems change. We will address such questions as: How did county size, geography, and demographics affect our outcomes? How did agency leadership, resources, and motivation affect our outcomes? How did initial county capacity for results-based accountability affect outcomes? Did the way each county organized themselves for the project and collaborated with their providers affect outcomes? What are the pros and cons of staff involvement at different levels of the agency? What process variables are associated with positive outcomes? What components of the intervention appeared most impactful? What did we learn from this project that can inform other projects focused on changing evaluation policy?
| |