|
Session Title: Evaluation Policy and Training Evaluation in Child Welfare
|
|
Panel Session 536 to be held in Room 104 in the Convention Center on Friday, Nov 7, 9:15 AM to 10:45 AM
|
|
Sponsored by the Human Services Evaluation TIG
|
| Chair(s): |
| Henry Ilian,
New York City Children's Services,
henry.ilian@dfa.state.ny.us
|
| Abstract:
This panel will address evaluation policy's influence on evaluation practice within child welfare systems. It posits that the absence of evaluation policy creates structural barriers to conducting training evaluation. In the absence of policy, training and training evaluation compete with all other organizational priorities for resources and managerial attention. This competition is governed by such factors as organizational structure, type of leadership, leadership turnover, organizational culture, and caseload size. Thereby the ability of evaluators to get data and to show how training is or is not influencing practice is frequently impeded. This conclusion is based on two sources of evidence: (1) an overview of the training evaluation literature; (2) lessons drawn from training evaluation within child welfare agencies in three localities, the state of Kentucky and the cities of Los Angeles and New York.
|
|
Effects of Leadership Change on Evaluation Efforts in Kentucky
|
| Anita Barbee,
University of Louisville,
anita.barbee@louisville.edu
|
|
Dr. Barbee was selected to participate in this panel because she has worked with one state child welfare agency conducting training evaluation over the past 16 years under four different governors, three different configurations of the agency and eight different commissioners. These changes have had an impact on organizational structure and culture, leadership style and expertise as well as case load size. All of these variables have affected access to decision makers, whether or not research and evaluation is valued, the evaluator's ability to obtain data from stressed and overworked employees, the quality of that data, the usefulness of the data to the agency, and the willingness of the agency to make changes based on the reported results. One solution to the problems would be the establishment of a formal evaluation policy so that such changes would not undermine the evaluation enterprise. This and other possible solutions will be discussed.
|
|
|
Evaluation Policy and Child Welfare Training Evaluation: Lessons from Los Angeles
|
| Todd Franke,
University of California Los Angeles,
tfranke@ucla.edu
|
|
Dr. Franke was selected for this panel as a result of his extensive work at the intersection of evaluation and child welfare issues. He has been involved with agencies that serve thousands of families representing unique geographic and cultural communities in California, particularly southern California counties. He is the PI for the evaluation of the First 5 Los Angeles funded Partnership for Families Initiative. Partnerships for Families is a five year secondary prevention initiative that is designed to prevent child maltreatment by creating opportunities for families, communities, and governmental entities to meaningfully partner in the prevention of child abuse and neglect throughout Los Angeles County. It is designed to assist at-risk families including pregnant and parenting teens. Dr. Franke also is the PI for the Small County Initiative for the past six years, which is designed to systematically evaluate the State of California's efforts to build and enhance child abuse and neglect prevention efforts in 11 rural counties in northern California. This effort required extensive collaboration with multiple county agencies and employed a mixed model approach to data collection. For the past several years he has been working with the Department of Children and Family Services (DCFS-Los Angeles) in evaluating their pre-service and in-service training for Child Protective Service (CPS) workers. In this capacity he has had the opportunity to work collaboratively with the Training Director and supervisors.
| |
|
Training Evaluation and Operational Imperatives at New York City's Children's Services
|
| Henry Ilian,
New York City Children's Services,
henry.ilian@dfa.state.ny.us
|
|
Dr. Ilian was selected to participate on this panel because of his work over twenty years in evaluating training at Children's Services, the New York City child welfare agency. He is strongly interested in the organizational dynamics--including change and persistence in organizational culture and the role of leadership--that permit or inhibit the evaluation of training. He will address lessons learned from efforts undertaken between 2002 and 2005 to conduct a multidimensional evaluation of a large-scale supervisory training program. The evaluation ultimately had to be abandoned, but the experience proved to be rich in insights into the conditions under which training evaluations are and are not successfully conducted within large child welfare organizations. Implications point to an evaluation policy that mediates between the competing imperatives of conducting an agency's ongoing work and ensuring the success of efforts to instill a mode of practice that meets nationally recognized standards.
| |