|
Evaluation of an Initiative to Reform Government Contracting for Human Services
|
| Presenter(s):
|
| Susan Wolfe, Susan Wolfe and Associates LLC, susan.wolfe@susanwolfeandassociates.net
|
| Kathryn Race, Race & Associates Ltd, krace@raceassociates.com
|
| Abstract:
Under a grant from the Wallace Foundation, the Donors Forum developed an initiative to overhaul the government contracting processes for human services in the State of Illinois and the City of Chicago. It is based on a framework of 15 principles in 6 areas and involves 40 practices. The focus of this paper is to describe the methodology used by two independent evaluators who collaborated to evaluate this effort. This methodology included administering a survey to key government respondents, using a modified Delphi method to obtain consensus and further explain survey results, site visits and interviews with key informants, document reviews, and a survey of stakeholders that included providers of human services, foundations and advocacy organizations. Our presentation will include discussion of the complexities and challenges encountered while conducting the evaluation, our collaborative approach, and highlight the applicability and generalizability of such a methodology for future evaluations of this nature.
|
|
Does What We Value Make a Difference in our Assessment of Implementation Fidelity?
|
| Presenter(s):
|
| Mary Styers, Magnolia Consulting LLC, mary@magnoliaconsulting.org
|
| Abstract:
Evaluators are continually tasked with making value decisions in the course of study design. In our decisions about implementation fidelity, we place value on specific observations (e.g., self-report, trained observer ratings) and measurement indicators (e.g., dosage, environment, observed use). Each value judgment can strongly impact how a study's implementation fidelity is conceptualized and understood. Yet across fields and within our field, researchers and evaluators tend to hold opposing values in the conceptualization and use of fidelity. As a consequence, there are different relations between fidelity and outcomes, leading to null or significant effects. Drawing on the literature and experiences from two elementary reading and mathematics program efficacy studies, this paper explores differences in fidelity measurements, fidelity variables, and offers recommendations for measuring fidelity.
|
| |