|
Unearthing Hidden Contexts in Evaluation Work: Using Brooks 5-Paths Analysis
|
| Presenter(s):
|
| Pauline Brooks, Independent Consultant, pbrooks_3@hotmail.com
|
| Abstract:
Under some circumstances, it is important for evaluators to access broader information concerning the context of programs. Circumstances likely to benefit from such broader contextual information include: situations where stakes are high, the problems being addressed are longstanding or deeply rooted, there is potential for high impact, the program is to be continued for another full round of funding or is to be scaled-up, etc. Increased understanding of the context can have influence on the evaluation focus, assumptions, methods, scope, processes, and eventually the evaluation findings and interpretations. This paper introduces an approach (Brooks 5-Paths Analysis) that evaluators and others can use in planning and collecting data concerning context. The approach focuses on five aspects of social context: history, laterality, accumulations, power dynamics, and resistance to change. Singularly and in combination, data concerning these five aspects of context can often refine the accuracy, relevance and meaningfulness of evaluations.
|
|
A Multi Method Approach for Assessing Fidelity to an Evidence-based Child Neglect Prevention Program
|
| Presenter(s):
|
| Jill Filene, James Bell Associates, filene@jbassoc.com
|
| Lauren Kass, James Bell Associates, kass@jbassoc.com
|
| Elliott Smith, Cornell University, egs1@cornell.edu
|
| Abstract:
Although there has been a growing emphasis on measuring fidelity to evidence-based programs, implementation evaluations tend to focus on quantitative elements of fidelity related to adherence and exposure (i.e., dosage). This paper will describe the process for developing a multi-site assessment of fidelity for an evaluation of the replication of a child neglect prevention program. To strengthen the quality of the findings, fidelity was assessed using a mixed-method approach to collecting qualitative and quantitative data from multiple data sources. In order to allow for the examination of the impact of fidelity on outcomes, a quantitative framework was developed that blends qualitative and quantitative fidelity data. Select findings will be presented about sites’ ability to implement the core components of the program, as well as the impact fidelity had on modifying family-level factors associated with child neglect. Lessons learned regarding developing fidelity assessment methods and applying them will be discussed.
|
|
Compliance is Improving, Now What? Using the Guskey Model in a Mixed Method Evaluation to Measure the Impact and Effectiveness of a National Technical Assistance Center
|
| Presenter(s):
|
| Paula Kohler, Western Michigan University, paula.kohler@wmich.edu
|
| June Gothberg, Western Michigan University, june.gothberg@wmich.edu
|
| Abstract:
This paper describes the evaluation process of the first five years of a national technical assistance center. The center was charged to help states build capacity to support and improve transition planning, services, and outcomes for youth with disabilities and disseminate information and provide technical assistance on scientifically-based research practices with an emphasis on building and sustaining state-level infrastructures of support and district-level demonstrations of effective transition methods for youth with disabilities. To determine the effectiveness of the center, a complex, multi-site multi-level mixed-method evaluation was conducted using federal compliance data (quantitative) and Gusky’s model for evaluating professional development (mixed-methods). Findings indicate effective implementation of the capacity building model used by the center as well as providing a method for evaluation that may be generalized to other complex technical assistance projects.
|
| | |