|
Using the Cynefin Framework in Evaluation Planning: A Case Study
|
| Presenter(s):
|
| Heather Britt, Independent Consultant, heather_britt_cairo@hotmail.com
|
| Abstract:
This paper provides a case study of the use of the Cynefin framework in evaluation planning. The Cynefin framework is a systems thinking tool that can be used to describe evaluation situations. Each of the four domains of the framework is characterized by different dynamics and corresponding types of inquiry. The characteristics of each domain suggest appropriate evaluation methods.
I outline five steps for using Cynefin to construct an evaluation design:
1. Identify evaluation users and purposes
2. Draft key evaluation questions
3. Assign each evaluation question to a domain of the Cynefin framework by asking 2 questions:
a. How can we know the answer to this evaluation question?
b. What is the nature of the relationship between elements within this situation?
4. Select appropriate evaluation methods for each evaluation question
5. Integrate the key questions and methods into a coherent evaluation design
|
|
Agent Based Modeling (ABM) Simulation for Program Evaluation
|
| Presenter(s):
|
| Stephen Magura, Western Michigan University, stephen.magura@wmich.edu
|
| Rainer Hilscher, Altarum Institute, rainer.hilscher@altarum.org
|
| Theodore Belding, TechTeam Government Solutions, ted.belding@newvectors.net
|
| Jonathan Morell, Vector Research Center, jonny.morell@newvectors.net
|
| Abstract:
Developing simulations to assist in program evaluation has great potential. Simulation is the use of a computer-based mathematical model to mimic a real world system so that the likelihood of various system outcomes can be estimated. Simulation can assist the evaluative process in situations where it is impractical to implement expensive and complex programs and ascertain their outcomes in real time. Given good theory and good data on the initial state of a system, simulation would allow a large variety of implementation scenarios to be explored “virtually,” to assist in deciding which programs with what attributes in which settings are likely to be effective. To illustrate, we outline a concept to apply a specific technique, agent-based modeling (ABM), to simulate the adoption of evidence-based practices into existing addiction treatment systems. A “quick” large-screen demonstration of swarm intelligence in predicting improvised explosive device (IED) threat areas will be included.
|
|
Using Systems Thinking Concepts in Evaluation of Complex Programs
|
| Presenter(s):
|
| William M Trochim, Cornell University, wmt1@cornell.edu
|
| Wanda Casillas, Cornell University, wdc23@cornell.edu
|
| Margaret Johnson, Cornell Univeristy, maj35@cornell.edu
|
| Jennifer Brown Urban, Montclair State University, urbanj@mail.montclair.edu
|
| Abstract:
A growing emphasis on systems thinking in evaluation increases the potential for conducting quality evaluation of programs nested in complex systems. Recent research efforts have led the development of strategic application of systems concepts such as the Systems Evaluation Protocol (SEP) for planning and implementation in the evaluation of nested programs. The SEP applies systems thinking concepts from evolutionary biology and developmental systems theory to program modeling, boundary analysis, and program/evaluation alignment in order to position the program and its evaluation within phases of a lifecycle that co-evolve in an ecological context. This paper presentation will discuss these theoretical concepts in depth, drawing examples from current evaluation work which utilizes a facilitated SEP.
|
| | |