|
Session Title: Using Logic Models to Support Organizational Alignment, Evaluation, and Learning: One Organization's Journey Toward a Culture of Evaluation
|
|
Panel Session 593 to be held in Capitol Ballroom Section 2 on Friday, Nov 7, 1:35 PM to 3:05 PM
|
|
Sponsored by the College Access Programs TIG
|
| Chair(s): |
| Michelle Jay,
University of South Carolina,
jaym@gwm.sc.edu
|
| Keren Zuniga McDowell,
Citizen Schools,
kerenzuniga@citizenschools.org
|
| Discussant(s):
|
| Michelle Jay,
University of South Carolina,
jaym@gwm.sc.edu
|
| Abstract:
This panel will discuss how one multi-program, multi-site organization used logic models to emerge from an organizational culture where programs operated in isolation, without cross-program outcomes measurement or sharing of data. The central theme of the panel will revolve around the development and use of a series of logic models, which were created collaboratively with organization stakeholders to ensure comprehensiveness and depth of understanding. Levels of success and lessons learned will be reviewed in three categories: strategic alignment of programs' measurement and outcomes; practical implementation of an effective and efficient evaluation policy; and, organizational growth and capacity building resulting from evaluation findings. Examples will be provided as to how logic models were used to align program theory and practice, inform evaluation design, and define program impact.
|
|
Aligning Program Practice and Outcomes Across Multiple College Access Programs
|
| Julie Crump,
The Education Resources Institute,
crump@teri.org
|
|
The use of logic models to align program theory, practice and outcomes will be described. In the last program year (2007-08), TERI delivered eight separate programs and services and piloted the delivery of three new programs. Prior to 2007-08 programs operated in isolation and had little communication with each other. As the organization adopted a new strategic plan and implemented a culture of evaluation, cross-program communication and measurement became integral. Logic models were used to define the organization's theoretical framework and used to align existing programs' activities and outcomes with the organization's theory of practice, as well as, with one another. When programs were aligned the gaps in outcomes measurement were clearly visible indicating where program activities were targeted but not measured making impact impossible to define.
|
|
|
Logic Models as a Tool for Establishing Efficient and Effective Evaluation Policy
|
| Keren Zuniga McDowell,
Citizen Schools,
kerenzuniga@citizenschools.org
|
| Keren Zuniga McDowell,
Citizen Schools,
kerenzuniga@citizenschools.org
|
|
This presentation will focus on how logic models were used to drive the development of an organization-wide evaluation policy and plan and discuss challenges that were encountered. Logic models were used to define the causal links between program inputs, activities, outcomes, and impact and the role of assumptions and external factors on program delivery within the framework of the organization's theory of practice. Key challenges to developing a culture of evaluation within a multi-program, multi-site organization for the very first time included: defining program outcomes, measuring short- and long-term program impact, building an internal evaluation infrastructure, engaging all stakeholders, and securing staff buy-in, all with limited resources and capacity.
| |
|
Creating a Culture of Learning Through Evaluation to Inform Systemic Change
|
| Adrian Haugabrook,
The Education Resources Institute,
haugabrook@teri.org
|
|
This presentation will discuss how TERI used logic models to drive organizational change through data-driven reflection and learning, as well as, advocate for the wide-spread development and integration of a culture of evaluation. Organizational emphasis was placed on the cycle of theory-practice-evaluation and the importance of using evaluation to inform program decision making and ongoing improvement of delivery. TERI found that logic models provided staff with a shared understanding of the organization's new program delivery model and were a helpful tool in defining cross-program accountability through outcomes measurement. The presentation will also describe how logic models were used to communicate organizational goals internally, as well as, externally with funding agencies, community partners, and service recipients.
| |