|
Session Title: High Flexibility and Low Fidelity: The Challenges of Evaluating Highly Adaptable Programs
|
|
Panel Session 824 to be held in Lone Star C on Saturday, Nov 13, 1:40 PM to 2:25 PM
|
|
Sponsored by the
|
| Chair(s): |
| Ann House, SRI International, ann.house@sri.com
|
| Discussant(s):
|
| Leslie Goodyear, National Science Foundation, lgoodyea@nsf.gov
|
| Abstract:
In an ideal world, many evaluation approaches prioritize clearly defining the purpose of the program being evaluated, understanding how the program is implemented, identifying targeted outcomes and following up with program participants. Yet, there are times when evaluators encounter programs that are, by design, flexible and open to adaptation; where strong program fidelity is not emphasized or seen as critical; and when participants are not readily identified. This panel will describe two projects that are distributed in their implementation and pose important evaluation challenges. Adobe Youth Voices is a global youth media program that relies on partners and distributed materials in implementation. Intel’s Elements courses are a series of teacher professional development opportunities which are freely available and intended to be used in ways that meet local training needs. The panelists will discuss issues of evaluator role, evaluation design, data collection, and analysis with regard to these adaptable programs.
|
|
Focusing on Quality: Designing Research for an Adaptable Program
|
| Ann House, SRI International, ann.house@sri.com
|
|
In 2009, Intel launched the Elements courses, a new series in the Teach professional development portfolio. The courses are free to the public, can be implemented through self-study or facilitated offerings, and can be adapted in schedule, platform and design. Rather than trying to describe all the different variations in course offerings and different possible course outcomes, this research was instead designed to provide a documentation of the strategic value of the course to sponsoring stakeholders, establish potential impacts and outcomes for course participants under positive implementation conditions, and identify some best practices and optimal conditions for implementation. The strategy of this approach, then, was to not describe all course uses and contexts since this is untraceable due to its open nature. Instead, this approach worked to establish potential positive impacts and outcomes, while providing implementation models that helped bring about those outcomes.
|
|
|
Finding the Core: Evaluating a Distributed Program
|
| Sophia Mansori, Education Development Center, smansori@edc.org
|
|
Adobe Youth Voices (AYV) is a global youth media initiative that empowers youth in underserved communities with real-world experiences and 21st century tools to communicate their ideas, exhibit their potential, and take action in their communities. AYV includes professional development for educators, access to resources, and communities of practice, delivered through varying channels to support educators in a wide range of settings. For the past 4 years, the program evaluation has worked to understand both the implementation and outcomes of this program as a whole as well as through its variations, which include: formal and informal education settings; face-to-face and online training; projects of different durations and intensity; and youth of different ages, cultures and geographies. Sophia Mansori will share her experience working to design and execute a cohesive evaluation for a program with so many changing and moving components.
| |