Return to search form  

Session Title: Multiple Levels of Evaluation: Organizational Learning in a Foundation-Supported Initiative

Panel Session 787 to be held in D'Alesandro Room on Saturday, November 10, 12:10 PM to 1:40 PM

Sponsored by the Non-profit and Foundations Evaluation TIG

Chair(s):

Andrea payday loan  Berger payday loan ,  American Institutes for Research,  aberger@air.org

Discussant(s):

Victor payday loan  Kuo payday loan ,  Bill & Melinda Gates Foundation,  victor.kuo@gatesfoundation.org

Abstract: How can a foundation encourage participant learning among school sites, foundation grantees, and external evaluators in one large initiative? This panel will discuss a nationwide initiative's unique organization for using data and evaluation findings. A lead evaluator will describe how multiple levels of evaluation activities, including monitoring, formative, and summative evaluation efforts, work together to meet different audience needs. A grantee representative will discuss how school data are used for accountability purposes within a network and for sharing with the foundation. A representative from the initiative's coordinating organization will discuss its unique role in connecting everyone around data sharing and learning from the evaluation. Finally, a foundation representative will discuss how all of these organizations meet the foundation's grant oversight needs. She will also talk about this initiative's unique organization of participants, how that differs from other initiatives, and what the benefits and drawbacks have been.

Collaboration Within a National Evaluation Andrea payday loan  Berger payday loan ,  American Institutes for Research,  aberger@air.org Two organizations, the American Institutes for Research (AIR) and SRI International (SRI), are partnering to conduct the ECHSI national evaluation. Andrea Berger has been the AIR evaluation director since its inception in 2002. She will discuss how collaborations with ECHSI participants have informed the evaluation. First, the evaluation team collaborates with JFF on two data collection activities: a school survey and the SIS, which houses student-level data. The evaluation team also formally and informally discusses the status of the ECHSs and the initiative overall with JFF. This close collaboration eases participants' reporting burden and keeps the evaluation team informed about the activities and challenges that are occurring. Annually, the evaluation team works with the foundation to reflect on data collection and reporting activities and develop approaches to keep the evaluation integral to the ECHSI.

Participating in and Learning From Evaluation Linda payday loan  Campbell payday loan ,  Center for Native American Educational Advancement,  lcampbell@antiochsea.edu Linda Campbell has overseen this grantee's work since the first round of ECHSI grants were made in 2002. Having been involved with this initiative since its inception, she has a perspective on how the evaluation work and interaction between the participants has grown and changed over the years. She will discuss what information needs are met by the national evaluation and by the work done by JFF. In particular, she will discuss her organization's involvement in the development of the SIS and in the accountability activities facilities by JFF (e.g., annual reviews with the foundation). She will also discuss her own organization's data collection and reporting from the perspective of working with rural and under-funded schools and districts, and with Native American students. Finally, Dr. Campbell will discuss the benefits and drawbacks of these multiple levels of collaboration.

Connecting all of the Evaluation Pieces to Facilitate Initiative Learning Leslie payday loan  Haynes payday loan ,  Jobs for the Future,  lhaynes@jff.org Leslie Haynes will discuss the many ways that JFF supports ECHSI participant information sharing and collaboration. Most of these supports stem from JFF's role as technical assistance provider. This role has evolved since 2002. JFF facilitates all of the annual reviews between the grantees and the foundation. These reviews represent an opportunity for foundation staff to learn about each intermediary's progress, collectively reflect on the work, and strategize about future plans. JFF also oversees the SIS and other initiative-wide data collection activities. These activities contribute to JFF's understanding of the overall initiative and its participants, placing JFF in a strong position to provide technical assistance and coordinate peer learning. JFF has also used its role as a technical assistance provider to support the activities of the national evaluation and to build learning opportunities around findings from the national evaluation.

Facilitating Connections Across the Initiative to Meet Funder Information Needs Yee-Ann payday loan  Cho payday loan ,  Bill & Melinda Gates Foundation,  yee-ann.cho@gatesfoundation.org Yee-Ann Cho is the foundation program officer overseeing the ECHSI. She will present her perspective as a consumer of the multiple levels of evaluation work. She will discuss how the foundation's evaluation efforts have evolved to include monitoring and formative and summative evaluation activities, and how the ECHSI evaluation exemplifies this multi-pronged approach. She will also discuss how these efforts ultimately meet the needs of the grantees, JFF, the foundation, and other audiences. The foundation supports the use of evaluation by grantees and JFF for learning and the annual review of grantee progress. Ms. Cho will also describe how the evaluation addresses learning goals articulated by the foundation's leadership. Finally, Ms. Cho will discuss the benefits of designing evaluations and program initiatives in tandem and will offer suggestions for how program officers and evaluators can work together to maximize evaluation use and learning.