| Session Title: Embedded Evaluation in Out of School Programs |
| Multipaper Session 723 to be held in Suwannee 13 on Saturday, Nov 14, 9:15 AM to 10:45 AM |
| Sponsored by the Pre-K - 12 Educational Evaluation TIG |
| Chair(s): |
| Patricia Jessup, InSites, pjessup@insites.org |
| Discussant(s): |
| Alyssa Na'im, Education Development Center Inc, anaim@edc.org |
| Abstract: Evaluation is embedded in each of the National Science Foundation funded programs discussed in this session in a different way with each program situated in a different out of school context. The first paper focuses on embedding evaluation in the program using a particular evaluation tool - concept mapping' with both teachers and students. The second paper focuses on embedding evaluation in a culturally responsive way that benefits both students and program leaders. The third paper looks at embedding evaluation in the after school community and education infrastructures to help sustain the program in both a school district and a community organization. How to integrate evaluation into programs and their contexts, embed evaluation, in ways that enhance the value of evaluation is an important issue in the formal K-12 education system as well. These papers provide important ways to strengthen evaluation in both formal and informal education settings. |
| Increasing Participant Investment in the Evaluation Process Through Concept Maps |
| Rachel Becker-Klein, PEER Associates, rachel@peerassociates.net |
| Andrew Powers, PEER Associates, andrew@peerassociates.net |
| The Community Science Investigators (CSI) program employs a professional development model of program delivery. CSI staff conduct workshops and ongoing professional development with classroom educators, in the areas of Geographic Information Systems (GIS), Augmented Reality (AR) and Service-learning (SL). In turn, the educators use their newfound knowledge in these three areas to structure an out-of-school time program for middle-school students that engages students in learning about science, technology, and nature. During the professional development training, CSI staff have educators fill out concept maps before and after learning about GIS, AR, and SL, in order to assess educators' knowledge in these areas. The educators' concept maps fulfill the purposes of: 1) assessing changes in educators' content knowledge; 2) increasing educators' investment in the evaluation process; and 3) demonstrating the value of using concept mapping to assess students' content knowledge. CSI staff expect that educators will have students complete concept maps over the course of the year. |
| Culturally-responsive Evaluation as Reciprocal Practice: How Evaluation Can Give-Back to Participants |
| Phyllis Ault, Northwest Regional Educational Laboratory, aultp@nwrel.org |
| This presentation examines implications of using culturally-responsive and transformative approaches to increase our understanding of how evaluation can contribute to improving outcomes for underrepresented students. Increasingly, evaluators recognize that cultural understanding is needed to derive meaning from student performance and measure changes over time. Yet cultural competence is arguably a minimum expectation of responsiveness to community norms and values. In projects such as those with American Indian students, evaluators should also be attentive to ways in which evaluation "gives-back" to students, enhancing their understanding or skills, and building on successes. This reciprocity in evaluation seeks to find methodologies enabling project leaders and evaluators to gauge the influence of a given project while simultaneously helping participants grow. Using the example of Salmon Camp Research Teams funded by the National Science Foundation, the presentation will discuss methodologies for data collection that have positively influenced participating students (all of whom have American Indian affiliation). |
| Using Evaluation to Encourage Sustainability of Out-of-School Programs |
| Patricia Jessup, InSites, pjessup@insites.org |
| Tirupalavanam Ganesh, Arizona State University, tganesh@asu.edu |
| Beverly Parsons, InSites, bparsons@insites.org |
| The three-year Learning through Engineering Design and Practice program (funded by the National Science Foundation) offers seventh and eighth grade students informal education experiences. The program, located in Arizona, includes an after-school component where participants learn engineering and information technology skills through activities such as simulating desert tortoise behaviors, researching and developing designs to mitigate the urban heat island, and designing autonomous rovers capable of navigating Mars-like terrain. They also participate in leadership development activities including summer activities serving as docents for younger children at the local science center and internships with a local water conservation project. The paper discusses the evaluation approaches used to investigate what students are learning, the differences in curriculum for out-of-school versus in-school programs, and issues of sustainability of the work. The evaluation blends external and embedded evaluation and is grounded in supportive relationships between the evaluators and the program director. |