Return to search form  

Session Title: Evaluating the Teaching of Program Evaluation: Student and Teacher Assessments

Panel Session 119 to be held in Calvert Ballroom Salon B on Wednesday, November 7, 4:30 PM to 6:00 PM

Sponsored by the Teaching of Evaluation TIG

Chair(s):

Katherine payday loan  McDonald payday loan ,  Portland State University,  kmcdona@pdx.edu

Abstract: Program evaluation has grown significantly since the 1960's (Rossi, Lipsey & Freeman 2004). As such, educating new program evaluators is an important aspect of promoting healthy growth of the field. However, we have conducted relatively few evaluations of the effectiveness of opportunities to learn about program evaluation (Trevisan, 2004). In this presentation, we will share findings from an evaluation of graduate students' learning in a program evaluation seminar and generate a vision for critical next steps. We assessed learning through student and teacher judgments of knowledge and skill demonstration. In some instances, we will draw comparisons between students and their peers not enrolled in the seminar and other evaluation novices in professional training opportunities (Stufflebeam & Wingate, 2005). Audience ideas and insights will be solicited throughout the presentation and implications for the field will be discussed interactively.

Ideas for Evaluating the Teaching and Learning of Program Evaluation Katherine payday loan  McDonald payday loan ,  Portland State University,  kmcdona@pdx.edu We will begin the presentation by asking participants to brainstorm ideas about how we can evaluate the teaching and learning of program evaluation. For example, we will ask participants to identify what questions we can ask and ideas for how these questions might be addressed. These ideas will be tracked on easel paper with post-it notes and organized thematically. After ideas have been generated, we will also ask participants to rank order each type of evaluative question in terms of its importance. These individual rankings will be examined to produce a group ranking of most to least important questions to consider in evaluating the teaching of program evaluation. This list will be used again in the final presentation to generate next steps.

Student Assessment of Gains in Knowledge of Program Evaluation Lauren payday loan  Denneson payday loan ,  Portland State University,  laured@pdx.edu Students' learning of program evaluation can be studied by examining their assessments of their knowledge. We gathered data on students' self-assessment of program evaluation knowledge through three means. First, students completed the Self-Assessment of Program Evaluation Expertise (Stufflebeam, 2001) at the beginning and end of the seminar; a group of students not in the seminar also completed this assessment. Second, students completed weekly reflections on course readings. Third, students rated the quality of their performance on assignments and reflected on factors that contributed to their learning at the end of the term. Here, we will present findings on students' perceived knowledge of the field. As appropriate, we draw comparisons between self-assessed knowledge of students in the seminar, students not in the seminar and individuals in professional development program evaluation workshops (Stufflebeam & Wingate, 2005). We discuss the benefits and limitations of examining students' assessment of their learning.

Teacher Assessment of Gains in Knowledge of Program Evaluation Tina payday loan  Taylor-Ritzler payday loan ,  University of Illinois, Chicago,  tritzler@uic.edu Students' learning of program evaluation is also examined through teachers' judgments of students' knowledge and skill. Here, we examine evidence for students' knowledge and skill through four means: weekly reflections on course readings, pre-evaluation proposals, and written and oral evaluation proposals. Specifically, we developed coding schemas (in collaboration with established evaluators) to assess students' evaluation-specific knowledge and skills and applied the schema to each data source. We will present thematic findings of teacher judgment of knowledge and skill application. Lastly, we will discuss the benefits and limitations of examining teacher assessment of students' learning.

Relationships Between Student and Teacher Assessment of Gains in Knowledge of Program Evaluation Margaret payday loan  Braun payday loan ,  Portland State University,  pdx01350@pdx.edu Student and teacher assessments of students' learning complement one another to more fully illuminate what, and how, students learn about program evaluation in a graduate seminar (Trevisan, 2004). That is, each source of data provide unique information about what, and how, students make knowledge and skill gains. Here, we examine relationships between these two forms of learning assessment. Specifically, we will examine each source's unique contribution to addressing what, and how, graduate students learn about program evaluation in a seminar on the topic along places of convergence and divergence of findings.

Generating a Vision for the Evaluation of Teaching Program Evaluation Shannon payday loan  Myrick payday loan ,  Portland State University,  shannonm@pdx.edu Here, we will integrate two sources of information that inform the evaluation of teaching program evaluation: (1) Previous evaluation on the topic (e.g., Febey & Coyne, 2007; Trevisan, 2004) and (2) Findings from Presentations 2, 3, and 4. From this synthesis, we will examine the current state of knowledge about the evaluation of teaching program evaluation and solicit audience input on the interpretation of this knowledge base. Then, we will compare that knowledge base to the audience-rated priorities for questions to answer generated earlier in the presentation. Through a participatory process, we will identify and reflect on knowledge gaps and begin visioning next steps for answering important questions in the evaluation of teaching program evaluation.