|
Session Title: Evaluations of Professional Development in Education: When 'Leave Them Smiling' Is Not Enough
|
|
Panel Session 884 to be held in Mineral Hall Section F on Saturday, Nov 8, 1:20 PM to 2:50 PM
|
|
Sponsored by the Pre-K - 12 Educational Evaluation TIG
|
| Chair(s): |
| Julie Morrison,
University of Cincinnati,
julie.morrison@uc.edu
|
| Abstract:
Although the evaluation of professional development is a critical component in the delivery of professional development, it is often poorly conceived. Notable improvements in student learning almost never take place without professional development (Guskey, 2000). We spend $5 to $12 billion annually on professional development (USDOE, 2008). This panel aims at helping evaluators design high-impact evaluations of professional development. The context for the current emphasis on the professional development of educators is presented in Presentation I. Presentation II provides an overview of Guskey's Model for evaluating professional development and how evaluative criteria and determination of merit can be integrated into the model. Presentation III explicates problems inherent in the current focus on participants' reactions to professional development. Presentation IV presents practical guidelines for improving the assessment of participants' reactions and improving evaluations of professional development. The last presentation highlights how new policies might ensure meaningful evaluation of professional development.
|
|
Context for Professional Development in Education: Policy Implications
|
| Imelda Castaneda-Emenaker,
University of Cincinnati,
castania@ucmail.uc.edu
|
| Julie Morrison,
University of Cincinnati,
julie.morrison@uc.edu
|
|
This presentation explores the historical and legal context for the emphasis on professional development (PD) of educators. The federal mandates embedded in No Child Left Behind and the long-standing requirement for professional development from Title I provide the context for creating more 'highly qualified' teachers. Compliance with federal, state, and district mandates have led educators to various models of PD, such as training, observation/assessment, development/improvement process, study groups, inquiry/action research, individually guided activities, and mentoring/coaching. Teacher PD comes at a considerable cost. More often, teacher PD competes with the financial allocations for other programs needed to be implemented to comply with the mandates. Although evaluation of professional development is identified as a critical component in the delivery of professional development, it is often poorly conceived, and has very limited, if not minimal, resource allocations. Future policy must include guidelines for high-impact evaluations of these often expensive, yet essential PD activities.
|
|
|
Integrating Evaluative Criteria and Merit Determination into the Evaluation of Professional Development
|
| Catherine Maltbie,
University of Cincinnati,
maltbicv@ucmail.uc.edu
|
| Julie Morrison,
University of Cincinnati,
julie.morrison@uc.edu
|
|
This presentation integrates Guskey's (2000) model for evaluating professional development with evaluative criteria and merit determination (Davidson, 2006). Guskey proposed a model for evaluating the impact of professional development that is comprised of five levels: (1) participants' reactions, (2) participants' learning, (3) organization support and change, (4) participants' use of new knowledge and skills, and (5) student learning outcomes. Generally, evaluators apply the program standards of feasibility, propriety, accuracy, and utility to uphold the value of evaluation activity. This presentation discusses how the evaluation program standards might be of special interest, or concern, at each of the five levels presented in Guskey's model.
| |
|
Removing the Rose-Colored Glasses: What are Participants' Reactions to Professional Development Really Saying?
|
| Janet Matulis,
University of Cincinnati,
jmatulis@ucmail.uc.edu
|
| Jerry Jordan,
University of Cincinnati,
jerry.jordan@uc.edu
|
|
Participants' evaluations of professional development workshops tend to be positive, in spite of the wide variations in the quality of these sessions. This presentation, informed by research in social psychology, highlights the significant dangers of misinterpreting data obtained from surveys of participants' reactions (Guskey's Level 1). This presentation discusses the various factors that confound the assessment of participants' reactions and provides evaluators with recommendations for maximizing the value of the post-workshop evaluations. Evaluation policy implications for measuring the impact of professional development evaluations will be discussed.
| |
|
Practical Guidelines for Improving the Evaluation of Professional Development in Education
|
| Julie Morrison,
University of Cincinnati,
julie.morrison@uc.edu
|
| Catherine Maltbie,
University of Cincinnati,
maltbicv@ucmail.uc.edu
|
|
Critical domains need to be established as standard practice in the evaluation of professional development workshops for educators. Beyond these standard domains, the evaluation should be carefully tailored to the unique professional development experience. The objective of the professional development experience (i.e., awareness, comprehension, application, analysis, synthesis, evaluation) and the expectations for learning (i.e., acquisition, fluency, generalization, and adaptation) need to be clarified. In light of the professional development objectives, various methods for assessing participants' reactions (and learning) will be presented, along with their advantages and disadvantages. The policy implications of improving the accuracy and utility of the evaluations of professional development for educators will be discussed.
| |
|
The Path Forward: Future Policy for the Evaluation of Professional Development for Educators
|
| Jerry Jordan,
University of Cincinnati,
jerry.jordan@uc.edu
|
| Imelda Castaneda-Emenaker,
University of Cincinnati,
castania@ucmail.uc.edu
|
|
Government (and organizational) policies should require the development of meaningful evaluative criteria for the evaluation of professional development. It is imperative that participants' Reactions (Level 1) NOT be misinterpreted as evidence of participants' learning (Level 2), organization support and change (Level 3), participants' use of new knowledge and skills (Level 4), and student learning outcomes (Level 5). Employing evaluative thinking in a traditional professional development evaluation model such as Guskey's could bring about a powerful tool in conducting high-impact, meaningful evaluations. Policies for the evaluation of professional development for educators should consider the breadth and depth of evaluation in terms of the issues for consideration and stakeholder participation, as well as provisions for optimal cost allocations.
| |