| Session Title: Using a Longitudinal Mixed Method Approach to Evaluate a Professional Development Community: A Case Study of the Forum for Western Pennsylvania School Superintendents |
| Multipaper Session 896 to be held in Suwannee 14 on Saturday, Nov 14, 3:30 PM to 5:00 PM |
| Sponsored by the Pre-K - 12 Educational Evaluation TIG |
| Chair(s): |
| Cynthia Tananis, University of Pittsburgh, tananis@pitt.edu |
| Abstract: Evaluators often assist programs in developing theories of action and processes for dealing with change. In this session, we present our longitudinal mixed-method evaluation of The Forum for Western Pennsylvania School Superintendents. In its thirteenth year, The Forum provides practical, professionally relevant strategies to alleviate the sense of isolation often accompanying the position of superintendent and to appreciate the complexity of issues increasingly facing the field of education. The Forum is a relationship-oriented and socially-interconnected professional development program that understands and impacts both the social contexts and effective practices that impact the educational task at macro- and micro- levels. In this session, we display several components of the overall Forum evaluation plan. Through our evaluation, we not only describe and analyze outcomes, but also portray the patterns of culture and interaction that express the entirety of the program's story. |
| The Role of Evaluation in Developing Program Theory: Using Retrospective Logic Modeling To Tell a Program's Story |
| Keith Trahan, University of Pittsburgh, kwt2@pitt.edu |
| Cara Ciminillo, University of Pittsburgh, ciminillo@gmail.com |
| Cynthia Tananis, University of Pittsburgh, tananis@pitt.edu |
| In this paper, we use a retrospective logic-modeling strategy to portray the Forum's theory of action, chronicle the evolution of the program, and inform future change. Our evaluation plan utilizes a longitudinal mixed-methods approach to help tell the program's story. Our meta-analysis of evaluation findings helps to position the Forum within the discourse of education professional development. We use the phrase community of educational practice and social action to identify the Forum as a hybrid of communities of practice and professional learning communities. The Forum is a relationship-oriented and socially-interconnected professional development program that understands and impacts both the social contexts and effective practices that impact the educational task at macro- and micro- levels. This paper illustrates the role of evaluation in developing program theory. |
| Organizational Evolution: Recognizing and Transferring Culture to Newcomers |
| Cynthia Tananis, University of Pittsburgh, tananis@pitt.edu |
| Keith Trahan, University of Pittsburgh, kwt2@pitt.edu |
| As the Forum has evolved over the last 14 years, it has matured as a special community of practice and learning. Members developed close, professionally intimate relationships that broke down the sense of isolation that is characteristic of the Superintendents. As long-timers age and retire from practice and newcomers are invited into the group, the nature of community within the organization changes. This paper examines these issues through the use of focus group data. |
| Telling the Forum's Membership Story: Portraying Change Using the Basics of Geographic Information Systems (GIS) |
| Keith Trahan, University of Pittsburgh, kwt2@pitt.edu |
| Justin Rodgers, University of Pittsburgh, jtr30@pitt.edu |
| In this paper, we discuss our use of geographic information systems to construct a visual narrative of program membership change. Since its founding in 1996, membership in The Forum for Western Pennsylvania School Superintendents has grown from a concentration around Pittsburgh to a broader regional presence. Our evaluation task is to research and display the Forum's changing presence in terms of member and district demographics and to highlight collegial networks that have grown out of the Forum. Thus, this work serves as a foundation for our future evaluation project in social network analysis. Using GIS we are able to create a more coherent portrayal of membership change than we are with standard report writing. While this application is a minimal use of the applicative power of GIS software, our aim is to show that evaluation practitioners with working knowledge of information systems can map membership change using GIS. |
| Leveraging Resources Through Evaluation |
| Cynthia Tananis, University of Pittsburgh, tananis@pitt.edu |
| This paper explores the ways in which initiatives from member Superintendents, provided with small seed-funding from the Forum, used evaluation capacity building to refine planning and implementation in ways that leveraged additional funding and matching resources to expand initiatives. Evaluation can help organizations build a compelling case of impact and further potential that can elicit expanded support from decision-makers and funders. |