| Session Title: Practicing What we Preach: Exploring the Transformative Potential of Evaluation Processes |
| Multipaper Session 337 to be held in International Ballroom B on Thursday, November 8, 11:15 AM to 12:45 PM |
| Sponsored by the Multiethnic Issues in Evaluation TIG |
| Chair(s): |
| Tanya Brown, Duquesne University, jaderunner98@gmail.com |
| Rodney Hopson, Duquesne University, hopson@duq.edu |
| Discussant(s): |
| Karen Kirkhart, Syracuse University, kirkhart@syr.edu |
| Stafford Hood, Arizona State University, stafford.hood@asu.edu |
| Abstract: How do our practices change once we acknowledge that learning within evaluation is dynamic and multi-directional? This question becomes even more prescient when we align it with dominant concerns of social justice and social change in evaluation practice, and the multiple learning capacities within the field. This panel, students of the AEA/DU Graduate Education Diversity Internship, provides accounts of dynamic learning processes that take place over the course of an evaluation. Each paper discusses how the evaluator navigated through the evaluation process, with special focus on one of the following: (1) attending to the interpersonal processes between evaluator and stakeholders; (2) utilizing theories of practice that uniquely address the concerns of the evaluation context; and (3) considering how the learning processes of a particular program inform and map onto the evaluation process and the evaluator's development. Discussants lift the presenters' learning experiences to proffer further lessons on evaluation process and its transformative potential. |
| Planting Collaborative Growth: Coalition Building as a Key Element to the Evaluation Process |
| Nia Davis, University of New Orleans, nkdavis@hlkn.tamu.edu |
| Since 1991, The United State Department of Justice has implemented Operation Weed and Seed (OWS) in sites across the country with the aim to simultaneously reduce crime and bolster community development. OWS New Orleans has adopted a coalition structure, comprised of representatives of organizations or community groups with a vested interest in the residential area designated as the community of focus. Unique to the coalition structure is the integration of the research and evaluation contacts, who collaborate with community representatives on a regular basis. Critical then, was an attunement to interpersonal relationships amongst coalition members, trust building, and the development of the community through the OWS initiative. This presentation highlights the evaluation activities employed to build cohesion and commitment to community development among coalition participants. The presenter also parallels this process to her own development as an evaluator. |
| An Analysis of Organizational Capacity and Research Inquires: Incorporating Cultural Competence in Evaluation Research Agendas |
| Milton Ortega, Portland State University, mao@pdx.edu |
| The literature on cultural competency in evaluation research has grown considerably over the last decade. However, relatively little has been done to implement coherent evaluation practices in accordance with cultural competency. This lack of attention to organizational capacity may be further echoed in the failure of some program evaluations to place cultural competency centrally in research agendas. The pursuit of cultural competence in research evaluation is further constrained by a lack of organizational and methodological approaches. This paper examines the organizational learning capacities of a research evaluation organization in its attempts to incorporate cultural competency in its own evaluation projects. The purpose of this analysis is to provide organizations with some recommendations in preparing for research inquires that promote cultural competency, with the hope that better understandings be attained. |
| Illuminating Community Meanings: Utilization of a Narrative Framework to Document Community Change |
| Josephine Sirineo, University of Michigan, jsirineo@umich.edu |
| The success of an evaluation is largely dependent on gathering information that accurately depicts a context under investigation. How people make sense of their environment (Weick, 1995) and how they choose to communicate their experiences to others are important concepts for evaluators to recognize throughout the evaluation process. Tobin (2005) notes how storytelling can be an integral component in program evaluation when it is used as a primary or secondary data gathering technique. This paper will present a framework that documents a process of applying the Most Significant Change methodology to a national, multi-site, cluster evaluation. The MSC technique is a systematic process for recording, collecting and analyzing stories around specific themes (Davies and Dart, 2005). Preliminary findings show that storytelling can demonstrate the dynamic nature of learning occurring at different levels (individual, group, and organization) and with varying intensities. |
| Evaluation of Non-Traditional Approaches for Preventing High School Dropout |
| Roderick Harris, University of Pittsburgh, rlh1914@yahoo.com |
| Experiential education is the process of actively engaging students in an authentic experience that will have benefits and consequences. Students make discoveries and experiment with knowledge themselves instead of hearing or reading about the experiences of others. Students also reflect on their experiences, thus developing new skills, new attitudes, and new theories or ways of thinking (Kraft & Sakofs, 1988). This presentation discusses the candid field experience of a novice evaluator who used an experiential approach to learn practical program evaluation within the context of a high school dropout prevention organization, Communities in Schools (CIS). Since 1985 CIS aims to help young people successfully transition out of high school and build on their potential. Building on the tenets of experiential education and the aims of CIS, the presenter will outline participatory methods used for evaluating an in-school program, and two different alternative learning academies. |