| Session Title: Conceptual and Empirical Advances in Evaluation Capacity Building |
| Multipaper Session 410 to be held in International Ballroom D on Thursday, November 8, 3:35 PM to 5:05 PM |
| Sponsored by the Evaluation Use TIG |
| Chair(s): |
| J Bradley Cousins, University of Ottawa, bcousins@uottawa.ca |
| Discussant(s): |
| Peter Dahler-Larsen, University of Southern Denmark, pdl@sam.sdu.dk |
| Abstract: Understanding evaluation capacity, and perhaps more importantly, how to go about fostering, developing or otherwise promoting it - that is, evaluation capacity building (ECB) - has captured the attention of many contributors to the field, not the least of whom are those involved in ECB practice. Yet conceptual advances and empirical research on ECB have been comparatively limited to date. Research and theory stand to benefit, for example, from broadening our understanding of evaluation capacity to include consequences of evaluation (Compton, Braizerman & Stockdill, 2002), and moving beyond organizational contexts to consider implications of ECB on a grander scale (Levin-Rozalis, Rosenstein & Cousins, forthcoming). This multiple paper session will showcase conceptual and empirical advances in ECB and contribute to the development of an agenda for ongoing inquiry that might usefully inform ECB practice. |
| Integrating Consequences of Evaluation into Evaluation Capacity Building Inquiry |
| J Bradley Cousins, University of Ottawa, bcousins@uottawa.ca |
| Swee C Goh, University of Ottawa, goh@admin.uottawa.ca |
| Catherine Elliott, University of Ottawa, elliott.young@sympatico.ca |
| A significant conceptual development in evaluation capacity building inquiry is reflected in a commitment to understand and foster the capacity to use evaluation (Compton, et al., 2002; Cousins et al. 2004). This paper describes the development and evolution of a conceptual framework intended to guide empirical inquiry that captures this perspective. The capacity to do evaluation is situated among conceptual consequences of evaluation and conditions that mediate such influences. Contributions to framework evolution of a concept mapping study and recent developments in the evaluation utilization literature are identified and implications for empirical inquiry are discussed. |
| Precarious Balance |
| Miri Levin-Rozalis, Ben-Gurion University of Negev, rozalis@internet-zahav.net |
| The rift in educational evaluation between the 'systems' vs. 'human agencies' approaches stemming from rival ontological conceptions reflects on Evaluation Capacity Building (ECB) processes of both evaluators and evaluee. An axiological 'ethical purposive' approach can help us to overcome this rift. The ends of inquiry into human activity are first ethical and political, concerning the worthwhile life and the just society and only secondarily epistemology or methodology. The main question for ECB is not "how to do" but "what for?" and thus "what difference should evaluation make?" The answers should lead our decisions concerning methodologies and processes that will keep the precarious balance between contradicting demands. |
| Evaluation Capacity Across Organizational Contexts: What are the Patterns? |
| Swee C Goh, University of Ottawa, goh@admin.uottawa.ca |
| Robert Lahey, REL Solutions Inc, relahey@rogers.com |
| Catherine Elliott, University of Ottawa, elliott.young@sympatico.ca |
| Jill Chouinard, University of Ottawa, jchou042@uottawa.ca |
| Keiko Kuji-Shikatani, Independent Consultant, kujikeiko@aol.com |
| This paper reports preliminary within- and cross-case results from a larger multiple case study designed to inquire into the nature and state of evaluation capacity and its antecedents. Of particular interest is the capacity to use evaluation and its relationship to the capacity to perform and/or oversee the function. Organizations included in this paper vary from agencies that fund community social services, to large federal government departments, to a secondary educational institution. Despite enormous variation in organizational context and evaluation capacity, within- and cross-case analyses are guided by a common conceptual framework. Emergent patterns are identified and discussed in terms of implications for ongoing inquiry. |
| A Case Study of a Successful School Evaluation Capacity Building Process |
| Barbara Rosenstein, Ben-Gurion University of Negev, rbarbara@netvision.net.il |
| Pnina Elal-Englert, Ben-Gurion University of Negev, rbarbara@netvision.net.il |
| This paper presents an evaluation capacity building (ECB) case of a school facing the challenge of mainstreaming children with special needs. The school has developed a successful program of ECB over a period of 10 years. We examine the case according to three separate but complimentary frameworks: first, that set out by Compton, Baizerman, Stockdill (2002) in which they distinguish between program evaluation practitioner and ECB practitioner. Second, we demonstrate King's (2002) prescriptions for successfully operating an ECB system in a school in relation to three stakeholders groups: teachers, administrators and opinion leaders; students and the curriculum. Third, we illustrate how the case fits Cousins' (2005) discussion of the role of the principal in school organizational learning. After thorough and systematic investigation, we believe that ECB developments at this school can serve as a model for other schools with similar contexts. |