| Session Title: Educational Evaluation in an Era of Globalization: New Tensions and Contradictions |
| Multipaper Session 297 to be held in Capitol Ballroom Section 4 on Thursday, Nov 6, 1:40 PM to 3:10 PM |
| Sponsored by the Theories of Evaluation TIG |
| Chair(s): |
| J Bradley Cousins, University of Ottawa, bcousins@uottawa.ca |
| Abstract: Nearly all aspects of modern life (e.g., culture, employment) are being changed as society is being increasingly globalized by market integration and the emergence of new global media and information technologies that permit the circulation of ideas, resources, and even individuals across boundaries. In this sense, evaluation theories and how practices are being 'globalized' as they are being influenced by the movement of ideas across national boundaries. Yet the extent to which and evaluation theories and practices are changing in response to globalization and other political and social changes has not been sufficiently examined. Using educational evaluation as the backdrop, this session is devoted to considering how these political and social changes create new contradictions and tensions for foundational evaluation issues. Each paper presents a particular turn on these contradictions and tensions in relation to fundamental issues, the role of science in evaluation, capacity building and monitoring, and democracy and participation. |
| Fundamental Evaluation Issues in a Global Society |
| Nick L Smith, Syracuse University, nlsmith@syr.edu |
| Fundamental issues are those underlying problems or choices that continually resurface in different guises throughout evaluation work. What are the proper roles for the evaluator as a professional? What is the nature of acceptable evidence in making evaluative judgments? Such issues are, by their very nature, never finally solved, but only temporarily resolved. Each resolution is highly dependent on a complex of interrelated historical, contextual, and cultural forces. This paper reviews 10 fundamental issues that will continue to reflect the changing nature of educational evaluation both within the US and globally. Resolutions to these issues are constantly being reshaped by the increasingly diverse, globalized nature of society; driving forces are briefly reviewed. Although common resolutions might facilitate cross-cultural evaluation collaboration, it is more likely that the evaluation profession will have to learn to flexibly accommodate to rapid redefinitions of evaluation theory, methods, and practice as the world becomes increasingly global. |
| Science, Evaluation, and Sense-Making |
| Melvin Mark, Pennsylvania State University, m5m@psu.edu |
| What is the role of educational evaluation, science and sense-making in a globalized society, and how can evaluation theory and practice be better suited for this role? I first briefly review the intersection between globalization-related trends and evaluation. I then offer observations on contemporary debates about the role of randomized experiments. The focal point of the paper is on a set of recommendations for future work, including: the need to develop procedures for macro-level evaluability assessment, to examine the readiness for evaluation in emerging policy and program areas; the value of developing models for communicating the rationale for choosing one (set of) potential evaluation questions) above others; the desirability of focusing on evaluation portfolios, rather than debating a single method or approach; and the importance of evaluators (and others) recognizing there is no single pathway by which evaluation can serve democracy - which should not be surprising in the face of mixed-model democratic governance. |
| Evaluation, Accountability and Performance Measurement in National Education Systems: Trends, Methods, And Issues |
| Irwin Feller, Pennsylvania State University, iqf@ems.psu.edu |
| Katherine Ryan, University of Illinois Urbana-Champaign, k-ryan6@uiuc.edu |
| The role of evaluation in relationship to accountability is receiving substantial attention and Ryan (2002, p. 453) has previously called upon the evaluation community to 'play an integral part in the study of, improvement of, and judgments' about the merit and worth of accountability systems. This paper is crafted both as a general and specific response to this call. It begins with a general overview of international trends toward new more formal requirements for accountability and performance measurement with emphasis on dominant 'global' requirements and precepts of the new public management about how public sector activities are to be managed, monitored, and measured. With consideration of the differentiated mix of interests of public and private institutions, the context of 'local' national systems of education and the evaluation of national K-12 and higher education systems, the paper concludes by discussing the implications of this analysis for the design and conduct of evaluations, and by posing additional policy and research questions. |
| Serving the Public Interest through Educational Evaluation: Salvaging Democracy by Rejecting Neo-Liberalism |
| Sandra Mathison, University of British Columbia, sandra.mathison@ubc.ca |
| Evaluation's role in the current educational reform movement is connected to both the neo-liberal agenda of free market capitalism and the neo-conservative agenda of Christian values. And, the State plays a significant role in facilitating the implementation of these agendas. These respective economic and moral neo agendas directly affect the purposes of educational evaluation, methodological choices, and conceptions of quality in education. Implicit in the neo agendas is the assumption that a select group of individuals is best positioned to determine what is in the public interest. However, I will argue that this agenda serves particular interests, the private and individual interests of a corporate ruling class, and that the public interest is not well served when evaluation is directed by these neo agendas. The paper also considers countervailing forces in evaluation, ones that support a grassroots, communitarian approach to the public interest. These alternatives and their potential for serving the public interest will be discussed. |