| Session Title: Competencies, Credentials, Certification, and Professional Development: Experiences and Issues for Evaluation Policy |
| Multipaper Session 850 to be held in Centennial Section C on Saturday, Nov 8, 10:45 AM to 12:15 PM |
| Sponsored by the Presidential Strand |
| Chair(s): |
| Melvin Mark, Pennsylvania State University, m5m@psu.edu |
| Discussant(s): |
| Thomas Schwandt, University of Illinois Urbana-Champaign, tschwand@uiuc.edu |
| Abstract: As is indicated by the AEA President's statement of the conference theme, various issues fall under the concept of "evaluation policy." Among these are a set of related areas of research and practice that have to do with evaluator, rather than evaluation, quality. These include the specification of evaluator competencies, the process of offering credentials or certification, and the nature of evaluation training. In this session, these issues are examined and discussed, drawing on experiences in four different countries. |
| Next Steps in the Use of the Essential Competencies for Program Evaluators |
| Jean King, University of Minnesota, kingx004@umn.edu |
| Development of the Essential Competencies for Program Evaluators (ECPE) began as an unfunded graduate student project at the University of Minnesota a decade ago. The resulting set of competencies, not yet sponsored by any professional organization, was published in draft form in 2002 and a more developed version in 2005. Since then it has been used in settings around the world. Three activities are appropriate next steps for ECPE development: (1) discussion and possible revision of certain competency statements, (2) formal validation of these competencies using the process outlined in Messick's unified theory, and (3) the development of rubrics for each of the competencies. Potential uses in evaluation policy include (1) establishing selection criteria for external evaluators in government agencies (for example, in New Zealand's Ministry of Education), (2) endorsement or accreditation of training courses and university-based programs, (3) endorsement or credentialing of evaluators, and (4) structuring of evaluation training. |
| Toward Professional Designations for Evaluators: The Canadian Experience |
| J Bradley Cousins, University of Ottawa, bcousins@uottawa.ca |
| Heather Buchanan, Jua Management Consulting Services, hbuchanan@jua.ca |
| Keiko Kuji Shikatani, Independent Consultant, kujikeiko@aol.com |
| Brigitte Maicher, Net Results and Associates, maicherb@nb.sympatico.ca |
| In May 2006 the Canadian Evaluation Society (CES) released an RFP for the development of an action plan for evaluator credentialing. The RFP process was in response to growing national interest in evaluation quality assurance and the professional development and renewal of qualified evaluation practitioners. The process marked the official launch of a year-long, multifaceted national consultation process on whether CES should develop and install a system of professional designations and if so what such a system should look like. Consultation ultimately led to a formal decision by CES National Council to commit to the development and implementation of a system of credentialing for evaluators including as a foundation, a cross-walk of evaluator competencies. This paper describes the development process for the system and plans for further development and implementation. Consideration will be given to challenges encountered along the way and lessons learnt for evaluation policy. |
| New Evaluation Policy for Schools in Japan |
| Masafumi Nagao, International Christian University, nagaom@icu.ac.jp |
| Starting this year (2008), all Japanese K-12 schools, including private ones, must carry out an annual self-evaluation and make public its results. In addition, the schools are advised to have the evaluation results reviewed by stakeholders. Annually at the end of the school year, each school must submit a report on its evaluation exercise to the local Board of Education, which should review the report and, if necessary and appropriate, take supportive action on it. All this results from of a June 2007 act passed in the Japanese national parliament, demanding from schools greater accountability and the practice of continuous improvement for providing quality education. This presentation will: discuss the rationale for this newly instituted policy; analyze factors which will determine its fate, including the reaction of schools and the massive requirement for evaluation training for teachers; and relate this Japanese experience to the broader issue of evaluation policy. |
| Where 'Connectedness to Others and the Land' is an Essential Competency for Evaluators: The Challenges of Building a System of Evaluation Professional Development in a Bi-Cultural Context |
| Kate McKegg, Knowledge Institute Ltd, kate.mckegg@xtra.co.nz |
| In the South Pacific, a diverse group of practitioner evaluators came together recently to form the Aotearoa New Zealand Evaluation Association (anzea). A motivating factor has been awareness by many evaluators of how different our evaluation practice is compared with much of what we read about from overseas. In particular, building connections is a key foundation for the practice of evaluation. Our histories matter here in New Zealand; where we are from and who our families and ancestors are matters. Establishing connection to others and to the land is a critical part of our cultural and evaluation practice. Our 'competence' is as much a function of our technical evaluation skills as it is our ability to connect. This paper will discuss some of the challenges that lie ahead for us in New Zealand as we work to weave this cultural competency into a system of professional evaluation development. |