|
Session Title: Evaluation in the Field: Developing Local Practitioners' Evaluative Thinking
|
|
Panel Session 859 to be held in Mineral Hall Section D on Saturday, Nov 8, 10:45 AM to 12:15 PM
|
|
Sponsored by the Organizational Learning and Evaluation Capacity Building TIG
|
| Chair(s): |
| Jennifer Greene,
University of Illinois Urbana-Champaign,
jcgreene@uiuc.edu
|
| Discussant(s):
|
| Jennifer Greene,
University of Illinois Urbana-Champaign,
jcgreene@uiuc.edu
|
| Abstract:
In this session, a distinctive model of evaluation practice is shared - a model that blends an educative and a capacity building intent with self-reflective evaluation practice and progress. A diverse group of R&D units and university-based evaluators in Sweden has been contracting with local municipal authorities to conduct an 18-month 'workshop' for staff in 6-8 municipal projects. The group meets once a month for one full day at a municipal location. At these meetings, the instructor offers short lectures on key topics in evaluation, and the group critically discusses evaluation progress on one or more of the projects involved. In between meetings, the staff in the projects works on their own evaluations. This distinctive model of evaluation, named Evaluation Verkstad Practice (EVP), offers promise to both enhance the defensibility of evaluation results in the region and to support more reflective evaluative thinking among professional municipal staff.
|
|
Evaluation Verkstad Practice (EVP): Basic Ideas, Structure and Content
|
| Elisabeth Beijer,
FoU i Väst,
elisabeth.beijer@gr.to
|
| Bengel Eriksson,
Karlstad University,
bengt-g.eriksson@kau.se
|
| Per-Ake Karlsson,
Borss University,
per-ake.karlsson@hb.se
|
| Tom Leissner,
Goteborg University,
tom.leissner@socwork.gu.se
|
|
Evaluation Verkstad Practice (EVP) constitutes an activity that accomplishes the combined purpose of conducting evaluations and developing competence to conduct evaluations, with the support of R&D units or university-based evaluators. The EVP featured in this presentation brings together groups of participants from different welfare organizations and workplaces. The participants have an assignment from their own organizations to conduct an evaluation of a specific object or program. The EVPs support the participants' evaluation activities through themed mini-lectures, and through a process of supervision, dialogue and reflection around issues that arise while the evaluations are in process. Further, the EVP supports the development of participant competence in evaluation more broadly, primarily through peer interaction and critical reflection. The evaluations conducted at the workshops are primarily internal, but with external support. EVP have a beneficial effect on the learning of evaluation methods by directly combining learning and conducting evaluation.
|
|
|
The Program Theory of Evaluation Verkstad Practice
|
| Kari Jess,
Malardalen University,
kari.jess@mdh.se
|
|
This paper explores the program theory of the Evaluation Verkstad Practice (EVP) as a form of 'learning by doing' practice or evaluation capacity building. The EVP facilitator should be experienced and skilled in evaluation, with knowledge about evaluation theory and multiple methodologies. The EVP participants should bring well-structured projects for evaluation, managerial support, and be at about the same stage in the project. It is an advantage if the projects to be evaluated are diverse in origin and scope, as this diversity enhances participant learning.
The purpose of the paper is to understand the premises and underlying rationale of the EVP. An EVP program theory can be formulated with a focus on the assumptions underlying (a) learning by doing, and (b) the role of the evaluator/instructor, which leads to (c) the development of learning organizations via 'the loop of learning'. (McLaughlin 1999, Stame 2004, van der Knaap 2004, Weiss 1997).
| |
|
Recruited or Appointed to Evaluation Verkstad Practice? A Reflection About Participation in Self-Reflective Evaluation Groups
|
| Laila Niklasson,
Malardalen University,
laila.niklasson@mdh.se
|
|
The model for implementing the Evaluation Verkstad Practice (EPV) takes the form of self-reflective evaluation groups. It also has a formal policy to guide decisions and actions, among them the issue of membership in the groups. In this paper, the recruitment of members to EPV is elaborated. Are members recruited or are they appointed? Who is making these decisions? How does recruitment and appointment affect the practical evaluation work? From a review of the EPVs conducted to date, a preliminary conclusion is that both recruitment and appointment have been used, reflecting both formal and ad hoc perspectives and several levels of decision making actors.
In this presentation, the effects of participant recruitment/appointment on the EPVs are only discussed from a theoretical perspective. The hypothesis presented is that choice of recruitment/appointment can affect dissemination of the project results, dependent on the formal authority/responsibility of the participant for reporting and thus dissemination.
| |
|
The Participant Perspective on Evaluation Verkstad Practice: Learning and Utilization
|
| Ove Karlsson Vestman,
Malardalen University,
ove.karlsson@mdh.se
|
|
With a view of evaluation as a practice of interpretation, argumentation, and decision making in interactional contexts, this presentation focuses on the significance of the Evaluation Verkstad Practice (EVP) for participant learning and utilization. The learning focus is on how participants have changed their conception of evaluation practice. Participant learning will be analyzed via three orders of learning: (1) assimilation or single-loop learning, as in increased knowledge; (2) accommodation or double-loop learning, involving critically assessing one's activity from new perspectives; and (3) learning that comprises the re-organization of earlier experiences and knowledge, generating a re-formulation of fundamental perspectives.
Participant utilization of their learning in the EVP will be analyzed via three venues of utilization: (1) instrumental utilization or how participants apply their learning in evaluation practice, (2) conceptual utilization or new ways of understanding their evaluation activity, and (3) utilization that involves changes organizational perspectives on and valuing of evaluation.
| |