|
Session Title: New Directions for Research on Evaluation
|
|
Panel Session 233 to be held in Texas F on Thursday, Nov 11, 9:15 AM to 10:45 AM
|
|
Sponsored by the Research on Evaluation TIG
|
| Chair(s): |
| John LaVelle, Claremont Graduate University, john.lavelle@cgu.edu
|
| Abstract:
In recent years the practice of evaluation has grown substantially, as evidenced by a rise in the number of professional evaluation organizations, AEA membership, the demand for evaluation services, and the number of number of universities offering training in evaluation (LaVelle & Donaldson, 2010). In tandem, a growing emphasis has been placed on the importance of research on evaluation. Mark (2007) proposed an organizing taxonomy that suggests four major areas that researchers of evaluation might explore: Context, Activities, Consequences, and Professional Issues. This panel will provide an introduction to Mark’s taxonomy, highlight current research in each section of the taxonomy, and provide an opportunity for the audience to brainstorm with the presenters to generate specific research ideas to guide future efforts.
|
|
Research on Evaluation Context: Examples and Ideas
|
| Michael Szanyi, Claremont Graduate University, michael.szanyi@cgu.edu
|
|
Mark (2007) defines evaluation context as the circumstances within which evaluation occurs and identifies systematically assessing the effect of context on evaluation practice as one focus in research on evaluation. This presentation will provide an introduction on the contextual piece of Mark’s (2007) framework. Examples will be given to illustrate research on evaluation context, such as Azzam’s study on evaluator responsiveness to stakeholder opinions about an evaluation’s design (2010). Ideas for future research on evaluation context will be empirically derived from a study that the presenter conducted where a needs assessment/interest survey was sent out to AEA members in fall 2009 and over 300 respondents suggested topics and questions related to evaluation context. The major themes and specific questions will be presented to offer research avenues on evaluation context so that we may improve our practice as well as help answer the calls for more research on evaluation.
|
|
|
Research on Evaluation Consequences: A Meta-analysis of Evaluation Use
|
| Mark Hansen, University of California, Los Angeles, markhansen@ucla.edu
|
| Anne Vo, University of California, Los Angeles, annevo@ucla.edu
|
|
One of the consequences of evaluation that has received greatest attention among theorists, practitioners, and researchers is evaluation utilization, which has been described as “the way in which an evaluation and information from the evaluation impacts the program that is being evaluated” (Alkin & Taut, 2003). Although there exists a rich body of literature on this topic, there have been only a few efforts to synthesize this work (e.g., Cousins & Leithwood, 1986; Johnson et al., 2009). Such reviews have been enormously helpful in summarizing the research and generating insights concerning conditions that contribute to use. However, it is noteworthy that a quantitative synthesis has not been conducted. Here, we seek to address this gap through a meta-analysis of studies that examined the relationship between evaluation characteristics and perceived usefulness. We describe the results of this synthesis and discuss the usefulness of meta-analysis as a tool for research on evaluation.
| |
|
Research on Professional Issues in Evaluation: Next Steps
|
| John LaVelle, Claremont Graduate University, john.lavelle@cgu.edu
|
|
The idea of professionalization and professional development is not new, and various professional identity topics have been discussed over the years. These discussions, however, have lacked a unifying framework to help the audience see the relationships between various professional topics. The presenter will draw from sociology to share Forsyth & Danisiewicz’s (1985) process model of professionalization, which will then be used to frame a discussion on recent research on professional topics. Potential topics to be shared include the presenters’ research on the recruitment & selection of evaluators and the preservice preparation of evaluators. The discussion will then be expanded to include topics such as competencies, professional regulation, and public awareness of evaluators.
| |