|
Session Title: Mixed Methods Contributions to Evaluation Quality
|
|
Panel Session 782 to be held in Lone Star A on Saturday, Nov 13, 10:55 AM to 12:25 PM
|
|
Sponsored by the Presidential Strand
|
| Chair(s): |
| Donna Mertens, Gallaudet University, donna.mertens@gallaudet.edu
|
| Discussant(s):
|
| Melvin Mark, Pennsylvania State University, m5m@psu.edu
|
| Kataraina Pipi, Independent Consultant, kpipi@xtra.co.nz
|
| Abstract:
Evaluators have been mixing methods for decades, or even longer. As the most applied of all social inquirers, evaluators base methodological decisions, in large part, on contextual practicality and design potency for answering evaluation questions. If both structured surveys and open-ended interviews are needed in a given context, then evaluators – unproblematically and without losing sleep – build these different methods into their design. So, it is not always apparent what the emerging ‘theory’ of mixing methods has to offer to the evaluation community. This session will present an argument that the conceptual ideas and aspirations of mixed methods inquiry can indeed make vital contributions to evaluation practice, and will focus this argument on mixed methods’ contributions to evaluation quality. The session will engage diverse understandings of the philosophical, theoretical, and methodological frameworks in mixed methods and their relevance to the meanings of evaluation quality, with illustrations from practical exemplars.
|
|
What is Quality Empirical Social Science From a Mixed Methods Perspective?
|
| Abbas Tashakkori, University of North Texas, abbas.tashakkori@unt.edu
|
|
The mixed methods community welcomes a rich diversity of philosophical, methodological, practical, and ethical views about quality. With diversity come challenges of reconciliation around what constitutes quality work, including explicit and implicit differences in criteria for evaluating the separate qualitative and quantitative strands, nomenclature, the relative importance and the separate versus integrated character of study components, and the necessity of evaluating inquiry consequences. Quality audits offer potential for common ground. Such audits would focus on mixed methods inquiry as a systemic attempt to answer questions, clearly differentiating four elements of such attempts: inputs (problems, data), processes (analysis), outcomes (inferences, recommendations), and consequences (impacts) of inquiry while recognizing that these might not operate in a linear manner. Issues in auditing the components of a mixed methods evaluation will be discussed, including the challenges of the evaluator as a stakeholder, divergence in quality assessors, and the role of outcomes/consequences.
|
|
|
Bridging Quality in Mixed Methods Inquiry With Evaluation Theory and Practice
|
| Donna Mertens, Gallaudet University, donna.mertens@gallaudet.edu
|
|
How well do the ideas about quality in contemporary mixed methods theory and practice translate to the challenges of quality in evaluation? How are the conceptualizations of quality similar and different in these two pluralistic domains and diverse communities of practice? What can evaluators learn about quality from mixed methodologists? This presentation will engage these questions primarily from a standpoint of quality as justice (House), but will also consider the quality dimensions of truth and beauty. In particular the mixed methods idea of quality audits as a vehicle for assessing inquiry quality will be engaged and critiqued for its applicability to the contexts and challenges of evaluation.
| |
|
How Does the Mix of Methods Enhance Evaluation Quality in the Field?
|
| Jori Hall, University of Georgia, jorihall@uga.edu
|
| Jennifer Greene, University of Illinois at Urbana-Champaign, jcgreene@illinois.edu
|
|
The contributions of mixed methods thinking to the quality of evaluation practice will be featured in this presentation. Using examples of mixed methods evaluation studies, the presentation will discuss the ways in which a mixed methodology can engage multiple dimensions of evaluation quality, from the methodological to the social, political, and aesthetic.
| |