|
Expanding the Evaluation Toolkit: Using Qualitative Comparative Analysis in Cross-Site Evaluation
|
| Presenter(s):
|
| Heather Kane, RTI International, hkane@rti.org
|
| Megan Lewis, RTI International, melewis@rti.org
|
| Pamela Williams, RTI International, ppiehota@rti.org
|
| Leila C Kahwati, National Center for Health Promotion and Disease Prevention, leila.kahwati@va.gov
|
| Abstract:
Evaluators are faced with two challenges when, conducting cross-site analyses of larger units such as organizations, communities or systems: 1) having too few cases; and 2) capturing multiple configurations indicating successful program effects, typically referred to as equifinality in organizational theory. To address this issue, we will describe how qualitative comparative analysis (QCA) bridges qualitative and quantitative analyses by combining in-depth knowledge gained from case studies with principles derived from Boolean algebra. This method can be used in evaluations containing at least 10-12 cases where a small sample size would preclude using other techniques. It also allows for multiple configurations of variables to account for program success, and thus may be a better reflection of complex implementation processes. We will describe an example of how it has been successfully used in a cross-site evaluation of the Veterans Health Administration's MOVE! Weight Management Program for Veterans
|
|
The Complexity of Practice: Participant Observation and Values-Engagement in a Responsive Evaluation of a Professional Development School Partnership
|
| Presenter(s):
|
| Melissa Freeman, University of Georgia, freeman9@uga.edu
|
| Jori Hall, University of Georgia, jorihall@uga.edu
|
| Abstract:
All social and professional practices are historically situated, evolving forms of acting and interacting. Evaluation, as a practice, is shaped by and shapes the practice evaluated. Our paper contributes to responsive and values-engaged evaluation approaches by reflecting on the space where these two practices intersect. The evaluative task was to document the nature of a partnership between a university and school district and how that partnership was being carried out in the form of a professional development school. Although other methods were used, we focus on the role participant observation, as an interactive and responsive form of engagement, played in the evaluation. Through two lenses: observing the partners and observing ourselves, we critically reflect on our decision-making processes, assessing their accomplishments and shortcomings. We conclude by considering how we might further our engagement as values-engaged evaluators in this context in ways that support the development of both practices.
|
|
Navigating Personal Values and Beliefs in the Analysis of Qualitative Data
|
| Presenter(s):
|
| Janet Usinger, University of Nevada, Reno, usingerj@unr.edu
|
| Stephanie Woolf, University of Nevada, Reno, swoolf@unr.edu
|
| Jafeth Sanchez, Washoe County School District, jsanchez@washoe.k12.nv.us
|
| Abstract:
Qualitative evaluators frequently argue how critical it is to represent the voice of the participant. Yet analysis of qualitative data is inherently a constructivist process. As evaluators interpret data collected through interviews, focus groups, observations, and field notes, how do they place their own values in abeyance? How do they not fall into the trap of analyzing data through their own personal perspectives? How do they avoid perpetuating stereotypes through superficial analysis? This paper presents an evaluator's reflective process in the analysis of qualitative data gathered longitudinally for six years to explore the process that 60 adolescents undertook as they socially constructed their career aspirations and the role that education played in that process. The study was a component of the evaluation of a State Gaining Early Awareness and Readiness for Undergraduate Programs (GEAR UP) project.
|
| | |