|
Session Title: What Constitutes Quality Evaluation of Development and Social Change: Values, Standards, Tradeoffs, and Consequences
|
|
Panel Session 202 to be held in Lone Star A on Thursday, Nov 11, 9:15 AM to 10:45 AM
|
|
Sponsored by the Presidential Strand
|
| Chair(s): |
| Indran Naidoo, Office of the South African Public Service Commission, indrann@opsc.gov.za
|
| Discussant(s):
|
| Patricia Rogers, Royal Melbourne Institute of Technology, patricia.rogers@rmit.edu.au
|
| Abstract:
How we view the processes of social change and development and what we consider quality evaluation of interventions to achieve these are inextricably linked. There is a risk that efforts to improve the quality of development evaluations will only support the evaluation of standardized, simple interventions. This may divert attention and resources from innovations that are more complex - i.e. emergent and unpredictable – and therefore inherently risky, yet critical if development is to be sustained and respond to urgent issues that require innovative responses. We need to move beyond the ongoing paradigmatic ‘tug-of-war’ about methodology to a deeper understanding of how change happens, better ways of evaluating ‘the complex’ without compromising quality standards and better understanding of the ways in which evaluation itself has an impact on the processes of social change. This issue has implications for other interventions that seek to bring about sustainable structural change.
|
|
Impact Evaluation: Serving Development?
|
| Zenda Ofir, Independent Consultant, zenda@evalnet.co.za
|
|
Recent studies note that development thinking makes use of too narrow a range of approaches to change, limiting creativity and contributing to insufficient interrogation of how change happens in complex development environments. At the same time the ongoing impact evaluation debates tend to focus on methodology rather than on the influences, values and standards that shape development and how its effectiveness is assessed.
This paper highlights the intensifying struggle to make space for a diversity of impact evaluation approaches that engage more deeply with how change is effected in challenging development contexts. Experiences during the process of establishing impact evaluation guidelines by the Network of Networks on Impact Evaluation (NONIE) reinforce arguments for dissecting our underlying assumptions when making choices, and support calls for reform of the development evaluation system.
|
|
|
The Social Transformation of Evaluation?
|
| Thomas Schwandt, University of Illinois at Urbana-Champaign, tschwand@illinois.edu
|
|
There is a growing literature critical of extant models of evaluation, and social science more generally, for their inability to successfully contribute to our understanding, appraisal, and capacity to solve the kinds of complex and dynamic social problems and desired social transformations that characterize the arenas in which evaluation (and social science) ought to have its greatest foothold (e.g. health-related fields, development, education).
This paper first briefly rehearses this criticism and then points to the kinds of alternatives to 'standard' ways of thinking of social science and evaluation that are being proposed including systems thinking and complexity science; implementation science; practice-based evidence; calls for 'public social science'; and proposals to revisit the theory-practice gap. The paper then turns to the question of whether different criteria might be needed to judge these new ways of practicing evaluation and social science.
| |
|
Values, Standards and Tradeoffs in the Evaluation of (Complex) Change Processes
|
| Irene Guijt, Learning by Design, iguijt@learningbydesign
|
|
While some discussions continue to centre on methodological superiority, with (quasi) experimental methods claiming the ‘rigor’ high ground, the need to understand how to assess ‘the complex’ has been fostering methodological innovations that capture change and performance from a broader perspective. All methodologies have their use, and all choices imply privileging some values over others. We therefore need to understand what values and quality standards are being upheld when making these choices. Tradeoffs are inevitable.
Thinking through the consequences of choice is crucial. Choices determine what is considered valid to see, define what is valued, and shape future thinking on development policies, processes and priorities. And eventually how development is understood. This paper will summarize the main lines of argument from recent conferences and highlight practical examples that reconcile an understanding of complex societal change processes with quality standards and rigor, ethical concerns, appropriateness and feasibility.
| |