|
Social Mechanisms, Theory-building and Evaluation Research
|
| Presenter(s):
|
| Brad Astbury, University of Melbourne, brad.astbury@unimelb.edu.au
|
| Abstract:
The relatively recent movement towards a focus on 'mechanisms' as a key analytical concept in the social sciences has started to spill over to the field of program and policy evaluation. This has occurred mainly through the introduction of the term 'mechanism' into the vocabulary of various forms of theory-driven evaluation (e.g. Pawson & Tilley, 1997; Weiss, 1997). However, there appears to be some ambiguity about the proper meaning and uses of mechanism-based thinking in both the social science and evaluation literature. In this paper I attempt to clarify what is meant by 'mechanisms' in the context of program evaluation. I also demonstrate, through illustrative case examples, how the process of investigating underlying generative mechanisms can aid theory-building in evaluation.
|
|
Evaluating a Carbon Monoxide Ordinance: Using Program Theory to Enhance External Validity
|
| Presenter(s):
|
| Huey Chen, Centers for Disease Control and Prevention, hbc2@cdc.gov
|
| Fuyuen Yip, Centers for Disease Control and Prevention, fay1@cdc.gov
|
| Shahed Iqbal, Centers for Disease Control and Prevention, geo6@cdc.gov
|
| Jacquelyn Clower, Centers for Disease Control and Prevention, flj@cdc.gov
|
| Paul Garbe, Centers for Disease Control and Prevention, plg2@cdc.gov
|
| Abstract:
Mecklenburg County of North Carolina passed a residential carbon monoxide (CO) alarm ordinance in 2002. An evaluation is being conducted to assess the effectiveness of the law in reducing CO poisoning. External validity is an important issue in this evaluation, because the findings from this evaluation will have important public health impact on the development of prevention messages and policy recommendations nationally. The paper will demonstrate that program theory is useful so that both internal and external validity issues are addressed. Program theory is used to identify the following contextual factors related to external validity: the enforcement of the ordinance, the capacity of the agency for enforcing the ordinance, the support from collaborating organizations, the residents' awareness of the ordinance, and the prevalence of the CO detectors in the county. Issues on how to integrate internal and external validity in an evaluation will also be systematically discussed.
|
|
Linking Network Structure With Project Performance
|
| Presenter(s):
|
| Boru Douthwaite, Challenge Program on Water and Food, bdouthwaite@gmail.com
|
| Sophie Alvarez, International Center for Tropical Agriculture, b.sophie.alvarez@gmail.com
|
| Abigail Barr, University of Oxford, abigail.barr@economics.ox.ac.uk
|
| Katherine Tehelen, International Center for Tropical Agriculture, k.tehelen@cgiar.org
|
| Abstract:
Innovation can be understood as an emergent property of people-based systems in which agents interact. These interactions can be represented by networks, showing agents as nodes, and interactions as the links between them. Research-for-development projects attempt to foster and support innovation through research. It follows that project network structure, in terms of the types of organizations that work together to implement projects, and their patterns of interaction, should correlate with measures of project success. This paper tests this premise using the network maps drawn by staff of 29 projects of the Challenge Program on Water and Food (CPWF). Performance is measured through a rating carried out by the program's research and development leadership. We conduct two types of network analysis. The first correlates individual project network structure with project performance. The second posits that an organization's performance will be affected by their location in the overall CPWF network in which they are embedded.
|
| | |