Search Form

Abstract: Checklists in evaluation are tools to support practitioners in the design, implementation, as well as metaevaluation of evaluations. They may not quite represent a methodology or theory in their own right, but commonly incorporate a set of components, dimensions, criteria, tasks, and strategies that are critical in the evaluation programs, personnel, systems, etc. (c.f. Scriven 2005). This session introduces two checklists and implications for their application in the field. First, Nadini Persaud introduces her checklist for conducting financial and economic analyses. Second, Daniela Schroeter will share her checklist for evaluating sustainability under consideration of the logic of evaluation and the social, economic, and environmental dimensions promoted in the field of sustainable development. Third, Tererai Trent and Otto Gustafson discuss challenges and opportunities in applying these checklists in evaluation practice on the example of an international development impact evaluation. Each paper will be scrutinized by an expert discussant.

A Cost Analysis Checklist Methodology for Use in Program Evaluations

Nadini

payday loan

Persaud

payday loan

, Western Michigan University, npersaud07@yahoo.com

A fundamental but often neglected aspect of professional evaluation is the exclusion of serious and sophisticated cost analysis studies. Knowing that a program is responsible for certain outcomes is of little value in a political environment. The quintessential question at the end of the day is -costs- (Chelimsky, 1997). Is the program cost-effective and cost-feasible and how cost-effective is it compared to similar programs? This paper will present a cost analysis checklist methodology that can be used by novice evaluators to conduct financial and economic analyses. The checklist is divided into 6 main sections: types of costs and benefits, valuation of costs and benefits, other issues that need to be considered in cost analysis, the discount rate, project appraisal methodologies, and reporting. The cost analytical methodology selected requires great care because procedures that are selected based on popularity, evaluator's familiarity, or ease of use may lead to fallacious evaluative conclusions.

The Logic and Methodology of Sustainability Evaluation: A Checklist Approach

Daniela C

payday loan

Schroeter

payday loan

, Western Michigan University, daniela.schroeter@wmich.edu

The logic of evaluation is generally subsumed within four steps: (i) identification of criteria, (ii) determination of performance standards on each dimension, (iii) collection and analysis of factual data on each dimension, and (iv) synthesis of results into statements of merit, worth, and significance (c.f. Scriven, 1982; Fournier, 1995). This checklist builds on the logic of evaluation and applies it specifically to the evaluation of sustainability. Evaluations of sustainability encompass a wide range of evaluands in numerous contexts, including for example policies, systems, institutions, communities, programs, and personnel. The checklist incorporates the relevant literature, research, and evaluations, merging best practices with current thought in the logic and methodology of sustainability evaluation. It is a tool for holding evaluands accountable and suggesting improvements, under consideration of growing demands on meeting consumers' needs as well as social, economic, and environmental aspects of evaluands.

The Validity and Utility of the Cost and Sustainability Checklists: A Field-Trial in an International Aid Evaluation

Otto

payday loan

Gustafson

payday loan

, Western Michigan University, ottonuke@yahoo.com

Evaluators are inundated with various checklists and other tools, some of which are more academic than pragmatic. In order to assess the validity and utility of evaluation checklists, it is imperative that they are field tested.. Additionally, when using checklists in developing countries it is necessary to examine whether these checklists are compatible with country-specific contexts and values as opposed to simply satisfying donor-driven accountability mechanisms. This paper will present a case study of the application of the previously discussed cost analysis and sustainability checklists in the context of an international aid program evaluation conducted in Africa. The presenters will reflect on the soundness and utility of both checklists as well as the challenges and advantages experienced integrating and implementing the checklists during the planning, execution, and metaevaluation of the evaluation. Finally, this paper will discuss implications of the field testing results for improvement.

Search Form