|
Sacrifices Must Be Made: Methodological Choices on a Shoestring Cost-Benefit Analysis Budget
|
| Presenter(s):
|
| Jeffrey Prottas, Brandeis University, prottas@brandeis.edu
|
| Melanie Gaiser, Brandeis University, melaniegaiser@myfairpoint.net
|
| Abstract:
Nonprofit organizations often do not have large budgets available for program evaluation. In order to perform nonprofit program cost-benefit evaluations, difficult choices may have to be made to enable the completion of a project within a relatively small budget. This paper presents a case that involves methodological decision-making in an environment where the use of a classic cost-benefit evaluation model is not possible. Resource constraints required tradeoffs in the methodology used to gather data and determine which costs to include in the analysis. The project team also worked with the nonprofit to determine how to address program planning costs and managers’ fears that including these costs in the analysis would preclude program replication. Finally, the paper addresses the steps taken and obstacles encountered due to the program’s structure; program costs are immediate, while the benefits will take several years to become apparent.
|
|
Evaluating the Cost-Benefit Impacts of a Cooking and Eating Healthy Program in Brazil
|
| Presenter(s):
|
| Miguel Fontes, John Snow, Brazil, m.fontes@johnsnow.com.br
|
| Lorena Vilarins, Social Service Industry, Brazil, lorena.vilarins@sesi.org.br
|
| Milton Souza, Social Service Industry, Brazil, milton.souza@sesi.org.br
|
| Rodrigo Laro, John Snow, Brazil, r.laro@johnsnow.com.br
|
| Fabrízio Pereira, Social Service Industry, Brazil, fpereira@sesi.org.br
|
| Morgana Rios, Social Service Industry, Brazil, morganarios.souza@gmail.com
|
| Diana Barbosa, Independent Consultant, tb.diana@yahoo.com.br
|
| Danielle Valverde, National Union of Municipal Education Managers, danielle_valverde@hotmail.com
|
| Abstract:
Objectives: In 2009, SESI implemented a Cooking and Eating Healthy Program in all 27 Brazilian states with 60,000 individuals. The objective of this evaluation is demonstrating its economic impact.
Methods: A national evaluative research was conducted (n=9,615). A Scale was generated based on 61 variables for eating/cooking Knowledge, Attitudes, Practices (KAP) of participants. A multivariate regression projected the impact of the Scale´s attributes (Conbranch alpha=0.83) on family income. A Gittinger matrix calculated total Net Present Value (NPV) and Benefit:Cost Ratio.
Results: Ex-ante mean of 56.2 points increased to 68.1 points for ex-post in final Scale (weighted range -129 to 129). Significant association between higher levels in Scale and reduction of food waste/expenditure was detected (coefficient 0.001; p-value<0.05). The NPV reached R$23 million after discounting (US$10 million equivalent) and Benefit:Cost Ratio was 4.2:1.0.
Conclusions: The Program demonstrated impact on KAP healthy cooking and eating and contributes to reduction of food waste.
|
|
Evaluating the Cost-Benefit Impacts of a Social Service Program in Brazil
|
| Presenter(s):
|
| Rodrigo Laro, John Snow, Brazil, r.laro@johnsnow.com.br
|
| Miguel Fontes, John Snow, Brazil, m.fontes@johnsnow.com.br
|
| Lorena Vilarins, Social Service Industry, Brazil, lorena.vilarins@sesi.org.br
|
| Diana Barbosa, Independent Consultant, tb.diana@yahoo.com.br
|
| Danielle Valverde, National Union of Municipal Education Managers, danielle_valverde@hotmail.com
|
| Abstract:
Objectives: In 2009, SESI implemented the Citizenship Rights Event in 30 Brazilian municipalities, offering 1.3 million individuals access to basic social and health services. The objective of this evaluation is demonstrating the economic impact of the program.
Methods: A national survey was conducted in November 2009 (n=9,729). A Scale was generated based on 15 types of services. A multivariate regression projected the impact of the Scale´s attributes on family income. A Gittinger matrix was used to calculate total Net Present Value (NPV) and Benefit:Cost Ratio.
Results: At the national level, ex-ante mean of -1.3 point increased to 12.0 points for ex-post in final Scale (range -65 to 65). Significant association between higher levels in Scale and family income was detected (coefficient 8.36; p-value<0.05). The NPV reached near US$22.0 million after discounting and Benefit:Cost Ratio 10.6:1.0.
Conclusion: The Program demonstrated impact on citizenship indicators and contributes to sustainability of local communities.
|
|
Getting to Impact for a Know Thyself Leader Development Program
|
| Presenter(s):
|
| Stacey Farber, Cincinnati Children's Hospital Medical Center, stacey.farber@cchmc.org
|
| Scott Steel, Cincinnati Children's Hospital Medical Center, scott.steel@cchmc.org
|
| Daniel McLinden, Cincinnati Children's Hospital Medical Center, daniel.mclinden@cchmc.org
|
| Britteny Howell, Cincinnati Children's Hospital Medical Center, britteny.howell@cchmc.org
|
| Janet Matulis, Cincinnati Children's Hospital Medical Center, janet.matulis@cchmc.org
|
| Abstract:
Calls for evidence of the impact and value reaped by staff training are pervasive in organizational cultures that encourage and support the development of its workforce. Combining the quantitative approach of Phillips and Phillips' (2007) Return on Investment (ROI) impact assessment methodology with the case study method approach of Brinkerhoff (2003), this paper presentation offers evaluators a model for determining and communicating to organizational leaders the impact (e.g., application, implementation, business measures) and value (i.e., ROI) of a leadership development program focused on building leader self-awareness. The nature of the program and the context in which it occurs (high accountability, mission-driven hospital during tough economic times) also adds a level of complexity that evaluators may find of interest.
|
|
Economic Analyses in Program Evaluation: First Approximations When Economists are Late on the Scene
|
| Presenter(s):
|
| Mustafa Karakus, Westat, mustafakarakus@westat.com
|
| Abstract:
To choose the most effective program among alternatives, economic analysis provides essential information. However, in setting up programs and designing program evaluations, economists are most often the last people on the scene. Thus, an economist is in a situation where s/he has to make the most effective use of administrative data and other program level information to implement sound economic evaluations. This study will discuss approaches to economic evaluations when availability of program data is less than ideal. The process starts with having the key program staff on board to understand the value of economic analyses. Then, all cost points in a program provide information in a uniform and systematic fashion through data collection tools designed by evaluators. Economic data collection tools and how to implement reasonable first approximations of program costs and benefits will be discussed. Example economic evaluations involving employment, career advancement, drug court, chronic homelessness programs will be presented.
|
| | | | |