|
Session Title: Evaluating Program Sustainability: A Context for Program Evaluation Over Time
|
|
Panel Session 247 to be held in Panzacola Section F3 on Thursday, Nov 12, 10:55 AM to 12:25 PM
|
|
Sponsored by the Health Evaluation TIG
|
| Chair(s): |
| Mary Ann Scheirer, Scheirer Consulting, maryann@scheirerconsulting.com
|
| Discussant(s):
|
| Mary Ann Scheirer, Scheirer Consulting, maryann@scheirerconsulting.com
|
| Abstract:
One context element for program evaluation is to consider the time point in a program's life cycle for each specific evaluation study. Evaluation should be used across the full range of a program or project's life cycle, from initial needs assessment and planning, to evaluating the sustainability of a program after its initial funding has closed. This panel will focus on methods for evaluating a potential end stage of the program life cycle: whether the program's activities, benefits, or other outcomes are sustained beyond its initial funding. We will present evaluations from several health and human service projects illustrating methods for evaluating sustainability, including the contexts affecting continuity across time. The panel will discuss issues concerning both the methods to evaluate sustainability, and what funders might do to foster greater sustainability of their programs.
|
|
Where Are They Now? Assessing the Sustainability of Foundation Grants
|
| Karen Horsch, Independent Consultant, khorsch@comcast.net
|
|
This presentation will focus on the methodology and lessons learned from conducting evaluations of the sustainability of grant-funded projects of two different health conversion foundations. The presentation will discuss the guiding evaluation questions, an overview of the methodological challenges to assessing sustainability and how they were addressed, and a description of the methodological approach which included web-based surveys as well as phone interviews.
|
|
|
Methodological Issues in Studying Sustainability of Social Programs
|
| Shimon Spiro, Tel Aviv University, shispi@post.tau.ac.il
|
| Rivka Savaya, Tel Aviv University, savaya@post.tau.ac.il
|
|
A number of issues were identified in a survey designed to test a complex predictive model of social program sustainability, administered to key informants in 200 projects. The issues included determining the boundaries of the population of programs to be sampled, and defining sustainability - aspects of time, size, institutionalization and the interrelationships among them. Other issues related to research protocol and instrument. Information collected and stored by foundations was sometimes inadequate, and key informants were not always able to provide information about financial aspects of their project and about developments over time. Furthermore, the attempt to test a complex predictive model with a limited sample of programs, proved to be a challenge. We will report on how we dealt with these issues, and look at the advantages and disadvantages of a survey of key informants, compared to in-depth case studies (an earlier stage of the same study).
| |
|
Sustainability Assessment and Targeted Technical Assistance
|
| Charles Gasper, Missouri Foundation for Health, cgasper@mffh.org
|
|
The Missouri Foundation for Health, like other funders, is strongly interested in the sustainability of the programs and organizations it supports. MFH is part of a smaller pool of funders that focus solely on development and expansion of programs versus providing ongoing support for more mature programming. As such, the ability of an organization to sustain funded programming is a strong interest of the Foundation, given its short cycle of funding. The Foundation staff conducted a literature review, developed and tested an instrument for assessment of nonprofits and their proposed programming. This has been integrated into the Foundation's proposal assessment process. The presentation addresses the development of the instrument and the lessons learned from the use of the instrument in identifying issues, which are addressed with technical assistance for funded programs from grant inception.
| |
|
Evaluating Program Sustainability in the Public Sector: Context as Structure
|
| Anne Hewitt, Mountainside Associates, hewittan@shu.edu
|
|
The number of new social programs introduced locally, regionally and nationally is staggering, but as many as 40% of all new initiatives are not sustained beyond the first few years. The challenge of creating a sustainable and viable public initiative remains an obstacle for many government agencies. To meet the challenge of public setting and environmental parameters, program sustainability evaluation requires integrating the political, regulatory, and policy contextual factors within the assessment framework. To highlight this evaluation approach, a case study of a five-year state initiative using three nationally recognized sustainability frameworks as assessment tools is presented. This evaluation study demonstrates the importance of aligning the political context factors with standard program monitoring and synthesizes the state actions that enabled continued program sustainability despite major political and economic challenges. Results from this review are shared to serve as a model for similar sustainability assessments in the public arena.
| |