|
Adapting Interdisciplinary Methods to Evaluate the Social Outcomes of Environmental Programs: Five Lessons from Minnesota
|
| Presenter(s):
|
| Karlyn Eckman, University of Minnesota, eckma001@umn.edu
|
| Erika Rivers, Minnesota Department of Natural Resources, erika.rivers@dnr.state.mn.us
|
| Kimberly Nuckles, Minnesota Pollution Control Agency, kimberly.nuckles@state.mn.us
|
| Valerie Were, University of Minnesota, were@umn.edu
|
| Abstract:
In the current economic climate, public agencies have fewer resources to plan, implement and evaluate environmental programs. In Minnesota agencies are under pressure from both state legislators and new federal mandates to demonstrate results from the investment of public resources. Evaluation practice varies widely across agencies, from costly and complex indicator matrices to absence of program evaluation. Researchers successfully adapted the Knowledge, Attitudes and Practices (KAP) study method (commonly used in international community health programs) to large and small environmental projects in Minnesota. The Minnesota KAP method has proved to be a low-cost, flexible and practical tool for both planning and evaluation. It has been tested with invasive species, urban stormwater, shoreland habitat, water quality, septic and other programs. In all cases, it met or exceeded the evaluation requirements of the users. Five lessons are presented from applied evaluation research on this interdisciplinary program evaluation methodology.
|
|
Methodological Reflections and Organizational Learning Generated by a Qualitative Evaluation of a Leadership Development Program of the U.S. National Park Service
|
| Presenter(s):
|
| Jennifer Jewiss, University of Vermont, jennifer.jewiss@uvm.edu
|
| Daniel Laven, National Park Service, daniel_laven@nps.gov
|
| Nora Mitchell, National Park Service, nora_mitchell@nps.gov
|
| Abstract:
“Many leadership programs … by definition … are ventures into the unknown, aspiring beyond the tried and true. From that point of view, evaluation is about discovery, understanding what is possible to achieve, and even helping to characterize new kinds of leadership.” (Leviton, p. xii, in The Handbook of Leadership Development Evaluation, 2007). This paper explores the notion of “evaluation as discovery” as it relates to a qualitative evaluation of the Superintendents Leadership Roundtable (SLR), an innovative leadership development program of the National Park Service Conservation Study Institute. The value of the study process and findings extended beyond an assessment of the program – surfacing important observations about the complex and challenging leadership role that superintendents play and the broader contexts in which these leaders and the SLR program operate. This presentation will reflect on the organizational learning generated as a result of the chosen research methods and stakeholder engagement process.
|
|
A Task for Superwoman: Pursuit of Truth and Justice Within Energy Impact Evaluations
|
| Presenter(s):
|
| Mary Sutter, Opinion Dynamics Corporation, msutter@opiniondynamics.com
|
| Abstract:
In the world of energy efficiency programs, the fate of millions of dollars in penalties or rewards often lies with the results of an impact evaluation. Yet, the methods to determine net impacts are often questioned by stakeholders, entities determining the penalties and rewards have difficulty understanding the ramifications of one method versus another, and the available methods are becoming less able to discern impacts well. This paper will discuss the tightrope between pushing the envelope to use newer methods and continually educating relevant stakeholders. We will highlight the struggle to assess programs with truth and justice while working within a regulatory framework where policy choices and evaluation methods may be at odds.
|
|
Continuous Improvement in Energy Efficiency: Helping Clients Shift From One-Shot Evaluations Towards Assessing the Effectiveness Within and Across Programs
|
| Presenter(s):
|
| Marjorie McRae, Research Into Action, marjorie@researchintoaction.com
|
| Jane Peters, Research Into Action, janep@researchintoaction.com
|
| Abstract:
In the field of energy conservation, continuous improvement is considered a best practice when managing energy efficiency programs. Process evaluations emphasizing continual feedback and nearly real-time data collection and analysis are recommended to utilities and energy commissions. This presentation will discuss examples of how evaluations provided fodder for the adoption of continuous improvement practices by organizations such as the Northwest Energy Alliance (NEEA), Energy Trust of Oregon, and New York State Energy Research & Development Agency (NYSERDA). It will also discuss how efforts to promote continuous improvement have been difficult to achieve within regulatory proceedings, as in California. This topic will be particularly relevant for evaluators working with resource conservation programs that must manage relationships with regulatory agencies or political organizations.
|
| | | |