|
Use of Mixed Methods to Evaluate Three States' Environmental Results Programs (ERPs)
|
| Presenter(s):
|
| John Heffelfinger, United States Environmental Protection Agency, heffelfinger.john@epa.gov
|
| Scott Bowles, United States Environmental Protection Agency, bowles.scott@epa.gov
|
| Abstract:
This evaluation examined the experience of three states, Delaware, Maine, and Rhode Island, which established Environmental Results Programs (ERPs) for the auto body repair sector. Auto body shops can pose a range of cross-media environmental and/or health concerns, including air emissions, water discharges, hazardous materials handling and waste management, and worker health and safety concerns. Each of these state’s ERPs incorporates voluntary self-certification by auto body shops, compliance assistance/training workshops, inspection of a random sample of facilities before and after program implementation, and use of statistical analyses to estimate overall compliance. While these states are similar in the types of programs they implemented, they differ in terms of several circumstances in the state that could affect ERP implementation, participation of auto body shops, and outcomes. The evaluation describes each state’s individual experience with their ERP program, focusing primarily on outcomes, changes in facility management practices, and costs.
|
|
The Statistically Valid Pilot: Taking Advantage of Unique Opportunities to Design and Implement Rigorous Program Evaluations
|
| Presenter(s):
|
| Tracy Dyke Redmond, Industrial Economics Inc, tdr@indecon.com
|
| Terell Lasane, United States Environmental Protection Agency, lasane.terell@epamail.epa.gov
|
| Abstract:
The task of ascertaining a program’s definitive effects is difficult to achieve in field settings where multiple factors pose rival explanations for a program’s causal impacts. In advance of a forthcoming regulation affecting small businesses, EPA planned to offer compliance assistance to regulated entities. To assess its results, EPA convened a group of the program’s managers, evaluation experts, and other stakeholders to design a statistically-valid program evaluation. The evaluation design includes random assignment to treatment and control groups, random selection from an identifiable universe, and a difference-in-differences analytical approach to analyze two comparison groups over time. This innovative approach to program evaluation design may yield impact evaluation results, which is rarely feasible in federal programs because of multiple rival causal explanations, limited fiscal resources, and other institutional restrictions. The lead evaluator will present the evaluation’s history, design, implementation, initial results, and potential applications of the design.
|
| |