| Session Title: New Evaluation Techniques For Estimating the Impacts of Science and Technology Innovations |
| Multipaper Session 485 to be held in Wekiwa 6 on Friday, Nov 13, 9:15 AM to 10:45 AM |
| Sponsored by the Research, Technology, and Development Evaluation TIG and the Costs, Effectiveness, Benefits, and Economics TIG |
| Chair(s): |
| Jerald Hage, University of Maryland, hage@socy.umd.edu |
| Abstract: Government concern about demonstrating the value of investments in science and technology has been heightened by the current economic crisis. This panel presents three novel approaches for evaluating the returns on investment in research and technology development from three distinct government agencies, two in the United States and one in Canada. Together these papers illustrate the importance of developing new measures for benefits of science and technology (S&T) innovations that move beyond the traditional economic measures of the dollar value of improved productivity and revenue from sales. The methods also address health, environmental, security, and knowledge benefits in quantitative as well as qualitative ways, and get at intermediate impacts as well as global impacts which helps attribute benefits to specific S&T programs. |
| A Credible Approach to Benefit-Cost Evaluation for Federal Energy Technology Programs |
| Gretchen Jordan, Sandia National Laboratories, gbjorda@sandia.gov |
| Rosalie Ruegg, TIA Consulting, ruegg@ec.rr.com |
| This paper describes a methodology that improves upon an already credible approach developed for a 2001 National Research Council study: "Energy Research at DOE: Was It worth It?" Three benefit-cost studies using this modified approach will be completed by the U.S. Department of Energy's Energy Efficiency and Renewable Energy in 2009. Economic performance metrics that are calculated are Net benefits, Benefit-cost ratio, and Internal Rate of Return. Benefits and costs for selected technology "winners" are calculated compared against the next best alternative. Additionally, an innovative "Cluster approach" is used that compares benefits of larger elements of a program to investment costs of the entire program. Environmental and Security benefits are also assessed, as are knowledge benefits. In contrast to the 2001 NRC study, the modified approach requires a case-by-case assessment of an array of ways additionality can occur, the difference that DOE made in the outcome. |
| Techniques for Evaluating Potential Benefits of New Scientific Instruments |
| Jonathan Mote, University of Maryland, jmote@socy.umd.edu |
| Aleia Clark, University of Maryland, alclark@socy.umd.edu |
| Jerald Hage, University of Maryland, hage@socy.umd.edu |
| This paper proposes a technique for evaluating the potential impacts of new scientific instruments in a way that avoids the pitfalls of "economic-only" cost-benefit analysis and meets the needs of the customer organization and Congress. The evaluation should convince Congress to fund a new suite instruments, the Hyperspectral Sounder (HES) on a new weather satellite to be launched by the National Oceanographic and Atmospheric Agency (NOAA). The evaluation will focus on improvements in warning time for severe weather events using more localized forecasting. More warning time will result in saved lives and reduced health consequences of sudden decreases in air quality. The context of the evaluation requires dealing with how the collection of regional weather data can be made compatible with current collection systems, and, of course, the question is what evidence can be marshaled for a system that is not yet operational. |
| A New Evaluation Strategy for Measuring the Returns on Investments in Medical Research: The Meso Level of the Treatment Sector |
| Jerald Hage, University of Maryland, hage@socy.umd.edu |
| Gretchen Jordan, Sandia National Laboratories, gbjorda@sandia.gov |
| Typically health evaluations are at either the micro level of a particular treatment or the macro level of a series of health benefits. With a small grant from the Canadian Academy of Health Sciences, we developed a new strategy that allows for the synthesis of evaluation of research findings from a variety of studies, treatment sector by treatment sector. The specific metrics of the framework are 1) health care impact by stage in the treatment process; 2) research investment by arenas within the production of medical knowledge within the specific treatment sector; 3)contributions to scientific knowledge; 4) network gaps in the production of innovative treatment protocols; and 5) economic and social benefits of medical research. Two unusual features are recognition that the advantages of alternatives kinds of research can be estimated and the potentiality of the valley of death in the transfer of medical research into health care products. |