Evaluating Research Capacity Development: The National Science Foundation's Experimental Program to Stimulate Competitive Research (EPSCoR)

Session Number: 616
Track: Research, Technology & Development Evaluation
Session Type: Panel
Session Chair: Brian Zuckerman [Science and Technology Policy Institute]
Discussant: Gretchen B Jordan [360 Innovation LLC]
Presenter 1: Brian Zuckerman [Science and Technology Policy Institute]
Presenter 2: Thomas W Jones [IDA Science and Technology Policy Institute]
Presenter 3: Brian Zuckerman [Science and Technology Policy Institute]
Presenter 4: Rachel A. Parker [U.S. Agency for International Development]
Time: Oct 17, 2014 (08:00 AM - 09:30 AM)
Room: Mineral A

Level: None
Abstract 1 Title: Session Overview: EPSCoR Origins and Evaluation Methods
Presentation Abstract 1: This paper presents background regarding the NSF EPSCoR program itself, including a synopsis of the program's legislative authorization, goals, and descriptive statistics regarding eligibility and participating jurisdictions. The EPSCoR program logic model is presented, identifying the legislatively-mandated goals --- increasing the competitiveness of investigators for NSF and other Federal funding and increasing participating jurisdictions' science and engineering research bases -- and the program's theory of action to reach those goals. The paper concludes by providing an overview of the methods that were used to conduct the evaluation.
Abstract 2 Title: Increasing Competitiveness of Investigators: Quantitative Approaches
Presentation Abstract 2: This paper describes two quantitative analyses conducted for the purpose of identifying EPSCoR impact on capacity development defined with respect to funding competitiveness. The first analysis used NSF awards and proposal data to compare the number of proposals submitted per investigator, success rates, award sizes, and awards per funded investigator for EPSCoR and non-EPSCoR jurisdictions. This comparison, by disaggregating the determinants of funding levels, identified the particular drivers of funding levels where EPSCoR and non-EPSCoR jurisdictions differ, and suggested hypotheses regarding EPSCoR impact. The second analysis used NSF awards data to conduct a time-series analysis of changes in funding levels in EPSCoR and non-EPSCoR jurisdictions, assessing the statistical significance of EPSCoR-related variables (years in program, number of EPSCoR-funded awards received).
Abstract 3 Title: Concentration Modeling
Presentation Abstract 3: The NSF Organic Act states that NSF should "... strengthen science and engineering research potential and education at all levels throughout the United States; and avoid undue concentration of such research and development." The NSF Organic Act, however, does not define "undue concentration". This paper describes techniques drawn from the economics literature that have only rarely been used for evaluation purposes (e.g., Halfmann and Leyesdorff 2010) for the purpose of computing measures of concentration of NSF funding across jurisdictions and examining correlations over time between levels of concentration and participation in the EPSCoR program. The paper concludes by describing simulations of how changes in programmatic approaches might influence the concentration of NSF funding.
Abstract 4 Title: Qualitative Approaches to Identifying Increased Research Competitiveness and Improved Science and Engineering Research Bases
Presentation Abstract 4: As the EPSCoR evaluation was a life-of-program assessment, STPI needed to collect and analyze data from more than years' worth of program documentation, including solicitations, proposals, and annual reports. This paper describes the coding approach taken and its implementation using the software package nVivo, focusing on EPSCoR education, outreach, and diversity (E/O/D) activities, institution-building activities, and innovation-promoting activities. The paper also describes the design and implementation of a survey of EPSCoR jurisdictions to collect quantifiable as well as qualitative data regarding program activities, outputs, and outcomes. The paper describes approaches taken to assess capacity development with respect both to competitiveness for funding and states' science and engineering research bases.
Other Authors: Brian Rieksts
IDA Science and Technology Policy Institute

Session Abstract: 

The NSF Act of 1950 stated that "it shall be an objective of the Foundation to strengthen research and education in the sciences and engineering, including independent research by individuals, throughout the United States, and to avoid undue concentration of such research and education. Since 1979, NSF has conducted a program intended to stimulate research activity in those parts of the country that have been less able to compete for NSF funds. In 2011, NSF asked the IDA Science and Technology Policy Institute (STPI) to conduct an evaluation of NSF EPSCoR. The objective of this evaluation was to perform an in-depth, life-of-program assessment of NSF EPSCoR activities and these activities' outputs and outcomes, and to provide recommendations for better targeting of available funding to those jurisdictions for which the EPSCoR investment could result in the largest incremental benefit to their research capacity. This panel presents the methodology used to conduct the evaluation.