| Session Title: Measuring Research Interdisciplinarity and Knowledge Diffusion |
| Multipaper Session 928 to be held in Malibu on Saturday, Nov 5, 12:35 PM to 2:05 PM |
| Sponsored by the Research, Technology, and Development Evaluation TIG |
| Chair(s): |
| Alan Porter, Georgia Tech and Search Technology Inc, alan.porter@isye.gatech.edu |
| Discussant(s): |
| Jan Youtie, Georgia Tech, jan.youtie@innovate.gatech.edu |
| Abstract: Interest in the attributes of cross-disciplinary research and in the distribution of research knowledge is strong. This has inspired introduction of several new measures of interdisciplinarity and research diffusion. This session brings together several explorations of the Integration and Diffusion scores, along with other measures and visualizations, to help understand their behavior. The first paper introduces the Diffusion score and examines its behavior in a substantial benchmarking exercise, augmented by in-depth lab studies. The second paper investigates development of a companion Integration score to gauge the diversity of patent sets. The third paper applies Integration and other scoring in research program assessment. It compares researcher and proposal level variations of the metric. We then discuss the opportunities and limitations in applying these measures on behalf of research evaluation -- e.g., sensitivities to disciplinary citation norms, Web of Science coverage, temporal distributions, and so forth. |
| A New Measure of Knowledge Diffusion |
| Stephen Carley, Georgia Tech, stephen.carley@gmail.com |
| Alan Porter, Georgia Tech and Search Technology Inc, alan.porter@isye.gatech.edu |
| The Diffusion score is a new interdisciplinary metric to assess the degree to which research is cited across disciplines. It is the analog to the Integration score, which measures diversity among a given publication's cited references. Together these metrics enable tracking the transfer of research knowledge across disciplines and citation generations. The two scores share a consistent formulation based on distribution of citations over Web of Science Subject Categories (SCs). The Integration score measures diversity among cited SCs; the Diffusion score measures diversity among citing SCs. Here we study the behavior of Integration and Diffusion scores for benchmark samples of research publications in six major fields (SCs) spanning interests of the National Academies Keck Futures Initiative (NAKFI). We also probe their behavior via two laboratory level analyses. Through long term observation of these labs, and interviews with their senior investigators, we explore "exactly what" Integration and Diffusion scores are tapping. |
| Analyzing the Effect of Interdisciplinary Research on Patent Evaluation: Case Studies in Nbs and Dsscs |
| Wenping Wang, Beijing Institute of Technology, wangwenping1009@gmail.com |
| Alan Porter, Georgia Tech and Search Technology Inc, alan.porter@isye.gatech.edu |
| Ismael Rafols, University of Sussex, i.rafols@sussex.ac.uk |
| Nils Newman, Intelligent Information Services, newman@iisco.com |
| Yun Liu, Beijing Institute of Technology, liuyun@bit.edu.cn |
| Policies facilitating interdisciplinary research (IDR) appear to be based more on conventional wisdom than empirical evidence. This study examines whether IDR leads to higher technological performance. Patents, as major outputs of technological invention, are adopted as the representative measure of technological performance. To look into the relationship between IDR and patent evaluation, we address "patent quality" and "IDR." Disciplinary diversity indicators of patents are developed from the properties of variety, balance, and similarity. Basing the research on patent abstract documents, we evaluate patent quality along three dimensions: technology, market, and legal. We then examine correlations between the diversity and patent quality measures. The case study builds on the emerging domains of Nanobiosensors and Dye-Sensitized Solar Cells. By devising patent metrics commensurate with publication measures, comparison also informs the relationship between research (publication) and patent activity. |
| Measuring Interdisciplinarity: A Unique Comparison Between the Researcher and Research Proposal |
| Asha Balakrishnan, IDA Science & Technology Policy Institute, abalakri@ida.org |
| Vanessa Pena, IDA Science & Technology Policy Institute, vpena@ida.org |
| Bhavya Lal, IDA Science & Technology Policy Institute, blal@ida.org |
| Measuring interdisciplinarity of the researchers and research teams is of major interest to agencies funding interdisciplinary research programs. One particular program funding potentially transformative research through interdisciplinary team science at a key R&D agency requested the Science and Technology Policy Institute (STPI) to conduct an assessment of the program's awards funded between FY2007 to FY2009. In this talk, we present a unique analysis that compares the interdisciplinarity of individual researchers compared with the interdisciplinarity of their awarded proposals funded by this particular program. We will describe the methodology behind measuring both the interdisciplinarity of the principle investigator's publication history and interdisciplinarity of the proposals, and compare how a PI's interdisciplinarity maps to his/her proposal interdisciplinarity. |