|
Session Title: Evaluating the Science of Discovery in Complex Health Systems: Challenges and Opportunities
|
|
Panel Session 387 to be held in BONHAM E on Thursday, Nov 11, 4:30 PM to 6:00 PM
|
|
Sponsored by the Research, Technology, and Development Evaluation TIG
|
| Chair(s): |
| Alison Buchan, University of British Columbia, abuchan@medd.med.ubc.ca
|
| Discussant(s):
|
| Alison Buchan, University of British Columbia, abuchan@medd.med.ubc.ca
|
| Abstract:
Complex health problems such as chronic disease or pandemics require knowledge that transcends disciplinary boundaries in order to generate solutions. Such transdisciplinary discovery requires researchers to work and collaborate across boundaries, combining elements of basic and applied science. At the same time, calls for more interdisciplinary health science acknowledge that there are few metrics to evaluate the products associated with these new ways of working. The Research on Academic Research (RoAR) initiative was established to evaluate the process of discovery and impact of collaboration that emerged through the Life Sciences Institute at the University of British Columbia, a state-of-the-art facility designed to support researchers - self-organized around specific health problems rather than disciplines. A logic model depicting the factors influencing such collaboration is presented along with a multi-method evaluation plan to assist understanding of the discovery process in this new environment and develop new metrics for assessing collaborative impact.
|
|
An Evaluation Framework for Advancing the Science of Evaluating Team Science: The Research on Academic Research Initiative (RoAR)
|
| Cameron Norman, University of Toronto, cameron.norman@utoronto.ca
|
| Timothy Huerta, Texas Tech University, tim.huerta@ttu.edu
|
| Sharon Mortimer, Michael Smith Foundation for Health Research, smortimer@msfhr.org
|
| Allan Best, Michael Smith Foundation for Health Research, allan.best@in-source.ca
|
| Alison Buchan, University of British Columbia, abuchan@medd.med.ubc.ca
|
|
Background: In 2006 the University of British Columbia opened the Life Sciences Institute (LSI), the first building of its size at UBC to be developed to support cross-disciplinary bio-sciences team research. The Research on Academic Research (RoAR) initiative was initiated in 2007 to serve as a platform for conducting exploratory research and evaluation of the effect that co-location of previously disparate researchers and institutional policies have on the organization and output of scientists.
Methods: A logic model was developed to guide a multi-method evaluation that aimed to assess the outcomes associated with the re-organization of the scientists from academic department buildings into integrated problem-based research groups. These methods included a survey of psychosocial issues and social networks, an examination of publication and grant application patterns, measures of physical proximity, and investigator interviews.
Discussion: Evaluation of team science activities requires a strategy that can address process and outcomes from multiple perspectives.
|
|
|
Advancing the Science of Evaluating Team Science: Psychosocial Factors and Related Outcomes From the RoAR Initiative
|
| Cameron Norman, University of Toronto, cameron.norman@utoronto.ca
|
| Timothy Huerta, Texas Tech University, tim.huerta@ttu.edu
|
| Sharon Mortimer, Michael Smith Foundation for Health Research, smortimer@msfhr.org
|
| Alison Buchan, University of British Columbia, abuchan@medd.med.ubc.ca
|
|
Background: Much of the research in basic science is investigator-driven and favors individual scientists or small groups. Team science is a different orientation, requiring new skills and knowledge. A survey was developed to assess the level of comfort and skills of investigators working in a team science environment over four years.
Method: Investigators with the Life Sciences Institute at the University of British Columbia were surveyed on their attitudes, knowledge, collaborative skills, and perceived benefits and risks associated with various models of research were taken.
Conclusions: The overall attitudes, skills and knowledge about how to do team science and its perceived value to academic work changed considerably over four years. Dr. Norman will draw on his background on health behavior change, systems science and evaluation research to discuss the measurement challenges in evaluating team science.
| |
|
Advancing the Science of Evaluating Team Science: Social Network Outcomes From the RoAR Initiative
|
| Timothy Huerta, Texas Tech University, tim.huerta@ttu.edu
|
| Cameron Norman, University of Toronto, cameron.norman@utoronto.ca
|
| Sharon Mortimer, Michael Smith Foundation for Health Research, smortimer@msfhr.org
|
| Alison Buchan, University of British Columbia, abuchan@medd.med.ubc.ca
|
|
Background: The shift from individual or small-group research to team science models requires shifts in social interaction patterns. Social network analysis can help evaluate these shifts.
Method: Participants were asked to indentify the investigators in the Life Sciences Institute (LSI) that they interacted with, the nature of those interactions and the degree to which it influenced their work in annual surveys over four years.
Results: An increase in the absolute numbers of collaborations within LSI was observed. While overall network density increased and the frequency of inter-departmental and inter-research group grew, the number of such increases suggest a developmental and slow growing process.
Conclusions: After four years the researchers in the LSI show significant alterations structure and processes used to guide interactions. Dr. Huerta will draws on more than a decade of experience with social network and systems research in exploring the role of social networks in evaluating team science.
Conclusions: Taking the Donabedian model of Structure-Process-Outcome indicates that the researchers in the LSI show significant alterations structure and processes used to guide interactions after four years.
| |
|
Advancing the Science of Evaluating Team Science: Scientometric-related Outcomes From the RoAR Initiative
|
| Sharon Mortimer, Michael Smith Foundation for Health Research, smortimer@msfhr.org
|
| Timothy Huerta, Texas Tech University, tim.huerta@ttu.edu
|
| Bianca Cervantes, University of British Columbia, bcervantes@exchange.ubc.ca
|
| Alison Buchan, University of British Columbia, abuchan@medd.med.ubc.ca
|
|
Background: The shift from individual or small-group research to team science models can be observed in the publications and grant application patterns of investigators over time.
Method: A comparison of funding by research group in the Life Sciences Institute (LSI) was completed for the major Canadian sources: Tri-Council (equivalent to NIH), peer-reviewed grants from other sources and research contracts.
Results: The data demonstrated a significant increase in Tri-Council funding in all groups, while comparison of publication metrics indicated potential shifts after 3 years.
Conclusions: Team science may be more promising as a discovery strategy within certain LSI areas. It is still too early to make definitive statements about the effect of team science model at the LSI with the exception of a rise in impact of papers in certain fields. Dr. Mortimer will expand on her role as a funder and researcher to discuss team science evaluation and its implications.
| |