| Session Title: Looking Inside the Research Center Black Box: Using Evaluation Research to Promote Organizational Effectiveness of Scientific Research Centers |
| Multipaper Session 376 to be held in International Ballroom E on Thursday, November 8, 1:55 PM to 3:25 PM |
| Sponsored by the Research, Technology, and Development Evaluation TIG |
| Chair(s): |
| Denis Gray, North Carolina State University, denis_gray@ncsu.edu |
| Discussant(s): |
| Gretchen Jordan, Sandia National Laboratories, gbjorda@sandia.gov |
| Abstract: Surprisingly, most of the evaluations of research centers (including collaborative centers) have been conducted at the program-level of analysis. That is, they have examined processes and outcomes for a whole program while ignoring variation across and within centers (e.g., for involved students). This session will show how one can move beyond this "black box" approach to evaluation by describing an ongoing NSF-sponsored center evaluation effort that attempts to improve center management through a combination of short term survey feedback methodology and a series of more targeted studies. These studies attempt to link specific center and/or individual-level variation with important center and stakeholder outcomes. Targeted studies will examine: factors that affect graduate students outcomes; role of leadership; factors affecting success of multi-institutional center partnerships; and factors affecting the success and survival of "graduated" centers. Implications for organizational learning and future research will be discussed. |
| Evaluating Leadership Development in an Research and Development (R&D) Context: Assessing Alpha, Beta, and Gamma Change |
| Bart Craig, North Carolina State University, bart_craig@ncsu.edu |
| Leaders in research and development (R&D) settings face specialized challenges that require them to motivate and support creative professionals, obtain resources in unstructured environments, and measure success in the absence of well-defined metrics. This presentation will report on the evaluation of a leadership development intervention specifically tailored to the needs of directors of industry-university cooperative research centers (IUCRCs). Directors received feedback and coaching based on results from 360-degree performance assessment and personality assessment. The 360-degree assessment instrument was readministered approximately four months after the initial feedback. The self-ratings from the two administrations will be compared for evidence of beta and gamma change using item response theory and confirmatory factor analysis. Alpha change will be assessed after controlling for beta and gamma change, if those are found to have occurred. Results will be interpreted in terms of implications for evaluating leadership development in this specialized setting. |
| A Multi-variate Study of Graduate Student Satisfaction and Other Outcomes Within Cooperative Research Centers |
| Jennifer Schneider, North Carolina State University, jsschnei@ncsu.edu |
| While it is assumed that the multidisciplinary, team-based, experiential elements of center-based training contribute to a variety of variety of educational advantages for graduate students including career opportunities, increased scholarly productivity, and development of soft skills (teamwork, communication, leadership), there is little empirical data to support these effects and less identifying which center features are truly instrumental. A cross-sectional predictive analysis was conducted to identify which individual center mechanisms positively or negatively influence graduate student outcomes. Data were collected from graduate students (n=190, 37% useable response rate) working in National Science Foundation's I/UCRC and STC programs (34 centers, 87% response rate) via a web-based questionnaire. Student outcomes include satisfaction, perceived skills, organizational commitment, scholarly achievements, career goals, and feelings of a competitive advantage. Results indicate that consistent and powerful predictive variables include: Multidisciplinary Center Experience, Experiential Expanded Center Experiences, Technical Project Involvement, and frequency of interactions with thesis/dissertation committee and Center industry members. |
| Enhancing Collaboration Between Historically Black Colleges and Universities (HBCU) and Research Extensive Universities |
| Andrea Lloyd, North Carolina State University, tejidos24@yahoo.com |
| Collaborative partnerships between historically black colleges and universities (HBCUs) and research extensive universities (REUs),often through involvement in centers, is regarded as a very promising mechanism for having the dual effect of maintaining the S&E focus of African-American students throughout their educational pursuits and of strengthening the institutions, HBCUs, that prepare a significant number of our African-American scientists (Tanaka & Gladney, 2004). The present research is a mixed methods study that explores the factors that are relevant to the adoption of a partnership strategy, the factors that are relevant to the success or failure of the sampled HBCU/REU partnerships and compares factors between HBCU/REU partnerships and REU/REU partnerships. Data has been collected from 29 partnerships participants across several universities. Key constructs to be examined include satisfaction, perceived obstacles and facilitators. |
| Predictors of Cooperative Research Center Post-Graduation Survival and Success |
| Lindsey McGowen, North Carolina State University, lindseycm@hotmail.com |
| Industry/University Cooperative Research Centers are supported by funding from NSF but like other center programs are expected to achieve self-sufficiency after a fixed term (ten years). However, there is little evidence about the extent to which government funded programs are able to make this transition. This study attempts to identify the factors that predict center survival and success after they have graduated from NSF funding. Archival data and qualitative interviews with Center Directors are used to explore the fate of I/UCRCs post graduation. The study examines infrastructure, transition planning, center management, faculty involvement, institutional factors, research area, industrial factors, and educational programs to determine if these constructs predict center success in terms of financial viability, industry engagement and support for multidisciplinary collaborative research, university support, faculty satisfaction and commitment, support for students, and technology transfer. |