| Session Title: Evaluation of Research, Technology, and Development (RT&D) to Illuminate Innovation |
| Multipaper Session 863 to be held in Wekiwa 6 on Saturday, Nov 14, 1:40 PM to 3:10 PM |
| Sponsored by the Research, Technology, and Development Evaluation TIG |
| Chair(s): |
| Shigeko Togashi, National Institute of Advanced Industrial Science and Technology, s-togashi@aist.go.jp |
| Abstract: In this session, we propose how we can measure outcomes of RT&D to illuminate innovation in a national research institute (AIST) and a national funding organization (NEDO) belonging to METI. At first, a strategy formation as an essential issue in the process of RT&D for innovation will be proposed. Then, an efforts to set a useful evaluation system by analyzing issues to be incorporated in AIST will be introduced; 1) efficient and practical evaluation with appropriate interval and self-evaluation, 2) novel evaluation indexes to evaluate the diversity of RT&D and human resource cultivation, 3) appropriate formation and selection scheme of reviewers for diversity and excellence, and 4) secure feedback action to reorganize RT&D agenda. Moreover, follow-up monitoring in NEDO will be mentioned in order to grasp the current statuses of project participants and their ex-post activities including further RT&D and commercialization for 5 years after the end of projects. |
| Strategy and Evaluation in Research, Technology and Development (RT&D) for Innovation |
| Naoto Kobayashi, National Institute of Advanced Industrial Science and Technology, naoto.kobayashi@aist.go.jp |
| Osamu Nakamura, Science and Technology Promotion Bureau, Nagasaki Prefectural Government, osamu.nakamura@pref.nagasaki.lg.jp |
| Kenta Ooi, National Institute of Advanced Industrial Science and Technology, k-ooi@aist.go.jp |
| A strategy formation is an essential issue in the process of RT&D for innovation. The strategy should reflect global, social, national, and technological points of view. RT&D programs/projects are to be planned, implemented and evaluated along this strategy in AIST. The evaluation should include (1) foresight, (2) process and progress, (3) output and outcome and (4) the young human resource cultivation. The evaluation of the strategy (meta-evaluation) should be, on the other hand, executed by checking the gap between scenario of the strategy and the real world. If a big gap is still observed between them after a fixed period (5~10 years), the reformation of the strategy is necessary. We have named this cycle SDCS (Strategy-Do-Check-Strategy) cycle [1], and the combined SDCS-PDCA cycle might be an effective tool for the realization of the RT&D programs originating from the research institute. [1] O.Nakamura et al., New Directions for Evaluations 118, pp.25-36 (2008). |
| Systematic Evaluation to Recognize Outcomes of the National Institute of Advanced Science and Technology (AIST) in Society |
| Kenta Ooi, National Institute of Advanced Industrial Science and Technology, k-ooi@aist.go.jp |
| Osamu Nakamura, Science and Technology Promotion Bureau, Nagasaki Prefectural Government, osamu.nakamura@pref.nagasaki.lg.jp |
| Shuichi Oka, National Institute of Advanced Industrial Science and Technology, s-oka@aist.go.jp |
| Koji Masuda, National Institute of Advanced Industrial Science and Technology, koji.masuda@aist.go.jp |
| Shigeko Togashi, National Institute of Advanced Industrial Science and Technology, s-togashi@aist.go.jp |
| Naomasa Nakajima, National Institute of Advanced Industrial Science and Technology, n-nakajima@aist.go.jp |
| With the introduction of an outcome-oriented strategic evaluation system specifically designed to foresee and monitor RT&D outcomes, research units in AIST have come to have a clear strategic agenda for formulating RT&D scenario toward outcome that contribute to AIST's mission. On the other hand, the present system needs some systematic modifications to recognize how AIST has been contributing toward the development of a sustainable society. We have been analyzing issues to be incorporated for a useful evaluation system; 1) efficient and practical evaluation with appropriate interval and self-evaluation, 2) novel evaluation indexes to evaluate the diversity of RT&D and human resource cultivation, 3) appropriate formation and selection scheme of reviewers for diversity and excellence, and 4) secure feedback action to reorganize RT&D agenda to maximize the total achievements of AIST. Systematic evaluation for impact assessment must be constructed so that AIST could act as a real innovation hub in society. |
| Utilization of Follow-up Monitoring Results of Projects for Accountability and Research, Technology and Development (RT&D) Management at New Energy and Industrial Technology Development Organization (NEDO) |
| Jun-ichi Yoshida, New Energy and Industrial Technology Development Organization, yoshidajni@nedo.go.jp |
| Tsutomu Kitagawa, New Energy and Industrial Technology Development Organization, kitagawattm@nedo.go.jp |
| Takahisa Yano, New Energy and Industrial Technology Development Organization, yanotkh@nedo.go.jp |
| New Energy and Industrial Technology Development Organization (NEDO) is Japan's largest public RT&D management organization for promoting wide range of industrial technologies. To ensure social accountability and to provide feedback for improving RT&D management, it is very important for funding agencies as NEDO to monitor the post-project activities of the private companies participated in the past projects toward the practical application of RT&D achievements. Thus, we conducted follow-up monitoring in order to grasp the current statuses of project participants and their ex-post activities including further RT&D and commercialization for 5 years. In this study, the analysis of the latest project evaluation results and follow-up monitoring data including some indicators for short-term outcomes are discussed. |