| Session Title: National Performance Evaluation System of Research and Development Programs in Korea: System and Applications |
| Multipaper Session 847 to be held in Pratt Room, Section A on Saturday, November 10, 3:30 PM to 5:00 PM |
| Sponsored by the Research, Technology, and Development Evaluation TIG |
| Chair(s): |
| Sang-Youb Lee, Korea Institute of Science and Technology Evaluation and Planning, sylee@kistep.re.kr |
| Discussant(s): |
| Jiyoung Park, Korea Institute of Science and Technology Evaluation and Planning, jypark@kistep.re.kr |
| Abstract: There has been increased requirement on the accountability and efficiency of national R&D programs in according to the increased R&D investment within the limited national resources. To cope with these issues, Korea is also trying to upgrade its performance evaluation system under the guidance of NSTC(National Science and Technology Committee). There are several points to be considered in Korean R&D evaluation system. Firstly individual program evaluation is working very well, but these evaluation methods are not systematically organized in the national evaluation system(NES). Secondly most of the current evaluation is focused on the monitoring of the R&D programs, therefore there is little in-depth performance evaluation to improve the programs. To deliver these problems, Korean government set up systematic performance evaluation methodology in national scale. In this session, several performance evaluation methods and applications were introduced |
| Method and Application of the Survey and Analysis of National Research and Development (R&D) Programs for the Performance Evaluation in Korea |
| Keun-Ha Chung, Korea Institute of Science and Technology Evaluation and Planning, khchung@kistep.re.kr |
| Hyejung Joo, Korea Institute of Science and Technology Evaluation and Planning, hjoo@kistep.re.kr |
| Herin Ahn, Korea Institute of Science and Technology Evaluation and Planning, herini@kistep.re.kr |
| The Survey and Analysis of national R&D programs has been performed by Korea Institute of S&T Evaluation and Planning (KISTEP) since 1999, in order to enhance the efficiency and effectiveness of national R&D investment as national R&D budget increases. Every government-funded research project is the subject of the survey and analysis, and Korea Research and Development Integrated Management System (KORDI) is used for collecting the required data such as program information, project information and performance information. Using the collected data, annual government R&D investment is analyzed in the area of socio-economic objectives, research activities, research performing sectors and technology classifications, etc. In this study, the result of the survey of the national R&D investment is analyzed and especially the feedback system of this survey and analysis to the annual performance evaluation and budget allocation for R&D programs is organized to maximize the efficiency of R&D investment. |
| The Performance Evaluation of Research and Development Programs in Korea |
| Seong-jin Kim, Korea Institute of Science and Technology Evaluation and Planning, shaqey@kistep.re.kr |
| Soon Cheon Byeon, Korea Institute of Science and Technology Evaluation and Planning, sbyeon@kistep.re.kr |
| Korea has a unique centralized overall evaluation and coordination system for the national research and development programs. Korea's national research and development evaluation system is administered by the National Science and Technology Council (NSTC). The evaluation of the important specific R&D programs are performed every three years, especially focusing on the performance of those programs, and the other programs are self-evaluated by each ministries followed by meta-evaluation performed by NSTC. The purpose of this study is to set up guidelines for the performance evaluation of the specific programs and self-evaluation of the other programs in the national R&D evaluation system. This study also includes evaluation methods for the national research and development programs. In conclusion, we hope that our guidelines and methods can contribute to the advancement of national R&D evaluation system. |
| Design of Metaevaluation Model for National Research and Development Programs in Korea |
| Young Soo Ryu, Korea Institute of Science and Technology Evaluation and Planning, soory@kistep.re.kr |
| Soon Cheon Byeon, Korea Institute of Science and Technology Evaluation and Planning, sbyeon@kistep.re.kr |
| Byung Dae Choi, Hanyang University, choibd@hanyang.ac.kr |
| This study presents Meta-Evaluation Model as an evaluation tool for National R&D programs by examining the essence of R&D evaluation. In Korea, an in-depth analysis of the national R&D has been carried out partially since 1999. And since 2006, targets of in-depth analysis have been reduced and the Meta-Evaluation has been performed for the efficiency of evaluation of National R&D programs. Now, as a 2nd year, it is the right time to study on the Design of Meta-Evaluation Model in depth. In this study, the characteristics of the meta-evaluation in R&D programs were derived considering the essence of R&D evaluation theoretically. Also a meta-evaluation model for National R&D programs was derived and the evaluation indicators were developed. The meta-evaluation model was also verified in the National R&D program evaluation system. The verification of evaluation indicators were performed by Delphi method utilizing meta-evaluation committee members for the National R&D program. |
| Development of the Evaluation Methodology for the Basic Research in Korea |
| Hyeyoung Yang, Korea Institute of Science and Technology Evaluation and Planning, hyyang@kistep.re.kr |
| Sangki Jeong, Korea Institute of Science and Technology Evaluation and Planning, sjeong@kistep.re.kr |
| The characteristics and outcomes of the basic research are quite different from those of the applied research or product development. However, the similar evaluation methodology has been generally applied to the basic research as compared to the others without consideration of its characteristics. This makes an obstacle to a quality improvement of the results of the basic research. The present study was undertaken to investigate the characteristics of the basic research and its outcomes through a survey of researchers in Korea and a network analysis methodology with SCI DB for the basic research evaluation. In this study, the implications will be presented about the basic research evaluation as well as the quality improvement of the outcomes of the basic research. |
| Performance Evaluation of Agriculture Research and Development Programs in Korea |
| Hoijong Jung, Korea Institute of Science and Technology Evaluation and Planning, jhj119@kistep.re.kr |
| Agriculture R&D programs are mainly supported by the government, the Ministry of Agriculture and Forest and the Rural Development Administration because of a long-term investment and a high risk of the private research. Therefore agricultural R&D programs of the agencies are important for improving agricultural technology in Korea. The purpose of the present report is to suggest a new revenue for improving efficiency of programs by reflecting unique evaluation indices on a portion of unique features of agriculture R&D programs and evaluating its performance. According to the features, the results reveal that the overall performance of R&D effort is more positive than what has been known. It could provide the direction of a specific program with a variety of information about the performance. This approach could be taken into account in improving the national evaluation system (NES) of government R&D programs. |