|
Session Title: Interim Evaluation: The Quality of Research and the Quality of Evaluation - Case Study of the FP7 Interim Evaluation
|
|
Panel Session 726 to be held in BONHAM E on Saturday, Nov 13, 8:00 AM to 9:30 AM
|
|
Sponsored by the Research, Technology, and Development Evaluation TIG
|
| Chair(s): |
| Neville Reeve, European Commission, neville.reeve@ec.europa.eu
|
| Abstract:
The 7th Framework Programme was a major step forward for the EU’s multiannual framework programmes for research and technology development. With a more than doubling of the overall budget, a duration of 7 rather than the previous 4 years and a vastly expanded range of research and policy related objectives, FP7 demonstrates the increasing importance of research amongst the policy aims of the EU. In keeping with these changes, the policy and decision makers have asked for a much enhanced approach to evaluation and monitoring. Major evaluation exercises are held every two to three years, supported by annually many different types of specific evaluation study at the level of the component research activities. This session looks at one such large-scale exercise: the FP7 Interim evaluation, which addresses specifically the questions of research quality, linking these with attempts to measure performance through progress against objectives.
|
|
Case Study of the FP7 Interim Evaluation: Notions of Quality in Evaluation Design
|
| Neville Reeve, European Commission, neville.reeve@ec.europa.eu
|
|
The interim evaluation of FP7 has involved a combination of different evaluation methods at different levels including modified peer review, analytical studies, statistical analysis and self-assessment. The aim is to provide good overall coverage and a balanced approach between the analysis of issues in depth and at the macro level. The presentation will expound on the benefits gained from the different approaches and the different notions of quality in designing the evaluation.
|
|
|
Evaluation Quality and Long-term Impact
|
| Peter Fisch, European Commission, peter.fisch@ec.europa.eu
|
|
The FP7 interim evaluation will not only provide a first look at the implementation of the programme it will also lay the basis for future EU research activities. Developing the terms of reference, the timing and the implementation of the evaluation has involved difficult trade-offs between getting ‘results now’ versus the types of analysis that are better carried out over the long–term. The presentation will explore these issues, examine the results of the evaluation and how these will be used in policy making.
| |
|
The Use of Independent Experts to Verify Quality and Strengthen Evaluation Process
|
| Wolfgang Polt, Joanneum Research, wolfgang.polt@joanneum.ac.at
|
|
The FP7 interim evaluation brings together a modified peer review style exercise based on the use of experts from across the spectrum of academic, public and commercial life as well as experts in the field of evaluation and science policy to support the process of generating data, analysis and for reasons of quality control. The presentation will examine the different roles played by the ‘independent experts’, examining questions of selection, role and their impact on the process.
| |
|
The Role of Stakeholders on the Quality of Evaluation Design, Implementation and Impact
|
| Iain Begg, London School of Economics,
|
|
The presentation will examine the different approaches for connecting with stakeholders in the FP7 interim evaluation. These include the preparation of the terms of reference, gathering evidence for the evaluation itself and disseminating the results of the evaluation to different audiences including policy makers and decision makers. The results of specific exercise used in the evaluation to ask the opinions of stakeholders will be described and analysed.
| |