Mintacím szerkesztése

Review of Competitive RDI Calls

Forum for Applicants and Evaluators

on calls promoting discovery research

József Pálinkás, President

Budapest, 25 January 2018

 Coordination of resources in a competitive
Mintacím szerkesztése
funding system The power of excellence
• coordinated management, flexible planning,
complex portfolio of calls
• well-balanced support to discovery research,
experimental development and innovation
• novel content, international competitiveness,
sustainability
• challenge-oriented rather than source-oriented
project funding
• practical applicability and marketability of RDI
results
Stakeholders of value-creating innovation
Mintacím szerkesztése
 Mintacím
The szerkesztése
system of RDI sources
Cohesion policy
instruments
Direct access to
Domestic sources international funding
Structural Funds
National Research, European Social Fund
Development and European Regional Horizon 2020
Innovation Fund Development Fund framework programme

Operational Programmes
Economic Development and Innovation
EUR 178.5 million
Operational Programme (EDIOP, in
approx. HUF 80 bn per year (approx. HUF 53.5 bn)
Hungarian: GINOP)
awarded to Hungarian businesses
Competitive Central-Hungary Operational
and research centres since 2014
Programme (CCHOP, in Hungarian:
VEKOP)

approx. HUF 750 bn between 2014 and 2020

Target groups,
Mintacím calls and funding decisions in the
szerkesztése
RDI calls (2015–2017)
 Major callsszerkesztése
Mintacím supporting discovery research and
aggregated annual budgets
PD: postdoctoral programme
K: research-initiated proposal HUF 16.5 bn
FK: research proposals by young researchers
NN: international research cooperation
ERC: ERC-related support HUF 12 bn
KH: publications with significant international impact
KKP: “Frontline” research excellence programme 2017 2018
HUF 8.8 bn
HUF 7.9 bn
HUF 5.7 bn K, FK, PD,
K, FK, PD, NN, KH,
2016 NN, KH ERC
2015 KKP
2014
K, PD, NN
K, PD, NN ERC K, PD
Mintacím szerkesztése

APPLICANTS
 Key figuresszerkesztése
Mintacím of the three main calls (K, FK, PD) supporting
discovery research in 2017
PD: postdoctoral programme
HUF bn K: research-initiated proposal
FK: research proposals by young researchers
25,00 700 660
20,63 600
20,00
500
15,00 400 332
9,32 300 264
10,00 204
6,51
3,00 3,96 200 109
5,00 1,37 90
100
0,00 0
TYPE K TYPE FK TYPE PD TYPE K TYPE FK TYPE PD

Submitted support request (HUF bn) Number of submitted Number of funded project
Amount awarded (HUF bn) project proposals proposals
 Beneficiaries of the three major calls (K, FK, PD) supporting
Mintacím szerkesztése
discovery research in 2017 by type of institution
HUF bn
25,00
21,23
20,00 791
800

15,00 600
10,98
404
10,00 400
6,20 226
4,16 155
5,00 3,08 200
0,43 0,20 16 8 45 14
0,69
0,00 0 BUSINESSES
HIGHER EDUCATION NETWORK OF BUSINESSES PUBLIC COLLECTIONS, HIGHER EDUCATION NETWORK OF PUBLIC COLLECTIONS,
HAS INSTITUTIONS OTHER PUBLIC INSTITUTIONS HAS INSTITUTIONS OTHER PUBLIC INSTITUTIONS

Submitted (HUF bn) Funded (HUF bn) Submitted Funded

 Beneficiaries of the three major calls (K, FK, PD) supporting
Mintacím szerkesztése
discovery research in 2017 by type of institution
HUF bn KKP: “Frontline”
KH: publications with significant international impact
12,00 11,46 80
10,00
62
60
8,00 52
45
6,00 40

4,00 3,00
20 12
1,18 1,00
2,00

0,00 0
TYPE KKP TYPE KH TYPE KKP TYPE KH
Submitted support request (HUF bn) Submitted Funded
Amount awarded (HUF bn)
 Beneficiaries of the new excellence call supporting discovery
Mintacím szerkesztése
research (KKP) in 2017 by type of institution
HUF bn
“Frontline”
8,00 25
25
6,29
6,00 20 18
4,66
15
4,00
10 7
1,73 5
2,00 1,27 5 2
0,51 0 0 0
0,00 0,00 0,00
0,00 0 HIGHER EDUCATION NETWORK OF BUSINESSES PUBLIC COLLECTIONS,
HIHGER EDUCATIONS NETWORK OF BUSINESSES PUBLIC COLLECTIONS, HAS INSTITUTIONS OTHER PUBLIC INTITUTIONS
HAS INSTITUTIONS OTHER PUBLIC INSTITUTIONS

Submitted (HUF bn) Funded (HUF bn) Submitted Funded

 Beneficiaries of the new excellence call (KH) supporting
Mintacím szerkesztése
discovery research in 2017 by type of institution
publications with significant international impact
HUF bn
Number
0,60 0,58 0,56 35
0,49 31
0,50 0,47 30 29
26
0,40 25 24

20
0,30
15
0,20
10
0,10 0,04 0,04 5 2 2
0,00 0,00
0 0
0,00 NETWORK OF
0 HIGHER EDUCATION NETWORK OF BUSINESSES PUBLIC COLLECTIONS,
HIGHER EDUCATION BUSINESSES PUBLIC COLLECTIONS, OTHER PUBLIC INSTITUTIONS
HAS INSTITUTIONS
HAS INSTITUTIONS OTHER PUBLIC INSTITUTIONS

Submitted (HUF bn) Funded (HUF bn) Submitted Funded

 Awarded
Mintacímprojects and funding in discovery research calls
szerkesztése
in 2017 by higher education institution
Eötvös Loránd University 68
University of Debrecen 50
Budapest University of Technology and Economics 42
University of Szeged 41
41
Semmelweis University 24 24
University of Pécs 24
24
University of Veterinary Medicine 4 4
University of Pannonia 4 4
2
University of Physical Education 2
7
Szent István University 7 8
Pázmány Péter Catholic University 8 3
University of Miskolc 3 4
Corvinus University of Budapest 4 1
Károli Gáspár University 1
HUF bn
0 1 2 3
 Awarded
Mintacímprojects and funding in discovery research calls
szerkesztése
in 2017 by HAS research institutions
HAS Research Centre for Natural Sciences 36
HAS Institute of Experimental Medicine 18
HAS Wigner Research Centre for Physics 19
HAS Biological Research Centre, Szeged 13
HAS Centre for Agricultural Research 15
HAS Research Centre for the Humanities 16
HAS Alfréd Rényi Institute of Mathematics 7
HAS Centre for Economic and Regional Studies 16
HAS Researsch Centre for Astronomy and Earth Sciences 9
HAS Centre for Ecological Research 7
HAS Research Institue for Linguistics 6
HAS Centre for Social Sciences 12
HAS Institue for Nuclear Research 5
HAS Institute for Computer Science and Control 5
HAS Office of submitted Research Groups 3
HAS Centre for Energy Research 3
0,00 0,50 1,00 1,50 HUF bn
 Age of principal
Mintacím investigators in discovery research calls
szerkesztése

2016 2017
338
339 322
326
299
241

189 172
144 118 125
127 121 113 113
92 94
67 68
59 48 43 50 39 48
35 35 42 34 23 14 12 17
81
1 0 0 1 1
<35 36-40 41-45 46-50 51-55 56-60 61-65 66-70 71-75 76-80 80< <35 36-40 41-45 46-50 51-55 56-60 61-65 66-70 71-75 76-80 80<
Submitted Funded Submitted Funded
 Age of principal
Mintacím investigators in discovery research calls
szerkesztése
by call type (2017)
Submitted Funded
169 178 63 59
157 143
135 45
103 39 35 40
88 90 33
70 26 26
43 39 13 14
5 18
7 8 2 51 1 1
1 1 1
<35 36-40 41-45 46-50 51-55 56-60 61-65 66-70 71-75 76-80 80< <35 36-40 41-45 46-50 51-55 56-60 61-65 66-70 71-75 76-80 80<

FK K PD
FK K PD

17 12 12
13
8 8
8 8 8 5
6 6 4
4 4 4 4 32 3 3
2 1 1 1 1
<35 36-40 41-45 46-50 51-55 56-60 61-65 66-70 71-75 76-80 80< <35 36-40 41-45 46-50 51-55 56-60 61-65 66-70 71-75 76-80 80<

KH KKP KH KKP
Mintacím szerkesztése

ASSESSMENT
 Evaluation system of RDI projects
Mintacím szerkesztése

expanded database of 2 to 5 independent panel assessment and

peer reviewers
a database of 15,000
peers, including
foreign experts
a set of assessment
criteria open to public
evaluations per decision proposal by
project proposal a steering committee
(depending on the or disciplinary panel
type of the call) by
anonymous peers

COMPREHENSIVE, MULTI-STAGE EVALUATION PROCESS

TRANSPARENT ASSESSMENT PROCEDURES AND CLEAR SET OF
CRITERIA
 Project proposals in the three discovery research calls (K, FK,
Mintacím szerkesztése
PD) split by the scientific councils (2017)
Number of submitted Number of funded
project proposals: project proposals: BTTK
Council of Humanities and Social Sciences

MFKMTK
AKÖFK BTTK AKÖFK BTTK Council of Mathematics, Physics, Chemistry,
21% 23% 20% 26% and Engineering

OBTK
OBTK MFKMTK OBTK MFKMTK Council of Medical and Biological Sciences
29% 27% 26% 28%
AKÖFK
Council of Agricultural, Environmental,
Ecological and Earth Sciences
 Project proposals of the KKP excellence call split by the four
Mintacím szerkesztése
scientific councils (2017)
“Frontline”
Number of submitted Number of funded BTTK
project proposals: project proposals: Council of Humanities and Social Sciences
AKÖFK BTTK AKÖFK BTTK
9% 9% 8% MFKMTK
8%
Council of Mathematics, Physics, Chemistry,
and Engineering
MFKMTK MFKMTK
34%
OBTK
OBTK 35% OBTK
47% Council of Medical and Biological Sciences
50%

AKÖFK
Council of Agricultural, Environmental,
Ecological and Earth Sciences
 Project
Mintacímproposals of the KH excellence call split by the four
szerkesztése
scientific councils (2017)
publications with significant international impact

Number of submitted Number of funded BTTK

project proposals: project proposals: Council of Humanities and Social Sciences
BTTK
2% MFKMTK
AKÖFK AKÖFK Council of Mathematics, Physics, Chemistry,
18% 19% and Engineering
MFKMTK MFKMTK
45% 44% OBTK
OBTK Council of Medical and Biological Sciences
35% OBTK
37%
AKÖFK
Council of Agricultural, Environmental,
Ecological and Earth Sciences
 Submitted research-initiated proposals (K, FK) per panel
Mintacím szerkesztése
member in the panels of the four councils (2017)
5,0
BTTK
4,1 Council of Humanities and Social Sciences
4,0
3,6
MFKMTK
3,1
3,0 2,9 2,9 Council of Mathematics, Physics,
2,6 Chemistry, and Engineering
2,4
2,1
2,0 OBTK
Council of Medical and Biological
Sciences
1,0

AKÖFK
0,0 Council of Agricultural, Environmental,
BTTK MFKMTK OBTK ÁKÖFK Ecological and Earth Sciences
2014-2016 2017
 Researcher-initiated proposals (K, FK) per panel member
Mintacím szerkesztése
submitted to the disciplinary panels in 2017
KM1
FIZ
GÉK
KM2
SUPRA
FT1
KLINO
KJS
KISOR
INFRA
MAT
FT2
PSP
IDEGT
AGR2
RGM
TRT
AGR1
AGR3
SZD
IVM
MOLBI
BIOIN
ÁJP
NYE
FIL
MIF
ÓKR
MKK

0,0 1,0 2,0 3,0 4,0 5,0 6,0 7,0

 Anonymous peer reviewers and their scores
Mintacím szerkesztése
in the K_17, FK_17 and PD_17 calls
reviewer
assessor
862
300 3,837 reviews in total
7.96

1,442 reviewers made 1 626 605

558
200 review only
408

100 272
229
149
reviews score
0 15 27 42 42

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 25
1 2 3 4 5 6 7 8 8,5 9 9,5 10
szakértők száma (fő)
Internationally good, with some shortcomings. (8)
Percentage of rejected assignments Internationally very good, with some shortcomings. (8.5)
2015 – 58% Internationally very good, results may have a substantial impact. (9)
Internationally excellent with respect to most criteria (9.5)
2016 – 46%
Outstanding proposal with respect to all criteria (10)
2017 – 51%
 Scores of project
Mintacím proposals in three different expert panels
szerkesztése
(2017)
rewiever Less optimal
51
46

7.29
Optimal 8.35
31 32
28
24 24 23 24
21 22

16 17 8.5 17 16
15
12 KKP
9 9
7
4 5
2 3 2
0 01 1 1
1 2 3 4 5 6 7 8 8,5 9 9,5 10
score
 Number
Mintacím and scores of assessed project proposals (2017)
szerkesztése

number of number of
reviews reviews
350 reviews/reviewer 30
reviews/reviewer
300 25
1 11
250 2
3 20 12
200 4 13
5 15 14
150 6 16
7 10
100 17
8
50 9 5 19
10 25
0 0
1 2 3 4 5 6 7 8 8,5 9 9,5 10 score 1 2 3 4 5 6 7 8 8,5 9 9,5 10

Number of reviews/reviewer Average score

1 8.3
2-5 8.0
6-10 7.7
11-15 7.2
16-25 8.2
 Mintacím
Guide szerkesztése
to high quality assessment
The assessment is more authoritative if

• the textual assessment is sufficiently elaborated

• the score accurately reflects the textual assessment

• it is based on facts and is free from assumption-based

conclusions

• it does not include any assessment criteria not listed in

the call
 Duration of the assessment of project proposals submitted to
Mintacím
discovery szerkesztése
research calls (2017)
Submission and adoption of
Call proposals, assignment of Anonymous peer reviews Panel assessment Duration of assessment
proposals to panels

PD_17 34 14 78 36

FK_17 35 14 54 59 124-128 calendar days

K_17 38 14 51 59

176 calendar
NN_17 57 14 154 8
days

KKP_17 85 14 41 11 66 calendar days

KH_17/1 30 14 98 8
120/73 calendar days
KH_17/2 48 14 51 8

0/02
 Assessment cost of project proposals submitted to
Mintacím szerkesztése
discovery research calls (2017)

NUMBER AMOUNT

ASSESSMENT COSTS 3,837 HUF 57.6 Mn

REMUNERATION WAIVED 1,369 HUF 20.5 Mn

REMUNERATION OF PANEL MEMBERS 519 HUF 68.2 Mn

Average cost of assessment required to prepare one funding decision

HUF 71.9 thousand
 What has been
Mintacím achieved of the NRDI Office’s commitments to
szerkesztése
the development of the funding system (Review of Competitive
Calls held 1 February 2017)

✓ • more funding opportunities for members of the researcher

community aged 40 or below (FK_17)
✓ • more important role of evaluation panels in the evaluation process
✓ • rearranged panel structure in consideration of the aspects of scientific
fields
✓ • renewed system of assessment criteria for more realistic scoring

✓
• more predictable and more efficient call process management
• higher number of project proposals per reviewer
 The new expert
Mintacím panels
szerkesztése
(29 panels until 2017, 27 panels from 2018) Goal: each reviewer should evaluate 6 to
10 project proposals in a call round
Expert panels of the Council of Humanities and Expert panels of the Council of Medical and Biological
Social Sciences Sciences
Law and Government Sciences, Political Science (ÁJP) Physiology, Pathophysiology, Pharmacology and Endocrinology
Economics (GAZD) (ÉLETTAN)
Literature (IROD) Genetics, Genomics, Bioinformatics and Systems Biology
Culture (KULT) (GÉNINFO)
Linguistics (NYELV) Neurosciences (IDEGT)
Psychology – Education (PSZINEV) Immunity, Cancer and Microbiology (IMMUN)
Archaeology (RÉG) Clinical Medicine (KLINO)
Society (TÁRS) Molecular and Structural Biology and Biochemistry (MOLBI)
History (TÖRT) Cellular and Developmental Biology (SEJTBIOL)
Expert panels of the Council of Physical Sciences
Expert panels of the Council of Complex Environmental
Informatics and Electrical Engineering (IVM)
Physics (FIZ) Sciences
Engineering, Metallurgy, Architecture and Transport Plant and animal breeding (NÖVÁLL)
Sciences (GÉK) Complex agricultural sciences (KOMPAGR)
Chemistry 1 (KM1) Earth sciences 1 (GEOINT)
Chemistry 2 (KM2) Earth sciences 2 (GEOEXT)
Mathematics and Computing Science (MAT) Ecology and evolution (ÖKOEVO)
Mintacím szerkesztése

NRDI FUND (2017)

ACHIEVEMENTS
 Funding
Mintacím disbursed from the National Research, Development and
szerkesztése
90,0 HUF bn Innovation Fund (NRDI Fund)
80,0

70,0

60,0

50,0

40,0 82,8 83,3

30,0
53,5
20,0 37,8 38,1 36,0 34,9
10,0 23,8

0,0
2010 2011 2012 2013 2014 2015 2016 2017
 Beneficiary target groups of the NRDI Fund in 2017
Mintacím szerkesztése

HUF bn 40,0
2015
35,0
2016
30,0 2017
25,0
20,0
15,0
10,0
5,0
0,0
Higher education Network of HAS Businesses Other Membership fees,
research insitutions State Treasury, fund
management fees
 Funding
Mintacímtoszerkesztése
higher education from the NRDI Fund in 2017
Budapesti Műszaki és Gazdaságtudományi Egyetem
Eötvös Loránd Tudományegyetem
Semmelweis Egyetem
Széchenyi István Egyetem
Debreceni Egyetem
Szegedi Tudományegyetem
Szent István Egyetem
Pécsi Tudományegyetem
Pázmány Péter Katolikus Egyetem
Óbudai Egyetem
Állatorvostudományi Egyetem
Pannon Egyetem
Soproni Egyetem
0,00 1,00 2,00 3,00 4,00
HUF
bn
 Funding
Mintacímtoszerkesztése
research centres from the NRDI Fund in 2017
MTA Természettudományi Kutatóközpont
MTA Kísérleti Orvostudományi Kutatóintézet
MTA Könyvtár és Információs Központ
MTA Wigner Fizikai Kutatóközpont
MTA Támogatott Kutatócsoportok Irodája
MTA Energiatudományi Kutatóközpont
MTA Szegedi Biológiai Kutatóközpont
MTA Agrártudományi Kutatóközpont
MTA Bölcsészettudományi Kutatóközpont
MTA Számítástechnikai és Automatizálási Kutatóintézet
MTA Közgazdaság- és Regionális Tudományi…
MTA Ökológiai Kutatóközpont
MTA Nyelvtudományi Intézet
MTA Csillagászati és Földtudományi Kutatóközpont
MTA Rényi Alfréd Matematikai Kutatóintézet
MTA Atommagkutató Intézet
MTA Társadalomtudományi Kutatóközpont
0,00 1,00 2,00
HUF bn
 Scientific
Mintacímperformance
szerkesztése of projects closed between 2015 and 2017
5000 4973 1400
4618 1307
4500
4077 1200
4000
3675 1055
3565
3500 984 970
3176 1000
906
3000
800 744
2500
2000 600
1500
400
1000 219 195 213
500 200 136
77 86
0 5 4 4
2015 2016 2017 0
KÖNYV
BOOK KÖNYVFEJEZET
CHAPTER SZABADALOM
PATENT ABSZTRAKT
ABSTRACT DISSZERTÁCIÓ
THESIS
Journal article This NKFI
Of this: support
NRDI Fundindicating
indicated as funder
2015 2016 2017

Quick report based on summary data from final

reports recorded in the EPR.
Mintacím szerkesztése

CALL FOR
PROPOSALS (2018)
2018 discovery
Mintacím research calls are now open
szerkesztése
with a total budget of HUF 12 billion

www.nkfih.gov.hu
 2018 discovery
Mintacím research calls – main changes
szerkesztése
• No age limit in the postdoctoral call, time from date of PhD is the only consideration

• Postdoc salary: gross HUF 350,000 /month + contributions payable by employer

• Host institutions are required to confirm in a statement that the infrastructural conditions
required for the project are available

• Summary table on the 2018 rules of project proposals submitted by the same principal
investigator at the same time or in addition to a running project funded under a previous
call

• Updated and clearer rules on the use of funds

 2018 discovery
Mintacím research calls – main changes
szerkesztése

General criteria for the assessment of final reports (as a pilot for OBTK)

1) Quality comes first (rather than quantity) in the assessment of scientific performance
2) Assessment is based on publications containing original observations in which the principal
investigator has significant authorship
3) Co-authored publications containing original observations can only be considered if the applicant
contributed to them and it was indicated so in the publication
4) Publications can only be considered for the assessment of final reports if the NRDI Office project
code was duly indicated
5) The final report of NN type projects can only be accepted, if at least 1 publication containing
original observations has been published together with the co-applicant
6) In the case of Open Access periodicals with no impact factor, the expectable reputation of the
periodical needs to be examined
 Assessment process of the 2018 discovery research calls
Mintacím szerkesztése
Submission
Announcement of the call Assessment of proposals Funding decisions Contracting
deadline

21/12-
PD_18 05/01 08/02 15/02-23/06 30/06 01/07-31/08

21/12-
FK_18 12/02 19/02-23/06 30/06 01/07-31/08
05/01

21/12-
K_18 05/01 14/02 21/02-23/06 30/06 01/07-31/08
NN_18
ANN_18 21/12-
15/02 22/02-23/06 30/06 01/07-31/08
SNN_18 05/01

05/04- 22/09- 05/1 06/11-30/11

KH_18 15/02 29/03 08/05 09/05-30/06 15/09
04/05 02/11 1
05/04- 22/09- 05/1 06/11-30/11
ERC_18 28/02 30/03 08/05 09/05-30/06 15/09
04/05 02/11 1

KKP_18 01/04 15/05 22/05-21/07 28/07 01/08-28/09

January February March April May June July August September October November December

0/02
 NRDI Fund
Mintacím disbursements as reflected in R&D spending figures
szerkesztése
on the basis of institutions’ data sheets submitted to the
Hungarian Central Statistical Office (HCSO)
Research universities’ R&D spending
(HCSO data) and amounts disbursed from NRDI Office initiatives
the NRDI Fund to them in 2016
1. Cooperation agreement with the HCSO
HUF 5.3 bn 2. Raising awareness of the importance of
Total R&D spending by
the 6 research universities Data are summarised by filling in OSAP data sheets correctly
the HCSO

R&D spending
in 2016 according to
HCSO data
3. Development of methodology proposals
Total amounts disbursed
by the NRDI Fund to the 6 Data are compared by 4. Involvement of a wider scope of data on
research universities in the HCSO
2016
research funding

HUF 14.5 bn
 Ever changing
Mintacím researcher community and
szerkesztése
research environment
“totem” (treats research infrastructure as own) “open science”

“manager”
“leader”
IF
Google Schoolar SCImago, Altmetrics
conference
F1000... on-line networking
Peer-review
“science only”
spin-off, start-up

WoS, Scopus “open access”

New initiatives to stimulate researcher performance adapted to the environment

Mintacím szerkesztése

www.nkfih.gov.hu