kom (2020) 0184 (forslag) - COMMISSION STAFF WORKING DOCUMENT Evaluation of the Innovation and Networks Executive Agency (INEA) Accompanying the document Report from the Commission to the European Parliament, the Council and the Court of Auditors Evaluation of the Consumers, Health, Agriculture and Food Executive Agency, the Executive Agency for Small and Medium-Size Enterprises, the Innovation and Networks Executive Agency, the Education, Audiovisual and Culture Executive Agency, the Research Executive Agency and the European Research Council Executive Agency

Europaudvalget 2020
KOM (2020) 0184
Offentligt

EUROPEAN

COMMISSION

Brussels, 29.4.2020

SWD(2020) 73 final

COMMISSION STAFF WORKING DOCUMENT

Evaluation of the Innovation and Networks Executive Agency (INEA)

Accompanying the document

Report from the Commission to the European Parliament, the Council and the Court of

Auditors

Evaluation of the Consumers, Health, Agriculture and Food Executive Agency, the

Executive Agency for Small and Medium-Size Enterprises, the Innovation and Networks

Executive Agency, the Education, Audiovisual and Culture Executive Agency, the

Research Executive Agency and the European Research Council Executive Agency

{COM(2020) 184 final} - {SWD(2020) 74 final} - {SWD(2020) 75 final} -

{SWD(2020) 76 final} - {SWD(2020) 77 final} - {SWD(2020) 78 final}

kom (2020) 0184 (forslag) - COMMISSION STAFF WORKING DOCUMENT Evaluation of the Innovation and Networks Executive Agency (INEA) Accompanying the document Report from the Commission to the European Parliament, the Council and the Court of Auditors Evaluation of the Consumers, Health, Agriculture and Food Executive Agency, the Executive Agency for Small and Medium-Size Enterprises, the Innovation and Networks Executive Agency, the Education, Audiovisual and Culture Executive Agency, the Research Executive Agency and the European Research Council Executive Agency

TABLE OF CONTENTS

NTRODUCTION

.................................................................................................................... 2

ACKGROUND

...................................................................................................................... 3

2.1 Description of the intervention and its objectives ............................................................ 3

2.2 Baseline and points of comparison ................................................................................... 4

MPLEMENTATION

TATE OF PLAY

....................................................................................... 5

ETHOD

.............................................................................................................................. 7

4.1 Short description of methodology .................................................................................... 7

4.2 Limitations and robustness of findings ............................................................................. 8

NALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS

................................................. 9

5.1

5.2

5.3

5.4

Effectiveness ................................................................................................................ 9

Efficiency................................................................................................................... 12

Coherence .................................................................................................................. 16

Retrospective Cost-Benefit Analysis ......................................................................... 17

ONCLUSIONS

.................................................................................................................... 19

NNEX

: E

VALUATION QUESTIONS

...................................................................................... 21

NTRODUCTION

The purpose of this periodical (every three years) evaluation of the operation of the

Innovation and Networks Executive Agency (INEA) is to assess INEA’s implementation of

the parts of the EU funding programmes (Connecting Europe Facility, Horizon 2020 and the

Trans-European Transport Network (TEN-T) and Marco Polo legacy programmes) which

have been entrusted to it.

INEA is governed by: (1) the Framework Regulation for executive agencies

; (2) the INEA

‘Act of Establishment’

, which creates the Agency and sets out its mandate; (3) the INEA Act

of Delegation

which specifies the tasks to be carried out by, and the powers delegated to, the

Agency in order to perform its mandate; and (4) the decision establishing the INEA Steering

Committee

In line with the Commission’s better regulation principles

, the evaluation applies several

standard evaluation criteria. The evaluation assesses whether the Agency has fulfilled its tasks

in an effective and efficient way, whether there are overlaps / gaps / inconsistencies in the

management of the programme portfolio by the Agency, and whether there is a clear

delineation of tasks between INEA and the parent DGs or other executive agencies

(coherence)

The evaluation also assesses whether the functioning of the Agency has yielded the expected

positive results as estimated in the Cost-Benefit Analysis for delegating task to the executive

Agency

and identify potential areas of improvement. To this end, the estimations of the Cost-

Benefit Analysis of 2013 have been tested to provide evidence on the validity of the

assumptions made in the ex-ante scenario by considering the actual costs and benefits of

programme implementation by the Executive Agency in a structured way. The aspects to be

covered by the Cost-Benefit Analysis are specified in Article 3(1) of the Framework

Regulation

and the Guidelines on establishing and operating executive agencies

Council Regulation 58/2003 of 19 December 2003 laying down the statute for executive agencies to be

entrusted with certain tasks in the management of Community programmes, OJ L 11 of 16 January 2003, p. 1.

Commission Implementing Decision 2013/801/EU of 23 December 2013 establishing the Innovation and

Networks Executive Agency and repealing Decision 2007/60/EC as amended by Decision 2008/593/EC.

Commission decision C (2013) 9235 of 23 December 2013 delegating powers to the Innovation and Networks

Executive Agency with a view to the performance of tasks linked to the implementation of Union programmes in

the field of transport, energy and telecommunications infrastructure and in the field of transport and energy

research and innovation comprising, in particular, implementation of appropriations entered in the general

budget of the Union.

Commission Decision C(2014)520 of 6 February 2014 appointing the members and observers of the Steering

Committee of the Innovation and Networks Executive Agency, as last amended by Commission Decision

C(2018) 769 final of 13 February 2018.

Commission’s Better Regulation Communication COM (2015) 215 final

and its accompanying Staff Working

Documents SWD (2015) 111 final and SWD (2015) 110 final.

The assessment of the evaluation criterion ‘EU added-value’, i.e. why the EU should act, is not perceived to be

a relevant criterion for the evaluation of INEA as INEA carries out tasks which the European Commission has

transferred to it. The EU added-value of the programmes that INEA manages is assessed in the context of the

programmes evaluations. The previous needs which INEA was meant to address and whether they still are

pertinent today (relevance) will be presented in the context of the evaluation.

Cost-Benefit analysis for the delegation of certain tasks regarding the implementation of Union Programmes

2014-2020 to the executive agencies - Final report for the Commission of 19 August 2013.

Identification of the tasks justifying outsourcing, a cost-benefit analysis which includes the costs of

coordination and checks, the impact on human resources, possible savings within the general budgetary

framework of the European Union, efficiency and flexibility in the implementation of outsourced tasks,

simplification of the procedures used, proximity of outsourced activities to final beneficiaries, visibility of the

The evaluation does not cover the achievements of the programmes managed by INEA, which

are subject to mid-term and ex-post evaluations themselves. The evaluation of INEA

nevertheless provides useful input for these programme evaluations, considering that the

performance of the Agency affects the efficiency and effectiveness of the programmes it

manages.

The evaluation examines the efficient use of resources and the effective achievement of the

tasks entrusted to the Agency. It looks in particular at whether the alignment of more coherent

programme portfolios with the Agency’s core competences and its brand identity delivered

the estimated qualitative benefits; whether the assembly of the management of different EU

programmes delivered the estimated synergies, simplification and economies of scale;

whether the pooling of instruments guaranteed consistent service delivery and whether there

is scope for simplification and further efficiency gains.

The evaluation covers all the tasks carried out by the Agency during the period from January

2014 to December 2016. The evaluation is supported by a study carried out by an external

contractor.

The results of this study are summarised hereafter and this evaluation will be presented to the

European Parliament, to the Council and to the Court of Auditors in accordance with Article

25(1) of the Framework Regulation.

The results will feed into the reflection to assess the opportunity of expanding/ modifying the

Agency’s mandate

in view of the delegation of the implementation of 2021-2027 EU

programmes.

ACKGROUND

2.1 Description of the intervention and its objectives

The outsourcing of certain management tasks to the Agency, as outlined in the Framework

Regulation and in INEA’s Act of Establishment,

intends to:



Allow the Commission to focus on its institutional tasks, i.e. tasks assigned to the

institutions by the Treaty which require discretionary powers in translating political

choices into action. Such institutional tasks should not be outsourced.



Enable the Commission to achieve the objectives of the delegated EU programmes

more effectively. According to the Cost-Benefit Analysis carried out in 2013, the

delegation of certain programme tasks to the Agency was estimated to be more cost-

efficient than an in-house scenario. The Act of Establishment stated that the alignment

of more coherent programme portfolios with the Agency’s core competences and its

brand identity would bring qualitative benefits. In addition, it projected that

assembling the management of different EU programmes would bring synergies,

simplification and economies of scale.

The original objective of the intervention is entrusting the Agency with the implementation of

several EU programmes in part or fully:

EU as promoter of the EU programme concerned and the need to maintain an adequate level of know-how inside

the Commission.

Appendix II of the Guidelines for the establishment and operation of executive agencies financed from the

Union budget (C (2014) 9109 from 2 December 2014; pp. 64-72).



Connecting Europe Facility: transport, energy and telecommunications,



Horizon 2020: transport and energy research (Part III Societal challenges of the

Specific Programme),



Legacy of the Trans-European Transport Network (TEN-T) programme,



Legacy of the Marco Polo II programme (transferred from the Executive Agency for

Competitiveness and Innovation (EACI), now Executive Agency for Small and

Medium-Sized Enterprises (EASME).

As of 1 May 2018, the mandate of the Agency was extended to include implementation of the

new Wifi4EU initiative as part of CEF Telecoms

The Agency works under the supervision of four parent Directorates-General:



DG Mobility and Transport (MOVE)

–

Lead parent DG;



DG Energy (ENER);



DG Communications Networks, Content and Technology (CNECT); and



DG Research and Innovation (RTD).

In implementing the programmes delegated to it, INEA is responsible for monitoring the

projects, making the necessary checks and recovery procedures, and performing budget

implementation tasks covering revenue and expenditure within the meaning of the EU

Financial Regulation

, and in particular:



Manage the operations and procedures leading to the adoption of Commission award

decisions and to the conclusion of grant agreements and manage the ensuing decisions

and agreements;



Provide support in programme implementation;



Perform all the operations required to launch contests and award prizes in accordance

with the EU Financial Regulation;



Conclude public procurement procedures and manages the ensuing contracts,

including the operations required to launch and conclude public procurement

procedures.

2.2 Baseline and points of comparison

The current evaluation of INEA operations during 2014-2016 assesses the actual costs and

benefits of programme implementation by INEA (executive agency scenario) when compared

with the alternative scenario of management by the Commission services (in-house scenario).

Accordingly, the reference point for the present INEA evaluation is the 2013 ex ante Cost-

benefit analysis, and the Specific Financial Statement

(SFS) of INEA.

Commission Decision C(2018) 1281 final of 27.2.2018 and C(2018) 6366 final of 4.10.2018 on amending

Decision C(2013)9235 delegating powers to the Innovation and Networks Executive Agency, as regards

promotion of internet connectivity in local communities.

Regulation (EU, Euratom) 2018/1046 of the European Parliament and of the Council of 18 July 2018 on the

financial rules applicable to the general budget of the Union, amending Regulations (EU) No 1296/2013,

(EU) No 1301/2013, (EU) No 1303/2013, (EU) No 1304/2013, (EU) No 1309/2013, (EU) No 1316/2013,

(EU) No 223/2014, (EU) No 283/2014, and Decision No 541/2014/EU and repealing Regulation (EU,

Euratom) No 966/2012, OJ L 193, 30.7.2018, p.1.

INEA’s SFS was updated in March 2018 in the context of the extension of INEA’s mandate to include the

delegation of the Wifi4EU programme under CEF Telecoms.

Wherever possible, the analysis of the INEA performance during the reference period assesses

the progress achieved since the two previous evaluations (of the precedent TEN-T EA),

covering the Agency’s operations

during 2008-2011 and 2011-2013.

It should be noted that over the period 2008-2011 the savings resulting from the delegation of

tasks to TEN-T EA have been evaluated at around EUR 8.6 million compared to the in-house

scenario. The estimated efficiency gains over the period 2011-2013 were of EUR 8.8 million

and of EUR 24.4 million over the period 2014-2016.

According to the SFS of INEA prepared in 2013 the estimated efficiency gains resulting from

programme delegation to INEA were of EUR 22.8 million over the period 2014-2016. The

total number of full time equivalents (FTEs) required to manage the relevant programmes in

year 2020, the peak programming year in terms of workload, was estimated at 318 for INEA.

The total initial operational budget entrusted to INEA in 2020 was estimated at around EUR

37 billion in both commitment and payment appropriations. Considerable efficiency gains

were expected as compared to the in-house scenario over the period, along with non-

quantifiable benefits such as improved quality of programme management and service

delivery, improved visibility of the EU programmes and proximity to beneficiaries.

MPLEMENTATION

TATE OF PLAY

In 2006, by Commission Decision 2007/60/EC

, the Commission established the Trans-

European Transport Network Executive Agency (TEN-T EA) and entrusted it with the

management of Community actions in the field of the trans-European transport network. The

initial duration of the Agency was limited to 31 December 2008. Then the main task of the

Agency was to manage the remaining open TEN-T projects and financing decisions under the

Multi-annual Financial Framework (MFF) 2000

–

2006.

Subsequently, in 2008, by Decision 2008/593/EC

, the Commission extended the period of

operation of the TEN-T EA to 31 December 2015 and redefined its objectives and tasks so

that it became responsible also for the implementation of financial aid from the trans-

European transport network budget under the MFF 2007-2013. Both the establishment and

extension of the TEN-T EA were preceded by dedicated cost-benefit analyses.

TEN-T EA managed a TEN-T programme budget of EUR 0.8 billion during the 2000-2006

MFF and EUR 7.8 billion during the 2007-2013 programming period. In the latter period the

Agency’s project portfolio consisted of 411

projects and by end 2013 its staff was around 100.

The Innovation and Networks Executive Agency (INEA) was set up on 23 December 2013

and is an autonomous legal entity since 1 January 2014. It is responsible for the management

of parts of the Connecting Europe Facility (CEF) and of Horizon 2020 as well as of the TEN-

T and the Marco Polo legacy programmes.

Commission Decision of 26 October 2006 establishing the Tran-European Transport Network Executive

Agency pursuant to Council Regulation (EC) No 58/2003 (OJ L 32, 6.2.2007, p.88).

Commission Decision of 11 July 2008 amending Decision No 2007/60/EC as regards the modification of the

tasks and the period of operation of the Trans-European Transport Network Executive Agency (OJ L 190,

18.7.2008, p. 35).

The new mandate also brought changes in the governance of the Agency as three new parent

DGs joined the Steering Committee. As stipulated in the Act of Delegation, a Memorandum

of Understanding was signed on 1 October 2014, which defines the modalities and procedures

of interaction between INEA and its parent DGs

. In addition, it brought a clearer

delimitation of the administrative activities between the Agency and its parent DGs. The

present evaluation assesses whether this distribution of roles has been complied with for the

2014-2016 period, as well as the effects of the intervention on the coherence and the

efficiency of

INEA’s

action.

The total operational budget earmarked for INEA for the MFF 2014-2020 is EUR 33.8

billion, of which EUR 28.5 billion is accounted for by the CEF and EUR 5.3 billion by

Horizon 2020. This means that INEA handles the largest budget of all the executive agencies

in the current financial period of 2014-2020.

The operational budget managed by INEA in 2014-2016 was approximately EUR 11.9 billion

in terms of commitments (EUR 3.6 billion in 2014, EUR 3.2 billion in 2015 and EUR 5.1

billion in 2016) and EUR 5.1 billion in terms of payments (EUR 0.6 billion in 2014, EUR 2

billion in 2015 and EUR 2.5 billion in 2016). The Agency’s administrative budget evolved

from EUR 13.1 million in 2014 to EUR 18.2 million in 2015 and EUR 21.5 million in 2016.

The total budget per head in terms of payments increased from EUR 3.97 million in 2014 to

10.75 million in 2015 and to EUR 11.11 million in 2016. The increasing volume of INEA’s

operational budget during this period outpaced the increase in the number of the Agency’s

staff.

The number of proposals received and evaluated in 2014-2016 was in total 3 490, i.e.

ery

close to the CBA estimations. The number of proposals managed per head varied between 3.1

in 2014, 8.3 in 2015 and 6.5 in 2016, which results mainly from the heavy frontloading of the

CEF Transport budget (2014 and 2015 calls, evaluated respectively in 2015 and 2016).

The number of INEA running projects increased from 496 in 2014 to 1026 in 2015 and 1383

in 2016. While the number of projects in 2014 was below the CBA estimations (which related

to the later than initially forecasted launch of the 2014 calls and later conclusion of grant

agreements), in 2015-2016 the actual number of running projects generally reached the CBA

estimations. Programme management costs (defined as the ratio between the administrative

and operational budget in payments) decreased from roughly 2% in 2014 to 0.9% in 2015 and

0.9% in 2016.

A series of organisational and procedural changes were introduced to render more efficient

the implementation of the delegated programmes under the MFF 2014-2020, notably in the

evaluation period:

Memorandum of Understanding between the Directorate-General for Mobility and Transport (DG MOVE), the

Directorate-General for Energy (DG ENER), the Directorate-General for Communication Networks, Content and

Technology (DG CNECT) and the Innovation and Networks Executive Agency (INEA) of 1 October 2014,

subsequently amended. The MoU is formed by four documents: Part I

–

General Provisions, Part IIa

–

Connecting Europe Facility, Part IIb

–

Horizon 2020, and Annex

–

Administrative and logistical support services

provided by REA. The General Provisions and Horizon 2020 parts of the MoU were updated in December 2015

to take into account the lessons-learned concerning programme implementation as well as administrative

developments within the Commission. The CEF part was updated in July 2018 notably also to take into account

the delegation of the Wifi4EU programme under CEF Telecoms to INEA on 1 May 2018.



Further simplification and improvement of grant management processes and

procedures, notably concerning the widespread use of the TEN-Tec portal

functionalities concerning submission and evaluation of proposals, preparation and

signature of Grant Agreements and reporting (in 2014, the Agency merged the

submission of the Strategic Action Plan (SAP) with the Action Status Report (ASR)

–

previously two separate documents);



The adoption of new business processes, Horizon 2020 IT tools and modes of service

delivery;



The adoption in April 2017 of a multi-annual human resource (HR) strategy, covering

the years 2016

–

2020, including an action plan for following up on the key findings

from the 2016 staff opinion survey. Based on this strategy, the Agency is

implementing specific measures related to staff engagement, training, internal

communication, career development opportunities, well-being and the balance

between work and private life, aimed at improving HR management and increasing the

effectiveness of the Agency’s operations. A staff retention policy has since been

approved in September 2018 to increase staff satisfaction levels and help reducing

staff turnover.

ETHOD

4.1 Short description of methodology

The evaluation was supported by a study carried out by an external contractor

The

methodology of the INEA evaluation was consistent with the approaches employed for

similar parallel evaluations of EACEA, EASME, ERCEA, CHAFEA and REA. The study

was structured around a series of evaluation questions available in Annex.

A combination of evaluation methods providing for the collection of qualitative and

quantitative information and evidence included:



Documentary review and desk research

concerning all relevant documentation

relating to INEA, including the legal basis of the executive agencies and of INEA, the

Memorandum of Understanding between the parent DGs and INEA (MoU), Annual

Work Programmes, Annual Activity Reports, European Court of Auditors (ECA) and

Internal Audit Service (IAS) reports and related documents, previous evaluations and

CBA studies, financial documents, and action plans; the evaluation relied also on the

results of the 2016 staff opinion survey;



Semi-structured or in-depth interviews

with EU staff and stakeholders of the

delegated programmes. 59 interviews were carried out with: staff of the parent DGs, of

other DGs represented in the INEA Steering Committee, and of INEA; project

applicants and beneficiaries; external experts contracted to carry out the external phase

of the CEF and Horizon 2020 evaluations;



Survey

addressed to stakeholders, experts and Member States’ officials involved in

the implementation of programmes managed by INEA. 1878 responses were received:

1404 from project applicants and beneficiaries; 382 from external experts; 79 from

programme committee members; and 13 from members of INEA’s CEF Transport

Advisory Group;

Study supporting the Evaluation of the Innovation and Networks Executive Agency (INEA) 2014-2016, Final

report 5 February 2019, performed by the Centre for Strategy & Evaluation Services (CSES), the Public Policy

and Management Institute (PPMI) and the Centre for Industrial Studies (CSIL).



A retrospective

cost–benefit analysis;



A comparative analysis and benchmarking of the various executive agencies was also

carried out against a set of (qualitative and quantitative) indicators. This enabled the

evaluation team to compare the performance of INEA effectively with that of other

executive agencies.

Case studies were used to complement the desk research and interview / survey programme

and to provide insights into the performance of the Agency on issues such as coherence and

maintenance of the know-how within the Commission, efficiency of simplification measures

etc. Those case studies have fed the different assessment questions.

4.2 Limitations and robustness of findings

A representative sample of Commission and INEA staff and stakeholders (both applicants of

rejected proposals and beneficiaries) and other officials / experts (external experts,

programme committee members and advisers) were consulted in the course of the evaluation.

Compared to the overall population reached, unsuccessful applicants participated less actively

in the survey than programme beneficiaries.

No sampling bias was observed as the profile of the respondents to the surveys was very

similar to the overall population, guaranteeing statistical representativeness. The non-response

bias (not all characteristics of the group that did not reply had been captured in full) was

mitigated through triangulation with the results of follow up interviews. The triangulation

approach, using multi-level and multi-stakeholder dimension in the data collection, ensured

the robustness and reliability of the data and information used to draw up conclusions in the

supporting study.

A mix of parent DGs and INEA managers and policy / project officers were interviewed

extensively. Some answers to evaluation questions, notably in the ‘Coherence’ section, rely

largely on interview and survey data. Some views from Commission and Agency staff

gathered from the interviews and mentioned in the study illustrate personal experience,

although they may not be representative of overall relations between the Commission and the

Agency.

Also, it should be noted that, in parallel to this evaluation, the European Court of Auditors

(ECA) carried out a performance audit to ascertain if INEA’s management of the EU

delegated programmes optimise their implementation. The audit focused on the 2014-2020

programming period and covered INEA’s set-up

and organisation, and its implementation of

the CEF programme. It involved inter alia the examination of the regulatory framework, the

responsibilities and the control systems in place as well as INEA’s

performance in the

management of the CEF programme and projects. In its report, the ECA inferred that INEA is

a well-organised Agency and that it is implementing the programmes delegated to it, in

particular the CEF, efficiently. However, the report also made a number of recommendations

to enhance the implementation of the Connecting Europe Facility and the overall operational

performance of the Agency, such as: a multi-annual programming of calls for proposals;

increased synergies between CEF and Horizon 2020 (Horizon Europe); a performance

framework to measure programme and project achievements; an upgraded and harmonised

procedure for the evaluation and selection of project proposals; to enhance the link between

funding and the achievement of project milestones; and to ensure more flexibility in the

allocation of staff among programmes within the Agency. The Commission takes the view

that the European Parliament and Council common understanding on the new CEF Regulation

is consistent with the ECA’s recommendations.

The Commission and INEA are already

addressing the ECA’s recommendations with a view to improving programme management

under the present MFF and in view of the CEF 2 programme.

NALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS

5.1 Effectiveness

For the purpose of this evaluation, effectiveness relates to how successful the Agency has

been in achieving or progressing towards its objectives.

The evaluation inferred that during 2014-2016 INEA operated according to the legal

framework that

established it. INEA’s activities, as planned in the annual work programmes

and reported in the annual activity reports, corresponded to the tasks set out in the

Commission’s establishment and delegation Decisions.

As provided in the delegation Decision, the modalities and procedures of interaction between

the Agency and its parent DGs are set out in the Memorandum of Understanding (MoU) of 1

October 2014. The MoU provides for a supervision strategy aimed at avoiding gaps or

duplication of efforts resulting from crossover between the policy-making, monitoring and

supervision tasks of the parent DGs and the execution tasks of the Agency. These provisions

have generally worked well and all parent DGs and INEA appreciate the effectiveness of the

collaboration.

In the evaluation period, INEA continuously improved its operations and achieved a high

level of overall effectiveness. The Agency was flexible and effective in addressing the key

challenges that occurred during the evaluation period, in particular:



the continuous phasing-out of its two legacy programmes TEN-T and Marco Polo (the

latter transferred from EACI, now EASME);



the evaluation and grant award process for the CEF Transport calls for proposals,

especially the first and largest ever CEF Transport call in 2014/2015 involving the

evaluation of almost 700 proposals for a final selection decision for funding of EUR

13.1 billion;



the gradual assumption of responsibilities on the implementation of the CEF Energy

and Telecoms programmes and the Horizon 2020 sub-programmes concerning

transport and energy;



the launch of a CEF synergies (transport and energy) call in 2016; and



the preparation as of 2016 of an innovative CEF Transport Blending call for EUR 1,35

billion, launched in 2017.

INEA has developed strong internal control standards, including on the management of

financial and human resources. The Agency established overarching control strategy with a

large number of control and reporting mechanisms that allow progress against objectives to be

closely monitored and risks to its operations to be prevented and mitigated in a timely and

effective way. INEA has in particular established a coherent anti-fraud strategy which it

pursues effectively. This is reflected in assessments undertaken by the Court of Auditors and

by the Commission’s Internal Audit Service.

In 2014-2016 INEA achieved almost all its targets in terms of key performance indicators

(KPI). Only the target for the net time-to-pay for the TEN-T programme 2007-2013 was

slightly exceeded in 2015 by 1 day (31 days instead of 30). The prompt conclusion of grant

agreements (measured in ‘Time-To-Grant’ –

TTG) varied, for Horizon 2020, between a

minimum of 216 days in 2015 and a maximum of 225 days in 2014 for Energy, and between

232 days in 2015 and 237 days in 2014 for Transport (still below the Horizon 2020 target of

245 days). Concerning the CEF programme, the longest sectorial TTG in 2015-2016 (no

grants were awarded and signed in 2014) were 264 days for CEF Transport, 249 days for CEF

Energy and 253 days for CEF Telecoms, all below the ceiling of 276 days. The share of grants

concluded within TTG targets, for both Horizon 2020 and CEF, reached nearly 100% for the

2014-2016 calls.

Regarding payments of grants, the average Time-to-Pay (TTP) stood well below the

contractual thresholds for all types of payments (pre-financing, interim and final payments) in

2014-2016 both for Horizon 2020 and CEF. Concerning the share of payments within

contractual limits, nearly 100% of all pre-financing, interim and final payments in 2014-2016

were executed on time.

INEA achieved full execution of its operational budget during 2014-2016, both in

commitment and payment appropriations; with the exception of the execution of commitment

appropriations for the TEN-T

and Marco Polo II programmes in 2014, both of which were

well below the target of 100% , although this target was subsequently met in 2015 and 2016.

With regard to the legality and regularity of transactions, the estimated residual multi-annual

error rate remained well below the threshold of 2% for the TEN-T and Marco Polo II

programmes in 2014-2016. Concerning CEF and Horizon 2020, this KPI was not yet

applicable and there were not enough audits in this period to draw any conclusions; however,

it is estimated that, by the end of the programme implementation, the error rate will be of 2%

cumulative for CEF while for Horizon 2020 it is established within the range of 2% - 5%.

For TEN-T, this was largely due to a lower than expected response to the 2013 Call for proposals.

Summary of the KPIs over the period 2014-16

2014

KPI’s

(1) Rate of execution of

commitment appropriations

(2) Rate of execution of

payment appropriations

(3) Time to grant (days)

(4) Net time to pay (Pre-

financing) (days)

(5) Error rate of payments at

ex-post control

Targets

Ten-T

Marco

Polo

CEF

Horizon

2020

Ten-T

2015

Marco

Polo

CEF

Horizon

2020

Ten-T

2016

Marco

Polo

CEF

Horizon

2020

100%

85%

60%

100%

N/A

100%

N/A

100%

N/A

100%

276

274

16.9

271

26.6

1.1 -

N/A

231

N/A

26.1

242

9.5

224

N/A

254

228

11.6

< 2%

0.80%

1.8%

N/A

0.78%

0.97%

N/A

0.80%

0.37%

N/A

Source: INEA activity reports 2014, 2015, 2016

Despite some shortcomings in call coordination between the parent DGs concerning the

evaluation period, INEA adapted well to peak periods of very high work-load. Such heavy

work-load was especially felt in 2015/2016 on the occasion of the 2014 and 2015 CEF

Transport calls for proposals, when the Agency was not yet fully staffed. Regular meetings

between the parent DGs and INEA resulted overtime in more efficient planning and

coordination of the calls, though continuous efforts on call coordination are necessary in the

run up to the last calls of the present MFF.

While providing recommendations for addressing minor shortcomings, the reports of the

independent observers accompanying the evaluations, for both CEF and Horizon 2020, were

consistently complimentary on the transparency and efficiency of the evaluation process.

Concerning Horizon 2020, INEA’s efforts to improve the quality of the evaluation process

were reflected in a very small share of evaluation review/redress cases filed and (fully or

partially) upheld compared to the number of proposals evaluated

The survey of INEA’s applicants and beneficiaries indicates satisfaction in their dealings with

the Agency, with a great majority of them considering that INEA has contributed to a

significant improvement of the management of the delegated programmes and helped to

develop better services. 90% of respondents stated that INEA managed the procedures

(applications, evaluation, contract negotiations, project monitoring, etc.) in relation to their

project very or quite efficiently. Only 8% of respondents indicated that during the application

process INEA was unresponsive to the queries or problems applicants raised and only 9%

indicated that the decision on the application was not taken within the acceptable timeframe.

It should be noted however, that applicants’ recommendations included a number of areas

where improvement may be necessary, for example:



establishing a “hotline” for National Contact Points/Committee members to obtain

instant information and advice on application procedures;



improving the feedback to applicants on the rejected proposals;



organising local information meetings on calls for proposals and other implementation

matters and making them accessible online;



more flexibility in reporting on project progress, both regarding how to submit and

save documentation (paper vs digital) and more use of electronic signatures;

In 2015 the redress case rate was 2.4%: 749 proposals evaluated; 18 redress cases submitted; 0 re-evaluations

(all evaluation results were confirmed). In 2016 the redress case rate was 2.6% and the re-evaluation case rate

was 0.14%: 734 proposals evaluated; 19 cases submitted; 1 re-evaluation done.



improving the participant portal and more workshops to update on progress of the

programmes

; and



guidance on how to better manage project risks.

Overall, external experts showed high levels of satisfaction with INEA’s selection and

contracting process, whether they worked on Horizon 2020, on CEF, or both. 58.8% of all

experts were very satisfied and an additional 32.4% were quite satisfied. Overall less than

10% were quite or very dissatisfied, or had no response. Experts working on Horizon 2020

alone had the highest level of satisfaction, with only 4.1% dissatisfied in any way (an

additional 4.5% did not respond or were unable to comment). By contrast, 9.4% of those

working on both CEF and Horizon 2020 were dissatisfied, as were 11.8% of those working on

CEF alone. Experts working on both projects were also least likely to be very satisfied

(46.9%) compared to those working on Horizon 2020 (61.5%) or CEF (60.3%) alone.

Experts overwhelmingly stated that evaluation / review consensus meetings are run efficiently

by INEA. While only 4.5% of the experts who replied to the survey said they were

dissatisfied, the proportion of experts who were very satisfied was 51.6%. Concerning

perceptions on the quality of evaluation/review activities, just 2% felt that they were low

quality, with 56.3% believing they were of high quality across the board.

The extension of the Agency’s mandate and change of name (from TEN-T

EA to INEA) in

2014 did not have an impact in respect of proximity to stakeholders and the visibility of the

EU as a promoter of the programmes delegated to it. In terms of proximity to the addressees,

INEA acted as a direct contact point between programme applicants and beneficiaries of EU

funding, as well as other stakeholders, and the Commission. The most regular contact takes

place between project beneficiaries and the respective project officers at INEA. Separation of

policy making and implementation benefits the beneficiaries insofar as they obtain specialised

advice and service. During the evaluation period, INEA made sustained efforts to boost

awareness of new funding opportunities for project promoters (both under CEF and Horizon

2020) and to consolidate a service-oriented communication. It also supported parent DGs by

giving visibility to the delegated programmes via success stories.

Concerning visibility of the EU as a promoter of the programmes entrusted to INEA, the

survey suggests that there is awareness amongst stakeholders that the Agency is acting under

powers delegated by the Commission. The Agency’s contributions to enhanced visibility of

the programmes included Info-days, project management workshops, pro-active use of social

media tools and good quality graphic and communication materials. Additionally, the website

of the Agency includes a significant amount of material related to the delegated programmes:

project information, brochures, maps and statistics. The many channels of communication

used by the Agency have resulted in the enhanced visibility of the programmes.

5.2 Efficiency

This section considers the relationship between the resources used by the Agency and the

output. The analysis, among other factors, also includes analysis of administrative and

regulatory burden and looks at aspects of simplification.

Applicants were also suggesting developing an online platform where INEA could showcase important

developments across different programme areas. INEA’s website is already a dissemination platform for the

managed portfolio of projects.

During 2014-2016 INEA proved to be an efficient and cost-effective structure for the

management of the delegated programmes. Its administrative budget decreased from 2% of

the operational budget based on payment appropriations in 2014 to 0.9% in 2016.

INEA’s actual operational budget during 2014-2016

(in commitment appropriations)

constituted EUR 11,788 billion, of which EUR 11,679 billion from EU budget and EUR

109,2 million related

to EEA/EFTA and third countries’ contributions (not considered in

INEA’s SFS). INEA’s administrative budget (including such contributions) increased from

EUR 13.1 million in 2014 to EUR 18.2 million in 2015 and EUR 21.5 million in 2016.

A detailed analysis of the INEA cost-efficiency and results of the Cost-benefit analysis is

presented under point 5.4.

Programme implementation cost (ratio between the administrative and operational

budget, percentage), payment appropriations

10,00

8,00

6,00

5,27

4,00

2,05

2,00

0,00

2014

INEA

2015

Average of the EAs

2016

0,89

4,72

7,97

Source: CSES, PPMI and CSIL, based on INEA Annual Activity Reports.

The number of proposals managed by INEA in 2014-2016 was 3490, very close to the CBA

forecast of 3576. The number of total running projects increased from 496 in 2014 to 1383 in

2016. During the evaluation period, the actual workload of INEA was higher than estimated

in the 2013 CBA due to factors beyond the Agency’s control that influenced its workload,

such as the average grant size, the number of proposals and grants, and the re-injection of

budget amounts. The

number of running projects ‘per operational head’ increased from 4.4 in

2014 to 7.6 in 2016 as a result of the first Calls of the 2014-2020 MFF. The ratio of budget

‘per operational head’ did not fluctuate much in terms of commitments (from EUR 31.8

million to EUR 27.8 million) but increased significantly in terms of payments (from EUR 5.4

million to EUR 13.5 million) in 2014-2016.

INEA Budget, staff and workload indicators, 2014-2016

(in € million, n° or %)

2014

2015

2016

Operational budget, commitments

Operational budget, payments

Administrative budget, commitments

Administrative budget, payments

Actual number of staff (at the end of the year)

Actual number of operational staff

Programme management cost* (commitments)

Programme management cost (payments)

3590

606

13.1

12.4

151

113

0.4%

2.0%

3251

2021

18.2

17.9

186

145

0.6%

0.9%

5057

2448

21.5

21.9

225

182

0.4%

0.9%

Budget ‘per operational head’ (commitments)

Budget ‘per operational head’

(payments)

Proposals received

Total running projects

Running projects ‘per operational head’

* Defined as the ratio between the administrative and operational budget.

31.8

5.4

479

496

4.4

22.4

13.9

1548

1026

7.1

27.8

13.5

1463

1383

7.6

Human Resource Management

In order to optimise allocation of human resources across its delegated Programmes, INEA

has developed a methodology to calculate its operational workload and staffing needs which

was approved by its Steering Committee in February 2018 and which has been kept

continuously updated. The methodology is based on three main workload drivers: proposals to

be evaluated, grant agreements to be signed, ongoing projects to be managed. While current

budgetary rules do not allow shifting Agency staff between Programmes, the work-load

assessment methodology proved useful to allocate staff more efficiently within Programme

sectors, in agreement with the parent DGs.

INEA had to cope with an increase of its staff turnover rate from 7.5% in 2015 to close to

10% in 2016 (the target level being 3%). INEA has actively addressed the underlying issues

and has elaborated a new staff retention policy with elements such as encouraging internal

staff mobility, announcing vacant positions internally and organising job shadowing. As a

consequence, INEA is now being more effective in retaining staff and filling vacancies, and at

the end of 2018 the vacancy rate had dropped to 5.4%.

The staff opinion survey carried out in 2016 revealed an increase in INEA’s staff satisfaction

compared to 2014. The staff engagement index is a composite indicator

that helps measure

how connected the staff is to the Agency and how committed to help it achieve its goals. In

2016, INEA had a staff engagement index result of 68%, above average of the other executive

agencies (66%) and the Commission (64%). With an improvement of 5% from 2014, it

clearly shows that good progress has been done in this area. The overall INEA staff job

satisfaction in 2016 was within the average of the Commission departments (71% in INEA

compared to 70% in the Commission) and the other executive agencies.

Several other indicators are evidences of the good results achieved by INEA in terms of HR

management. 62% of INEA staff felt the Agency was a modern and attractive workplace, 6%

higher than the Commission average (56%). 95% of employees at INEA said they had a clear

understanding of the Agency’s purpose in 2016, a 9% increase compared with 2014. Other

examples are: the managers’ approach to the staff (staff has a much better perception

management, especially middle management, as compared to the 2014 survey, while it is

important to note that a new management structure was just evolving in 2014) and training

and opportunities for staff to advance their professional development.

•

The staff engagement index is composed of the following seven individual questions:

I have the appropriate and timely information to do my work well;

My colleagues are committed to doing quality work;

I have a clear understanding of what is expected from me at work;

I have recently received recognition or praise for good work;

I feel that my opinion is valued;

My manager seems to care about me as a person;

My line manager helps me to identify my training and development needs.

At the same time, INEA staff perceived less positively the offer of training and development

activities and the opportunities for their mobility and career development.

To follow up these staff survey results, as mentioned under section 3, the Agency adopted in

April 2017 a comprehensive multi-annual HR strategy followed by a staff retention action

plan in September 2018. The evolution of key indicators for staff satisfaction – after the

evaluated period – is encouraging.

In the latest staff survey, which was carried out in the 4

quarter of 2018 (thus outside of

the evaluation period), INEA managed to increase for the third consecutive time the staff

engagement index

from 61% in 2014 to 68% in 2016 and 70% in 2018. Also, the overall

job satisfaction, even if it is remains lower than the Commission average (77%), has also

increased to 75%. The results of this new staff opinion survey also show points for

attention: decreasing staff satisfaction with workload levels, work recognition and

performance management as well as lower satisfaction rates concerning the Agency’s

middle management. These matters will have to be addressed by the Agency together with

its parent DGs in the most appropriate way and will feed into a new follow-up action plan.

Overall, the size and structure of INEA, and the level of resources in each of its departments,

were appropriate to its mandate and delegated tasks during the evaluation period. The

organisational structure of INEA was, as of the start of its mandate in 2014, based on separate

departments for each of the delegated programmes

–

CEF (Department C) and Horizon 2020

(Department H)

–

as well as for Programme support and resources (Department R). Compared

with the previous TEN-T EA, a new hierarchical level was therefore introduced in the form of

heads of department function, which led to the need for staff to adapt to new internal reporting

and approval circuits. Department C has 3 units for CEF Transport, reflecting the substantial

respective grant budget delegated to INEA, split by funding priority and geographical

coverage, as well as 1 unit each for CEF energy and CEF telecoms (formerly one single unit).

Department H has 2 units respectively for transport and energy research. Department R

comprises 4 units responsible for the Agency’s horizontal tasks

such as planning and

coordination of the calls and communication, administration and finance, HR and IT and

logistics, as well as the internal control function. INEA’s organisational structure was

reviewed and effectively aligned to cope with changes, such as the evolution of its mandate,

notably with the creation of a dedicated unit for CEF Telecoms on the occasion of the

delegation of the WiFi4EU programme to INEA.

During 2014-2016, INEA, in cooperation with the Commission, continued the optimisation of

its procedures and programme management functions and introduced a number of

simplifications. Concerning Horizon 2020, the improvements and simplifications concerned

wider use of remote evaluation of proposals, the introduction of a single set of rules,

electronic signature of grant agreements, the Participant Portal as the one-stop-shop for

interactions with participants, single reimbursement rate, flat rate for indirect costs, etc. With

regard to CEF, these concern the streamlining of submission of progress reports by the

beneficiaries (Action Status Reports), e-submission and evaluation modules in the TEN-tec

portal, harmonisation of Grant Agreements etc. Following its successful testing end 2015,

Speedwell has become as of 1st January 2016, the standard tool to process administrative

payment paperless.

In July 2014 the Agency revised its Manual of Procedures. Furthermore, an exercise for

streamlining and updating the internal procedures and guidelines for the new programmes was

initiated. For document management, the use of paperless workflows was extended. A

specific Agency Task Force was created which led to a concrete action plan to increase

paperless procedures.

5.3 Coherence

Coherence looks at any overlaps and complementarities within the programme portfolio

managed by the Agency or delimitation of responsibilities between the Agency and its parent

DGs.

In the 2014-2016 period, INEA demonstrated flexibility in implementing a diverse thematic

programming portfolio and being answerable to four different parent DGs. The evaluation did

not find any evidence of overlaps, duplications, gaps or inconsistencies within the INEA

programme portfolio. The Memorandum of Understanding as last amended in July 2018

provides for a clearer and appropriate delimitation of responsibilities and tasks between INEA

and its parent DGs. However, due consideration should be given to the fact that supervision

tasks (e.g. financial, auditing, planning, reporting, coordination etc) have increased

significantly as of the start of the present MFF. Hence, the necessary supervision capacity in

the parent DGs (with benchmarking among DGs) needs to be ensured.

Since INEA was established around the concept of network industries and related research

and innovation, there is a strong coherence between the CEF and Horizon 2020 programmes

delegated to INEA notably in respect of the energy and transport sectors. Concerning the

transport sector in particular, the Commission benefitted from the continuity of support to

TEN-T infrastructure actions within the same Agency since 2006, as well as the taking over

by INEA of the Marco Polo programme from EACI (now EASME).

The coherence in the programmes managed by INEA also helped to exploit synergies to the

extent possible. The mid-term evaluation of the CEF programme

noted that, on the one

hand, CEF synergies have been effective at programme level, notably through common

implementation by INEA with the same business processes, 3-sector coordination, common

CEF coordination committee of Member States, and common Work Programmes for CEF

Financial Instruments, all of which generate economies of scale and simplification. On the

other hand, the legal basis of the CEF programme is not sufficiently flexible to allow funding

of large-scale infrastructure projects combining 2 or 3 sectors; hence the limited success of

the CEF synergies (Transport and Energy) call for proposals of 2016. The Common

Understanding on CEF II provides for facilitation of synergies within the programme which

would benefit from a coherent set-up of the programmes managed by INEA.

With regard to synergies between Horizon 2020 and CEF, despite some intrinsic difficulties

linked with the legal bases of both programmes, different business processes and distinct type

of support projects (research vs deployment and / or infrastructure building), INEA and the

parent DGs endeavoured to take initiatives in this field. INEA launched a reflection exercise

concerning the transport sector which led to a report being presented to the Steering

Committee in 2017. Several actions were taken at project level including technical meetings

with the parent DGs, TRIMIS collaboration, and a synergies webpage in INEA website. As a

result of this exercise INEA Horizon 2020 projects had a greater presence at the TEN-T days

COM(2018) 66 final of 14.2.2018, Report from the Commission to the European Parliament, the Council, the

European Economic and Social Committee and the Committee of the Regions on the mid-term evaluation of the

Connecting Europe Facility (CEF).

organised by DG MOVE (as of the 2017 edition). In addition, there is a higher degree of

coordination between Horizon 2020 and CEF Work Programmes. Several proposals applying

for CEF Transport funding have referred to results from Horizon 2020 projects. A similar

exercise is currently being carried out in the Energy sector.

The evaluation however stressed that INEA would gain from a closer alignment of

procedures, as appropriate, between the infrastructure programme (CEF) and the research

programme (Horizon Europe) in terms of call launching, submission of proposals, evaluation

and selection process and grant awarding for the next MFF. Work is being developed in this

respect in the context of the corporate model grant agreement and e-grants process.

INEA has a very good working relationship with parent DGs which are in general satisfied

with the levels of formal and informal communication taking place in a regular and structured

manner. The fact that some of the Agency’s personnel, mainly at management level, is either

seconded from one of the parent DGs (in 2018, 19 of 277 staff) or has previously worked

therein is an important asset in this respect.

Despite the inherent difficulties linked with reporting on programme implementation (grant

funding) on the variety of delegated programmes and sub-programmes, INEA has been

proactive and constructive in providing policy feedback to parent DGs. This is the case

notably on lessons-learned from project management and project funding. However, the

evaluation found that, as the projects supported at the start of the programmes are unfolding in

the second half of the present MFF, INEA could use its knowledge of programme

implementation to further help Commission policy-making notably on the preparation of the

Work Programmes. INEA has also been proactive at proposing a framework for the provision

of KPIs on CEF implementation, as well as preparing Country and Corridor reports (notably

on CEF Transport) and brochures on achievements and successful stories. While

improvements were found necessary on the provision of real-time information on programme

and project progress, this is being addressed and improved constantly through the close

cooperation between the parent DGs and INEA.

5.4 Retrospective Cost-Benefit Analysis

The retrospective Cost–Benefit Analysis for the period 2014-2016 was carried out based on

the results of the 2013 ex-ante CBA, the assumptions laid down in the Specific Financial

Statement (SFS) and the actual costs of INEA.

The 2013 ex-ante CBA estimated that the delegation of tasks to INEA would result in savings

of around EUR 54 million over the 2014-2020 Multi-Annual Financial Framework when

compared to the management by the Commission services (in-house scenario). For the period

2014-2016 such savings were estimated at around EUR 16.1 million. The savings estimate in

the INEA SFS for the period 2014-2016 was in the order of EUR 22.8 million

The retrospective CBA revealed that the actual costs of the executive agency approach in

2014-2016 were much lower than the costs of the in-house scenario estimated both in the ex-

ante CBA and in the SFS. The actual cost savings constituted EUR 24.4 million (or 29% of

the estimated costs under the in-house scenario)

The retrospective CBA is based on the SFS version of November 2017

–