kom (2019) 0133 - Ingen titel

Europaudvalget 2019
KOM (2019) 0133
Offentligt

EUROPEAN

COMMISSION

Brussels, 12.3.2019

SWD(2019) 97 final

COMMISSION STAFF WORKING DOCUMENT

on the Evaluation of the operation of Council Regulation (EU) 2016/369 on the provision

of emergency support within the Union

Accompanying the document

Report from the Commission to the Council on the Evaluation of the Regulation (EU)

2016/369 on the provision of emergency support within the Union

{COM(2019) 133 final}

kom (2019) 0133 - Ingen titel

Table of contents

INTRODUCTION ................................................................................................................................ 5

BACKGROUND TO THE INTERVENTION ..................................................................................... 6

IMPLEMENTATION / STATE OF PLAY ........................................................................................ 16

METHOD............................................................................................................................................ 23

ANALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS .......................................... 25

5.1.

5.2.

5.4.

5.5.

5.6.

Relevance ...................................................................................................................... 25

Coherence and complementarity ................................................................................... 31

Efficiency ...................................................................................................................... 41

EU added value ............................................................................................................. 45

Sustainability ................................................................................................................. 46

CONCLUSIONS ................................................................................................................................. 47

6.1.

6.2.

6.3.

6.4.

6.5.

6.6.

6.7.

Relevance ...................................................................................................................... 48

Coherence ...................................................................................................................... 48

Effectiveness ................................................................................................................. 49

Efficiency ...................................................................................................................... 50

EU added value ............................................................................................................. 51

Sustainability ................................................................................................................. 51

Lessons learned from the evaluation ............................................................................. 52

ANNEX 1: PROCEDURAL INFORMATION ............................................................................................ 55

ANNEX 2: INTERVENTION LOGIC OF THE ESI ACTIONS ................................................................. 60

ANNEX 3: STAKEHOLDER CONSULTATION....................................................................................... 61

ANNEX 4: LIST OF EVALUATION QUESTIONS ................................................................................... 64

ANNEX 5: RESULTS OF THE OPEN PUBLIC CONSULTATION ......................................................... 66

ANNEX 6: OVERVIEW OF ESI FUNDED ACTIONS PER PARTNER AND AREAS OF

INTERVENTION ......................................................................................................................................... 68

ANNEX 7: METHODS AND ANALYTICAL MODELS .......................................................................... 69

kom (2019) 0133 - Ingen titel

Glossary

Term or acronym

AMIF

ASB

Meaning or definition

Asylum, Migration and Integration Fund

Arbeiter-Samariter-Bund - The Workers' Samaritan

Federation

Directorate-General for Civil Protection and Humanitarian

Aid Operations

Directorate-General for Migration and Home Affairs

Directorate-General for Regional and Urban Policy

The Danish Refugee Council

Emergency Assistance Grant Scheme

The European Regional Development Fund

The European Social Fund

Emergency Support Instrument

Emergency Support Operational Priorities

Support to Integration & Accommodation

The European Union Solidarity Fund

Financial and Administrative Framework Agreement

Final consumption expenditure

The Fund for European Aid to the Most Deprived

Framework Partnership Agreement

The International Federation of Red Cross and Red

Crescent Societies

The International Rescue Committee

International Organisation

International Organization for Migration

Internal Security Fund

Ministry of Migration Policy

Multipurpose Cash Transfer

DG ECHO

DG HOME

DG REGIO

DRC

EMAS

ERDF

ESF

ESI

ESOP

ESTIA

EUSF

FAFA

FCE

FEAD

FPA

IFRC

IRC

IOM

ISF

MoMP

MPCT

kom (2019) 0133 - Ingen titel

MSF

NFI

NGOs

NRC

SRSS

STC

TDH

UAMs

UCPM

UNHCR

VAT

Médecins Sans Frontières

Non-Food Items

Non-governmental organisations

Norwegian Refugee Council

Structural Support Service

Save the Children

Terre des Hommes

Unaccompanied Minors

The Union Civil Protection Mechanism

United Nations High Commissioner for Refugees

Value-Added Taxes

kom (2019) 0133 - Ingen titel

1. I

NTRODUCTION

1.1.

Purpose of the evaluation

This Staff Working Document presents the results of the evaluation of the operation of

Council Regulation (EU) 2016/369 on the provision of emergency support within the

Union

(hereafter 'the Regulation'), implemented

by the European Commission’s

Directorate-General for European Civil Protection and Humanitarian Aid Operations

(DG ECHO). The Regulation was adopted and simultaneously activated on 16 March

2016 for a period of three years to provide a rapid response to the humanitarian needs

arising from the influx of refugees and migrants into the Union. While the Regulation

does not establish it explicitly,

de facto

it creates a financial instrument, referred to in this

document as ‘Emergency Support Instrument’ (ESI).

Article 8(1) of the Regulation, provides that a report shall be presented by the

Commission to the Council twelve months after the activation of the ESI. A report

fulfilling this requirement was submitted to the Council on 15 March 2017

Furthermore, in accordance with Article 8(2) of the Regulation, the Commission is

required to submit

“an evaluation of the operation of the Regulation to the Council by

the end of the current activation in March 2019 and to put forward suggestions for the

future of the Regulation, and where appropriate, proposals to amend or terminate it”.

The evaluation has also been carried out in accordance with Article 30(4) of the Financial

Regulation

The purpose of the evaluation is therefore to assess the implementation and performance

of the actions funded by ESI and implemented in Greece, since 16 March 2016, and

hence to examine the extent to which the Regulation is fit-for-purpose, based on the

experience gained from the implementation of such actions. The evaluation consequently

seeks to identify any gaps, as well as strengths and weaknesses, of the current legislative

framework and in the operational response.

The evaluation is mainly based on an external study

that was carried out between July

and October 2018, which builds on an internal evaluation exercise carried by the

Commission, and it provides the most important source of evidence for this document.

1.2.

Scope of the evaluation

As set out in the Evaluation Roadmap

, the scope of the evaluation covers the actions

implemented by ESI in Greece since its activation (from 16 March 2016 up to 15

OJ L 70, 16.3.2016, p. 1–6.

Report from the Commission to the Council on the implementation of Regulation (EU) 2016/369 on the

provision of emergency support within the Union, of 15 March 2017- COM(2017) 131 final

Which stipulates that all programmes or activities where the resources mobilised exceed EUR 5 million,

must be the subject of an interim and/or ex post evaluation in order to verify that they were consistent with

the objectives set. OJ L 298 of 26.10.2012.

Evaluation of the operation of Regulation (EU) 2016/369 on the provision of emergency support in the

Union (ICF Consulting Services Ltd):

https://ec.europa.eu/echo/funding-evaluations/evaluations/thematic-

evaluations_en

kom (2019) 0133 - Ingen titel

October 2018)

looking at the following criteria established in the Commission's Better

Regulation guidelines

: (i) relevance, (ii) effectiveness, (iii) efficiency, (iv) coherence, (v)

EU added value. Given the particular nature of the intervention under ESI, the evaluation

will also consider a further criterion, namely sustainability.

2. B

ACKGROUND TO THE INTERVENTION

2.1.

Description of the intervention and its objectives

Background: a humanitarian emergency within the EU

In 2015 and during the first months of 2016, close to 1.4 million persons in need of

international protection and migrants made their way to the EU via the Eastern and

Central Mediterranean route

. This movement has been considered the "largest

movement of displaced people through European borders since World War Two"

. This

flow of migrants and refugees affected the capacity of countries along the migratory

routes to cope with humanitarian and protection needs.

In 2015, 850 000 people crossed from Turkey to Greece through the Aegean and

Dodecanese sea, mainly Syrians (59%), Afghans (24%) and Iraqis (8%), but also from

other countries, such as Iran, Pakistan, North and Central Africa. These events unfolded

at a time when Greece was struggling with the effects of an unprecedented economic and

financial crisis.

In November 2015, the former Yugoslav Republic of Macedonia (FYROM) announced

that only Syrians, Iraqis and Afghans would be allowed to cross, creating tensions at the

border with migrants and refugees from other nationalities. Circumstances changed

drastically as of mid-February 2016 with the progressive establishement of border

restrictions along the Western Balkans route, leading to an effective closure of the route

at the beginning of March 2016.

As a result, more than 60 000 people became stranded in Greece, with around 8,500

people at Idomeni alone (a Greek village of about 150 inhabitants located close to the

north border with the Republic of North Macedonia ). The situation shifted from Greece

being a transit country to becoming a host country.

This put a huge strain on the already limited resources of the national authorities to

respond to this humanitarian emergency. For instance, Greece did not have a dedicated

department dealing with migration issues (i.e. the Ministry of Migration Policy (MoMP)

was established in October 2016)

and faced a limited coordination amongst the different

The published Roadmap of the evaluation can be consulted at:

https://ec.europa.eu/info/law/better-

regulation/initiatives/ares-2017-4710263_en

The better regulation guidelines set out the principles that the European Commission follows when

preparing new initiatives and proposals and when managing and evaluating existing legislation, and can be

consulted at

https://ec.europa.eu/info/law/law-making-process/planning-and-proposing-law/better-

regulation-why-and-how/better-regulation-guidelines-and-toolbox_en

Data provided by IOM, available at

http://migration.iom.int/europe/.

UNHCR Regional Refugee and Migrant Response Plan for Europe, 2016.

European parliament. 2017. International protection in Greece Background information for the LIBE

Committee delegation to Greece 22-25 May 2017. Available at:

https://www.ecoi.net/en/file/local/1401618/1226_1497249698_ipol-stu-2017-583145-en.pdf.

kom (2019) 0133 - Ingen titel

ministries with competences in that area.

The capacity to provide accommodation to the

newcomers proved to be insufficient, with several important gaps in the availability and

access to services for asylum seekers. Water and sanitation facilities were insufficient.

The humanitarian situation was exacerbated by limited access to legal aid and translation

services to asylum seekers, or of humanitarian and social protection to vulnerable groups,

such as women, children and unaccompanied minors.

Figure 1 and 2 below present a timeline, as developed in the support study, on the

different relevant stages that unfolded in Greece between 2015 and 2018. Overall, both

provide a summary of the sequence of events and situation in Greece, the available

resources at the time, as well as the policies and measures undertaken at EU level to help

the Greek authorities address the emergency.

OCHA. 2016.

Greece: “Europe’s lack of political will creating serious suffering for thousands of

migrants in Greece” –

UN rights expert. Available at:

https://www.ohchr.org/EN/NewsEvents/Pages/DisplayNews.aspx?NewsID=19976&LangID=E

kom (2019) 0133 - Ingen titel

Figure 1.

Timeline of events between 2015 and 2016

kom (2019) 0133 - Ingen titel

Figure 2.

Timeline of events between 2017 and 2018

This figure represents the timeline of events in 2017 and 2018, showing ESI’s intevention in relation to other EU instruments

and political events at

national and European levels.

kom (2019) 0133 - Ingen titel

The need for a new instrument to provide emergency support within the EU

The EU has well-established financial instruments in place to assist Member States in

responding to different types of internal challenges and to express solidarity with

disaster-stricken regions. These include for example: the Union Civil Protection

Mechanism (UCPM)

; the Asylum, Migration and Integration Fund (AMIF)

; the

Internal Security Fund (ISF)

; and the European Union Solidarity Fund (EUSF)

, the

European Social Fund (ESF)

, the European Regional Development Fund (ERDF)

, the

Fund for European Aid to the Most Deprived (FEAD)

and the Health Programme

18.

However, as most of these intruments rely on shared or indirect management

implementation modalities

, none of them is fully suitable to address wide-ranging

humanitarian needs resulting from disasters of exceptional scale and impact, which

consequently overwhelm the existing national capacities of Member States. Thus, at the

peak of the refugee crisis in Greece, there was no EU instrument available that allowed

for the direct implementation of operations through humanitarian partners.

Decision No 1313/2013/EU of the European Parliament and of the Council of 17 December 2013 on a

Union Civil Protection Mechanism he Union Civil Protection Mechanism (OJ L 347, 20.12.2013, p. 924).

Regulation (EU) No 516/2014 of the European Parliament and the Council of 16 April 2014 establishing

the Asylum, Migration and Integration Fund, amending Council Decision 2008/381/EC and repealing

Decisions No 573/2007/EC and No 575/2007/EC of the European Parliament and of the Council and

Council Decision 2007/435/EC (OJ L 150, 20.05.2014, p.168).

Regulation (EU) No 513/2014 of the European Parliament and of the Council of 16 April 2014

establishing, as part of the Internal Security Fund, the instrument for financial support for police

cooperation, preventing and combating crime, and crisis management and repealing Council Decision

2007/125/JHA (OJ L 150, 20.5.2014, p. 93) and Regulation (EU) No 515/2014 of the European Parliament

and of the Council of 16 April 2014 establishing, as part of the Internal Security Fund, the instrument for

financial support for external borders and visa and repealing Decision No 574/2007/EC (OJ L 150,

20.5.2014, p. 143).

Council Regulation (EC) No 2012/2002 of 11 November 2002 establishing the European Union

Solidarity Fund (OJ L 311, 14.11.2002, p.3).

Council Regulation (EU) No 1304/2013 of 17 December 2013 on the European Social Fund (OJ L 347,

20.12.2013, p. 470–486)

Regulation (EU) No 1304/2013 of the European Parliament and of the Council of 17 December 2013 on

the European Social Fund and repealing Council Regulation (EC) No 1081/2006 (OJ L 347, 20.12.2013, p.

470–48).

Regulation (EU) No 223/2014 of the European Parliament and of the Council of 11 March 2014 on the

Fund for European Aid to the Most Deprived (OJ L 72, 12.3.2014, p. 1–41).

Regulation (EU) No 282/2014 of the European Parliament and of the Council of 11 March 2014 on the

establishment of a third Programme for the Union's action in the field of health (2014-2020) and repealing

Decision No 1350/2007/EC.

There are different ways to implement the EU budget, depending on the variable level of implication of

the European Commission in its implementation. In direct management, the Commission is in charge of all

EU budget implementation tasks, which are performed directly by its departments either at headquarters or

in the EU delegations or through European executive agencies. In indirect management the Commission

entrusts budget implementation tasks to partner countries (or to bodies designated by them), international

organisations or development agencies of EU Member States. In shared management the Commission

delegates implementation tasks to the EU Member States. For more info see:

https://ec.europa.eu/europeaid/funding/about-funding-and-procedures/how-do-we-offer-funding_en

kom (2019) 0133 - Ingen titel

While Council Regulation (EC) No 1257/96 on humanitarian aid

allows the European

Union to provide assistance, relief and protection to people affected by natural or man-

made disasters and similar emergencies in third countries, its scope does not include the

EU. Therefore, while the Commission is able to draw rapidly on the humanitarian

capacities and expertise of more than 200 UN partners, international organisations and

non-governmental organisations outside of the EU, there was no legal basis to do so

within the EU at the beginning of 2016.

Consequently, as a response to the increased number of arrivals in the summer of 2015,

and in support to the Greek authorities, several national and international NGOs and

groups of volunteers had already started to provide assistance to refugees and migrants

arriving in Greece. However, administrative and operational modalities of the available

EU instruments made it difficult for humanitarian actors to be funded

. In particular, the

AMIF National Programme, where the majority of resources would have been available,

was not ready to fund at that moment any National or International NGOs, nor

International Organization, who could provide crucial support to the national response in

an EU Member State.

In this context, on 19 February 2016 the European Council called the Commission to 'put

in place the capacity for the EU to provide humanitarian assistance internally, in

cooperation with organisations such as the United Nations High Commissioner for

Refugees (UNHCR), to support countries facing large numbers of refugees and migrants,

building on the experience of the EU Humanitarian Aid and Civil Protection

department'.

On 2 March 2016, the Commission adopted its Proposal for a Council Regulation on the

provision of emergency support within the Union

, subsequently the Council Regulation

(EU) 2016/369 was adopted by the Council

on 15 March 2016

and pursuant to Article

9(2) it was activated for a period of three years for the management of the humanitarian

impact of the refugee and migration crisis within the EU. As a result, the Commission

had an instrument that could potentially finance its humanitarian partners to directly

implement actions within the EU and consequently boost Member States’ capacity and

support the expertise available for the humanitarian response to the the crisis.

Objectives and Scope of Council Regulation (EU) 2016/369

The general objective of the ESI, as stated in Article 3(1) of the Regulation, is to

"provide needs-based emergency response, complementing the response of the affected

Member States, aimed at preserving life, preventing and alleviating human suffering, and

maintaining human dignity…". According to Art 3(3) the emergency support provided

under the Regulation must be granted and implemented in compliance with the

OJ L 163, 2.7.1996, p. 1.

Under the AMIF emergency assistance component EMAS, the basic act for AMIF and ISF do not limit

funding to particular entities. It has been a policy choice to introduce in the annual work programme

(AWP) the principle that eligible applicant organisations can only be Member States, International

organisations and EU agencies.

Stefania Kalogeraki. 2018. Volunteering for Refugees and Asylum Seekers in Greece. Available at:

https://link.springer.com/chapter/10.1007/978-3-319-73335-7_7

European Council Conclusions EUCO 1/16, Brussels, 18-19 February 2016.

COM (2016) 115 final, Brussels, 2.03.2016.

https://www.consilium.europa.eu/en/press/press-releases/2016/03/15/refugee-crisis-emergency-support/

OJ L 70, 16.3.2016, p. 1

kom (2019) 0133 - Ingen titel

fundamental humanitarian principles of humanity, neutrality, impartiality and

independence.

According to Article 3(2) of the Regulation, emergency support may include any of the

humanitarian actions eligible for Union financing pursuant to Article 2, 3 and 4 of

Regulation (EC) No 1257/96, and may therefore encompass assistance, relief and, where

necessary, protection operations to save and preserve life in disasters or in their

immediate aftermath.

The Regulation builds upon the know-how of the Commission in the field of

humanitarian assistance - acquired through EU funded actions implemented in third

countries - and sets out the framework within which emergency support may be

provided, namely when the following preconditions are met:

In the event of an ongoing or potential natural or man-made disaster;

Where the exceptional scale and impact of the disaster is such that it gives rise to

severe wide-ranging humanitarian consequences in one or more Member States;

and



In exceptional circumstances where no other instrument available to Member

States and to the Union is sufficient.

Concerning its geographical scope, emergency support may be provided in the territory

of the EU Member States affected by the above conditions, and in the event of such a

disaster, the Council, based on a Commission proposal, would decide whether or not to

activate the emergency support and the duration of its activation.

In terms of implementation, the actions funded by ESI shall be carried out by the

Commission or by humanitarian partners selected by the Commission, including non-

governmental organisations, specialised services of the Member States or international

agencies and organisations having the requisite expertise, which have concluded a

Framework Partnership Agreement (FPA) with the Commission or fall in the scope of

the Financial and Administrative Framework Agreement (FAFA).

According to Article 1(2) of the Regulation, emergency support provided under ESI is in

support of, and complementary to, the actions of the affected Member State; therefore

close cooperation and consultation shall be maintained with the affected Member State

–

a stronger coordination requirement than is imposed on humanitarian operations outside

the EU. In addition, synergies and complementarity must be sought with other Union

instruments under which some form of emergency assistance may be offered. In the case

of humanitarian actions in Greece, this would primarily relate to voluntary in-kind

contributions provided under the UCPM and emergency assistance under AMIF and ISF.

The activation of the emergency support for the refugee and migration crisis

Following the Council’s decision to activate the ESI for a three-year

period, a Financing

Decision

was adopted on 15 April 2016 for the first year of actions funded by ESI. The

latter specified the maximum EU contribution for emergency support actions and defined

the specific objectives of actions funded by ESI in Greece and in any other affected

Member State. Two consecutive Financing Decisions followed for the 2017

and 2018

implementation periods, in which the same objectives were maintained, mainly:



C (2016) 2214 final, Brussels, 15.04.2016

(

2017) 763 final, Brussels, 14.02.2017

C (2017) 8295 final, Brussels, 12.12.2017

kom (2019) 0133 - Ingen titel



To meet the basic needs of refugees and migrants in the Union through the provision

of multi-sectorial support. The multi-sectorial response included the following

sectors of intervention: 1) food assistance; 2) distribution of non-food items; 3)

provision of shelter; 4) healthcare, including psychosocial support; 5) water,

sanitation and hygiene (WASH); 6) protection services; and 7) educational services

appropriate to the emergency context;



To provide technical assistance to the extent required for the management of

emergency support.

The programming of ESI was defined annually in the Emergency Support Operational

Priorities document (ESOPs).

Each ESOP served as guidance for the humanitarian

organisations to prepare their proposals for actions to be implemented in Greece. The

funding was allocated to the selected partners, following technical assessments

undertaken by the Commission experts.

All the humanitarian partners selected to implement ESI’s actions in Greece were

humanitarian organisations with whom the Commission had a signed Framework

Partnership Agreement (FPA). These are hereafter referred to in this document as

Commission’s partners.

The detailed

intervention logic

of the ESI is provided in Annex 2, where the scope,

overall objective and activities of the Instrument are presented and linked. The rationale

for ESI’s intervention was

based on the overall assumption that no other EU/ national

instrument was ready to provide the emergency support needed, at least not adequately to

the nature and scale of the needs, and that a ‘no action’ approach would have had severe

negative consequences not only on Greece but on the EU as a whole. The rationale for

the activation in Greece was related to different factors, including the high number of

arrivals, the closure of the Western Balkan borders and the fact that Greece shifted from

being a transit country to a host country, while it was already previously presenting

insufficient national capacity to address the basic needs of the newly arrived. In terms of

inputs, the maximum EU contribution allocated to the ESI, EUR 650 million, was meant

to be spread over the three years of the instrument's activation. Given that the ESI

focused on ‘life-saving’ emergency support, it was therefore assumed that the allocation

and selection of the Commission’s partners and their actions, together with the

Commission’s

experience on delivering humanitarian support, would contribute to the

provision of swift response. In this regard, the support study has confirmed that indeed

no other EU/ national instrument was able to deliver emergency support at this scale and

speed, and that the ESI’s implementation period has allowed for the immediate delivery

of a response.

Regarding the main activities to be implemented and undertaken by the ESI, the main

assumptions were based on the fact that Commission’s partners with appropriate

expertise, were available on the ground, or ready to be quickly deployed, to provide a

response which was timely and, more importantly, based on needs assessments. Overall,

a total of 29 actions were implemented by 18 Commission’s partners. The

support study

confirmed that the implemented actions prioritised the sectors in which support was the

most urgently needed, and that these also included coordination with national authorities,

other EU instruments, Commission’s partners, local implementing

partners and other

organisations involved. Furthermore, the support study indicated that in terms of outputs

ESOPs and Financing Decisions are published in ECHO website.

kom (2019) 0133 - Ingen titel

the actions implemented by ESI reached a cumulative total of over 146 thousand persons

by early 2018, mainly through activities related to protection, shelter and settlement, food

security and livelihoods, education, health, coordination, WASH and multi-purpose cash

transfers (MPCT) for basic needs assistance. With the caveat that within this cumulative

figure, beneficiaries might be counted twice, and there were nevertheless some

challenges faced at the initial stage of the ESI activation with regard to the coordination

with and between national authorities, which are further developed in section 5.

In terms of outcomes, actions funded by ESI were expected to address the most urgent

needs in terms of food, shelter and healthcare and to overall improve access to different

services, including accommodation protection measures and education for children. At

the same time, actions funded by ESI would aim to be complementary to actions

implemented by national authorities and civil society actors, while providing capacity-

building, and paving the way for a successful handover of the ESI-funded response to

other instruments. As a result, the assistance to and protection needs of the final

beneficiaries would have been met to the extent possible, and their dignity restored.

Regarding the successful handover, the support study identified shortcomings in this

regard, including the full development of a clear exit strategy to support a successful

handover to national authorities and ensure sustainability of actions. It must however be

noted that many actions funded by ESI were still on-going at the time the support study

was undertaken. Therefore, most of the exit strategies were still under development, and

as a result it was too early to assess their success and sustainability of exit strategies.

Nevertheless, several measures have also been undertaken specifically in order to

enhance the short- and longer-term sustainability of actions funded by ESI: these are

further described in section 5.

Finally, with regard to the expected impacts, the ESI was expected to prevent and

alleviate human suffering as a result of the emergency, and restore and maintain dignity

of the affected populations. In addition, actions funded by ESI were also expected to

–

indirectly-

support the Member State’s integration policy and measures, for example by

creating the pre-conditions for integration for beneficiaries. The main assumptions on

this related to the availability of a national integration framework/ strategy in the host

society and the implementation of such measures and framework by the authorities. In

this regard, the support study highlighted the absence of a well-developed national

integration strategy, which consequently affected the extent to which beneficiaries can

make a smooth transition from receiving emergency support to starting their integration

into the new host society. The Commission however considers that while these

challenges could indeed affect the final expected impact regarding the integration of

beneficiaries, these are nevertheless considered to be part of the situational context, and

not part of the nature of the ESI as an instrument. These challenges are described in

section 5.

2.2.

Baseline and points of comparison.

The ESI was set up as a generic instrument (i.e. to address the humanitarian

consequences of any natural or man-made disaster in the EU), not as a specific tool to

deal with humanitarian consequences of the migration and refugee crisis. Due to the

immediate need to provide a rapid response, no impact assessment or in-depth

stakeholder consultation took place prior to the creation of the ESI (i.e. back in 2016).

In order to address the absence of a baseline to implement the first EU emergency

support intervention within the EU, the Commission carried out a broad consultation at

different levels to develop a needs assessment. The latter involved not only the different

kom (2019) 0133 - Ingen titel

relevant Commission services active in regard to the migration crisis, but also a wide

range of stakeholders, including humanitarian partners and other donors.

The United Nations’ Regional Refugee and Migrant Response Plan for Europe, launched

in January 2016

, was key to build points of comparison and the responsible

organisations (i.e. International Organization for Migration (IOM) and United Nations

High Commissioner for Refugees (UNHCR)) were widely consulted throughout the

process. This plan identified the main needs and provided a quantification of financial

resources needed per sector and per year. The 2016 response was divided in seven sectors

namely protection, education, site management support, health and nutrition, food

security, shelter/ NFIs items and WASH, thus covering all the sectors with the highest

reported needs.

The 2017 response targeted the same sectors but taking into

consideration the new contextual developments and emerging needs. Similarly the 2018

response followed up on actions covering the main sectors identified throughout 2017;

however, in 2018 the need to plan and prepare for the hand-over of actions funded by ESI

to the national authorities was introduced. A description of the evolution of the identified

needs and priorities is further described in section 3.

Other sources of information were considered in the support study as point of

comparison. However, it must be emphasised that Greece’s situation in 2015/2016 was

unprecedented. Greece was already facing many challenges as a transit country of

migrants and refugees, paired with a deep economic crisis. After becoming a host country

with an increased number of arrivals, the national authorities had to deal not only with

the most urgent humanitarian needs of the new arrivals, but also with the medium and

long-term needs of those who were now stranded and therefore remaining in Greece.

Greece therefore faced the need to significantly and urgently increase its reception

capacity, to move from temporary accommodation to more long-term housing, as well as

to address the health, education and protection needs of refugee and migrant children and

the eventual integration of the beneficiaries, for example into the labour market and

public services for the adults.

Overall, the situation of refugees and migrants in Greece in March 2016- hence before

the activation of ESI- and their main needs can be summarised as follows

There was insufficient accommodation capacity and a lack of basic humanitarian

services (i.e. food, water and sanitation facilities).

While some facilities were reported to be adequate for short-term

accommodation, there was a lack of long-term accommodation facilities.

The vast majority of the camps were located outside urban areas, often far from

service providers and with difficult access to public transport.

The Regional Refugee and Migrant Response Plan, launched by the Office of the United Nations High

Commissioner for Refugees (UNHCR), the International Organization for Migration (IOM) and 72 other

partners, aimed to play a key role in ensuring more efficient operations and a better coordinated response to

the migration challenges. Available at: UNHCR Regional Refugee and Migrant Response Plan for Europe,

January 2016

UNHCR. 2016. Regional refugee and migrant response plan for Europe January to December 2016.

Available at: http://reporting.unhcr.org/sites/default/files/RRMRP%20Europe%20Jan.-Dec%202016%20-

%20Revision%20May%202016.pdf

Amnesty International. 2016. Greece: Chaos erupts at Idomeni border as Balkans route shut down.

Available at: htpps://www.amnesty.org/en/press-releases/2016/02/idomeni-border-crisis/

https://reliefweb.int/sites/reliefweb.int/files/resources/62216.pdf

kom (2019) 0133 - Ingen titel

In a number of camps, there was no official management of the site leading to a

lack of registration of refugees and migrants.

There was a lack of special shelters for vulnerable people including

unaccompanied minors, elderly, victims of sexual and gender based violence

(SGBV) and individuals suffering from serious mental health disorders.

The lack of security in the camps was highlighted by national and international

NGOs. Violent incidents, including SGBV were reported in several sites.

The lack of systematic provision of information to refugees and migrants on their

rights and obligations was also observed by some NGOs on the ground.

In the islands, there was a significant lack of accommodation capacity and

refugees and migrants were sleeping in tents unsuitable for the winter. Access to

food and water was also reported to be insufficient

3. I

MPLEMENTATION

STATE OF

LAY

This section provides a brief overview of the actions funded by ESI over the 2016-2018

period.

Following its activation in Greece, a total of EUR 700 million was intially set as the

maximum contribution of the EU for the provision of emergency support to Greece for

ESI’s three-year

activation period (2016-2019). The 2016 Financing Decision initially

provided for a maximum contribution of EUR 300 million, however the budgetary

authority ultimately allocated EUR 250 million. Both the 2017 and the 2018 Financing

Decisions set the maximum contribution of the Union at EUR 200 million for each year

respectively, bringing the overall total to EUR 650 million.

These funds were allocated to deal with the fluctuating number of people in need of

protection in Greece. In addition, support was to be provided to a Member State already

severely affected by the impact of the economic crisis, where significant unmet

humanitarian needs were present in the sectors of food assistance, shelter, healthcare,

water, sanitation, hygiene (WASH), non-food items (NFIs), education and protection.

However, shortly after the adoption of the Regulation, the situation on the Eastern

Mediterranean route significantly changed. Following the adoption of the EU-Turkey

Statement of 18 March 2016

, irregular crossings from Turkey to Greece decreased

sharply (from 1,800 daily arrivals in early 2016 to 73 daily arrivals after the Statement)

Despite seasonal peaks and a surge of arrivals by land at the border with Turkey since

ECRE. Greece. Conditions in reception facilities. Available at:

http://www.asylumineurope.org/reports/country/greece/reception-conditions/housing/conditions-reception-

facilities

ECRE. Greece. Conditions in reception facilities. Available at:

http://www.asylumineurope.org/reports/country/greece/reception-conditions/housing/conditions-reception-

facilities

ECRE. Conditions in reception facilities. Available at:

http://www.asylumineurope.org/reports/country/greece/reception-conditions/housing/conditions-reception-

facilities

See http://www.consilium.europa.eu/en/press/press-releases/2016/03/07-eu-turkey-meeting-statement/

Based on figures available until 22.09.2017 according to data provided by Greek authorities to the

Commission.

kom (2019) 0133 - Ingen titel

late 2017, the average daily arrivals in 2017 and 2018 have nevertheless ranged between

100 and 135 persons per day.

Actions implemented under the ESI and geographical coverage

Within the three-year activation period ESI has funded 29 operational actions

implemented by 18 Commission’s partners amounting to EUR 644.5 million

(as shown

in Figure 1 below). The actions included activities in the field of provision of shelter, site

management, cash assistance, food aid, distribution of non-food items, protection,

education, health services, including psychosocial support. An overview (table) of the

actions funded by ESI, by funding and Commission’s partners is provided in Annex 6.

Figure 3.

ESI contribution to operational actions implemented in Greece

At the beginning of the activation, in March 2016, while ESI was mainly funding actions

on the Greek mainland (hereafter referred to as ‘mainland’), specific unmet needs on the

islands were also targeted. This was the case for emergency accommodation capacity

during winter (including hotel places and rub-halls), complementary food distribution,

protection and some education in emergency activities. Most of the needs on the islands,

where the so-called

“hotspots”

were located, were targeted by other EU funding

instruments such as AMIF and ISF.

However, in 2017 under the agreement with the Greek authorities and in view of the

phasing out of the ESI in 2019 and the need to prepare a handover, it was decided that

ESI would be active in the mainland only, with the exception of the cash and rental

accommodation schemes on the islands.

Evolution of programmatic priorities

During 2016, the humanitarian and emergency situation in Greece was characterised by

several changes in terms of population movement. From a high number of arrivals

While EUR 650 million was set as the maximum EU contribution for ESI actions, the EUR 644.5 million

represent the funding provided for the implementation of

operational

actions, while the remaining budget

has been allocated to cover the Commission’s technical

assistance for the management of the support

provided by ESI. As established in the Financing Decisions, in 2016 and 2017 a total of 1% of the total

maximum contribution was allocated each year for technical assistance, whereas in 2018 0.5% of the

budget was allocated to technical assistance.

“The 'hotspot approach' was presented by the Commission as part of the European Agenda on Migration

of April 2015, when record numbers of refugees, asylum-seekers and other migrants flocked to the EU.

The 'hotspots'

–

first reception facilities

–

aim to better coordinate EU agencies' and national authorities'

efforts at the external borders of the EU, on initial reception, identification, registration and fingerprinting

of asylum-seekers and migrants. Currently,

only Greece and Italy host hotspots”. Source:

European

Parliament Briefing on the State of Play of Hotspots at EU External borders.

kom (2019) 0133 - Ingen titel

reported at the end of 2015 and beginning of 2016 and closed borders

along the Balkan’s

route, which prompted a wide-ranging and multi-sectoral emergency assistance to meet

basic needs of people in transit, the situation changed after the EU-Turkey Statement,

which resulted in a sharp reduction in the number of arrivals.

ESI activation focused on providing support to meet the basic needs of the people

stranded in Greece. Before ESI activation, living conditions in existing accommodation

facilities were assessed as sub-standard with needs remaining high in terms of food, non-

food items, protection, medical care, WASH, non-formal education,

shelter/accommodation (including winterisation) and unaccompanied minors (UAMs) . It

is against this scenario, and following an agreement on the priorities between the

Commission and the relevant national authorities, that the first tranche of EUR 247.5

million was allocated in 2016 for emergency support actions to address the

aforementioned sectors. As a result, most beneficiaries of actions funded by ESI

(hereafter ‘beneficiaries’) were

accommodated in 40 formal and informal sites/camps on

the mainland and in two sites on the islands (Karatepe and Souda).

While the initial 2016 ESI budget allocation was made in emergency mode and in the

absence of an agreed governmental response plan, in early 2017 the Commission services

(Structural Support Service (SRSS), DG HOME and DG ECHO) and the national

authorities (Ministry of Migration Policy (MoMP), Ministry of Economy and Ministry of

Defence in particular) agreed on the operational priorities and on a strategic response

plan for 2017. The discussions resulted in the first Financial Plan presented by the

national authorities in February 2017. Overall, the Financial Plan provided the main

elements of the approach to be implemented during the given year, the main needs and

areas to be covered (as indicated and requested by the national authorities), an indicative

budget breakdown per relevant actor and per EU funding instrument, as well as the

Guiding Principles for all the actions funded in Greece.

The subsequent 2017 ESI funding allocation amounted to EUR 198 million and was

made in line with the priorities established in the Financial Plan 2017, which included a

gradual transition of beneficiaries from sites to urban rental accommodation, and an

increased focus on multi-purpose cash transfers (MPCT). The combination of rental

accommodation and MPCT resulted in the programme called Emergency Support to

Integration & Accommodation (ESTIA), which is the flagship programme of ESI. The

response plan also included a handover of activities to the national authorities, mainly

those actions providing shelter for UAMs and all actions carried out on islands (with the

exception of partial cash transfers and rental accommodation schemes) with the support

of the AMIF National Programme as of August 2017.

The third and last allocation of ESI funding amounted to EUR 199 million. Its

accompanying 2018 Financial Plan reflected the need to plan and prepare for the

handover of actions funded by ESI to the national authorities, and namely of the ESTIA

programme which absorbed the majority of the 2018 ESI funds (EUR 167.5 million) to

provide 27 000 accommodation places and cash assistance to more than 65 000 asylum

seekers. The transition of actions to the national authorities was also planned in other

sectors, with health assistance in sites already starting as of July 2018.

Overall, the actions implemented by ESI evolved along and in parallel with the needs and

the changing situation in Greece i.e. from an initial emergency phase in which the short

term needs of refugees/migrants needed to be covered and the administrative capacities

needed to be developed and reinforced (e.g. creation of the Ministry of Migration

Policy); to a second phase of joint planning with the national authorities, where the

humanitarian response was consolidated, for example by reducing the number of partners

kom (2019) 0133 - Ingen titel

and interventions and by initiating the handover phase; and finally, to a last

implementation phase where by the end of 2018/beginning of 2019 all actions funded by

ESI would be completed, and the most important building blocks would be handed over

to other sources of EU funding, with the ultimate aim being the handover of actions to

the national authorities. For instance on 20 December 2018 agreements were signed for

the two main programmes funded by the Emergency Support Instrument (ESTIA and

Site Management Support for the mainland), that ensure funding from AMIF Emergency

Assistance as of January 2019.

Figure 4 below provides an overview of the evolution of priorities and activities

according to the different ESOPs developed throughout 2016 -2018.

Figure 4. Three-year ESOPs evolution overview.

NB:

The maximum contribution of each ESOPs budget includes the budget allocation for the provision of

the Commission’s technical assistance for the management actions funded by ESI.

The following sub-sections below provide the implementation and state of play of the

actions funded by ESI according to the specific priorities and sectors.

Figure 5. ESI Funding per sector (as of 31/12/2018)

kom (2019) 0133 - Ingen titel

CCOMMODATION

The Commission allocated EUR 300 million (representing the highest share of ESI

funding and taking into account the following sectors: shelter & settlements, WASH and

support to operations) to its

Commission’s

partners (i.e. UNHCR, IOM, IRC, IFRC,

ASB, DRC, NRC, CARE, OXFAM and TDH

) to provide accomodation and shelter

support for all beneficiaires in Greece. Accomodation included both categories: places in

sites and rental accommodation scheme (apartments). It is important to note that at the

time of activation, insufficient shelters were available to house refugees and migrants and

most sites had to be set up from scratch, which made the initial actions costly. Moreover,

on top of the funding granted to actions in the area of shelter and accommodation, ESI

also funded actions that aimed at providing and maintaining WASH facilities in

accommodation places. A total of EUR 21 million from the ESI budget was used to fund

WASH actions during the period 2016-2018 (see Figure 5). Out of the total allocation

under accomodation, over EUR 116 million were allocated to

Commission’s

partners to

build sites/camps and assist in their management, including coordination, care,

maintenance and community mobilization. This has allowed the creation of some 20 000

places in permanent camps and some 10 000 to 15 000 temporary emergency

accommodation places all over mainland

As part of the 2017 Financial Plan, the national authorities decided for a gradual

transition of beneficiaries from temporary reception facilities (sites) to urban rental

accommodation. Internally, the European Commission services decided that the AMIF-

funded rental accommodation scheme would be handed over to ESI, changing its main

purpose from being a functional tool of the relocation policy, to a vulnerability-based

reception system. UNHCR was identified as the most suitable framework partner to run

such an action, labelled as the

“ESTIA programme”.

Over EUR 184 million were

allocated to the ESTIA accommodation programme to provide up to 52 000 rental

accommodation places between 2017 and 2018 (up to 25 000 in 2017 and up to 27 000 in

2018).

ULTIPURPOSE

ASH

CHEME

The Commission allocated EUR 122 million to multi-purpose cash assistance for

beneficiaires in Greece throughout the 2017-2018 period, representing the second highest

share of ESIs funding

. At the beginning of the implementation, all assistance had been

provided in kind (food rations, non-food items) by several actors funded under different

instruments.

As of July 2016 and in agreement with the national authorities, ESI started funding

actions to build up a multi-purpose cash scheme in Greece via several

Commission’s

United Nations High Commissioner for Refugees (UNHCR), International Organisation for Migration

(IOM), the International Federation of Red Cross and Red Crescent Societies (IFRC), the International

Rescue Committee (IRC), the Danish Refugee Council (DRC), Arbeiter-Samariter-Bund (ASB), Save the

Children (STC), Norwegian Refugee Council (NRC), Care International (CARE), and Terre des Hommes

(TDH).

Greek Council for Refugees

–

types of accommodation, available at

http://www.asylumineurope.org/reports/country/greece/reception-conditions/housing/types-

accommodation

The multipurpose cash scheme was introduced in April 2017 and fully implemented nationwide in May

2017. The total allocation (between 2017-2018) amounts to EUR 122 million. Before the scheme was

introduced, a total of EUR 22 million was allocated to different activities involving cash transfers in 2016.

kom (2019) 0133 - Ingen titel

partners. The aim of this first phase was to build up the coverage of multi-purpose cash

assistance and the operational capacity of partners. By the end of 2016, with the addition

of a UNHCR-implemented cash project, the total cash assistance provided to beneficiares

in Greece increased, replacing in-kind distribution of relief items except food aid. By

February 2017, over 35 000 people living in 55 locations in the mainland and on the

islands, as well as in two urban locations, had received cash assistance, which was

provided through pre-paid cards. The monthly cash assistance scheme is only covering

basic needs and is geographically restricted to Greece. By injecting cash into the local

economy, it impacts economic recovery in the host country and supports increasing the

self-reliance of its beneficiaries. Major progress was achieved in 2017 through the

establishment of a single delivery cash-based assistance: by December 2017, 37 600

people living in 92 locations in mainland and on the Greek islands have received cash

assistance. From April 2017 to October 2018, 90 351 eligible individuals have received

cash assistance in Greece at least once

ROTECTION

The protection sector was the third largest funded by the ESI. The Commission allocated

EUR 108 million for protection related activities in Greece, both in sites and in urban

contexts in complementarity to the rental accommodation scheme. Funded actions

included the provision of psycho-social support, child-friendly spaces, case management

systems, family tracing and care for UAMs. Two-thirds of children residing in camps

benefited from child-friendly spaces, where children could regain a sense of normality,

and the ones more at risk can get specialised support. Ten safe zones with a total capacity

for 300 UAMs were available in ten sites on the mainland. Other actions included

protection of women (including from Sex and Gender Based Violence), elderly and

disabled persons, legal support to refugees and migrants, as well as to ensure that

refugees and migrants were well informed on their status and rights. It is estimated that a

cumulative total of over 200 000 persons benefitted from protection services for the

whole ESI activation period, with the caveat that estimation on the number of

beneficiaries reached is strongly limited by the fact that there could be a significant share

of double counting- this is further explained in section 5.3.

ELIVERY OF HEALTH SERVICES

EUR 31.5 million, representing the fourth largest share of

ESI’s

funding, were allocated

to provide primary health care, specialised healthcare (including mental health), psycho-

social support and referral to hospital for people located in camps, both on mainland and

islands, as well as referral and translation support in the urban context of Athens. An

average of over 10 000 primary health care consultations were provided every month by

Commission’s

partners (i.e. IFRC, ASB, MDM-BE and MDM-GR

). Vaccination for

children was provided, including for those attending schools. Around 1 000 mental health

consultations took place every month, a service much needed due to the traumatic

experiences of most beneficiaires. In 2018 health

Commission’s

partners handed over

health care in sites to the Ministry of Health structures under the DG HOME funded

"Philos programme".

Source: UNHCR data,

https://data2.unhcr.org/en/documents/download/66253

Arbeiter-Samariter-Bund (ASB), the International Federation of Red Cross and Red Crescent Societies

(IFRC) and Médecins du Monde (MDM).

kom (2019) 0133 - Ingen titel

NACCOMPANIED

INORS

(UAM

)

Between April 2016 and July 2017, ESI continued to scale up its financial support for the

provision of adequate accommodation and accompanying care services for UAMs

stranded in Greece. By July 2017, ESI was providing financial support to cover the

management of approximately 1 000 places for UAMs in dedicated shelters in Greece,

for a total amount of EUR 22.8 million. As previously mentioned, from August 2017, the

responsibility for the provision of financial support to continue the management of these

shelters was transferred to the national authorities, under AMIF funding of the National

Programme. ESI continued to provide financial support to run ten temporary safe zones

of UAMs in camps. Safe zones provide short-term, temporary accommodation and a full

range of care services for UAMs, pending the availability of more adequate

accommodation place in dedicated UAMs shelters

DUCATION

Since April 2016, ESI provided over EUR 25 million in support of access to formal and

non-formal education, representing the fifth largest share of

ESI’s

funding by sector.

Actions under formal education included transport by school buses between camps and

local schools throughout the academic year, and supporting the implementation of the

Ministry of Education's strategy to ensure access and integration of refugee children in

the formal education system via afternoon bridging classes (DYEP programme). All

refugee sites across the mainland were incorporated into this strategy. In addition, ESI

also funded actions, mainly through UNICEF, to support the work of the Ministry of

Education and coordinating organisations working with refugees to enrol children living

in urban areas into the local school, and to provide training to local school teachers in

multicultural schoolroom management skills. By June 2018, 8 000 refugee and migrant

5-17 year old children were enrolled in primary and secondary education.

Funding was also allocated to

Commission’s

partners to provide non-formal education to

children and adults residing in refugee camps and urban areas across Greece. Activities

provided included basic maths and literacy lessons, and classes in Greek and English and

mother tongue language. Around 9 000 children have benefited from these activities.

THER MEASURES

As further complementary measures, over EUR 53 million were allocated to ensure the

delivery of food, nutritional services for children, distribution of non-food items

(blankets, clothing, rain ponchos, hats, gloves, etc) and coordination activities such as

organisation of working groups and production of information material.

Three Commission humanitarian field experts were redeployed to monitor and coordinate

the actions funded by ESI in Greece. The presence of these experts was crucial to support

the Commission’s partners in their work and to liaise with the national authorities. All in

all, the 29 actions were monitored through at least 65 field missions. In addition, some of

the

Commission’s

global thematic experts (e.g. on cash, health services, protection and

gender issues) were also sent to Greece over twelve times to provide strategic

orientations policy in their areas of expertise.

Overall, while it is too early to assess the impact of the actions funded by ESI in the mid

and long-term run, in the

Commission’s

view

–

based on the available qualitative and

quantitative data-

ESI’s

actions have significantly contributed to meeting the needs of the

kom (2019) 0133 - Ingen titel

target population during the period concerned, complementing crucially the related

actions of the Greek authorities and civil society organisations.

4. M

ETHOD

4.1.

Short description of methodology

This Staff Working Document builds on an independent evaluation conducted by an

external contractor, as well as on an internal evaluation exercise developed by the

Commission on the actions carried out by the ESI. The internal evaluation exercise was

undertaken from January 2018 to February 2018, in view of the possibility of ESI being

included in the future post 2020 Multiannual Financial Framework (MFF) and in view of

the Better Regulation Guidelines. Given the timing of the internal evaluation exercise,

the latter focused on assessing the Instrument as a whole, i.e. without fully assessing in

detail the design and results of ESI's funded actions, as many of these were still ongoing.

Different research methods and tools were used by the internal evaluation, including

literature and document review, as well as a stakeholder consultation involving all

relevant actors (See Annex 3). In order to complement the preliminary findings of the

internal evaluation, an external evaluation was launched. The latter identified and

targeted the gaps of the internal evaluation (i.e. the mapping and analysis of the ESI’s

implemented actions and their available results) and complemented its preliminary

findings, by further assessing the actions implemented as part of the operational

response.

The external evaluation (hereafter ‘support study’) was carried out between July 2018

and October 2018. The methodology used by the both the support study and the

Commission’s internal evaluation are coherent with the guidelines provided in the Terms

of Reference (ToR), as adopted by the Inter-service Steering Group (ISG) and as

indicated in the Evaluation Roadmap. The results of the internal evaluation exercise,

which complement the support study, were added and are presented as a single

evaluation exercise by the support study.

A desk review was used to inform the support study, which provided the answers to the

evaluation questions identified by the Commission in the ToR and was based on a

detailed evaluation framework developed for this evaluation, which was informed by all

data sources and research tools put in place during the evaluation.

The support study also made use of complementary research methods to ensure a robust

assessment of the findings, and hence the reliability and validity of the evidence and to

verify and triangulate the different sources of the evaluation results, as well as to address

potential bias and to ensure objectivity.

For example, in addition to a stakeholder consultation, the variety of data sources and

research tools included documentary and literature review, as presented below:



A large literature review, including:

A set of 166 documents (i.e. legislation, regulatory documents, the

Emergency Support Operational Priorities (ESOPs), Financial Decisions

and Plans, project portfolio analysis, proposals, monitoring reports,

amongst others):

kom (2019) 0133 - Ingen titel

35 publicly available documents to capture information gathered by third

parties, such as UNHCR, IOM and other Commission services. In

addition, databases from UNHCR, IOM and the Greek authorities were

explored and used for this analysis.



The mapping of the 29 actions funded by ESI, including their proposal

assessment, as well as rejected proposals.

The stakeholder consultation covered a wide range of relevant actors involved in actions

funded by ESI in order to capture their views and inputs and it was composed of the

following activities:



A total of 73 key informant interviews

(with EU officials, Greek authorities,

ESI’s Commission’s partners and local implementing partners)

Two different online surveys:

The first one targeting the Commission’s

partners and their local

implementing partners who have carried out actions in Greece and;

The second one consisted of a mini mobile survey of main beneficiaries.



Ten focus groups with beneficiaries;

Three field missions to Greece which included interviews and project visits.

An overview of the stakeholders reached and consulted is provided in Annex 3.

An open public consultation (OPC) was also carried out for three months (January 2018

–

March 2018), for which 153 respondents from all over Europe provided feedback on

EU funds in the area of security, including amongst other issues, the provision of

humanitarian support at large scale for emergency situations. An analysis of the relevant

replies is provided in Annex 5.

4.2.

Limitations and robustness of findings

Several actions funded by ESI were still ongoing at the time the support study was

carried out. Thus the unavailability of some final project reports was a limitation.

Mitigation measures were therefore undertaken and all available primary data for all

actions funded by ESI that could be collected was analysed. The stakeholder strategy

therefore focused on carefully identifying and reaching a strategic sample of stakeholders

aimed at targeting those that have had a significant involvement in actions funded by

ESI, provided valuable findings. Furthermore, it was also too early to assess the

successful handover of ESI actions to the national authorities or the implementation of

activities with other sources of EU funding such as AMIF/EMAS. At the time of

developing this report, grant agreements with UNHCR and IOM were signed by the

Commission (DG HOME) to ensure the continuation of EU funding for the ESTIA

programme and site management support actions

–

following the end of ESIs three year

activation period - and before the actions are handed over to the authorities.

43 key informant interviews were undertaken by the external evaluators, while 30 interviews were

undertaken as a part of the internal evaluation exercise done by the Commission.

kom (2019) 0133 - Ingen titel

The lack of quantitative data was another limitation, particularly to inform the efficiency

criteria. This is also related to the fact that there has not been any other operation

providing emergency humanitarian support at the same scale and within the EU that

could be used for potential comparison. Therefore, the external evaluator developed a

series of indicators to assess the cost-effectiveness of the ESI in Greece. The indicators

compared different costs in relation to actions funded by ESI, with the costs of other EU

interventions in Egypt, Jordan, Lebanon and Turkey (see section 5.4). In addition,

emphasis on qualitative data, particularly on the stakeholder consultation findings was

placed in order to triangulate the findings and provide valid conclusions

Finally, it should be highlighted that the findings provided by the support study and this

document are mainly based on the sole case study on the activation of ESI in Greece.

The implementation of ESI actions was influenced by the specific context of one single

Member State, therefore, the extent to which this evaluation was able to assess the

instrument’s broader adequacy for emergency situations elsewhere in the EU is somehow

limited to such single case study.

5. A

NALYSIS AND ANSWERS TO THE EVALUATION QUESTIONS

The support study was based on 17 evaluation questions (as shown in annex 4) and

organised around the five main evaluation criteria presented in the European

Commission's Better Regulation Guidelines, complemented by the criterion of

sustainability. The findings are presented in this section according to each criterion.

5.1.

5.1.1.

Relevance

PROVIDE EMERGENCY SUPPORT

ELEVANCE OF THE REGULATION AS A TOOL TO

WITHIN THE

NION

The ESI was relevant and necessary to provide the capacity to address wide-ranging

humanitarian needs within the Union resulting from natural or man-made disasters,

within just the general limitation of assessing the Regulation’s broader relevance on the

basis of a single case study. The conclusion of the support study underlined the swift

activation time of ESI, the ability to mobilise a large proportion of funding and hence a

swifter implementation of the actions (through an established framework of

Commission’s

partners on the ground) and the diversity of such actions. Furthermore,

there was a general agreement among stakeholders that the Commission’s contribution

through the ESI activation was in line with its mandate and relevant to an emergency

response within the EU.

The document review and the consultation with stakeholders showed that ESI was the

most relevant tool to deliver an emergency response of such nature and scale within the

EU. In Greece, ESI supported a Member State for which the refugee and migration influx

added critical challenges to their national system already struggling with a difficult socio-

economic and employment situation, in addition to the emerging and urgent humanitarian

needs of the new arrivals. Survey results of Commission’s partners

and their local

implementing partners, confirmed this finding as the vast majority of respondents (80%)

considered the use of the ESI was relevant to respond to the humanitarian needs spurred

by the sudden influx of refugees and migrants into the EU.

kom (2019) 0133 - Ingen titel

Figure 6. Relevance of the ESI to respond to emerging humanitarian needs in Greece, and in the EU

Source: DG ECHO. 2018. Survey of framework partners and local implementing partners. N=38

Some of the main distinguishing factors of the ESI were its rapid response time, i.e. the

speed with which the ESI was activated and actions launched on the ground once the

situation was deemed an emergency; the Commission’s implementation method and

longstanding experience in humanitarian emergencies i.e. through the humanitarian

Commission’s partners; as well as the breadth of actions it was able to fund. This allowed

to commit funds directly to the Commission’s partners, already present in Greece (which

in result collaborated with local implementing partners already operating on the ground),

as opposed to through national/ regional authorities. Although different EU funding

instruments were available for Greece (e.g. the EUSAF, AMIF, ISF, FEAD), the

document review and other research tools showed that none of these specifically fully

targeted the evolving large scale humanitarian needs (See section 5.2). More importantly,

such funds primarily relied on the administrative and operational capacities of a Greek

government, who was under severe stress in financial and economic terms.

The swift proposal, adoption and activation of ESI meant that there was a quick response

to the humanitarian consequences of the emergency. Evidence shows that, following the

quick agreement by the Council on the Regulation, ESI was operational in a very short

time period. Overall, it took 19 days to activate and within the subsequent three weeks

the first Commission’s partners’ contracts were signed and funding released.

5.1.2.

ELEVANCE

OF THE

ESI

INTERVENTION WHEN CONSIDERING

REECE

’

PARTICULAR SITUATION AND NEEDS

;

AND THE CORRECT ASSUMPTION OF SUCH

NEEDS

ESI was also a relevant tool for an emergency such as the one faced by Greece. The ESI

intervention was appropriate and valuable considering the situation in Greece, which

was, at the time, characterised by an overwhelmed national capacity to respond and

address in a timely manner the scale of the needs on the ground. The national authorities

were facing a multitude of national challenges. The nature of these were not only

ongoing economic and related to institutional reforms, but also related to the asylum

system and procedure. The evidence collected by the support study further concluded that

both the sudden increase in the number of asylum seekers and migrants paired with

European Commission (EU) 2 March 2016. Proposal for a Council Regulation on the provision of

Emergency support within the Union

kom (2019) 0133 - Ingen titel

institutional and administrative challenges faced by the national authorities, made ESI

intervention indispensable.

The Member State’s limited capacity was underscored by the support study, which found

that although other EU funding instruments were available, Greece was facing issues

regarding the rapid implementation and financial absorption of these funds (mostly

AMIF National Programme). Research conducted by the support study and based on

Key Informant Interviews compared the response to the refugee emergency in Greece

(using ESI), Bulgaria and Italy (using EMAS)

. It was found that the response in

Bulgaria was slower and less flexible than the response in Greece, particularly because

of the different regulatory framework of the instrument, paired with the complex public

procurement procedures, including the development of specification of the needs. For

example, in Bulgaria, the general procurement process takes six months on average.

Therefore, the process between the identification of the ‘need’ to intervene and the

delivery of goods or services to the beneficiaries, typically, can take up to one year,

which in practice means that the length of the procedures is a significantly long period to

provide an emergency response.

Also, the majority of the Commission’s partners and implementing

partners surveyed

(75%) in the context of the evaluation indicated that ESI had been a relevant tool to

respond to the situation in Greece. Out of these respondents, 53% agreed that the specific

objectives of ESI were relevant, or at least somewhat relevant, to the needs in Greece,

with 63% of them considering the objectives to be relevant to the specific needs of the

target group identified (see Figure 6 above).

Given the emergency of the situation and the need for a rapid response, at the initial stage

of the activation, a rapid needs assessment was undertaken, which consisted of mapping

the existing EU funds, to minimise duplication and increase the relevance of ESI within

the context of Greece, as well as needs assessment undertaken in the field

based on a

broad dialogue with relevant stakeholders at all levels.

Subsequently, and as described in Section 3, each yearly ESOP undertook a needs

assessment, and the priorities of each ESOP evolved according to the changing situation

and needs in Greece during the implementation period

. As underlined by the support

study, each of the ESOPs was based on robust needs and risk assessments conducted

jointly by the Commission services, national authorities and Commission’s partners. In

addition, specific needs assessments were conducted for all actions funded by ESI. For

example, while at the central level, the Commission conducted needs assessments mainly

through field missions and dialogue with relevant stakeholders, at the governmental and

While findings are reported for Bulgaria, the support study does not mention any remarkable findings

related to the relevance for Italy.

The initial assessments took into account the Emergency Response Coordination Centre (ERCC) needs

assessments, the results provided by the early Commission reports undertaken on the refugee crisis in

Greece, and the needs assessment and priorities provided by DG ECHO partners active in the Member

State. Overall, the evaluation findings showed that a preliminary needs assessment on Greece was actually

undertaken in early February 2016 by the ERCC following the activation of the EU Civil Protection

Mechanism in Greece.

Thus, ESI initial needs assessment considered those assessments undertaken and provided by several

Commission’s partners active in Greece, such as: Action Aid, Caritas, Christian

Aid, DRC, IOM, IRC,

MDM, MSF, NRC, Oxfam, REACH, Red Cross Movement, Save the Children, Solidarities International,

UNHCR, UNICEF, as well as the Greek authorities and the European Commission.

See further flowchart of Annex 1 “Safeguards for spending EU funds and ensure quality under the ESI”

kom (2019) 0133 - Ingen titel

the local level,

each of the Commission’s partners had to undertake and provide a needs

assessment for each of their proposed actions. Therefore, each of the ESOPs contained a

clear analysis of needs as well as a risk assessment, identifying potential constraints and

limitations, to be taken into consideration when implementing actions on the ground. The

support study also indicated that while at the beginning of the activation, the

Commission’s partners considered that the development process of the initial needs

assessments was unclear, the process considerably improved in the subsequent years, by

providing better clarity on the process and links between such assessments and funding

allocations.

Thus, in agreement and cooperation with the Greek authorities, the Commission

prioritised the sectors of shelter, food, health and protection

. As shown in section 3,

given that the needs were multi-faceted, these had to be addressed from various angles,

hence by a multi-sectoral approach. The actions funded by ESI were therefore

implemented through a multi-sectoral response, although priorities were formulated in a

way to also address niche support if relevant. Thanks to years of expertise of

humanitarian activity outside of the EU, the Commission has defined policy guidelines

for each main humanitarian sector to ensure high quality and coherence of the funded

actions. Overall, the Commission’s partners involved in the ESIs response have not only

adhered to these standards and practices, but have been often active contributors to the

definition of policies, as the Commission involves its humanitarian partners in the

definition of policies through extensive consultation. In this sense, the Commission

considered that the direct implementation of ESI actions through its experienced

humanitarian partners was relevant, as it allowed the Commission to ensure adherence to

its own humanitarian standards and good practices. The guidelines were applied to the

same extent for the current ESI activation as for actions in third countries

. They were

referred to in the ESOPs, were taken into account in the selection of projects, and were

used as a reference for project monitoring

Concerning ESI funding, this was consistent over the intervention period and the

mapping of actions funded by ESI showed that all main sectors were overall well

covered, with accommodation, food assistance and protection identified recurrently as

top needs. The support study however underlined that a few needs identified on the

ground during the implementation, in particular those of specific vulnerable groups and

of beneficiaries on the islands could have benefited from more support. The former

mainly related to mental health services, interpretation and cultural mediation services.

While these areas were identified

in the 2017 & 2018 ESOPs, the Commission’s partners

did not propose enough actions for funding to cover these specific needs. The absence of

proposals for actions was due either to the lack of national capacity in the Member State,

See further flowchart of Annex 1 “Safeguards for spending EU funds and ensure quality under the ESI”

A repository of DG ECHO’s humanitarian policies is available at

:https://ec.europa.eu/echo/what/humanitarian-aid_en - by sector

Aid efficiency, Capacity building, Cash-

based assistance, Disaster risk reduction, Education in emergencies, Food assistance, Gender- and age-

sensitive aid, Health, Humanitarian air services, Needs assessments, Nutrition, Protection, Resilience,

Shelter and settlements, Social protection , Water, sanitation and hygiene.

The sector guidelines are designed to fit all emergency contexts, and focus on the delivery of emergency

services to targeted people. Thus, they must be flexible to the context where the emergency response is

delivered. To be kept in mind is that the ESI implementing partners are the same as for EU emergency

response in third countries, which contributes to ensuring a proper interpretation for and application to the

specific context.

See further flowchart of Annex 1 “Safeguards for spending EU funds and ensure quality under the ESI”

kom (2019) 0133 - Ingen titel

and/or to the lack of sufficient resources of partners to provide for specific services.

Regarding the needs on the islands, it was agreed in 2017, in cooperation with the

national authorities, that the EU instruments AMIF/EMAS would cover the services in

the “hotspots”.

Unavoidable challenges were nevertheless present. One of them was the constantly

changing situation on the ground which hindered the accurate assessment of needs,

capacities and number of beneficiaries to be targeted. In this regard, the constant and

close monitoring undertaken by the Commission field experts played a key role to better

assess the response and the adaptation to the changing conditions on the ground.

Although the evaluation found there were slight discrepancies between UNHCR data on

people of concern compared to the numbers of beneficiaries reached by actions funded

by ESI, the findings also showed that the ESI-targeted beneficiaries aligned well with the

needs assessments carried out throughout the actions and UNHCR data. Nevertheless,

during ESI’s

activation it has been difficult to determine the exact number of persons of

concern given the type of emergency and the constantly fluctuating number of

arrivals/departures to and from Greece.

5.1.3.

LEXIBILITY

OF THE LEGAL FRAMEWORK TO APPROPRIATELY ADAPT THE

OPERATIONAL RESPONSE TO

THE CHANGING CONDITIONS AND NEEDS IN THE

EMBER

TATE CONCERNED

;

AND TO THE CHANGING CONDITIONS ON THE

GROUND

The ESI legal framework was found to be appropriate and flexible for such an

operational response in Greece, given the nature of the emergency and the ever-changing

needs on the ground. This flexibility can be attributed to the framework (design and

implementation) but also to the working method of ESI, where actions were regulated by

the agreements between the Commission and its partners, with the former providing more

flexibility

It must be further emphasised that the Regulation was not designed to specifically

respond to a migration crisis, and thus it could potentially be used for different type of

emergencies within the European Union (please see section 2). The Regulation therefore

provides for a wider scope to ensure the necessary flexibility to adapt to the national and

operational context.

Overall, there was a general agreement by all stakeholders consulted (Commission,

national authorities, Commission’s partners and implementing

partners) that ESI's legal

framework allowed for a rapid and flexible response, just as it is the case for emergencies

in third countries. Such adaptability was evidenced by the reallocation of funding,

modification requests and contingency planning

developed over the course of ESI’s

activation. The set of rules governing the ESI actions allows for the introduction of

modifications in case of changed circumstances. Yet, some elements of the actions

needed prior Commission’s approval in order for these

to take effect or be implemented

(e.g. the funding amount , the duration of the action, the results to be achieved); other

were considered as non-essential changes and therefore these could be unilaterally

modified by the Commission’s partner. For instance,

a partner has the flexibility to make

transfers between different budget headings in their budget.

See further flowchart of Annex 1 “Safeguards for spending EU funds and ensure quality under the ESI”

kom (2019) 0133 - Ingen titel

ESI partners made use of this flexibility, particularly at the beginning of the operations in

2016. The first 17 contracts signed in 2016 in the first months of the activation saw 58

amendments. This mirrored the difficulties of operational planning in the year 2016. In

the following two years, once the initial response had been deployed, the operational

environment was better known and the response could be better structured, the number of

amendments dropped to eight in 2017 and thirteen in 2018.

The Commission considers that the legal framework of the ESI is in principle applicable

to other Member States facing a similar crisis, provided that the Council activates the use

of the instrument. The support study nevertheless highlighted that there is room for

improvement in the case of a future activation, in terms of better mapping the capacities

of local actors such as municipalities and civil society organisations, preferably before an

ESI intervention or at least at an early stage thereof. Indeed some local partners

considered that they had been insufficiently consulted at the very early stage of the

activation. Additionally the study called for a clearer division of responsibilities between

the various stakeholders involved in an ESI activation, for example by early

identification (i.e. at an early stage of ESI intervention) of the key actors and their roles

(at EU, national and local level) to reinforce the sustainability of an intervention in the

long run and in view of improving the coordination and cooperation between all the

actors involved. It must however be noted that operating in crises situations does not

always allow for such a degree of forward planning, in particular as the ESI operates by

definition in a Member State that is overwhelmed.

5.1.4.

XTENT

TO WHICH

ESI

WORKING METHOD

(

CHANNELLING AID THROUGH

HUMANITARIAN PARTNERS

)

WAS RELEVANT TO ADDRESS THE HUMANITARIAN

NEEDS THAT RESULTED FROM THE REFUGEE AND MIGRATION CRISIS

Channelling the actions through the Commission’s partners has ensured a timely and

tailored response to the emerging humanitarian needs of the beneficiaries targeted in

Greece. Moreover, it allowed the Commission to fund organisations which were not

eligible or available through other instruments. International NGOs for instance, which

played an important role in the response, were not funded directly by the Commission

through other instruments. This method was relevant, as it allowed the NGOs to take

swift action to address the most urgent humanitarian needs, given the limited capacities

of the national authorities as well as that of the local and civil society organisations, to

respond to the situation at the beginning of ESI’s

activation. It must be underlined that

the capacity of the national authorities was overwhelmed at the beginning of 2016. The

Member State had to deal with an influx of refugees and migrants that was

unprecedented. Special measures had been taken to cope with the situation such as using

the armed forces to create shelter and provide services, but there was nevertheless a lack

of sufficient humanitarian capacity and expertise in the country, which had not been

prepared for such an exceptional situation. Thus the immediate provision of expertise and

funding was the most viable solution to respond to the emergency situation and to allow

the national authorities to strengthen their structures and to gradually increase the

capacity of national entities including civil society organisations.

This was corroborated by 91% of the Commission’s partners surveyed, who considered

ESI’s working method to be relevant or somewhat relevant to address needs, with none

deeming it irrelevant. Opinions on the possibility of working directly with Member State

authorities were divided, with 48% of respondents indicating it would have been relevant

or somewhat relevant to implement ESI’s actions through the Member State authorities,

while 43% considered it would be somewhat irrelevant or irrelevant. However, it must be

kom (2019) 0133 - Ingen titel

highlighted that the legal basis of ESI does not allow for its actions to be directly

implemented by the Member State authorities, yet close cooperation with the affected

Member State is a requirement (see section 5.2 below). This is consistent with the fact

that emergency support is to be provided by the ESI when a Member State’s capacity to

respond to the humanitarian consequences of a crisis is limited and/or overwhelmed- as

was the case of Greece. In addition, at the time ESI was activated, Greece was also under

an institutional restructuring process, including the creation of the Ministry of Migration

Policy, established by mid- 2016.

The support study also underscored some issues regarding the involvement of local

organisations. These were mainly identified at the initial activation phase, although local

implementing partners considered that they could have been better involved to provide a

better understanding of the local context and needs. The context and situation in Greece

must be again noted in this regard, as there was no Greek organisation which had signed

a framework partnership agreement with the Commission before the beginning of 2016,

and thus it was not possible to directly fund Greek civil society organisations through

ESI. This issue was addressed and the involvement of Greek civil society organisations

gradually increased, on the one hand as leading Commission’s partners increasingly

collaborated with local organisations on the ground for the implementation of the actions,

and on the other hand when selection criteria for Commission’s partners eligible to

receive emergency support funding were adapted in 2017 in order to allow local Greek

organisations to become ESI FPA partners. At the end of 2018, Greek FPA partners and

local implementing partners played a key role in the management of the response and are

expected to continue doing so, provided other sources of funding such as the AMIF

national programme will be accessible.

5.2.

5.2.1.

Coherence and complementarity

ESI

ACTIONS ADHERED TO THE HUMANITARIAN

XTENT

TO WHICH THE

PRINCIPLES

The actions funded by ESI were implemented in respect of the humanitarian principles of

neutrality, humanity, impartiality and independence, as required by Article 3 of Council

Regulation (EU) 2016/369. This was validated by the support study findings, which

showed that the Commission assessed both at the proposal stage and during the

consistent monitoring of the activities, the adherence of the proposed actions, their design

and implementation to the humanitarian principles.

The majority of stakeholders consulted (Commission, national authorities, Commission’s

partners and implementing partners) highlighted that no discrimination among the

beneficiaries of ESI actions has been found. On the contrary, every person who was

seeking asylum in Greece was eligible for ESI funded assistance. Furthermore, the

Commission’s partners implementing ESI actions all had a solid track record of applying

the humanitarian principles, as they have undergone a robust certification

. Finally, both

NGOs can apply for the signature of Framework Partnership Agreement for humanitarian aid and

emergency support if they comply with the criteria established under the Humanitarian Aid

Regulation, the Financial Regulation and on the basis of its Rules of Application. The selection criteria

consist of a number of eligibility and suitability criteria. The full set of criteria, the applied

benchmarks, the application form and the supporting documentation are available at

http://dgecho-

partners-

kom (2019) 0133 - Ingen titel

exit strategies and handover arrangements to the national and local authorities (including

municipalities) included capacity-building activities with explicit reference to the

humanitarian principles. However, the implementation on the ground continued to be

affected by fluctuating numbers of beneficiaries, security challenges and the lack of

accommodation facilities, which was progressively tackled through establishment of new

camps/sites.

Despite the challenges faced and the complex situation involving partners, beneficiaries

and national authorities, ESI proved to be a relevant and flexible instrument to deal with

this exceptional situation in an EU Member State. The majority of the stakeholders

consulted

(i.e. Commission, national authorities and Commission’s partners) considered

that actions funded by ESI respected the fundamental humanitarian principles. However,

the support study underlined that some of the Commission’s partners perceived some

challenges while applying the principles of independence and neutrality. This issue was

mainly related to the requests and decisions made by the national authorities on the

delivery of humanitarian aid

–

including for example the location and establishment of

camps/sites. While the selection of sites, in particular of temporary facilities, was

considered sometimes as unsatisfactory (since the camps/sites were located in remote

places or needed to be developed) at least at the beginning of the activation, balanced

decisions had to be made between the need for shelter and meeting national standards.

Such decisions however, together with their assessment, had to be done in cooperation by

the national authorities and negotiated with local authorities both for practical purposes

and given the legal requirement for such coordination in the ESI Regulation.

In addition, it should be noted that working in an EU Member State is inevitably different

from working outside the EU. Within the EU, and according to the principle of loyal

cooperation, the Commission has an obligation to cooperate and coordinate with the

Member State concerned, also in accordance with Article 3 (4) of the ESI Regulation (a

more stringent requirement than the one applicable in third countries). At the beginning

of the activation, this particular obligation presented a challenge for humanitarian

partners, in particular NGOs, which typically limit their interaction with governmental

authorities in third countries to a minimum. In the case of Greece this was different. The

national authorities (i.e. later the Ministry of Migration Policy) were at all times in the

lead of the response, even if their necessary legislative framework and sufficient

capacities were not fully developed at the beginning of the crisis but gradually developed

throughout ESI’s activation. Greece, as an EU Member State, took the necessary

decisions to shape the national response in the medium and long-term, also in view of

taking over the ESI operations by 2019. Some partners interpreted this as a limitation of

their operational independence, an issue also raised in the support study. In the

Commission’s view this was not the case, as the actions developed by ESI were in full

respect of the humanitarian principles (as also proven by the evidence provided in the

support study), therefore the limitation raised by the partners was eventually a matter of

diverging views on certain aspects of the response.

At the same time, the Regulation explicitly states that close cooperation and consultation

with the affected Member State has to be ensured. Given that the ESI was a new

instrument, through which emergency humanitarian-style support has been provided for

the first time in the EU, the Commission’s partners faced a different situation and

helpdesk.eu/_media/become_a_partner/fpa_application_questionnaire_for_website_en_13_02_2015.p

kom (2019) 0133 - Ingen titel

context. They also had to learn how to work in a new context and with the national

authorities of an EU Member State, and

vice versa.

The cooperation and collaboration of

all the actors involved was nevertheless substantially improved during the

implementation of actions funded by ESI. In this regard, stakeholders consulted (i.e.

national authorities, Commission’s partners) highlighted the Commission’s coordinating

role (particularly DG ECHO with regards to ESI-funded actions) to achieve an effective

working method between all the relevant donors and organisations involved at EU and

national level. In addition, the Commission (DG ECHO in particular for ESI-funded

actions) played a key role in mediating between the traditional humanitarian actors

positions and those of the national authorities which were driven, by nature, by strategic

imperatives for example for what concerns the locations of camps (a scattered location of

small camps has prevailed in order to better distribute the burden between different

regions of the country).

In the opinion of some of the Commission’s partners consulted, the limited intervention

on the islands could be perceived as compromising the principle of humanity. In this

regard, it must be noted that both ESI and AMIF funding for actions in the islands was

available and implemented throughout the period of the support study. However,

attempts to significantly increase reception capacity on the islands were unsuccessful.

For instance EUR 3 million for UNHCR from the ESI budget were committed for the

creation of an additional reception facility on the island of Chios. As no agreement on the

location for the facility was reached, the latter was never built. As of 2017, ESI’s main

focus would therefore primarily concentrate on providing assistance on the mainland in

order to create the necessary capacity for the decongestion of the islands, which was the

best strategy to cope with the low but continuous influx. This agreement is reflected in

the Financial Plans for 2017 and 2018 which present the responsibilities of the various

EU instruments as regards the financing of activities in relation to the refugee and

migration crisis. This division of tasks was appropriate to guarantee coordination and

ensure complementary, EMAS funding were already scaling up the response on the

islands, where the migration hotspots were concentrated. As of 2017, there was a clear

need to boost the response in the mainland by creating sufficient hosting capacity to

facilitate the transfers from the islands and, as a consequence, to alleviate the pressure on

islands.

5.2.2.

XTENT

TO WHICH THE

ESI

IS COMPLEMENTARY TO OTHER RELATED

OR NATIONAL INSTRUMENTS AND OVERLAPS IDENTIFIED

AND

Article 6 of the Regulation provides that the ESI funded interventions must seek

synergies and complementarities with other EU Instruments. In this regard, ESI has

overall achieved a high level of coherence with other relevant EU instruments and it did

not have any evident overlaps with other EU instruments that could deliver assistance to

Greece.

When analysing the extent to which these EU instruments either overlap or complement

the ESI, the following differences are noted: the instruments of Humanitarian Aid

(Council Regulation 1257/96) and the UCPM (Decision No 1313/2013/EU) have guiding

principles that are very similar to ESI (provision of relief to most affected by cases of

man-made and/or natural disasters). Nevertheless, the geographical scope of the former is

different, as the Humanitarian Aid Regulation foresees a geographical limitation:

assistance can be provided

outside

the Union only. On the other hand, the UCPM is

mainly limited to in-kind assistance and/ or human resources (teams of experts) and it

depends on Member States' voluntary contributions to the affected country. Assistance

kom (2019) 0133 - Ingen titel

has to be requested by the affected country and delivered by specialised Member State

agencies. In the case of Greece, the Mechanism was activated twice in 2015 and 2016

with over 200 000 in-kind donations of relief items and equipment from over 20 other

EU Member States. By the time the emergency support Regulation was proposed by the

Commission, it was clear that the UCPM had reached its limits and further substantial

assistance, in terms of further capacity to provide additional humanitarian assistance for

the mid and/or long term, could have not been provided by other Member States, who in

many cases were also hosting a high number of refugees and migrants.

The EUSF (Council Regulation 2012/2002) targets natural disasters only, thus excluding

man-made ones, under which category falls the flow of refugees and migrants

experienced in Greece since 2015, and for which the ESI was activated in March 2016.

The FEAD (Regulation (EU) 223/2014) focuses on reducing poverty in the Member

States. The third EU Health Programme (2014-2020) also supported actions to address

the migrant health issues in Greece, amongst other countries.

Finally, AMIF (Regulation (EU) No 516/2014) and the ISF (Regulation EU 513/2014

and Regulation EU 515/2014) do not particularly focus on the provision of humanitarian

aid, emergency relief or poverty-reduction assistance. Instead, AMIF's goal is to support

the management of migration flows and to strengthen a common European approach to

asylum and immigration, mostly through support to the development of structures,

systems and organisational capacities. The ISF on the other hand aims to enhance

security within the EU by both strengthening police cooperation and combating crime, as

well as supporting integrated border management and a common visa policy. The support

study underlined that while some stakeholders (Commission and Commission’s partners)

raised similarities with other EU funding instruments, it is mainly EMAS the instrument

which presents more similarities. In fact,the mandate of EMAS is defined as providing

additional emergency support to address urgent and specific needs arising, inter alia, also

from “heavy migratory pressure in one or more Member States characterised by a large

and disproportionate inflow of third-country nationals, which places significant and

urgent demands on their reception and detention facilities, asylum systems and

procedures”.

Overall the complementary with EMAS has been guaranteed thanks to:



The timing: when EMAS was used to assist vulnerable refugees and migrants in

Greece, ESI was not yet available.

The channels: a significant proportion of the EMAS funding in the 2015-2017

period went directly to the national authorities and to international organisations,

while ESI funding went also to NGOs and the Red Cross family organisations,

partners which are not eligible under AMIF EMAS

The division of responsibilities: DG ECHO primarily concentrating on the

provision of emergency assistance on the mainland of Greece and DG HOME

providing assistance through the National Programmes and EMAS in the island

territories, where the migration hotspots were located.



Under the emergency assistance component EMAS, the basic act for AMIF and ISF do not limit funding

to particular entities. It has been a policy choice to introduce in the annual work programme (AWP) the

principle that eligible applicant organisations can only be Member States, International organisations and

EU agencies.

kom (2019) 0133 - Ingen titel

Furthermore, partner organisations on the ground ensured the complementarity of the

actions with the local authorities and with other international organisations by

demonstrating at action proposals stage to have a full knowledge of the context and of the

response already ongoing on the ground. Proposals have been rejected by the

Commission where a risk of overlap existed. Monthly updates have been requested to the

partner organisations, which allowed the Commission to map the actions by sector and to

continue checking any possible overlap and to guarantee constant coordination.

Moreover, a series of coordination mechanisms between the relevant Commission

services and national authorities were established to ensure coherence between the ESI

and other EU funding instruments. The main result of the effective coordination between

all relevant actors is reflected in the Financial Plans, where the main needs and areas to

be covered by ESI, as indicated and requested by the national authorities, including an

indicative budget breakdown per relevant actor and per EU funding instrument were

provided. As of 2018, a Steering Committee was also established with the involvement

of relevant Commission Services and the National Authorities (i.e. Ministry of Migration

Policy, Ministry of Economy, Ministry of Labour, Ministry of Education etc.).

Furthermore, strict and regular monitoring of the Commission field experts, together with

the coordinating role of the SRSS, also highlighted by many stakeholders consulted,

facilitated the complementarities between the various EU Instruments.

Overall coordination was provided by the SRSS at the institutional level. The

Commission (through DG ECHO) organised regular coordination meetings with the

national authorities and different Ministries, such as the Ministry of Economy and the

newly established MoMP as well as regular meetings with humanitarian organisations.

The coordination at the Commission level also involved regular (firstly weekly later

monthly) coordination meetings between DG ECHO and DG HOME at different levels

of hierarchy as well as exchanges with other DGs for the provision of social services

(funded by ESIF)

–DG

EMPL and DG REGIO in particular.

Finally, no overlaps were encountered with national instruments, as there were no

available national instruments that could possibly target and address the scale of the

humanitarian needs on the ground. This was firmly corroborated by all the stakeholders

consulted, including the national authorities.

Overall, it can therefore be concluded that ESI interventions inside the EU require a high

degree of complementarity and coordination with the existing EU funding mechanisms

and the authorities of the affected Member State. A balance has to be found to ensure that

the experience and expectations from the humanitarian partners are channelled through

the appropriate channels and delivered in a principled, yet effective way.

5.3.

5.3.1.

Effectiveness

XTENT

TO WHICH THE

EGULATION OBJECTIVES HAVE BEEN ACHIEVED

PARTICULAR AS REGARDS MEETING THE HUMANITARIAN NEEDS OF REFUGEES

AND MIGRANTS IN THE DIFFERENT SECTORS OF INTERVENTION

The identified humanitarian needs of refugees and migrants in Greece were met and/or

their situation improved, when compared to the starting point in 2016, as a result of ESI’s

actions. There was an evident upgrade of people’s living conditions in terms of shelter,

WASH, immediate living conditions and their access to cash assistance (replacing in-

kind assistance). For a detailed list of achievements per sector, please see section 3 above

kom (2019) 0133 - Ingen titel

(implementation/state of play). The support study evidence shows that most of the

actions funded by ESI addressed the identified humanitarian needs, achieved and

surpassed their output targets

with many reaching a higher number of beneficiaries than

planned. In summary, the support study provides an analysis of three different aspects

related to ESI actions:



Result indicators of the ESI actions: All the sectors, except for the WASH

activities, have on average surpassed the targets set in terms of the number of

services to be provided, number of sites to be built/ conditions improved or the

number of support mechanisms to be implemented. In 86% of the indicators (266

out of 308) the targets set were achieved or exceeded, and around 60% of such

achievements related to the beneficiary level targets.

Outcomes and improvement of the beneficiaries’ situation: achievements are

mixed. The absolute majority of outcome indicators set in the ESI actions related

to the improvement of the beneficiaries’ situation. The achievement of outcome

indicators set at the project level has been positive, but less so in comparison to

the result indicators.

Number of beneficiaries reached: Overall, the support study states that all ESI

actions have reached a higher number of beneficiaries than originally planned,

and with a very wide coverage of needs and managing to improve their living

conditions if compared to the starting point in 2016. For instance, it is estimated

that the ESTIA multi-purpose cash transfer programme cumulatively had around

104 000 beneficiaries compared to the estimated 114 000- 143 000 people in need

in Greece in the 2017-2018 period. The overachievement of targets however

could be linked to the overly cautious initial estimations, alternatively to the

fluctuations in the number of people arriving in Greece requiring assistance, or

potential double counting in the number of beneficiaries. Due to the multi-

sectorial nature of actions funded by ESI, and to the way project templates are

structured, it is not possible to obtain a number of unique beneficiaries supported

by the ESI funding. The analysis of the average number of beneficiaries reached

per action per gender shows that 60% of reached beneficiaries were women and

40% men on average. Women remained consistently in the majority amongst the

ESI beneficiaries in 2016, 2017 and 2018. Comparing the latter with UNHCR

data on people in need of assistance in Greece (1) shows a different profile, as

amongst the overall number of people in need, 60% were men and 40% women.

This means that the ESI reached more women than in the overall target

population, which is positive as women tend to be more vulnerable in the

emergency situations.

Cumulative numbers of people in need of assistance in Greece, by age and sex, 2017 and

2018

Number

Adults

Children

66,583

44,311

60%

40%

men

women

Number

65,909

44,985

59%

41%



Table 1.

Targets refer to the most vulnerable people as identified by the needs assessments and specified in the

ESOPs. The quality of the response is the focus of the project selection process and monitoring, which

ensure that projects are specified and deliver according to standards and good practice as expressed by the

sector guidelines.

kom (2019) 0133 - Ingen titel

Total

110,894

100%

total

110,894

100%

Source: UNHCR. Population data

–

UNHCR estimates based on enrolment. Cumulative number

of unique people in need for years 2017 and 2018.

In addition, there was a broad agreement between the stakeholders consulted (i.e.

Commission, national authorities,

Commission’s

partners and implementing partners)

that the identified and targeted humanitarian needs in the different sectors of intervention

were met by the actions funded by ESI in a timely manner.

On the other hand, the beneficiaries’ satisfaction varied across the services provided by

ESI’s actions, as pointed out by the support study, particularly regarding cash assistance.

Beneficiaries’

feedback collected by the support study in September 2018

(Figure 7

below), and through other means of verification such as partners-implemented surveys,

indicates that with the support of ESI some of their needs were met to a certain degree of

satisfaction.

On average, across the services received, 50% of beneficiaries surveyed in the evaluation

were either very happy or happy with the services received. However, to be noted is the

varying degree of satisfaction with the different types of support received, with high

approval ratings for services in the education, protection assistance and shelter/

accommodation. Half of the respondents who received healthcare were satisfied with the

support provided while around one third of the respondents were not happy with the

health assistance received. In the shelter sector, in general, those who were happy with

the support received and those who believed that their needs had not been covered were

relatively equally distributed (47% and 42% respectively).

Only a third of the beneficiaries were happy with the food services they received. Cash

services seem to have satisfied few beneficiaries, with about half of the participants not

being happy with the programme and considering that the support received did not

respond to their needs. Only a third of the cash beneficiaries perceived that their needs

were covered with the support received. Mainly beneficiaries complained that the

amounts received were not sufficient.

This finding appears in line with the trends observe in other Commission-funded cash

actions in third countries, and somehow expected as a result to such a direct question,

however the results of the beneficiaries’ survey contradict the feedback provided by ESI

partners and the results of monitoring missions undertaken by DG ECHO.

It is important to underline that the cash amounts distributed were established as a

contribution to the beneficiaries’ family basket, and not meant to fully substitute a

family’s income, as the question of the mini survey undertaken by the support study

seems to suggest. Furthermore, the amount provided corresponds to the amount given to

vulnerable Greek families by the social protection system and that the Minimum

Expenditure Basket was agreed between members of the Cash Working Group in Greece

and the Greek Government. Additionally, in order to pursue an objective of

sustainability, and allow a possible handover/takeover of the cash assistance at the end of

ESI operations, the Commission and the Greek Government decided to align the cash

transfers, so as to equal the amounts provided by the Social Solidarity Income to the

The survey link was sent to 10,901 households (i.e. to the head of the household) of the ESTIA

programmes covering the five languages of the survey through UNHCR, 6,803 households received it

(63%) and 1,788 responded to the survey. After cleaning the data and removing duplicates, 1,316 responses

were considered valid representing a response rate of 19%.

kom (2019) 0133 - Ingen titel

Greek citizens, and to also avoid inequalities between the beneficiaries and the Greek

citizens. The results of the survey may however indicate that the amounts are too low and

it is a matter that may require further analysis.

Figure 7. Does the support you receive (good and services) responds to your needs and (if relevant) the

needs of your family/ (difference of views between respondents on the mainland and the islands)

5.3.2.

NABLING

OR HAMPERING FACTORS INFLUENCING THE ACHIEVEMENT OF THE

OBJECTIVES

As described above in section 3 (“evolution of programmatic priorities”) the ESOPs were

constantly updated and adapted to the evolving needs and established priorities. The

support study concluded that the flexibility of ESI was one of the main enabling factors

for the achievement of the objectives. It allowed to adapt the ongoing actions to the

changing conditions on the ground, which amongst others included the fluctuating

numbers of beneficiaries. In addition, the swift and holistic approach of the actions

funded by ESI to address the needs of beneficiaries and their engagement in the delivery

of assistance were also highlighted as enabling factors. The clearest example of such a

flexibility approach is represented by the choice to support the systemic shift from in-

camp to urban accommodation policy and from in-kind assistance to cash.

A number of challenges were also identified, both at the strategic and operational level,

which influenced the achievement of the objectives. According to the support study,

kom (2019) 0133 - Ingen titel

more than half of Commission’s partners and implementing partners (58%) indicated

having encountered challenges while implementing their ESI-funded actions in Greece.

This is corroborated by the analysis of the project final documentation (Single Forms)

where the framework partners identified having faced challenges affecting the

implementation of the ESI actions in 22 out of 29 actions. The most frequently

mentioned challenge, across the various sources of evidence, relates to:



At the strategic level:

absence of a plan and/or strategy

at an early point of the

crisis by the national authorities to respond to the large influx of refugees and

migrants into the country. There were also limited adequate resources and

experience in emergency assistance and asylum issues by the national authorities,

to steer the activities in a consistent manner. This has improved as the ESI

implementation progressed, especially with the establishment of a single Ministry

responsible for migration matters, which did provide the necessary guidance as of

2017.



At a more operational level:

Framework partners faced a situation of

poor and fluctuating data on the

beneficiaries

and the nature of their needs, which made it difficult to plan the

individual project actions. This has somewhat improved over time as the

migration situation has stabilised and better data emerged from the national

authorities and UNHCR.

National authorities were considered to insufficiently manage the

security

the various mainland sites. Over time, the situation was addressed and has

improved.

Several

procedural and administrative obstacles

were encountered,

including delays in obtaining permits to access locations and start planned

actions, legislative and administrative challenges as procedures did not always

permit the swift action that is needed in an emergency context, which in turn

led to delays in the provision of services by the ESI actions. Examples given

included challenges in determining site locations and the numbers of refugees

and migrants per site, with changing plans causing delays and changes in the

implementation of actions.

However, as actions funded by ESI matured, the positive effects of developing

constructive relationships between the national authorities in Greece, local NGOs, local

authorities and other partners at the local level was also stressed by the support study.

The cooperation with the Greek municipalities played a key role for the activities

implemented by ESI, in particular as part of the ESTIA programme.

5.3.3.

XTENT TO WHICH THE

CTIONS FUNDED BY

ESI’

IMPLEMENTATION METHOD

(

CHANNELLING AID THROUGH HUMANITARIAN PARTNERS

)

ALLOWS TO

EFFECTIVELY ADDRESS THE HUMANITARIAN NEEDS THAT AROSE AS A RESULT OF

THE REFUGEE AND MIGRATION CRISIS WITHIN THE

Channelling the actions through the Commission’s partners

allowed ESI to timely and

effectively fulfil the humanitarian needs of the targeted beneficiaries (see section 5.1.4).

Overall, the working method was pertinent and effective when considering the numerous

challenges faced by both the national authorities and the local civil society organisations

to provide a rapid, coordinated and effective response to the emergency.

kom (2019) 0133 - Ingen titel

This view was also shared by the Commission’s partners and implementing partners as

most of them (90%) agreed that channelling ESI’s actions

through humanitarian partners

was effective or somewhat effective. Only 35% consider that it would have been

effective or somewhat effective doing it through the Member State authorities.

Figure 8.

Share of respondents that considered having ESI’s actions implemented

by Member State

authorities vs. Commission’s partners would allow for a good, timely and effective response

The support study findings also showed that the close presence and extensive follow-up

by the Commission experts in the field, was instrumental in the effectiveness of ESI

overall.

5.3.4.

XTENT TO WHICH THE

CTIONS FUNDED BY

ESI

FOLLOWED THE

VISIBILITY

CRITERIA AS SET OUT BY THE

OMMUNICATION AND

ISIBILITY

ANUAL FOR

UROPEAN

NION

FUNDED

UMANITARIAN

CTIONS

ESI funded actions provided a clear communications and visibility plan to ensure the

relevant activities met the EU visibility Criteria for EU funded Humanitarian Aid actions.

With the activation of ESI in March 2016, communicating the impact of EU actions in

Greece towards EU taxpayers, beneficiaries, local communities and stakeholders became

essential. Commission’s partners overall adhered to the communication and visibility

requirements, while the Commission provided continuous guidance and monitoring,

leading to significant progress throughout ESI activation.

This was corroborated by the actions mapping done by the support study, which found

that while the visibility of ESI funded actions varied, overall Commission’s partners

applied the visibility guidelines extensively, with many of the partners going beyond the

mandatory visibility requirements. Although some issues were encountered during the

first months of actions funded by ESI as underscored by the support study, these mainly

related to security concerns (especially shortly after the EU-Turkey Statement). The

visibility however gradually improved over time. As shown by Figure 5 below, 95% of

the surveyed partners reported both being aware of the Commission’s visibility

requirements and having managed to implement these under the actions funded by ESI,

15% of respondents reported having faced obstacles while doing so, whereas most of

them (80%) stated that they did not face any obstacles.

kom (2019) 0133 - Ingen titel

Figure 9.

Share of awareness on the Commission’s visibility requirements, their implementation and

obstacles

5.4.

5.4.1.

Efficiency

XTENT

TO WHICH THE

CTIONS FUNDED BY

ESI

WERE IMPLEMENTED IN A

TIMELY AND COST

EFFECTIVE WAY

TAKING ACCOUNT OF THE SPECIFIC

CONDITIONS OF THE EMERGENCY OPERATIONS IN

REECE

? - A

HAT

FACTORS AFFECTED THE EFFICIENCY OF THE RESPONSE AND TO WHAT EXTENT

Overall the actions funded by ESI are considered to be cost-effective and the budget

allocation was appropriate to meet the humanitarian needs. Furthermore, the efficiency

the actions overall improved over time. For example, while in 2016, ESI’s focus was to

primarily respond to the emergency in a timely manner, concentrating on lifesaving

activities, in 2017 and 2018, the Commission adopted a different approach to the crisis to

avoid gaps and overlaps and consolidated the response by reducing the number of

partners.

As a result, the 2017 ESOP promoted the “one partner per sector and per site”

strategy avoiding duplication of activities. Similarly to findings of the effectiveness of

the actions, the support study underlined that the flexibility of ESI also contributed to

improve the efficiency of actions funded by ESI, by allowing the Commission’s partners

to adapt to the changing needs on the ground. For example, evidence on the flexibility

has shown that modification requests allowed the Commission’s partners to adapt

the

actions to evolving needs, in particular for 2016 contracts. The majority of partners have

requested modifications during the implementation of their actions. In 60% of the cases

these concerned an adaptation of the strategic operations, such as upgrading the container

sites, adapting the food offer based on the needs of the beneficiaries, scaling up of winter

kits distribution or increasing the legal aid provision to beneficiaries. The Commission’s

partners also requested adaptations to the timing and budget of their actions. The

flexibility in modifying and extending the duration of the ESI actions is a welcome

element that was raised by most of the partners as improving the cost-effectiveness of the

response.

The assessment provided by the support study was mostly based on qualitative indicators

and stakeholder views, mainly given the fact that ESI is a relatively new EU support

instrument and it has only been activated, for the first time in Greece. Hence, there is no

basis for comparison with previous EU interventions in any other Member State. The

support study however tried to provide quantitative evidence by developing a series of

indicators to look at the cost-effectiveness of similar crises targeted by EU humanitarian

aid actions, however implemented in third-countries, such as Egypt, Jordan, Lebanon,

and Turkey. Through this exercise different levels of economic development were taken

kom (2019) 0133 - Ingen titel

into consideration by weighing the costs using final consumption expenditure (FCE) per

capita and GDP per capita.

The support study developed three indicators to assess the costs effectiveness of ESI:



The first indicator compared the cost per beneficiary of ESI in Greece with the

cost per beneficiary of other Commission (DG ECHO) interventions in Egypt,

Jordan, Lebanon and Turkey. The costs in the group of countries of reference was

then indexed to the consumer price level of Greece (using the FCE).

The second indicator provided a comparison of the alpha ratio across the

countries and indicated the share of the cash and in-kind modalities that is directly

transferred to the beneficiaries. A higher alpha ratio indicated better cost-

effectiveness.

The third indicator, finally compared the average share of specific costs per

project in Greece and in the other countries, with a higher share of specific cost

indicating a better efficiency of the actions.



As a result of such exercise, the support study concluded that the average cost of

implementation per beneficiary of the emergency support activities in Greece was lower

than in the other countries (EUR 792 vs EUR 1 375) once the prices were adjusted to the

Greek prices level (FCE). The analysis however showed different level of cost-efficiency

depending on the different sectors, hence suggesting that while a high cost-efficiency was

found in some sectors (e.g. health services, basic needs assistance (e.g. MPCT) and Food

Security and Livelihoods (FSL)), more efficiency could have been achieved in other

sectors (e.g. protection and accommodation services). This first basis for comparison,

however should be interpreted with caution as other factors such as the country taxes, the

price of the goods bought on international market, exchange rates and import taxes, etc.

were not considered. Beyond the current exercise of evaluating the activation of the ESI

for Greece, some of the calculations done in this context could usefully serve as an input

to further work on assessing cost-effectiveness.

Figure 10. Average cost per beneficiary, per sector, 2016 - 2018

The support study findings overall showed that actions funded by ESI were

proportionally less efficient in 2016 due to higher support costs to initiate all the

activities in Greece (set up of camps/sites etc.) however the efficiency improved in the

subsequent years. This is consistent with the situation in Greece at the beginning of the

kom (2019) 0133 - Ingen titel

activation, as the Member State had not seen a previous and major presence of

humanitarian actors, and the Commission’s partners first had to establish a presence on

the ground.

In addition, an indication of the efforts made to improve the efficiency of the actions, is

reflected in the Commission’s application of strict monitoring requirements on ESI

actions. For instance, UNHCR was required to conduct monthly certification of refugees

and migrants, something which is not needed as part of humanitarian responses

elsewhere. This specific requirement overall helped to keep track of the extent to which

targets were being met within the available budget. Throughout the project cycle the

Commission (DG ECHO) undertook regular site visits to monitor the implementation of

the ESI actions, including close scrutiny of ongoing activities and making

recommendations to improve efficiency. This was also reflected in some of the actions’

modification requests whereby specific suggestions were made to improve the efficiency

of the activities and ensure that lessons learned from previous visits/ actions were taken

into consideration. In addition, Commission’s field experts played a key role in following

the implementation of technical solutions on the ground and in proposing more cost-

effective alternatives where and whenever necessary. For instance, on several occasions

expensive technical solutions were revised by the Commission’s experts to reduce the

costs.

Coordination

was also key to ensure efficiency. The support study’s stakeholder

consultation findings show that one effort to ensure cost-effectiveness was the UNHCR-

led inter-agency coordination mechanism, with all relevant humanitarian organisations,

as it allowed to better coordinate activities implemented by different organisations and

avoid overlaps, as well as for both the best practices and ‘know-how’ exchange.

In addition, the support study findings also identified other measures that the

Commission’s partners

and local implementing partners put in place to ensure and/or

maximise the cost-effectiveness of their ESIs actions (see also Figure 11). The main

measures included:



Cooperation with the private sector. For instance, the private sector was involved

through civil engineering and architectural studies.

The use of innovative method and new technologies. For instance, Mercy Corps

developed an App called ‘refugee.info’ which

gave beneficiaries access to a range

of services in their desired language.

Partner monitoring systems, including the use of efficiency-related indicators.

Community based participatory approaches to allow beneficiaries to support

themselves and their peers.

kom (2019) 0133 - Ingen titel

Figure 11.

Measures implemented by framework partners and local implementing partners to

maximise the cost-effectiveness of actions funded by ESI

Regarding the factors that affected the efficiency of actions funded by ESI, these were

both of external and internal nature. The external factors, as by the document review and

the stakeholders consulted, included the overall overwhelmed national capacity and lack

of a well-defined national strategy in 2016 to respond to the influx of migrants and

refugees, paired with the limited infrastructure and services to deal with refugees and

asylum seekers, as well as the national requirements and standards set by the national

authorities regarding the setting up of sites/camps, which increased the building,

administration, transportation and logistical costs. Some examples provided by the

support study related to the dispersion of camps/sites, their capacity, the use of private

land or private buildings, the national standards requests on specific model of containers,

the last-minute closure of camps already set-up, among others. Furthermore, the costs of

operating in Greece were higher when compared to EU interventions in third countries,

as ESI had to adhere to EU and national legal frameworks and procedures. Challenges

identified

were particularly related to the Member State’s specific context and legal

framework, such as a high level of taxation and social security obligations resulting from

Greek labour law, lack of sufficient provision of public services close to sites which lead

to additional transportation costs, amongst others.

On the other hand, internal factors affecting the efficiency, were also identified by the

support study. It should be however noted that the Commission mitigated such

challenges by requesting its partners to undertake certain measures to improve the

efficiency of their actions. These included for example: the progressive reduction of

expatriate staff and instead increase the employment of local staff and volunteers,

(including migrant volunteers), grassroots movements; the creation of partnerships with

local NGOs (a total of 69 different implementing partners participated in ESI’s actions,

25 local NGOs and 10 local authorities); strict monitoring of ESI’s actions to boost cost-

effective alternatives (e.g. UNHCR monthly certification of refugees and migrants in

ESTIA).

Finally, coordination between the relevant actors (including partners, local organisations

and local authorities, such as municipalities) was also key to ensure the efficiency of

actions funded by ESI. Such coordination clearly improved in 2017 and 2018.

kom (2019) 0133 - Ingen titel

5.5.

5.5.1.

EU added value

ADDED VALUE OF THE

ESI

ACTIVATION IN

REECE

- W

HAT IS THE

SPECIFIC

ADDED VALUE

CTIONS FUNDED BY

ESI

AND WORKING METHODS

REECE

ESI demonstrated a clear EU added value given that its actions could not have been

implemented by any other national instrument and/or actor - as there was no national

equivalent instrument available that could have successfully addressed the needs, nor did

the concerned Member State have the capacity to address the scale of the resulting

humanitarian crisis.

Although other EU funding instruments and tools were active in Greece (e.g. AMIF/

EMAS), overall during its three year activation, ESI addressed a clear gap in the

provision of humanitarian emergency support, as it provided a timely support, for large-

scale actions to address the basic needs of beneficiaries in an emergency context, by

channelling the response through certified humanitarian partners, which could not have

been funded directly by other EU instruments and thus providing a holistic response. As

shown by Figure 12 below, the vast majority of the Commission’s partners and

implementing partners (78%) considered that the emergent needs - resulting from the

sudden influx of refugees and migrants into Greece- would have not been met without

the activation of ESI.

Figure 12. Share of respondents that considered actions could not have been implemented without ESI

and/or the needs could not have been met in Greece without ESI

Share of respondents who considered that actions

could/could not have been implemented by the

concerned Member State

Share of respondents who consider that the emerging

needs in Greece could/could not have been met

without ESI's activation

The main specific characteristics, as also highlighted by the support study, that boosted

the added value of actions funded by ESI were notably: ESI's focus on humanitarian

support; the flexibility of the instrument to adapt to evolving needs; the fast response by

which ESI was activated and the actions then implemented; the expertise of

Commission’s partners and consequently the transfer of knowledge and the capacity

building component included in all ESI’s actions (for both local organisations

and

national authorities);

as well as the Commission’s expertise and technical assistance on

the ground. ESI replicated the Commission’s successful and unique

modus operandi

deploying humanitarian field experts to Greece (by DG ECHO). The three Commission

Greek-speaking field experts deployed to Greece were the interface between the

Commission, the Greek authorities and the partners. Thanks to their long-standing

expertise in managing and monitoring the implementation of humanitarian operations in

kom (2019) 0133 - Ingen titel

the field, they were able to address implementation challenges directly, which improved

coordination and speeded up implementation.

5.6. Sustainability

5.6.1.

XTENT TO WHICH THE

ESI

ACTIONS CAN CONTINUE AFTER THE END OF THE

THREE

YEAR ACTIVATION ON THE BASIS OF NATIONAL FUNDING

AND RELYING ON

THE NATIONAL PROGRAMMES UNDER

AMIF

AND

ISF,

OR OTHER SOURCES OF

FUNDING

(

SUCH AS

EUSF, ESF, ERDF

AND

FEAD)

The ESI activation is due to end in March 2019, thus at the time the support study was

undertaken actions funded by

ESI were still ongoing. Similarly, while some of ESI’s

actions were handed over to the national authorities (e.g. shelters for UAMs, primary

healthcare) the transition process is still ongoing.

Although sustainability and exit strategy are not one of the main objectives, and not

formally referred to in the ESI Regulation, the support study nevertheless showed that the

Commission did undertake a number of measures to ensure that partners considered

sustainability of their actions throughout the ESI activation period. As of 2017 the

Commission consolidated the ESI operations by reducing the number of partners and

streamlining sectoral interventions. For example, the 2018 ESOP did consider and

referred to the sustainability and exit strategies. In addition, the Commission has

continued to stress the sustainability of the actions as part of the coordination

mechanisms (involving other Commission services and national authorities). The

analysis of actions

funded by ESI also showed that Commission’s partners considered

sustainability in the design of their actions. The latter particularly focused through the

collaboration with local implementing partners, training and capacity building of local

authorities and organisations, and capacity building of beneficiaries.

While overall, it is therefore too early to assess the long-term

sustainability of ESI’s

actions, the following actions have nevertheless taken place to enhance the short- and

longer-term sustainability the actions:



EU funding has been ensured for the continuation of the actions, mainly through

AMIF/EMAS, and the Commission has been working to ensure a smooth

handover of the actions between its services. On 20 December 2018 the

Commission (DG HOME) signed grants with UNHCR and IOM to continue the

financing of the ESTIA programme and the site management support in 2019.

Moreover, the ESTIA programme was set up in a way to facilitate handover to

other funding sources and the national authorities.

A better and matured coordination has resulted from the implementation of

actions funded by ESI between Commission services, national and local

authorities Commission’s partners and local organisations. Cooperation with local

authorities included the municipalities, which was a key cooperation factor for

ESI’s activities, in particular under the ESTIA programme. Monthly meetings of

a steering committee chaired by DG HOME with the participation of the Greek

authorities, UNHCR and IOM monitor progress in handing over responsibilities

to the Greek authorities.



kom (2019) 0133 - Ingen titel



As explained in section 5.5 above, there has been an embedded transfer of

knowledge and capacity building through actions funded by ESI, provided to both

local organisations and national authorities.

Positive examples highlighted by the support study also included: the use of a transitional

period for the successful handover of activities as part of actions funded by ESI exit

strategies and improved pre-conditions for the integration into the host society of

beneficiaries being granted an international protection or humanitarian status. All these

elements play a key factor for the sustainability and continuance of the actions.

It should also be noted that additional support, outside the scope of ESI, has been

provided in parallel by the Commission to the national authorities, with a twofold aim:

on the one hand to move from a crisis/emergency mode (following three-year activation

of ESI) towards a more stable approach to manage the refugee and migrant influx; and on

the other hand, to strengthen the national administrative and operational capacity (mainly

that of the Reception and Identification Service (RIS) and the Department for the

Protection of Asylum Seekers (DPAS) under the Ministry of Migration Policy (MoMP))

to enable them to provide accommodation and basic services to asylum seekers and

refugees. This support goes hand in hand with the view and aim to ensure the

sustainability of the actions implemented by ESI in Greece.

The challenges regarding sustainability raised by the stakeholders consulted in order to

ensure the sustainability of actions funded by ESI, remained the lack of a well-defined

national strategy, including specific sectoral strategies, to continue to respond to the

needs, as well as the limited implementation by the national authorities of integration

measures. In the Commission’s view, these challenges are part of the situational context

of Greece and are considered as ‘external’ since they are not embedded in the nature of

the ESI and/or its Regulation. The Commission nevertheless continues to work on the

handover of the operations with all relevant actors.

6. C

ONCLUSIONS

The objective of the evaluation of Council Regulation 2016/369 of 15 March 2016 on the

provision of emergency support within the Union, was to assess, on the basis of its first

activation, whether it was suitable to support EU Member States in the humanitarian

response to an overwhelming emergency. The evaluation did not assess wider

implications of the aid provided given the urgent nature of the needs addressed and their

evolving nature.

The evaluation found that Council regulation 2016/369 allowed the Commission to

rapidly react to an emergency in a Member State that was overwhelmed by the crisis, by

mobilising the support of humanitarian partners and possibly Member States’ specialised

agencies through EU funding. The results of this evaluation and the support study

indicate that the legislative framework is suitable and flexible enough to support a

Member State and express European solidarity by building on the humanitarian expertise

of the European Commission.

The evaluation indicates that none of the challenges encountered were a direct

consequence of the legislative framework of ESI. At the same time the support study and

the Commission’s evaluation confirmed that a humanitarian intervention

in a Member

State poses challenges of an operational nature. However, in the Commission’s

experience, some of the challenges are also encountered in non-EU interventions. In the

case of Greece, some of them can be explained by the fact that the Regulation was new

kom (2019) 0133 - Ingen titel

and hence activated for the first time, while others related to the need of better

coordination and planning. As these challenges may occur in future activations, in

particular as the Regulation is aimed to be activated for a Member State that would be

overwhelmed by a crisis, in future activations it will be therefore necessary to address

such challenges, as early as possible, at the outset of the activation and in close

cooperation with the affected Member State.

Based on the findings of the

support study, the Commission’s overall conclusion is that

the Emergency Support Regulation is a suitable tool to address large-scale humanitarian

emergencies within the European Union. The activation of the Regulation and the actions

implemented as a result of the activation in Greece have achieved its objectives, and ESI

has therefore allowed Greece to address the emergency humanitarian needs which it

faced as a result of the refugee and migration crisis. The activation of ESI will cease in

March 2019, thus there is an on-going and well-advanced transition of the activities to

national authorities, but ESI remains a valid EU emergency support instrument in the

future.

6.1.

Relevance

The evidence provided shows that the intervention of ESI has been both relevant to

respond to the migration and refugee crisis in the EU, and particularly relevant to address

its consequences in Greece, as it provided the much-needed additional capacity for the

affected Member State to swiftly respond to the particular scale of the emerging

humanitarian needs. The vast majority of the stakeholders consulted agreed that ESI’s

working method has also proven to be relevant for the implementation of humanitarian

actions, allowing international organisations and NGOs to provide a fast and effective

response to the crisis, and at the same time support the Member State’s national

authorities to address the humanitarian needs. As a result of the lessons learnt throughout

this first and so far only activation of ESI in the EU, there is nevertheless room for

improvement in the case of a future activation, such as, ensuring a faster involvement of

the local organisations already active in the field, and avoiding duplication and/or

decentralisation of activities. On the other hand, ESI’s legislative

framework has allowed

the actions to evolve and adapt according to the changing needs of the crisis, hence

showing the flexibility of the instrument. The flexibility demonstrated by ESI and its

legal framework, shows that, in principle, ESI is applicable to other Member States and

other type of emergencies.

6.2.

Coherence

Internal coherence of ESI and its implemented actions is deemed satisfactory. The

evidence provided by the support study shows that ESI has complemented the actions

undertaken both by the national authorities, as well as by other EU instruments and no

major overlaps were encountered during the implementation of actions funded by ESI.

Overall, while some coordination challenges were faced at the early stage of the ESI’s

activation, the coordination mechanism established between the different Commission

services (SRSS, DG HOME and DG ECHO), Commission’s partners and national

authorities matured and improved over the activation period and was a key factor to

avoid overlaps. In particular, the coordination role of the Commission between all

relevant actors and the adoption and implementation of the Financial Plans, were

considered as a key achievement that positively contributed to the coordination of the EU

response to the refugee crisis.

kom (2019) 0133 - Ingen titel

External coherence was also satisfactory, as actions funded by ESI were designed and

implemented in compliance with the four humanitarian principles of humanity,

neutrality, independence and impartiality. The Commission consistently monitored the

implementation of the actions ensuring their adherence to the humanitarian principles.

All actions were implemented by experienced humanitarian organisations, which fully

adhere to the principles. In some cases there were divergent views on the design of the

response, and it was not always possible for the Commission to fully reconcile the

positions of humanitarian partners with those of the Greek authorities. But every effort

was made to find viable solutions. In the Commission’s view these challenges and

decisions did not jeopardise the operational independence of humanitarian partners.

6.3.

Effectiveness

Both the qualitative and quantitative evidence showed that actions funded by ESI were

overall effective and have addressed the identified humanitarian needs. The vast majority

of the stakeholders consulted considered ESI as an effective instrument, which reached

its objectives in providing immediate and speedy relief to the targeted beneficiaries.

More importantly, the situation of the latter improved when compared to the starting

point in 2016. This was linked to the fact that ESI’s working method (i.e. channelling the

implementation of actions through Commission’s partners) also proved to be effective in

order to provide a fast and flexible response. In addition, one of the key factors for the

success of ESI actions, as pointed out by the support study findings, was the regular

monitoring activities by the Commission field experts, ensuring that the needs were

effectively covered.

Furthermore, most of ESI’s actions fully

achieved their outputs and reached a higher

number of beneficiaries than planned. The overachievement of targets however could be

linked to the cautious initial estimations, or alternatively to the fluctuations in the number

of people arriving in Greece requiring assistance. Overall, actions funded by ESI have

positively contributed to preserving lives, preventing and alleviating human suffering, as

well as improving dignity of life of the targeted affected people. By December 2018, the

ESI-supported actions were able to address several sectorial needs of the target

population in the mainland, with the provision of accommodation, multi-purpose cash

transfers, water, sanitation, hygiene services, health and protection and education.

As an example, the flagship initiative ESTIA programme managed to provide housing to

over 50 000 people, as well as pre-paid cash cards to more than 65 000 refugees and

migrants. The cooperation developed with local authorities, in particular with

municipalities, played a key role to reach the main objectives of the programme.

Similarly, the Site Management Support provided by ESI-funded partners allowed to

build-up the reception capacity in the mainland in a more structured, dignified and

regulated way, with the creation of some 20 000 places in permanent camps and some

10 000 to 15 000 temporary emergency accommodation places all over mainland.

The consultation with the final beneficiaries however shows that, although generally

speaking their level of satisfaction regarding the assistance provided by actions funded

by ESI is positive, the results were mixed, as their degree of satisfaction varied across the

services received. Most notably beneficiaries were less satisfied regarding the amount of

the cash assistance provided by

ESI’s actions. It must be however noted that the amount

established for the cash support provided by ESI was agreed between members of the

Cash Working Group in Greece and the Greek authorities. Furthermore, the amount

corresponds to the amount provided by the Social Solidarity Income to the Greek

kom (2019) 0133 - Ingen titel

citizens. While the results of the survey may indicate that the amounts are too low, in the

Commission’s view this is a matter that may require further analysis.

While there were no major obstacles to the effectiveness of actions funded by ESI in

Greece, the support study underscored some challenges such as the overwhelmed

national capacity at the beginning of the crisis, as well as the limited strategy and

coordination of authorities at the early stage of the activation.

In the Commission’s

opinion, such challenges are nevertheless perceived as ‘external’ challenges/factors, as

these are not directly embedded in the nature or legislative framework of ESI, but linked

to the situational context of the Member State. These include for example: at the strategic

level, the volatile situation in Greece at the time of ESI’s activation, coupled with feeble

coordination between national authorities and resulting from the lack of a well-defined

national strategic approach in 2016 to manage the refugee and migration crisis. In this

regard, to ensure the effectiveness of the actions, a close cooperation and coordination

between all the relevant actors was deemed essential in ESI’s intervention in Greece.

Overall, the close cooperation between the relevant Commission services, national

authorities and Commission’s partners was fundamental to facilitate coordination

between all actors involved (in addition to the establishment of a Steering Committee

between the Commission Services and the national authorities). The involvement of the

national authorities and local organisations was likewise identified by the support study

as a success factor.

At the operational level, challenges were encountered regarding the fluctuating data on

the number of beneficiaries. This was however counterbalanced by the flexibility of the

ESI to adapt its actions to the changing needs on the ground, paired to the close

monitoring undertaken by the Commission field experts on the ground. Finally, the

visibility requirements of EU actions were overall achieved and no major issues were

encountered, as Commission’s partners generally adhered and managed to fulfil these.

6.4.

Efficiency

Based on the evidence provided by the support study the efficiency of the actions funded

by ESI was adequate and it improved over the years. The allocation of ESI’s budget was

timely and effective and the budget allocation under the Financial Decisions reflected the

targeted needs and provided a flexible time framework.

While Commission’s

partners did not face major obstacles to ensure the rationalisation of

actions funded by ESI, the cost-effectiveness

was also influenced by the same ‘external’

factors aforementioned, i.e. due to the situational context of Greece, (e.g. the limited

national capacity, strategy and coordination between the national authorities). These

challenges, as highlighted by the support study, were also associated to the following

issues: the limited infrastructure and services to welcome the high

–

and fluctuating-

number of refugees and migrants, the national authorities’ approach to disperse the

camps/sites around the country and the national standards to be met by the services

provided. As a result, the findings of the support study showed that ESI’s actions

were

proportionally less efficient in 2016 due to higher support costs to initiate all the

activities in Greece (set up of camps/sites etc.). This is consistent with the situation in

Greece at the beginning of ESI’s activation, as the Member State had not

seen major

presence of humanitarian actors before, and the Commission’s partners first had to

establish several of their actions from scratch. Nevertheless, the cost-efficiency improved

over the activation period and benefited from the flexibility of the instrument to adapt

and/or adjust its actions according to the changing needs.

kom (2019) 0133 - Ingen titel

Internal factors, on the other hand, were also encountered, such as the high presence of

expatriate staff. These factors were however mitigated by the Commission by requesting

its partners to undertake certain measures to improve the efficiency of their actions. For

example, encouraging partners to increase the employment of local staff and volunteers,

the creation of partnerships with local NGOs, strict monitoring of actions funded by ESI

to boost cost-effective alternatives etc.

While comparing

ESI’s

intervention in Greece to humanitarian response in third

countries, adjustments to contextual factors must be carried out. Thus, the comparison

done by the support study to similar interventions in Egypt, Jordan, Lebanon and Turkey

showed that the average cost of implementation of certain emergency support activities

in Greece was overall lower. However, the analysis shows different levels of efficiency

across the different sectors.

Commission’s partners nevertheless endeavoured to adopt

economies of scale in the design and implementation of the actions funded by ESI and

thus increasing ESI's efficiency over time.

6.5.

EU added value

The intervention of ESI provided a concrete EU added value to address the crisis, given

that there was no available national instrument to respond to the crisis in the affected

Member State. Similarly, the actions funded by ESI could not have been implemented by

any other EU instrument - at least not with a similar scale and/or within the similar

timeframe. Consequently, ESI’s intervention in Greece addressed the emerging

humanitarian needs that could not have been met by neither national nor other EU

instruments.

Four main characteristics of ESI boosted the EU added value of its actions: 1) the

flexibility of the instrument to adapt to evolving needs, 2) the fast response of the

intervention, 3) the adhered “know-how” and expertise on humanitarian actions provided

by ESI’s response throughout the Commission’s partners and services, including the

Commission field experts (DG ECHO) presence on the ground and 4) the rapid

mobilisation of NGOs that could not be funded by any other EU instrument.

Moreover, the EU added value also comprised the transfer of knowledge that ESI funded

actions have provided to the local organisations and to the government actors as a result

of its intervention. The support study nonetheless suggests that the EU added value could

be further boosted by the early involvement of local non-governmental organisations, in

the case of a future ESI activation.

6.6.

Sustainability

In terms of the sustainability of the actions funded by ESI, given that the latter were still

ongoing at the time the support study was carried out and these will end by March 2019

(as stipulated by the Regulation), it was too early for the support study to assess this

specific criterion fully. Nevertheless, although the sustainability of the actions was not

one of the primary objectives of the Regulation, the preliminary findings show that all

Commission’s partners considered sustainability in the design and implementation of

their actions. Similarly, sustainability and exit strategies were reflected in the ESOP 2018

annual strategy.

kom (2019) 0133 - Ingen titel

The experience of the Commission with

nexus

approaches

in humanitarian responses

overall points to many challenges in ensuring the sustainability of the emergency

operations implemented in the field. These are mainly due to the limitations encountered

in the country concerned, such as national capacity, both at financial and infrastructure

level, as well as that of their civil structures and organisations. Moreover, in most of the

cases, one of the main challenges is the absence of other funding instruments to ensure

the continuation or takeover of such operations.

In the case of Greece, the three-year activation period of ESI has marked considerable

progress on services provided to refugees and migrants. At present, the sustainability of

the actions funded by ESI has been ensured mainly by: safeguarding the EU funding for

the continuation of the actions through other EU Instruments (AMIF/EMAS grant

agreements for 2019 were signed with UNHCR, IOM, and UNICEF on 20 December

2018); by enhancing the coordination between all relevant actors, i.e. Commission

services, national and local authorities (including municipalities), partners and local

organisations as a result of ESI’s actions and; by the embedded transfer of knowledge

and capacity building to both local organisations and national authorities. The additional

actions funded through other EU instruments, for example under the SRSS, to reinforce

the national capacity of the authorities, also complement the sustainability element of the

actions implemented by ESI in Greece. Such actions and results provide interesting

lessons learned for nexus approaches elsewhere.

The Commission has overall continued ensuring the smooth handover of ESI’s actions

and to put emphasis on the need for close cooperation and coordination between the

relevant actors, considered essential to guarantee the sustainability of the actions.

6.7.

Lessons learned from the evaluation

Both the support study and the Commission’s evaluation point to the fact that the

activation and implementation of ESI funded actions in Greece has been overall

successful. The establishment and activation of ESI was relevant given the overwhelming

humanitarian consequences in the Member State concerned at the time of the activation,

in order to swiftly and effectively

address a situation of such scale. The Instrument’s

working method and its legal framework have allowed for a flexible, fast response

implemented through experienced humanitarian organisations, who have collaborated

with both the national authorities and local organisations, in return providing them with

capacity building and transfer of knowledge to enhance the sustainability of the actions.

Close cooperation between the relevant Commission services, the national authorities and

Commission’s partners,

as well as the constant monitoring undertaken by the

Commission in the field, have proven to be key for the efficient and effective

implementation of the actions. Overall, consultations at all levels showed that

stakeholders considered the ESI activation and actions relevant and effective to target the

humanitarian needs on the ground and these have also reached.

However, it has to be acknowledged that both the support study and the evaluation have

important limitations. The legal obligation to report back to the Council at the end of the

The humanitarian-development

nexus

approach predicates that poverty, conflict, fragility and forced

displacement are deeply interlinked and must be addressed in a coherent and comprehensive way. In

May 2017 the Council adopted conclusion on the operationalization of the nexus approach. See

https://www.consilium.europa.eu/media/24010/nexus-st09383en17.pdf

kom (2019) 0133 - Ingen titel

activation meant that research for the support study and this evaluation started only two

years after the activation of the Regulation and long before the final outcomes of the ESI

operations could be assessed. Many final reports were still outstanding at the time this

report has been drafted and will only become available later in 2019. While the handover

of activities to other EU funding sources has taken place, medium-term sustainability and

efficiency will have to be assessed on the basis of costs incurred which will only be

known in 2019. Thus, the assessment of the suitability and performance of the instrument

will need to be further complemented in light of the final results.

As ESI is a new instrument, which has been activated for the first time, there are

consequently areas for improvement in case the need for a future activation arises. As

evidenced by the findings of the support study, as well as by the Commission’s

evaluation, the following issues would need to be considered:

1. The activation of ESI should be accompanied by an obligation to develop an

organisational framework outlining roles and responsibilities, tailored to

each country of activation

The support study and the Commission’s own data and qualitative

assessment

indicates that the legal framework is appropriate for the tasks described, despite a

number of external challenges. Nevertheless, in order to ensure a clear division of

roles and a good coordination between all relevant actors involved, since the very

early stage of the activation, ESI should be accompanied by an obligation to develop,

an organisational framework outlining capacities, roles and responsibilities of all

actors involved and tailored to each country of activation.

2. ESI should include an explicit requirement for the development of an overall

exit strategy, as well as specific exit strategies by sector and at action level

While the 2018 ESOP did consider and referred to the sustainability and exit

strategies of ESIs actions and the support

study further confirmed that Commission’s

partners did undertake a number of measures to ensure the sustainability of their

actions- and even though it was too early for the support study and this evaluation to

assess the sustainability of the actions, the Commission considers that in order to

address any foreseen challenges regarding the sustainability of ESI’s actions, in the

case of a future activation, the Commission and the affected Member State should set

up operational strategies and ensure the availability of follow-up funding. These two

elements would enhance the chances of a successful exit strategy.

3. Further synergies with other funding mechanisms to complement emergency

activities with longer-term actions should be sought systematically as part of

any future activation

At the early stages of the ESI’s activation, some challenges were encountered

regarding the cooperation and coordination between relevant actors. These

coordination mechanisms nevertheless, matured and improved throughout the

activation period. And while no clear overlaps or duplications were identified

between national and/or EU instruments- also thanks to the close coordination

between the main stakeholders, the mapping of funding and joint planning- it is

considered that for any possible future activation, further complementarities and

synergies could however be sought to provide a more holistic response. Thus, to

ensure and improve the impact of ESI-funded activities, as well as to avoid any

potential duplication of efforts, further synergies with other funding mechanisms to

kom (2019) 0133 - Ingen titel

complement emergency activities with longer-term actions should continue to be

sought systematically as part of any future activation.

4. Local partners should be more systematically involved in future ESI

activations

As showed by the evaluation, some of the local implementing partners and

Commission’s partners stressed that a better mapping and use of the local

organisations could have benefited the ESI’s activation at an early stage. Thus, for a

better contextual understanding in the Member State concerned, local partners

already active in the country of activation should be identified, at an early stage of the

activation, and more systematically involved, potentially as implementing partners.

5. In any future activation, ESI should continue to aim for cost-effectiveness

among the different sectors targeted and by seeking further economies of

scale.

The evaluation showed that the cost-effectiveness of the activities varied amongst the

different sectors under which

ESI’s activities were implemented. And while overall

this evaluation found that the efficiency of the actions funded by ESI was adequate

and it improved over the years, there would be always room for improvement and

thus the ESI should continuously aim for the cost-effectiveness of its actions.

In the case of Greece, one of the key factors that positively influenced and ultimately

improved the cost-effectiveness of the implemented actions, was the close monitoring

on the situation on the ground. As explained by this evaluation, the latter was

regularly undertaken by the Commissions field experts. Thus the Commission

considers that in any future ESI activation, and whenever relevant, the Commission

should continue ensuring the presence and support of its humanitarian experts in the

field, whose presence on the ground ensures the operational coordination and

technical support necessary for the success of its operations.

Based on the findings of the evaluation as presented in this document and according to

Article 8 of the Regulation, as the next step, the Commission will present a report to the

Council concerning the future of the Regulation and, where appropriate, proposals to

amend or terminate it.

kom (2019) 0133 - Ingen titel

NNEX

1: P

ROCEDURAL INFORMATION

EAD

DG, D

ECIDE

LANNING

/CWP

REFERENCES

European Civil Protection and Humanitarian Aid Operations (ECHO)

LANNING

EFERENCE

PLAN/2017/1561

RGANISATION AND TIMING

The support study was supervised by an Interservice Steering Group (ISG), with

members from DG BUDG, DG EMPL, DG HOME, DG NEAR, DG REGIO, DG

SANTE, SG and DG ECHO. In total, two meetings were held with the ISG during the

overall process of the support study.

Timetable



1st meeting of ISG (ToR):

Start of evaluation contract:

Launch of OPC:

Close of OPC:

ISG meeting on draft Final Report:

16 October 2017

6 July 2018

10 January 2018

9 March 2018

11 October 2018

Publication of external report:

https://ec.europa.eu/echo/funding-

evaluations/evaluations/thematic-evaluations_en

XCEPTIONS TO THE BETTER REGULATION GUIDELINES

The Better Regulation Guidelines were applied in full.

ONSULTATION OF THE

RSB

The present section provides an overview of the information and evidence that has been

further incorporated to the Staff Working Document, following the consultation of the

Regulatory Scrutiny Board (RSB) by which recommendations were provided for the

improvement of the document.

ACKGROUND

Following the inclusion of the Evaluation of the Emergency Support Instrument (ESI) in

the 2018 Business Plan of the RSB, the ESI’s external evaluation was submitted together

with a draft Commission Staff Working Document (SWD) to the RSB. On 16 January

2019, the RSB provided a positive (second) opinion to the Commission regarding the

Staff working Document. The RSB’s main considerations and how they were addressed

are presented in the following table:

kom (2019) 0133 - Ingen titel

No. Point raised by RSB

1.a To what extent the

approach to

humanitarian aid was

guided by existing

Commission

guidelines

1.b

Whether the

guidelines need to be

adapted for

emergency support

within the European

Union

3.a

The comparison with

EU humanitarian

activities in third

countries uses some

ambiguous language

Whether all numbers

and targets reached

correspond to the

delivery of

qualitative outcomes

Comment

The Commission’s sector guidelines

were applied to the same extent for

ESI as for actions in third countries.

They are based on good practice,

which ensures quality of

implemented actions. They are

referred to in the ESOPs, are taken

into account in the selection of

projects to be funded, and are used as

a reference for project monitoring.

The sector guidelines are designed to

fit all emergency contexts, and focus

on the delivery of emergency

services to targeted people. Thus,

they must be flexible to the context

where the emergency response is

delivered. To be kept in mind is that

the ESI implementing partners are

the same as for EU emergency

response in third countries, which

contributes to ensuring a proper

interpretation for and application to

the specific context.

The comparison concerns cost-

effectiveness. Ambiguity eliminated.

Section of SWD

Sub-section 5.1.2

(para. 6) amended.

Annex 1 amended.

Explanatory

footnote added to

section 5.1.2

Sub-section 6.4

(para. 4) rephrased

3.b

3.c

What do these

numbers mean in

view of the evolving

needs?

An analysis of some

critical dimensions

such as security

could yield more

Numbers and targets reached refer to

the most vulnerable people as

identified by the needs assessments

and specified in the ESOPs. The

quality of the response is the focus of

the project selection process and

monitoring, which ensure that

projects are specified and deliver

according to standards and good

practice as expressed by the sector

guidelines.

The relevance of targets was ensured

by a continuous revision of needs

assessments, which were taken into

account for the annual ESOPs.

There are two possible

interpretations of this point:

1) Security in the camps. At the

baseline (before ESI activation)

Explanatory

footnote added in

sub-section 5.3.1.

As for ensuring

quality of actions,

see flowchart of

Annex 1.

This is explained

under sub-section

5.1.2 (para. 5)

Concerning

baseline, see sub-

section 2.2 (bullet

list)

kom (2019) 0133 - Ingen titel

No. Point raised by RSB

lessons for

addressing future

emergency situations

The report could

improve readability

of the timelines of

events and of the

intervention logic

Comment

there were reports of lack of

security in the camps, which was

mitigated by protection activities

under the ESI. However, to be

kept in mind is that enforcement

of security measures addressing

violence is the competence of the

Member State concerned.

2) The ESI was included in the

Open Public Consultation for the

security cluster, as it addresses

the refugee crisis in the EU.

However, the ESI (a) focuses on

emergency aid based on the

humanitarian principles, and (b)

addresses disasters that

overwhelm the capacity of an EU

Member State. Thus, security

issues are outside the scope of the

ESI (and are within the mandate

of DG HOME), and refer

specifically to the current

activation rather than the overall

instrument.

The text of the two charts is small,

and it should be ensured that the

reader can get a quick overview. The

chart for the intervention logic is

provided in annex. However, the

main components and logic are

explained in a more digestible format

in the main document.

Section of SWD

Intervention logic:

Subsection 2.1

(end)

Timeline: Main

milestones are

provided before the

detailed chart (sub-

section 2.1).

Concerning points 1.a), 3.a); 3.b), the flowchart below is provided for clarification.

Particularly, it shows how the (policy) sector guidelines are linked to the different steps

of the process for emergency aid under the ESI. Moreover; it illustrates how needs

assessments relate to the humanitarian principles and how they are linked to the priority

setting of the ESOPs.

kom (2019) 0133 - Ingen titel

Safeguards for spending EU funds and ensure quality under the ESI

Monitoring

reports

Partner

selection

Identi fication

of needs

Priority setting

Project

selection

Project

monitoring

Evaluation

Framework

agreements

Needs

assessment

framework

ESOPs

Selection criteria

Monitoring

framework

Evaluation report

Humanitarian

principles

The Regulation

Risk assessment

Sector guidelines

Audit

Audit reports

kom (2019) 0133 - Ingen titel

VIDENCE

SOURCES AND QUALITY

The evaluation relies entirely on an external evaluation carried out by ICF Consulting

Services Ltd, which included the use of a series of research tools specifically developed

and tailored for the purpose of capturing the views and input of all relevant stakeholders

of the activities funded by ESI and providing triangulated conclusions. These research

tools included:



A large literature review, including:

A set of 166 documents regarding regulatory documents, Operational

Priorities (ESOPs), Financial decisions and plans, DG ECHO mission

reports, audit reports, meeting minutes, communication and visibility

plans and beneficiary surveys and:

35 publicly available documents to capture information gathered by third

parties such as UNHCR, IOM and other Commissions services. In

addition, databases from UNHCR, IOM and the Greek authorities were

explored and used for this analysis.



The mapping of 29 ESI funded actions and the proposal assessment of the funded

actions as well as those actions rejected.

73 key informant interviews conducted at different stages of the evaluation and

involving one to two interviewees per interviewing session;

Three field missions to Greece. The missions included interviews, ten focus

groups and project visits.

One online survey

targeting the Commission’s partners and local implementing

partners

One mini-mobile survey targeting final beneficiaries of the ESTIA programme

(hence focusing on the accommodation and cash component)

An Open Public Consultation (OPC) on the Commission’s

website from January

to March 2018; and

Overall, the evaluation results are considered as valid, as most findings are confirmed by

multiple sources of evidence.

kom (2019) 0133 - Ingen titel

NNEX

2: I

NTERVENTION

OGIC OF THE

ESI

ACTIONS

Following the submission of the support study, the ESI committed the 100% of its budget, amounting to EUR 643 600 000.

kom (2019) 0133 - Ingen titel

NNEX

3: S

TAKEHOLDER CONSULTATION

1. Objective and scope of the stakeholder consultation

The objective of the stakeholder consultation was to provide all relevant stakeholders

with the opportunity of expressing their views on the main topics of the evaluation.

The consultation, which was developed through a number of different methods, covered

the following topics:



The

relevance

of ESI’s as a tool for emergency response within the EU, both in

terms of its legal framework and implemented operations, its working method, as

well as the relevance of its intervention in Greece, according to the evolving

humanitarian needs and context.



The

coherence and complementarity

of ESI to other EU and/or national

instruments, as well as its adherence to the humanitarian principles.



The

effectiveness

of the actions implemented by ESI in achieving or contributing

to the Regulation objectives and to meet the humanitarian needs targeted; the

effectiveness of its working method and any factors contributing or hampering the

former.



The

efficiency

of the ESI’s implemented

actions;



The

EU added value

of the intervention of ESI in Greece, including that of its

implemented operations and working method;



The

sustainability

of ESI’s actions and the extent to which they could continue

after the end of the three-year activation of the ESI, through other funding

sources.

2. Stakeholder mapping

Stakeholders were identified through a literature review and exchanges with the

Commission. These include the following groups, also reflected in Figure A1.1 below:



EU officials:

this group of stakeholders relate to EU officials involved in the

design and implementation of the ESI or in other support mechanisms

implemented on the ground. Their consultation aimed to understand the

coherence of ESI’s actions with other EU instruments. These

included: DG

ECHO officials, DG HOME officials, DG SG officials and representatives of the

Structural Reform Support Service (SRSS).



National authorities:

these included Greek national authorities that had a major

influence on the implementation of ESI actions in the country. A selected number

of public authorities at national, regional and local level were consulted to gather

their views on ESI intervention.

kom (2019) 0133 - Ingen titel



Commission’s partners and their local implementing partners:

these included

the selected humanitarian organisations, INGOs and NGOs that had implemented

ESI’s actions on the ground. As one the main type of stakeholders to be

consulted, the partners and their implementing partners were selected on the basis

of the number of actions implemented and the amount of funding received.



Civil Society Organisations:

these included non-governmental organisations

(NGOs), universities and research organisations, media representatives, social

partners, professional associations and other organisations that played an

important role in undertaking complementary function in areas such as good

governance advocacy. Their consultation aimed at understanding the context, as

well as the relevance, effectiveness and added value of ESI’s actions.



Beneficiaries:

these are at the centre ESIs actions and included the target

population but also actors that have indirectly benefited from actions funded by

ESI such as the wider community. Their consultation aimed at understanding how

ESI’s intervention helped and met their needs.



Other actors:

these included other (humanitarian) actors who did not receive ESI

funding but were present in the country and pursuing humanitarian activities. The

consultation with this type of stakeholders aimed at understanding how

coordination on the ground

worked and their perspectives on ESI’s effectiveness

and added value.



Private sector:

these are those that played an increasing role in emergency

response working for or in coordination with humanitarian actors, for example

those private entities providing goods and services such as personnel, equipment

or logistics services (e.g. Cash Based Interventions (CBIs)). Their consultation

aimed at exploring how ESI’s actions engaged with them while retaining its

independent character and how partnerships worked in practice.

Figure A1.1: ESI’s evaluation stakeholder mapping

kom (2019) 0133 - Ingen titel

3. Methods and tools

The topics and consultation tools were tailored for each stakeholder group. The specific

methods and tools to gather information from the different stakeholders were chosen on

the basis of their role and diversity, as well as their influence, impact, awareness and

accessibility. A key principle of the consultation was to strike an adequate balance

between representativeness of stakeholders and gathering the relevant insights to answer

the evaluation questions. The table below provides an overview of the methods that were

used to consult each stakeholder group:

Table A1.1 Overview of the different tools used according to type of stakeholder

Research tools

Coherence &

complementarity

Evaluation criteria

EU Added Value

Fieldwork focus

groups

Key informant

Interviews

Mobile survey

Online Survey



Beneficiaries

EU officials



National Authorities



Framework partners



Local implementing



partners

Civil Society Organisations



*



Private sector



*



Other humanitarian



*

actors

Note: * The external evaluation team reached out to different members this stakeholder group, however

some or none of them responded to the requests to discuss this evaluation

Sustainability



Effectiveness

Fieldwork

interviews

Relevance

Efficiency

kom (2019) 0133 - Ingen titel

NNEX

4: L

IST OF EVALUATION QUESTIONS

ELEVANCE



EQ1

To what extent is the Emergency Support Instrument fit-for-purpose as a

tool for emergency response within the Union?

EQ2

To what extent was the ESI intervention appropriate when considering

Greece's particular situation and needs (e.g. type of needs, assessment of

capacities, number of beneficiaries)?

EQ3.

To what extent has the legal framework demonstrated to be flexible enough

to appropriately adapt the operational response to the changing conditions and

needs in the Member State concerned?

EQ4.

To what extent is the method of working (channelling aid through

humanitarian partners) relevant for addressing the humanitarian consequences of

the refugee and migration crisis within the EU?

EQ5.

To what extent were

the assumptions of ESI’s intervention in Greece

correct (type of needs, assessment of capacities and number of beneficiaries)?

EQ6.

To what extent has the legal framework shown to be flexible enough to

allow for appropriately adapting the operational response to the changing

conditions on the ground? Is the Instrument’s flexibility applicable in any given

Member State?

OHERENCE AND COMPLEMENTARITY



EQ7.

To what extent has the ESI intervention adhered to the humanitarian

principles (humanity, neutrality, impartiality and independence)? When assessing

the implemented operations?

EQ8.

To what extent is the Emergency Support Instrument complementary to

other related EU instruments, and are there any overlaps?

FFECTIVENESS



EQ9.

To what extent have the objectives of the Regulation and the specific

objectives of the activation been achieved, in particular as regards meeting the

humanitarian needs of refugees and migrants in the different sectors of

intervention?

EQ10.

Were there any particular factors influencing the achievement of these

objectives?

EQ11.

To what extent did actions funded by ESI’ implementation method (i.e.

channelling aid through humanitarian partners) allow to effectively address the

humanitarian needs?

EQ12.

To what extent have the ESI-funded operations achieved EU visibility as

set out by the Communication and Visibility Manual for European Union-funded

Humanitarian Aid Actions?



kom (2019) 0133 - Ingen titel

FFICIENCY



EQ13.

To what extent were the actions funded by ESI implemented in a timely

and cost-effective way, taking account of the specific conditions of the emergency

operations in Greece?

EQ14.

What factors affected the cost-effectiveness of the response and to what

extent?

ADDED VALUE



EQ15.

What was the EU added value of the ESI activation in Greece?

EQ16.

What is the specific EU added value of the ESI’s operations and their

working method in Greece?

USTAINABILITY



EQ17.

To what extent can actions currently funded under the ESI continue after

the end of the three-year activation on the basis of national funding, and relying

on the national programmes under AMIF and ISF, or other sources of funding

such as EUSF, ESF, ERDF and FEAD??

kom (2019) 0133 - Ingen titel

NNEX

5: R

ESULTS OF THE

PEN

UBLIC

ONSULTATION

An open public consultation (OPC) on EU funds in the area of security was carried out

for three months (January 2018

–

March 2018) The aim was to collect the views of all

interested parties on the EU budget in view of the next generation of financial

programmes for the post-2020 Multiannual Financial Framework. The topics of the

consultation included: Borders and security, Fraud prevention, Humanitarian aid and

civil protection, Business and industry, Digital economy and society, Economy, finance

and the euro, Energy, Research and innovation- and Transport

. Hence, the Emergency

Support Instrument was also covered by this consultation.

A total of 153 respondents from all over Europe (of which 114 were organisations and 39

individuals) provided feedback on EU funds in the area of security, including amongst

other issues, the provision of humanitarian support at large scale for emergency

situations. As part of the consultation, the following relevant questions on the emergency

support were addressed to the general public and the figures show the results.

Firstly, respondents were asked to indicate their experience with one or more funds or

programmes. As shown in Figure 1 below, respondents seem to be most familiar with the

Internal Security Fund for police (40.5%) whereas only 6.5% of the respondents were

familiar with the Emergency Support instrument. The low awareness/experience of the

instrument might a linked to the fact that the latter was only introduced in March 2016.

Figure 1:

Experience of respondents with programmes/funds

Source: Factual summary of the public consultation on the EU long-term

In addition, respondents were asked which challenge(s) respondents considered

important and how successful the current EU programmes/funds are at addressing these

challenges. Figure 2, shows that 68.6% of the respondents considered the provision of

humanitarian support in large-scale emergency situations as important or rather

important, while 25.5% considered that the challenge has been fully or fairly well

addressed.

More information is available at:

https://ec.europa.eu/info/consultations/public-consultation-eu-funds-

area-security_en

kom (2019) 0133 - Ingen titel

Figure 2:

Importance of policy challenges and how well current programmes/funds

address them

Source: Factual summary of the public consultation on the EU long-term

Regarding the EU added value that the current EU programmes/funds provide compared

to those the Member State could achieve at national, regional and local level, 68.0% of

respondents considered that the EU programmes/funds do add value to a large or fairly

large extent compared to what Member States could achieve on their own. While 34

respondents (22.2%) considered that the current programmes/funds add value to some

extent only, and three respondents (2%) considered them not to add any value at all.

When enquiring respondents to provide their opinion regarding the obstacles that prevent

the current EU programmes and fund to meet their objectives, the results showed that

71.9% of respondents considered "Complex procedures leading to a high administrative

burden and delays" the most important obstacle preventing current programmes/funds

from successfully achieving their objectives. The latter obstacle was followed by the

"lack of flexibility to react to unforeseen circumstances or new priorities" which was

considered by 56.2% of respondents consider a challenge to a large or fairly large extent.

Finally, preliminary steps were identified by the consultation for respondents to provide

their views on which of these could help reduce the administrative burdens the current

programmes/funds pose to beneficiaries. Results showed that "fewer, clearer and simpler

rules" was the most preferred option- as chosen by 78.4% of respondents considering this

at least to a fairly large extent) while "simpler application and reporting procedures" was

also considered by 73.2% of respondents.

kom (2019) 0133 - Ingen titel

NNEX

6: O

VERVIEW

INTERVENTION

ESI

FUNDED ACTIONS PER PARTNER AND AREAS OF

(As of 31/12/2018)

Partner

OXFAM-NL (NOVIB)

STC-UK

DRC-DK

IRC-UK

UNHCR-CH

MDM-BE

ASB-DE

FICR-CH

NRC-NO

IOM-CH

CARE-DE

MCE-UK

UNICEF-US

TDH-CH

MDM-EL

METADRASI

SPANISH RC

Smile of the Child

Contract Amount in

Euro

13 500 000

9 651 000

48 404 000

30 780 000

369 170 000

13 550 000

18 625 000

23 000 000

16 400 000

56 400 000

3 726 000

6 567 782

22 796 218

5 830 000

900 000

1 500 000

2 000 000

800 000

Areas of intervention

Multi-sectorial assistance in Epirus region

Child protection, education in emergencies

Site management, protection, food aid, shelter

Cash assistance, site management, WASH

Cash programme, accommodation scheme, multi-sectorial response,

coordination

Health

Shelter

Multi-sectorial response, cash, health

Food assistance (Chios), shelter, basic services in Chios

Camp management activities, shelter, transport, UAM

Protection activities

Cash assistance

Education, UAM, child protection activities

Mental health for urban caseload

Interpretation services to facilitate access to healthcare system,

protection activities, education

Health, protection activities

Protection activities

TOTAL

643 600 000

kom (2019) 0133 - Ingen titel

NNEX

7: M

ETHODS AND ANALYTICAL MODELS

A. Research framework

A variety of data sources were used to build a rich and comprehensive evidence base for this

evaluation covering a wide range of stakeholders. Overall, approximately 200 documents

were reviewed and 73 interviews were undertaken. Direct engagement with beneficiaries took

place through ten focus groups, nine interviews and a mini mobile survey (Figure 13).

Figure 13.

Overview of the approach to the evaluation

Source: ICF. 2018.

1. Internal evaluation review and gap analysis

As part of an internal exercise, the Commission conducted an internal evaluation of the ESI

Regulation covering the period 2016-2017. As a first step, the external evaluation team

reviewed the internal evaluation including the main findings reported in the draft report and

underpinning data and information.

2. Documentary and literature review

As part of the desk review, the evaluation team looked at a range of secondary sources of

evidence.



Literature review:

35 publicly available documents were reviewed to capture

information gathered by third parties such as the United Nations High Commissioner

for Refugees (UNHCR), the International Organization for Migration (IOM) and other

Commissions services. In addition, databases from UNHCR, IOM and the Greek

authorities were explored and used for this analysis.



ESI documentation review:

166 documents were reviewed as part of this evaluation,

these include regulatory documents, Operational Priorities (ESOPs), Financial

Decisions and plans, DG ECHO mission reports, audit reports, meeting minutes,

communication and visibility plans, and beneficiary surveys.

kom (2019) 0133 - Ingen titel



ESI portfolio analysis:

data extracted from DG ECHO EVA and HOPE databases

were analysed to create a series of tables and charts that are presented throughout this

report. The analysis focused on elements such as Resilience marker, Gender/Age

marker, beneficiaries, sectors, transfer modalities, project duration, location, and costs.



ESI project mapping:

all 29 ESI actions were reviewed through a mapping of

information extracted from DG ECHO SingleForms

and operation sheet (FichOps)



ESI Dashboard analysis:

decisions and reasons for proposals’ approval or refusal

were analysed and a typology of actions accepted or rejected was created.

3. Stakeholder consultation

As part of the consultation stage a mini mobile survey, field visits and Key Informant

Interviews were undertaken in parallel to engage with a maximum of stakeholders.



Mini mobile survey of beneficiaries:

a survey focusing on the ESI support and more

specifically on Cash and Accommodation under the Support to Integration &

Accommodation (ESTIA) programme, was created in English and translated into

French, Arabic, Farsi and Kurmanji to cover the majority of the beneficiaries (81%).

The survey link was sent to 10,901 households (i.e. head of the household) of the

ESTIA programmes covering the five languages of the survey through UNHCR,

6,803 households received it (63%) and 1,788 responded to the survey. After cleaning

the data and removing duplicates, 1,316 responses were considered valid representing

a response rate of 19%. In addition, one focus group and four beneficiary interviews

were conducted to gather additional qualitative information on Cash;



Field visits:

two field visits were conducted with a specific thematic and evaluative

focus. The first visit, called “The

handover of the ESI health projects to National

Authorities”

focused on the transition of the health activities from DG ECHO to the

national authorities, which is primarily linked to the evaluation criteria of coherence

and sustainability. The second visit, called “ESI

accommodation projects under the

ESTIA programme

–

working with local NGOs and municipalities”

focused on the

accommodation component of the ESTIA programme, and the evaluation criteria of

relevance, effectiveness, efficiency and sustainability. For each field visit, the external

evaluation team engaged with a wide range of stakeholders (26 interviews were

undertaken), including beneficiaries (five focus groups and five interviews were

EVA and HOPE are DG ECHO interactive dashboards that provide information about each action financed by

a contract signed by the partner.

Main instrument of Framework Partnership Agreement used by framework partners to submit requests,

modification requests, interim reports and final reports.

Communication and recording tool that captures the main aspects of the operational analysis and follow-up

made by Field Experts, Desk Officers, Desk Assistants and Financial Officers on the humanitarian actions all

along the action’s cycle.

UNHCR. 2018. Data on languages spoken.

A sample of 10,901 cases were contacted by UNHCR out of 25,000 registered cases. The sample was selected

on the basis of the availability of the contact details and the language spoken.

kom (2019) 0133 - Ingen titel

conducted)

and visited several sites. In addition, four focus groups were conducted

with 23 children to discuss the education component of ESI’s operations.



Key Informant Interviews:

17 additional interviews were organised to complement

the interviews conducted as part of the internal evaluation and the field visits.

Interviews were conducted with EU officials, Greek authorities, framework partners

and local implementing partners. In the present report, these will be referred to as

Second round of Key Informant Interviews.

4. Validity of findings

Complementary research methods were used to enhance the reliability and validity of the data

collected and to provide the basis for cross-verification, corroboration and triangulation of the

evaluation results. The vested interests of different stakeholder groups were taken into

account to address potential bias and to ensure objectivity. However, as with any evaluation,

there were limitations to the methodologies applied, which are summarised in Table 2 below.

Given these methodological caveats and limitations, caution was exercised when interpreting

data and producing findings.

Table 2.

Tasks

ESI documentation

Limitations to methodologies applied

Issues encountered

Incomplete financial information.

Steps taken

Additional information requested directly to DG

ECHO, DG HOME and framework partners.

Lack of clarity on whether the information

relates to DG ECHO or DG HOME.

Information mapped into excel templates to allow

for comparison.

Uneven provision of detail in the different

reports.

Data used with caution and triangulated with other

data sources (interviews, field visits).

No clear structure of documents (e.g. meeting

minutes, mission reports).

Portfolio analysis

Limitations with beneficiary data (i.e. double Several other external sources were used to

counting).

corroborate the data (UNHCR, Greek Asylum

services).

Incomplete information for projects where the

final report is not yet available.

The quantitative analysis was supported by

qualitative information from other data sources

(interviews, field visits).

Inconsistencies in type and amount

information available in partner reports.

of Data used with caution and triangulated with other

data sources (interviews, field visits).

corroborated

with

Project mapping

Data mainly at output-level, less at outcome Quantitative information

level.

Portfolio analysis.

Limited data to inform cost-effectiveness and

efficiency assessments.

Mini mobile survey

Coverage based on the languages spoken and Focus groups with women were conducted while in

availability of contact details.

the field to capture women’s perspective on cash.

Overrepresentation of men.

Duplicates were removed.

Two focus groups and three interviews with beneficiaries were conducted as part of the accommodation case

study and three focus groups and one interview with beneficiaries were conducted as part of the health case

study.

kom (2019) 0133 - Ingen titel

Tasks

Issues encountered

Steps taken

Several beneficiaries completed the survey The base taken for analysis is the number of

more than once.

respondents to each question, therefore it varies

greatly between questions.

Number of valid responses vary greatly by

question.

Field visits

Few sites visited, as the camps are scattered Sites that were not visited were discussed during

throughout Greece.

the interviews with framework partners and local

implementing

partners.

Information

was

As framework partners were no longer active triangulated with project documentation and

in the sites visited, they had limited control literature review.

over the organisation of the site visits

including focus groups.

In sites, the team requested the support of the ESI

SMS partners to organise the focus groups.

Specific topic guides were developed for the

different stakeholder’s groups consulted. During

the analysis, the information collected was

contextualised; the differences in the contexts and

views were factored in. Data was cross-checked

with other data sources.

Key

Informant Diversity of interviewees consulted (DG

Interviews

ECHO,

framework

partners,

local

implementing partners, National Authorities

etc.) with more or less informed knowledge of

the ESI interventions and some vested

interests.

kom (2019) 0133 - Ingen titel

B. Efficiency analysis

The use of the final consumption expenditure and GDP per capita to compare ESI with

DG ECHO actions in other countries

As it was the first time the ESI was activated in Europe, no comparison could be made with

other Member States. Nonetheless, four third countries were selected as a reference to assess

the cost-effectiveness of ESI, namely Egypt, Jordan, Lebanon, and Turkey. These countries

all welcomed large numbers of refugees from the Middle East and faced some similar

challenges as those encountered in Greece. The different levels of economic development

were taken into consideration through final consumption expenditure (FCE) per capita and

GDP per capita. Figure 14 indicates that the average price level of the four countries equals

merely 32% of Greece’s level (Greece

representing 100%). In other words, consumption of

similar goods in Greece is nearly three times more expensive than in other countries. In the

same vein, the average GDP per capita of the four countries equals 30% of Greece’s GDP per

capita. This supports the assumption that the wage level in Greece is roughly three times

higher than the averages in other countries.

Figure 14.

Final consumption expenditure and GDP per capita (average from 2015 to 2017)

Source: ICF. 2018. World Bank database. Indicators: (1) Households and NPISHs final consumption expenditure per capita

(constant 2010 US$), (2) GDP per capita (constant 2010 US$), available at:

https://data.worldbank.org/,

extracted on

26/09/2018

Note: this graph represents the share of the final consumption expenditure per capita and GDP per capita of Egypt, Jordan,

Lebanon and Turkey, as a share of the final consumption expenditure per capita and GDP per capita of Greece.

The average cost of implementation of the emergency support activities in Greece is EUR 813

per beneficiary while it costs on average EUR 379 per beneficiary in other countries.

However, once the prices are adjusted to the Greek price level (FCE), the average cost per

beneficiary in the other countries is EUR 1 375 per beneficiary (Figure 15). The adjusted

price is on average three times the original prices of consumption in the countries of

kom (2019) 0133 - Ingen titel

reference. Although it provides a base for comparison, this should be interpreted with caution;

the rate might be biased as many other factors affect the costs such as the country taxes, the

price of the goods bought on international market, exchange rates and import taxes, etc.

Figure 15.

Average cost per beneficiary, per sector, 2016 - 2018

Source: ICF. 2018. Data extracted from HOPE/EVA databases on 15.10.2018. Note: Some sectors were not covered in all

five countries.

Quantitative indicators used to assess the cost-effectiveness of the ESI in Greece



The first indicator compares the

cost per beneficiary

of the ESI in Greece with the

cost per beneficiary of other DG ECHO interventions in Egypt, Jordan, Lebanon and

Turkey. The costs in the group of countries of reference is then indexed to the

consumer price level of Greece (using the FCE). Although not perfectly accurate, this

gives an indication of the cost-effectiveness of the actions based on similar modus

operandi (i.e. DG ECHO through framework partners) with the same objective (i.e.

save life) for similar target groups (i.e. refugees).



The second indicator compares the

average share of specific costs per project

Greece and in the other countries, with a higher share of specific cost indicating a

better efficiency of the actions.



And lastly, the

alpha ratio

of the Basic needs assistance (e.g. MPCT) sector indicates

the share of the cash that is directly transferred to the beneficiaries. A higher alpha

ratio indicating a better cost-effectiveness.

kom (2019) 0133 - Ingen titel

Methodology of the calculation of the cost per beneficiary

Data sources

Two datasets have been used to

calculate the cost per beneficiary indicator: the “Transfer

Modalities” and the “Beneficiaries” datasets extracted from the HOPE dashboard. The table

below shows the variables that have been used for the calculations and the source of each

variable.

The

indicators used from the “Transfer Modalities” are:



Result Direct Specific Cost:

as given in the single form “Estimated

incurred total

amount”.

In theory, this should represent the amount of cost directly transferred to the

implementation of the result.

For example:

Project of sensitisation in refugee camps:

◦

Result 1: sensitisation of heath matters to the refugees (Health).

◦

Result 2: installation of sanitation equipment (WASH).

The cost for the running of the local office, the transport of the staff to the refugee

camps are seen as common direct support costs. The cost of the sanitation equipment

is a direct cost of result 2.



Contract Amount Signed by Partner:

The initial ECHO contribution to the total

costs of the project.



Result Contract Amount Signed by Partner: Initial ECHO contribution to each result



Result Individual Beneficiaries:

the number of beneficiaries, both individual

beneficiaries and beneficiaries reached through households, at result level.

From the “Beneficiaries” dataset we have only extracted the

total cost,

which represent the

total cost of the project.

Methodology

To estimate the cost per beneficiary for each result, the total cost of each result was divided

by the number of individuals reached. However, the

total cost per result,

which was not

available in the datasets, was estimated by the weight of ECHO’s contribution to each result.

Which was also equal to the weight of each Result Direct Specific Cost. Hence, the total cost

of each results includes the direct support costs common to all results and the indirect costs.

Analysis of the share of direct specific costs

This was confirmed by emails exchanged with colleagues from DG ECHO analytics. However, some financial

statements extracted from the Dashboard shows that it’s not always

the case. For instance, the financial

statement of the project ECHO/CHD/BUD/2014/91002 shows that support costs common to all results are

included in the direct costs of the actions.

kom (2019) 0133 - Ingen titel

Data sources

The two same datasets have been used to calculate the share of direct support cost and direct

specific costs: the “Transfer Modality” dataset and the “Beneficiaries” extracted from HOPE.

The indicators used are:



Total cost:

The total cost of the project



Total cost per result:

Total cost of each result (including the indirect common costs

and other indirect costs).



Contract amount signed by partner:

amount of ECHO contribution to the total cost

of the project



Result contract amount signed by partner:

amount of ECHO contribution per result



Result Direct Specific Cost:

the “net” cost of each result, this excludes the indirect

costs and the common direct cost of implementation

Methodology

In theory, the sum of the “result direct specific costs” (the pure or net costs of the results)

should give an indication of the amount of money used to implement the action (directly

transferred to beneficiaries).

Hence, the difference between the total cost of the project (a) and the sum of “direct specific

cost” (d) should give a relatively good estimation of the “direct support costs” as described in

section 10.2 of the Single Form. Note that Section 10.2 was introduced in late 2017 only.

The share of direct support costs (e) is hence equal to the estimated total direct support costs

divided by the total cost.

Analysis of the Alpha ratio

The alpha ratio is the transfer value divided by the total costs. In other words, it represents the

share of the total costs that is directly transferred to the beneficiaries in the form of cash,

voucher or in kind.

Sources

The two same datasets have

been used to calculate the alpha ratio: the “Transfer Modality”

dataset and the “Beneficiaries” extracted from HOPE.

The indicators used are



TM name:

gives the types of transfer modality used per result



TM Direct Specific Cost:

the direct specific cost incurred to each transfer modality

type



Total cost per result:

Total cost of each result (including the indirect common costs

and other indirect costs).



The database of the transfer modality gives an additional disaggregation of each result

per transfer modality type. For instance, the table below shows a project that

kom (2019) 0133 - Ingen titel

implemented two results in different sectors. The two results used cash as a transfer

modality. However, the full amount of the costs was not given in cash, a share of it

was used for other purpose

and is referred as “no transfer”.

Result number

Sector

Nutrition

WASH

Total costs per result

622,120

528,140

TM Name

cash

In kind

No transfer

cash

No transfer

TM Direct Specific Cost

11,912

32,696

483,386

14,714

433,773

Methodology

As a first step, we have summed up for each result, the values of the TM direct specific cost

that are given either in cash, in voucher of in kind (a+b for result 1, c alone for result 2). This

gives the total value of the transfer, per result. Then, to obtain the alpha ratio, we have divided

the value of the transfer by the total cost per result: (a+b)/c and d/e. This gives the share of the

total cost that has been directly transferred to beneficiaries in the form of cash, in kind or

vouchers.

kom (2019) 0133 - Ingen titel

C. Analytical framework

Evaluation

Criterion

Relevance

Internal/

External/

Both

Internal

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

EQ1. To what extent is

the Emergency Support

Instrument (ESI) relevant

as a tool for emergency

response within the

Union?

EQ2. To what extent was

the ESI intervention

appropriate when

considering Greece's

particular situation and

needs (e.g. type of needs,

assessment of capacities,

number of beneficiaries)?

JC1.1 The ESI is necessary to

provide the capacity for the Union

to address wide-ranging

humanitarian needs resulting from

natural or manmade disasters.

JC2.1 The intervention under the

instrument was appropriate for the

MS situation, as the latter had

already exhausted all available

capacity/resources.

Internal

JC2.2 The ESI actions were

appropriate and took into account

the emerging needs in the Member

State (Greece) and targeted the

relevant beneficiaries

Description of ESI, its objectives,

characteristics, etc.

Evidence of ESI (instrument) being developed

based on assessments of the context and needs

Perceptions/opinions of relevant stakeholders

regarding the relevance of the instrument

Description of the crisis and context

Needs identified by DG ECHO (ESOPs) vs

needs identified by other actors (HOME, UN,

NGO, authorities) in their respective needs

assessment

Evidence of ESI (legal framework, ESOPs)

being based on an assessment of capacity and

gaps in response (i.e. existing instrument)

Perception of relevant stakeholders on the

national capacity to respond to the crisis

Evidence of ESI actions being based on an

assessment of existing national and EU

initiatives and gaps in response

Evidence of ESI actions being based on

assessments of the country's context and needs

Number of beneficiaries targeted by the

actions/partners vs. an estimate of number of

people/beneficiaries in need

Qualitative

Quantitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

Perceptions/opinions of relevant stakeholders

regarding the relevance of ESI actions vs their

needs assessment in EL

Qualitative

JC2.3 Partners' needs assessments

have correctly analysed the needs

of the target beneficiaries.

JC2.4 Budget allocations have

taken account of the partners' needs

assessments

Data on beneficiaries to be presented by

sector/need and location

Type and level of needs identified by partners

vs needs identified by other actors in their

respective needs assessment

Evidence of quality (e.g., level of involvement

of key stakeholders and final beneficiaries,

robust methods being used, etc.)and

comprehensiveness (e.g. inclusion of sectors,

vulnerable groups, gender, etc.) of ESI actions'

needs assessments

Absolute amount and share of ECHO budget

allocated to ESI (instrument) compared to total

funding allocated to the refugee and migration

influx

Description of DG ECHO mechanism for

funding allocation

Evidence that DG ECHO’s budget size was

commensurate to ESI objectives and expected

outcomes

Quantitative

Qualitative

Quantitative

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

Internal

EQ3. To what extent has

the legal framework

demonstrated to be

flexible enough to

appropriately adapt the

operational response to

the changing conditions

and needs in the Member

State concerned?

JC3.1 ESI's legal framework

allowed for the optimal

implementation of the actions, and

remained relevant throughout the

changing conditions and needs in

the Member State

Evidence that DG ECHO’s budget allocations

considered the actions of other donors

Evidence that the action selection and funding

allocation was based on needs

Evolution of the needs overtime

Changes in ECHO strategy over the period

(ESOPs, Financial Decisions, Financial plans)

Perceptions/opinions of relevant stakeholders

regarding the relevance of the ESI vs their

needs assessment in EL

Perceptions/opinions of relevant stakeholders

regarding the flexibility of the ESI and if such

flexibility allowed the instrument to remain

relevant despite EL changing needs

Perception of stakeholders indicating that the

ESI actions were relevant and coherent with

ESI objectives

Evidence of coherence between ESOPs and

ESI actions

Description of ECHO working method in the

context of Greece

Perception of stakeholders on the relevance of

the working method

Reasons why some partners/potential partners

did not apply for funding under ESI

Qualitative

JC3.2 The ESI implemented actions

were relevant to the ESI objectives

Qualitative

Both

EQ4. To what extent is

the method of working

(channelling aid through

humanitarian partners)

relevant to address the

humanitarian

JC4.1 Channelling the actions

through the FPA has ensured a

timely and tailored response to the

emerging humanitarian needs of the

targeted beneficiaries

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

Both

consequences of the

refugee and migration

crisis within the EU?

EQ5. To what extent were

the assumptions of the

ESI intervention in

Greece correct (type of

needs, assessment of

capacities and number of

beneficiaries)?

Perception of stakeholders on the relevance of

the partner selection

JC5.1 The majority of the

assumptions at the start of the ESI

operations at the operational level

leading to funding the individual

projects were correct: the type and

scale of needs, the assessment of

capacities (of the national

authorities and the implementing

partners

–

framework and local)

and the anticipated number of

beneficiaries were assessed mostly

appropriately

JC5.2 At the project level, the

target beneficiaries outlined in the

Single Forms align with other

(official, high quality) needs

assessments

JC5.3 Projects provided a clear

definition of the target group, their

level of vulnerability, and a

rationale for targeting the

population

Type and scale of needs in different sectors

Estimates of number of people with different

types of need (WASH, food, shelter, etc.)

Estimates of number of beneficiaries in need

by type of beneficiary, location

The assumptions based on the baseline

situation in early 2016 correctly anticipated the

developments in 2016, 2017 and 2018 in

relation to the Greek authorities’ capacities to

provide humanitarian aid

Opinions of relevant stakeholders on the

degree of the correctness of the main

assumptions for ESI intervention in Greece

Number of beneficiaries targeted by DG

ECHO partners by type of beneficiary / type of

need / location as listed in Single Forms

Number of beneficiaries and needs identified

by other actors

Rating of the quality of the needs assessments

by the evaluation team

Number of ESI actions including a

vulnerability assessment

Qualitative

Quantitative

Qualitative

Quantitative

Qualitative

Quantitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

JC5.4 The needs identified in the

majority of ESI projects align

appropriately with the needs

described by beneficiaries

JC5.5 The emergency support

provided in the majority of the ESI

projects was appropriate to the

particular context, gaps in support

and needs identified

Both

EQ6. To what extent has

the legal framework

shown to be flexible

enough to allow for

appropriately adapting the

operational response to

the changing conditions

on the ground? Is the

Instrument’s flexibility

applicable in any given

Member State?

JC6.1 The ESI legal framework has

shown sufficient flexibility to

enable the ESI funded projects to

respond to the changing conditions

Alignment of needs assessment with needs

identified elsewhere

Alignment of needs with needs reported by

beneficiaries

Evidence that needs assessments informed the

decision to activate ESI as well as funding

decisions

Opinions of relevant stakeholders on the

relevance of the ESI actions in relation to

needs, gaps and context

Evolution of the ESI funded activities

compared to the evolution of the needs of the

beneficiaries on the ground and of the Greek

authorities’ capacities to respond

The changes experienced by ESI actions (both

at strategic and operational levels) and the

influence of legal framework on such changes

(as well as other factors)

Qualitative

Quantitative

Qualitative

JC6.2 The features allowing the

flexibility of the ESI instrument

were maximised by DG ECHO,

partners and national authorities in

Greece across the projects

examined (i.e. quick response time,

low administrative burden)

Number of modification request submitted by

partners

Evidence that the partners' modifications were

based on evolving needs

Opinions of relevant stakeholders on the

degree of flexibility provided by the legal

framework

Quantitative

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

Coherence

and

comple-

mentarity

Both

EQ7. To what extent have

the ESI projects adhered

to the humanitarian

principles (humanity,

neutrality, impartiality

and independence)?

JC6.3 The flexibility of the ESI

instrument allows it to be easily

adapted to other Member States

JC6.4 Individual ESI project

approaches are easily transferred to

other Member States

JC7.1 The ESIs funded actions

match (or do not contradict) the

humanitarian principles

Opinions of relevant stakeholders on the extent

of application of ESI in other Member States

Opinions of relevant stakeholders on the extent

of application of the activities funded under

ESI in other Member States

ESI regulation and DG ECHO policy

framework regarding principles and policy on

Humanitarian Principles

Key informant perspectives on coherence

between the ESI actions implemented and the

humanitarian principles

References to / coverage of the principles in

the action design (Single Forms) and the

documentation of DG ECHO’s

Qualitative

EQ8. To what extent is

the Emergency Support

Instrument

JC7.2 At proposal stage, the actions

have been designed to adhere to the

humanitarian principles and the

selection of actions identified and

addressed any possible concerns

beforehand

JC7.3 In the implementation phase,

any tensions between humanitarian

principles and practicalities of

delivering ESI funded activities

were successfully resolved

JC8.1 The actions funded through

the ESI could not have been

implemented by any other EU or

Quantitative

Evidence of tensions identified and how they

have been resolved

Qualitative

Number of actions linked (or covering) ESIs

specific objectives

–

as described in the Single

Forms

Quantitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

complementary to other

related EU instruments ,

and are there any

overlaps?

national instrument

JC8.2 The funded actions do not

overlap with actions

implemented/funded under other

EU or national instruments.

JC8.3 The implementation of ESI

actions had not had any direct

negative consequence on actions

implemented through other EU and

national instruments

JC8.4 ESIs budget allocation also

considered the funding provided by

other EU and/or national

instruments.

JC8.5 Coordination mechanisms

have been established to coordinate

the intervention/ operations

between the main actors. i.e. DG

ECHO, partners and national

authorities

JC9.1 Following the ESI activation

Perceptions/opinion of relevant stakeholders

on the complementarity of ESI actions

Perceptions/opinion of relevant stakeholders -

the (non-existing) overlaps between ESIs

actions and other EU and/or national

instruments

Evidence of duplication in the actions

managed by DG ECHO and those managed by

other EU and international actors

Perceptions/opinion of relevant stakeholders

on potential negative consequences of ESI

activation in Greece

Qualitative

Break down of the funding sources

Evidence of coordination with EU and national

actors for the allocation of funding

Perceptions/opinion of relevant stakeholders

on the ESI funding allocation

Evidence of coordination mechanisms at the

EU, national and local and thematic levels

Perception of relevant stakeholders on the

coordination and how it could have been

improved

Main outcomes and results indicators of the

Quantitative

Qualitative

Effective-

Both

EQ9. To what extent have

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

ness

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

the objectives of the

Regulation, and the

specific objectives of the

activation been achieved,

in particular as regards

meeting the humanitarian

needs of refugees and

migrants in the different

sectors of intervention?

the identified humanitarian needs of

refugees and migrants were met,

and/or these improved as a result of

the ESI actions.

actions

Estimate of total number of refugees and

migrants in need of humanitarian assistance

(according the relevant sectors).

Data on beneficiaries to be presented by

sector/need and location

Number of beneficiaries targeted vs number of

beneficiaries reached

Perceptions/opinion of relevant stakeholders

on the achievement of ESI

Perceptions/opinion of relevant stakeholders

on the implementation of ESI actions

Outcomes and impacts of ESI actions as

reported by implementing partners,

beneficiaries and external observers

Opinions of relevant stakeholders (including

beneficiaries) on the funded actions’

contribution to addressing needs

Description of the key stakeholder cooperation

mechanisms developed for the implementation

of ESI actions (refer to EQ8)

Opinions of the relevant stakeholders on the

level and success of cooperation between

stakeholders at different levels

Quantitative

Qualitative

JC9.2 ESI projects were

implemented as planned on the

ground

JC9.3 ESI projects met / improved

the identified humanitarian needs of

refugees and migrants

Quantitative

Qualitative

JC9.4 Success factors were

maximised by DG ECHO, DG

ECHO partners and national

authorities across the majority of

ESI projects

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

JC9.5 Appropriate technical and

other support and monitoring was

provided by DG ECHO to the

majority of ESI projects

JC9.6 Actions provided a clear

Communications and visibility plan

to ensure the relevant activities met

the EU visibility Criteria for EU

funded Humanitarian Aid actions.

EQ10. Were there any

particular factors

influencing the

JC10.1 Obstacles to the

effectiveness of ESI activities have

been addressed appropriately and

Typology of critical success factors

Description of components of DG ECHO’s

technical and other support delivered to the

ESI actions

Description of the components of monitoring

and evaluation arrangements across the ESI

actions

Perception of relevant stakeholders on DG

ECHO technical support and monitoring

practices

Number of Communication Plans developed

(number of communication plans submitted by

partners)

Number and type of communication activities

implemented by sector

Information / Description on the

communication plans meeting the EU visibility

requirements

Measures taken to avoid confusion between

DG ECHO and other EU instrument

Perception of relevant stakeholders on DG

ECHO visibility

Evidence of obstacles to effectiveness

identified by partners and DG ECHO in their

reports

Qualitative

Quantitative

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

achievement of these

objectives?

effectively across the majority of

ESI projects

Perceptions/opinion of relevant stakeholders

enabling or hampering factors regarding the

achievement of the ESI objectives and/or

action results

Number of DG ECHO framework partners

which had a presence / could operate in Greece

in 2016

Share of framework partners which proposed

ESI actions following the activation of ESI in

Greece

Number of local implementing partner used by

ESI actions

ESI budget at the instrument level and at

individual action level was implemented as

planned

Opinions of relevant stakeholders on the

effectiveness of the ESI implementation

method

Response time and flexibility of the emergency

support under AMIF in Italy and Bulgaria

Opinions of relevant stakeholders (DG

HOME) on AMIF / EMAS implementation vs

ESI

Qualitative

EQ11. To what extent did

ESI-funded actions

implementation method

(i.e. channelling aid

through humanitarian

partners) allow to

effectively address the

humanitarian needs

JC11.1 Channelling the actions

through framework partners

allowed the ESI projects to timely

and effectively fulfil the

humanitarian needs of the targeted

beneficiaries

Quantitative

Qualitative

JC11.2 Comparison with the

emergency support under AMIF

(including EMAS) in Italy shows a

slower and less flexible response

compared to ESI projects in Greece

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

EQ12. To what extent

have the ESI-funded

operations achieved EU

visibility as set out by the

Communication and

Visibility Manual for

European Union-funded

Humanitarian Aid

Actions?

JC11.3 The change of the

implementation selection of

framework partners was effective in

ensuring the right selection of

partners to implement the ESI

actions

JC12.1 The visibility of ESI funded

operations is high in Greece

Opinions of relevant stakeholders on the

changes in the partner selection criteria

Qualitative

JC12.2 The Manual is used

extensively by FPAs in ESI funded

activities

Efficiency

Both

EQ13. To what extent

were the ESI-funded

actions implemented in a

timely and cost-effective

way, taking account of the

specific conditions of the

JC13.1 DG ECHO’s response was

timely and flexible

Evidence of the type and volume of ESI

activities implemented supporting the visibility

objective

Evidence of the level of awareness of DG

ECHO activities within Greece (amongst

national stakeholders and final beneficiaries)

Obstacles to visibility

Number of actions reporting the

implementation of visibility activities and the

use of the Manual

Evidence of effective use of the

communication and visibility manual within

DG ECHO funded actions

Information on the budget planning and

timelines, for example as described by the

ESOPs

Evidence of flexibility mechanism being in

place to adapt to the changing needs (in terms

of strategy implementation and funding)

Quantitative

Qualitative

Quantitative

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

emergency operations in

Greece?

JC13.2 The allocated budget was

proportionate to the targets set

JC13.3 ESIs financial decisions and

ESOPs were designed according to

the specific needs and time

constraints

JC13.4 Humanitarian actions

funded by DG ECHO were cost-

effective

Perception of relevant stakeholders on the

timeliness of DG ECHO funding

Rationale of the selection of actions- how the

selection considered the available budget, vs

the budget requested and timeliness

Rationale of the partners' design of their

action's budgets and timelines- including the

selection of implementing partners

Perception of relevant stakeholders on the

budget allocated to Greece

Info on the ESI funding process and

development of the financial planning

Qualitative

Quantitative evidence that actions funded by

DG ECHO were cost effective (e.g. cost per

beneficiary, alpha-ratio)

Qualitative evidence that actions funded by

DG ECHO were cost effective

Evidence that partners have a good

understanding of factors affecting the cost-

effectiveness of their actions, and take these

into account during design and implementation

stage

Evidence of key actions undertaken to ensure

cost-effectiveness

Quantitative

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

JC13.5 DG ECHO promoted,

developed and implemented cost

effective approaches in ESI projects

JC13.6 DG ECHO balanced cost in

relation to making strategic choices

about its portfolio of assistance

JC13.7 DG ECHO took appropriate

actions to ensure cost-effectiveness

throughout the project cycle

Evidence that largest indirect cost drivers were

well managed

Evidence of DG ECHO supporting innovative

solutions and best practices to improve cost

effectiveness of its funded actions

Evidence of DG ECHO cooperating and

coordinating with the national and local

authorities to improve the cost effectiveness of

its funded actions

Evidence of DG ECHO cooperating and

coordinating with other organisations and the

private sector to improve the cost effectiveness

of its funded actions

Evidence of DG ECHO promoting best

practices driving cost effectiveness, e.g. multi

sectoral approaches, consortium, transfer

modalities, using local partners etc.

Evidence of DG ECHO considering cost-

effectiveness when preparing the ESOPs and

when discussing the requests put in by the

Greek authorities

Evidence of DG ECHO considering economy,

efficiency and cost-effectiveness in selecting

partner proposals and negotiating contracts

(prior to the action)

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

EU Added

Value

Internal

EQ14 What factors

affected the cost-

effectiveness of the

response and to what

extent?

EQ15. What was the EU

added value of the ESIs

activation in Greece?

Evidence of DG ECHO monitoring efficiency

and cost-effectiveness of partners and taking

appropriate actions to ensure cost-effectiveness

throughout the action cycle (during

implementation of the action)

Evidence that lessons learned were considered

throughout the action cycle and good practice

being shared and taken into account to drive

efficiency

JC14.1 No major impediments were Underspending reported or found in any of the

encountered while ensuring the

ESI actions

cost-effectiveness of the actions

Any factors influencing the efficiency of the

ESI actions

JC15.1 The ESI has a clear EU

added value when compared to

existing national instruments

JC15.2 The actions funded through

the ESI could not have been

implemented by any other national

actor

JC15.3 Certain characteristics of

the ESI make it distinctive to any

other available national instruments

JC16.1 Actions financed by the ESI

on the ground have a clear added

Perceptions/opinion of relevant stakeholders

on the added value of ESI compared to other

instrument activated in Greece

Perceptions/opinion of relevant stakeholders

Qualitative

Quantitative

Qualitative

External

EQ16. What is the

specific EU added value

Perceptions/opinion of relevant stakeholders

on specific characteristics differentiating ESI

from other instruments

ESI funding in Greece compared to other EU /

donor funding to address the problem

Qualitative

Quantitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

of the ESI’s operations

and their working method

in Greece?

value compared to actions financed

by other EU instruments and/or

other relevant donors

JC16.2 The actions funded through

the ESI could not have been

implemented by any other national

actor

JC16.3 The instrument had specific

characteristics which distinguished

it from other EU support

mechanisms

Perception of key stakeholders on the added

value of the activities implemented through

ESI

Perceptions of key informants on factors

hindering the maximisation of EU added value

Evidence of results that could not have been

achieved without a coordinated effort at the

EU level

Perceptions of key informants on whether

other instruments could have achieved the

same results

Analysis of the typology of characteristics

distinguishing ESI from other relevant

interventions (especially voluntary in-kind

contributions provided under the UCPM and

emergency assistance under AMIF and ISF)

Perceptions of key informants on specific

aspects contributing to the EU added value

Evidence that partners are systematically

considering sustainability issues in the ESI

actions (i.e. exit strategy)

Evidence of exit strategies

Evidence of handover of ESI activities

Evidence of success factors leading to

increased sustainability at the action level

Qualitative

Sustain-

ability

External

EQ17. To what extent can

actions currently funded

under the ESI continue

after the end of the three-

year activation on the

basis of national funding,

and relying on the

national programmes

JC17.1 The majority of ESI

projects have built in the

sustainability considerations into

the project design and

implementation

JC17.2 Where relevant, the

majority of ESI funded actions are

handed appropriately over to the

Qualitative

kom (2019) 0133 - Ingen titel

Evaluation

Criterion

Internal/

External/

Both

Evaluation Question

Judgement criterion

Indicators

Quant/Qual

under AMIF and ISF, or

other sources of funding

such as EUSF, ESF,

ERDF and FEAD?

relevant EU /Greek authorities

JC17.3 Key factors leading to

sustainability have been maximised

across the majority of ESI projects

JC 17.4 Specific actions have been

taken on the ground to ensure a

nexus between ESI funding and

other EU funds (e.g. AMIF, ESF,

etc., see also Coherence EQ)

Factors challenging sustainability/handover of

activities

Analysis of the typology of measures and

approaches to increase sustainability

Evidence of building capacity at national and/

or local level; local community engagement

and participation

Qualitative