See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/255271281

The effectiveness of Extreme Rainfall Alerts in predicting surface water

flooding in England and Wales

Article  in  International Journal of Climatology · September 2012

DOI: 10.1002/joc.2391

CITATIONS READS

33 157

4 authors:

Anthony Hurford Sally J. Priest

Centre for Alternative Technology Middlesex University, UK
19 PUBLICATIONS   261 CITATIONS    52 PUBLICATIONS   1,387 CITATIONS   

SEE PROFILE SEE PROFILE

Dennis John Parker Darren Lumbroso

Middlesex University, UK HR Wallingford
147 PUBLICATIONS   2,656 CITATIONS    107 PUBLICATIONS   665 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Stepping Up: Water-Energy-Food View project

Global flood forecasts View project

All content following this page was uploaded by Darren Lumbroso on 15 October 2017.

The user has requested enhancement of the downloaded file.

INTERNATIONAL JOURNAL OF CLIMATOLOGY
Int. J. Climatol. (2011)
Published online in Wiley Online Library
(wileyonlinelibrary.com) DOI: 10.1002/joc.2391

Short Communication
The effectiveness of extreme rainfall alerts in predicting
surface water flooding in England and Wales
A. P. Hurford,a* S. J. Priest,b D. J. Parkerb and D. M. Lumbrosoa
aHR Wallingford, Howbery Park, Wallingford, Oxfordshire OX10 8BA, UK
b Flood Hazard Research Centre, Middlesex University, London N14 4YZ, UK

ABSTRACT: The extreme rainfall alert (ERA) pilot was a response to the Pitt Review’s recommendation that ‘the Met
Office and Environment Agency (EA) provide an early assessment of the costs, benefits and feasibility of techniques
which can predict where rain will fall and where surface water flooding will occur’. In April 2009, the ERA became an
operational service provided to the EA professional partners and Category 1 and 2 responders – organizations with legal
responsibilities to respond to flooding. ERAs are intended as a first step towards warnings for surface water flooding
(SWF). This article assesses the effectiveness of the ERA pilot at predicting SWF. ERAs relating to three case study
areas and issued during the pilot were compared with reports of SWF, identified from the data available relating to all
sources of flooding. While data were found to be lacking in quality and consistency, it is possible to provide an initial
assessment of the effectiveness of the ERA pilot. The existing ERA rainfall thresholds do not relate directly to SWF in
all areas. As the ERA service specifically intends to warn for rainfall likely to cause severe SWF in urban areas, it is
impossible to draw strong conclusions without further investigating the magnitude of flooding which occurred both when
an ERA was and was not issued. As SWF magnitude and extent data were extremely sparse, it is recommended that
systematic collection of comprehensive data relating to flood events should be implemented nationally. This will facilitate
more accurate investigations of the links between rainfall intensity and SWF, which it is vital to understand in order to
provide the most effective surface water flood warnings possible. Copyright  2011 Royal Meteorological Society

KEY WORDS extreme rainfall alerts; surface water flooding; pluvial flooding; flood warning; warning effectiveness
Received 23 November 2010; Revised 24 May 2011; Accepted 3 June 2011

1. Introduction 2008; Golding, 2009) to take account of positional

The damaging floods of summer 2007 in England and
Wales led to the Pitt Review recommending that ‘the
Met Office and Environment Agency (EA) provide an
early assessment of the costs, benefits and feasibility of
techniques which can predict where rain will fall and
where surface water flooding will occur’ (Pitt, 2008). It
is believed that approximately two thirds of the summer
2007 flooding was surface water rather than fluvial
in nature (Environment Agency, 2007; Pitt, 2008). No
warning service was in place for this type of flooding,
caused by intense rainfall combined with saturated or
urbanized ground surfaces. The primary dependence
of surface water flooding (SWF) on intense rainfall
makes the successful prediction of such rainfall events
fundamental to the development of a warning service.
The Meteorological Office (MO) has developed
cutting-edge probabilistic forecasting techniques, based
on a process called time-lagged fuzzy jiggling (Orrell,
* Correspondence to: A. P. Hurford,, HR Wallingford, Howbery Park,
Wallingford, Oxfordshire OX10 8BA, UK.
E-mail: a.hurford@hrwallingford.co.uk
uncertainty in a forecast, while successive model runs
account for developmental (i.e. temporal) uncertainty.
Using the MO 4 km numerical weather prediction (NWP)
model, forecasters use an estimate of a typical error in
the positioning of the rainfall (e.g. 40 km) for a particular
forecast length. Then, the domain is tiled into 36 × 36 km
squares. The squares are then moved around into every
possible position on the 4-km model grid within the
bounds of the typical forecast error (e.g. 40 km in each
direction gives 20 × 20 possible positions). For each
square, all the positions in which there is an ‘extreme’
rainfall total (e.g. >40 mm in 3 h) are added up and
divided by the total number of possible positions to give
the probability of an extreme event occurring somewhere
within that square (back at its original position). This
is done for each new forecast and the final probability
is made up of a combination of the probabilities from
the latest and previous forecasts (so called ‘time-lagged’
ensemble) with some weighting applied to give more
credence to the more recent forecasts for the shorter fore-
cast lengths (Roberts, pers. comm.). These techniques are
able to predict, with around 50% accuracy (Roberts et al.,

Copyright  2011 Royal Meteorological Society

 A. P. HURFORD et al.

2008), when one of a number of thresholds for ‘extreme’ Table I. FEH 1-in-30-year rainfall depths (mm) for eight cities
rainfall are likely to be exceeded somewhere within a and rainfall thresholds (mm) defined for the use by the ERA
geographically defined polygon. pilot service.
The forecasting techniques described above formed
Storm duration (h)
the basis, first for a pilot and since April 2009 for an
operational Extreme Rainfall Alert (ERA) service [Flood 1 3 6
Forecasting Centre (FFC), 2010], provided to the EA
professional partners and Category 1 and 2 responders. Central London 37.3 48.4 67.4
Category 1 and 2 responders are those organizations that Manchester 32.5 42.2 58.5
have responsibilities to respond to flooding under the Birmingham 31.6 42.1 60.4
Newcastle 26.3 38.2 61.2
Civil Contingencies Act 2004 [Her Majesty’s (HM) Gov-
Bristol 31.5 43.8 66.3
ernment, 2004] and include such organizations as local Cardiff 31.2 43.9 67.7
authorities, the EA, emergency services and utilities com- Plymouth 32.6 46.2 71.8
panies. Glasgow 24.1 35.4 57.6
The ERA service was not intended to be an operational ERA thresholds (mm) 30 40 50
surface water flood warning service, but rather a step
towards such a service. The ERA service provides
warnings of extreme rainfall and defines it as those • Advisory alert – 10% probability
intensities likely to cause severe SWF in urban areas. • Early alert – 20% probability
The objective of this article was to assess the effec- • Imminent alert – 40% probability
tiveness of the ERA service as a surface water flood
warning. This assessment is based on an investigation of In a typical event, the three types of alert would
the links between the issuance of ERAs during the ERA be expected to be issued in sequence – advisory, early
pilot and observed surface water flood events rather than then imminent – as the lead time of the forecast event
a preliminary assessment of the performance of ERAs in decreases and the likelihood of occurrence increases
predicting exceedance of rainfall thresholds as previously (Roberts et al., 2008). As the ERA service has become
assessed by Roberts et al. (2008). The research was spon- operational, the types of alerts issued have been revised
sored by the EA/Defra and focused on the pilot period of but remain based on the level of probability (FFC, 2010).
the ERA service – 1 July 2008 to 28 February 2009.

2. Defining ‘extreme’ rainfall
Intensity thresholds for the ERA service were defined
by a scoping study (Halcrow, 2008). These thresholds
assumed that rainfall intensities exceeding the 1-in-30-
year return period with which most urban drainage sys-
tems are designed to cope would overwhelm the system
and cause SWF. On this basis, Flood Estimation Hand-
book (FEH) (Institute of Hydrology, 1999) rainfall inten-
sities for eight major UK urban centres were averaged and
rounded resulting in the thresholds as shown in Table I
(Halcrow, 2008).
Forecasting of where and when these extreme intensity
rainfall thresholds will be met is not the sole challenge
in providing timely and accurate warnings for SWF,
although it is certainly the most technically difficult
owing to the highly complex and chaotic nature of
atmospheric systems. Major considerations in defining
thresholds which are relevant to the generation of SWF
include the characteristics of the catchment on which the
rainfall is falling and the severity of SWF for which it is
economically viable to provide warnings.
3. Methodology
This section describes the process during the study from
the selection of suitable case study areas for the collection
and processing of data and the analysis of those data to
provide the results discussed in the next section.
3.1. Selection of case study areas
ERAs present a range of information relating to the
forecast timing and intensity of the rainfall event (FFC,
2010). They also show a polygon overlaid on a map of the
UK defining the geographic area within which the prob-
ability of occurrence is above a certain threshold – the
minimum for issuing of an alert being 10% probability.
Three case study areas were selected for this research by
plotting all the polygons defined in the ERAs being anal-
ysed on a GIS map of England and Wales and analysing
the number of ERA events (a forecast extreme rainfall
‘event’ might have more than one alert issued in relation
to it, as the probability of occurrence changes or as the
predicted location changes) which related to each county.
Figure 1 shows the resulting map highlighting in black
the three case study areas:

• Cumbria,
2.1. Function of the ERA pilot service
• Southern Wales, and
During the ERA pilot phase, three types of alert were • South-west England.
issued based on the predicted probability of the extreme
rainfall event occurring at the county level. The proba- It was recognized that many of the weather systems
bilities used to trigger the issuing of an alert were: affecting southern Wales also affect the south-west of

Copyright  2011 Royal Meteorological Society Int. J. Climatol. (2011)

 RAINFALL ALERTS FOR PREDICTING SWF

Figure 1. Map of the number of ERAs referring to each county.

England and some ERAs referred to both areas as being

within the polygon where the forecast extreme rainfall
could occur (Figure 2). It was considered inappropriate
to choose only one of these areas to use as a case study as
both scored high for ERA events which applied to them.
Data relating to events in the three case study areas were
provided by 52 ERAs.

3.2. Data availability, collection and processing

Contact was made with relevant regional EA offices to
enquire about the availability of flood event data and
to utilize existing contacts between the EA and the
professional partners or Category 1 and 2 responders who
might gather these data. Figure 2. Example of ERA warning polygon (marked A) covering both
South Wales and Southwest England, the Met Office.
3.3. Flood event data
It was possible to obtain 1393 reports of flooding from from those which were more likely to have originated
all sources. These (largely qualitative) data were analysed from another source, such as fluvial flooding, coastal
to filter those likely to have been surface water events flooding or leaks from water mains. This analysis utilizes

Copyright  2011 Royal Meteorological Society Int. J. Climatol. (2011)

 A. P. HURFORD et al.

the information available in relation to each event, which

is variable, but always includes at least a location (the
majority with grid reference, otherwise with place name).
The best quality data are available from the fire and
rescue service, as these often include time of call-out.
The problem with these data is that log reports relating
to flooding can also include events such as burst pipes
or leaking washing machines for which the service is
sometimes mobilized.

3.4. Identifying surface water flood events

The data obtained are of a limited quality for this
kind of study, in terms of their consistency and their
lack of detail regarding flood timings and magnitude,
but are nonetheless currently the best available. These
facts dictate that sophisticated analysis for filtering is
impractical although best use is made of the available
data for a high-level assessment.
The primary analysis was carried out by plotting
the flood location data in a GIS and assessing the
likelihood of the source being surface water. The main
criterion for assessment is the proximity of the event
location to a watercourse of some kind (being close
to a water course suggesting a fluvial flood event), as
shown on 1 : 10 000 scale Ordnance Survey mapping.
The following other factors, based on broad assumptions,
are all taken into account as no further information is
available:
Figure 3. Location of all flood events with symbol coding for those
• proximity to the coastline would suggest the possibility events assumed to be SWF or flooding from other sources.
of coastal flooding, so such events are ruled out
as SWF; approach and more appropriate than including events
• incident angle of watercourses/drainage channels in the subsequent analysis which are unlikely to be
approaching the location – where the flood location is SWF. Inclusion of events unlikely to be SWF and
on the outside bend of a small watercourse, this would therefore unrelated to the ERAs would be likely to
more likely suggest a fluvial event; decrease the apparent success rate of the ERA service
• ‘sinking’ and ‘issuing’ marked on map relating to in predicting SWF.
watercourses on either side of a location – flooding Using the above method, the initial total of 1393 flood
was considered to be more likely upstream (i.e. in the reports were reduced to 286 surface water flood events for
vicinity of ‘sinking’) due to overloading of a point further analysis, representing 20.5% of the original data
where water enters a hydraulic flow control (such as a set. The full data set is shown in Figure 3, with different
culvert). Issuing is where a water course emerges from symbols representing the SWF events and flooding from
a feature such as a culvert and this was considered other sources.
unlikely to be point vulnerable to flooding due to the Once surface water flood events were identified, these
limit of flow through the culvert (for example) – the were cross-referenced both spatially, using a GIS analysis
presence of either of these was taken as an indicator of where SWF events intersected ERA area polygons and
that the event was not SWF as defined by the EA for temporally with a manual comparison of time periods to
the purposes of this study (Priest et al., 2010); which ERAs referred.
• mention of blocked drains in the incident descrip-
tion – this is considered SWF as there is no way to
discriminate between blocked drains and surcharging
4. Results and discussion
drains even where mentioned; and
• mention of ‘out of bank flows’ or rain causing a river The effectiveness of the ERA system in predicting SWF
to increase flow – this is not considered as SWF. may be assessed by considering the timing and location
separately. This allows the complex issues of whether an
There are no doubt occasions where an event which ERA was linked to SWF in time or space or both can be
could have been SWF was excluded on the basis of considered independently – an alert may be successful in
the above factors, but this is considered a conservative predicting the flooding in terms of the time of occurrence

Copyright  2011 Royal Meteorological Society Int. J. Climatol. (2011)

 RAINFALL ALERTS FOR PREDICTING SWF

Table II. Explanation of the values in Tables III and IV.

Advisory issued Early issued Imminent issued ERA not issued

Flood hit Number of alerts which correctly predicted SWF events Number of Dates on
which SWF events
occurred but no ERA
was issued
Flood missed Number of alerts which missed SWF events geographically, i.e. outside alert polygon –
geographically
No flood event Number of alerts which could not be correlated with a SWF event –

while missing the location geographically. Likewise, the Table III. Contingency table for the individual and combined
geographical location may be correctly predicted, but the case study areas relating ERAs issued to SWF events.
timing incorrect.
It is acknowledged that the ERA system predicts the Advisory Early Imminent ERA not
timing of rainfall threshold exceedence rather than asso- issued issued issued issued
ciated flood event occurrence, but where timings were Cumbria
incorrect, they were sufficiently incorrect to conclude that Flood hit 2 1 0 1
the system had not worked effectively as a SWF warning Flood missed 2 1 0 –
system on that occasion. geographically
No flood event 10 4 0 –
4.1. Linking ERAs to observed SWF events Southern Wales
To assess the links between issuance of ERAs and Flood hit 4 5 0 17
occurrence of observed SWF events, each alert issued Flood missed 4 5 0 –
(regardless of type/level) was considered individually, geographically
even when relating to the same rainfall event. This No flood event 4 1 0 –
is felt to be a fair and practical method of assess- South-West England
ment: fair, as a decision is made each time an alert Flood event 3 2 1 4
is issued and practical, as for this type of analysis Flood missed 3 2 0 –
the decision must be assumed to be independent of all geographically
those made previously to avoid complications relating No flood event 4 2 1 –
to the success of individual alerts in predicting SWF. Combined sum of three case study areas
The results of the analysis examine the links between Flood event 9 8 1 22
ERAs issued and observed SWF. Table II serves as an Flood missed 9 8 0 –
geographically
explanation of the result data in subsequent Tables III
No flood event 18 7 1 –
and IV.
While the indications given by results in Table III for
individual case study area vary, by combining the data
a clearer picture emerges. The final section of Table III occasions when SWF events occurred after imminent
clearly shows that the advisory alerts issued with longer alerts would be higher than after early alerts purely on
lead times (and therefore expected to be less accurate; the basis of a higher probability forecast of extreme
Roberts et al., 2008) were less often linked to SWF rainfall occurring. The disagreement of the data with
events than the early and imminent alerts issued closer this is likely to be due to the low number of occasions
to the time of the storm commencement. Similarly, the when imminent alerts were issued, but this would need
early alerts were less often linked to flooding than the to be confirmed by analysis of a more extensive data
imminent alerts. set. The incidence of SWF occurring even where rainfall
These figures can be compared with data produced thresholds are met depends of course, on a range of
by the MO’s own verification of the ERA pilot system hydrological factors such as antecedent soil moisture (in
(Roberts et al., 2008) in which the success of the system non-urban environments). It is important to note that
was analysed in terms of rainfall actually meeting the no data are available regarding the magnitude of the
thresholds as predicted. Table IV shows the MO’s data observed surface water flood events used as input for
with an additional column to show data from this this analysis.
assessment, linking the ERAs to flooding. The MO’s verification data are based on much more
Although based on limited data in terms of both quality extensive data regarding rainfall events than that on
and quantity, these figures suggest that the ERA system which this analysis linking to flooding is based. Although
has performed better in terms of ‘predicting’ SWF, than it there are uncertainties associated with the accuracy of
has in terms of predicting rainfall threshold exceedances radar rainfall data, it can be assumed that these data
accurately. It would be expected that the percentage of are more accurate than the reports of flooding available,

Copyright  2011 Royal Meteorological Society Int. J. Climatol. (2011)

 A. P. HURFORD et al.
Table IV. Success of ERAs in predicting breaches of rainfall thresholds and links to SWF events.
ERA product issued Probability of
exceeding rainfall
threshold (%)
Advisory/guidance 10
Early alert 20
Imminent alert 40
Table V. Date-based assessment of links between ERAs and
SWF in individual case study areas.
ERA issued ERA not issued
Flood event 10 23
No flood event 19 – clusions without further investigating the magnitude
being at least less subjective and by nature quantitative.
The MO found that ERAs were only issued on 50% of
occasions when ERA rainfall thresholds were met. This
indicates that more alerts (twice as many) should have
been issued.
Table V shows data relating only to the dates when
SWF occurred or ERAs were issued. These data suggest
that ERA products were only issued for 30% of dates
when SWF occurred. This number is approximate and
intended only to give a figure for comparison with
the success in predicting rainfall which exceeded ERA
thresholds. As the analysis was carried out on whole
case study areas rather than individual counties and as
the ERA area polygons are often extensive, a single alert
could relate to more than one case study area, particularly
in the case of southern Wales and South West England.
Although this was not explicitly investigated, these cases
are not likely to account for more than one or two of the
alerts issued.
5. Conclusions
Data quantity and quality are factors limiting the accuracy
of the research reported here, although it proved possible
to develop a methodology and to provide an initial
assessment of the effectiveness of the ERA service as
a surface water flood warning.
It is recognized that the criteria for selecting surface
water flood events from the greater number of flood
reports are imperfect, but it was necessary to filter those
events which could be considered most likely to have
resulted from surface water. For this analysis, the EA
required that flooding from drainage systems was not
to be considered as SWF, leading to the assessment of
flooding related to drainge features as not relevant to this
analysis.
The combined conclusion from the results reported,
taking into account the extremely limited quality and
quantity of data available, is that the existing ERA
rainfall thresholds do not relate directly to SWF in
Copyright  2011 Royal Meteorological Society
Percentage of Percentage of
occasions that rainfall occasions that SWF
thresholds were met events occurred
14 33
26 53
50 50
all areas. However, no data were available relating
to the magnitude of the flood events used for this
analysis. As the ERA service specifically intends to
warn for rainfall likely to cause severe events in
urban areas, it is impossible to draw strong con-
of flooding which occurred both when an ERA was
and was not issued. Many of the SWF events anal-
ysed for this study were in urban areas, although not
in the middle of major cities, and so the relevance
of the ERA system is open to some debate regard-
ing the level of urbanization to which it should be
applicable.
The research carried out has highlighted the need
to introduce and maintain a consistent quality of data
relating to flood events to facilitate further investigation
of the links between rainfall intensity and flooding
magnitudes and durations. The implications for the ERA
service are that it has proved to be an effective first
step in moving towards a warning service for SWF
but requires development to provide more focused and
accurate warnings with a sufficient lead time for an
effective response to be implemented.
Acknowledgements
The research was a Science Project (SC080034) spon-
sored by the Environment Agency. The authors are grate-
ful to staff from the Environment Agency and Fire and
Rescue services who provided their data for this research
and to members of the project board – Katharine Evans,
Jim Walker, Mel Andrews, Faye Burrows, Tony Deakin,
Andy Fraser, Joanne Grimshaw, Chris Strong and Claire
Sunshine.
References
Environment Agency. 2007. Review of 2007 summer floods, Environ-
ment Agency, Bristol. Available at http://publications.environment-
agency.gov.uk/pdf/GEHO1107BNMI-e-e.pdf (Accessed 23 August
2010).
Flood Forecasting Centre (FFC). 2010. Extreme Rainfall Alert (ERA)
Service User Guide. Available at http://www.ffc-environment-
agency.metoffice.gov.uk/media/pdf/FFC ERA User Guide.pdf
(Accessed 23 August 2010).
Golding B. 2009. Long lead time flood warnings: reality or fantasy?
Meteorological Applications 16: 3–12.
Halcrow. 2008. Proposed Pluvial Flooding Trial Service, Draft Report
prepared for the Met Office. April 2008.
HM Government. 2004. Civil Contingencies Act, 2004. 2004c.36.
Available at http://www.opsi.gov.uk/acts/acts2004/ukpga 20040036
en 1 (Accessed 7 December 2009).
Int. J. Climatol. (2011)
 RAINFALL ALERTS FOR PREDICTING SWF
Institute of Hydrology. 1999. Flood Estimation Handbook. Institute of
Hydrology: Wallingford.
Orrell R. 2008. Extreme Rainfall Alert Service, Presentation at the
SNIFFER-SEPA pluvial flooding meeting, Edinburgh, 25 November
2008. Available at http://www.rise-frm.org.uk/exe/download.asp?
rise/ERAscience Orrell.pdf (Accessed 1 September 2010).
Pitt M. 2008. Learning Lessons from the 2007 Floods. The Pitt Review
Cabinet Office: London.
Copyright  2011 Royal Meteorological Society
View publication stats
Priest SJ, Parker DJ, Hurford AP, Pardoe JL, McCarthy SS, Tapsell
SM. 2010. Surface Water Flood Warning Scoping Project – Final
Report, Environment Agency Science Programme Report, Science
Report, SC080034/SR1.
Roberts N, Pierce C, Mittermaier M. 2008. Preliminary performance
evaluation of the Extreme Rainfall Alert Service, Summary of
performance for the period from 2 July to 31 October 2008 inclusive,
Met Office, Exeter, UK.
Int. J. Climatol. (2011)