Comparison of Damage Assessment Maps

Derived from Very High Spatial Resolution

Satellite and Aerial Imagery Produced for
the Haiti 2010 Earthquake
Christina Corbane,a) Daniela Carrion,a) Guido Lemoine,a)
and Marco Brogliaa)

Following the devastating M7.2 earthquake that affected Haiti on 12 January

2010 two types of building damage assessment maps were produced: 1) area-
based damage assessments using pre- and post-event satellite imagery and 2)
detailed building-by-building damage assessments using post-event aerial pho-
tography. In this paper, we compare the reliability and the usability of area-
based damage assessment maps from satellite imagery with respect to the
detailed damage assessment from aerial data. The main objective is to better
understand how cooperative rapid mapping can steer the more detailed assess-
ments that are typical in determining postdisaster recovery and reconstruction
efforts. The results of these experiments indicate that damage assessment maps
based on satellite data are capable of capturing the damage pattern, mainly in
areas with a high level of damaged and many collapsed structures. However,
these maps cannot provide the level of information needed for the quantification
of damage intensity. [DOI: 10.1193/1.3630223]

INTRODUCTION
The Haiti earthquake resulted in extensive damage to building structures. In the after-
math, an unprecedented effort was undertaken by scientists and engineers from all over the
world to exploit the postdisaster remote sensing data for mapping the physical damage to
structures and for aiding in the recovery effort. This unprecedented response was further
stimulated by the fact that unrestricted access to very detailed and up-to-date remotely
sensed imagery was quickly made available to anyone willing to help. The effort to process
these huge data sets was facilitated by novel collaborative systems, many of them available
in the public domain.
Building damage is one of the primary causes of human casualties (Hengjian et al.
2003). The ability to rapidly determine the impact of an earthquake on structures is very im-
portant for emergency planning, human recovery, subsequent reconstruction efforts, and
future disaster risk preparedness. Field surveys enable an accurate damage assessment but
they may require considerable time and resources and are generally nonexhaustive; field
surveys are usually conducted on a sample basis because it is practically impossible to

a)
European Commission, Joint Research Centre, Ispra, Italy

S199
Earthquake Spectra, Volume 27, No. S1, pages S199–S218, October 2011; V
C 2011, Earthquake Engineering Research Institute
S200 CORBANE ET AL.

survey all individual buildings in the field, especially for large affected areas. Alternatively,
post-earthquake images acquired via satellite and aerial sensors have become valuable data
sources for damage detection (Turker and Sumer 2008).
A variety of studies have been carried out in the last few years to assess damages using
remotely sensed imaging technologies. The majority of these image analysis studies
obtained damage assessments based on medium or fine spatial resolution satellite images
(Ping and Xiaxin 2005, Rathje et al. 2005) and analyzed the changes between pre- and post-
earthquake images. These studies concluded that remotely sensed imagery can help in
delineating damaged regions, primarily in heavily affected areas. With the availability of a
variety of very high spatial resolution (VHR) satellite data (pixel < 1 m) both automated
analysis and visual interpretation were employed for the identification of damage in built-up
environments. The following studies fall under the automated analysis category: Liu et al.
(2004) used textural features, Matsuoka et al. (2004) exploited edge-based techniques,
Gusella et al. (2005) developed an object-based approach for determining the number of
collapsed structures, and Brunner et al. (2010) proposed a data fusion approach using VHR
optical and synthetic aperture radar (SAR) imagery. Studies involving visual interpretation
include the following works: Chiroiu and André (2001), Chiroiu (2005), Saito et al.
(2005),Yamazaki et al. (2005). All of these studies concluded that photo-interpretation
could be used as a reliable technique to assess building damage in VHR satellite imagery.
Even though the advent of VHR satellite data has brought new insights into the assessment
of damaged buildings, digital aerial photography with a decimetric spatial resolution is still
a commonly used source of information for retrieving detailed damage assessments at the
local scale. Both automated analysis and visual interpretation are also applied and tested on
aerial data. Turker et al. (2004, 2008) introduced an approach for detecting collapsed build-
ings using the relationship between buildings and their cast shadows. Rezaeian and Gruen
(2007) developed a novel approach for feature extraction of damaged buildings based on a
comparison of stereo pairs of aerial images before and after an earthquake. Ogawa and
Yamazaki (2000) evaluated the damage situation after the 1995 Kobe, Japan, earthquake
using photo-interpretation of aerial data. Hasegawa et al. (2000) investigated the influence
of differences in structure type, building density, and human interpreter on the photo-inter-
pretation results.
Despite the numerous automated approaches for the extraction of structural damage
from satellite and aerial imagery, classical photo-interpretation remains the most commonly
used method for a reliable assessment of damage.
In the case of the Haiti earthquake, both satellite and aerial imagery were analyzed by
visual interpretation for damage identification and mapping. The key novelty compared to
earlier disastrous earthquakes (e.g., Bam, Iran, in 2003, Chengdu, China, in 2008) was the
almost immediate (less than 24 hours after the disaster) availability of VHR imagery
acquired by GeoEye (0.5 m spatial resolution) and its immediate release into the public do-
main. Furthermore, the availability of pre-earthquake VHR imagery made visual pre- and
post-event comparison possible. Several rapid mapping initiatives used the publicly avail-
able images to produce area-based damage maps that were delivered to emergency response
coordinators within the first week following the earthquake. Five days after the earthquake
(from January 17 onwards) aerial campaigns managed by Google Inc., and later the World
COMPARISON OF DAMAGE ASSESSMENT MAPS S201

Bank-ImageCAT-RIT (Rochester Institute of Technology) Remote Sensing Mission,

acquired image coverages of the most-affected administrative units (e.g., Port-au-Prince,
Carrefour, Léogâne, Gressier, Petit-Goâve, and Grand-Goâve) at a spatial resolution of 0.15
m. With all this imagery in hand, a comprehensive building-by-building damage assessment
was performed on the basis of a visual comparison between pre-earthquake satellite images
and post-earthquake aerial photography. This extensive effort was carried out jointly by
teams from the United Nations’ Operational Satellite Applications Programme (UNOSAT),
the European Commission’s Joint Research Centre (JRC), and the World Bank/ImageCAT
in support of the Post-Disaster Needs Assessment (PDNA), and resulted in a point-based
damage assessment for 13 different administrative units in Haiti.
The key differences between the rapid mapping outputs and the detailed aerial-based
assessment, apart from their intended end uses, are timing, completeness, and accuracy
(Corbane et al. 2011). The accuracy of the assessment is mostly dependent on the differen-
ces in spatial resolution between images acquired by satellites and aerial platforms. Follow-
ing the release of the Google Inc. aerial photography, the JRC team performed a cross-anal-
ysis of the point-based damage assessments on a sample of grid cells using both the satellite
and aerial images. This analysis was carried out by the same interpretation team, so no bias
was introduced by a change of operators. Time constraints for this cross-analysis were the
same in both cases. On average, five to eight times more damage points were identified in
the aerial images depending on the type of built-up structure and the local earthquake
impact. The overall accuracy of the aerial-based damage assessment was estimated to be
65% (Corbane et al. 2011, Lemoine 2010) in comparison to field survey results in particular
considering GPS-tagged ground reference photos. This low accuracy was attributed to the
omission of damage cases that were difficult to assess in near-nadir looking imagery, resid-
ual operator errors, and the general subjectivity expected in a visual interpretation exercise
involving many operators. The significantly lower damage detection rates in the satellite im-
agery, however, show that absolute damage figures from this source would constitute a
gross underestimation of the real damage. The results of the detailed building damage
assessment were used as an input to the PDNA and, more specifically, to the Damage and
Loss Assessment (DaLA) for the housing sector. It was the first time that the PDNA was
based on a damage assessment produced with remotely sensed data. The report also
included information on the accuracy of the estimates. Despite the level of error, the
detailed damage assessment from remotely sensed data represents the only exhaustive
source of information on damage, and it was delivered well in time for the Donors’ Confer-
ence in New York on 31 March 2010.
This study examines the reliability and usability of area-based damage assessment pat-
terns derived from satellite imagery as compared to the detailed point-based damage assess-
ment derived from aerial data. While reliability is generally defined as the degree to which
the information contained in a map is consistent, dependable, and repeatable, usability is
directly related to the communication of the map to third parties. It can be defined as the
degree to which a map provides relevant information.
For the experiment four area-based damage maps, covering a common area in Port-au-
Prince were analyzed (Figures 1 and 2). The four maps were provided by four different
well-known and experienced emergency mapping service providers and were publicly
S202 CORBANE ET AL.

Figure 1. Location of the study area in Port-au-Prince.

available on the Web. We assumed that the general experience of the different photo-inter-
preter teams was equal. As a reference, the detailed point-based damage assessment was
used. The analysis focused on the pairwise comparison of each of the four area-based dam-
age assessments with respect to the point-based map instead of an intercomparison of the
maps or an absolute accuracy check with respect to the ground data acquired. Consequently,
different map comparison strategies were explored including the traditional agreement/
disagreement method and other methods that look at the spatial distribution of damage cate-
gories between the map pairs.

Figure 2. The four area-based damage assessment (DA) maps derived from satellite imagery
with modified colors and without original legends.
COMPARISON OF DAMAGE ASSESSMENT MAPS S203

This paper is organized as follows: First, the dataset used in the experiment is described,
and then the different approaches used to compare maps are presented. The results of each type
of analysis are then presented and followed by a discussion on the potential advantages and lim-
itations of satellite-derived damage assessment maps, both in terms of reliability and usability.

DATA SET DESCRIPTION

AREA-BASED DAMAGE ASSESSMENT MAPS DERIVED FROM SATELLITE

IMAGERY
Out of the many satellite-derived damage assessment map products produced immedi-
ately after the 12 January 2010 Haiti earthquake, four area-based maps extending over a
common area of approximately 5 x 4 km2 in Port-au-Prince (centre coordinate: 18.544 N,
72.337 W, Figure 1) were selected. To keep bias to a minimum, each map from the collabo-
rating map providers was identified by a number, the original colors used in the maps were
modified, and the map legends were not published (Figure 2).
The four maps had a number of similarities and dissimilarities. In terms of similarities:
i) All of the four maps were based on the same input 0.5 m resolution satellite image
(GeoEye) for the identification of damage.
ii) The same methodology, computer-aided visual interpretation of pre- and post-earth-
quake imagery was used for the generation of the maps.
Regarding the dissimilarities:
i) The mapping units used for representing the damage categories were not the same.
Maps 1 and 2 used an urban block representation whereas Maps 3 and 4 used a square
grid representation. Furthermore, the urban blocks were defined differently in the first
two maps and the sizes of the grids differed between the two grid-based maps (i.e.,
square cells measured 250 x 250 m2 in Map 3 and 200 x 200 m2 in Map 4).
ii) The categorization of damage levels differed, both in the labeling of the damage
categories and in the number of classes. For instance, in Map 2, the different dam-
age categories were defined according to the proportion of buildings showing visi-
ble damage (e.g., 0–10% of the urban block is damaged). In Map 4, the actual
counting of visibly damaged or destroyed structures was used for defining the dam-
age categories (e.g., more than 15 damaged or destroyed structures per 200 m2
cell). Furthermore, in Map 3, the damage categories were based on an increasing
scale of damage grades, from low to high visible damage, without any explicit ref-
erence to the quantitative rules governing the classification.
The dissimilarities presented above introduced challenges for the analysis, especially
for the comparison of the maps with the point-based damage map.

POINT-BASED DAMAGE ASSESSMENT MAP DERIVED FROM AERIAL

IMAGERY
As a reference for the analysis, the detailed point-based damage assessment derived
from aerial imagery was used (Figure 3). This product was generated by computer-aided
S204 CORBANE ET AL.

Figure 3. Point-based building-by-building DA using post-event aerial imagery. The classifica-

tion of damage is based on the European Macroseismic Scale (EMS) 1998. The road network is
reproduced from the OpenStreetMap data set (OSM 2010).

visual interpretation of pre-earthquake satellite imagery and post-earthquake aerial imagery.

The methodology consisted of the assignment of different damage grades to individual
buildings, using the European Macroseismic Scale 1998 (EMS-98; Grünthal 2001). This
scale defines five damage grades: Grades 1 to 5 should ideally represent a progressive
increase in the strength of shaking for different types of masonry and reinforced-concrete
buildings. The damage descriptions for these two building types were specifically selected
because they most closely represented the types of construction in Haiti.
The main qualitative criteria for recognizing damage grades on overhead or nadir im-
agery include, but are not limited to, the shifting of building perimeters, a lack of definition
of perimeter walls, or obvious debris surrounding the damaged buildings. However, for
some types of failure, using only nadir imagery will result in missed damage assignments.
For example, many soft-story building failures were observed in the Haiti earthquake. These
failures usually occur when the upper stories of a building collapse directly onto the first or
bottom story. They are common when the upper stories are constructed to be much stiffer
than a more open or softer bottom story. In the nadir images, these types of failures are
extremely difficult to identify without the use of oblique imagery or ground survey data. In
COMPARISON OF DAMAGE ASSESSMENT MAPS S205

addition, more minor or moderate damage to buildings that does not result in building col-
lapse is also difficult to identify in the nadir images (Corbane et al. 2011).
Accordingly, the damage levels used for marking individual buildings on the aerial pho-
tographs were limited to the higher grades of the EMS-98 scale (e.g., substantial to heavy
damage [EMS Grade 3], very heavy damage [EMS Grade 4], and destruction [EMS Grade
5]).
All building centroids were marked, including those that did not exhibit visible damage
(EMS Grade 1). The overall point data set consisted of more than 300,000 labeled points. A
key advantage of having such an exhaustive point data set was the ability to generate abso-
lute and relative damage statistics for any arbitrary spatial sampling. The absolute damage
density is defined as the number of damaged buildings per sampling unit, whereas the rela-
tive damage density is equal to the percentage of damaged buildings relative to all buildings
in the sampling unit. Note that the exhaustive building labeling did not diversify by building
type and neither did the map products. The absolute number of buildings and the relative
density of damage can be observed in Figure 3, which shows an overview of the point-
based aerial assessment. Occurrences of Damage Grades 3, 4, and 5 showed a pattern that is
distinctively different from the overall building density in the area of interest. For instance,
the lower left quadrant of the map shows the downtown commercial district (Morne a Tuff)
with a low density of buildings but with a high occurrence of damage (i.e., low absolute
damage density but high relative damage density). The lower central map area includes the
heavily affected Fort National and Saint Antoine neighborhoods, which showed the highest
absolute and relative damage densities because buildings in this area were generally small
and very densely packed. Other areas showed strong local variation in both absolute and rel-
ative damage densities.
The point-based damage assessment map was validated against a variety of reference
data sets including combined GPS photo-surveys that encompassed over 6,000 validation
locations selected according to a stratified sampling schema taking into account the three
following criteria: population density, building density, and land-use classes. The break-
down of the number of buildings assessed can be found in Table 1.

Table 1. Breakdown by commune of the number of buildings surveyed in the field using
ground photos

Commune Number of buildings surveyed on the ground

Croix-des-Bouquets 343
Cite Soleil 243
Carrefour 1,305
Léogâne 97
Delmas 735
Tabarre 648
Pétion-Ville 464
Gressier 50
Port-au-Prince 2,607
S206 CORBANE ET AL.

Cross-comparison of survey point labels and the damage labels from the photo-interpre-
tation exercise produced an overall level of accuracy of 65% (Corbane et al. 2011, Lemoine
2010). The validation activities showed that one of the major problems is the omission of
damaged buildings that have pancaked or suffered from so-called soft-story failure,
whereby a lower floor has collapsed but the upper structure of the building has remained
intact. Typical commission errors occur in city areas where unfinished construction work is
classified as damaged. The comparison also showed that photo-interpretation is more accu-
rate for Damage Grades 4 and 5 only. The high level of confusion in Damage Grade 3 can
be explained by the fact that the outer structure of buildings suffering this level of damage
remains mostly intact. Debris in the immediate surrounding area is used as an indicator for
Grade 3, but appears to be unreliable in this respect. For the purpose of this study, damage
Grades 4 and 5 were grouped together because these grade groups include houses that are
beyond repair (i.e., “total losses” in a reconstruction sense) and are the most likely locations
where human victims are present (i.e., a focus for search and rescue).

MAP COMPARISON METHODOLOGY

The focus in this study was on a pairwise comparison of each of the four area-based
damage assessments with respect to the point-based map instead of an intercomparison of
the maps or an absolute accuracy check with respect to the ground data acquired. The partic-
ularity of this approach resides in the assessment of the potential advantages and limitations
of satellite-derived damage assessment maps, both in terms of reliability and usability, con-
sidering their similarities and differences. A three-step methodology was developed and is
illustrated in Figure 4.

Figure 4. The three-step strategy to compare satellite-derived maps to aerial-derived DA maps.

COMPARISON OF DAMAGE ASSESSMENT MAPS S207

STATISTICAL DISTRIBUTION OF DAMAGE CLASSES

Descriptive Statistics
In the first step, the consistency of the statistical distribution of the point-based damage
assessment across all four area-based maps was checked prior to the direct map comparison.
This step highlighted significant anomalies present in the classification of damage grades in
the satellite-derived damage assessment (DA) maps. For that purpose, the percentage of
Damage Grades 1, 3 and 4 and 5 of the point-based damage assessment was computed for
each damage category of the four area-based maps.

Aggregation of the Point-Based Data into Area-Based Maps

The map comparison problem required that the compared maps have the same number
of classes and that the classes were labeled in the same manner. To cope with the dissimi-
larities of the area-based maps, mainly in terms of heterogeneous mapping units and clas-
sification of damage categories, it was necessary to aggregate the point-based data set into
four grid- or urban block-based maps using a unique aggregation rule. Some aggregation
rules have been tested, including the four most common methods of ranging of intervals:
equal steps, quantiles, standard deviation, and natural breaks that were applied to the
actual percentage of Damage Grades 1, 3, and 4 and 5. These tests were used for finally
selecting the midpoint values of the distribution of percentages for Damage Grades 4 and
5 as the aggregation criterion. The two advantages of this rule were that it prevented
favoring one satellite-derived map over the other and that it guaranteed the maximum
achievable accuracy for a given satellite-derived map. The four aerial-derived maps result-
ing from the transformation of the point-based DA into area-based DA had the same num-
ber of categories and the same labels as the four satellite-derived maps under analysis.
This allowed a direct pairwise comparison between satellite- and aerial-derived damage
assessments.

ANALYSIS OF MAPS’ CHARACTERISTICS: COMPOSITION AND

CONFIGURATION
After Gustafson (1998) and Csillag and Boots (2004) it is possible to identify two sets
of characteristics of categorical maps: composition, which relates to the agreement/disagree-
ment between the different map categories, and configuration, which refers to characteris-
tics of spatial distribution of the map categories.
Map composition analysis is based on comparing each area-based aerial map derived
from the aggregation of the point-based data set to its respective area-based satellite map.
This classical approach tends to look for the degree of agreement/disagreement between sat-
ellite-aerial DA map pairs based on a cell-by-cell or block-by-block comparison. The typi-
cal measure of agreement/disagreement between map categories is the overall agreement
computed from the contingency table (Congalton 2001, Foody 2002). More sophisticated
measures of agreement between maps, also commonly based on the contingency table, have
been defined in the literature. The simplest are based on v2 (Rees 2008) and its various
modifications such as Pearson’s u and C and Tschuprow’s T (Agresti 2002), but one of the
simplest, and probably the most widely used, is Cramér’s V (Cramér 1999). Equivalent to
S208 CORBANE ET AL.

the well-known Kappa statistics, Hagen (2003) created another indicator, K Fuzzy, in which
he used the fuzzy set theory to consider fuzziness of location and fuzziness of category for
map comparison. A review summarizing all the techniques in a single paper is needed due
to the existence of a variety of analytical methods for studying maps’ composition. Despite
the availability of a range of competing and more contemporary measures, the standard
reporting convention, recommended by many researchers, remains the overall agreement
measure (Congalton 1991).
Compared to the previous approach, which accounts for compositional differences
between damage categories only, the analysis of map configuration is used to character-
ize the match (or mismatch) between the spatial distribution of damage categories or the
damage patterns (Csillag and Boots 2004, Hagen-Zanker et al. 2005). A typical way to
quantify the map pattern is the computation of the spatial autocorrelation, that is, the
degree to which characteristics at one location are similar (or dissimilar) to those nearby.
Several measures can be used for quantifying spatial autocorrelation. The choice of mea-
sure depends on the properties of the data and the need to quantify the pattern at the
local or global scale.
Both the characteristics of maps’ composition and configuration were studied in the two
subsequent steps of the map comparison methodology.

Composition Analysis
The contingency tables were built for each pair of area-based aerial map and area-
based satellite map through a cross-tabulation of damage categories represented in the
two maps being compared. In addition to the measure of overall agreement, 2-D cross-
correlation analysis (Gonzalez and Woods 1992) was also employed to estimate the
degree of correlation between the pairs of satellite-aerial DA maps. For two maps show-
ing a good matching of damage categories, the resulting correlation coefficient r will be
close to 1. Inversely, a low correspondence between damage categories will result in a
value of r close to 0.

Configuration analysis
Here, damage patterns between categorical data sets at each individual location were
compared, therefore the Local Moran Index (LMI; Anselin 1995) was chosen because it is
used extensively to identify and quantify local spatial patterns. Statistical considerations are
beyond the scope of this paper but are available in the following references (Anselin 1995,
Getis and Ord 1996). The following describes the LMI statistic in a simplified manner: A
high positive LMI value implies that the location under study has similarly high or low val-
ues as its neighbors, thus the locations exhibit a pattern. Spatial patterns include high-high
clusters (high values in a high value neighborhood) and low-low clusters (low values in a
low value neighborhood). In damage assessment maps, low-low clusters are clusters with a
low occurrence of damaged buildings, while high-high clusters can be regarded as clusters
with a high occurrence of damaged buildings. A high negative LMI value means that the
location under study is a spatial outlier. Spatial outliers are those values that are obviously
different from the values of their surrounding locations (Lalor ad Zhang 2001). Spatial out-
liers include high-low and low-high outliers.
COMPARISON OF DAMAGE ASSESSMENT MAPS S209

To complement the comparison of damage patterns between the pairs of satellite-aerial

DA maps, the 2-D cross-correlation of the resulting LMI maps was also computed to esti-
mate the degree of correlation between the damage patterns exhibited.

RESULTS AND DISCUSSION

STATISTICAL DISTRIBUTION OF DAMAGE CLASSES

Figure 5 presents the distribution of the point-based DA damage grades as a function of
area-based DA damage grades for Map 3. For a given area-based DA damage grade, the
percentages of buildings from the point-based DA labeled with Damage Grade 4 and 5
(very heavy damage and destruction), Damage Grade 3 (substantial to heavy damage) and
Damage Grade 1 (no visible damage) are computed. In each case, the trend that emerged
when computing the distribution of damage classes between the point-based and area-based
DA maps was that Damage Grade 4 and 5 increased for high damage grades and Damage
Grade 1 decreased for high damage grades. Analysis of the spatial distribution of the point-
based DA across the four area-based maps also exhibited these tendencies. This provides
evidence that there is a consistent and logical assignment of damage labels to the satellite-
derived products independent of the mapping unit (grid or urban block), the classification
methodology, and the number of damage categories.
The aggregation rule used to transform the point-based DA data into grid- or urban
block-based maps was based on these findings. Consequently, the percentage of Damage
Grade 4 and 5 was used as the aggregation criterion (Figure 6, i.e., the point-based DA was
transformed into a relative DA density map). For illustration, consider an area-based satel-
lite-derived map with three damage categories (slight, medium, and significant) and the
computed percentage of point-based Damage Grade 4 and 5 in each of these three catego-
ries: 14% slight damage, 22% medium, and 44% significant. The aggregation rule defined
three class intervals using these percentages as the midpoint values.

Figure 5. Map 3: distribution of point-based building damage (in percent) as a function of

increasing damage grades for the area-based DA (low, medium, and high damage).
S210 CORBANE ET AL.

Figure 6. Example of the aggregation of the point-based aerial-derived DA into an area-based

map using the urban-block mapping unit.

RESULTS OF MAP COMPOSITION ANALYSIS

The aggregation technique described above was used to convert the point-based DA
into four area-based aerial DA maps (Figure 7) in order to be able to compare them directly
to their respective area-based satellite DA maps, in terms of map composition.
Measures of overall agreement were computed using contingency tables for each of the
area-based satellite maps (Maps 1, 2, 3, and 4) using the area-based aerial DA maps as a refer-
ence. Contingency tables were built for each pair of area-based aerial map and area-based

Figure 7. The four point-based aerial DA maps aggregated into area-based DA. These maps
have the same number of categories and the same labels as the four satellite-derived DA maps
under analysis (ref. Figure 2).
COMPARISON OF DAMAGE ASSESSMENT MAPS S211

satellite map through a cross-tabulation of damage categories represented in the two maps
compared, in order to evaluate the agreement between satellite- and aerial-based assessments.
Both Table 1 and Figure 8 illustrate the analysis of agreement/disagreement between
the pairs of satellite-aerial DA maps. They point out an overall low agreement between sat-
ellite- and aerial-derived damage assessments, both for the urban block-based maps (Maps
1 and 2) and the grid-based maps (Maps 3 and 4). Comparing the patterns in agreement/dis-
agreement (Figure 8) with the distribution of both the building density and damage density
of Figure 3, the following observations can be made:
- Map 1 shows a clustering of disagreement within areas of significant absolute dam-
age density (Fort National and Saint Antoine in Figure 3), whereas Map 2 shows a
higher overall agreement in regions of low building density but of high density of
significant damage (Morne a Tuff in Figure 3; i.e., it appears to outline relative dam-
age density well);
- Maps 3 and 4 exhibit more or less the same, though somewhat random, distribution
of agreement/disagreement. However, Maps 3 and 4 show opposite results with clus-
ters of agreement in Map 3 and of disagreement in Map 4 within areas with a low
density of buildings but with a high relative damage density. This is mainly the case
in the sector of Morne a Tuff characterized by a low absolute damage density and
high relative damage density.
- All four maps show disagreement in areas characterized by a high density of build-
ings and a high density of significant damage. This underlines the difficulties of satel-
lite image visual interpretation for building damage in regions with a high density of
buildings, many of which correspond to informal settlement areas (e.g., slums).
In addition to the overall agreement measures computed, the correlation coefficient was
computed using the 2-D cross-correlation statistic (Table 2). The correlation coefficients
computed supported the contingency table results, which showed a little correlation between

Figure 8. Result of direct cell-by-cell and block-by-block comparison for the four pairs of satel-
lite-aerial DA maps.
S212 CORBANE ET AL.

Table 2. Results of the overall agreement and correlation coefficient statistics computed for
each of the four satellite DA maps using the aerial DA as a reference

Overall agreement Correlation coefficient (r)

MAP 1 0.47 0.56

MAP 2 0.58 0.42
MAP 3 0.44 0.56
MAP 4 0.51 0.75

damage categories of satellite and aerial DA maps. However, a relatively high correlation
was computed for the grid-based Map 4 (r ¼ 0.75). Closer inspection showed that this map
contained a large number of cells (247 out of a total of 523 cells) with undamaged build-
ings. Since the 2-D cross-correlation was based on an average of all cell labels (transformed
into codes for computational purposes) within the two maps being compared, the presence
of such a large number of cells with undamaged building labels had a significant influence
on the global correlation coefficient r, resulting in the high correlation coefficient obtained
for Map 4. As evident in Table 2, there was no trend between the overall agreement mea-
sure computed and the correlation coefficients. This was due to differences in intrinsic com-
putational properties of the two measures: While the overall agreement takes into account
the amount of correct matching between damage categories (it incorporates the major diago-
nal and excludes the omission and commission errors of the contingency table), the correla-
tion coefficient considers the correlation between all the cells composing the pairs of maps
under assessment. Here, the correlation coefficients had the advantage of being applicable
to both the maps’ composition and configuration analysis, thus allowing the comparison of
the results of the two approaches.
The comparison between satellite and aerial DA maps following the analysis of the map
composition approach uncovered an overall low consistency both for grid-based and urban
block-based satellite-derived maps. The results published in the joint report entitled Build-
ing Damage Assessment Report - Haiti Earthquake 12 January 2010 Joint Damage Assess-
ment (Kemper et al. 2010), showed that the satellite-based DA underestimated damage by
at least a factor of five, compared to the aerial-based DA. Thus, the low consistency
obtained from our analysis can mostly be explained by damage underestimation and is sup-
ported by several studies showing that even buildings categorized as having Grade 4 dam-
age are not always distinguishable on VHR satellite images (Saito et al. 2004, Yamazaki
et al. 2005). To better illustrate the degree of damage underestimation in the satellite-
derived maps, the percentages of omission errors were computed for high damage grades.
The results showed that omission errors ranged between 40% (Map 2) and 80% (Map 3),
thus supporting the observations given in the Building Damage Assessment Report (Kemper
et al. 2010).
These results suggest a low reliability of satellite-derived DA maps with respect to aerial
DA maps. In terms of usability, the high underestimation of damage obtained on the satel-
lite imagery demonstrates the limitation of this type of imagery for detailed damage enumer-
ation exercises and suggests the need for imagery whose spatial characteristics are similar
to those of 0.15 m aerial images.
COMPARISON OF DAMAGE ASSESSMENT MAPS S213

RESULTS OF MAP CONFIGURATION ANALYSIS

Table 3 presents the LMI map correlation coefficients. These statistics highlight the
degree of correspondence between patterns of damage categories in the pairs of satellite-aer-
ial DA maps. Damage patterns of two pairs of aerial and satellite DA maps are shown in
Figure 9. They were obtained by computing the spatial autocorrelation using the LMI.
These results uncover a fairly good correlation between damage patterns exhibited on the
satellite DA maps with reference to the aerial DA maps. Higher correlation is observed for
Maps 1 and 2, which use urban blocks as their mapping units (r ¼ 0.86 for Map 1 and 0.88
for Map 2). For instance, Map 2 recorded the highest LMI correlation where two major
types of clusters can be distinguished: one corresponding to high levels of damaged build-
ings (HH) and one corresponding to low levels of damaged buildings (LL).
Figure 9a and Figure 9b give a spatial account of the similarities between the distribu-
tions of these clusters on the aerial- and satellite-based DA maps. Inversely, the lowest LMI
correlation was observed in Map 3, where large differences in the spatial distributions of the
clusters are visible (Figure 9c and Figure 9d): the aerial DA map exhibits a large cluster of
buildings with a high level of damage (HH) in the lower-left quadrant, which does not
appear in the satellite DA map.
Comparing the map configuration between satellite and aerial DA maps suggests the
following:
- All maps show a spatial patterning of damage categories, mainly for high and low
levels of damages. Damage patterns are likely related to building type and construc-
tion material, the spacing between structures, and the local soil and underground con-
ditions. Unfortunately, no information on these variables is available and they cannot
be tested to explain these local differences. More detailed investigations may provide
better insight into the processes that influence the generation of the observed patterns
of structural damage in the satellite and aerial images.
- In terms of reliability, damage patterns are better captured using urban block-based
representations than grid-based representations. This might be related to the fact that
maps based on urban blocks are better at grouping homogeneous urban areas and
may, therefore, be better at representing the spatial differences between the different
damage categories and the related patterns.
- In terms of usability, VHR satellite imagery is capable of capturing the relative dam-
age intensity pattern observed in aerial imagery. In that sense, even if VHR satellite

Table 3. Correlation coefficients of LMI maps computed between the four pairs of satellite-aer-
ial DA maps

Correlation coefficient of LMI maps

MAP 1 0.86
MAP 2 0.88
MAP 3 0.23
MAP 4 0.61
S214 CORBANE ET AL.

Figure 9. Results of spatial autocorrelation statistics based on the Local Moran Index (LMI).
The four maps represent significant values of LMI summarized into four main types of cluster.
Figures 9a and 9b correspond to LMI computations for the area-based aerial damage assessment
and its corresponding satellite DA Map 2, respectively. Figures 9c and 9d correspond to LMI
computations for the area-based aerial damage assessment and its corresponding satellite DA
Map 3, respectively.

imagery cannot be used for accurately interpreting the damage intensity, it is a useful
source for providing an overview of the damage pattern.
CONCLUSIONS
The motivation behind this study was to assess the reliability and usability of building
damage assessments derived from VHR satellite imagery as compared to the detailed dam-
age assessments derived from aerial imagery using information gathered during the 2010
Haiti earthquake. Satellite-based damage maps are generally available shortly after the rele-
vant satellite image becomes available, whereas aerial assessments are more detailed but
require much more time to generate. Furthermore, aerial data may not be available in many
post-earthquake contexts. Therefore, the 2010 Haiti earthquake provided a unique opportu-
nity to study the relationship between image spatial resolution and the related value-added
COMPARISON OF DAMAGE ASSESSMENT MAPS S215

analysis outputs. A three-step map comparison strategy was developed and implemented on
four area-based satellite maps using point-based damage assessment maps derived from aer-
ial imagery as a reference. The main findings of this work are summarized in terms of the
advantages and the limitations of damage assessment maps derived from satellite imagery
specifically related to their reliability and usability.
The following conclusions are made with respect to the reliability of satellite-derived
maps for building damage assessments:
- Computer-aided visual interpretation of satellite imagery can produce maps depicting
overall patterns in damage grades quickly and efficiently. This was verified for Dam-
age Grade 1 (no visible damage) and for the combined Damage Grades 4 and 5
(defined as very heavy damage and destruction, respectively).
- The analysis of map composition (i.e., direct cell-by-cell or block-by-block compari-
son), demonstrated that the consistency of satellite-derived products with respect to
the aerial-derived assessment is very low both for grid-based and urban block-based
maps. This was evidenced by the average overall agreement of 0.5, and the average
correlation coefficient of 0.54 computed for the four analyzed maps.
- The analysis of map configuration, based on the spatial distribution of damage cate-
gories, revealed the potential usefulness of VHR satellite imagery in capturing the
overall spatial pattern of building damage. Urban block-based maps achieve greater
reliability than grid-based maps in terms of damage patterns.
The following conclusions are made with respect to the usability of satellite-derived
maps for building damage assessments:
- Current state-of-the-art satellite imagery at sub-meter spatial resolution is essential to
allow the rapid generation of maps showing general damage patterns in a post-earth-
quake impact analysis context. However, even with spatial resolutions of 0.5 m it is
difficult to interpret building damage accurately and to generate overall damage esti-
mates reliably, either as absolute or relative damage density indicators. This limita-
tion is largely due to the near-nadir viewing configurations of orbiting satellites.
After the 2010 Haiti earthquake, this difficulty was exacerbated by the relatively
small structure of the buildings found within the study area. Based on the study’s
results, a detailed damage enumeration exercise requires, at least, imagery whose
spatial characteristics are similar to those of 0.15 m aerial images.
- Cooperative satellite-based rapid mapping for damage assessments produces a rea-
sonable representation of the overall damage pattern, especially if the aggregation is
based on an urban block mapping unit. The timeliness of such map products make
them useful for focusing the detailed damage assessments based on aerial imagery.
In a similar way, they can contribute to improving sampling scheme designs for field
data collection by contributing to the stratification and/or the clustering of the sam-
ples. Today, rapid postdisaster image acquisition is routine, whereas aerial imagery
is not guaranteed.
This research contributed significantly to our team’s work to develop consistent valida-
tion approaches and criteria for producing emergency maps. The situation after the Haiti
earthquake provided unique insights into the types of reference data available and
S216 CORBANE ET AL.

alternative cooperative analysis approaches. Our current research focuses on understanding

the limitations of using computer-aided visual interpretation, testing alternative sampling
designs, and the use of automated image processing routines to highlight damage artifacts.
We are working with colleagues from UNOSAT and the World Bank to put together stand-
ard operational procedures in case of future disasters.

ACKNOWLEDGMENTS
The authors would like to acknowledge SAFER and G-MOSAIC RGR (Rapid Geospa-
tial Reporting) GMES projects and all the institutions that performed the damage assess-
ments, mainly SERTIT, German Aerospace Center (DLR), JRC, European Union Satellite
Centre (EUSC) and e-Geos in collaboration with UN Cartographic Section. Special thanks
to our colleagues from UNITAR/UNOSAT and the World Bank. Finally our gratitude goes
to Conrad Bielski for his collaboration on this work and for his constructive comments.
Any opinions, findings, conclusions, or recommendations expressed in this material are
those of the authors and do not necessarily reflect the view of the Joint Research Centre.

REFERENCES
Agresti, A., 2002. Categorical Data Analysis, 2nd Edition, Wiley-Interscience, Hoboken, NJ,
734 pp.
Anselin, L., 1995. Local indicators of spatial association – LISA, Geographical Analysis 27,
93–115.
Brunner, D., Lemoine, G., and Bruzzone, L., 2010. Earthquake damage assessment of buildings
using VHR optical and SAR imagery, IEEE Transactions on Geoscience and Remote Sens-
ing 48, 2403–2419.
Chiroiu, L., 2005. Damage assessment of the 2003 Bam, Iran, earthquake using Ikonos imagery,
Earthquake Spectra 21, S219–S224.
Chiroiu, L. and André, G., 2001. Damage assessment using high resolution satellite imagery:
Application to 2001 Bhuj, India earthquake, RiskWorld, Reports and Papers, http://www.risk-
world.com/Nreports/2001/Bhuj,India,earthquake2001.PD, posted September 2001.
Congalton, R. G., 1991. A review of assessing the accuracy of classifications of remotely sensed
data, Remote Sensing of Environment 37, 35–46.
Congalton, R. G., 2001. Accuracy assessment and validation of remotely sensed and other spa-
tial information, International Journal of Wildland Fire 10, 321–328.
Corbane, C., Saito, K., Dell’Oro, L., Gill, S., Piard, B., Huyck, C., Kemper, T., Lemoine, G.,
Spence, R., Krishnan, R., Bjorgo, E., Senegas, O., Ghesquiere, F., Lallemant, D., Evans, G.,
Gartley, R., Toro, J., Ghosh, S., Svekla, W., Adams, B., and Eguchi, R., 2011. A comprehen-
sive analysis of building damage in the January 12, 2010 Mw7 Haiti earthquake using high-
resolution satellite and aerial imagery, Photogrammetric Engineering and Remote Sensing
(PE&RS) 77(10).
Cramér, H., 1999. Mathematical Methods of Statistics, Princeton University Press, Princeton,
NJ, 575 pp.
Csillag, F. and Boots B., 2004. Toward comparing maps as spatial processes, in Developments
in Spatial Data Handling (P. F. Fisher, ed.), Springer, Berlin, Germany, 641–652.
Foody, G. M., 2002. Status of land cover classification accuracy assessment, Remote Sensing of
Environment 80, 185–201.
COMPARISON OF DAMAGE ASSESSMENT MAPS S217

Getis, A. and Ord, J. K., 1996. Local spatial statistics: An overview, in Spatial Analysis: Model-
ing in a GIS Environment (P. Longley and L. Batty, eds.), John Wiley & Sons, Inc., New
York, NY, 261–277.
Gonzalez, R. C. and Woods, R. E., 1992. Digital Image Processing, Addison-Wesley, Reading,
MA, 954 pp.
Grünthal, G. (ed.), 2001. European Macroseismic Scale 1998 (EMS-98), Centre Europèen de
Géodynamique et de Séismologie, Luxembourg.
Gusella, L., Adams, B. J., Bitelli, G., Huyck, C. K., and Mognol, A., 2005. Object-oriented
image understanding and post-earthquake damage assessment for the 2003 Bam, Iran, earth-
quake, Earthquake Spectra 21, S225–S238.
Gustafson, E. J., 1998. Quantifying landscape spatial pattern: What is the state of the art? Eco-
systems 1, 143–156.
Hagen-Zanker, A., Straatman, B., and Uljee, I., 2005. Further developments of a fuzzy set map
comparison approach, International Journal of Geographical Information Science 19,
769–785.
Hagen, A., 2003. Fuzzy set approach to assessing similarity of categorical maps, International
Journal of Geographical Information Science 17, 235–249.
Hasegawa, H., Yamazaki, F., Matsuoka M., and Seikimoto, I., 2000. Determination of building
damage due to earthquakes using aerial television images, Paper No. 1722, 12th World Con-
ference on Earthquake Engineering, January 30–February 4, 2000, Auckland, New Zealand.
Hengjian, L., Kohiyama, M., Horie, K., Maki, N., Hayashi, H., and Tanaka, S., 2003. Building
damage and casualties after an earthquake, Natural Hazards 29, 387–403.
Kemper, T., Lemoine, G., Corbane C., Gill, S., Eguchi, R., Bjorgo, E., and Dell’Oro, L., 2010.
Building Damage Assessment Report - Haiti Earthquake 12 January 2010 Joint Damage
Assessment, JRC59706.
Lalor, G. C. and Zhang, C., 2001. Multivariate outlier detection and remediation in geochemical
databases, Science of the Total Environment 281, 99–109.
Lemoine, G., 2010. Validation of building damage assessments based on post-Haiti 2010 earth-
quake imagery using multi-source reference data, VALgEO 2010, 2nd International Workshop
on Validation of Geo-Information Products for Crisis Management, October 11–13, 2010,
Ispra, Italy.
Liu, J.-H., Shan, X.-J., and Yin, J.-Y., 2004. Automatic recognition of damaged town buildings
caused by earthquake using remote sensing information: Taking the 2001 Bhuj, India, earth-
quake and the 1976 Tangshan, China, earthquake as examples, Acta Seismologica Sinica 17,
686–696.
Matsuoka, M., Vu, T. T., and Yamazaki, F., 2004. Automated damage detection and visualiza-
tion of the 2003 Bam, Iran earthquake using high-resolution satellite images, 25th Asian
Conference on Remote Sensing, November 22–26, 2004, Chiang Mai, Thailand.
Ogawa, N. and Yamazaki, F., 2000. Photo-interpretation of buildings damage due to earth-
quakes using aerial photographs, Paper No. 1906, 12th World Conference on Earthquake En-
gineering, January 30–February 4, 2000, Auckland, New Zealand.
Open Street Map (OSM), 2010, Haiti Crisis Map, http://haiti.openstreetmap.nl/, accessed July
2, 2010.
Ping, L. and Xiaxin, T., 2005. Quantitative earthquake damage detection from changes in
remote sensing images - a case study, IEEE International Geoscience and Remote Sensing
Symposium, July 25–29, 2005, Seoul, South Korea.
S218 CORBANE ET AL.

Rathje, E. M., Crawford, M., Woo K., and Neuenschwander, A., 2005. Damage patterns from
satellite images from the 2003 Bam, Iran, earthquake, Earthquake Spectra 21, S295–307.
Rees, W. G., 2008. Comparing the spatial content of thematic maps, International Journal of
Remote Sensing 29, 3833–3844.
Rezaeian, M. and Gruen, A., 2007. Automatic classification of collapsed buildings using object
and image space features, in Geomatics Solutions for Disaster Management (J. Li, S. Zlata-
nova, and A. G. Fabbri, eds.), Springer, Berlin, Germany, 135–148.
Saito, K., Spence, R., Eeri, M., Going, C., and Markus, M., 2004. Using high-resolution satellite
images for post earthquake building damage assessment: A study following the 26.1.01
Gujurat earthquake, Earthquake Spectra 20, 145–169.
Saito, K., Spence R., and Foley, T., 2005. Visual damage assessment using high-resolution sat-
ellite images following the 2003 Bam, Iran, earthquake, Earthquake Spectra 21, S309–S318.
Turker, M. and San, B. T., 2004. Detection of collapsed buildings caused by the 1999 Izmit,
Turkey earthquake through digital analysis of post-event aerial photographs, International
Journal of Remote Sensing 25, 4701–4714.
Turker, M. and Sumer, E., 2008. Building-based damage detection due to earthquake using the
watershed segmentation of the post-event aerial images, International Journal of Remote
Sensing 29, 3073–3089.
Yamazaki, F., Yano, Y., and Matsuoka, M., 2005. Visual damage interpretation of buildings in
Bam City using QuickBird images following the 2003 Bam, Iran, earthquake, Earthquake
Spectra 21, S329–S336.
(Received 9 August 2010; accepted 23 January 2011)