1

Assessment of the Quality of

Digital Terrain Models
Joachim Hhle
Aalborg University
2
Disposition
1.

Introduction
2.

Acquisition and processing methods
3.

Methods of checking DTMs
4.

Reference data, sample size, distribution of errors
5.

Robust quality measures
6.

Other quality parameters of DTMs
7.

Improving the quality of DTMs
8.

Visualization of DTM quality
9.

Results of projects
10.

Requirements of various applications
11.

Summary
3
Airborne

laser scanning (ALS)
Digital photogrammetry
Acquisition of DEMs by ALS and Photogrammetry
Source: Toposys Source: Intergraph
2. Acquisition and processing methods
4
Processing

steps at DTM generation from ALS
Position
(DGPS, IMU)
Calibration data
Distance,
looking angle
X, Y, Z (WGS84)
Projection
Filtering
Regular DTM
Reduction of amount of data
2. Acquisition and processing methods
5 Filtering

of ALS data
( ) ( )

+
+ >
+ s s
<
=
0
1
1
1
) (
b
g r a
h g r
h g r g
g r
r w
Progressive densification
Surface-based filters
where
r

residuals to the approx. surface
g

defines the maximal residual
h

gives the range of residuals
a, b

parameters of the weight function
2. Acquisition and processing methods
6
DSM generation by means of automated photogrammetry
Source: Kraus 1996
2. Acquisition and processing methods
7
Filtering

of the DEM data
Filters and their

parameters Result

of filtering
All elevations in the red areas

are

removed
Source: Hhle, Technical report #3 of National Survey & Cadastre, 2009
cell size
cell height
2. Acquisition and processing methods
8
Definitions and terms
Definitions: DTM, DSM, DEM, DCM
Forms of DTMs: Point cloud, gridded DTM, TIN, contour lines
Terrain types: Open terrain, built-up areas, forested areas,
Quality: Accuracy, density, completeness
Checking/assessment: Production (Quality Assurance), final product
(Quality Control)
Improving/editing: Manually/automated
Updating
3. Methods of checking DTMs
9
Checking

of the accuracy

in general
Well defined points are best for accuracy checking
Reference points of superior accuracy are required
Absolute accuracy is important to the users of DEMs
Updating of DTMs

require the original data
3. Methods of checking DTMs
10
Accuracy

measures

for vertical

errors

at normal distribution
3. Methods of checking DTMs
11
Checking

the normality

of the error

distribution
Histogram of the errors h
Q-Q plot for the distribution of h
3. Methods of checking DTMs
12
Horizontal

accuracy

measures

at normal distribution
3. Methods of checking DTMs
13
Roof

types usable

for the derivation of planimetric accuracy
3. Methods of checking DTMs
14
Hip house & cross gable house
Building roofs

for derivation of check points
Combination of gable roofs
pyramid hip roof
Combination of gable roofs &
Check point
Laser foot print
Source: Hhle & Pedersen,
Proceedings of Accuracy 2010
Ah
Ap
o
h=ptan
3. Methods of checking DTMs
15
number of check points
accuracy of the reference values
confidence interval for accuracy measures
Mean and Standard Deviation
Various

details

in the assessment
confidence level 95%
4. Reference data, confidence interval, confidence level, sample

size
16
Median, 68% & 95% quantiles
5. Robust accuracy measures
Practical example
95%
17
Robust accuracy

measures

for vertical

errors
5. Robust accuracy measures
18
Flow chart

for decision on

type of accuracy

measures
The Quantile-Quantile

plot (Q-Q plot) will

reveal

a non-normality

in the distribution of errors.
Robust accuracy

measures

have to be

applied

at non-normal

distribution of h
i

.
Q-Q plot
DEM data checkpoints
interpolate elevation at checkpoint
calculate dh
i
calculate RMSE, mean, std. dev.,
number of blunders (N),
median, NMAD and their
confidence interval (CI)
generate histogram & Q-Q plot of dh
i
normal
distributionof dh
i
?
visual check
mean~median ?
std. dev.~NMAD ?
standard accuracy
measures:
RMSE, mean, std. dev., N
yes
no
compute quantiles of abs(dh
i
)
Q(0.68), Q(0.95) and their CIs
robust accuracy measures :
median, Q(0.68), Q(0.95) and their CIs
5. Robust accuracy measures
19
Distance between grid points and terrain points Density of ALS data
Source: Karel

& Kraus
6. Other quality parameters of DTMs
Density

and completeness
-

ALS data
20
7. Checking

and improving

the quality

of DTMs
Removing of points
within the outlines of
buildings and the
surroundings zones
Use

of GIS data and GIS techniques
21
Use

of photogrammetry -

1
DEM errors (dh) are found by
matching patches of two aerial
images.
7. Checking

and improving

the quality

of DTMs
Back-projection

method
22
Two

orthoimage method
h b
h
dX
b
h
dX dh
1
2

|
.
|

\
|
+
|
.
|

\
|
~
dh

elevation error
h

height above DEM
b

basis
dXparallax between two
corresponding orthoimage patches
An error in the DEM results in
a parallax between the two
orthoimages, which are
derived from overlapping aerial
images. The parallax (dX) is
automatically

measured.
7. Checking

and improving

the quality

of DTMs
Use

of photogrammetry -

2
23
2D Plot
Profile
exaggerated
Automated
Photogrammetry
Laserscanning
Profile
Manual
Photogrammetry
Filtered away
Terrain
2.0 m
0.2 m
Contour lines
on top of
orthoimage
Perspective
view
Profiles of elevation
data from different
acquisition systems
8. Visualization

of the DTM quality
Orthoimage, contour

lines, perspective

views

and profiles
24
Plot of absolute

horizontal

errors
8. Visualization

of the DTM quality
25
Tools

for checking, completion

& visualization
Photogrammetric workstation
with stereo display
DTM editor program DTMaster

Stereo
8. Visualization

of the DTM quality
26
Specification of the National Survey and Cadastre of
Denmark (KMS) for a nationwide DEM
9. Results

of projects

1a
27
Assessment

of quality

at the Danish DTM/DSM
Reference patches for the
absolute vertical error of the
Danish DEM
Source: KMS
9. Results

of projects

1b
28
Source:
Thommesen, 2010
Errors in the DTM
Hydrologic
applications
require an
accurate and
complete DTM.
Errors can be
created during
filtering and
interpolation.
Profile data

reference data by
ground surveying

DTM profile
9. Results

of projects

1c
29
Assessment of quality in DDH

& corridor mapping
Planimetric accuracy
Source: Fugro

A/S
Point density
COWI A/S
Number of points in 10mx10m cells
Spatial ground target Target in profile data
9. Results

of projects

-

2
30
Automatic assessment of DTM quality by means of photogrammetry
EuroSDR

project Automated checking and improving of DEMs

(2006)
The left figure depicts the
method to be used for low
accurate DEMs
The right figure depicts the
method to be used for
accurate DEMs
Reference data can be
derived by means of
images from low flying
altitude or by GPS
9. Results

of projects

-

3
31

E

=0.08m,
E

=0.26m

N

=0.16m,
N

=0.18m
Histogram and
normal distribution
curves
Accuracy measures:
95% confidence ellipse
(bivariate

normal)
Ellipse parameters:
Semi-axes: a = 0.64m, b= 0.44m
Rotation angle:

= -0.70 gon
9. Results

of projects

4a
Horizontal errors at ALS ( normal distribution)
32
Easting
[m]
Northing
[m]
Radial
[m]
Median 0.07
[0.06, 0.18]
0.11
[0.10, 0.19]
0.24
[0.22, 0.30]
Q(0.68) 0.26
[0.24, 0.31]
0.21
[0.20, 0.28]
0.32
[0.31, 0.38]
Q(0.95) 0.45
[0.43, 0.73]
0.43
[0.39, 0.68]
0.56
[0.54, 0.74]
Q(0.95)95% quantile, [95% confidence interval]
9. Results

of projects

4b
Horizontal errors at ALS (non-normal distribution)
33
Application Requirements
contour lines of topographic maps
(contour interval=0.5m)
DTM (0.15), TIN
spot elevations, break lines
production of orthophotos
with GSD=0.4m
DTM (1.75), grid
bridges (polygons, mass point elevations)
production of true orthophotos
with GSD0.1m
DTM (0.15), grid
wire-frame 3D city model
slope and erosivity maps DTM (0.15m), grid
measuring from single images DTM (0.3), grid
photo-realistic 3D landscape models DSM (0.5), grid of high density, vegetation, buildings
3D city models DSM (0.15), TIN, buildings incl. roof structures
hydrologic modelling DTM (0.2), TIN
dykes, culverts, pumping stations, locks
10.

Requirements of various applications
Accuracy, data structure and additional data
34
Some DEM applications
require images

in order to:
-classify vegetation & soil
types
-extract buildings
-map objects (e.g. bridges,
dykes, underpasses, roofs)
DEM
I
M
A
G
E
Slope &
erosivity
map
Contour
Lines
S
oblique
false color
True
orthoimage
Photo-realistic
3D-city &
landscape
model
Ortho
image
Point clouds from
ALS and/or
photogrammetry
Mono-
plotting
Classification
3D-
building
model
Hydrologic
model
DEM
10.

Requirements of various applications
Image data in applications
35
10.

Summary
Vertical and horizontal accuracy has to be
assessed
Robust accuracy measures have to be applied at
non-normal distribution of errors
Density and completeness are important quality
parameters
Requirements of different applications have to be
considered in the assessment of quality
Aerial imagery can be used in generation,
checking, improving and updating of DTMs