2012 5th International Congress on Image and Signal Processing (CISP 2012)
Design and Implementation of Aerial Remote Sensing
Image Processing Software System
Chunxiao Zhang, Zhaorong Lin, Gaojin Wen, Qian Zhang, Xiu Liu, Guangbiao Wang
Beijing Institute of Space Mechanics & Electricity
Beijing, P. R. China
Abstract—An image processing software system framework for
aerial remote sensing is designed and implemented in this paper.
Different from commercial remote sensing processing software
for satellite platforms, a serial of functions, including
radiometric correction, geometric correction, image mosaic and
fusion, are proposed referring to the feature of the aerial images.
These functions can be freely composed to form a process
pipeline with a friendly interactive user interface. Moreover, an
automatic optimization method for processing pipeline is put
forward based on the image quality evaluation. Besides, this
system can yield valuable feedbacks to improve the imaging
system. Its performance is verified by processing real aerial
images. It is shown that the involved functional algorithms are
robust against the instability of airborne platform, the variety of
terrain, and other interfering factors resulting in imaging
distortions.
Keywords-aerial remote sensing; image processing pipeline;
distortion correction; image mosaic
I. INTRODUCTION
The aerial remote sensing image processing has attracted
much attention with the booming applications of unmanned
vehicles. It is a critical technique to display aerial remote
sensing advantages. Comparing with satellite’s platform,
airborne platform can admit more loads at a lower price, as
well as more flexibly cover interesting regions with a higher
resolution and less time. It greatly benefits timely acquiring
detailed information of local district, could be used in city
planning, traffic surveillance, reconnaissance, etc. [1].
Aerial images have the following features. The covering
area of a single-frame image is too small to provide enough
information, due to the low altitude of airborne platforms. It is
necessary to perform the image mosaic [2]. However, the
perspectives at various views present some differences caused
by variety of terrain, increasing the difficulties of image
mosaic. Besides, the imaging distortion could be aggravated as
the deviation of flight status when the airborne platform
becomes unstable. All of these aspects make it impossible to
employ the current mature software of remote sensing image
for the aerial image processing, like ENVI, ERDAS, etc.
This paper presents a design and implementation of aerial
remote sensing image processing software system, taking the
above mentioned imaging features into account. This system
achieves many functions, such as radiometric correction,
geometric correction, automatic image mosaic, etc. Moreover,
978-1-4673-0964-6/12/$31.00 ©2012 IEEE
it allows users to assemble functions into a pipeline and
automatically completes all the operations. If the user’s
pipeline is not rational, the system also could provide an
optimal pipeline after an optimization analysis. Besides, it
could feed back the valuable information to the imaging
system for improving imaging quality. This kind of modular
programming style benefits a further development, for meeting
users’ requirements.
II. FRAMEWORK OF SOFTWARE AND ITS PERIPHERAL
SYSTEMS
Fig.1 is the framework of the designed software system and
its peripheral imaging and control systems. The raw data of
acquired images involves the image data, as well as the status
data at that moment, including aircrafts’ pose/position
parameters, and imaging parameters. The status data is
meaningful to an analysis of geometric-distortion mechanism,
helping to eliminate these distortions. After importing data, the
acquired image is displayed in a window. The processing
pipeline could be designed by users or in a predefined manner,
and all the operations would be taken as the scheduled pipeline
step by step. The quality-evaluation module can be used to
give a comprehensive judgment. Proper pre-processing
operations could be carried out according to the evaluation
result, such as image enhancement, defocusing deblurring,
linear motion deblurring, etc. The modules of radiometric
correction, geometric correction, image mosaic and fusion
could obtain satisfying results providing the accurate status
data. If not up to the desired level, this software will estimate
the status and give an advice to update the work of imaging
system. The whole system is an integrated process, from data
acquirement, processing to feedback control, each part of
which interacts to finally generate high-quality mosaic images.
Meanwhile, the software reserves a function-extended
interface to fulfill more particular applications according to
users’ special demands. So it is potential to a further update
and development.
If the user has no idea to assemble functions into a pipeline,
the pipeline could be automatically designed by an
optimization process, see Fig. 2. The objective function for the
optimization is an evaluation of quality difference between the
original image and its quality-improved version, involving the
evaluation indexes and the quality-improved warping models.
This process is to iteratively calculate evaluation indexes and
update parameters of warping models. In order to avoid a local
1034
minimum, currently, a genetic algorithm is applied in this
system. Then the parameters of warping models could be
identified naturally while the optimal solution is obtained, and
so does the evaluation indexes. Since this process involves
more than one evaluation indexes and quality-improved
models, how to set the weight of each part would influence the
final solution. Therefore, the weights in this system could be
adjusted as the users’ will in order to underline a certain
performance.
Figure 1. Framework of this software system and its peripheral imaging and
control systems
Figure 2. An optimal pipeline design based on an evaluation function
III. IMPLEMENTATION OF THE SOFTWARE SYSTEM
A. Image-quality evaluation module
Image quality directly determines the accuracy and
adequacy of remote sensing information. This module could
calculate definition, contrast, SNR (signal to noise ratio),
information entropy, edge-radiation distortion, gain-radiation
distortion, skewness, steepness, etc. Furthermore, these
evaluation indexes could be weighed to yield a comprehensive
assessment and provide feedbacks to adjust imaging
parameters, like integral time, aperture, focal length, etc.
B. Preprocessing module
This module comprises basic denoising operations, image
enhancing operations, as well as some deblurring operations.
Because the blurring process could be formulated a
convolution of a clear image and a blurring function, the
deblurring could be regarded as a deconvolution operation [3].
Besides, there are usually clouds or hazes affecting imaging
quality, so the operations of removing clouds and hazes are
supplied, in the methods of Ref.[4] and Ref.[5].
C. Radiometric correction modal
There are lots of mature approaches for radiometric
correction, generally classified into three classes: radiometric
cross calibration, MTF correction, and relative radiometric
correction. The first class does not require an accurate
atmosphere measurement. Because when the corrected satellite
image is given, the uncorrected image could be calibrated
through a comparison. MTF could be estimated by the
atmosphere model, or an error propagation model [6]. Relative
radiometric correction is to remove the boundary artifacts, by
equalization curves, matrix matching, etc.
D. Geometric correction module
Since the comparatively low altitude of airborne platform,
geometric distortions could be caused by the variety of terrain
particularly, aggravating with the instability of platform in
flight. So it is more complicated to mosaic aerial images than
satellite images. Geometric correction could be classified into
two groups according to whether given status parameters or
not. When the status parameters are known, including the pose
and position information, DEM, etc., the correction model
could be directly reconstructed, called a forward correction.
Contrarily, a backward correction while unknown the status
parameters, is to estimate the correction model by measuring
some values in image against their correspondences in a
reference system [7]. The measured value is conventionally the
ground control point (GCP), assigned automatically or
manually. The precision of GCPs directly affects the
performance of geometric correction. The GCPs should be
equally distributed and marked features.
1) Forward Correction
For the photo-mechanical scanning system, the distortion
in the tangential direction caused by a systemic error could be
corrected in advance. The variety of flight parameters is
random, which could result in complex distortions. The
variety of roll angle could produce the different starting-
1035
position deviations for each tangential scanning line,
presenting twist distortions of ground objects. The variety of
yaw angle could lead to negative directional distortions. The
variety of pitching angle could change the overlapping rate,
presenting the trapezium scanning plane. In addition, because
the scanning speed is set to be related with the ratio of
velocity to altitude, when the flight altitude deviates from the
assigned track, the scale in the tangential direction will
change. While the flight velocity varies, the scale along the
track will vary. Hence, the accurate parameters provided by
GPS/INS, could help to reconstruct exact models for
eliminating distortions [8]. Meanwhile, the distortions caused
by hypsography could be reduced by adopting the projection-
error correction method with the known DEM data. However,
all of these parameters are not accurate in real flights, and the
exact mathematical models are unavailable. Therefore, the
forward correction is usually a primitive step followed by a
refined correction.
2) Backward Correction.
The transformation for geometric correction could be
estimated by resolving the collinearity equation, given the
GCPs and their corresponding points in image. GCPs should
meet the following commands: nearly even distribution,
remarkable and stable points, a certain amount. GCPs could be
indicated manually or automatically where matches are found
between the distorted image and corrected satellite image by a
matching method. If the transformation is assumed to be a
polynomial function, the first-order function could describe a
linear mapping, like scaling and displacement, and the second-
order (higher-order) could present a nonlinear mapping. The n-
order polynomial function could perform a satisfying
correction, when the camera keeps a small squint angle to scan
a flat region, finding more than 6× (n+1)(n+2)/2 GCPs. If the
transformation is assumed to be an affine matrix, the
correction could be guaranteed under the parallel projection,
which is suitable for the camera with a small zenith angle and
a long focus, where the central projection is approximate to a
parallel projection.
E. Image mosaic module
Image mosaic of aerial remote sensing is more
challengeable than stitching of satellite images, so the stricter
conditions are required to stitch a high-quality image. Before
going on image stitching, two conditions would be checked.
One is that the overlapping rate in the forward and lateral
directions should be respectively up to a level. The other is that
the zenith angles of two adjacent images should be close
enough. Then, the common regions of two images could be
identified by the phase-correlation method based on power
spectrums, and their relative positions are generally estimated
[9]. The two images could be aligned in general and followed
by the refined stitching.
After a general alignment, registration is carried out during
the aligned regions. The feature-based registration is robust
against view variety, where the feature points and lines could
be detected by many approaches, such as Harris corner, SIFT,
SURF, Hough line, etc. [10].
Considering the projection diversity of an identical scene
among these images, a region-based registration method is
used, where the polynomial/affine transformation is performed
on each local region instead of the whole region. This method
divides the scene into segments to perform an alignment on
each part, possessing several advantages: the complexity of
transformation of each segment is greatly reduced; it is
convenient to achieve a parallel speedup; the distortion caused
by variety of terrain is relieved after performing a
transformation on an individual region. The overlapping
regions could be smoothed by differently weighing
interpolation.
F. Image fusion
Image fusion is to develop respective advantage of each
imaging sensor, yielding a better performance. Three modes
have been fused each other, including multi-spectrum images
and panchromatic images, panchromatic images and infrared
images, infrared images and multi-spectrum images.
IV. EXPERIMENTAL EXAMPLES
Our system is tested by a set of aerial remote sensing
images. “Batch process” could be chosen to indicate the
single-frame process or multiple-frame process. For the former,
“Open file” would load an aerial remote sensing image (shown
in Fig.3), as well as the flight and imaging parameters. The
pipeline is automatically designed to perform a deblurring
operation and a geometric correction, according to the
evaluation indexes. The result of removing defocusing blurring
is shown in Fig.4. Since obtaining the status parameters, the
geometric correction could be carried out by modeling
distortions, and the corrected result is shown in Fig.5. These
corrected images could be stitched to a mosaic, as shown in
Fig.6. There are no obvious artifacts at the overlapping regions.
For “Batch process”, the aerial images are loaded together and
the stitching result can be automatically obtained through a
process pipeline. Fig.7 shows another stitched result after a
batch process.
V. CONCLUSIONS
This paper presents a design and implementation of image
processing software for aerial remote sensing. This software
employs a modular programming style, achieving a serial of
operations, such as radiometric correction, geometric
correction, image mosaic and fusion, etc. Particularly, the
pipeline design could not only allow users to assemble the
operations, but also automatically select optimal operations
and the related parameters. Meanwhile, the comprehensive
evaluations could produce a valuable feedback to improve the
performance of imaging system. Targeting the difficulties of
aerial remote sensing image processing, the involved
algorithms are robust against the instability of platform, the
variety of terrain. This software is stable and practical,
possessing strong versatility and a wide range of promotional
prospect in engineering applications.
1036
Figure 3. loading an aerial remote sensing image

Figure 7. another stitched image after a batch process

ACKNOWLEDGMENT
This work is supported by the National Natural Science
Foundation of P. R. China under the Grant No.60903116 and
the Science and Technology Project of Shenzhen under the
grant No. ZD201006100023A, ZYC201006130310A.
Figure 4. the image removing defocusing blurring
REFERENCES
[1] L. Zebedin, A. Klaus, B. Gruber-Geymayer, and K. Karner, “Towards
3D map generation from digital aerial images,” ISPRS Journal of
photogrammetry & remote sensing, vol. 60, pp. 413-427, August 2006.
[2] C. Li, J. Guo, Y. Xia, “Image auto-mosaic for large angle of obliquity
close-range sequential images to realize cyber city,” 2008 International
symposiums on Information processing, IEEE 2008, pp: 384-388..
[3] K.T. Lay, A. K. Katsaggelos, “Image identification and restoration
based on the expectation-maximization algorithm,” Optical
Engineering, Vol. 29, pp.436-445, 1990.
[4] Y.J. Huang, S.G. Wang, J.Y. Liu, D. Chen, “Research on removing
cloud to the remote sensing images,” Chinese Journal of Scientific
Instrument, vol. 24(z2), pp. 490-491, 2003.
[5] K.M. He, J. Sun and X.O. Tang, “Single image haze removal using dark
channel prior,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 33(12), pp. 2341-2353, 2011..
[6] X.Y. Li, X.F. Gu, T. Yu, “Image-derived MTF method and MTF
Figure 5. the image after a geometric correction
compensation for CBERS-02B WFI imager,” Journal of Remote
Sensing, vol. 13(3), pp. 371-384, 2009.
[7] A. Goshtasby, “Registration of images with geometric distortion,” IEEE
Transactions on Geoscience and Remote Sensing, vol. 26(1), pp. 60-64,
1988.
[8] M. Baumker and F.J. Heimes, New calibration and computing method
for direct georeferencing of image and scanner data using the position
and angular data of a hybrid inertial navigation system. Hannover:
OEEPE workshop, 2001.
[9] Q. Chen, M. Defrise, F. Deconinck, “Symmetric phase-only matched
filtering of Fourier-Mellin transform for image registration and
recognition,” IEEE Transactions on Pattern Analysis and Machine
Intelligence, vol. 12, pp. 1156-1168, 1994.
[10] B. Zitova, J. Flusser, “Image registration methods: A survey,” Image
Figure 6. the stitched image and Vision Computing, vol. 21, pp. 977-1000, 2003.

1037