ANALYSIS OF THE INFRARED IMAGES TO DETECT POWER LINES

Kazuo Yamamoto Kimio Yamada

Electronic Navigation Research Institute, Ministry of Tkansport
6-38-1 Shinliawa Mitaka, Tokyo 181, Japan
yamamoto@enri.go.j p kyamada@enri.go. j !p

ABSTRACT
An image sensor system consisting of an infrared (IR) cam-
era and a color video camera is presented to detect and
identify flight obstacles especially power lines. Images in-
cluding obstacles are gathered by the sensor system. The
IR images demonstrated the advantage over color images
in detecting power lines in many environments. Several
imaging filters are applied to the IR images to enhance the
target-tubackground contrast and to suppress the noise in
the images. Fusion of IR and color images is also tried t o
generate a virtually enhanced image. These image process-
ing techniques proved to be valid to facilitate the algorithm
development for detecting and identifying flight obstacles.

1. INTRODUCTION
Helicopters and small aircraft often fly at low altitude in
the visual flight rules for surveillance or to scatter agricul-
tural chemicals. They often crash against obstacles such as
a power transmitting line or a tall pole because these obsta-
cles are not visible in dark background. We have begun to
develop a forward surveillance and collision warning system
which can automatically detect and identify obstacles and
can provide a pilot with enhanced forward images.
The collision warning system is widely studied. A laser
radar was introduced to detect power lines [l]. However, im-
provement should be done to reduce the laser power in “eye
safe” level in keeping the coverage. A Millimeter-wave radar
forward sensor was proposed[2], and looks very promising
in the future. However, it takes a long time to synthesize a
forward image from the radar echo even by a powerful com-
puter. A method was presented to utilize the polarization of
the radar returns from power lines for obstacle detection[3].
But further work should be needed to discriminate clutters.
An image sensor, especially a high resolution infrared
(IR) camera can provide a forward image instantly and
vividly, which is important for pilots to identify what is
lying ahead. So, even if radars mentioned above are em-
ployed for forward surveillance, we believe that an image
sensor is still necessary for the warning system.
The first objective of this paper is to describe the image
sensor system that we have constructed for our collision
warning system. The system, consisting of an IR camera,
a color video camera and peripheral equipment, is installed
on the ground or on an aircraft to gather images. From the
images, the influence of the background luminance or ma-
Fig.1 Configuration of the image sensor system
terial upon the targets-to-background contrast is discussed.
The coverage of the IR camera is also described.
The second objective of this paper is to present several
image processing techniques to enhance the contrast of a
target and to suppress the noise, which are indispensable
to develop a dependable algorithm for the collision warning
system. Several imageing filters as the sharpening filter,
the Median filter and the directional filter are evaluated.
Fusion of the images front the IR and color cameras is also
attempted to produce a virtually enhanced image.
2. THE IMAGE SENSOR SYSTEM
Fig.1 shows the configuration of the image sensor sys-
tem. The system consists of an infrared (IR) camera, a
video camera, a laser distance meter, video recorders and a
personal computer (PC).
The IR and the video cameras must have a high resolution
sensing elements because the primary target is thin wires
(5 to 30” in diameter). We chose an IR camera (MIT-
SUBISHI IR-M500) which can detect 3 - 5 pm IR with 512
x 512 PtSi Schottky-Barrier elements. These elements are
reported to have a wide dynamic range, which is important
for our application.
As a video camera, a 3CCD color video camera with
750 scanning resolution (:SONY DXC-950) was used. Im-

1997 IEEE TENCON - Speech and Image Technologies for Computing and Telecommunications 343
 (a) Color Images
Fig.2 Color and E
t Images obtained at Sagami-ko
Fig.3 IR Image obtained at Baba
ages gathered by these sensors are recorded by the video
recorders and by the PC through image acquisition boards.
The laser distance meter is used to measure the distance to
an obstacle.
The PC is also used to evaluate several imaging filters to
enhance the target-to-background contrast and to suppress
the noise in the images. The sensor system is placed on the
ground or on a twin turbo-prop aircraft (Beechcraft l3-99)
to gather the images including power lines.
3. MEASURED IMAGES
Fig.2 shows an example of the IR and color images measured
on the ground at Sagami-ko, which is about 50km west
from downtown Tokyo. The distance t o obstacles (power
lines) is approximately 500m. The background mountain
and building are about 800m away. The power lines could
be seen by eyes and in the color image where the lines run
in the sky, but could never be detected when the lines are
in front of the mountain.
1997 IEEE TENCON - Speech and Image Technologies for Computing and Telecommunications
344
(b) IR Image
In the IR image, the power lines could be detected what-
ever the background is, although the targetrto-background
contrast is not so high. To enhance the contrast, the cam-
era sensitivity was adjusted. Then the lines were enhanced
where the background is the mountain. However, the lines
was extinguished where they are in the sky.
Fig.2 suggests that the IR camera can greatly enhance
the target if the camera sensitivity is properly adjusted. IR
images were then collected in many different backgrounds,
distances and weather conditions to test the validity of the
camera.
Fig.3 is the IR image obtained at Baba, which is about
74km west from downtown Tokyo. The distances from the
sensor to farther power lines and to low voltage distribution
lines are approximately 450m and 80m respectively. The
background mountain is about 500m away. The power lines
and distribution lines appeared in the IR image although
the contrast of the power lines is not very high. In the
color image however, no power lines were displayed, while
the distribution lines were barely detected, although it is
not shown here. This is a typical example that the dark
background of foliage is very close to the target. In such a
case, it is extremely difficult for pilots to discern obstacles.
Fig.4 is the image observed by an aircraft a t the altitude
about 5000 feet. This image was obtained in winter at the
suburbs of Kushiro, a city in Hokkaido Japan. Then, the
background is snow. In spite of long distance observation,
the line could be clearly displayed in the IR image, This
is partly because snow is depicted very dark in the IR im-
age and partly because this camera can also detect infrared
included in the sun light which is scattered by the lines.
In the color image however, no power lines were detected
although it is not shown here.
Fig.5 is an example where even an IR image fails to
show targets. The figure was obtained at Hakusyu, about
23km north west from Kofu, where a power tower is shown
vaguely. The weather was drizzly and the distance to the
tower was about 1000m. Power lines were not detected both

Fig.4 IR Image obtained at Kushiro
Fig.5 IR Image obtained at Hakusyu
by IR and color cameras.
Above figures proved that IR images have an advantage
over color images in detecting power lines in many environ-
ments. However, obstacles were not detected even in IR
images in some adverse conditions as Fig.5.
The contrasts of the targets in gathered images have been
classified as the function of distance from the sensor to the
target. The result showed that the contrast became too low
to detect target when the distance exceeded about 800m in
most images. Of course, there were some exceptions as
Fig.4, but this value gives us outlook of our system’s cov-
erage.
4. IMAGE PROCESSING FOR CONTRAST
ENHANCEMENT AND NOISE
SUPPRESSION
The IR images proved to be valid to detect power lines,
but the target-to-background contrast was not very high
in most images. Then the contrast of a target was mea-
1997 IEEE TENCON - Speech and Image Technologies for Computing and Telecommunications
Fig.6 Processed Image (Original : Fig.P(b))
sued first, because this v i h e will be u k d as a reference for
automatic target detection and identification.
The contrast C of a target can be calculated as
c = Li. - Lb
where Lt is the target luminance and Lb the background
luminance in which each value goes from 0 (dark) to 255
(light).
For example, the contrast C of the power lines in Fig.2(b)
varied from 3 to 10 along the lines. These values are not
necessarily enough to detect and identify a target, because
the contrast between the noise and the background also
varied from 5 to 10 in most part of the image, and the noise
included in the images may be detected as false targets.
In addition to this. in imany video monitors, images are
displayed in 0 to 255 (8-bit) gray scale. It w a s pointed out
that in watching monitors, the contrast within 2 to 3 cannot
be discerned in most human eyes[4]. This means that the
identification of obstacles by pilots may be difficult in such
contrast as 3 to 10. Some image processing techniques must
then be introduced to enlhance the contrast of the targets
and to suppress the noise in the original images.
Fig.6 is a processed image where the original image
Fig.2(b) went through the sharpening filter then the Me-
dian filter. The sharpening filter is a kind of high pass filter
which can enhance edges between different regions. The
Median filter can smooth an image, while preserving edges
larger than the kernel dimensions.
Power lines became clearer than those in the original Fig-
ure. An overhead ground wire, which was never detected in
the original image was also detected slightly in the figure.
This filter amplified the noise in the background too, but it
was considerably suppressed by the Median filter. In Fig.6
therefore, the contrast C was enhanced by about 15 to 25
in most portions along the lines. We can say that Fig.6 is
better to use as a reference for warning system than the
Original image Fig.S(b).
Fig. 7 is another example to enhance contrast and sup-
press noise, where the origiinal image (Fig.3) was sharpened,
then it went through the directional filter. The directional
filter selectively enhances iimage features having specific di-
345

Fig.? Processed Image IOrieinal : Fig.3)
Fig.8 Fused Image (Fig.S(a) + (b))
rection components. Since the power lines are running al-
most horizontally, the direction of filtering was set to be
horizontal. By these filters, the contrast C of the lines in-
creased to 10 - 20 from the original contrast 5-10. On the
other hand, these filters did not increase noise in the back-
ground.
Figs. 6 and 7 show that the combination of the sharpen-
ing filter and the Median filter or the directional filter gives
us images with enhanced contrast and suppressed noise.
These processes took about 1 to 3 seconds by the Window
95 PC with 260MKz processor.
As stated previously, the target-to-background contrast
of IR images can be enhanced by the adjustment of the
camera sensitivity when the background is a mountain. On
the other hand, power lines can be detected clearly in color
images when the background is sky. So, IR and color image
fusion is attempted to improve the contrast of the target in
any backgrounds.
Fig.8 shows the fused image where the power lines are
clearer than those in original IR images Fig.2(a) and (b).
This method will not need long processing time, but exact
camera location and view point settings must be done in
order t o merge two images accurately.
1997 IEEE TENCON - Speech and Image Technologies for Computing and Telecommunications
346
The image processing techniques presented here are ex-
pected to simplify the total algorithm to detect and identify
obstacles and to show enhanced forward images including
the obstacles.
5. CONCLUSION
The performance of the image sensor system to detect
power lines was described to develop a total system for air-
craft collision warning in visual flight. The images gathered
by the sensor system revealed that the inhared (IR) camera
we introduced, detected power lines even when they could
not be found by eyes or by a color camera. The coverage
of the IR camera was about 800m. However, the target-tc-
background contrast of the IR images was not enough to
be used as a reference for automatic target detection and
identification.
Several image processing techniques were applied to en-
hance the contrast of the targets and to suppress the noise
in the original IR images. Imaging filters as the sharpening
filter, the Median filter and the directional filter proved to
be effective to enhance the contrast by about twice as high
as that of the original images and to suppress the noise.
We are now developing the total algorithm to automati-
cally detect and identify obstacles from the observed images
and to provide a pilot with enhanced forward images includ-
ing obstacles. The image processing techniques presented
here will greatly facilitate the development of our algorithm.
6. REFERENCES
[l]H. Saito, M. Doshida, Y. Mine, “Obstacle warning lider,”
Technical Report of IEICE Japan Vol. SANE93-89,
pp.1-6, Fev.1994, (Japanese).
[2] L. D. Almsted, R. C. Becker, R. E. Zelenka, “Affordable
MMW Aircraft Collision Avoidance System,” Proc.
SPIE, Vo1.3098, pp.57-63, April.1997.
[3]K. Yamamoto, K. Yamada, “The polarization of the
radar echo from power lines,” Int. Symp. Antennas
(JINA96), ISSN 1168-3848, pp.233-236, Nov.1996.
[4] J. Silverman, J.M. Mooney, V.E. Vickers, “Display of
wide dynamic range infrared images from PtSi Schottky
barrier cameras,” Optical Eng., 29-2, Feb.1990.