Intelligent vision system

for quality classification of airport lamp prisms

Jakub Suder, Kacper Podbucki, Tomasz Marciniak, Adam Dąbrowski
Poznan University of Technology, Faculty of Automatic Control, Robotics, and Electrical Engineering
Institute of Automatic Control and Robotics
Division of Electronic Systems and Signal Processing
Jana Pawła II 24, 60-965 Poznań, Poland
e-mail: jakub.a.suder@doctorate.put.poznan.pl

Abstract—The article presents a concept of the analysis of

mechanical wear of prisms in the in-pavement airport lamps. The
solution is based on image processing technique that requires an
appropriate selection of parameters due to the specificity of the
objects. During the experimental tests, a database consisting of 316
photos of IDM airport lamps mounted in the airport areas was
used. The proposed solution using an artificial neural network
allows for the classification of lamps with an efficiency of 81.4%.

Keywords-vision system; airport; lighting control; neural
networks; artificial intelligence; Hough transform
I. INTRODUCTION
Due to growing requirements of the aviation agencies,
airports all over the world are increasingly burdened with tasks
to improve safety of the air operations [1]. One of the important
aspects in this field is the correct lighting of the airport areas,
especially runways, where the most critical air operations occur.
For this purpose, measuring devices are built [2, 3], which are
able to measure the luminous efficiency of each lamp, assess its
wear and then classify it as suitable for further use or requiring
replacement [4]. Reduction of the luminous efficiency occurs
through weather conditions and the operation of the runway
(e.g., through sticky, powdered rubber from aircraft tires).
Machines that keep runways and taxiways clean may also
damage lamp housings and their prisms, especially during snow
removal. Metal brushes scratch the lamp holders and chip their
prisms, making it necessary to replace the damaged lamps.
For example, on the runway of the Poznań-Ławica airport,
there are 356 in-pavement lamps, distributed over a distance of
approx. 2.5 km. Lamp performance is specified in the European
Aviation Safety Agency (EASA) standard in the Chapter U —
Colours for aeronautical ground lights, markings, signs and
panels [1]. This document presents the intensity of illumination
of individual lights and their dependence on the angle of
incidence. The given values differ depending on the lamp type
and the light color. The standards also define the requirements
relating to the airport category. One of the most important
parameters is the minimum luminous intensity for the main
beam, which is in a different angular range depending on the
lamp type. It should be noted that stricter guidelines have been
provided for the runway than for other airport areas, which is
justified by the safety requirements of air operations. Fig. 1
shows a lamp damaged after the winter season and a new lamp.
Figure 1. An example of a built-in airport lamp, damaged (left) and new
(right) [2]
Quality of airport lamps can be assessed stationary (using
a light goniometer) or mobile (using luminous intensity
measuring platforms [2] or drones [5, 6]). The stationary method
is a very time-consuming as it requires disassembly of the lamps.
The use of measurement platforms or drones, due to the short
time gaps between air operations, ranging from 5 to 10 minutes,
necessitates short occupations of the runway areas.
An alternative mobile method is the use of a vision system
that allows analyzing the degree of mechanical wear of lamps,
mainly their prisms. An advantage of such solution, in relation
to the luminous intensity measurement, is the precise assessment
of the degree of degradation of the prisms’ frontal plane. The
presented solution can also be a supplement to the sensory
luminous intensity measurement system. Image acquisition and
processing can be performed with the use of embedded systems
[7, 8], using dedicated vision sensors [9-11]. Fig. 2 shows
a general concept of the described mobile vision system.
Figure 2. Concept of intelligent vision system for the analysis of mechanical
wear of the prisms of in-pavement airport lamps
The paper presents various stages of image processing of the
built-in airport lamps. For the final classification assessment,
a two-layer neural network with forward feed was used, based
on the sigmoidal function in the hidden layer and the softmax
function (normalized exponential function) in the output layer.
Experimental research was carried out using the proprietary
database and the Matlab 2022a environment [12].

This research was funded partly by the 2022 subvention and partly with
the SMART4ALL EU Horizon 2020 project, Grant Agreement No 872614.
 II. GENERAL CONCEPT OF PROPOSED VISION SYSTEM
The general scheme of the image processing is shown in
Figure 3. Processing can be divided into two stages:
 detection of the area of interest (ROI)
 decision-making process based on a neural network.
Figure 3. General scheme of the processing process
The prism localization and detection is shown in Figure 4.
Lamp locating is performed with the circular Hough transform
to detect the lamp and then its position and orientation. The
prism area is detected with morphological operations and high-
pass filtering using the Sobel mask.
Figure 4. Diagram of the detection of lamp and prism
III. DATA SET FOR EXPERIMENTAL TESTS
The in-pavement lamps database consisting of 316 images
was prepared for the experimental research. The photos were
taken in different weather conditions and at different times of
the day, which makes it possible to test the operation of the
algorithms in many situations. Attempts were made to obtain
different positions of the lamps in the photographs and different
backgrounds (asphalt, concrete, or a white background in the
case of a lamp in the area of road markings on the airfield).
Ultimately, the photos have a resolution of 49283264 pixels
and a 24-bit color depth. Their format is JPG with the RGB
color space. Figures 5, 6 and 7 show examples of different types
of in-pavement airport lamps collected in the database.
Figure 5. Taxiway Center Line Lamp IDM5582
Figure 6. Runway center line lamp IDM4582
Figure 7. Touchdown zone lamp IDM4671
The lamps have different positions and brightness in the photos.
Additionally, some of them were switched on (the runway
centerline lamps work in both directions, the taxiway only in
one direction, according to the regulations). Such diversity is
valuable because the algorithms can be fine-tuned, allowing for
more accurate inspection, regardless of the conditions.
IV. LAMP AND PRISM DETECTION PROCESS
The lamp detection and prism localization starts with loading
an image from the database, scaled to 15% of its original size
(reduction of the data amount is critical for the neural network
training [11]). Then the image is converted to grayscale. On this
basis, the lamp is searched for in the image using the circular
Hough transform. Its determination is carried out by means of
appropriately selected ranges of the radius value of the lamp
being sought. The function uses the Circular Hough Transform
(CHT) algorithm to find the circles in the image. This approach
is used because of its resistance to noise, occlusion, and
fluctuating illumination.
 Then, smaller circles are searched for, which are airport
lighting mounting bolts. The pixel span has been selected
experimentally and optimally for search applications. These
operations are followed by an appropriate rotation of the input
image in order to obtain the same position for all photos. In
addition, the use of the primary image allows for obtaining
better quality of data necessary for further steps. Fig. 8 shows
the result of the lamp detection operation (blue circle) and its
orientation based on bolts (green circles).
Figure 8. The result of the operation of detecting the lamp (blue circle) and
its orientation (green circles)
The next operation limits the working area by cropping to
a part of the image, on which the prism is located. This image
is converted to grayscale and the edges are detected. Then
morphological operations and filtering with the help of the
Sobel operator are made to determine the position of the
searched prism, which also allows to reduce the ROI.
The finally used Sobel mask has the form:
0 0 0
[1 1 1]
0 0 0
Based on the conducted experiments, we focused on finding
horizontal edges as it was the optimum solution for our
application. As a result, distortions and small edges, which
caused errors in the further processing steps, have been partially
eliminated.
For the Sobel mask in diamond form, the obtained was
completely devoid of reference points, which made it
impossible for the algorithm to determine the position of the
prism. Fig. 9 shows the final result of detecting the prism
location of the tested built-in lamp.
Figure 9. The result of the prism detection operation
Table I summarizes the effectiveness of the various stages
of the algorithm's operation.
TABLE I. IDENTIFICATION EFFECTIVENESS
Efficiency of identification
Type of operation
Correct identification Incorrect identification
Lamp detection 90,81% 9,09%
Lamp orientation 90,21% 9,79%
V. PRISM QUALITY CLASSIFICATION
The two-layer feed-forward network implemented using
Open Neural Network Start app (nnstart in Matlab
environment), based on the sigmoidal function in the hidden
layer and the softmax function (normalized exponential
function) in the output layer, is able to correctly classify the
vectors when it contains a sufficient number of neurons in the
hidden layer. The network is trained using the scaled back
propagation of the conjugate gradient. Entry to the network is
the I matrix with dimensions of 1,440,000×284, which contains
284 samples, each of which with 1,440,000 elements (image
vector). The target matrix T dimensioned 3 × 284 also
represents 284 samples but with 3 elements (prism
classification). The database is limited at this stage to
284 images due to incorrect performance in the earlier stages
(no lamp detected). The first class (Fig. 10) are undamaged
prisms, the second are those that are no longer suitable for
operation (Fig. 11). The third group (Fig. 12) consists of images
when it is impossible to judge the wear of the prism. Images in
this group are most often the result of incorrect earlier stages of
finding the location of the lamp and prism. Fig. 13 shows the
ratio of the class distribution in our database.
Figure 10. Examples of tarnished prisms but with no mechanical damage
Figure 11. Examples of damaged prisms
Figure 12. Examples of improper prism segmentation
35
58
191
Class I Class II Class III
Figure 13. The ratio of the class distribution in the database
 Samples entering the network input are randomly divided
with each call, maintaining the following proportions:
 Training: 70% − 198 images
These images are fed to the network during its training, during
which it is adjusted according to its error.
 Validation: 15% − 43 images
The above samples are used to measure the network
generalization and to stop the training process when the value
of the generalization stops improving. This is indicated by the
increment of the cross-entropy error of the validation samples.
 Testing: 15% − 43 images
This part of the process does not affect the training of the neural
network, thus it is an independent indicator of performance
during and after training.
Figure 14. Neural network confusion matrices
The results of the trained neural network turned out to be
very satisfactory, where 81.4% of correct classifications were
obtained during the tests (Figure 14).
VI. SUMMARY
The paper presents an intelligent vision system and the
performed experiments of quality classification for the prisms
of in-pavement airport lamps. The assessment was made with
the use of a proprietary database prepared in cooperation with
Poznań-Ławica Airport, Poland. The specificity of airport
lamps requires proper selection of parameters for image
processing algorithms and functioning of the neural network.
Improvement of the ROI detection efficiency can be sought
with lighting systems when taking photos, which is currently
subject of the further research. The accuracy obtained in the
presented classification process is 81.4% and can be improved
by increasing the number of photos in the database, balancing
the number of photos in each class, which the authors also
intend to do.
The experiments were performed using the standard Open
Neural Network Start app in the Matlab environment and
standard parameters of neural network operation were used. In
the future, the authors plan to expand the database and examine
the impact of neural network parameters on the efficiency and
speed of the segmentation and classification process.
REFERENCES
[1] Certification Specifications (CS) and Guideline Material (GM) for
Aerodrome Design Edition 3, Annex to Decision No. 2016/027/R of the
EASA Executive Director, European Aviation Safety Agency. Available
online: https://www.easa.europa.eu/downloads/21730/en (accessed on 4
July 2022).
[2] Suder J., Maciejewski P., Podbucki K., Marciniak T., Dąbrowski A.,
Platforma pomiarowa do badania jakości działania lamp lotniskowych,
Pomiary Automatyka Robotyka, R. 23, No. 2/2019, pp. 5-13,
https://doi.org/10.14313/PAR_232/5
[3] Suder J, Podbucki K, Marciniak T, Dąbrowski A. Spectrum sensors for
detecting type of airport lamps in a light photometry system. Opto-
Electronics Review. 2021;29(4):133–140.
https://doi.org/10.24425/opelre.2021.139383
[4] Podbucki K., Suder J., Marciniak T., Dąbrowski A., Elektroniczna
matryca pomiarowa do badania lamp lotniskowych, Przegląd
Elektrotechniczny, No. 02/2021, pp. 47-51,
https://doi.org/10.15199/48.2021.02.12
[5] Sitompul D. S. D., Surya F. E., Suhandi F. P., Zakaria H., "Runway Edge
Light Photometry System by Using Drone-Mounted Instrument," 2019
International Symposium on Electronics and Smart Devices (ISESD),
2019, pp. 1-5, doi: 10.1109/ISESD.2019.8909498.
[6] Sitompul D. S. D., Surya F. E., Suhandi F. P., Zakaria H., "Horizontal
Scanning Method by Drone Mounted Photodiode Array for Runway Edge
Light Photometry," 2019 International Seminar on Intelligent Technology
and Its Applications (ISITIA), 2019, pp. 41-45, doi:
10.1109/ISITIA.2019.8937211.
[7] Barua B., Gomes C., Baghe S., Sisodia J., A Self-Driving Car
Implementation using Computer Vision for Detection and Navigation,
2019 International Conference on Intelligent Computing and Control
Systems (ICCS), 2019, pp. 271-274, doi:
10.1109/ICCS45141.2019.9065627
[8] Podbucki K., Suder J., Marciniak T., Dąbrowski A., CCTV based system
for detection of anti-virus masks, 2020 Signal Processing: Algorithms,
Architectures, Arrangements, and Applications (SPA), 2020, pp. 87-91,
doi: 10.23919/SPA50552.2020.9241303
[9] Suder, J.; Podbucki, K.; Marciniak, T.; Dąbrowski, A. Low Complexity
Lane Detection Methods for Light Photometry System. Electronics 2021,
10, 1665. https://doi.org/10.3390/electronics10141665
[10] Suder, J., Możliwości przetwarzania sekwencji wizyjnych w systemach
wbudowanych, Przegląd Elektrotechniczny, No. 01/2022, pp. 188-191,
doi: 10.15199/48.2022.01.41
[11] K. Podbucki, J. Suder, T. Marciniak and A. Dabrowski, "Evaluation of
Embedded Devices for Real- Time Video Lane Detection," 2022 29th
International Conference on Mixed Design of Integrated Circuits and
System (MIXDES), 2022, pp. 187-191, doi:
10.23919/MIXDES55591.2022.9838167.
[12] MATLAB 2022a Documentation Available
online:https://www.mathworks.com/help/matlab/release-notes.html.
(accessed on 4 July 2022).