2021 International Conference on System Science and Engineering (ICSSE)
A Computer Vision System
for Power Transmission Line Inspection Robot
Huu Tho Tran1,2
1Ho
Chi Minh City Uni. of Technology
2Vietnam National University HCM
2021 International Conference on System Science and Engineering (ICSSE) | 978-1-6654-4848-2/21/$31.00 ©2021 IEEE | DOI: 10.1109/ICSSE52999.2021.9538440
Ho Chi Minh City, Vietnam
tho.tranhuu@hcmut.edu.vn
Quang Huy Tran1,2
1Ho
Chi Minh City Uni. of Technology
2Vietnam National University HCM
Ho Chi Minh City, Vietnam
huy.tranlhp20699@hcmut.edu.vn
Abstract— Periodic inspections of power grids to prevent
power outages and promptly handle potential risks are one of
the most important tasks of the electricity industry. However,
this is a tedious, time-consuming and dangerous job as all
current inspection methods are carried out manually. This
paper proposes a computer vision system that assists inspection
robots working on overhead high voltage transmission lines to
operate autonomously. Our system performs three main
functions. The first is to detect obstacles by using YOLOv4, a
state-of-the-art object detection technique so that the robot can
determine how to properly overcome obstacles. The second is to
estimate the distance to the obstacle by using linear regression
technique so that the robot can determine the exact time to
overcome the object. The third is to detect wire defects based on
the wire edges by using image processing techniques. Our
achieved performance of the system: detecting obstacles with
mAP@0.5 equal to 98.65%, estimating distance to objects with
average mean absolute error equal to 0.81cm in the range from
20cm to 100cm, and detecting wire defects with precision equal
to 90.24% and recall equal to 86.05%. Our computer vision
system is accurate and reliable, ready to integrate with the robot
in real life. Inspection robots with this system will make the
inspection of power lines faster and simpler, which saves time,
maintenance costs and labor.
Keywords—power transmission line inspection, deep learning,
computer vision system, distance estimation, wire defect detection
I. INTRODUCTION
Computer vision system for power transmission line
inspection robot is a novel research topic. There have been
several research articles on this topic, however, these studies
still have many shortcomings. For example, in [1], the system
uses the classical object detection model Support Vector
Machine (SVM) and Histogram of Oriented Gradients (HOG)
to detect obstacles, which gives inaccurate results and cannot
detect objects at long distances. Also, the distance estimation
algorithm in [1] uses the position of the object in the frame,
which may lead to inaccurate results because the position of
the object in the frame depends on many volatile factors such
as the curvature of the string, camera placement and camera
viewing angle. Wire defect detection algorithm in [1] is
susceptible to background noise.
In [2], the authors used HOG descriptor in combination
with Logistic Regression to detect electric towers inside the
image. This method has the advantage of low computational
978-1-6654-4848-2/21/$31.00 ©2021 IEEE
Authorized licensed use limited to: China University of Petroleum. Downloaded on November 17,2023 at 05:58:58 UTC from IEEE Xplore. Restrictions apply.
Minh Quan Tran1,2
1Ho
Chi Minh City Uni. of Technology
2Vietnam National University HCM
Ho Chi Minh City, Vietnam
quan.tran.minh@hcmut.edu.vn
Viet Cuong Pham1,2
1Ho
Chi Minh City Uni. of Technology
2Vietnam National University HCM
Ho Chi Minh City, Vietnam
pvcuong@hcmut.edu.vn
cost but comes with high False Positive rate (22%) and high
False Negative rate (60%). The reason for these high error
rates is because the HOG descriptor is not able to capture all
the important features of the objects. Thus, Logistic
Regression cannot discriminate the objects and background
in captured image.
In the field of wire defect detection, Wang et al. [3]
proposed a method based on Fast R-CNN to extract the
features and pass them to SVM classifier to detect faults on
the transmission line, the algorithm was able to obtain high
accuracy in this task but comes with considerably high
computational cost.
For the distance estimation task using a single camera,
Megalingam et al. [4] used a traditional method of pinhole
model to measure the distance via a single camera. There are
also several sophisticated methods to estimate distance using
stereo vision algorithms which are discussed by Tippetts et
al. [5]. These algorithms come with different computational
times, and some of them have attained high accuracy and
computational time fast enough for real-time application.
In this paper, we propose a computer vision system that
assists inspection robots working on overhead high voltage
transmission lines to operate autonomously. We use
YOLOv4 [6] object detection model which is a deep
convolution neural network model instead of HOG descriptor
used in [1], [2] to improve the object detection accuracy since
the YOLOv4 model is one of the best in terms of accuracy
and especially designed to work in real-time. Then, we
employ image processing techniques to furthermore fine-tune
the bounding boxes and use these bounding boxes as input for
the pinhole distance estimators similar to those used in [4].
This approach gives us a boost in accuracy with regard to
distance measurement when being compared with the result
in [1]. For the wire defect detection, we use traditional image
processing methods to reduce the computing time in
comparison to [3] and still be able to retain the accuracy to
some extent. The paper focuses on the three most common
obstacles on overhead high voltage transmission lines:
marker balls, dampers and clamps, as shown in Fig. 1.
289
 2021 International Conference on System Science and Engineering (ICSSE)
Fig. 1. Ostacles used in the paper. From left to right: marker ball, damper,
clamp.
The rest of the paper consists of four sections as follows.
Section II introduces our proposed methods in detail, which
include obstacle detection, distance estimation and wire defect
detection. Section III shows experiment results to evaluate the
performance of the methods. Section IV does a comparison
with a method in [1]. Finally, we conclude the paper in Section
V.
II. METHODS
A. Obstacle Detection
Obstacle detection is an important task in helping the
robot determine how to properly overcome obstacles and to
provide the input to the distance estimation algorithm. We use
a state-of-the-art object detection model YOLOv4 which is
very fast and accurate.
The dataset for training the model includes images taken
at many positions with different distances to an obstacle. To
reduce overfitting, we use image augmentation techniques
like changing random background, Gaussian blur, changing
gamma, adding random noise, shifting, rotation and contrast
limited adaptive histogram equalization (CLAHE).
B. Distance Estimation
In order for the robot to know when it is close enough to
pass an obstacle, it needs to know the distance to the obstacle.
The computer vision system acts as the robot's eye, receiving
the image of the obstacle and estimating the distance from the
camera to the obstacle, thereby deducing the distance from
the robot to the obstacle. An idea of the distance estimation
is shown below.
1) Calculate distance from object width in the image
We can approximate our camera as an ideal pinhole-
camera model to estimate distance from the camera to a
known width object. An ideal pinhole-camera formula to
calculate distance to a known width can be represented as (1):
𝑓∗𝑊𝑜 ∗𝑤𝑖
𝐷=
𝑤𝑜 ∗𝑤𝑠
where D (mm) is the distance from camera to object, f (mm)
is the focal length of the camera, 𝑊𝑜 (𝑚𝑚) is the real object
width, 𝑤𝑜 (px) is the object width represented in the image,
𝑤𝑖 (px) is the image width, 𝑤𝑠 (mm) is the sensor width of
camera.
With a known camera and obstacle, we can simplify (1)
into (2):
1
𝐷=𝑘∗
𝑤𝑜
where k is a constant depending on the camera parameters and
obstacle size. The distance from camera to object and the
inverse of object width extracted from the image are indeed
having a linear relationship, hence we can estimate the
Authorized licensed use limited to: China University of Petroleum. Downloaded on November 17,2023 at 05:58:58 UTC from IEEE Xplore. Restrictions apply.
distance using linear regression technique. In practice, due to
the error in calculating the object width and the error when
measuring the real distance, the intercept of a regresion line
can be non-zero.
The object width can be calculated from the bounding box
output by the obstacle detection model. However, although
the bounding boxes output from the YOLOv4 model are very
accurate, they are not always perfect. To improve the
accuracy, we do not calculate the object width directly from
the bounding box. Bounding box helps to narrow the area
containing the object, then we use the following methods to
calculate the object width.
2) Calculate object width in the image
a) Canny edge method
We get the width of the object in the image by detecting
the outer edge of the object. To achieve this goal, we use
Canny edge detection technique.
To detect the outer edge of the object in image, we extend
the bounding box output from the object detection model with
a fixed ratio (10%). Then, we perform Canny edge detection
in the extended bounding box to detect the outer edge of
object and furthermore calculate the width of the object in
pixel.
However, sometimes when the background contains
strong edges, these edges can be incorrectly detected as the
outer edge of the object in the image.
b) Circle with the strongest edge method
For objects containing round edges like marker balls and
dampers, we use the radius of the circle with the strongest
edge instead of the width of the object for more accuracy.
To achieve the goal of detecting the circle with the
strongest edge, first, we take advantage of Canny edge
detection algorithm to detect edges in the image. We do not
use fixed threshold parameters for the Canny edge detector
but iterate to find the highest threshold possible over the
possible region of Canny threshold values. After detecting the
edges inside the image by Canny edge detection, we use
Hough circle detection to identify whether there is any strong
circle in this edge image
If there exists at least a circle, we go back and increase
threshold value for Canny edge detection to check for the
existence possibility of stronger circle edges in the original
image. Otherwise, if there are not any strong circles, we go
(1) back to the highest Canny threshold which still produces at
least a circle when performing Hough circle detection
algorithm and use this threshold to detect the circle with the
strongest edge in the image.
To reduce the computational complexity of the
computation when iterating over possible region of Canny
threshold, we use binary search idea for finding the highest
Canny threshold with the procedure as follows:
 Step 1: Choose min and max value of high threshold for
Canny Edge Detection algorithm. We initialize L = min
(2) and R = max.
 Step 2: If 𝐿 < 𝑅 we jump to step 3, otherwise we jump
to step 5.
290
 2021 International Conference on System Science and Engineering (ICSSE)

𝐿+𝑅
 Step 3: We set 𝑢𝑝𝑝𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 = 𝑟𝑜𝑢𝑛𝑑( ) and
2
𝑢𝑝𝑝𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑
𝑙𝑜𝑤𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 = and perform Canny
2
edge detection followed by Hough circle detection with
the calculated Canny thresholds.
 Step 4: If Hough circle detection cannot find any strong
circle, we set 𝑅 = 𝑢𝑝𝑝𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 and move to step
5. Otherwise we set 𝐿 = 𝑢𝑝𝑝𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 + 1 and
jump to step 2.
 Step 5: We set 𝑢𝑝𝑝𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 = 𝑅 − 1 and
𝑅−1
𝑙𝑜𝑤𝑒𝑟 𝑡ℎ𝑟𝑒𝑠ℎ𝑜𝑙𝑑 = and then perform Canny edge
2
detection followed by Hough circle detection and return
the detected circle as the strongest edge circle detected
in the image.
The results of circle detections are illustrated in Fig. 2. (a)

(a)

(b)

(b)
Fig. 2. Circle detection results of (a) marker ball; (b) damper

3) Linear regression model training

The training set includes actual distances and inverse
object widths obtained from images taken at 81 different
positions in a range of 20cm to 100cm, each location spaced
apart 1cm. After training, we obtain best fit lines for the data (c)
points, as shown in Fig. 3.
Fig. 3. Regression lines of (a) marker ball; (b) damper; (c) clamp

C. Wire Defect Detection

Wire defect is one of the most common defects in power
transmission systems. Timely detection and repair will
prevent major losses. The wire defect detection algorithm
will help the robot know if the wire in front of it is damaged.
The algorithm is presented as follows.
First, we convert an image to grayscale for easier
handling. In order to process more accurately, instead of

291
Authorized licensed use limited to: China University of Petroleum. Downloaded on November 17,2023 at 05:58:58 UTC from IEEE Xplore. Restrictions apply.
 2021 International Conference on System Science and Engineering (ICSSE)
processing the entire frame, we focus on a small region
containing the wire. We extract a fixed narrow region of the
frame that always contains the wire segment, as shown in
Fig. 4.
Fig. 4. Fixed narrow region of the frame that always contains the wire
segment
However, the extracted region is still quite large. We use
Template Matching technique to track the wire segment in the
extracted region. Template Matching technique works well in
this case because the shape of the wire segment in the
extracted region does not change much and is completely
different from the background. The template and the
matching result are shown in Fig. 5 and 6. The matched
region will be processed in our next step.
Fig. 5. Template for tracking wire
Fig. 6. Region obtained after using template matching technique
The following processing steps are used to determine if
the tracked wire is defective:
 Step 1: Segment the wire out of the background by using
adaptive threshold technique. Fig. 7 shows the result of
this step.
Fig. 7. The result after segmenting the wire out of the background
 Step 2: Apply opening transformation to remove
background noise. Fig. 8 shows the result of this step.
292
Authorized licensed use limited to: China University of Petroleum. Downloaded on November 17,2023 at 05:58:58 UTC from IEEE Xplore. Restrictions apply.
Fig. 8. The result after applying opening transformation
 Step 3: Extract wire boundary by using Canny Edge
Detector. Fig. 9 shows the result of this step.
Fig. 9. Wire boundary
 Step 4: Calculate the number of pixels of wire edges by
summing all white pixels in the image after the edge is
extracted. If the number of pixels of the wire edges is
higher than the upper threshold or lower than the lower
threshold, the wire segment is defective. The threshold
is selected based on a decision scheme as shown in Fig.
10. In this scheme, we choose the upper threshold equal
to 36 and the lower threshold equal to 45.
Fig. 10. Decision scheme to choose the defect threshold.
III. EXPERIMENT RESULTS
In this section, we present experiment results to evaluate
the performance of our methods. We divide the section into
four parts. Section A gives an introduction to datasets.
Section B presents evaluation criteria to evaluate the
methods. Section C shows the results on the dataset. Finally,
we compare our method with the method in [1] in Section D.
A. Dataset
Our dataset for training and evaluating the obstacle
detection model includes 3500 images per class, of which
3000 training images are augmented from 81 images taken at
positions with different distances to the obstacle, and 500
images taken with different backgrounds and brightness are
used for evaluating. Image augmentation techniques are
outlined in Section II-A. To verify the effectiveness of the
wire defect detection algorithm, a dataset of 212 different
wire segments is used.
B. Evaluation Criteria
For obstacle detection, we use evaluation metrics such as
precision, recall, F1-score, mean average precision
(mAP@0.5) and average IoU. We also test the performance
of the model for each class using metrics such as True
Positive (TP), False Positive (FP) and Average Precision
(AP).
 2021 International Conference on System Science and Engineering (ICSSE)

For distance estimation, we calculate Mean Absolute TABLE IV

Error (MAE) and Root Mean Square Error (RMSE) between PERFORMANCE OF WIRE DEFECT DETECTION
the estimated distances and the actual distances at 81 different
TP FP FN Precision Recall F1-score
positions, in the range from 20cm to 100cm, each position is
37 4 6 90.24% 86.05% 88.10%
1cm apart. MAE and RMSE are given by (3) and (4):

1
𝑀𝐴𝐸 = ∑𝑁
𝑖=1 |𝑦
̂𝑖 − 𝑦𝑖 | (3)
𝑁

𝑁 2
∑ (𝑦̂𝑖 −𝑦𝑖 )
𝑅𝑀𝑆𝐸 = √ 𝑖=1 (4)
𝑁

where N is the number of data, 𝑦̂𝑖 is the ⅈ𝑡ℎ estimated

distance and 𝑦𝑖 is the ⅈ𝑡ℎ actual distance.
To evaluate the performance of wire defect detection, we
also use metrics such as precision, recall and F1-score.
Precision indicates reliability if the system detects a defect,
while recall expresses the ability to find all defects in the
dataset. F1-score is the harmonic mean of precision and recall
that takes both two metrics into account.
C. Results
We test the obstacle detection model on the test set
obtained from the dataset in Section III-A. The results are
shown in Table I and II:
Fig. 11. Some wire defect detection results
The high precision shows that defect results are reliable. The
result of the recall indicates that our system can detect most
of defects. Some defects that do not change edges of the wire
are not detected. Fig. 12 shows an example.
(a) (b)
Fig. 12. An example of an undetected defect. (a) Wire segment defect. (b)
Edges of the wire segment.

TABLE I IV. DISCUSSION

PERFORMANCE OF OSTACLE DETECTION MODEL FOR EACH CLASS Comparisons of obstacle detection and distance
Class name AP TP FP estimation with the method in [1] are shown in Fig. 13 and
Marker ball 100% 499 0 Table V:
Damper 98.34% 445 18
Clamp 97.60% 452 138

TABLE II
OVERALL PERFORMANCE OF OBSTACLE DETECTION MODEL

Average
Precision Recall F1-score mAP@0.5
IoU
90% 95% 92% 84.61% 98.65%

The extremely high results of AP show that the model can

detect obstacles precisely. The high result of average IoU
indicates that detected bounding boxes are very accurate.
The performance of distance estimation is shown in Table
III:

TABLE III
PERFORMANCE OF DISTANCE ESTIMATION

MAE RMSE Fig. 13. Compare results of obstacle detection. Left: Our method. Right:
Marker ball 0.88cm 1.15cm Method in [1]
Damper 0.89cm 1.19cm
TABLE V
Clamp 0.66cm 0.83cm
COMPARISON OF RMSE OF DISTANCE ESTIMATION
The errors of all three classes are low enough for the robot to Method in [1] Our method
overcome the obstacles, especially the closer the camera is to (20cm – 55cm) (20cm – 100cm)
the object the smaller the error and the more stable the Marker ball 2.02cm 1.15cm
estimated distance. Damper 1.83cm 1.19cm
The performance of wire defect detection on the dataset Clamp 2.27cm 0.83cm
is shown in Table IV and some results are illustrated in
Fig. 11.

293
Authorized licensed use limited to: China University of Petroleum. Downloaded on November 17,2023 at 05:58:58 UTC from IEEE Xplore. Restrictions apply.
 2021 International Conference on System Science and Engineering (ICSSE)
Clearly, our method gives more accurate bounding boxes and
better estimated distance. Our estimated range is also wider,
up to 100cm.
V. CONCLUSION
Our computer vision system is accurate and reliable,
ready to assist inspection robots working on overhead high
voltage transmission lines to operate autonomously. The
results we have achieved include detecting obstacles with
mAP@0.5 of 98.65% and average IoU of 84.61%, estimating
the distance to the obstacle with average MAE of 0.81cm in
the range from 20cm to 100cm, and detecting wire defects
with precision equal to 90.24% and recall equal to 86.05%.
Robots with this system will make the inspection of power
lines faster and simpler, which saves time, maintenance costs
and labor.
ACKNOWLEDGEMENT
We would like to thank Ho Chi Minh City University of
Technology (HCMUT), VNU-HCM for the support of time
and facilities for this study.
294
Authorized licensed use limited to: China University of Petroleum. Downloaded on November 17,2023 at 05:58:58 UTC from IEEE Xplore. Restrictions apply.
REFERENCES
[1] C. S. Pham, T. N. Vo and V. C. Pham, “A Computer Vision System for
Overhead Line Inspection Robots,” Ho Chi Minh City University of
Technology, Vietnam, 2020.
[2] C. Martinez, C. Sampedro, A. Chauhan, J. F. Collumeau, and P.
Campoy, “The Power Line Inspection Software (PoLIS): A versatile
system for automating power line inspection,” Engineering
Applications of Artificial Intelligence, vol. 71, pp. 293–314, 2018.
[3] Y. Wang, Q. Li, and B. Chen, “Image classification towards
transmission line fault detection via learning deep quality-aware fine-
grained categorization,” Journal of Visual Communication and Image
Representation, vol. 64, p. 102647, 2019.
[4] R. K. Megalingam, V. Shriram, B. Likhith, G. Rajesh, and S. Ghanta,
“Monocular distance estimation using pinhole camera approximation
to avoid vehicle crash and back-over accidents,” 2016 10th
International Conference on Intelligent Systems and Control (ISCO),
2016.
[5] B. Tippetts, D. J. Lee, K. Lillywhite, and J. Archibald, “Review of
stereo vision algorithms and their suitability for resource-limited
systems,” Journal of Real-Time Image Processing, vol. 11, no. 1, pp.
5–25, 2013.
[6] A. Bochkovskiy, C.-Y. Wang, H.-Y. M. Liao, “YOLOv4: Optimal
Speed and Accuracy of Object Detection,” 2020, arXiv:2004.10934.
[7] J. Canny, "A Computational Approach to Edge Detection," in IEEE
Transactions on Pattern Analysis and Machine Intelligence, vol.
PAMI-8, no. 6, pp. 679-698, Nov. 1986, doi:
10.1109/TPAMI.1986.4767851.