You are on page 1of 6

2022 2nd International Conference on Intelligent Technologies (CONIT)

Karnataka, India. June 24-26, 2022

Crack Detection and Classification for Reinforced


Concrete Structures using Deep Learning
Alonica Villanueva Juan Karlo B. Balba Carlo D. Beceril Jaira Lyne G. Belza
Computer Engineering Computer Engineering Computer Engineering Computer Engineering
Department Department Department Department
Technological Institute of the Technological Institute of the Technological Institute of the Technological Institute of the
Philippines Philippines Philippines Philippines
Quezon City, Philippines Quezon City, Philippines Quezon City, Philippines Quezon City, Philippines
avillanueva.cpe@tip.edu.ph jkbalba@gmail.com cbeceril101@gmail.com Jairalynebelza@gmail.com

Maria Cecilia A. Venal Menchie M. Rosales


2022 International Conference on Intelligent Technologies (CONIT) | 978-1-6654-8407-7/22/$31.00 ©2022 IEEE | DOI: 10.1109/CONIT55038.2022.9848129

Roel Ian P. Tagle


Computer Engineering Department Computer Engineering Department
Computer Engineering Department
Technological Institute of the Technological Institute of the
Technological Institute of the Philippines
Philippines Philippines
Quezon City, Philippines
Quezon City, Philippines Quezon City, Philippines
roeliantagle@gmail.com
cecilia.venal@tip.edu.ph Mmiranda.cpe@tip.edu.ph

Abstract—Building cracks such as gaping cracks, separation, [8], same as the proposed study which uses You Only See
and horizontal cracks are a few types of cracks that possess a Once (YOLO) v2 that identifies cracks on the roads [9] and
severe issue on reinforced concrete; hence, the earlier the identifies the crack type and its class name, U-Net was used in
detection the cheaper the repairs. Numerous studies about crack Pixel-level crack detection [10] that identifies segmentation
detection considered VGG16, and Faster R-CNN because the through a neural network accuracy which is above 92%,
severity level is crucial to each construction company business detection of cracks in Electroluminescence Images using
owner. This paper aims to build a deep learning model using Steerable evidence filter(SEF)-CNN [11], and others uses
Yolov3 that can detect a crack in reinforce concrete structures wireless camera network in detecting asphalt pavement cracks
and categorize a medium, severe, or very severe crack using an
[12].
android application. An android application was developed
instead of using an expensive Ultrasonic Pulse Velocity in the A different deep learning approach can also be used to
market to detect the severity of the crack on the concrete. detect the crack in the pipeline [13]. In contrast, YOLO v3 uses
a one-stage system that loses accuracy when achieving high
The overall accuracy summary of the android application is detection speeds [14]. Another model that takes place, which is
93.33%, while the kappa value is .97. Therefore, the deep VGG16 in 2020 [15], says it is much more accurate than the
learning model and android application produced an accurate YOLO algorithm, and VGG19 is a good choice in real-time
calculation in detecting the crack and determining its crack
detection [16]. One of the papers showed that Yolov3 has a
classification.
clear advantage for pavement crack detection speed [17]
Keywords— Crack, detection, crack classification, crack because of its fastest way and can produce a high accuracy rate
detection, YOLOv3, deep learning, and severity in detecting cracks and was tested on the railway bridge [18].
Also, one of the researchers creates output and increases the
accuracy using Faster R-CNN [14].
I. INTRODUCTION
Due to the increasing labor cost and the increasing number The cracks in the buildings are a significant problem
of structures built by the client every year, most inspections are nowadays [19]. Crack in reinforced concrete structures is
done annually up to every other year in the Philippines [1]. undesirable aesthetically, it affects the structure's durability,
Thus, many studies were created to automatically detect the and it leads to changes in stiffness and change force
cracks, like the study which uses magnetic fields to detect a distribution in the structure. It gives the feeling that the
crack in the underground pipeline [2]. Nevertheless, the construction is unsafe. Thus, the system detects cracked and
process of crack detection and classification based on the non-cracked concrete structures and could classify the crack
physical attributes of the surface area in the building is the based on cracked severity of damage using the android
basis of generating the created study. application compared to another study that classifies the crack
damage such as horizontal, vertical, branch, and diagonal
Deep Convolutional Neural Network (CNN) was used in cracks [20]. Therefore, the proposed system categorizes the
different studies in detecting cracks such as pavement cracks crack in its severity level, such as medium, severe, and very
[3], [4], [5], [6]. Exploration of deep learning models in crack severe, based on the Control of Cracking in Concrete
detection was given importance by utilizing NB-CNN. Naive Structures standard [21]. It uses the Convolutional Deep
Bayes [7] for nuclear power plant, CrackNet-V on 3D asphalt Learning model using YOLO v3. A total of 1105 datasets used

978-1-6654-8407-7/22/$31.00 ©2022 IEEE 1

Authorized licensed use limited to: UNIVERSIDAD DE SANTIAGO DE CHILE. Downloaded on April 19,2023 at 22:01:34 UTC from IEEE Xplore. Restrictions apply.
for this algorithm were categorized as medium, severe, or very Fig. 2 shows the contents of the buttons shown in Fig. 1.
severe, unlike the previous study, which classified the crack as The image on the left is the content of the gallery button in the
wide, shallow, and fine crack [8]. Some categories include main menu; hence, to view all the previous images generated
longitudinal, linear, and alligator crack [9]. by the application. The second image from the left shows the
full-sized image from the gallery. The third image shows the
II. METHODOLOGY walkthrough instructions on how to operate the application.
Lastly, the image on the right shows the contents of the about
The methodology comprises an android application, deep us button on the main menu explaining why the application is
learning algorithm, and processes such as architecture and designed.
datasets as an overall software design.
B. Deep Learning
A. Application Design Convolutional Neural Network (ConvNet/CNN) is a deep
This section discusses the android application design. The learning algorithm that can take in an input image, assign
Android Application is an aid for crack detection and importance to various aspects/objects in the image, and
classification for building inspectors and safety officers used. differentiate one from the other [22]. The pre-processing
The application allows the user to detect and classify cracks on required in a ConvNet is much lower than in different
reinforced concrete structures with the help of the smartphone's classification algorithms [23]. While in primitive methods,
camera. It runs on an Android device with a minimum version filters are hand-engineered, with enough training, ConvNets
of Android 6.0. The application design is based on the can learn these filters/characteristics [24]. The innovation of
interviewed guiding matrix used by the Department of Public convolutional neural networks automatically learns many
Works and Highways, as shown in Fig. 1. filters in parallel specific to a training dataset under the
constraints of a particular predictive modeling problem, such as
image classification. The result is particular features that can be
detected anywhere on input images. It only concludes that even
with multiple sizes of cracks on an image, the YOLO algorithm
could detect it [25].
The YOLOv3 Architecture is in Fig. 3.

Fig. 1. Android Application Design

Fig. 1 shows the software design of the Android


application. The image on the left is the first screen of the
application to get started. The image on the right shows the
application's main menu containing four (4) buttons. The
capture button on the top left is for image capture, wherein
permission to open the device camera is needed to begin
capture for the first time of use. The gallery button on the top
right is used to view the previous output images of the Fig. 3. YOLOv3Network Architecture [26]
application. The user manual button on the lower left is used to
get some insights into the guide and How-tos of the The image in Fig. 3 shows above is YOLO's Architecture.
application. Lastly, the about us button on the lower right The uniqueness of YOLO makes use of only convolutional
opens a page about why the application is designed. As shown layers, making it a fully convolutional network (FCN). It has
in Fig. 2 is the Main Menu contents. 75 convolutional layers, with skip connections and up-
sampling layers. No form of pooling is used, and a
convolutional layer with stride 2 is used to down-sample the
feature maps. This helps in preventing the loss of low-level
features often attributed to pooling. The architecture boasts of
residual skip connections and up-sampling. The most salient
feature of v3 is that it makes detections at three different scales.
YOLO is a fully convolutional network, and its eventual output
is generated by applying a 1 x 1 kernel on a feature map. In
YOLO v3, the detection is done using 1 x 1 detection kernels
on feature maps of three different sizes at three different places
Fig. 2. Main Menu Contents
in the network. YOLO v3 predicts at three scales, which are

Authorized licensed use limited to: UNIVERSIDAD DE SANTIAGO DE CHILE. Downloaded on April 19,2023 at 22:01:34 UTC from IEEE Xplore. Restrictions apply.
precisely given by down-sampling the dimensions of the input Fig.5 shows the visual presentation of the block diagram in
image by 32 strides, 16, and 8, respectively [26]. figure 2.16. First, the N x N grid is generated wherein each grid
has its bounding boxes and confidence level, including a
C. Dataset probability class-map for different classifications. Lastly, the
The dataset used to train the deep neural network model final detection outputs the confirmed predicted image. A
comprises 1105 images. The image dataset used in this design sample detected cracked image is shown in Fig. 6.
is SDNET2018 by Maguire, Marc: Dorafshan, Sattar: and
Thomas, Robert J [27] for machine learning applications. This
dataset contains 56,000 crack and non-crack concrete
structures using a 16 Megapixel Nikon digital camera. Cracks
on this dataset are as narrow as 0.06mm and wide as 25mm.
Fig. 4 explains how the data or datasets are processed
throughout the system.

Fig. 6. YOLO Validation

Fig. 6 shows that the YOLO algorithm can detect all 3


classifications, namely medium, severe, or very severe crack,
during the validation of a concrete image with crack from
google images. This also shows that even with multiple sizes
of cracks on an image, the YOLO algorithm could detect it.
Fig. 4. Data Process Diagram of YOLO D. Software Design
Fig. 4 shows the process of crack detection and The android applications’ front-end development uses
classification on the server. The process starts with the input Android Studio, while its back-end development is Python
image in an indefinite size. The image is then resized to 240 x programming using Tensorflow. The system is pure software
240 dimensions; hence, generating grid cells dividing into N x deployed in a server-based mobile application that uses
N squares wherein each square predicts bounding boxes with YOLOv2. Below is the systematic procedure of YOLOv3 in
different confidence scores and the class probability of the the server. The software design diagram is in Fig. 7.
bounding boxes. These confidence scores reflect how confident
the bounding box contains a crack. Lastly, using the process of
non-max suppression, the bounding boxes having a low
confidence score are removed, leaving the bounding box with
the highest confidence score reflecting the image as a classified
output. Fig. 5 shows the image block process.

Fig. 7. Software Design Diagram

Fig. 7 shows an application will serve as an interface for


the user to run the system. First, the captured image from the
device is uploaded to the server and uses the trained graph
model generated in Tensorflow via a convolutional network
algorithm; this is where the detection and classification take
place. Lastly, the classified image is downloaded back to the
display device for output, as shown in Fig. 8.
Fig. 5. Image View Per Process Block

Authorized licensed use limited to: UNIVERSIDAD DE SANTIAGO DE CHILE. Downloaded on April 19,2023 at 22:01:34 UTC from IEEE Xplore. Restrictions apply.
serves as a backbone for crack detection while the
classification processes are through TensorFlow.
Confusion Matrix allows visualization of the classification
performance, as shown in Table 2. The confusion matrix
helped compute overall accuracy, false-positive rate, false-
negative rate, true positive, and true negative. A true positive is
an outcome where the classification model correctly classified
the positive class. Similarly, a true opposite is an outcome
where the model correctly classified the negative class. A false
positive is an outcome where the model incorrectly predicts the
positive class. Also, a false negative is an outcome where the
model incorrectly classified the negative class. Since
classification is a unique machine-learning problem, the
confusion matrix is essential in analyzing the performance of
the classification model on a test set for which true values are
Fig. 8. Systematic Procedure of the YOLO Algorithm
known [28].
Fig. 8 shows how the YOLO algorithm can detect a
specific object on an image. First, identifying the bounding TABLE II. CONFUSION MATRIX OF CRACK DETECTION
boxes for an instance of a crack is determined. In the YOLO Actual
algorithm, the image is divided into N x N squares, and each Very Overall
Tests Medium Severe
squared predicts five bounding boxes with different aspect Severe Classification
Medium 9 0 2 11
ratios. However, not every bounding box has an object on it or Severe 1 10 0 11
has an instance of crack on it. The bounding box having thicker Classified
Very
0 0 8 8
lines represented by a confidence score is the detected crack Severe
Actual
object of the model. Hence, the confidence score reflects how Overall
10 10 10 30
likely the box contains a crack. Thus, discarding the bounding = denotes the number of correctly identified cracks
boxes with a low confidence score allows the system to easily Legend:
= denotes the total number of trials presented in crack
detection
detect and classify the instance of a crack on a concrete. = denotes the number of incorrectly identified cracks

III. TESTING AND RESULTS During the testing process after the training of data sets,
The primary function of the mobile application is to detect Table 2 shows the number of identified cracks, the total
and classify concrete cracks by capturing the image. Hence, the number of trials presented in crack detection, and the number
result is compared to a crack comparator for visualization and of incorrectly identified cracks.
to confirm the accuracy of the application. The test results The summary of True Positive Rate, True Negative Rate,
compare the manual severity level using the crack comparator False Positive Rate, and False Negative Rate for Crack
and the automated severity level generated by the application Detection is shown in Table 3.
basing the interviewed guiding matrix from DPWH. Testing
and results used the confusion matrix and the Kappa equation.
TABLE III. SUMMARY OF DETECTION RATE
The test results used different test images as a sample:
True True False False
crack images with medium to very severe levels of severity are Object Positive Negative Positive Negative
shown in Table 1. Rate Rate Rate Rate
Crack 100.00% 100.00% 0.00% 0.00%
No Crack 100.00% 90.00% 0.00% 10.00%
TABLE I. SAMPLE SIZE OF EACH LABELED CONCRETE CRACKS
Kappa 0.97
Label Dataset count
Medium Crack 350 Table 3 shows the summary of all detection percentages for
Severe Crack 405 crack detection. True positive means detection occurs correctly
Very Severe Crack 350
with an object in a wall. True negative means no crack detected
Total 1105
in a wall that has no object in it. False-positive means no crack
detection occurs even a there is an object(crack) in the wall.
Table 1 shows the sample size used to train the dataset; the
False-negative means a detection occurs when no object(crack)
SDNET Database provides an image database for the different
is on the wall.
concrete crack images. Hence, the database contains 56,000
other images, with or without cracks. Labeled data used are The predicted fraction of values of a class that does not
1105 images, as shown in Table 3. All images were converted belong to the class represented in Table 3 resulted in errors of
to grayscale and resized to the exact dimension of 256-by-256 commission. They are a measure of false positives. False-
pixels. The training process consists of getting the positive rate (FPR) is calculated as the number of incorrect
characteristics of each image along with the labeled cracked; positive predictions divided by classification overall [29]. The
thus, the data is used to create and train. The resulting training rows of the confusion matrix show the error of commission

Authorized licensed use limited to: UNIVERSIDAD DE SANTIAGO DE CHILE. Downloaded on April 19,2023 at 22:01:34 UTC from IEEE Xplore. Restrictions apply.
except for the values along the diagonal. The predicted value compared from the crack comparator's actual measurement,
that belongs to a class but is predicted in a different class which matches the android application's reading. The android
represents the error or omission. They are a measure of false application testing to each crack classification set successfully
negatives. In computing the false-negative rate in which the gives a summary accuracy percentage of 93.3%.
columns of the confusion matrix represent, except for the
values along the main diagonal. Identifying both False Positive Exploration of the deep learning models can increase the
and False Negative Rates is important in reducing error [29]. accuracy rate and the kappa coefficient value to improve the
True Positive is calculated as the number of correct positive importance of this study in detecting crack classification. Thus,
predictions divided by the classification overall [29]. The True the android application may be run without an internet
Negative rate measures the proportion of correctly identified connection.
negatives [29]. The kappa coefficient measures the agreement ACKNOWLEDGMENT
between classification and truth values. Kappa is always less
The authors would like to thank Faculty Members under
than or equal to one (1). A kappa value of one (1) represents a
Computer Engineering Department for giving valuable ideas
perfect agreement, while a value of zero (0) represents no
and suggestions in the final implementation of the project.
agreement. According to Statistics HowTo 2018 [30], the
Kappa statistic varies from zero (0) to one (1), where: • 0 = No REFERENCES
agreement • 0.01 – 0.20 = Slight agreement. • 0.21 – 0.40 = [1] J. I. C. Agency, "PAVEMENT INSPECTION GUIDELINE," Ministry
Fair agreement. • 0.41 – 0.60 = Moderate agreement. • 0.61 – of Transport Republic of Tajikistan, Japan, 2016.
0.80 = Substantial agreement. • 0.81 – 0.99 = Almost perfect or [2] et al., H. M. (2013). Detection Method of Cracks by Using Magnetic
Near-perfect agreement • 1 = Perfect agreement. Fields in. 2013 10th International Conference on Ubiquitous Robots and
Ambient Intelligence (URAI), 734-737.
The equation below shows the computation for the kappa [3] L. Z. et al., "ROAD CRACK DETECTION USING DEEP
value: CONVOLUTIONAL NEURAL NETWORK," IEEE, pp. 3708-3712,
2016.
[4] F.-C. C. a. M. R. Jahanshahi, "NB-CNN: Deep Learning-Based Crack
(1) Detection," IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS,
pp. 4392-4400, 2018.
Where: [5] e. a. Gang Yao, "Crack Detection of Concrete Surface based on
x i is the class number Convolutional Neural Networks," 2018 International Conference on
Machine Learning and Cybernetics, Chengdu, China, pp. 246-250,
x N is the total number of classified values compared to 2018.
truth values [6] A. J. Balaji and et al., "A Machine Learning-Based Approach to Crack,"
IEEE, 2018.
x mi, i is the number of values belonging to the truth [7] F.-C. Chen and M. R. Jahanshahi, "NB-CNN: Deep Learning-Based
class i that have also been classified as class I (i.e., Crack Detection," IEEE TRANSACTIONS ON INDUSTRIAL
values found along the diagonal of the confusion ELECTRONICS, pp. 4392-4400, 2018.
matrix) [8] Y. Fei and et al., "Pixel-Level Cracking Detection on 3D Asphalt
Pavement Images Through Deep-Learning Based CrackNet-V," IEEE
x Ci is the total number of predicted values belonging to TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS,
class i pp. 273-284, 2020.
[9] V. Mandal, L. Uong, and Y. Adu-Gyamfi, "Automated Road Crack
x Gi is the total number of truth values belonging to a Detection Using Deep Convolutional Neural Networks," 2018 IEEE
class International Conference on Big Data (Big Data), pp. 5212-5215, 2018.
[10] J. Cheng and et al., "Pixel-level Crack Detection using U-Net,"
The first testing conducted a series of 20 medium crack TENCON 2018 - 2018 IEEE Region 10 Conference, pp. 0462-0466,
images using an android application with 95% accuracy. The 2018.
second testing showed 20 severe crack images with 90% [11] H. Chen, S. Wang and J. Xing, "Detection of Cracks in
accuracy. The Third test conducted a 20 series of Very Severe Electroluminescence Images by Fusing Deep," IEEE, pp. 2565-2569,
2019.
crack images with 100% accuracy. The overall accuracy
summary of the android application is 93.33%, while the kappa [12] W. Cao and et.al, "Deep Discriminant Learning-based Asphalt Road
Cracks Detection via Wireless Camera Network," IEEE, pp. 53-58,
value is .97, as shown in Table 3. Therefore, the mobile 2019.
application produced an accurate calculation in detecting the [13] R. Jafari, A. Gegov, S. Razvarz, and B. Vatchova, "Deep Learning for
crack and determining its crack classification. Pipeline Damage Detection: an Overview of the Concepts and a Survey
of the State-of-the-Art," 2020 IEEE 10th International Conference on
Intelligent Systems, pp. 178-182, 2020.
IV. CONCLUSIONS AND RECOMMENDATIONS
[14] J. Yang and et al., "Infrared Thermal Imaging-Based Crack Detection
In this paper, out of the 1105 datasets, the cracks were Using Deep Learning," IEEE, pp. 182060-182077, 2019.
categorized as a medium, severe, and severe and were tested [15] Z. Qu, J. Mei, L. Liu and D.-Y. Zhou, "Crack Detection of Concrete
with different crack classifications. Yolov3 deep learning Pavement With Cross-Entropy Loss Function and Improved VGG16
algorithm was used in detecting and classifying cracks in Network Model," IEEE, pp. 54564-54573, 2020.
reinforcement concrete structures. The model was deployed in [16] A. Kathuria, "What's new in YOLO v3?," 23 April 2018. [Online].
Available: https://towardsdatascience.com/yolo-v3-object-detection-
the server and can be accessed by the android application. 53fb7d3bfe6b.
Several tests were conducted using the android application and

Authorized licensed use limited to: UNIVERSIDAD DE SANTIAGO DE CHILE. Downloaded on April 19,2023 at 22:01:34 UTC from IEEE Xplore. Restrictions apply.
[17] M. Nie and et.al, "Pavement Crack Detection based on YOLO v3," 2019 convolutional-neural-networks-the-eli5-way-3bd2b1164a53. [Accessed
2nd International Conference on Safety Produce Informatization 3 March 2022].
(IICSPI), pp. 327-330, 2019. [25] J. Brownlee, "How Do Convolutional Layers Work in Deep Learning
[18] G. XingQi, L. Quan, Z. MeiLing and J. HuiFeng, "Analysis and Test of Neural Networks?," Machine Learning Mastery Making Developers
Concrete Surface Crack of Railway Bridge Based On Deep Learning," Awesome at Machine Learning, 17 April 2019. [Online]. Available:
2020 IEEE 5th Information Technology and Mechatronics Engineering https://machinelearningmastery.com/convolutional-layers-for-deep-
Conference, pp. 437-442, 2020. learning-neural-
[19] A survey on problems encountered in current concrete construction and networks/#:~:text=The%20innovation%20of%20convolutional%20neur
the potential benefits of self-healing cementitious materials, 2018. al%20networks%20is%20the,that%20can%20be%20detected%20anywh
ere%20on%20input%20images.. [Accessed 9 March 2022].
[20] P. Kumar, S. Batchu, N. Swamy and S. R. Kota, "Real-Time Concrete
Damage Detection Using Deep Learning for High Rise Structures," [26] A. Kathuria, "What's new in YOLO v3?," towards data science, 23 April
Open Access Journal, p. 20, 2021. 2018. [Online]. Available: https://towardsdatascience.com/yolo-v3-
object-detection-53fb7d3bfe6b. [Accessed 9 March 2022].
[21] A. C. 224, "224R-01 Control of Cracking in Concrete Structures".
[27] S. Dorafshan, R. J. Thomas and M. Maguire, "SDNET2018: An
[22] P. S. Sinha, "Convolution Neural Network," inblog, 7 November 2020.
Annotated Image Dataset for Non-Contact Concrete Crack Detection
[Online]. Available: https://blog.ineuron.ai/Convolution-Neural-
Using Deep Convolutional Neural Networks," Data in Brief, vol. 21, p.
Network- 4, 2018.
AIKFjDUgt4#:~:text=What%20is%20Convolutional%20Neural%20Net
work%20A%20Convolutional%20Neural,be%20able%20to%20differen [28] D. T. N. &. J. G. Chicco, "The Matthews correlation coefficient (MCC)
tiate%20one%20from%20the%20other.. [Accessed 9 March 2022]. is more reliable than balanced accuracy, bookmaker informedness, and
markedness in two-class confusion matrix evaluation.," BioData Mining,
[23] "ProjectPro," 10 September 2021. [Online]. Available: pp. 14, 13, 2021.
https://www.projectpro.io/recipes/what-are-
convnets#:~:text=The%20pre- [29] M. Ribalta, C. Mateu, R. Bejar, E. Rubion, L. Echeverria, F. J. V.
processing%20required%20in%20the%20ConvNet%20is%20much,is% Alegre, and L. Corominas, "Sediment Level Prediction of a Combined
20a%20stack%20of%20layer_conv_2d%20and%20layer_max_pooling_ Sewer System Using," MDPI, pp. 1-15, 2021.
2d%20layers.. [Accessed 9 March 2022]. [30] Stephanie Glen. "Total Population Sampling"
[24] S. Saha, "towards data science," 16 December 2018. [Online]. From StatisticsHowTo.com: Elementary Statistics for the rest of
Available: https://towardsdatascience.com/a-comprehensive-guide-to- us! https://www.statisticshowto.com/total-population-sampling/

Authorized licensed use limited to: UNIVERSIDAD DE SANTIAGO DE CHILE. Downloaded on April 19,2023 at 22:01:34 UTC from IEEE Xplore. Restrictions apply.

You might also like