Mahadi Final Presentation

Road Damage Detection and Classification
using Deep Neural Network
Supervised By
Presented By
Dr. Kaushik Deb
Md. Mahadi hasan
Professor
1604020
Dept. of CSE, CUET
Course Code: CSE-400

Course Title: Project and
Thesis 1
Contents
 Goal
 Motivation
 Application
 Literature Survey
 Challenges
 Research objective
 Proposed methodology
 Result and discussion
 Contribution
 Response to research objective
 Conclusion
 Limitation & Future Work
 References
2 Department of CSE, CUET
Goal
 Detection and Classification of road damages using Deep Neural
Network
(a) (b)
Figure1: Detection and classification of road damage. (a) Test image from RDD-2020 dataset,
(b) Test image from CUET Road.

Motivation
Opportunity to reduce road accident

• To reduce road accident by monitoring road
surface
Great Research Opportunity

• To generate bounding boxes in damage region
• To classify these boxes
Overcoming limitation of previous work

• To detect overlapping multiple damage region in
same image [3]
• To improve accuracy of previous model [3]
[3] D. Arya et al., "Transfer learning-based road damage detection for multiple
countries, "arXiv preprint arXiv:2008.13101, 2020

Applications
Road Surface Monitoring
Automated Driving Car
Smart Application
Mapping Damage road
https://www.tno.nl/media/2360/onderhoud_weg_240.jpg?rnd=1
https://gcn.com/-/media/GIG/GCN/Redesign/Articles/2018/May/automatedvehicle.png
https://onlinelibrary.wiley.com/doi/abs/10.1111/mice.12387 5 Department of CSE, CUET
Literature Survey
No Methodology Outcome Limitations
[1]  They used MobileNet-v2  Bounding boxes on damage  Failed to detect
as a backbone transfer reign multiple overlapping
learning model damage region in
 They used SSD as  Classification of these same image
detection head damage region
[2]  They used MobileNet-v2  Bounding boxes on damage  Straggles to deal

and Inception-v2 as a reign with close and small
backbone transfer learning damage region
model  Classification of these  Failed to detect
 They used SSD as damage region multiple damage
detection head region in same image
[1] D. Arya et al., ‘Transfer learning-based road damage detection for multiple countries,’ CoRR, vol. abs/2008.13101, 2020. arXiv:
2008.13101. [Online].Available: https://arxiv.org/abs/2008.13101
[2] J. Singh and S. Shekhar, ‘Road damage detection and classification in smartphone captured images using faster R-CNN,’
CoRR, vol. abs/1811.04535,2018. arXiv: 1811.04535. [Online]. Available: http://arxiv.org/abs/1811.04535. .

Literature Survey
[3]  They used VGG-16 as a  Bounding boxes on damage  Failed to detect
backbone transfer learning reign multiple damage
model region in same
 They used Faster RCNN  Classification of these image
as detection head damage region
[5]  They used Resnet-50 as a  Bounding boxes on damage  Mask accuracy is

backbone transfer learning region not high. In some
model  Generate mask in damage images mask is not
 They used Mask RCNN region accurate in damage
algorithm as detection region
head
[3] H. Maeda, Y. Sekimoto, T. Seto, T. Kashiyama and H. Omata, ‘Road damage detection using deep neural networks with
images captured through a smartphone,’ CoRR, vol. abs/1801.09454, 2018. arXiv: 1801.09454
[5] Q. Zhang, X. Chang and S. B. Bian, ‘Road-damage-detection segmentation algorithm based on improved mask rcnn,’
IEEE Access, vol. 8, pp. 6997–7004, 2020. doi:10.1109/ACCESS.2020.2964055.

Literature Survey
[6]  They used a customized CNN  Bounding boxes on  Does not gives its best
for feature extraction and trained damage region performance in low
their model with three different  Generate mask in light and low
learning rate and three different damage region resolution images
optimizer
 They used Mask R-CNN
algorithm as detection head
[7]  They used a modified version of  Bounding boxes on  They used only 573
DenseNet-121 named Proposed damage region images for training
Net as a backbone transfer  Generate mask in and 120 images for
learning model damage region testing
 They used Mask R-CNN
algorithm as detection head
[6] P. Kumar, A. Sharma and S. R. Kota, ‘Automatic multiclass instance segmentation of road damage using deep learning
model,’ IEEE Access,vol. 9, pp. 90 330–90 345, 2021. doi: 10.1109/ACCESS.2021.3090961.
[7] S. Shim and G.-C. Cho, ‘Lightweight semantic segmentation for road-surface damage detection based on multiscale
learning,’ IEEE Access, vol. 8, pp. 102 680–102 690, 2020. doi: 10.1109/ACCESS.2020.2998427.

Challenges
To detect damage region in

varying weather and
lighting condition
Correctly labeling the data
To detect overlapping multiple

damage region in same image
Image Source:
http://www.roadsafetyuae.com/assets/uploads/2014/12/Topic-Pic-Bad-Weather.jpg
https://paperswithcode.com/media/thumbnails/task/task-0000000480-281654b9.jpg

Research Objective
 Failed to detect multiple overlapping different damage region in
same image [1]
 To achieve better accuracy[1]
[1] D. Arya et al., ‘Transfer learning-based road damage detection for

multiple countries,’ CoRR, vol. abs/2008.13101, 2020. arXiv:
2008.13101. [Online].Available: https://arxiv.org/abs/2008.13101

Proposed Methodology
Figure 2: Proposed methodology to detect and classify road damages.

Proposed Methodology
Figure 3: Faster RCNN architecture.
Figure 4: SSD architecture.

Results & Discussion
 Dataset Description
We have used two datasets to evaluate our proposed method.
1. Road Damage Detection (RDD-2020) dataset
• Train with 9,000 images with 18,462 labels
• Validation with 6,603 labels
• Test with 1000 labels
27500
22500
17500
12500
7500
2500
D00 D10 D20 D40 Total
India 1555 68 2021 3187 6831
Czech 988 399 161 197 1745
Japan 4049 3979 6199 2243 16470
Total 6592 4446 8381 5627 25046
India Czech Japan Total
Figure 5: RDD-2020 sample images. Figure 6: Dataset description of RDD-2020 dataset.

 Dataset Description
2. CUET road dataset
• Test with 829 labels
850
750
650
550
450
350
250
150
50
D00 D10 D20 D40 Total
No of labels 123 92 278 336 829
Figure 7: CUET road sample images. Figure 8: Dataset description of CUET road dataset.

 We have used 6 different models to train and evaluate with RDD-2020
and CUET road images.
Table 1: List of six models trained with RDD-2020 dataset.
Model no. Backbone Network Detection Head

1 ResNet-50 Faster RCNN
2 ResNet-101 Faster RCNN
3 MobileNet-v1 SSD
4 MobileNet-v2 SSD
5 ResNet-50 SSD
6 EfficientNet EfficientDet

Model Hyperparameters
Hyperparameters:
 Batch Size: 4
 Optimizer: SGD with Momentum
 Loss Function: Sparse Categorical Cross Entropy
 Learning Rate: 0.01
 IoU threshold: 0.6
 Steps: 25000

Performance Evaluation
 Confusion matrix and performance score of Faster RCNN with
ResNet-50 using RDD-2020 test images
(a) ( b)
Figure 9: Evaluation of Faster RCNN with ResNet-50 using RDD-2020 dataset.(a) Confusion Matrix,
(b) performance score.

Performance Evaluation
 Confusion matrix and performance score of Faster RCNN with
ResNet-50 using CUET road images
(a) (b)
Figure 10: Evaluation of Faster RCNN with ResNet-50 using CUET road images.(a) Confusion Matrix, (b)
performance score.

 Detection and classification result of RDD-2020 dataset
(a) (b) (c)
Figure 12: Test images result using Faster RCNN with ResNet-50. (a) Image from Japan,
(b) Image from India, (c) Image from Czech.

 Detection and classification result of CUET road dataset
(a) (b) (c)
Figure 11: CUET road images test result using Faster RCNN with ResNet-50.
(a) & (b) Classify damage region correctly, (c) Incorrectly classified damage
region.

 Detection and classification of six models in
same image
a) Faster RCNN with ResNet-50
b) Faster RCNN with ResNet-101
c) SSD with MobileNet-v1
d) SSD with MobileNet-v2
e) SSD with ResNet-50
f) EfficientDet
(a) (b)
(d) (e) (f)

(c)
Figure 13: Detection and classification result of six models in same image.

 Performance score on RDD-2020 dataset test images of six models
0.85
0.65
0.45
0.25
0.05
Precission for Recall for D00 F1 score for Precission for Recall for D10 F1 score for Precission for Recall for D20 F1 score for Precission for Recall for D40 F1 score for
D00 D00 D10 D10 D20 D20 D40 D40
Faster 0.47 0.48 0.48 0.52 0.66 0.58 0.43 0.71 0.54 0.41 0.18 0.24
RCNN
with
Resnet-50
Faster 0.58 0.39 0.47 0.42 0.39 0.41 0.29 0.72 0.41 0.57 0.25 0.35
RCNN
with
Resnet-
101
SSD with 0.25 0.19 0.22 0.59 0.17 0.26 0.25 0.64 0.36 0.1 0.06 0.07
Mobilenet
V1
SSD with 0.44 0.44 0.44 0.04 0.01 0.01 0.32 0.74 0.45 0.15 0.07 0.1
Mobilenet Faster RCNN with Resnet-50 Faster RCNN with Resnet-101 SSD with Mobilenet V1 SSD with Mobilenet V2 SSD with Resnet-50 EfficientDet
V2
SSD with 0.23 0.12 0.15 0 0 0 0.22 0.79 0.35 0.12 0.04 0.06
Resnet-50
Efficient- 0.63 0.09 0.15 0.26 0.18 0.21 0.2 0.86 0.33 0.5 0.03 0.06
Det
Figure 14: Bar chart representation of performance score on RDD-2020 Dataset’s images of six
models.
 Optimizers:
Table-2: F1 score of Faster RCNN with ResNet-50 for 4 classes on three different Optimizers
Model F1 Score mAP Numeber of GFLops
params (M)
SGD with 0.48 0.58 0.24 0.54

Momentum
RMS Prop 0.22 0.26 0.07 0.37
Adam 0.47 0.42 0.33 0.41
Figure 15: Bar chart representation of F1 score on three different optimizer

 Performance score on RDD-2020 dataset test images of six models in
different batch size
Faster RCNN with ResNet-50
0.75
0.65
0.55
0.45
0.35
0.25
0.15
0.05
Precission for Recall for D00 F1 score for Precission for Recall for D10 F1 score for Precission for Recall for D20 F1 score for Precission for Recall F1 score for
D00 D00 D10 D10 D20 D20 D40 D40
Batch Size 4 0.47 0.48 0.48 0.52 0.66 0.58 0.43 0.71 0.54 0.41 0.18 0.24
Batch Size 8 0.47 0.47 0.47 0.51 0.65 0.57 0.42 0.68 0.52 0.41 0.17 0.24
Batch Siize 16 0.46 0.45 0.45 0.51 0.63 0.56 0.42 0.67 0.52 0.4 0.16 0.23
Batch Size 32 0.45 0.43 0.44 0.51 0.6 0.55 0.41 0.66 0.51 0.38 0.17 0.23
Batch Size 4 Batch Size 8 Batch Siize 16 Batch Size 32
Figure 16: Bar chart representation of performance score on RDD-2020 Dataset of six models in
different batch size.

Table 3: Comparison of f1 score with existing literature
using RDD-2020 dataset
Model name D00 D10 D20 D40

D. Arya et al. [1] 0.37 0.22 0.52 0.17
Faster RCNN with 0.48 0.58 0.54 0.24
ResNet-50
Faster RCNN with 0.47 0.41 0.41 0.35
ResNet-101
SSD with MobileNet-v1 0.22 0.26 0.36 0.07
SSD with MobileNet-v2 0.44 0.01 0.45 0.1
SSD with ResNet-50 0.15 0.0 0.35 0.06
EfficientDet 0.15 0.21 0.33 0.06
[1] D. Arya et al., ‘Transfer learning-based road damage detection for multiple
countries,’ CoRR, vol. abs/2008.13101, 2020. arXiv: 2008.13101. [Online].
Available: https://arxiv.org/abs/2008.13101

 Performance score on CUET road images of six models
1.1
0.9
0.7
0.5
0.3
0.1
Precission for Recall for D00 F1 score for Precission for Recall for D10 F1 score for Precission for Recall for D20 F1 score for Precission for Recall for D40 F1 score for
D00 D00 D10 D10 D20 D20 D40 D40
Faster 0.38 0.26 0.31 0.25 0.05 0.09 0.35 0.44 0.39 0.54 0.61 0.57
RCNN with
Resnet-50
Faster 0.39 0.1 0.16 0.48 0.11 0.18 0.38 0.85 0.52 0.54 0.24 0.33
RCNN with
Resnet-101
SSD with 0.16 0.14 0.15 0.27 0.04 0.07 0.37 0.85 0.52 0.45 0.09 0.14
Mobilenet
V1
SSD with 0.36 0.11 0.16 0.64 0.2 0.3 0.37 0.87 0.52 0.54 0.19 0.28
Mobilenet
V2
SSD with 0.42 0.04 0.07 0 0 0 0.34 0.99 0.51 0.06 0 0.01
Resnet-50
Faster RCNN with Resnet-50 Faster RCNN with Resnet-101 SSD with Mobilenet V1 SSD with Mobilenet V2 SSD with Resnet-50 EfficientDet
EfficientDet 0.29 0.06 0.1 0.14 0.01 0.02 0.33 0.79 0.47 0.5 0.2 0.29
Figure 17: Performance score on CUET road images of six models

Contribution
Overcome limitation of a literature using
RDD-2020 dataset
• Detect multiple overlapping damage region in same
image
• Better F1 score in all four classes than the literature [1]
Build a Dataset
• Make a small dataset of CUET Road
• Evaluate six different models with CUET road
images

Response to Research Objective
 We detect multiple overlapping damage region using Faster RCNN
with ResNet-50 and Faster RCNN with ResNet-101
 Achieve better f1 score than previous model [1]
Table 3: Comparison of f1 score with existing literature
Model name D00 D10 D20 D40
D. Arya et al. [1] 0.37 0.22 0.52 0.17
Faster RCNN with ResNet-50 0.48 0.58 0.54 0.24
Faster RCNN with ResNet-101 0.47 0.41 0.41 0.35
(a) (b) (c)
Figure 18: Detection and classification result (a) Result from paper [1], (b) Result using Faster
RCNN with ResNet-50, (c) Result using Faster RCNN with ResNet-101

Conclusion
 Our models can detect and classify overlapping of multiple damage
region in same image using Faster RCNN with ResNet-50 and Faster
RCNN with ResNet-101.
 We evaluate our models with CUET road images.
 We have found a maximum f1 score 0.48 for longitudinal class, 0.58
for transverse class and 0.54 for alligator crack using faster RCNN
with ResNet-50 and 0.35 for pothole using Faster RCNN with
ResNet-101.

Limitation & Future Work
 Limitation:
Low performance
• For class D40, low f1 score
Misclassification
• Performance in unseen data like CUET
road image is not good enough
 Future Work:
Train our models with large
Bangladeshi dataset
Applied to Bangladesh transportation

system
.
Improve accuracy

References
[1] D. Arya et al., ‘Transfer learning-based road damage detection for multiple countries,’ CoRR,
vol. abs/2008.13101, 2020. arXiv: 2008.13101. [Online].
Available:https://arxiv.org/abs/2008.13101.
[2] J. Singh and S. Shekhar, ‘Road damage detection and classification in smartphone captured
images using faster R-CNN,’ CoRR, vol. abs/1811.04535,2018. arXiv: 1811.04535. [Online].
Available: http://arxiv.org/abs/1811.04535.
[3] H. Maeda, Y. Sekimoto, T. Seto, T. Kashiyama and H. Omata, ‘Road damage detection using
deep neural networks with images captured through a smartphone,’ CoRR, vol. abs/1801.09454,
2018. arXiv: 1801.09454.
[4] M. S. Arman, M. M. Hasan, F. Sadia, A. K. Shakir, K. Sarker and F. A.Himu, ‘Detection and
classification of road damage using r-cnn and faster r-cnn: A deep learning approach,’ in Cyber
Security and Computer Science, T.Bhuiyan, M. M. Rahman and M. A. Ali, Eds., Cham: Springer
International Publishing, 2020, pp. 730–741, isbn: 978-3-030-52856-0.
[5] Q. Zhang, X. Chang and S. B. Bian, ‘Road-damage-detection segmentation algorithm based
on improved mask rcnn,’ IEEE Access, vol. 8, pp. 6997–7004, 2020.
doi:10.1109/ACCESS.2020.2964055.
[6] P. Kumar, A. Sharma and S. R. Kota, ‘Automatic multiclass instance segmentation of road
damage using deep learning model,’ IEEE Access,vol. 9, pp. 90 330–90 345, 2021. doi:
10.1109/ACCESS.2021.3090961.

References
[7] S. Shim and G.-C. Cho, ‘Lightweight semantic segmentation for road-surface damage
detection based on multiscale learning,’ IEEE Access, vol. 8, pp. 102 680–102 690, 2020. doi:
10.1109/ACCESS.2020.2998427.
[8] S. Karimzadeh, M. Ghasemi, M. Matsuoka, K. Yagi and A. C. Zulfikar, ‘A deep learning
model for road damage detection after an earthquake based on synthetic aperture radar (sar) and
field datasets,’ IEEE Journal of Selected Topics in Applied Earth Observations and Remote
Sensing, vol. 15,pp. 5753–5765, 2022.
[9] K. Zhao, J. Liu, Q. Wang, X. Wu and J. Tu, ‘Road damage detection from post-disaster high-
resolution remote sensing images based on tld framework,’ IEEE Access, vol. 10, pp. 43 552–43
561, 2022. doi: 10.1109/ACCESS.2022.3169031.
[10] R. Girshick, J. Donahue, T. Darrell and J. Malik, ‘Rich feature hierarchies for accurate
object detection and semantic segmentation,’ in Proceedings of the IEEE conference on computer
vision and pattern recognition, 2014, pp. 580–587.
[11] R. Girshick, ‘Fast r-cnn,’ in Proceedings of the IEEE international conference on computer
vision, 2015, pp. 1440–1448.
[12] S. Ren, K. He, R. Girshick and J. Sun, ‘Faster r-cnn: Towards real-time object detection with
region proposal networks,’ Advances in neural information processing systems, vol. 28, 2015.
[13] K. He, G. Gkioxari, P. Dollár and R. Girshick, ‘Mask r-cnn,’ in Proceedings of the IEEE
international conference on computer vision, 2017, pp. 2961–2969.

References
[14] D. Arya et al., ‘Global road damage detection: State-of-the-art solutions,’ in 2020 IEEE
International Conference on Big Data (Big Data), IEEE, 2020, pp. 5533–5539.
[15]T. Akiba, S. Suzuki and K. Fukuda, ‘Extremely large minibatch SGD: training resnet-50 on
imagenet in 15 minutes,’ CoRR, vol. abs/1711.04325, 2017.arXiv: 1711.04325. [Online].
Available: http://arxiv.org/abs/171104325.

THANK YOU

Appendix – Backbone Network
Figure 22: Architecture of ResNet-50

Appendix – Backbone Network
Figure 23: Identity block
Figure 24:Convolutional block

Appendix - Dataset Description
Figure 19: RDD-2020 dataset description

Appendix -Implementation
Figure 20: Conversion from xml to csv file Figure 21: Conversion from csv to TfRecord file

Appendix -Implementation
Figure 22: xml file Figure 23: csv file

Models Performance score using RDD-2020 Dataset
Table 4: Performance score of six models using RDD-2020 dataset

Appendix – Experimental Results
 Performance score using RDD-2020 Dataset on different batch size
Model Name Batch Size Class precision Recall F1 score
D00 0.47 0.48 0.48

D10 0.52 0.66 0.58
4 D20 0.43 0.71 0.54
D40 0.41 0.18 0.24
D00 0.47 0.47 0.47

D10 0.51 0.65 0.57
8 D20 0.42 0.68 0.52
D40 0.41 0.17 0.24
Faster RCNN with ResNet-50
D00 0.46 0.45 0.45
D10 0.51 0.63 0.56
16 D20 0.42 0.67 0.52
D40 0.4 0.16 0.23
D00 0.45 0.43 0.44

D10 0.51 0.6 0.55
32 D20 0.41 0.66 0.51
D40 0.38 0.17 0.23
Table 5: Performance score of six models on different batch size using RDD-2020 dataset

Models Performance score using CUET road images
Table 6: Performance score of six models using CUET road images

Appendix –Performance on Faster RCNN with ResNet-101
 Evaluate with RDD-2020 Road

images-
 Batch Size: 4
 Optimizer: SGD with Momentum
 Loss Function: Sparse
Categorical Cross Entropy
 Learning Rate: 0.01
 IoU_threshold: 0.6
 Epochs: 25000
 Momentum_optimizer_value: 0.9
Figure 25: Evaluation of faster RCNN with ResNet-101 in RDD-2020 test images. (a) Confusion
Matrix (b)ROC curve (c) performance score

Appendix –Performance on SSD with MobileNet v1
images-
 Batch Size: 4
 Optimizer: SGD with
Momentum
 Epochs: 25000
 Momentum_optimizer_value:
0.9
Figure 26: Evaluation of SSD with MobileNet-v1 in RDD-2020 test images. (a) Confusion

images-
 Batch Size: 4
 Epochs: 25000
Figure 27: Evaluation of SSD with MobileNet-v2 in RDD-2020 test images. (a) Confusion

Appendix –Performance on SSD with ResNet-50
images-
 Batch Size: 4
 Epochs: 25000
Figure 28: Evaluation of SSD with ResNet-50 in RDD-2020 test images. (a) Confusion Matrix
(b)ROC curve (c) performance score

Appendix –Performance on EfficientDet
images-
 Batch Size: 4
 Optimizer: SGD with
Momentum
 Epochs: 25000
 Momentum_optimizer_value:
0.9
Figure 29: Evaluation of EfficientDet in RDD-2020 test images. (a) Confusion Matrix (b)
ROC curve (c) performance score

Appendix –Performance on Faster RCNN with ResNet-101
 Evaluate with CUET Road images-
 Batch Size: 4
 Loss Function: Sparse Categorical Cross Entropy
 Epochs: 25000
Figure 30: Evaluation of Faster RCNN with ResNet-101 in CUET test images. (a) Confusion
Matrix (b) performance score
Appendix –Performance on SSD with MobileNet-v1
 Batch Size: 4
 Epochs: 25000
Figure 31: Evaluation of SSD with MobileNet v1 in CUET test images. (a) Confusion Matrix (b)
performance score
 Batch Size: 4
 Epochs: 25000
Figure 32: Evaluation of SSD with MobileNet v2 in CUET test images. (a) Confusion Matrix (b)
performance score
Appendix –Performance on SSD with ResNet-50
 Batch Size: 4
 Epochs: 25000
Figure 33: Evaluation of SSD with ResNet-50 in CUET test images. (a) Confusion Matrix (b)
performance score
Appendix –Performance on EfficientDet
 Batch Size: 4
 Epochs: 25000
Figure 34: Evaluation of EfficientDet in CUET test images. (a) Confusion Matrix (b)
performance score
Appendix
Why F1 score is used ?
Precision:
 Precision tells us how many of the correctly predicted cases actually turned out to
be positive.
 This would determine whether our model is reliable or not
Recall:
 Recall tells us how many of the actual
positive cases we were able to predict
correctly with our model.
 also known as Sensitivity or true positive rate

Appendix
 Precision is a useful metric in cases

where False Positive is a higher
concern than False Negatives.
 music or video recommendation systems
 e-commerce websites
 Spam detection
 Recall is a useful metric in cases where

False Negative is a higher concern than
False Positive.
 In medical cases where it doesn’t matter

whether we raise a false alarm but the
actual positive cases should not go
undetected

Appendix
F1-Score:
 In practice, when we try to increase the precision of our model, the recall goes
down, and vice-versa. The F1-score captures both in a single value.
 F1-score is a harmonic mean of Precision and Recall, and so it gives a combined

idea about these two metrics.
 It is maximum when Precision is equal to Recall.

Appendix
Guideline to collect image dataset?
1. Collect images of the object from different angles and perspective.
2. Gather images of the object in variable lighting condition
3. Gather object with different object size and distances from object
4. Ensure images are clearly visible
5. Images should be collected in variable view points and variable scales
6. Collect high quality images
7. Avoid images with excessive size
8. Label Every Object of Interest in Every Image
9. Label the Entirety of an Object
10. Label Occluded Objects
11. Create Tight Bounding Boxes
12. Create Specific Label Names
13. Maintain Clear Labeling Instructions
14. Use These Labeling Tools
• CVAT RectLabel (for Mac)
• Roboflow LabelImg

Gradient Descent (GD)
Gradient Descent (GD) characteristics:
 Take all the data point to calculate the loss function.

 Per epoch only one iteration happen.
 When we have huge record such as (10 million) we require a large RAM.
 Need large amount of recourses.
 Computationally expensive.
 Need less time to converge global minima.
For example we have 10K records. We train our network in 20 epoch.
n=10K records

Stochastic Gradient Descent (GD)
2.Stochastic Gradient decent:

 Instant of taking all records they take only one record per epoch.
 So here 10K iteration per epoch.
 If we have a huge amount of record, we need not require a large size of RAM
 No need large amount of recourses.
 Computationally inexpensive.
 Need very much time to converge global minima.

Mini Batch Stochastic Gradient decent
3. Mini Batch Stochastic Gradient decent:
 Instant of taking all records they take some record

in a branch per epoch.
 So if we have 10K records. And take branch size

=100, then 10,000/100=10 iteration per epoch.
 If we have a huge amount of record, we need not

require a large size of RAM as GD.
 No need large amount of recourses.
 Computationally less expensive than Gradient

decent but more than Stochastic Gradient decent.
 Need an average time to converge global minima.

Loss Curve
Figure 34: Loss curve in Faster RCNN

Loss Curve
Figure : Total Loss

MobileNet
Figure :Depthwise Separable Convolution layer in MobileNet

EfficientDet
Figure :Width , depth, resolution scaling in EfficientDet

Damage Class
D00(Longitudinal Crack) D10(Transverse Crack)
D20(Aligator Crack) D40(pothole Crack)

Evaluate with 1000 vs 6603 labels for SSD with MobileNet-v2

Mahadi Final Presentation

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Mahadi Final Presentation

Uploaded by

Copyright:

Available Formats

Road Damage Detection and Classification

using Deep Neural Network

Course Code: CSE-400

3 Department of CSE, CUET

Opportunity to reduce road accident

Great Research Opportunity

Overcoming limitation of previous work

4 Department of CSE, CUET

Road Surface Monitoring

Automated Driving Car

Mapping Damage road

[2]  They used MobileNet-v2  Bounding boxes on damage  Straggles to deal

6 Department of CSE, CUET

[5]  They used Resnet-50 as a  Bounding boxes on damage  Mask accuracy is

7 Department of CSE, CUET

8 Department of CSE, CUET

To detect damage region in

Correctly labeling the data

To detect overlapping multiple

9 Department of CSE, CUET

[1] D. Arya et al., ‘Transfer learning-based road damage detection for

10 Department of CSE, CUET

Figure 2: Proposed methodology to detect and classify road damages.

11 Department of CSE, CUET

Figure 3: Faster RCNN architecture.

Figure 4: SSD architecture.

12 Department of CSE, CUET

India Czech Japan Total

Figure 5: RDD-2020 sample images. Figure 6: Dataset description of RDD-2020 dataset.

13 Department of CSE, CUET

14 Department of CSE, CUET

Table 1: List of six models trained with RDD-2020 dataset.

Model no. Backbone Network Detection Head

15 Department of CSE, CUET

16 Department of CSE, CUET

17 Department of CSE, CUET

18 Department of CSE, CUET

 Detection and classification result of RDD-2020 dataset

(a) (b) (c)

19 Department of CSE, CUET

 Detection and classification result of CUET road dataset

(a) (b) (c)

20 Department of CSE, CUET

(d) (e) (f)

21 Department of CSE, CUET

 Performance score on RDD-2020 dataset test images of six models

SGD with 0.48 0.58 0.24 0.54

RMS Prop 0.22 0.26 0.07 0.37

Adam 0.47 0.42 0.33 0.41

Figure 15: Bar chart representation of F1 score on three different optimizer

23 Department of CSE, CUET

Batch Size 4 Batch Size 8 Batch Siize 16 Batch Size 32

24 Department of CSE, CUET

Model name D00 D10 D20 D40

25 Department of CSE, CUET

Figure 17: Performance score on CUET road images of six models

26 Department of CSE, CUET

27 Department of CSE, CUET

(a) (b) (c)

28 Department of CSE, CUET

29 Department of CSE, CUET

Applied to Bangladesh transportation