See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/360080345

Using computer vision to recognize construction material: A Trustworthy

Dataset Perspective

Article  in  Resources Conservation and Recycling · August 2022

DOI: 10.1016/j.resconrec.2022.106362

CITATIONS READS

0 56

2 authors:

Sun Ying Zhaolin Gu

Chonqging University of Posts and Telecommunication Xi'an Jiaotong University
2 PUBLICATIONS   2 CITATIONS    263 PUBLICATIONS   3,308 CITATIONS   

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

National Natural Science Foundation of China ( No. 51006077) View project

VCAD Project View project

All content following this page was uploaded by Sun Ying on 08 August 2022.

The user has requested enhancement of the downloaded file.

 Resources, Conservation & Recycling 183 (2022) 106362

Contents lists available at ScienceDirect

Resources, Conservation & Recycling

journal homepage: www.elsevier.com/locate/resconrec

Full length article

Using computer vision to recognize construction material: A Trustworthy

Dataset Perspective
Ying Sun a, b, Zhaolin Gu a, *
a
School of Human Settlement and Civil Engineering, Xi”an Jiaotong University, Xi”an 710049, China
b
College of Media and Arts, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

A R T I C L E I N F O A B S T R A C T

Keywords: Data quality is vital in machine learning models, and data scientists spend a substantial amount of time on data
Data Quality quality certificates before model training. With the objective of building trustworthy construction material
Construction Material Datasets datasets in this study, we first built construction material datasets, namely those for concrete, brick, metal, wood,
Cross Review
and stone data, by collecting images of different construction materials. In particular, these datasets include the
Deep Convolutional Neural Network
main construction material (i.e., concrete, brick, metal, wood, and stone), background, and uncertainty cate­
gories. Subsequently, we propose a novel cross-review framework that applies the entire corresponding datasets
as the input (e.g., brick datasets) and returns clean datasets. Subsequently, several state-of-the-art deep con­
volutional neural networks such as VGG16, GoogleNet, and ResNet were selected to verify the quality and
effectiveness of the construction material datasets under two pipelines: friendly mobile devices and unfriendly
architectures. The experimental results show that the proposed construction waste material datasets have
satisfactory quality and can be effectively recognized by these state-of-the-art deep neural network models.
Additionally, the results indicate the necessity to clean the data before training a deep-learning model. Further
research should be combining a novel deep construction material recognition model with the cross-review
framework to further improve the recognition performance.

1. Introduction
Recently, research has been focused on sustainability because the
government is concerned with sustainable development and the
increasing energy crisis. In particular, the aim of sustainable architec­
ture is to design or construct environment-friendly buildings by maxi­
mizing the utilization of resources and minimizing the damage to the
environment, including that caused by construction waste, pollution,
and emissions (Ma and Zhang, 2020; Ragheb et al., 2016; Seadon, 2010;
Su, 2020). Additionally, sustainable buildings also generate benefits for
the use of secondhand or recycled resources such as wasted brick, metal,
and wood, thereby easing the extra demand for construction resources to
some extent.
A large amount of construction and demolition waste (over 10 billion
tons) is known to be generated every year globally. As the largest
contributor, China generates more than two billion tons annually
(Zheng et al., 2017), which has a serious impact on the environment (e.
g., greenhouse gas emissions, resource consumption, and raw material
consumption) (Ding et al., 2016; Llatas et al., 0000; Ma and Zhang,
2020; Wang et al., 2018). Moreover, the report from the National
Development and Reform Commission of China indicates that recycled
construction and demolition waste accounts for only 5% of the total
waste, and the remainder is illegally dumped in landfills. To this end,
governments and industries have attempted to solve issues related to the
construction and demolition waste (Lu et al., 2021; Wu et al., 2016). For
example, some cities in China (e.g., Suzhou, Shenzhen, Qingdao) have
made efforts to develop recycling equipment for the construction and
demolition waste.
Construction waste recycling is essential in waste management. Su
(Su, 2020) studied the multi-agent evolutionary decision-making pro­
cess for the construction waste management. Su also pointed out the
recognition of construction waste material (e.g. classification) is
significantly important for recycling the construction waste, as shown in
Figure 1. In addition, Davis et al. (2021) also pointed out that waste
sorting using technologies that automatically identify different mate­
rials has the potential to assist in classifying construction demolition and
reduce costs. To this end, although the datasets used in this paper are
intact, it is still meaningful for construction waste recognition: (i) with

* Corresponding author.
E-mail address: guzhaoln@mail.xjtu.edu.cn (Z. Gu).

https://doi.org/10.1016/j.resconrec.2022.106362
Received 9 December 2021; Received in revised form 1 April 2022; Accepted 11 April 2022
Available online 20 April 2022
0921-3449/© 2022 Elsevier B.V. All rights reserved.
Y. Sun and Z. Gu
Fig. 1. Main Process of Construction Waste Recycling
the success of the deep learning technique, many promising deep
learning models have already been used and designed to recognize
construction waste. The famous dataset ImageNet is widely used to
pre-train different models and later transfer knowledge to other tasks
(Chu et al., 2016). In this case, the similar dataset can be collected for
construction waste recycling, where the deep learning models are first
pre-train on construction material dataste. Then the pre-trained models
can be fine-tuned when there is a real construction waste dataset
available, which can further improve the recognition performance and
this technology has been widely used in the different domains. So, it is
meaningful to collect construction material dataset for construction
waste recognition; (ii) labeling dataset is very expensive and
time-consuming. Same performance can be achieved by using less data
when using pre-training technologies. In summary, building a
high-quality construction material dataset is very meaningful for con­
struction waste recycling. In particular, the research community has
focused on computer vision-based methods owing to their potential to
overcome the limitations of manual sorting. To this end, several com­
puter vision methods have been proposed to recognize construction
waste materials by automatically capturing visual information and
on-site images or videos. From a traditional machine learning perspec­
tive, Zhu and Brilakis (2010) proposed an approach to identify concrete
material regions using machine learning techniques. In particular, the
concrete image is initially divided into several regions; subsequently, the
color/texture of each region is extracted, and these features are
considered as the input to train the machine learning classifier. In 2014,
Son et al. (2014) studied the performance of six different classifiers and
the potential of ensemble classification on three construction materials:
concrete, steel, and wood. Additionally, these models were employed
based on voting ensemble using the HSI color space feature. Subse­
quently, they evaluated the experiments based on the metrics of accu­
racy, precision, sensitivity, and average score values, wherein the
ensemble classifier achieved the best performance. Xiao et al. (2019)
applied the NIR hyperspectral technology to extract and classify typical
construction waste.
In contrast, with the development of artificial intelligence, deep-
learning-based algorithms have further enhanced the capability of
computer vision for processing and analyzing construction waste ma­
terials. Toğaçar et al. (2020) initially proposed an autoencoder network
to learn meaningful construction waste features, and then used a support
vector machine (SVM) to recognize construction waste. Recently, Davis
et al. (2021) investigated a deep convolutional neural network (CNN) to
recognize seven typical construction waste materials, including the
single and mixed disposal types. Lu et al. (2022) proposed an approach
2
Resources, Conservation & Recycling 183 (2022) 106362
for recognizing the composition of construction waste in its original
cluttered state using semantic segmentation, a technique that can
deliver fine-grained information such as the types and corresponding
pixel areas of waste materials in images. Chen et al. (2021) proposed a
hybrid framework to automate the estimation of the composition of
construction waste. Furthermore, Dong et al. (2022) proposed a novel
boundary-aware transformer model for recognizing the fine-grained
composition of construction waste mixtures.
The aforementioned research focuses on image classification and
semantic segmentation, which only identifies whether a given image
belongs to one of the several predetermined waste categories or simul­
taneously distinguishes and locates multiple waste materials in the same
image. However, only a few studies have focused on building high-
quality construction waste datasets to further improve the perfor­
mance of recognition instead of traditional model-centric methods. In
this study, we propose a data-driven, end-to-end solution to achieve
high-quality data for construction waste recycling. In particular, this
framework formulates the data cleaning procedure as a cross-review
process; and for each candidate construction material image, the
cross-review framework provides a score for each image, which in turn
enables the real/outlier selection of corresponding datasets. This
framework is flexible, and various expert models can be used. In
particular, we make the following contributions.
• To fill the knowledge gap, we formulate a cross-review framework to
achieve high-quality construction material datasets.
• The method inventively adapts a novel cross-review framework to
study a data-centric model for construction material recognition,
achieving a contactless approach that can be employed in different
applications with respect to construction waste recycling, e.g., object
detection, semantic segmentation, etc.
• This study investigates the possibility of a data-centric model for the
construction management domain and thus provides good practices
for other researchers involving similar tasks, e.g. solid waste man­
agement, etc.
2. Related Works
2.1. Construction waste recognition based on classification
Construction waste classification aims to classify a given construc­
tion waste image into predefined classes. Significant efforts have been
made to recognize construction waste such as plastics, metals, wood,
and concrete. Traditionally, construction waste material features need
Y. Sun and Z. Gu
to be manually designed and fed into machine learning models, such as
SVMs (Toğaçar et al., 2020; Wang et al., 2019) and ensemble models
(Son et al., 2014). However, handcrafting features for different con­
struction waste materials limits the applicability of these traditional
approaches.
In contrast, with the success of deep learning, CNN is a widely used
approach to support the construction waste recognition (Chen et al.,
2021; Davis et al., 2021; Toğaçar et al., 2020) owing to its capacity to
capture meaningful features. Davis et al. (2021) investigated a deep
CNN to recognize seven typical construction waste materials, including
the single and mixed disposal types. Chen et al. (2021) proposed a
hybrid approach by combing visual features captured from
DenseNet-169 and physical features, namely, the weight and waste
depth for unattended gauging in construction waste mixtures.
2.2. Construction waste recognition based on object detection / semantic
segmentation
Notably, the construction waste material classification only focuses
on whether an image contains a certain material category but fails to
provide information of finer granularity with respect to the location,
geometry, and boundaries. Recently, an increasing number of re­
searchers have shifted their focus from image classification to applica­
tions of object detection or semantic segmentation Ku et al. (2021);
Kujala et al. (2015); Wang et al. (2019). Ku et al. (2021) investigated a
grasp detection approach based on an R-CNN to process construction
and demolition waste. In contrast, Wang et al. (2019) used both a faster
R-CNN and a mask R-CNN to train a construction waste detection model,
enabling robots to recycle nails, screws, residual pipes, and cables on
construction sites. We observed that previous research mainly focused
on detecting separate construction waste objects under relatively
well-controlled conditions. While such research is helpful for waste
segregation in semi-structured environments such as recovery facilities,
it fails to work in scenarios where heterogeneous materials are randomly
mixed, as in truck-loaded construction waste.
To address this issue, Lu et al. (2022) proposed an approach based on
DeepLabv3+ to recognize composited material components from clut­
tered construction waste mixtures, which demonstrated the feasibility of
semantic segmentation for distinguishing highly unstructured materials
in mixed states. However, its precision is still not sufficiently high for
practical applications in construction waste management, primarily
because of a deficiency in the boundary detection. To enable the
recognition of the fine-grained composition for waste mixtures, Dong
et al. (2022) proposed a boundary-aware semantic segmentation model
to depict the edges among different waste materials. Such
boundary-aware precise waste segmentation can potentially be achieved
by a transformer, which is a DL framework that is gaining considerable
attention in the field of computer vision.
In summary, although significant efforts have been made for con­
struction waste recognition, these studies have mainly focused on the
model performance, namely, the classification accuracy. Only few
studies have focused on the data quality, which we aim to solve in this
study.
2.3. Construction material strength prediction
Compared with computer vision-based methods, some existing
studies have focused on the material strength prediction based on
regression approaches (Ahmad et al., 2021; Khan et al., 2022; Song
et al., 2021). Song et al. (2021) investigated the compressive strength
prediction problem of concrete using gene expression programming,
artificial neural networks, and decision trees. In particular, 98 data
points were collected from the experimental approach, and seven
different features (cement, fly, ash, superplasticizer, coarse aggregate,
fine aggregate, water, and days) were treated as inputs to predict the
compressive strength. Ahmad (Ahmad et al., 2021) further studied the
3
Resources, Conservation & Recycling 183 (2022) 106362
individual and ensemble approaches to estimate the compressive
strength by considering the cement content, fine, coarse aggregate,
water, binder-to-water ratio, fly ash, and superplasticizer as the input.
The results show that the ensemble model achieved a better result than
the decision tree and gene expression programming methods. Khan
et al. (2022) developed a Multiphysics model to predict the strength of a
concrete-filled steel tube column based on an artificial neural network.
Although these studies conducted their research based on regression,
which is completely different from ours, they provide a promising way
to improve the data quality of the properties of concrete by considering
its compressive strength.
2.4. Data Quality for Solid Waste Management
Despite the fact that few studies have focused on building high-
quality construction waste dataset, some existing studies have attemp­
ted to collect high-quality datasets for municipal solid wastes manage­
ment (Iqbal et al., 2020; Olay-Romero et al., 2020; Wang et al., 2021).
Considering that effective municipal solid waste (MSW) management is
deeply influenced by the methodology, quality of data, and robustness of
the analysis. Iqbal et al. (2020) pointed out the quality of input data is
critical to producing reliable results from and Life cycle assessment
study. In this context, it is strongly advised to use primary data when­
ever possible and to use data quality analysis methods like sensitivity
and uncertainty analysis to guarantee the data quality of input.
Furthermore, Olay-Romero et al. (2020) treated Mexico as a case study
and discovered that increasingly complex sets of indicators play an
important role in municipal solid waste management (MSWM). The data
quality was evaluated according to the methodology developed by
Turcott Cervantes et al. (2018), which considers the following criteria:
a) origin of the data, b) level of uncertainty, c) temporal coverage, and
spatial coverage. Recently, Wang et al. (2021) further investigated how
different quality of collected data affects the performance of data-driven
models, such as linear regression and Artificial Neural Network, where
authors also pointed out that collecting high-quality primary MSW data,
using consistent measurement and recording methodologies are likely to
be prohibitively time and labor-consuming. Compared with constructing
high-quality municipal solid waste dataset, we intend to bridge this gap
for construction material and propose a novel framework for building
high-quality construction material dataset.
3. Methodology
In this section, we introduce our construction material datasets,
including the data collection and annotation, data selection, training
criteria, and data augmentation.
3.1. Data Collection and Data Annotation
To the best of our knowledge, deep learning is a data-hungry model.
A recognition model usually requires large, comprehensive, and high-
quality datasets for improved performance. However, Son et al.
(2014) already investigated this field and generated data for three types
of construction material (i.e., steel, concrete, and wood): (1) these
datasets have limited construction materials and are not available on­
line; and (2) the quality of the datasets need to be improved further. To
this end, considering the limited categories of construction materials
and no available datasets, five datasets (one each for concrete, brick,
metal, wood, and stone) were collected and generated.
There are several environmental factors that affect the surface colors
of construction materials, such as weather conditions (e.g., clear, rain,
cloudy), time variation of a day or season, and vision when collecting
these data; these factors lead to a large uncertainty in the appearance of
construction materials. To make the classifier more robust to such un­
certainties, 285, 300, 487, 340, and 106 photographs were captured for
concrete recognition, metal recognition, wood recognition, brick
Y. Sun and Z. Gu
recognition, and stone recognition under different environmental fac­
tors, respectively. Additionally, an EOS-200D camera was used to collect
construction material data. These photographs include different images
of five construction components (e.g., concrete, metal, wood, brick, and
stone) collected to build high-quality construction material datasets and
4
Resources, Conservation & Recycling 183 (2022) 106362
Fig. 2. (a) An example of a construction material image for con­
crete recognition; (b) examples of sub-regions for concrete recog­
nition; (c) an example of a construction material image for brick
recognition; (d) examples of sub-regions for brick recognition; (e)
an example of a construction material image for metal recognition;
(f) examples of sub-regions for metal recognition; (g) an example
of a construction material image for wood recognition; (h) exam­
ples of sub-regions for wood recognition; (i) an example of a
construction material image for stone recognition; and (j) exam­
ples of sub-regions for stone recognition (Note: The first, second,
and third rows in the (b), (d), (f), (h), and (j) panels of this figure
show the examples of sub-regions that were labeled as the material
of interest, the background, and unable to say whether the sub-
region was the material of interest or the background,
respectively.)
validate the robustness and effectiveness of the corresponding con­
struction material dataset using a state-of-the-art deep classifier. More­
over, when these photographs were captured, we also considered the
distance to ensure that all images contained the full structures.
Specifically, each photograph was separated into subsets of 32 × 32
Y. Sun and Z. Gu
Fig. 3. Data Selection through Learning
Table 1
Criteria for the selection of training, validation, and testing images
Number of Images Percentage
Training 4900 70%
Validation 700 10%
Testing 1400 20%
Total 7000 100%
pixels. Each subset was classified and labeled into the categories of
materials of interest, background, and uncertainty; the uncertainty
category implies that the subset cannot be distinguished between the
material of interest or background. The first row of Figure 2 (b, d, f, h, j)
shows the subsets labeled as concrete, brick, metal, wood, and stone
datasets. The second and third rows of Figure 2 (b, d, f, h, j) show the
subsets wherein the main object or the background (uncertainty) cannot
be determined and the background. In particular, for different datasets,
the background comprises all types of locations wherein the construc­
tion materials are located; namely, the sky, windows, soil, trees, pipes,
and other backgrounds. In the present study, we selected 7000 images
for each class of different construction materials to train the different
deep CNNs to verify the effectiveness, validity, and reliability of the
collected construction material datasets.
3.2. Data Selection
After data collection and annotation, we proposed a novel cross-
review framework that could provide high-quality construction mate­
rial datasets through a learning process, as shown in Fig. 3. In this phase,
the goal is to assign each construction material image j to the training
datasets and delete images with a score below the threshold, wherein the
specific image is classified as real or an outlier.
The dataset quality is vital in training the state-of-the-art deep-
learning models. Although significant efforts have been made toward
the model-centric image recognition of construction waste, no studies
have been conducted on the data-centric image recognition of con­
struction waste. Thus, we hypothesize that higher quality datasets will
achieve higher accuracy.
To conduct data selection, we first divide our construction material
dataset D into N non-overlapping meta-datasets, that is, MetaSeti , where
i = 1, 2,…, N. To this end, we select state-of-the-art image classification
models, such as the VGG Network, ResNet, and GoogleNet, to train the N
expert models Expert
̃ i : i = 1, 2, 3, …, N with respect to the N meta-
5
Resources, Conservation & Recycling 183 (2022) 106362
datasets (notably, each expert model Expert ̃ i will only train on 1/N of
each entire dataset). The Expert model can be formulated as
∑ ( )
̃ i = argmin
Expert ̃ i
L dj , Expert (1)
̃
Experti dj ∈MetaSeti
where L denotes the objective function. After every expert model is
trained on its metadata, the entire dataset can be evaluated.
The dj of each image should be contained in only one meta-dataset.
This dataset was assumed to be MetaSet2 ; we subsequently conducted an
evaluation of dj on all expert models except expert model 2 because it
was considered to be separate from the meta-dataset MetaSet2 that it had
already been trained with. After conducting all the evaluations, we
computed the scores of dj in the metric of the target task, achieving N −
1 scores from N − 1 different expert models. The metric to be used here
can be F1, accuracy, or precision for our construction waste image
recognition task. Finally, we formulated the score of dj as follows:
1 ∑ ( ( ))
sj = ̃ i dj ,
Expert (2)
N − 1 i∈(1,2,3,…,N),i∕
=2
with all the computed scores, we obtained the final score for each sample
in the dataset. To this end, we then decided whether each sample was
real or an outlier based on the sj score, which was referred to as voting,
and can be formulated as follows:
{
keep if sj > δ
Datasets = (3)
delete otherwise
where δ denotes the threshold in terms of the corresponding metric. We
refer to our data selection approach as a cross-review, as shown in
Fig. 3. In the proposed approach, the expert models perform their
evaluation in a crossed manner, which effectively achieves the corre­
sponding high-quality datasets. Additionally, we conducted an evalua­
tion of each sample to understand the fluctuation of each sample and
alleviate the fluctuation of the specific expert model. In particular, our
cross-review data selection approach can effectively achieve high-
quality datasets.
3.3. Training Criteria
After data selection, 7000 images were obtained for each material
dataset. Thus, 70% of the images were chosen for training, 10% for
validation, and the remaining 20% images for testing. Thus, we selected
4900, 700, and 1400 images for the training, validation, and model
Y. Sun and Z. Gu Resources, Conservation & Recycling 183 (2022) 106362

Fig. 4. Effect of Data Selection

Table 2
Deep Convolutional Neural Networks Models
Mobile Architecture Year Main Contribution Parameters Depth Category References
Architecture Name

VGG 2014 Homogenous topology Uses small size kernel 138M 19 Spatial Exploitation Simonyan and
Zisserman (2014)
GoogleNet 2015 Introduced block concept Split transform and merge 4M 22 Spatial Exploitation Szegedy et al. (2015)
idea
Inception-V3 2015 Handles the problem of a representation bottleneck 23.6M 159 Depth + width Szegedy et al.
Replace large size filters with small filters (2016b)
Unfriendly ResNet 2016 Residual learning Identity mapping based skip 25.6M 152 Depth + multi-path He et al. (2016)
connections
Inception-V4 2016 Split transformand merge idea 35M 70 Depth + width Szegedy et al.
(2016a)
Inception 2016 Uses split transform merge idea and residual links 55.8M 572 Depth + width + Szegedy et al.
-ResNet multi-path (2016a)
DeseNet 2017 Cross-layer information flow 25.6M 190 Multi-path Huang et al. (2017)
Xception 2017 Depth wise convolution followed by point wise 22.8M 126 Width Chollet (0000)
convolution
SeNet 2017 Models inter-dependencies between feature-maps 27.5M 152 Feature-map Hu et al. (2018)
exploitation
NASNet 2018 Search for an architectural building block on a small 22.6M — Depth Zoph et al. (2018)
dataset Transfer the block to a larger dataset
SqueezeNet 2016 Smaller CNN architectures 4.8M 18 Feature-map Iandola et al. (2016)
exploitation
MobileNet 2017 Mobile and embedded vision applications Depthwise 3.2M 88 Width Howard et al. (2017)
separable convolutions
ShuffleNet 2017 Computation-efficient CNN architecture Pointwise 4.2M 50 Deepth + Feature-map Zhang et al. (2018)
group convolution Channel shuffle exploitation
Friendly MobileNetV2 2018 Inverted residual structure 3.4M 88 Depth + Width Sandler et al. (2018)
ShuffleNetV2 2018 Efficient Network Design 1.37M 50 Width + Feature-map Ma et al. (2018)

testing, respectively. The statistics for the training, validation, and
testing datasets of each class are presented in Table 1.
3.4. Data Augmentation
Image data augmentation is vital in training the deep-learning
models, and can generate many modified photos to improve the
robustness of the deep learning model. Here the functions TORCH.
UTILS.DATA and TORCHVISION.TRANSFORMS from Pytorch deep-
learning library are used to generate the image tensor data through
data augmentation techniques, thereby ensuring that the trained deep-
learning model is reliably robust. To make the model more robust to

6
Y. Sun and Z. Gu Resources, Conservation & Recycling 183 (2022) 106362

Table 3
Evaluation of Concrete Datasets
Mobile Architecture Architectures Categories Performance Evaluation(%) Average(%)

Accuracy F1 Accuracy Precision Recall F1

Unfriendly VGG16 Concrete 98.143 99.063 96.585 96.588 96.585 96.581

Background 96.841 98.395
Uncertainty 94.767 97.313
GoogleNet Concrete 99.000 99.497 97.278 97.284 97.278 97.274
Background 97.272 98.617
Uncertainty 95.556 97.727
Inception-V3 Concrete 98.357 99.172 97.039 97.040 97.039 97.037
Background 96.770 98.358
Uncertainty 95.986 97.952
ResNet50 Concrete 98.929 99.461 97.373 97.374 97.373 97.370
Background 97.128 98.543
Uncertainty 96.057 97.989
Inception-V4 Concrete 98.928 99.461 97.206 97.210 97.206 97.204
Background 96.482 98.209
Uncertainty 96.200 98.064
Inception-ResNet Concrete 98.571 99.280 96.800 96.809 96.800 96.796
Background 96.697 98.321
Uncertainty 95.125 97.501
DenseNet Concrete 98.571 99.281 97.254 97.253 97.254 97.251
Background 97.487 98.727
Uncertainty 95.699 97.802
Xception Concrete 98.857 99.425 97.162 97.158 97.162 97.155
Background 96.626 98.284
Uncertainty 95.986 97.952
SeNet50 Concrete 98.643 99.317 97.755 97.762 97.755 97.756
Background 97.128 98.543
Uncertainty 97.491 98.730
NASNet Concrete 98.714 99.353 97.135 97.143 97.135 97.130
Background 97.344 98.654
Uncertainty 95.341 97.615
Friendly SqueezeNet Concrete 98.429 99.208 96.896 96.895 96.896 96.892
Background 96.913 98.432
Uncertainty 95.341 97.615
MobileNet Concrete 97.643 98.807 96.872 96.871 96.872 96.871
Background 97.272 98.617
Uncertainty 95.699 97.802
ShuffleNet Concrete 98.643 99.317 97.039 97.038 97.039 97.036
Background 96.770 98.358
Uncertainty 95.699 97.802
MobileNetV2 Concrete 98.500 99.244 96.777 96.774 96.777 96.771
Background 96.698 99.321
Uncertainty 95.125 97.502
ShuffleNetV2 Concrete 98.214 99.099 96.944 96.942 96.944 96.940
Background 97.200 98.580
Uncertainty 95.412 97.652

various input object sizes and shapes, we applied three different
augmentation approaches for training, which are given as follows.
3.4.1. Flipping
Many studies have proven that flipping is a useful technique for
training deep-learning models. Therefore, batch data were horizontally
flipped with a probability of 0.5 for our construction material datasets.
3.4.2. Cropping
Cropping is another basic data augmentation method that achieves a
batch of image tensors by cropping the central patch of each image”.
Compared with the translations, we obtained a smaller image after
cropping, whereas the translations retained the original image size. To
this end, we randomly cropped each sample to a size of 32 ×32 from
padded 40 × 40.
3.4.3. Rotation
Rotation is also a basic operation when conducting data augmenta­
tion, which can be realized by rotating the photo by x-degrees on an axis
in a clockwise or inversed clockwise direction, where x ∈ [1,359]. In our
study, we rotated each sample by r ∈ [− 5, 5] degrees in batches.
4. Experiments
4.1. Implementation
All experiments were conducted using Pytorch 1.4 that is a deep-
learning library installed on a PC server under Python 3.6 environ­
ments. In particular, a single NVIDIA TITAN-V graphical processing unit
(GPU) was used to train the deep-learning classifier. Furthermore, this
PC server has a RAM capacity of 64.0 GB and holds a 3.60 GHz Intelx­
Corec i7-9700K processor with 16 logical threads and 16 MB of cache
memory.
4.2. Evaluation Metric
To evaluate the experimental results from the deep-learning model,
the confusion matrix, which includes the true positive (TP), true nega­
tive (TN), false positive (FP), and false negative (FN) values, was used. In
our case, TP represents images that can be correctly recognized as
construction materials by the deep-learning classifier. FP denotes that
the deep learning model treats the background or uncertainty images as
construction materials. Moreover, FN indicates that the deep learning
model recognizes the construction image as the background or uncer

7
Y. Sun and Z. Gu Resources, Conservation & Recycling 183 (2022) 106362

Table 4
Evaluation of Brick Datasets
Mobile Architecture Architectures Categories Performance Evaluation(%) Average(%)

Accuracy F1 Accuracy Precision Recall F1

Unfriendly VGG16 Brick 97.714 98.840 97.000 97.007 97.000 97.001

Background 98.143 99.063
Uncertainty 95.143 97.515
GoogleNet Brick 97.714 98.844 97.667 97.666 97.667 97.666
Background 99.000 99.497
Uncertainty 96.286 98.108
Inception-V3 Brick 98.000 99.990 97.476 97.477 97.476 97.476
Background 98.286 99.135
Uncertainty 96.143 98.033
ResNet50 Brick 98.143 99.063 97.857 97.859 97.857 97.858
Background 98.571 99.281
Uncertainty 96.857 98.403
Inception-V4 Brick 97.429 98.698 97.286 97.286 97.286 97.285
Background 98.857 99.425
Uncertainty 95.571 97.736
Inception-ResNet Brick 98.000 98.990 97.429 97.427 97.429 97.426
Background 98.571 99.281
Uncertainty 95.714 97.810
DenseNet Brick 98.000 98.990 97.667 97.663 97.667 97.662
Background 99.286 99.642
Uncertainty 95.714 97.810
Xception Brick 97.714 98.844 97.429 97.431 97.429 97.429
Background 98.714 99.353
Uncertainty 95.857 97.885
SeNet50 Brick 98.571 99.281 98.000 98.003 98.000 97.999
Background 98.714 99.352
Uncertainty 96.714 98.330
NASNet Brick 98.000 98.990 97.286 97.284 97.286 97.284
Background 98.286 99.135
Uncertainty 95.471 97.736
Friendly SqueezeNet Brick 98.000 98.989 97.714 97.717 97.714 97.714
Background 98.857 99.425
Uncertainty 96.286 98.108
MobileNet Brick 97.571 98.771 97.048 97.055 97.048 97.048
Background 98.429 99.208
Uncertainty 95.143 97.511
ShuffleNet Brick 97.857 98.917 97.285 97.294 97.285 97.285
Background 98.714 99.353
Uncertainty 95.286 97.586
MobileNetV2 Brick 98.857 99.425 97.619 97.634 97.619 97.617
Background 98.429 99.208
Uncertainty 95.571 97.736
ShuffleNetV2 Brick 97.143 98.551 97.048 97.048 97.048 97.047
Background 98.571 99.281
Uncertainty 95.429 97.661

tainty image. TN indicates that the deep learning model correctly rec­ • Accuracy: Accuracy measures the correctness of the deep learning
ognizes the background or uncertainty image. This is similar to the model, which computes the ratio of the number of correctly recog­
combined dataset. In this case, the precision, recall, accuracy and F1- nized construction materials to the total number of testing samples.
score were used to evaluate the classification performance based on
(TP + TN)
the testing set and confusion matrix. Specifically, the scikit-learn library Accuracy = (6)
(TP + FP + FN + TN)
is used to realize these four metrics (Pedregosa and Varoquaux, 0000).
The four performance metrics were as follows:
• F1-score: F1-score is the weighted average of the precision and
• Precision: Precision measures the correctness of a model by calcu­
recall. Therefore, this metric considers both the FP and FN.
lating the ratio of correctly recognized construction images to the
∑N
total images recognized as construction material. i=1 2TPi
F1-Score = ∑N (7)
∑N i=1 2TP i + FPi + FNi
TPi
Precision = ∑N i=1 (4)
i=1 TPi + FPi

• Recall (Sensitivity): Recall measures the ratio of correctly recog­ 4.3. Effects of Data Selections
nized construction images to the total number of sampling images.
∑N We initially studied the effect of our cross-review data selection
TPi
Recall = ∑L i=1 (5) strategy on the concrete, brick, and metal datasets. Specifically, we
i=1 TPi + FNi chose VGG16 as our expert model and used the classification accuracy as
our metric. Finally, we conducted the evaluation with and without the
data selection strategy using the VGG16, GoogleNet, Inception-V3,

8
Y. Sun and Z. Gu Resources, Conservation & Recycling 183 (2022) 106362

Table 5
Evaluation of Metal DataSets
Mobile Architecture Architectures Categories Performance Evaluation(%) Average(%)

Accuracy F1 Accuracy Precision Recall F1

Unfriendly VGG16 Metal 95.286 97.586 94.905 94.961 94.905 94.899

Background 99.000 99.497
Uncertainty 90.429 94.974
GoogleNet Metal 96.143 98.034 95.524 95.592 95.524 95.523
Background 99.143 99.569
Uncertainty 91.286 95.444
Inception-V3 Metal 92.714 96.219 94.714 94.732 94.714 94.722
Background 98.714 99.353
Uncertainty 92.714 96.219
ResNet50 Metal 96.000 97.959 95.238 95.297 95.238 95.232
Background 99.143 99.569
Uncertainty 90.571 95.052
Inception-V4 Metal 93.714 96.755 94.571 94.588 94.571 94.574
Background 98.714 99.353
Uncertainty 91.286 95.444
Inception-ResNet Metal 92.571 96.142 94.095 94.101 94.095 94.096
Background 98.714 99.353
Uncertainty 91.000 95.288
DenseNet Metal 94.571 97.210 95.000 95.013 95.000 94.996
Background 99.000 99.497
Uncertainty 91.429 95.522
Xception Metal 95.000 97.436 95.238 95.265 95.238 95.244
Background 98.286 99.135
Uncertainty 92.429 96.065
SeNet50 Metal 94.714 97.285 95.426 95.439 95.426 95.431
Background 98.857 99.425
Uncertainty 92.714 96.219
NASNet Metal 95.143 97.511 95.429 95.447 95.429 95.428
Background 99.000 99.497
Uncertainty 92.142 95.911
Friendly SqueezeNet Metal 93.857 96.831 95.190 95.179 95.190 95.180
Background 99.714 99.857
Uncertainty 92.000 95.833
MobileNet Metal 95.143 97.511 94.810 94.848 94.810 94.808
Background 98.427 99.208
Uncertainty 90.857 95.210
ShuffleNet Metal 94.857 97.361 94.238 94.298 94.238 94.240
Background 98.000 98.990
Uncertainty 89.857 94.658
MobileNetV2 Metal 95.000 97.436 94.620 94.676 94.620 94.617
Background 98.571 99.281
Uncertainty 90.286 94.895
ShuffleNetV2 Metal 94.857 97.360 94.190 94.267 94.190 94.180
Background 99.000 99.497
Uncertainty 88.714 94.019

ResNet50, and DenseNet models, respectively.

e‖wj ‖‖a ‖ψ (θj )
(i)
Fig. 4 plots the classification accuracy of VGG16, GoogleNet, p(i) (8)
j =
Inception-V3, ResNet50, and DenseNet with and without the data se­ e‖wj ‖‖a(i) ‖ψ (θj ) + ∑ e‖wl ‖‖a(i) ‖cos(θl )
l∕
=j
lection strategy. Notably, when we did not use data selection (i.e., w/o
data selection in Fig. 4), these models achieved bad performance and a ( ) ( )
ψ θj = ( − 1)k cos mθj − 2k, θj ∈ [kπ / m, (k + 1)π / m] (9)
big differences were observed between the accuracy of different models.
In contrast, when our data selection strategy was applied (i.e., w/ data where k ∈ [0, m − 1] is an integer and m controls the margin among the
selection in Fig. 4), the accuracy considerably improved and was stable classes. When m = 1, the L-Softmax loss is reduced to the original
between the different models. In addition, we also conduct experiments Softmax loss. A relatively difficult learning objective can be defined by
and use precision as evaluation metric. The results are consistent with adjusting the margin m between classes, which can effectively avoid
the performance achieved by accuracy. This indicates that our data se­ overfitting.
lection strategy can provide high quality training, testing, and validation
datasets. 4.4.1. Training Scenarios
We conducted these experiments under different training scenarios
using six different construction-material datasets. In particular, we
4.4. Deep Convolutional Neural Network Models
conducted experiments on the combined construction-material datasets.

To further verify the quality, effectiveness, validity, and reliability of

1. Scenario 1: Evaluate concrete datasets (i.e., Concrete, Background,
the proposed construction material datasets, we conducted several ex­
and Uncertain) on the selected state-of-the-art deep classifiers.
periments on state-of-the-art deep CNN models. The detailed models are
2. Scenario 2: Evaluate brick datasets (i.e., Brick, Background, and
listed in Table 2. Additionally, to further improve the model perfor­
Uncertain) on the selected state-of-the-art deep classifiers.
mance, we used the large-margin softmax (LSoftmax) loss to replace
softmax loss to avoid overfitting. Eq. 8 defines the L-softmax loss.

9
Y. Sun and Z. Gu Resources, Conservation & Recycling 183 (2022) 106362

Table 6
Evaluation of Wood Datasets
Mobile Architecture Architectures Categories Performance Evaluation(%) Average(%)

Accuracy F1 Accuracy Precision Recall F1

Unfriendly VGG16 Wood 99.714 99.857 97.885 97.887 97.885 97.886

Background 97.079 98.518
Uncertainty 96.866 98.408
GoogleNet Wood 99.786 99.893 97.932 97.942 97.932 97.932
Background 98.006 98.993
Uncertainty 96.011 97.965
Inception-V3 Wood 99.571 99.785 97.766 97.767 97.766 97.766
Background 96.795 98.371
Uncertainty 96.937 98.445
ResNet50 Wood 99.929 99.964 98.146 98.146 98.146 98.146
Background 97.079 98.518
Uncertainty 97.436 98.701
Inception-V4 Wood 99.643 99.821 97.980 97.983 97.980 97.981
Background 97.151 98.554
Uncertainty 97.150 98.554
Inception-ResNet Wood 99.357 99.678 97.219 97.233 97.219 97.222
Background 95.513 97.705
Uncertainty 96.795 98.371
DenseNet Wood 99.714 99.857 97.790 97.796 97.790 97.790
Background 96.368 98.150
Uncertainty 97.293 98.628
Xception Wood 99.786 99.892 97.789 97.791 97.789 97.790
Background 97.151 98.554
Uncertainty 96.439 98.187
SeNet50 Wood 99.857 99.929 98.146 98.156 98.146 98.147
Background 96.510 98.224
Uncertainty 98.077 99.029
NASNet Wood 99.857 99.929 98.075 98.074 98.075 98.074
Background 97.293 98.638
Uncertainty 97.079 98.518
Friendly SqueezeNet Wood 99.643 99.821 97.671 97.673 97.671 97.671
Background 97.080 98.518
Uncertainty 96.296 98.113
MobileNet Wood 99.571 99.785 97.552 97.557 97.552 97.553
Background 96.225 98.076
Uncertainty 96.886 98.408
ShuffleNet Wood 99.571 99.785 97.552 97.557 97.552 97.553
Background 96.225 98.076
Uncertainty 96.886 98.408
MobileNetV2 Wood 99.857 99.928 97.243 97.244 97.243 97.242
Background 96.509 98.224
Uncertainty 95.370 97.630
ShuffleNetV2 Wood 99.642 99.821 97.338 97.337 97.338 97.337
Background 96.581 98.261
Uncertainty 95.798 97.854

3. Scenario 3: Evaluate metal datasets (i.e., Metal, Background, and
Uncertain) on the selected state-of-the-art deep classifiers.
4. Scenario 4: Evaluate wood datasets (i.e., Wood, Background, and
Uncertain) on the selected state-of-the-art deep classifiers.
5. Scenario 5: Evaluate stone datasets (i.e., Stone, Background, and
Uncertain) on the selected state-of-the-art deep classifiers.
6. Scenario 6: Evaluate combined datasets (i.e., Concrete, Brick, Metal,
Wood, and Stone) on the selected state-of-the-art deep classifiers.
4.4.2. Experiment Results
Evalution on Concrete Datasets: Concrete is a basic material in
modern construction material, making it a basic material to be recog­
nized by the deep classifier. Table shows the evaluation of concrete
datasets under deep classifiers. Specifically, we first conducted experi­
ments on unfriendly mobile architectures. We observed that SeNet50
achieves the best performance compared with the other architectures,
with average accuracy, precision, recall, and F1-score values of 97.755,
97.762, 97.755, and 97.756%, respectively. Conversely, VGG16 ach­
ieved the worst results, with accuracy, precision, recall, and F1-score
values of 96.585, 96.588, 96.585, and 96.581%, respectively. Addi­
tionally, considering the performance evaluation of the concrete, Goo­
gleNet achieved the best accuracy of 99.000%, and the worst accuracy
was observed from the VGG16 architecture, (98.143%). However, these
concrete datasets can still be recognized using the VGG model. Although
different models show different performances, the difference in the
performance between different models is not large. This indicates that
the proposed concrete datasets are of good quality and can be recog­
nized by different models.
Additionally, from the model-efficient viewpoint, we also evaluated
the proposed concrete datasets on mobile-friendly architectures, which
are also presented in Table 3. ShuffleNet achieved the best results with
accuracy, precision, recall, and F1-score values of 97.039, 97.038,
97.039, and 97.036%, respectively. MobileNetV2 achieved the worst
performance, but its performance was acceptable, with accuracy, pre­
cision, recall, and F1-score values of 96.777, 96.774, 96.777, and
96.771%, respectively.
Notably, state-of-the-art deep classifiers can recognize concrete from
the background and uncertainty, which verifies the effectiveness, reli­
ability, and identifiability of our concrete dataset.
Evaluation of Brick Datasets: Brick is known to play an essential
role in modern construction materials; therefore, it is important for the
deep classifier to distinguish it from other objects (e.g., background and
uncertainty). To this end, we conducted experiments to evaluate the
effectiveness of the proposed brick dataset. The results are listed in

10
Y. Sun and Z. Gu Resources, Conservation & Recycling 183 (2022) 106362

Table 7
Evaluation of Stone Datasets
Mobile Architecture Architectures Categories Performance Evaluation(%) Average(%)

Accuracy F1 Accuracy Precision Recall F1

Unfriendly VGG16 Stone 97.286 98.624 97.810 97.814 97.810 97.810

Background 98.714 99.353
Uncertainty 97.429 98.698
GoogleNet Stone 97.714 98.844 98.333 98.337 98.333 98.334
Background 99.143 99.570
Uncertainty 98.143 99.063
Inception-V3 Stone 97.286 98.624 97.667 97.670 97.667 97.668
Background 98.571 99.281
Uncertainty 97.143 98.551
ResNet50 Stone 97.571 98.771 98.095 98.095 98.095 98.094
Background 99.143 99.569
Uncertainty 97.571 98.770
Inception-V4 Stone 97.286 98.624 97.857 97.859 97.857 97.857
Background 99.000 99.497
Uncertainty 97.286 98.624
Inception-ResNet Stone 96.571 98.256 97.286 97.304 97.286 97.290
Background 98.000 98.989
Uncertainty 97.286 98.624
DenseNet Stone 97.143 98.550 98.000 98.008 98.000 98.002
Background 98.857 99.425
Uncertainty 98.000 98.989
Xception Stone 98.000 98.989 98.333 98.333 98.333 98.333
Background 99.286 99.641
Uncertainty 97.714 98.844
SeNet50 Stone 97.429 98.698 98.190 98.203 98.190 98.193
Background 98.857 99.425
Uncertainty 98.286 99.135
NASNet Stone 98.143 99.063 98.286 98.287 98.286 98.286
Background 99.143 99.569
Uncertainty 97.571 98.770
Friendly SqueezeNet Stone 97.286 98.624 98.000 98.001 98.000 97.999
Background 99.286 99.641
Uncertainty 97.428 98.698
MobileNet Stone 97.714 98.844 97.667 97.670 97.667 97.668
Background 98.285 99.135
Uncertainty 97.000 98.477
ShuffleNet Stone 97.143 98.551 97.381 97.386 97.381 97.383
Background 98.285 99.135
Uncertainty 96.714 98.330
MobileNetV2 Stone 96.571 98.256 97.143 97.145 97.143 97.143
Background 98.714 99.353
Uncertainty 96.143 98.034
ShuffleNetV2 Stone 97.143 98.550 98.143 98.162 98.143 98.146
Background 98.714 99.353
Uncertainty 98.571 99.281

Table 4. Notably, SeNet50 achieved the best results for the brick data­
sets, with accuracy, precision, recall, and F1-score values of 98.000,
98.003, 98.000, and 97.999%, respectively. Moreover, although VGG16
achieves the worst performance, this model still performs reasonably
well, with accuracy, precision, recall, and F1-score values of 97.000,
97.007, 97.000, and 97.001%, respectively. Additionally, although
VGG16, GoogleNet, and Xception have the worst accuracy, the brick
material can be satisfactorily recognized by these models.
Furthermore, we evaluated the performance of mobile-friendly ar­
chitectures. The results are presented in Table 4. Notably, SqueezeNet
achieved the best performance, with accuracy, precision, recall, and F1-
score values of 97.714, 97.717, 97.714, and 97.714%, respectively.
MobileNet and ShuffleNetV2 achieved the worst performance compared
with the other architectures, but still showed relatively reasonable
performance.
All the aforementioned experiments demonstrate the effectiveness,
reliability, and quality of the proposed brick datasets.
Evaluation of Metal Datasets: Subsequently, we evaluated the
metal datasets using different deep classifiers; Table 5 presents the re­
sults. Notably, GoogleNet achieves the best average performance,
wherein the accuracy, precision, recall, and F1-score values were
95.524, 95.592, 95.524, and 95.523%, respectively. Meanwhile,
Inception-ResNet achieved the worst average performance, with accu­
racy, precision, recall, and F1-score values of 94.095, 94.101%,
94.095%, and 94.096%, respectively.
Additionally, mobile-friendly architecture was used to evaluate and
train the metal datasets. The results are listed in Table 5. We observed
that SqueezeNet achieved the best average performance with accuracy,
precision, recall, and F1-score values of 95.190, 95.179, 95.190, and
95.180%, respectively. ShuffleNetV2 achieved the worst average per­
formance; however, the performance was still acceptable, with accu­
racy, precision, recall, and F1-score values of 94.190, 94.267, 94.190,
and 94.180%, respectively.
Although the performance of the state-of-the-art model is lower than
those of concrete and brick, the performances of these models verifies
the effectiveness, reliability, and quality of the proposed metal datasets.
Evaluation of Wood Datasets: Compared with the other materials,
we conducted subsequent experiments to evaluate the performance of
the wood datasets. Table 6 presents these results. SeNet50 achieved the
best performance in terms of average metrics with accuracy, precision,
recall, and F1-score values of 98.146, 98.156, 98.146, and 98.147%,
respectively. Inception-ResNet achieves the worst average performance,
with accuracy, precision, recall, and F1-score values of 97.219, 97.233,
97.219, and 99.929%, respectively.

11
Y. Sun and Z. Gu Resources, Conservation & Recycling 183 (2022) 106362

Table 8
Evaluation of Combined Datasets under Mobile Unfriendly Architecture
Architectures Categories Performance Evaluation Average(%)

Accuracy F1 Accuracy Precision Recall F1

VGG16 Concrete 99.857 99.929 99.600 99.602 99.600 99.600

Brick 99.857 99.928
Metal 99.571 99.785
Wood 99.714 99.857
Stone 99.000 99.497
GoogleNet Concrete 99.857 99.928 99.657 99.658 99.657 99.657
Brick 99.857 99.928
Metal 99.714 99.856
Wood 99.714 99.855
Stone 99.143 99.570
Inception-V3 Concrete 99.571 99.785 99.429 99.430 99.429 99.428
Brick 99.714 99.856
Metal 99.571 99.785
Wood 99.571 99.785
Stone 99.714 99.353
ResNet50 Concrete 99.571 99.856 99.457 99.460 99.457 99.457
Brick 99.714 99.856
Metal 99.714 99.857
Wood 99.571 99.785
Stone 99.714 99.353
Inception-V4 Concrete 99.714 99.857 99.543 99.546 99.543 99.543
Brick 99.714 99.856
Metal 99.714 99.857
Wood 99.714 99.857
Stone 98.857 99.425
Inception-ResNet Concrete 99.286 99.642 99.343 99.343 99.343 99.342
Brick 99.857 99.929
Metal 99.285 99.641
Wood 99.571 99.785
Stone 98.714 99.353
DenseNet Concrete 99.714 99.857 99.686 99.687 99.686 99.685
Brick 99.857 99.929
Metal 99.857 99.928
Wood 99.857 99.928
Stone 98.143 99.570
Xception Concrete 99.571 99.785 99.543 99.544 99.543 99.542
Brick 99.857 99.929
Metal 99.714 99.857
Wood 99.857 99.928
Stone 98.714 99.353
SeNet50 Concrete 99.571 99.785 99.457 99.460 99.457 99.457
Brick 99.714 99.856
Metal 99.714 99.857
Wood 99.571 99.785
Stone 98.714 99.353
NASNet Concrete 99.857 99.928 99.629 99.630 99.629 99.628
Brick 100.000 100.000
Metal 99.571 99.785
Wood 99.857 99.928
Stone 98.857 99.425

Additionally, we also evaluated the mobile-friendly architectures;
the results are presented in Table 6. Notably, SqueezeNet achieved the
best average performance, with accuracy, precision, recall, and F1-score
values of 97.671, 97.673%, 97.671, and 97.671%, respectively. Mobi­
lenetV2 achieved the worst average performance, but its performance
was still relatively reasonable, with accuracy, precision, recall, and F1-
score values of 97.243, 97.244, 97.243, and 97.242%, respectively.
In this setting, we also verified the effectiveness, reliability, and
quality of the proposed wood dataset.
Evaluation of Stone Datasets: Considering the stone datasets,
although only few constructions directly use stone, it still plays an
essential role in constituting construction materials. In this case, further
experiments were conducted on stone datasets. Tables 7 present these
results; GoogleNet was observed to achieve the best results, with accu­
racy, precision, recall, and F1-score values of 98.333, 98.337, 98.333,
and 98.334%, respectively. Moreover, although Inception-ResNet ach­
ieved the worst performance, this model also showed a relatively
reasonable performance for this dataset, with accuracy, precision, recall,
and F1-score values of 97.286, 97.304%, 97.286, and 97.290%,
respectively.
We then conducted experiments on different mobile-friendly archi­
tectures. The results are listed in Table 7. Notably, ShuffleNetV2 ach­
ieved the best performance, wherein the accuracy, precision, recall, and
F1-score values were 98.143, 98.162, 98.143, and 98.146%, respec­
tively. MobileNetV2 achieved the worst performance compared with the
other architectures, but it could still be used to recognize stones. Finally,
all the aforementioned experiments demonstrated the effectiveness,
reliability, and quality of the proposed stone datasets.
Evaluation of the Combined Datasets: In this section, we first
select the images of the main construction material (i.e., concrete, brick,
metal, wood, and stone) from the five construction material datasets to
build new datasets, called, the combined datasets. Subsequently, the
aforementioned state-of-the-art deep learning classifiers are used to
verify whether these models can distinguish between these different
construction materials under a PC device setting. The results are listed in
Table 8. DenseNet achieved the best average performance, wherein the

12
Y. Sun and Z. Gu Resources, Conservation & Recycling 183 (2022) 106362

Table 9
Evaluation of Combined DataSets under Mobile Friendly Architectures
Architectures Categories Performance Evaluation Average(%)

Accuracy F1 Accuracy Precision Recall F1

SqueezeNet Concrete 99.857 99.929 99.600 99.601 99.600 99.599

Brick 100.000 100.000
Metal 99.571 99.785
Wood 99.714 99.857
Stone 98.857 99.425
MobileNet Concrete 99.857 99.929 99.457 99.460 99.457 99.457
Brick 99.857 99.928
Metal 99.428 99.713
Wood 99.714 99.857
Stone 98.429 99.208
ShuffleNet Concrete 99.857 99.929 99.428 99.429 99.428 99.428
Brick 99.714 99.857
Metal 99.000 99.487
Wood 99.857 99.928
Stone 98.714 99.353
MobileNetV2 Concrete 99.714 99.857 99.371 99.372 99.371 99.370
Brick 100.000 100.000
Metal 98.857 99.425
Wood 99.857 99.928
Stone 98.428 99.208
ShuffleNetV2 Concrete 99.857 99.928 99.371 99.374 99.371 99.371
Brick 99.571 99.785
Metal 99.429 99.713
Wood 99.714 99.856
Stone 98.286 99.135

accuracy, precision, recall, and F1-score values were 99.686, 99.687,
99.686, and 99.685%, respectively. Inception-ResNet achieved the
worst average performance, wherein the accuracy, precision, recall, and
F1-score values were 99.343, 99.343, 99.343, and 99.342%, respec­
tively. Additionally, with respect to the specified performance evalua­
tion, NASNet achieved the best accuracy performance for the following
different objects: 99.857, 100.000, 99.571, 99.857, and 98.857% for
concrete, brick, metal, wood, and stone, respectively. The worst accu­
racy performance was achieved by Inception-ResNet architecture, with
accuracy values of 99.286, 99.857, 99.285, 99.571, and 98.714% for
concrete, brick, metal, wood, and stone, respectively. However, the
main construction material can still be recognized using the Inception-
ResNet model.
Additionally, we also evaluated the proposed combined datasets on
the mobile device setting, whose results are presented in Table 9.
Notably, SqueezeNet achieved the best average performance, wherein
the accuracy, precision, recall, and F1-score values were 99.600, 99.601,
99.6001, and 99.599%, respectively. MobilenetV2 achieved the worst
average performance, but its performance was still relatively reason­
ably, wherein the accuracy, precision, recall, and F1-score values were
99.371, 99.372, 99.371, and 99.370%, respectively. Meanwhile, from a
performance evaluation perspective, SqueezeNet achieved the best ac­
curacy performance, with accuracy values of 99.857, 100.000, 99.571,
and 98.857% for concrete, brick, wood, and stone, respectively. Shuf­
fleNet achieves the worst accuracy performance: with accuracy values of
99.857, 99.714, 99.000, 99.857, and 98.714% for concrete, brick, metal,
wood, and stone, respectively. In summary, these experiments verified
the effectiveness, reliability, and identifiability of the proposed datasets.
6. Conclusions
Recycling construction and demolition waste is an ever-increasing
problem that continues to contribute to landfills. A large amount of
construction waste can be reused if separated. Manually sorting is very
expensive and time-consuming. The recent success of deep learning
makes automatic sorting possible. To achieve high-performance auto­
matic sorting (e.g., classification) model, this research has shown that it
is possible to propose a data-centric model instead of the traditional
model-centric model to improve the classification performance. The
experiments results (refer to Fig. 4) show the possibility of improving the
performance through data cleaning and also verify the effectiveness of
the proposed cross-review framework.
The development of the technologies described in this paper would
lead to considerable changes in contemporary construction and demo­
lition waste management. They have the potential to assist project
managers (waste champions) and waste stream managers avoid incor­
rect waste type sorting. This could further promote automatic waste
classification. An outcome of this nature would be of considerable
benefit to construction and demolition practitioners, constructors, and
the wider community.
CRediT authorship contribution statement
Ying Sun: Data curation, Writing – original draft, Investigation.
Zhaolin Gu: Supervision, Writing – review & editing.
Declaration of Competing Interest

5. Limitations and Future Work
This study has two limitations that could be addressed in future
research. First, it is of interest to design a novel deep construction ma­
terial recognition model by combining the cross-review framework to
further improve the recognition performance. Second, the current con­
struction waste material datasets work well for image classification
tasks. In contrast, a similar framework is required to improve the data
quality for construction waste object detection and semantic
segmentation.
The authors declare that they have no known competing financial
interests or personal relationships that cdocould have appeared to in­
fluence the work reported in this paper.
Acknowledgments
This work was supported by the project of Basic Work of the Ministry
of Science and Technology of China (No. SB2013FY112500)

13
Y. Sun and Z. Gu Resources, Conservation & Recycling 183 (2022) 106362

References Pedregosa, F., Varoquaux, G.,. Scikit-learn: Machine learning in python.

Ragheb, A., El-Shimy, H., Ragheb, G., 2016. Green architecture: A concept of
sustainability. Procedia-Social and Behavioral Sciences 216 (6). https://doi.org/
Ahmad, A., Farooq, F., Niewiadomski, P., Ostrowski, K., Akbar, A., Aslam, F.,
10.1016/j.sbspro.2015.12.075.
Alyousef, R., 2021. Prediction of compressive strength of fly ash based concrete
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C., 2018. Mobilenetv2:
using individual and ensemble algorithm. Materials 14 (4), 794. https://doi.org/
Inverted residuals and linear bottlenecks. Proceedings of the IEEE conference on
10.3390/ma14040794.
computer vision and pattern recognition, pp. 4510–4520. https://doi.org/10.1109/
Chen, J., Lu, W., Xue, F., 2021. looking beneath the surface: A visual-physical feature
CVPR.2018.00474.
hybrid approach for unattended gauging of construction waste composition. Journal
Seadon, J.K., 2010. Sustainable waste management systems. Journal of Cleaner
of Environmental Management 286, 112233. https://doi.org/10.1016/j.
Production 18 (16-17), 1639–1651. https://doi.org/10.1016/j.jclepro.2010.07.009.
jenvman.2021.112233.
Simonyan, K., Zisserman, A., 2014. Very deep convolutional networks for large-scale
Chollet, F.,. Xception: Deep learning with depthwise separable convolutions.
image recognition. arXiv preprint arXiv:1409.1556.
Chu, B., Madhavan, V., Beijbom, O., Hoffman, J., Darrell, T., 2016. Best practices for
Son, H., Kim, C., Hwang, N., Kim, C., Kang, Y., 2014. Classification of major construction
fine-tuning visual classifiers to new domains. European conference on computer
materials in construction environments using ensemble classifiers. Advanced
vision. Springer, pp. 435–442.
Engineering Informatics 28 (1), 1–10. https://doi.org/10.1016/j.aei.2013.10.001.
Davis, P., Aziz, F., Newaz, M.T., Sher, W., Simon, L., 2021. The classification of
Song, Hongwei, Ahmad, Ayaz, Farooq, Furqan, Ostrowski, Krzysztof Adam,
construction waste material using a deep convolutional neural network. Automation
Maślak, Mariusz, Czarnecki, Slawomir, Aslam, Fahid, 2021. Predicting the
in Construction 122, 103481. https://doi.org/10.1016/j.autcon.2020.103481.
compressive strength of concrete with fly ash admixture using machine learning
Ding, Z., Yi, G., Tam, V.W., Huang, T., 2016. A system dynamics-based environmental
algorithms. Construction and Building Materials 308, 125021.
performance simulation of construction waste reduction management in china.
Su, Y., 2020. Multi-agent evolutionary game in the recycling utilization of construction
Waste management 51, 130–141. https://doi.org/10.1016/j.wasman.2016.03.001.
waste. Science of The Total Environment 738, 139826. https://doi.org/10.1016/j.
Dong, Z., Chen, J., Lu, W., 2022. Computer vision to recognize construction waste
scitotenv.2020.139826.
compositions: A novel boundary-aware transformer (bat) model. Journal of
Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A., 2016. Inception-v4, inception-resnet and
environmental management 305, 114405. https://doi.org/10.1016/j.
the impact of residual connections on learning. arXiv preprint arXiv:1602.07261.
jenvman.2021.114405.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D.,
He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition.
Vanhoucke, V., Rabinovich, A., 2015. Going deeper with convolutions. Proceedings
Proceedings of the IEEE conference on computer vision and pattern recognition,
of the IEEE conference on computer vision and pattern recognition, pp. 1–9. https://
pp. 770–778. https://doi.org/10.1109/CVPR.2016.90.
doi.org/10.1109/CVPR.2015.7298594.
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T.,
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z., 2016. Rethinking the
Andreetto, M., Adam, H., 2017. Mobilenets: Efficient convolutional neural networks
inception architecture for computer vision. Proceedings of the IEEE conference on
for mobile vision applications. arXiv preprint arXiv:1704.04861.
computer vision and pattern recognition, pp. 2818–2826. https://doi.org/10.1109/
Hu, J., Shen, L., Sun, G., 2018. Squeeze-and-excitation networks. Proceedings of the IEEE
CVPR.2016.308.
conference on computer vision and pattern recognition, pp. 7132–7141.
Toğaçar, Mesut, Ergen, Burhan, Cömert, Zafer, 2020. Waste classification using
Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q., 2017. Densely connected
AutoEncoder network with integrated feature selection method in convolutional
convolutional networks. Proceedings of the IEEE conference on computer vision and
neural network models. Measurement 153, 107459.
pattern recognition, pp. 4700–4708.
Turcott Cervantes, D. E., et al., 2018. Sistema de indicadores para la evaluación integral y
Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K., 2016.
control de la gestión de residuos municipales.
Squeezenet: Alexnet-level accuracy with 50x fewer parameters and< 0.5 mb model
Wang, D., Tang, Y.-T., He, J., Yang, F., Robinson, D., 2021. Generalized models to predict
size. arXiv preprint arXiv:1602.07360.
the lower heating value (lhv) of municipal solid waste (msw). Energy 216, 119279.
Iqbal, A., Liu, X., Chen, G.-H., 2020. Municipal solid waste: Review of best practices in
https://doi.org/10.1016/j.energy.2020.119279.
application of life cycle assessment and sustainable management techniques. Science
Wang, J., Wu, H., Duan, H., Zillante, G., Zuo, J., Yuan, H., 2018. Combining life cycle
of The Total Environment 729, 138622.
assessment and building information modelling to account for carbon emission of
Khan, S., Ali Khan, M., Zafar, A., Javed, M.F., Aslam, F., Musarat, M.A., Vatin, N.I., 2022.
building demolition waste: A case study. Journal of cleaner production 172,
Predicting the ultimate axial capacity of uniaxially loaded cfst columns using
3154–3166. https://doi.org/10.1016/j.jclepro.2017.11.087.
multiphysics artificial intelligence. Materials 15 (1), 39. https://doi.org/10.3390/
Wang, Z., Li, H., Zhang, X., 2019. Construction waste recycling robot for nails and
ma15010039.
screws: Computer vision technology and neural network approach. Automation in
Ku, Y., Yang, J., Fang, H., Xiao, W., Zhuang, J., 2021. Deep learning of grasping detection
Construction 97, 220–228. https://doi.org/10.1016/j.autcon.2018.11.009.
for a robot used in sorting construction and demolition waste. Journal of Material
Wu, H., Wang, J., Duan, H., Ouyang, L., Huang, W., Zuo, J., 2016. An innovative
Cycles and Waste Management 23 (1), 84–95. https://doi.org/10.1007/s10163-020-
approach to managing demolition waste via gis (geographic information system): a
01098-z.
case study in shenzhen city, china. Journal of Cleaner Production 112, 494–503.
Kujala, J. V., Lukka, T. J., Holopainen, H., 2015. Picking a conveyor clean by an
https://doi.org/10.1016/j.jclepro.2015.08.096.
autonomously learning robot. arXiv preprint arXiv:1511.07608.
Xiao, W., Yang, J., Fang, H., Zhuang, J., Ku, Y., 2019. A robust classification algorithm
Llatas, C., Bizcocho, N., Soust-Verdaguer, B., Montes, M., Qui,. An lca-based model for
for separation of construction waste using nir hyperspectral system. Waste
assessing prevention versus non-prevention of construction waste in buildings.
Management 90, 1–9. https://doi.org/10.1016/j.wasman.2019.04.036.
Lu, W., Chen, J., Xue, F., 2022. Using computer vision to recognize composition of
Zhang, X., Zhou, X., Lin, M., Sun, J., 2018. Shufflenet: An extremely efficient
construction waste mixtures: A semantic segmentation approach. Resources,
convolutional neural network for mobile devices. Proceedings of the IEEE conference
Conservation and Recycling 178, 106022. https://doi.org/10.1016/j.
on computer vision and pattern recognition, pp. 6848–6856. https://doi.org/
resconrec.2021.106022.
10.1109/CVPR.2018.00716.
Lu, W., Yuan, L., Xue, F., 2021. Investigating the bulk density of construction waste: A
Zheng, L., Wu, H., Zhang, H., Duan, H., Wang, J., Jiang, W., Dong, B., Liu, G., Zuo, J.,
big data-driven approach. Resources, Conservation and Recycling 169, 105480.
Song, Q., 2017. Characterizing the generation and flows of construction and
https://doi.org/10.1016/j.resconrec.2021.105480.
demolition waste in china. Construction and Building Materials 136, 405–413.
Ma, L., Zhang, L., 2020. Evolutionary game analysis of construction waste recycling
https://doi.org/10.1016/j.conbuildmat.2017.01.055.
management in china. Resources, Conservation and Recycling 161, 104863. https://
Zhu, Z., Brilakis, I., 2010. Parameter optimization for automated concrete detection in
doi.org/10.1016/j.resconrec.2020.104863.
image data. Automation in Construction 19 (7), 944–953. https://doi.org/10.1016/
Ma, N., Zhang, X., Zheng, H., Sun, J., 2018. Shufflenet v2: Practical guidelines for
j.autcon.2010.06.008.
efficient cnn architecture design. Proceedings of the European conference on
Zoph, B., Vasudevan, V., Shlens, J., Le, Q.V., 2018. Learning transferable architectures
computer vision (ECCV), pp. 116–131.
for scalable image recognition. Proceedings of the IEEE conference on computer
Olay-Romero, E., Turcott-Cervantes, D.E., del Consuelo Hernández-Berriel, M., de
vision and pattern recognition, pp. 8697–8710. https://doi.org/10.1109/
Cortázar, A.L.-G., Cuartas-Hernández, M., de la Rosa-Gómez, I., 2020. Technical
CVPR.2018.00907.
indicators to improve municipal solid waste management in developing countries: A
case in mexico. Waste Management 107, 201–210. https://doi.org/10.1016/j.
wasman.2020.03.039.

14

View publication stats