Renewable and Sustainable Energy Reviews 130 (2020) 109979

Contents lists available at ScienceDirect

Renewable and Sustainable Energy Reviews

journal homepage: http://www.elsevier.com/locate/rser

Revolutionising building inspection techniques to meet large-scale energy

demands: A review of the state-of-the-art
M. Hasan Shariq *, Ben Richard Hughes
Department of Mechanical and Aerospace Engineering, University of Strathclyde, United Kingdom

A R T I C L E I N F O A B S T R A C T

Keywords: The building sector is responsible for 40% of the overall energy consumption in the EU. Building defects, such as
Infrared thermography heat losses, moisture, and air leakages, inevitably causes inefficient space heating or cooling, which accounts
Building inspection considerably towards this high energy consumption and associated greenhouse gas emissions. In order to meet
Building energy audit
the EU’s 2050 carbon reduction targets, building inspection techniques need to be revolutionised. Current
Real-time thermal modelling
Photogrammetry
methods rely on terrestrial or hand-held infrared thermography (IRT) to detect building defects. However, for a
Drone 3D mapping large-scale inspection, these methods are generally labour-intensive, time-consuming, costly and often ineffi­
cient. The aim of this paper is to highlight the possibility of integrating various state-of-the-art technologies and
computational methods with IRT including drones, photogrammetry and AI. This paper presents a compre­
hensive review of relevant scientific papers and recent developments in such technologies that can retrofit the
existing manually intensive methods. Among the findings of this research, feasibility of monocular thermo­
graphic photogrammetry integrated on a drone (quadcopter) promises a time-efficient, cost-effective and near-
autonomous solution to large-scale building inspections.

The Energy Performance of Buildings Directive (EPBD) in 2018

1. Introduction
Building performance is a concept that quantifies the sustainability
of a building to be energy efficient [1]. Defects in the envelopes of
buildings, such as air leakages, heat losses, moisture, damaged thermal
insulation, etc. are the foremost reason that affects building perfor­
mances [2], consequently increasing energy consumptions and inevi­
tably greenhouse gas emissions. The building sector accounts for 36% of
Carbon Dioxide (CO2) emissions in the EU, where one-third of energy
consumption results from the non-domestic buildings. At present, almost
75% of the building stock is energy inefficient, and more than 75% of the
overall building stock is envisaged to remain in use beyond 2050 [3,4]
particularly in the public sector. Among the types of energy demand
within the EU’s building stock, space heating accounts for the largest
proportion with around 68% of energy usage in residential buildings
[5], while space and industry process heating accounts for above 70% of
energy consumption in industries [6]. This demands an urgent and
effective action to cut emissions in the building sector where emissions
have not fallen significantly in recent years. Retrofit energy-efficient
intervention strategies are therefore vital to meet ener­
gy/CO2-emissions targets.
introduced new targets to “accelerate cost-effective renovation of
existing buildings, with the vision of a decarbonised building stock by
2050” [7]. As a result, noticeably more researches have emerged
addressing heating and cooling in buildings and industries to substan­
tially boost energy performances and reduce carbon footprint. However,
building energy modelling and the corresponding adoption of
energy-efficiency interventions at scale remains limited by the cost and
time taken to survey relevant buildings and generate accurate models
from high granularity data on the building structure and façade. Existing
manually intensive approaches to internal building surveys are invasive,
inefficient and/or access-limited. This presents time and cost barriers to
building energy modelling in most contexts and circumstances.
Thermal cameras detect infrared (IR) radiation that is emitted from
an object’s surface and convert it into a thermal image consisting of
temperature gradients that can be interpreted [8]. Over the recent years,
there have been a large number of studies in structural thermography
primarily focused on detecting water/heat leakages and other faults in
building envelopes that affect energy efficiency in buildings [9]. This
includes a non-destructive means of inspection of the interior as well as
the exterior of a building. The size and complexity of a building play an
important role during an inspection. Hence, the efficiency of thermal

* Corresponding author.
E-mail address: hasan.shariq@strath.ac.uk (M.H. Shariq).

https://doi.org/10.1016/j.rser.2020.109979
Received 25 September 2019; Received in revised form 3 June 2020; Accepted 6 June 2020
Available online 18 June 2020
1364-0321/© 2020 Elsevier Ltd. All rights reserved.
M.H. Shariq and B.R. Hughes
Abbreviations
AI Artificial Intelligence
ANN Artificial Neural Network
BEM Building Energy Modelling
BIM Building Information Modelling
CNN Convolutional Neural Network
CO2 Carbon Dioxide
DEM Digital Elevation Model
DoF Degree of Freedom
DPI Dots Per Inch
EU European Union
FDD Fault Detection and Diagnosis
analysis of buildings is not limited to the resolution of thermography
results or its accuracy, but also the speed of inspection and the cost.
Lagüela et al. [10] developed a technique to accurately map ther­
mographic imagery onto digital imagery semi-automatically. Two
thermographic images capturing the same region were first combined to
register a thermographic mosaic and create a dense thermal image,
which was then fused with the corresponding high-resolution digital
(photographic) imagery using a dense surface modelling algorithm. This
helped produce a high thermal- and geometric resolution of an ‘ortho­
thermogram’ (a thermal 3D model) as shown in Fig. 1. This method
helps minimise time to qualitatively identify and assess a defect.
The authors concluded that infrared thermography is a self-sufficient
technique for building inspection if the thermographic imagery is pre­
sented upon a digital 3D model. However, although it is applicable for
both, indoors and outdoors, this approach is time-consuming consid­
ering an inspection of a large building due to terrestrial scanning
methods.
Fox et al. [11] explored building defect detection using external and
internal thermography. Their main focus was on building damages
related to missing insulation, cracks, heat losses, thermal bridging, and
moisture defects. The authors investigated and compared the traditional
(i) walk-through thermography, in which the thermographer walks
around the building while recording all external building, against the
emerging (ii) ‘walk-past’ methodology, which requires the thermogra­
pher to capture the images of only the designated dwelling one at a time.
Later, all images are compared together qualitatively. The authors
concluded the walk-past method to be much quicker and cheaper to
perform. However, this approach could be more efficient if the images
were captured and compared digitally and automatically. The authors
also acknowledged that despite a much greater number of defects were
detected using a walk-past approach, there were still many defects
unfound since the thermography was performed from single external
elevations.
Barreira et al. [12] and more recently, Cai et al. [13] investigated IRT
passive techniques to detect and assess the effect of leakages in build­
ings. The latter authors outlined an energy-saving detection method for
buildings energy efficiency through thermal imager and infrared image
Fig. 1. Visible camera image (a) of a building fused with its thermographic image (b) to produce an Orthothermogram (c) [10].
2
Renewable and Sustainable Energy Reviews 130 (2020) 109979
GPR Ground Penetrating Radar
GPU Graphical Processing Unit
IMU Inertial Measurement Unit
ISO International Standards Organization
GPS Global Positioning System
IRT Infrared Thermography
LiDAR Light Detection and Ranging
RGB-D Red Green Blue - Depth
SfM Structure from Motion
SLAM Simultaneous Localisation and Mapping
TLS Terrestrial Laser Scanning
UAS Unmanned Aerial Systems
UAV Unmanned Aerial Vehicle
processing. A digital image processing method was presented to effec­
tively eliminate noise interference (of thermal imagery) through a But­
terworth low pass filter, which notably speeds up image processing and
noise cancellation. This opens up a variety of opportunities and appli­
cations if this approach is built on, for example, employing an un­
manned aerial vehicle for building thermography.
Recent review papers related to building energy inspection largely
presented the developments, methods, challenges, and future trends of
fault detection and diagnosis. Kirimtat and Krejcar [9] summarised the
existing methods and applications of IRT-based building inspections.
The authors highlighted the future scope of IRT based inspection by
outlining several procedural techniques and suggested image processing
methods to better visualise surface defects. Zhao et al. [14] discussed
existing methods of Artificial Intelligence (AI) driven fault detection and
diagnosis (FDD) and stressed on the lack of research focus towards
AI-based FDD in buildings despite its potentially high favourable scope.
Kim and Katipamula [15] reviewed FDD methods from 2004 to 2017
and argued the lack of automation in most building inspections and
suggested that future work should focus to eliminate the need for
manual fault identification through trained Artificial Neural Networks
(ANNs). Lucchi [16] comprehensively discussed the IRT applicability in
building energy audit while investigating a variety of methods and
challenges associated with the existing IRT inspections. The authors also
suggested smart sensors for indoor monitoring and environmental con­
trol. Rakha and Gorodetsky [17] emphasised on using Unmanned Aerial
Systems (UAS) for inspections of the built environment to reduce
manual workflow of the energy audit process. Although a few studies
used drone-based thermal inspections of buildings, the authors argued
that its scope demands further experimentation-based research and
standardisation. Hence, among the five review papers discussed above,
each identified specific limitation(s) of existing building energy audit
techniques and suggested a novel methodology that could contribute
towards an efficient building inspection.
However, to the authors’ knowledge, no review or research paper
related to building energy has been carried out that specifically discusses
multiple innovative technologies from diverse fields that can collec­
tively be integrated to revolutionise the building inspection process for a
M.H. Shariq and B.R. Hughes
large-scale survey. Moreover, despite the urgent demand to rapidly
lower energy consumption, research focus currently lacks investigating
an automated, versatile, and yet cost-effective solution for building
energy audit to solve the energy crisis in the building sector.
Hence, the objective of this review paper is to discuss and evaluate
the feasibility of integrating state-of-the-art multidisciplinary technol­
ogies and trending computational methods that can transform thermal
inspection of existing buildings in the most time- and cost-friendly
manner in order to rapidly improve the thermal efficiency of existing
buildings. A large number of recent research papers from 2012 to 2019
on diverse yet relevant areas are highlighted and evaluated. The feasi­
bility and applicability of their innovative methodologies are then
examined for applying on potentially large or complex building struc­
tures considering the time and cost barrier.
2. Review of technologies relevant to building inspection
The following sub-sections review various technologies and
computational methods that promise direct or indirect benefit and op­
portunities for building inspection, which could enable a low cost and
speedy energy audit of entire building envelopes regardless of building
size or complexity.
2.1. Model reconstruction through laser scanning
In most cases for building thermography, identifying and relocating
the defects require a three-dimensional (3D) model of the target build­
ing. This helps to map the exact location of all defected areas onto a
three-dimensional model of the interior or exterior of the buildings to
help visualise and inspect the defects quantitatively. Recently, a large
amount of work has been done to optimise LiDAR sensing and mapping
processes. The applications and purposes of LiDAR usually range from
generating point clouds for three-dimensional (3D) modelling of in­
doors, the exterior of buildings or structures, as well as for producing
accurate elevation or topographic maps of farmlands, forests, water­
ways, etc. For the purpose of this review, laser scanning applicable for
only building indoor and/or façades are deliberated.
Garwood et al. [18] investigated the potential application of model
reconstruction for Building Energy Modelling (BEM) by producing
detailed 3D models of a large industrial building using LiDAR-based
terrestrial laser scanning (TLS) method. The authors had to capture at
86 distinct positions manually around the building, which took five
working days. Point cloud data of a single scan and the combined fusion
of 86 scans are shown in Fig. 2. In addition to the time taken by the laser
scanning, it also took an extensive amount of manual post-processing
due to a large amount of data from the point cloud. The authors
acknowledged that “an opportunity exists for increasing the speed of 3D
geometry creation of existing industrial buildings for application in BEM
and subsequent thermal simulation”.
A similar study was performed by Son et al. [19] with an emphasis on
reducing the number of laser scans to achieve the same result. The au­
thors managed to reduce the amount of laser scanning, and yet achieve
Fig. 2. (a) Point cloud data of a single LiDAR scan, (b) Fusion of 86 scans of entire building [18].
3
Renewable and Sustainable Energy Reviews 130 (2020) 109979
the desired result, by mapping each nearby thermal imagery with a
common point cloud data with the help of camera-based digital mapping
techniques. Fig. 3 shows the mapping of texture onto 3D point clouds
and then mapping of thermal imagery onto the 3D model. The idea
works well for an average-sized building, however, this method of
‘image matching’ would take exponentially more time as the size or
complexity of the building increases. This is because the graphics need
to be mapped manually each time in order to generate a 3D model.
Hence, despite the thermographic data mapping time is reduced, the
overall time barrier has not been resolved.
In order to improve the time taken to produce 3D models, Ochmann
et al. [20], in 2016, presented a parametric-based automatic approach
for 3D model reconstruction from indoor point clouds. A 3D recon­
structed model was produced that combined different scans from
different rooms together to automatically detect the wall openings such
as doors and windows. Although these methodologies reduced the time
for post-processing, the approach however required laser scans from
fixed positions in every room and a manual room segmentation to start
the process. In their next research, the authors [21] presented a revised
and novel approach to automatically reconstruct a completely volu­
metric 3D model of buildings from oriented point clouds. The new
technique eliminated the need for manual room segmentation, as the
authors integrated probabilistic assumptions to auto-detect room spaces,
walls, doors, etc. This approach, however, is not applicable for recon­
structing exterior of buildings.
In a very recent investigation by Zhan et al. [22], a ‘diffraction op­
tical element’ (DOE)-based structured-light method has been proposed
to capture 3D objects as point-clouds with extremely high resolution.
This method relies on structured laser patterns to capture perfectly
aligned grid-points of the object and then applying a pre-trained
‘convolution neural network’ (CNN) to identify pattern-element. This
helps construct a highly accurate 3D model of the scanned object.
Although the results show great potential in 3D scanning for a variety of
applications (particularly those that require higher-level accuracy with
real-time 3D model reconstruction), this technique also requires scans
from fixed locations and will be too time-consuming for scanning a
building for inspection purposes.
To enable a mobile LiDAR system, Park et al. [23] investigated and
proposed a unique solution to the motion distortion in LiDAR scanning,
which can help in efficient loop closures during mapping. The authors
used LiDAR fusion for a Continuous-Time Simultaneous Localisation and
Mapping (i.e. CT-SLAM) system using local mapping for
continuous-time trajectory, and global mapping for loop closure and
global map building. The experiments proved that with the help of
simultaneous map fusion and continuous-time trajectory, the measure­
ments storage to obtain an optimised 3D model is no longer required,
and motion distortion can be effectively compensated in real-time,
which makes this framework to be time- and cost-effective. However,
the amount of computing resources required to process the data presents
many challenges for rapid inspection of buildings at larger-scale.
In conclusion, LiDAR surveying and mapping provide the best ac­
curacy and high-density results [24]. However, among the range of
M.H. Shariq and B.R. Hughes
Fig. 3. (a) Mapping texture on point clouds, (b) mapping thermal imagery onto final 3D model [19].
real-world applications where LiDAR is being deployed, continuous
real-time optimisations (such as loop closures, motion distortion and
sparse data correction) [23], as well as post-processing (such as data
refinement, noise and misalignment correction and 3D mapping) [18]
are required that demand high computing power (and hence time) as
well as more human time. Hence, laser scanning for building energy
audit applications shows lesser scope for a large-scale inspection, where
a speedier and less memory-intensive solution is highly preferable.
2.2. Model reconstruction through photogrammetry
In recent years, technological advancements in digital imagery, ste­
reo vision, RGB-Depth (RGB-D) cameras, and image-mapping algo­
rithms have provided a massive scope for photogrammetry in three-
dimensional (3D) model reconstruction. Photogrammetry is a mature
technology that has existed for over a century [25]. It relies on ‘Structure
from Motion’ (SfM) and image-stitching techniques to capture and stitch
overlapped images to form a 3D model of a targeted scene. A variety of
photogrammetric methods have emerged in the last decade that tend to
provide a low-cost and high-resolution possibility to real-time 3D
scanning and mapping for a large range of applications [26] including
building interiors and façades. Three of such photogrammetry methods,
i.e. (a) Monocular photogrammetry, (b) Stereophotogrammetry, and (c)
RGB-D photogrammetry are reviewed in the following sub-sections.
2.2.1. Monocular photogrammetry
In 2015, Kaufman et al. [27] presented a technique to produce a 3D
model of a static object through a regular DSLR camera by digitally
combining multiple photos taken from different angles around that
object. The authors achieved this without any complex hardware or
software using a two-DoF turntable that captured multiple images at
different angles. The images were then filtered by masking them from
Fig. 4. Dense 3D point clouds and model reconstruction of a building model (b) being created through real-time 3D scanning via a monocular camera by hand
(a) [29].
4
Renewable and Sustainable Energy Reviews 130 (2020) 109979
the background, and a 3D photo-textured mesh image was formed with
the help of an image processing software called PhotoScan Pro. The
authors managed to produce a detailed and textured 3D model of an
object through a single DSLR camera, a simple setup, and user-friendly
software. Though, this technique was only applicable to small and
portable objects that can be placed and rotated on a turntable.
However, with the advancements in computer vision and open-
source photogrammetry algorithms, Pizzoli et al. [28] developed a
technique to produce dense and accurate point cloud with depth map
through a single moving camera. This probabilistic monocular depth
measurement method allowed a real-time 3D mapping of the sur­
roundings. An example can be seen in Fig. 4 (extracted from YouTube
video [29] published by the author) that shows a 3D dense map being
produced in real-time as it is being scanned through an ordinary camera.
The lack of any additional sensor (such as IR depth) helps in low memory
usage and faster processing of 3D reconstruction. This feature shows
high potential in 3D mapping of large-scale buildings through remotely
controlled mobile robot or an unmanned aerial vehicle (drone) with an
on-board processor for all the computational tasks. In the context of
building inspections, fusing thermal imagery with the camera result (or
stitching beforehand) would require more concurrent memory require­
ment, which will reduce the speed of construction. However, a delayed
real-time model reconstruction that requires moderate computational
memory and low cost is certainly more applicable and beneficial in
large-scale building inspection than a near-instant reconstruction with
heavy computation and high cost.
Similarly, Sch€ops et al. [30] proposed a smartphone-based real-time
monocular model reconstruction technique that performs all its
computation through the GPU of the mobile device. This near-instant
reconstruction allows the user to visualise (during the scan) the areas
that are yet to be reconstructed. Although, the low-resolution (i.e. 320 �
240) of stereo images helps in achieving a low computationally intensive
M.H. Shariq and B.R. Hughes Renewable and Sustainable Energy Reviews 130 (2020) 109979

and rapid 3D model reconstruction, building inspection applications real-time even in darkness. To optimise the use of Kinect sensor, espe­
would ideally require a denser resolution. However, considering the cially for large scale mapping, Esparza et al. [39] investigated and
large-scale and outdoor adaptability of this approach, a low-resolution developed a tool that integrates multiple camera calibration with point
fusion of thermal and RGB images can still produce a comprehensible cloud using the newer Kinect V2. The novel technique matches multiple
3D orthothermogram that can be used for a quick qualitative inspection 3D points from various Kinect V2 sensors to obtain intrinsic and extrinsic
of building façades before proceeding with an extensive inspection or parameters for the cameras (sensors) while attaching the results accu­
diagnosis, if necessary. rately. This allows a 360� field of view for mapping in real-time (if four
Kinect V2 sensors are arranged 90� apart) to reconstruct a 3D model of a
2.2.2. Stereophotogrammetry room instantly, as shown in Fig. 6.
Krutikova et al. [31] investigated a 3D model reconstruction tech­ Liu et al. [40] highlighted an experimented method for Simultaneous
nique through stereo images. A human face was used as an example of a Localisation and Mapping (SLAM) using RGB-D sensor using
geometrically complex shape to produce its 3D model through two open-source software. A mobile robot was used to capture an indoor
cameras (small distance apart) that took pictures simultaneously. The environment using a Kinect sensor to obtain colour and depth images.
distortion in the two images was resolved by locating and aligning two Wheel odometry data was processed in real-time in order to minimise
corresponding points of both images. A similar investigation that also localisation errors, mainly in the momentary absence of visual data. The
aimed to reconstruct the 3D model through stereo images was carried authors concluded that due to some inevitable errors in the localisation,
out by Fu et al. [32], which showed more promising results. A novel different SLAM approaches need to be used for different environments.
spatial-temporal stereo matching algorithm was developed using a Zollho€fer et al. [41] investigated and analysed in detail the latest
space-time speckle projection. The authors’ approach is highly techniques for 3D reconstruction using RGB-D cameras. The authors
time-efficient and can produce multiple 3D models accurately within a outlined that more than a thousand papers have been published related
second, which shows potential to be applied for a speedy unmanned to 3D RGB-D mapping since the release of the Kinect sensor. After
aerial inspection of building exteriors. analysing a variety of approaches for multiple scenarios and applica­
Pire et al. [33] presented a feature-based SLAM method for mapping tions, the authors concluded that despite the developments, a range of
through stereo imagery. A novel methodology called S-PTAM (Stereo
Parallel Tracking and Mapping) technique was used that utilises local
stereo images to track feature points and produce a 3D map while per­
forming loop detection and closures in real-time. This framework can be
used in large and complex environments, both indoors and outdoors,
with the help of loop closure module. Since sudden jerks or unusual
camera movements can affect localisation and hence the results, an IMU
can be integrated to compensate for such errors in real-time.
A high accuracy stereo based disparity map and 3D surface recon­
struction were investigated by Hamzah et al. [34]. The authors intro­
duced an improved stereo matching algorithm that helped in reducing
noise from image data obtained to ensure reliable results from pixel
comparison before creating a disparity map. The authors concluded that
this algorithm can be applied for 3D surface reconstruction for more
accurate mapping through stereo vision images. Hadfield et al. [35]
presented a top-down approach for a similar stereo reconstruction. Fig. 5
shows the output reconstruction of three random places in comparison
with the ground truth images. The authors’ technique of creating
high-quality disparity maps to be efficiently applicable in both indoor
and outdoor environments.

2.2.3. RGB-D photogrammetry

Namitha et al. [36] explored 3D reconstruction through the Micro­
soft Kinect RGB-D sensor [37,38]. A real-time Kinect Fusion algorithm
was used to combine depth maps with the RGB colour information to
Fig. 6. Point cloud fusion via four Kinect sensors [39].
create 3D point clouds. This allowed the 3D model to be reconstructed in

Fig. 5. Random examples of outdoor dataset with actual image (left), the output of investigated reconstruction (middle), and the ground truth reconstruction (see
full high-res image) [35].

5
M.H. Shariq and B.R. Hughes
challenges is remaining to be solved or retrofitted. Among these include
(1) the need for more efficient methods for geometry optimisation, (2)
the potential for applying 3D reconstruction in high-level applications,
and (3) integrating machine learning, more specifically deep neural
networks, on 3D data for qualitative and quantitative visual analysis to
make 3D reconstruction applications truly real-time and mostly
autonomous.
Guclu and Can [42] proposed a real-time SLAM based mapping
system through RGB-D sensor for large indoor environments. Through
frame-to-frame motion keypoints filtration and feature matching algo­
rithms, loop closures are performed before a map is produced in
real-time using trajectory estimation. A detailed flowchart describing
the proposed mapping algorithm is presented in Fig. 7. The authors
concluded that the framework allows large-scale and complex indoor
environments to be mapped in real-time using CPU. Further improve­
ments were suggested to improve the efficiency and speed of mapping
that includes the integration of deep features and implementation of this
system on Graphical Processing Unit (GPU) instead. Hence, with the
addition of GPU, this real-time SLAM system potentially equipped with a
thermal camera can be effectively used for large-scale indoor IRT-based
building inspections.
In conclusion, although the monocular mapping technique is
extremely low cost [43], it generally also requires a lot of
post-processing time and high computational requirement [44], except
if traded off for a lower resolution model reconstruction, which can be
obtained in real-time [30]. While the stereo camera technique signifi­
cantly reduces most of the heavy computational requirement, they are
more applicable in real-time applications [32]. However, stereo­
photogrammetry is limited by parallax effects (non-matching pairs) in
close-ranged mapping and non-planar environments, which causes
inaccurate results [45]. This limitation is resolved through the RGB-D
sensor that provides depth information along with colour imagery
mapped together to produce a high-resolution 3D model. Mapping
through RGB-D sensors, such as Kinect [37], not only provides a low-cost
solution to 3D mapping but can also replace the need for LiDAR mapping
in most applications, including aerial mapping through drones. With
respect to building inspections, although none of the recent
photogrammetry-based investigations incorporated energy modelling or
inspection, some studies [28,33] showed highly promising features that
can be integrated to assist in real-time building energy modelling tasks.
Hence, 3D model reconstruction (using photogrammetry techniques)
superimposed with thermographic imagery (discussed in Section 2.5)
can enable swift building energy modelling and help fill in these gaps.
2.3. Simultaneous localisation and mapping (SLAM)
Simultaneous Localisation and Mapping (SLAM) have been
commonly used in the field of Mobile Robotics. It allows a robot to locate
Fig. 7. Detailed RGB-D algorithm for 3D map reconstruction [42].
6
Renewable and Sustainable Energy Reviews 130 (2020) 109979
itself while simultaneously map (reconstruct) a 3D model of any un­
known indoor environment through a range of sensors and camera.
Borrmann et al. [46] proposed and implemented a structural thermog­
raphy system that captures thermal imagery and produces 3D point
clouds through a mobile robot. This remotely controllable robot con­
sisted of a laser scanner, thermal sensor, IMU module, GPS, and odom­
eter sensors that could perform SLAM and infrared thermography
concurrently. Although, this technique is efficient, automatic, and
time-saving, there are still a few downsides in the context of building
inspection applications. For example, this robot cannot possibly reach
and sense the hidden areas that are inaccessible for the wheeled robot,
perhaps due to height, shape, or location of the area. Secondly, due to
laser-scanning, the whole process is costly and highly time-consuming.
Similarly, Kim et al. [47] presented a technique to register 3D point
clouds through an autonomous SLAM-driven mobile robot. The authors
integrated a hybrid scanning system with a DSLR camera and a LiDAR
system mounted on a mobile robot platform. This four-wheeled auton­
omous robot is programmed to navigate inside a building while avoiding
while performing SLAM using a dynamic scanning process. According to
the authors, this SLAM-driven robotic system is applicable for both in­
door and outdoor mapping of buildings autonomously. However, the
final three-dimensional (3D) point clouds are generated using a static
scan to achieve the required accuracy, which can be counter-productive
in the context of large-scale mapping of buildings for inspection
purposes.
Yang et al. [48] presented a multi-camera visual SLAM based
approach for autonomous navigation through micro aerial vehicles. This
framework integrates imagery data from multiple cameras that track
navigation of micro aerial vehicles (MAVs) autonomously, which
enabled the drone to perform accurate localisation in any indoor envi­
ronment. However, due to the sole use of cameras for SLAM purposes,
the focus was more towards localisation rather than an accurate map­
ping of the three-dimensional indoor environment. Hence, with the
addition of point cloud generation, this methodology may as well be
used for real-time aerial inspection of buildings.
Zhang and Singh [49] investigated one of the most promising
intertial-SLAM techniques in 2017, especially useful for high-speed
navigation purposes. The authors highlighted a low-drift odometry
approach with real-time mapping using data from a 6-DoF IMU com­
bined with LiDAR point clouds. The method estimates the motion of
moving 3D LiDAR while compensating for the distortion of misaligned
point clouds (due to high speed) with the help of an odometry algorithm
simultaneously, prior to mapping. In 2018 [50], the authors further
added a camera to the framework of motion estimation that helped in
detecting geometric features more accurately at high frequency in a
coarse to fine manner, which enabled the localisation, mapping and
navigation to occur in dark and texture-less environments while
manoeuvring aggressively at high speeds. Two examples of 3D mapping
M.H. Shariq and B.R. Hughes
through this technique are shown in Fig. 8 that shows 3D reconstructed
model of two entire floors, each scanned in 2 min, through a hand-held
scanner. This method (after additional work) can allow an automated or
remotely controlled aerial mapping of building interiors at high speeds
for a rapid qualitative building inspection.
In conclusion, after evaluating a range of investigations and frame­
works, simultaneous localisation and mapping (SLAM) techniques show
a promising scope for a real-time indoor 3D mapping and can potentially
be used for producing thermographic 3D models (as reviewed in Section
2.5) of building envelopes for inspection. Since SLAM applications have
been noticeably limited to indoor environments, further research and
development opportunity exist that can allow SLAM equally applicable
for an exterior envelop of an entire building.
2.4. Drone surveys
Aerial surveys that used fixed-wing aircraft or helicopters have been
around since the 1980s [8]. However, in recent years, there has been an
increase in the use of multi-rotor drones (or UAVs) for surveillance and
monitoring applications [51–57] due to their long-ranged inspection,
applications for aerial photogrammetry, efficient data logging, dynamic
control, and mobility. One of the many challenges of such inspection is
the ability of the drone to reliably perform in extreme and/or unknown
environments. A variety of studies are reviewed that relate to drone
operation in various building or structural inspections.
Lucchi [16] reviewed IRT-based inspections of buildings among
almost 150 literature sources from the last fifty-five years. Different
passive thermographic approaches; aerial, automated fly-past (i.e. via
drone), street pass-by, perimeter walk-around, walk-through, etc. were
compared. The author (in reference to Fox et al. [8], who referred to an
older study [58] of 2012) reiterates that drones for energy audit pose a
variety of drawbacks including vibrations, reduced spatial resolution
and accuracy, the cost of equipment, and licensing restrictions. How­
ever, thanks to the advancements in UAVs, these problems have already
been resolved over recent years. For example, vibrations are eliminated
through a 3-axis gimbal and anti-vibration dampers. Likewise, the other
mentioned drawbacks such as spatial resolution, accuracy and cost are
also overcome through the recent advancements in photogrammetry
systems, as reviewed previously in Section 2.2.
Zorbas et al. [55] discussed a range of solutions to minimise cost in
the operation of drone surveys, which primarily includes energy con­
sumption. As the drone’s altitude increases to observe a larger area, the
energy consumption rises with it. The authors hence developed mathe­
matical models to compute the optimal placement coverage area for a
drone to hover to extract information of a target area. Several areas to
further improve and optimise the energy-consumption were also high­
lighted by the authors, including the possible addition of static sensors to
enhance localisation. This presents an opportunity for an unmanned
drone to perform thermographic 3D model reconstruction of buildings
Fig. 8. 3D model reconstruction through hand-held LiDAR [50].
7
Renewable and Sustainable Energy Reviews 130 (2020) 109979
interior and façades efficiently by ensuring the longest possible flight
time.
Seo et al. [59] investigated a drone-based bridge inspection meth­
odology for damage detection. In their study, a bridge and an abutment
in South Dakota was inspected as a subject through a DJI Phantom 4
drone through which various damages were effectively detected through
a real-time inspection. The authors observed an improved
image-resolution and damage detection capability as well as the
considerably lower cost of the drone-based inspection compared to other
conventional inspection methods. Similar aerial inspections can provide
an effective solution to inspecting high-rise buildings and/or unreach­
able areas of a building envelope.
Yang et al. [48], as also discussed in Section 2.3, performed real-time
mapping through a micro aerial vehicle. Their approach overcomes a
variety of limitations including inaccessibility of scanning hidden areas,
and real-time point-cloud mapping. The only gap, in the context of
building inspection, is the lack of thermographic 3D modelling on the 3D
reconstruction, which is an achievable integration as further reviewed in
Section 2.5.
Gao et al. [60] introduced a novel and low-cost approach to attain
high-resolution imagery to produce ortho-images and topographic data
of a terrain. Multi-view photogrammetry was used to compute
three-dimensional vertices of homologous image points from two adja­
cent photographs of the target area. The authors managed to achieve a
resolution of 0.065 of digital elevation model (DEM) while capturing the
images through a UAV. This helps in detecting faults/defects more
effectively and accurately. However, despite this system was specifically
designed to map and reconstruct terrains, this method has the potential
to be applied to building exteriors for model reconstruction.
Similarly, Entrop and Vasenev [61] featured a real-time thermog­
raphy approach by combining thermal sensors with a UAV in order to
monitor, assess and detect flaws in building structures (through thermal
imagery) remotely in real-time. This innovative technique by the au­
thors does indeed solve the time and cost problem in detecting flaws in
building structures, however, the two major shortcomings of this process
are that (1) the pilot cannot practically ensure about sensing every
portion of a large or complex building, and second (2) due to lack of 3D
mapping, once a flaw has been identified, there is no way to save, record
or relocate the same location again except by remembering it. Hence, for
a large, complex or unknown target building, the heat/water leakages
may not be relocatable.
In conclusion, drone-based surveys have proven to be a low-cost and
highly productive solution for a large range of applications related to
inspections and fault detection. However, despite the promising scope of
drone-based inspections for buildings, research focus currently lacks the
investigations to standardise an automated fly-past passive thermal in­
spection of building interiors and exteriors for fault detection. In a
detailed review on automated building inspection through drones,
Rakha and Gorodetsky [17] argued that the innovation of drones with
M.H. Shariq and B.R. Hughes Renewable and Sustainable Energy Reviews 130 (2020) 109979

thermal cameras for surveying has presented a massive opportunity can be addressed and resolved through a less computationally
towards automated building inspection with minimal manual workflow demanding methodology by replacing LiDAR with stereo depth IR sen­
and post-processing. This can allow speedy energy audits on large sors to capture real-time depth map using stereo photogrammetry, as
buildings at a considerably reduced cost. discussed in Section 2.2.
Another similar framework was highlighted by Aguilera et al. [65]
that fused point clouds from 3D laser scanner with IR images to produce
2.5. Thermographic 3D modelling
thermographic 3D models to study building energy performance. This
orthothermogram is generated automatically using a feature extraction
Recent developments towards building information modelling (BIM)
method from the registration of multiple IR and thermographic images.
have allowed building inspection to be performed non-destructively and
The results showed high accuracy in both 3D mapped result (with a
time-efficiently. This is achieved through a virtual integration of
point cloud resolution of 10 mm) and thermographic and metric infor­
building geometry during its inspection. Infrared thermography can be
mation (with 0.1 � C thermal resolution). However, this methodology
fused with point clouds of a building to produce a thermographic 3D
would be unreasonably time-consuming for a large or complex building.
model, also known as an orthothermogram [10]. This orthothermog­
An indoor mobile robot based mapping technique was examined by
raphy represents a 3D map of the façade or any building structure along
Lo�pez-Ferna�ndez et al. [66] that combined infrared thermography with
with its thermographic texture (most often with radiometric data) that
laser scanning to produce 3D orthothermogram of indoor environments.
allow defects, such as water leakages, heat losses, etc., to be detected
A two-wheeled (plus castor) mobile robot was integrated with IMU and
and analysed effectively and remotely. The orthothermogram could be
wheel odometry to autonomously navigate and scan walls through IR
mapped through either laser scanning or photogrammetry. Recent re­
thermal camera and laser scanner. After data acquisition and
searches in thermographic 3D modelling and results from both mapping
image-point cloud registration, a radiometric orthothermogram was
methods are reviewed in this section, along with their feasibility and
produced of an indoor of a building as shown from outside in Fig. 10.
scope in large-scale building inspection.
With the advancements in drone surveying as discussed in Section 2.4,
Lagüela et al. [10,62,63] developed a novel framework to automat­
this inspection methodology can be integrated on a UAV to reconstruct
ically map infrared thermal imagery onto point clouds of a building.
high-resolution 3D thermal models and perform qualitative fault anal­
Image processing was used for both thermographic imagery and
ysis on building indoors and exterior façades more accessibly.
laser-scanned point clouds in order to align them through a line features
to produce a thermographic 3D model, which represents thermographic
information mapped on to a 3D model. A flowchart outlining the process
to create textured as-built BIM and orthothermogram is shown in Fig. 9a
and resulting thermographic 3D models are shown in Figs. 1 and 9b. At
the current state, this method is not applicable for a large or complex
building as well as indoor environments due to lack of autonomy in
mapping.
Wang et al. [64] outlined a hybrid method for thermal modelling
using LiDAR system using existing building envelopes. A 3D laser
scanner and an IR camera were used to map and produce 3D thermo­
graphic models from fixed scans. However, due to the huge amount of
concurrent and post-processing with laser scanning, this thermographic Fig. 10. An indoor orthothermogram of a hall shown from outside [66].
3D modelling approach cannot be applied in real-time. This limitation

Fig. 9. (a) The process to create 3D orthothermograms for BIM [63] (b) Thermographic 3D Model (left) and its corresponding RGB 3D Model (right) [63].

8
M.H. Shariq and B.R. Hughes
However, with recent developments in photogrammetric mapping,
the 3D modelling technique through laser scanning can be replaced with
photogrammetry in order to provide a more real-time and automated
scope to thermographic BIM methodology [67]. Aguilera et al. [68]
investigated a method to produce 3D thermal models to inspect building
façades without the need for laser scanning. Thermographic images
were mapped irrespective of their affine deformations such as scale,
rotation, translation, tilt angle, etc. The resulted thermographic 3D
model displayed both metric and thermal information. This novel
technique allows a remote low-cost thermal inspection with high reso­
lution and accuracy of point clouds (with ground sampling distance
(GSD) of 15 mm), which can be further improved with a higher DPI
camera to match or even exceed the typical resolution from a LiDAR
result. The authors also highlighted a forthcoming challenge, which is to
apply this framework to be performed using an unmanned aerial vehicle
(UAV) to enable thermal inspection of the complete building envelope,
especially for complex buildings structures.
Cao et al. [69] studied a real-time approach for thermographic 3D
modelling by using an integrated depth and thermal sensor. Using an IR
sensor and an RGB-D Kinect sensor [37], the authors implemented
camera pose estimation and pixel alignment techniques to produce
radiometric thermographic 3D models of the scanned objects (such as a
human, chair, mug, etc.). The results present high-resolution radio­
metric information (�2 � C) with a range of 0� C–400 � C. However, apart
from a potential addition of mapped RGB imagery, further improve­
ments are needed, especially towards increasing the range of sensing
and mapping of walls, where changes in pixel information may be
minimal, in order to enable such framework for building modelling and
inspection in real-time.
Müller and Kroll [70] also investigated a similar technique that
utilised a depth camera, RGB camera, and a thermal camera to produce
3D point clouds texturised with thermal data. The authors performed
experiments with a variety of conditions (such as sensing distance, re­
flections, thermal emissivity, camera movements/speeds, etc.) that
could affect the accuracy of thermal 3D reconstruction through a
hand-held 3D scanning approach. The results showed that the (1) rapid
movement of the camera during mapping, (2) exceeding sensing dis­
tance of over 50 m, and (3) non-perpendicular thermal sensing, all
caused considerable errors and distortions in 3D reconstruction and/or
its overlaid thermal representation. Although a close-ranged, slow-­
paced, normal-to-surface scan, would produce a high-fidelity 3D ther­
mal reconstruction, this technique cannot be applied for building
inspections as it will be immensely time-consuming and daunting.
Similarly, Sels et al. [71] also achieved highly accurate results of thermal
superimposition on 3D mapped models. The authors used structured
light method (through a fixed projector) and an RGB-Depth camera to
produce an accurate 3D reconstruction of various objects. Despite ac­
curate thermographic 3D models, this technique is not applicable in
real-time 3D model reconstruction since the structured light method
requires stationary (fixed camera) scans for mapping.
A leading-edge development by Vidas and Moghadam [72] back in
Fig. 11. 3D reconstruction of an HVAC system showing (a) superimposed radiometric data with colour and only RGB colour model (b) scanned using HeatWave [72].
(For interpretation of the references to colour in this figure legend, the reader is referred to the Web version of this article.)
9
Renewable and Sustainable Energy Reviews 130 (2020) 109979
2013 highlighted a novel solution to real-time thermal inspections for
energy audits. The authors presented a handheld device, called Heat­
Wave, capable of recording thermal and digital imagery while creating a
photogrammetric 3D model of the scanned visuals in real-time. The
light-weight handheld device consisted of an IR range sensor, RGB
camera and a thermal camera and as it maps in real-time it enabled users
to view an overlay of thermal highlights of the 3D reconstruction with
and without the colour imagery. An example of which is shown in
Fig. 11. This system is evidently a highly efficient solution to an energy
audit of buildings in real-time. However, with the limitation of the
physical reach of human, this device is not suitable for thermal inspec­
tion of a high-rise exterior of building structures or out-of-reach high
indoor areas. A remotely controlled Unmanned Aerial Vehicle (UAV)
shall overcome this limitation.
In conclusion, orthothermography (or thermographic 3D modelling)
allows an easy, accessible, accurate, and a cost-effective approach to
study building envelopes for detecting defects [69,73]. The two common
methods to produce these thermal 3D models include point cloud
scanning through a (1) laser scanner, such as a LiDAR [10,64–66], or (2)
an IR depth sensor, such as an RGB-D sensor [69–72]. The latter method
has been more widely used in the recent years [67] due to considerable
advancements in computing power and photogrammetry (as discussed
in Section 2.2) and much lower cost than a LiDAR-based solution [43].
Ferna�ndez et al. [68] demonstrated most relevance due to the real-time
and remote approach to produce orthothermogram, along with frame­
works by Aguilera et al. [68] and Cao et al. [69] that presented a
low-cost real-time building inspection approach through
photogrammetry-based 3D thermal modelling, however, none of the
techniques showed potential to thermal map and inspect a large and
complex 3D building in a time-efficient way. Hence, there is an evident
need and scope for non-handheld, aerial and ideally automated photo­
grammetric approach for a time and cost-efficient real-time building
energy inspection and automated fault diagnosis.
2.6. Damage detection through computer vision and machine learning
This section will review recent investigations and developments to­
wards building interior/exterior and/or civil infrastructure inspection
(for defect detection) through image-processing techniques, essentially
with the help of machine learning algorithms. A large number of studies
have been carried out in recent years for the classification of defects
from digital image data using computer vision and ANN (Artificial
Neural Networks). Although computer vision techniques for remote
building inspection have been investigated for more than two decades
[74], due to more recent advances in computing power and handling of
big data, CNNs (Convolutional Neural Networks) have been the back­
bone of most computer-vision decision making. Current developments
and researches that apply CNNs with computer vision for defect detec­
tion primarily aim to automate the process in order to reduce time and
cost of inspections [75].
Xu et al. [76] investigated a framework to detect cracks in metal
M.H. Shariq and B.R. Hughes
surfaces through infrared thermal image-processing. Previously, detec­
tion of cracks on metal was performed manually by observing a com­
plete thermal video, which was extremely time-consuming. The studied
method used superpixel algorithm to filter and process a raw IR thermal
imagery and identify cracks on metal surfaces automatically and at high
speed. The authors proposed that this framework has a scope to be
applied to other materials and applications to allow automatic, accurate
and time-efficient inspections.
Galantucci and Fatiguso [77] proposed a non-destructive survey
technique for historical buildings using digital photogrammetry,
image-processing and 3D surface model reconstruction. The authors
used digital image processing techniques with an edge-extraction filter
to identify cracks, lacks and erosions on scanned surfaces. The authors
concluded that it was impossible to automate the fault identification
process without human intervention for validation. This is due to the
environmental factors, physical variables, inconsistencies in defect im­
ages, etc. Hence, it is evident that machine learning techniques (such as
ANNs) need to be implemented in order to automate the fault identifi­
cation process (to detect moisture and heat losses) with high accuracy.
Similarly, Wang et al. [78] investigated and developed a
cost-effective mobile-enabled deep-learning-based automatic damage
detection of historic buildings. CNNs were first trained for 30,000 iter­
ations and 2.5 h with 500 images of a variety of experimental cases that
included spalling and efflorescence. High accuracy of CNN output with
reliable and consistent identification was achieved to detect and locate
spalling and fluorescence damages instantly. The damages on the sur­
face/façade seen through a mobile (smartphone) camera were detected
and identified by CNN algorithm in real-time. These damage types were
highlighted and displayed on the mobile screen as shown in Fig. 12. This
instantaneous, cost-effective automatic fault detection system is highly
applicable and particularly advantageous for a potential real-time and
roving fault detection for time- and cost-efficient qualitative building
inspection at a large-scale.
Hoskere et al. [79] examined a vision-based post-earthquake struc­
tural inspection of civil infrastructure through CNN. The automated
defect classification technique through computer-vision could identify
spalling, corrosion, and various type of cracks in a structure with an
accuracy of 86.7%. The authors suggested that this approach can be
deployed on a UAV for an automatic and real-time structural inspection
to reduce both time and cost during a large-scale inspection.
In relation to saving building energy, a novel deep learning-based
fault detection approach to detect faults in a refrigerant flow air-
conditioning system was investigated by Guo et al. [80] that per­
formed an unsupervised fault identification framework with high ac­
curacy. Hence, trained CNNs can certainly be used for detection of a
larger variety of damages (using larger training data set) in existing
Fig. 12. Damages identified by CNN and displayed in real-time through a smartphone [78].
10
Renewable and Sustainable Energy Reviews 130 (2020) 109979
building envelopes, plus, the real-time wireless mobile-based damage
detection feature notably allows the feasibility of a drone-based
remotely controlled aerial inspection as well.
In conclusion, machine learning with image processing approach for
fault detection promises to provide an accurate, time-efficient, and
automated means of fault detection in buildings and other structures.
Since ANNs can be trained to provide meaningful output from any set
and form of data, there is virtually no limitation on the applications of
this methodology. As also argued by Refs. [14,15], building inspections
demand such automation to speed up the process as well as reduce the
cost of inspections for large buildings. The innovative solution by Hos­
kere et al. [79] shows a significant scope in a real-time, automatic and
remote building energy inspection, provided the potential addition of
‘real-time 3D model reconstruction’ and ‘aerial inspection’.
The above sections studied various techniques that can be useful for
building inspection in terms of large-scale efficiency. Table 1 presents a
synthetic summary of the advantages and disadvantages of these tech­
nologies and methods if integrated with an IRT-based inspection.
3. Results and discussion
3.1. Research focus within building inspections
The state-of-the-art methods and technologies reviewed in chapter 2
focused on diverse applications. However, in context of building or
structural inspections, only three studies showed IRT-based inspection
integrated with other technology to enhance the speed or efficiency of
the inspection. For example, an indoor thermal inspection performed via
autonomous mobile robot [66], a real-time and automatic structural
inspection driven by machine learning [79], damage detection of
bridges through drones [59].
Since, this review is essentially focused on a larger-scale problem
with the demand to rapidly cut down CO2 emissions from the overall
non-residential building sector of the EU, 26 recent research papers from
year 2012–2019 relevant to building (and/or structural) inspection
application [10,11,13,16,18,19,46,59,61–66,68,69,71,72,76–82] are
evaluated through a radar chart in Fig. 13 that shows the areas that each
of these researches are focused on.
As seen from the graph, the common features of a building inspection
system include thermographic 3D representation, 3D model recon­
struction and real-time inspection. However, very less focus has been
there towards real-time 3D model reconstruction, drone-based inspec­
tion and integration of automation or machine learning frameworks.
Addressing these areas and incorporating them in building inspection
will massively help in building energy performance at a very large-scale,
as it will allow cost and time-efficient inspection of entire buildings
M.H. Shariq and B.R. Hughes Renewable and Sustainable Energy Reviews 130 (2020) 109979

Table 1
Advantages and disadvantages of integrating various surveying methods with IRT-based inspection.
Technology/Method Advantages Disadvantages

Laser Scanning � High-resolution point cloud � High cost

Monocular Photogrammetry
Stereo-photogrammetry
RGB-D Photogrammetry
Simultaneous Localisation and
Mapping (SLAM)
Drones
Thermographic 3D Modelling
Computer vision and machine
learning
� Accurate model reconstruction
� Low-cost (single camera)
� Lesser equipment makes it suitable for aerial surveying
� Less computationally demanding than laser scanning
� Suitable for real-time 3D mapping applications
� Less computationally intensive
� More accurate results than other photogrammetry methods
� Plain walls are easily detected thanks to IR depth sensor.
� Can operate in low-light conditions
� Allows real-time 3D mapping
� Provides scope for automation (e.g. drone-based obstacle avoid­
ance in unknown indoor environments while mapping)
� Allows autonomous real-time 3D mapping capability
� Able to capture areas inaccessible by humans (hidden indoor
regions, façades of high-rise buildings)
� Allows swift detection of faults with precise location
� Allows quantitative analysis and diagnosis of faults.
� Scope for automating fault detection and identification
� Less human resources (including thermographic inspectors)
required
� Not suitable for aerial mapping
� Lower resolution (as compared to LiDAR results)
� Plain featureless walls are reconstructed inaccurately
� Not suitable in low-light conditions
� Close ranged mapping produces parallax effects that result in inaccurate
model reconstruction
� Relatively high computationally intensive
� Noise (from IR depth sensing) require for further post-processing of
results
� Existing methods rely on laser scanning, which is not suitable for aerial
surveying.
� Switching to photogrammetry will affect reliability and accuracy of
localisation
� Flight time, especially when equipped with additional mapping
hardware (lasers/IR depth sensors/cameras/thermal cameras/etc)
� Superimposing thermal imagery onto point clouds is computationally
demanding and time-consuming
� Large amounts of training and testing data need to be prepared for a
vast variety of faults

regardless of their size. limitations of camera-based mapping (especially of building envelopes

3.2. Analysis techniques
It has been observed that most of the recent researches had been
focused largely on quantitative measurements in thermal inspections.
Fig. 14 shows the percentage distribution of qualitative vs. quantitative
methods used by recent papers on fault detection in buildings [9].
However, for an extensive energy audit of entire buildings, more
research needs to emerge focussing on a qualitative approach to find and
locate faults in building envelopes in the least amount of time and
extract a projected intensity of each to determine which faults need
greater or urgent attention. Addition of automation in qualitative in­
spections may eliminate the requirement of a hindering 32 h of training-
based certification criteria prescribed by the standards ISO 18436 [83]
and ISO 6781 [84] for a level 1 IRT-based inspection and evaluation of
its results [16]. By all means, quantitative analysis will most often be
required and should be applied for a fault to be assessed and diagnosed
accurately. However, research priority must now shift towards
increasing efficiency in qualitative techniques to allow more automated
and intuitive interpretation of thermographic data for swift large-scale
inspections.
3.3. LiDAR vs. state-of-the-art photogrammetry
Although LiDAR-based 3D mapping provides a highly robust, accu­
rate and dense point cloud [85], application of LiDAR for building in­
spections should be questioned. As this review emphasises on a rapid
cost-effective solution for large scale building energy audit, one must
argue on the requirement of a very high-resolution 3D model as a trade
off with lower resolution but rapid and real-time inspection for
large-scale building inspection purposes. Thanks to the recent
computing developments, the state-of-the-art photogrammetry tech­
niques have shown extremely promising results with relatively
high-density point cloud with textured RGB data overlaid achievable in
real-time [86–89]. Digital photogrammetry shows more scope in
real-time mapping than using LiDAR scanners in many applications [90]
including rapid model reconstruction and drone-based mapping.
Photogrammetry, unlike LiDAR-based 3D mapping, is not only less
computationally demanding and relatively faster, but also cost-effective.
Monocular photogrammetry techniques allow remote sensing and real-
time mapping ability using a simple camera. One of the common
and indoor corridors) is to perform accurately against a plain wall that
has no distinct features [91,92]. However, an integration of IMU can
eliminate errors of visual odometry (by supplementing IMU-based cor­
rections) and help in more accurate image stitching, especially during
cornering against plain walls. Furthermore, supplementary laser pat­
terns can be projected at any untextured wall or surface to help with
image stitching, as presented by Y.F. Xu [93] and Liu et al. [94].
Considering the recent leading-edge developments in photogram­
metry [24], building inspection through real-time photogrammetry can
surely save a substantial amount of time and cost of building inspection
for the applications that require 3D model reconstructions with an
overlaid radiometric thermal imagery. Hence, in a large-scale context, a
high-speed photogrammetric model reconstruction is a favourable
trade-off against a higher-resolution laser-based reconstruction for
building inspections.
3.4. Advancements in real-time thermographic 3D modelling
Thanks to the open-source research contributions in photogram­
metry [95–102] and the relevant open-source computer-vision (and/or
photogrammetry) software frameworks [103–105], there is a promising
scope of feature matching algorithms that can be applied to pairs of
thermal images that may help achieve ‘thermographic photogrammetry’
possibilities. To date, thermal images are being superimposed to an
existing point-cloud of a 3D model. However, the high-resolution ther­
mal images are inevitably reduced to the resolution of point clouds that
causes partial omission of useful thermographic data.
An opportunity exists to stitch thermal imagery with colour images
with a pixel-wise accuracy to produce an orthothermogram using SfM
instead of overlaying the two sets of images (or texturising the point
clouds) discretely. A similar technique has been successfully shown by
Gan et al. [106] through homography transformation that uses regional
edge features extracted from stereo colour imagery that corresponds
accurately on the thermal imagery that contains those features.
Although the authors implemented this technique for an agricultural
application, overlaying thermal images on an edge-feature representa­
tion of a corresponding colour image shows strong potential in photo­
grammetry for building inspections as it eliminates the post-processes of
superimposing thermal imagery to create a high-quality
orthothermogram.
Real-time thermographic 3D modelling can enable a speedy

11
M.H. Shariq and B.R. Hughes
Fig. 13. Number of papers from the year 2012–2019 relevant to building inspection containing each feature.
Fig. 14. Distribution of quantitative vs qualitative methods used in fault
detection by recent papers [9].
quantitative inspection of façades and indoors. However, to achieve the
inspection in real-time, existing thermographic 3D mapping techniques
[69–72] shall be switched to monocular photogrammetry [28,30] to
reduce real-time computational processing of the depth sensor (or laser
scanner).
3.5. Integration of drones, automation and AI
Among the trending and fruitful technological developments, drones
and machine learning have revolutionised automation for many in­
dustries at a whole new level, particularly during the last decade. The
recent integration of drones for building energy audit applications has
shown positive results [17]. Therefore, the use of drones for inspections
provide a promising scope in reducing the time (and hence the cost) of
building energy audit, especially for large buildings, as well as over­
coming the problem with inaccessible areas. However, drones for 3D
mapping and thermographic surveying purposes are equipped with
necessary hardware, which would reduce its flight time drastically.
Hence, the selection for appropriate type of inspection method and
equipment (for example: Monocular photogrammetry as oppose to laser
scanning) is imperative for a drone-based thermographic mapping and
inspection.
Likewise, machine learning techniques have shown impressive re­
sults, both with data-driven and knowledge-driven methods [14].
However, from the analysis in section 3.1, only 3 out of 26 studies (12%)
related to building or structural inspection focused on artificial intelli­
gence (AI) for fault detection in the last 8 years. Hence, although
incorporating machine learning system would require a massive amount
of time and data, a gap and an opportunity definitely exist for an
automated building inspection application.
In conclusion, UAVs (drones) along with AI-based fault detection
12
Renewable and Sustainable Energy Reviews 130 (2020) 109979
diagnosis can provide an unsupervised autonomous inspection of
buildings that can assist significantly in identifying faults and conse­
quently aiding in reducing overall CO2 emissions from buildings far
more efficiently.
3.6. Challenges and potential solutions
Integrating technologies and retrofitting inspection methods with
IRT, such as thermographic 3D modelling using monocular photo­
grammetry through a drone, does indeed promise to be a time- and cost-
effective solution to building energy inspection. However, there are
several methodological challenges anticipated from this potential
solution.
Firstly, the on-board processor/computer mounted on the drone (e.g.
a quadcopter) will be required to capture and process an enormous
amount of digital data in real-time, that includes recording and accu­
mulating high-quality visual feed from visible and thermal cameras,
storing a corresponding depth map data from IR sensors (or image-
stitching thousands of feature points in real-time), 3D reconstruction
using point clouds, visual odometry-based loop closures, IMU-aided
corrections, superimposing thermal imagery, apart from the live trans­
mission to the on-ground computer. A powerful processor can achieve
this, though, it would also consume more battery power hence reducing
the flight time of the drone. However, with the nearing standardisation
of fifth-generation (5G) communication technology with exceptionally
high bandwidth, much-increased connection density and reduced en­
ergy consumption that is ideally suited for IoT applications [107], most
of the computationally burdensome processes mentioned above can
easily be eliminated. The on-board processor of the drone shall offload
the ‘Big Data’ near-instantly in real-time through 5G to a high per­
forming recipient computer (or on cloud computing servers) to carry out
all the resource-intensive computation in real-time including 3D model
reconstruction, thermal modelling, energy modelling calculations, etc.
for a thorough building energy audit.
Secondly, implementation of machine learning algorithms for fault
detection (or identification), whether real-time or offline, can be highly
time-consuming due to the need for large training data set, manual trials
and experimentations. However, research focus can instead initially be
made towards qualitative analysis instead of a quantitative one as the
latter would require more precise and larger training data to produce an
accurate fault identification or measurement system [14]. Whereas,
qualitative fault detection would require simpler computer vision al­
gorithms with a relatively much lesser amount of sample training data
M.H. Shariq and B.R. Hughes
that can enable an unsupervised automatic detection of common faults
in building envelopes.
Thirdly, the average maximum flight time of state-of-the-art pro­
fessional and commercial drones is merely 25–30 min, which may cause
hindrance for large-scale aerial mapping and inspection. Hence, a hybrid
mobility can be considered, such as an addition of a wheeled mobile
robot (such as [40,46,47,66]) designated to perform thermographic
photogrammetry of the accessible lower areas, and also act as a charging
base for the drone. This would allow the drone to be used for longer
periods as it would be required only for the inaccessible high areas. This
will also enable a more complete 3D map of an indoor environment as
the ground robot will capture areas that are inaccessible or unsafe for a
drone to manoeuvre around and vice-versa. Optionally, the drone can be
programmed to land on the autonomous mobile robot platform for
self-charging. Additionally, to improve the time and speed of 3D
reconstruction, instead of still photos, a drone could record videos of the
building envelopes at a higher speed, whether indoor or exterior, from
both RGB and thermal cameras. After the necessary image processing
and feature extraction, the pair of unblur images (extracted from the
video) can then be used to reconstruct a superimposed 3D thermo­
graphic model through SfM method.
4. Conclusions and recommendations
The objective of this constructive review was primarily to highlight
the potential opportunities that exist in retrofitting building energy in­
spection techniques that can help in reducing excess carbon emissions at
a large-scale and rapidly improving building energy performances. A
variety of recent papers from 2012 to 2019 were reviewed related to
state-of-the-art technologies and trending computational methodologies
that can be integrated towards energy inspection of buildings including
photogrammetry, SLAM, LiDAR mapping, thermographic 3D modelling,
drone surveys and AI. Integrating these technologies with IRT-based
building inspections allow a swift, efficient and less manually inten­
sive audit of buildings at a large-scale.
The key findings of this review paper are summarised as follows:
� There has been an evident lack of research and development towards
integrated or hybrid solutions that could combine and utilise a va­
riety of Industry 4.0 technologies with IRT.
� Monocular photogrammetry showed highly promising scope thanks
to its minimal hardware requirement and relatively low computa­
tional processing time for mapping [89].
� Thermographic 3D model can be generated by superimposing ther­
mal camera imagery with monocular photogrammetry (during 3D
mapping). This allows a real-time 3D reconstruction of building en­
velopes for BEM and quantitative energy analysis.
� Integration of drones with IRT and 3D mapping allows an unre­
stricted aerial survey with qualitative inspection in real-time, and a
complete 3D model of the building (or room) for a quantitative
building energy modelling [17].
� LiDAR (or laser scanners) are accurate and highly reliable for a single
façade inspection or a high-resolution terrestrial scanning [24,85].
However, in the context of large-scale or high-speed building sur­
vey/inspection, LiDAR based thermography can be excessively
time-consuming, computationally resource intensive [18].
� With the advancements in digital photography including high-
definition thermal cameras, the real-time data processing during
remote (aerial) mapping can be computationally demanding. How­
ever, 5G-based data communication is vital to overcome this po­
tential restriction [107].
� Simultaneous Localisation and Mapping (SLAM) techniques rely on
LiDAR-based mapping and navigation, which prevents real-time
dense reconstruction through a drone. Switching to photogram­
metric mapping can ease computational requirements and allow
semi-automatic real-time aerial mapping with autonomous obstacle
13
Renewable and Sustainable Energy Reviews 130 (2020) 109979
avoidance capability. Further research is required to assess the ac­
curacy and reliability of localisation using photogrammetric
methods.
� Artificial Intelligence (AI) promises a scope for automated fault
detection and analysis in the near future based on the recent de­
velopments [14,15,77,78] as reviewed in chapter 2. Qualitative in­
spections of building façades can potentially be developed and
performed near-autonomously through a drone with minimum
human involvement [79]. This would reduce the number of certified
inspectors required during a large-scale survey.
Declaration of competing interest
The authors declare that they have no known competing financial
interests or personal relationships that could have appeared to influence
the work reported in this paper.
Acknowledgement
The support by the Mechanical and Aerospace Engineering depart­
ment of Strathclyde University is gratefully acknowledged. The state­
ments made herein are solely the responsibility of the authors.
References
[1] de Wilde P. Ten questions concerning building performance analysis. Build
Environ 2019. https://doi.org/10.1016/j.buildenv.2019.02.019.
[2] Alencastro J, Fuertes A, Fox A, De Wilde P. The impact of defects on energy
performance of buildings: quality management in social housing developments.
Energy Procedia 2019;158:4357–62. https://doi.org/10.1016/j.
egypro.2019.01.784. Elsevier Ltd.
[3] Energy efficient buildings | Energy n.d. https://ec.europa.eu/energy/topi
cs/energy-efficiency/energy-efficient-buildings_en?redir¼1. [Accessed 24
September 2019].
[4] Li Y, Kubicki S, Guerriero A, Rezgui Y. Review of building energy performance
certification schemes towards future improvement. Renew Sustain Energy Rev
2019;113:109244. https://doi.org/10.1016/j.rser.2019.109244.
[5] Energy consumption by end-use | Energy n.d (accessed April 21, 2020), https
://ec.europa.eu/energy/content/energy-consumption-end-use_en?redir¼1.
[6] Heating and cooling | Energy n.d. https://ec.europa.eu/energy/topics/energy-
efficiency/heating-and-cooling_en?redir¼1. [Accessed 21 April 2020].
[7] Energy performance of buildings directive | Energy n.d (accessed April 21, 2020),
https://ec.europa.eu/energy/topics/energy-efficiency/energy-efficient-building
s/energy-performance-buildings-directive_en?redir¼1.
[8] Fox M, Coley D, Goodhew S, De Wilde P. Thermography methodologies for
detecting energy related building defects. Renew Sustain Energy Rev 2014.
https://doi.org/10.1016/j.rser.2014.07.188.
[9] Kirimtat A, Krejcar O. A review of infrared thermography for the investigation of
building envelopes: advances and prospects. Energy Build 2018. https://doi.org/
10.1016/j.enbuild.2018.07.052.
[10] Lagüela S, Armesto J, Arias P, Herr�aez J. Automation of thermographic 3D
modelling through image fusion and image matching techniques. Autom
ConStruct 2012. https://doi.org/10.1016/j.autcon.2012.05.011.
[11] Fox M, Goodhew S, De Wilde P. Building defect detection: external versus internal
thermography. Build Environ 2016. https://doi.org/10.1016/j.
buildenv.2016.06.011.
[12] Barreira E, Almeida RMSF, Moreira M. An infrared thermography passive
approach to assess the effect of leakage points in buildings. Energy Build 2017.
https://doi.org/10.1016/j.enbuild.2017.02.009.
[13] Cai W, Wen X, Wang S, Wang L. A real-time detection method of building energy
efficiency based on image processing. J Vis Commun Image Represent 2019.
https://doi.org/10.1016/j.jvcir.2019.02.032.
[14] Zhao Y, Li T, Zhang X, Zhang C. Artificial intelligence-based fault detection and
diagnosis methods for building energy systems: advantages, challenges and the
future. Renew Sustain Energy Rev 2019. https://doi.org/10.1016/j.
rser.2019.04.021.
[15] Kim W, Katipamula S. A review of fault detection and diagnostics methods for
building systems. Sci Technol Built Environ 2018. https://doi.org/10.1080/
23744731.2017.1318008.
[16] Lucchi E. Applications of the infrared thermography in the energy audit of
buildings: a review. Renew Sustain Energy Rev 2018. https://doi.org/10.1016/j.
rser.2017.10.031.
[17] Rakha T, Gorodetsky A. Review of Unmanned Aerial System (UAS) applications in
the built environment: towards automated building inspection procedures using
drones. Autom ConStruct 2018. https://doi.org/10.1016/j.autcon.2018.05.002.
[18] Garwood TL, Hughes BR, O’Connor D, Calautit JK, Oates MR, Hodgson T.
A framework for producing gbXML building geometry from Point Clouds for
M.H. Shariq and B.R. Hughes
accurate and efficient Building Energy Modelling. Appl Energy 2018. https://doi.
org/10.1016/j.apenergy.2018.04.046.
[19] Son H, Lee S, Kim C. Automated 3D model reconstruction to support energy-
efficiency. Procedia Eng 2016. https://doi.org/10.1016/j.proeng.2016.04.046.
[20] Ochmann S, Vock R, Wessel R, Klein R. Automatic reconstruction of parametric
building models from indoor point clouds. Comput Graph 2016. https://doi.org/
10.1016/j.cag.2015.07.008.
[21] Ochmann S, Vock R, Klein R. Automatic reconstruction of fully volumetric 3D
building models from oriented point clouds. ISPRS J Photogrammetry Remote
Sens 2019. https://doi.org/10.1016/j.isprsjprs.2019.03.017.
[22] Song Z, Tang S, Gu F, Shi C, Feng J. DOE-based structured-light method for
accurate 3D sensing. Optic Laser Eng 2019. https://doi.org/10.1016/j.
optlaseng.2019.02.009.
[23] Park C, Moghadam P, Kim S, Elfes A, Fookes C, Sridharan S. Elastic LiDAR fusion:
dense map-Centric continuous-time SLAM. Proc. - IEEE Int. Conf. Robot. Autom.
2018. https://doi.org/10.1109/ICRA.2018.8462915.
[24] El-Din Fawzy H. 3D laser scanning and close-range photogrammetry for buildings
documentation: a hybrid technique towards a better accuracy. Alexandria Eng J
2019;58:1191–204. https://doi.org/10.1016/j.aej.2019.10.003.
[25] Tang P, Vick S, Chen J, German Paal S. Surveying, geomatics, and 3D
reconstruction. Infrastruct. Comput. Vis.. Elsevier; 2020. p. 13–64. https://doi.
org/10.1016/b978-0-12-815503-5.00002-4.
[26] Colomina I, Molina P. Unmanned aerial systems for photogrammetry and remote
sensing: a review. ISPRS J Photogrammetry Remote Sens 2014. https://doi.org/
10.1016/j.isprsjprs.2014.02.013.
[27] Kaufman J, Rennie AEW, Clement M. Single camera photogrammetry for reverse
engineering and fabrication of ancient and modern artifacts. Procedia CIRP 2015.
https://doi.org/10.1016/j.procir.2015.01.073.
[28] Pizzoli M, Forster C, Scaramuzza D. REMODE: probabilistic, monocular dense
reconstruction in real time. Proc. - IEEE Int. Conf. Robot. Autom. 2014. https://
doi.org/10.1109/ICRA.2014.6907233.
[29] REMODE: Probabilistic. Monocular dense reconstruction in real time - YouTube
n.d. https://www.youtube.com/watch?v¼QTKd5UWCG0Q. [Accessed 21 April
2020].
[30] Sch€ops T, Sattler T, H€ane C, Pollefeys M. Large-scale outdoor 3D reconstruction
on a mobile device. Comput Vis Image Understand 2017. https://doi.org/
10.1016/j.cviu.2016.09.007.
[31] Krutikova O, Sisojevs A, Kovalovs M. Creation of a depth map from stereo images
of faces for 3D model reconstruction. Procedia Comput. Sci. 2016. https://doi.
org/10.1016/j.procs.2017.01.159.
[32] Fu K, Xie Y, Jing H, Zhu J. Fast spatial–temporal stereo matching for 3D face
reconstruction under speckle pattern projection. Image Vis Comput 2019. https://
doi.org/10.1016/j.imavis.2019.02.007.
[33] Pire T, Fischer T, Castro G, De Crist�oforis P, Civera J, Jacobo Berlles J. S-PTAM:
Stereo Parallel tracking and mapping. Robot Autonom Syst 2017. https://doi.org/
10.1016/j.robot.2017.03.019.
[34] Hamzah RA, Kadmin AF, Hamid MS, Ghani SFA, Ibrahim H. Improvement of
stereo matching algorithm for 3D surface reconstruction. Signal Process Image
Commun 2018. https://doi.org/10.1016/j.image.2018.04.001.
[35] Hadfield S, Lebeda K, Bowden R. Stereo reconstruction using top-down cues.
Comput Vis Image Understand 2017. https://doi.org/10.1016/j.
cviu.2016.08.001.
[36] Namitha N, Vaitheeswaran SM, Jayasree VK, Bharat MK. Point cloud mapping
measurements using kinect RGB-D sensor and kinect fusion for visual odometry.
Procedia Comput. Sci. 2016. https://doi.org/10.1016/j.procs.2016.06.044.
[37] Kinect XBOX n.d. https://www.xbox.com/en-US/kinect. [Accessed 21 April
2020].
[38] Zhang Z. Microsoft kinect sensor and its effect. IEEE Multimed 2012. https://doi.
org/10.1109/MMUL.2012.24.
[39] C�ordova-Esparza DM, Terven JR, Jim� enez-Hern� andez H, Herrera-Navarro AM.
A multiple camera calibration and point cloud fusion tool for Kinect V2. Sci
Comput Program 2017. https://doi.org/10.1016/j.scico.2016.11.004.
[40] Liu Y, Zhu M, Zhang H. Processed RGB-D slam using open-source software. Proc. -
2017 IEEE Int. Conf. Comput. Sci. Eng. IEEE/IFIP Int. Conf. Embed. Ubiquitous
Comput. CSE EUC 2017 2017. https://doi.org/10.1109/CSE-EUC.2017.115.
[41] Zollh€ofer M, Stotko P, G€
orlitz A, Theobalt C, Nießner M, Klein R, et al. State of the
art on 3D reconstruction with RGB-D cameras. Comput Graph Forum 2018.
https://doi.org/10.1111/cgf.13386.
[42] Guclu O. Can AB. k-SLAM: a fast RGB-D SLAM approach for large indoor
environments. Comput Vis Image Understand 2019. https://doi.org/10.1016/j.
cviu.2019.04.005.
[43] Jalandoni A, Domingo I, Taçon PSC. Testing the value of low-cost Structure-from-
Motion (SfM) photogrammetry for metric and visual analysis of rock art.
J Archaeol Sci Reports 2018;17:605–16. https://doi.org/10.1016/j.
jasrep.2017.12.020.
[44] Bandini F, Sunding TP, Linde J, Smith O, Jensen IK, K€ oppl CJ, et al. Unmanned
Aerial System (UAS) observations of water surface elevation in a small stream:
comparison of radar altimetry, LIDAR and photogrammetry techniques. Remote
Sens Environ 2020;237:111487. https://doi.org/10.1016/j.rse.2019.111487.
[45] Dornaika F, Chung R. Mosaicking images with parallax. Signal Process Image
Commun 2004;19:771–86. https://doi.org/10.1016/j.image.2004.06.008.
[46] Borrmann D, Nüchter A, Dakulovi�c M, Maurovi�c I, Petrovi�c I, Osmankovi�c D,
et al. A mobile robot based system for fully automated thermal 3D mapping. Adv
Eng Inf 2014. https://doi.org/10.1016/j.aei.2014.06.002.
14
Renewable and Sustainable Energy Reviews 130 (2020) 109979
[47] Kim P, Chen J, Cho YK. SLAM-driven robotic mapping and registration of 3D
point clouds. Autom ConStruct 2018. https://doi.org/10.1016/j.
autcon.2018.01.009.
[48] Yang S, Scherer SA, Yi X, Zell A. Multi-camera visual SLAM for autonomous
navigation of micro aerial vehicles. Robot Autonom Syst 2017. https://doi.org/
10.1016/j.robot.2017.03.018.
[49] Zhang J, Singh S. Low-drift and real-time lidar odometry and mapping. Aut
Robots 2017. https://doi.org/10.1007/s10514-016-9548-2.
[50] Zhang J, Singh S. Laser–visual–inertial odometry and mapping with high
robustness and low drift. J Field Robot 2018. https://doi.org/10.1002/
rob.21809.
[51] Gallardo-Saavedra S, Hern� andez-Callejo L, Duque-Perez O. Technological review
of the instrumentation used in aerial thermographic inspection of photovoltaic
plants. Renew Sustain Energy Rev 2018. https://doi.org/10.1016/j.
rser.2018.05.027.
[52] Liardon JL, Hostettler L, Zulliger L, Kangur K, Gujja Shaik NS, Barry DA. Lake
imaging and monitoring aerial drone. HardwareX 2018. https://doi.org/
10.1016/j.ohx.2017.10.003.
[53] Gomes I, Peteiro L, Bueno-Pardo J, Albuquerque R, P� erez-Jorge S, Oliveira ER,
et al. What’s a picture really worth? On the use of drone aerial imagery to
estimate intertidal rocky shore mussel demographic parameters. Estuar Coast
Shelf Sci 2018. https://doi.org/10.1016/j.ecss.2018.08.020.
[54] Almeida DRA, Broadbent EN, Zambrano AMA, Wilkinson BE, Ferreira ME,
Chazdon R, et al. Monitoring the structure of forest restoration plantations with a
drone-lidar system. Int J Appl Earth Obs Geoinf 2019. https://doi.org/10.1016/j.
jag.2019.03.014.
[55] Zorbas D, Di Puglia Pugliese L, Razafindralambo T, Guerriero F. Optimal drone
placement and cost-efficient target coverage. J Netw Comput Appl 2016. https://
doi.org/10.1016/j.jnca.2016.08.009.
[56] Urbanov� a P, Jurda M, Vojtí�sek T, Krajsa J. Using drone-mounted cameras for on-
site body documentation: 3D mapping and active survey. Forensic Sci Int 2017.
https://doi.org/10.1016/j.forsciint.2017.10.027.
[57] Kumar NM, Sudhakar K, Samykano M, Jayaseelan V. On the technologies
empowering drones for intelligent monitoring of solar photovoltaic power plants.
Procedia Comput. Sci. 2018. https://doi.org/10.1016/j.procs.2018.07.087.
[58] Eschmann C, Kuo CM, Kuo CH, Boller C. Unmanned aircraft systems for remote
building inspection and monitoring. Proc. 6th Eur. Work. - Struct. Heal. Monit.
2012, EWSHM 2012 2012.
[59] Seo J, Duque L, Wacker J. Drone-enabled bridge inspection methodology and
application. Autom ConStruct 2018. https://doi.org/10.1016/j.
autcon.2018.06.006.
[60] Gao M, Xu X, Klinger Y, Van Der Woerd J, Tapponnier P. High-resolution
mapping based on an Unmanned Aerial Vehicle (UAV) to capture paleoseismic
offsets along the Altyn-Tagh fault, China. Sci Rep 2017. https://doi.org/10.1038/
s41598-017-08119-2.
[61] Entrop AG, Vasenev A. Infrared drones in the construction industry: designing a
protocol for building thermography procedures. Energy Procedia 2017. https://
doi.org/10.1016/j.egypro.2017.09.636.
[62] Lagüela S, Díaz-Vilari~no L, Armesto J, Arias P. Thermographic 3D models as the
foundation for building information models. https://doi.org/10.21611/qirt.2012.
180; 2012.
[63] Lagüela S, Díaz-Vilari~no L, Martínez J, Armesto J. Automatic thermographic and
RGB texture of as-built BIM for energy rehabilitation purposes. Autom ConStruct
2013. https://doi.org/10.1016/j.autcon.2012.12.013.
[64] Wang C, Cho YK, Gai M. As-Is 3D thermal modeling for existing building
envelopes using a hybrid LIDAR system. J Comput Civ Eng 2013. https://doi.org/
10.1061/(ASCE)CP.1943-5487.0000273.
[65] Gonz� alez-Aguilera D, Rodriguez-Gonzalvez P, Armesto J, Lagüela S. Novel
approach to 3D thermography and energy efficiency evaluation. Energy Build
2012. https://doi.org/10.1016/j.enbuild.2012.07.023.
[66] L�
opez-Fern� andez L, Lagüela S, Gonz� alez-Aguilera D, Lorenzo H. Thermographic
and mobile indoor mapping for the computation of energy losses in buildings.
Indoor Built Environ 2017. https://doi.org/10.1177/1420326X16638912.
[67] Solla M, Riveiro B. Non-destructive techniques for the evaluation of structures
and infrastructure. https://doi.org/10.1201/b19024; 2016.
[68] Gonz� alez-Aguilera D, Lagüela S, Rodríguez-Gonz� alvez P, Hern�andez-L�opez D.
Image-based thermographic modeling for assessing energy efficiency of buildings
façades. Energy Build 2013. https://doi.org/10.1016/j.enbuild.2013.05.040.
[69] Cao Y, Xu B, Ye Z, Yang J, Cao Y, Tisse C-L, et al. Depth and thermal sensor fusion
to enhance 3D thermographic reconstruction. Optic Express 2018. https://doi.
org/10.1364/oe.26.008179.
[70] Ordonez Muller A, Kroll A. Generating high fidelity 3-D thermograms with a
handheld real-time thermal imaging system. IEEE Sensor J 2017. https://doi.org/
10.1109/JSEN.2016.2621166.
[71] Sels S, Verspeek S, Ribbens B, Bogaerts B, Vanlanduit S, Penne R, et al. A CAD
matching method for 3D thermography of complex objects. Infrared Phys Technol
2019. https://doi.org/10.1016/j.infrared.2019.04.014.
[72] Vidas S, Moghadam P. HeatWave: a handheld 3D thermography system for
energy auditing. Energy Build 2013. https://doi.org/10.1016/j.
enbuild.2013.07.030.
[73] Gonz� alez-Jorge H, Lagüela S, Krelling P, Armesto J, Martínez-S� anchez J. Single
image rectification of thermal images for geometric studies in façade inspections.
Infrared Phys Technol 2012. https://doi.org/10.1016/j.infrared.2012.05.003.
[74] Paterson AM, Dowling GR, Chamberlain DA. Building inspection: can computer
vision help? Autom ConStruct 1997. https://doi.org/10.1016/S0926-5805(97)
00031-9.
M.H. Shariq and B.R. Hughes
[75] Spencer BF, Hoskere V, Narazaki Y. Advances in computer vision-based civil
infrastructure inspection and monitoring. Engineering 2019. https://doi.org/
10.1016/j.eng.2018.11.030.
[76] Xu C, Xie J, Chen G, Huang W. An infrared thermal image processing framework
based on superpixel algorithm to detect cracks on metal surface. Infrared Phys
Technol 2014. https://doi.org/10.1016/j.infrared.2014.08.002.
[77] Galantucci RA, Fatiguso F. Advanced damage detection techniques in historical
buildings using digital photogrammetry and 3D surface anlysis. J Cult Herit 2019.
https://doi.org/10.1016/j.culher.2018.09.014.
[78] Wang N, Zhao X, Zhao P, Zhang Y, Zou Z, Ou J. Automatic damage detection of
historic masonry buildings based on mobile deep learning. Autom ConStruct
2019;103:53–66. https://doi.org/10.1016/j.autcon.2019.03.003.
[79] Narazaki Y, Hoskere V, Hoang TA, Fujino Y, Sakurai A, Spencer BF. Vision-based
automated bridge component recognition with high-level scene consistency.
Comput Civ Infrastruct Eng 2019. https://doi.org/10.1111/mice.12505.
[80] Guo Y, Tan Z, Chen H, Li G, Wang J, Huang R, et al. Deep learning-based fault
diagnosis of variable refrigerant flow air-conditioning system for building energy
saving. Appl Energy 2018. https://doi.org/10.1016/j.apenergy.2018.05.075.
[81] Edis E, Flores-Colen I, De Brito J. Building thermography: detection of
delamination of adhered Ceramic Claddings using the passive approach.
J Nondestr Eval 2015. https://doi.org/10.1007/s10921-014-0268-2.
[82] Fox M, Coley D, Goodhew S, De Wilde P. Time-lapse thermography for building
defect detection. Energy Build 2015. https://doi.org/10.1016/j.
enbuild.2015.01.021.
[83] ISO. ISO 17359:2003. Condition monitoring and diagnostics of machines —
general guidelines. 2003.
[84] Standardization IO for. Thermal Insulation - qualitative detection of thermal
irregularities in building envelopes - infrared method. Int Organ 2009. https://
doi.org/10.1021/es0620181.
[85] Mukupa W, Roberts GW, Hancock CM, Al-Manasir K. A review of the use of
terrestrial laser scanning application for change detection and deformation
monitoring of structures. Surv Rev 2017;49:99–116. https://doi.org/10.1080/
00396265.2015.1133039.
[86] Jones CA, Church E. Photogrammetry is for everyone: structure-from-motion
software user experiences in archaeology. J Archaeol Sci Reports 2020;30:
102261. https://doi.org/10.1016/j.jasrep.2020.102261.
[87] Heipke C, Madden M, Li Z, Dowman I. Theme issue “State-of-the-art in
photogrammetry, remote sensing and spatial information science. ISPRS J
Photogrammetry Remote Sens 2016;115:1–2. https://doi.org/10.1016/j.
isprsjprs.2016.03.006.
[88] Bemis SP, Micklethwaite S, Turner D, James MR, Akciz S, Thiele S T, et al.
Ground-based and UAV-Based photogrammetry: a multi-scale, high-resolution
mapping tool for structural geology and paleoseismology. J Struct Geol 2014;69:
163–78. https://doi.org/10.1016/j.jsg.2014.10.007.
[89] Filippelli SK, Lefsky MA, Rocca ME. Comparison and integration of lidar and
photogrammetric point clouds for mapping pre-fire forest structure. Remote Sens
Environ 2019;224:154–66. https://doi.org/10.1016/j.rse.2019.01.029.
15
Renewable and Sustainable Energy Reviews 130 (2020) 109979
[90] Aicardi I, Chiabrando F, Maria Lingua A, Noardo F. Recent trends in cultural
heritage 3D survey: the photogrammetric computer vision approach. J Cult Herit
2018;32:257–66. https://doi.org/10.1016/j.culher.2017.11.006.
[91] Mathys A, Semal P, Brecko J, Van den Spiegel D. Improving 3D photogrammetry
models through spectral imaging: tooth enamel as a case study. PloS One 2019;
14:e0220949. https://doi.org/10.1371/journal.pone.0220949.
[92] Mallison H, Wings O. Photogrammetry in paleontology - a practical guide.
J Paleontol Tech 2014.
[93] Xu YF. A photogrammetry-based experimental modal analysis method by
tracking visible laser spots. Meas J Int Meas Confed 2020;151:106963. https://
doi.org/10.1016/j.measurement.2019.106963.
[94] Zhou C, Liu K, Wang S, Wei S, Fan X. 3D shape measurement of a ground surface
optical element using band-pass random patterns projection. Chin Optic Lett Vol
13, Issue 8, Pp 081101- 2015;13:081101.
[95] Lowe DG. Distinctive image features from scale-invariant keypoints. Int J Comput
Vis 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94.
[96] Bay H, Ess A, Tuytelaars T, Van Gool L. Speeded-up robust features (SURF).
Comput Vis Image Understand 2008. https://doi.org/10.1016/j.
cviu.2007.09.014.
[97] Rosten E, Drummond T. Machine learning for high-speed corner detection. Lect
Notes Comput Sci (including Subser Lect Notes Artif Intell Lect Notes
Bioinformatics) 2006. https://doi.org/10.1007/11744023_34.
[98] Calonder M, Lepetit V, Strecha C, Fua P. BRIEF: binary robust independent
elementary features. Lect Notes Comput Sci (including Subser Lect Notes Artif
Intell Lect Notes Bioinformatics) 2010. https://doi.org/10.1007/978-3-642-
15561-1_56.
[99] Rublee E, Rabaud V, Konolige K, Bradski G. ORB: and efficient alternative to SIFT
and SURF. 2011 IEEE Int Conf Comput Vis 2011. https://doi.org/10.1002/
art.38045.
[100] Mur-Artal R, Montiel JMM, Tardos JD. ORB-SLAM: a versatile and accurate
monocular SLAM system. IEEE Trans Robot 2015. https://doi.org/10.1109/
TRO.2015.2463671.
[101] Mur-Artal R, Tardos JD. ORB-SLAM2: an open-source SLAM system for
monocular, stereo, and RGB-D cameras. IEEE Trans Robot 2017. https://doi.org/
10.1109/TRO.2017.2705103.
[102] Labb�e M, Michaud F. RTAB-Map as an open-source lidar and visual simultaneous
localization and mapping library for large-scale and long-term online operation.
J Field Robot 2019. https://doi.org/10.1002/rob.21831.
[103] AliceVision | meshroom - 3D reconstruction software n.d (accessed April 22,
2020), https://alicevision.org/#meshroom.
[104] Open3D – a modern library for 3D data processing n.d (accessed April 22, 2020),
http://www.open3d.org/.
[105] OpenCV nd. https://opencv.org/. [Accessed 22 April 2020].
[106] Gan H, Lee WS, Alchanatis V. A photogrammetry-based image registration
method for multi-camera systems – with applications in images of a tree crop.
Biosyst Eng 2018. https://doi.org/10.1016/j.biosystemseng.2018.06.013.
[107] Prerna D, Tekchandani R, Kumar N. Device-to-device content caching techniques
in 5G: a taxonomy, solutions, and challenges. Comput Commun 2020;153:48–84.
https://doi.org/10.1016/j.comcom.2020.01.057.