ISPRS Journal of Photogrammetry and Remote Sensing 139 (2018) 146–153

Contents lists available at ScienceDirect

ISPRS Journal of Photogrammetry and Remote Sensing

journal homepage: www.elsevier.com/locate/isprsjprs

Accurate facade feature extraction method for buildings from

three-dimensional point cloud data considering structural information
Yongzhi Wang a, Yuqing Ma a, A-xing Zhu b,c,d,e,f,⇑, Hui Zhao a, Lixia Liao a
a
School of Architectural and Surveying & Mapping Engineering, Jiangxi University of Science and Technology, Ganzhou, China
b
Key Laboratory of Virtual Geographic Environment, Nanjing Normal University, Nanjing, China
c
State Key Laboratory Cultivation Base of Geographical Environment Evolution, Nanjing, China
d
State Key Laboratory of Resources and Environmental Information System, Institute of Geographical Sciences and Natural Resources Research, Chinese Academy of Sciences (CAS),
Beijing, China
e
Department of Geography, University of Wisconsin-Madison, Madison, WI, USA
f
Jiangsu Center for Collaborative Innovation in Geographical Information Resource Development and Application, Nanjing, China

a r t i c l e i n f o a b s t r a c t

Article history: Facade features represent segmentations of building surfaces and can serve as a building framework.
Received 23 June 2017 Extracting facade features from three-dimensional (3D) point cloud data (3D PCD) is an efficient method
Received in revised form 21 November 2017 for 3D building modeling. By combining the advantages of 3D PCD and two-dimensional optical images,
Accepted 21 November 2017
this study describes the creation of a highly accurate building facade feature extraction method from 3D
Available online 17 March 2018
PCD with a focus on structural information. The new extraction method involves three major steps:
image feature extraction, exploration of the mapping method between the image features and 3D PCD,
Keywords:
and optimization of the initial 3D PCD facade features considering structural information. Results show
Three-dimensional point cloud data
Two-dimensional optical images
that the new method can extract the 3D PCD facade features of buildings more accurately and continu-
Structural information ously. The new method is validated using a case study. In addition, the effectiveness of the new method is
Facade features demonstrated by comparing it with the range image-extraction method and the optical image-extraction
Three-dimensional building modeling method in the absence of structural information. The 3D PCD facade features extracted by the new
method can be applied in many fields, such as 3D building modeling and building information modeling.
Ó 2017 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier
B.V. All rights reserved.

1. Introduction research problem due to the amount of valuable information

With the development of urban planning, cultural documenta-
tion, smart city construction, and building information modeling
(BIM), the requirement for realistic three-dimensional (3D) build-
ing models has increased (Sajadian, 2014; Cheng et al., 2016;
Jordana et al., 2016; Turk, 2016; Aljumaily et al., 2017). Facade fea-
tures represent a building surface segmentations, such as bound-
aries of windows, doors, and edges of other significant structural
elements, which play an important role in realistic 3D building
modeling (Laefer et al., 2011; Hinks et al., 2013). Facade features
provide an easy, flexible, and economic way to reconstruct large
scale geometric models (Ceylan et al., 2012; AlHalawani et al.,
2013). Methods for extracting the facade features of buildings
based on 3D point cloud data (3D PCD) are becoming an important
⇑ Corresponding author at: Key Laboratory of Virtual Geographic Environment,
Nanjing Normal University, Nanjing, China.
E-mail addresses: gisstarnnu@gmail.com (Y. Wang), azhu@wisc.edu (A-x. Zhu).
(including corners, edges, and borders) contained in building 3D
PCD, (An et al., 2013).
Various methods have been proposed for facade feature extrac-
tion from 3D PCD. They mainly involve either direct or indirect
extraction. In direct extraction methods, facade features are
extracted from raw or pretreated 3D PCD directly by calculating
the geometric information, such as distance changes, normal
changes, curvature changes, and the density of the 3D PCD
(Rutzinger et al., 2009; Tan and Cheng, 2015). Given that the stor-
age of 3D PCD is unstructured, these methods often use random
sample consensus (RANSAC), principal component analysis, or
fuzzy clustering approaches to segment and cluster the 3D PCD
(Schnabel et al., 2007; Biosca and Lerma, 2008; Pang et al., 2015).
However, these approaches are only capable of extracting building
boundary and contour features (Bauer et al., 2003; Zolanvari and
Laefer, 2016). In addition these approaches do not easily simulta-
neously analyze the feature relationships between different 3D
PCD clusters. Therefore, given that direct extraction methods use

https://doi.org/10.1016/j.isprsjprs.2017.11.015
0924-2716/Ó 2017 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rights reserved.
 Y. Wang et al. / ISPRS Journal of Photogrammetry and Remote Sensing 139 (2018) 146–153
only unstructured 3D PCD to extract facade features, their preci-
sion is often very low (Rutzinger et al., 2009; Zolanvari and
Laefer, 2016).
In indirect extraction methods, to acquire features with high
accuracy, other information, such as range images, optical images,
and geo-information, are often included in the facade feature
extraction process (Yang et al., 2014, 2016; Gilani et al., 2015).
Indirect extraction methods can be divided into two types. The first
type constructs range images using 3D PCD first, then extracts fea-
tures from the range images with the aid of digital-image process-
ing and then maps the range-image features to the 3D PCD (Tan
and Cheng, 2015). This type of indirect extraction methods applies
range images to extract facade features from 3D PCD to enable
detection of edge features of heterogeneous regions. However, this
type of indirect extraction methods requires transformation of 3D
PCD into two-dimensional (2D) range images. Additionally, only
range information associated with 3D PCD is preserved during
the transformation process, with loss of substantial amounts of
boundary information. Therefore, this type of indirect extraction
methods excludes many facade features.
The second type of indirect extraction methods uses 2D optical
images obtained by a camera registered on a 3D laser scanner to
assist facade feature extraction (Gilani et al., 2015). Considering
that high-resolution 2D optical images can supple important infor-
mation, the second type of indirect extraction methods can acquire
extraction results that are more accurate than the first type (Wang
et al., 2013; Li et al., 2013; Gilani et al., 2015). However, 2D image
processing algorithms are still used to extract 2D features in the
second type (Yang et al., 2014, 2016), which does not take advan-
tage of the textural and geometric structural information con-
tained in the 2D optical images of building facades during the
extraction process, resulting in the extraction of many non-
features while some important facade-features are ignored.
The 2D optical images contain rich textural and geometric
structural information that can be used to help extract facade fea-
tures after adding scales (Fig. 1). The images of building facades
contain regular and repetitive textural structural information, such
as wall surfaces covered with the same painting material. In addi-
tion, there are many window and texture edges with rich simple
geometric structural information, such as line segments. This
structural information can be used to describe the spatial distribu-
tion and internal variation of images. By applying statistical analy-
sis to the structural information of adjacent areas, the marching
directions of information can be determined efficiently during
the feature extraction process, and these directions are important
for improving the feature extraction accuracy (Li et al., 2012;
Zhou and Yin, 2013). Thus, structural information can be used in
facade feature extraction for 3D PCD but has not been sufficiently
utilized in current methods (Truong-Hong et al., 2012; Vo et al.,
2015; Aljumaily et al., 2015). This leads to many weaknesses,
Textural structural
information
Fig. 1. Schematic diagram of textural and geometric structural information in a facade image.
147
including decreased accuracy of feature extraction from 2D images
of building facades, lack of rigorous mapping models between
2D-image features and 3D PCD, and lack of reasonable optimal
treatment of building facade features of 3D PCD. Therefore, it is
necessary to investigate methods allowing for higher-accuracy
feature extraction from building facades for 3D PCD.
By combining the advantages of both 3D PCD and 2D optical
images, we achieved a highly accurate building facade feature
extraction method from 3D PCD that considers structural informa-
tion. Section 2 introduces the basic idea and details of the method.
Section 3 describes a case study that extracts the facade features
from the 3D PCD of the School of Architectural and Surveying &
Mapping Engineering (SASME) building, Jiangxi University of
Science and Technology (JXUST). The effectiveness, value settings,
and potential limitations of the method are discussed in Section 4.
In Section 5, we deliver our conclusions on the validity and accu-
racy of the method. Our results show that this method can provide
data support for 3D modeling and BIM construction in urban plan-
ning applications.
2. Methodology
2.1. Basic idea and overall design
The 3D PCD of buildings is unstructured, and the features
directly extracted from the 3D PCD often exhibit low geometrical
continuity. 2D optical images, which can be mapped to the real-
world scale according to the 3D PCD, contain substantial amounts
of geometric and textural structural information. This information
can be used to describe the spatial distribution and internal varia-
tion of image elements, such as line segment features. By applying
statistical analysis to the structural information of adjacent areas,
the marching directions of information, such as line segments,
can be determined efficiently in the feature extraction process,
and these directions are very important for improving the accuracy
of feature extraction. Therefore, introducing the structural infor-
mation contained in 2D optical images to the process of feature
extraction of building facades from 3D PCD should be feasible
and effective.
The overall design of the new method involves three major
steps (Fig. 2). First, extract the image features of building facades
based on structural information. In this step, the textural and geo-
metric structural information is used to extract image features
accurately. Second, explore the mapping method between the
image features and the 3D PCD. In this step, we combine image
features of building facades and the 3D PCD and then extract the
initial 3D PCD facade features based on the image features. Finally,
optimize the initial 3D PCD facade features that take into consider-
ation the structural information. In this step, we propose an
Geometric structural
information
148 Y. Wang et al. / ISPRS Journal of Photogrammetry and Remote Sensing 139 (2018) 146–153

High resolution Pretreated 3D point

optical images cloud data

Step1: Step2
Image feature extraction of Structural Mapping method between image
building facades information of features and 3D PCD
building images
Building image pretreatment Textural Analysis of internal and external
and facade structure structural parameters of the camera after
decomposition information registration

Geometric Mapping model of image features

Line segment extraction of and 3D PCD
Structural
building facades
information
Initial 3D PCD features of building
Line segments validation ……
facades

Step3

3D PCD features of building facades based on image features

Initial 3D PCD
Optimizing method for initial The final 3D PCD
features of
3D PCD features based on features of
building
RANSCA building facades
facades

Case study and discussion of the new feature extraction method for
building facades from 3D PCD considering structural information

Fig. 2. Flowchart of the new facade feature extraction method for buildings.

optimizing method for the 3D PCD features of building facades.
This step also addresses the noise and discontinuous phenomena
of initial 3D PCD features.
2.2. Image features of building facade extraction considering structural
information
Currently, numerous classical edge detection algorithms, such
as the Roberts operator, the Sobel operator, the Laplacian operator,
and the Canny operator, exist for the extraction of 2D image fea-
tures of building facades (Canny, 1986; von Gioi et al., 2010,
2012; Maini and Aggarwal, 2009). These algorithms usually use
differential operators that check derivative changes to detect alter-
ations in gray values surrounding an image. These algorithms can
be directly or indirectly (with some adjustments) used to extract
building facade features; however, the facade features extracted
by these algorithms are incomplete and inaccurate due to their
not using facade structure information. To address these problems,
we developed a new method for extracting image features of build-
ing facades based on structural information.
The key processes associated with this extraction method are as
follows. (1) Preprocess images. A Gaussian filter with a window
height and width of 3 pixels is used to suppress noise and
smoothen out the input images. This step involves zooming out
from high-resolution images to a suitable scale according to the
scale-zooming factor. (2) Remove texture. The relative total
variation (RTV) method is used to filter windows to decompose
and remove the texture of building facades (Li et al., 2012). RTV
regulates texture removal and contains general pixel-wise win-
dowed total-variation measurements and the novel windowed
inherent variation. In addition, RTV produces texture structure
images. (3) Calculate gradient. Gray-change-detection operators,
such as Cany, are used to compute the gradient magnitude and
direction at each pixel and obtain optimized gradient maps. (4)
Detect line segments. EDLines, which is a famous line segment
detector, is used to detect features contained in gradient maps
(Akinlar and Topal, 2011; Von Gioi et al., 2012). These line segment
detection algorithms make full use of neighboring pixel-gradient
magnitudes and directions. By calculating or setting the appropri-
ate detection parameters, such as the gradient threshold and scale
factor which are suggested to be set to 5.22 and 0.8 according to
Akinlar and Topal (2011) and Von Gioi et al. (2012), these line seg-
ment detection algorithms can detect the pixels that exhibit alter-
ations in peak gray values more accurately. By using Edge Drawing,
which is a novel and recently-proposed edge detection algorithm,
to connect these pixels, we can obtain additional line segments
that constitute the image features of building facades (Akinlar
and Topal (2011)). (5) Validate line segments of image features.
Validation is performed using the ‘‘Number of False Alarms” (NFA)
of a line segment, which was defined by Desolneux et al. (2000)
and is widely used in many line segment detection methods
(Von Gioi et al., 2010, 2012; Akinlar and Topal, 2011). The NFA
value of A, which is a segment of length ‘‘n” with ‘‘k” points, in
an image of N
N pixels is calculated by Formula (1) below. A is
considered a validated line segment if its NFA  ¢, which is recom-
mended to be set to 1 by Desolneux et al. (2000).
 Y. Wang et al. / ISPRS Journal of Photogrammetry and Remote Sensing 139 (2018) 146–153 149

n  
X n The following steps describe the mapping process:
NFAðn; kÞ ¼ N4 pi ð1  pÞni ð1Þ
i¼k
i
(1) The pixel coordinates of the image features are recorded as a
set (U, V), which is traversed to analyze every pixel, (Ui, Vi),
In Formula (1), ‘‘p” is defined as follows: if two line segments, P
to determine its status as a feature pixel (if the pixel belongs
and Q, exhibit the same directionality, the precision (1/n) of angles
to a line segment, it is considered a feature pixel). All feature
(P) and (Q) are within p/n degrees of each other.
pixels are saved to set (UC, VC).
Compared with classical feature extraction operators, the new
(2) Coordinates are registered from the 3D PCD to 2D optical
method for extracting the image features of building facades uses
images acquired by a camera fixed to the 3D laser scanner.
the textural and geometric structural information of building
The intrinsic (M1) and external parameters (M2) of the cam-
images more effectively. Furthermore, the new method can main-
era are optimized using 3D PCD-processing software, such as
tain better geometry continuity for the extracted features than
RiSCAN PRO. The 3D PCD set is traversed, and every point
other algorithms on building facade feature extraction. In addition,
coordinate value (XWi, YWi, ZWi) is introduced into conver-
the new extraction method can detect valid features for images
sion formula 2, which converts the 3D PCD from geospatial
that exhibit subtle levels of gray changes. The validity of the image
coordinates (XW, YW, ZW) to camera coordinates (XC, YC,
features extracted from building facades indicates that this method
ZC), followed by analysis of the camera coordinates (XCi,
is an effective framework and basis for the next step of feature
YCi, ZCi) of each 3D PCD, especially the optical axis value,
extraction.
ZCi. We then introduce each the geospatial coordinate value
(XWi, YWi, ZWi) and the ZCi value of each point into conversion
2.3. Mapping between image features and 3D PCD formula 3 to calculate the corresponding image pixel-
spatial-coordinate value (ui, vi).
Image features extracted from building facades represent a ser-
ies of feature pixels. The image feature is a set of 2D pixels. There- ½X C YC ZC 11 ¼ M 2  ½X W YW ZW 11 ð2Þ
fore, the method used to map between the 2D image features and
the 3D PCD can be simplified into mapping from the 2D plane to Z C  ½u v 11 ¼ M 1  M 2  ½X W YW ZW 11 ð3Þ
the 3D space. Because each pixel coordinate (u, v) may correspond
to more than one 3D point (x, y, z), the mapping relationship
between pixels of image features and points of the 3D PCD is (3) If this coordinate (ui, vi) is contained within the feature pixel
one-to-many. set (UC, VC), the 3D-laser points corresponding to this image
This section describes the method used to map between image pixel represent the feature points in the 3D PCD and are
features of building facades and the 3D PCD. The aim of this pro- saved along with the corresponding image pixels into a 3D
cess is to discover corresponding 3D PCD features according to PCD feature-point set (XWC, YWC, ZWC).
the 2D image features of building facades. The flowchart is shown (4) By executing step (2) and (3) iteratively, we obtain the initial
in Fig. 3. 3D PCD features of building facades.

2.4. The initial 3D PCD facade feature optimization

There are many incomplete and discontinuous features in initial

Image features Original 3D PCD M1, Mounting,
3D PCD facade features, resulting in numerous potential problems.
(U,V) (XW,YW,ZW) COP, SOP
First, target buildings are often obstructed by other objects and
their shadows, which introduce numerous faulty and discontinu-
ous features into the extracted image features of building facades.
Additionally, due to the influence of glass transmission, projection
Traverse pixels Traverse 3D PCD
(Ui,Vi) (XWi,YWi,ZWi) parallax, and differences in sampling density, image-feature pixels
might correspond to more than one 3D point, which can introduce
Formula(1) noise during the 3D PCD feature extraction process. Therefore,
No The corresponding optimizing the initial 3D PCD facade features being extracted is
camera coordidate of important to more accurately represent the structural profile of
Is a feature laser point (XCi,YCi,ZCi)
building facades.
pixel ?
Here, we developed a 3D PCD facade feature optimization
Obtain ZCi method based on the RANSAC algorithm. Because of the robustness
Yes
No of random sampling uniformity to noisy and external points, the
Formula(2) RANSAC algorithm is widely used in geometric element detection,
The set of image The corresponding pixel including for 3D-plane or line detection (Schnabel et al., 2007;
feature pixels coordidate of laser point Pang et al., 2015). Based on these characteristics, a geometric
(UC,VC) (ui,vi) element-extraction method from 3D PCD was previously proposed
(Schnabel et al., 2007). Building facades contain obvious geometric
features, such as intersection lines between different walls and
Does (ui,vi) exist in edges of windows and gates; therefore, it is feasible to introduce
(UC,VC)? the RANSAC algorithm to optimize initial 3D PCD facade features.
The process of 3D PCD facade-feature optimization is described
Yes as follows:
The set of initial 3D PCD
features (XWC,YWC,ZWC)
(1) Regard the initial 3D PCD facade features extracted via
methods described in Section 2.3 as a candidate point set,
Fig. 3. Flowchart of mapping between 2D image features and 3D PCD. Q = {qj}, with two points, {q1, q2}, randomly selected from Q.
150 Y. Wang et al. / ISPRS Journal of Photogrammetry and Remote Sensing 139 (2018) 146–153

(2) Construct a line marked as I according to the two points

{q1, q2}.
(3) Calculate the distance, dj, from other points, qj, except
points, {q1, q2}, in the set Q to line, I, and record the point
into a set, DI, if its distance, dj, is smaller than a distance
threshold, n. Set the number of points in DI, as the score of
line, I.
(4) Repeat processes (1) to (3) M times and select the line, I⁄,
with the highest score (the value of M is computed by
formula (4)).
logð1  WÞ
M¼
logð1  ð1  gÞ2 Þ
In this formula, g is the proportion of the points out of range of
line, I⁄, and W is the chosen probability after sampling M times.
(5) Record line, I⁄, and remove points in set, DI, from the candi-
date point set, Q.
(6) Repeat processes (1)–(5) until the line cannot be constructed
from the remaining points or when the score is greater than
a score threshold.
3. Case study
We used Riegl VZ1000 (with point density set to 1 cm) and
Nikon D610 (with resolution set to 20 mm) to capture the 3D
PCD and 2D optical images of the SASME building in JXUST. This
data was used to demonstrate the validity of the proposed extrac-
tion method. After pretreating the test data to remove noise and
mosaicity, we used the new method to extract the 3D PCD facade
features of the SASME building. With the goal of showing the
framework information of the building in a small amount of 3D
PCD, the results (Fig. 4) are visualized by the visualization toolkit
ð4Þ (VTK).
The new extraction method proposed by this paper can acquire
more accurate and continuous 3D PCD facade features. The
extracted 3D PCD facade features are consistent with the building
geometric edges, such as the window and wall edges. These 3D
PCD facade features can represent the building facade structures
comprehensively and accurately, as shown in Fig. 4. Thus, based
on these facade features, we can describe the overall building using
3D PCD concisely. Furthermore, these 3D PCD facade features can
serve as important structural constraints for 3D building modeling.

(a)

(b) (c)
Fig. 4. Final 3D PCD facade features extracted by the new method. (a) Overall 3D PCD facade features of the SASME building. (b and c) Partial detail of the 3D PCD belonging to
the facade features (red) overlapping with the original 3D PCD. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of
this article.)
 Y. Wang et al. / ISPRS Journal of Photogrammetry and Remote Sensing 139 (2018) 146–153 151

4. Discussion presented a swell phenomenon, describing the presence of numer-

4.1. Effectiveness of the new method
To verify the effectiveness and accuracy of the extraction
method proposed here, we tested it on a portion of the SASME
building (Fig. 5a) and conducted three experiments using different
extraction methods, including the range-image extraction method
and the optical-image method in the absence of structural infor-
mation. The 3D PCD facade features extracted by the three meth-
ods are shown in Fig. 5(b)–(d).
The range-image method returned very few line segments from
the 3D PCD facade features, and some window edges were not
extracted effectively. The optical-image method in the absence of
structural information was capable of extracting detailed features,
such as window edges, although it also extracted scrappy line seg-
ments that were not facade features. Additionally, the line seg-
ments of the facade features extracted by this method often
ous non-feature points adjacent to the facade feature line seg-
ments. Due to this phenomenon, the continuousness of the
extracted facade feature line segments was very weak. However,
the method presented in this study extracted facade feature line
segments that fit the real line segments very well and maintained
high degrees of continuity. Moreover, this method reduced the
amount of scrappy line segments and eliminated the swell phe-
nomenon. These results indicated that the new method was able
to more completely and accurately extract the facade features of
buildings.
In addition, the new method is more robust. Some areas of the
target object have no 3D PCD owing to the occlusion (by other
objects) or reflection (by glass) during data collection. Fake bound-
aries are generated in these areas with missing data. The current
exaction methods cannot easily to distinguish the difference
between the real boundaries and these fake boundaries based on
just the 3D PCD. Thus, these fake boundaries are treated as facade

Fig. 5. Comparisons of facade features extracted by different methods. (a) The original 3D PCD overlapping with the 2D optical images. (b) 3D PCD facade features in red color
extracted by range-image method. (c) 3D PCD facade features in red color extracted by the optical-image method in the absence of structural information. (d) 3D PCD facade
features in red color extracted by the new extraction method proposed by this paper. (For interpretation of the references to color in this figure legend, the reader is referred
to the web version of this article.)
152 Y. Wang et al. / ISPRS Journal of Photogrammetry and Remote Sensing 139 (2018) 146–153

features and result in some false detection and extraction, as
shown in Fig. 5(b) and (c). By comparison, the new method can
eliminate most of the false detection, as shown in Fig. 5(d), indicat-
ing the effectiveness of the new method further.
Furthermore, the statistical analysis of facade feature informa-
tion (Table 1) for the three extraction methods was performed
based on Fig. 5. A quantifiable assessment was completed by refer-
encing Truong-Hong and Laefer’s (2013) work. Table 1 shows the
following conclusions: the range-image method can only extract
few true facade features with more false features; the optical-
image method without structural information can extract most
true facade features, but numerous false facade features; and the
new method can extract most true facade features with few false
features.
4.2. Value setting of optimization parameters
Three important parameters in the initial 3D PCD facade feature
optimization process include distance threshold, n, chosen proba-
bility, W, and proportion, g, of out-of-range points. Our analysis
indicated that a distance threshold (0.02%) of the length of the
diagonal line of the bounding box associated with the candidate
point set, Q, is optimal. The chosen probability, W, is suggested
as 0.99, and the proportion, g, is suggested as 0.9. Based on these
values, we selected partial initial 3D PCD facade features of the
SASME building as test data (Fig. 6, black), and used the new
proposed optimization method get the final 3D PCD facade features
(Fig. 6, red).
In the initial 3D PCD facade features, many non-features (at the
top of Fig. 6) are extracted by the false detection, and some non-
feature points are included because of the swell phenomenon (at
the bottom of Fig. 6). However, in the optimized 3D PCD facade fea-
tures, the non-features and swell phenomenon are mostly elimi-
nated. Obviously, we obtain accurate 3D PCD facade features
after the optimization process.
4.3. Potential limitations of this approach
By using the textural and geometric structural information con-
tained in the optical images, the proposed extraction method
improved the accuracy and continuousness of the resulting facade
features. However, this method still has some potential limitations.
When the textures of building facades become increasing complex,
the amount of scrappy line segments extracted will increase. Addi-
tionally, when there are multiple types of geometric structures in
the building facades, the accuracy of the final facade features will
be reduced. Finally, the occlusion can result in missing data in
LiDAR point cloud data and images. False detection from the miss-
ing data is hardly eliminated. These potential limitations will be
investigated further in the future.
5. Conclusion

Table 1 This paper proposed a building facade feature extraction

Information of facade features derived by different methods.
method from 3D PCD where textural and geometric structural
Feature extraction Actual Number of true Number of false information is introduced into three processes: extraction of image
method number of facade features facade features features of building facades, mapping between 2D image features
facade extracted extracted
features
and 3D PCD, and optimization of the initial 3D PCD features. By
combining the respective advantages of 3D PCD and 2D optical
The range-image 45 6 13
extraction method
images, the new method can extract more accurate and continuous
The optical-image 45 36 55 3D PCD building facade features. The effectiveness of the new
method without method was demonstrated by comparing its results with those of
structural classical methods (range-image extraction and optical-image
information
extraction in the absence of structural information). The 3D PCD
The new extraction 45 41 7
method facade features extracted using the new method constitute valu-
able support for 3D modeling and BIM construction in urban areas.

Acknowledgement

This study is supported by the National Natural Science Founda-

tion of China (No. 41601429), the Natural Science Foundation of
Jiangxi Provincial Department of Science and Technology (No.
20171BAB203028). The authors would also like to acknowledge
the contributions of Mercedes Endres and David Cushley to the
spelling and grammar check for this paper.

References

Akinlar, C., Topal, C., 2011. EDLines: A real-time line segment detector with a false
detection control. Pattern Recogn. Lett. 32, 1633–1642.
Aljumaily, H., Laefer, D.F., Asce, M., Cuadra, D., 2015. Big-data approach for three-
dimensional building extraction from aerial laser scanning. J. Comput. Civil Eng.
30 (3).
Aljumaily, H., Laefer, D.F., Asce, M., Cuadra, D., 2017. Urban point cloud mining
based on density clustering and MapReduce. J. Comput. Civil Eng. 31 (5), 215–
219.
AlHalawani, S., Yang, Y., Liu, H., Mitra, N.J., 2013. Interactive facades analysis and
synthesis of semi-regular facades. Comput. Graphics Forum 32, 215–224.
An, Y., Li, Z.L., Shao, C., 2013. Feature extraction from 3D point cloud data based on
discrete curves. Math. Probl. Eng. 2013, 290740.
Bauer, J., Karner, K., Schindler, K., Klaus, A., Zach, C., 2003. Segmentation of building
models from dense 3D point-clouds. In: Proceedings of the 27th Workshop of
the Austrian Association for Pattern Recognition. OAGM, Laxenburg, Austria, pp.
Fig. 6. Facade features after optimization using the suggested values. 253–259.
 Y. Wang et al. / ISPRS Journal of Photogrammetry and Remote Sensing 139 (2018) 146–153
Biosca, J.M., Lerma, J.L., 2008. Unsupervised robust planar segmentation of
terrestrial laser scanner point clouds based on fuzzy clustering methods. J.
Photogramm. Rem. Sens. 63, 84–98.
Canny, J., 1986. A computational approach to edge detection. IEEE Trans. Pattern
Anal. Mach. Intel. PAMI-8, 679–698.
Ceylan, D., Mitra, N.J., Li, H., Weise, T., Pauly, M., 2012. Factored facade acquisition
using symmetric line arrangements. Comput. Graphics Forum 31, 671–680.
Cheng, L., Wang, Y., Chen, Y., Li, M., 2016. Using LiDAR for digital documentation of
ancient city walls. J. Cult. Heritage 17, 188–193.
Desolneux, A., Moisan, L., Morel, J.M., 2000. Meaningful alignments. Int. J. Comput.
Vis. 40 (1), 7–23.
Gilani, S.A.N., Awrangjeb, M., Lu, G., 2015. Fusion of lidar data and multispectral
imagery for effective building detection based on graph and connected
component analysis. ISPRS – Int. Arch. Photogramm., Rem. Sens. Spatial
Inform. Sci. XL-3/W2, 65–72.
Hinks, T., Carr, H., Truong-Hong, L., Laefer, D.F., Asce, M., 2013. Point cloud data
conversion into solid models via point-based voxelization. J. Surv. Eng. 139 (2),
72–83.
Jordana, T.R., Goetcheus, C.L., Madden, M., 2016. Point cloud mapping methods for
documenting cultural landscape features at the Wormsloe state historic site,
Savannah, Georgia, USA. ISPRS – Int. Arch. Photogramm., Rem. Sens. Spatial
Inform. Sci. XLI-B5, 277–280.
Laefer, D.F., Truong-Hong, L., Fitzgerald, M., 2011. Processing of terrestrial laser
scanning point cloud data for computational modelling of building facades.
Recent Patents Comput. Sci. 4, 16–29.
Li, X., Yan, Q., Xia, Y., Jia, J., 2012. Structure extraction from texture via relative total
variation. ACM Trans. Graphics 31 (6), 131–139.
Li, H., Zhong, C., Hu, X.G., Xiao, L., Huang, X., 2013. New methodologies for precise
building boundary extraction from LiDAR data and high resolution image.
Sensor Rev. 33 (2), 157–165.
Maini, R., Aggarwal, H., 2009. Study and comparison of various image edge
detection techniques. Int. J. Image Process. 3 (1), 1–11.
Pang, S.Y., Liu, Y.W., Zuo, Z.Q., Chen, Z.F., 2015. Combination of region growing and
TIN edge segmentation for extraction of geometric features on building facades.
Geomatics Inform. Sci. Wuhan Univ. 40 (1), 102–106.
Rutzinger, M., Elberink, S.O., Pu, S., Vosselman, G., 2009. Automatic extraction of
vertical walls from mobile and airborne laser scanning data. In: Bretar, F.,
Pierrot-Deseiligny, M., Vosselman, G. (Eds.), Laserscanning ’09, ISPRS Archives,
XXXVIII-3/W8, pp. 7–11.
153
Sajadian, M., 2014. A data driven method for building reconstruction from LiDAR
point clouds. In: The 1st ISPRS International Conference on Geospatial
Information Research, ISPRS Archives, XL-2/W3, pp. 15–17.
Schnabel, R., Wahl, R., Klein, R., 2007. Efficient RANSAC for point-cloud shape
detection. Comput. Graphics Forum 26, 214–226.
Tan, K., Cheng, X.J., 2015. Dual-threshold algorithm for intensity image edge
extraction of terrestrial laser scanning point cloud. J. Tongji Univ. (Nat. Sci.) 43
(9), 1425–1431 (in Chinese).
Truong-Hong, L., Laefer, D.F., Asce, M., Hinks, T., Carr, H., 2012. Flying voxel method
with Delaunay triangulation criterion for facade/feature detection for
computation. J. Comput. Civil Eng. 26 (6), 691–707.
Truong-Hong, L., Laefer, D.F., 2013. Validating computational models from laser
scanning data for historic facades. J. Test. Eval. 41 (3), 481–496.
Turk, Ž., 2016. Ten questions concerning building information modelling. Build.
Environ. 107, 274–284.
Vo, A.V., Truong-Hong, L., Laefer, D.F., Bertolotto, M., 2015. Octree-based region
growing for point cloud segmentation. ISPRS J. Photogramm. Remote Sens. 104,
88–100.
Von Gioi, R.G., Jakubowicz, J., Morel, J.M., Randall, G., 2012. LSD: a line segment
detector. Image Process. On Line 2, 35–55.
Von Gioi, R.G., Jakubowicz, J., Morel, J.M., Randall, G., 2010. LSD: a fast line segment
detector with a false detection control. IEEE Trans. Pattern Anal. Mach. Intell. 32,
722–732.
Wang, Y., Ewert, D., Schilberg, D., Jeschke, S., 2013. Edge extraction by merging 3D
point cloud and 2D image data. In: Proceedings of the 10th International
Conference & Expo on Emerging Technologies for a Smarter World (CEWIT),
New York, USA, pp. 21–22.
Yang, L., Sheng, Y.H., Wang, B., 2014. LiDAR data reduction assisted by optical image
for 3D building reconstruction. Optik – Int. J. Light Electron Opt. 125, 6282–
6286.
Yang, L., Sheng, Y.H., Wang, B., 2016. 3D reconstruction of building facade with
fused data of terrestrial LiDAR data and optical image. Optik – Int. J. Light
Electron Opt. 127, 2165–2168.
Zhou, S.R., Yin, J.P., 2013. LBP texture feature based on Haar characteristics. J. Softw.
24 (8), 1909–1926 (in Chinese).
Zolanvari, S.M.I., Laefer, D.F., 2016. Slicing Method for curved façade and window
extraction from point clouds. J. Photogramm. Rem. Sens. 119, 334–346.