A HYBRID SYSTEM FOR SEMI-AUTOMATIC 3D BUILDING MODELING

Jiann-Yeou Rau1 ,Liang-Chien Chen2 ,Tee-Ann Teo3 ,Yi-Hsien Tseng4 ,Szu-jen Chen5
1Assistant Professor, 4Professor, Department of Geomatics, National Cheng Kung University,
No. 1, University Road, Tainan City 701, TAIWAN
{jyrau, tseng}@mail.ncku.edu.tw, (886) 6-2757575~63839, ~63835
2Professor & 3Ph.D Candidate, Center for Space and Remote Sensing Research, National Central
University, No. 300, Jhongda Road, Jhong-Li, Taoyuan 320, TAIWAN
{lcchen, ann}@csrsr.ncu.edu.tw, (886)3-4227151 ~ 57651, 57622 and 57623
5Officer, Ministry of the Interior
No.5, Syujhou Road, Jhongjheng District, Taipei City 100, TAIWAN
babbler@gps.moi.gov.tw, (886)2-23566102

KEY WORDS: 3D Building Modeling, 2D Maps, Semi-Automatic

ABSTRACT: There are two major strategies for building modeling, i.e. data-driven and
modeldriven approaches. Those two methods have their own advantage. For example, as for
spherical dome the extraction of structure lines from images or LiDAR in the data-driven approach
is not practical and impossible. But it is easy to adopt a parametric model for modeling a spheroid.
In the paper, we propose a hybrid system that combines the data-driven and model-driven
approaches for building modeling. It fuses both approaches’ excellence and compensate for each
other to increase the degree of completeness of modeling. The input data is the airborne stereo-
imagery with orientation parameters. During building reconstruction, depend on the type of
buildings the operator may select either data-driven or model-driven approach. For data-driven, the
3D structure lines have to be measured and we can adopt the Split-Merge-Shape algorithm to
reconstruct its detailed rooftop structure, such as gable, hipped, cross gable and cross hipped, etc.
For roof type in cylindroids or spheroid, the model-driven approach is suggested that an
intermediate parametric model will be utilized and converted to a generalized polyhedral model at
the end. In the measurement of 3D structure lines from roof corners and 3D point from rooftop, a
naked-eye stereo-measurement GUI without special hardware was designed to accomplish the
whole process. Experimental results demonstrate that the proposed scheme can generate most kinds
of building models effectively.

1. INTRODUCTION

Digital 3-D building models are useful in many fields, such as location-based service,
trueorthophoto generation (Rau et al., 2002), urban planning, addressing of mobile phone stations,
virtual tourism, real estate, etc. The generation of reliable and accurate 3-D building models thus
becomes important. It is especially that Microsoft and Google provided their location-based service
on the internet since 2005. One can realize the demand of 3D building models will be considerable
grow in the near future.

From the operation point of view, the generation of 3D building models can be classified into a
manual operation, an automatic approach (Brenner, 2000) and a semi-automatic approach (Grün &
Wang, 2001; Rau & Chen, 2003). In order to improve the automation, the data-driven and
modeldriven strategies were adopted in most of the researches.

For manual operations, building models were created by manually measured from a Digital
Photogrammetric Workstation (DPW) (Stereo Analyst, 2007). The operator is responsible for the
measurement of roof corners even occlusion existed. This procedure is tedious and inefficient,
especially for connected buildings in densely built-up area that topological error is unavoidable.
Since airborne laser scanner data (LiDAR) can provide high density of three-dimensional point
clouds of ground surface, many algorithms for automatic reconstruction of 3-D building models
were proposed. Although a detail and accurate rooftop structure could be obtained, the problem is a
precise building boundary cannot be derived from LiDAR data directly. A more practical solution
that integrates the LiDAR data with 2D topographic map (Brenner, 2000) or aerial imagery (Chen,
et al., 2006) were thus proposed. By means of data fusion, a detailed roof structure with accurate
building boundary could be achieved.

Grün & Wang (1998, 2001) proposed a model-driven semi-automatic approach for 3-D building
modeling. The proposed system has been proven practical and effective not only for buildings but
also for roads, rivers, parking lots, ships, etc. However, some requirements are necessary, such as
(1) the measurement of hidden corners caused by building occlusions is necessary to assure the
completeness of a roof unit, and (2) the digitizing sequence of boundary points is restricted to be
point-wise.

In this paper, we propose a hybrid scheme by combing data-driven and model-driven approach for
building model reconstruction. The creation of 3D structure lines was performed by manual
stereomeasurement of roof-corners without specific digitization sequence. In case of occlusion,
only the visible part of the roof edge was measured. It means that an incomplete structure line is
acceptable in building modeling. Since the generated building model is in polyhedron, other
rooftops in cylindroids or spheroid will be reconstructed by means of parametric building model. A
group of generalized polyhedral models will be produced at the end. In the proposed system, a GUI
interface without special hardware, such as stereo-monitor and 3D mouse, was designed for naked-
eye stereo-measurement.

2. METHODLODGY

Fig. 1 is the diagram of the proposed system for building modeling. In which, two major
components were designed, i.e. the Naked-Eye Stereo-Measurement GUI and Building
Reconstruction. The major input data of the system is the airborne stereo-imagery with exterior
orientation parameters (EOPs). An optional 2D topo-map with building boundary can accelerate the
whole process but not a requirement. The designed GUI interface is based on the photogrammetric
top-down and bottom-up technique. The system also provides some tools for the editing of roof
corner coordinates, adding 3D structure lines or 3D points for the refinement of rooftop structure.
In building reconstruction, depend on the type of target roof a suitable approach will be adopted,
e.g. the parametric modeling and the SMS algorithm. The final product of building reconstruction is
polyhedron. If further refinement is necessary, the operator can add more 3D polygon (roof), 3D
structure lines, or 3D points to achieve a detailed rooftop structure.

2.1 The Naked-Eye Stereo-Measurement GUI

The purpose of designing such a Naked-Eye Stereo-Measurement GUI is to create a thin-client web
based 3D mapping system for internet collaboration. Since 3D building modeling is a
timeconsuming and labor-intensive work, the increasing of operators could increase the production
efficiency. If there are N operators, we will need N datasets and Personal Computers. If special
hardware, such as stereo-monitor and 3D mouse, is required during stereo-measurement, the cost
will be increased significantly. Thus, an alternative way is to design a naked-eye
stereomeasurement GUI without special hardware in order to keep the cost down.
Fig. 2 illustrates the conceptual scheme of the
stereo-measurement based on photogrammetric
top-down and bottom-up techniques (Rau et
al., 2004). The operator digitize one point on
the reference imagery (x1,y1). Given any initial
ground height (Z1) and utilizes the inverse
collinearity equations we can calculate its
corresponding ground coordinates (X1,Y1). In
the meantime, we can back-project (X1,Y1,Z1)
to the search imagery and get its corresponding
image coordinate (x2,y2) using the collinearity
equations. If the ground height (Z1) is not
correct the image feature on both images will
be different. The operator may need to adjust
the ground height and inspect the location on
the search imagery until it locates at the
conjugate point on the ground. In the end, we
can get the 3D coordinates of one point. When
digitizing on the other corner of the same roof-
edge with same height, the operator need only
focus on the reference imagery without
changing of ground height. The result will
reflect on the search image automatically.
Fig. 1. Diagram of the proposed system.

Fig. 3 depicts the designed GUI interface. In which the left window is used for arameters setup and
control panel. The upper-middle window is the ground plan of the building models. The upperright
window is a 3D view of the processed buildings and all visible images for treated buildings. The
lower two windows are the left and right images chosen from all visible images. Fig. 4~6 illustrates
some examples of the provided editing tools. Figure 4 illustrates a tool for digitizing a flat roof. It
start from add one point on the left image but its height may not correct (Fig. 4-A). Then we can
adjust its height and verify the back-projection result on the right image until they correspond to the
same location (Fig. 4-B). If the roof height does not change, the digitization of other roof corners
can focus on the left image only (Fig. 4-C~F). For adding 3D poly-line, the procedure is the same
as adding 3D polygon except for the closing of the start and the final vertices.

Fig. 2. Stereo measurement scheme. Figure 3. The designed GUI.

 Figure 4. A tool for digitizing rooftop by means of naked-eye stereo-measurement.

2.2 Building Model Reconstruction

In this paper, the result after building reconstruction is the polyhedral building models. Each roof
patch is a planar surface with vertical wall. In principle, the SMS algorithm needs 3D structural
lines for building model reconstruction. However, for cylindrical and spherical dome, many
roofpatches are required to describe its surface. It means that a large quantity of structure lines is
necessary. In the proposed system, we thus adopt the model-driven approach to model this kind of
building. A temporary parametric model is utilized and converted into a number of polyhedrons at
the end. Thus, the required labor can be reduced significantly and a promising result can be
achieved. Except for pyramidal roof that can directly reconstructed by one 3D point and its ground
plan, the other types of roof were reconstructed by the Split-Merge-Shape method. A detail
description of the SMS algorithm can be found in the literature (Rau & Chen, 2003). In this paper,
we focus only on the parametric building modeling.

In the parametric building modeling, only rooftops in cylindroids or spherical dome were
considered. The cylindrical roof has a vertical cross-section in circle and a ground plan in
rectangular. For spherical dome, it has a vertical and horizontal cross-section in circle. In
polyhedral building modeling, the cylindrical roof was generalized as a gambrel roof and the
spherical dome was transformed into a combination of quadrilateral patches. Equation (1) is used
for parameter fitting for the cross-section of a cylindrical roof after rotation with respect to its
principal axis. In which, three unknown parameters were introduced, i.e. the radius (r) of the
cylinder and the coordinates (a, b) of its center. Equation (2) is used for parameter fitting of a
spherical dome. In which, four unknown parameters were introduced, i.e. the radius (r) of the
sphere and the coordinates (a, b, c) of its center. In the end, a transformation from parametric model
to polyhedral model is necessary and a certain degree of error will be introduced. It was
implemented by a given threshold, called tolerance error, as shown in Fig. 5.
3. CASE STUDY

The case study site is located around the Taipei City Government with an area of 5,376 hectares.
The airborne imagery was taken by the UltraCamD digital camera with an image scale about
1:13,800 and a ground sampling distance (GSD) around 12.4 cm. In the paper, four case studies
were depicted to demonstrate the ability of the proposed system.

Fig. 6 and Fig.7 illustrate two case studies that adopt the proposed SMS method for building
modeling. Normally, one can digitize the structural lines from the highest roof to the lowest one.
For the highest roof, the surrounding eaves are always visible, so a 3D polygon is suggested. Fig. 6-
A and Fig. 7-A are the stereo-imagery overlaid with the digitized 3D polygons (in blue color) and
3D structural lines (in magenta color), respectively. Fig. 6-B and Fig. 7-B are the 3D view of the
3D roofs and 3D structural lines before building reconstruction. One can realize that the 3D
linesegments are not complete and the hidden corners were not measured. Fig. 6-C and Fig. 7-C are
the results after building reconstruction. In case study 2, i.e. Fig. 7, there are two additional 3D
points digitized and two pyramidal roofs were reconstructed. Those results demonstrate that the
performance and capability of the invented SMS method is quite well.

Fig. 8 demonstrates another two case studies based on the model-driven approach. Case 3 and case
4 are applied for cylindrical roof and spherical dome, respectively. For cylindrical roof, only one
planar roof with one ridge line is required, as shown in Fig. 8. In the meantime, for the modeling of
a spherical dome, i.e. Fig. 9, one needs only to digitize its circular boundary and one 3D point on its
rooftop. For isolated buildings, a pre-processing for regularization could be applied before
parameter fitting, in order to obtain a reasonable result. For example, the central location of a
spherical dome can be estimated by averaging the planimetric coordinates of all vertices. A
tolerance error of 10 cm was utilized during transformation. The results demonstrate that a
promising result was achieved by the propose approach. Since the operational procedure is
intuitive, an inexperience operator can adapt the system in a short time.

(A)
(A)

(B) (C) (B) (C)

Figure 6. Case study 1 of building modeling Figure 7. Case study 2 with two pyramids.
result.

4. CONCLUSIONS

This paper presents a prototype of naked-eye stereo-measurement system for 3D building

modeling. The kernel for building reconstruction is a hybrid method that integrates data-driven and
 modeldriven approach. For data-driven approach, the SMS method was adopted, which utilizes 3D
Figure 6. Case study 1 of building modeling result.

Figure 7. Case study 2 with two pyramids. structural line for building modeling. Since the 3D
structural line is the basic feature of line graph in conventional 2D mapping, thus the adopted
technique could be integrate with digital photogrammetric workstation seamlessly. For model-
driven approach, two kinds of models were implemented, i.e. spherical dome and cylindrical roof.
It is estimated that the proposed hybrid system for building modeling can be applied for more 99%
of all buildings in Taiwan.

Figure 8. Case study 3 for cylindrical roof Figure 9. Case study 4 for spherical dome
modeling. modeling.

ACKNOWLEDGEMENT

This paper was partial supported by the Ministry of the Interior and the National Science Council
(NSC-95-2221-E-008-101), Taiwan. The authors would like to thank the Taipei City Government
for providing the digital topographic map, UltraCamD digital images with aero-triangulation data.

REFERENCES

1. Brenner, C., 2000. “Towards fully automatic generation of city models”, IAPRS, Vol. 33, Part.
B3, Amsterdam, Netherlands, pp.84-92.
2. Chen, L.C., Teo, T. A., Hsieh, C. H., and Rau, J. Y., 2006, Reconstruction of Building Models
with Curvilinear Boundaries from Laser Scanner and Aerial Imagery, Lecture Notes in
Computer Science. LNCS 4319,pp.24-33.
3. Grün, A. and X. Wang, 1998, CC-Modeler: A Topology Generator for 3-D Building Models,
IJPRS, Vol.53, pp.286-295.
4. Grün, A. and X. Wang, 2001. “News from CyberCity-Modeler”, in Automatic Extraction of
Man-Made Object from Aerial and Space Images (III), (Eds. Baltsavias, E.P., A. Grüen, and
L.Van Gool), Centro Stefano Franscini, Monte Verita, Ascona, 2001, pp.93-102.
5. Rau, J. Y., Chen, N.Y, and Chen, L. C., 2002, “True Orthophoto Generation of Built-Up Areas
Using Multi-View Images”, Photogrammetric Engineering and Remote Sensing, Vol. 68, No.
6, pp. 581-588.
6. Rau, J. Y., Chen, L. C., and Wang, G. H., 2004, “An Interactive Scheme for Building Modeling
Using the Split-Merge-Shape Algorithm”, International Archives of Photogrammetry and
Remote Sensing, Vol. 35, No. B3, pp. 584-589.
7. Rau, J. Y., and Chen, L. C., 2003, “Robust Reconstruction of Building Models from Three-
Dimensional Line Segments”, Photogrammetric Engineering and Remote Sensing, Vol.
69.No.2, pp. 181-188.
8. Stereo Analyst, Leica Geosystems, 2007, http://gi.leica-eosystems.com/LGISub1x30x0.aspx,
(Access: April, 30, 2007)