CONVR 2012 Proceedings

CONVR 2012
Proceedings of 12
th
International Conference
on Construction Applications of Virtual Reality

November 1-2, 2012, Taipei Taiwan

Edited by
Yu-Cheng Lin
National Taipei University of Technology, Taiwan
Shih-Chung Jessy Kang
National Taiwan University, Taiwan

12th International Conference on Construction Application of Virtual Reality
Organizations
Organizer

National Taiwan University
Supporting Organizations and Sponsors

National Science Council

Bureau of Foreign Trade

Ministry of Education

Smart Aging Alliance
Autodesk Taiwan Ltd.

CECI Engineering Consultants, Inc., Taiwan

Chien Kuo Construction Co.,Ltd.

China Engineering Consultants Inc., Taiwan

Moh and Associates, Inc.

Sinotech Engineering Consultants, Ltd.

ISBN: 978-986-03-4289-5
All rights reserved 2012
No part of this work may be reproduced, stored in a retrieval system, or transmitted in any
form or by any means, electronic, mechanical, photocopying, microfilming, recording or
otherwise, without written permission from the Publisher, with the exception of any material
supplied specifically for the purpose of being entered and executed on a computer system, for
exclusive use by the purchaser of the work.

Published and printed at National Taiwan University Press, Taiwan, by University Publishing
Service.
November 1-2, 2012, Taipei, Taiwan
International Advisory Committee:

Shang-Hsien Hsieh (Chair), National Taiwan University, Taiwan
Chimay J. Anumba, The Pennsylvania State University, USA
Nashwan Dawood, Teesside University, UK
Feniosky Pea-Mora, Columbia University, USA
Lucio Soibelman, University of Southern California, USA
Xiang-Yu Wang, Curtin University, Australia
Nobuyoshi Yabuki, Osaka University, Japan

Organizing Committee:

Shih-Chung Jessy Kang (Chair), National Taiwan University, Taiwan
Hung-Ming Chen (Co-Chair), National Taiwan University of Science and Technology, Taiwan
Albert Y. Chen (General Secretary), National Taiwan University, Taiwan
Yu-Cheng Lin (Publication), National Taipei University of Technology, Taiwan
Po-Han Chen (Program), National Taiwan University, Taiwan
Su-Ling Fan (Local Arrangement), Tamkang University, Taiwan
Chien-Cheng Chou (Website), National Central University, Taiwan
I-Chen Wu (Registration), National Kaohsiung University of Applied Sciences, Taiwan
Kuo-Liang Kevin Lin, I-Shou University, Taiwan

i
International Scientific Committee:

Changbum Ahn, University of Nebraska-Lincoln, USA
Burcu Akinci, Carnegie Mellon University, USA
Homan Bahnassi, Concordia University, Canada
Amir Behzadan, University of Central Florida, USA
Manfred Breit, University of Applied Sciences Northwestern Switzerland, Switzerland
Ioannis Brilakis, University of Cambridge, UK
Gerhard Girmscheid, ETH Zrich, Switzerland
Mani Golparvar-Fard, Virginia Tech, USA
Jie Gong, Rutgers, The State University of New Jersey, USA
Ning Gu, The University of Newcastle, Australia
SangUk Han, University of Illinois, USA
Sangwon Han, University of Seoul, Korea
YoMing Hsieh, National Taiwan University of Science and Technology, Taiwan
Vineet Kamat, University of Michigan, USA
Puteri Shireen Jahn Kassim, International Islamic University Malaysia, Malaysia
Hiam Khoury, American University of Beirut, Lebanon
Tsuneya Kurihara, Central Research Laboratory, Hitachi Ltd., Japan
Kincho Law, Stanford University, USA
Ghang Lee, Yonsei University, Korea
SangHyun Lee, University of Michigan, Ann-Arbor, USA
Fernanda Leite, The University of Texas at Austin, USA
KenYu Lin, University of Washington, USA
Emmajane Mantle, University of Glamorgan, UK
Katsuhiko Muramoto, Pennsylvania State University, USA
Abbas Rashidi, Georgia Institute of Technology, USA
XinYi Song, Georgia Institute of Technology, USA
Kenichi Sugihara, Gifu-Keizai University, Japan
Georg Suter, Vienna University of Technology, Australia
PingBo Tang, Arizona State University, USA
Menghan Tsai, National Taiwan University, Taiwan
Nobuyoshi Yabuki, Osaka University, Japan
Zhenhua Zhu, Concordia University, Canada
ii
Preface

We are very pleased to hold the 12th International Conference on Construction Applications
of Virtual Reality (CONVR 2012) at National Taiwan University, Taipei, Taiwan. The first
CONVR was held in Teesside, UK in 2000, and the conferences since have been held in
Chalmers, Sweden (2001), Blacksburg, VA, USA (2003), Lisbon, Portugal (2004), Durham,
UK (2005), Orlando, FL, USA (2006), Penn State, PA, USA (2007), Kuala Lumpur, Malaysia
(2008), Sydney, Australia (2009), Sendai, Japan (2010), and Weimar, Germany (2011). We
would like to invite you this year to Taipei, a city well known for its welcoming attitude
towards foreign visitors to Taiwan.
Information and communication technology has advanced rapidly and with the utilization of
computer-based visualization is quite developed and widely applied. The progress of Virtual
Reality (VR), Augmented Reality (AR), and Building Information Models (BIM) is
particularly swift. These forward-looking technologies offer the potential for considerable
benefits at all stages of implementation, from initial planning and conceptual design to facility
management and operations.
CONVR 2012 will bring together researchers and practitioners from a variety of fields such as
architecture, civil engineering, and construction to exchange ideas and methods for new VR,
AR, and BIM applications. The aim of this conference is to report ideas, improvements,
products, and applications that are innovative and for international research activities and
strategies within the fields of Architecture, Engineering, and Construction (AEC).
In response to our Call for Papers, we received 102 abstracts, of which 64 were finally
accepted by the International Scientific Committee after two rounds of rigorous review. The
proceedings include all 64 accepted papers and four keynote speakers. We are very pleased
with the quality of conference papers and wish to thank the authors for all their efforts in
contributing to our conference.
We hope that CONVR 2012 and the publication of the proceedings will contribute to the
development and application of information and communication technology for AEC long
into the future. Finally, we hope that you enjoy the conference papers and will join us at future
CONVR events.

With Warmest Regards,

Yu-Cheng Lin (Editor, National Taipei University of Technology)
Shih-Chung Jessy Kang (Conference Chair, National Taiwan University)

iii

TABLE OF CONTENTS
Organizations ........................................................................................................... i
Preface .................................................................................................................... iii
Keynote Speech
CREATING SMARTER FACILITIES AND INFRASTRUCTURE
USING SENSORS, DATA ANALYTICS, VISUALIZATION AND
INTELLIGENT DECISION SUPPORT: OPPORTUNITIES,
CHALLENGES, AND RECENT RESEARCH AT CARNEGIE
MELLON UNIVERSITY ..................................................................................... 1
James H. Garrett
TAIWANS BUILDING INFORMATION MANAGEMENT
DEVELOPMENT, OPPORTUNITIES AND CHALLENGES FROM
AN INDUSTRYS APPLICATION PERSPECTIVE ........................................... 3
Richard Moh
VISUALISATION AND OPTIMISATION TOOLS FOR CO
2

REDUCTION IN BUILT ENVIRONMENT ........................................................ 5
Nashwan Dawood
CONTENT-DRIVEN USER EXPERIENCE ....................................................... 7
Shuen-Huei Guan

I. Building Information Modeling (BIM) .......................................... 9
INTEGRATIVE APPLICATION OF BIM AND TAIWAN GREEN
BUILDING STANDARDS TO
BUILDING ENVELOPES ................................................................................. 11
Po-Han Chen, Po-Chuan Chuang & Meng-Shen Kan
THE DEVELOPMENT OF WEB BIM-BASED INTERFACE
COMMUNICATION SYSTEM IN CONSTRUCTION .................................... 19
Yu-Chih Su, Meng-Chung Lee, Chih-Yuan Li & Yu-Cheng Lin
COUNTABLE BENEFITS OF VISUALIZING ONE-OF-A-KIND
PROCESSES ....................................................................................................... 29
Hans-Joachim Bargstaedt
DIGITAL MEDIATED TECHNIQUES FOR THE KNOWLEDGE OF
THE EVOLUTIONARY PECULIARITIES OF THE BUILT
v
ENVIRONMENT: THREE CASE STUDIES .................................................... 37
Danilo Di Mascio
APPLYING BUILDING INFORMATION MODELING IN
EVALUATING BUILDING ENVELOPE ENERGY PERFORMANCE .......... 47
Cheng-Yuan Hsieh & I-Chen Wu
BIM SUPPORTED DATA VISUALIZATION STRATEGIES FOR
FACILITY MANAGEMENT ............................................................................. 54
Anne Anderson, Ken-Yu Lin & Carrie Sturts Dossick
WORK PLANNING BASED ON A BUILDING INFORMATION
MODEL .............................................................................................................. 64
Sebastian Hollermann & Hans-Joachim Bargstdt
TRANSFORMATION AND INTEGRATION OF BUILDING
INFORMATION MODEL FOR DISASTER MITIGATION ............................ 73
Chia-Ying Lin & Chien-Cheng Chou
DEVELOPMENT OF A VISUAL DOMESTIC ENERGY
ASSESSMENT TOOL ........................................................................................ 80
A. Mhalas, T. Crosbie & N. Dawood
STUDYING CCTV COVERAGE IN AN MRT STATION USING
BIM-BASED VR APPROACH .......................................................................... 90
Huan-Ting Chen, Si-Wei Wu & Shang-Hsien Hsieh
A 3D-WEB ENABLED FRAMEWORK FOR DELIVERING DIGITAL
FACILITY ASSET MANAGEMENT ................................................................ 99
Edvinas Rasys, Michael Hodds & N. N. Dawood
BUILDING-ENVELOPE-BASED INTEGRATION OF ENERGY
SAVING AND COST EFFECTIVENESS USING BIM .................................. 107
Po-Han Chen, Long Chan & Jeong-Shing Lee
BIN-BASED APPLICATION DEVELOPMENT OF UNDERGROUND
MRT STATION EMERGENCY EVACUATION-TIME EVALUATION
FOR TAIPEI MRT DESIGN CHECKING ....................................................... 113
C.H. Wang, R. Moh, S.M. Kang, W.L. Lee, T.J. Pei & S.H. Hsieh
INFORMATION CAPTURE AND REPRESENTATION OF
MODEL-BASED DESIGN REVIEW PROCESS IN VIRTUAL
ENVIRONMENTS ........................................................................................... 122
Li Wang, Junlin Yi & Fernanda Leite
vi
A BIM-BASED VISUALIZATION SYSTEM FOR STEEL ERECTION
PROJECTS SCHEDULING ............................................................................. 132
Hsiang-Wei Lu, Wan-Li Lee, Chih-Chuan Lin & Po-Ming Huang
INTEGRATION OF BIM AND CONSTRUCTION SIMULATION FOR
SCHEDULING ................................................................................................. 142
Ming-Tsung Lee, Cheng-Yi Chen, Shao-Wei Weng, Shih-Hsu Wang,
Wei-Chih Wang, Jang-Jeng Liu, Yuan-Yuan Cheng, Ming-Feng Yeh,
Bing-Yi Lin & Cheng-Ju Kung
ROBUST MATERIAL RECOGNITION FOR AUTOMATED
BUILDING INFORMATION MODELING FROM UNORDERED SITE
IMAGE COLLECTIONS ................................................................................. 152
Mani Golparvar-Fard, Andrey Dimitrov & Feniosky Pea-Mora
BIM-BASED JOB HAZARD ANALYSIS ....................................................... 162
Jrgen Melzner & Hans-Joachim Bargstdt
AUTOMATED AS-BUILT MODELING WITH SPATIAL AND
VISUAL DATA FUSION ................................................................................. 172
Zhenhua Zhu
IMMERSIVE VIRTUAL REALITY SYSTEM FOR BIM .............................. 183
Julian Kang, Adithya Ganapathi & Hussam Nseir
MAPPING BIM MODELS AND 3D GIS MODELS USING
INSTANCE-BASED AND LINGUISTIC METHOD ...................................... 187
Yichuan Deng & Jack Chin Pang Cheng
A STUDY ON DATA EXTRACTION FROM BIM MODELS FOR
KNOWLEDGE MANAGEMENT ................................................................... 198
C.L. Hung & S.A. Chou
USER REQUIREMENTS FOR MOBILE AR AND BIM UTILIZATION
IN BUILDING LIFE CYCLE MANAGEMENT ............................................. 203
Timo Kuula, Kalevi Piira, Anu Seisto, Mika Hakkarainen & Charles
Woodward
EXPLORE PATTERNS OF HUMAN-COMPUTER INTERACTIONS
IN PARAMETRIC DESIGN ENVIRONMENTS ............................................ 212
Rongrong Yu, Ning Gu & Michael Ostwald
vii
II. Augmented Reality (AR) and Sensing ...................................... 223
AUGMENTED REALITY TECHNOLOGIES FOR AEC PROJECTS: A
LITERATURE REVIEW .................................................................................. 225
Sarah Rankouhi & Lloyd M Waugh
AUGMENTED REALITY MARKERS AS SPATIAL INDICES FOR
INDOOR MOBILE AECFM APPLICATIONS ............................................... 235
Chen Feng & Vineet Kamat
REAL-TIME MICRO PLANNER FOR CRANE OPERATIONS ................... 243
Wei Han Hung & Shih Chung Kang
ENHANCED HD
4
AR (HYBRID 4-DIMENSIONAL AUGMENTED
REALITY) FOR UBIQUITOUS CONTEXT-AWARE AEC/FM
APPLICATIONS ............................................................................................... 253
Hyojoon Bae, Mani Golparvar-Fard & Jules White
COMMUNICATION SUPPORT SYSTEM FOR WORKERS AND
SUPERVISORS BY MULTIPLE-CAMERA HANDLING AND
AUGMENTED REALITY TECHNIQUES ..................................................... 263
Takehiro Urano, Tsuneya Kurihara, Yuki Watanabe, Tatsuhiko
Kagehiro, Tetsuya Tsubokura, Fumio Hatori & Yuichi Yashiro
IMPROVING THE BENEFIT OF VIRTUAL REALITY SESSION
DOCUMENTATION THROUGH AUGMENTED REALITY ........................ 271
Martin Heinig, Axel Friedewald & Hermann Ldding
MOBILE AUGMENTED REALITY: BRIDGING THE GAP
BETWEEN REAL AND VIRTUAL IN SUSTANINABLITY
EDUCATION .................................................................................................... 282
Katsuhiko Muramoto & Vui Huynh
A FRAMEWORK FOR UTILIZING CONTEXT-AWARE
AUGMENTED REALITY VISUALIZATION IN ENGINEERING
EDUCATION .................................................................................................... 292
Amir H. Behzadan, Vineet R. Kamat
2C - MOBILE COLLABORATIVE FIRE HAZARD DETECTION
SYSTEM ........................................................................................................... 300
Hugo M. da Silva, Joo P. Jacob, Antnio F. Coelho & Rui Rodrigues
VIRTUAL PROTOTYPING FOR ROBOTIC FABRICATION OF
REBAR CAGES IN MANUFACTURED CONCRETE
CONSTRUCTION ............................................................................................ 309
Akash Garg & Vineet R. Kamat
viii
PERFORMANCE BETWEEN GENDERS IN ASSEMBLY TASK
RELEVANT TO MEMORIZATION AFTER TRAINING USING AR
AND MANUAL PRINTS ................................................................................. 319
Lei Hou & Xiangyu Wang
EFFECTIVENESS OF VIRTUAL REALITY ON PRESENTING FIELD
MONITORING DATA ...................................................................................... 323
Yo-Ming Hsieh & Ya-Sue Liu
SOLAR RADIATION SURVEY METHOD WITH IMAGE STITCHING
............................................................................................................................. 332
Yoshihiro Yasumuro, Takashi Shimomukai, Hiroshige Dan & Masahiko
Fuyuki
PHOTOGRAMMETRIC MODELING AND SLING LENGTH
MEASUREMENTS OF A RIGGING SYSTEM ENGINEERED FOR
INDUSTRIAL MODULE ERECTION ............................................................ 340
Ming-Fung Francis Siu, Sheng Mao, Ming Lu & Simaan Abourizk
NAVIGATION IN VIRTUAL REALITY USING MICROSOFT KINECT
............................................................................................................................. 350
Florian Kammergruber, Andreas Ebner & Willibald Gnthner
EMOTIONAL 3D EXPERIENCE IN AUGMENTED HERITAGE
REALITY THROUGH ADVANCED TECHNOLOGY FUSION ................... 360
Matteo Fabbri, Emanuele Borasio, Daniel Blersch & Christoph
Froehlich

III. 3D/4D Modeling ......................................................................... 367
4D VISUALIZATION OF RAILROAD TRANSPORT OPERATIONS
OF EQUIPMENT TO A POWER GENERATION FACILITY ........................ 369
Brandon S. Reisser & John C. Hildreth
A SPACE MODEL FOR PERSONAL LIGHTING CONTROL ...................... 379
Georg Suter, Filip Petrushevski & Milos Sipetic
AUTOMATIC GENERATION OF 3D BUILDING MODELS BY
STRAIGHT SKELETON COMPUTATION .................................................... 389
Kenichi Sugihapa & Zhen-jiang Shen
A 3D CAD ENGINE WITH DATA MODELS BASED ON
INTERNATIONAL STANDARDS .................................................................. 399
ix
Satoshi Kubota, Etsuji Kitagawa, Kantaro Monobe, Kenji Nakamura &
Shigenori Tanaka
EMERGENCY MEDICAL DISPATCH: A CASE STUDY OF NEW
TAIPEI CITY .................................................................................................... 407
Hsiao-Hsuan Liu & Albert Y. Chen

IV. Real Time Visualization ............................................................. 415
FROM PHYSICAL TO VIRTUAL: REAL-TIME IMMERSIVE
VISUALISATIONS FROM AN ARCHITECTS WORKING MODEL ......... 417
Gerhard Schubert, Christoph Anthes, Dieter Kranzlmller & Frank
Petzold
3D MODEL VR APPLIED IN TOURISM PLANNING-TAIWAN
TAOYUAN HAKKA FOLK TOURISM PLANNING FOR EXAMPLE ....... 427
Ren-Jwo Tsay & Jian-Yuan Chen
INCREASED INTERACTION WITH MULTI-USER VIRTUAL
REALITY IN CONSTRUCTION PROJECTS ................................................ 434
Janne Porkka, Nusrat Jung, Sunil Suwal, Pivi Jvj, Anssi Savisalo,
Jani Pivnen & Jarkko Sireeni
A HEATSTROKE PREVENTION SYSTEM FOR CONSTRUCTION
WORKERS USING SIMULATION AND VR ................................................ 443
Nobuyoshi Yabuki, Takuya Onoue, Tomohiro Fukuda & Shinji Yoshida
VIRTUAL REALITY: FACTORS DETERMINING SPATIAL
PRESENCE, COMPREHENSION AND MEMORY ...................................... 451
Bimal Balakrishnan, Danielle Oprean, Brad Martin & Melina Smith
DIGITAL ECOLOGIES: A SHIFTING PARADIGM FOR
CONSTRUCTION ............................................................................................ 460
Robert Amor & Dermott McMeel
SERIOUS PLAY: INTUITIVE ARCHITECTURAL CONCEPTUAL
DESIGN WITH IMMEDIATE STRUCTURAL FEEDBACK AND
ECO-NOMICAL AND ECOLOGICAL PERFORMANCE
PREDICTIONS ................................................................................................. 467
Manfred Breit, Li Huang, Frank Lang, Fabian Ritter & Andr
Borrmann
VIRTUAL REALITY AND SCIENTIFIC VISUALIZATION APPLIED
IN TECHNICAL TRAINING IN BRAZILIAN CONSTRUCTION ............... 476
Fabiano Stange & Sergio Scheer
x
SURFING IN THE VIRTUAL DIGITAL LIBRARY A LINKED
DATABASE STRUCTURE FOR INTUITIVE
INFORMATION-SEEKING BEHAVIOR OF CHILDREN ............................ 486
Ko-Chiu Wu
AUTOMATED 3D HUMAN SKELETON EXTRACTION USING
RANGE CAMERAS FOR SAFETY ACTION SAMPLING .......................... 494
SangUk Han, Madhav Achar, SangHyun Lee & Feniosky Pea-Mora

V. Simulation and Planning ............................................................. 503
SIMULATION MODELLING FOR ORGANIZING THE BUILDING
MACHINERY WORK AT THE CONSTRUCTION SITE .............................. 505
Alexander Ginzburg
SIMULATION OF PROCESS INTERACTION FOR QUALITY
ASSURANCE DURING CONSTRUCTION .................................................. 510
Vitali Kochkine, Jrgen Schweitzer, Ilka von Gsseln & Nils Rinke
GENERATING WORKSPACE REQUIREMENTS IN A FINISHING
EXECUTION PHASE ...................................................................................... 521
Trang Dang, Amir Elmahdi & Hans-Joachim Bargstdt
ONSITE PLANNING OF 3D SCANNING FOR OUTDOOR
CONSTRUCTIONS WITH MATHEMATICAL PROGRAMMING .............. 532
Hiroshige Dan, Yoshihiro Yasumuro, Taisuke Ishigaki & Tatsuaki
Nishigata
GLOBAL PATH PLANNING IN 4D ENVIRONMENTS USING
TOPOLOGICAL MAPPING ............................................................................ 542
Vitaly Semenov, Konstantine Kazakov & Vladislav Zolotov
LOGISTICS AND LAYOUT PLANNING OF CONSTRUCTION
EQUIPMENT ON A VR-MULTI-TOUCH-TABLET ...................................... 552
Andreas Ebner, Florian Kammergruber, Tim Horenburg & Willibald
Gnthner
EXPLORING USER EXPERIENCE OF WAYFINDING IN A LARGE
AND COMPLEX HOSPITAL .......................................................................... 561
Mi Jeong Kim, Sooyeon Han & Xiangyu Wang
A SIMULATION DRIVEN VISUALIZATION APPROACH FOR
CONSTRUCTION MOBILE RESOURCES PLANNING .............................. 571
Amr El-Nimr, Yasser Mohamed & Simaan AbouRizk
xi
DEVELOPMENT OF GRID-BASED NETWORK MODEL FOR PATH
FINDING IN REAL DYNAMIC ENVIRONMENTS ..................................... 579
Tatsuru Tomii & Koji Makanae
Author Index ....................................................................................................... 587
xii
KEYNOTE SPEECH I

Prof. James H. Garrett, Thomas Lord Professor and Head in the Department of Civil and Environmental
Engineering, Carnegie Mellon University, USA.
Title: Creating smarter facilities and infrastructure using sensors, data analytics, visualization and intelligent
decision support: opportunities, challenges, and recent research at Carnegie Mellon University
Abstract:
The U.S. infrastructure is a trillion dollar investment, defined broadly to include road systems and bridges, water
distribution systems, water treatment plants, power distribution systems, telecommunication network systems,
commercial and industrial facilities, etc. In spite of the enormous investments made in these systems and their
importance to the US economy, they are in a significantly deteriorated state. Sensing technologies, data
modeling and mining approaches, advanced visualization and decision support can be deployed to improve the
performance and/or reduce the life-cycle cost and societal impacts of all life-cycle phases of facilities and
infrastructure systems.
The Pennsylvania Smarter Infrastructure Incubator at Carnegie Mellon, which includes the Center for Sensed
Critical Infrastructure, the IBM Smarter Infrastructure Lab at Carnegie Mellon, the Facility/Infrastructure
Information Modeling and Visualization Lab and the Facility/Infrastructure Analytics Lab, is performing
research on what actionable information about the condition and usage of our facilities and infrastructure
systems is needed and how best to capture, model and reason about it so as to improve the maintenance and
operation of this infrastructure.
In this talk, I will give a brief overview of the research going on in the PSII and then present more details about
several projects: 1) a framework for gathering and integrating data about HVAC systems, stored in a number of
standardized data models, so that large numbers of available analyses, decision support functions and
visualizations can be readily conducted using the collected and integrated data; 2) an approach for gathering and
visualizing work orders in facilities so as to better understand maintenance hotspots in a facility and where
building information models may need to be updated to remain accurate; 3) an approach for using multiple laser
scans in time to identify variations between reality and building information models; 4) an approach for
evaluating how well a building information model can improve the accuracy of indoor localization algorithms,
which are extremely important for emergency response and facility management; 5) representing and visualizing,
in building information models, the data about damaged building components collected using laser scanners and
other sensor systems; and 6) the use of 3D immersive visualization for supporting decision making regarding the
design and evaluation of energy efficient buildings.
This work has been done in collaboration with my colleagues Professors Burcu Akinci, Mario Berges and
Semiha Ergan in our Advanced Infrastructure Systems Group at Carnegie Mellon and our students: Xuesong
(Pine) Liu, Asli Akcamete, Te Gao, Saurabh Taneja, Engin Anil, and Sheryl Yang.

1
Bio of Prof. Garrett:
James H. Garrett, Jr. is the Thomas Lord Professor of
Civil and Environmental Engineering at Carnegie Mellon
University. He has been the head of the department
since 2006. He is currently a faculty-co-director of the
Pennsylvania Smarter Infrastructure Incubator (PSII),
which is a research center aimed at creating and
evaluating sensing, data analytics and intelligent decision
support for improving the construction, management and
operation of infrastructure systems. Garrett also currently
serves as the co-Chief Editor of the ASCE Journal of
Computing in Civil Engineering. Garrett received his BS
(1982), MS (1983) and PhD (1986) in Civil Engineering
from Carnegie Mellon University. He is a registered
professional engineer in the state of Texas.
Garretts research and teaching interests are oriented
toward applications of sensors and sensor systems to civil infrastructure condition assessment; application of
data mining and machine learning techniques for infrastructure management problems in civil and environmental
engineering; mobile hardware/software systems for field applications; representations and processing strategies
to support the usage of engineering codes, standards, and specifications; knowledge-based decision support
systems. Garrett has published his research in over 60 refereed journal articles, over 80 refereed conference
papers, over 90 other conference papers and 10 sections or chapters in books or monographs.
Garrett was awarded the 2007 Steven J. Fenves Award for Systems Research at Carnegie Mellon, the 2006
ASCE Computing in Civil Engineering Award, and the ASCE Journal of Computing in Civil Engineering Best
Paper Award in 2001 for the paper he co-authored with Han Kiliccote, entitled "Standards Usage Language
(SUL): An Abstraction Boundary between Design Systems and Standards Processors." He is a co-recipient of the
1993 ASCE Wellington Prize for his paper entitled "Knowledge-Based Design of Signalized Intersections,"
which he co-authored with Rahim Benekohal and Jeffrey Linkenheld. He is also a co-recipient of the 1990
ASCE Moisseiff Award for his paper entitled "Knowledge-Based Standard-Independent Member Design", which
he co-authored with Steven J. Fenves. In 1994, he was also a Humboldt Stipendiat and spent 6 months at the
University of Karlsruhe and the Technical University Munich.

2
KEYNOTE SPEECH II

Mr. Richard Moh, Senior Vice President, Corporate Development , Special Assistant to the Chairman,
MAA Group Consulting Engineers, Taiwan
Title: Taiwans Building Information Management Development, Opportunities and Challenges from an
Industrys Application Perspective
Abstract:
Due to the advancement of computer software and hardware, Building Information Management (BIM) is
revolutionizing the Architecture, Engineering and Construction (AEC) industrys practices worldwide. Plenary
Session II will introduce the developments, opportunities and challenges of BIM development in Taiwan through
project examples carried out by an engineering consulting firm. The presentation will share how an
engineering consulting firm adopts the BIM concept under current industry environment and the challenges
faced in near future.

3
Bio of Mr. Moh:
Richard Moh is the Corporate Development Senior
Vice President and Special Assistant to the Chairman
of MAA Group Consulting Engineers. MAA Group is
a leading 900 people multidisciplinary engineering
consulting firm in East/Southeast Asia established in
1975. He is a holder of Project Management
Professional and is the Chairman of Young Engineers
Committee and Vice Chairman of International &
Cross Strait Affairs Committee of the Chinese
Association of Engineering Consultants. He is also
the Director of Construction Management Association
of the Republic of China. Educated with an
undergraduate degree in civil engineering and a
masters degree in engineering management at Cornell
University and an MBA degree from Wharton School
of Management, and having lived in many countries, Richard brings a unique set of skills to the company
management. Richard is responsible of the corporate development of MAA Group, which includes general
management improvements and the development of new ideas such as Building Information Modeling (BIM).
Since 2009, he has been leading the company in the commitment of adopting BIM in its services to achieve the
vision of integrating engineering and architecture disciplines.

4
KEYNOTE SPEECH III

Prof. Nashwan Dawood, Professor and Director of TFI, Teesside University, UK
Title: Visualisation and Optimisation Tools for CO2 Reduction in Built Environment
Abstract:
While many cities in Europe and in the world have issued climate change policies and set targets for reduction of
CO2 emissions, their implementation in built environment and in particular urban planning practice remains a
major challenge. Devising effective decision systems which support CO2 emissions reduction demands a
systems approach which enables different actors -policy makers, planners, engineers, consultants, and
inhabitants- to correlate a diversity of problems, spanning across distinct domains and geographic scales.
Visualisation and optimisation technologies are set to play a major role in enhancing the decision making
processes and to improve our understating of the consequences of different intervention strategies. The
presentation will include experience from two EU projects that have bee awarded to Teesside University in
conjunction with EU academics and industry partners.
The EU projects to be presented, INTUBE (Intelligent Analysis of Energy Information) and SEMANCO
(Semantic Technologies for Carbon Reduction in Urban Planning). The technical contents of these projects are
based on the integration of energy related open data structured according to standards, semantically modelled
and interoperable with a set of tools for visualizing, simulating and analyzing the multiple interrelationships
between factors determining CO2 production. For the SEMANCO project, a Semantic Energy Information
Framework (SEIF) is being developed to model the energy-related knowledge planners and decision makers
need. The tools interoperating with the framework will support systems innovation and include available
technologies, enhancements to existing open source platforms, and new technological solutions. SEMANCO
carries out an analysis requirements to support the application of the tools by the different stakeholders involved
in energy related urban planning. The development of the visualisation and optimisation tools and methods will
be informed by three case study scenarios in Spain, UK and Denmark which will cover three geographical scales
-neighbourhood, municipal and regional- including both existing and new urban areas. The case studies will
identify the relevant indicators and the interrelationship between factors contributing to CO2 production in the
analysed urban areas. Based on this analysis, tools and methods are being developed as the project progress.
Their application within the cases of study will demonstrate quantifiable and significant reduction of energy
consumption and CO2 emissions achieved through ICT and will make it possible to assess their social impact.
Keywords: Visualisation, Pptimisation, CO2 reduction in built environment

5
Bio of Prof. Dawood:
Professor Dawood, initiator of CONVR conference series, is
currently Director of the Centre for Construction Research &
Innovation (CCIR) and Professor of Construction Management
and IT at the University of Teesside, UK. He is also Director of
the Technology Futures Institute, through which the
engineering and technology research is structured and
supported. This role includes responsibility for developing and
promoting research polices throughout the institution.
Professor Nashwan Dawood is a specialist in project
construction management and the application of IT in the
construction process. This has ranged across a number of
research topics including BIM technologies and processes,
sustainability, Information Technologies and Systems
(5D,VR,ICT for energy efficiency), risk management,
intelligent decision support systems, cost forecasting and
control and business processes.
Professor Dawood has extensive experience of leading
internationally recognised research work in BIM technology and processes and in the application of 5D
modeling in construction processes, and has successfully generated peer reviewed funded projects from the
Engineering and Physical Sciences Research Council, the Technology Strategy Board, and EU / Framework
Programme. This has resulted in more than 200 peer reviewed journal and conferences publications
Professor Dawood also has significant and long-standing experience of working with major industrial partners in
the UK and internationally to develop and apply research results as part of further collaborative projects. In
particular he is currently running international research and development projects in South Korea, Japan, Qatar,
Europe and USA in the areas of 5D modeling, serious game engine technologies application to training and ICT
(Information and Communication technologies) for energy efficient buildings . He is also regularly invited to be
a key note presenter in a international events.

6
KEYNOTE SPEECH IV

Mr. Shuen-Huei Guan,R&D manager of Digimax Inc., Taiwan
Title: Content-Driven User Experience
Abstract:
By a narrow definition, virtual reality is an artificial environment that is presented in such a way that audiences
believe and accept it as a real environment. Virtual reality aims to provide, enhance or even create a perceptible
world that is firmly believed to exist whether it is real or imaginary. It is a science about perception, recognition,
and presentation that permits real-time experiences and improvisation.
Content presentation that allows real-time experiences and improvisation is about emotions, context and
storytelling. Human beings convey their experiences via a wide-range of methods, and the application of
computer generated animations, like virtual reality, is just one of them. I would like to share my experience on
how to create "reality" through digital content.

7
Bio of Mr. Guan:
Shuen-Huei Guan is R&D manager of Digimax Inc., an animation
studio located at Taipei/Taiwan, aiming to produce influencing
feature animations/films. He has been involved in animation
industry for eight years, participating titles of "Adventures the
National Palace Museum", "Quantum Quest: A Cassini Space
Odyssey", opening ceremony of 2010 Asian Games, and others.
Shuen-Huei Guan is currently Ph.D. candidate in Graduate Institute
of Networking Multimedia, National Taiwan University. During
production in Digimax, he also published several works to ACM
SIGGRAPH, IEEE journal, and other related conferences.

8

I. BUILDING INFORMATION MODELING (BIM)

9
10

INTEGRATIVE APPLICATION OF BIM AND TAIWAN GREEN
BUILDING STANDARDS TO BUILDING ENVELOPES
Po-Han Chen, Po-Chuan Chuang & Meng-Shen Kan,
Department of Civil Engineering, National Taiwan University
ABSTRACT: The world climate has changed dramatically within these decades. Many countries around the world
are now trying to find out possible ways to reduce environmental impact, and have established their own
environmental regulations. In construction field, green building is considered a practical technology to achieve
this goal. However, it takes a lot of time and effort to qualify for green building standard due to dealing with
massive parameter calculation, and the energy-saving efficiency can't be straightly calculated, either. In order to
improve this difficulty, the study integrates BIM (Building Information Modeling) with its comprehensive building
information and Taiwan's green building standard (EEWH-NC) to conduct information transferring and energy
consumption calculation of building envelope (ENVLOAD). The result will be directly shown on the developed
interface, and case study verifies it's within 5% margin of error. With this integration system developed in this study,
the designer can save a lot of time and think about better building design. In the future, the integration can work
with more complex building design, attracting more potential users using BIM and make the most of its building
information.
KEYWORDS: BIM, green building, ENVLOAD, Automation.
1. INTRODUCTION
According to IPCC (Intergovernmental panel on climate change)'s report, the world temperature has climbed up
rapidly. The increase amount of green house gases, especially carbon dioxide, is mainly coming from the overuse
of fossil fuel and the changing form of land use. The statistical data collected from National Cheng Kung
University shows the carbon dioxide emitted from construction industry accounts for 28.8% of total carbon
emission in Taiwan. Thus, developing sustainable engineering in Taiwan construction industry has become a very
critical issue. In green building design, many countries have developed their own green standards, techniques, and
provide ratings for qualification. But in real construction use, designer tends to use design which is previously
accepted by authority because they can't instantly calculate the energy saving efficiency nor know if the new
design meets all the standards. ( ZENG Xu - dong , ZHAO Ang2006). The calculation takes too much time and
effort. In some cases, the energy saving performance is even worse than those non-adopting energy saving designs.
Now with computer-aided design technology, software including BIM and energy simulation software have been
used in building designing stage, but there hasn't been any software designed for Taiwan's green building
assessment. The energy saving efficiency and evaluation of green building are still relying on manual calculation,
which not only wastes too much time but also has higher occurrence of error. Also, with more large scale and
complex construction project being developed, it is believed that a computerized, paperless, and automatic
working environment will be the future trend. Many international and Taiwan consulting companies are investing
in this development.
In order to improve the massive calculation and time spent on checking data, the study integrates BIM with
Taiwans green building standard to conduct the automatic calculation process. The energy consumption of
building envelop can be directly shown on the developed interface within very short time, saving a lot of time and
increasing design quality and efficiency. The proposed interface is expected to be a plug-in program attached in the
BIM - Revit Architecture. After architecture design finishes, the energy consumption of building envelop
(ENVLOAD), HSC (Heat Source Capacity), and EAC (Electronic Air Cleaners) can be automatically calculated
and shown. Also, each parameter will be provided for designer as reference if theres something that needs to be
corrected or checked. The study also wants to create a developing base of integrating BIM and Taiwans green
building standard for further research.
2. LITERATURE REVIEW
In recent years, there are more and more studies focusing on the environmental assessment issues of buildings.
Wang(2005), Chang(2005), and Lo(2006) report that water and electricity expense accounts for the most part in
buildings life cycle cost. Therefore, how to control the energy consumption of air-conditioning system, which
11

influences the water and electricity cost the most, is the key point of controlling the total life cycle energy cost.
There are different parts of the buildings including the building envelope, indoor space, and roofs, etc., among
them, the design of building envelope is probably the most key strategies in the construction of buildings. An
integrated thermal model was proposed by Morrissey and Home (2010), who use life cycle costing approach to
an extensive sample of dominant house designs to investigate life cycle costs in a cool temperate climate in
Melbourne, Australia. Results suggest that most cost-effective building design is always more energy efficient.
Therefore, it can be concluded that early energy efficiency evaluation of new buildings is quite important in
terms of cost and energy saving. Kneifel (2009) ran a total of 576 energy simulations for 12 prototypical
buildings in 16 cities in States, the results show conventional energy efficiency technologies can be used to
decrease energy use in new commercial buildings by 2030% on average and up to over 40% for some building
types and locations. The above studies all show good examples of why the evaluation of energy-saving
efficiency of buildings is so important. A well-designed energy-saving and heat insulation system in the building
designing stage can largely save energy and thus protect the environment. Taiwans green building evaluation
system includes nine indicators: bio-diversity, greenery, soil-water content, energy savings, CO2 emission
reduction, construction waste reduction, water resource, garbage and sewage improvements, indoor environment
quality which are categorized by ecology, energy saving, waste reduction and health (EEHW). Many new-built
buildings in Taiwan are dedicated to achieve those standards.
2.1 ENVLOAD
ENVLOAD is an indicator used for predicting the annual cooling and heating loads of office and residential
buildings. The calculation methodology proposed by Lin (1990) has its significance on energy load assessment
because users can calculate the cooling and heating load on their own rather than asking for professional help.
ENVLOAD has now been adopted in many studies to evaluate the energy loads of buildings. Lai (2008) used
ENVLOAD calculation and integrated it with natural ventilation system of office buildings to propose a
conservation coefficient system, encouraging more adequate building window designs. In Huangs study (2002),
it is not only used and developed as a simplified methodology to evaluate the heat load of air-conditioning
system, but also propose a predicting system of air-conditioning capacity. Its widely application and integration
can still be further developed and expected. The calculation process will be explained as follows: It is calculated
using linear regression equations. These equations are very simple and acceptable because they consist of only
two weather variables (temperature difference and solar heat gain) and three building variables (insulation, solar
shading and thermal capacity). In multiple regression analysis, the object function (the energy consumption of air
conditioning) is simulated by dynamic computer program HASP8001, and the constants and partial regression
coefficients are obtained through variance analysis between the values of object function and the above
theoretical predictor variables. To ensure the wide adaptability for building models and climatic contexts, 27
office building spaces and 54 residential building with distinct design in orientations, constructions, glass areas,
floor levels, types were used for simulation models, and 18 climates distributed throughout Taiwan, Korea, Japan
and the U.S. for weather input. The statistical samples include 486 office buildings and 972 residential buildings.
The result showed the R2 values of the multiple regression equation were between 0.848 to 0.972, which
illustrating the high accuracy to this method. It can be widely applied for international climates, which has
successfully achieved the goal of simplicity and wide adaptability (Lin, 1990). ENVLOAD calculation and
parameter definition are shown as follows:
ENvL0AB = au + a1 u +a2 L BB + aS (Hk IEk) (2.1)
u = ui Ac (2.2)
Ac = a +b Tu - c Tu
2
(2.3)
Tu = ui L (2.4)
L =
ui Ai + u.S ui Ai
AFp + Iox
(2.5)
Nk =
|(Ki pi Ai) +u.uSS (ui Ai)] + u.S |(Ki pi Ai) +u.uSS (ui Ai)]
Ap
(2.6)
The parameter of calculating ENVLOAD can be categorized into two types:
12

Climate Related Parameters: Temperature and Solar Heat
Buiding Parameters: Heat Capacity, Sheltering and Heat Insolation
2.2 BIM (Building Information Modeling)
BIM (Building Information Model) refers to software which contains massive building information data
including architectural, structural, MEP (Mechanical, Electrical, and Plumbing) systems information. It can be
used in the designing, constructing, and the operation stage of building. With its visualized building model
automatically built within the software, users can easily see how the design works with other part of the building.
BIM can be seen as a database of a project, which is the most useful advantage of BIM. BIM is now widely used
in cost analysis, spatial conflict check, scheduling, work simulation, and energy-saving simulation (Hartmann, T.,
J. Gao, et al., 2008).
3. RESEARCH METHODOLOGY
After retrieving data from BIM, it is firstly categorized and changed into specific format. After categorizing the
data, compile it and conduct the calculation process. Detail data selection process is stated as follow:
3.1 Using API in Data Selection
3.1.1 Heat Conductivity (Ui) of Building Envelope in Air-Conditioning Area
In order to calculate the Ui, rooms will be categorized according to air-conditioning and non air-conditioning
area. Then the roof, wall, and window will be separated apart from the room to calculate the heat conductivity of
each building part. There are many building materials, each of them has its own Ui value, thus the automation of
calculation process is really important in terms of time saving.
3.1.2 Area of Building Envelope (Ai) in Air-Conditioning Area
The data selection process is similar to Ui. The difference is the process of switching imperial unit (aquare foot)
to metric unit(M2), which is the standard unit of Taiwans green building standard.
3.1.3 Heat Conductivity (Ui) of Building Envelope in Non Air-Conditioning Area and Area of Building
Envelope (Ai) in Non Air-Conditioning Area
Both the data selection processes are the same as Ui and Ai, when it is categorized as non air-conditioning area,
same selection process will be made to conduct the data calculation.
13

Fig.1 Calculation Flow Chart

3.2 Data coming in Indirect Process
3.2.1 Total air-conditioning floor area (AFp) in perimeter area and the total air-conditioning floor area
in interior area (AFi)
The interior and perimeter area are closely related, the data selection process should be done simultaneously. The
definition of perimeter area is the area summed up by the area at a distance of 5 meters inside from the joint line.
The Revit can automatically calculate the floor area of each room and floor except the perimeter area, thus the
study develop the program and conduct the calculation. The formulas of calculating perimeter area are listed
below:
Inteiioi Aiea of 0ne Layei Builuing = (a - 1u) (b - 1u) (3.1)

Peiimetei Aiea of 0ne Layei Builuing = a b - (4.1) (3.2)
Parameter a refers to building length and b refers to width. After calculating the interior area, perimeter area can
be easily calculated according to formula (4.2). The study only considers simple building shape including
rectangular. Also, deciding the accurate coordination of building is another important contribution of this study.
The judgment process will be transferred to programming codes, thus the perimeter area can be calculated.

The programming adjustment of each coordinate is a critical step. The study proposed eight criteria, four
adjustment direction to conduct the adjustment process. The start point will be set as reference point, through the
counterclockwise data checking process, both of the x value and y value will minus 5 units, thus the shape will
be formed and conduct the next procedure. After having those coordinate data, they will be used in formula
(3.3)~(3.5) to calculate the interior floor area.
x
n+1
x
1
(3.3)
14

y
n+1
y
1
(3.4)
A
1
2
(x
y
+1
- y
x
+1
)
n
=1
(3.5)

After having the interior floor area, perimeter area can be calculated using formula (3.2). The highest floor also
belongs to perimeter area, thus the AFi and AFp calculations will be shown in formula (3.6) and (3.7).

AFi = Inteiioi Aiea of 0ne Layei Builuing (Numbei of Floois - 1) (3.6)

AFp = Total Flooi Aiea - AFi
= Peiimetei Aiea of 0ne Layei Builuing
(Numbei of Floois - 1) + Bighest Flooi Aiea
(3.7)
3.2.2 The cooling degree hours (DH) and the building located areas insolation hours (IHk) Calculation
DH and IHk will be different according to different regions including altitude and building direction. The
building and the geographical information are not difficult to have, but the building direction adjustment will be
more complex. The judgment process is shown below:
In real construction project, the building direction is not always in perfect N-S or E-W direction. Thus it takes
quite much time in data transferring and calculating. With this proposed methodology of deciding the building
direction, it really saves a lot of time. The previous manual table checking method is similar to the x-y
coordination. The y value can be found as long as x value is determined, then the coordinate will be transferred
using code mode and integrated into the program. The table checking process is shown as follows:
With the interface proposed from this study, it will be quite convenient for user to adopt in real project use
because most of the table-checking and calculation will be done by automation of program, all the necessary data
are within the program. When using the plug-in interface, user can select the building type and input information,
within one click, the developed interface will instantly show the calculation results and each value.
3.2.3 The building envelop Insulation Gain Coefficient (Mk) of each direction, sun shading
coefficient(Ki), Solar Heat Gain Rate(i)
In calculating the three values, the glass-covered area, Ki and i should be firstly calculated. Some of the value
calculation has been introduced in previous part. Here are the simplified calculation process of Ki and i:
3.2.4 Sun Shading Coefficient(Ki)
Ki value is related to building outside shelter. The study chooses horizontal shelter, and its characteristic will be
considered in the calculation process. The modification coefficient can be retrieved by table checking, thus the
depth ratio can be calculated by measuring the lengh of window and the depth of sun shade.
3.2.5 Solar Heat Gain Rate
The glass material in Revit database doesnt contain the attribute, thus the study create a new attribute and input
the value in order to conduct more comprehensive calculation.
3.2.6 The cooling energy consumption of ENVLOAD (a0) and partialregression coefficient(a1, a2, and
a3)
The value of a0, a1, a2, and a3 will be different according to building types. The building type selection will be
designed in the input interface, as long as user choose the accurate building type, the program will automatically
conduct the calculation. Thus those coefficients are all been set within the program.
15

3.2.7 Indoor Heat (G), average indoor heat (Gi), and average indoor heat gain (Tu)
The calculation of annual indoor heat will need the value of average indoor heat (Gi) and average indoor heat
gain ( Ac). The Gi is also different according to building types. Similar to the calculation of a0~a3, user will only
have to choose the accurate building type, the program will automatically conduct the calculation.
4. VERIFICATION
The input building data is a 11-floor office building located in Taipei city, Taiwan. After establishing the
necessary building information into Revit, the verification process is conducted. The comparison result between
true value and calculation results is shown in Fig.2. The comparison ENVLOAD is within 5% margin of error.
Fig.2 Verification Result
5. CONCLUSION
The study develops an interface using BIM with its massive building information and conducts the
energy-saving process which automatically calculates the ENVLOAD, showing the possibility of using BIM in
calculating energy-saving efficiency of building envelop. The study includes the following contributions:

5.1 Showing the possibility of integrating BIM with other software
The study shows the application value of using BIM in building design process, making the most of using the
database in BIM, attracting more potential owners to use and develop BIM technology.
5.2 Instantly calculate ENVLOAD
The proposed interface can automatically calculate the ENVLOAD within a very short period of time, saving a
lot of time and effort compared with previous manual calculation process. Once theres any design change, user
will only have to correct the data then the result can be instantly shown on the interface.
5.3 The integration of BIM and green building standard
The regulations in building energy-saving technical book are integrated with BIM in this study to conduct the
automatic calculation process. Thus users wont have to manually check and deal with many complicated data,
lowering the occurrence of error and promoting the use of BIM.
16

5.4 The integration of software/interface will be the new trend
Though Revit Architecture is quite useful for designers, it is still not accepted by many users because of
application restriction. The proposed interface in this study is a good performance for user to input the necessary
data and automatically conduct the data calculation. Autodesk company will soon released a new version of BIM,
which is called Building Design Suite to combine the other software into one single BIM software. The study
shows the similar trend with software integrating development.
5.5 Provide a developing base for future studies
The developing process and difficulties are all well recorded and stated in this study, thus it can be a very good
reference for future studies. With this developing basis, future study can try to incorporate more software and
standard for user to adopt the technology.
6. RECOMMENDATIONS
The study has developed the integration process and interface, but the system is still not comprehensive enough
and can be further developed to meet different kinds of needs, thus for future study here are the
recommendations:

6.1 Integrate with more green building standards
There are other green standards including the lightening system which can be integrated with the proposed
interface. With the lightening system data in BIM, the indicator related to lightening system can be easily
calculated and conduct the energy-saving calculation.

6.2 Calculate floor area with air-conditioning system in perimeter area.
The study fails to check if the air-conditioning or non air-conditioning area is located in the perimeter area,
therefore the floor area in perimeter area cant be precisely and straightly calculated. There are more and more
building with complex exterior design, thus the process should be improved, which can be done by extracting
more data from BIM to conduct more complex floor area calculation.

6.3 Evaluate the cost efficiency of other building information
Other building information including mechanical system can all be integrated into the cost efficiency calculation.
The owner can predict and know how much expense he can save.

6.4 Automatically create necessary report and diagram which are needed for
qualifying green building standard.
The study has accomplished the automation calculation of ENVLOAD, and the results data are all saved in the
developed interface. With this characteristic, the report and diagrams and be created by designing the standard
output format, thus saving a lot of time in preparation process.
7. REFERENCES
Chang, Chia-Ruei (2005). Research on the Life-cycle Costs of Gymnasiums - A Case Study on the Gymnasiums
of National Taiwan University, Department of civil engineering, National Taiwan University, R.O.C Taiwan.
Hartmann, T., J. Gao, et al. (2008). Areas of Application for 3D and 4D Models on Construction Projects.
Journal of Construction Engineering and Management 134(10): 776-785.
Huang, Kuo-Tsang (2002). A Study on the Simplified Method of Evaluating the Air-conditioning Capacity
17

---Evaluating the Capacity of Chiller base on ENVLOAD, Department of Architecture, National Cheng Kung
University, R.O.C Taiwan
Joshua Kneifel (2009), Life-cycle Carbon and Cost Analysis of Energy Efficiency Measures in NewCommercial
Buildings, Office of Applied Economics, Building and Fire Research Laboratory, National Institute of Standards
and Technology, Gaithersburg, United States
Lai, Po-Hung (2008). Assessment on Ventilation Performance for Envload in Office Buildings, Department of
Architecture, National Cheng Kung University, R.O.C Taiwan
Lin, Hsien-Te (1990). An Evaluation Method for The Energy Consumption of Building Air Conditioning-Lin's
Simplified Method , Department of Architecture, National Cheng Kung University, R.O.C Taiwan
Lo,Yu-Ying(2006) . A Study of Maintenance Costs and Strategies of Student Dormitories Using Life Cycle Cost
Concept Department of civil engineering, National Taiwan University, R.O.C Taiwan
Morrissey J. and Horne R.E. (2010), Life Cycle Cost Implications of Energy Efficiency Measures in New
Residential Buildings, Centre for Design, RMIT University, Melbourne, Australia.
Wang, Ya-Huei (2005). Analysis on life-cycle cost of experimental school building in the universityemphasis
on the stage of maintenance management Department of civil engineering, National Taiwan University, R.O.C
Taiwan.
ZENG Xu - dong , Z. A. (2006). Study on the Application of Energy Efficiency Building Design Based on BIM
Technology. College of Architecture and Urban Planning, Chong-qing University, ChongqingChina.

18

THE DEVELOPMENT OF WEB BIM-BASED INTERFACE
COMMUNICATION SYSTEM IN CONSTRUCTION
Yu-Chih Su
National Taipei University of Technology, Department of Civil Engineering, Taiwan
Meng-Chung Lee & Chih-Yuan Li
Chien Kuo Construction Company, LTD, Taiwan
Yu-Cheng Lin
National Taipei University of Technology, Department of Civil Engineering, Taiwan
ABSTRACT: Various problems occurred while simply using construction drawing to communicate and respond
among participants during construction phase. When information is unclear, general constructor need to discuss
and obtain sufficient information from other project participants. Nowadays, email correspondence, one of the
most convenient ways, is adapted as a communication tool between participants, but project participants cannot
communicate interactively by using text and pictures. Moreover, it cannot record and manage the communication
information during the process. With the advent of the Building Information Modeling (BIM), BIM digitally
contains precise information and relevant data which includes the description of 3D object-oriented CAD that
can help reducing cognitive differences. Therefore, the aim of the study is developing the Web BIM-based
Interface Communication System. It is not only providing participants to publish the latest BIM on the web
instantly, but also give stakeholders the accessibility to BIM model via web browser. Furthermore, project
participants can propose or answer questions by selecting precise component of the BIM model. Feedbacks can
be saved back to both system database and BIM model. In the end, this communication system will be
implemented to a selected project in Taiwan to verify our proposed methodology, and benefits, limitations, and
suggestions for future research.
KEYWORDS: Building Information Modeling, Web-based System, Communication Management
1. INTRODUTION
Construction projects, which are characterized by extreme complexity and non-standardized production,
differ in that they are designed and executed to meet individual owner needs. Thus, effectively communicating
project-related interfaces are essential to successful construction management. Effectively communicating and
managing interfaces can improve interface communication (IC) for interface management (IM) in construction,
thereby eliminating unnecessary mistakes. Effective interface information sharing allows project participants to
identify existing interfaces and solve interface problems. Without IC, poorly coordinated and controlled
boundary conditions among project participants can cause such interface problems as design errors, part
mismatch, coordination difficulties, and construction conflicts. Conventional interface communication methods
include face-to-face meetings, telephone communication, and virtual design and construction (VDC). However, a
typical problem encountered during conventional communication is that discussions may not be effectively
understand and shared with others special regard to interfaces of CAD issues.
Building information modeling (BIM) is a new industry term referring to parametric 3D computer-aided
design (CAD) technologies and processes in the AEC industry (Taylor and Bernstein, 2009). During the
construction phase, participants typically communicate the interfaces through text-based document or e-mail way
for sharing interface information with other participants. Notably, IC is an information-intensive task in which
extremely useful information is made available to participants. In order to assist involved participants in
exchanging and responding interface information special related to interfaces of 3D BIM models, the primary
objective of this study is to facilitate interface sharing and management during the construction phase.
Identifying, tracking, controlling, and managing interface events and problems are critical tasks in IM. This
study develops the WEB BIM-based Interface Communication (BIMIC) System for engineers to enhance
interface information sharing and tracking efficiency. Notably, this study integrates novel web-based
communicating platform and the 3D BIM approach to communicate and manage interfaces in a 3D BIM
environment. The main function of the BIM approach in this study is 3D illustration and mapping of interfaces.
The BIM approach retains interface information in a digital format, and facilitates easy illustrated and updating
interface information in the 3D BIM environment. By utilizing the 3D BIM models, project participants can
19

obtain an overview of previous and current interface events in a given project and manage interfaces.
Furthermore, project participants can track and access the most recent information for any interface, change, or
conflict during the construction phase. Interfaces can be updated rapidly and made available to each participant
via the 3D BIM environment during the construction phase. This research is a pilot study to apply the BIMIC
system for IC during a building project in Taiwan, and analyzes and discusses the entire IM process
Interface communication and management in construction affects the cost, the scheduling, and the quality of
projects, both directly and indirectly. The interfaces can be identified and tracked in interface management
involved general contractor and all participants to improve the construction process and minimize deleterious
change, and promote beneficial change. Only limited research has examined interface communication and
management system in construction. (1) Senthilkumar et al. (2010) presented a web-based system for interface
management for construction projects during the design phase. (2) Siao and lin (2012) presented a web-based
system integrated with multilevel interface matrix for interface management during the construction phase.
Wide application of BIM is due to its various benefits throughout project design and planning, construction
and facilities management. There are many benefits of BIM cited in the previous work in supporting decisions
and improving processes throughout the lifecycle of a project (Eastman et al., 2008; Shen and Issa 2010;
Vanlande et al. 2008; Becerik-Gerber and Rice 2010). Related to the preconstruction phase of a project, these
benefits include identification of design conflicts prior to construction, enabling the prefabrication of
components prior to construction, accurate cost estimation, and accurate geometric representation of all parts of a
facility (Li et al., 2006; Eastman et al., 2008; Shen and Issa 2010; staub-french and khanzode 2007: Goedert and
Meadati 2008). During the construction phase, these benefits include less rework, reduction in requests for
information and change orders, communication through visualization, improved productivity in scheduling,
faster and more effective construction management with easier information exchange (Eastman et al., 2008;
Hardin 2009; Matta and Kam 2010; Azhar 2011). During the operation phase, these benefits will include control
of facilities management progress, integrated life-cycle data, rapid and accurate information of updating and
changing activities, more effective facility management with easier information exchange (Eastman et al., 2008;
Hardin 2009; staub-french and khanzode 2007; Manning and Messner 2008; Underwood and Isikdag 2010;
Laura et al. 2012; Burcin et al. 2012). Despite many articles and system developments in academic and practice
literature, there is a lack of systematic approaches to communicate and manage interfaces special to BIM-related
discussion in the 3D visual environment. To solve this problem, the proposed web BIM-based Interface
Management (BIMIC) system is developed to enhance interface communication performance.
3. SYSTEM FRAMEWORK
The correctness of the BIM model will directly affect the IC operations in BIMIC system. In order to
avoid too many users to use BIM models simultaneously that affect the accuracy of the BIM models, users can
update the information of the BIM elements database directly in BIMIC system. The latest information in the
BIM elements database will be updated again when BIM models content changes. In this framework, the
information of BIM can be saved and be updated in BIM elements database in BIMIC system without accessing
the BIM models directly. IC operations do not need all the building information and only presents the
information needed maintenance although BIM model may cover the whole building the information. Therefore,
during the pre-IC progress, the BIM engineer is responsible for making the decision regarding to export the
DWF (Design Web Format) file of BIM model and save as BIM models source in the BIMIC system in
advanced based on requirement of the IC operations. The features and benefits of DWF format cant only retain
building information-related of BIM model, but also the file size is smaller than general BIM models file.
Integrated with DWF file as 3D BIM illustration, BIMIC system can be improved and enhanced in system
performance when user access BIM elements database. Furthermore, the BIM elements database in the server
can keep and remain accurate information of BIM models. The BIMIC system server supports four distinct
layers, each with its own responsibilities: management, database, application and presentation (see Fig. 1). This
following section describes the distinct layers in the BIMIC system
Management layer is to provide the BIM engineers to develop, modify and edit the BIM models. Only BIM
engineers allow edit and export information of BIM models into BIM elements database directly and export BIM
model into DWF files in the server side. Integrated the BIM API programming, Management layer of the BIMIC
system can let data in the BIM models export to BIM elements database. Furthermore, interface information of
20

related BIM can also be written in the BIM elements database in the Management layer of the BIMIC system.
There are two databases in the database layer of the BIMIC system. They are BIMIC database and BIM
elements database. The BIMIC database storages all communication records of interfaces. BIM elements
database storages complete BIM-related components information regarding to the related interfaces. The BIM
information is read-only in the BIMIC system. The information of BIM models cannot be input or modified in
the BIMIC system. Furthermore, there is a relationship between BIMIC database and BIM elements database.
The application layer defines various applications for major system and API modules. These applications
offer indexing, BIM model data updating and transferring, status visualization, and report generation functions.
The application layer will integrate and utilize BIM software to open the BIM models by developed Design
Review API modules. When the API modules in the BIMIC system to receive request from the client. Finally,
the application layer can automatically make data acquisition and analysis of BIM models based on request, and
then send the results back to the client side.
The presentation layer is the main implementation platform of BIMIC System. During the IC progress, the
project manager and project participants can use the PC or tablet (client side) and utilities BIMIC system for IC
operation. The presentation layer includes displaying the location information of BIM model automatically,
listing interface communication records, illustrating different respond regarding to problems associated with the
BIM-related components automatically.

Fig. 1: System Framework

4. SYSTEM IMPLEMENT
In this study, BIM is used as an information model in the BIMIC system. One purpose is to utilize BIM models
as the visualization tool of interfaces. The BIM models are applied in the BIMIC system to link and relate to
interface information (such as event descriptions and location). Autodesk Revit Architecture and Revit MEP
were used to create 3D BIM models. Autodesk Design Review was used to read BIM files (DWF files). Interface
information integration with the BIM components in 3D BIM models was achieved by using Autodesk Revit
application programming interface (API) and Microsoft Visual Basic.Net (VB.Net) programming language. This
following section demonstrates the implementation modules in the BIMIC system.
Authority Management Module
The authority management module is an access control mechanism preventing unauthorized users from entering
system or retrieving sensitive related information. The BIMIC system requires all project participants to register.
To register, users provide a unique User ID and password for authentication. As interface information or reports
required by different project participants and different interfaces vary, general contractors have different access
21

rights and authorities than other parties. There are three types of users in this study. They are system
administrator, project managers, and project participants.
Interface Edition Module
This interface edition module lets users edit related BIM-based interface information associated with related
perspective BIM information (or attached files). Furthermore, the information includes the proposer name,
related responded participants and contact mail information. This interface edition module is a BIM-based
communication platform similar to instant messaging or forum which can online exchange opinions and ideas on
a particular interface. Users associated with specific interfaces can post questions, responses and comments,
thereby generating a permanent record of discussions regarding specific interfaces. This module is to provider
project participants to respond or reply for discussing specific interfaces with attaching files regarding to the
BIM components in the 3D BIM models (see Fig 2)
Interface Search Module
The interface search module allows offer indexing, full text search, element ID search and location/area search
functions. This module provides the user to search the interfaces easily and quickly based on the user
requirement (such as the interface submitter, release time of interface, related component ID numbers of BIM
models, and floor of building). The user also can search related interfaces via selecting elements in BIM model.

Fig. 2: The Interfaces Edition in the BIMIC System
The BIMIC system is based on the Microsoft Windows Server 2008 operating system with an Internet
Information Server (IIS) as the web server. The BIMIC system consists of three different user areas project
participants, project manager, BIM engineer and system administrator areas. Access to the BIMIC system is
controlled by passwords. In order to integrate system with BIM model, the system develops the following API
modules:
Automated BIM Information Import API Module
This API module provides to import the basic information of selected BIM components into the editing form
automatically without manual data entry when user is ready to edit interface and click BIM components in the
3D BIM models (see Fig 3).
BIM Highlighted Visualization API Module
The module provides to illustrate relevant highlighted components with the bright colors to let user quickly and
22

effectively access the relevant BIM components of interfaces regarding to the location and condition if the BIM
components in the 3D BIM models are selected as the relevant description of interface, (see Fig 4 and Fig 5).

Fig. 3: The Selection of BIM components in the 3D BIM models in the BIMIC System

Fig. 4: The BIM Highlighted Visualization in the BIMIC System

23

Fig. 5: Responding Interfaces with Highlighted BIM Models
BIM Element Model Link Interface API Module
This module is mainly to provide the link functionality to access the latest BIM models in the BIMIC system
through updated whole or separated DWF file. The two databases are linked with element ID index thought data
mapping. Therefore, the information can utilized for data association by data mapping to retrieve complete
interface information based on element ID index between two databases (see Fig 6).

Fig. 6: The linkage between BIMIC database and BIM elements database
BIM Model Perspective Information API Module
This module provides the user to save the current viewing BIM model perspective information submitted by
users (such as view position, direction, elevation, distance, and zoom information). When user enter the system
and click the interface and then can quickly access the same 3D view position and direction for a clear
understanding of interface relevant to BIM components in the 3D BIM models (see Fig. 7).
24

Fig. 7: BIM Model Perspective Information in the BIMIC System
5. DISCUSSION
The general contractor encouraged all participants to utilize the BIMIC system as communicate interface. The
BIMIC system was utilized in the office-building project to verify its efficacy and demonstrate its IC
effectiveness with BIM models. During tracking interfaces phase, all interfaces are identified by responsible
participants or project managers. Before the IC phase, BIM engineer initially utilizes Autodesk Revit
Architecture or Revit MEP to create BIM models then export to DWF file format in the BIMIC system. During
interface identification phase, project participants edit interface problems, select BIM model (DWF files), and
link the interface problems associated with the 3D BIM models components. These interfaces include
descriptions of unconfirmed problems, detailed situation descriptions, and explanations of interface problem
solutions. Finally, the engineer submits issue with associated BIM components and to the BIMIC system for
approval. After approval is obtained from the project manager, related responsible participants will reply their
comments to BIMIC system. When processed interface is tacked, the system shows the latest status and result for
each interface. Furthermore, participants can access related interfaces directly by clicking on BIM components in
the 3D BIM models. All interfaces are centralized and stored in the central database to avoid redundancy. Table 1
shows comparison of current approach and proposed system.
The BIMIC system provided solutions to interface-related problems, and supported all interfaces and changes
during construction. The principal advantages of the BIMIC system are as follows. (1) The BIMIC system
allowed project participants to track and manage integrated information with BIM models during construction
phase. (2) The BIMIC system allowed participants to collaborate and communicate interface issues associated
with the BIM components in the 3D BIM models. (3) The BIMIC system enabled participants to identify
interfaces and changes to solve problems in advance. (4) The BIMIC system enhanced interface communication
and management easily and effectively in the web-based 3D BIM environment.

25

Table 1: Comparison of Current Approach and Proposed System

Item

Current Approach Proposed Approach
Discuss interface
Use the paper-based document or
e-mail system for discussion

Utilize the BIMIC system to
communicate interfaces integrated
with BIM models through the web
browser
Illustrate the interface
Illustrate the interface in the text
form attached with files

Illustrate the interface in the text
form integrated with BIM models
attached with files
Share interface
Use e-mail system for interface
sharing

Utilize the BIMIC system to
communicate interfaces integrated
with BIM models to share
interfaces with others
Track interface
Paper-based document or e-mail for
track

Track the interfaces record directly
in the BIMIC system
Search interface Search interface using e-mail system
Search interface from the BIMIC
system or the components of BIM
models (DWF files)
The primary limitations to applying the BIMIC system were as follows:
In this study, the Autodesk Design Review is selected as a web version of the BIM model Viewer in the
BIMIC system. However, the required functions of the system need to be developed without the support of
the API although the Autodesk Design Review provides more complete API environment compared to
other software.
If the whole BIM model is presented directly to the page on the BIMIC system, it will affect the
smoothness and effectiveness of the system operation. Therefore, the whole model is necessary to split into
parts of BIM model based on the user requirement to review BIM models effectively.
The general contractor is selected as to apply and use the system to communicate with other project
participants supported by top management in the general contractor. Regard to other project participants
(such as architects, PCM, and owners), it will need to develop suitable mechanisms and regulations to
require all project participants using the same system to communicate project-related interface for IM.
6. CONCLUSION
The application of IC integrated with the BIM approach for building projects during the construction phase is
discussed in this work. This study implements the novel web-based BIMIC system for all project participants as
an interface communication platform integrated with BIM models through the web browser without purchasing
BIM commercial software. The web-based BIMIC system enhances interface communication effectively through
3D BIM environment. The BIMIC system provides insight into factors impacting IC activities, which in turn
assists participants in managing interfaces to improve construction management performance. The collection of
interface events and problems by the BIMIC system allows projects participants and engineers to view 3D BIM
models and related information of interfaces in the 3D BIM environment.
The BIMIC system use BIMIC database to apply and analyze information from BIM models directly based on
the SQL syntax. The BIMIC system updates the latest information in the BIM models automatically
synchronized to the BIM elements database. All required information in the BIM models automatically
synchronized to the BIM elements database based on required information for IC by the API development. The
BIMIC system will retain existing data and update only the changed data after synchronization if the exported
information already exists in the BIM elements database. The main characteristic of the BIMIC system is to
provide topic description while communicating with components that are relevant to 3D BIM model and
26

associated 3D view for project participants to quickly understand problems associated with BIM models through
the web browser. When users respond to the interface discussion, they will view the same view angle and
relevant components of BIM models assigned by interface proposed user. The proposed solution can reduce the
cognitive differences of interfaces discussion among project participants. When interfaces all related to
component selection, making the interface theme and BIM models relevant, effective integration of the course of
the discussion in the BIM model to improve the status of the communication of information scattered in the past.
For the participants are not familiar with BIM software interfaces, the BIMIC system is designed to simplify and
easily system operation by the API development (such as perspective storage, related components and image
viewing) to increase the system willingness use. Furthermore, all topics that are stored in the BIMIC system will
be able to use this system to automatically output to common formats (such as Word, Excel, and PDF) and other
electronic files for the usage of paper-based communication.
7. REFERENCES
Eastman, C., Teicholz, P., Sacks, R., & Liston, K. 2008. BIM handbook: A guide to building information
modeling for owners, managers, designers, engineers and contractors, Hoboken, NJ: Wiley.
Shen, Z. and Issa, R. 2010. Quantitative evaluation of the BIM assisted construction detailed cost estimates,
ITCON 15: 234-257 [accessed 22 May 2011] Available from Internet: < http://www.itcon.org >.
Vanlande, R., Nicolle, C., & Cruz, C. 2008. IFC and building lifecycle management. Automation in Construction,
134(7), 7078.
Becerik-Gerber, B. and Rice, S. 2010. The perceived value of building information modeling in the US building
industry, ITCON 15: 185-201. [accessed 20 May 2011] Available from Internet: < http://www.itcon.org >.
Li, G., Ding, L., and Wang, J. 2006. Construction project control in virtual reality: a case study, Journal of
Applied Science, 3 (13), 2724-2732.
Staub-French, S., Khanzode, A. 2007. 3D and 4D modeling for design and construction coordination: issues and
lessons learned, ITCON 12: 381407 [accessed 5 June 2011] Available from Internet: < http://www.itcon.org >.
Goedert, J. D. and Meadati, A. G. F. 2008. Integrating Construction Process Documentation into Building
Information Modeling, Journal of Construction Engineering and Management, 134 (7), 509516.
Matta, C. and Kam, C. 2010. The GSA'sBIM Pilot Program, Presentation by GSA PBS Office of the Chief
Architect, http://bim.arch.gatech.edu/data/reference/gsaBIM.pdf 20058 (last accessed on March 21, 2010).
Hardin, B. 2009. BIM and construction management: proven tools, methods, and workflows. John Wiley & Sons,
New Jersey.
Azhar, S. 2011. Building Information Modeling (BIM): trends, benefits, risks, and challenges for the AEC
industry, Leadership and Management in Engineering 11(3): 241-252.
Umit, Isikdag and Jason, Underwood 2010. Two design patterns for facilitating building information
model-based synchronous collaboration. Automation in Construction. 19(5), 544-553.
Laura, K., Nan L., Burcin B.G. 2012. Imaged-based verification of as-built documentation of operational
buildings. Journal of Automation in Construction, 21 (1): 161-171.
Manning, R. and Messner, J. 2008. Case studies in BIM implementation for programming of healthcare facilities,
ITCON 13: 446457 [accessed 27 March 2011] Available from Internet: < http://www.itcon.org >.
Fox, S., and Hietanen, J. 2007. Interorganizational use of building information models: potential for
automational informational and transformational effects. Construction Management and Economics, 25,
289-296.
Burcin, B. G., Farrokh, J., Nan, L., and Gulben, C. (2012). Application Areas and Data Requirements for
BIM-Enabled Facilities Management. Journal of Construction Engineering and Management, 138(3), 431-442.
Tse T. K., Wong K. A. and Wong K. F. 2005. The Utilisation of Building Information Models in nD Modelling: A
27

Study of Data Interfacing and Adoption Barriers, ITcon 10: 85-110 [accessed 16 May 2011] Available from
Internet: < http://www.itcon.org >.
Ding, L.Y., Zhou, Y., Luo, H.B., and Wu, X.G. 2012. Using nD technology to develop an integrated construction
management system for city rail transit construction. Automation in Construction, 21 (7), 64-73.
Fu, C., Aouad, G., Lee, A., Ponting, A., and Wu, S. 2006. IFC model viewer to support nD model application,
Automation in Construction, 15 (2), 178-185.
Chau, K., Anson, M., Zhang, J. 2004. Four-dimensional visualization of construction scheduling and site
utilization, Journal of Construction Engineering and Management. 130 (4): 598-606.
Siao, F. C. and Lin, Y. C. 2012. Enhancing Construction Interface Management Using Multilevel Interface
Matrix. Journal of Civil Engineering and Management, 18(1): 133-144.
Senthilkumar, Venkatachalam, Varghese, Koshy and Chandran, Ajai. (2010). A Web-based System for Design
Interface Management of Construction Projects. Automation in Construction, 19(2), 197-212.

28

29
COUNTABLE BENEFITS OF VISUALIZING ONE-OF-A-KIND
PROCESSES
Bargstdt, Hans-Joachim
Professor (Bauhaus-Universitt Weimar, Germany), Visiting Professor (National Taiwan University, Taiwan)
ABSTRACT: The Construction processes are unique in several aspects. Many projects are of the one-of-a-kind
type, which means that the planning and the preparation of work are individual for each project. Also most
construction projects are broken down into small-scale work packages. The packages are then performed by
different contractors and subcontractors, all working on their own, although their working place is one and the
same site.
The documents, which are produced even within the scope of a standard building project, have developed to such
a huge amount that nobody can examine them thoroughly and bear in mind all necessary details. When using
Building Information Modelling, Simulation Environments and Augmented Reality, the different persons in charge
of the numerous work packages can be addressed individually. So the actual situation is very favourable for
applying innovative tools and methods.
The paper deals with the multiple scales and objectives, which the persons on site have in order to get all
necessary information out of a BIM data base at the right time. A major focus is to show various aspects and views
on the innovative planning tools for the different addressees. Also BIM offers various presentations of the results
of in-depth and highly sophisticated work preparation. In order to enhance the process of realising what is given
in the execution plans drawn from BIM, the perception of the different stakeholders on site are regarded. The
aspects are then further developed to respect the underlying general on-site demand, which is the matching
between the as-planned and the to-be-built status, and which then is transposed to the as-built status.
Finally an overview is given for the benefits, which the people on site will gain from a proper visualisation of the
necessary information about the detailed construction process. This also includes some estimation about the real
cost benefits.
KEYWORDS: building information, one-of-a-kind-process, visualization, transfer of information, complete
design, site management
1. OBJECTIVES IN DESIGNING ONE-OF-A-KIND-PRODUCTS
1.1 The typical one-of-a-kind product
The construction process is unique in several aspects. Many projects are one-of-a-kind, so that the planning and
preparation of work is highly individualized. Still there are similarities to other branches as ship building, plant
layout and others. Therefore it is interesting to reflect on the benefits of visualization of one-of-a-kind processes in
a broader perspective. Still the focus of this paper shall be on the application in construction and building
processes.
One-of-a-kind is defined as
some project, process or
production, which is special
and unique in some degree and
has no other exactly like it. It
stays in contrary to series items
or series productions. The
one-of-a-kind products show
several of the following
features, as worked out by
Franz and Pitsch in the
ASIM-Arbeitsgruppe Unikat-
prozesse (ASIM 2011) in
Germany, Fig. 1.

Fig. 1: Indicators for one-of-a-kind production (ASIM 2011)

30
The planning of a building
according to this is
one-of-a-kind and highly
complex. It involves quite a
number of participants. Already
the client is often diversified
and acting with several
representatives, for example
representing the specialists for
different building functions in
the future service life of the
building, the financing etc. It is
not seldom, that already the
client himself changes his objectives and his priorities several times within the duration of the project, Fig. 2.
Several architects, engineers, project managers and other specialists, representatives from different authorities are
involved, each with his own perspective of what could be the best solution. Then contractors and subcontractors
are bound, who will interpret the so far given drawings and specifications according to their perception.
1.2 The clients objectives
The planning of a modern building is not
sequential any more (from first sketch to
preliminary design to final drawings, shop
drawings and site preparation). Today many
aspects and details of the final result are prepared
at different stages throughout the planning
process, no matter whether they are due at that
phase or not. Fig. 3 indicates by the coloring,
that often some details for the later finishing are
decided already during the initial phase of idea.
And on the other hand it is common, that the final
design has not been finished yet, when the
structural work is already completed. This is also
due to the very long process of designing and
building these one-of-a-kind products, because
during the long period of planning and
construction new ideas, aspects and constraints
emerge.
The concept of Building information modeling (BIM) allows to capture all the required information within one
system. By using sophisticated viewers and different filters it makes it easy to get an overview about the current
status of planning as well as details of the design. Still the process is not deterministic, but rather governed by
reflections, considering alternatives and different options throughout the process. Even if the final solution seems
rather determined, a good architect or engineer tends to even finalize the planning and construction by giving it last
corrections.
A challenging task for any design engineer is to keep up with the objectives of the client. Very often we find that
the determination of the clients is not very specific, so that a good project manager has to guide the client through
all the different phases and alternatives. Sometimes he even has to prevent his client from being too open for more
and more alterations. Many clients show the prioritization shown in Fig. 2, which changes in time. According to
that figure the first choice for clients is the function and the quality of the building. At preliminary design stages
the time frame is often not fixed, whereas the costs are in a middle position. When it comes to the bidding and
award process, the costs, i.e. contract price becomes first priority. Second is the time until completion, whereas the
quality standards are written in the contract documents and therefore are not given special attention. During the
realization of the building the construction time becomes crucial. Quality and quality inspections are next, whereas
sometimes the cost items are postponed to be resolved in later negotiations. Finally, after completion, the most
sensible area for a client is quality and long life quality of the building. So this comes back on place one. Costs then
will be negotiated very hard, whereas the time aspect, no matter, whether the milestones were met or not, is no
longer an item or consideration.
Fig. 3: no sequential detailing of design
Fig. 2: Objectives in the life cycle of a building

31
2. THE ROLE OF VIZUALISATION IN CONSTRUCTION
2.1 Objectives of visualization
The well-known German architect, Mein-
hard von Gerkhan, once said by showing the
new design of the Berlin main railway
station, Fig. 4: when presenting the visuali-
zation of a future building, it is not the
intention to show the building as will it be
in reality, with people on the platform, with
advertisings, with shops and with all the
different installations. The visualization
shall demonstrate the main idea of the
design, the perspectives, the lines going
through, the shape of the larger bodies etc.
So he admitted that the aim was not an
objective representation of the building
within its future context, but an alteration,
not to say manipulation, which will merely
be seen later in reality.
This demonstrates very clearly, that visualization has many different aspects. Visualized objects can transport
much more information in a more comprehensive way than any other medium. And man is able to capture much
more of the information than in other ways, for example as written documents, text, drawings, descriptions etc.
Knowing that the visualization of objects allows the transfer of complex information, it can be used in
construction for the following purposes:
Check on the geometric input data (volumes, boundaries, Finite Element mesh etc.).
show input and output data as diagrams, charts, schedules, using different forms of presentation.
Check completeness of data for quantity take-off, order lists.
Show the different steps of construction as a movie.
Allow to play a game of construction or alteration of the design.
Simulate the behavior of the building in order to optimize different aspects.
Offer the data for augmented reality for use on site.

2.2 Completeness of the design
One major challenge in construction is the fact that not all information will be fixed at the beginning of
construction. It is part of the uniqueness of the one-of-a-kind processes in construction, that the information is
competed as the construction progresses. This is due to the fact, that the resources for planning are limited as
well as some decisions, for example the flower arrangement for the opening ceremony, just need not to be
decided much earlier than during the last week of construction.
In the following reflections about the benefits of visualization the usual limitations and traditions in the
construction process as legal regulations, administrative issues and traditional habits will be as much as possible
disregarded in order to open the mind for the possibilities of visualization on the basis of BIM.
3. STATUS QUO IN CONSTRUCTION CONCRETIZATION
3.1 Availability of information on site
When pointing out the benefits of BIM for construction industry, Eastman and alii (Eastman 2011) describe the
current processes in construction in detail. Thus they demonstrate that a thorough understanding is necessary in
order to show, in which way an owner, an engineer or a contractor can benefit from the gathered information.
Also it shows which part of information should be delivered by the different parties. Von Both and Kindsvater
(vonBoth 2012) have reported on a recent survey on the acceptance of BIM in the German AEC market. It shows,
that the need for information exists by all involved parties, but the access to the hardware and software systems
is lacking. Also there is a need for easier access to the stored data and without having to furnish the whole set of
Fig. 4: architectural 3D-scetch of Berlin main railway station

32
software packages. In Mefisto, a lead project of the German Federal Ministry of Education and Research
(Mefisto2012), the access and decentralized storage of data is subject of developing standardized information
containers, which can be accessed by reading and writing by different parties. However by designing pilot
projects on the Mefisto platform it showed, that the participants tend to keep as much of their detailed
information as possible within their own area and they didnt share it on the platform, if not needed.
When it comes to the construction site, often the
understanding of the delivered drawings and
specifications require quite a big amount of analysis and
common discussion, Fig. 5. The times needed for these
processes are seldom counted, since it is going along
with other tasks and duties during the day of a site
manager and his team.
In construction there are different perspectives on the
schedule. The two most important in construction
processes are the owners view and the contractors view.
The owners or clients view is more on the dates of
delivery, milestones and other fixed dates. This can be
named as time scheduling. The construction company,
which is more interested in the processes, is developing
the construction schedule, which also shows the rate of
progress in its activities. The rate of progress allows the manager to follow the work on site much closer and
already during the first steps of work. Also the construction manager can analyze the different steps of work and
check on the plausibility of the overall layout. Visualization of the separate work steps is helping to understand
complicate geometry as well as complex assembling operations.
3.2 Searching for solutions
When looking at a design process from the beginning, many aspects are dealt with on the basis of rough sketches.
Those sketches are drawn in order to act as a basis for discussion. The engineer explains his ideas and is used to
improve them in a dialog with colleagues and partners. Some later aspects of the design can also be determined
right from the beginning. So it is usual in hotel construction, that the client already builds a sample bathroom for
early choice of material and demonstration of colors and lights. Other architects use standard details, which they
either adapt themselves during the execution planning or which they prescribe for the execution.
From the point of view of the project
manager it is important, that a feasible
process pattern is established as early as
possible and that at the beginning of the
realization on site one approved version of
the execution model exist, according to
which the work can be performed, Fig. 6.
It is issue for good project management
practice, that the manger always keeps the
running of the project within this set of
accepted and feasible documents. If
changes occur, these changes shall not
affect the realization unless the changed
set of information (drawings, model) is
approved and will replace the former set
of approved drawings.
4. CURRENT CHALLENGES
4.1 Traditional views
In this paragraph some major challenges are discussed, which have to be faced especially by the site personnel
when working with BIM and thus with a more elaborate preparation of site operations.
Fig. 5: typical construction site analysis meeting
Fig. 6: 3D-model for work operation on site

33
The perceptions about the different languages to communicate on construction processes on site are still very
different. They reach from drawings (2D or 2D-extractions from 3D) and specifications about quite a thorough 3
years education, additional training and acquired experience (where no specific drawings are required or
demanded any more), to additional specifications, supervision and controlling of the work (by the ones who have
read the specifications) and up to different ways of suggestions and guidelines provided by vendors and other
service units. In some cases samples of the required work standard or samples of bad workmanship are furnished
in order to better shape the outcome. Still the most important means of information are the oral instruction and
the minute of meetings on site.
All these habits of delivering and accepting information are difficult to change at one time. They not only have
their fixed place in many informal descriptions of the processes, they are also controlled by different simple tools
on site, which are hard to change. So the BIM-oriented processes, although being developed to change many
work patterns even on site, must still be set up such that the used work pattern can be followed by the traditional
workforce, which is actually shifting the demand on an adopted BIM-representation rather than on the
workforce.
The main characteristic of a construction process is the time component. Within the representation of the whole
construction time often only some crucial minutes, major events or typical cycles earn the most interest.
Therefore any BIM viewer for the construction processes must be able to show the different stages of a virtual
presentation, run and stop the virtual process at any moment and reverse it as well. A lot of valuable information
from virtual construction is just gained by analyzing the virtual scenery in different speed and by then changing
the setting for the real world.
The perspective of the different stakeholders on the virtual representation is very diverse (Bargstaedt 2012). Each
specialist is interested in different details and has to evaluate them. The architect intends to give an overall view
and shows the matching of shapes and colors. The structural engineer looks for the bearing elements and their
joints. By using the common data base clash detection can be automated. But this does not relieve the engineer
from first developing a scheme to define the main routes and positions for pipes and cables within the building.
Only within this scheme the proper design of pipes, circuits and cables makes sense. In practice already large
teams have met to solve all conflicts detected by a clash detection program, but without being given a general
scheme and first having established rules for conflicts and intersections. Then hours of common meetings on the
virtual model are spent to detect and solve all clashes. So easily several days of common engineering work are
summed up in order to prevent later hazards on site. But this detection and elimination of hundreds of clashes by
visualization and with the help of clash detection programs is not breaking news, if the simple and beforehand
definition of schemes and rules for the design corridors would not have produced any conflict at all.
4.2 Visualizing processes and its current obstacles
The main demand for
visualizing processes in
construction is not the actual
status, but the future ongoing of
the project. On different levels
of detailing the supervisor, the
site manager as well as foremen
and other site personnel need to
check on the information for
future action. Therefore the
status of the site at specific dates
in the future is most important
for them. This is a major
requirement, for example
facilitated in BIM, for example
in the project MEFISTO, Fig. 7
(MEFISTO 2012).
One aspect to be mentioned in
this context is, that most site
engineers and managers are kept
Fig. 7: high rise building modeled according to its construction processes

34
too busy as to really have enough lead time to get involved in the preparation of the upcoming activities. So
pushing information to the site, when it is not yet time to be troubled with it, is not useful. Site managers rather
work in pull-mode. That means that only at the time, when they have some time available for going through the
next steps of work, they will search for the necessary information.
5. BENEFITS OF VISUALIZATION
5.1 General remarks
When discussing the benefits of visualization and the use of BIM, a broad variety of arguments can be found in
opposite to it. So it should be remembered, what and where benefits are already gained today by using BIM and
its visualization tools.
One main point is the evaluation of alternatives. Good engineering practice needs the discussion about different
solutions, about alternatives to the first sketch, it needs the common development of alternatives and their
advantages and disadvantages. By using good visualization tools the proposals are more easily to be understood,
so that the best alternative can be chosen.
Often the decision about one alternative has to be taken on the basis of best available information. In this case
the further process of optimization of this alternative will also be supported by good and easy to be understood
documentation. Often in practice just the fact that a decision was taken for one specific alternative has been the
starting point of an effective optimization process in order to make the best out of this proposal.
Any visualization of construction processes tends to show the planning results of an undisturbed production.
That is because any disturbance, any obstacle, will, if observed within a simulation or a visualization, most likely
result in a longer time for production, a shut down or an interruption of the production. Activities, which have to
be repeated due to quality problems or other faulty work, are very complicated to model and tend to be constraint
and inflexible in their appearance. The reality procures such a tremendous amount of possible disturbances and
obstacles, that it is not realistic to model them in detail. Therefore only the consequences are modeled, which are
first of all the time elapsed without effective production. .
Processes, which are designed from detail to the whole, tend to be modeled with a too short time demand.
Looking on an undisturbed and optimized process on site yields durations, which are too small. Therefore in
construction practice, the art of estimating required time, includes a realistic estimation of buffer times at several
levels. For example when calculating the crane capacity for formwork, the existing crane capacity should not be
used for more than 50 or 60 % of the available time. When designing the crane process for assembling work, the
calculated crane time should not exceed more than 85 % of the installed crane capacity. In many processes on
site buffer time of 50 to 80 % of the total time are realistic. The analysis by visualizing the process can bring
improvements in time of a high percentage. Any site manager, who wants to transfer these calculated results into
reality has to decide, how much of the calculated saving time can be incorporated on this specific site.
On site different kind of skilled people are active. On traditional German construction sites, the craftspeople are
highly skilled and experienced. Even if the current development shows, that more and more young,
inexperienced or unmotivated people are met on site, there is still a large reservoir of experience and a tendency
of the people that they want to understand what they are doing. Since most of them are educated within this old
established environment of construction sites, where site manager and foremen organize their work, the tendency
of bringing in ideas for improvement are still considerable. But the site demands are becoming more and more
complex. Many situations arise on site, where the proposals and alternatives of the people on site cannot been
evaluate any more without thorough understanding of the underlying complex overall system. Here visualization
tools are needed, which show the complexity of the building and also the consequences of any changes.
5.2 Indirect benefits
There is already a large area where visualization of construction processes has become standard. It is in the field
of standard items for outfitting and finishing trades. The supply industry has been very active in the past to
develop articles which make assembling easier and more perfect. For example standard installation units are
mostly delivered together with detailed drawings about how to install them, how to assemble and connect them
with the building. Many suppliers offer this information via internet, so that the craftsman can look up on the
internet and ask questions, get assembling explanations for assembling. Some suppliers, for example several
international formwork specialists, already offer quite a range of educational and instruction movies via internet.

35
Here the challenge for the site manager is not to generate and get the information. The challenge is to recognize
the need for education and training among his workers on site and then get the right, pinpointed movie, power
point show or sketch right on the spot.
Indirect benefit can be gained by
analyzing more thoroughly in less time and thus avoiding later mistakes,
showing the processes in order to activate also the intuitive control of sequences and relationships,
committing oneself to a more thorough site preparation, simply by making the effort to visualize it,
general and easy improvement of personal knowledge and expertise
One hindering and costly aspect is, when the crews do not exactly know how to do the job. This might result in
quite a number of visible activities, showing the crew occupied with studying the drawings, with discussing the
solution on site. Then slack time occurs, the quantity of work performed diminishes and becomes less intensive,
sometimes without the foreman and the site manager really noticing it. Therefore visualization done in advance,
giving a good image of what has to be performed, is a key factor for efficient production especially in
one-of-a-kind-processes.
The average ratio of salary for workers compared to engineers in Germany is between 1 : 2 and 1 : 4. This makes
it more expensive to spend an engineering hour instead of a workers hour. The number of engineers in
comparison to the number of workers attached to certain processes, however, is between 1 : 10 and 1 : 30. So it
is much more economical to invest in the higher effort for engineering skills and preparation of drawings,
schedules and visualization. However, not in every area there are enough skilled engineers available. Then it is
still difficult to set the main focus on work preparation well in advance and to use the appropriate visualization
tools.
5.3 Direct cost benefits
Proving cost benefits in
one-of-a-kind processes seems to be
extremely difficult, because there is
no repetition of processes. So it
doesnt allow for alternative runs,
one without and the other with the
installed improvement measures.
This difficulty is inherent in most
areas of construction projects as
well as in project management in
general.
In most cases the cost accounting for site operations is not very elaborate. Fig. 8 shows some possible ways of
rework during the lifecycle of a building element. During the scheduled activity for that element there are
different stages, at which rework is performed still within the regular work timeframe (second start, corrections)
and will not be accounted for separately. Still during internal supervision (refinish), during and after final
acceptance by the client (punch list) and even sometimes during the warranty period (repair) often the costs of
rework are not attached to the phase of work, but are still allocated to the cost unit of the original building
element. Only after financial closure of the project the arising costs for rework are written to the warranty budget.
That makes it difficult to exactly pinpoint the costs as well as the benefits by saved time and money due to better
work preparation and visualization.
In terms of quantifiable benefits the following should be taken into consideration, as Bargstaedt and
Spieckermann stated in (ASIM 2011):
Discover hidden costs because of wrong layout planning and dimensioning.
Reduce the necessary site area by optimizing the process planning.
Reduce the necessary equipment and its service time on the basis of visualized planning.
Reduce the risk of liquidated damages by gaining more buffer time during operation.
Have better proof in case of alterations, disturbance, delays and other obstacles.
Check on and improve the key values for performance ratio of workmen and equipment.
Fig. 8: typical accounting costs along rework activities on an element

36
Nevertheless these aspects are still difficult to measure, if the cost accounting is not done according to the
estimated schemes. Having established the ideal duration of activities and processes, this is the measure line for
deviations. Non-optimum processes can result in a lack of quality as well as in unforeseen idle time. Most
dangerous is the lack of efficiency, where the crews work at a slower path than normal, or where the equipment
is used below its expectations. In these cases the loss of time might not be obvious at first glance, but still can be
measured clearly, if times are taken from the daily performance. If comparing the actual process with the planned
processes, the differences become transparent. By taking the visualization of a process, this standard sequence
and velocity can be set as a reference for the real work.
6. SUMMARY
Construction is a typical one-of-a-kind operation for one-of-a-kind products. Therefore information and
communication play a major role in disseminating the necessary knowledge to all involved parties. Here
visualization offers new perspectives for better, easier and more effectively instruction the people on site. It also
assists the site in doing better work panning and for the evaluation of alternatives and optimization.
The benefits by using a thorough visualization of the construction processes are non-direct as well as direct.
Although most accounting procedures on site are not setup such that they differentiate between the original work
items and different areas of rework, there are still a part of quantifiable benefits. Since concerning most of the
site people work in pull-mode, the information must easily be kept available at the time of requirement by site
managers and workmen.
7. REFERENCES
Arbeitsgruppe Unikatprozesse (ASIM 2011) in ASIM Fachgruppe Simulation in Produktion und Logistik.
http://www.asim-gi.org/index.php?id=192
Bargstaedt, H.-J. (Bargstaedt 2012) Transparency of Construction Knowledge in BIM-oriented Processes.
Published electronically, 14
th
International Conference on Computing in Civil and Building Engineering, Moskow,
2012
Eastman, C., Teicholz, P., Sacks, R. and Liston, K. (Eastman2011) BIM Handbook. A guide to building
information modeling for owners, managers, designers, engineers, and contractors, 2
nd
edition. Wiley, Hoboken
von Both, P. and Kindsvater, A. (vonBoth 2012) Potentials and barriers for implementing BIOM in the German
AEC market Results of a current market analysis. Published electronically, 14
th
International Conference on
Computing in Civil and Building Engineering, Moskow, 2012
Scherer, R. J., Schapke, S. E. and Tauscher, H. (MEFISTO 2012): MEFISTO: Management- Fhrung
Information Simulation im Bauwesen. Tagungsband 2. Mefisto-Kongress, 13. October 2011, Dresden

Figures not generated by the author :
Fig. 4: http://www.tu-cottbus.de/theoriederarchitektur/Wolke/deu/Themen/061%2B062/Krueger/abb09-1_
Gewoelbe Entwurf.jpg
Fig. 5: http://news.agc.org/wp-content/uploads/2011/02/Cape-Site-Meeting-686x458.jpg
Fig. 6: http://www.pm-akademie.de/pm_academy/img/content/CS_M36-D-Baustelle-5-1-504x321.jpg
Fig. 7: Research report Mefisto ErgB_B-5[1].1, 2011


DIGITAL MEDIATED TECHNIQUES FOR THE KNOWLEDGE OF
THE EVOLUTIONARY PECULIARITIES OF THE BUILT
ENVIRONMENT: THREE CASE STUDIES
Danilo Di Mascio
Department of Architecture,Faculty of Architecture, G. d'Annunzio University, Pescara-Chieti, Italy
ABSTRACT: The following work is part of a Ph.D. thesis, having as its general objective the use of ICT in the
knowledge and documentation of five peculiarities of the built environment. The built environment is
acknowledged as a value both as material resource and cultural heritage to be preserved. I believe that digital
technologies give the opportunity to expand the knowledge of complex artifacts and reflect on traditional
problems in a different way. In this paper I will deal with the interpretation, representation and communication
of the features defined as "evolutionary". With the analysis of these features I want to pay attention to the time
variable declined in two different ways, both united by a progression of human actions over in time. At first, I
consider the transformations that have affected an artifact or a settlement during the course of centuries or
years, and have influenced the present configuration; secondly I examine the various realization phases of an
artifact, characterized by a montage of its technical elements. To represent and communicate the different phases
of construction I do not use dynamic motion, but a sequence of static single frames, where each frame describes
a phase and can be described in turn and fulfilled by other textual information. Consequently, I will present three
case studies: the medieval village of Castelnuovo (Italy), damaged by an earthquake; a raw earth house, located
in Figuig (Morocco); a trabocco, a pile dwelling construction used for fishing, located along Abruzzos coast
(Italy). Then, for each case study I will present the objectives, some issues that have to be faced and a
methodology that involves the use of software such as AutoCAD and 3ds Max. In the latter part of the work, the
results will be presented and commented.
KEYWORDS: 3d modeling, construction, evolution, montage, built environment, representation, communication.
1. INTRODUCTION
This work is part of a Ph.D. thesis (successfully completed in May 2012. The Research has received the Doctor
Europaeus award, an additional title obtained after meeting the requirements established in 1992 by the European
University Association and by the Norms and Regulations of the University "G. D'Annunzio" Chieti-Pescara
related to the Ph.D.) and its general objective is related to the identification and testing of new methods, tools and
techniques, with particular focus on ICT, knowledge and documentation of five characteristics of the historical and
contemporary built environment. Some topics of the thesis have been published as conference papers, while others
are being published.
In the first chapter of the thesis, through the analysis of international documents (such as international charters on
conservation and restoration or documents published by organizations such as UNESCO or ICOMOS), the built
environment has been recognized as a valuable material and cultural resource to be maintained and enhanced for
the present and future generations. Often the interventions on the built heritage are being made without any
adequate documentation on the basis of its technical and cultural characteristics. This method of procedure causes
changes in various aspects with there are the cultural, aesthetic, structural and functional ones. These errors are
caused by the negligence of those who intervene in these situations, but also by various difficulties raised by the
artifact or the settlement where works have to be done.
I believe that ICT provide methods e tools and, together with the traditional use and theories, they enable to tackle
and resolve difficulties and to expand and record the knowledge of these artifacts. Thanks to these new digital
technologies it is possible to reflect on traditional problems in a new way. In this research the term knowledge
means the understanding of the built environment and in particular the peculiarities analyzed.
The analysis of international documents and some personal considerations have identified five characteristics,
material and immaterial, important to improve the understanding, management and valorization of the historical
and contemporary built heritage; the aim is to preserve and/or enhance physical objects and intangible aspects in
respect of the cultural qualities. The five selected characteristics are: morphological, constructive, evolutionary,
perceptual and relational peculiarities. These peculiarities also define the limits of the field of investigation of the
research to specific theories and concepts for the study of the built environment and to determined categories of
37

ICT. In this paper I will focus on the evolutionary peculiarities.
2. OBJECTIVE: THE EVOLUTIONARY PECULIARITIES
With the analysis of the evolutionary peculiarities I want to pay attention to the time variable declined/interpreted
in two different ways, both united by a progression of human actions in time, in fact one of the meanings of the
word "evolution" in the Oxford Dictionary is "the gradual development of something". In the first interpretation I
consider the transformations that have affected a building or a settlement during the course of centuries or years,
and have influenced the current configuration. For example, it is possible to take into consideration interventions
such as the demolition or extension of a dwelling to modify existing functions or accommodate new ones. In the
second interpretation it is possible to consider the various building stages of an artifact, characterized by the
assembly or progressive construction of the technical elements. During a lapse of time it is possible to assist to the
development (or processing) of more or less quick, more or less in-depth features often bind among them. In fact,
an operation of extension of a dwelling unit to accommodate the new functions is a variation of morphological,
structural, perceptual aspects in almost all cases.
In summary it is believed that the study of the transformations and assembling and constructive phases may be
useful to understand and learn:
development of a settlement during a given period, with all the variables connected to it;
materials and construction method of an artifact.
3. METHODOLOGICAL APPROACH
The working method has been developed taking into consideration three essential actions in the path of
understanding and documenting the evolution of built features: interpret, represent and communicate.
The interpretation is an essential action when facing the study of the built environment because often the available
documents (i.e. recoverable data through archival researches and those achievable during a site inspection) are not
sufficient to fill in all the gaps. Therefore it is needed a creative interpretation to overcome some obscure points
that otherwise would block the progress of the research. Within this research I intend to show situations with the
action of representing, otherwise they cannot be seen in reality. The representation will allow to give form to a
discussion so as to verify if interpretations and hypothesis are correct or if its necessary to modify and rectify. In
the creative interpretation process it is possible to move along the elaboration of various hypotheses, realizing
different two-dimensional graphic works and three-dimensional models. The creation of this material can be
possible by using both the existing documentation and intuitions that can come from the comparative analysis with
similar artifacts or by the application of knowledge belonging to a personal cultural background and gained both in
architecture and in other fields. With the action of communicating it is possible to indicate the intention to transmit
to others in a clear and effective way information that are represented for the first time and that will be used for
further studies.
The interpretation stage follows the preliminary stage of data collection (including archival research) and analysis
of an existing documentation, which may consist of historical documents and drawings produced during site
inspections (when it is possible to make them). The experiences of other studies have shown that each working
phase is not autonomous but is always characterized by multiple actions. Moreover, the process does not develop
itself in a linear manner but it follows a continuous interpretation, representation, evaluation of options leading to
a progressive modification and refinement of the works. For example, the interpretation and representation phases
are closely linked.
The main digital technologies selected for this study belong to the categories of CAAD and three-dimensional
modelers. The CAAD are especially useful to create metrically precise two-dimensional drawings and 3D
modelers to create three-dimensional digital models of the objects of study, organize objects through layers and
render images. Laser scanning technologies were not used because they were not deemed fit to tackle the
challenges posed by individual case studies: more precise details related to this choice will be described in the
section referring to the discussion of the results.
After defining the main actions to be taken and digital technologies to be used, it was necessary to define a method
of representation and organization of the clear and effective elements, useful in the interpretation and
38

communication stages.
Terms and concepts such as time, sequence assembly, which characterize the evolutionary peculiarities, directs us
to the movie sector where it is possible to find the same words used in a slightly different way. The use of film
techniques to communicate in a clear and immediate way the assembling phases of an artifact is not a new
procedure and does not belong to recent years. In 1941 Ray Harryhausen, the famous American film producer, best
known for being the creator of special effects (a true expertise in the technique called stop-motion, and his most
famous movies are the ones with mythological creatures animated with this technique e.i. the Argonauts) created
a stop-motion movie titled "How to Bridge a Gorge". In this animation, designed to educate the American troops
on how to assembly a wooden bridge, Harryhausen demonstrates effective "three-dimensional possibilities of
animation as a visual teaching system. A science applied in the study of mechanical assemblies, installation and
repair of complicated instruments, strategic and operational problems with defined conditions. Harryhausen
states that these techniques are effective in reducing the learning time to a minimum because they are very
communicative. In this first example, we are dealing with an animation that demonstrates the effectiveness of a
movie in communicating constructive information, but even more interesting for the goals of this research are the
experiments of Eadweard Muybridge, English photographer pioneer of the photography of the movement. In 1878
Muybridge performed an experiment called "Horse in Motion" (and followed by many other experiments) with
whom he proved that while the horse was running there was an instant moment in which all four paws would be off
the ground. This demonstration was made through a sequence of photographs taken by 24 cameras, each operated
by a cord struck by the passage of the animal. Hence with this method Muybridge breaks down a continuous action
in a sequence of still images. All these images illustrates the movements made by a horse in the race. This is
exactly our goal, or rather tell/record the stages of transformation and/or installation/construction of a settlement
and/or an artifact through the breaking down of these phases in a sequence of static images. In fact, to represent and
communicate the different stages of transformation/construction we will not use dynamic animations, but a
sequence of single static frames (remember that dynamic animation is nothing but a sequence of still images
played back at a certain speed to give the illusion of movement), each frame will show a particular time and may in
turn be described and supplemented by other textual information.
The word "mediated" in the title of this paper (digital mediated techniques [...]) points out that in the methodology
used the digital techniques are mediated both by a traditional architectural knowledge (in particular construction
techniques and materials), and by theories belonging to the cinematographic world.
Three case studies have been chosen at two different scales (two of which are on the same scale of the artifact but
with a totally different construction technique) because they allow to deal with various problems linked to the
evolutionary peculiarities:
the scale of the settlement: the fortified town of Castelnuovo (Abruzzo, Italy);
the scale of the building: a house in clay (Figuig, Morocco) and a trabocco of pier (Abruzzo, Italy).
4. THE CASE STUDIES: OBJECTIVES, ISSUES AND METHODOLOGIES
In this chapter there will be presented three case studies. Each case study has been introduced with a short
general overview followed by: main objectives, questions to be faced and used methodology. In the case studies I
will mainly focus on the assembling sequences (that represent the evolutionary peculiarities), and not just on the
digital three-dimensional reconstruction of the artifact, since this topic is more keen on the constructive
peculiarities. However, in general it is possible to state that the digital reconstruction is a process that owns an
autonomous value, independent from further analysis, because the same reconstruction process contributes to
deepen and broaden the knowledge of the artifact, it is an interactive process of continuous construction and
evaluation.
4.1 The first case study: The fortified centre of Castelnuovo (Abruzzo, Italy)
4.1.1 General description
The tragic earthquake that hit the regional capital LAquila caused damages of various entity in more than 50 little
villages, including Castelnuovo, a borough of the village of San Pio delle Camere, which counts less than two
hundred inhabitants.
Castelnuovo is locate within the Piana di Navelli, but in a barycentric position. The village rises on the top of a hill
39

where it dominates the entire landscape, today marked also by an important road. This privileged position inserted,
during the Middle Age, Castelnuovo, and in particular its fortified centre, inside the spotting and defense system of
the valley of LAquila.
From the XII century to present days, the village has known various growing phases. In this research, we have
dealt with the fortified centre, the most ancient and important area of the village, which is an urban settlement
inspired to the Roman castrum. This peculiarity, that distinguishes Castelnuovo from the other nearby towns,
derives probably from the closeness of the ancient Roman settlement of Peltuinum.
4.1.2 Main objectives
The main objectives of this working phase are the following:
to understand and document the morphological-functional transformation of the fortified centre of
Castelnuovo;
to individuate, represent and describe the criticalities, which are the aspects that have altered the original
plan.
4.1.3 Issues to be faced
The main issues to be faced mainly come from the lack of documentation that might help the understanding of the
morphological transformations occurred in the fortified centre during the centuries. To identify the criticalities it
was only possible to use some available information, given the impossibility of visiting the village because of the
dangerous buildings.
4.1.4 Analysis of the morphological-functional transformations
The model used for the stratigraphic reading is the result of creative digital reconstruction documented in (Di
Mascio, 2010). Starting from the model created in the previous phase, it has been possible to make a multilayered
reading to better understand and document the morphological-functional transformations of the village: these
transformations have affected mainly the increase of the heights of the buildings. In the past, most of the residential
units were mainly constituted by two rooms: one on the ground level (usually a stable), laid approximately about
1m under the street level (Fig. 1 - image 1) and one at the upper floor, used as a dwelling (Fig. 1 !
- image 2). During the course of centuries, new needs have brought to the change of the destination use
and volumes, in particular there is an increase of the number of floors up to five (Fig. 1 - image 3). In addition to
the traditional pitched roofs, it is possible to trace also some terraces and loggias (Fig. 1 - image 4). The result of
this first reading shows us a complex urban pattern, constituted of buildings which are very different from each
other, what concerns both the historical period and the constructive techniques.
4.1.5 Analysis of the criticalities
In another three-dimensional model, based on the previous one and defined as criticality model (Fig. 1 - right), we
have identified the spontaneous transformations, such as the abandonment or demolition of parts of the buildings,
or the closure of important pathways. In order to make the communication clearer and more effective, it has been
chosen not to use the complete model of the village, but a synthesis previously elaborated. The perimeter of the
buildings is the same, as well as the perimeters of the single residential units. To represent the buildings belonging
to the original system of the village, it has been chosen the white color, which is neutral. To highlight the critical
issues, the symbols used have been the following: red colored solids to indicate the additions outside of the original
perimeter of the fortified center (including the volumes that close the paths); red colored and translucent solids that
identify the demolitions and the anomalous spaces; empty walls to report the abandoned buildings.
The previous analysis have made clear that in designing a new village, it is necessary to maintain some
qualities/important features that have been lost over the centuries. Initially, for example, all the paths had an exit,
while during the years, three of the six secondary way outs have been closed.
40

Fig. 1: (left) Stratigraphic reading of the morphological transformations of the fortified centre during the course of
centuries; (right) the 3D model with the representation of the different criticalities.
4.2 The second case study: A raw earth dwelling (Figuig, Marocco)
The analyzed building is located in the city of Figuig in the eastern area of Morocco, to the border with Algeria.
The city, built around an oasis, consists of seven ksour which are typical fortified villages in north Africa. In a ksar
there are collective structures such as barns, shops, religious buildings and private homes. The homes have courts
and they are all juxtaposed to form a compact tissue which facilitates defensive actions. The main constructive
material for all the structures is the adobe, which are bricks made of soil and water; in some cases it is possible to
find the cut stone.
To suppose and document the realization phases of the building. The information should be visually communicated
in a clear and effective way; each further phase has to clearly demonstrate which are the new elements and how
they are linked to the present ones.
The main problems are represented by:
lack of detailed information about the different realization phases, and difficulty in obtaining new specific
data on the Figuig buildings;
identification of an adequate method to document and communicate the various construction phases.
4.2.4 Methodology
For a clear and effective communication I have decided to represent the single construction phases directly in their
final configuration and through static images. By considering the same model realized to analyze and document
the constructive characteristics of the dwelling, it has been set and rendered a series of images that document and
represent the building sequence. The entire process has been summarized through ten phases (Fig. 2): the first
phase coincides with the masonry foundation structure; the last conclusive phase ends with the superior horizontal
closure in earth, sand and lime. The building has some openings that related it with other adjacent routes that are
part of another dwelling unit included in the same property. This further information that consider the artifact as
part of a building set and not as an isolated episode, have been represented with semitransparent synthetic volumes.
To keep the same scale of the building it has been chosen a shot able to clearly document and include all the
installation phases. With this method, each image represents a frame of a sequence that, just like in cinematography,
has the aim of narrating the process of the installation phases of the artifact. The decision to use static images
instead of a movie comes from the necessity of including the graphic works among a paper documentation, and to
complete it with textual description and information. This does not preclude the possibility of creating an
animation with the same frames, that could be inserted in a website or in a presentation video.
41

Fig. 2: The 10 frames documenting the installation sequence of raw earth dwelling.
42

4.3 The third case study: A trabocco of pier (Abruzzo, Italy)
The trabocchi are pile-dwelling constructions, particularly suitable to environmental situations, such as marshy
lands and subject to flood or lake shores, located along Abruzzos coast (a region in central Italy, on the Adriatic
Coast). These artifacts date back probably to the XVII century when some familiar groups coming from the
Centre-North of Europe (in particular Germany and France) settled in the area between the villages of San Vito
Chietino and Rocca San Giovanni, localities near the coasts devastated several years before by the earthquakes and
seaquakes (Cupido, 2003). Through the wits and the handicraft knowledge, these foreign families overcome their
seafaring lacks (as they were not able to swim or arrange a boat), by building a system that could allow them to fish
close by the seashore (at that time fishing was the only livelihood source), even then, due to the rough or
non-limpid sea, it was not possible to use harpoons and fishing spears. For the maintenance and construction of
new trabocchi, the constructors (called traboccanti) used to manage a small raft of reeds fastened together, pushed
with a long stick. With this raft they used to carry trunks and other materials necessary to their construction.
(Forlani, 2005) In the past times along the Abruzzo coast there were about 50 trabocchi, but unfortunately
nowadays more than half remained and have been preserved in different ways. All the trabocchi on the coast
belong by now to private owners, except one, the trabocco of Punta Turchinio (so called because of the place
where it is located), owned by a public entity, and maintained by San Vito Chietino Council.
To hypothesize, represent and document the installation phases of a trabocco of pier:
the information must be provided in a visually clear and effective way with technical elements easily
recognizable;
each phase has to clearly demonstrate which are the new elements and how they are linked to the present
ones.
The available documentation and information are not sufficient to elaborate an assembling sequence, therefore it
was necessary to hypothesize the single phases.
4.3.4 Methodology
As first step, I have researched and collected documents that contain specific information on the constructive
phases of the trabocchi; the collected documentation is the result of previous publications and researches
containing short descriptions and hand drawings, which show some constructive phases of a typical trabocco. The
images summarize the constructive phases in a few sequences hence from an image to another many elements are
added that make difficult a clear comprehension both of the insertion order and the existing relationship between
an element and another. In the 29 trabocchi located along Abruzzo's coast, the constructive systems and the many
technical elements are repeated in various artifacts, but the traboccchi are not similar to each other, they are all
unique. For this reason, it was decided to document the assembling phases of a typical trabocco, taken as example.
The trabocco Turchinio, case study of a previous research (Di Mascio, 2009), with its abundance of structural
elements of the platform, the high number of ropes and slings, and with its long footbridge was considered too
complex to communicate the information in a clear and effective way; but the knowledge background achieved
during the morphological and constructive analysis of the Turchinio (the structural engineers collaborated for the
structural part), together with the direct observation of other trabocchi (and to the documentation collected in the
preliminary phase), allowed to digitally built a new trabocco. The new virtual construction exemplifies the
assembling phases of a typical trabocco of pier, bringing out which are the used elements and their assembling
order. Among the different considered constructive methods it has been chosen the following because it represents
the most simple prototype, related to the number (and type) of essential elements. As in the previous case study the
single images show the elements in their final position, hence already installed and without supporting elements.
The only exception is the figure number three, that shows the positioning of a provisional pole in water through a
beam linked to the pole on the ground through provisional ropes and cables. Pictures 9 and 10 show the assembling
order of the definitive elements, hence without provisional supporting ropes and cables. Hence, the picture 11
shows the cables in their final permanent configuration.
43

Also for the trabocco, as well as for the Figuig dwelling, it was chosen a adequate shot to contain all the
progressive assembling phases of the construction. But unlike the previous work, each rendered image does not
coincide with the subdivision of the typical layers of a technological breakdown. The different constructive system
of the trabocco, made of point elements, piles and beams needs to be divided into layers according to another order.
The used classification scheme coincides with the assembling order of the elements. To clarify and prepare the
sequence of the images, it was used another typical technique of the film sector, the storyboard. The structure and
the succession of the images has been analyzed through handmade drawings. To allow a correct and clear
interpretation of the various phases, it was decided to divide the process into 16 frames (Fig. 3).
Fig. 3: The 16 frames documenting the assembling sequence of a trabocco of pier.
5. DISCUSSIONS OF THE RESULTS
One of the main objectives of the research was to define, experiment and propose a methodology for the
comprehension and documentation of the evolutionary peculiarities of an artifact or a settlement. The
experimentation of the three case studies suggests some reflections on tools and methods used. When the available
documents on the building under study were not sufficient it was useful to identify buildings with similar
characteristics (we refer to the case study Figuig) and to made a comparison providing some useful information for
further research. For Castelnuovo 3D models have been prepared and rendered images have been made
representing a hypothesis of multilayer reading, useful to better understand and document the morphological and
functional changes and the criticality of the village. For the raw earth dwelling in Figuig it was possible to use a
model created to analyze and document the constructive peculiarities of the artifact, at this stage it has been set and
rendered a series of images documenting the various stages of implementation of the building. To maintain the
same scale of the building it has been chosen a fixed frame that managed to clearly document and to include all the
phases of implementation. For the trabocco the phases of assembly were organized and represented. The trabocco
being a wooden construction on stilts, it has very different characteristics and structural elements from the house of
44

Figuig. Given the complexity of the structure, non-standard, composed of many elements in wood, it was
necessary to prepare a documentation similar to a storyboard to correctly interpret and translate the descriptive
information in images. Also in this case has been selected a frame to document and include all the phases of the
assembly. Without a representation of the stages through an organized sequence of images would be impossible to
communicate the same information with other traditional methods. A multidisciplinary approach (architectural
knowledge, methods and tools of digital technologies and theories of cinematography) has allowed us to formulate
a clear methodology to support the various phases of work, starting from the interpretation and the representation
and ending with the communication. The aim to create a sequence of images documenting the construction of an
artifact is to communicate accurate information and not to entertain. If the information are improperly transferred
the sequence will have no informational value, but will be a simple graphic exercise, not a job of an architect or
engineer. It is not advisable to show work done without a proper interpretation of construction to students or other
interested parties, since it would provide inaccurate information that could be confusing. The following table
summarizes the positive use of digital technologies in the three case studies of this research and compared with
traditional techniques:
DIGITAL TECHNOLOGIES TRADIZIONAL TECHNOLOGIES
test and evaluate various hypotheses quickly modifying
some parts of the same model;
the documents produced are easily updatable and expandable
over time;
create a three dimensional model that allows you to observe
the objects of study from any angle and any distance, the
degree of detail of the model can be easily deepened at any
time and at any point using the same three-dimensional object;
a three-dimensional model allows a better assessment of the
construction aspects because it simulates the position of the
technical elements in a 3D space and the relationship that
exists between each technical element;
the documents produced can be a base for further
experimentation.
to modify drawings produced in the traditional way
(perspectives and axonometric) and models is a lengthy and
complex process;
traditional drawings and models are very difficult to amend
or enlarge unless they are recreated with new information;
traditional drawings (elevations and axonometric views) are
bound to definite points of view, so provide partial and limited
visions; models are bound to the defined scale of its creation;
to create a scaled detailed model has high costs (time,
materials and equipment); traditional drawings doesnt allow to
correctly estimate the position and relations between the
different technical elements;
drawings and models produced in the traditional way can
hardly be the basis for other drawings, but are only of useful
references.
The benefit of the methodology developed and proposed in this paper can be summarized as follows:
all the produced images are the representation, communication and completion of the textual descriptions
and they make the understanding of the transformation, construction and assembling phases clearer and
more efficient, mainly if compared with the only written text;
the graphic sequences produced make evident the relations among the various elements, defining who is
in relationship with what; in fact, the new elements introduced during each phase are connected or
overlapped to the existing ones;
a better understanding of the relations among the technical elements allows to reduce the mistakes and to
improve the recovery and maintenance interventions;
to each image there could be descriptive information on the type and quantity of used materials, since
these are information useful for a Life Cycle Assessment (LCA);
the comprehension and documentation of the montage/assembly phases represent a useful support also
during the dismantling/demolition phases.
6. CONCLUSIONS
In this paper it has been presented a methodology for the comprehension and documentation of the evolutionary
peculiarities of the historic and contemporary built environment and its application to three case studies. In the
understanding and representation/documentation of these peculiarities proper use of digital tools and methods
45

must be supported by the background and technical knowledge of an architect that allows to correctly document its
construction aspects and avoid making an animation that would have the value of a simple graphic exercise. In fact
there often are detailed animations only in the graphic part, while the order and the assembling mode if the
elements are in part or totally wrong. Furthermore, in the architecture field is often used an empirical approach for
the creation of static rendering and animations, while on the contrary theories and techniques from the filmmaking
world are very useful in providing a theoretical basis to support various experiments. As it happens in other studies
a multidisciplinary and trans-disciplinary approach, along with theories and methods of information technology is
useful in an innovative way to tackle old and new problems. Each building belonging to the built heritage presents
construction characteristics and problems that are often unique. A creative multidisciplinary approach allows us to
understand and document features like the evolutionary peculiarities which are difficult to analyze and represent
only with traditional methods and tools. Even though the elaborated methods are adequate to study artifacts that
present similar features of the analyzed case studies, it is though that they could be widened and deepened further
on. The basic theoretical approach is always valid, but the built environment presents different situations that
require the definition of specific working choices. Other theoretical studies and practical experiments for new case
studies are still in progress.
7. ACKNOWLEDGMENTS
I would like to thank Professor Maria Cristina Forlani for permitting me to study these subjects and my sister Ilaria
Di Mascio for helping me with the translation of this paper from Italian to English.
8. REFERENCES
As, I., Schodek, D. (2008). Dynamic digital representations in architecture. Visions in motion. Abingdon, Oxon,
United Kingdom: Taylor & Francis.
Cupido, P. (2003). Trabocchi, traboccanti e briganti. Ortona (Ch), Italy: Edizioni Menab.
Di Mascio, D. (2012) "ICT in the knowledge and in the documentation of the peculiarities of the historical and
contemporary built environment. Methodological paths and case studies", PhD thesis, Scuola Superiore G.
dAnnunzio School of Advanced Studies, G..d'Annunzio University, Chieti-Pescara, Italy.
Di Mascio, D. (2010). Preserving memories with digital media: a methodology for the reconstruction of
Castelnuovo Village, CAADRIA2010: New Frontiers (15th International Conference Proceedings on Computer
Aided Architectural Design Research in Asia) Hong Kong, China, 7-10 April 2010, pp. 83-92.
Di Mascio, D. (2009). Digital Reconstruction and Analysis of Turchinios trabocco: A method of digital
reconstruction of a complex structure as a way to improve our knowledge of a cultural heritage artifact, Digitizing
Architecture: Formalization and Content [4th International Conference Proceedings of the Arab Society for
Computer Aided Architectural Design (ASCAAD 2009)], Manama, Kingdom of Bahrain, 11-12 May 2009, pp.
177-189.
Eadweard Muybridge (2012). Encyclopdia Britannica Online,
<http://www.britannica.com/EBchecked/topic/399928/Eadweard-Muybridge> (last accessed 10 June 2012)
Evolution, in Oxford Dictionaries, The world's most trusted dictionaries, < http://www.oxforddictionaries.com/>
(last accessed 06 June 2012).
Forlani, M. C. (2005). I trabocchi, archetipi costruttivi della leggerezza. Il Progetto Sostenibile, 8.
Knight, M., Bandyopadhyay, S., Berridge, P. and Brown, A.: 2001, Digital Hindcasting - Critical Analysis through
Virtual Reconstruction, Architectural Information Management (19th eCAADe Conference Proceedings) Helsinki
(Finland) 29-31 August 2001, pp. 529-533.
Munari, B. (2008) Design e comunicazione visiva. Bari, Italy: Editori Laterza.
Sparkhill Ray Harryhausen. The early years collection. [DVD]. London, United Kingdom: Sony Pictures.
Thomas, M., Penz, F. (2003). Architectures of illusion. From Motion Pictures to Navigable Interactive
Environmets. Bristol, United Kingdoms: Intellect Books.
46

APPLYING BUILDING INFORMATION MODELING IN EVALUATING
BUILDING ENVELOPE ENERGY PERFORMANCE
Cheng-Yuan Hsieh & I-Chen Wu
Department of Civil Engineering, National Kaohsiung University of Applied Sciences, Taiwan
ABSTRACT: The energy efficiency of new constructions and existing buildings can be determined via energy
performance evaluation. Low energy consumption and reduced carbon dioxide emissions can be achieved if the
energy-saving index is taken into account when a building is being designed. Building envelopes can be considered
the most significant factors among the various energy impact factors. For example, reduced use of heating,
ventilation, and air conditioning (HVAC) and lighting equipment can be achieved by simply considering the
interaction between the building envelope and the surrounding environment, such as ventilation, sunshade, and
ambient lighting. Currently, in Taiwan, this evaluation is conducted manually. This approach is time-consuming
because of the need to constantly consult tables and regulations. It is also susceptible to human error. This
research discusses, in detail, the role of the building envelope in influencing the energy performance of the
building itself, and demonstrates the use of Building Information Modeling (BIM) for visual and automated
evaluation of building energy performance so as to assist planners in determining whether energy-saving
standards have been effectively met. In this research, the feasibility of using BIM for building energy performance
evaluation was examined by applying this approach to an engineering project example. This system showed
effectiveness in assisting planners in identifying and understanding the possible blind spots affecting the
achievement of energy-saving requirements of the designed buildings, and enabled the examination of further
design modifications to optimize energy-saving effects.
KEYWORDS: Building Information Modeling (BIM), Building Energy Performance, Energy Consumption,
Carbon Reduction, Building Envelope
1. INTRODUCTION
In order to cope with projected climate changes brought about by the enhanced greenhouse effect, countries around
the world have invested efforts into the development of new energy resources and energy-saving techniques,
alongside efforts to reduce carbon emissions. Research has found that the operation of buildings accounts for up to
40 percent of global CO2 emissions. Buildings designed with energy-saving considerations have the potential to
achieve fifty to sixty percent energy savings during their entire life cycle. In the past, most architects designed
buildings with little regard for the energy performance of buildings. Nowadays building energy conservation has
become a priority. Generally, performance evaluation of a buildings energy can be divided into five parts: (1) the
building envelope; (2) the air conditioning and ventilation; (3) the water heating system; (4) dynamic equipment
and (5) illumination. Although various factors affect the energy performance of a building, the building envelope
plays the most significant role. For example, higher solar radiation received from the building envelope implies
greater total solar heat gain and hence higher energy requirements for cooling. When designing an air conditioning
system, knowledge of the total radiation striking a surface over a specified period of time is required (ASHRAE,
2009). Various studies have been conducted involving the application of energy analysis of building envelopes.
For example, Chang (2006) employed Lins Method (Lin, 1990) (Yen et al., 2005) (Liao et al., 2006) (Lin et al.,
2007) and the Building Energy Evaluation Program (BEEP-II) to analyze the Envelope Load (ENVLOAD) of
different opening ratios of windows, different directions of buildings and Lightweight Aggregate Concrete
envelopes. Arno Schuster (2008) conducted an energy analysis by considering the thermo economic concept of
exergy. Exergy analysis takes into account the quality of energy and creates a holistic view of the building
incorporating the buildings form, materials and technical systems. Normally, energy analyses involve the use of a
wide range of data and in various modes, including 3D geometry, weather and construction materials. In
response to this problem, this research implemented specific modules based on the BIM system to assist planners
in considering engineering information synthetically when trying to evaluate the energy performance of buildings
accurately. BIM is a computer model database of building design information, which may also contain information
about the buildings construction, management, operation and maintenance (Wu et al. 2012). BIM systems can
access required data from BIM models, and allow complicated numerical data to be displayed effectively in an
easily understandable visual format. In the traditional architectural workflow, energy performance assessment is
mostly done subsequent to the architects design, and design changes greatly reduced efficiency and increased
costs when implemented during construction. This system assists planners in identifying and understanding the
47

possible blind spots when attempting to achieve energy-saving requirements of the designed buildings, and
enables further design modifications to be incorporated to optimize energy-saving effects in the early stages of the
buildings development. All of the application functions were implemented on AECOsim Energy Simulator
which supports visualization of the 3D building model and provides capabilities for energy analysis and
numerical data inputs. This research also applied the technology to a real engineering project example to verify its
feasibility.
2. METHODOLOGY
Fig.1 shows the concept of the building envelope energy performance evaluation adopted in this research. The
methodology can be divided into three parts: (1) Building Information Modeling; (2) Building Envelope Energy
Performance Index; (3) Energy Performance Evaluation. Further discussion is provided in the following sections.
2.1 Building Information Modeling
BIM is a new computer technique that is part of a comprehensive systems approach to the planning, design,
construction, management, operation, maintenance, and use of buildings (Eastman, 2008). In this study, we
employ the concept of BIM in information integration and information visualization for energy performance
evaluation. One of the goals of BIM is to make the cooperation between stakeholders in the building process more
efficient. This is achieved by storing relevant design information of each step in the design process. The ability of
the model to serve as an interdependent, multi-disciplinary data repository enables new approaches on integrating
performance analysis into the design process. Parameters defined during the design process can be accessed and
edited during the design process and utilized for energy performance calculation. Information on building
elements, area, building type, climate, HVAC, materials, date, time and energy standards was stored in the BIM
model we developed for environment analysis, summary reporting, energy analysis and day lighting analysis.
2.2 Building Envelope Energy Performance Index
Following the Taiwan Building Technique Regulation and engineering design requirements, this research
evaluated building energy performance in terms of four aspects of the building envelope energy performance
index for energy performance evaluation: (1) Average Thermal Transmittance; (2) Solar Heat Gain Coefficient;
(3) Reflection Rate of Visible Light and (4) Ratio of Equivalent Transparency.
2.3 Energy Performance Evaluation
We analyzed the calculation results to determine if the project would pass the building energy performance
evaluation. We hoped that this research would help the designer to design better ventilation operations to achieve
reduced energy consumption and carbon dioxide emissions.

Fig.1 Conceptual framework of research
48

3. BUILDING ENERGY PERFORMANCE INDEX
The building envelope is the most important impact factor with regards to energy efficiency. If a building designer
can control the building environment and optimize the building envelope energy saving factor, good energy saving
designs can be achieved, which will lead to lower requirements for air conditioning, ventilation and illumination,
and reduce energy waste and carbon dioxide emissions. The focus of this research was on developing an efficient
and accurate method to assess the building envelope in terms of energy-efficiency and performance. Specifically,
this study aimed to achieve, in our example, five on the building energy performance index, as described in the
literature and building codes.
3.1 Average Thermal Transmittance (Uar and Uaw)
U-Value is the measure of the rate of heat loss through a material. Thus, all aspects of building design should strive
for the lowest U-Values possible because the lower the U-value the less heat that escapes. According to Rules 311
and 312 in Taiwans building technique regulations for roofs, the value Uar must be lower than 1.0 W/(m2/k). For
external walls Rule 310 states that the Uaw value must be lower than 3.5 W/(m2/k). These are illustrated in Fig. 2.
AVERAGE THERMAL
TRANSMITTANCE

UAR

UAR=(UR1*AR1+UR2*AR2)/(
AR1+AR2)
UAR < 1.0 W/(M2/K)

AVERAGE THERMAL
TRANSMITTANCE

UAW

UAW=(AWI+UWI)/AWI
UAW <3.5 W/(M2/K)

Fig.2 Illustration of Average Thermal Transmittance
3.2 Solar Heat Gain Coefficient (HWs)
The Solar Heat Gain Coefficient represents the extent to which solar radiation heat penetrates through the
translucent roof skylights into the interior of the building. According to Rule 308 in Taiwans building technique
regulations, HWs must be lower than the standard HWsc. This is illustrated in Fig. 3.
SOLAR HEAT GAIN
COEFFICIENT

HWS

HWS=((1.0-KHI)*KSI*I*A
GI)/
AGI
HWS<HWSC

Fig.3 Illustration of Solar Heat Gain Coefficient
49

3.3 Reflection Rate of Visible Light (Gri)
The reflection rate of visible light is defined as bouncing back of light from a reflective surface. According to
Rule 308 of Taiwans building technique regulations, Gri must be lower than the standard value of 0.25 (Fig. 4).
REFLECTION RATE OF
VISIBLE LIGHT

GRI

GRI=RVI
GRI <0.25I=1~N

Fig.4 Illustration of Reflection Rate of Visible Light
3.4 Ratio of Equivalent Transparency (Req)
The Ratio of Equivalent Transparency (Req) is a basic index for assessing whether a building is energy-efficient.
Req is the ratio of the area of the total building envelope to the area of the envelopes equivalent transparency. The
standard value will be different in different zones and latitude in Taiwan; for example, southern <18%, middle
<15%, northern <13%.
RATIO OF EQUIVALENT
TRANSPARENCY

REQ

REQ = AEQ / AEN
AEQ = AGI X FK X KI X FVI
+ AGSI X FK X KI X FVI/
AEN
REQ< REQS

Fig.5 Ratio of Equivalent Transparency
4. EXAMPLE
4.1 Background
A dormitory engineering project in the Yanchao campus at the National Kaohsiung University of Applied
Sciences, completed in 2010, was used as a test case. This project was divided into three 7-floor buildings. This
study focused on Building C for conducting the building envelope energy performance analysis and simulation,
and the building envelope energy performance index was applied to assess the standard was achieved.
4.2 Modeling
AECOsim Energy Simulator was used to develop BIM model. Using AECOsim, planners can build 3D models
based on 3D plan generated from CAD system quickly and accurately. Essential parameters for energy
performance evaluation such as the environment, weather, material, area, and date, were set using the Data
Manager and System Manager. Finally, the BIM model was used to execute the building envelope energy
performance analysis and simulation.
4.3 Analysis
As shown in Table 1, the analysis results were: (1) Average Thermal Transmittance Uar:0.99 W/(K), (2)
Average Thermal Transmittance Uaw:3.49 W/(K), (3) Solar Heat Gain Coefficient HWs:0, (4) Reflection
Rate of Visible Light Gri:0.15 and (5) Ratio of Equivalent Transparency Req: 6.8%. For this example, all five
50

building envelope energy performance indices met the required standards.
Table 1: Analysis of results for Building envelope energy performance index
Analysis of results for Building envelope energy performance index
Uar =
( Ar1*Ur1+Ar2*
Ur2) /(Ar1+Ar2)
Ari = 810 m2,
Uri1= 0.99, Uri2
= 3.52
0.99<1.0
Pass
Uaw =
(Awi+Uwi) /
Awi
Awi = 1652
m2, Uri = 3.49
3.49<3.50
Pass
HWs =
((1.0-Khi)*Ksi*i*Agi)
/ Agi
Because no skylight, so
its 0
Pass
Gri = Rvi
Rvi = 0.15
0.15<0.25
Pass
Req = Aeq / Aen
Aeq=(Agi*fk*Ki*fvi+
Agsi x fk x Ki x fvi) /Aen
Aen = 2976 m2
Aeq=200.91
0.068 = 6.8%
Pass

Uar Uaw

HWs Req
51

Gri

5. CONCLUSION
Five building envelope energy performance indices were implemented using a BIM system and AECOsim
Energy Simulator to evaluate the energy performance of a building Building C of the dormitory engineering
project in the Yanchao Campus of National Kaohsiung University of Applied Sciences. This research employed
the concept of BIM to integrate building and environmental parameters, and then to visualize the results of
analysis and simulation for five building envelope energy performance indices, so as to assist planners to clearly
understand the energy efficiency of a target building. Various energy-saving factors and species were considered
comprehensively: living people, water heating systems, lighting, air conditioning; factors such as the building
operation type, quantity, the environment and weather, which would affect the calculation results of building
energy performance indices, were included. In our further work, we will attempt to implement another building
energy performance index which will include considerations of air conditioning and ventilation systems, water
heating systems, dynamic equipment and illumination to assist planners to accurately and comprehensively
evaluate building energy performance.
6. REFERENCES
Arno, S and Frank, T., (2008). "Building Information Model Based Energy/Exergy Performance Assessment in
Early Design Stages", Automation in Construction, Vol. 18 (2), pp. 153-163.
ASHRAE (2009). ASHRAE Handbook of Fundamentals (SI Edition), American Society of Heating, Refrigeration,
and Air-Conditioning Engineers, Atlanta, GA, USA.
Chang, C. L., (2006). "The Effects of the Direction and Windows Open Ratio of Lightweight Aggregate Concrete
Buildings on the Energy Conservation Index and the Estimations of Economical Benefits", Master Thesis,
Department of Applied Mathematics, National Chung Hsing University.
Chang, S., Lee, B. C., Chen, P. Y. and Wu, I. C., (2011). "Visual Req Calculation Tool for Green Building
Evaluation in Taiwan", Proceedings of the 28th International Symposium on Automation and Robotics in
Construction, June 29th - July 2nd, Seoul, Korea.
Eastman, C., Teicholz, P., Sacks, R., and Liston, K., (2008). "BIM Handbook: A Guide to Building Information
Modeling for Owners, Managers, Designers, Engineers and Contractors", Wiley, 504 pages
Liao, H. W., Jane, K. C., Chang, C. L., (2006). "Building Shell to the Use of Lightweight and Normal Weight
Concrete Girders Shell Energy Consumption Analysis and Economic Evaluation", Proceedings of the 3rd
Cross-strait Conference on Production and Application Technology of Lightweight Aggregate Concrete,
November 1st - 4th, Hubei, China.
Lin, H. T., (1990). "An Evaluation Method for The Energy Consumption of Building Air Conditioning Lin's
Simplified Method", Journal of Architecture, Vol. 1, pp. 49-72.
Lin, J. T., (2007). "The Effects of the Direction, the Material of Walls, the Open Ratio of Windows, the Material of
Windows, and the Sunshade on the Energy Conservation Index", Master Thesis, Department of Civil Engineering,
National Chung Hsing University.
Smith, D. K., and Tardif, M., (2009). "Building Information Modeling: A Strategic Implementation Guide for
Architects, Engineers, Constructors, and Real Estate Asset Managers", Wiley, 216 Pages.
Wu, I. C., and Hsieh, S. H., (2012). "A Framework for Facilitating Multi-dimensional Information Integration,
Management and Visualization in Engineering Projects", Automation in Construction, Vol. 23, pp. 71-86.
52

Yen, T., Chen, H. J., Liao, H.W., Huang, C. H. and Chang, C. L., (2005). "The Development and Application of
Economic Evaluation of Lightweight Aggregate Concrete Wall Panels", Architecture and Building Research
Institute, Ministry of the Interior.

53

BIM SUPPORTED DATA VISUALIZATION STRATEGIES FOR
FACILITY MANAGEMENT
Anne Anderson, Ken-Yu Lin & Carrie Sturts Dossick
University of Washington, Seattle, Washington
ABSTRACT: As much interest as there is to leverage BIM for facility management and operation, it is still unclear
how existing practices can be adjusted to realize the potential of BIM. More is still yet to be learned about the
current facility management processes and practices and how the life cycle of facility information informs the
strategic planning of BIM adoption. Based on the findings of a COBie pilot case study conducted at the University
of Washington, this paper provides a detailed review of the information needs and challenges that stem from the
everyday facility management and operation activities at this large organization. Fourteen Maintenance and
Alterations Department personnel were interviewed as a part of the case study to identify: (1) current information
system related issues that cripple timely responses to maintenance and repair requests, and (2) wish list items for an
ideal facility management information system. Lack of access to as-built information ranks as the top issue for
current systems whereas making that information more accessible through a user-friendly system supported by
up-to-date technologies is the most desirable item on the wish list. The paper also discusses a fictitious scenario to
illustrate the identified issues in their real world contexts and explore possible triggers for the lack of as-built
information. The triggers range from poorly integrated information systems to the imperfect fulfillment of
consultant service agreements. The authors argue that facility management requires specific BIM data
visualization strategies that can reflect the highly specialized domains and systems involved in the everyday
maintenance and operation activities. GIS and BIM stand out as the two enablers for a user-friendly information
system. The system could potentially guide the maintenance and operation personnel to obtain the needed asset
information through the visualization of color coded spaces, systems, and system components without
overwhelming users with unnecessary information.
KEYWORDS: Facility management, maintenance and operation, COBie, BIM, information systems, data
visualization
1. INTRODUCTION
Facility owners are challenged by not only the initial capital cost to build the facilities but also the operation and
maintenance costs that last decades. While Building Information Modeling (BIM) is being researched extensively
to bring benefits to the design, planning and construction of these facilities, less is known about its challenges and
enablers for facility management especially when it comes to large organizations. Many universities, for example,
are large organizations that rely on numerous legacy systems to operate and manage hundreds of buildings and
large networks of utilities. As much interest as there is to leverage BIM for facility management and operation, it is
still unclear how existing contractual, technological, procedural and cultural practices can be adjusted to realize the
potential of BIM (Anderson et al, 2012).
Although data schema such as COBie (Construction-Operations Building information exchange) has been
proposed to guide the preparation of facility data for BIM (East, 2007), BIM oriented software in FM is still
nascent. The development BIM for FM software is where BIM for design and construction was five to ten years
ago. We anticipate that there will be many software changes in upcoming years. One of the objectives of this
research is to understand the information needs for maintenance crews so that the industry can develop BIM
systems that support FM work. Much is still yet to be understood about current facility management processes and
practices and how facility information is created, transferred, indexed, retrieved, and applied to inform the
strategic plan of BIM adoption. Based on the findings of a case study, this paper provides a detailed review of the
information needs and challenges that stem from the everyday facility management and operation activities.
The case study involves one large U.S. higher education institution, the University of Washington, which owns and
manages more than 300 buildings across its main Seattle campus. Initial findings of the case study have been
reported in the 2012 Construction Research Congress conference and this paper concentrates on the analyzed
results. Specifically, in this paper, the authors report findings from the case study that are relevant to the
Maintenance and Alterations Department at the University of Washingtons Facilities Services. The authors
explain the background of the case study, present and illustrate major issues that the departments personnel have
in relation to existing IT infrastructure, and proposes corresponding BIM/GIS data visualization strategies based
54

on some of the wish list items identified for the information system in the case study.
2. CASE STUDY BACKGROUND
The case study is a part of a pilot COBie project at the University of Washington (UW) that aims to understand
and improve the general practices of sharing information among the major facility construction, commissioning,
and operation stakeholders. Although for newer projects, much of the project information is available digitally,
some of the University of Washingtons facility operation and maintenance manuals are still in hard copies.
As-built drawings in the form of expensive and bulky Mylar copies are also common, due to the record keeping
requirements set forth by the state. Recently, these requirements were dropped and the state now permits digital
document submissions. The UWs Capital Projects Office is taking this opportunity to review and revise
submittal requirements for new buildings on campus and reviewing the hand over process as a building is
commissioned and turned over to facilities services. In this digital transition, where paper documents are scanned
and stored in digital records, digital project information is overwhelming to sift through and the non-digital
information is difficult to retrieve and not readily available when it is needed by the operation personnel
(Anderson et al, 2012).
The pilot COBie project, Foster School of Business Phase II, was motivated by a 2011 COAA (Construction
Owners Association of America) initiative that engages in a series of case studies across the U.S. to measure and
assess the actual impact of COBie for the capital facilities industry. Because the case study focuses on the
information hand-offs between construction and operation, it involves mainly the Capital Projects Office and the
Facilities Services at the University of Washington, although only the latter is at the center of discussion in this
paper. Capital Projects Office serves as the major project management provider for planning and developing the
University of Washingtons physical facilities, which once built are operated and maintained by Facilities
Services.
While the Facilities Services supports a wide range of assets and programs on the Universitys campus, most
case study participants are from selected trades (e.g. plumbing, electrical, and refrigeration) at its Maintenance
and Alterations Department. The selection of these trades corresponds to the systems that form the core data sets
in COBie. Fourteen Maintenance and Alterations personnel ranging from the department director, zone manager,
to trade specific technician participated in the case study and provided their perceptions through intensive
interviews. The participation list based on the different participation roles is shown in Table 1. Each interview
was administered to cover three topics, including the interviewees (1) issues with the current information system,
(2) reaction to proposed COBie/BIM solutions for future University of Washington capital projects, and (3) wish
list items that make up an ideal information system. All interviews took place in the summer of 2011 and each
lasted 45 to 60 minutes. Interviews were recorded, transcribed, coded and analyzed for emerging themes.
Table 1: Maintenance and Alterations Department Interview List
Participant Role Number of Participants Zones Covered
Director 1 All campus
Zone Supervisor 1 Central and southwest campus
Zone Manager 1 Health Science
Maintenance Coordinator 1 Health Science
Electrical Lead 2 Northeast campus, Health Science
Plumbing Lead 3 All campus
Metal Trades Lead 2 Northeast campus, Health Science
Refrigeration Lead 3 All campus

55

3. ISSUES WITH CURRENT INFORMATION SYSTEM
3.1 Top Five Issues Identified Based On the Interview Data
Issues captured from interviewing the Maintenance and Alterations personnel were prioritized based on their
appearance frequencies in the interview data and their impacts upon the personnels work. If an issue causes a
work activity to stop or creates additional work for the activity, it is considered to have the highest level of
impact. Table 2 lists the top five issues that were identified through the interviews. Other issues not listed are less
significant and are related to interoperability problems between specific systems, the challenge to preserve expert
knowledge, and the inconsistency among different vendor systems when accessing parts and service information.

Table 2: Top 5 issues with current information system (ranked according to impact on work)
No. Issue Example
1 Lack of access to current as-builts is
problematic
Technicians go to field with little or no knowledge of
what is out there.
2 Software is not user-friendly, especially
for non-computer-savvy technicians
The computerized maintenance management system is
not very intuitive and requires many mouse clicks just for
printing.
3 Maintenance personnel do not have
input during design/construction
Maintenance personnel have seen costly repairs that were
due to the incorrect installation originally.
4 Redundancy of effort when using the
computerized maintenance management
system
Many of the assets or attributes were not imported into the
computerized maintenance management system and so
the technicians need to obtain equipment in the field and
enter corresponding information into the system.
5 Facilities Services does not have
up-to-date technology
Technicians do not have iPads or smart phones to access
information in the field and have to print out work orders
on paper.

3.2 Current Maintenance and Operation Work Process
The following discussion intends to provide a glimpse into the current maintenance and operation work process in
order to, through a fictitious scenario, illustrate the listed issues in their real world contexts. At the University of
Washington, anyone affiliated with the University can make a maintenance/repair request through phones or online
using the work routing and control system FSWorks as shown in Figure 1. Most buildings have dedicated
coordinators that the building occupants can contact when having problems and these coordinators are the heaviest
users of this system. On average, there are more than 100 maintenance/repair requests daily when the school is in
regular session. After a request is submitted, Facilities Services staff will create a work order in the computerized
maintenance management system, AiM, to track the work handling process and to route the work order to the
appropriate shops or crews for attention.
A crew lead who receives the work order then plans the work schedule and assigns it to a crew member. Ideally, the
crew member should be able to obtain information from AiM for the building and asset involved in the request. In
the case that the crew member does not see any asset information (e.g. make, model, serial number) in AiM for the
work order assigned, the member would check to see if the shop or the Campus Engineering Records web portal
has any asset related operation and maintenance manuals or plans, or visit the site to collect necessary asset
information (Marsters, 2011). Seeking asset information as such is crucial to support the maintenance/report work
but it can also be a very time consuming task. This is especially true for the plumbers who try to figure out valve
locations or valve isolation conditions when the plans do not have this information.
Taking the maintenance/repair request in Figure 1 as an example, the specific request is about the heating problem
at PNCCRE (Pacific Northwest Center for Construction Research and Education), a lab managed by the
56

Construction Management Department that is 3 miles (4.8 km) northeast of the University of Washingtons Seattle
campus. The lab is known to its most occupants as PNCCRE, a name that is not recognized by the FSWorks system
or Facilities Services personnel. Therefore, when the crew member assigned for the maintenance/repair work
receives the request, the member could not locate asset related information in AiM or the shop. The crew member
alternatively looks up the lab online and learns that the lab is located within the Magnuson Park at Sand Point. The
crew member then goes on to the web portal managed by the Campus Engineering Records Office and searches for
archived drawings for PNCCRE using synonymous building names including PNCCRE, Magnuson, and
Sand Point. Finally, the crew member retrieves a list of buildings and documents archived for these buildings,
with one of the buildings very likely being the PNCCRE lab, as shown in Figure 2.
Figure 1. An example maintenance/repair request made in FSWorks

The crew member confirms the building name with a Facilities Services colleague and also learns that there are
more than 1000 archived documents, with some dated back to 1938, for the lab. Unfortunately, for the most part,
the documents do not seem to be listed following a logical order. However, the web portal does export the
document search results into an Excel file for easier browsing and data filtering, despite the facts that not all
technicians are computer savvy and probably will not be able to take advantage of all what the spreadsheet could
offer. Being a creative thinker, the crew member does some research on the labs history and notices that the lab
was last renovated in 2008.
With this additional information, at about 1/5 from the bottom of the list, the crew member is able to narrow down
a number of mechanical drawings that seem to apply. The drawings are labeled as the construction set for
PNCCREs Phase 2 project, with a time stamp 03.28.2008. One of the sheets for the level 2 HVAC plan is
illustrated in Figure 3. However, the Campus Engineering Records Office web portal does not have any
maintenance and operation manuals online, making it necessary for the crew member to plan a site visit or go to the
Campus Engineering Records Office to look for the paper copies. According to the Campus Engineering Records
Offices website, as of September 2011, 98.93% of total active drawings and 39.58% of total
specifications/manuals held by the office are available online.
57

After evaluating the two viable options, the crew member determines to do a site investigation since some of the
mechanical drawings are available and can serve as a starting point. The crew member is experienced and has some
ideas in mind what could go wrong and stop the heater from functioning properly. It is very likely that somewhere
at the lab the responsible zone-level airflow control VAV box is not working as it is supposed to. Because there are
no mobile devices to access the drawings on site, the crew member prints out the needed mechanical drawings on
papers and brings the paper copies to the lab for investigation and troubleshooting. Based on the drawings, the
VAV box responsible for the heating problem is on the hallway, outside the reception and kitchen area with easy
access.

Figure 2. An example maintenance/repair request made in FSWorks

Once in the lab, the crew member soon learns that the VAV box in question is at a different location that requires
access to one of the lab offices. Most of the ductwork at the PNCCRE lab is exposed and it is easy to spot this
change. However, some of the campus buildings have hidden mechanical systems and it is usually quite a
challenge to determine the valve locations or which valve controls what areas if no correct as-built drawings are
available. The crew member finally is able to obtain the asset information from the correct VAV box during the site
visit. Since Facilities Services personnel are not equipped with mobile computing devices to access asset
information as needed, the crew member has to return to the shop to search for the maintenance and operation
manual from Johnson Controls for the identified VAV box. The crew member also remembers to enter the asset
information in the computerized maintenance management system AiM for future references. According to the
troubleshooting guide in the manual, there are seven conditions that could stop a heaters operation because of a
malfunctioning VAV box. Being aware of all the possible situations, the crew member gathers tools and parts that
might be helpful and heads back to the lab, hoping to nail down a solution. In the best case, the crew member will
be able to repair the VAV box and close the work request. In the worst case, it is not the VAV box and that the
investigation has to start a new thread.
58

3.3 Why Lack of Access to As-Built Information
In the scenario discussed, the drawings retrieved from the Campus Engineering Records Office are not the as-built
drawings and so the expectation is that the final locations of the mechanical systems could be different. Apparently,
should the as-built drawings be more available and accessible to the Facilities Services personnel, similar
situations would have been avoided. In addition, in the discussed scenario, it is possible that the needed as-built
drawings are still yet to be scanned at the Campus Engineering Records Office and so they are not available on the
web portal. It is also possible that the design consultant never handed off the correct drawings to the Campus
Engineering Records Office. At the University of Washington, the Campus Engineering Records Office is not
responsible for evaluating the completeness, compliance, and acceptability of submitted consultant CAD drawings.
The actual as-built drawing specifies the relocation of the VAV box in question. Many hand written or drawn notes
also appear throughout the as-built drawings and could be very useful for the long-term maintenance and operation
of the facility.
As illustrated, lack of access to current as-built information is problematic and has many different triggers that
range from poorly integrated information systems to the imperfect fulfillment of consultant service agreements. At
the University of Washington, facility information is not at one place. Rather, the information if available is
distributed in different legacy systems that are often also not integrated. This makes it much more laborious for the
maintenance and operation personnel to search for bits and pieces of facility or asset information. Some systems
are great for their specialized purposes but are unnecessarily cumbersome for other purposes. Taking the
computerized maintenance and management system AiM as an example, the system provides strong supports to
process and cost tracking (which is very useful for reporting and monitoring) but is overly complicated for simple
tasks such as printing assigned maintenance requests by the shop technicians. Even if the entire IT infrastructure
could be well designed and integrated, a mechanism to validate the as-built information is still lacking.
Other aspects that could support the facilities maintenance and operation activities as a whole include: (1)
providing intelligent maintenance request guidance to the occupants in order to obtain useful factual facility asset
information from the occupants, (2) supporting the update of construction drawings so that hand written notes or
sketches can be recognizable in the as-built documents, (3) identifying trade specific knowledge by looking into
how a particular type of maintenance request has been handled and using that knowledge to speed up the response
time for similar maintenance/repair requests, and (4) structuring facility asset information in a way that it can
correspond to the information need of a particular trade or for a specific system without the inclusion of unrelated
detail.
4. ENVISIONING FACILITY MANAGEMENT DATA VISUALIZATION
STRATEGIES
When a building is handed over to Facilities Services, information from the design and construction phases is
disaggregated and distributed across the Universitys IT infrastructure (Anderson et al. 2012). The authors posit
that distribution of information is necessary because each jurisdiction, being experts with respect to their particular
information sets, is most qualified to maintain that information. For example, once new backflow prevention
assemblies come online with a new building, the plumbing shop incorporates them into a database system called
Tokay used to track backflow testing and provide testing reports to the City of Seattle. The distribution of
information across the IT infrastructure, however, creates the issue of having to search in various locations for bits
and pieces of information as described earlier. While the Universitys Records Department is regarded as the
central repository for facility information, the information is sometimes not there (especially if stored and
maintained by individual shops), or if it is there, may be difficult to retrieve due to the number of documents
associated with each building. The fictitious scenario described in Section 3 illustrates how naming conventions
can be an obstacle when searching for information in the Records repository since the scenarios building location
is variously referred to as PNCCRE, Sand Point and Magnuson Park. A future intelligent maintenance request may
include a geographic location pinpointed on the GIS interface by the occupant requesting work that would provide
a visual rather than textual building location for the maintenance technician. Once the technician navigates to the
building in the GIS system, clicking on assets within the building may provide links to asset information
maintained by the various shops.
Interviewing staff from Maintenance and Alterations Department not only helped identify pressing issues as they
relate to the current information systems but also aided the gathering of wish list items that helps us imagine an
ideal information system. Similar to how the issues are prioritized in Table 2, the top five wish list items (excluding
the ones that are more related to organizational issues) are ordered from the most to least frequent occurrences in
59

the data in Table 3.
Table 3: Potential Technologies for Ideal Information Systems that Support Maintenance and Operation
No. Wish List Potentially Supporting Technologies
1 Make information more easily accessible COBie, BIM, mobile devices
2 User-friendly system Improve user interface, COBie, GIS, BIM, mobile
devices
3 Up-to-date technology COBie, BIM, equipment tagging, mobile devices
4 Associate new equipment with supporting
systems in the building model
BIM
5 Visualization of systems BIM, GIS

Based on the case study findings and an in-depth understanding of the existing working process at the case study
organization University of Washington, the authors argue that facility management requires specific BIM data
visualization strategies that can reflect the highly specialized domains and systems involved in the everyday
maintenance and operation activities. As indicated in Table 3, among the potential technologies that could be used
to support the information system for facility management, GIS and BIM stand out as the enablers for a
user-friendly system that orients its users through the visualization of systems. The authors envision that a 3D GIS
system will serve as the main interface for facilities management personnel to locate the building space that needs
attention. The GIS mapping data will be linked to the COBie-based asset data in BIM and a maintenance
technician will be able to see all systems or equipment that are related to the use and operation of the located space.
For trouble shooting and information gathering, the technician will be able to select the system or equipment in
question and then retrieves asset information through a 2D, color coded view of the system or equipment
components. Figure 3 summarizes the envisioned data visualization strategies.

Figure 3. Data visualization strategy for facility management
(Images are modified from University of Washington Capital Project Office, 2011)
60

Figure 4 shows a COBie-based BIM in which a VAV box is selected on the basement floor plan of the case study
building. Figure 5 shows a fire alarm device selected in the 3D systems view of the buildings third level. When a
technician clicks on an asset in the BIM, the asset information appears in the Properties window as shown on the
left-hand side of the figures. Asset information includes manufacturer, model, supplier contact info, and Omniclass
number. In this example, the systems are grouped and color-coded using the American Public Works Association
(APWA) Uniform Color Code that Facilities Services crews are already familiar with, e.g. yellow represents
gas/oil and blue represents potable water.
Figure 4. COBie-based asset data in BIM plan view
Figure 5. COBie-based asset data in BIM 3D view
61

The need to visualize an entire system as built can be critical, for example when a system must be shut down for
repair or remodel. Without correct as-builts, a system in another part of the building may unknowingly tap into the
first system. At a University where research labs supporting various experiments are located campus-wide, an
accidental shutdown can be devastating. According to the interview data, Facilities Services personnel are
accustomed to working with 2D drawings on paper, such as shop drawings or as-builts, and would prefer to see
systems information in a 2D format. It follows that the 2D floor plans that comprise the GIS system would be an
intuitive interface for Facilities Services. In addition, with the access and navigation being similar to Google maps
an application that many are familiar with the GIS interface will probably be easy to learn. One interviewee
stated, What I would like to see is how Google does their maps. You can get closer and narrow in, so take a
building and make a map [showing] electrical, mechanical, carpentry, etc. zoom in to a light fixture or outlet.
Touch it and information would be right there in front of you.
5. DISCUSSION AND CONCLUSION
One of our main findings is that mapping, not modeling, supports the visualization information needs of facilities
maintenance crews. They need to 1) know what assets are in a room, 2) where to find those assets (e.g. which
ceiling tile to pop), and 3) how are the assets in a room connected to the building systems (e.g, which air handler
feeds the room, which electrical panel?). For maintenance tasks, BIMs often provide too much information, which
is visually difficult to understand in 3D. Maintenance crews appear to need more schematic visualizations of how
the systems are connected, and where each asset resides in a building.
Additionally, the make, model and serial number of the equipment is only the first step in the troubleshooting
process. Although this is critical information, it is only useful in that it leads the crew to operations manuals and
other asset documentation that contains parts lists and performance specifications that help them understand and
troubleshoot what might be wrong with the system in question. Currently, visual information, such as drawings and
records are stored separately from manuals, which are stored separately from the work order files and asset data.
In thinking through system integration architecture, one of the drawbacks of the GIS/BIM system is that it requires
set up and maintenance to ensure links between GIS and associated data remain intact when information is updated
or deleted when information becomes outdated. One of the concerns expressed in the Facility Services personnel
interviews was that new technology might burden them with additional administrative tasks, which translates to
less time on task in the field (the opposite of what the goals of these information systems should be). Consequently,
a plan for how the data are accessed and maintained is a critical part of the development of integrated systems for
Facilities Services. A University of Washington GIS Specialist demonstrated in an interview the ease with which a
photo can be taken using a mobile device and immediately attached to a location on the Universitys GIS system.
This feature could be very useful for technicians in the field as a quick and easy way to document an as-built
condition, but the possibility of information overload (too many photos) and the need to maintain this information
would need to be addressed. Years after the photo is taken, it may no longer be applicable.
In conclusion, based on the findings of a COBie pilot case study conducted at the University of Washington, in this
paper the authors discuss the information needs and working processes that drive the everyday facility
management and operation activities. Among the top five issues identified, lack of access to current as-built
information remains the biggest concern and can be triggered by a range of aspects, from poorly integrated
information systems to the imperfect fulfillment of consultant service agreements. While data schema such as
COBie might address information exchange and reduce the need to reenter information that was already provided
in earlier phases of the project, GIS and BIM really stand out as the two enablers for a user-friendly information
system that could potentially guide the maintenance and operation personnel to obtain the needed asset
information through the visualization of color coded spaces, systems, and system components without
overwhelmingly unnecessary information, sifting through record as-build documents, and help connect work
orders to assets. As the GIS and BIM visualization strategies for COBie data are introduced to the Facilities
Services during the next phase of the Foster School of Business case study, researchers will be tracking use of the
data for preventive maintenance and repair activities in the building. Follow-up interviews are planned to
determine if the systems were used as anticipated and develop new interface designs that explore the potential for
BIM and GIS identified in this paper.
62

6. ACKNOWLEDGEMENTS
This research is funded by the UW Capital Projects Office and UW Facilities Services as part of the COAA COBie
pilot project initiative and broader UW lean initiative. We want to acknowledge the support of and our deep
appreciation for UW Capital Projects staff as well as UW employees and Foster School of Business Project
personnel interviewed for this research.
7. REFERENCES
Anderson, A., Marsters, A., Dossick, C., and Neff, G. Construction to Operations Exchange: Challenges of
Implementing COBie and BIM in a Large Owner Organization. Published electronically, Proceedings of
Construction Research Congress 2012, West Lafayette, IN, USA. May 21 23, 2012.
East, E. W. (2007). Construction Operations Building Information Exchange (COBIE): Requirements Definition
and Pilot Implementation Standard. Ft. Belvoir, Defense Technical Information Center.
Marsters, A. Obstacles of Implementing 6D Information Exchange: a Case Study of COBie at the University of
Washington, M. Thesis, University of Washington, 2011.
University of Washington Capital Project Office. CAD/BIM/GIS Floor Plan System, PowerPoint presentation,
November 3, 2011, University of Washington, Seattle, WA, USA.
63

WORK PLANNING BASED ON A BUILDING INFORMATION MODEL
Sebastian Hollermann & Hans-Joachim Bargstdt
Bauhaus-Universitt Weimar, Germany
ABSTRACT: In this paper, the author show how to improve scheduling based on a building information model.
Building information models (BIM) are not yet common in construction and as a result, have not been fully taken
advantage of. Delayed completion and increasing costs during execution indicates that a more detailed planning
and risk analyse for construction projects is needed. Therefore, as much as possible information should be used
for decision making. Easer accessible and more information can be stored in a building information model.
Through formalising routine activities computer can support processes. These intelligent tools lead in shorter
time to a more detailed planning. The one of a kind character and increasing complexity of construction projects
requires focusing on problems and not on routine activities. Therefore, digital object-oriented product models
such as a BIM should be used to generate schedules. Rule based or case based algorithm could be used for
generating the (directed acyclic graph) structure of a schedule. Through simulation experiments the project
duration and capacity utilisation of resources can be calculated. In this paper, the author demonstrates how
rules are created and used to generate the structure of schedules. The object representation of the
object-oriented product model as input is the decisive factor for the algorithm to generate the project schedule.
Within this work algorithms for generating a schedule out of a product model like a building information model
have been developed. This is an on-going research. Therefore, this paper presents the overall concepts, research
objectives and the methodology. First analyses of prospects for improvements of scheduling are shown.
KEYWORDS: Building Information Model, Construction management, Production scheduling
1. INTRODUCTION
1.1 Problem Statement and Motivation
Construction projects are unique. This one-of-a-kind character requires that a new construction model and a new
schedule needs to be developed at the beginning of every project. However, they are always made up of several
types of building elements and processes. A project has several models. A construction flow model is developed
out of the construction model and construction process pattern. A common way to display this schedule is a bar
chart. A schedule is not linked to the underlying information. Therefore, changed information will not change the
schedule. Furthermore, the same information is often stored several times in the models. These cause
inconsistent information in a project.
The size and complexity of construction projects increase. Therefore, planning and execution of these projects
become more challenging. Furthermore, for such complex projects, the availability of an experienced site
manager is limited. Nevertheless, quite a number of alternative approaches to build a project are thinkable. The
base of choosing a schedule is an analysis of different scenarios. This means that many probable scenarios
should be analysed. In this case, a fast generation of scenarios would save time and money.
Today in construction management, a lot of time is spent on repeating work steps. Detailed scheduling requires
the consideration of all building elements and their construction methods. Linking each building element to a
process and relating resources to activities requires repetition and therefore a lot of time. Furthermore, repetitive
activities are often prone to error. Routine activities are time consuming. Through that, in the end, there is not
enough time to focus on the core of engineering problems.
1.2 Research Objectives
The primary objectives of this research are:
Formalizing of frequently repeated activities in scheduling so that the focus lies on scheduling particular
and project specific problems.
Automatically generating one simulation model for scheduling as a basis for simulation experiments,
thus allowing for different alternative scenarios and their bottle necks to be analysed easy.
Traceable decision making in scheduling, meaning that dependencies between activities in the
construction flow should be built into the schedule, especially for automatically or semi automatically
generated schedules.
64

1.3 Scope of Research
This research project is focused on increasing the reliability of schedules and estimated costs through a faster
analysis of several alternatives which are based on a building information model. Therefore, routine activities
should be formalized. Through that the focus could be on the core of engineering problems. The object-oriented
concept of modelling is the base. This allows the parametric modelling of objects and input of additional
information.
1.4 Methodology
The strategy used in this research is based on formalizing routine processes of scheduling. The plan of this
strategy can be divided into four phases:
Phase 1: Literature review
Exploring available concepts of construction model based scheduling
Phase 2: Process Analysis
Investigating processes of work flow planning in construction
Characterisation and evaluation of process dependencies
Phase 3: Framework Development
In this phase, based on the knowledge obtained in Phase 1 and the identified dependencies in Phases 2,
a concept for a framework is developed. This concept gets implemented in a preproduction model.
Furthermore in this phase concepts of making decision traceable get develop.
Phase 4: Validation
The model is validated through the use of a case study.
Only the first two phases are covered in this paper.
2. STATE OF THE ART
2.1 Building information model
A building information model is an object-oriented, three dimensional product model. The geometrical shape, the
position and orientation is described by parameters. A building information model can include other information
besides geometric information. Material could be such an attribute of an object. Requirements for such an
integrated product model are already described in (Abeln, 1995).
The construction flow can be shown as a 4D visualisation. For a 4D visualisation, a three dimensional
object-oriented product model has to be created and linked to activities of schedule.
The Industry Foundation Classes (IFC), developed by buildingSMART (International Alliance for
Interoperability, IAI), is a neutral and open specification for building information models. It is registered as the
international standard ISO 16739. The base class IfcProduct relates to a geometric or spatial context of objects.
The IFC base class for processes, IfcProcess, is subdivided into tasks, events and procedures.
2.2 Process model
The Mefisto research project proposed using BPMN models as a formal description for construction processes. A
process-based simulation library can be established. Process templates can be used to break the project schedule
down into one or more levels (Scherer und Ismail, 2011). The proposal requires a Build Information Model
manual linked with a project schedule and information about available resources as input.
Within the research project of Mefisto Benevolenskiy et al. discussed the ontology-based model and the use of
process patterns and rules in the configuration of construction processes (Benevolenskiy et al., 2011). Therefore,
they develop a process configurator. It provides the defined ontology specifications, an initial knowledge base of
process patterns, basic process rules and a set of construction process rules for the subdomain structural
concrete works.
65

Huhnt proposes for a more detailed construction flow model process pattern (Huhnt und Richter, 2010).
Therefore, construction process must be modelled with their activities and events. Process patterns were linked
to building elements for a 4D visualisation. A 4D visualisation of a schedule helps to check the workflow. A
methodology to break down a building into components and to assign processes is shown in (Huhnt et al., 2010).
Thereby different colours help to understand a 4D visualisation.
Within the research project, ForBAU Wu recognized that, in process pattern, formalized construction processes
represent the knowledge of companies (Wu et al., 2010b, Wu et al., 2010a). Furthermore, she proposes a general
activity package scheme which encapsulates an activity and all information associated with it.
The advantage of process pattern is that technological constraints of repetitive processes are always considered.
Process pattern can be recycled in several projects. A process pattern based scheduling is with the same effort
more detailed compared to traditional scheduling.
2.3 Linking Product and Process Data
Model based construction sequence planning is introduced by (Tulke und Hanff, 2007). The approach aims to
speed up the preparation of 4D visualisation by using 3D models. Out of a 3D CAD and a room book they
generated a project data base. An estimator creates the bill of quantities based on the product model. The link
between estimating and scheduling is the bill of quantities. Subsets of the bill of quantities are associated with
activities in the time schedule.
CISmo (CAD-integrated simulation modelling for the simulation of construction processes) is a modelling
environment for the creation of agent-based simulation models for the simulation of construction processes
(Kugler et al., 2011). It is integrated into the CAD environment, AutoCAD Architecture. It uses only
3-dimonsional product models and a process data base. CISmo focuses on the link between the process and the
product data.
In order to prepare the necessary input data for simulation, the research project Forbau (Gnthner und Borrmann,
2011, page 182) developed the Preparator application. This allows for product model components to be
interactively assigned to their corresponding building elements (Wu et al., 2010b; Wu et al., 2010a).
The research project, Mefisto (Management Fhrung Information Simulation im Bauwesen), has developed
the SiteSim-Editor for preparing input data for simulation. It can be used for logistics (Habenicht et al., 2011)
and for processes (Knig und Marx, 2011). Objects can be linked interactively. Storage areas and materials for
logistics, as well as process pattern to building elements, can be linked.
Diverging from previous studies (Wu et al., 2010b; Wu et al., 2010a; Knig und Marx, 2011), information out of
building models has to be prepared manually for simulation. Additional information of a building information
model is not used. Only geometric information is used. A concept for an automated generation of Construction
Schedules based on Case-Based Reasoning is shown by Tauscher and Mikulakova (Tauscher, 2011; Tauscher
et al., 2009; Mikulakova, 2011).
2.4 Process simulation in construction planning
The role of Simulation in Construction Management is analysed by AbouRizk et al. (AbouRizk, 2010. ). They
propose special purpose simulation systems rather than universal tools. His research program explores the use of
HLA-based integrated large-scale models in construction. He proposes distributed simulation for complex
large-scale construction models, real-time input and integrated simulation.
Forbau is using a constraint-based simulation. For each configuration, a Monte-Carlo analysis consisting of 1000
simulation runs is performed. Each run results in a different schedule (Wu et al., 2010a).
Beiert has implemented a constraint-based simulation concept for construction planning in a discrete event
simulation (DES) framework (Beiert, 2012). The concept should allow for the consideration of experienced
planners knowledge, especially when it comes to practical execution strategies.
Manually linked objects are the basis for creating a schedule through the use of simulation. Schedules are the
results of simulations. This is not necessarily the optimal solution of the schedule. The results only include the
start and finish of activities but not dependencies on constraints and the links to predecessors. Therefore, it is not
66

possible to check such a schedule.
To understand a schedule and the processes, it would be necessary to understand the dependencies. As a result, it
is not possible to check the schedule. Furthermore, it is not possible to make adjustments without running the
whole simulation. Especially because of the fact that experienced site managers are more effective than any
simulation tool can be. Process simulation in construction should be a tool to support site management.
Therefore, it is necessary to understand the results or, rather, the dependencies. Another problem is the use of the
construction model and other associated information as a basis for construction flow simulation. The information
out of building models has to be prepared manually for simulation. Additional information from a building
information model is not used, yet. Only geometric information is used.
3. BUILDING INFORMATION MODEL BASED SCHEDULING
A lot of information is needed for scheduling in a construction process. Information about the construction
model entails the geometry of building elements and their positions. This gives an initial structure for the
construction flow of a construction process. The information about the material of a building element is
necessary when choosing a construction method. Based on this Information about the structure and material
properties of the construction, a first solution for a construction flow model can be generated.

Figure 1: link between activity and building element in a 4D-Visualisation
The generation of solutions of a construction flow model has been created manually so far. For a 4D
Visualization, the building information model gets linked with the construction flow model. Figure 1 shows a 4D
Visualization for a bridge. The building element pier is linked with an activity. The dynamic activity is
highlighted in yellow. Different processes can have different colour-codes.
For an automatically generation of a schedule the dependencies between activities has to be analysed. The three
categories of dependencies are discretionary, resources and logical. Typically, bridge constructions are linear and
have a lot of dependencies between activities because of their complexity. Therefore, I have analysed the
schedules of bridge constructions and have characterised the dependencies between activities in the schedules. I
categorised the dependencies into geometry, processes, logistics, resources, and approval dependencies. Figure 2
shows the results. Most dependencies between activities are because of processes or geometry.
schedule
Building information model
67

Figure 2: Dependencies of activities in a bridge construction scheduling
Figure 3 shows the identified workflow of planning a construction flow. The first step is to create the building
information model. The structural analyse of the construction gives the first dependencies for the construction
flow. The next step in construction planning is to choose a construction method. The decision of which
construction method should be used can be based on rules or by an experienced civil engineer. For example, the
volume of a building element indicates which construction method is most efficient. The chosen construction
method defines which process pattern are needed in the schedule. The process pattern describes the process
workflow, needed resources for each activity and the algorithm for estimating the construction method.

Figure 3: Workflow of scheduling
Work sections can be created by splitting or combining building elements. Splitting up building elements creates
new building elements and changes the structures of the building. Therefore, the construction model has to be
adjusted and a new structural analysis is required. Combining building elements or single processes of
construction methods for producing building elements is a common approach of creating work sections.
Combining building elements into one activity is often needed to execute this activity without interruption.
Figure 4 shows the identified Association between building elements and the construction method. Each building
element can be built with one or more construction method, and each construction method has one or more
activity.
Creating construction flow model Simulation
68

Figure 4: link between activity and building element
The association between building elements and construction can be stored in a database. If a building element
can be built with more than one construction method, rules for choosing a construction method can be defined. A
construction method requires one or more activity. The workflow sequence of a construction method could be
linear or parallel. This structure belongs to the construction method and is modelled in the process pattern. The
analysis of bridge construction schedules shows that most processes are linear. One example of this is the
process of producing building elements out of concrete, as shown in Figure 5.

Figure 5: constraints for the schedule out of work sections
Some activities in construction processes cannot be interrupted. Pouring concrete and launching a scaffold are
examples for such activities. To save fixed costs for establishing a concrete pump three times or having complete
truckloads of concrete, activities can be combined into work sections. The red border line in Figure 5 shows
three processes of concreting which are combined. For saving fix costs, the three processes of pouring as a whole
should not be interrupted. Therefore, the dependencies between these activities are a finish to start relationship
with a direct start of the next activity. In fact, these combined activities form one new non-interruptible activity.
4. PROCESS ANALYSES
4.1 Environment for construction scheduling
The workflow of scheduling in construction is
dynamic. The amount of parties involved and
the possible organisation structures in the
planning process indicate that. Therefore, a
dynamic environment is required. With a
toolbox of different components, such a
dynamic environment could be modelled. A
common form of communication is required so
that these components can work together.
Figure 6 shows the developed
distributed environment for construction
scheduling. It is implemented in Tecnomatix
Plant Simulation from Siemens. PM, FM and
CM are independent working components
called federates. FOM is the backbone of the
network. Because the FOM is coordinating
federates, the network is called federation.

Figure 6: distributed environment for construction scheduling
69

Federates are only communicating through the FOM. Therefore, federates can be replaced by others with a
different logic. Furthermore, different parties can develop and work on different federates. Beside that, it is also
possible that these federates are running on different computers in different place. The FOM is not storing
information. It is only coordinating federates in the federation. Therefore it is a distributed model.
The duties of federates in Figure 6 are corresponding to the stages in Figure 3. The federate PM in Figure
6 is dealing with the product model. Therefore, it is equivalent to the first two stages of Figure 3. The federate
CM is correspondent to stages three and four in Figure 3. The construction flow simulation is implemented in the
federate FM. Apart from that, each stage of Figure 3 could be implemented in one federate. Furthermore,
federates for weather simulation or breakdowns of machinery can be added.
4.2 Dependencies of activities
The identified dependencies between activities can be characterised into three groups as shown in Figure 7. The
first group are discretionary dependencies. These dependencies are according to the judgement of a person in
authority on what is necessary in each particular situation. It is not possible to describe these dependencies with
general rules. In computer assisted processes, therefore, user interfaces are required. The second group are the
dependencies of resources. In a process pattern, the dependencies between resources can be modelled. Rules can
define the required resources for an activity. The availability of resources determines the duration of the
construction process. The construction flow structure is not directly influenced. The third group are logical
dependencies. The analysis shows that most of the dependencies in construction are logical ones. These
dependencies can be described with rules. The logical dependencies come from the bearing structure of a
construction and from technological constraints of construction processes.

Figure 7: groups of dependencies between activities in construction scheduling
In time schedules of construction project, all dependencies cause relationships between activities. In other words,
the dependencies are in a schedule implemented as successors or predecessors of activities. Thus the structure of
the construction flow model gets created.
The quality of the construction flow model is responsible for the quality of the schedule. Rules for frequently
used dependencies in construction models facilitate construction scheduling. Project specific dependencies
always have to be modelled by a person in authority. Therefore, rules for creating relationships automatically
help the user to focus on specific problems.
4.3 Rules for creating relationships
As shown in Figure 2 most dependencies between activities are because of processes or geometry. Thru
analysing these dependencies I identified that most of them are caused by the same reason. Therefore the focus
in this paper is on rules which describe the geometric dependencies between objects. In Table 1 are shown
different strategies for creating geometric relationships between building elements and examples are given.
Table 1: Strategies for creating geometric relationships between building elements
Strategies Example
Default as stored in the Product Modell-file
Direction bottom-up, top down, from left to right, from right to left
Attributes by Volume, by length, by type
Random without any regular pattern
Discretionary decided according to the judgement of a person in authority, user interface
70

The Strategies Direction is depending on the geometry of the building elements. For a fast modelling of building
elements often bounding boxes as defined in ISO 10303-42 (1992) are used to describe complex objects. In
Construction the bounding box of a building element is often equal to the geometry of the building element. In
this case geometric rules can use the coordinates of the bounding box. For building elements where the bounding
box is not equal to the bounding box other methods to analysing the geometry has to be used. With a Solid
Model representation better results can be generated.

Figure 8: Bounding box representation of building elements
Rules for creating geometric relationships between building elements has to define the direction and reference
points of the building elements such as a corner or the centre of gravity. Figure 8 shows two bounding boxes of
building elements. If a rule defines the direction as x positive and the centre of gravity as reference point the
order is first building element 1 and then building element 2. For better results the topological structure must be
availeble. Therefore a solid model is needed.
5. CONCLUSION AND OUTLOOK
The paper presents a work in progress. The approach is to improve a building information model based
scheduling. This paper presents the concepts, research objects and the methodology. The workflow for
construction scheduling is described and analysed. Dependencies between activities in schedules are
characterised into three groups. The tasks of building information model based scheduling are identified.
Requirements for building information models are defined. The next steps in this research project are to improve
the prototypical development framework. Furthermore, traceable decision-making will be implemented. Finally,
the developed framework has to be validated. Therefore, several investigations of case studies are necessary.
71

6. REFERENCES
Abeln, O. (1995). CAD-Referenzmodell. Stuttgart, Teubner.
AbouRizk, S. (2010). Role of Simulation in Construction Engineering and Management. Journal of Construction
Engineering and Management, Vol. 136 No. 10, 1140-1153.
Beiert, U.. Constraint-basierte Simulation zur Terminplanung von Ausfhrungsprozessen Weimar, Weimar, Verl.
der Bauhaus-Universitt
Benevolenskiy, A.; Katranuschkov, P.; Scherer, R. (2011). Ontology-Based Configuration of Construction
Processes Using Process. EG-ICE Workshop 2011, Twente,
Gnthner, W. A. and Borrmann, A. (2011). Digitale Baustelle-innovativer Planen, effizienter Ausfhren. Berlin,
Springer.
Habenicht, I.; Spieckermann, S.; Knig, M. (2011). Simulation der Baustellenlogistik am Beispiel eines
Flugsteigs. 2. Mefisto-Kongress, Dresden, Inst. fr Bauinformatik, 133144.
Huhnt, W.; Richter, S. (2010). Eine Modellierungsmethodik fr Ausfhrungsterminplne im Praxistest. 2533.
Huhnt, W.; Richter, S.; Wallner, S.; Habashi, T.; Krmer, T. (2010)Data management for animation of
construction processes. Advanced Engineering Informatics, Vol. 24 No. 4, 404416.
Knig, M.; Marx, A. (2011). Aufbereitung von Multimodellen fr Simulationsstudien zur Ausfhrungsplanung. 2.
Mefisto-Kongress, Dresden, Inst. fr Bauinformatik, 93106.
Kugler, M.; Kordi, B.; Franz, V.; Samkari, K. (2011). Linking product and process data in the modeling
environment 'CISMO'. CONVR 2011, Weimar, Bauhaus-Universitt Weimar, 171181.
Mikulakova, E. (2011). Wissensbasierte Bauablaufplanung mit Fallbasiertem Schlieen. Weimar,
Bauhaus-Universitt Weimar.
Scherer, R.; Ismail, A. (2011). Process-based simulation Library for construction project planning. Winter
Simulation Conference, Phoenix, IEEE Service Center,
Tauscher, E.; Mikulakova, E.; Beucke, K.; Knig, M. (2009). Automated Generation of Construction Schedules
Based on the IFC Object Model. Austin, Texas, ASCE; American Society of Civil Engineers, 66.
Tauscher, E. (2011). Vom Bauwerksinformationsmodell zur Terminplanung Weimar, Verlag der
Bauhaus-Universitt.
Tulke, J. (2010). Kollaborative Terminplanung auf Basis von Bauwerksinformationsmodellen. Weimar, Verlag
der Bauhaus-Universitt.
Tulke, J.; Hanff, J. (2007). 4D construction sequence planing - new process and data model. CIB-W78 24th
International Conference on Information Technology in Construction, Maribor, Faculty of Civil Engineering,
7984.
Wu, I.-C.; Borrmann, A.; Beiert, U.; Knig, M.; Rank, E. (2010a)Bridge construction schedule generation with
pattern-based construction methods and constraint-based simulation. Advanced Engineering Informatics, Vol. 24
No. 4, 379388.
Wu, I.-C.; Borrmann, A.; Rank, E.; Beiert, U.; Knig, M. (2010b). A Pattern-Based Approach for Facilitating
Schedule Generation and Cost Analysis in Bridge Construction Projects. Istanbul, CRC Press/Taylor & Francis
Group, 93100.

72

Transformation and Integration of Building Information Model for Disaster
Mitigation
Chia-Ying Lin & Chien-Cheng Chou
National Central University, Taiwan
ABSTRACT: BIM techniques attempt to bring the entire life cycle of a building together, including the design,
construction, operation and maintenance phases. Although many researchers investigate BIM environments,
most of them are concerned with the design and construction phases. With the plentiful information carried by
BIM, developing applications of the operation and maintenance phases can be a potential research trend. For
example, mitigating disaster impact is one of the critical issues in applying BIM in the post-construction phase.
During the occurrence of a disaster, first responders need up-to-date information to support their decision-making
process. On the other hand, they also need simplified information to mitigate the loss as soon as possible. In
BIM, it promise to update information in all views and remain all versions to ensure the newest and the most
complete data. However, in order to meet the demands of first responders, the information from the BIM still
needs to be simplified and transformed. In this research, model driven architecture (MDA) framework is applied
to process the BIM information in order to fit in with the needs of first responders. MDA is one of the software
developing methodologies specifically for used in the computer science field. In the software developing process,
the first thing is using system analysis and design techniques to capture the demands of stakeholders. During this
stage, some models such as use case models, class diagrams, and so on, are produced to map the concepts into the
system. Then programmers develop the software according to the models. While referring to the MDA, the
mechanism is mainly based on the model transformation. Meta Object Facility (MOF), as the backbone of MDA,
provides a universal way of describing modeling constructs. Clarifying the definition of models and meta-models
makes it easier to correlate and exchange model information across different models. It enables MDA to accelerate
the software developing process as well as lighten the work load of maintenance of the software. Since BIM is also
regarded as a model, the authors define a new mapping for BIM-related model structure and try to assist
information extraction from BIM by this new approach. The BIM for a specific building, e.g. a concrete plant, is
promoted to the model level in order to generate application program for this building. Such concept facilitates
development of different application software around BIM environment, and, especially, for disaster mitigation in
this research.
KEYWORDS: BIM, Model-Driven Architecture, Meta Object Facility, Disaster Mitigation.
1. INTRODUCTION
Building Information Model (BIM) is a promising technology that can provide a communication and information
exchange platform for all stakeholders involved in a building, as well as create a 3D display environment to clarify
a buildings virtual representations. Many researchers study BIM-related topics, such as design and engineering,
linking to analysis tools, energy innovations, facility management, and so on (Becerik-Gerber and Kensek, 2010).
However, most studies are concerned with the design and construction (D&C) phases of a building, few studies
deal with issues pertaining to the operation and maintenance (O&M) phases (Akcamete et al., 2010)(Vanlande et
al., 2008). Because costs associated with the O&M phases of a building are always higher than those with the
D&C phases (Gallaher et al., 2004), and because BIM is becoming a mature software tool capable of supporting all
kinds of information-intensive activities, applying the BIM technology to the O&M phases is desired.
Although researchers have shown that a building construction project and its subsequent facility maintenance
projects are dissociated (Vanlande et al., 2008), it is obvious that the applications in the O&M phases require the
information from the D&C phases of a building. In the D&C phases, BIM applications are designed to use
parametric and object-oriented modeling techniques to represent each individual building element and help
associated processes. Applications in the O&M phases may require not only original BIM-related information
but the enhanced one that can be employed to record and manage various activities during O&M. Example
applications include facility management systems and disaster management systems. Current practices require
contractors provide a buildings handover information such as Construction-Operations Building information
exchange (COBie) for later usage in the O&M phases, but they usually spend minimal efforts so as to satisfy the
contract (Akcamete et al., 2010). Hence, a better information bridge may be needed in order to provide
transformation and customization of the handover information from the D&C phases to the O&M phases.
73

Applications in the O&M phases possess several unique requirements regarding interpretation of BIM. Some
BIM information should be accessed and handled via an easier method while the other should be further processed
through use of a more complicated interface. In other words, customization of BIM is needed for the O&M
phases. In fact, the utilization of BIM should follow the open/closed principle (OCP) open for extension, but
closed for modification (Meyer, 1988) as well. In the closed side, for example, structural elements in a building
will not be changed during their entire life cycle (Hassanain et al. 2001); and thus, for application developers in the
O&M phases, it may be a good way to operate such information at the class level, not at the metaclass level to
avoid future modification. By operating BIM at the metaclass level, we mean that O&M application developers
regard the structural elements as objects and use generalized classes such as Room, Column, and Wall to access
each individual element. When facility managers would like to query one specific rooms maintenance history,
application developers need to select all rooms maintenance objects and then filter out the records not belonging
to the room specified. It is an indirect approach, and because the structural elements are constant, developers
actually can regard each element as a distinct class in order to use an object to represent the current condition of an
element at a specific time instance. In this way, all BIM or IFC-related functions are encapsulated in one class for
an element. Application developers can have a more simple data access interface to manipulate O&M-related
information. The traditional approach regards the structural elements or other elements infrequently changed as
objects regardless of the D&C or O&M phases; and thus, such O&M applications may use additional redundant
codes to retrieve information.
In the open side of the OCP, current materials and equipment information stored in BIM cannot accommodate
various needs during the O&M phases, e.g., spare parts list management, which is an essential function from the
facility management perspective. An extension mechanism of BIM may be needed and has been proposed by
several studies (CRC, 2007). Among the O&M-related activities, the concept of rooms and zones forms the
central management unit, which is just one of the derived attributes and is not the focus during D&C. Proper
spatial containment of related building elements is essential in the O&M phases (East et al., 2012). In the D&C
phases, architects design the form or the placement of each building element. But facility managers need to know
the location of a specific device in order to perform maintenance work. A typical room consists of many building
elements, and in BIM tools such as Revit, designers need to draw walls, windows, and doors first, and then specify
the boundary of a room to link these building elements. Additional furniture or equipment elements may need to
be attached to some rooms during the O&M phases. The traditional approach to transferring handover data
requires manually create polylines of a building instead of directly using the digital building information used in
the D&C phases. The traditional approach needs couple days and is labor-intensive. Even if computerized tools
can be provided to assist in the transformation process, such tools describe only the geometric information and do
not consider the integrity of each building element as a room or zone in the O&M phases.
Finally, several studies have recognized the lack of 3D visualization capabilities in most applications in the O&M
phases (Akcamete et al., 2010). BIM, like a spatial analysis engine (Akcamete et al., 2010), can assist facility
management applications in analysis of spatial relationships between these building elements. The other issue is
that the O&M phases consist of many different applications, such as disaster mitigation, security, community care,
and so on. In these fields domain-specific requirements exist, and BIM cannot cover all the fields which need
building information as a possible input source.
In this research, the encapsulation of BIM (E-BIM) was designed following the model-driven architecture (MDA)
technique for automatic and comprehensive transformation and customization of the handover information from
the D&C phases to the O&M phases. A metamodel hierarchy of a building was constructed to encapsulate the
building information, not simply transforming data into an open format, but playing the role of an interface
users can manipulate the building information through the model instead of directly modifying it. Regarding to
the designed metamodel hierarchy, we also attempted to present spatial relationships to reshape data structures of
maintenance data. The proposed MDA approach was utilized with the hierarchy to generate codes for further
applications of building information.
2. RELATED WORK
Nowadays, the software development process is facing more and more complicated situations. Not only
requirements become more dynamic but different platforms can be selected as the working environment. To
solve these problems, MDA was proposed by Object Management Group (OMG), mainly for the purpose of
integration and interoperability (Soley, 2000). This approach basically uses a series of formal models to help
the software development process.
74

To take an overview of MDA, at first the separation of concerns should be introduced. Three MDA viewpoints
of a system are described as follows: (1) Computation Independent Viewpoint (CIV), (2) Platform Independent
Viewpoint (PIV), and (3) Platform Specific Viewpoint (PSV). CIV contains none of the computer-related
processing details and only focuses on the business requirements; PIV deals with the operations of a system but
does not contain the details for a specific platform; PSV integrates PIV with the details of using a particular
platform as the development environment (Miller and Mukerji, 2003).
As shown in Figure 1, four layers and their transformation mechanisms are identified based on these views. A
computation independent model (CIM) is constructed from CIV. Based on the CIM a platform independent
model (PIM) is created. With the sufficiently complete and precise PIM, a platform specific model (PSM) can
be generated by using model-to-model transformation mechanisms, and the specific code model which can be
viewed as implementation can be automatically transformed from the PSM.
OMG defines a standard Meta Object Facility to provide metadata management and modelling language
definitions. MOF is used with a metamodel hierarchy shown in Figure 2. A run-time system can be
interpreted by a UML model, since the UML model is designed by people, there can be many UML models from
different perspectives. To describe a UML model, we need to define UML descriptions and notations as a
communicating method. The UML definitions are also based on the definitions of MOF. With the
descriptions of higher layers, the lower layer can be clearly explained.

Fig. 1: MDA Process

3. RESEARCH APPROACH
Because of the wide use of BIM, building information is available for stakeholders in the O&M phases.
However, the BIM data structure is not easy to be used for users and needs extra efforts for application
developers to write programs for further use of such information. In order to offer a better way for the extended
use, this research defines a metamodel hierarchy for a building to interpret their various elements.
Figure 3 shows the sample definitions of the metamodel hierarchy of a residential building. This hierarchy is
designed as the basis of the proposed approach.
Computation Independent Model (CIM)
Platform Independent Model (PIM)
Platform Specific
Model (PSM)
Specific Code
Model
Platform Specific
Model (PSM)
Specific Code
Model
Skilled transformation
Automated model-to-
model transformation
Automated model-to-
code transformation
75

Fig. 2: Metamodel hierarchy
Fig. 3: Metamodel hierarchy of a building

M3
M2
M1
M0
Real world Instance
Model layer
Metamodel layer
Metameta model layer
Run-time system
UML
MOF
UML model
<<instance of>>
<<instance of>>
<<instance of>>
Building
Residential Building Factory Apartment Floor Household Room
Living room
Bedroom
Kitchen
Toilet
Door
Wall
Front door
Back door
Residential Building A
As Slot
As Floor
As Living room
As Bedroom
As Kitchen
As Toilet
Aggregation
Inheritance
M1
M2
M3
Metameta
Model
Meta Model
Model
M0
Real World
Object
The timeline (life cycle) of the specific building A
BuildingSuperObject
<<instance of>>
<<instance of>>
<<instance of>>
76

M3 layer: For different perspectives, the contents in M3 layer should be viewed as different roles. Inside
this hierarchy, the M3 layer describes basic components of a building, such as doors, walls, windows, and
so on. These components exist in any type of building, in other words, they are essential for a building;
as a result they are designed to be in the M3 layer. Outside this hierarchy which means extended
applications without sufficient understanding of the building internal structure and BIM programming for
them, the layers below and those basic components are encapsulated due to OCP. Building information is
available for them through the BuildingSuperObject but its data structure cannot be changed from the
outside, that is, BuildingSuperObject is an interface as a communicating bridge.
M2 layer: This layer is instantiated from basic components from the M3 layer, as well as different types of
buildings and their basic units are described here. Different types of buildings may have
rooms/zones/spaces for different purposes, and rooms are classified by purposes as a basic unit. For
example, an apartment is classified as one type of residential buildings. Bedrooms and living rooms may
only exist in a residential building, not in a factory.
M1 layer: In the M1 layer, a model means a specific building and is constructed here. Most of the
static components are from higher layers. Also, the constant spatial components of a building are
described here, since most of the time the floor plan of a building will not be changed after the D&C
phases.
M0 layer: Finally, the instances of M1s building are stretched by time as a timeline and record the whole
life cycle, which means the building information covering the D&C phases to the O&M phases exists in
this layer.

Suppose BIM data of a specific building is available. First its spatial relationship and the usage of spaces are
re-drawn as the model shown in the M1 layer of Figure 3. The class library of this building can then be
automatically generated by the model. Since attributes are also predesigned to be contained in the model,
extended applications can use data they need through the model.
In Figure 4, the E-BIM process was designed referred to the MDA process; the original BIM of the specific
building can be viewed as the CIM level, and then it will be transferred to E-BIM form preparing to be extended
by different applications. Different application domains are considered PSM in MDA process; therefore, specific
domain knowledge is used to be the additional information and taken as the foundation of the specific
applications.

4. EXAMPLE
Since the more floors and rooms a building contains, the more complicated the problem is. A three-floor
apartment is presented in this section as a simple example to show the contents in metamodel hierarchy of an
apartment building B. Some details are also omitted for clearly interpreting the proposed approach. The
relationship between contents and the expected application of the metamodel hierarchy are also interpreted in
this section.
Figure 5 presents a sample model constructed following the metamodel hierarchy in previous section. In
apartment building B, there are three floors and an elevator. There are two households in each floor, and the
rule of the address assignment is the floor number, plus the serial number. Each household is assumed to have
three rooms, i.e., living room, bedroom and toilet. In Figure 5, rooms in address 02 are omitted. These
rooms define attributes from their designed usage respectively. In addition, from the spatial perspective, B
can be viewed in two dimensions: horizontal and vertical. The horizontal dimension is equivalent to the floor
concept; the vertical dimension, named slot here, is composed of rooms located on the same vertical line.
Usually the floor plan of a household is the same as its neighbour upstairs or downstairs in an apartment;
therefore, rooms of the same purpose belong to the same slot. However, the elevator is independent of a floor
or a slot since it is movable.
The model described above becomes a basis for automatic code generation; a class library of B is constructed
for extension. The extended applications can use BIM data through this model more specifically, users of
77

these applications access BIM data through this model instead of directly operating with BIM. Since it is not
easy to cross the threshold of the programming issues for BIM applications, it is believed that using this model
will assist the development of extended allocation for BIM.

Fig. 4: The E-BIM process referred to the MDA process
Fig. 5: The model of a specific apartment building B

The original BIM of the specific building
E-BIM of the specific building
Specific domain
Knowledge
Specific
application
Specific domain
Knowledge
Specific
application
Apartment B
Floor 1
Floor 2
Floor 3
Address 101
Address 102
Address 201
Address 202
Address 301
Address 302
101s living room
101s bedroom
101s toilet
201s living room
201s bedroom
201s toilet
301s living room
301s bedroom
301s toilet
Slot 01 space a
Slot 01 space b
Slot 01 space c
living room
bedroom
toilet
Elevator
78

5. Conclusion
This research has proposed a software architecture to integrate the static (BIM) information with the dynamic,
O&M-related data. The MDA technique was utilized to store and synthesize the data. The BIM information
is used to show the geometry aspect of the building. MDA has been successfully applied to other industries
applications, including the AEC industry. Further enhancement of the system is needed in order to integrate
more dynamic information from different facility management domains and/or other BIM programs.
6. REFERENCES
Akcamete, A., Akinci, B., and Garrett, J.H. (2010). Potential utilization of building information models for
planning maintenance activities. In: Tizani, W. (Ed.). Proceedings of the International Conference on Computing
in Civil and Building Engineering, Nottingham, UK.
Becerik-Gerber, B. and Kensek, K. (2010). Building Information Modeling in Architecture, Engineering, and
Construction: Emerging Research Directions and Trends, Journal of Professional Issues in Engineering Education
and Practice, 136(3), pp.139-147.
CRC for Construction Innovation. (2007). Adopting BIM for facilities management: Solutions for managing the
Sydney Opera House, Brisbane, Queensland, Australia.
East, E. W., Nisbet, N., and Liebich, T. (2012). The Facility Management Handover Model View, Journal of
Computing in Civil Engineering, in press.
Gallaher, M.P., OConnor, A.C., Dettbarn, Jr., J.L., and Gilday, L.T. (2004). Cost Analysis of Inadequate
Interoperability in the U.S. Capital Facilities Industry, National Institute of Standards and Technology,
Gaithersburg, Maryland, USA.
Hassanain, M.A., Froese, T.M., and Vanier, D.J. (2001). Development of a maintenance management model based
on IAI standards, Artificial Intelligence in Engineering, 15, pp. 177-193.
Jardim-Goncalves, R., Grilo, A., and Steiger-Garcao, A. (2006). Challenging the interoperability between
computers in industry with MDA and SOA, Computers in Industry, 57, pp. 679-689.
Meyer, B. (1988). Object-Oriented Software Construction, Prentice Hall.
Miller, J. and Mukerji, J. (2003). MDA Guide Version 1.0.1, Object Management Group,
http://www.enterprise-architecture.info/Images/MDA/MDA%20Guide%20v1-0-1.pdf, accessed August 2011.
Soley, R. (2000). Model Driven Architecture, White Paper, Draft 3.2, Object Management Group.
Vanlande, R., Nicolle, C., and Cruz, C. (2008). IFC and Building Lifecycle Management, Automation in
Construction, 18, pp. 70-78.

79

DEVELOPMENT OF A VISUAL DOMESTIC ENERGY ASSESSMENT
TOOL
Mhalas A, Crosbie T & Dawood N
Centre for Construction Innovation and Research, Teesside University, Middlesbrough, TS1 3BA, UK.
ABSTRACT: The UK has ambitious CO
2
emission reduction targets of 20% by 2020 and 80% by 2050. More
energy is used in housing than either road transport or industry. The energy used in homes accounts for more than
a quarter of energy use and CO
2
emissions in the UK and the UK has some of the least energy efficient housing
stock in Europe. Therefore it is imperative to improve the energy performance of the existing domestic building
stock and fully exploit renewable energy sources. The problem is that it is difficult for decision makers to assess
which options for reducing CO
2
emissions have the most potential. There are a number of databases holding
information about the domestic building stock in the UK. This research investigates how this information with the
aid of information technology and visualisation systems can be used to support informed decision making in
energy related urban planning. A pilot tool is being developed to estimate the environmental and financial
implications of different approaches to reducing the energy consumption and CO
2
emissions from domestic
buildings at the neighbourhood level. The tool will support town planners, local authorities and social housing
providers in the development of the policies and interventions necessary reduce the CO
2
emissions from domestic
building stock. In this way the research seeks to help town planners and local authorities to meet CO
2
emission
reduction targets.
KEYWORDS: Building Stock, GIS, Building Physics Based Model, Domestic Energy Assessment, Standard
Assessment Procedure.
1. INTRODUCTION
Buildings contribute almost a half of all CO
2
emissions in UK. Of those emissions 26% come from approximately
26 million residential dwellings and 18% come from 2 million non-domestic buildings which include commercial,
public sector and industrial buildings. It is expected that about 75% of the existing domestic stock will be still
present in 2050 (Wright, 2008).
With the rising interest in tackling climate change and subsequent to the 1992 Kyoto Protocol there is a growing
incentive to develop sustainable societies through increased use of renewable energy sources and reducing energy
demand thereby reducing the CO
2
emissions to the atmosphere. The UKs commitment under the protocol is for a
reduction in greenhouse gas emissions of 12.5% from 1990 levels by 2012. The UK government in its Climate
Change Bill has legally committed to reduce its carbon emissions by 80% by 2050 over its 1990 baseline (H.M.
Government, 2008). The UK government is also committed to meet the EU target to obtain 15% of energy from
renewable sources by 2020 (House of Lords, 2008). Putting things into perspective, carbon emissions reduction
from the existing built environment is likely to be a key component of meeting the overall 80% CO
2
emissions
reduction target.
The 2002/91/EC Energy Performance of Buildings Directive requires all EU member states to establish a
national energy calculation methodology to measure the energy performance of buildings. National Indicator (NI)
186 requires every UK local authority to report annually the percentage reduction in CO
2
emissions per capita from
their area includes public and private housing (DECC, 2009). The success of NI 186 depends on the ability of local
authorities to identify individual properties which could benefit from energy efficiency measures, in addition to
making broad citywide estimates. However, one of the problems in reporting and monitoring effectiveness of
energy efficiency measures is the lack of consistent, publicly available tools and methods for calculating the
potential carbon savings (Gupta, 2009).
In light of the above, this paper presents the findings of the research based on a hypothesis that visual aid
techniques can better estimate baseline carbon emissions and assist in energy related urban planning. The research
aims to use visualisation techniques integrated with databases in Geographic Information System (GIS) to estimate
baseline CO
2
emissions and identify the CO
2
reduction potential of urban energy efficiency and renewable energy
interventions.
80

GIS has traditionally been seen as a means of displaying information having spatial relationship. However, with
the advancement in GIS based software including data structuring and storing, analyses, model development and
geo-processing capabilities further coupled with developer tools, it is now possible to develop customized
applications. As the intention of this research is to demonstrate a methodology, it is only intended to develop a
low level prototype. This paper gives a brief description of input requirements, the output from the process and
the illustrations of their use in case studies.
2. ENERGY MODELS
All energy modelling is based on some form of input data with which to calculate or simulate energy
consumption. This data is available at various levels of detail, resulting in the use of different modelling
techniques which seek to take advantage of the available information. Techniques to model energy consumption
can be broadly categorised into top-down and bottom-up approaches.
2.1 Top-down Approach
Top down approaches work at an aggregated level and are typically used to investigate the relationship between
the energy sector and economy at large. Top down approaches can be further classified into econometric and
technological approaches. Econometric models focus on energy use in relationship to variables such as income,
fuel prices, and gross domestic product to express the connection between the energy sector and economic output.
Technological models attribute levels of energy consumption to broad characteristics of the entire housing stock
such as appliance ownership trends (Kavgic et al., 2010). The strength of top-down modelling is that it can be
based on historical aggregated data, which is widely available and simple to use (Swan and Ugursal, 2009).
However, these models have no inherent capability to model discontinuous advances in technology. Also, the
lack of detail concerning the energy use of individual buildings makes it difficult to identify how current
approaches to the reduction of energy consumption can be improved (MIT, 1997).
2.2 Bottom-up Approach
The bottom-up approach consists of models that use input data from a hierarchal level less than that of the sector
as a whole. Energy consumption from highly disaggregated units such as individual houses, or groups of houses
is accounted for and then extrapolated to regional or national level.
Statistical modelling is an approach that relies on historical information and regression analysis which is used to
attribute building energy consumption to particular end-uses. Once the relationships between end-uses and
energy consumption have been established, the model can be used to estimate the energy consumption of
dwellings representative of the residential stock. Even though, this method models energy consumption it does
not provide much detail or flexibility and therefore its capacity to evaluate the impact of a wide range of energy
conservation scenarios is restricted (Swan and Ugursal, 2009).
Building physics based models are another approach that rely on detailed housing information. This approach is
based on a sample of houses representative of the national housing stock and utilises a building energy
calculation method to estimate delivered energy. These models require input composed of quantitative data on
variables such as the characteristics of space heating systems, walls, roof, floor, windows, doors along with their
thermal characteristics (U-values), internal temperatures and heating patterns, ventilation rates, energy
consumption of appliances, number of occupants, external temperatures, etc. The combination of building
physics and empirical data from housing surveys, national data sets and assumptions about buildings operation,
give modellers the means to estimate energy consumption in dwellings for the past, present, and future. By
developing different scenarios, the models have the potential to be used to assess the impact of specific carbon
reduction measures on the overall energy demand, which can be used as part of an evidence based approach to
medium to long term energy supply strategy. These models are seen as useful tools to provide for policymakers
with estimates of the effectiveness of policies and can help to identify technological measures for end-use energy
efficiency (Kavgic et al., 2010).
Some of the key UK building physics models are:
x Building Research Establishments Housing Model for Energy Studies (BREHOMES) (Shorrock and
Dunster, 1997);
81

x The Johnston Model (Johnston, 2003);
x The UK Domestic Carbon Model (UKDCM) (Boardman, et al., 2005);
x The DECarb model (Natarajan and Levermore, 2007);
x The Domestic Energy Carbon Counting and Carbon Reduction Model (DECoRuM) (Gupta, 2009);
and
x The Energy and Environmental Prediction (EEP) (Jones et al., 2007).
All these models have the same energy calculation engine BRE Domestic Energy Model (BREDEM), modified
to varying degrees. Bottom-up methods appear more appropriate in the context of this paper as these models can
account for changes in end-use technologies. They attempt to account for the complex interactions that exist
between the different end-uses of energy, and incorporate changes in ownership, substitution effects and saturation
effects. As the data is physically measurable, it enables bottom-up methods to be accurately extrapolated to allow
for future changes in technical efficiency or technical change (Cheng & Steemers, 2011). This enables greater
understanding of the way in which the individual end-uses of energy influence housing stock energy use and CO
2

emissions and can explicitly assess and quantify the impact that different technological advances may have on
energy use and CO
2
emissions.
Though BREDEM has been at the center of several energy related research described above, it is not without
limitations or shortcomings. One of the most important aspects is that, these models lack transparency and
quantification of inherent uncertainties. The resolution of spatial coverage is another concern with these models.
The local authorities, planning agencies and energy consultants are more interested in predicting energy use of
housing stock at the level of city, region and local authorities (Cheng & Steemers, 2011). This is so that the local
government can use this information to target areas with lower energy efficiency through energy efficiency and
renewable energy interventions.
BREDEM also underpins the Standard Assessment Procedure (SAP) which is the recommended tool of the
Department of Energy and Climate Change (DECC) for assessing and comparing the energy and environmental
performance of dwellings. In 1994 SAP was cited in Part L of the Building Regulations for England and Wales
as a means of assessing dwelling performance. SAP is now UKs National Calculation Methodology, meeting
one of the requirements of the Energy Performance of Buildings Directive. SAP provides indicators of energy
performance through energy consumption per unit floor area, an energy cost rating (the SAP rating), an
Environmental Impact rating based on CO
2
emissions (the EI rating) and a Dwelling CO
2
Emission Rate (DER)
(DECC/BRE, 2010). The calculation is based on the energy balance of the dwellings taking into account the
following factors:
x Materials used for construction of the main aspects of the dwelling such as walls, windows and roofs
x Thermal insulation characteristics of the building fabric
x Ventilation characteristics of the dwelling and ventilation equipment
x Efficiency and control of the heating system(s)
x Gains into the dwellings from solar, metabolism, cooking and lights and appliances
x Fuel used to provide space and water heating, ventilation and lighting
x Energy for space cooling, if applicable
x Renewable energy technologies
3. MODEL DESCRIPTION
As mentioned earlier, for this research, bottom-up methods are more appropriate to establish the baseline energy
consumption and hence the CO
2
emission data. For this purpose, we have used BREDEM based SAP as our core
energy calculation engine. To calculate the energy cost rating, SAP requires an input of over 80 different items of
data for each dwelling listed in Table 1. This data is typically easily available for new developments; however,
82

for existing dwellings, most of this data has to be gathered through site surveys. A detailed property survey by a
trained assessor can last for at least 30 minutes (Rylatt et al., 2003). Thus collecting this data for each dwelling
and then aggregating for locality, town, city, region, etc. can be time hugely consuming and an expensive process.
To overcome this challenge, this research makes use of information from aerial and terrestrial imagery, published
databases such as Homes Energy Efficiency Database (HEED), household surveys such as English House
Condition Survey (EHCS), census and the Office of National Statistics (ONS) and identify the parameters
required to calculate SAP rating through desktop studies using GIS based modelling. In addition, this research
identifies the energy consumption and hence carbon emission reduction potential of the area through use of
energy efficiency and renewable energy technologies. The overall concept of the research is demonstrated using
the function modeling diagram below.

Fig. 1: Overall Concept to Calculate Baseline Energy Consumption and Energy Reduction Potential
3.1 Creating Domestic Dwelling Models
One of the first requirements of the process is to develop domestic dwelling models which represent the types of
dwellings present in a neighbourhood. Based on the input parameters required by SAP, the dwelling model will
be built on the geometry (dimensions) of the dwelling, the physics (building fabric) and the type of usage in the
dwelling. This is depicted below in Figure 2.

Fig. 2: Creating a Domestic Dwelling Model.
This research has identified the various data sources required to develop the domestic dwelling models. The
detailed list of parameters and the sources of their data is described in Table 1.
Table 1: Data Required for Model Development and the Sources of Data
Data Category Information Required Data Source
Location Degree day region
Height above sea level (m)
Mean wind speed
Level of Overshading
Vector map of the area from Ordinance Survey
Data tables provided in SAP
Raster imagery from Ordinance Survey and Google
Maps
A1.1
DevelopDwelling
GeometryModel
Requestfrom
townplanners,
local authorities,
andsocial housing
providersto
reducecarbon
emissionsfrom
domestic
dwellings ina
neighbourhood
Featurecollectiondataset
(vectormaps) fromLandmap/
Ordinance SurveyinGIS
Software
A1.2
DevelopDwelling
PhysicsModel
Aerial Imagery(rastermaps)
fromLandmap/Ordinance
SurveyandlinkforGoogle
StreetviewintegratedinGIS
software
Dimensions of the
dwellingsuchas
floorarea,height
andperimeter
HouseArchetypesfrom
EnglishHousingSurveyand
otherliteraturereview
A1.3
DevelopDewlling
UsageModel
Housetype
informationsuch
as terraced,
detached,flats,
etc.
Numberof peoplelivingin
various typesof dwellings
fromNeighbourhood
Statistics
Estimatesonboiler,
radiatorcontrols,heating
regimefromIndices of
Deprivation
Completeddwelling
model storedas attributes
of vectormaps
Repeatthetaskfor
otherdwellings inthe
neighbourhood Standard
assumptionsfrom
RdSAP
83

Type Dwelling detachment (mid or end
terraced, semidetached, detached, flat,
etc.)
Dwelling Age (Before 1900, 1900-1929,
1930-1949 1950-1966, 1967-1975,
1976-1982, 1983-1990, 1991-1995,
1996-2002, 2003-2006, 2007 Onwards)

Vector map of the area from Landmap, raster imagery
from Ordinance Survey and Google Maps
Dwelling dimensions Number of storeys
Floor area and perimeter
Height of each storey
Area of the roof
Area of the exposed walls and windows
Terrestrial imagery from Google Maps
Vector map from Landmap
Built fabric U Value for walls and windows
U Value for floor and roof
Draught proofing
Type of window and door frame
Orientation of windows
Inferred from age of the building and raster imagery
Inferred from age of the dwelling
Inferred from economic deprivation data from ONS
Terrestrial imagery from Google Maps
Maps
Type of ventilation Pressure test results
Number of flues, chimneys
Number of fans and vents
Presence of mechanical ventilation
Assumed no pressure test undertaken
Maps
Inferred from age of the dwelling
Assumed no mechanical ventilation
Type of water heating Type of water heater (gas, oil or solid fuel
boiler, electric immersion.)
If hot water tank present its volume,
thickness of insulation, thermostat and
insulation of primary pipework
HEED

Default SAP values depending on age of the dwelling
Type of space heating Heating system type e.g. open, sealed,
etc. and fuel e.g. gas, oil, soild fuel,
electric, etc.
Efficiency of the heating system
Type of heating controls (programmers,
thermostats)
HEED and EHCS

Inferred based on HEED and EHCS data
Cooking Type of cooking system (electric cooker,
cooking range, etc.)
Lighting Total number of lighting outlets and
proportion of low energy lights
Inferred based on economic deprivation data from
ONS, HEED and EHCS data
84

Occupancy Number of occupants
Heating periods
Demand temperatures
Level of use of hot water, lights and
cooking (average, below average and
above average)
Census data from ONS
ONS
ONS
ONS
The vector maps obtained from Ordinance Survey has various attribute information attached with the point, lines
and polygon features. The above information required for energy modeling can also be attributed to the dwelling
polygons. This can be achieved through the developer techniques available with GIS based software. In our case,
we have developed a userform in ArcGIS 10 using Visual Basic for Applications (VBA). ArcGIS also supports
use of other applications such as VB.Net and Java. The userform developed for this research is shown in Figure
3 below.
Figure 3: Userform to input data and develop dwelling models
One of the significant advantages of using GIS based software is that the data entry process can be replicated for
several dwellings if they have similar characteristics. This is particularly useful in undertaking the energy
assessment for long rows of terraced houses, where several features other than the dwelling dimensions will
typically be similar. This expedites the data entry process and hence identification of energy consumption over
larger areas can be undertaken relatively quickly.
3.2 Calculate Baseline Energy Consumption
Subsequent to developing the dwelling models in a neighbourhood, the next stage is calculation of baseline
energy consumption. Again, with the built-in developer capabilities of GIS softwares, the empirical equations
within the SAP calculation engine can be developed. The dwelling models developed feed in the required input
parameters to these empirical equations. The resultant data is the energy consumption values. As mentioned
earlier, this research intends to demonstrate this methodology through development of a prototype, an MS-Excel
based calculation tool is developed which is simpler compared to the detailed programming required in VB.Net
and/or Java which is currently beyond the capacities of researchers of this project. The development of this tool
85

is seen as future work.
The lowest geographical level on which the data from the Office of National Statistics is aggregated is the Lower
Level Super Output Area (LLSOA). LLSOAs have a minimum population of 1,000 equating to around 400
households. An energy calculation output for one such LLSOA is depicted in Figure 4 below.
Figure 4: Baseline Energy Consumption Calculation on LLSOA Level
3.3 Quantify Carbon Reduction Potential
Subsequent to identifying the baseline energy consumption and associated CO
2
levels, the next target is to
identify the carbon reduction potential of the area. It is therefore important to identify the key strategies that can
have significant impact on carbon emission reductions. These include (Gupta, 2009):
x Improve the energy efficiency standard of the dwellings through double glazing of windows,
insulation of solid and cavity walls, insulation of roof and draught proofing and increase uptake of
energy efficient space heating and controls and low energy lighting.
x Installation of renewable energy interventions such as solar panels, wind turbines, micro wind turbines
and low carbon installations such as combined heat and power and air and ground source heat pumps.
While we have developed the dwelling models, we have attributed a large amount of information to the existing
vector maps in GIS. This information can be selected using Structured Query Language (SQL), a powerful
language used to select one or more criteria that can consist of attributes, operators, and calculations which is
now commonly a part of GIS software.
Figure 5: Houses with South Facing Roof Capable of Housing Solar Panels
86

This querying capability of GIS software could be harnessed to identify dwellings of particular type. As an
example, we can identify all dwellings with south facing roof and their roof areas so as to identify the solar
potential of the area. An example of this type of selection is shown in Figure 5. Similarly, dwellings having
sufficiently large gardens have a potential for installation of ground source heat pumps. Using querying options,
these types of dwellings can be identified. This is shown in Figure 6.
Figure 6: Dwellings with potential for Ground Source Heat Pumps
Similar to the baseline energy equations, the empirical equations related to energy efficiency, renewable energy
and low carbon technologies are currently being investigated and a MS-Excel based calculation tool, similar to
the one developed based on SAP is being developed.
4. THE CASE STUDY
To validate the approach of identifying baseline energy consumption through use of visual aid and databases
integrated within a GIS, system, an empirical case study has been undertaken. For this purpose, it was decided
that some dwellings currently being managed by a social housing provider be used. This method was selected as
the social housing providers have to maintain an energy performance certificate (EPC) for their properties. Thus
the energy performance of the dwellings estimated by the method in this research can be checked with the energy
performance calculated from actual survey of the property. The following Table 2 shows a comparison of the
results. To get a better idea a range of properties of various age and detachment were selected.
Due to data protection requirements, the exact number of the property cannot be provided. However, all the
properties are located within the Middlesbrough, UK and their street names and post codes have been provided.
Table 2: Case Study Results
Dwelling Estimated Energy
Consumption (kWh/m
2
)
Calculated Energy
Consumption
(kWh/m
2
)
% Difference
Ammerston Road, TS1 5LL 198.54 175.00 +13.45
Bow Street(1), TS1 4BS 306.77 283.26 +8.29
Bowley Walk, TS1 5NB 189.47 190.08 -0.003
Ayresome Green Lane TS5
4DZ
322.94 381.69 -15.39
Carey Close(1) TS1 5NH 351.69 380.96 -7.68
Alwent Road(1) TS1 5LJ 180.22 191.62 -5.95
87

Albany Street TS1 4DB 283.64 298.78 -5.07
Aske Road, TS1 4DA 268.75 242.86 +10.66
Alwent Road(2) TS1 5LJ 178.80 187.52 -4.65
CareyClose(2) TS1 5NH 269.54 286.21 -5.82
Bow Street(2) TS1 4BU 338.30 380.96 -11.20
The above results indicate that the energy calculation undertaken from data inferred through aerial and terrestrial
imagery, databases, surveys and national statistics are within 15% of the values obtained through actual site
surveys. Further analyses is also being carried out to identify the difference in intermediate parameters such as
total gains, losses, mean internal temperatures, energy required for space and water heating and energy required
for lights and appliances. This should give us a better understanding of how these values behave in greater detail.
5. CONCLUSION
This paper has described in brief a method for estimation of baseline energy consumption and associated CO
2

emissions eliminating the need to visit the properties, thus achieving significant savings in time and cost. Further,
with the use of GIS, once the models are developed, this information is saved within a central database. This
database can be used not only for analyses of energy consumption, but for other purposes such as alleviating fuel
poverty. The approach has also been validated through undertaking a case study. The approach is of significant
use to the local authorities, regional and town planners, energy consultants, etc. in making energy related urban
planning decisions.
6. REFERENCES
Boardman, B., Darby, S., Killip, G., Hinnells, M., Jardine, C., Palmer, S., et al. (2005). 40% House. ECI:
University of Oxford.
Cheng, V., & Steemers, K. (2011). Modelling domestic energy consumption at district scale: A tool to support
national and local energy policies. Environmental Modelling & Software, Vol. 26, 1186-1198.
DECC. (2009). Community Energy Saving Programme. Department of Environment and Climate Change
Consultation.
DECC/BRE. (2010). The Government's Standard Assessment Procedure for Energy Rating of Dwellings. Watford:
Building Research Establishment.
Gupta, R. (2009). Moving towards low-carbon buildings and cities: experiences from Oxford, UK. International
Journal of Low-Carbon Technologies, Vol. 4, 159-168.
H.M. Government. (2008). Climate Change Act. London: The Stationery Office Limited.
House of Lords. (2008). The EUs Target for Renewable Energy: 20% by 2020. London: The Stationery Office
Limited.
Johnston, D. (2003). A physically based energy and carbon dioxide energy model of the UK housing stock. PhD
Thesis, Leeds Metropolitan University.
Jones, P., Patterson, J., & Lannon, S. (2007). Modelling the built environment at an urban scale - energy and health
impacts in relation to housing. Landscape and Urban Planning, Vol. 83, 39-49.
Kavgic, M., Mavrogianni, A., Mumovic, D., Summerfield, A., Stevanovic, Z., & Djurovic-Petrovic, M. (2010). A
review of bottom-up building stock models for energy consumption in the residential sector. Building and
Environment, Vol. 45, 1683-1697.
MIT. (1997). Energy technology availability: review of longer term scenarios for the development and deployment
of climate friendly technologies. Cambridge, USA: Massachusetts Institute of Technology Energy Laboratory.
88

Natarajan, S., & Levermore, G. J. (2007). Predicting future UK housing stock and carbon emissions. Energy Policy,
Vol. 35 No. 1, 5719-5727.
Rylatt, R.M., Gadsden, S.J., Lomas, K.J., (2003). Methods of predicting urban domestic energy demand with
reduced datasers: a review and a new GIS based approach. Building Services Engineering Research and
Technology, Vol 24 No. 2, 93-102.
Shorrock, L. D., & Dunster, J. E. (1997). The physically-based model BREHOMES and its use in deriving
scenarios for the energy use and carbon dioxide emissions of the UK housing stock. Energy Policy, Vol 25 No. 12,
1027-1037.
Swan, L. G., & Ugursal, V. I. (2009). Modeling of end-use energy consumption in the residential sector: A review
of modeling techniques. Renewable and Sustainable Energy Reviews, Vol. 13, 1819-1835.
Wright, A. (2008). What is the relationship between built form and energy use in dwellings? Energy Policy, Vol. 36,
4544-4547.

89

STUDYING CCTV COVERAGE IN AN MRT STATION USING
BIM-BASED VR APPROACH
Huan-Ting Chen, Si-Wei Wu & Shang-Hsien Hsieh
ABSTRACT: Nowadays, Closed Circuit Television (CCTV) systems are an effective way of monitoring traffic,
preventing crime, and ensuring transport safety in many MRT (Mass Rapid Transit) stations. However, the
effectiveness of CCTV coverage is often achieved through experience and by trial-and-error, instead of being
evaluated and visualized using a robust approach. In this research, a Virtual Reality (VR) approach to this problem
is presented. This approach is based on Building Information Modeling (BIM) technology and is capable of
simulating CCTV systems in a VR environment in order to evaluate the CCTV coverage. Using this approach, the
VR environment of an MRT station is firstly represented by a BIM model. Next, a method for simulating varifocal
CCTV lenses in order to attain different fields of view was developed, allowing real CCTV views to be
approximated by adjusting the parametric properties of simulated CCTV cameras in the BIM model. Finally, the
overall coverage of the CCTV systems for an MRT station can be demonstrated visually and studied in the station's
BIM model. In addition, an API (Application Programming Interface) plug-in program for the Autodesk Revit
Architecture was developed to facilitate the parametric modeling of CCTV systems and the evaluation of the CCTV
coverage.
KEYWORDS: Building Information Modeling (BIM), Closed Circuit Television (CCTV), Mass Rapid Transit
(MRT)
1. INTRODUCTION
Nowadays, Closed Circuit Television (CCTV) is widely used in many public spaces in order to prevent crime and
monitor traffic events. Currently, most CCTV systems are designed based on 2D diagrams such as floor plans.
This may cause many design conflicts due to a lack of the third dimension in the design space, and may even lead
to overlapping CCTV coverage. Additionally, both CCTV screen views and CCTV coverage are unknown before
construction is completely finished.
Using Building Information Modeling (BIM) technology, an accurate building model with precise geometry and
other relevant data can be constructed in a virtual environment throughout the lifecycle of the building (Eastman
et al., 2011: 1-2). Now that the 3D modeling and visualization provided by the BIM software can effectively
discover and solve some hidden problems that previously could not be detected until construction was finished.
By using different viewpoints, reviewers are able to conduct interference checking between systems and thus
reduce unnecessary design conflicts. It is also feasible to simulate CCTV systems using some related tools in
BIM software in order to help users inspect the monitoring situation. The use of an existing BIM model to
examine the coverage of a CCTV system can be a highly efficient and low cost auxiliary tool in the construction
of such systems.
The objective of this research is to provide a better VR approach to evaluate the coverage of CCTV systems. The
expected benefits can be summarized as follows: Firstly, to enable better organization and coordination after the
initial design phase of CCTV systems (i.e., to reduce design conflicts and attain better coverage). Secondly, to
reduce the additional time required to setup CCTV cameras during construction. Thirdly, to provide a better
medium for communication between CCTV system designers and project owners.
In addition, an MRT station was chosen as a case study in order to demonstrate the results of this research. In
order to fulfill the different spatial requirements of an MRT station, it is better to consider the arrangement of
CCTV systems in advance. Using the Autodesk Revit Architecture (Autodesk, Inc., 2010a) (hereafter shortened
to Revit) along with the BIM model of an MRT station, BIM technology was introduced to integrate CCTV
systems into the virtual building space of an MRT station. Finally, virtual CCTV screen views were simulated so
that the CCTV coverage could be clearly visualized and studied in the virtual environment. This approach
proved to be effective and can be widely applied to other building spaces under similar conditions.
90

2. MODELING A CCTV SYSTEM IN A BIM ENVIRONMENT
This section explains the process of modeling a CCTV system in a BIM model. Revit has been a popular and
competitive product in the BIM software market in Taiwan. It is widely used in most construction projects in
Taiwan; therefore, this research aims at developing a complete solution in Revit for simulating CCTV systems in
BIM models in order to achieve the desired objectives.
2.1 Simulating varifocal lenses of CCTV cameras
A complete CCTV system is composed of video cameras and TV monitors. This section introduces how the Revit
camera works, as well as how it can serve as a CCTV camera in order to generate virtual screen views on TV
monitors. Since both Revit cameras and CCTV cameras can produce perspective views on the screen, it is an
appropriate and convenient option in the simulation of CCTV systems. Virtual screen views can also be
automatically generated by setting up Revit cameras with several significant parameters applied.
Different CCTV cameras in a building have different fields of view (FOV). Based on the optical properties of
camera lenses, different FOVs can be achieved by changing their focal lengths. Thus, the first step in the
simulation of CCTV systems in BIM models is to come up with an approach to simulate the varifocal lenses of
CCTV cameras.
Revit cameras lack parameters concerning focal length (i.e., it is not possible to change the focal length of a
certain Revit camera by setting the element parameter). One possible solution is to apply a pre-calculated render
image size to the 3D view of a Revit camera, which corresponds to the desired FOV (i.e., the larger the render
image size, the larger the FOV, and vice versa). Hence, it is necessary to figure out the relationship between the
focal length (i.e., FOV) and the render image size. According to the geometric relationship shown below:
Focol lcngt =
1
2
Film Jimcnsion
tan
F0I
2
(1)
in which, the focal length can be obtained by inputting the FOV into Eq. (1) while the default film dimension is
fixed (i.e., the larger the focal length, the smaller the FOV, and vice versa). The width of the render image size
can now be calculated by using the principle of similar figures (see Fig. 1 below). Thus, the render image size is
inversely proportional to the focal length. The height of the render image can also be obtained by multiplying by
the screen aspect ratio of 3/4.

Lens

Fig. 1: The principle of similar figures between the render image size and the film dimension
2.2 Simulating virtual CCTV screen views
Virtual CCTV screen views can be simulated by using Revit camera views if appropriate parameters are applied.
There are several significant parameters to be configured in the CCTV system and it is necessary to relate these
parameters to the parameters of the Revit camera. Parameters of the CCTV system that are most relevant in this
research include FOV, camera height, target height, subject height, subject distance, and real angle of depression.
After analyzing these parameters and determining their reasonable ranges, an optimized real angle of depression
for each CCTV camera can be found based on the spatial relationship shown in Fig. 2. By setting up a CCTV
camera with a predetermined optimized real angle of depression, the construction efficiency is enhanced as the
set up time is reduced. The following paragraphs explain each parameter while its reasonable range will vary
from case to case.
91

h
H
D
FOV
h

Fig. 1: The spatial relationship of CCTV camera parameters
1. Field of view (FOV):
As mentioned in section 2.1, a CCTV camera uses a varifocal lens to attain different FOVs which are determined
according to its requirements, location, etc. Since this research aims to examine the existing design of CCTV
systems, all FOVs of CCTV cameras should be predefined by CCTV designers.
2. Camera height (H):
The camera height relates to the ceiling height. This dimension is measured from the floor elevation to the
ceiling elevation. The corresponding parameter of the Revit camera is eye elevation.
3. Target height (h):
Normally, the target of CCTV cameras are people, therefore, the average height of humans is adopted as the
target height. The corresponding parameter of the Revit camera is target elevation.
4. Subject height (%) and Subject distance (D):
Rotakin, developed by the Home Office in the UK, is the only device specified in BSI EN 50132-7: 1996 for
testing CCTV camera performance. The Home Office also offers an operation manual (Cohen et al., 2009: 8-9)
that defines the concepts of subject height and subject distance. Depending on the nature of the human activity to
be observed, five general observation categories have been defined for different purposes, including monitoring
and control, detection, observation, recognition, and identification. These five categories are based on the
relative size of a person that appears on a screen (see Fig. 3). This is defined as the subject height (i.e., a
percentage of the height of a person divided by the height of the screen). For a specified subject height, the
maximum distance from which a person can be seen on the screen is defined as the subject distance (i.e., a
person would be too small to be seen on the screen beyond the subject distance for the specified subject height).
An independent consultancy company provides an online lens calculator based on the standards of the Home
Office, LensCalc (CCTV Advisory Service, 2008), to help determine the subject distance required to achieve
specified subject heights for different combinations of sensor size, subject height, and lens focal length (see
Table 1).

Fig. 3: The four out of five general observation categories in accordance with different subject heights
(Cohen et al., 2009: 8-9)
92

Table 1: Subject distances for different combinations of the subject height and the FOV

5. Real angle of depression ():
The Real angle of depression can be referred to as the end result once the camera height, target height, and
subject distance are specified. An optimized real angle of depression can be determined for each CCTV camera
based on spatial relationship (see Fig. 2).
After importing 2D CAD files of CCTV plan views and alignment with floor plan views in BIM models, Revit
cameras can be properly positioned with reference to the layout of the two overlapped plan views. Virtual CCTV
screen views can also be automatically generated after inputting all of the required parameters (see Fig. 4).

Fig. 4: A virtual CCTV screen view simulated by means of the Revit camera
In addition to the parameters above, the CCTV coverage is also shown in Fig. 2. The red trapezoidal area reveals
the coverage area of a certain CCTV camera. The measure of that red trapezoidal area can also be calculated and
displayed in BIM models using Revit (as discussed in Section 3 below).
2.3 Developing a plug-in program with the Revit API
Autodesk has released the Revit API (Autodesk, Inc., 2010b) to allow users to develop plug-in programs in order
to automate repetitive tasks, thus extending the core functionality of Revit. With Revit API, it is likely to perform
customizations based on users requirements.
In order to enhance the performance of this software in the evaluation of CCTV coverage, two main tasks were
taken into consideration: Firstly, it is more succinct to integrate all related functions and steps in Revit to a single
interface because modeling CCTV systems in BIM models requires a sequence of procedures. Secondly, it is more
time efficient for users to configure many parameters due to the fact that there may be many CCTV cameras
required even in a single building space.
The plug-in program used in this research, called CCTV setup advisor for Revit cameras (hereafter shortened to
CCTV setup advisor), has two main functions developed with BIM technology: Firstly, simulating varifocal
CCTV camera lenses by means of Revit cameras. Secondly, configuring the parameters of Revit cameras in order
to obtain virtual CCTV screen views and optimized real angles of depression. Revit cameras in the Revit API are
read-only; therefore, no parameter of Revit cameras can be modified through external API commands. Thus, it is
advisable for users to manually type in the values of the parameters suggested by the plug-in program.
93

Fig. 5 below shows the user interface of the plug-in program. The upper section simulates varifocal CCTV
camera lenses, which helps to determine the render image size depending on different FOVs. The lower section
optimizes the real angle of depression, which suggests parameters for Revit cameras based on user inputs and
then calculates the real angle of depression.

Fig. 5: The user interface of the CCTV setup advisor for Revit cameras
3. A CASE STUDY WITH A COMPLETE BIM MODEL OF AN MRT STATION
After successfully modeling a CCTV system in a BIM-based virtual building space, an approach to studying
CCTV coverage is introduced in this section. To make the BIM-based virtual environment correspond better to a
real situation, a complete BIM model of an MRT station under construction in Taipei city was adopted as a case
study. Furthermore, in order to examine any design conflicts arising from the BIM model, several components in
an MRT station that may block CCTV camera projection are also modeled as parametric elements. These include
smoke curtains and ceiling sign boards.
3.1 Visual representation of CCTV coverage
Even with the highly developed VR technology used today in the construction industry, CCTV systems are more
often designed in a 2D environment. As CCTV systems have been introduced in 3D BIM models in this research,
it is convenient to display the overall situation and the CCTV coverage area through both 2D and 3D VR
approaches.
In order to display visual representations of CCTV coverage, it is essential to define the actual area covered by
CCTV cameras. According to Fig. 2 the red trapezoidal area reveals the coverage area of a certain CCTV camera.
Although the red trapezoidal area can be calculated through a mathematical approach, it is more efficient to use
the Filled region function in Revit. Once an area formed by a closed loop is specified, the corresponding filled
region can be shown on the floor plan view, and the measure of the area can be calculated automatically.
After creating all filled regions based on each CCTV camera in the MRT station, it is crucial to check if there
are any overlapping areas among those filled regions. If so, the filled regions with overlapping areas must be
redrawn as a closed loop area to obtain accurate calculations of the area. This may take a lot of time to do,
especially when there are many CCTV cameras in an MRT station; it is however, a necessary step.
Finally, the coverage ratio can be determined by dividing the overall floor area in the MRT station by the total
area of all filled regions. Then the overall situation of the CCTV coverage in the MRT station can be displayed
94

on both 2D floor plan views and 3D model views. Fig. 6 and Fig. 7 show the CCTV coverage in 2D floor plan
views in the concourse level and in the platform level respectively. Fig. 8 and Fig. 9 show the CCTV coverage in
3D model views in the concourse level and in the platform level respectively.

Fig. 6: The CCTV coverage of the concourse level shown in a 2D floor plan view
(Coverage ratio = 25.73%)

Fig. 7: The CCTV coverage of the platform level shown in a 2D floor plan view

Fig. 8: The CCTV coverage of the concourse level shown in a 3D model view
95

Fig. 9: The CCTV coverage of the platform level shown in a 3D model view
3.2 Checking for design conflicts
Prior to the use of BIM, interferences between systems had many difficulties to overcome and were
time-consuming. BIM technology is now likely to detect clashes in advance through more sensible visual
presentations and more seamless collaboration platforms. (Eastman et al., 2011: 272-273)
One of the aims of this research is to inspect whether a desired location in which a CCTV camera is installed
would be a suitable one in the future. Now that CCTV screen views can be simulated via Revit cameras, it is
simple to check if anything blocks the CCTV camera projection in the BIM model. Examples of these are smoke
curtains and ceiling sign boards, which can be modeled as parametric elements in the BIM model of the MRT
station. Such design conflicts must be discovered in advance and eliminated to reduce the construction time
required to set up CCTV cameras.
When considering this topic, it is difficult to give a precise definition of design conflicts due to the fact that how
seriously something blocks the CCTV camera projection depends on subjective assessment by the operators. A
more objective way to evaluate whether a there is a clash is to use the optimized real angle of depression
suggested by the CCTV setup advisor. Once an optimized real angle of depression is applied, most interference
can be avoided. If there is still an obstacle blocking the CCTV camera projection, this implies that the
parameters of the CCTV camera should be reset, including the camera height, target height, FOV, or even its
original location.
Fig. 10 shows a virtual CCTV screen view where almost half of the view is blocked by smoke curtains and
ceilings. After adjusting several parameters of the Revit camera with the assistance of the CCTV setup advisor, a
clearer view can be obtained as shown in Fig. 11. The setting up of a CCTV camera must be based on users
requirements. The simulation of CCTV screen views offers an approach for CCTV designers to conduct both
detailed modifications and subtle adjustments prior to the real construction.
96

Fig. 10: A virtual CCTV screen view before the
checking of design conflicts
Fig. 11: A virtual CCTV screen view after the
checking of design conflicts
4. CONCULSIONS AND SUGGESTIONS
BIM models offer either a more obvious visualization (i.e., 3D instead of 2D) or a more organized management
(i.e., a collaboration platform); therefore, it is possible to reduce most of the associated uncertainties time and
time again with model reviews in virtual space. When CCTV systems are designed and planned in accordance
with other room and spatial requirements, BIM models can at least provide owners with an overall blueprint in a
virtual space by reviewing CCTV coverage and simulated CCTV screen views.
This research has outlined an approach to modeling CCTV systems in virtual building spaces. The emphasis was
placed on offering a visual representation of the CCTV coverage in a BIM-based virtual environment. By
simulating varifocal lenses and configuring the parameters of Revit cameras, the simulated virtual CCTV screen
views provide a better visual demonstration of the working system. This is advantageous in the checking of
design conflicts and effective communication between owners and contractors. The filled regions in the 3D
environment are also more apparent, allowing a more accurate visual evaluation of CCTV coverage. Additionally,
the plug-in program developed by Revit API, (i.e., CCTV setup advisor) is capable of processing repetitive tasks
such as inputting the values of parameters. Finally, in the case study of an MRT station, all the proposed methods
were examined and were shown to enhance the performance and efficiency.
As spatial requirements vary from case to case, it is desirable to make virtual spaces closer to reality and make
BIM models more comprehensive and sensible. Thus, the LOD (Level of development) of BIM models must be
higher than before. The higher the LOD that can be achieved, the lower the possibility of design alterations. That
is, a higher LOD results in a higher reliability of predesigned CCTV systems and their simulations.
5. ACKNOWLEDGEMENT
We wish to thank for the following individuals for their generous assistance. Firstly, our research was greatly
facilitated by numerous government agencies; we thank them all sincerely. Our thanks and appreciation for
highly professional work are due to the National Science Council, Taiwan, who granted us a College Student
Research Training Fellowship and due to the Department of Rapid Transit Systems, Taipei City Government,
who granted us all related data, drawings, and materials needed for the research. Secondly, we would like to
show gratitude to Sinotech Engineering Consultants, Ltd. for offering us a complete BIM model of an MRT
station as our case study. Finally, we are grateful to ST Electronics (Taiwan) Limited, who supported with
technical assistance of CCTV systems and its practical experiences.
6. REFERENCES
Autodesk, Inc. (2010a). Revit Architecture 2011 Users Guide. The United States: Autodesk, Inc.
Last accessed 13th Sep 2012.
Available: http://images.autodesk.com/adsk/files/revit_architecture_2011_user_guide_en.pdf.
97

Autodesk, Inc. (2010b). Revit 2011 API Develpers Guide. The United States: Autodesk, Inc.
Last accessed 13th Sep 2012.
Available: http://images.autodesk.com/adsk/files/revit2011sdk0.exe.
CCTV Advisory Service. (2008). LensCalc - online lens calculator. Last accessed 13th Sep 2012.
Available: http://www.cctv-information.co.uk/lenscalc/.
Cohen, N., Gattuso, J., MacLennan-Brown, K. (2009). CCTV Operational Requirements Manual 2009. The
United Kingdom: Home Office Scientific Development Branch. Last accessed 13th Sep 2012.
Available: http://www.nactso.gov.uk/SiteCollectionDocuments/ManagingTheRisk/
28_09_CCTV_OR_Manual2835.pdf.
Eastman, C., Liston, K., Sacks, R., Teicholz, P. (2011). BIM Handbook: A guide to building information modeling
for owners, managers, designers, engineers & contractors. 2nd ed. The United States: John Wiley & Sons.

98

A 3D-WEB ENABLED FRAMEWORK FOR DELIVERING DIGITAL
FACILITY ASSET MANAGEMENT
Edvinas Rasys & Michael Hodds
Centre for Construction Innovation & Research, Teesside University, UK;
N.N. Dawood
CCIR (Centre for Construction Innovation & Research), Teesside University, UK;
ABSTRACT: Efficient digital asset management is crucial to the successful execution of projects in the oil and gas,
and civil engineering sectors. As information is dispersed between different stakeholders during the project
lifecycle stages, exchange of the project data often involves diverse standards, protocols and business practices.
Cloud based web applications have recently shown a big progress in providing a quick and convenient access to
data regardless of the users location.
Multiple methods and frameworks exist for delivering 3D content on the web, but a single globally accepted
standard has yet to be developed. Most modern web browsers support WebGL hardware accelerated rendering of
3D content, but there is still a gap in transferring model data from the design applications to a format acceptable
by WebGL.
The framework proposed in this paper addresses several issues in collaboratively managing engineering
information. Class library based storage and information exchange dynamically defines the data model, and
allows limiting and transforming the flow of information from different project stakeholders. An efficient and
flexible data storage layer allows context and area based 3D information retrieval and loading alongside other
digital asset information. The selected Web3D framework enables the partial delivery of the 3D content to a web
browser; reducing the amount of data that the client has to parse.
The proposed integration framework is expected to be of value to oil and gas and civil engineering companies,
working with large amounts of data. 3D visualization can be used as another dimension when searching for
information. It enables training personnel without visiting the remote or not yet built facilities, planning
construction, maintenance or decommissioning work by providing relevant information regarding the facility
assets and opens prospects for additional augmented/mixed reality features.
KEYWORDS: project lifecycle, engineering information integration, Class Library, Web3D, WebGL
1. INTRODUCTION
It is common for the project contractor in the AEC and Oil & Gas industries to share information and
responsibilities with the owner and subcontractors (Schramm et al. 2010), which may be located in different parts
of the country or even different parts of the world. A multitude of 2D/3D modeling and data aggregation software
packages exist and each of them uses different internal data models, making data transfers nontrivial. Although
data modeling and exchange standards (e.g. ISO15926, IFC) exist, lots of proprietary file formats cause problems
when transferring data between different software packages and even tools from the same software vendor are not
guaranteed to provide seamless information exchange.
Most data warehouses, integration systems and BIM servers use relational databases (RDBMSs) for storing
project data. Their effectiveness is somewhat limited by the amount of data that the systems can store. While
transactional RDBMSs seem to be perfectly fit for online transaction processing solutions, data warehousing
systems often seem to require enormous computing resources to provide reasonable response times. As the
databases tend to be highly normalized, querying such systems becomes impractical and additional steps (e.g.
de-normalization, creating materialized views) are needed (Brazhnik and Jones, 2007).
Web is a universal platform, independent of the computer architecture, operating system used and even of the local
machine performance to some extent as well. Enterprises have capable web portals able to deliver engineering data,
documents, media files in a secure and convenient way (Samdani and Till, 2007) and the trend of moving from the
desktop to the web applications does not seem to be changing. The development of cloud infrastructure enables
companies to create flexible and scalable systems that can be accessed from all over the world on different devices.
99

3D on the web is still developing as there are multiple competing standards and proprietary solutions to deliver 3D
content to the client browsers. It is one of the key areas in providing visualization capabilities in oil and gas facility
management applications and BIM servers. It would enable better planning of work processes, automated
management capabilities, improvement in training and maintenance; while BIM users are waiting for tools that
would enable light-weight design anywhere capabilities on tablet and smart-phone application interfaces
(Eastman et al. 2011).
With those issues and challenges in mind this research project aims to create an integration framework that would:
be deployed on the cloud infrastructure and provide a web front-end interface,
utilize an efficient storage layer for storing different types of engineering data,
be driven by an ontology, which is called a Class Library in the field of oil and gas projects,
integrate a current Web3D technology to deliver the 3D content to the client browser.
The data model and data management approaches discussed in this paper have been derived from the oil & gas
engineering data management field, but the ideas can be easily applied to other applications, which are using
similar data modeling concepts.
The rest of the paper is dedicated to show the achievements in implementing the project objectives. An efficient
semi-structured data storage layer has been chosen to support the proposed framework and it has been shown to
cope better with a large dataset than a comparable RDBMS implementation. The class library implementation
provides a flexible way of defining project data requirements. A WebGL based technology has been taken as a base
for the further research and development of the 3D visualization implementation, which does not use browser
plugins.
2. FRAMEWORK
2.1 Semi-structured database layer
2.1.1 Data model
A new term NoSQL databases (Leavitt, 2010) has been gaining popularity and it refers to a range of data storage
systems, which do not implement the relational data model and do not support SQL query language. In this project
data consistency and availability are the most important properties as the system has to perform well with large
datasets and has to be designed to be scalable horizontally in a predictable manner.
One possible way to represent and store the object data is the JavaScript Object Notation (JSON). Such document
(also called semi-structured) storage makes the planning and development of the database schema easier as it
closely resembles the human readable representation of the data as shown in Table 1. Variable number of attributes
allows the system development to be aligned with the agile project management. The system can be started small
and provide basic functionality initially; over time additional functions can be added as the database layer can be
easily extended without taking the existing application offline. JSON is similar to XML as these formats are both
self-describing, but JSON is more compact and querying JSON objects does not need a separate language to access
the tree elements.
Table 1: Human readable and JSON representation of the data compared.
Human readable data JavaScript Object Notation (JSON)
Pump (P-101)
Class : Pump
ID : P-101
Description : Cold water pump, floor 2
Pressure Max: 80 psi
Type : electric
Voltage : 380 V

{ "ID" : " P-101",
"Name" : " Pump P-101",
"Description" : "Cold water pump, floor 2",
"Class" : "Pump",
"Attributes" : [
{Name : Pressure Max, Value : 80, UOM : psi"},
{Name : Type, Value : "electric},
{Name : Voltage, Value : 380, UOM : V"},
]}

With client JavaScript code already present on majority interactive web applications, usage of client side
JavaScript libraries like Backbone.js or Knockout.js simplifies the development of rich client interfaces. Having
this format at the storage layer, combined with a JavaScript based application server such as Node.js, means that
100

JSON format could be the only form of information needed in the whole path from the database to the client
browser. This saves a lot of development time as data can be extracted from the database, manipulated and passed
to the client without the need to change its format.
As JSON data format seems suitable for the database layer several JSON based databases were considered and one
called MongoDB (10gen Inc., 2012) was chosen as the storage engine. MongoDB supports multiple indexes and
has a query language similar to SQL. It is designed to be scalable and features like GridFS for storing large files
and geospatial indexing seem good candidates to use when developing 3D integration.
Modeling data in JSON format in MongoDB has its own advantages and disadvantages. Since the database lacks
the functionality of joins, it means that linking data together needs to be done at the application layer. This is
partially offset by the rich data model capabilities. As JSON supports hierarchical data structure with deep nesting,
some relationships can be embedded inside objects (see Figure 1).
This model represents commonly found entities in the oil and gas facility management applications. Functional
equipment Objects can have a variable number of attributes and represent equipment that is required to do certain
tasks pumps, valves, vessels, pipes, etc. Documents can be drawings containing Objects, data sheets, operation
manuals or some other form of electronic records. These entities can be linked, as Documents can contain
references to Objects and vice versa. Both Objects and Documents can have Attributes describing some
operational parameters or properties of the entities. Since usually there are a lot less Documents than Objects in the
system, their identifiers have been embedded in the Object JSON storage, but the relationship has to be modeled as
a separate table in the relational schema.
2.1.2 Performance benchmarks
Having modeled Object, Document and Attribute data in MongoDB, a comparable relational data model was
implemented in MySQL database. MySQL was chosen as a relational database for comparison, as it is also an open
source database; it is very widely used and can run on multiple operating systems.
Performance benchmarks were done on Amazons AWS cloud infrastructure - EC2 large machine instances, each
with 2 cores and 7.5GB of memory. As the performance of Elastic Block Storage (EBS) is not consistent, tests
were executed three times (on three different virtual machines) and the average results were calculated.

The operating system used was Ubuntu 11.10 64bit, MongoDB version was 2.0.2 and MySQL version was 5.1.58.
Every MongoDB collection and MySQL table had two indexes one for the primary key and one secondary index.
Test scenarios were created to emulate a large data process job for importing equipment Objects and their
Attributes into the system with a huge number of write and/or read operations taking place sequentially. For these
tests it was assumed that Object has 100 Attributes. As it was a prototype test data store model, the numbers should
not be treated as absolute values, but rather as relative performance indicators.
Fig. 1: JSON model (left) and relational schema (right) compared
101

To test the database write performance a single insert operation was executed for both MongoDB and MySQL
repeatedly. The horizontal axis represents the number of equipment Object records in the database. The graph (see
Figure 2) shows the consistent performance of both databases regardless of the number of records with MongoDB
being 3-4 times faster than MySQL:

Attributes were stored in the database as a separate process and the results there were very similar. Another test to
evaluate read performance a test was performed involving retrieving the Object along with its Attributes. For
MySQL a select operation using a join was involved, MongoDB in this case needs only one operation since the
Attributes are stored inside the Object data structure. The graph is different here as MySQL showed very good
performance until there were about 5 million object records (50 million Attributes). From that point the
performance decreased very fast. MongoDB showed a nearly linear performance decrease and outperformed
MySQL by a considerable margin on a larger scale.

The performance benchmarks proved that JSON is a feasible way of storing project data in the database, with the
MongoDB being faster than MySQL with large datasets. Thus MongoDB has been chosen as the database storage
engine for both data and the Class Library systems.
2.2 Class Library
The class library is an ontology that defines the information requirements for the safe and profitable operation of
the project equipment.
Class library information is hard to store to a fixed schema of a relational database effectively, due to classes
having highly varied information requirements.
Most class library data can be grouped into one of three distinct types of data: a means of identification, a means of
grouping with similar classes, and explicit links to other classes.
Fig. 2: Comparison of the database performance of the write operations
Fig. 3. Comparison of the database performance of the read operations using joins
102

Table 2: An example class, describing the attribute requirements for a pump. The attributes themselves are classes,
which identify acceptable units of measure, and could have other validation rules.
Human readable data JavaScript Object Notation (JSON)
Pump (P-101)
Class : Pump
ID : P-101
Description : Cold water pump, floor 2
Pressure Max: 80 psi
Type : electric
Voltage : 380 V

{ Alias: [
{ Label: "Class", Value : "Pump" }
{ Label: "Description", Value : "Cold water pump" }, ]
Association: [
{ Label: Type, Value: Functional Class }, ]
Mapping: {
Attributes : [
ID,
Description,
Pressure Max,
Type,
Voltage,
],
}}

By grouping data into the three distinct categories, a class can easily have additional data added without interfering
with the schema of other classes. Storing data in a long format as opposed to a wide, column-based approach
allows for much simpler and tidier presentation of class data, removing redundant gaps in information.

Class storage with a no SQL database such as MongoDB offers very high performance indexing, and allows
powerful queries to be built up easily. The query language permits matching individual fields such as an Alias
Value, or a combination of fields such as a Label - Value pair. Paired with the ability to use regular expressions for
value matches, full text searching over classes can be trivial to implement, and is easy to control.

In the example class, additional aliases could be added with domain specific means of identifying the pump.
Querying by an alias can require a specific label and value pair, or can require only a match by value, allowing text
searching of classes by all available aliases. Queries for association matches can combine multiple associations via
Boolean logic, to allow broad or highly specific groups of classes to be easily fetched.

2.3 3D model visualization
There are multiple ways to display 3D content inside web browsers. Most of them are implemented using browser
plugins (Behr et al. 2009), which create a separate virtual environment within a browser, isolate the DOM tree,
events and CSS features (Sons et al. 2010). WebGL is emerging as one of the most popular base frameworks for the
3D content presentation (Ortiz Jr., 2010), and is supported natively on several browsers (Chrome, Firefox, Opera,
Safari). WebGL itself is a low level JavaScript wrapper over OpenGL 2.0 ES functions, and this means it is quite
difficult to transfer content from the engineering design applications to the web applications utilizing WebGL.
Numerous libraries that use WebGL as their rendering engine have been created. They abstract the low level
functions into APIs, which deal with vertices, objects and scene graphs. While some of them try to tie the existing
standards, e.g., X3D (Behr et al. 2009), others deviate from that and focus on making the most of the underlying
platform and the rendering engine.
SceneJS is one of such libraries as its main data format is JSON. It creates a JSON scene graph which can be easily
manipulated by a JavaScript application engine and employs a lot of optimizations to increase the rendering
performance (SceneJS group, 2012). As the main framework engine is based on JavaScript, this seems like a
natural choice for the 3D model integration.
An open source BIM Surfer project, which uses SceneJS as the visualization engine, already exists (BIM network,
2011). While no known design or CAD applications produce SceneJS JSON file as the export format, another open
source BIMserver project (BIM network, 2012) can be used as transformation step. This allows taking a model
from a design application (i.e. REVIT), exporting it to IFC format, importing it into BIM server and exporting it as
the SceneJS JSON format, which is then accepted by the BIM Surfer (Figure 4):
103

While being a good starting point, the BIM surfer lacks the integration capabilities and does not provide additional
object properties for the 3D model components. The aim of this project is to link the 3D model component with the
additional engineering data coming from other sources. This should allow a two way communication between the
integration framework and the 3D model clicking on the item in the component list should highlight the select
object and vice versa selecting an item in the 3D model should bring up the data, associated data with that piece
of equipment (Figure 5):
It has also been observed that even specialized modeling software with dedicated professional graphics hardware
struggle to display large oil and gas facility models with good detail and reasonable refresh rates. There are
multiple ways of improving visualization experience. While some geometry simplification algorithms exist, the
authors of this paper propose splitting the 3D model into components and storing them separately. Since the target
users are not architects but rather plant operators, the business scenarios usually do not need the full facility view.
The usual approach to show a single component on the screen is to load the whole model up and then hide the
Fig. 4: Transfer of the 3D data of the model
Fig. 5: Different views of the same data
104

unneeded components (see Figure 6). A lot more efficient method would be to only load the component(s) that the
person is interested in such feature would make 3D visualization of the component even on portable devices with
simple graphical hardware.
3. DISCUSSION AND FUTURE WORK
Dynamic nature of the SceneJS scene graph means that it is possible to construct partial 3D models grouped by
systems, disciplines, locations, types of equipment, etc. Thus it will be possible to construct a view of all
equipment objects, belonging, for example, to a single discipline, even if the objects are not geometrically
connected or linked in some way on the model.
Since every node in the SceneJS scene graph is in JSON, it is possible to directly store each node in MongoDB and
index it by some of its attributes. Mapping between the IFC identifiers and Digital Object Identifiers (handles)
used in the integration framework will be needed. MongoDB also supports 2D geospatial indexing so queries
involving bounding box or circle selectors are implemented at the database level. Such feature would be very
useful when planning some construction, repair or maintenance work as it would allow easily selecting objects
which are close to a particular point in space.

There are several issues in WebGL 3D implementation. Since it is a JavaScript based library, it heavily depends on
the features and execution speed that the language provides. There are more efficient ways of structuring the 3D
content than a scene graph. Most game engines have their own 3D graphical engines, optimized for specific tasks
and graphics hardware. Scene graph does make sense on the web, as the browser DOM object itself is a tree
structure or graph. While this might not be the best performing 3D rendering approach, JavaScript performance is
increasing with each browser engine generation, so the speed is only expected to increase.

Some security flaws have been identified as an independent security company published a report (Forshaw, 2011)
pointing out security issues present in the current WebGL implementation, which allows malicious code to be run
and either make the machine unusable or put users data, privacy and security at risk. This issue has been
preventing Microsoft and some other major players from fully embracing WebGL. At the moment the only way to
add WebGL support to the Internet Explorer browsers is through the use of a plugin, but as the technology matures
and underlying issues are resolved, Microsoft will be not able to ignore the native browser 3D support for long.
4. CONCLUSIONS
The proposed framework based on a semi-structured database (MongoDB) has proven to be flexible and efficient.
JSON based data model enables storing, linking, manipulating and displaying various types of engineering
information for a rich and effective, 3D-capable digital facility asset management application.
Fig. 6: Current way of isolating components of the model
105

5. REFERENCES
10gen Inc. (2012). MongoDB. Available at: http://www.mongodb.org/ [Accessed: July 15, 2012].
BIM network (2011). BIM Surfer. Available at: http://bimsurfer.org/ [Accessed: July 15, 2012].
BIM network (2012). BIMserver. Available at: http://bimserver.org/ [Accessed: July 15, 2012].
Behr J., Eschler P., Jung Y. and Zollner M. (2009). X3DOM - A DOM-based HTML5/ X3D integration model.
Proceedings of Web3D 2009 The 14th International Conference on Web3D Technology, 127-135.
Brazhnik, O. and Jones, J.F. (2007). Anatomy of data integration. Journal of Biomedical Informatics, 40(3),
252-269.
Eastman C., Teicholz P., Sacks R. and Liston K. (2011). BIM Handbook: A Guide to Building Information
Modeling for Owners, Managers, Designers, Engineers and Contractors. L. Khelmani, ed., Wiley.
Forshaw, J. (2011). WebGL - A New Dimension for Browser Exploitation. Available at:
http://www.contextis.com/resources/blog/webgl/ [Accessed: July 15, 2012].
Leavitt, N. (2010). Will NoSQL Databases Live Up to Their Promise? Computer, 43(2), 12-14.
Ortiz Jr., S. (2010). Is 3D Finally Ready for the Web? Computer, 43(1), 14-16.
Samdani, K. and Till, A. (2007). Engineering portals add significant value to E&P project delivery capabilities.
World Oil, 228(11), 8.
SceneJS group (2012). 3D Scene Graph Engine for WebGL. Available at: http://scenejs.org/ [Accessed: July 15,
2012].
Schramm, C., Meiner, A. and Weidinger, G. (2010). Contracting strategies in the oil and gas industry. 3R
International, 1, 33-36.
Sons K., Klein F., Rubinstein D., Byelozyorov S. and Slusallek P. (2010). XML3D: interactive 3D graphics for the
web. Web3D 10 Proceedings of the 15th International Conference on Web 3D Technology. ACM, 175-184.

106

BUILDING-ENVELOPE-BASED INTEGRATION OF ENERGY SAVING
AND COST EFFECTIVENESS USING BIM
Po-HanChen, Chan Long & Jeong-Shing Lee
ABSTRACT: Green building, which refers to a building which is built without great consumption of natural and
energy resources during its life cycle, and such an idea is regarded to be adopted from the early stage such as
choosing materials and design to the stage of construction, management and even demolition. In other words, the
main difference between a traditional building and a green building is that a future building is expected to better
save the environment, reduce energy consumption and soon. To fulfill it, however, we need to pay for it, namely a
higher cost. Thus, it is considered that cost and benefit must be taken into account when we are going to make an
investment in new measures which improve the state of constructions. In this way, life-cycle costs analysis (LCCA)
can provide the investors or operators with the best suggestions over the cost benefit. It is all well known that the
consumption of energy is the second most expensive cost after a building is finished, and comes into operation, and
now we can use the financial profit from LCC to conserve energy like other green buildings in the market which are
eco-friendly. Also, what if the building owner has a limited budget or a requirement for the capability of the energy
saving system? Under these situations, which project should the owner choose to meet the cost benefit requirement
or to conserve the most energy efficiently? This research proposes to use BIM to make a construction model
first.Secondly, we use software to estimate the approximate cost of the building and analyze the yearly energy cost
through the modelautomatically. Thus, with these two things, we can obtain information about what are the best
components and materials for different building and provide owners with good advice.
KEYWORDS:BIM (Building Information Modeling)Life-cycle CostsEnergy Consumption CostsInitial
Construction CostsAutomation
1. INTRODUCTION
In Taiwan, area for new buildings every year averages a little more than 40 million meters square and life cycle of
them can be as long as 40 to 50 years. And from the researches on life cycle capital analysis, it was found that, for
most buildings, operation capital is placed first, responsible for 50% to 80%, and from which we can see how
important it is to keep the lift cycle capital under control and that is definitely the true value of the green building.
Moreover, designs for construction are miscellaneous, so the efficiency of the project highly depends on buildings
every single component such as equipments. In addition, some experts suggested that the energy conservation will
have a better efficiency if faade design of a building can be set as the top priority, and the bigger the scale of the
building is the energy conservation efficiency of equipment design and that of management project will become
better.
Energy conservation adopted in construction probably is the best weapon for environmentally sustainable cities
and improvement of living standard. From perspectives of experts around the world over the design of constructive
energy conservation, it is generally considered to be an object that is designed to reduce as much energy
consumption as possible by taking a variety of factors into consideration such as climate, ventilation, light source
and noise management so that everything surrounding is in favor of the building. And on account of the aforesaid
factors, materials needed for green buildings will be different from those needed for normal ones, for example,
materials for green buildings are more heat-isolated and light-source efficient and they are more expensive, so
green buildings will cost more than transitional buildings. From the angle of short-term benefits, such a cost rise
will not be favorable to green buildings. Having said that, green buildings also have their bright side which the
energy conservation efficiency in their life cycle period is significant that can even cover the initial cost rise. So it
is all about up-front cost versus life-cycle cost regarding the option between the green building and the traditional
building, and it is worth discussing how people make their discretion for the choice.
Nowadays constructive energy consumption simulators such as eQUEST, EnergyPlus and Ecotect can be used for
quantifying the energy consumption cost. Taking eQUEST for an example, it needs particulars such as 8760 hours
sunshine intensity, temperature, shape of buildings, interior space and nature of materials. In recent years, some
companies like TSMC have already advocated for using eQUEST for calculation of energy consumption, and 8760
hours TMY2 climate information has been collected in Taipei, Taichung, Kaohsiung and Hsinchu so it can be input
107

into eQUEST directly (LAM HIN DAK, WONG KOC CHIAN, 2004).
On the other hand, owing to the fact that constructive projects are becoming larger and more complicated, in order
to meet the demand of the design team for information management, technologies such as 3D, 4D and BIM
(Building Information Modeling) have been springing up and BIM is believed to be the trend for the future. And it
is also found that more and more constructive consultant companies, aboard and domestically, starts using
simulators for building assessment such as CECI and SIOTECH, both of them are using products developed by
Autodesk Revit, and some of companies such as TSMC even develops their own software system. Generally, the
application of the software ranges from cost assessment, space conflict analysis, arrangement, operation
simulation to constructive energy consumption. (Hartmann, T., J. Gao, ento al. , 2008)
2. BUILDING LIFE-CYCLE COST ANALYSIS
The analysis of life cycle cost mainly focuses on all costs and profits involved in the whole building life-cycle
period, estimating products price with a rather practical way. However, this method is not perfect yet as this kind
of technology is still not mature enough and it is little commonly used so far. The cost related to construction is
separated into three kinds in buildings life cycle(see Sieglinde Fuller 2005): initial construction cost, operation
cost (energy consumption and energy maintenance), demolition cost, replacement cost and residual values.
(seeSieglinde K Fuller et al.1995)At present, the main culprit for people to see if green building is worth being
invested in still highly depends on the extra cost needed for the construction stage, plus how efficient the energy
saving system is. (see Greg Kats et al. 2003) As a matter of fact, according to the information from studies over
the cost for building life cycle, the cost for constructing and maintaining are both accounting for the majority.
Moreover, as mentioned in GRAHAM IVEs thesis(see Graham Ive 2006), it is believed that provided that the
loan cost is excluded, then the operation cost will surpass the initial construction cost. This fact is also consistent
with the examples found in studies about building life cycle. So upon further analysis, it is found that after the
calculation of the cost, the ratio of the operation cost to the initial construction cost is between 1:3 and 1:1.5. (see
Sheet 1) Thus, this essay is going to probe into these two costs, pinpointing the problems so that we can have a
clear understanding of the value of green building.
Table 1Building life-cycle costs table
Author(Publication) Building Lifespan(years) Initial Costs Operation Costs Other Costs
Ya-Hui W (2005) School Building 47 26% 73% 1%
Chia-Ruei, C (2005) Stadium 55 53% 46% 1%
Yu-Ting, L (2006) Student Dorm 49 42% 55.5% 2.5%
Graham, I (2006) Commercial Office 20 25% 75% -
40% 60% -
Chun-Ta T et al. (2010) Government Offices 30 60%-80% 40%-20%
3. BUILDING INFORMATION MODELING(BIM)
BIM stands for Building information modeling and Building information model, a process which generates and
manages construction information during its life cycle, and this process will produce a BIM product which is an
object-orientated CAD system. The system embodies two concepts; they are virtual building and project
database. These two concepts additionally encompass building geometry, spatial relationships, geographic
information, quantities and properties of building components and all other information related to
construction.(see Ghang Lee et al. 2005, Ian Howell et al. 2005) As mentioned before, there is a fundamental
difference in design between a green building and a traditional building, so what customers are mainly concerned
about is the extra cost put in during the initial construction, such as setup of a sun shading board, modification of
window types and a heat-proof roof to name a few. Thus, we can work out the initial construction cost and other
construction aspects to get the best analysis through operating BIM.
108

4. ENERGY SIMULATION ANALYSIS
At the moment, software for energy simulation analysis, a kind of program which is mainly responsible for
anticipation of a building annual energy consumption and simulation of real situation of a building visually, has
been commonly used and commonly applied to early design of buildings.(see Richard Paradis 2010) Energy
simulation software will take a few aspects into consideration when it works, including a buildings geometric
shape, direction, materials, faade design, climatic information, interior circumstances, owners activities and
schedule, air conditioning, lighting system and all other information related to the building.(see Donald R.
Wulfinghoff et al.2009) Through the analysis, energy consumption and annual operation cost can be calculated
so that the best advise can be given.
The research will also use Revit API to integrate Autodesk Revit with other software to gain access of the
information to the objects which are entered into the software by users when they are using BIM to operate
Autodesk Revit such as the coordinate, geometry and space parameter. After that, a project with the lowest life
cycle cost or a compromise project will be generated from the integration and will be shown on Revits operation
interface so users can go for their favorite choice (see Fig1).
Fig.1 BIM to energy analysis diagram
5. COMPROMISE PROGRAMMING
In 1974, Yu and Leitmann conceived Compromise Decision, and it was later developed to Compromise
Programming for multi-criteria decision-making. Multi-criteria decision-making problem, in fact, is an
expansion of mono-criteria decision-making problem, and the distinction between them is that the multi-criteria
decision-making copes with at least two objective parameters at the same time, but the mono-criteria
decision-making merely focuses on one objective parameter, and both of them aims to work out the maximum or
the minimum optimal solution of their objective parameter in feasible solution region. And the formula below is
the mathematical representation of the multi-criteria decision-making problem:
M|n Z = { w
p n
=1
(
|
-
|
)
p
]
1
p
,
--------------------(eq. 1)
where
| represents n criteria or objectives
w
|
corresponds to a weight of a particular criteria or objective
p represent the importance of the maximal deviation form the ideal point( p =1,2, )
the best value for criteria , respectively(also referred to as positive ideals)
|
feasible solution
In this research, the distance parameter p will be set as 2 to weigh the distinction between the feasible solution and
the ideal solution for the purpose of users convenience to acquire the compromise solution by direct visual method.
The picture (see Fig.2)below is for showing the geometrical meaning thereof:
109

Fig.2 Compromise Programming geometrical diagram
6. CONCLUSION AND SUGGESTION
The gist of this research is to roughly estimate building life cycle cost by operating BIM information model plus
energy consumption analysis software (eQUEST) and to delve into the feasibility of the use of BIM for green
building design. Originally, the design of BIM software was for managing the data of entire building life cycle,
but at the moment it is only applied to building life cycle. In this research, we integrate and manage BIM data,
conducting a series of assessment from building design to operation management by using BIM 3D to show the
practical use of BIM in the entire life cycle.
6.1 Conclusion
The following are the main five conclusions from this research:
1. Designing green building more quickly with more cost-effectiveness
The system developed in the research can estimate construction cost and electricity consumption cost effectively
and quickly, so the time spent for analyzing building energy consumption can be reduced substantially and
buildings can be designed greener more quickly and cost-effectively. In addition, data is processed by the
system, so mistakes caused by data imputing will also plummet compared to manual data imputing.
2. Building life cycle cost analysis can be conducted more easily
From the reference about building life cycle cost analysis included in this research, we know that Initial
construction stage and Operation stage dominate the total cost of the whole building life cycle. Thus, in effect,
when analyzing life cycle cost of a building, more attention should be put into the aforementioned two stages
because once we know the costs for them, we can almost assess the entire building life cycle cost.
3. Life cycle cost analysis can be conducted more quickly by BIM
From the above conclusion, we know that we can estimate the rough cost of the entire life cycle cost from Initial
construction stage and Operation stage, and whats more, it is a piece of cake to work out the aforementioned
tow costs by BIM and energy consumption analysis software. Thus, the use of the database of BIM and the
energy consumption analysis software for life cycle cost analysis can save a plenty of time and reduce substantial
workload.
4. Through mathematical calculation, the optimal design for building projects can be found
In the past, life cycle cost analysis involved substantial calculations of parameters and it was time-consuming.
110

However, the system in this research estimates construction cost and energy cost that represent a high proportion
of life cycle cost through BIM and energy consumption analysis software, and simplifies the analysis process so
the workload is reduced and additionally feasible projects are found. Due to the fact that there are a lot of
feasible projects, compromise theory will be used to find 10 compromise projects that best fit the idea project.
After that we will calculate the life cycle cost of every project, so that the design project with the lowest life
cycle cost can be found.
6.2 Suggestion
Due to the limited time and the limited capacity of this research, there is still a plenty of theories which are not
mentioned or discussed yet in this thesis. As a result of that, we would like to make some suggestions for the future
researchers so that they can take them as reference when conducting a research of this field. And the following part
is the suggestions:
1. This research does not and it is also impossible for it to cover all the potential construction materials and
designs for construction design as new construction materials or construction designs keep being refined in the
market. But it is not a weighty matter as when there is any new component or construction design involved in life
cycle cost analysis in the future, just follow the same theory to build BIM model and the analysis can be carried
out.
2. Although the data needed for setting up the database in this research is based on the information quoted
from domestic theses and the practical data provided from domestic companies, technologies develop with rapid
progress. So, there is still a lot of room for improvement. In the future, it is hoped that the condition can be made to
fit the practical condition more by refining the system.
3. When the research is running and calculating the data of all feasible projects, if there are a lot of
information of construction materials or methods to be dealt with, the result of feasible projects can come up to
thirty thousand plus, and even though when it comes to finding the optimal project by using compromise theory, it
is also done according to feasible solution area, so it is time-consuming in spite of using programs. As a result of
that, if it is possible, Artificial Neural Network and Genetic Algorithms can be used for filtering the result first
before working out the feasible solution, so that the time for calculation can be reduced.
4. The research does not use the BIM data fully on grounds of limited time, for example: The area
estimation function of BIM: a model that can be used to find the carbon dioxide emission from a new building
during its construction stage. Or we can take more energy consumptions from other equipments such as elevators,
photocopying machine or other energy consumption equipments into consideration depending on different project
we are handling, so that the accuracy and the objectivity of the analysis for building life cycle cost can increase.
In addition, it is advised that the future researchers can conduct a data processing for more complicated building
designs, so that the system can cope with more types of building models.
7. REFERENCES
Amy S. Rushing, Joshua D. Kneifel, Barbara C. Lippiatt(2010), Energy Price Indices and Discount Factors for
Life-Cycle Cost Analysis 2010
Architecture and Building Research Institute, Ministry of the Interior, http://www.moi.gov.tw/english/index.aspx
Bansal,V. K. and Pal,M.(2008). Generating, Evaluating, and Visualizing Construction Schedule withGeographic
Information Systems, Journal of Computing in Civil Engineering, Vol. 22, No. 4, 233-242.
BIM and Cost Estimating, Autodesk,
http://www.consortech.com/bim2/documents/BIM_cost_estimating_EN.pdf
Chia-Ruei C (2005), Research on the life-cycle costs of gymnasiums - A case study on the gymnasiums of
National Taiwan University, NTU
Chau,K. W., Anson,M.and Zhang, J.P.(2004). Four-Dimensional Visualization of Construction Scheduling and
Site Utilization, Journal of Construction Engineering and Management, Vol. 130, No. 4, 598-606.
Chau,K. W., Anson,M.and Zhang, J.P.(2005a). 4D dynamic construction management and visualization software:
111

1. Development, Automation in Construction, Vol. 14, No. 4, 512-524.
Chau,K. W., Anson,M.anddeSaram, D.D.(2005b). 4D dynamic construction management and visualization
software: 2. Site trial, Automation in Construction, Vol. 14, No. 4, 525-536.
Chuah,Y.K., Lee, K.P.andLee,W.S.(2003). The Study of Air-Conditioner and Lighting Energy Saving Standard,
Air-Conditioners and Refrigerators, Vol.24, 99-104.
Crawley,D. B., Hand,J. W.,Kummert,M.and Griffith, B.T. (2008). Contrasting the capabilities of building energy
performance simulation programs, Building and Environment, Vol. 43, No.4, 661-673.
Charles J.K(2008), Sustainable Construction Green Building Design and Delivery, P174-175
Chuck E, Building Information Modeling, Digital Building Lab, http://bim.arch.gatech.edu/?id=402
Donald R.W, Prof. Rajan R, Dr. Vishal G, Dr. Jyotirmay M(2009), Energy Simulation, Energy Conservation
Building Code Tip Sheet , Version 1.0(Reprinted)
Fu,C., Aouad,G., Lee A., Marshall-Ponting A.J. and Wu, S.(2006). IFC model viewer to support nD model
application, Automation in Construction, Vol. 15, No. 2, 178-185.
GhangL , Rafael S , Charles M.E(2006), Specifying parametric building object behavior (BOB) for a building
information modeling system, Automation in Construction 15, 758 776
Greg K.E(2003), The Costs and Financial Benefits of Green Buildings, A Report to Californias Sustainable
Building Task Force, October
Graham I(2006), Re-examining the costs and value ratios of owning and occupying buildings, Building Research
& Information, 34:3, 230-245
Huang, J.R.(2006). Applying MD Model to Develop a Contract-Item Based Scheduling System for Construction
Projects, Master Thesis, National Cheng Kung University
Ian H and Bob B (2005), Building Information Modeling Two Years Later Huge Potential, Some Success and
Several Limitations
Jongeling,R.andOlofsson, T. (2007). A method for planning of work-flow by combined use of location-based
scheduling and 4D CAD, Automation in Construction, Vol. 16, No. 2, 189-198.
Koo B. and Fischer M. (2000). Feasibility Study of 4D CAD in Commercial Construction, Journal of
Construction Engineering and Management, Vol. 126, No. 4, 251-260.
Lauren B. R and VittalS.A(2011), Greening Project Management Practices for Sustainable Construction Journal
of Management in Engineering, Vol. 27, No. 1, pp. 48-57, January
Richard P(2010), Energy Analysis Tools, National Institute of Building Sciences,
http://www.wbdg.org/resources/energyanalysis.php
Multi-Objective Optimization, Wiki, http://en.wikipedia.org/wiki/Multi-objective_optimization
SieglindeK.F , Stephen R. P(1996), Life-Cycle Costing Manual for the Federal Energy Management Program
P1-2
SieglindeK.F(2010), Life-Cycle Cost Analysis (LCCA) http://www.wbdg.org/resources/lcca.php
Wiki Energy Conservation, 1995
Ya-Hui W(2005), Analysis on life-cycle cost of experimental school building in the university emphasis on
the stage of maintenance management, NTU
Yu-Ying L (2010), Life Cycle Cost Analysis and Maintenance Strategy for Student Dormitory Buildings, NTU
Yu-An T (2007), Operational Management of University Library Based on Life-Cycle Cost Analysis, NTU
112

BIM-BASED APPLICATION DEVELOPMENT OF UNDERGROUND
MRT STATION EMERGENCY EVACUATION-TIME EVALUATION
FOR TAIPEI MRT DESIGN CHECKING
C.H. Wang, R. Moh, S.M. Kang & W.L. Lee
Moh and Associates Inc., Taiwan
T.J. Pei & S.H. Hsieh
Department of Civil Engineering, National Taiwan University, Taiwan
ABSTRACT: The facility capacity (number of stairways, escalators, platform width, e-ticketing gates, etc) and
evacuation time of a MRT station are two key requirements that need to be determined prior to carrying out the
subsequent detailed design of the MRT station. The facility capacity is determined during the planning/basic
design stage based on the estimated annual passenger volume of the station. With the determined facility
capacity, it is then required to carry out design checking for the emergency evacuation time needed in the MRT
station during the subsequent design stages. The process must be repeated every time a change order occurs.
To meet the objective in improving design efficiency and lower cost due to repeated works, Moh and Associates,
Inc. (MAA) engaged with the Research Center for Building & Infrastructure Information Modeling and
Management in the National Taiwan University to develop the Underground MRT Station Emergency
Evacuation-Time Evaluation application through the application programming interface (API) provided by
Autodesks Revit Architecture. With the BIM model as the base model of an MRT underground station, the
BIM-based application captures the necessary facility capacity information from the model and through
programmed formulas, emergency evacuation-time evaluation reports are generated automatically. This
software development has met the objective of cost reduction due to repeated works and improved overall design
quality.
KEYWORDS: BIM API, Facility Capacity, MRT Station, Emergency Evacuation
1. INTRODUCTION
The adoption and application of Building Information Modeling (BIM) has become the new trend in the
construction industry. With the advancement of computer and software technologies, information from various
architecture and engineering disciplines can be integrated throughout a project lifecycle, thereby decreasing
mistakes and misinterpretation of information flow and interface management among all stakeholders. In
addition, BIM model allows for easier 3D virtualization of design interfaces, digitalization of architecture
information, and design drawing production, which results in improved project cost, schedule and quality control.
Using BIM model as the base, design efficiency can be further enhanced through the programming development
using application programming interface (API).
The design of MRT Stations is a complex process that requires a close collaboration among many engineering
disciplines and architects. The complex interface management among the design team and the demands of the
client often results in repeated design work, which is often unavoidable. For every repeated design process, the
facility capacity and emergency evacuation-time evaluation must be recalculated to ensure the design meets the
project capacity and safety requirements, which results in large amounts of working hours. To improve the
work efficiency and to improve the design process, Moh and Associates, Inc. (MAA) and the Research Center
for Building & Infrastructure Information Modeling and Management of the National Taiwan University (NTU)
developed the BIM-based Underground MRT Station Emergency Evacuation-Time Evaluation application
(called EETE) using the API of Autodesks Revit Architecture.
2. DEVELOPMENT METHODOLOGY
2.1 Objectives
The facility capacity (number of stairways, escalators, platform width, e-ticketing gates, etc) and evacuation time
of a MRT station are two key requirements that need to be determined prior to carrying out the subsequent
detailed design of the MRT station. The facility capacity is determined during the planning/basic design stage
113

based on the estimated annual passenger volume of the station. With the determined facility capacity, it is then
required to carry out design checking for the emergency evacuation time needed in the MRT station during the
subsequent design stages. The design check process must be repeated every time a change order occurs. The
development of EETE aims to reduce the working hours and improve overall design quality during each design
checking process. Specifically EETE aims to achieve:
1. Capturing facility capacity requirements from the BIM model prior to design phase
2. Carrying out automated design checking of Emergency Evacuation-Time during each stage of
design (including change orders)
3. Generating Emergency Evacuation-Time Evaluation report during each stage of design (including
change orders)
The development concept is described in Figure 1.

Figure 1 API-based EETE development concept
2.2 Development Contents
EETE is developed for design checking usage during detailed design phase of the MRT station design practice in
Taiwan. EETE is a plug-in based on the API of Revit Architecture 2012. EETE comprises of two main
parts, inputting pre-design requirements and extracting post-design parameters, of which both are compared
through the design checking report. Pre-design requirements are calculated through the combination of user
input of relevant design parameters and pre-programmed design specifications. Post-design parameters
produced for checking includes the following:
1. Facility Capacity and Emergency Evacuation-time parameters:
i. User input of the stations required annual passenger volume.
ii. Calculated station facility capacity (such as number of e-ticketing booths, entrance
doors, elevators, escalators, doors, public restrooms toilets, etc.).
iii. Station platform width and passenger capacity
iv. Emergency evacuation time
2. Station reverberation time.

114

Figure 2 EETEDevelopment Process
The relevant specifications used in EETE development are adopted from the 2004 System Planning Manual of
Taipei Metropolitan Mass Rapid Transit System: Volume 4 Fixed Facility Requirements publication by the
Department of Rapid Transit Systems of Taipei City Government. The detailed specifications are as follows:
1. Section 4.4.3 Transit Traveler: Ticketing Facility
Section 4.4.3.16 Ticketing Gate Requirements
2. Section 4.4.4 Station Design Specification: Platform Design Specification
Section 4.4.4.3 Platform Width Design Under Normal and Emergency Conditions
3. Section 4.4.4 Station Design Specification: Elevator and Escalator Design Specification
Section 4.4.4.28 Escalator Capacity Under Normal and Emergency Conditions
Section 4.4.4.35 Stairway Capacity Under Normal and Emergency Conditions (55 cm per step)
4. Section 4.4.5.16 Public Toilet Design Specification
5. Section 4.4.9 Evacuation in Public Areas
Section 4.4.9.3 Station Emergency Evacuation Time
Section 4.4.9.8 Emergency Evacuation Time Calculation Formula for Island and Side Platforms
Section 4.4.9.9 Evacuation time to emergency exit door from the lowest level in a multi-floor
underground station must be less than 4 minutes. For each extra floor, 2
minutes are added for evacuation time to safety area (for three level underground
stations, total evacuation time is 8 minutes)

3. EETE KEY DESCRIPTIONS
3.1 Component Selection and Parameter Input and Retrieval
In order for EETE to accurately retrieve and count the identified facility capacity (e-ticketing booths, ticketing
gates, escalators, elevators, public toilets, etc), it is a necessity to create facility capacity components in the BIM
model. Each component in the BIM model has its own parameters. EETE identifies these components
through the ID numbers that are specified in the BIM model.
3.2 Retrieval of Geometric Information
Platform width cannot be calculated nor retrieved directly from component parameters, instead it should be
retrieved through the use of geometrizing the three-dimensional BIM model. The geometrizing of the BIM
model begins with the identification of spatial boundaries of the station platform, followed with the identification
of facility capacities components location and geometric shape. Under the platform width specifications, the
distances between platform spatial boundaries and the facility capacities are then calculated. Figure 3
115

describes the platform width calculation.

Figure 3. Platform Width Calculation
3.3 Emergency Evacuation Time Estimation
To estimate the emergency evacuation time, the evacuation route must be first identified. The route is drawn as
a line function in the BIM model, as shown in Figure 4, in accordance to Section 4.4.9 Evacuation in Public
Areas of the 2004 System Planning Manual of Taipei Metropolitan Mass Rapid Transit System: Volume 4
Fixed Facility Requirements publication by the Department of Rapid Transit Systems of Taipei City
Government.

Figure 4. Emergency Evacuation Routes
3.4 Noise Absorption Estimation
According to the Taipei MRT specification, noise absorption formula is:
Noise absorption = Material surface area x noise absorption coefficient.
Surface area of identified material type can be calculated through the default function in Revit Architecture 2012.
With the noise absorption coefficient from the specification, the noise absorption can be calculated for the
various types of materials used in the station ceiling, wall and floor.
4. EETE API CALCULATION PROCESS
After the pre-design requirements (design parameter inputs and pre-programmed design specifications) are
completed, the post-design checking process involves the retrieval of relevant post design information and the
performance of emergency evacuation-time evaluation report. The programming language is C# in
the .NET environment. The complete process is described in the following steps:
A Emergency Evacuation Route
B Emergency Evacuation Route
C Emergency Evacuation Route
116

1. Define design parameters: input annual estimated passenger volume, platform parameters (see
figure 7), and calculate facility capacity requirements according to design specifications.
2. Compare the facility capacity retrieved from the BIM model with the required facility capacity
(defined as design parameters in step 1).
3. Create evacuation route and checking the evacuation time as regulated by the specifications.
4. Generate design checking report
Pre-design requirements can be completed before the BIM model is completed. For each version of BIM
model created during the design stage, facility capacities can be checked to ensure that the model meets the
design and facility capacity requirements as regulated in the specifications. Figure 5 shows the main page of
EETE. Figure 6 shows the inputs of estimated peak and off-peak passenger volume per hour and estimated
annual passenger volume. Figure 7 shows the required platform parameters including train headway interval,
platform width, passenger volume between stations, etc.

Figure 5 EETE Main Window

Figure 6 Input Window of Passenger Volume
117

Figure 7 Input Window of Platform Parameters
Based on the target estimated annual passenger volume, the required facility capacity is calculated and compared
to the facility capacity retrieved from the BIM model. The comparison is show in the EETEchecking table in
Figure 8. The right hand side buttons are for showing the detailed calculation process.

Figure 8 EETE Facility Capacity Checking Table
The minimum platform width can be calculated according to the annual estimated passenger volume. The
calculated minimum platform width is compared to the BIM models platform components width, allowing an
easy checking mechanism for any discrepancies to occur. The BIM model platform component is identified
with a categorized ID number for easy identification of its location and relevant information. This is shown in
Figure 9.
118

Figure 9 EETE Platform Width Calculation
After creating the evacuation route in the BIM model, the checking function in EETE will calculate the shortest
distance from the pre-identified starting point and exit point. EETE will also calculate the passing and waiting
time according to the passageway widths on each floor to come up with the total evacuation time. Figure 10
shows the evacuation time calculation.

Figure 10 EETE Calculation of Emergency Evacuation Time
Noise Absorption Estimation is based on the wall, ceiling, floor and opening areas retrieved from the BIM model
multiplied by absorption coefficients inputs in EETE. Figure 11 shows the Noise Absorption Estimation
calculation.
119

Figure 11 EETE Noise Absorption Estimation
Finally, all the design checking items are consolidated into a report that can be exported into Microsoft Word
files, as show in Figure 12. The report will act as the design checking for emergency evacuation-time
evaluation for the engineers and project managers. Any adjustment needed can be done on the BIM model and
rechecking can be easily re-performed through EETE.

Figure 12 EETE Design Checking Report
5. CONCLUSION
This paper describes the development of an API-aided design process in a BIM environment. It is an example
of how the combination of engineering, adoption of BIM and programming can help the design process to
maintain quality work in an efficient manner while decreasing unnecessary working hours and risks of mistakes.
The adoption of the EETE has helped the companys design team to focus more on important design issues.
6. REFERENCES
Department of Rapid Transit Systems of Taipei City Government, (2004), System Planning Manual of Taipei
Metropolitan Mass Rapid Transit System: System Facility Requirement, Vol.3 (in Chinese).
120

Department of Rapid Transit Systems of Taipei City Government, (2004), System Planning Manual of Taipei
Metropolitan Mass Rapid Transit System, Fixed Facility Requirements, Vol.4 (in Chinese).
Kang, S.M, Yang N.D Yang, and Moh Richard (2011), Application of Building Information Modeling on Mass
Rapid Transit Design (in Chinese).
Taipei City Government, Design Specification on System Noise and Vibration of Mass Rapid Transit System (in
Chinese).
Tsai S.L., Calculation of Basic Facility of the Mass Rapid Transit Station, (in Chinese).

121

INFORMATION CAPTURE AND REPRESENTATION OF
MODEL-BASED DESIGN REVIEW PROCESS IN VIRTUAL
ENVIRONMENTS
Li Wang, Junlin Yi & Fernanda Leite
University of Texas at Austin, United States
ABSTRACT: In recent years, the utilization of computer-aided information and communication technology such
as Building Information Modeling (BIM) has become a prevailing trend in the Architectural, Engineering and
Construction (AEC) industry. The data-richness nature of BIM enables creating, storing, processing and
representing parametric information as well as establishing connections between model components and related
information. Researchers have discussed BIM strengths in managing the product information, but little is
discussed regarding capturing and representing process information in a computer-interpretable manner, such as
notations and markups generated during the model-based design review process. Documentation of this process
information, if any, is usually conducted in the form of text, paper drawings with markups or digital snapshots of
model components with markups. While product information is well-represented in 3D or nD modeling, process
information representation remains in 2D. Design coordination is typically conducted in a model-based
environment, but since no linkage is established between review markups and model components, the knowledge
and information generated during the design review process cannot be used in further model-based analysis.
Therefore, this research aims to explore the possibility and potential benefits of capturing process information
generated in model-based design review and visualizing it in virtual environments. The envisioned model-based
design review environment enables efficient management of process information with model-based tag functions,
as well as searching and grouping functions. This research was conducted in three phases: requirements
elicitation, system design and prototype demonstration. The functional requirements of the envisioned system were
identified based on interviews with three BIM engineers and represented with Use Cases. A prototype system with
the desired functions was designed and implemented. The proposed system described in this paper indicates that
process information can be captured and represented with connections to the model, providing important insights
on innovative approaches for information capture and representation in model-based design review.
KEYWORDS: Building Information Modeling (BIM), Model-based Design Review, Information Capture and
Representation
1. INTRODUCTION
Historically, when design drawings were only drafted in 2D, the majority of Mechanical, Electrical, Plumbing and
Fire Protection (MEPF) coordination and design review was performed by overlaying two drawings on a light
table and contractor/subcontractors visually identifying constructability issues, which is known as Sequential
Composite Overlay Process (SCOP) (Korman et al. 2003). This 2D-based constructability review process is very
time-consuming and error-prone, and the information generated during the process is difficult to capture and store
for future use. A great portion of the construction knowledge is generated and used in the design review process,
which can be utilized as data sources for construction knowledge formalization (Wang and Leite 2012).
Construction knowledge formalization plays an important role in design-construction integration in the design
phase (Fischer 2006). Fischer and Tatum (1997) developed an expert system based on expert interviews. This type
of information capture has its disadvantages in large-scale implementation. In recent years, the utilization of
computer-aided information and communication technology such as Building Information Modeling (BIM) has
become a prevailing trend in the Architectural, Engineering and Construction (AEC) industry. The data-richness
nature of BIM enables creating, storing, processing and representing parametric information as well as establishing
connections between model components and related information (Eastman et al. 2008). With the assistance of BIM,
a huge amount of time can be saved by using automated clash detection to identify design conflicts. Researchers
and practitioners have widely discussed and utilized BIM in managing product information of a facility, but little is
discussed regarding capturing and representing process information such as discussions and solutions brought up
during the model-based design review process. Documentation of this process information, if any, is usually
conducted in the form of text, paper drawings with markups or digital snapshots of the model components along
with notes. While product information is well-represented in 3D or nD modeling, process information
representation remains in 2D. Nowadays, design coordination is typically conducted in a model-based
environment, but since no linkage is established between review markups and model components, the knowledge
122

and information generated during the design review process cannot be used in further model-based analysis.
This research aims at exploring the feasibility of capturing process information generated in model-based design
review through BIM and visualizing it in virtual environments. The envisioned model-based design review
environment not only enables efficient management of process information with advanced model-based tag
functions but also serves as a basis for semi-automated construction knowledge extraction and formalization which
will be the focus of a future phase of this research. This research proposes an information capture and
representation approach using BIM to map design review comments with the building model. A prototype
application was used to demonstrate the feasibility and advantages of our approach. This paper documents
research progress, including requirements elicitation, system design and a prototype demonstration.
2. MOTIVATION
Constructability review is an iterative process and can be considered as a source for gathering constructability data
and formalizing construction knowledge (Wang and Leite 2012). The information generated in the constructability
review process, if being documented properly, can be utilized as a basis for constructability knowledge
formalization and reasoning mechanisms development. Therefore, it is important for the construction team to
document constructability issues with explicit explanation of rules, as well as the comparison of the proposed
solutions with the original design. However, it has been observed that the information generated during the design
review process was either not documented or not properly documented (Wang and Leite 2012). In most cases, the
BIM engineer posed an issue and the construction team came up with a solution, which would be represented in an
updated model. As long as the clashes are resolved, what changes are made and why those changes are made are
not the questions valued by the construction team. Many changes were only represented in an updated model and
reside in the minds of associated participants. If knowledge is not documented, it is difficult to learn from it.
Detailed documentation of the current construability review process is essential for formalizing reasoning
mechanisms to reduce the reliance on tacit knowledge of the subcontractor in the design phase.
3. RELATED WORK
In the field of knowledge management, it is widely accepted that approximately 80% of what individuals know is
in the form of tacit knowledge (Nonaka and Takeuchi 1995). Hanlon and Sanvido (1995) further confirmed that
83% of the knowledge used in constructability reviews lies in the heads of experts participated in the process.
Explicit knowledge can be articulated in written form, such as guidelines and rules of thumb, while tacit
knowledge exists only in the heads of industry experts, mostly based on experience (Polanyi 1962, Nonaka and
Takeuchi 1995, Inkpen and Dinur 1998, Cook and Brown 1999). Explicit knowledge is easier to share and
communicate than tacit knowledge, while tacit knowledge is often the most valuable type of knowledge (Nonaka
and Takeuchi 1995). Only 50% of design firms and 10% of construction companies have a documented
constructability philosophy (Uhlik and Lores 1998; Arditi et al. 2002). The efforts of enhancing the
constructability of designs are usually by the use of some common approaches, such as of peer reviews, lessons
learned, brainstorming sessions and procedural guidance. The implementation of these approaches greatly relies
on expertise of individuals involved making the process challenging to generalize or automate.
In order to reach some extent of automation in constructability checking in design, constructability knowledge
needs to be formalized; important tacit knowledge needs to be transformed into explicit knowledge so as to be
easily conveyed and utilized. Expert systems were used to formalize constructability knowledge (Hendrickson et
al. 1987, Fisher and Rajan 1996), but these systems are cumbersome to apply if they are not explicitly related to a
computer-based representation of the facility design that is shared with the design team. Construction knowledge
was historically provided in various forms including formal means such as specifications, work procedures,
construction codes, task sequences and informal forms such as verbal suggestions, experience-driven
decision-making and evidences from on-site execution. Formal knowledge representations were mainly
paper-based documents and informal knowledge was seldom properly documented. Such scattered and implicit
representation of knowledge makes it difficult for knowledge formalization, integration or transformation. As
computer-based solutions are gradually replacing paper-based processes, information exchanging protocols are
also greatly altered. Therefore, several constructability review and construction planning systems have been
developed based on a product model. Examples include OARPLAN developed by Darwiche et al. (1988),
integrating CAD with construction-schedule generation (Cherneff et al. 1991), automating constructability
reasoning with a geometrical and topological project model (Fischer 1993), and COKE (Construction Knowledge
Expert) developed by Fischer and Tatum (1997). The fundamental challenge for the development of these
123

automated constructability review and improvement methods is to find an effective method for construction
knowledge extraction and formalization. Current approaches of knowledge formalization are mostly
manual-based, making it difficult to implement on large-scale projects, since it requires considerable manual
input upfront.
The review of background research and initial discussions with industry professionals indicated that, although
available, construction knowledge is very difficult to be formalized to the extent necessary to incorporate
explicitly in design models. Significant research is still needed for a new generation of techniques and tools with
the ability to automatically capture useful construction data for knowledge formalization and reasoning
mechanisms development so as to assist incorporation of semi-automated constructability checking in the
context of design solutions (Soibelman and Hyunjoo 2002, Fischer 2006).
4. RESEARCH APPROACH
This research was conducted in three phases: requirements elicitation, system design and prototype
demonstration. The functional requirements of the envisioned system were identified based on interviews with
three BIM engineers in two construction organizations, who have adequate experience using BIM for design
review on multiple projects. During the semi-structured interviews, the BIM engineers were asked to describe
the standard procedures and challenges of using BIM for design review including information preparation before
the coordination meetings, information management and documentation during coordination meetings, as well as
information update and maintenance after coordination meetings. The standard procedures explained by the BIM
engineers helped the authors understand the basic BIM-related tasks required for design review, and the
challenges identified revealed the limitations of current tools. Based on such information, functional
requirements of the envisioned system were identified and represented with Use Cases, which are commonly
used for capturing functional requirements in software and systems engineering (Wiegers 2003). A process
information capture and representation system with the desired functions was designed and prototyped as a
plugin of a widely used BIM application, Autodesk Navisworks Manage 2012. The plugin was developed using
Visual Studio C# 2008. The graphical user interface (GUI) was designed under .NET 3.5 Framework. The .NET
APIs of Autodesk Navisworks Manage 2012 were used to implement most functions relevant to connecting
models with process information generated during design review. Furthermore, the feasibility of capturing and
representing certain process information using the developed plugin was demonstrated on Windows 7 operating
system on personal computers (with 1.60-2.7GHz processor and 4-8 GB RAM), as well as on the high-resolution
touch screen system, Lasso, developed at the Texas Advance Computing Center (TACC)s Visualization
Laboratory. Lasso is a touch-sensitive tiled display that consists of six 46" monitors in a 2x3 grid configuration
at a resolution of 12.4 megapixels (5760x2160) and supports human-computer interaction utilizing a PQ Labs
32-point multi-touch infrared perimeter and a Microsoft Kinect (Westing et al. 2011).
5. PROCESS INFORMATION CAPTURE AND REPRESENTATION SYSTEM
This research aims at exploring the feasibility and potential of capturing information generated during the design
review process and representing such information in a computer-interpretable manner. The developed prototype
system is called TagPlus. This section presents the current research progress on the system development which
includes requirements elicitation, system design and prototype demonstration.
5.1 Requirements Elicitation
Based on the results of expert interviews, preferable functions include basic tag functions such as creating,
saving, editing and deleting tags, as well as more advanced functions such as establishing links between tags and
associated model components, semantic search of tags or comments, grouping tags with certain hierarchies and
exporting tag information for future use. These functional requirements are described using Use Cases, which are
used to identify the major tasks performed by users of a system (Wiegers 2003). In other words, Uses Cases
show how a system will/can be used. The Use Case Diagram (Fig. 1) illustrates the basic functions that the
system supports. In Unified Modeling Language (UML), actors are parties outside the system that interact with
the system, represented as a stick figure in the Use Case Diagram. A use case is a high-level description of an
activity that will be performed on a system, represented by an oval with the name of the use case inside. A
sequence of individual steps, also known as a flow of events, may be required for a use case. One use case may
involve several different sequences of steps, which are referred as scenarios. Connecting lines between actors
and use cases show which actors participate in a certain use case. In TagPlus, the process information
124

generated during design review is captured and represented with tags that can be directly linked to specific
model elements. The primary actors involved are specified as a user (the one who uses the TagPlus system,
usually a BIM engineer), tag library (the data structure that stores all tag information), and model that is under
review. The explanation of the main use cases shown in Fig.1 is provided as follows.
TagPlus System
Create Tag
Delete Tag
Save Tag
Link Tag to Model
Edit Tag
Search Tag
Export Tag Report
User
Model
<<includes >>
Tag Library
Change Comment
<<includes >>
Change Tag Name
<<includes>>
Group Tag

Fig. 1: Use case diagram for the TagPlus system
Create Tag: When there is a discussion or decision needs to be made regarding certain parts of the model, a tag
can be created. The user needs to input information such as tag name and comment; the date when a tag is
created is automatically documented; the interrelation between the tag and the model elements is specified by the
user by selecting certain piece(s) of the model when creating the tag. This information is automatically saved
when a tag is created. A detailed description of this use case is shown in Table 1.
Delete Tag: When a tag is no longer needed, it can be removed from the tag library. Select a certain tag and
remove it from the tag list.
Save Tag: The tag information can be saved as additional data in the model file so that the tag information will
not be lost when a user exits the system.
Edit Tag: The tag information can be modified after it is created or saved. This use case further includes two
subcases, change tag name and change comment. The revised information will then be updated in the tag
library.
Search Tag: The user may search for tags with certain tag name, keywords in the comments, specific created date
or specific model elements with tags. When the matching tags are found, they will be listed in the Active Tag
View window. If a tag is selected, the summary information (tag name, comment, number of associated
geometry and date created) of that particular tag will be presented with the associated assemblies highlighted in
the model. The user can also hide all other parts that are not highlighted in order to quickly locate the related
model elements.
Group Tag: For the convenience of the users when organizing the tags, the grouping function is provided so that
the user can merge several tags into one folder or combining several folders into a folder with higher hierarchy.
Export Tag Report: In addition to saving tag information as part of the model, this information can also be
125

exported and saved in other formats such as CSV. The user can select specific items to export with a particular
focus. Such external information storage may ease future manipulation or utilization of the data.
Table 1: Description of the Create Tag use case
Use Case Name Create Tag
Description User creates a tag by selecting related model elements, inputting tag name and comments, and saving
the above information as a tag. The system verifies the input information and interrelation between the
tag and the model and adds the new data item (newly created tag) into the existing tag library.
Actors User, model, tag library
Preconditions 1. A valid model is open.
2. The TagPlus system is successfully loaded.
Post-conditions 1. The tag library is updated with tag information added to the original model file.
2. The displaying color of the tagged model elements changes to yellow.
Normal Flow of
Events
1. User selects the model element(s) that he/she wants to tag;
2. User inputs tag name;
3. User inputs comments;
4. User submits the request of adding the new tag;
5. TagPlus validates the inputs;
6. TagPlus saves the tag to the existing tag library and changes the color of the tagged model elements
to yellow.
Alternative
Flows
1a. User fails to select any model elements;
2a. User fails to input a tag name;
3a. User fails to input any comments;
5a. No model selection is found;
5b. No tag name is found;
6a. The system sends a warning message saying No model item is selected and tag construction fails.
6b. The system sends a warning message saying No model item is selected and tag construction fails.

5.2 System Design
The proposed information capture and representation system with the functions identified above was prototyped
as a plugin of a widely used BIM application, Autodesk Navisworks Manage 2012. The plugin was developed
using Visual Studio C# 2008. Graphical User Interface (GUI) was designed under .NET 3.5 Framework. Besides,
Autodesk Navisworks Manage 2012s .NET APIs were used to implement most functions relevant to connecting
models with tags. As shown in Fig.2, the system architecture is mainly composed of three layers: (1) Graphical
User Interface (GUI), (2) Basic Data Structure of the Plugin, and (3) Store and Runtime Data Structures, which are
explained in details in the following sections.
126

Fig.2: Software architecture of the TagPlus system

5.2.1 Graphical User Interface (GUI)
The plugins GUI is designed within .NET 3.5 Framework. The Windows Form class of .NET is the basic
framework of the GUI, and all other sub window and menu list are created using build-in objects from .NET library
System. As is shown in Fig. 3, the plugin user interface is composed of three main components: Main Menu, Tag
Operation Region, and Active Tag View. The main menu offers operations like saving tags, searching tags, and
editing tags. Tag operation region is the most used part since all important tag operations are found here. Users can
create new tags, save and delete existing tags, and select a certain tag to review the tag information and have the
associated assemblies highlighted in the model. The Active Tag View region presents tag information according
to different viewing demands such as showing the relevant tags that are linked to specific model elements or
selected tags or search results or all tags.
127

Fig.3: GUI of the plugin

5.2.2 Basic Tag Data Structure of the Plugin
A new C# data structure tag is designed as the data structure to store all tag-related information. Its data members
include data types like strings (build-in data type of C#) and modelItemCollection (offered by Autodesk
Navisworks Manage 2012 API). Information like tag name, date created, and tag comment are stored as strings.
All models associated with the tag are stored as a collection in the data type modelItemCollection. To support
multiple tags, the authors used the generic data type List<type_name> of C# since it offers common array
operations such as add, remove, and can dynamically change its own length.
5.2.3 Store and Runtime Data Structures
The APIs of Autodesk Navisworks Manage 2012 plays an important role in connecting the models and the tags.
ModelItemCollection is a build-in class of the .NET API of Autodesk Navisworks Manage 2012. This data
structure allows the operations of reference to a collection of models, and thus it can be considered as a pointer to
an array of models. By making ModelItemCollection a data member of the tag class, model elements can be
directly connected to tags. More importantly, the TimelinerTask class was used to store the information of the tag.
Since all changes made by the users are within the temporary memory space of the plugin, which will be released
when plugin is closed, it is important to find a storage format that can be saved into files on disk. Ultimately,
TimelinerTask, which has data structures including a chain structure composed of strings and a
modelItemCollection, was used to store the tags. TimelinerTask is typically used to store the information of
construction sequences. In order to distinguish these two kinds of uses, a redundant string TAG was added to
the top of the string chain of TimelinerTasks that are used to save tags. Thus, the plugin can automatically detect
tags from all TimelinerTasks and generate a tag library.

5.3 Prototype Demonstration
The prototype plugin was demonstrated on Windows 7 operating system on personal computers and Lasso
(described previously). All functional requirements identified in section 5.1 were implemented. As compared to
the tag functions of current tools, the most important improvement of the proposed system is enabling the
connections between tag and associated model element(s). Consequently, the tags that contain process
information generated during the design review process can be stored as parametric information of the model
instead of static 2D snapshots that have no connections to the model. Changes that are made or need to be made
can be easily associated with model elements, which ease the efforts of organizing and managing the design
review comments by manually referencing the model. In addition, searching and grouping functions also
provides support to process information management in design review. Fig.4 shows an example of demonstrating
the search function with the prototype plugin. There are two tags already created in the model: tag 1 with
128

comment 1 and tag 2 with comment 2. If we perform a search of tags that contain 2 in the comment, the result
listed in the Active Tag View will just include tag 2, and the associated model element of tag 2 will be
highlighted in blue. Such search word can be changed to move or lift or any word that makes sense.

Fig.4: Snapshots of Search Comment function
The prototype plugin was also demonstrated on the touch screen system Lasso (Fig.5). Fig.5 shows an example
of selecting a wall assembly with the touch function for tag construction. The semantic tag information can be
input using the soft keyboard or writing pad supported by the Lasso system.

Fig.6: Prototype demonstration on the touch screen system
129

6. DISCUSSIONS AND CONCLUSIONS
This research aims at exploring the feasibility and potentials of capturing and representing process information in
the model-based design review process. Since the design review (also referred to as constructability review in
this paper) is an iterative process, the information (such as constructability issues identification and proposed
solution justification) generated during this process involves significant amount of construction knowledge
articulation, exchange and sharing. Such information, if documented properly, can be utilized as a basis for
constructability knowledge formalization and reasoning mechanisms development. Through a preliminary
prototype development and demonstration, this research initiates a new approach of capturing model-based
process information using BIM and an auxiliary tool with augmented tag functions. This section discusses the
potential benefits and applications of the proposed system, as well as the limitations of the TagPlus system at
its current state.
In current design review practice, process information is seldom documented formally. Such process information
may include detailed explanation of constructability issues, the steps taken to identify certain clashes, especially
soft clashes, and the causal relationship between clashes and proposed solutions. Effectively capturing this
process information is essential for developing structured construction knowledge and accurate reasoning
mechanisms for resolving various constructability issues. Current commercial software only supports creating
tags or notations on static viewpoints of the model, which are basically 2D screenshots. No connection is
established between the tags and the model elements, which makes it difficult, if not impossible, to develop
reasoning mechanisms using those tags and screenshots. The relationship between the proposed solution and
associated model elements needs to be studied and extracted as computer-interpretable rules so as to support
construction knowledge formalization and integration in early design. The system described in this paper
indicates that process information can be captured and represented with connections to the model, which serves
as a basis for advanced development of information capture tools with similar purposes. With the prototype
system, BIM engineers can explicitly visualize what model components a tag refers to and all the tags that are
attached to certain assemblies. They can also group and search for specific information as needed. The potential
of implementing the envisioned information capture system on a touch screen, which can be remotely accessed
with mobile devices such as smart phones or tablets, further strengthens its power of capturing information from
multiple participants through multiple sources in a collaborative virtual environment.
Since the system development is only at an early deployment phase and the focus of this paper is to explore the
possibility of the proposed information capture approach, the prototype system stills needs significant
improvements before it can be used as a data capture tool for construction knowledge formalization. The data to
be captured in design review needs to be specified, through additional expert interviews and case studies, which
will be the focus of a future phase of this research. In the current prototype, it is generalized as comment,
which needs to be classified into different categories of information or in terms of data requirements. However,
the results of this paper still provide important insights on innovative approaches for information capture and
representation in model-based design review.

7. REFERENCES
Arditi, D., Elhassan, A. and Toklu, Y. C. (2002) Constructability analysis in the design firm. Journal of
Construction Engineering and Management, ASCE, New York, NY, 128, 117126.
Cherneff, J., Logcher, R., and Sriram, D. (1991). Integrating CAD with construction-schedule generation.
Journal of Computing in Civil Engineering, 5(1), 64-84.
Cook, S. D., and Brown, J. S. (1999). Bridging epistemologies: The generative dance between organizational
knowledge and organizational knowing. Org. Sci., 10(4), 381400.
Darwiche, A., Levitt, R. E., and Hayes-Roth, B. (1988). OARPLAN: generating project plans by reasoning about
objects, actions and resources. AI EDAM, 2(3), 169-181.
130

Eastman, C., Teicholtz, P., Sacks, R. and Liston, K. (2008). BIM Handbook: A Guide to Building Information
Modeling for Owners, Managers, Designers, Engineers and Contractors. Hoboken, NJ, Wiley & Sons.
Fischer, M. (l993). "Automating constructability reasoning with a geometrical and topological project model."
Computing Systems in Engineering, 4(2-3), 179-192.
Fischer, M. (2006). Formalizing Construction Knowledge for Concurrent Performance-Based Design. In Smith,
I. (Ed.). Intelligent Computing in Engineering and Architecture, Springer, New York, NY, 186-205.
Fischer, M. and Tatum, C.B. (1997). Characteristics of Design-Relevant Constructibility Knowledge. J. Constr.
Eng. & Mgt., ASCE, New York, NY, 123(3), 253260.
Fisher, D.J. and Rajan, N. (1996). Automated constructibility analysis of work-zone traffic-control planning. J.
Constr. Eng. & Mgt., ASCE, New York, NY, 122(1), 3643.
Hanlon, E., and Sanvido, V. (1995). Constructability information classification scheme. J. Constr. Eng. & Mgt.,
ASCE, New York, NY, 121(4), 337345.
Hendrickson, C., Zozaya-Goranza, C., Rehak, D., Baracco-Miller, E. and Lim, P. (1987). Expert System for
Construction Planning. J. of Comp. in Civ. Eng., 1 (4), 253-269.
Inkpen, A. C., and Dinur, A. (1998). Knowledge management processes and international joint ventures. Org.
Sci., 9(4), 454468.
Nonaka, I. and Takeuchi, H. (1995). The Knowledge Creating Company: How Japanese Companies Create the
Dynamics of Innovation. Oxford University Press, New York, NY.
Polanyi, M. (1962). Personal knowledge-Towards a post critical philosophy. Routledge and Kegan Paul,
London.
Soibelman, L. and Hyunjoo, K. (2002). Data preparation process for construction knowledge generation
through knowledge discovery in databases. Journal of Computing in Civil Engineering, 16, 39.
Korman, T.M., Fischer, M., and Tatum, C.B., (2003). Knowledge and Reasoning for MEP Coordination. J.
Constr. Eng. & Mgt., ASCE, New York, NY, 129(6), 627-634.
Uhlik, F.T. and Lores, G.V. (1998). Assessment of constructability practices among general contractors.
Journal of Architectural Engineering, 4, 113123.
Wang, L. and Leite, F. (2012) Toward Process-aware Building Information Modeling for Dynamic Design and
Management of Construction Processes. In: Proceedings of the 19th Annual Workshop of the European Group
for Intelligent Computing in Engineering (EG-ICE). Munich, Germany: Technische Universitt Mnchen.
Westing, B., Urick, B., Esteva, M., Rojas F, and Xu, W. (2011). Integrating multi-touch in high-resolution
display environments. In State of the Practice Reports (SC '11). ACM, New York, NY, USA, Article 8 , 9 pages.
Wiegers, K. E. (2003). Software Requirements (2nd ed.). Redmond: Microsoft Press.

131

A BIM-BASED VISUALIZATION SYSTEM FOR STEEL ERECTION
PROJECTS SCHEDULING
Hsiang-Wei Lu, Wan-Li Lee, Chih-Chuan Lin & Po-Ming Huang
ABSTRACT: This article presents a practical system to develop scheduling for Structural steel erection projects.
The Visual ESS (Visual Management System for Erection of Steel Structures) system is based on BIM (Building
Information Model) techniques with a focus on context visualization of schedule. The system is aimed to improve
scalability and maintainability of data aggregation for designers, producers and managers of whom participate in
steel erection projects. The system has an integrated user interface that allows users to schedule data and develop
3D models of steel structures; it also can be used by project managers to schedule steel erection projects.
Application model for Visual ESS in construction processes is also being discussed in the later part of this article.
KEYWORDS: BIM, project scheduling, steel erection, interactive interface, Tekla Structures, structural Steel
1. PREFACE
The steel structure works can be divided into Steel Factory Manufacturing and On Site Installation, how to
manage the process of installation works and how the production process of steel factory information will affect
the project schedule, construction quality and cost. In a construction project before installation of Structural Steel,
the project manager has to prepare works as following:
Seeking Installation Group
Generating Installation Instruction
Preparing Erection and Installation Tools
Generating Delivery Planning
Generating Component Storage Plan
Generating Installation Process
Generating Quality Control Regulation
Generating Working Schedule
Throughout the duration of the project, the project manager has to constantly be communicating with
manufacturing, which is very tiring. With the responsibility to fulfill the scheduled deadline, the project manager
also has to make sure that, the structural steels are produced and delivered on time.
To finish these works precisely, the designers, contractor, project manager and structural steel manufactory have to
constantly communicate tiredly. The traditional working process was based on 2D shop drawings, worksheets,
emails, telephone and fax to communicate and confirm. A structural steel component needs many 2D diagrams (for
example plan, elevation and section) and refers to the installation process, which confirms the particular serial
number, model number and position. If we want to know the advance information, for example, the production
status, installation status or order status of any component, we have to use many interface and spend a lot of time.
After complex and repeated works, we are still not sure whether the information is correct or not. In fact, these
incidents of information going missing cause many engineering lawsuits. Engineering information is complex, and
if we follow the traditional way of transferring data, mans memory, cognition and translation will cause
insufficient and distorted information.
Aimed towards this issue, this article will talk about CECI How to use BIM technology to improve the production
and installation method of a Taiwan local structural steel manufactory. We developed an application program
called Visual ESSVisual Management System for Erection of Steel Structures). According to our experience of
the Visual ESS development, we show how to use BIM technology to integrate information of structure steel
installation works. We try to help the manufacturing and onsite construction group manage information while
132

working together so they can provide correct information for project managers, which help them make decisions.
These works improve working quality, reduce unscheduled delay and repeated works.
2. DESIGN OF THE VISUAL ESS SYSTEM
Visual ESS completely uses the extension and organization of BIMs (Building Information Modeling)
information structure, and tries to integrate all kinds of information in structural steel installations with a 3D model.
First, the system limits change the data structure in manufacturing side and construction side. Second, it develops
a framework that is extendable and maintainable. The user interface has to focus on visualization and interaction.
We tried to use interactive interface to arrange structural steel components in the 3D model. Therefore, the
manufacturing side and construction side can use 3D models to review and manage their works intuitively.
2.1 System framework
Visual ESS completely uses the extension and organization of BIMs (Building Information Modeling)
information structure, and tries to integrate all kinds of information in structural steel installations with a 3D model.
First, the system limits change the data structure in manufacturing side and construction side. Second, it develops
a framework that is extendable and maintainable. The user interface has to focus on visualization and interaction.
We tried to use interactive interface to arrange structural steel components in the 3D model. Therefore, the
manufacturing side and construction side can use 3D models to review and manage their works intuitively.

Fig. 1: Visual ESS system framework

2.2 Messaging Integration Mechanism
Visual ESS has an interchange with the structural steel processing system in manufacturing, which updates the
attribute of the structural steel component. Therefore, this requires the collaboration of the different operating
groups, such as public works, administrative, manufacturing or external teams over topics such as the design and
constructionthey follow as a short operation unit. In the "Flexible link" mode, the protocol interface and
information exchange mechanism allows for smooth access to all system information and achieves subsequent
expansion flexibility.
Visual ESS
Data Access
Components for
manufacturer
Data Access
Components
for 3D model
Construction side

Components of
3D Model
Heterogeneous /Remote system

Manufacture
System
Design side
Manufacture
side
Manufacture
processing
Database
Integrated
Database Data sync controller
133

2.2.1
2.2.2 Flexible Link
In order to find a location in the city, we habitually use a house address for positioning. Similarly, in searching for
an object in the BIM model, we will use the object ID as an index. BIM models need to integrate the 3D model of
the cooperative unit, which due to their respective requirements have also developed their own unique coding rules.
Imposed object identifier values are required to uphold the principles of the Uniform Code. The principle of the
fear of imperfection cannot be taken into too much consideration from the parties; if the two continuously debate
unit change in both IT core systems, the cost will be high.
Is there a better solution for these topics? Regarding the principle of unified coding, the process must be
compatible with both the coding rules of the collaborating units, and finally the development of a progressive
manner in order to reach the overall target applications. Visual ESS uses RDB (Relational Database), combined
with object-oriented technology interface concept, to design the flexibility link interface (hereafter referred to as
"flexible link"), see the build diagram shown below.

Fig. 2: Flexible Link
The flexible link meets the multiple response mechanisms for lifting the component identification value; its data
structures use a relational database that extend to many types, as shown in Figure 3. Collaborating units only
provide one or more elastic link protocol interface (see Section 2.2.2) while Visual ESS is easily integrated.

Fig. 3: Relation between Visual ESS and database
2.2.3 Flexible Link Protocol Interface
The design concept of flexible link protocol interface is based on the object-oriented technology, with related
regulated specifications and principles. The main protocols are summarized as follows:
a). To obtain the functions of the corresponding table of the latest lifting each other component identification
value; system to get the accuracy in order to protect the lifting components corresponding relationship.
Visual ESS
Cooperation Information
System
Installation
component 1
Installation
component 2
Installation
component 3
Component from
vendor A
Component from
vendor B
Component from
vendor C
Visual ESS
Flexible-link
interface

Synchronous Controller
Integrated
Database
Data access Components
Heterogeneous /Remote
system
Application
Document
Database
134

b). Installation components ID to obtain or store the information of the components in accordance with the
Visual ESS; identify the value of the conversion work to a cooperative unit to interface components deal with.
The major principles aforementioned in this agreement are important to maintain open communication to achieve
minimum requirements, to ensure the accuracy of the installation components, to identify the value of the
corresponding relations and to exchange information. The above-mentioned components depict an expanded way
to build with less impact on the operation of the existing system

2.3 Synchronization Mechanism
Visual ESS has a synchronization mechanism between the 3D model and production process data, which is
designed according to the 2.2.1 mentioned flexible link model. The synchronization relationship of whole system
was shown in Figure 4. Please refer to their specific designs in the following sections.

Fig. 4: Operation image

2.3.1 3D Model Synchronization Mechanism
The 3D model synchronization mechanism of Designer sides installation component is shown in Figure 5 and the
operation kernel of Visual ESS is BIM. Tekla Structures is currently used as a modeling tool. The model data
access components in the diagram is a flexible link protocol interface; its responsibility is obtaining the attribute
of installation component model, and to provide a synchronization controller for data synchronization.

Fig. 5: 3D model synchronization mechanism
Sync Controller
New or Update Model

Tekla Structures Open
API get components
detail
Installation
component
Building Model
(Tekla Structures)
Data access
Components
For 3D Model
Sync Database
Integrated
Database
135

2.3.2 Manufactories Data Synchronization Mechanism
Usually, processing system between Structural steel manufactory is different. Via Visual ESS "synchronization
controller, data can be synchronized with many manufactories data access components (flexible-link interface)
without affect to the operation of process systems of each manufacturer, as shown in Figure 6.

Fig. 6: Manufactories data synchronization mechanism
2.4 Development and Deployment Environment
Visual ESS was constructed on Microsoft Windows and Tekla Structures (Structural Steel modeling application
software), Microsoft Visual Studio, combined with the .Net Framework 3.5, Tekla Structures Open API (as 3D
model access program control interface) and the Microsoft SQL Server 2005.Deployed in the operating system, it
is required to install Microsoft Net Framework 3.5, Microsoft SQL Server 2005 database and Tekla Structures 16.0
(Viewer licensed version above).
3. THE USER INTERFACE DESIGN
The interactive design is to meet users' needs, and to develop new and appropriate use of experience, technology
and human seamless link. BIM technology is a very important issue, calling into question how to use the 3D
interface to provide visual information and correct intuitive operations, and to assist the user in management or
decision-making.
Visual ESS provides a 3D model interface (hereinafter referred to as 3D interface) and listed view interface
(hereinafter referred to as table interface). In a real-time interaction platform, users can directly select the
component in a 3D model, which information will be sync to the table interface for follow-up handler. On the other
hand, the user selected an item on table interface, which will sync a selected mark in model. At the same time, the
system will be based on different information or status attached to the lifting component, paint different colors in
the selected components in model to help users to intuitively grasp the lifting progress or other components of the
information contained in project. The following features are the main function of Visual ESS.
Sync Controller

A Manufacture
Data Access
Components
B Manufacture
Data Access
Components

A process system
for Hoisting

A Manufacture
process Database
B Manufacture
process Database
B process system
for Hoisting
Integrated
Database
136

3.1 The Installation Scheduling
Schedule planning is not only installation of the preliminary work of the project, but the implementation of the
works shall be updated at any time with the status of appropriate adjustments. Visual ESS installation schedule
management provides a real-time interactive 3D interface and the interface table columns, which uses different
colors to distinguish the component status between ready, installed or overdue on schedule. Due to that
interface, the project manager can focus on lifting schedule planning issues, simplify the traditional repeating
checking works on 2D drawings, control the lifting component coding table and memory scheduling component of
the job, to avoid artificial experience summary of integration possible negligence by the multi-information. The
interface is shown in Figure 7.

Fig. 7: Installation report (Scheduling)
In addition, the project manager can view the progress of the daily installation, as shown in Figure 8; this service
lists the date of completion and required component information, but also show the immediate installation state of
the component to the different colors displayed in the 3D model, so that project managers to quickly grasp the
progress of implementation.

Fig. 8: Installation daily report
Sync
137

3.2 Lifting Time Inquiry
The lifting time inquiry feature allows project managers to query lifting time of the component or display the
assemble status in any period, which help engineers catch up the lifting of progress and review of the lifting
operations.

Fig. 9: Lifting time inquiry interface

3.3 Production Status Inquiry
Visual ESS provides a feature that can trace the production status of component, so the project manager can
monitor the production status of components in manufacturing. The components may have the problems of
delivery or production; therefore, the project manager can adjust the schedule of lifting components or change the
lifting order to keep the project on schedule. The interface is shown in Figure 10.

Fig. 10: Production status inquiry interface
3.4 Delivery Planning Application and Management
Delivery planning is a key to link the factory and construction site. Many failures are caused by the lack of delivery
planning, the lack of the arrival report, and the repeated mistake of applying a missing component. To avoid these
Sync
Sync
138

failures, Visual ESS uses visual interface to integrate information, so project managers can catch up on problems
easily. These functions can be divided into two parts as follows:
In delivery planning application, the project manager wants to apply for shipments of installation components to
join the table interface, and display the component information immediately. At the same time, 3D interface
display components with different colors corresponding to the process status. Project manager verify that the
components are ready or not, and then apply for delivery. Subsequently, the system generates a delivery planning
application form in order to save time and avoid errors. The interface is shown in Figure 11.

Fig. 11: Delivery planning application interface
After the delivery planning management, the component will be delivered to the construction site, and construction
group have to report arrival status. Therefore, the project manager can handle the logistics of component easily.
The interface is shown in Figure 12.

Fig. 12: Delivery planning management interface
Sync
Sync
139

4. VISUAL ESS APPLIED TO OVERLAP THE ENGINEERING PROCESS
Traditional construction process is divided into design, construction and operation management in a linear way, as
shown in Figure 13. After BIM technology is applied, the construction threads can be overlapped, which means the
processing time can be compressed as shown in Figure 14. BIM benefited communication between construction
groups in an earlier stage of project, and construction groups can also get the point and detail via reviewing the
BIM model. Therefore, project quality can be improved and construction time can be shortened.

Fig. 13: Traditional working process

Fig. 14: Overlapping working process after BIM apply

Installation operation is a main item of construction work and critical path of construction projects schedule. If
BIM is applied into the construction process, which changes traditional working process from linear to nonlinear
at the end of the design stage, project manager can start to planning lifting cooperation, using Visual ESS
synchronization mechanism between design, process and lifting schedule, and assist the project manager making a
better decision under a visual and intuitive management interface, see Fig. 15.

Fig. 15: Visual ESS overlapping threads
5. CONCLUSION
This article used the development of the Visual ESS based on BIM technology as an example to discuss the
management level of structure steel installation and 3D model combined with the interactive user interface for
visual project management application.
BIM application development has two issues: a). Data synchronization between construction groups, this article
provides an idea called flexible link, which uses a proxy interface for data synchronization and system message
delivery. It effectively reduces the coupling between the different systems so the system can easily expand and be
maintained. The synchronized mechanism provides a flexible management interface between traditional and
visualized method. b). Working in 3D platform detailed information which can synchronize with traditional
engineering worksheets.
Design
Construction
Operations Management
Rusty Skilled
Construction
Operations Management
Design
Construction
Install
Design
Operation Management
Visual ESS
Data Sync
Coordination and management
140

This article uses Visual ESS interface design as an example and discussed practice lifting schedule management,
lifting time query, the process status inquiries, and the application and management of transportation plans. This
article using the perspective from construction project to explore the positioning of the Visual ESS and applied
nonlinear engineering process by a visual integration of the communication mechanism, which assisted the project
managers to make a better decision of scheduling in order to enhance lifting efficiency and reduce construction
costs.

6. REFERENCES
Public Construction Commission Executive Yuan, R.O.C. (2009) Construction and testing standard of structural
steel, http://erp.pcc.gov.tw/pccap2/BIZSfront/upload/article/c6c_6_2_8.pdf.
Charles M. Eastman, Chuck Eastman, Paul Teicholz, Rafael Sacks.(2008) BIM handbook: a guide to building
information modeling for owners, managers, designers, engineers, and contractors, John Wiley and Sons.
Succar, B. (2009) Building information modeling framework: A research and delivery foundation for industry
stakeholders, Automation in Construction, 18, 357-375.

141

INTEGRATION OF BIM AND CONSTRUCTION SIMULATION FOR
SCHEDULING
Ming-Tsung Lee
National Synchrotron Research Center, Taiwan
Cheng-Yi Chen & Shao-Wei Weng
National Chiao Tung University, Taiwan
Shih-Hsu Wang
R.O.C Military Academy, Taiwan
Wei-Chih Wang
National Chiao Tung University, Taiwan
Jang-Jeng Liu, Yuan-Yuan Cheng & Ming-Feng Yeh
National Synchrotron Research Center, Taiwan
Bing-Yi Lin & Cheng-Ju Kung
Concord Tech Co., Ltd, Taiwan
ABSTRACT: Numerous construction practitioners and researchers have been devoted to exploring the potential
applications of Building Information Model (BIM). Developing a four-dimensional (4D) model by linking the
BIMs three-dimensional (3D) components with activities from the schedule has been recognized as a promising
application. However, the current BIM-based 4D model is limited in not using BIMs capabilities effectively. This
study proposes a new model by using the BIMs quantity take-off results in building structure components (such
as reinforcing rebar, form, and concrete) to support site-level construction simulation and then generate an
activity-level project schedule. The resulted project schedule is next linked to the BIM components for 4D model.
Site-level resource competitions (such as crews and forms) and uncertainties of task durations are incorporated
into the construction simulation. The proposed model is applied to a reinforced concrete (RC) building project
located in northern Taiwan. The application results have demonstrated the model strengths.
KEYWORDS: Building Information Modeling (BIM), 4D model, construction simulation, project schedule.
1. INTRODUCTION
In the fields of construction and civil engineering, two-dimensional (2D) drawings have typically been used to
illustrate each phase of a construction project. However, as construction grows increasingly complex and
involves multiple elements, conventional 2D drawings are often unable to adequately express the ideas of the
designer or resolve problems between conflicting interfaces. Due to the further development of drawing
technology, computer aided design (CAD) has been upgraded from 2D to 3D models. The rapid advance of
information and communication technology has accelerated the electronization and digitization of construction
processes (Hsieh, 2010). With 3D models providing a basic framework, engineering data is integrated, digitized,
and expressed through building information modeling (BIM) a topic of growing interest in academia and the
engineering industry (Wu and Hsieh, 2008; Chen, 2009; Zhang and Hu, 2011). Combining BIM models and
project schedules into so-called 4D models has become an extremely popular topic in the field of BIM, because
4D models can contribute to schedule management through their approach to visualization (Wu and Hsieh, 2008;
Koo and Fisher, 2000). First, the 3D model and project schedule are produced (for example, Microsoft Projector
or Primavera P3 software can be used to generate relevant diagrams). Next, progress simulators (such as
SmarkPlaint Review or NavisWorks) are used to combine graphic components and scheduling elements resulting
in a 4D model (Koo and Fisher, 2000; Chen, 2009). As the project progresses, this 4D model employs animation
techniques to color-code finished and unfinished components, thereby illustrating how each element of the
project is progressing. Some researchers have proposed that managers can utilize 3D displays to examine the
logical order of each component and modify the project schedule as needed (Koo and Fisher, 2000; Zhang and
Hu, 2011).
Current BIM-based 4D models rely mainly on data from their 3D components; an approach that is similar to
previous 4D methods. This means that the functions of BIM in relation to calculating and storing data have not
142

yet been fully realized. Based on BIM calculations, this study develops a computer-simulated scheduling model
that integrates BIM with task-levels. This model considers the competition between human and mechanical
resources, factors in the uncertainty of construction time (expressed using statistical distribution), analyzes the
time required for project completion, and produces an activity-level project schedule (bar chart or network
diagram). The schedule is then integrated with the 3D components of BIM, to produce the 4D visualization
model.
This paper is organized as follows: Chapter 2 reviews 4D and STROBOSCOPE computer simulation; Chapter 3
proposes the integration of BIM and STROBOSCOPE. Chapter 4 applies the model to a case study of a
construction project in northern Taiwan. Research focuses on the structural engineering (including concrete
works and formwork engineering) of the project. The case study results verify the feasibility of the model
constructed by this study. Lastly, we provide recommendations for future research.
This chapter focuses on 4D models, BIM, and computer simulation for construction projects.
2.1 4D models
A 4D model integrates construction elements (3D) with the project schedule, and uses various visualization
methods to illustrate the temporal and spatial relationships among these elements. This visualized interface
provides managers with a considerable amount of supplementary information, which facilitates further analysis,
planning, and communication within the project (Hsieh, 2005). Many universities and research institutes have
participated in the development of Construction Simulation Toolkit, such as the combination of Walkthru and
Primavers P3 by Jacobus Technology, producing several types of commercial software. For example, the Center
for Integrated Facility Engineering (CIFE) used the Jacobus system to develop the 4D CAD system (Mckinney
et al., 1996McKinney and Fisher, 1998Koo and Fisher, 2000). Tsing-hua University in China produced the
4D-GCPSU system, which takes into account construction resources (Zhang et al., 2000; Chau et al., 2005). The
academic sector of Taiwan has also been engaged in developing 4D models. National Taiwan University (NTU)
developed a A4D model that is applied to structural steel and building structures (Hsieh, 2005; Lu, 2006), and
the VisPMIS project estimation system (Wu, 2007). Cheng Kung University proposed the concept of
multi-dimensional objects and conducted a series of 4D-related studies (Hsu, 2006; Chen, 2009; Feng, et al.,
2010).
Construction projects are usually lengthy in schedule and involve many operational elements. For this reason,
BIM, which is capable of storing and computing large amounts of data, was incorporated early into 4D
development (Tanyer and Aouad, 2005; Popov, 2010; Zhang and Hu, 2011). In terms of building a 4D model,
however, the main function of BIM is to provide 3D components, which is not unlike conventional 4D
approaches (although computational speeds are significantly higher). In other words, it appears that the data
processing capacities of BIM have not yet been fully utilized. Tanyer and Aouad (2005) identified many
weaknesses that must be overcome if 4D models are to be more widely implemented. One key difficulty is the
inability of current 4D models to incorporate the details and procedures of the construction site. This study
successful resolved this issue.
2.2 Computer simulation of construction operations
Computer simulation of construction operations can be used to solve risk-related problems in an uncertain
environment (Halpin and Riggs, 1992; Martinez, 1996). Computer simulation has been applied to scheduling and
costing problems at both the project and site levels. In 1996, based on the CYCLONE model (Cyclic Operations
Network)(Halpin and Riggs, 1992), Professor Julio Martinez of the University of Michigan developed
STROBOSCOPE (STate and ResOurce Based Simulation of COnstruction Processes). The main difference
between STROBOSCOPE and other CYCLONE-like software is that STROBOSCOPE mainly uses
programming language to simulate problems, rather than relying on modeling elements. Although this software
does provide interface programs for modeling elements (see Table 1), the elements are used mainly to aid users
in writing programs (Martinez, 1996; Wang et al. 2000).
STROBOSCOPE provides two key simulation features: (1) It is capable of providing detailed records of both
countable and uncountable resources (such as equipment code, weight, horsepower, and unit price); (2) It is
capable of capturing relevant data from the dynamic simulation process, such as truck waiting times, frequency
143

of operations, and the timing of the last system login. The programming language of STROBOSCOPE comprises
element definitions, element attributes, and control elements (see Martinez (1996) for further details). In recent
years, CYCLONE and STROBOSCOPE have been integrated with 3D models, using visualization techniques to
plan for equipment, materials, and other elements of the construction process. This provides valuable insight into
decision-making with regard to site layout and equipment (Kamat and Martinez, 2007; Chen and Huang, 2011).
Table 1: Basic components of STROBOSCOPE
Graphic Name Function

Queue Resource waiting point

Normal
When the previous operation is completed, the task it represents can be commenced
immediately.

Combi The task cannot be commenced until the conditions set are satisfied.

Link
Connects the elements on the network diagram, and indicates the direction of the
flow of resources.
A

Assembler
Combines two or more different types of resources into a composite resource (such
as cement+ water+ aggregate concrete)

AssemblerLink Indicates that other types of resources are linked to or attached to this resource

Fork
Probability is used to determine successive activities (only one successive activity
can be selected at any one time)

Consolidator Resources will flow through once they have accumulated to a set quantity
Data source: (Martinez, 1996; Wang et al., 2000)
3. A SCHEDULING MODEL COMBINING BIM AND STROBOSCOPE
Fig. 1 presents the framework of the proposed model, comprising four parts: (1) construction of BIM model; (2)
Management Interface for the Estimation of Project Schedule System (MIEPSS); (3) computer simulation; and
(4) production of schedule and 4D model. The following sections provide a brief explanation of each component.
3.1 Construction of BIM model
Autodesk Revit Architecture is used to construct the BIM model. After the model is completed, amount data
such as the rebar, formwork, and the concrete used in columns, beams, walls, and slabs are recorded in the BIM
table (shown in Fig. 2). From this point on, the model exports quantitative tables (in .txt files, according to
building level and type) which are read by the Access database.
3.2 Management interface for the estimation of project schedule system
MIEPSS is the operational center of the entire model; developed using the Visual Basic. Its main function is to
integrate the information from BIM, the Access database, STROBOSCOPE, Excel, and MS-Project. The system
comprises two modules, the login screen and the main display. These modules, as well as the relationships
between this estimation system and each type of software, are illustrated in Fig. 1.
1. Login module: The main function is to export BIM calculations to the Access database to commence
integration. The integrated Access tables are then linked to the system to provide data for the estimation of
project schedule.
2. Main display module: There are three steps to the operation of this module: Input construction conditions,
estimate project schedule, and produce construction schedule.
(1) Input construction conditions: These conditions include unit time, formwork equipment, number of
workers, number of cranes, whether overtime is required, etc.
(2) Estimate project schedule: After information related to rebar, formwork, and concrete of columns,
beams, walls, and slabs is exported from the Access database, the model combines it with construction
144

condition to calculate optimal, most probable, and most pessimistic estimates of project schedule. For
example, the duration of material would be calculated by amount, which is taken out from Access
database, and the unit rate and number of crane that are defined by users (see equation 1). Moreover,
these estimates are inputted into the pre-defined computer simulation code, which is transmitted to
STROBOSCOPE as a .str file. Computer simulation is then commenced to calculate the overall project
schedule.
duration of material erection = amount of material / (unit rate of crane number of crane) (1)
(3) Produce project schedule: Simulated work time estimates are compiled to generate the project
schedule. The schedule displays data such as task name, time required for completion, start time, end
time, prerequisite tasks, float, average/minimum/maximum time resulting from 10,000 simulation
iterations; and waiting times for rebar workers, formwork workers, and cranes. Users then export the
schedule (Excel) from Microsoft Project software.

Fig. 1: Model framework

Fig. 2: BIM amount data
3.3 Computer simulation
Using the features of STROBOSCOPE computer simulation, this study conducted simulated analysis of
construction work. The data required for this simulation is as follows: (1) process network diagrams of
construction tasks for STROBOSCOPE simulation; (2) unit time for each worker involved in each task (the
amount of work that can be completed by one worker in a single unit of time); (3) the amount of materials
required for each task (such as the amount of column rebar required, in tones); (4) number of rebar and
formwork workers on each shift; (5) formwork quantity.
145

Through statistical distribution (such as the distributions of optimal, most probable, and most pessimistic time
estimates), STROBOSCOPE considers the influence of uncertainty on work time and analyzes the competition
between human and mechanical resources. Through multiple simulation cycles (such as 1,000 simulation
iterations), STROBOSCOPE can objectively estimate project schedule and automatically generate a probability
curve for project completion.
3.4 Production of schedule and 4D model
When a simulation has been completed and exported to Microsoft Project through Excel, managers can group
multiple tasks into (milestone) activities using MS Project. The 3D components of BIM are then integrated with
each task (or activity). The schedule is exported to NavisWorks and a 4D model is produced. The methodology
at this stage is similar to conventional methods.
4. CASE STUDY
The construction site in this study was in a science park in northern Taiwan. The construction project was a
two-level underground basement structure (built using reinforced concrete) with a raft foundation. The building
floor area was 14, 874 square meters, and the construction budget was approximately NTD 222 million. The
following sections explain how the model developed in this study was applied to this project.
4.1 Building the BIM model
The completed BIM model for the construction project is shown in Fig. 3. The model shows the length, width,
height, area, and volume of the rebar, formwork, and concrete for each element of construction (column, beam,
wall, and slab). The amounts for some elements can be directly calculated using model data. For example, the
number of reinforcing bars (rebar) for columns was calculated by estimating the rebar volume. The amount of
concrete required was obtained by calculating internal volume (=length x width x height). However, calculations
for some elements required supplementary information.
For example, the calculation of formwork quantity required knowledge of beam depth and the width of the
adjoining wall. Formwork quantity for slabs can be directly calculated from the slab area shown in the
model. Table 2 shows the BIM calculations for each element of construction. After the supplementary
information was inputted, we devised equations to enable automatic calculations, as shown in Table 2 (Chen,
2011). The formulas for this study did not consider protective cover and problems such as lap splicing and hooks.
Therefore, a degree of error is to be expected.
The quantity tables were exported to .txt files (through the export data function of the BIM model) and viewed
using Access. The quantity table for this case study is shown in Fig. 4.
4.2 Obtaining construction site information
Information was obtained from the construction company to simulate the construction site, including the
construction process, the number of workers, and unit time for each task, the quantity of materials for each task,
the number of cranes, and quantity of formwork.

Fig. 3: BIM model of case study
146

Table 2: Information required and equations for BIM calculations
Component Details
Data required for calculations
Equations
Data contained in model Supplementary data required
Column
Column rebar Estimate of rebar volume NA
Estimate of rebar volume 7.85 (convert weight)
Column formwork Length, width, height
Beam depth and width of
adjoining wall
(length 2 width 2 width of adjoining
wall) (length beam depth)
Column concrete Volume NA Volume
Wall
Wall rebar Estimate of rebar volume NA Estimate of rebar volume 7.85
Wall formwork area
Formwork single-sided or
double-sided
Area formwork single-side (or double-side)
Wall concrete volume NA Area
Beam
Beam rebar Estimate of rebar volume NA Estimate of rebar volume 7.85
Beam side
formwork
Width and formwork
removal length
NA
if(ground beam, 0 m, width formwork removal
length)
Beam bottom
formwork
Beam height and
formwork removal length
Slab thickness and
side formwork area
Formwork removal length (beam depthslab
thickness) side formwork area
Beam concrete Volume NA Area
Slab
Slab rebar Estimate of rebar volume NA Estimate of rebar volume 7.85
Slab formwork area NA if(bottom slab, 0 m, area)
Slab concrete Volume NA volume

Fig. 4: View of quantity table
4.3 Management interface for estimation of project schedule
After the quantity table was exported from BIM to the Access database, we were able to start the MIEPSS. Users
must input the project name (which is also the file name for the project database). According to the project name,
the system extracts quantitative data from the respective project database. Users then click the button to enter the
home page (as shown in Fig.5)
After entering the home page, users can input the unit time for each construction task (the most optimistic, most
probable, and most pessimistic estimates), formwork quantity, number of workers, number of cranes, and
whether overtime is required or not.
After Execute Computer Simulation is selected on the home page, the system automatically estimates project
schedule according to the table in Fig. 5, producing estimates for the most optimistic, most probable, and most
pessimistic length of time to completion. This information is then automatically transferred to the simulation
programming code. The programming code is stored as .str text files and activated using STROBOSCOPE.
147

Fig. 5: System home page
The construction project in question used conventional RC construction methods. Based on the construction
plans of contractors and sub-contractors, we developed a construction process, which was illustrated using
STROBOSCOPE diagrams (as shown in Fig. 6). This figure illustrates the logical order of construction tasks and
resources (such as the sequential relationship between formwork quantity and each construction task). See Chen
(2011) for further explanation on the various nodes in the diagram. The construction process can be briefly
summarized as follows:
1. The tasks on each level of the construction site included layout, column rebar hoisting, column rebar binding,
column formwork assemble, wall formwork assemble (front side), wall rebar hoisting, wall rebar binding,
beam bottom formwork assemble, beam side formwork assemble, slab formwork assemble, beam and slab
rebar hoisting, beam rebar binding, slab rebar binding, mechanical/ electrical/plumbing systems, supervision
and inspection, concrete pouring, and concrete curing).
2. There were six workgroups for formwork: Group 1 handled column formwork, Groups 2 and 3 handled wall
formwork (front and back), Group 4 was responsible for beam bottom formwork, Group 5 for beam side
formwork, and Group 6 for slab formwork. Rebar was handled by a single workgroup.
3. The work time for Groups 1-6 was pre-calculated using the unit time of each worker, the number of
formwork workers, and the formwork quantity. The work time for the rebar workgroup was calculated in a
similar fashion.
After STROBOSCOPE is activated, users need only click the Run Simulation button to commence computer
simulation. Users must manually save the simulation results in .txt format to the file path, which allows the
system to read and extract the data. In the case study, 1,000 simulation iterations required approximately 12.72
seconds.
When users return to the home page and click View Simulation Results, the system converts the .txt files of
simulation results into a visualized interface, as shown in Fig. 7. The outcome is the average value resulting from
1,000 simulation iterations, and includes the overall project schedule, start and end times for each construction
task, float, etc. The average project schedule for this case study was calculated at 215.33 days; the maximum
value was 236.81 days and the minimum value was 198.63 days. The results of the 1,000 STROBOSCOPE
simulation iterations are saved as .STO files. Users can open these files using Excel and convert the data into
cumulative probability curves for project schedule, as shown in Fig. 8.
4.4 Producing the schedule and 4D model
Users select Export Schedule to Microsoft Project in the lower right corner of the screen shown in Fig.8. The
system then automatically converts the schedule into Excel format and opens MS Project software. Users must
manually open the correct Excel file using the MS Project interface. The schedule can be modified, if so desired,
by grouping tasks into activities, to produce a schedule, as shown in Fig. 9. The construction schedule and BIM
3D model are both inputted into NavisWorks Manage software to complete the 4D model simulation.
148

FLOW
layout
START
qm1
column
rebar
binding
rebar
workers
qs1x
formwork
worker
wall
formwork
(front side)
qf2
wall
formwork
wall
rebar
binding
formwork
worker
column
formwork
assemble
qs1
column
formwork
qf1
qs2
beam
rebar
binding
qs2x
wall
formwork
(back side)
wall
formwork
qs3
beam
bottom
formwork
qf3
beam side
formwork
qf4
qf6
qf5
slab
formwork
ep1
beam
formwork
slab
formwork
formwork
worker
formwork
worker
formwork
worker
supervision
and
inspection
beam
formwork
concrete
pouring
pass
concrete
curing
1 day
concrete
curing
14 day
release
side
formworks
release
bottom
formwork
select
New area
END
crane
column
rebar
hoisting
cn1
wall
rebar
hoisting
cn2
beam and
slab rebar
hoisting
cn3
slab
rebar
binding
MEP
systems
qs4

Fig. 6: Construction process network diagram of case study

Fig. 7: View Simulation Results
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
200 205 210 215 220 225 230 235 240 245
A
c
c
u
m
u
l
a
t
i
o
n
P
r
o
b
a
b
i
l
i
t
y
(
%
)
Duration (Day)

Fig. 8: Cumulative probability curve for project schedule
149

Fig. 9: Project schedule produced using model
5. CONCLUSIONS AND RECOMMENDATIONS
This study developed an integrated BIM and computer simulation model. The model inputs BIM calculations
into the STROBOSCOPE simulation module. Total project schedule and the length of time required for each
construction task are determined using an objective simulation process. This information is then exported to
scheduling software to produce the project schedule. Lastly, NavisWorks is employed to view the 4D simulation
of the construction schedule.
Various decision-making parameters (such as formwork quantity and whether overtime is required or not) can be
modified in the system, which can help managers to devise better strategies and produce more effective
construction plans. As evidenced by the case study, the main contribution of our research was to demonstrate that
BIM-produced calculations can be inputted into computer simulation tools to aid contractors and managers in
scheduling construction. This strengthens the practical application and value of BIM models.
The system developed by this study still has a number of limitations, which may be remedied by future research:
(1) This study considered only RC engineering; future research could apply the model to other types of
construction (such as SC, SRC, precast concrete and partially recast concrete). (2) The model in this study was
applied only to structural engineering; future studies could apply the model to other phases of construction (such
as grading, excavation, ceilings, partitioning and landscaping) to build a more complete project schedule. (3)
Other catch-up strategies could be evaluated in computer simulation. For example, we might consider the
influence of various construction zones on overall project schedule. (4) The focus of this study was on the
project schedule; future studies could incorporate cost factors, facilitating simultaneous analysis of both cost and
scheduling issues. (5) The usability of the computer programs employed in the model could be improved.
6. ACKNOWLEDGMENTS
This study was funded by the National Science Council (project number: NSC100-2622-E-009-017-CC3). The
authors would also like to thank the following persons and organizations: Mr. Ping-I Lin and Mr. Chung-Chieh
Hsieh of Concord Tech for their support with BIM technology; Autodesk for providing BIM software, and Ms.
Mei-Yi Chu of Chiao-Tung University for her assistance in constructing BIM models. We are also grateful for
the valuable information and experience offered by the engineers and other professionals involved in the case
study.
7. REFERENCES
Chau K. W., Anson M., and Zhang J. P. (2005). 4D dynamic construction management and visualization
software: 1. Development, Automation in Construction, Vol. 14, No. 4, 512-524.
Chen C. Y. (2011). Applying BIM and Simulation to Schedule Construction Projects, Master's Thesis,
Department of Civil Engineering, National Chiao Tung University, Hsinchu, Taiwan.
Chen H. M. and Huang P. H. (2011). 3D AR-based Simulation for Construction Operation, Proceedings of the
Conference on Computer Applications in Civil and Hydraulic Engineering (CCACHE 2011), Kaohsiung,
Taiwan, 148-157.
150

Chen, Y. J. (2009). The Development of Projects Information Model and the Applications of PIM Model-based
System, Doctoral Dissertation, Department of Civil Engineering, National Cheng Kung University, Tainan,
Taiwan.
Feng C. W., Chen Y. J., and Huang J. R. (2010). Using the MD CAD model to develop the time-cost integrated
schedule for construction projects, Automation in Construction, Vol. 19, No. 3, 347-356.
Halpin D. W. and Riggs L. S. (1992). Planning and Analysis of Construction Operations, John Wiley & Sons,
New York.
Hsieh D. Y. (2005). 4D Simulation System for Steel Structure Construction, Master's Thesis, Department of
Civil Engineering, National Taiwan University, Taipei, Taiwan.
Hsieh S. H. (2010). Special Issues on BIM, The Magazine of The Chinese Institute of Civil and Hydraulic
Engineering, Vol. 37, No. 5, 6.
Hsu S. K. (2006). Using Multi-Dimensional Graphic Components to Generate the Schedule for the Construction
Project, Master's Thesis, Department of Civil Engineering, National Cheng Kung University, Tainan, Taiwan.
Kamat V. R. and Martinez J. C. (2007). Variable-speed object motion in 3D visualizations of discrete-event
construction simulation models, Electronic Journal of Information Technology in Construction, Vol. 12,
293-305.
Koo B. and Fisher M. (2000). Feasibility study of 4D CAD in commercial construction, Journal of Construction
Engineering and Management, Vol. 126, No. 4, 251-260.
Lu M. C. (2006). 4D Simulation System for Building Construction, Master's Thesis, Department of Civil
Engineering, National Taiwan University, Taipei, Taiwan.
Martinez J. C. (1996), STROBOSCOPEState and Resource Based Simulation of Construction Processes, Ph.D.
Dissertation, University of Michigan, Ann Arbor, Michigan.
McKinney K. and Fisher M. (1998). Generating, evaluating and visualizing construction schedules with CAD
tools, Automation in Construction, Vol. 7, No. 6, 433447.
McKinney K., Kim J., Fischer M., and Howard C. (1996). Interactive 4D-CAD, Proceedings of the Third
Congress on Computing in Civil Engineering, ASCE, Anaheim, CA, 383-389.
Popov V., Juocevicius V., Migilinskas D., Ustinovichius L., and Mikalauskas S. (2010). The use of a virtual
building design and construction model for developing an effective project concept in 5D environment,
Automation in Construction, Vol. 19, No. 1, 66-72.
Tanyer, A. M. and Aouad, G. (2005). Moving beyond the fourth dimension with an IFC-based single project,
Automation in Construction, Vol. 14, No. 1, 15-32.
Wang W. C., Liu J. J., and Wu C. H. (2000). A Novel Computer Simulation Syntax in Construction
STROBOSCOPE, Construction Management Association of the Republic of China, Vol. 45, 58-60.
Wu I. C. (2007). Integration, Management and Visualization of Multi-Dimensional Information in Engineering
Projects, Doctoral Dissertation, Department of Civil Engineering, National Taiwan University, Taipei, Taiwan.
Wu I. C. and Hsieh S. H. (2008). The Contemporary Analysis and Development Tendency of 4D Construction
Management System, Construction Management Association of the Republic of China, Vol. 74, 23-34.
Zhang J. P., Anson M. and Wang Q. (2000). A new 4D management approach to construction planning and site
space utilization, Proceedings of the 8th International Conference on Computing in Civil and Building
Engineering (ICCCBE-VIII), Stanford, 15-22.
Zhang J. P. and Hu Z. Z. (2011). BIM- and 4D-based integrated solution of analysis and management for
conflicts and structural safety problems during construction: 1. Principles and methodologies, Automation in
Construction, Vol. 20, No. 2, 155-166.
151

ROBUST MATERIAL RECOGNITION FOR AUTOMATED BUILDING
INFORMATION MODELING FROM UNORDERED SITE IMAGE
COLLECTIONS
Mani Golparvar-Fard
Virginia Tech, United States
Andrey Dimitrov & Feniosky Pea-Mora
Columbia University, United States
ABSTRACT: Automatically generating Building Information Models from site images requires specific semantic
information to be associated with building elements, which in the case of materials such information can only be
derived from appearance-based data. Currently, the state-of-the-art texture recognition algorithms in computer
vision community are very promising (reaching over 95% accuracy), yet they have mainly been tested in strictly
controlled conditions and often they do not perform well with images collected from construction sites (dropping to
70% accuracy and lower). In addition, there is no benchmark that validates their performance under real-world
construction site conditions. To address these limitations, we propose a new robust vision-based method for
material detection and classification from single images taken under unknown viewpoint and illumination
conditions. In the proposed algorithm, material appearance is modeled in form of texture by joint probability
distribution of filter responses and Hue-Saturation-Value (HSV) color values. This distribution is modeled by a
frequency histogram of filter response and HSV color clusters. Finally the material histograms are classified using
a multiple binary Support Vector Machine (SVM) classifier. Classification performance is compared with the
state-of-the-art algorithms both in computer vision and AEC communities. Experiments are conducted on 20
typical construction materials with more than 150 images per category. The effects of various parameters such as
the choice of the filter bank and the size of the histograms are discussed. Overall, an average accuracy of 94% in
material classification is reported. The results further expose the limitations of the state-of-the-art classification
algorithms under real world conditions and define a new benchmark for robust classification as a practical tool.
The perceived benefits and limitations of the proposed method for automated generation of semantically-rich 3D
models are also discussed.
KEYWORDS: Material Recognition, Image-based 3D Reconstruction, 3D Point Cloud Models, Building
Information Models, Texton, Support Vector Machine
1. INTRODUCTION
Material detection and classification is an important part of any automated vision-based system for generating
semantically-rich as-built 3D models. In addition to 3D geometrical information, automatically generating
Building Information Models (BIM) requires specific semantic information to be associated with building
elements, which in the case of materials such information can only be derived from appearance-based data.
Current state-of-the-art practice for generating BIM primarily focuses on using laser scanners or image-based 3D
reconstruction to generate 3D point cloud models (e.g., Brilakis et al. 2010, Tang et al. 2010, 2011). These methods
which are either manual or semi-automated (Tang et al. 2010) can generate the required geometrical information
for proper BIM; however for generating semantic information which is a necessity for BIM, both
appearance-based data and spatial relationship between elements (e.g., beam supported by column) need to be
added. In this paper, we focus primarily on the problem of automated material detection and classification.
Automated material detection not only helps with deriving the required appearance-based data required for
semantically rich as-built 3D modeling, but it also helps with segmentation of elements for automatic geometrical
modeling as well as automatic detection of building elements for construction progress monitoring purposes. In the
following, first the process of automated building information modeling, 3D modeling for construction progress
monitoring, and the need for material recognition is discussed in detail. Next, the state-of-the-art techniques for
material detection and classification both in the Architecture/Engineering/Construction (AEC) and computer
vision communities are overviewed and their limitations are presented. A new vision-based method for material
detection and classification is presented next and is also accompanied with exhaustive validation experiments to
benchmark the performance against the state-of-the-art algorithms both in computer vision and AEC communities.
A comprehensive dataset and a set of validation methods that can be used in the field for development and
benchmarking of future algorithms are also introduced. The perceived benefits and limitations of the proposed
152

method in the form of open research challenges are presented. Detailed performance data, the experimental and
validation codes, the benchmarking dataset, along with additional supplementary material for the proposed
vision-based method can be found at http://www.raamac.cee.vt.edu/materialclassification.
2. AUTOMATED CREATION OF BIM AND PROGRESS MONITORING
3D modeling of the as-built environment is used by the AEC industry in a variety of engineering analysis scenarios.
Significant applications include progress monitoring of construction sites, quality control of fabrication and on-site
assembly, energy performance assessment, and structural integrity evaluation. The modeling process mainly
consists of three sequential steps: data collection, modeling, and analysis. In current practice, these steps are
performed manually by surveyors, designers, and engineers. Such manual tasks can be time-consuming,
prohibitively expensive, and prone to errors. While the analysis stage is fairly quick, taking several hours to
complete, data collection and modeling can be the bottlenecks of the process. In particular, the data collection can
spread over a few days, nonetheless the modeling stage can span over multiple weeks or even months. Additionally,
modeling often tends to be specific to certain analysis, making the application of one model to multiple analyses
very challenging. In consequence, the applicability of as-built modeling has been traditionally restricted to high
latency analysis, where results need not be updated frequently. In fast changing environments such as construction
sites, due to the difficulty in rapidly updating 3D models, model-based assessment methods for purposes such as
progress or quality monitoring have had very limited applications. There is a need for a low-cost, reliable, and
automated method for as-built modeling. This method should quickly generate and update accurate and complete
semantically-rich models in a master format that is translatable to any engineering scenario and can be widely
applied across all construction projects. The development of such a 3D modeling method also needs to be
accompanied with a low-cost and fast data collection process that is able to generate accurate and complete data
sets for modeling purposes.
Over the past decade, research on as-built modeling has primarily focused on data collection and modeling
techniques that can generate 3D geometrical information. Cheok et al. (2000) is among the early works that
demonstrated how LADAR (Laser Distance and Ranging) can be used as an as-built 3D modeling tool for
construction monitoring purposes. This led to more systematic studies of structured light data collection
techniques (Gordon and Akinci 2005, Tang et al. 2010, 2011). Several research studies have focused on showing
that Terrestrial Laser Scanning (TLS) techniques provide sufficient accuracy for use in construction dimensional
surveying and Quality Assurance / Quality Control (QA/QC) (Anil et al. 2011, Tang et al. 2011, Cheok et al. 2009).
Another line of research focuses on automated registration of 3D laser scanning point clouds with BIM (e.g.,
Bosch 2011, Belton et al. 2011, Tang et al. 2010) for the purpose of controlling the quality of the data collection.
While collecting data for as-built modeling using laser scanners can generate very accurate and complete
geometrical models, in most cases the approach suffers from the lack of semantic information; in particular
information related to construction materials. In other words, there is a need for distinguishing different materials
(e.g., formwork versus finished concrete) during the 3D modeling process for the purpose of progress or quality
monitoring. Nonetheless, the laser scanning point clouds typically do not provide the data necessary for such
analysis.
Over the past few years, cheap and high-resolution digital cameras, extensive data storage capacities, and the
availability of Internet connections on construction sites have enabled capturing and transmitting information
about construction performance on a truly massive scale. Photographic data has become the favored
documentation medium, as it contains rich information (geometry and appearance), and data is collected quickly
and inexpensively. Furthermore, the data collection process can be conducted by any on-site observer, with little
training. In the meantime, exciting research progress has been made on techniques that can assist with 3D point
cloud modeling of construction sites or individual building elements using digital imagery. For example,
Golparvar-Fard et al. (2012 & 2010) presented a new algorithm based on Structure-from-Motion (SfM) for
generating dense 3D point cloud models from unordered imagery (Snavely et al. 2008, Candrall et al. 2011).
Brilakis et al. (2011) also presented a method for generating sparse 3D point cloud models of infrastructure using
video streams. Despite the recent advancements, there are still several main research problems that need to be
addressed in the production of semantically-rich Solid Geometric Models (SGM) compatible with the Industry
Foundation Classes (IFC) format for BIM:
(1) Data Segmentation: techniques in recognition are required to segment a 3D point cloud dataset based on
geometric and appearance information into distinct subsets;
(2) Object Placement: techniques in geometrical modeling and recognition are needed to populate the scene with
153

distinct IFC objects based on the segmented subsets;
(3) Inter-object Relationships: techniques in recognition are required to identify the physical relationships between
IFC objects in the scene.
Ongoing research on image-based 3D modeling techniques can benefit from the appearance information to assist
in two steps during the production of semantically-rich SGMs; in particular (1) by segmentation of point clouds
based on appearance information so each subset can be separately used for extracting geometric information, and
(2) by recognizing construction materials to semantically label each generated 3D element. Fig. 1 shows the
process of automatically generating BIM of as-built scenes.
Fig. 1: Automated Generation of BIM is a process that requires both 3D reconstruction and recognition techniques
to generate geometry, segment and assess context, and finally identify and place BIM objects in a scene.
3. STATE-OF-THE-ART RESEARCH IN MATERIAL RECOGNITION
3.1 Computer Vision Domain
The need for extracting semantic information from a scene has yielded two main computer vision approaches:
object-based and material-based. Although object recognition systems have made significant advances in recent
years (Liu et al. 2010), they tend to overlook material specificity. In the context of construction projects, object
shape and material have a weak correlation. Similar shapes can be made up of different construction materials; e.g.,
a foundation element as well as formwork section can both be shaped are rectangular boxes. Also objects of a same
material can have high shape variability; e.g., a concrete column can be rectangular as well as circular. Thus,
material detection and classification can be seen as the dominant factor for extracting of semantic information
necessary both for geometric modeling and semantic-labeling purposes.
In the context of material detection and classification, a variety of feature based strategies have emerged and
proven applicable to more loosely defined scenes. They are usually coupled with machine learning techniques for
training and testing purposes. Popular features include filter responses (Leung and Malik 2001, Shmid 2001,
Geusebroek et al. 2003) and image patches (Ponce et al. 2006), and have been coupled with a Nearest Neighbor
(NN) learning (Varma and Zisserman 2005, 2009). More recently, state-of-the art Bayesian network learning (Liu
et al. 2010) has been applied to object recognition with features such as Scale Invariant Feature Tranforms (SIFT)
(Lowe 2004), Histogram of Oriented Gradients (HOG) (Dalal and Triggs 2005), jet, and curvature (Liu et al. 2010).
Although they have shown promising results reaching over 95% accuracy, yet they do not perform well with
images collected from construction sites. The authors conducted exhaustive experiments and in most cases
accuracy dropped to 70% and lower using real-world construction imagery. Brostow at al. (2008) have proposed to
incorporate SfM point cloud information to appearance features for recognition and segmentation, however only
accuracies of below 70% have been reported from real-world data. Caputo et al. (2010) have shown that Support
Vector Machine (SVM) learning can improve on NN, it is still not sufficient for a robust real-world recognition
system. Detection in computer vision has been tested mostly on controlled and/or limited photographic data sets
like CUReT (Columbia-Utrecht Reflectance and Texture Database), KTH-TIPS2 (Caputo et al. 2010), and
154

Brodatz textures (http://www.ux.uis.no/~tranden/brodatz.html) and University of Illinois at Urbana-Champaigns
textures (http://www-cvr.ai.uiuc.edu/ponce_grp/data/). Dissatisfaction with these benchmarks has driven the
emergence of real world data sets, like Flicker Materials Database (Sharan et al. 2009), to take into account
inconsistencies in data quality and context. In this respect, there is still no comprehensive construction materials
library that could be used for training and testing of material detection and classification algorithms for the purpose
of semantically-rich as-built 3D modeling.
3.2 Architecture/Engineering/Construction Domain
Automated sensing has been a topic of interest in the construction research field for some time. Automated
recognition of materials in a construction setting has been of particular importance to construction progress
monitoring and quality control. Brilakis et al. (2005, 2006, 2010) is the first to use recognition techniques to
facilitate image retrieval as well as identify building elements from 2D images. Kim et al. (2002) developed a
scanning technique that utilizes laser scanner point cloud data to automatically identify the sizes of stone
aggregates. Despite the good performance of these algorithms, there is still a need for research on material
recognition techniques that with a very high reliability can detect distinct construction materials under variety of
illumination, viewpoint, and site conditions.
3.3 Limitations of Current Approaches
The overall limitations of the state-of-the-art approaches for material recognition both in computer vision and AEC
communities are as follows:
1. Lack of systematic data collection and comprehensive datasets of material recognition in uncontrolled
construction environments;
2. Lack of robust automated material recognition techniques that maintain a high level of accuracy in
uncontrolled construction environments; and
3. Lack of integration and cross validation of multiple feature types for increased robustness and accuracy
in detection and classification.
To support automated generation of BIM and/or generate models for progress and quality monitoring purposes,
there is a need for new material detection and classification techniques that can address these limitations. In the
following sections, a new method and a comprehensive dataset for material detection and classification is
introduced. The performance of our proposed method is also compared against the state-of-the-art methods in
both computer vision and AEC communities.
4. PROPOSED IMAGE-BASED MATERIAL RECOGNITION METHOD
Our proposed image-based material recognition algorithm is based on statistical distribution of filter responses
over the images in form of primitives such as edges, spots, waves and Hue-Saturation-Value (HSV) color values,
and is divided into two stages of learning and classification. Fig. 2 shows an overview of the learning and
classification components of our proposed algorithm.
Fig. 2: The overview on the learning and classification components of our proposed material recognition method.
155

In the learning stage of our algorithm, training images are initially converted to gray-scale and their intensities
are normalized to have zero mean and unit standard deviations. Next, these images are convolved with a
comprehensive filter back (Leung and Malik 2001) to generate filter responses (See Fig.3). The filter bank
consists of 48 filters, first and second derivatives of Gaussian at 6 orientation and 3 scales (total of 36), 8
Laplacian of Gaussian filters, and 4 Gaussians. The scales of filters in this filter bank range between =1 to 10.
Next, following a bag-of-words model, we cluster the filter responses to generate codebooks of material
appearances. The cluster centers (codewords) will be Textons which are visual atoms of the visual perception of
materials (Leung and Malik 2001, Cola and Dana 2001). Thus, the texture of the materials will be defined by
modeling the texton frequencies learned from training images. For all images in the training dataset, the filter
responses are concatenated. Next, a fixed number of texton cluster centers are computed using the k-means
clustering algorithm (k the number of visual words). After quantizing individual features into k visual words, the
distribution of visual words per image is calculated for all of the different features by assigning each pixel in the
image to the nearest visual word index and forming the histogram over the frequency of the visual words.
The histogram of texton frequencies is then used to form the codebooks and the material models that are
corresponding to our training images. In our work, HSV colors are also incorporated to leverage the color
information for a robust material detection and classification. It was hypothesized that since HSV instead of
standard R, G, and B color channels can separately model the brightness, it can provide a higher degree of
invariance to varying illumination conditions on the sites or the time images are captured. Similar to the texture
recognition, for all images in the training datasets, the HSV color values are aggregated and clustered into HSV
textons using the k-means clustering algorithm. Next, a histogram of color textons, i.e., the frequency with which
each color texton occurs in the images, forms the codebook corresponding to each training image. The texture
and color histograms are then concatenated to generate a unique codebook for each image.
Our visual representation based on codebooks for construction materials is leveraged by training discriminative
machine learning models of material categories. We train non-linear binary Support Vector Machine (SVM)
classifiers with ^2 kernels for each material category independently. SVM classifiers are discriminative binary
classifiers that optimize a decision boundary between two classes. Note that we choose a non-linear ^2 kernel
due to its suitability for classification of histograms. In order to extend the binary classification decision of each
SVM classifier to multiple classes, we adopt the one-versus-all multi-class classification scheme. When training
the SVM classifier that corresponds to each material class, we set all the examples from that class as positive and
the examples from all other classes as negatives. The result of the training process is one binary SVM classifier
per material of interest. Given a novel testing image, we apply all binary classifiers and select the material class
corresponding to the classifier with the highest classification score.
Fig. 3: The textural features of each pixel are described by convolving the original image with the 48 filters
proposed by Leung and Malik (2001).
5. EXPERIMENTS AND VALIDATION METRICS
5.1 Data Collection and Experimental Setup
Due to the lack of existing datasets for benchmarking performance of material recognition algorithm in real-world
conditions, it was necessary to create a new comprehensive Construction Materials Library (CML) that enables
recognition of large quantity of material types recorded in a variety of as-built contexts. This dataset is for both
training and testing purposes so that it can be released to the community for further development and validation of
new algorithms. For this purpose, we collected 3,000 material samples in the library is an uncompressed color
image of resolution 200200 pixels. These images were recorded from seven different scenes (i.e., 2 existing
156

building and 5 construction sites). In order to create a comprehensive dataset with varying degrees of viewpoint,
scale, and illumination changes, these images were collected over the span of seven months. Due to various
possible appearances of materials, a set of factors were accounted for in each material category: surface condition,
scale, and illumination.
Material Categories: The CML comprises 20 major construction material types. A full list of these materials
can be seen in Figure 4.
Sub-Categories: A course material definition is broken down to several representative sub-types, depending of
the material. For example, concrete can be present in a scene as smooth finish, course finish, good condition, or
deteriorated. Additionally, each material exhibits a specific appearance under different lighting conditions. Two
to four lighting conditions per material where documented.
Poses: Scale and orientation variability is documented for each sub-category. For a digital resolution of 200
pixels, images representing an approximate physical length of 12", 24" and 48" were gathered. Three to five
view orientations in respect to the surface normal were also considered. Thus, a total of 9-15 poses make up each
material sub-category. The entire dataset is made public at: http://www.raamac.cee.vt.edu/materailclassification.
Fig. 4: The Construction Materials Library contains the following materials: Asphalt, Brick, Cement (Granular
and Smooth), Concrete (Cast-in-Place and Precast), Foliage, Form Work, Grass, Gravel, Marble, Metal Grills,
Paving, Soil (Compact, Loose, Mulch, and Vegetation), Stone (Granular), Limestone, Wood.
5.2 Performance Evaluation Measures
To quantify and benchmark the performance of the material recognition algorithm, we plot the Precision-Recall
curve and use confusion matrix to represent average classification accuracies. These metrics are both set-based
measures; i.e., they evaluate the quality of an unordered set of data entries. In the context of material recognition
and classification, we define each as follows:
5.2.1 Precision-Recall Curve
To facilitate comparing the overall average performance of the variations of the proposed material recognition and
classification algorithm over a particular set of images, individual recognition class precision values are
interpolated to a set of standard recall levels (0 to 1 in increments of 0.1). Here, precision is the fraction of
retrieved instances that are relevant to the particular classification, while recall is the fraction of relevant instances
that are retrieved. Thus, precision and recall are calculated as follows:
TP
precision
TP FP

(1)
FN TP
TP
recall

(2)
where in TP is the number of True Positives, FN is the number of False Negatives and FP is the number of False
Positives. For instance, if the algorithm detects and classifies the material properly, it will be a TP; if a material
instance is incorrectly detected and classified under another material class, it will be a FP. When a material instance
is not detected and classified under the material class, then the instance is a FN. The particular rule used to
interpolate precision at recall level i is to use the maximum precision obtained from the detection class for any
recall level great than or equal to i. For each recall level, the precision is calculated; then the values are connected
and plotted to form a curve.
157

5.2.2 Confusion Matrix
In our experiments, average accuracy of detection and classification is measured per material using confusion
matrix. Each index in the confusion matrix shows for each pair of segmented category <c
1
,c
2
>, how many material
instances from c
2
were incorrectly assigned to c
1
. Each column of the confusion matrices represents the predicted
material category and each row represents the actual material category. The segmented True Positives (TP), False
Positives (FP), and False Negatives (FN) are compared and the percentages of the correctly predicted categories
with respect to the actual category are calculated using the above formulas and represented in each row.
6. EXPERIMENTAL RESULTS
In our research, we used the CML to benchmark the performance of the proposed material detection and
classification. The performance of the proposed method is validated against the state-the-art algorithms. In total,
seven categories of features were tested (see Table 1). The filter based approach of Leung and Malik (2001) was
implemented in its original 48-dimensional form (LM) as well as in its 18-dimensional rotationally invariant
form (rLM). Image patch (IP) features (Varma and Zisserman 2009) were implemented with a 77 pixel
neighborhood as it produces the best results in accordance to the literature. For color based features, HSV
formats are used to create 3-dimensional feature collections. Finally, we tested two combinations of these
categories by concatenating their frequency histograms.
For all cases, the data was split into two subsets: a learning set and a classification (testing) set. Thus, a randomly
selected 70% of the CML samples were allocated to learning and the remaining 30% were used for testing the
classification performance. Ten such splits where performed for each approach and the average performance was
recorded. To assess the computation cost of classification versus accuracy, several clustering granularities were
tested. Granularity defines the number of bins in each training and testing histogram. Table 1 shows a summary
of the results.
Table 1: Effect of cluster granularity on average accuracy (%) of different features.
Feature
Texton Clusters per Material
3 4 5 7 10 15
LM 76.6 78.1 80.1 81.3
rLM 73.6 76.9 76.3 78.2
IP 56.8 63.0 64.6 64.2
HSV 87.2 89.1 89.2 89.3
RGB 80.2 84.0 87.1 87.1
LM(10) + HSV 92.8 93.1 94.0 91.0
LM(15) + HSV 92.8 93.5 93.5 91.2
LM: Leung and Malik filter bank; rLM: 18-dimensional rotationally invariant form of LM filter bank; IP: Image patch.
We can first note that the performance of the LM filter bank for the CML dataset is well under the one reported
on the CURet dataset. The average of 82.5% classification rate is much lower than the 95%+ rate as reported in
the literature (Varma and Zisserman 2005&2009). This implies that this approach by itself is not robust enough
for use in the context of as-built modeling for unordered photographs. Similarly, the (Brilakis et al. 2005)
algorithm underperforms when tested against the CML benchmark, with an average of 52.8% (See the
precision-recall graph in Fig.5).
Fig. 5: Precision and recall plot of our algorithm with respect to other material classification methods.
158

All features achieve best performance with 5-7 clusters per material. More clusters can result in small
improvements at a much higher computation cost. The intensity based features hover around the 60%-80% range
with LM showing the highest average precision. However, color based features perform even better, ranging
between 85% and 89%. A significant improvement is shown when LM features are combined with HSV features.
The results of between 91% and 94% remain steady, regardless of the granularity of clustering of HSV. This
approach proves to be the most efficient as well, as a high accuracy is maintained even with 3 clusters per
material.
Fig. 6 shows the confusion matrix on the performance of our proposed material detection and classification
method. As observed in this figure, most material classifications have very high average accuracies (above 90%).
In several categories such as cement and concrete classes, there is still 2-7% confusion among them. For these
categories, human observation only based on the 200200 images in the library may not result in right
classification, which is primarily related to the absence of context information.

Fig. 6: The confusion matrix shows the average accuracies of material classification.
7. DISCUSSION ON THE PROPOSED METHOD AND RESEARCH CHALLENGES
This study presented a robust material detection and classification method and the first comprehensive dataset for
semantically-rich as-built 3D modeling purposes. The average accuracies of 94% across all materials promise the
applicability of the proposed method. Such results also indicate the robustness of the method to dynamic changes
of illumination, viewpoint, camera resolution, and scale which is typical for unordered construction image
collections. While this paper presented the initial steps towards processing unordered image collection for the
purpose of material recognition, several critical challenges remain. Some of the open research problems include:
Impact of Image Quality: The effect of different compression qualities and image sources (e.g., SLR
cameras, video cameras, cell phones, wed based content) on classification performance;
Spatial Resolution: The effect of the number of available pixels on feature extraction and classification
performance;
Multiple Images: The effect of leveraging multiple images registered to reconstructed point cloud
models through various voting/scoring methods;
Optimal Machine Learning Algorithms: More exhaustive experiments with state-of-the-art machine
learning algorithms to determine their optimal application to material detection and classification data.
159

8. CONCLUSION AND FUTURE WORK
This paper presented a robust material detection and classification method for semantically-rich as-built 3D
modeling and construction monitoring purposes. A Construction Materials Library was formulated to train and
test the proposed method. This real-world dataset exposed the limitations of current material detection and
classification methods and validated the performance of the proposed algorithm. From our testing, an average
classification rate of 94% was achieved. The addition of HSV color channels as a feature for training results in a
more robust classification algorithm, and can be implemented as critical process in an automated as-build 3D
modeling system. Our ongoing research is focused on exploring the application of the proposed material
detection and classification for segmentation of 3D point cloud models. Segmenting point cloud models into
recognizable shapes and materials is a stepping stone in detecting and placing complete construction elements in
a Building Information Model. Furthermore, material detection and classification performance could be
improved by exploring the incorporation of 3D geometrical information. The geometric surface properties of a
given surface could supply additional descriptive features that can be integrated along with the aforementioned
image based features. These are being explored as part of ongoing research and results will be presented soon.
9. REFERENCES
Anil E. B., Tang P., Akinci B., and Huber D. (2011). Assessment of Quality of As-Is Building Information Models
Generated from Point Clouds Using Deviation Analysis. Proceedings of SPIE, San Jose, California, USA.
Belton D., Gibson A., Stansby B., Tingay S. J., and Bae K. (2011). Modelling of an inexpensive 9M satellite dish
from 3D point clouds captured by terrestrial laser scanners, ISPRS Workshop Laser Scanning 2011. Calgary,
Canada: International Society for Photogrammetry and Remote Sensing.
Bosch F. (2011). Automated Recognition of 3D CAD Model Objects in Laser Scans and Calculation of As-built
Dimensions for Dimensional Compliance Control in Construction, Elsevier Journal of Advanced Engineering
Informatics, 24( 1), 107-118.
Brilakis, I., and Soibelman, L. (2006). Multi-Modal Image Retrieval from Construction Databases and
Model-Based Systems, Journal of Construction Engineering and Management, American Society of Civil
Engineers, Volume 132, Issue 7, 777 785.
Brilakis, I., Soibelman, L. and Shinagawa, Y. (2005). Material-Based Construction Site Image Retrieval, Journal
of Computing in Civil Engineering, American Society of Civil Engineers, Volume 19, Issue 4, 341 355.
Brilakis, I., German, S. and Zhu, Z. (2011). Visual pattern recognition models for remote sensing of civil
infrastructure, Journal of Computing in Civil Engineering, American Society of Civil Engineers, Volume 25, Issue
5, 388 393.
Brostow G.J., Shotton J., Fauqueur J., Cipolla R. (2008). Segmentation and Recognition using Structure from
Motion Point Clouds, ECCV (1): 44-57.
Caputo B., Hayman E., Fritz M., and Eklundh J. (2010). Classifying materials in the real world. Image and Vision
Computing, 28:150163.
Cheok, G. S., Filliben, J. J., and Lytle, A. M. (2009). Guidelines for accepting 2D building plans. NIST
Interagency/Internal Report (NISTIR) - 7638.
Cheok, G. S., Stone, W. C., Lipman, R. R., Witzgall, C. (2000). LADARs for Construction Assessment and Update,
Journal of Computing in Civil Engineering Automation in Construction, Vol. 9, No. 5, 463 477.
Crandall D., Owens A., Snavely N., and Huttenlocher D. (2011). Discrete-Continuous Optimization for
Large-Scale Structure from Motion, Computer Vision and Pattern Recognition (CVPR).
Cula O. G. and Dana K. J. (2001). Compact Representation of Bidirectional Texture Functions, In IEEE
Conference on Computer Vision and Pattern Recognition, 10411047.
Dalal N. and Triggs B. (2005). Histograms of Oriented Gradients for Human Detection, In Proceedings of IEEE
Conference Computer Vision and Pattern Recognition , San Diego, USA, 886 - 893.
160

Geusebroek J.M., Smeulders A.W.M. and J. van de Weijer (2003). Fast Anisotropic Gauss Filtering. IEEE
Transactions on Image Processing, 12(8), 938-943.
UGolparvar-Fard, M.U, Pea-Mora, F., and Savarese, S. (2012). Automated model-based progress monitoring using
unordered daily construction photographs and IFC as-planned models. ASCE Journal of Computing in Civil
Engineering, http://dx.doi.org/10.1061/(ASCE)CP.1943-5487.0000205 (In press).
UGolparvar-Fard, M.U, Pea-Mora, F. and Savarese, S. (2010). D
4
AR 4 Dimensional augmented reality - tools for
automated remote progress tracking and support of decision-enabling tasks in the AEC/FM industry. Proc., The
6th Int. Conf. on Innovations in AEC Special Session - Transformative machine vision for AEC, State College, PA.
Golparvar-Fard M., Pea-Mora F., and Savarese S. (2009). D4AR A 4-dimensional augmented reality model for
automating construction progress data collection, processing, and communication, Journal of Information
Technology in Construction, 14, 129 153.
Gordon C. and Akinci B. (2005). Technology and process assessment of using LADAR and embedded sensing for
construction quality control. Construction Research Congress.
Lazebnik S., Schmid C., and Ponce J. (2006). A Discriminative Framework for Texture and Object Recognition
Using Local Image Features, Toward Category-Level Object Recognition, Springer-Verlag Lecture Notes in
Computer Science vol. 4170, 423 442.
Leite, F., Akcamete, A., Akinci, B., Atasoy, G. and Ergan, S. (2011). Analysis of Modeling Effort and Impact of
Different Levels of Detail in Building Information Models, Automation in Construction, vol. 20, 601609.
Leung T. and Malik J. (2001). Representing and Recognizing the Visual Appearance of Materials using
Three-Dimensional Textons, International Journal of Computer Vision, 43(1), 29-44.
Lowe D. G. (2004). Distinctive image features from scale-invariant keypoints, International Journal of Computer
Vision, 60, 2, 91-110.
Liu C., Sharan L., Adelson E. H. and Rosenholtz R. (2010). Exploring Features in a Bayesian Framework for
Material Recognition, IEEE Conference on Computer Vision and Pattern Recognition.
Ponce J., Hebert M., Schmid C., and Zisserman A. (2006). A Discriminative Framework for Texture and Object
Recognition Using Local Image Features, Toward Category-Level Object Recognition, Springer-Verlag Lecture
Notes in Computer Science vol. 4170, 423 442.
Schmid C. (2001). Constructing models for content-based image retrieval, Proceedings of the IEEE Conference on
Computer Vision and Pattern Recognition, volume 2, 39-45.
Sharan, L., Rosenholtz, R., & Adelson, E. H. (2009). Material perception: What can you see in a brief glance?
Journal of Vision, 9(8):784, 2009.
Snavely N., Seitz S. M., Szeliski R. (2008). Skeletal Sets for Efficient Structure from Motion, Computer Vision
and Pattern Recognition (CVPR).
Tang, P. Huber D. and Akinci, B. (2011). Quantification of Edge Loss of Laser Scanned Data at Spatial
Discontinuities, Journal of Computing in Civil Engineering. Volume 25, Issue 1, 31-42.
Tang, P., Huber, D., Akinci, B., Lipman, R., and Lytle, A. (2010). Automatic Reconstruction of As-built Building
Information Models from Laser-Scanned Point Clouds: A Review of Related Techniques, Automation in
Construction, 19, 829-843.
Varma M. and Zisserman A. (2005). A Statistical Approach to texture classification from single images,
International Journal of Computer Vision: Special Issue on Texture Analysis and Synthesis.
Varma M. and Zisserman A. (2009). A Statistical Approach to Material Classification using Image Patch
Exemplars, IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 31, 2032 2047.
Zhu, Z. and Brilakis, I. (2010). Concrete Column Recognition in Images and Videos, Journal of Computing in
Civil Engineering, American Society of Civil Engineers, Volume 24, Issue 6, 478 487.
161

BIM-BASED JOB HAZARD ANALYSIS
Jrgen Melzner & Hans-Joachim Bargstdt
Bauhaus-Universitt Weimar, Germany
ABSTRACT: The accident rate in the construction industry is the highest rate among all industries. In Germany,
more than 100 construction workers die every year at work. This figure shows that safety in construction business
is still a problem which must be faced. In most cases, safety-planning is based on checklists and manual
description which are not closely related to the actual and specific construction object. Safety planning in
construction is a challenging task because of the large number of parties involved, the constantly changing
conditions and the complexity of buildings. The nature of construction projects are separate stages in the planning
process. This leads to unequal information distribution among the involved work planners. The objective
evaluation of safety-planning methods regarding qualitative and quantitative factors could be considerably
improved by applying innovative and integrated safety-planning tools. Modern technologies, such as Building
Information Modeling (BIM), are offering an object-orientated planning approach towards the projects lifecycle.
Current job site conditions are unsafe due to a lack of information for safety planning in the work preparation
process. This paper solves this problem by applying a objct-orientated and process-orientated job hazard analysis
based on Building Information Models (BIM). The proposed rule-based system can detect a safety hazard early on
in the design and planning process. The scope of the research in this paper is limited to safety hazards in shell
construction. However, future research investigations aim to expand the application fields of the proposed
framework.
KEYWORDS: Building Information Modeling, Construction Management, Rule-Interpretation, Safety Planning
1. INTRODUCTION
The current high accident rate in the construction industry compared with all other industries emphasizes that
safety issues are not considered satisfied in the construction industry.
Modeling and visualization of construction processes play a key role in construction planning, reducing the risk of
faulty management of budget, time, quality, and safety issues in the construction project (Song et. al. 2006)
(AbouRizk et. al. 1992). Construction projects are complex due to the many different aspects of the planning phase
which must be considered. The most common problems in construction arise due to uncontrolled processes, which
are a result of changed conditions or visualization models that have not been taken into account.
The importance of construction project planning has been acknowledged (Faniran et. al. 1999). The project
planning process has a significant impact on the success of the construction projects (Proporowitz 2008). However,
there is generally not enough time available for the work preparation process. This dilemma occurs due to the fact
that the company executing the project is involved too late in the projects development. Also, the planning
engineer is unable to spend enough time on the development process and lacks the necessary tools to evaluate all
boundary conditions and safety requirements associated with the different execution methods of the project.
Especially in construction practice, the safety planning process and the execution planning process are considered
independently due to their different responsibilities and the lack of an existing integrated product model.
Researches in safety sciences already recognized the causes of injuries in construction, and numerous preventative
solutions have been developed. New tools and methods are required to improve construction site safety (Zhang et
al. 2012). This paper proposes a safety planning support system based on a Building Information Modeling. The
scope of this paper focuses on a rule-interpretation based on the German safety standard and best practice solutions.
The guidelines will translate and be implemented into a fictive example of a construction project.
The paper is structured in the following way: Section 2 gives an overview about the topics Building Information
Modeling (BIM), safety management and knowledge representation. Section 3 explains the problems in the
construction safety management and the necessity of improvements. Section 4 describes the applied research
framework and the architecture of the rule-based system. An description of the object- and process-oriented job
hazard analysis follows in Section 5. Further, description of future work and conclusion is provided.
162

2. BACKGROUND
2.1 Building Information Modeling
According to Eastman et al. is BIM a modeling technology and associated set of processes to produce,
communicate, and analyze building models. Building models are characterized by:
Building components that are represented with digital representations (objects) that carry computable
graphic and data attributes that identify them to software applications, as well as parametric rules that
allow them to be manipulated in an intelligent fashion.
Components that include data that describe how they behave, as needed for analyses and work processes,
for example, takeoff, specification, and energy analysis.
Consistent and non-redundant data such that changes to component data are represented in all views of
the component and the assemblies of which it is a part.
Coordinated data such that all views of a model are represented in a coordinated way.
(Eastman et al. 2011)
The Associated General Contractors of America (AGC) describe BIM as a data-rich, object-oriented, intelligent
and parametric digital representation of a facility (AGC 2010). The most well-known building model is the
Industry Foundation Class (IFC). It is an object-oriented model for the description of building structures (Tauscher
2011). Building information models contain all the data to describe the building with all its components. With the
help of the IFC as an interface, the information from the model is taken for further processing. This interface
represents the relationships of the objects that describe the structure and its attributes.
Support construction management in the early planning phase of Building Information Modeling (BIM) has been
acknowledged as an appropriate tool (Hartmann et al. 2008). The gap between the construction process and health
and safety issues can be closed through the use of Building Information Models through the integration of safety
requirements into the program (Sulankivi et al. 2010).
Xue et al. (2012) shows the collaborative work and project management in architecture, engineering and
construction (AEC) projects in the past decade. They reviewed 83 journal papers and concluded that with the
advantages of IT related technologies such as nD modeling, intelligent agent and virtual collaborative environment
technology is becoming more and more popular in AEC projects. Hartmann et al. (2008) shows a review of about
50 case studies and analyzed the projects regarding 3D/4D model applications. The analysis of the reviewed case
studies shows that 3D/4D Models have been applied successfully in the construction industrys Virtual Design in
Construction (VDC). Even in safety planning, Building Information Models can be used to improve safety
performance onsite as well in the planning phase (Zhou et al. 2012).
Digital building models such as BIM are widely used in the architecture, engineering, and construction (AEC)
industries in design and construction. The advantages of using BIM for safety planning can be summarized in the
following way:
Identification of safety hazards based on the model
Communication of safety risks to the workers
Detections of special conflicts
4D-visualiszation of safety equipment

2.2 Safety management
Health and safety management for job sites is part of the construction project management. Safety planning in
the construction industry is complicated due to the dynamic nature of the construction environment and the
active involvement of many different stakeholders. Construction is typically seen as a one-of-a-kind business.
Each project is built for the first time and only very few building types are constructed repeatedly.
Consequently, construction planning has to start again with the initiation of a new project. Traditionally, safety
planning in construction is separated from the work planning phase, which can lead to (1) decisions in the design
and layout planning process being made independently of safety issues, and (2) the project layout changing
throughout the construction phases.

163

The contractor is responsible for construction site safety (Gambatese et al. 2008, BaustellV 1998). This
obligation also includes the introduction of a health and safety coordinator if the contractor is unable to carry out
the task himself due to his lack of his expertise. Although the introduction of a safety and health coordinator will
cause additional costs of up to 0.3-1.0 % of the construction budget, savings are expected as a result of reduced
construction interferences, lower accident and loss rates/costs, better coordination, and the sharing of
construction equipment (Bargstdt and Steinmetzger 2010). Furthermore, the implementation of a coordinator
does not exempt the contractor from the responsibility for maintaining safe job sites.
The practice of health and safety in construction is characterized by the manual observation of 2D drawings.
Safety analysis is mostly based on the finished product represented in the drawing. The problem in the practiced
work flow is that hazards arising in or from temporary construction stages, such as fall hazards that are caused
by unfinished exterior walls, will or cannot be addressed with appropriate and safe measures. In fact, safety
planning in the world wide construction industry has to be improved (Melzner et al. 2012b). It is in the nature of
the construction business that quality planning and work preparation is directly linked to the knowledge of the
executing person. Therefore, knowledge-based decision support tools can help humans to improve the planning
standard by, for example, assisting the safety planning engineer in making better and safer decisions.

2.3 Knowledge Representation
Knowledge-based systems support people in solving complex problems by deriving recommendations for action
from a knowledge base. So-called if-then relationships can reflect human knowledge and make it
machine-readable. Knowledge-based systems are part of the field of artificial intelligence. The methods and
procedures of knowledge processing make this knowledge explicit and link it to formal knowledge holders. Thus,
access to experience allows for the use and re-use of knowledge in a project (Gehle 2006).
Knowledge-based systems use different methods to draw possible conclusions. Mikulakova presents a
case-based reasoning (CBR) method to determine possible execution sequences of construction processes. She
proposed a knowledge-based system to formalize execution problems in case problems and compares these with
previously stored projects (Mikulkov 2010, Mikulkov et al. 2010).
A further method of knowledge-representation is the constraint satisfaction paradigm. Beiert proposed a
constraint-based approach for creating possible execution alternatives of outfitting processes in construction.
(Beiert 2010). She used a constraint-based approach to describe restrictions of outfitting processes. Her approach
contains the description of so called hard and soft constraints. Hard constraints take technological dependencies
into consideration. By using soft constraints, it is possible to regard conditions that do not have to be completely
fulfilled.
Rule-based systems are not based on actual cases. Rules are generalized in the form if A then B. Contrary to CBR,
such rules are more general laws which allow for conclusions to be drawn for specific situations. Rules often have
to be entered directly by human experts into the system.

3. PROBLEM DESCRIPTION
Nowadays, safety management in most real-world scenarios is characterized by ad-hoc decision-making based
on observed situations on job sites. This workflow causes an unsafe environment because of a poorly developed
safety concept for each specific construction project. Many dangerous situations on job sites can be eliminated
with a project-specific job hazard analysis (Wang and Boukamp 2009). A manual hazard analysis process can be
time-consuming when floor plans of several units consisting of many different drawings and sections are taken
from a variety of building plans (Blickling 2003). Work preparation is directly linked to the knowledge of the
executing person. Only very experienced planners can imagine the real spatial conditions of job sites based on 2D
drawings. A hazard analysis based on a virtual model of the building can improve the safety conditions.
However, working with one central building model leads to further challenging objectives. It is in the nature of the
construction business that many parties are involved in the projects development phase. Furthermore, the detailed
design of the building will change many times before and during construction. Many details must be entered
multiple times. This often leads to redundant or incorrect data, loss of quality, and a higher expenditure of time
which result in higher costs.
164

Therefore, knowledge-based decision support tools based on a virtual building model can help humans to improve
the planning standard by, for example, assisting the safety planning engineer in making better and safer decisions.
Therefore, the proposed framework brings together the best practices and legal requirements in a knowledge-base
and applies it on an object-oriented building model.

4. KNOWLEDGE-BASED CONCEPT
This paper introduces a knowledge-based decision support system for safety planning in construction management,
based on a Building Information Model (BIM). The approach uses rules to check the building model in regard to
pre-defined attributes.
The architecture of a rule-based system consists of three parts (Fig. 1), and the input data is generated from three
different sources. The building objects will be derived from the building information model. The data includes an
objects ID, its name, and its geometrical and schedule information.
Fig. 1: Architecture of the rule-based system
By assigning one or more activity or sub-activity to each object, the entire workflow to build an object is
represented. The next step in this approach is to design the rule-base for a geometrical and construction process
related hazard. Based on annual reports from the German accident insurance, the accidents in the construction
industry were analyzed. Through the analyzed statistics, the accident black spot could be identified. Legal textual
regulations on construction safety and other free defined rules drawn from previously finished projects completed
the input-data base.
The findings from these statistics, the legal regulations and the best-practice solutions will be interpreted in
table-based rules. These build the knowledge-base. The knowledge will be represented using production rules like
if A then B. Theoretically, there is no numerical restriction for the constraints which will lead to a rule-execution.
Finally, the rule engine applies the stored rules on facts derived from the building model and process model. Facts
are attributes of the objects.
The objective of this approach is the application of basic knowledge of construction management in the work
preparation process. In order to achieve this objective, four steps must be defined:
1. Obtain the relevant best-practice rules and safety standards in table-based rules.
2. Describe this empirical knowledge in a rule-based system.
3. Develop abstract solutions depending on geometrical conditions.
4. Implement the solutions by adapting them for a specific project in a case study.
165

5. OBJECT- AND PROCESS-ORIENTED JOB HAZARD ANALYSIS
The first step in this concept is the description of the object-oriented job hazard analysis. Rozenfeld et al. proposed
a hazard analysis by describing all possible loss-of-control events in a knowledge-base. They developed the
Construction Job Safety Analysis (CJSA) based on three major steps (Rozenfeld et al. 2010). Step 1 identifies
hazards that occur during the execution of construction processes. Step 2 assesses the probability of occurrences of
each hazard. Finally, step 3 assesses the degree of severity for each accident.
In this paper, two different types of hazard sources are assumed. Firstly, geometrically caused hazards can derive
from the shape, location or size of an object. For example, the likelihood of falling from the edge of a slab is higher
if the work place is near the edge. Secondly, hazards can be caused by the construction process. A common method
to structure objects and processes in a construction project is working with the work breakdown structure (WBS)
and process patterns (Melzner et al. 2012c). The decomposition of erection processes helps to structure and to
analyze the hazards regarding objects and processes (Fig.).
Fig.2: Decomposition of building elements

After the decomposition of the building into single objects, the construction method can be allocated. At least one
activity is needed in order to build an object. Normally, construction processes are divided into several
sub-activities. The process of building a cast-in-place concrete column is decomposed into five sub-activities such
as reinforcing, forming, concreting, curing and stripping (Fig.2).
The next step in configuring the knowledge-base is to capture the object and the geometry-related hazards. An
example of this characteristic is shown in Figure 3. Based on the attributes of the objects in the 3D-building model,
so-called geometry-related hazards can be deducted. Examples for that are the
x distance to the building edge,
x distance to the lower level,
x distance to the ground floor ,
x distance to other objects,
x distance to holes in the floor and
x distance to deepening in the floor.
166

Fig.3: Object /geometry-related hazards
Next, the knowledge-base will be generated regarding the construction method-related hazards. There are different
methods which can be used to build identical construction objects. Different methods lead to different
process-related hazards which must also be taken into consideration. There are two different way to build concrete
columns; through assembling pre-cast columns or cast-in-place concrete columns. Hazards in this category of
building concrete columns are explained again (Fig. 4).
Hazards in regard to the building process in this situation are
x hazards through installing the reinforcement
x hazards through installing formwork
a) Falling from the scaffold
b) Getting hit by a form board
x hazards through concreting
a) Falling from the scaffold
b) Getting injured by a vibrator.

167

Fig.4: Construction method-related hazards
6. FUTURE WORK
The presented framework enables an expansion to other tasks in work preparation of construction projects.
Further investigations will be necessary in order to develop the rules for other applications in site management
e.g. temporal structures such as temporal steel columns or scaffolds.
Another approach is to expand the knowledge base in order to compare different solutions if there is more the
one possible result. Exemplarily for the evaluation of different fall protection methods, a hierarchical approach
was introduced (Melzner et al. 2012a). In this approach, five main criteria have to be chosen to reflect the overall
goal and the best fall protection system for working on slabs. These five criteria are split into further sub criteria
and, together with the overall goal, shape the model of the AHP. The safety measures shown here are taken in
order to prevent construction workers from falling off slabs: barriers, personal safety equipment, scaffolds and
side protection are taken into account. The list of criteria is a representative and generic sample of factors
whichhold importance in various decision making problems concerning fall protection (Fig. 5).

168

Fig. 5: Influencing factors for the selection of fall protection systems (Melzner et al. 2012a)
By integrating a multi-criteria decision support system in the BIM-based construction job site approach a holistic
framework will developed.

7. CONCLUSION
This research outlines an application for an automated rule-based safety checking system for construction job
site hazard detection and mitigation, based on German safety standards. The implementation of this system will
be shown in fictive examples dealing with objects form a high rise building. The paper shows the work flow of
job site hazard analysis regarding object and geometry-related hazards, as well as construction method-related
hazards. The approach is customizable to local safety regulations and individual best practice solutions.
A validation of the applied prevention methods on real construction projects is necessary. The presented study
and results are an initial first contribution for future work towards more detailed and real-world construction
environments.
8. ACKNOWLEDGEMENT
The described approach is partly supported by the The Bavarian Construction Industry Association (Bayerischer
Bauindustrieverband) and Construction Company W. Markgraf GmbH & Co KG. I express my thanks for the
possibility to develop this approach.
9. REFERENCES
Abourizk, S., Halpin, D.W., Lutz, J.D. (1992). State of the art in construction simulation, Proceedings of the 1992
Winter Simulation Conference, Arlington, VA, 1992, 1271-1277.
Associated General Contractors of America (AGC). (2010). The contractors guide to BIM. Lincoln, NE.
Retrieved June 22, 2012, from www.agcnebuilders.com/documents/BIMGuide.pdf
Bargstdt, H. and Steinmetzger, R. (2010). Grundlagen des Baubetriebswesens, Skriptum zur Vorlesung,
169

Bauhaus-Univeritt, Schriften der Professur Baubetrieb und Bauverfahren, Weimar.
Beiert, U. (2010). Constraint-basierte Simulation zur Terminplanung von Ausfhrungsprozessen. Reprsentation
baubetrieblichen Wissens mittels Soft Constraints, Dissertation, Bauhaus-Universitt Weimar, Fakultt
Bauingenieurwesen, Professur Baubetrieb und Bauverfahren, Weimar.
Blickling, A. (2003). Die Verwendung von virtuellen 3D-Modellen bei der SiGeKo-Planung auf Baustellen.
3. Fachtagung Sicherheit auf Baustellen, ed H.-J. Bargstdt, Weimar, 7378.
Eastman, C. M., P. Teicholz, R. Sacks, and K. Liston. (2011). BIM handbook: A guide to building information
modeling for owners, managers, designers, engineers and contractors. 2nd ed. Hoboken, NJ: Wiley.
Faniran, F.O., Love, P.E.D., and Li, H. (1999). Optimal allocation of construction planning resources, Journal of
Construction Engineering and Management, Vol. 125, no. 5, 311319.
Gambatese, J.A., Behm, M. and Rajendran, S. (2008). Designs role in construction accident causality and
prevention: Perspectives from an expert panel, Safety Science, vol. 46, no. 4, 675691.
Gehle, M. (2006). Internationales Wissensmanagement - Zur Steigerung der Flexibilitt und Schlagkraft
wissensintensiver Unternehmen, Dt. Universitts-Verlag. Wiesbaden.
Hartmann, T., Gao J., and Fischer, M. (2008). Areas of Application for 3D and 4D Models on Construction
Projects. In: Journal of Construction Engineering and Management, Vol. 134, no. 10, 776785.
Mikulkov, E. (2010). Wissensbasierte Bauablaufplanung mit Fallbasiertem Schlieen. Dissertation,
Bauhaus-Universitt Weimar, Fakultt Bauingenieurwesen, Weimar, Germany.
Mikulkov, E.; Knig, M.; Tauscher, E. and Beucke, K. (2010). Knowledge-based schedule generation and
evaluation, In: Advanced Engineering Informatics, Vol. 24, No. 4, 389403.
Melzner, J., Hollermann, S., and Bargstdt, H.-J. (2012a). A multi-criteria decision-support approach for
construction planning, 9th European Conference on Product and Process Modelling, Reykjavik, Iceland.
Melzner, J., Zhang, S., Teizer, J., Hollermann, S., and Bargstdt, H.-J. (2012b). Safety planning based on an
object-orientated building model, EG-ICE Conference 2012, Munich, Germany.
Melzner, J., Hollermann, S., Elmahdi, A., Le H.H., and Bargstdt, H.-J. (2012c). Pattern-Based Process
Modeling for Construction Management, 14th International Conference on Computing in Civil and Building
Engineering; Moscow, Russia.
Proporowitz, A. (2008). Baubetrieb Bauwirtschaft, Hanser-Verlag, Munich, Germany.
Rozenfeld, O., Sacks, R., Rosenfeld, Y., and Baum, H. (2010). Construction Job Safety Analysis, In: Safety
Science, Vol. 48, No. 4, 491498.
Song, P., Wang, S., and Abourizk, S. (2006). A virtual shop modeling system for industrial fabrication shops,
Simulation Modelling Practice and Theory, Vol. 14, 2006, 649-662.
Sulankivi, K., Khknen, K., Mkel, T., and Kiviniemi. M. (2010). 4D-BIM for Construction Safety Planning. In
Proceedings of the CIB 2010 World Congress. Manchester, UK.
Tauscher, E. (2011). Vom Bauwerksinformationsmodell zur Terminplanung : ein Modell zur Generierung von
Bauablaufplnen, Dissertation, Bauhaus-Universitt Weimar, Fakultt Bauingenieurwesen, Weimar, Germany.
Verordnung ber Sicherheit und Gesundheitsschutz auf Baustellen. BaustellV. (1998). Retrieved June 15, 2012,
from http://www.gesetze-im-internet.de/bundesrecht/baustellv/gesamt.pdf
Wang, H.-H., and Boukamp, F. (2009). Ontology-based Job Hazard Analysis Support. Computing in Civil
Engineering: Proceedings of the 2009 ASCE International Workshop on Computing in Civil Engineering, Austin,
Texas, 676684.
Xue, X., Shen, Q., Fan, H., Li, H. and Fan, S. (2012). IT supported collaborative work in A/E/C projects: A
ten-year review, Automation in Construction, vol. 21, 19.
170

Zhang, S., Teizer, J., Lee, J.K., Eastman, C., Venugopal, M. (2012). Building information modeling (BIM) and
safety: Automatic safety checking of construction models and schedules, In: Automation in Construction, 2012.
doi:10.1016/j.autcon.2012.05.006
Zhou, W., Whyte, J. and Sacks, R. (2012). Construction safety and digital design: A review, Automation in
Construction, vol. 22, 102111.
171

AUTOMATED AS-BUILT MODELING WITH SPATIAL AND VISUAL
DATA FUSION
Zhenhua Zhu
Department of Building, Civil, and Environmental Engineering, Concordia University, Canada
ABSTRACT: As-built models are useful in many civil infrastructure assessment and management tasks, including
renovation/retrofit planning, maintenance scheduling, progress monitoring, etc. However, the current as-built
modeling process requires a lot of manual editing and correction work. The large volume of manual work
increases the modeling cost, and therefore limits the use of as-built models in the vast majority of projects. In order
to overcome these limitations, this paper presents a novel method for civil infrastructure as-built modeling. The
method relies on the fusion of the spatial and visual data collected from a portable sensing camera, Kinect, which
can provide a stream of point clouds with per-point color information in real time. Under the method, pairs of the
retrieved point clouds are first registered by matching their spatial features and iteratively approximating their
closest points. Meanwhile, civil infrastructure elements, such as columns, are recognized based on their visual
patterns. The recognition results can be used to label the points in the clouds. This way, the points with the same
label can be grouped and then modeled separately. So far, the proposed method has been tested with a pilot study.
The results showed its effectiveness and promise in civil infrastructure as-built modeling.
KEYWORDS: As-built modeling, data fusion, cloud registration, object recognition, automation, Kinect camera
1. INTRODUCTION
Civil infrastructure as-built models record the infrastructure's existing conditions, including the actual details of
the architectural, structural, and mechanical, electrical, and plumbing (MEP) elements (IBC, 2011). Therefore,
these models are useful for multiple infrastructure assessment and management tasks, such as project quality
control, renovation/retrofit planning, and maintenance scheduling. For example, as-built models could be used to
identify and quantify design/construction errors or deviations, which significantly reduced the amount of rework
during the construction phase of a project (Liu et al. 2012). Also, as-built models could facilitate the MEP design
coordination for renovating and retrofitting civil infrastructure, which was expected to reduce the MEP
field-to-finish workflow by 60% (ClearEdge3D, 2012).
Although civil infrastructure as-built models have proved useful in many project-related applications, the current
process of creating such models requires a lot of manual editing and correction work. Previous studies indicated
that the overall modeling process could take up over two thirds of manual modeling effort (Sternberg et al. 2004;
Jaselskis et al. 2005). Similar findings were also reported by professional modelers. So far, even when modeling
simple civil infrastructure, the majority of the modeling effort has been exhausted on the manual conversion of
as-built data (images, point clouds, etc.) into an object-oriented model (Brilakis et al. 2011).
This large volume of manual editing and correction work increases the modeling cost, and makes the overall
modeling process time-consuming, which limits the use of as-built models in the vast majority of construction and
renovation/retrofit projects (Brilakis et al. 2011). Pettee (2005) observed that the generation of the as-built modes
was not a value adding task for most general contractors, unless the as-built modeling cost was significantly
reduced, and the models could be constantly updated and closely reviewed during construction.
In order to facilitate the current as-built modeling process, this paper presents a novel method, which relies on the
fusion of two different types of sensing data (i.e. point clouds and video frames) to model the as-built conditions
of civil infrastructure. Under the method, as-built data are first collected from a portable sensing camera, Kinect.
The camera can provide a stream of point clouds with per-point color information almost in real time. Then, the
retrieved point clouds are registered by matching their spatial features and iteratively approximating their closest
points. Civil infrastructure elements, such as columns, are recognized based on their unique visual patterns. The
visual recognition results of the infrastructure elements can provide the points in the clouds with certain semantic
labels, such as "column". This way, the cloud points can be automatically categorized into multiple parts and
modeled separately.
172

Compared with most existing modeling techniques, which only made use of one type of sensing data, the
proposed method can significantly reduce the as-built sensing time for civil infrastructure, and recognize civil
infrastructure elements from the sensing data in a fast way. So far, a pilot study has been performed to test the
method. The results from the study have shown the effectiveness and promise of the method in civil infrastructure
as-built modeling, especially in the area of building indoor environments modeling.
2. RELATED WORK
The current process for civil infrastructure as-built modeling starts with the collection of infrastructure as-built
conditions using remote sensing devices. Typically, the as-built data are collected from multiple locations.
Therefore, they need to be registered under one coordinate system (i.e. 3D reconstruction). Also, consider the
sensing data do not contain any semantic or high-level geometric information (e.g. whether there is a column in
the sensing data and if yes, what is the shape of this column) (Tang et al. 2010). Civil infrastructure elements need
to be further recognized from the sensing data (i.e. infrastructure element recognition), so that they can be used
for domain related problem solving.
Multiple research studies have been initiated to address the manual and time-consuming nature of the current
as-built modeling process. Based on the types of sensing data they work on, existing research studies can be
mainly divided into two categories. In the first category, the studies focus on the spatial data (i.e. point clouds)
directly captured by laser scanners, while the studies in the second category work on the visual data (images or
videos) captured by digital or video cameras.
2.1 Point cloud based modeling
Laser scanners can capture the detailed as-built conditions of civil infrastructure. One laser scan may contain
millions of three dimensional (3D) points. Using this rich information, Okorn et al. (2010) proposed an idea of
creating building floor plans by projecting the collected points onto a vertical (z-) axis and a ground (x-y) plane.
In addition to the modeling of floor plans, Xiong and Huber (2010) also employed the conditional random fields
to model infrastructure elements with planar patches, such as building walls. The openings on the walls can be
further located by considering the low data density around the openings (Ripperda and Brenner, 2009) or using a
support vector machine (SVM) classifier (Adan and Huber, 2011).
So far, there has been little work specifically developed to automate the modeling of civil infrastructure with
point clouds. However, several general point-based object recognition methods were created. These methods
established the fundamentals for the recognition of different types of civil infrastructure elements, which could
facilitate the overall infrastructure modeling process and relieve the manual recognition of 3D points from the
clouds. The recognition of these methods mainly relied on the points' spatial features. These features can be
described globally or locally (Stiene et al. 2006). Global descriptors, such as the ones using the multi-shell
extended Gaussian images (Wang et al. 2007) or rotation invariant spherical harmonic representations (Kazhdan
et al. 2003) captured all the geometrical characteristics of a desired object. Therefore, they are discriminative but
not robust to cluttered scenes (Patterson et al. 2008). Local descriptors, such as the ones using the shapeme
histogram (Shan et al. 2006) or semi-local shape features (Huber et al. 2004), improved the recognition
robustness, but they were computationally complex (Bosche and Haas, 2008).
2.2 Image/video based modeling
Laser scanners are expensive and not portable. Therefore, the studies in the second category target on the visual
data (images or videos) from digital or video cameras as an affordable and portable modeling alternative. Several
critical techniques have been created, including the structure-from-motion (Snavely et al. 2006), multi-view stereo
matching (Furukawa and Ponce, 2010), and simultaneous location and mapping (Durrant-Whyte and Bailey.
2006). Built upon them, many image/video-based approaches for modeling civil infrastructure were established,
and real-time 3D reconstruction of built environments with these approaches became possible (Geiger et al. 2011).
For example, 3D models of high visual quality can be automatically derived even from single facade images of
arbitrary resolutions (Mller et al. 2007). Also, Pollefeys et al. (2008) used the videos from a moving vehicle to
model urban environments.
173

Similar to the point cloud based modeling, object recognition methods from images or videos can be used to
facilitate the modeling of civil infrastructure. There are multiple recognition cues that can be adopted, including
color, texture, shape, and/or local invariant features. Take the research studies performed in construction as an
example. Neto et al. (2002) used color values to detect structural elements in digital pictures. Brilakis et al. (2006)
presented the concept of "material signatures", which are based on the retrieval of color and texture patterns.
Koch and Brilakis (2011) detected potholes in asphalt images, since the potholes have elliptical shapes.
Although many methods have been developed to recognize objects from images or videos, the robust recognition
of civil infrastructure elements for the infrastructure as-built modeling has not achieved yet. One main limitation
lies in the fact that the object recognition results from images or videos are limited to two-dimensional (2D)
space. Therefore, it is difficult to directly use these recognition results for the 3D infrastructure as-built modeling,
with the depth information missing .
2.3 Comparison between two categories of modeling techniques
Existing modeling techniques, no matter point cloud based or image/video based, only relied on one single type
of sensing data for modeling civil infrastructure as-built conditions. As a result, they have the benefits and
limitations associated with the type of sensing data they work on. Specifically, point cloud based modeling
methods can accurately model the detailed civil infrastructure as-built conditions, but the process of capturing
as-built data is time-consuming. Most laser scanners are not portable and the setup of a laser scanner also takes
time (Foltz, 2000). This is especially true when modeling building interiors, where a laser scanner has to be set
up at many locations to scan the building due to the existence of building interior partitions. Adan and Huber
(2011) reported that the complete scan of 40 rooms needs 225 locations. In addition, the recognition of civil
infrastructure elements in the point cloud based modeling solely relied on the spatial features of the point clouds.
Therefore, the recognition capability of civil infrastructure elements is limited. For example, it is difficult for the
point based recognition methods to differentiate between a concrete column and a wooden column, if both of
them have the same shape and size.
Compared with the point cloud based modeling, the object recognition in the image/video based modeling can be
performed using multiple recognition cues, including color, texture, shape, etc. Multiple recognition cues
increase the recognition capabilities of the methods, and make them possible to recognize civil infrastructure
elements even with different materials. The main limitation of the image/video based modeling technique lies in
its modeling accuracy. Although the modeling accuracy of the technique can rival that of laser scanning when
delicate textures are available (Seitz et al. 2006), overall, it is not accurate enough for modeling civil
infrastructure in the architectural, engineering and construction (AEC) domain (Adan and Huber, 2011). Also, the
3D reconstruction methods in the image/video based modeling are not always robust. They may fail when
experiencing texture-poor civil infrastructure elements , such as walls and floors.
All the benefits and limitations of two categories of modeling techniques have been summarized in Table 1.
According to the table, it can be found that both modeling techniques are complementary to each other.
Therefore, an optimum solution for civil infrastructure as-built modeling is expected to combine both techniques:
3D reconstruction is performed through the registration of point clouds, while civil infrastructure elements are
recognized from images or videos. This way, the limitations of both modeling techniques can be overcome.
Table 1: Benefits and limitations of two categories of modeling methods
Point-cloud based modeling Image/video based modeling
3D reconstruction Accurate and detailed Not accurate and detailed
Infrastructure element recognition One recognition cue only (spatial
feature)
Multiple recognition cues (color,
texture, shape, etc.)
3. PROPOSED METHOD
This paper proposes a novel method for modeling the as-built conditions of civil infrastructure through the
fusion of spatial and visual data. The method consists of three main components: 1) point clouds registration for
174

3D reconstruction, 2) civil infrastructure element recognition, and 3) civil infrastructure element modeling. First,
a portable visual and spatial sensing camera, Kinect, is used to capture civil infrastructure conditions. The camera
provides a stream of point clouds with per-point color information. The retrieved sensing data are reorganized into
two parts: point clouds and video frames. Each point in a cloud maintains a one-on-one correspondence to a
pixel in a video frame. Then, point clouds are registered by matching their spatial features and iteratively
approximating their closest points. On the other hand, civil infrastructure elements, such as columns and walls,
are recognized based on their visual patterns. The recognition results are used to classify the points in the point
clouds with certain semantic labels, such as "columns" or "walls". The points with the same label can be grouped
and modeled individually, since they belong to the same type of civil infrastructure elements. The whole
framework has been illustrated in Fig. 1, and details are given below.
Fig. 1: Framework of the proposed method
3.1 Point clouds registration
It is common that civil infrastructure needs to be surveyed with multiple scans to capture its complete as-built
conditions. Each scan produces a point cloud. The registration here is to merge and align the 3D point clouds
from multiple scans into one single point cloud under a pre-defined global coordinate system. This way, the
intersecting areas between any point clouds can overlap perfectly (Fig. 2).

Video
frames
Point
clouds
Registration
Data processing
Element
recognition
Recognized
elements
Registered
point clouds
Element modeling
Modeled
elements
Hybrid sensing
Civil
infrastructure
Point cloud scans
with color values
Data preparation
Reorganization
Point
clouds
Video
frames
1:1
175

Fig. 2: Point cloud registration from two scans
In this paper, the global coordinate system is defined as the coordinate system used by the point cloud in the first
scan (i.e. scan 1). The point clouds in other scans (scan 2, 3, ...) are aligned to the cloud in scan 1. The alignment
estimation is performed through two steps. First, for scan , the alignment of its point cloud to
the point cloud from its preceding scan (scan ) is estimated. This is referred to as pair-wise alignment.
When the pair-wise alignment information is available, the alignment of one point cloud to the global coordinate
system can be simply calculated through multiple pair-wise alignment matrix multiplication (Fig. 3).
Fig. 3: Alignment of point cloud scans to the global coordinate system
(a) Point cloud A (b) Point cloud B
(c) Point cloud A and B before registration
(d) Point cloud A and B after registration
Point
cloud
scan 1
Point
cloud
scan 2
Point
cloud
scan n
Point
cloud
scan 3
[ ] ...
12
= M
[ ] ...
23
= M
34
M
n n
M
1
23 12 13
M M M =
n n n
M M M M
1 23 12 1
......

=
j i
M --- 4 x 4 matrix used to align the point cloud j to the point cloud i
176

The pair-wise alignment matrix between two neighboring point clouds is estimated following the steps described
in Fig. 4. First, the key points and features are individually retrieved for each point cloud. The key points are the
interest points, such as corners in the scene. The features are the vectors that can describe certain special
properties of the points. Here, the fast point feature histograms (FPFH) descriptor is used, since it significantly
reduces the computational complexity and provides the possibility for real-time or near real-time applications
(Rush, 2010). Based on the key points and features retrieved, the features that are common in both point clouds
can be found, which form pairs of the matched features between two point clouds. This way, the overlapping
parts of both point clouds can be determined and the alignment matrix from one point cloud to the other is
estimated. However, this alignment matrix may be not accurate enough due to the potential error in finding the
pairs of corresponding features. For this reason, the matrix is further refined with the iterative closest point (ICP)
algorithm (Besl and McKay, 1992).
Fig. 4: Pair-wise alignment
3.2 Infrastructure element recognition
The registration makes the point clouds from multiple scans aligned and merged to one point cloud. As
mentioned in the previous section, the 3D points do not have any semantic or high-level geometric information.
In order to retrieve this information, civil infrastructure elements contained in the point cloud need to be
recognized and modeled. However, the direct recognition and modeling of the elements from the point clouds has
proven difficult and computationally complex, unless there is detailed prior information for the elements, such as
the as-planned civil infrastructure model, as a reference (Bosche and Hass, 2008). In this paper, the problem is
overcome through the recognition of civil infrastructure elements in the video frames provided in the stage of
data preparation in Fig. 1.
The recognition of civil infrastructure elements mainly relied on their visual patterns. Most civil infrastructure
elements, such as columns, beams, and walls, are characterized with simple geometric and material
configurations. For this reason, the contour and material features of the elements are used as the recognition cues.
A general approach for recognizing civil infrastructure elements in a video frame follows four basic steps (Fig. 5):
1) retrieving contour features through edge/line detection; 2) extracting texture features with digital filtering, 3)
classifying the material information of the elements with machine learning, and 4) locating civil infrastructure
elements based on their topological configurations of the contour features and material information in the video
frame. This approach can be applied for the recognition of different types of civil infrastructure elements with
appropriate and slight customization. For example, concrete columns (rectangular or circular) in a video frame
are dominated by long near-vertical lines (contour features) and concrete surfaces (material information).
Therefore, concrete columns can be located by searching such cues in the video frame. This idea of recognizing
concrete columns has been implemented by the writer. More details can be found in the writer's pervious work
(Zhu and Brilakis, 2010).

Point
cloud A
Key-point
retrieval
Feature
description
Key
points
FPFH
features
Feature
matching
at key
points
Pairs of
feature
points
Initial
alignment
estimation
ICP
alignment
Point
cloud B
Key-point
retrieval
Feature
description
Key
points
FPFH
features
Initial
alignment
matrix
Refined
alignment
matrix
177

Fig. 5: Civil infrastructure element recognition
The recognition results are further used to label the 3D points in the point clouds, according to the one-on-one
point-pixel relationship between the point clouds and video frames (Fig. 1). This way, the 3D points in the point
clouds can be identified with different element labels. An element label, such as "wall" or "column", indicates
the type of civil infrastructure elements. For the points with different element labels, it means that they belong to
different types of civil infrastructure elements, and as a result they can be modeled accordingly.
3.3 Infrastructure element modeling
The purpose of modeling civil infrastructure elements is to define and organize the 3D points in the point cloud
using a software neutral format. In order to do so, the points with the same element label are grouped first. Then,
for each set of grouped points, a geometric primitive that represents the geometry of that set of the grouped
points is created using the least square orthogonal distance fitting. The parameters of the geometric primitive are
retrieved to calculate the attributes of the element. The attributes include but are not limited to local placement,
dimension, cross section area, etc. Finally, the element and its attributes are matched to their best fit entities
specified in the Industrial Foundation Classes (IFC) data schema (Nisbet and Liebich, 2007).
Fig. 6: Modeling of a concrete column
Take the modeling of a column for an example (Fig. 6). Suppose a circular column is recognized in a video
frame. Then, the 3D points in the corresponding point cloud are identified. A cylinder geometric primitive are
created. The attributes of the column, such as radius, depth, and extruded direction, are retrieved based on the
properties of the cylinder geometric primitive. After that, an IfcColumn-type entity will be instantiated for this
column. For each retrieved column attribute, its corresponding entity is also created and attached to this
IfcColumn entity. All the entities are organized as illustrated in Fig. 6.
The final result from the proposed method is an object-oriented model with civil infrastructure elements
recognized and organized following the IFC specifications. The IFC specifications were developed by
buildingSMART to facilitate interoperability in the architecture, engineering and construction industry (IBC,
2011). Therefore, this model is expected to be seamlessly integrated to existing software tools for multiple
engineering applications, such as structural integrity analysis, building energy efficiency analysis, etc.
Video
frames
Element
labels
Material
classification
Texture
retrieval
Material
information
Texture
features
Element
locating
Contour
retrieval
Contour
features
IfcProductRepresentation
IfcElement
IfcRoot
IfcColumn
ifcObjectPlacement
178

4. PILOT STUDY
The proposed method has been partially implemented using Microsoft Visual C++ with the support of OpenCV
(Open Source Computer Vision) and PCL (Point Cloud Library). Modeling the indoor environments of a
building (EV. building) in Concordia University, Canada was selected as a pilot study to test the proposed
method. The as-built conditions of the building was captured by a Kinect camera (Fig. 7). The maximum
resolution of the camera is 640x480, which means it can capture 307,200 points per frame. The frames per
second (FPS) for capturing point clouds could reach 30 Hz. One example of the captured point clouds with color
information was illustrated in Fig. 8.
Fig. 7: Kinect camera connected to a laptop Fig. 8: Point cloud with per-point color information
Fig. 9 showed the point cloud without the color information (Fig. 9a) and the corresponding color image (Fig. 9b)
that were reorganized from the data captured by the camera. The results of using the proposed method in the
pilot study were illustrated in Fig. 10 - 12. Fig. 10 was the result of the registration of two point clouds viewed
from the top. Fig. 11 showed the recognition of a concrete column in a video frame (2D), and the recognition of
the column points in the corresponding point cloud (3D).
Fig. 9: Reorganized data
(a) Point cloud without color information (b) Color image
179

Fig. 10: Point clouds registration: unregistered two point clouds (left) and registered point clouds (right)
Fig. 11:Concrete column recognition: recognition in 2D (left) and recognition in 3D (right)
5. CONCLUSIONS AND FUTURE WORK
Civil infrastructure as-built models are useful for multiple infrastructure assessment and management tasks, but
the current process of creating such models is labor-intensive and time consuming. In order to facilitate the
current as-built modeling process, this paper presents an automated method to model civil infrastructure as-built
conditions. The method relies on the fusion of two different types of sensing data, 3D point clouds and video
frames captured by a portable sensing camera, Kinect. The camera can provide a stream of point clouds and video
frames, where each 3D point corresponds to a 2D video pixel. The method consists of three main steps: 1) point
clouds registration, 2) visual recognition of civil infrastructure elements, and 3) IFC-based infrastructure
elements modeling. The final result produced by the method is expected to be an object-oriented model with
civil infrastructure elements recognized and organized following the IFC specifications.
So far, a pilot study has been performed to test the proposed method. The pilot study selected an office in the EV
building in Concordia University, Canada as the test environments. The results from the study have shown that
the method could successfully register the point clouds from multiple scans, recognize concrete columns in video
frames, and identify 3D concrete column points in the point cloud based on the recognition results. Compared
with existing modeling techniques, which only made use of one type of remote sensing data, the proposed
method can significantly reduce the as-built sensing time for civil infrastructure, and recognize and model civil
infrastructure elements from the sensing data in a fast way. The effectiveness of the method from the test
indicates the promise of the method in civil infrastructure as-built modeling, especially in the area of indoor
environments modeling.
Future work will consider the following aspects: 1) integrating the recognition of other types of civil
infrastructure elements into the method, 2) implementing the work of infrastructure element modeling, and 3)
comprehensively testing the proposed method with more test scenarios.
180

6. ACKNOWLEDGEMENT
This paper is based in part upon work supported by the National Science and Engineering Research Council
(NSERC) of Canada. Any opinions, findings, and conclusions or recommendations expressed in this paper are
those of the author(s) and do not necessarily reflect the views of the NSERC.
7. REFERENCES
Adan, A. and Huber, D. (2011). 3D reconstruction of interior wall surfaces under occlusion and clutter. Proc. of
3DPVT 2011, Hangzhou, China.
Besl, P. and McKay, H. (1992). "A method for registration of 3-D shapes." IEEE Transactions on Pattern Analysis
and Machine Intelligence, 14(2): 239 - 256.
Bosche, F. and Haas, C. (2008). Automated retrieval of 3D CAD model objects in construction range images,
Auto. in Constr., 17(4): 499-512.
Brilakis, I., Soibelman, L., and Shinagawa, Y. (2006). "Construction Site Image Retrieval Based on Material
Cluster Recognition", Journal of Advanced Engineering Informatics, 20(4): 443 - 452.
Brilakis, I., German, S. and Zhu, Z. (2011) "Visual pattern recognition models for remote sensing of civil
infrastructure", Journal of Computing in Civil Engineering, 25(5): 388 393.
ClearEdge3D. (2012). "3D imaging services reduces MEP workflow on Pharma project by 60% using EdgeWise
Plant 3.0." < http://www.clearedge3d.com/> (June 10, 2012)
Durrant-Whyte, H. and Bailey, T. (2006). Simultaneous localization and mapping, IEEE Robotics and
Automation Magazine, 13(2): 99 110.
Foltz, B. (2000). Application: 3D laser scanner provides benefits for PennDOT bridge and rockface surveys.
Prof. Surv., 20(5): 2228.
Furukawa, Y. and Ponce, J. (2010). Accurate, dense, and robust multi-view stereopsis, IEEE Trans. on Pattern
Analysis and Machine Intelligence, 32(8): 1362-1376.
Geiger, A., Zeigler, J. and Stiller, C. (2011). Stereoscan: Dense 3D reconstruction in real-time, Proc of IEEE
Intelligent Vehicles Symposium, BadenBaden, Germany, 2011.
Huber, D., Kapuria, A., Donamukkala, R., and Hebert, M. (2004). Parts-based 3D object recognition, Proc. of
CVPR 2004, Washington DC.
Institute for BIM in Canada (IBC). (2011). Environmental scan of BIM tools and standards, Canadian
Construction Association
Kazhdan, M., Funkhouser, T. and Rusinkiewicz, S. (2003). Rotation invariant spherical harmonic representation
of 3D shape descriptors, Symposium on Geometry Processing, pp. 156-164.
Koch, C. and Brilakis, I. (2011) "Pothole Detection in Asphalt Pavement Images", Journal of Advanced
Engineering Informatics, Elsevier, 25(3): 507 515.
Liu, X., Eybpoosh, M., and Akinci, B. (2012). "Developing as-built building information model using
construction process history captured by a laser scanner and a camera." Construction Research Congress(CRC)
2012, West Lafayette, IN., May, 21-23, 2012, pp:1232 - 1241.
Murphy, M., McGovern, E., and Pavia, S. (2006). "The processing of laser scan data for the analysis of historic
structures in Ireland". Proc. of the 7th international symposium on virtual reality, archaeology and culture
heritage, Nicosia, Cyprus, 30th Oct - 4th November 2006, pp: 135-139.
Mller, P., Zeng, G., Wonka, P. and Van Gool, L. (2007). "Image-based procedural modeling of facades". ACM
Transactions on Graphics, Vol. 26, No. 3, Article 85, pp: 1-9.
Neto, J. A., Arditi, D., and Evens, M. W. (2002). Using colors to detect structural components in digital
181

pictures. Comput. Aided Civ. Infrastruct. Eng., 17, 6167.
Nisbet, N. and Liebich, T. (2007). "ifcXML implementation guide." International Alliance for Interoperability,
<http://www.stanford.edu/group/narratives/classes/08-09/CEE215/ReferenceLibrary/Industry%20Foundation%2
0Classes%20(IFC)/ifcXML%20General/ifcXML%20Implementation%20Guide.pdf > (June 10, 2012)
Patterson, A. IV, Mordohai, P. and Daniilidis, K. (2008). Object detection from large-scale 3D datasets using
bottom-up and top-down descriptors, Proc. of ECCV 2008, Marseille, France.
Pettee, S (2005). "As-builts - problems and proposed solutions." CM eJournal, First Quarter: 1-19.
<http://cmaanet.org/files/as-built.pdf>(June 10, 2012)
Pollefeys, M., Nister, D., Frahm, J.-M., Akbarzadeh, A., Mordohai, P., Clipp, B., Engels, C., Gallup, D., Kim,
S.-J., Merrell, P., Salmi, C., Sinha, S., Talton, B., Wang, L., Yang, Q., Stewenius, H., Yang, R., Welch, G., Towles,
H. (2008). "Detailed real-time urban 3D reconstruction from video". International Journal of Computer Vision,
78(2): 143 - 167.
Ripperda, N. and Brenner, C. (2009). "Application of a formal grammar to facade reconstruction in
semiautomatic and automatic environments". Proc. of AGILE conference on geographic information science,
Hannover, Germany, 2009.
Rusu, R. (2010). " Semantic 3D object maps for everyday manipulation in human living environments." Ph.D.
thesis, TECHNISCHE UNIVERSITT MNCHEN.
Seitz, S., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. (2006). A comparison and evaluation of
multi-view stereo reconstruction algorithms, Proc. of CVPR 2006, New York, NY.
Shan, Y., Sawhney, H., Matei, B., and Kumar, R. (2006). Shapeme histogram projection and matching for partial
object recognition, IEEE Trans. on Pattern Analysis and Machine Intelligence, 28 (4): 568-577.
Snavely, N., Seitz, S., and Szeliski, R. (2006). Photo tourism: exploring image collections in 3D, Proc of
SIGGRAPH 2006, Boston, MA.
Stiene, S.,Lingemann, K.,Nuchter, A., and Hertzberg, J. (2006). Contour-based object detection in range images,
Proc. of 3DPVT 2006, Chapel-Hill, NC.
Sternberg, H., Kersten, T., and Kinzel, R. (2004). Terrestrial 3D laser scanning - data acquisition and object
modeling for industrial as-built documentation and architectural applications. Proc. of ISPRS, Vol. XXXV, pp.
942-947.
Tang, P., Huber, D., Akinci, B. Lipman, R., and Lytle, A. (2010). Automatic reconstruction of as-built building
information models from laser-scanned point clouds: a review of related techniques, Automation in Construction,
19(7): 829-843.
Jaselskis, E.J., Gao, Z., Walters, R. (2005). Improving transportation projects using laser scanning, J. Constr.
Eng. & Manage. 131(3): 377-384.
Okorn, B., Xiong, X., Akinci, B. and Huber, D. (2010). Toward automated modeling of floor plans, Proc. of
3DPVT 2010, Paris, France.
Wang, D., Zhang, J., Wong, H., and Li, Y. (2007). 3D model retrieval based on multi-shell extended gaussian
image, Lecture Notes in Computer Science: Advances in Visual Information Systems, pp. 426-437
Xiong, X. and Huber, D. (2010). Using context to create semantic 3D models of indoor environments, Proc. of
BMVC 2010, Aberystwyth, UK.
182

IMMERSIVE VIRTUAL REALITY SYSTEM FOR BIM
Julian Kang & Adithya Ganapathi
Texas A&M University, U.S.A.
Hussam Nseir
Satterfield and Pontikes Construction, U.S.A.
ABSTRACT: Issues relative to reviewing the 3D Building Information Model (BIM) during the coordination
meetings were on the rise, as more construction professionals using BIM. The view of the model, however, is
naturally affected by the size of the computer monitor especially when multiple team members review the model at
the same time. Obviously the image of a 3D model projected on a large screen would enhance the team members
experience in reviewing the model. If users are surrounded by multiple large screens displaying the
viewer-centered images of the 3D model, they would even feel that they are within the 3D model to some extent.
The CAVE system, first introduced at the SIGGRAPH in 1992, demonstrated that users were indeed getting some
sense of presence in the 3D model when they were surround by 3 walls of screens and floor projecting the
viewer-centered image, which was interactively reacting to the users motion. The CAVE system therefore is
expected to facilitate construction professionals to browse the Building Information Model (BIM) with enhanced
sense of presence. Most CAVE systems developed so far, however, are designed for a single user. While one person
is browsing the model, others are just watching him/her working with the CAVE system. Most CAVE systems use
their own format to contain 3D model, therefore it would take time to convert a BIM file along with its information
for the CAVE systems. The research team at Texas A&M University developed an Immersive Virtual Reality system
using commercial BIM applications in order to handle these challenges. Unlike other CAVE systems, they used the
commercial BIM application, Autodesk NavisWorks, to generate viewer-centered images of the 3D model and
project them on three walls consisting of 12 LCD monitors. This paper presents how this system was developed.
The paper also presents some lessons the research team learned while industry BIM specialists evaluated the
BIM/CAVE system.
KEYWORDS: BIM, Immersive Virtual Reality, CAVE
1. INTRODUCTION
For years, miniature models have been used to enhance our understanding of the architects design intention. As
3D CAD technology was getting advanced, many contractors started using it to visualize the architects design
intention in 3D world. Compared to using miniature models, construction professionals found it less time
consuming, more flexible and efficient to use 3D CAD technology for visually presenting the buildings to be built.
With the advance of Building Information Modeling (BIM) technology in recent years, it is not difficult now to
find those contractors using BIM during preconstruction coordination meetings to detect clashes between the
mechanical, electrical, and plumbing (MEP) components and other building components.
Browsing the 3D computer model of the buildings to be built for clash detection, however, is naturally affected by
the size of the computer monitor especially when multiple team members are reviewing the model at the same time
during preconstruction coordination meetings.
Obviously the 3D model presented on a large screen would enhance the team members experience in reviewing
clashes between the building components. If users are surrounded by multiple large screens displaying the
viewer-centered images of the 3D computer model, they would even feel that they are within the computer model
to some extent.
This subjective feeling of being in the model, which is also defined as the sense of presence, is often affected
by the size of the computer display (De Leo, et al. 2011; Usoh et al. 2000). The Cave Automatic Virtual
Environment (CAVE), first introduced at the SIGGRAPH in 1992, demonstrated that users were indeed getting
some sense of presence in the 3D model when they were surround by 3 walls of screens and floor projecting the
viewer-centered image, which was interactively reacting to the users motion.
The CAVE system is a room of 10 feet wide, 10 feet long, and 9 feet tall, where a viewer-centered high-resolution
computer images are projected on the walls and the floor. Although one may get only a limited amount of sense of
183

presence in the CAVE system (Sutcliffe et al. 2008), its capacity of getting the users exposed to a virtual world can
be used for various applications. For instance, the CAVE system could be used to walk through a destroyed nuclear
power plant that is emitting radiations (Brown-VanHoozer et al., 1996). It could be used to train the Hubble space
telescope flight team (Loftin and Kenney, 1995) or medical team (Lee et al. 2007).
It is reasonable to anticipate that the CAVE system could facilitate construction professionals to browse the
Building Information Model (BIM) with enhanced sense of presence. Those who are in the CAVE system looking
around the BIM may feel that they are actually walking in the building.
2. MOTIVATION
The original CAVE used the Silicon Graphics Graphic Library to create the image of the surrounding objects.
The polygons depicting the surrounding objects were coded along with other functions of the CAVE system and
compiled into a binary code (Cruz-Neira et al. 1992; Cruz-Neira et al. 1993). Another CAVE system developed
at Virginia Tech used the OpenGL Performer .PFB file format to keep the polygon data in a computer file. The
OpenGL Performer is a C++ simulation and visualization graphics library written by SGI.
Although some efforts were made to develop computer applications that can facilitate to transform the 3D model,
for example, produced using Autodesk 3D Studio into the CAVE system
(http://www.sv.vt.edu/classes/ESM4714/cave/prgintro.html), the robust transfer of a 3D model and associated
information into the CAVE system is often non-trivial at best as noted by the research team in the Immersive
Environments Laboratory (IEL) at Pennsylvania State University (Otto et. al 2005). It is reasonable to assume
that the process of transforming the BIM data into the CAVE system is not simple. Engineering data attached to
the BIM could be lost while they get transformed because of lack of interoperability between major BIM
applications and CAVE systems. These challenges hindered professionals in the construction industry from best
utilizing the CAVE system.
One then might be wondering if we can fabricate the CAVE system using commercial BIM applications. If it is
possible, we might be able to use the CAVE for BIM without worrying about the file transformation issues or
any data loss. For example, if the CAVE system can be developed on top of Autodesk NavisWorks, the file
conversion process would be seamless because NavisWorks is designed to open the model created by major BIM
applications. Data loss during the file conversion process would be minimized because NavisWorks can open the
model saved in as many as 19 different data formats.
3. FABRICATION
In Fall 2011, in order to remedy the shortcomings of the existing CAVE systems in terms of reviewing the
Building Information Models (BIM), we fabricated the BIM CAVE system using three video walls and Autodesk
NavisWorks. Autodesk NavisWorks is one of the popular BIM reviewing application. As shown in Figure 1, each
video wall consisting of four 46-inch ultra-thin bezel LCD flat monitors is placed to create a partial octagonal
space.

Figure 1. Immersive environment created using video walls and Autodesk NavisWorks

184

Each video wall is connected to a dedicated personal computer. Autodesk NavisWorks is use to create the
rendered images of the building model in real time. In order to produce the viewer-centered images of the
building model, we developed a computer application that controls the camera position and aiming angle in the
NavisWorks model. The NavisWorks Application Programming Interface (API) was used to enforce three
computers to place the camera at the same position in the NavisWorks model. The cameras aiming angle in each
NavisWorks model is determined by the interior angle between two video walls. On each computer, the same
NavisWorks model is loaded. The computer assigned to the video wall in center is designated as master computer,
which is communicating with the user for the camera location in the model. Two computers on each side of the
master computer keep monitoring the camera location in the master computer, and place its camera at the same
location in the model. The collection of these rendered images projected on each video wall is creating the
viewer-centered image of the NavisWorks model.
4. EVALUATION
Our attempt to create an immersive environment using a commercial BIM reviewing application was a success.
It created the viewer-centered images of the NavisWorks model almost in real time as the users navigated around
the model. The camera position and its aiming angle were successfully controlled in each NavisWorks instance.
There was slight time gab in the camera walk between the master computer and two client computers, but it did
not bother the users too much in terms of gaining the sense of presence.
Six BIM specialists in the construction industry were invited to take a look at our system. One architect, one
BIM manger, and four project managers examined how our system created the viewer-centered images of the
NavisWorks model in real time. All participating evaluators agreed that the viewer-centered images produced put
them in an immersive environment to some extent.
However, they also addressed some shortcomings of our system mainly caused by the lack of the ceiling with the
top screen attached. Since most construction practitioners use BIM for clash detection between the MEP system
and other building components, it is critically important to facilitate the users to take look at the MEP system.
When the top screen is not available, users need to tilt the camera angle up in order to look at the MEP system.
When the user tilts up angle of the camera in the master model, the entire viewer-centered images in our systems
get slanted, which could mess up the users sense of gravity, and result in the reduction of their sense of presence
while reviewing the model. One can easily assume the adversarial impact caused by this shortcoming. We are
currently investigating various ways of placing the LCD monitors on top of the octagonal space.
BIM has been used to discuss the constructability issues. Some advanced contractors combine their model with
the construction schedule to produce a 4D construction simulation, which enables them to see the spatial
relationship between the building components to be built, equipments they need to use, and the space where
these equipments are placed. The use of 4D construction simulation enables them to make proactive decisions on
the constructability issues and get the job site activities executed as seamlessly as possible. Industry
professionals examined our system noted that it would be help construction professionals discuss the
constructability issues if the construction sequence is presented in our system in real time. We are currently
upgrading our application to gain the control over the timeline animation in NavisWorks. Upon gaining the
control over timeline animation, users might be able to see the entire construction sequence as if they are on the
job site.
5. CONCLUSION
We attempted to create an immersive environment where users can review the Building Information Model with
enhanced sense of presence. In order to remedy shortcomings of the existing CAVE system in terms of
understanding BIM data file, we fabricated a new CAVE system using a commercial BIM application, Autodesk
NavisWorks. Once the same NavisWorks model is loaded on three computers connected to each video wall of
our CAVE system, the plug-in application we developed synchronizes the came position in each NavisWorks
instance. It then enforces NavisWorks to produce the rendered images of the model with different aiming angle
in real time to produce the viewer-centered images of the model.
Industry professionals who examined our system were satisfied with the speed of generating the viewer-centered
images of the NavisWorks model projected in three video walls. However, they also pointed out the
shortcomings of our system. One of these shortcomings is mainly caused by the lack of the ceiling with the top
185

screen attached. When the user tilts up the camera in order to review clashes between the MEP system and other
building components on the ceiling, the entire images projected on the video walls get slanted, which could
result in the reduction of the users sense of presence. We are currently investigating various ways of placing the
LCD monitors on top of the semi-octagonal space, but stitching the images projected on the rectangular top
screens and those images projected on the video walls seamlessly in order to create a viewer-centered image
would be challenging.
The advantage of our system is that it can be easily extended. With 7 video walls, for example, our system could
create a full octagonal space where the project team can get fully surrounded by the viewer-centered images
projected on the video walls. Our system presented in the paper is a big step towards to a BIM/CAVE where
users can get exposed to a full immersive virtual environment for Building Information Modeling.
6. REFERENCES
Brown-VanHoozer, S. A., Singleterry, R. C., and King, R. W., 1996. Virtual Environments for Nuclear Power Plant
Design, Nuclear Plant Instrumentation, Control and Human-Machine Interface Technologies, American Nuclear
Society International Topical Meeting (NPIC & HMIT 96).
Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., Kenyon, R. V., and Hart, J. C., 1992. The CAVE: Audio Visual
Experience Automatic Virtual Environment, Communications of the ACM, 35 (6), pp. 64-72.
Cruz-Neira, C., Sandin, D. J., DeFanti, T. A., 1993. Surround-Screen Projection-Based Virtual Reality: The Design
and Implementation of the CAVE, SIGGRAPH '93 Proceedings of the 20th Annual Conference on Computer
Graphics and Interactive Techniques.
De Leo, G., Goodman, K. S., Radici, E., Secrhist, S. R. and Mastaglio, T. W. (2011). Level of Presence in
Team-Building Activities: Gaming Component in Virtual Environments, The International Journal of Multimedia
& Its Applications (IJMA), 3(2), pp. 1-10.
Lee, C. H., Liu, A., Del Castillo, S., Bowyer, M., Alverson, D., Muniz, G., and Caudell, T., 2007. Towards and
Immersive Virtual Environment for Medical Team Training, Studies In Health Technology and Informatics, V. 125,
pp. 274-279.
Loftin, R. B. and Kenney, P. J., 1995. Training the Hubble Space Telescope Flight Team, IEEE Computer Graphics
and Applications, 15(5), pp. 31-37.
Otto, G. H., Messner, J. I., and Kalisperis, L., 2005. Expanding the Boundaries of Virtual Reality for Building
Design and Construction, ASCE International Conference on Computing in Civil Engineering. Cancun, Mexico.
Sutcliffe, A., Gault, B., Fernando, T., and Tan, K., 2008. Investigating Interaction in CAVE Virtual Environments,
ACM Transactions on Computer-Human Interaction, 13(2), pp. 235-267.
Usoh, M., Catena, E., Arman, S., and Slater, M., 2000. Using Presence Questionnaires in Reality, Presence, 9(5),
pp. 497-503.

186

MAPPING BIM MODELS AND 3D GIS MODELS USING
INSTANCE-BASED AND LINGUISTIC METHODS
Yichuan DENG & Jack Chin Pang CHENG
The Hong Kong University of Science and Technology
ABSTRACT: Three-dimensional (3D) digital models are the basis for storing and managing building
information in building information modeling (BIM). BIM not only provides 3D geometric information but also
the semantic information about component types (e.g. walls, columns) and attributes. At the same time, people in
the geographical information system (GIS) domain increasingly use 3D building models to visualize
geographical objects and to support activities like urban planning and noise management. The 3D building
models in the BIM domain and those in the GIS domain share many similar features and the conversion between
them could reduce the effort of re-modeling. While there are many studies about interoperability among various
BIM standards, the study of integration between BIM standards and GIS standards is still lacking.
In recent years, CityGML (City Geography Markup Language) has been developed in the GIS community as a
data standard to represent the geometry and semantics of city objects. The conversion between BIM models and
3D CityGML models is challenging due to the different terminologies and modeling paradigm of the two
standards. This paper presents our attempts to develop mappings between BIM models in Industry Foundation
Classes (IFC) standard and 3D GIS models in CityGML standard. Both instance-based method and linguistic
method were applied in the study. In the instance-based method, a single component (e.g. walls, windows) of a
building was selected and its representation in IFC models and CityGML models were compared. The matching
process not only compared the data structures and terminologies, but also related the numeric values in the two
representations which sometimes involved element decomposition and coordinate transformation. In the linguistic
method, entities and attributes from the IFC schema and the CityGML schema were extracted and compared in
pairs using linguistic techniques and similarity analysis measures. The preliminary results using the two methods
are compared and presented in this paper.
KEYWORDS: Building information modeling (BIM); Interoperability; Schema mapping; City Geography
Markup Language (CityGML)
1. BACKGROUND OF RESEARCH
Building Information Models (BIM) are digital models that could allow people to create, store and exchange data
inside for the purpose of better collaboration and cost saving (Eastman et al., 2008). The construction industry has
seen its revolution of exchange medium from paper works to digital formats. However, problems still exist in the
information exchange between different stakeholders in the construction process due to the high cost for
re-entering the data and possible loss of data during the process. The BIM models, which are rich in semantic
information, could allow stakeholders access to various information about the project, thus fasten the decision
making process. On the other side, people in the geography domain also digitalized the geographic information in
the Geographic Information System (GIS). Data standards like the Geography Markup Language (GML) have
been widely used to store and exchange the geographic information. Though the data integration between
different data standards has been studied by various researchers, the data integration between BIM models and
GIS models still need further study. The mutual demand of the data exchange creates the need for the data
integration of BIM models and GIS models. To achieve this, the integration between the data standards of BIM
models and GIS models must be realized to reduce the effort of re-entering of the data.
Industry Foundation Classes (IFC) is one of the key data exchange standards for BIM models. It is based on
EXPRESS language and was initiated by buildingSMART (formerly International Alliance for Interoperability,
IAI) in 1994. IFC has been formally registered as an ISO standard as ISO/PAS 16739. It supports object-oriented
three-dimensional models which are rich in semantics. IFC allows people to share information on a common
standard, thus allowing a seamless data exchange process. It is now supported by most of the major BIM software
(Benner et al., 2005), for example, the Revit Architecture from the Autodesk. IFC supports various kinds of 3D
geometric representation of building parts. The representation may be either one of or the combination of
Constructive Solid Geometry (CSG), Swept Solid, and Boundary Representation (BRep).
The City Geography Markup Language (CityGML), on the other hand, is a new data standard in the GIS domain.
187

It was developed by Special Interest Group 3D (SIG 3D) of the SID North Rhine-Westphalia. CityGML was
accepted as the Open Geospatial Consortium (OGC) standard in August, 2008. From its initiative, CityGML was
intended to be a unified geographic information exchange standard. It inherits many of the entities from GML and
adds more features to represent the city objects, the botany and water bodies. Furthermore, not only does CityGML
represent the 3D geometric information of objects, like the other OGC standards do, but it also contains semantic
information. It could carry the information about the attributes of the objects (e.g. name, width, height, and
orientation) and the relationship between different objects. It is an object-oriented standard with five Levels of
Details (LoDs). The LoDs vary from LoD 0, which is basically the regional model, to LoD 4, which could
represent the interior building model. The definition of different LoDs could broaden the application area for this
data standard. For LoD 1 to LoD 4, CityGML only uses BRep to represent objects, which is different from that of
IFC.
The integration of BIM models and GIS models is mainly the integration of different data standards. As IFC and
CityGML are the two common data standards in the AEC domain and the GIS domain, this paper focuses on the
integration for these two standards. Previous attempts by other scholars to realize this include unified data
standards like IFG (Industry Fundamental Classes for GIS) and a mega ontology that merges both data standards
(van Berlo and de Laat, 2011, El-Mekawy and stman, 2010). This paper would present the utilization of text
mining techniques for ontology mapping between these two standards along with an instance-based method for
ontology mapping. The following sections are structured as follows: Section 2 reviews the previous attempts for
ontology mapping. Section 3 and Section 4 present the methodology implemented for instance-based mapping and
linguistic-based mapping. Section 5 discusses the results and compares the two methods.
The topic of ontology mapping is a critical topic in many domains, such as e-commerce, semantic web and data
integration (Shvaiko and Euzenat, 2005). A survey conducted by Noy identified the three processes of ontology
mapping as the mapping discovery, formal mapping declaration, and reasoning with mappings (Noy, 2004).
While the linguistic method could help facilitating the mapping discovery, the instance-based method could be
utilized in both discovery and mapping declaration. In the AEC domain, many prior attempts have been made to
integrate IFC and other schemas, such as CIS/2 (CIMSteel Integration Standards) and AEX (Automating
Equipment Information Exchange) using different approaches (Lipman, 2009, Hijazi et al., 2009, El-Mekawy
and stman, 2010, Isikdag and Zlatanova, 2009, Pan et al., 2008).
Mapping discovery could be performed by either manual methods or semi-automated methods, according to the
participation of computer-aided techniques. Due to the different purposes and structures of different schemas, it
is extremely difficult to perform mapping discovery automatically (Lipman, 2009). One manual mapping
example was reported by Lipman from NIST, when he tried to map IFC with CIS/2. Lipman used a pragmatic
approach by manual inspection of the entities inside the schemas (Lipman, 2009). As reported by Lipman (2009),
some entities are obviously related according to their entity names, for instance, the IFC entity IfcCartesianPoint
and the CIS/2 entity Cartesian_point. Mapping according to entity names is therefore one approach to find
related entities in heterogeneous ontology.
Guaranteeing the accuracy, though, the manual method is time-consuming and requires domain experts to find
out the true matches. Researchers have introduced computer-aided ways to find out related entities in other
schema. For example, Wang et al. (2008) have introduced ways to find out related terms between IFC and AEX
by applying domain constraints. The domain constraints are the expanded definition and explanation for a single
entity. For example, the constraints has casing, works on air, and has rotation could be assigned to the
entity fan. Applying domain constraints could narrow down the potential matches; however, it also requires
broad domain knowledge and the boundary of the domain constraints is hard to control. Some scholars also
compared the structure of the entities and the attributes inside the entity in order to find out similar entities. A
mapping between different IFC versions was also done by Wang et al. (2007), in which they compared the entity
structure and attributes of the entity to find out the changes in different versions of IFC (Wang et al., 2007). This
kind of mapping, however, could only apply to schemas with enough similarity in structure and is not applicable
for IFC and CityGML, which are in two different domains and have diverse entity definitions.
As CityGML is a schema which is rich in semantic information, people have attempted to map CityGML entities
to IFC entities. Some researchers introduced a unified building model (UBM) to map the entities in the two
schemas (El-Mekawy and stman, 2010). The UBM basically merges the two schemas, creating a mega schema
which defines all the entities again. The UMB may help the mapping; however, it is not efficient to create
188

another huge schema, especially when duplications are rare between the schemas. Nagel et al. (2009) proposed
some conceptual requirements for transforming information from GIS to IFC. They used a two-process approach
to achieve the data transformation. First, all the GIS models were transformed into CityGML format. Then, they
developed transformation from CityGML to IFC. They discovered that, the transformation process may include a
1-to-n mapping between CityGML entities and IFC entities. Also, patterns must be developed in order to realize
the transformation from BRep to CSG in IFC (Nagel et al., 2009). For instance, the connection part of two walls
is difficult to be recognized when converting from BRep to CSG. As CityGML only employs BRep, while IFC
uses various ways including BRep, CSG, and swept solid, the mapping between the two schemas would
inevitably contain the problem of transforming from BRep to CSG, swept solid, and vice versa. In Wu and Hsieh
(2007), they introduced a method for converting CSG in IFC to BRep in GML using coordinate system
transformation matrix.
Both CityGML and IFC have rich definitions and annotations inside the schemas, which could provide
constraints for the corresponding entities. The entity definitions provide the opportunity for applying text mining
techniques in the mapping discovery phase by comparing the relatedness of the entities. The entity pairs with the
high relatedness are more likely to be true matches. Prior approaches to utilize the text mining techniques in
schema mapping include Pan et al. (2008) and Cheng et al. (2008). They employed a document corpus for the
comparison and compared the entity definitions to a corpus. None of the approaches have considered the direct
comparison of the entity definitions.
Our approach to map IFC and CityGML is based on two methodologies: the instance-based method and the
linguistic method. As both IFC and CityGML use a component-based method to represent building objects, the
instance-based mapping could be performed on a component basis. By looking at the instance of different
building components (e.g. wall, door, or window), the mapping could be developed. To facilitate the mapping
discovery process, a linguistic method, which would use text mining techniques to discover the relatedness of the
entities by comparing their definitions, was introduced.
3.1 Research Methodology for Instance-based Method
The IFC schema and CityGML schema are component-based, meaning that they define objects and their
semantic on a components basis. This component-based representation ensures different parts of a building could
be marked and the information related to those parts could be stored (Ibrahim et al., 2004). Information is related
to components, which provide opportunity for further implication, such as the cost analysis, and automatic
generation of BOQ (bill of quantities). Instance-based mapping between the IFC and CityGML is therefore
conducted on a component basis. The mapping constitutes two parts: the semantic information mapping and
geometric representation mapping. The major challenges for the mapping include (1) the transformation of the
coordinating system, (2) the conversion from BRep to CSG, Swept Solid, and vice versa, (3) the possible loss of
information due to schema difference, (4) the transformation between different LoDs in CityGML, and (5) the
possible 1-to-n mapping between the components (Wu and Hsieh, 2007).
The IFC expression of an object contains the geometric information of the object represented in a local
coordinate system (Lipman, 2009) as well as the semantic information of the object, such as the Global ID,
modification history, the scale of the object, manufacturer, and material used. The entities in an EXPRESS
format of IFC files would be referenced to other entities by a referencing system, which make it possible to form
a tree structure for each object containing all the relevant information. Take a window as an example. A typical
window can be expressed in a tree-like structure as shown in Figure 1. The IfcOwnerHistory defines the owner of
the object, the software used to create it, and modification history. The local coordinate system of the object and
its referencing coordinate system are defined in IfcLocalPlacement. All the objects in an IFC schema have their
own local placement system while the local coordinate system refers to a higher level coordinate system.
IfcAxis2Placement3d is used to define the origin of coordinates and the orientation of axis. The
IfcRepresentation entity would be used to define the geometry of the object using BRep, CSG or Swept Solid.

189

Figure 1 Representation of a window in IFC

Figure 2 Semantic (left) and geometric (right) representation of a window in CityGML
The representation of a window in CityGML is stored in the XML format. The CityGML object representation
also includes the semantic information, such as the Global ID, the scale of the objects, and material. However,
there exists no entity to represent the information in the IfcOwnerHistory, so when mapping IfcOwnerHistory,
the information would be lost. The corresponding CityGML expression of semantic information is shown in
Figure 2 (left). The geometry information is represented in a BRep representation, as shown in Figure 2 (right).
Basically the mapping from IFC points to CityGML points constitutes a transformation of different coordinate
systems.
If the representation method in IFC is CSG or Swept Solid, rather than BRep, the CSG and Swept Solid would
be transformed into points in BRep utilizing the method mentioned in (Wu and Hsieh, 2007). Then, given the
origin and axis orientation of local coordinate system and its referencing system, the coordinate system
transformation matrix M and the origin difference could be calculated. The points in CityGML, which are in a
World Coordinating System, could be calculated as
x x
y y
z z
C I
C I M
C I
( (
( (
= + A
( (
( (

where vector C is the coordinate in CityGML and vector I is the coordinate in IFC.
3.2 Research methodology for Linguistic-based Method
The linguistic method would utilize the concept of relatedness analysis in the text mining domain to facilitate the
discovery of similar entities from the two schemas. By calculating the similarity of entity descriptions and
definitions, the relatedness of entities could be discovered. To evaluate the relatedness of the text, the Cosine
Similarity, Jaccard Similarity Coefficient, and Market Basket Model were calculated.
The first step for the similarity calculation is the definition extraction. Of the 1068 entities defined in the IFC
schema, 1008 have descriptions and definitions. And 68 entities of the 1008 entities were found to have two
definitions from different sources, which are IAI (buildingSMART), ISO (International Standard Organization),
CSS1 (Cascading Style Sheets, level 1), and BS (British Standard). The multiple definitions from various sources
may broaden the scope; however, it would also cause trouble to the linguistic analysis. The detailed
Ifcwindow(ID, #ownerhistory, name, discription, #local placement, # representation,
Height, width)
#Ifcownerhistory()
#Ifclocalplacement(#place related to, #relative placement)
#Ifcproduct()
#ifcaxis2placement3d(#location,#axis,#reference direction)
#(x,y,z)
#(x,y,z)
#(a,b,c)
#ifcrepresentation
#ifcpolyloop(P1,P2,P3,P4)
#(a1,a2,a3)
#(b1,b2,b3)
#(c1,c2,c3)
#(d1,d2,d3)
<bldg:opening>
<bldg:Window>
<xbuilding:GlobalId value=D"/>
<gen:doubleAttribute
name="OverallWidth">
<gen:value>Width</gen:value>
</gen:doubleAttribute>
<gen:doubleAttribute
name="OverallHeight">
<gen:value>Height</gen:value>
</gen:doubleAttribute>
<bldg:lod4MultiSurface>
<gml:MultiSurface>
<gml:surfaceMember>
<gml:Polygon>
<gml:exterior>
<gml:LinearRing>
<gml:posList srsDimension="3">
s1, s2, s3, t1, t2, t3, u1, u2, u3, s1,
s2, s3
</gml:posList>
</gml:LinearRing>
</gml:exterior>
190

Figure 3 Illustration of feature vector
specifications of IFC entities are stored in HTML format (Hyper Text Markup Language). Therefore, entity
definitions were extracted from HTML files and grouped into a well-formed XML (Extensible Markup
Language) file using Java.
For the entities in CityGML schema, considering their referencing to the GML schemas, 607 entities with
definitions were found in the documentation or annotation of the CityGML and GML schemas. The definitions
for each entity could be extracted directly from the schemas because the CityGML schema is represented in the
XSD (XML Schema Definition) format, which is a special type of XML.
The entity definitions were then tokenized in two steps: stop words removal and stemming. Stop words are those
words that occur so often in the documents that they may be not as relevant to the query as the query to the
document (Wilbur and Sirotkin, 1992). Including the stop words in the comparison may harm the accuracy, so all
the stop words in the definitions were removed beforehand. A general stop words list was used which contains
450 commonly seen words, for instance, is, for, and to. After removing the stop words, the entity
definitions were stemmed so that the form of the words would not affect the comparison. Stemming is the
process of changing the words to their stem or base form. For example, definition is changed to defin and
coordinated is changed to coordin after the stemming. A stem of the word would not be similar to the
morphological form of the word, but it would ensure all the related word would have the same stem. This study
adopted the Porter Stemming algorithm, which is a commonly used stemming algorithm for finding out the stem
of the word (Porter, 2001).
After the tokenization process, the definitions were ready for relatedness analysis. Each definition was
formalized into a feature vector for further analysis. If concept n (e.g. window) appears m times in the definition,
the n-th value of the feature vector of definition i would be m, as illustrated in Figure 3.
Once the feature vectors were generated for each of the definitions, a one-to-one comparison between entities
from IFC and entities from CityGML was performed by calculating the Cosine Similarity, Jaccard Similarity
Coefficient and Market Basket Model scores.
The Cosine Similarity is a measurement of similarity in the field of text mining. It is a non-Euclidean distance
measure between two vectors (Nahm et al., 2002). If two vectors v
i
and v
j
are given, the Cosine Similarity of the
two vectors could be expressed as:
( , )
i j
i j
v v
Sim i j
v v

By its nature, the Cosine Similarity is in a range of [0, 1]. Herein, the maximum score of 1 indicates that the two
concepts i and j have almost identical features and have the highest similarity.
The Jaccard Similarity Coefficient is a measure of the overlap between the feature vectors v
i
an v
j
of two
concepts (Roussinov and Zhao, 2003). Let N
11
be the number of features that both feature vectors contain
non-zero value, N
10
be the number of features which appear in v
i
and do not appear in v
j
, N
01
be the number of
features which appear in v
j
and do not appear in v
i
. The similarity between the concepts i and j could be
calculated by the following equation:
11
11 10 01
( , )
N
Sim i j
N N N
=
+ +

Definition of ifcwindow in IFC: Construction for closing a vertical or near vertical opening
Definition of windowtype in CityGML: Type for windows in walls .
191

The Jaccard Similarity Coefficient score could not exceed 1, which indicates that the two feature vectors are the
same. The Jaccard Similarity Coefficient score is a popular measure for the relatedness analysis of a
term-to-term similarity. It could return the overlapping situation of two terms, thus it is an efficient way to
compare the two terms (Cheng et al., 2008).
The Market Basket Model is another data-mining technique to calculate the similarity of two concepts (Metcalfe
and Cantner, 2003). Let N
11 ,
N
10 ,
N
01
have the same definitions as those for Jaccard Similarity Coefficient. Given
two feature vectors v
i
and v
j
, the associate rule i to j without absolute notation could be calculated as
11 01 11
11 10
( , )
N N N
Sim i j
N N n
+
=
+

Herein, the number n is the number of all features. The Market Basket Model would return a result that is in the
range of (-1, 1). The number of -1 means all the features that appear in concept j do not appear in concept i. The
value of 1 is another boundary value as the value of N
11
+N
01
could not be 0 (Cheng et al., 2008).
By looking at the scores and the ranks according to the scores, similar entities could be discovered, of which the
results would be discussed in the next section. In order to improve the accuracy, the Term Frequency (tf) and
Inverse Document Frequency (idf) were also considered in the calculation. The method implementing tf*idf
would improve the result by looking into the frequency of a word in a single document while at the same time
considering the inverse proportion of the word in the whole document pool (Ramos, 2003). The tf*idf would
decide a weight for each word in each definition considering the frequency it appears in that definition and all
the definitions. The words which are rarely seen in all the documents would be assigned a higher weight while
the common words would be assigned a smaller one. Given a single document d which belongs to a document
pool D, the general formula to calculate tf*idf for a single word w is as follows (Salton and Buckley, 1988,
Ramos, 2003)
( )
, ,
* log /
w d w D
tf idf f D f =

where
, w d
f is the frequency of w in the document d, D is the size of the document pool and
, w D
f is the
number of documents in which w appears.
The results for the similarity scores with and without tf*idf considerations were evaluated and compared using
the precision and the recall measures. One true match would come from the instance-based method, by looking
at the hierarchy structure of the matching entities. The other true match would come from the mappings from
domain experts, where the experts would check the random pairs of matches and mark out the true matches. The
precision and recall of the linguistic method could be calculated as

where the predicted matches are the results by the linguistic method and the true matches are given by the two
methods mentioned above (Pan et al., 2008, Cheng et al., 2008).
4. RESULTS AND DISCUSSION
In the linguistic method, 653132 (1076x607) pairs of entities from IFC and those from CityGML were generated
for similarity score comparison. To evaluate the precision and recall of the results, the true matches from the
instance-based method and from domain experts were generated.
4.1 The relationship between instance-based method and linguistic-based method
The instance-based method looks inside the representation of the objects in IFC and CityGML. The mapping
192

could be done level by level, as shown in Figure 1 and Figure 2. The entities on the same level (e.g. ifcwindow vs.
windowtype) were thus considered as true matches.
CityObject
+creationDate: xs:date[0]
+terminationDate: xs:date[0]
_Opening
Window Appearance
+theme: XS:string[0..1]
appearance
0..*
0..*
generalizeTo
0..*
0..*
gen: _GenericAttribute
+name: xs: string[1]
gen:_genericAttribute
ExternalReference
+informationSystem: xs:anyURI[0..1]
1
0..*
gml:MultiSurface
lod4MultiSurface 0..1
*
lod3MultiSurface
0..1
0..*
gml:AbstractGeometricPrimitiveType
+metaDataProperty[0..*]
+description[0..1]
+descriptionReference[0..1]
+identifier[0..1]
+name[0..*]
surfacemember surfacemembers
0..*
0..*
0..*
0..1

Figure 4 Informal UML diagram for window entity in CityGML
Table 1 The entities for representing a window in IFC and CityGML
Level IFC Entities CityGML Entities
Object ifcwindow opening
window
Middle ifcownerhistory MultiSurface
ifclocalplacement surfaceMember
ifcproductdefinitionshape Polygon
ifcshaperepresentation
ifcgeometricrepresentationsubcontext
ifcmappeditem
ifcrepresentationmap
ifcaxis2placement3d
ifcshaperepresentation
ifcgeometricset
Value ifcpolyline LinearRing
ifccartesianpoint exterior
The mapping between IFC and CityGML was performed on a component basis, from which the mapping
algorithm would be discovered by the instance-based method. By comparing the different representing
geometries for the objects in IFC and CityGML, the mapping between CSG and Swept Solid to BRep could be
achieved. The instance-based method helps to locate the similar entities in the two schemas. Moreover, the
linguistic method could provide a guide for locating the potential true matches by comparing the definitions. The
linguistic method could generate a list of candidates for a particular entity, so that the searching space for the
instance-based method could be reduced.
The hierarchy structure from instance-based method for IFC and CityGML are not as redundant as those in the
schemas. The informal Unified Modeling Language (UML) of a window entity in CityGML is shown in Figure 4.
As the UML diagram shows, the Window entity inherits all the attributes from CityObject entity in CityGML,
along with the relationship with other entities. The actual object representation hierarchy in a real CityGML file
is shown in Figure 2, which suggests a more concise structure. The same hierarchy inside an IFC file could also
be generated, which is shown in Figure 1.
To acquire the true matches, the entities in the representation hierarchy were divided into three levels: the object
level, the middle level, and the value level. Take a window as an example. The levels of the IFC representation
and CityGML representation are shown in Table 1. The entities on the same level were considered to be true
193

matches.

4.2 The comparison between different similarity measures
The Cosine Similarity, Jaccard Similarity Coefficient and Market Basket Model were used to measure the
relatedness results in the linguistic-based method. The Term Frequency and Inverse Document Frequency were
also considered in the Cosine Similarity. Based on the true matches from the instance-based method, the
precision and recall of the different measures could be calculated.
The Cosine Similarity scores for the entities of window representation in IFC and CityGML are shown in Table 2.
The entities on the same level, e.g. ifccartesianpoint and LinearRing, tend to have a higher similarity score. This
relatedness could not be discovered purely based on the name of the entities.
As shown in Figure 5, Cosine Similarity resulted in a higher precision and recall rate than Jaccard Similarity
Coefficient and Market Basket Model in general. The Jaccard Similarity Coefficient and Market Basket Model
are based on the duplication of words in the comparing texts. If the two schemas utilize different sets of
vocabularies, the Jaccard Similarity Coefficient and Market Basket Model could not return a high score. The
Cosine Similarity, on the other hand, not only considers the duplication of the words, but also considers the
duplication times of the words. Given two sets of vocabularies, the Cosine Similarity is expected to generate a
higher score. The definitions of CityGML entities utilize a vocabulary of 1734 words, while those of IFC entities
utilize 1603 words. The two schemas only share a list of 726 words, which are 42% of the words in CityGML
and 45% of the words in IFC. Given such a low duplication in vocabulary, the Jaccard Similarity Coefficient and
Market Basket Model could not get a high score. The influence of the vocabulary to Cosine Similarity would not
be as large as that of Jaccard Similarity Coefficient and Market Basket Model.
With a higher precision and recall rate, Cosine Similarity was then used to study the effect of the tf*idf
consideration. As shown in Figure 5, the Cosine Similarity without the tf*idf consideration showed a higher
recall, which indicates more accuracy of the candidates. However, the results with tf*idf considerations indicated
that the precision with tf*idf was higher, which means that tf*idf consideration could better local the candidates.
To see the difference between the two, the ranking of the entities according to their similarity score was also
considered. The results are shown in Figure 6.
Table 2 Cosine Similairty result for IfcWindow and window entity in CityGML
opening window MultiSurface surfaceMember Polygon exterior LinearRing
ifcwindow 0.05505 0.10328 0.20943 0.04495 0.04032 0.07875 0.05064
ifclocalplacement 0.04029 0.07559 0.06131 0.00000 0.02951 0.00000 0.03706
ifcproductdefinitionshape 0.00000 0.00000 0.13498 0.00000 0.04331 0.00000 0.05439
ifcshaperepresentation 0.02686 0.12599 0.20438 0.04386 0.07870 0.00000 0.02471
Ifcgeometricrepresentation
subcontext
0.00000 0.05164 0.04189 0.04495 0.04032 0.03937 0.05064
ifcmappeditem 0.00000 0.00000 0.05648 0.00000 0.02719 0.00000 0.00000
ifcrepresentationmap 0.04103 0.03849 0.06244 0.03350 0.06011 0.00000 0.15097
ifcaxis2placement3d 0.03604 0.05345 0.05484 0.04652 0.03960 0.00000 0.19890
ifcshaperepresentation 0.02686 0.12599 0.20438 0.04386 0.07870 0.00000 0.02471
ifcgeometricset 0.00000 0.00000 0.06363 0.10242 0.12251 0.05981 0.00000
ifcpolyline 0.00000 0.00000 0.04336 0.00000 0.04174 0.04076 0.15724
ifccartesianpoint 0.00000 0.04082 0.09934 0.03553 0.09564 0.00000 0.18605
194

Figure 5 Comparison of precision (left) and recall (right) of different similarity measures

Figure 6 Comparison of precision (left) and recall (right) of Cosine Similarity according to ranking with and
without tf*idf consideration
As shown in Figure 6, the rankings according to CityGML entities generated a lower precision and recall. This is
because the number of IFC entities is greater than that of CityGML entities. Given the same matching, the true
matches would be bigger. Also, the rankings of Cosine Similarity with or without tf*idf showed almost the same
result. We could only conclude that the tf*idf method could improve the precision of the results. This result also
indicates that when utilizing the linguistic method, the ranks of entities according to similarity scores should also
be considered.
There are a lot of terminologies in the vocabulary of IFC and CityGML schemas. Of the 2611 words that appear
in all the definitions, 81.7% of them only appear less
than 10 times in all the documents. If all the definitions
are considered to be one document, 78.3% of all words
only appear less than 10 times. The high frequency of
rare words reduces the effect of tf*idf measure, thus
making the recall of the Cosine Similarity with tf*idf
low.
This paper presents two methods to map IFC entities
with CityGML entities. The relatedness analysis from
the text mining domain was conducted using the
descriptions and definitions of entities from the two
schemas. The mapping algorithm between IFC and
CityGML entities was developed to transfer both the semantic information and geometric information. The
transformation from CSG or Swept Solid, which is commonly seen in IFC, to BRep, which is representation
geometric in CityGML, was also considered. In order to facilitate the mapping process, the linguistic-based
195

method was introduced and evaluated.
The result shows that in the linguistic method the Cosine Similarity scores could yield a better result comparing
to the Jaccard Similarity Coefficient and Market Basket Model. In order to improve the results, the tf*idf
weighting was considered in the similarity score calculation. The evaluation was then conducted considering
both the scores and the ranking. The tf*idf consideration was found to generate results with a better precision
rate.
The hierarchy structure generated in the instance-based method was used to perform the evaluation of the
linguistic-based results. When performing mappings between IFC and CityGML utilizing the instance-based
method, the mapping discovery is limited to the instances collected. Changes of schema versions or applications
may affect the results of the instance-based mapping. The linguistic method, however, could provide potential
mappings between two schemas mainly based on the semantics of entities. It could narrow the search space for
the true matches. Linguistic methods usually only consider the entity names. The linguistic method introduced in
this paper compared the definitions of entities from IFC and CityGML schemas. The related entities which do
not have similar entity names could also be discovered using the linguistic method proposed in this paper.
This paper does not consider the schema structure for the linguistic method, nor the transformation of different
Levels of Detail in CityGML. Furthermore, the entity names were not included in the relatedness analysis in this
study. These issues will be addressed in the future research.
6. REFERENCES
Benner, J., Geiger, A. & Leinemann, K. (2005) Flexible generation of semantic 3D building models..
Cheng, C. P., Lau, G. T., Pan, J., Law, K. H. & Jones, A. (2008) Domain-specific ontology mapping by
corpus-based semantic similarity..
Nagel, C., Stadler, A. & Kolbe, T. H. (2009) Conceptual requirements for the automatic reconstruction of building
information models from uninterpreted 3D models. The International Archives of the Photogrammetry, Remote
Sensing and Spatial Information Sciences, Vol. 34, Part XXX
Eastman, C. M., Teicholz, P., Sacks, R. & Liston, K. (2008) BIM handbook, Wiley Online Library.
El-Mekawy, M. & stman, A. (2010) Semantic Mapping: an Ontology Engineering Method for Integrating
Building Models in IFC and CITYGML. Proceedings of the 3rd ISDE Digital Earth Summit,, 12-14.
Hijazi, I., Ehlers, M., Zlatanova, S. & Isikdag, U. (2009a) IFC to CityGML transformation framework for
geo-analysis: a water utility network case..
Ibrahim, M., Krawczyk, R. & Schipporeit, G. (2004) Two approaches to BIM: a comparative study..
Isikdag, U. & Zlatanova, S. (2009) Towards defining a framework for automatic generation of buildings in
CityGML using building Information Models. 3D Geo-Information Sciences,, 79-96.
Jiayi, P., Cheng, C. P. J., Lau, G. T. & Law, K. H. (2008) Utilizing Statistical Semantic Similarity Techniques for
Ontology Mapping--with Applications to AEC Standard Models. Tsinghua Science & Technology, 13, 217-222.
Lipman, R. (2009) Details of the mapping between the CIS/2 and IFC product data models for structural steel.
Journal of Information Technology in Construction, 14, 1-13.
Metcalfe, J. S. & Cantner, U. (2003) Change, transformation, and development, Physica Verlag.
Nagel, C., Stadler, A. & Kolbe, T. H. (2009) Conceptual requirements for the automatic reconstruction of building
information models from uninterpreted 3D models..
Nahm, U. Y., Bilenko, M. & Mooney, R. J. (2002) Two approaches to handling noisy variation in text mining.,
Citeseer.
Noy, N. F. (2004) Semantic integration: a survey of ontology-based approaches. ACM Sigmod Record, 33, 65-70.
Porter, M. (2001) Snowball: A language for stemming algorithms..
196

Ramos, J. (2003) Using tf-idf to determine word relevance in document queries..
Roussinov, D. & Zhao, J. L. (2003) Automatic discovery of similarity relationships through Web mining. Decision
Support Systems, 35, 149-166.
Salton, G. & Buckley, C. (1988) Term-weighting approaches in automatic text retrieval. Information processing &
management, 24, 513-523.
Shvaiko, P. & Euzenat, J. (2005) A survey of schema-based matching approaches. Journal on Data Semantics IV,,
146-171.
van Berlo, L. & de Laat, R. (2011) Integration of BIM and GIS: The development of the CityGML GeoBIM
extension. Advances in 3D Geo-Information Sciences,, 211-227.
Wang, H., Akinci, B. & Garrett Jr, J. H. (2007) Formalism for detecting version differences in data models.
Journal of Computing in Civil Engineering, 21, 321.
Wang, H., Akinci, B., Garrett Jr, J. H. & Reed, K. A. (2008) Formalism for applying domain constraints in
domain-oriented schema matching. Journal of Computing in Civil Engineering, 22, 170.
Wilbur, W. J. & Sirotkin, K. (1992) The automatic identification of stop words. Journal of information science, 18,
45-55.
Wu, I. C. & Hsieh, S. H. (2007) Transformation from IFC data model to GML data model: methodology and tool
development. Journal of the Chinese Institute of Engineers, 30, 1085-1090.
197

A STUDY ON DATA EXTRACTION FROM BIM MODELS FOR
KNOWLEDGE MANAGEMENT
C.L. Hung & S.A. Chou
Sinotech Engineering Consultants, Ltd., Taiwan
ABSTRACT: BIM models contain geometric and non-geometric information for engineering constructions in
the means of digital format. Through manipulating BIM models, engineers can obtain rich and consistent data for
further uses as compared to 2D drawings. Therefore building BIM models sometimes become one of the mandates
in an architectural construction project. Since BIM models can provide not only spatial relations for constructions
but also quantities, construction sequences, costs and other professional knowledge, the procedures, requirements,
regulations and guidelines for feeding data into BIM models have been widely discussed in recent years and many
useful practices have been proposed. Yet actions when having BIM model such as extracting data from BIM
models to form a knowledge base for an enterprise did not receive much attention so far. How BIM models can
evolve to become company asset to help engineers in engineering design phase is surveyed in this research.
KEYWORDS: BIM, Knowledge Repository, Data Extraction, Knowledge Management
1. FOREWORD
The key factor affecting an enterprises competence lies in the preservation and utilization of its past useful
experience, namely knowledge. For engineering design firms, 2D CAD system is a powerful tool for mass
productions. 3D design is also welcomed along with the development of computer hardware and applications.
Sophisticated designs with irregular shapes or curves are only possible with 3D design software to reveal the
intentions of architects and designers. The Gymnasiums of Peking built for Olympic Games 2008 are good
examples of 3D design. 3D design software are becoming smarter than ever that not only design results are kept in
digital models but also the imaginations, knowledge for construction and etc. are kept internally in 3D models.
How these useful experiences, or knowledge, can be extracted from digital models to be an enterprises asset is the
goal of this research.
BIM (Building Information Modeling), the processes in creating 3D digital models for construction and its
product - BIM models, are changing our way of doing constructions. The accumulation of knowledge through
interpreting BIM models will be a quite different attempt to keep enterprise assets. This might not be the final stage
for knowledge management, but no doubt managing BIM models will be a milestone for engineering firms.
2. THE ESSENCE OF BIM
BIM illustrates the process of creating parametric 3D digital models BIM models. BIM models are rich-content
digital archives containing not only geometric data such as dimensions and coordinates, but also design intents
such as construction sequences, performance data, runnig costs, suppliers, contact persons and etc. With
geometric and non-geometric information, BIM models are useful in different stages of a construction for
perspective purposes. Value-added applications can also be developed based on BIM models to further boost
their values. [Wikipedia, 2012]
BIM models, as a common language for construction teams, play a vital role in mitigating mis-understanding.
But this benefit is not cultivated without any efforts. To fully explore the value of using BIM, it is expected that
BIM technology be used in all phases in a constructions life cycle. Through information sharing from design
stage to construction and finally operation stage, construction industries will perform like production lines to
achieve better efficiency, better quality and higher productivity. The use of BIM no wonder is a giant step for
construction engineering.
BIM represents not only models produced through collaborative work but also the methodology of adopting
BIM. Therefore BIM is treated as a business secret in advanced engineering firms. How to preserve BIM models
and workflows for future use is of great importance in major engineering firms in Taiwan. Similar researches in
constructing BIM-based KM (Knowledge Management) are also about to evolve in some research institutes. For
example a study relating BIM as knowledge repository is conducted in Georgia Institute of Technology in
2010[ASCE, 2010]. As BIM is still in development in many areas around the world, KM using BIM is now
198

mostly confined as personal use or limited to single project only. Even though most enterprises do not list BIM
model management and related knowledge extraction as their goals in BIM adoption, some leading firms have
noticed this trend. For example, Sumex Design, a company based in USA, has offered a service named
BIM-based knowledge management already.
3. DATA EXCHANGE USING IFC
There are quite a few BIM-enabled software packages but all of them come with own proprietary file formats.
Although such formats are good at performance issues but are not suitable for knowledge preservation or
extraction. In order to keep BIM assets at hand, some public-owned or open formats need to be adopted for
archiving BIM data. Possible formats include IFC, CityGML, gbXML and so on. Among them, IFC is
recognized as the most promising one for BIM data exchange.
IFC (Industry Foundation Class) is proposed by IAI (Industry Alliance for Interoperability) in 1995 in order to
set a standard for describing all the information required for an engineering construction. Through commonly
used IFC format, different team members using different software can work together seamlessly through data
sharing and conversion.
IFC is an open standard based on ISO 10303, which allows software vendors to develop their own software to
communicate with others through IFC. IFC can be viewed as a standard syntax in defining geometric and
non-geometric objects using Express language. The latest version is IFC2X4 while IFC is still under
development. IFC is actually composed of many schemas in accordance with object-oriented principles. This
schema contains 4 classes Select, Class, Type and Property Set. IFC is structured hierarchically into 4 layers:
Resource Layer, Core Layer, Interoperability Layer and Domain /Application Layer). [IAI, 2007]
A sample data using IFC format to express a data structure is show in Fig-1. The interpretation of IFC is not
difficult since there are only 4 major data types in an IFC file -Defined Type, Enumeration, Select Type and
Entity. Ladder principle also applies, which means class reference to same level or lower level is allowed to
realize data inheritance. In this research, we use IFC as the data format to store and retrieve data and act as our
knowledge bases.
Fig. 1: A sample IFC files header
4. APPLICATION OF BIM
Usually BIM is considered a method, not a product to deliver. But somehow we still need to have pre-defined
purposes before BIM-related job starts. The process of creating/modifying BIM models is itself a valuable
knowledge to learn, but what left for archiving is the product - BIM models. Although BIM models are usually
built for different purposes, the value of each BIM model differs depending on the contents or information built
into it. To extract information from BIM models, we need to know what have been included in each BIM
model. From our experience and clients demands, followings are 4 general uses of BIM - visualization, collision
detection, quantity take-off and 4D simulations.
199

Visualization
In terms of visual effects, 3D models are more realistic and attractive than 2D drawings. With BIM models, the
size, dimension, location and geometry-related information can be better revealed for discussion and mutual
understanding of spatial relationships. A typical school building in 3D model is shown in Fig. 2.
Collision Detection
Spatial interferences between different systems are common faults found under 2D or even 3D CAD systems.
Not until solid modeling introduced into CAD systems has this problem eased through spatial interference check,
or collision detection (CD). CD is mostly useful in examining MEP (Mechanical, Electrical and Plumbing)
systems when space or height is limited. A typical snapshot is shown in Fig. 3 that pipes are detected to be
interfered before actual construction starts.
Quantity Take-off
BIM model is built as similar to real structure as possible. When the level of details (LOD) is high enough to
offer information for further engineering purposes such as quantity calculations, data for bill of quantity (BOQ)
can be easily extracted from BIM model to check against manual calculations. Not only quantities, but other
digital calculation such as area calculations can also be achieved with BIM models.
4D Simulation
Typical 4D simulations need input from 3D models and schedules to illustrate the sequence for construction. 4D
simulation not only acts as a media for discussion but also points a way for improving construction arrangements
and reducing risks. Through visual examination, 4D simulations outperform paper-based or 2D drawing-based
schedule control. A well-designed BIM model containing 4D data no doubt is a great source for knowledge
extraction
5. PREPRATION FOR KNOWLEDGE EXTRACTION FROM BIM
BIM is a content-rich product produced by engineers for different purposes as described in last section. Even
though these BIM models can be treated as an enterprises asset, proper processing is still inevitable to extract
knowledge embedded in BIM models. A scenario targeting a new building is proposed in the following to
explain how knowledge can be extracted from BIM models and preserved for sharing.
Creation of BIM model
Here we use a model shown in Fig. 4 as our test target for extracting knowledge from. This is a model for a
5-story building in a university located in eastern Taiwan. Model was built with Revit Architecture 2012. For
simplicity, only structural and architectural components were built.
Fig. 2: BIM Model for visual effect Fig. 3: Collision detected between pipe
200

BIM service platform
BIM models need to be converted into non-proprietary formats for further use. IFC is select for this purpose in
our research. Also a platform is designed for interpreting IFC files and extracting information from IFC files.
This BIM service platform is designed to support multiple engineering projects in a way that all BIM models are
stored project by project and system by system. In Fig. 5 our sample project, a school buildings BIM, has been
converted into IFC format and stored in our BIM service platform for architectural and structural parts.
In order to view the contents stored in the BIM service platform, we developed a web-based Browser to view
IFC files without using any commercial packages. This Browser is developed using HTML5 Canvas and WebGL
and runs well under several Internet web browsers. Some basic functions such as rotating, panning, scaling have
been developed as shown in Fig. 6.
Value-added usage in facility management
Each component in a BIM model is identified uniquely with a code named GUID, a text string composed of
numbers and texts. In order to find out a components location within a building, GUID code can be translated
into QR Code to be recognized by QR Code scanners. With smart devices like smart phones, we can get detailed
information for facility management by scanning QR Code and link to a database storing information retrieved
from BIM models like Fig. 7. On-situ maintenance operations will be easier with this extended usage of BIM
models.
The use of QR Code from GUID ID generated with BIM models helps not only in maintenance but also for other
operations such as inventory of properties. The combination of mobile devices and BIM models is just the
beginning of a scenario for information everywhere.
Fig. 4: BIM model of a school building
Fig. 5: IFC BIM service platform
Fig. 6: an IFC BIM model viewer developed in this
research
Fig. 7: Facility management using QR code
201

6. DISCUSSIONS
Knowledge is valuable and reusable and this is the reason knowledge is different from information which only
static data are stored in BIM models. Not until the embedded information can be retrieved and managed will
BIM models really demonstrate the use in assisting engineers. But the extraction of knowledge from BIM
models is not easy because it needs very technical persons to examine and analyze the information in BIM
models. Just like a position named CKO (Chief Knowledge Officer), this job needs a BIM knowledge manager
[Youtube, 2012]. BIM knowledge manager is neither a BIM model manager nor a BIM modeler. BIM
knowledge manager is a coordinator who can help interpreting design and construction intentions and relay those
concerns to other BIM engineers. BIM will become an enterprises assets only when BIM knowledge managers
function properly. In most construction firms, BIM are getting more widely used and the issues of knowledge
extraction will become more obvious as time goes by. How and when to have BIM knowledge systematically
managed depend on each firms decisions.
7. REFERENCES
Wikipedia (2012). Visited at http://zh.wikipedia.org/wiki/BIM.
ASCE, BIM and Concrete Formwork Repository (2010). Download at
http://ascpro0.ascweb.org/archives/cd/2010/paper/CERT177002010.pdf
IAI, Industry Foundation Classes Release 2x3 (2007). See http://www.buildingsmart- tech.org
Youtube, BIM Specification and Knowledge Management (2012). Visited at
http://www.youtube.com/watch?v=G-k3Lw6CEQE
202

USER REQUIREMENTS FOR MOBILE AR AND BIM UTILIZATION
IN BUILDING LIFE CYCLE MANAGEMENT
Timo Kuula, Kalevi Piira, Anu Seisto, Mika Hakkarainen & Charles Woodward
VTT Technical Research Centre of Finland
ABSTRACT: The aim of our research is to bring the information content existing with Building Information
Models (BIM) to mobile users in building lifecycle applications by means of Augmented Reality (AR). This article
presents the first phase of our study, analysing the requirements that were collected from users based on
participatory design principle. With the involvement of the users in the design process, we aimed at obtaining
information of applications that would have real benefits in the building maintenance work. As a result of this user
study, the most potential application ideas were listed. The ideas most relevant for this study were the ones in
which AR and BIM are combined. These include for example fault diagnosis, mobile route guidance inside the
building, viewing interiors of building through different user profiles, and taking photos and linking them to the 3D
model. The first prototype version of the application is implemented using the user data as background and
supportive information. A requirement specification for the system architecture and tracking technology for the
indoors AR implementation is presented. In the next phases of the project, the prototype application will be tested
and evaluated by the end-users in actual field tests with a real world pilot case. Hence, in the second phase of the
study the users will evaluate concrete prototypes built based on their own descriptions.
KEYWORDS: Augmented Reality (AR), mobile AR, Building Information Models (BIM), building facility
management, user study, participatory design.
1. INTRODUCTION
While the use of Building Information Models (BIM) has been gaining increasing popularity in planning and
monitoring of building construction work, much less attention has been placed in BIM applications in buildings
life-cycle management. Compiled with great effort by various parties in the planning phase of the building, BIMs
contain a great deal of valuable information that could later serve different functions of monitoring, maintenance
and repair personnel as well.
On the other hand, mobile applications have been recognised as a most powerful tool to assist facility managers
and other related personnel in their jobs (see e.g. Gheisari and Irizarry 2011). As todays practices involve team
work with various nomad workers, the individual building responsibles may change frequently, requiring
up-to-date information of facilities previously unknown them. Mobile Augmented Reality (AR) has been proposed
as means to bring the facilitys underlying digital information, e.g. maintenance instructions and real time process
data, to the mobile users access (Siltanen et. al 2007).
Mobile Augmented Reality has been widely explored for outdoors applications such as architectural planning. In a
study by Woodward and Hakkarainen (2011), mobile AR was piloted in various cases for visualization of planned
building projects. Ultimately, their work enables comparison of 4D BIM defined daily work plans with the actual
situation on the construction site. Recently, mobile AR has been applied in real decision making by communal
officers, viewing augmented presentations of proposed construction plans at street level views around the town
(Olsson et al. 2012). This work also involved user experience (UX) evaluation of the applications. In a similar
study by Allen et al. (2011), user acceptance for the technology was also evaluated highly positive.
However, mobile AR for building life cycle applications has not been studied much at all, one general reason being
the lack of accurate positioning infrastructure inside buildings (GPS only works ourtdoors). For the same reason,
consumer level AR world browsing applications such as Layar and Wikitude are designed for outdoors
applications only.
The goal of our DigiSpaces project is to combine accurate indoors positioning and vision based tracking
technology with a mobile AR interface, to provide access to various BIM information serving buildings life-cycle
applications. Work similar to ours, combining mobile AR with BIM information, has been carried out by Gheisari
et al. (2012); however no technical detail of it published yet. In particular, no research at all has apparently been
conducted on user involvement in creating this kind of applications. In the present article, we concentrate on the
user requirements, while system implementation aspects will be discussed in future articles.
203

As industrial partners in the DigiSpaces project, we have, besides Nokia, the construction company Skanska, the
building services consulting company Granlund, the AEC engineering and consulting company Pyry, the BIM
design software company Tekla (Trimble), and the BIM analysis and verification software company Solibri. This
article presents the user requirements for a mobile AR system based on BIM related information, obtained from
user studies and interviews with the participating companies. The user study conducted is mainly based on
participatory design (PD) approach.
2. PARTICIPATORY DESIGN
Companies are more and more interested in collaborative innovation with customers and users. Different methods
for user participation in product development have been developed for this, e.g., in the fields of information
systems science, human-computer interaction, marketing, innovation and service science.

Most of the user participation methods allow either rich face-to-face communication among a small group of
participants or limited feedback from a large number of people, e.g., via online surveys. In addition, social media
tools have been made available for involving the users into the innovation process (Nkki, 2008). Despite the
known benefits of user involvement, integration of real end-users in innovation activities remains a difficult task
(Schumacher and Feurstein 2010). For companies, it is challenging to find the customer innovators and establish
ties with them in a cost-effective manner (Nambisan, 2002). User participation is lacking especially in the early
phases of the innovation process (Flstad, 2009) although early user involvement would be most beneficial for a
successful design process (Gardner, 1999; Madsen and Borgholm, 1999).

There is no single methodology for user involvement in system design; instead, there are various approaches
stemming from different disciplines (Kujala, 2003). In user centric design (UCD), the user is involved as an
essential part of the design process. The user centric design process should begin at the very early stages of the
project, usually when the initial concept for the product or system is being formulated (ISO 9241-210:2010). The
standard determines four phases: 1) Understanding and specifying the context of use, 2) Specifying the user
requirements, 3) Producing design solutions and 4) Evaluating the design. When repeating these phases
iteratively, feedback from users becomes a critical source of information.

Participatory design (PD) is an approach that gives users an active role in the design process and pays attention to
different roles, power structures and democracy that are involved in systems design especially at workplaces
(Schuler and Namioka, 1993; Greenbaum and Kyng, 1991). Both user centric design and participatory design are
led by professional designers in a company that defines the innovation target. In user centric design, the users do
not necessarily have any contact with each other whereas in participatory design users participate in design
workshops and thus meet other users as well as designers and developers.

In this study, PD was used as the main user-centric approach but some elements of the UCD were combined with it.
As this study aims at providing a new way of working for the maintenance personnel, it was of specific interest and
importance to involve the actual end-users in the design process. This was also the main reason for choosing PD as
the main theoretical framework, as it is important to consider not only the design of a new application but also how
the working practices are affected and how motivated the end-users are.
3. USER IDEAS FOR THE APPLICATION
The design process was started together with all participating companies, but three companies (Granlund, Skanska
and Pyry) were involved in bringing user groups into the first phase of the application design process. As the first
step, company-specific workshops were organized with the participating companies in order to find the most
appropriate end-user group for the development of the mobile AR application. The workshops were arranged at the
end of 2011.
As a result, building maintenance personnel were seen as the most appropriate group to work with. This group was
seen as the most potential end users of applications that were relevant for the project goals, i.e. utilizing a location
based mobile AR service indoors. Suitable people working with maintenance needed to be found for the next step.
Three of the participating companies named their representatives for forthcoming group interviews. Additionally,
one company wanted to include the viewpoints of building instructors in the interviews.

204

Four group interviews were conducted in January and February 2012. Altogether 12 people participated in the
interviews, 24 persons in a single interview. During the interviews, ideas and comments related to the tablet
application were collected with a wide scope, not necessarily limiting to maintenance work, BIM or AR. The
interviewees were allowed to express freely their wishes for any mobile applications that would be useful and
beneficial for carrying out their work. All of the interviews were recorded for analysis.
The aim of this first phase of involving the users was to collect user ideas in order to support and guide the
technical development of the AR tablet application. The ideas were taken into account and partly included in the
system requirements of the application. Since this phase of user study was conducted without hands-on access to
the application software system, the role of user ideas was preliminary and supportive, and worked as a
communication tool between participating companies and system developers.
The focus of the project is in utilization of BIM as data source for AR application, thus the ideas with BIM as
primary data source will be emphasised in the development of the actual system. In the second phase of the user
study, users will have the opportunity to test a prototype of the system. The concrete experience of holding a
mobile device and using an application in real settings will provide an important aspect of the user experience.
As in the first phase the users could express any wishes for the application, it is possible that they change their
minds to some extent when they are actually testing the application. For getting a realistic view of the motivation
of the users to actually use an application, both parts of the design process are very important.
3.1 Analysis of the ideas
Altogether 43 ideas were collected and listed from the group interviews. The ideas were then categorized from two
viewpoints:
1) the data source(s) related to the idea;
2) the role of AR in the idea.
Categorization based on the role of AR revealed nine ideas that were not relevant for the focus of this study, having
no need for any type of locationing. Additionally, five ideas were unclear or combined with other ideas. After this
filtering, 29 relevant ideas remained on the list. The participating companies were also asked to prioritize the ideas
from the users viewpoint (value to user). Finally, 16 ideas were prioritized by three companies (Granlund,
Skanska, Pyry). More specific descriptions of these categorizations are presented in sub-chapters 3.1.1 and 3.1.2.
3.1.1 Data sources related to the idea - BIM Table
Since the use of BIM information was in the main focus of the study, it was chosen as the defining technology for
the data source based categorization. The ideas were further placed in four categories: 1) Ideas including BIM as
data source; 2) Ideas including BIM and one other data source (BIM+); 3) Ideas including BIM and several data
sources (BIM+++); 4) Ideas not including BIM. The other data sources included sources such as building
automation system, facility management system and building service book. Based on this categorization, BIM was
included in 23 ideas out of 29 relevant ideas. The largest category was the second one, BIM+, with 14 ideas. Six of
the ideas did not include BIM as data source. The BIM Table is presented in Figure 1.

Fig. 1: Categorization of 29 relevant ideas based on data source (BIM Table).
205

3.1.2 The role of AR and company priorities
The role of AR was examined and evaluated in every idea. Four further categories were constructed: 1) AR in
major role; 2) AR in supporting role; 3) Tracking of the target device, but no AR view required; 4) Locationing of
the building area (no AR). Based on this analysis, 18 ideas with BIM as data source included AR in major or
supportive role. Thus, 18 ideas (62%) of all 29 relevant ideas supported the original idea of combining AR and
BIM in this project. Six ideas included only tracking of the target device, and one idea included locationing of the
building area alone.
As the final step, the companies prioritized the ideas by evaluating the value to the user on a scale 1-3
(1=obligatory feature; 2=important feature; 3=less important feature). All the ideas with average value < 2
were listed as prioritized. Altogether 16 ideas were prioritized. 13 out of 16 prioritized ideas included AR in major
or supportive role and seven prioritized ideas included AR in major role (see Figure 2). The 16 prioritized ideas are
presented more specifically in chapter 4.1 (Table 1 and 2) with practical examples.

Fig. 2: Categorization of ideas based on data source, role of AR and company priorities.
4. SYSTEM REQUIREMENTS
Next we present the planned system architecture of the DigiSpaces (DS) software system. It contains three
application modules:
1) The DS_BIM_Studio application is used for inputting and updating the BIM data from various sources, and
adding external information to the BIM model. The application can also show user feedback data that has been
collected from the site, like photos and reports.
2) The DS_AR_Studio application is used for defining locationing and tracking specific information to the project ,
all linked to the BIM (see below for some details). The software supports creating a reconstruction file for the
tracking, as well as indoor positioning mapping to the common project coordinate system.
3) DS_Browser is the application providing the mobile AR view and access to BIM (and other) information on the
site. The application shows the 3D BIM rendered in correct size and orientation in the users video view. The
DS_Browser application provides access to the BIM information related to the 3D model, as well as user feedback
to the application database.
The two first (Studio) applications are typically used at the office, and the mobile user interacts only with the
DS_Browser application. See Figure 3.
206

Fig. 3: DigiSpaces software system architecture.
For Positioning technology we are using both Nokias HAIP (based on incoming signal angle) and WLAN
(triangulation), and for 6DOF model & feature based tracking. The Tracking initialization data utilizes of one or
several of the following methods: 3D model simplified from BIM for tracking purposes; images or video
sequences of the environment; fingerprinting to match sensor data with previously measured groung truth;
combined with the floor plan and other informatio of the building, All this data is in principle available also for
Tracking recovery wth the DS_Bwowser, by WLAN server connection or ultimately as cloud based servce.
The graphical user interface is designed for tablet use. The user is able to get more information from the model by
browsing the models part list. The selected part is highlighted in the AR and VR view. The user is also able to
select interesting parts from the visualization views (AR/VR) by clicking the model in the required point. The part
is highlighted in the view and in the part list and additional part information is shown to the user. The user is able to
add feedback to the part like mark a maintenance procedure being done, etc.
4.1 User ideas and system requirements
In the DigiSpaces project plan, the main research emphasis is on BIM-related functionalities that can be utilized in
AR view. Thus, in our case user ideas which do not utilize BIM as data source have low priority in the pilot system
design process. The user ideas are mainly related to the user on site and therefore the DS_Browser application for
the mobile user on the site is the most important module in the case of this study. Obviously, the requirements
especially concerning other data sources affect the design and implementation of DS_BIM_Studio as well.
In the user studies, one of the first observations was that the users wanted to have also other kind of information
available on the mobile device than just geo located BIM information. Thus, the Virtual Reality mode was
included in the DS_Browser implementation plans as well. In the Virtual Reality (VR) view the model is
visualized as it is with the addition of a floor plan. In this mode the system can provide the user location
information in the whole positioning/tracking area, also allowing the user to view the BIM in any desired view.
After reviewing the 16 user ideas in relation to the project plan requirements, new prioritizing was made in order to
guide the design of the pilot system. Note that the software system based on this first phase of user study does not
fulfil all the project plan requirements. The idea is to build a system for piloting and demonstrating the
functionalities. In the second phase, more functionalities will be added to the system. Below are two tables
showing how 16 user ideas have been prioritized for the design of the DS_Browser application. Firstly, Table 1
lists the high priority functionalities for DS_Browser.

207

Table 1: High priority functionalities for the pilot system (bold text: highest importance)
Idea Example Data source Role of AR
1) BIM based fault diagnosis. Water pipe is leaking, in room 1067, where is the
shut-off valve.
BIM supporting
2) Viewing the faulted device affected
zone in 3D.
Where is the faulted air handling unit and what is its
affected area.
BIM supporting
3) Mobile route guidance for walking
inside the building.
Floor plan related mobile route guidance helps to
navigate to the conference room.
BIM supporting
4) Building performance and related
problem monitoring in 3D.
A service man using tablet can see outside the
building in which part of the building for example the
temperature levels have been too high in this summer.
BIM+ major
5) Viewing of target spaces inside building
or important issues inside the space for
different user profiles (cleaner, security
guard, technician, etc.).
Cleaner can see what his/her cleaning area in this
building is. Service man can see the sensor values in
this room or what is broken in this room.
BIM+ major
6) Rebuild phase related design: how to
get the information (photos etc.) from the
construction site to the different interest
groups.
Workers in the construction site update the model for
example by taking supporting photos (with direction
information).
BIM+ major
7) Alarm location inside the building. Duty officer receives the alarm and the application helps
him to navigate to the right place, locate the device and
get some instruction.
BIM+ supporting
8) BIM based virtual model for building
management purposes.
Building maintenance person can use some building
functions from BIM based model.
BIM+ tracking only
9) Building automation related solution for
finding differences between design values
and measurements values.
Service man can check using the tablet camera if some
measurement value in this room differs from the design
value.
BIM+++ supporting
10) Tablet based audit applications. Auditor can do the auditing comments / marks (utilising
indoor positioning) when walking in the building.
BIM+++ major
11) Installer application for informing others
that these are already installed.
Installer updates the BIM online or after the working day
that these devices / systems are installed.
BIM+++ major
Altogether 11 ideas are seen as high priority functionalities. The ideas 1, 2, 3, 4, 5 and 6 (highlighted in bold) are of
highest importance. This is due to categorization based on data source and role of AR.
Ideas 1, 2 and 3 include BIM as the only data source, thus they are fundamentally of highest priority from the
perspectives of this study. The first two ideas are related to fault diagnosis. Idea 3 (Mobile route guidance for
walking inside the building) is related to the primary function of DS_Browser: Tracking of the location inside the
space.
Ideas 4, 5 and 6 have one data source in addition to BIM, but AR is seen as being in a major role in the ideas. These
are therefore of highest importance as well. In idea 4, the additional data source could be for example building
automation system. Idea 6 (how to get the information (photos etc.) from the construction site to the different
interest groups) includes important functionality for DS_Browser, i.e. taking photos and linking them to a 3D
model with location, direction and time. Note that the original idea is related to rebuild phase, not maintenance
work, but the functionality can very well be utilized in maintenance work as well.
The rest of the ideas (not of highest importance) with more than one additional data source and/or AR in no major
role will be processed after the six most important functionalities have been realized.
The remaining five functionalities are not utilising BIM as data source. Based on the project plan requirements,
these ideas have low priority in this project. Additionally, idea 14 (visualizing of real estate service area borders
and snow ploughing routes) is not actually related to indoors tracking, and is therefore out of the scope of this
project. Table 2 contains a list of low priority functionalities.

208

Table 2: Low priority functionalities.
Idea Example Data source Role of AR
12) Tablet based usage diary for building
sector professionals.
Building sector professionals can handle fault related
usage diary in mobile device.
No BIM major
13) Tablet based service request. Office worker see the problem in the building and sends
the service request. The indoor positioning based tablet
application insert the right location information
automatically to the service request.
No BIM supporting
14) Visualizing of real estate service area
borders and snow ploughing routes.
GPS based AR application for informing real estate
service borders.
No BIM major
15) Utilizing of technologies like QR code,
RFID, etc. also in no BIM applications
(alternative or support option for AR).
Identification of devices, spaces etc. for no BIM based
AR applications in general.
No BIM tracking only
16) Visualising building life cycle projects. Building owner is renovating the building and the spaces
must be in use at the certain time. Tablet application
shows what the situation is now and what should be the
degree of readiness.
No BIM locationing only

5. DISCUSSION
According to the original DigiSpaces project plan, AR applications that are utilizing BIM as the main data source
are on the main focus of this project. However, based on the user interviews and analysis, only four out of 29
original ideas included BIM as the only data source. The company representatives prioritized three of these four
ideas, including fault diagnosis and route guidance inside the building. The latter idea including indoor tracking is
considered as one of the primary functionalities of the software system. These ideas will thus be of primary interest
when realising the pilot system. The majority of the ideas arising from the users, 19 ideas altogether, included at
least one additional data source to BIM, and this must be taken into consideration when designing the pilot system
and additional development after piloting. Six ideas did not contain any BIM information and were therefore
considered as secondary functionalities in the further development of applications in this project.
User ideas that are utilising only or mainly BIM as data source and have AR in major or supporting role are the
most important functionalities in the pilot implementation phase. The three functionalities including BIM as the
only data source, and furthermore, three functionalities with one additional data source to BIM and with AR in
major role are considered as highest priority in implementation (see Table 1). Even more specific prioritization can
still be made based on the role of AR and number / type of other data sources than BIM (i.e. the lesser the other
sources, the higher the priority).
The design of pilot system is currently in process and technical possibilities as well as project schedule will be
affecting the amount and quality of the chosen functionalities. The technical system requirements may also alter
during the design process. The role of the user ideas in this phase of the project is supportive in nature; the system
developers will evaluate the ideas in relation to technological progress of the system and time schedules. As the
users typically dont have the technological competence required for evaluating the possibilities to build
prototypes based on their wishes, it is the responsibility of the developers to consider which features are included
in the prototype to be tested in the second phase of the study. In addition, the user ideas in this phase are still
preliminary: the testing in next phases with the actual pilot system will reveal new and more accurate user needs,
and possibly question some of the ideas presented in this phase.
6. CONCLUSIONS
In this article we have presented the results of a user study based on participatory design, in which guidelines for
a mobile Augmented Reality (AR) system utilizing Building Information Models (BIMs) were created for the
buildings life cycle management applications. As one of the main conclusions of the study, users can provide
valuable information and ideas for the design process.
209

Based on earlier experiences (e.g. Seisto et al. 2011), the motivation of the user to actually start using new
technology for work purposes is highly affected through the involvement of users in the design process. Therefore
it is valuable to take into account the user perspective already from the beginning of the design process and guide
the process into a direction that also recognizes the motivation of the users. In this project the users involved in the
group interviews were very enthusiastic in discussing how technology could be utilized in everyday work duties.
They provided a large number of good quality ideas effortlessly and generally saw that a mobile tablet application
would be easily utilized in their work.
In our case, the users provided altogether 43 ideas which were categorized and prioritized in several steps, leading
into 11 ideas highly relevant for this project. These ideas will be utilized as guidelines for the piloting phase. User
ideas may not always be usable as such but analysis and refining can often link them into system requirements.
This may be time consuming, but ensures that all the stake holders (i.e. end users, developers and the steering
group of the project) involved in the process share a common goal. This supports the basic principle of
participatory design related to different roles, power structures and democracy that are involved in systems design.
The categorization based on data source, role of augmented reality, and participating company priorities proved to
be useful in analysis of the ideas. Several of the user proposed ideas, including many outside the original project
scope, were taken into account in the final application design. This expands our work from just technological
implementation to providing solid grounds also for the second phase of the study when involving the users in
piloting in real work environments.
7. REFERENCES

Allen M., Regenbrecht H., Abbott M. (2011). Smart-phone augmented reality for public participation in urban
planning. Proc. OZCHI11, Nov 28 - Dec 2, 2011, Canberra, Australia.
Flstad, A. (2009). Co-creation through user feedback in an online Living Lab: A case example. Proceedings of the
International Workshop on Open Design Spaces. International Reports on Socio-Informatics, 6(2), pp. 43-55.
Gardner, J. (1999). Strengthening the focus on users working practices. Communications of the ACM, 42 (5), pp.
79-82.
Gheisari M. and Irizarry J. (2011), Investigating facility managers decision making process through a situation
awareness approach. International Journal of Facility management, Vol 2, No 1.
Gheisari M., Williams G., Eastman C., Walker B. (2012), Info-Spot: low-cost, mobile AR aapproaches for
accessing building information for Situation Awareness Enhnacement. Manuscript, submitted July 2012.
Greenbaum, J. and Kyng, M. (eds.) (1991). Design at work: cooperative design of computer systems, Lawrence
Erlbaum, Hillsdale, New Jersey. 294 p.
Kujala, S. (2003). User Involvement: a review of the benefits and challenges. Behaviour & Information Society
(22) 1, pp. 1-16.
Madsen K.H. and Borgholm, T. (1999) Cooperative usability practices. Communications of the ACM, 42 (5), pp.
91-97.
Nambisan, S. (2002) Designing Virtual Customer Environments for New Product Development: Toward a Theory.
Academy of Management Review, 27(3), 392413.
Nkki, P., Antikainen, M. and Virtanen, T. (2008) Participatory design in an open web lab Owela. In the workshop
on Distributed participatory design, CHI, May 0510, 2008, Florence, Italy.
Olsson T., Savisalo A., Hakkarainen M., Woodward C. (2012). User evaluation of mobile augmented reality in
architectural planning. eWork and eBusiness in Architecture, Engineering and Construction, Gudnason G. &
Scherer R. (eds.), ECPPM 2012, Reykjavik, Island, Jul 25-27, 2012, pp. 733-740.
Sanders, E. B.-N. and Dandavate, U. (1999) Design for experiencing: New tools. In Proceedings of the first
international conference on Design & Emotion. (Overbeeke C.J. and Hekkert P. eds.), Delft University of
Technology, Delft, the Netherlands.
210

Schuler, D. and Namioka, A. (Eds.) (1993) Participatory Design: Principles and Practices, Lawrence Erlbaum
Associates.
Schumacher, J. and Feurstein, K. (2007) Living Labs the user as co-creator. In: ICE 2007 Proceedings: 13th
International Conference on Concurrent Enterprising, Sophia Antipolis, France.
Seisto, A., Federley, M., Kuula, T., Paavilainen, J. & Vihavainen, S. (2011). Involving the end-users in the
development of language learning material. International Journal of Mobile and Blended Learning, 3(2),
April-June 2011. pp. 4356.
Siltanen P., Karhela T., Woodward C., Savioja P. (2007), Augmented reality for plant lifecycle management. 13th
International Conference on Concurrent Enterprising (ICE2007), Sophia.Antropolis, France, 4-6 June 2007.
Woodward C., Hakkarainen M. (2011). Mobile mixed reality system for architectural and construction site
visualization. Augmented Reality - Some Emerging Application Areas, Andrew Yeh Ching Nee (ed.), InTech,
ISBN 978-953-307-422-1, pp. 115-130.
211

EXPLORE PATTERNS OF HUMAN-COMPUTER INTERACTIONS IN
PARAMETRIC DESIGN ENVIRONMENTS
Rongrong Yu, Ning Gu & Michael Ostwald
University of Newcastle, Australia
ABSTRACT: Current literature shows that there is a lack of empirical evidence support the understanding of
human-computer interaction (HCI) process in parametric design environments (PDEs). Parametric design is a
new digital design method increasingly applied in architecture, and characterized by parametric relationship
control, rule algorithm design and multiple solution generation. The dynamic nature of parametric design
indicates frequent interactions between designers and design environments. Therefore, we are wondering if there
are particular designers behavior patterns of HCI in PDEs. To explore the issue, this study proposes a pilot study
in which two students are involved to complete a design task using parametric tools. Furthermore, method of
protocol analysis is applied to analyze the video-recorded data of their design processes. Based on Schns study
on designers cognitive behavior (reflection in action and seeing-moving-seeing), we develop a coding
scheme for analyzing designers behavior from three levels: intention level, action level and perceptual level.
During HCI process in PDEs, designers activities transfer among these three levels. For the intention level,
Geros FBS model is introduced to identify designers thinking processes: FBS model has been suggested to be able
to capture most of the meaningful design processes and indicate clear transition between design events. For the
action level, both geometry modeling activities and specific actions happened in PDEs are categorized in order to
analyze the action happened. Perceptual level is suggested to be the accumulation of generating new intentions,
which plays important role in the process of interaction between designers and PDEs. All of the three levels
contain two types of activities: design knowledge based and rule algorithm based. Through protocol analysis,
some interaction results and patterns of inspiring and reflecting behavior is identified and discussed.
KEYWORDS: Human computer interaction, parametric design environments, behavior patterns, protocol
analysis, FBS model
1. INTRODUCTION
Parametric design has become increasing popular in architecture design field recent years. Its dynamic feature
enable designers to have more flexible and controllable design environments (Schnabel, 2007). In parametric
design environments (PDEs), designers not only work on geometry model but also make rule relationship, which is
quite different from what they do in traditional modeling environments. Undoubtedly, by reflecting instantly on the
changing parameters or parametric rule relationship, designers tend to have more frequent and closer interaction
with design environments. Cross (2011) believes that expert designer have more intense, reflective interaction
with representations of problem and solutions, and they can shift quickly between design instances. Therefore, we
are wondering what role does parametric design tools plays during the interaction process between designers and
PDEs; and are there any particular designers behavior patterns of human-computer interaction (HCI) in
parametric design process? Previous studies show that parametric design environments advanced design process in
a variety of ways (Iordanova et al., 2009, Burry and Holzer, 2009, Schnabel, 2007). However, there is a lack of
evidence regarding the interaction process between designer and parametric design environments. To address the
issue, this paper proposed a pilot study in which two designers are involved to complete a design task in PDEs.
Then method of protocol analysis is applied to explore designers behavior patterns of their interaction with PDEs.
From protocol analysis, some interaction results and patterns of inspiring and reflecting behavior is identified and
discussed.
2. BACKGROUND
2.1 Human computer interaction in design field
From mid-1980s, human-computer interaction (HCI) has become a specific term extend to emphasis all aspects of
interaction between user and computer rather than just the design of interface (Preece, 1995). There are varieties of
definitions, a typical of them is a set of processes, dialogues, and actions through which a human user employs
and interacts with a computer. (Baecker and Buxton, 1987, p.40). HCI has been studied broadly across disciplines
such like computer science, phycology and cognition, industrial design field. Though there are numbers of
212

research study from users perspective (Persons and Foa, 1984, Isen et al., 1991) , HCI research mainly focus on
the technical aspect such like applications in virtual reality (Hudlicka, 2003) in design domain. With the
development of digital design in architecture field, it is in urgent need of HCI study for exploring a better
interaction way which may beneficial for design process. In this study, we mainly focus on cognition aspect of
designers behavior in their interaction process with parametric design tools rather than a broader scope of the
computer function core or interface design inside the software.
2.2 Protocol analysis of parametric design
Parametric design is a new digital design method increasingly applied in architecture, and characterized by
parametric relationship control, rule algorithm design and multiple solution generation (Karle and Kelly, 2011,
Abdelsalam, 2009). According to Woodbury (2010), it supports the creation, management and organization of
complex digital design models. By changing parameters of an object, particular instances can be altered or created
from a potentially infinite range of possibilities (Kolarevic, 2003). Previous studies on designers behaviors in
PDEs show that parametric tools advance design processes in a variety of ways. For instance, evidence shows that
the generation of ideas is positively influenced in PDEs. Particularly, in Iordanova et al.s (2009) experiment on
generative methods, ideas were shown to be generated rapidly while they also emerge simultaneously as variations.
Moreover, Schnabel (2007) shows that PDEs are beneficial for generating unpredicted events and can be
responsible for accommodating changes. However, researchers have typically studied design behavior in PDEs
mostly by observing students interactions with PDEs in design studios or workshops. Arguably, this approach can
hardly provide an in-depth understanding of designers behaviors in PDEs. This empirical gap will be addressed in
the present study by adopting the method of protocol analysis.
Protocol analysis is a method widely used for cognitive studies into designers behavior during design processes
(Cross, 2001, Cross et al., 1996, Ericsson and Simon, 1980). It has been applied across a variety of design
environments, for instance, by Kim and Maher (2005) to compare GUI (Graphic User Interface) and TUI (Tangible
User Interface) environments and by Bilda and Demirkan (Bilda and Demirkan, 2003) to study sketching and the
CAD environment. The general procedure of protocol analysis is that the protocol data (in this case, the
video-recorded information of designers behavior) collected from the experiment is transcribed and segmented; a
customized coding scheme is then applied to categories segments. In the following section, we will develop a
customized coding scheme which is capable of coding three levels of designers interaction activities, where the
characteristics of parametric design are reflected.
3. HUMAN COMPUTER INTERACTION PROCESS IN PARAEMTRIC DESIGN
ENVRIONMENTS
In PDEs, the interactions between designer and computer tends to be more frequent than in traditional design
environments due to the dynamic feature of parametric design. Evidence suggests that frequent shifts of attention
or activity mode likely lead to more creative design outcome (Cross, 2011). Therefore, we are interested in
exploring how the interaction processes happened in PDEs.
3.1 From designers cognition perspective
Design is a purposeful activity to create artifacts or change the world in order to suit peoples need (Gero, 1990).
Sharp et al. (2007) defines seven cognition processes in HCI: attention, perception/recognition, memory, learning,
reading, speaking/listening, and problem solving. During parametric design process, designers attention,
recognition, perception, memory and problem solving are the typically cognitive processes happen in PDEs.
Figure 1 illustrates the cognitive process in PDEs, which is developed based on Barbers (1988) information
process model. In PDEs, designers attention (we call it intention in this study to compare with action) is the
original driving force for design concept development. Intention is inspired by both responds from parametric
design interface in real time and designers memory. The information or responds are encoded, compared and
selected, and finally designers make a decision and take actions to response to the computer. The information
encoded is a recognition process, where designers recognize and start to analyses the information or design
problem. Multiple design solution can be developed in parallel in PDEs, so the perception process of comparing
and selecting from multiple alternatives happens frequently. The problem solving process of taking execution is a
reflection of previous memory and attention. Usually, designers develop concept by deriving knowledge from
the previous experiences and design knowledge stored in the memory (Demirkan and Afacan, 2012). Therefore,
all the recognition, perception and action of solving problem constantly accumulate and interact with designers
213

intention and memory, which reflects on the final decision making.
Fig. 1: Designers cognition model (After Barber, 1988)
3.2 Parametric design as a design media
Design media has significant effect on designers thinking process (Chen, 2001, Mitchell, 2003). A number of
studies have suggested that sketch assist with design thinking as an effective design media (Black, 1990, Schn,
1983). With increasing application of digital design tools, scholars start to study on the influence of computational
design tools on design process. For instance, Fallman (2003) claims that CAD tools can reflect designers thinking
process as a thinking media just like what sketch does in traditional design environment. Parametric design as a
new media applied in architecture design, its influence on design thinking is an urgent gap to be filled.
Parametric design basically is to make a logical parametric relationship reflect designers concept, which can be
called design of design (Burry, 2011). This feature adds an extra layer of thinking over the traditional way of
designers thinking. In HCI study, user interface plays important role in the interaction process (Norman, 1992).
Parametric design tools usually have two types of interface: Geometry modeling interface and Data management
interface. On data management interface, designers set/change parameters and make rule relationship; while on
geometry modeling interface, designers view and evaluate the model. The two types of interface facilitate media to
report and delivery information effectively. During parametric design process, designer will shift between the two
types of interface, which potentially provoke more frequent interactions between designers and computers.
3.3 Dialogue between designer and computer in PDEs
From previous discussion, we know that designers interact with PDEs by taking actions driven by their intentions
and perceptual activities. Designers intention is a responds/reflection to the instant feedback through the execution
of action (Schn, 1992, Goldschmidt and Porter, 2004). During parametric design process, designers get
inspirations from what they see on the screen; at the same time, they reflect on what they see and what they do by
taking action of making rules. In this process, the perceptual activity plays an important role in facilitating design
thinking. Schn describes this process as seeing-moving-seeing---- the designer sees what is there on the
screen or their sketch, draws or model in relation to it, and sees what they have produced, this process informs
further design (Schn, 1992). Figure 2 illustrates the interaction process between designer and computer in PDEs:
designers intention is the driven force of design action, which is making parametric rule relationship; the
generated geometry model inspires designers new intention. The whole process is circular that consistent to build
up the design problems and solutions and inform further the design forward.

214

Fig. 2: Interaction process between designer and computer in PDEs
4. A PILOT STUDY
4.1 Experiment setting
Aiming to provide better understanding of HIC process in PDEs, this study proposes a pilot study to use protocol
analysis explore designers behavior patterns. In devising an experiment to collect protocol data from PDEs, 2
students are recruited to complete a design task using commercial parametric design software (Grasshopper in this
study) in 60 minutes. Each of participants, all master of architecture students, has had at least two years of
parametric design experience. The expectation is that some typical design behavior patterns regarding HCI in
PDEs will be identified.
The parametric environment is a computer, a pen and paper, with two video cameras (Figure 3). The design task is
to generate a conceptual form for the tower part of a high-rise building. During the design process, both think
aloud and retrospective method are applied to collect protocol data. Designers verbalization and design actions
are video-recorded for later use as protocol data. Generally, the two students show a good ability of manipulates
forms in Grasshopper as well as representing the advantages of parametric design. However, their design strategy
is quite differentdesigner A uses a top-down method (in which the designer considers mainly the faade of the
building) and the designer B is bottom-up (in which the designer considers hotel rooms as a unit and focus on the
combination of these in the overall tower form).
Fig. 3: Design environments
4.2 Protocol analysis
Based on the discussion in section 3.3, the interaction between designer and design environment includes
designers intention, action and their perceptual activities. For the pilot study of the two students, we use a
customized coding scheme consist of three levels of designers interaction behavior in PDEs: intention level,
action level and perceptual level. Based on related works in parametric design such as those by de Boissieu et al.,
(2011) and Woodbury (2010), there are two levels of typical activities in parametric design process: activities
based on design knowledge and activities based on rule algorithms. In PDEs, besides thinking from the perspective
of design knowledge, designers also consider from the aspect of rule algorithm. At the rule algorithm level,
designers think about the way to make use of parametric tools to serve their design concept. For instance, they will
consider the establishment of parametric relationship, the selection of component to achieve certain purpose, etc.
Additionally, most design knowledge level protocols are produced in modeling interface, while rule algorithm
level protocols usually happen in scripting/data management interface. Furthermore, there are two types of
215

activities: New means the first time a design instance appears; while Old means a design instance appears after
the first time. During design process, it is very likely that a new intention is inspired by an old instance. That is
also an interesting process potentially introducing new variables and reconstructing design problems.
4.2.1 Intention level
In order to analyses data of designers intention, Geros FBS model (2004) is introduced as the foundation for
developing coding category. The reason is that FBS model is claimed to be able to capture most of the meaningful
design processes (Kan and Gero, 2009) and provide clear transitions between design instances. The FBS model
contains three classes of variables: Function (F), Behavior (B) and Structure (S). Function represents the design
intentions or purposes; behavior represents how the structure of an artifact achieves its functions; structure
represents the components that make up an artifact and their relationships. The rule algorithm level of intention
mainly from perspective of rulemaking or scripting based on the consideration of function, behavior and
structure. Table 1 is the coding category of intention level.
Table 1: Coding scheme of intention level
Category Name Sub-category ID Description
Function (F) Requirement-New Design knowledge F-K-Rn The first time read design brief
Requirement-Old Design knowledge F-K-Ro Revisiting the requirement
Intention-New Design knowledge F-K-In Initial definition or interpretation of function
Structural Behaviour (Bs) Intention-New Design knowledge Bs-K-In Behaviour interpreted from structures
Rule algorithm Bs-R-In Behaviour From existing script
Expected behaviour (Be)

Intention-New Design knowledge Be-K-In Expected behaviour from design knowledge
Rule algorithm Be-R-In Expected behaviour form rule making
Intention- Old Design knowledge Be-K-Io Change the expected behaviour from design knowledge
Rule algorithm Be-R-Io Change of expected behaviour from rule making
Structure (S) Intention-New Design knowledge S-K -In Expected structure, about size, colour or shape
Intention- Old Design knowledge S-K -Io Change of structure
4.2.2 Action level
Action level of designers activities contains three main categories: Geometry, Parameter and Relationship. The
parameter setting /changing and rule relationship making/changing is typical activities happened in a parametric
design process. From analyzing, the role parametric tools plays will be highlighted. Table 2 is the coding category
of action level.
Table 2: Coding scheme of action level
Geometry

Action-New

Design knowledge G-K-An Initial making of geometry or talking about modeling method
Rule algorithm G-R-An Set component for the purpose of making geometry
216

Action-Old

Design knowledge G-K-Ao Changing existed geometry or modeling method
Rule algorithm G-R-Ao Change component for the purpose of changing geometry
Parameter

Action-New Rule algorithm P-R-An Setting parameters
Action-Old Rule algorithm P-R-Ao Changing or resetting parameters
Relationship

Action-New Rule algorithm R-R-An Setting parametric relationship
Action-Old Rule algorithm R-R-Ao Changing parametric relationship
4.2.3 Perceptual level
Perceptual behavior means designers attention or reorganization of geometry elements or parametric relationship.
Perceptual activity is one of the important factors in the interaction process between designers and parametric
environments, which is the accumulation of generating new intentions. Meanwhile, a new perceptual activity
has close relationship with appearance of unexpected discovery (Suwa et al., 1999). From Schns theory of
seeing-moving-seeing, seeing is an activity of perceptual level. Perceptual activity calls designers attention
on particular design elements or aspect, which may lead to reconstruct of design problem or introduce new
variables. That potentially provides new design directions.
Table 3: Coding scheme of action level
Attention Perceptual-New Design knowledge A-K-Pn Attention or recognition of geometry element for the first time
Rule algorithm A-R-Pn Attention or recognition of rule relationships for the first time
Perceptual-Old Design knowledge A-K-Po Attention or recognition of geometry element after the first time
Rule algorithm A-R-Po Attention or recognition of rule relationships after the first time
5. RESULTS
For the pilot study of the two students, we use the developed coding scheme to code the cognitive behavior of the
designers. The segmentation is according to semantic meaning in terms of function, behavior and structure, or
action meaning. There are respectively 221 and 206 segments from the two protocol data, and over 80% of the
meaningful design processes can be coded. As shown in table 4, student 1 and student 2 have different coding
coverage of the three levels. Student 1 has more intention and less action compared to student 2s design process.
Both of the students have the least coverage of perception coding.
Table 4: General coding coverage
Design time Number of segments Coded segments Intention level Action level Perception level
Student1 36 mins 221 201 40.2% 33.6% 26.2%
Student 2 49 mins 206 181 28.4% 46.1% 25.5%

217

Fig. 4: Three levels of design activities of student 1 along the whole design session
Fig. 5: Three levels of design activities of student 2 along the whole design session

5.1 Interaction analysis
Designers interaction with PDEs is represented by the transition between designers intention, action and
perceptual activities. Figure 4&5 illustrate the frequency occurrence of these three levels along the whole design
processes. For both students, the three levels of designers behavior transfer frequently during design process.
Meanwhile, designers intention and action usually appears together, and perceptual activities appear
independently. Furthermore, there are more perceptual levels of activities appear towards the end of design session,
which means that designers have more attention of the parametric model in the end; at the action level, both
students are active from beginning to the end; while at the intention level, both students are more active at the
beginning of the session. That also match statement of Zeiler et al. (2007) that most of the important design
decisions are made at the early design stage. Crosss claim that expert designers tend to have more intense,
reflective interactions and they have ability to shift easily and rapidly between doing and seeing (Cross, 2011).
Therefore, the frequent transition between three levels is beneficial for developing design ideas.

5.2 Potential patterns from combined codes
5.2.1 Behavior patterns of inspiring
From previous discussion, in parametric design process, designers get instant inspiration from the rule-controlled
parametric model. Patterns of inspiring refers to the activities designer get inspiring from what they see on the
screen. Here we use combined code to explore these higher level of understanding on parametric design process.
Combined code of this pattern includes transition from perceptual level to intention level. Figure 6 illustrates all
the transitions happen in both of the students design processes which exclude certain transitions happened only in
one students design process; the horizontal axis demonstrates all the transitions happen in both of the students
design processes; vertical axis represents the percentage coverage of the particular transition occupied in all the
transitions from perceptual level to intention level.
Figure 6 shows that both the student 1 and student 2 have the dominant pattern of A-K-Po>S-KIn,
A-K-Pn>S-K-Io and A-K-Po>S-K-Io. From protocol analysis, the most frequent transition is perceptual attention
to a structure related intention, that probably because that by inspecting of geometry model or scripting, designers
would have more intention of changing the geometry model. The most obvious pattern is A-K-Po>S-KIn, which
means that by revisiting the model, designers generate new intention of making structure from design knowledge
perspective.

218

Fig. 6: Patterns of inspiration
5.2.2 Behavior pattern of reflection
From previous discussion, designers tend to reflect on the instant feedback they get from parametric model or
scripting interface. Patterns of reflection refer to designers activity of what they do and think reflect on what they
see from the screen. There are two types of reflection: reflection in seeing and reflection in doing. Reflection in
seeing includes designers activity transfer from perceptual level to action level or perceptual level to intention
level then to action level; while reflection in doing includes designers activity transfer from action level to
perceptual level.
1. Reflection in seeing
Reflection in seeing means designers action reflects on what they see from the screen. Figure 7 illustrates all the
transitions happen in both of the students design processes which exclude certain transitions happened only in one
students design process. The dominant pattern in this level is A-K-Po>R-R-Ao, A-K-Pn>P-R-Ao and
A-K-Pn>G-K-Ao, which suggest that there are more rule related action reflect in what designers see in the
modeling interface. The most frequent appear pattern is A-K-Po>R-R-Ao, which means designers change
parametric relationship depend on their perception of geometry modeling.

Fig. 7: Patterns of reflection in seeing
2. Reflection in doing
Reflection in doing means designers reflection on what they do. Schns (1992) concept of reflection in action
suggests that designers tend to reflect on what they do. Transition of this level is from action level to perceptual
level. Figure 8 illustrates all the transitions happen in both of the students design processes which exclude certain
transitions happened only in one students design process. The dominant coded pattern in this level is
P-R-Ao>A-K-Pn, which means that designers have more perceptual reflection after they change a parameter.

219

Fig. 8: Patterns of reflection in doing
This study proposed a pilot study aim to explore designers behavior patterns in terms of HCI in PDEs. Through
protocol analysis, designers interaction among three activity levels of intention, action and perceptual is
discussed: (1) For both students, the three levels of designers behavior transfer frequently during design process;
designers intention and action usually appears together, and perceptual activities appear independently. (2)
There are more perceptual levels of activities appear towards the end of design session, which means that
designers have more attention of the parametric model in the end; at the action level, both students are active
from beginning to the end; while at the intention level, both students are more active at the beginning of the
session. (3) Three levels of interaction patterns have been identified by analyzing of combined code: patterns of
inspiring, reflection in seeing and reflection in doing. In the inspiring level, most designers intention regarding
design knowledge is inspired by geometry model; while in the reflection level, designers tend to reflect on their
rule algorithm activities. The results of this study have provided more understanding of designers interaction
behavior in PDEs. Moreover, some particular patterns which may benefit for designers interaction with PDEs
has been proposed and discussed.
The next stage of this study is to conduct a main study with a larger number of designers in order to identify
designers behavior patterns in terms of HCI. In the main study, we will compare designers behavior in PDEs
with traditional modeling environments to further explore the role of parametric design plays in HCI. Results of
the main study will help to explore factors affect designers interaction with PDEs.
7. REFERENCES
Abdelsalam, M. (2009). The Use of the Smart Geometry through Various Design Processes: Using the
programming platform (parametric features) and generative components. Proceedings of the Arab Society for
Computer Aided Architectural Design (ASCAAD 2009) Manama, Kingdom of Bahrain. 297-304.
Baecker, R. M.and Buxton, W. A. S. (eds.) (1987). Readings in human-computer interaction: A multidisciplinary
approach: Morgan Kaufmann Publishers.
Barber, P. J. (1988). Applied Cognitive Psychology: An Information-Processing Framework, Methuen.
Bilda, Z.and Demirkan, H. (2003). An insight on designers' sketching activities in traditional versus digital media.
Design Studies, 24, 27-50.
Black, A. (1990). Visible Planning on Paper and on Screen. Behavior and Info. Technology, 9, 283296.
Burry, J.and Holzer, D. (2009). Sharing Design Space: Remote Concurrent Shared Parametric Modeling.
Proceedings of 27th eCAADe Conference Istanbul, Turkey. 333-340.
Burry, M. (ed.) (2011). Scripting cultures: architectural design and programming, UK: Wiley.
Chen, S.-C. (2001). The Role of Design Creativity in Computer Media. Proceedings of 19th eCAADe
Conference Helsinki (Finland). 226-231.
220

Cross, N. (2001). Design Cognition: Results from Protocol and other Empirical Studies of Design Activity. In:
CHARLES, M. E., MCCRACKEN, W. M. & WENDY, C. N. (eds.) Design Knowing and Learning: Cognition in
Design Education. Oxford: Elsevier Science.
Cross, N. (2011). Design thinking : understanding how designers think and work, Oxford ; New York, Berg
Publishers.
Cross, N., Dorst, K.and Christiaans, H. (1996). Analysing design activity, Chichester ; New York, Wiley.
de Boissieu, A., Lecourtois, C.and Guna, F. (2011). Operation of parametric modelling and/or operation of
architectural conception?: Expressing relationships in parametric modelling. Proceeding of 29th eCAADe
Conference Slovenia. University of Ljubljana, 530-538
Demirkan, H.and Afacan, Y. (2012). Assessing creativity in design education: Analysis of creativity factors in the
first-year design studio. Design Studies, 33, 262-278.
Ericsson, K. A.and Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215-251.
Fallman, D. (2003). Design-oriented human-computer interaction. Proceedings of the SIGCHI conference on
Human factors in computing systems Florida, USA. 225-232.
Gero, J. (1990). Design prototypes: a knowledge representation schema for design. AI Magazine, 11, 2636.
Gero, J.and Kannengiesser, U. (2004). The situated function-behaviour-structure framework. Design Studies, 25,
373-391.
Goldschmidt, G.and Porter, W. L. (eds.) (2004). Design Representation: Springer.
Hudlicka, E. (2003). To feel or not to feel: The role of affect in humancomputer interaction. International Journal
of Human-Computer Studies, 59, 1-32.
Iordanova, I., Tidafi, T., Guit, M., De Paoli, G.and Lachapelle, J. (2009). Parametric methods of exploration and
creativity during architectural design: A Case study in the design studio. Processdings of CAADFutures 2009.
423-439.
Isen, A. M., Rosenzweig, A. S.and Young, M. J. (1991). The influence of positive affect on clinical problem
solving. Med Decis Making, 11, 221-227.
Kan, J. W. T.and Gero, J. S. (2009). Using the FBS ontology to capture semantic design information in design
protocol studies. In: MCDONNELL, J. & LLOYD, P. (eds.) About: Designing. Analysing Design Meetings. CRC
Press.
Karle, D.and Kelly, B. (2011). Parametric Thinking. Proceedings of ACADIA Regional 2011 Conference.
109-113.
Kim, M. J.and Maher, M. L. (2005). Creative Design and Spatial Cognition in a Tangible User Interface
Environment. In: GERO, J. & MAHER, M. L. (eds.) Computational and Cognitive Models of Creative Design VI.
University of Sydney.
Kolarevic, B. (2003). Architecture in the digital age : design and manufacturing, New York, NY, Spon Press.
Mitchell, W. J. (ed.) (2003). Beyond Productivity: Information Technology, Innovation, and Creativity,
Washington, D.C.
Norman, D. A. (1992). Turn Signals Are the Facial Expressions of Automobiles, Addison-Wesley.
Persons, J. B.and Foa, E. B. (1984). Processing of fearful and neutral information by obsessive-compulsives.
Behav Res Ther, 22, 259-265.
Preece, J. (1995). Human-computer interaction, Wokingham, England ; Reading, Mass., Addison-Wesley Pub. Co.
Schn, D. A. (1983). The reflective practitioner: how professionals think in action, Basic Books.
221

Schn, D. A. (1992). Designing as reflective conversation with the materials of a design situation.
Knowledge-Based Systems, 5, 3-14.
Schnabel, M. A. (2007). Parametric Designing in Architecture. Proceedings of the 12th International Conference
on Computer Aided Architectural Design Futures (CAAD Future2007) Sydney, Australia. 237-250.
Sharp, H., Rogers, Y., Preece, J.and Books24x7 IT Pro Collection. (2007). Interaction design beyond
human-computer interaction. 2nd ed. Chichester ; Hoboken, NJ: Wiley.
Suwa, M., Gero, J.and Purcell, T. (1999). Unexpected Discoveries And S-Invention Of Design Requirements: A
Key To Creative Designs In: GERO, J. S. & MAHER, L. M. (eds.) Computational Models of Creative Design IV.
University of Sydney,: Key Centre of Design Computing and Cognition.
Woodbury, R. (2010). Elements of Parametric Design, Taylor and Francis.
Zeiler, W., Savanovic, P.and Quanjel, E. (2007). Design decision support for the conceptual phase of the design
process. Proceeding of International association of societies of design research (IASDR07) Hongkong. 1-15.

222

II. AUGMENTED REALITY (AR) AND SENSING

223
224
AUGMENTED REALITY TECHNOLOGIES FOR AEC PROJECTS: A
LITERATURE REVIEW
Rankouhi, S. & Waugh, L.M.
University of New Brunswick, Fredericton, N.B., Canada
ABSTRACT: Since 2006, researchers at the University of New Brunswick have used virtual and augmented reality
technologies to document the construction phase of projects for facility owners. That research has identified the
following beneficial capabilities: virtual site visits, comparing as-built and as-planned status of projects, pre-
empting schedule disputes, enhancing collaboration opportunities, and planning/training for similar projects. This
paper provides an expanded foundation for future research by presenting a review of augmented reality technology.
The review is based on articles found within three well-known journals in architecture, engineering and construction
(AEC) until the end of the year 2011: the Journal of Automation in Construction, the Journal of Information
Technology in Construction, and the ASCE Journal of Construction Engineering and Management. The review
further narrows the literature within these journals by considering only those 65 articles found through a key word
search for augmented reality. The selected journal articles are classified within the following dimensions: target
audience (e.g., workers, design team, project managers, building system engineers, etc.), project phase (initiation,
design, procurement, construction, maintenance), stage of technology maturity (technical issues, proposed system
development, and application demonstration), application area (visualization or simulation, information access or
evaluation, communication or collaboration, and training or safety), and hardware (e.g., immersive or mobile). The
number of articles within these dimensions are used to identify maturing and emerging trends in the literature as
well as to synthesize the current state-of-the-art of augmented reality research in the AEC industry. In summary, the
AR literature has increasingly focused on the demonstration of visualization and simulation applications for use
during the construction phase that address issues faced by on-site workers.
KEYWORDS: Augmented reality, AEC industry, project phase, construction management, literature review
1. INTRODUCTION
The complex nature of the architecture, engineering and construction (AEC) industry and its appetite for access to
information for evaluation, communication and collaboration, increases the industrys demand for information
technologies. Recent visualization technologies are ideal in this environment. Augmented reality (AR), one of the
newly developed visualization techniques, typically combines a virtual environment with a real environment by
superimposing real images (of actual construction) on the digital images from a virtual model (of planned
construction).
Augmented reality has many advantages, for example allowing the observer to interact with both the actual and the
virtual objects and to monitor the construction progress by comparing the as-planned and as-built status of the
project. Various applications of augmented reality have been recommended for AEC industry such as those by
Dunston et al. (2005), Golparvar et al. (2009), and Vineet et al. (2011); these applications demonstrate the potential
of this technology for future use in this domain.
This paper presents a literature review of augmented reality technologies in the AEC industry up to and including
the year 2011. The aims of this review are (1) to synthesize the current state-of-the-art of augmented reality
technologies for AEC construction projects, and (2) to identify key application areas which could highly impact the
AEC industry. This goal is accomplished by classifying the literature in categories defined by the authors. Finally, a
summary of the important points and conclusions are presented.
As indicated in Figure 1, this paper describes: our selection of three journals and subsequently 65 articles on the
topic of augmented reality in the AEC industry (section 2), our review of the articles and identification of their
characteristics (section 3), our definition of relevant categories for the classification of the articles (section 4), and
our classification of the articles in the defined categories (section 5). Section 6 presents our conclusion. This
research methodology is similar to the methodology used by Aziz et al. (2011).

225

Figure 1: Research methodology
2. SELECTION OF THE JOURNALS AND ARTICLES
2.1 Journal Selection
Three diverse academic journals were selected within the domain of construction engineering and management to
record the evolution of AR technology in the AEC industry: the Journal of Automation in Construction (AIC), the
Journal of Information Technology in Construction (ITcon), and the ASCE Journal of Construction Engineering and
Management (CEM). Selection of these journals was based on their prominence in the field of construction
engineering and management research. AIC and CEM are SCI-indexed and included in the SCI-Expanded database
and ITcon is a well-recognized open access journal on the use of IT in architecture and civil engineering.
2.2 Article Selection
A total of 94 articles were found in these three journals using the search phrase augmented reality. The AIC search
resulted in 61 articles, ITcon resulted in 21 articles, and CEM resulted in 12 articles, for a total of 94 articles. After
excluding articles that were published in 2012 (due to the lack of a full year at this time) and articles such as
Calendars, Editors Notes, Subject Index, and Contents of Volume, the total number of selected articles was 65.
These 65 articles were distributed among the three journals as follows: AIC had 37 articles, ITcon had 18 articles
and CEM had 10 articles. The article selection process is depicted in Figure 2.
Figure 2: The process of selecting journal articles
2. Selection of the
journals and articles
3. Review/identification
of article characteristics
5. Categorization of
the articles
4. Definition of
categories
Selection of three prominent academic journals which
provide diversity within the area of construction
engineering and management
Science Direct: J. of Automation in
Construction (AIC)
ITcon: J. of Information Technology
in Construction (ITcon)
ASCE: J. of Construction Engineering
and Management (CEM)
Search each Journal for the phrase Augmented reality in
the full bibliographic record.
AIC: 61 articles
ITcon: 21 articles
CEM: 12 articles
Total: 94 articles
Exclude articles that were published in 2012, as well as
articles such as Calendars, Editors Notes, Subject Index,
and Contents of Volume.
AIC: 37 articles
ITcon: 18 articles
CEM: 10 articles
Total: 65 articles
226

3. REVIEW AND IDENTIFICATION OF THE ARTICLE CHARACTERISTICS

The number of articles by journal and by year are depicted in Figure 3. Among the three journals, AIC has the
highest number of articles (57%) while ITcon and CEM have 28% and 15% respectively. The maximum number of
articles in a single year were published in both 2008 and 2011 (18 or 28%).

Figure 3: (a) Total number of articles by journal; (b) Total number of articles by year
Figure 4 presents the number of articles by journal and year of publication. The chart shows high numbers of articles
for individual years in both the AIC (11 in 2011) and the ITcon (10 in 2008) journals.

Figure 4: Number of articles by journal and year of publication
227
The final characteristic identified in this section is the number of articles based on the first authors country of
residence. With 34 articles, first authors residing in the USA have the highest number of the articles about AR
technology in the AEC industry among the other countries. The remaining counts are: 9 first authors residing in
Australia; 5 in South Korea; 2 in each of Canada, Germany, Japan, Spain, and UK; and 1 in each of Austria, Greece,
Israel, Netherlands, Saudi Arabia, Taiwan, and Turkey.
4. DEFINITION OF CATEGORIES

To better understand and further segregate the literature, we defined categories to be used in this paper; each article
was then compared to these defined categories for identification of its principal focus area or to determine the
percentage of articles including reference to that classification.
Due to the complexity of construction projects and the collaborative nature of the AEC industry, the application of
AR systems could have a wide range of target audiences. To classify these articles the following audiences were
chosen: (1) workers, e.g., machine operators and technicians, (2) design team, e.g., architects, interior and exterior
designers, (3) project managers and other construction professionals, (4) building systems engineers, e.g., structural,
mechanical, and electrical engineers, (5) site developers, e.g., planners, civil engineers, and landscape architects, (6)
engineering students, (7) stakeholders, e.g., clients, and building owners (K. Muramoto, 2008). If an article
proposed a change in the work of one of these audiences, it was assigned to that subcategory for this classification.
The life cycle of a construction project consists of a sequence of steps or project phases to be completed in order to
reach project goals and objectives. These phases are defined by N. Dawood (2009) as: (1) initiation and outline
design, (2) design development, (3) [procurement], contract and pre-construction, (4) construction, and (5)
maintenance.
From a stage of technology maturity perspective, the articles are divided in three subcategories: (1) technical issues,
e.g., investigation of tracking, positioning and orienting issues for AR-based technology for steel column inspection
(D. H. Shin et al., 2010), (2) proposed system development, e.g., development of ARVISCOPE (AR animation
scripting language) and ROVER a mobile computing framework for information modeling and simulation of
construction operation (Behzadan et al. 2011), and (3) application demonstration, e.g., application of D
4
AR for
construction progress monitoring (Golparvar-Fard et al. 2011), application of AR Training System (ARTS) for
training the operation of heavy construction equipment (X. Wang et al., 2007).
Augmented reality technology has many applications in the AEC industry. We group AR application areas in the
AEC industry as follows: (1) visualization or simulation, (2) information access or evaluation, (3) communication or
collaboration, and (4) training or safety.
Augmented reality technology, which typically layers virtual information on a real scene, utilizes different hardware
(personal computers (PC), laptops, head mounted displays (HMD), GPS, data gloves, smartboards, etc.) and
software (AutoCAD, Photoshop, AC3D, 3D Studio, building information model (BIM), etc.). From a user
experience perspective AR environments are: (1) immersive or (2) desktop-based, i.e., non-immersive. Devices such
as HMD and data-gloves create immersive AR systems, in which users feel immersed in a virtual environment just
as they usually feel in a real environment. Due to the improving performance of handheld devices and recent
solutions to technical difficulties such as tracking, there is an opportunity for augmented reality systems to become
portable, as well there is a growing interest in the use of mobile AR applications. From a technology device
perspective augmented reality devices are classified as (1) mobile or (2) stationary.
5. CATEGORIZATION OF THE ARTICLES
This section discusses the classification of the current state of AR technology literature in the AEC industry. The
228
articles are classified based on their principal focus and each article is counted once (except for section 5.1 where
articles may be counted more than once).
5.1 Target Audience
In this section instead of giving the number of articles with a principal focus on a subcategory, we report the
percentage of articles including reference to that subcategory, since in this section each article may refer to more
than one subcategory. Figure 5 presents the percentage of articles by target audience. The results indicate that 30%
of articles include reference to workers as the target audience (as defined in section 4), 25% refer to design teams,
24% refer to project managers, 22% refer to site developers, 9% refer to students, and 6% refer to stakeholders.

Figure 5: Percentage of articles by target audiences
5.2 Project Phase
The number of the articles by project phase is depicted in Figure 6. Twenty articles (31%) have a principal focus on
the construction phase and 11 articles (17%) have a principal focus on the design phase. There are no articles with a
principal focus solely on the procurement phase. Nine (4+1+4, 14%) articles focus on two project phases, and 5
articles (8%) focus on all 5 project phases.

Figure 6: Number of articles by project phase
229
Figure 7 illustrates the number of articles for each project phase by year of publication. In this diagram articles with
a focus on multiple phases are excluded (reducing the total to 42 articles). The highest number of articles in a single
year is for the construction phase in the year 2011. The focus on the design phase of a project started with 1 selected
article in 1999 and reached its highest number (4 articles) in the year 2008. Figure 6 and Figure 7 show that the
highest number of articles occur in the design and construction phases of a project for AR technologies and
applications.

Figure 7: Number of articles by project phase and year (articles spanning multiple phases are excluded)
5.3 Stage of Technology Maturity
Figure 8 illustrates the number of articles within each stage-of-technology-maturity subcategory. As shown, 29
articles (45%) have a principal focus on augmented reality application demonstration in AEC industry, while 14
articles (21%) and 10 articles (15%) have a principal focus on AR proposed system development and technical
issues respectively. Eleven articles (17%) have a focus on multiple areas (i.e., more than one of the previous stages);
these multiple areas are typically a combination of application demonstration and proposed system development.
230

Figure 8: Number of articles by stage of technology maturity
5.4 Application Area
Figure 9 presents application areas for AR technologies in the AEC industry. As shown, 19 articles (29%) have a
principal focus on visualization and simulation as an application area for AR technology. Information access and
evaluation, communication and collaboration, and training and safety subcategories have 15 articles (23%), 10
articles (15%) and 8 (12%) articles respectively. Six articles focus on multiple application areas while these
subcategories were not applicable for 7 articles.

Figure 9: Application areas for AR technology
231
5.5 Hardware
Within the hardware category, 16 articles had a principal focus on immersive AR technologies, 34 articles (52%)
had a principal focus on desktop-based AR technologies, while 15 articles were not applicable. Figure 10 presents
the number of articles with immersive and desktop-based technologies as a principal focus by year.

Figure 10: Immersive and desktop-based AR technology by year
Figure 11 presents the number of articles within the hardware category that had a principal focus on stationary and
on mobile AR technologies in the AEC industry. The diagram implies an increasing trend in mobile AR
technologies in AEC industry. Of the selected articles, one was published in 2000 that discussed mobile AR
technology, while 7 articles (10%) focus on mobile AR technologies in the year 2011; 22 articles were not
applicable to this category.
Figure 11: Number of articles for mobile and stationary AR technology by year
232
6. CONCLUSION
A structured methodology was used to
identify 65 articles on the topic of augmented
reality from 3 prominent AEC industry
journals. The first article was published in
1999; a significant increase in the number of
articles on this topic occurred during 2008. In
addition to statistics on the counts of articles
by year and the first authors country of
residence, five interpreted categories were
developed for classification of these articles:
target audience, project phase, stage of
technology maturity, application area, and
hardware (including both user experience and
device). Articles were classified and insight
on the current state of AR technology in AEC
industry research was gained. The following
results are concluded for the categories shown
in Figure 12.
Journals: AIC has the highest overall
number of articles among the journals,
while ITcon has the second highest
number of articles. The maximum
number of AR technology articles
published in these 3 journals in a single
year, occurred in both 2008 and 2010.
First authors: USA was the dominant
residence of the first authors.
Target audience: The most frequent focus
is the workers (e.g., machine operators
and technicians).
Project phase: The most frequent focus is
the construction phase with the design
phase being next with approximately half
as many articles; this is most obvious in
recent years. Twenty-two articles cover
two phases (e.g., initiation/design) or all
phases.
Technology maturity: The majority of
articles focus on AR application
demonstration in the AEC industry, with
approximately half as many focusing on
proposed system developments.
Application areas: Over half of the
articles had a principal focus on
visualization/simulation or information access/evaluation.
Hardware system: A spike in the number of articles focusing on mobile technologies occurred in 2010. From
the perspective of immersive and desktop-based technologies most of the articles discussed desktop-based
technologies.
Figure12:LiteratureReviewSummary
233

In summary, the AR literature has increasingly focused on the demonstration of visualization or simulation
applications for use during the construction phase that address issues faced by on-site workers. With respect to other
target audiences, design team, project managers, and building systems engineers (as opposed to workers)
were strong contenders.
We predict expansion of AR technologies from a principal role in the construction and design phases to other phases
(especially the procurement and maintenance phases) as the ability to compare virtual models with previous virtual
models (and realities with previous realities) rather than the narrow focus of comparing a current construction phase
reality with a final design phase model. We predict continued growth in the use of mobile devices to display (and
capture) models and realities.
We also speculate that the cost of immersive hardware is, and will continue to be, an impediment to its widespread
use.
7. REFERENCES
Aziz F., Rankin J. H., and Waugh L. M. (2011). Information management research in the construction industry,
CSCE 3rd International/9th Construction Specialty Conference, Ottawa, Ontario.
Behzadan A. H. and Kamat V. R. (2011). Integrated information modeling and visual simulation of engineering
operations using dynamic augmented reality scene graphs. Journal of Information Technology in Construction,
ITcon, Vol. 16, 259-278.
Dawood N. (2009). VR-Roadmap: a vision for 2030 in the built environment, Journal of Information Technology in
Construction, ITcon, Vol. 14, 489-506.
Dunston P. S. and Wang X. (2005). Mixed reality-based visualization interfaces for architecture, engineering, and
construction industry. ASCE Journal of Construction Engineering and Management, ASCE, Vol. 131, No. 12, 1301-
1309.
Golparvar-Fard M., Pea-Mora F., and Savarese S. (2009). D4AR A 4-dimensional augmented reality model for
automating construction progress monitoring, data collection, processing and communication, Journal of
Information Technology in Construction, ITcon, Vol. 14, 129-153.
Golparvar-Fard M., Pea-Mora F., and Savarese S. (2011). Integrated sequential as-built and as-planned
representation with D4AR tools in support of decision-making tasks in the AEC/FM industry, ASCE Journal of
Construction Engineering and Management, ASCE, Vol. 137, No. 12, 1099-1116.
Kamat V. R., Martinez J. C., Fischer M., Golparvar-Fard M., Pea-Mora F., and Savarese S. (2011). Research in
visualization techniques for field construction, ASCE Journal of Construction Engineering and Management, ASCE,
Vol. 137, No. 10, 853-862.
Muramoto K., Jemtrud M., Kumar S., Balakrishnan B., and Wiley D. (2008). Emerging technology in tele-
collaborative design studio between Pennsylvania State University and Carleton University, Journal of Information
Technology in Construction, ITcon, Vol. 13, 660-673.
Shin D. H. and Dunston P. S. (2010). Technology development needs for advancing augmented reality-based
inspection, Journal of Automation in Construction, Elsevier, Vol. 19, 69-182.
Wang X. and Dunston P. S. (2007). Design, strategies, and issues toward an augmented reality-based construction
training platform, Journal of Information Technology in Construction, ITcon, Vol. 12, 363-380.
234

AUGMENTED REALITY MARKERS AS SPATIAL INDICES FOR
INDOOR MOBILE AECFM APPLICATIONS
Chen Feng & Vineet R. Kamat
University of Michigan, Ann Arbor, USA
ABSTRACT: This paper presents a new methodology for utilizing Augmented Reality (AR) fiducial markers on
mobile devices (smart phone/tablet) for indoor Architecture, Engineering, Construction, and Facilities
Management (AECFM) applications such as navigation and inspection. On one hand, previous efforts on such
applications focused mainly on exploring traditional non-visual-sensor-based methods to track user's position
continuously, ignoring the advantage that in most of the built environment, human inspectors can easily navigate
themselves to destinations given that they can make correct decisions at a set of discrete critical spatial locations
(corner of hallway, stairs etc.). On the other hand, traditionally fiducial markers are extensively used to recover
user's pose and thus serve as table-top AR display surface, benefiting from its cost-efficiency and high flexibility. A
different view that combines these two observations is to attach these markers at critical locations whose global
positions and orientations are known in advance, treating markers as spatial indices which help the automatic
identification of key locations. Upon recognizing the marker, as well as estimating the relative pose between the
user and marker, user's pose in the global coordinate frame can be calculated. Then decisions can be
automatically made and users are guided to their destinations by 3D graphical instructions. An example
application for complex indoor environment way-finding is built on Android platform and tested which
demonstrates the efficiency of the proposed method.
KEYWORDS: Context-aware computing; Fiducial Marker; AprilTag; Indoor Navigation; Registration
1. INTRODUCTION
Context-aware information delivery has been recognized as a critical component in many Architecture,
Engineering and Construction, and Facilities Management (AECFM) applications (Anumba & Aziz, 2006;
Khoury & Kamat, 2009; Andoh, et al., 2012). Identification of a users location is one of the most fundamental and
extensively studied problems in this area. Solutions that achieve good balance between cost and accuracy could
lead to meaningful productivity improvement in applications such as facility management, construction inspection,
and indoor way finding.
Previously, researchers attention has focused more on traditional non-visual-sensor-based methods, such as
Global Positioning System (GPS), Inertial Measurement Unit (IMU) (Akula, et al., 2011), Radio Frequency
Identification (RFID) (Sanpechuda & Kovavisaruch, 2008; Andoh, et al., 2012), Wireless Local Area Network
(WLAN) (Aziz, et al., 2005), and Ultra-Wide Band (UWB) (Teizer, et al., 2008). Recently computer vision and
robotics communities have proposed methods such as Simultaneous Localization and Mapping (SLAM) (Thrun,
2008) and Visual Registration (Olson, 2011), utilizing visual sensors such as camera or lidar. Although most of
these methods can track a users 3D position continuously, they also have their own disadvantages respectively
such as indoor unavailability, tracking drift, large infrastructure/special hardware requirement, cost inefficiency,
and high computational power requirement.
However, in many of those methods AEC domain applications, the fact that the end-user is human being makes
human intelligence not only a non-negligible but rather important ingredient to achieve a good cost-accuracy
balance (Akula, et al., 2011). This is different from robot navigation since robots need to know their own location
at any moment for further decision making, while a human already possesses the ability of maintaining its own
location within a certain range. For example, given a typical hallway with limited turnings, a human can easily
navigate from one end point to the other; while a robot may need help of continuous SLAM. Only when the
hallway contains lots of turnings and exit stairs, navigation becomes useful for humans, which is also true for
outdoor road navigation. This observation in fact suggests that continuous tracking of users position might not be
necessary in many AEC applications; for human inspectors, to automatically extract discrete-spatial-distributed
information could be sufficient to accomplish their jobs faster and better.
Naturally, marker based visual registration method for Augmented Reality (AR) becomes a good candidate for
such discrete localization (Olson, 2011; Feng & Kamat, 2012). Registration problem means how to find the
relative position and orientation between two coordinate systems in AR (Azuma, 1997). Thus, beside
235

correspondences between different AR markers and discrete locations, these types of methods also provide
estimation of not only position but also orientation. This additional dimension of information could help to obtain
better visualization of extracted information and hence extend the application domain to where traditional sensor
based methods such as RFID could not reach.
In the meantime, mobile devices are becoming more and more popular recently. Especially for smart phones and
tablets, camera, CPU and even GPU are standard configuration, which unsurprisingly makes them ideal equipment
for ubiquitous visual computing. Combining those observations, the authors were motivated to explore a new
method utilizing AR markers as spatial indices to create links between physical locations and virtual information
stored in databases, which runs on mobile devices. Our contribution in this paper is mainly a general computing
framework as well as an indoor way-finding application based on such a framework. The fact that more and more
people have smart devices makes it easier for people to access information using this method than all previous
methods such as RFID and UWB.
The remainder of this paper is structured as follows. In section 2, previous work on both traditional and visual
sensor based methods is reviewed; in section 3, the general computing framework of AR marker as spatial index is
explained in detail; in section 4 an indoor way finding application based on that framework is developed; section 5
describes a set of experiments conducted to prove the new indoor way finding methods efficiency; and finally
section 6 presents the conclusions of the paper.
2. PREVIOUS WORK
As mentioned before, researchers have extensively studied two types of localization techniques for context aware
computing, i.e. traditional non-visual-sensor-based methods, as well as newly emerging visual-sensor-based
methods.
Among the first type, GPS is mainly used for outdoor scenarios; IMUs tracking drift issue requires error
correction either by human (Akula, et al., 2011) or by combining with other methods such as AR marker (Feng &
Kamat, 2012); RFID-based methods usually depend on large infrastructure (i.e. enough RFID tags must be
available) and also requires special tag reader which is not easily accessible by common people (Sanpechuda &
Kovavisaruch, 2008; Andoh, et al., 2012); WLAN-based methods also require large number of footprints (Aziz,
et al., 2005); UWB-based methods generally cost too much (Teizer, et al., 2008; Khoury & Kamat, 2009).
Besides, none of these methods can easily provide instantaneous orientation information (even though there is
angular sensors such as gyroscope, electrical compass or accelerometer, they themselves come with problems
such as noise and sensitivity to the environment), which makes them not optimal for further 3D visualization
purpose.
On the contrary, the second type of methods directly output orientation along with position information. By
analyzing images captured from visual-sensors such as a simple webcam, the visual registration methods can
recover that sensors pose (position and orientation). Based on their different assumptions on the environment
(i.e. the surrounding world where visual registration is going to be performed), these algorithms can be classified
into two groups (Lepetit & Fua, 2005; Feng & Kamat, 2012): known environment vs. unknown environment
(see Fig. 1, our proposed indoor way-finding application contains a module that can be classified as fiducial
marker-based method).
The first group of visual registration algorithms is less computation-consuming and easier to design since the
only unknown is the users pose. Because they have been well studied, and many related powerful algorithms
have been proposed in the last two decades, it is more realistic to apply them for solving real-world engineering
issues. Within this class of methods, they can be further categorized into two groups: planar environment vs.
non-planar environment. The first group is again easier to design because of the simple assumption made
regarding the environmenta planar structure with known visual features. The second group is more often
applied in a controlled environment with limited space, such as a small manufacturing workspace.
In the indoor way-finding applications, the authors chose to take advantage of plane-based methods since planar
structures are abundant in buildings, construction sites, and other built environments where engineering
operations are conducted, which makes this type of method very convenient to apply. In addition, a planar
structure can simply be an image printed out on a piece of paper and attached to a wall/floor of a corridor, with
negligible cost.
236

Plane-based methods can be further classified based on different visual features they adopt: fiducial marker vs.
natural marker. A fiducial marker is composed of a set of visual features that are easy to extract and provide
reliable, easy to exploit measurements for the pose estimation (Lepetit & Fua, 2005). Usually those features are
a set of black and white patterns forming simple geometry by circles, straight lines, or sharp corners and edges.
Well known fiducial markers include ARToolKit (Kato & Billinghurst, 1999) and the newly proposed AprilTag
(Olson, 2011).
Distinct from a fiducial marker, a natural marker does not require special predefined visual features. Instead, it
treats any visual features in the same way. In this sense, almost any common image, ranging from a natural view
to a company logo, can immediately be used as a natural marker. Even though natural marker methods have
advantages of more accurate and stable as well as larger tolerance to partial occlusion, its relatively higher
computational requirement than fiducial marker methods limits its sphere of application to high-end smart
mobile devices. Taking all these factors into consideration, the authors chose to apply fiducial markers in the
indoor way finding application so as to lower the requirement of its targeted devices.
3. METHODOLOGY
In this section, a general computing framework is described, representing the authors proposed idea to use AR
marker as spatial index which links between physical location and virtual information related to that location.
Since the framework is generic, the descriptions are not fixed to any specific algorithms. As long as an algorithm
can meet the requirements of each module to be described below, it can fit into the framework. This makes it
possible to adapt new algorithms for existing system with little difficulties.
The system overview is shown in Fig. 2. The system operates based on the following procedure. Firstly, an image
potentially containing an AR marker is captured by the camera on the mobile device. This image is then sent to the
marker recognition module. If the image contains an AR marker and it is recognized by the marker recognition
module, then the ID of the AR marker is sent to the database as a key value to search for attached information.
Simultaneously, the estimated pose of the mobile device relative to the marker is sent to the 3D visualization
module. This visualization module then takes the estimated pose to render the information sent back from the
database on the screen of the mobile device for further decision making by its end-user. In order to achieve such
functionalities, each module needs to conform to certain requirements, which are explained in more detail below.
3.1 Physical Space
Generally speaking, the physical space in this framework, as the targeted environment of such a system, could be
any indoor scene. For example, a complex shopping mall, an international airport, a subway station, a big
warehouse or a building construction site. In fact, in some application scenarios, even outdoor scene such as a
public park could be equipped with such a system.
Visual Registration Methods
Unknown
Environment
(SLAM)
Known Environment
Non-planar
Environment
(CAD Model Based)
Fiducial
Marker-Based
Methods
Natural
Marker-Based
Methods
Planar Environment

Fig. 1: A brief taxonomy of visual registration methods based on assumptions on environment.
237

In order to make the original physical space compliant with the system, AR markers must be attached to a discrete
set of locations which are critical to the application. The AR markers should have their own IDs which serve as the
spatial indices of that set of locations. The marker recognition algorithm corresponding to those markers should
contain two functions: 1) the ability to extract the ID of a marker presents in an image; 2) the ability to recover the
pose of that image, i.e. the mobile device, relative to the marker. Some possible choices of marker could be
AprilTag (Olson, 2011) or KEG Tag (Feng & Kamat, 2012).
Even though the requirement of one-to-one mapping between AR markers and physical locations makes our
approach slightly infrastructure dependent, the facts that AR marker is flexible and easy to use/install, cost
efficient, and more importantly capable of providing orientation information make it more ideal than RFID tag.
3.2 Mobile Device
A mobile device is the core of such a system. Its camera serves as the main input where information enters into the
system. While cameras are part of the standard configuration of common mobile devices such as mobile phone or
tablet, a rear facing camera setup is more desirable than a front facing one, since the direction of sight of the
camera could be aligned with users viewing direction more naturally so that user could see the 3D visualization
more conveniently.
Also, the 3D visualization module naturally locates within the mobile devices. It visualizes the information sent
from the database in the means of augmented reality by pose estimated from the marker recognition module. Here
an implementation choice must be made for any application conforming to this framework: whether the marker
recognition module should locate locally within mobile device or remotely in the server/cloud. This usually
depends on the invoking frequency of this computation framework. For some applications, the whole procedure
needs to be performed for every frame of the live video stream captured by mobile devices camera. In this
situation, a local marker recognition module is more reasonable, otherwise the captured image needs to be
transferred to the server/cloud side very frequently, which will increase the data volume transferred through the
mobile network resulting in higher cost and longer latency. For some other applications, the invoking frequency is
relatively low, say around 1 time per minute. Then turning the marker registration module into an online service
could be a good choice.
3.3 Database
A database is the foundation of such a system. With physical locations mapped into IDs through AR markers, the
database module is very flexible and can be implemented from almost all kinds of databases available, such as
MySQL, SQLite or even a self-defined text file. Similarly, the storage location of this database is also an
Cloud/Local
Database(BIM, etc.)
Physical Space
1. Captured image
containing a marker
2. Recognized
location ID
Mobile Device
Marker
Recognition
3D
Visualization
4. Extracted information
attached to the ID
3. Estimated pose
relative to the marker
Fig. 2: System overview of AR marker as spatial index.
238

implementation choice, depends on the size of the whole database as well as the size of information to be sent back
to the 3D visualization module. For relatively smaller-size database, say less than 1 Mb, downloading the whole
database from cloud/server side to mobile device once and for all is a reasonable choice. Otherwise, server-side
storage of the whole database could be more efficient, as long as the size of the information to be sent back in step
4 of Fig. 2 is relatively small.
4. WAYFINDING APPLICATION
As mentioned previously, the above methodology is best suitable for the decision making process which involves
a set of discrete spatial locations. Indoor way finding is one good example application for which the above
methodology could be very helpful.
Consider the following first-person scenario: you are a new student on the first day of class. You were given a
room number which is the first class you are going to take. But the building is large and unfamiliar, and even
though you entered it for a few times previously, you still find it difficult to find the correct direction. You try to
look for an indoor floor plan view of the building but it takes you more than 5 minutes and you fail to understand
the map. Since it is already time for class, very few people are available for inquiring the directions. When you
finally find someone for help, the instructions you get are go along this hallway and turn left at room 1318, then
walk to the end of the hallway and turn right. Then pass the exit door and go upstairs to second floor and turn
right again. After another two exit doors and a right turn is your destination. After you find the room probably
half of the class has passed. Similar situations happen frequently inside many complex buildings for a person
who is unfamiliar with the environment or doesnt have a good sense of direction, and even for someone who
works/studies inside the building every day since s/he has never been to the room before.
In this case one can see that the most useful piece of information, i.e. the instruction which greatly influences the
decision making process, contains such a set of discrete locations (e.g. room 1318, exit door, corner, stair, etc.) as
described before. It is natural to consider that an AR marker should be attached to each of such critical spatial
locations to map these locations into spatial indices. In the database module, the position and orientation of each
marker under the buildings global coordinate system is stored, as well as positions of all the rooms of the
building. Thus an oriented graph is generated from this information, with nodes as rooms and markers, edges as
instructions of how to move from starting node to ending node, edge weights as lengths of physical paths (see
Fig. 3, shaded large circle means marker node, small circle means room node). There are two situations for
adding edge to the graph:
1. When there is a physical path between a room node A and a marker node B without passing other
markers, an edge from B to A can be added to the graph; No edge from room node to marker node
should be added;
2. When there is a physical path between a marker node C and a marker node D without passing other
markers, an edge from C to D, as well as an edge from D to C, should be added to the graph.
0
1 3
2 4
5
Turn right
Turn left
Move
backwards
Marker 2
Marker 4 Marker 0
Marker 3
Destination
Shortest Path
Graph Edge
Marker Node
Room Node Marker Pose

Fig. 3: An example of oriented graph for indoor way finding application.
239

The instruction stored within each edge, say edge starting from node A to node B, is the method of how to move
from A to B assuming user is currently facing towards the marker. For example, as the orientation of marker 2
shown as an arrow in the detailed floor plan view of marker 2 in Fig. 3, instruction stored in edge from marker 2
to marker 0 could be turn left, since when the user gets this instruction s/he should be looking at this marker 2.
After the graph is constructed, the user is asked to select a node as destination. Then the Dijkstra algorithm
(Dijkstra, 1959) is performed on the graph to compute the shortest path from all other marker nodes to this node.
Thus whenever user comes to a marker, the marker recognition module will extract the corresponding spatial
index and then find the marker node in the graph. Then the shortest-path edge starting from this node is retrieved
and the moving instruction is displayed graphically in the 3D visualization module (see Fig. 4). When user is
standing at the marker node which has a direct edge pointing to the destination, special logo is shown (the
animated eye shown at the top-right corner of the left image in Fig. 4) to remind user to slow down and look
carefully around the surroundings since s/he is very close to the destination room.
The authors implemented the described indoor way finding application on Android platform. Here the two
design choices are made as follows. Marker recognition module is located in the mobile device, since real-time
performance is desired; database is stored in server side as a simple self-defined text file describing the whole
graph of a building (termed as a map file). This application assumes that user is aware of which building s/he is
in. When user selects the building, the corresponding map file will be downloaded to the mobile device. The
marker recognition module in this application adopts the AprilTag (Olson, 2011). As mentioned in section 2, the
reason to adopt this specific algorithm is that AprilTag is proved to be superior than previous proposed fiducial
markers in the sense of speed, accuracy and tolerance to critical view conditions (long distance, large viewing
angle, partial occlusion etc.).
The authors have made this application, named as Mobile AR Navigator (MARvigator), publicly available online
at http://www.umich.edu/~cforrest/upload/marvigator/. Although similar methods or systems have been proposed
before (Kalkusch, et al., 2002; Wagner & Schmalstieg, 2003; Augmented Reality & Assistive Technology
Laboratory of NUS, 2011), MARvigator is superior since it takes advantage of the state-of-the-art fiducial
marker system and is implemented conforming to the general framework described previously, resulting in faster
real-time performance (about 15 frame-per-second as shown in Fig. 4), high ease of use and flexibility (could be
easily setup at any complicated indoor environment or some outdoor environment).
In order to show MARvigators efficiency in helping people find destinations, a set of experiments is conducted
on the first floor of a building on the authors university, which is known on the authors campus for its complex
network of corridors and hallways. Seventeen apriltags are placed on critical locations among the region whose
positions and orientations are shown in Fig. 5. Six target positions are selected (room 1351, 1318, 1069, 1188,
1040 and 1504) which need to be sequentially found by each of the 10 experiment volunteers, among which 6
volunteers work/study within a part of this region and the other 4 have never been to this region or are not
familiar with it.
Each volunteer starts from nearby marker 0. They are instructed on the usage of MARvigator for one minute
before the experiment. The time for each volunteers reaching target room is recorded with resolution of half a
minute. In order to reduce to the minimum the influence of other factors such as variations in walking speed and
sense of direction, volunteers are asked to switch between using and not using MARvigator during the way
Fig. 4: Two screenshots showing graphical instruction of how to move to the destination (Best viewed in color).
240

finding. For example, if a volunteer start to find the first target room 1351 with the help of MARvigator, then
s/he should not use MARvigator but any other common means of way finding (looking at the map on some part
of the building or asking other people for help) to discover the second target room 1318, and vice versa. In the
experiment, 5 people start with MARvigator and the other 5 people start without.
The experiment results are shown in Table 1. The shaded grid shows the time to find the target room assisted
with MARvigator. From these results it is clear that MARvigator does help people to find destinations faster.
Notice that for the first two targets, using MARvigator took slightly longer time. This could be explained by the
fact that the user needs time to learn how to use the application. Also the first two targets are very close to the
start position and can be found easily. However, by comparing the average time of finding later targets,
MARvigators efficiency is better highlighted.
Table 1: Experimental results for the 10 volunteers (shaded cells shows the time with MARvigator assistance).
Time(min)
Volunteer who starts with
MARvigator
Volunteer who starts w/o MARvigator
Target A B C D E F G H I J Average
1351 0.5 1 1 1 0.5 0.5 0.5 0.5 0.5 0.5 0.80 0.50
1318 0.5 0.5 0.5 1 0.5 0.5 1 1 1 0.5 0.60 0.80
1069 0.5 1 1 1 1 0.5 2 5 2 2 0.90 2.30
1188 2 2 7 4 2 1.5 2 4 3 2 3.40 2.50
1040 1 2 1 1 2 2 0.5 1 4 4 1.40 2.30
1504 3 3 2 2 2 2 1 2 3 2 2.40 2.00
6. CONCLUSION
In conclusion, the proposed general computing framework of using AR marker as spatial index to link physical
locations with virtual information related to that location offers a new perspective of utilizing AR markers. The
indoor way finding application MARvigator conforming to this framework is proved by experiments to be very
1
2
3
0
4
5
6
7
8 9 10 12
11 14
13
15
16
1504
1351
1318
1069
1188
1040
Target Room
Marker
Starting Position

Fig. 5: Indoor way finding experiment setup.
241

efficient and convenient. Future research directions could be adopting more advanced computer vision and
machine learning algorithms to replace the marker recognition module with landmark recognition module which
further improves the infrastructure dependency of the framework.
7. REFERENCES
Akula, M. et al., 2011. Integration of Infrastructure Based Positioning Systems and Inertial Navigation for
Ubiquitous Context-Aware Engineering Applications. Advanced Engineering Informatics, 25(4), ICCCBE 2010
Special Issue on Computing in Civil and Building Engineering, pp. 640-655.
Andoh, A. R., Su, X. & Cai, H., 2012. A Boundary Condition-based Algorithm for Locating Construction Site
Objects Using RFID and GPS. Proceedings of 2012 Construction Research Congress, West Lafayette, IN.
Anumba, C. & Aziz, Z., 2006. Case Studies of Intelligent Context-Aware Services Delivery in AEC/FM. Lecture
Notes in Computer Science, Vol 4200, pp. 23-31.
Augmented Reality & Assistive Technology Laboratory of NUS, 2011. AR Navi NUS. [Online]
Available at: http://serve.me.nus.edu.sg/ongsk/ARLab/arnavi.htm [Accessed 12 06 2012].
Aziz, Z. et al., 2005. Context Aware Information Delivery for On-Site Construction Operations. Proceedings of the
22nd CIB-W78 Conference on Information Technology in Construction, Dresden, Germany, p. 321327.
Azuma, R., 1997. A survey of augmented reality. Presence:Teleoperators and Virtual Environments, pp. 355-385.
Dijkstra, E., 1959. A note on two problems in connexion with graphs. Numerische mathematik, Vol 1, pp. 269-271.
Feng, C. & Kamat, V. R., 2012. A plane tracker for AEC-automation applications. Proceedings of 2012
International Symposium on Robotics and Automation in Construction, Eindhoven, NL.
Kalkusch, M. et al., 2002. Structured visual markers for indoor pathfinding. Proceedings of the First IEEE
International Workshop On Augmented Reality Toolkit, Darmstadt, Germany.
Kato, H. & Billinghurst, M., 1999. Marker tracking and hmd calibration for a video-based augmented reality
conferencing system. Proceedings of the 2nd International Workshop on Augmented Reality, pp. 85-94.
Khoury, H. M. & Kamat, V. R., 2009. Evaluation of position tracking technologies for user localization in indoor
construction environments. Automation in Construction, Volume 18, pp. 444-457.
Lepetit, V. & Fua, P., 2005. Monocular Model-Based 3D Tracking of Rigid Objects. Foundations and Trends in
Computer Graphics and Vision, pp. 1-89.
Olson, E., 2011. AprilTag: A robust and flexible visual fiducial system. Proceedings of IEEE International
Conference on Robotics and Automation, pp. 3400-3407.
Sanpechuda, T. & Kovavisaruch, L., 2008. A review of RFID localization: Applications and techniques.
Proceedings of 5th International Conference on Electrical Engineering/Electronics, Computer,
Telecommunications and Information Technology, Krabi, pp. 769-772.
Teizer, J., Venugopal, M. & Walia, A., 2008. Ultrawideband for automated real-time three-dimensional location
sensing for workforce, equipment, and material positioning and tracking. Transportation Research Record:
Journal of the Transportation Research Board, Vol 2081, pp. 56-64.
Thrun, S., 2008. Simultaneous localization and mapping. Robotics and cognitive approaches to spatial mapping,
pp. 13-41.
Wagner, D. & Schmalstieg, D., 2003. First Steps Towards Handheld Augmented Reality. Proceedings of the 7th
International Symposium on Wearable Computers (ISWC'2003), White Plains, New York, USA..
242

REAL-TIME MICRO PLANNER FOR CRANE OPERATIONS
Wei Han Hung & Shih Chung Kang
Computer-Aided Engineering Group, Department of Civil Engineering, National Taiwan University, Taiwan.
ABSTRACT: This research proposed a real-time micro planner for crane operations, which is can allow engineers
to perform detailed erection simulation and automatic erection path finding in micro construction planning. Four
major and critical components have been developed and implemented in this research, which are (1) the
configurable crane models; (2) the automatic collision boundaries generator; (3) the erection path planner; (4)
and the space-emphasized human-machine interface. The configurable crane modeling method utilizes multibody
dynamics to modularized cranes into three modules, which can be reused and efficiently reassembled to versatile
types of cranes for real-time simulations; the collision boundaries generator converts a site model into
appropriate collision boundaries by a iteratively clustering method developed based on K-means clustering
method according to the objects dimension and position in the construction site; the erection path planner
improves the C-space construction by the four strategies proposed in this research by majorly considering the
lifting capacities of cranes and collision detection strategies. It can allow engineers to perform erection path
planning efficiently; the space-emphasized human-machine interface integrated stereoscopic vision and
kinesthetic vision to provide an intuitive and realistic viewing experiences for engineers and crane operators. Via
the proposed system, detailed erection simulations therefore can be easily setup to help finding potential problems
and re-generate alternate erection plans during or before the erection. The developed visualization methods are
also expected to improve the comprehensibility of spatial relationship and information of erection environment.
KEYWORDS: crane erection, path planning, real-time simulation, virtual construction.
1. INTRODUCTION
The use of cranes is becoming more versatile and yet challenging in modern construction projects (Kang and
Miranda, 2006). The rigging objects are now larger, heavier, and come in more varieties and yet the erection times
and cost need to be reduced. Most of the work-related items for cranes are in the critical path in a project. Any
delay in these tasks may often result in a delay of the overall project. Furthermore, some erection tasks are very
risky and difficult that the process time can be more than half or one days. Such erection tasks hence require a high
accuracy and efficiency plan before the tasks begin.
During the last decade, researchers have developed computer-aided tools and visualization methods to improve the
accuracy, efficiency, and safety of crane erection activities from different aspects (Rppel and Schatz, 2011; Yang
et al., 2010; Zhang and Hammad, 2012). These successful applications have shown the advantages and benefits
brought from these novel technologies and methods applied in construction. Detailed simulation and visualization
of crane erections and automatic erection path planning are two growing research trends (Chi et al., 2007;
Garca-Fernndez et al., 2008; Kang and Miranda, 2009; Chang et al., 2012; Yoo et al., 2012; Zhang and Hammad,
2012). By simulating detailed motions and behaviors of crane operations, engineers are able to find potential
problems and conflicts of erection tasks; the automatic path planning methods can find feasible and collided-free
path efficiently for critical erection tasks. These researches are trying to reduce unexpected accidents and
additional cost caused by the empirical misjudgment of crane erections.
However, a construction site is dynamical that unexpected situations are happened during the erection. These
changes can sometimes largely influence the erection plans including the located position of cranes, the required
loading capacity, and the efficiency of planed erection task (Xu and Li, 2012). A micro plan is the plan that made
dynamically before or during the erection, which are made according to the changes of the construction conditions.
Traditionally in micro construction planning, engineers try to figure out feasible logistics and erection plan by
using 2D paper-based tools. They have to image the scenario of the erection process, which largely relies on the
personal experiences of engineers. To handle the daily and unexpected changes during erection, micro plans must
be made efficiently and effectively on-site in order to minimize the additional cost and delay of the project.
However, the lack of an effective tool for authoring and presenting detailed erection process and activities is the
critical problem in current micro construction planning for crane erections.
This research aimed at developing a computer system to present and describe crane erection activities for micro
construction planning. The proposed system is expected to allow engineers efficiently build the virtual
243

environment for simulating detailed erection process on-site. The system can also perform erection path planning
via the defined erection scenario in the virtual environment, therefore engineers can use it to find feasible erection
path or evaluate the planned erection path. Project managers can monitor the process of the erection plan generated
by the real-time crane erection simulation. Crane operators can also use it to rehearsal and practice critical and
dangerous erection task before the actual task starts.
2. CHALLENGES OF MICRO PLANNER
To support real-time erection simulation and erection path planning in micro construction planning stage, there are
several critical challenges need to be overcame. The major challenge is the insufficient time to prepare the required
environment and information for erection activities. This can affect many aspects the development of an effective
and useful planner for crane operations:
Construction of the crane model for real-time simulation: In order to provide real-time simulation for crane
erection activity in the micro planning stage, a virtual crane need to be modeled and constructed in a short time and
support real-time simulation, which can also help crane selection by different capacities and properties. The lifting
objects are different to different erection task. Their lifting styles are also different due to the various properties and
the features of the lifting object, and most of them need additional lifting devices to balance the loads. These
factors need to be considered and simulated in the model. The used crane may be also changed to different types.
Generation of accurate and efficient collision boundaries: Collision detection is an important and fundamental
component to the real-time simulation of erection activity. It checks if construction machines (e.g. cranes and
excavators) are collided with environment objects (e.g. beams and floors) or lifting objects. Similar to real
construction site, the 3D model of the construction site is different at different schedule. Furthermore, there are
usually thousands or more elements in a 3D model of construction site. It makes the generation process of the
collision boundaries difficult and usually requires lots of manual works to modify and simplify the original 3D
models. The efficiency of the generation process is the major problem since engineers need to make decision
quickly due to the changes of the construction site in micro construction planning. In addition, the generated
collision boundaries must retain certain accuracy and also support real-time collision detection for crane erection
simulation.
Preparation work of erection path planning: In order to help finding feasible and reliable erection paths, an
erection path planner utilized computational methods needs to be developed. However, finding collided-free
erection path by existing and common methods usually required certain preparation works to construct required
information for path finding, which is C-space construction (Lozano-Perez, 1983). For a more accurate and
reliable erection path finding, it usually needs to take hours to complete the process, which is difficult to be applied
in micro construction planning.
Insufficient spatial information delivered by traditional visualization methods: In order to find potential problems
and safety issue of erection plans, engineers and crane operators need to clearly comprehend the spatial
relationship between cranes, the lifting object, and the environment objects in the construction site (Al-Hussein et
al., 2001). Traditional 3D visualization can deliver realistic views of erection, however there is still a gap for
humans to percept accurate and correct space information by simply projecting 3D screen on a 2D screen
(Jourdam et al., 2004; Birkfellner et al., 2003). Traditional method is lack of the two important and intuitive
factors that we human usually do in spatial information perception, which are the motion parallax and the
binocular vision (Wheatstone, 1838; Cutting, 1986). The major problems to deliver these behaviors are the various
sizes of screens and the real-time capturing of the viewers position. Traditional method uses fixed angle of the
field-of-view to render the virtual world, which will generate distorted images to viewers when the size of the
actual screen and the position of the viewer are changed.
This research addressed the problems and developed critical methods of a real-time micro planner for crane
operations. The methods can provide a systematic and modularized crane modeling method, which can reduce the
development time of the crane model and also make the developed crane model reusable and extendable. The
methods can also convert a 3D site model retrieved from a 4D model application or a BIM model into appropriate
collision boundaries for crane erection simulation. They also increase the usability of the erection path planning by
reducing the preparation process and time. Finally, they improve the comprehensibility of spatial information of
the visualization method by a space-emphasized human machine interface.
244

3. ARCHITECTURE OF THE MICRO PLANNER
There are four major components in the proposed system, which are (1) the configurable crane models, (2) the
automatic collision boundaries generator, (3) the efficient erection path planner, and (4) the space-emphasized
human-machine interface. Figure 1 shows the architecture of the proposed system, the users (engineers, project
managers, or crane operators) first defined the information of the erection activity that is going to be simulated.
The information includes the applied crane to perform the erection activity; the target element to be erected or
lifted of the task; and environment status of the construction site. The system then uses the information to build up
the corresponding virtual construction scene.
Collision
Boundaries
Generator
BIM
Configurable
Cranemodels
Erec onPath
Planning
Real- me
Erec onAc vity
Simula on
Human-Machine
Interface
Users
Erec onAc vityDefini on
Crane
TaskTarget
Environment
VirtualConstruc onScene
ControllableCraneEn ty
EnvironmentEn es
TargetEn ty

Fig. 1: The architecture of the micro planner.
The virtual construction scene is composed of a controllable crane entity, a target entity, and environment entities.
An entity represents an object in the virtual environment that simulates the object in the real world. The system
retrieves the corresponding information from a configurable crane model database to build the controllable crane
entity, which can be operated by users. The information of the crane includes its type, dimension, weights, and
other physics properties required for generation of the detailed simulation. The configurable crane mode database
is established by using the proposed modeling method which modularizes cranes.
The environment entities are built from the BIM and represent the static elements and objects in the construction
site. The system retrieves the 3D model of the construction site from its BIM according to the erection activitys
specified schedule. Then the proposed generator generates proper collision boundaries of the retrieved 3D model
to the environment entities.
After the virtual construction scene was built, users can utilize it to perform real-time erection simulation by
operating the controllable crane entity. The real-time erection simulation simulates the detailed and dynamical
behaviors of the crane according to the defined crane model; since the BIM model contains the information of the
target element including storage placement and the destination, users can also utilize the developed virtual
construction scene to perform erection path planning. The erection path planner automatically finds collided-free
erection path according to the cranes located position for users to help make erection plan. The planed erection
path can also be integrated and visualized in the real-time erection simulation to rehearsal and evaluate its
feasibility; crane operators can practice and follow the planed erection path when erection task is difficult and
risky.
In the final, the proposed human-machine interface visualizes the erection simulation and the found feasible
erection path on screen in real-time. The interface provides intuitive and realistic depth perception behavior by
projecting accurate and correct images on screen according to users head position.
245

4. CONFIGUABLE CRANE MODELING
To efficiently model a crane, a systematic method must be developed and applied. A typical crane and its lifting
devices can be divided into multiple movable parts: the main body (or base) of a crane, cabin (rotatable part which
usually connects with booms), one or multiple extendable booms, trolley, pulley, cable, hook, an optional mobile
platform (for a mobile crane), lifting object, and lifting devices. These parts should be grouped and modularized to
make the crane configurable so that it can easily be adapted, modified, and reused for different crane capacities,
erection tasks, and purposes.
Figure 2 shows the common types of cranes used in construction. The similar parts of each type of crane are
marked using different colors. We modularized a crane by three major modules: the manipulation module, the
suspension module, and the lifted-objected module. The manipulation module represents the main body of a crane
and differs for different types of cranes; the suspension module contains a trolley, cables, a block actor, and a hook;
and the lifted-object module includes the object to be erected and optional lifting devices.
Mobile crane with
telescopic booms
Crawler crane Luffing crane Tower crane

Fig. 2: Cranes are divided into three modules: (1) the manipulation module (the red part); (2) the suspension
module (the green part); and (3) the lifted-object module (the blue part).
The connecting relationships between each part of crane module are represented using equations in which physical
laws are catered for. In this component, we utilized the method of multibody dynamics to simplify a complex crane
model, and used Position-based Dynamics (PBD) as the solver to compute the simulated result of a crane (Erleben
et al., 2005; Mller et al., 2007). By the configurable modeling method, we can easily define crane model for
different types of crane with different lifting object and lifting devices.
5. AUTOMATICS BOUNDARY GENERATOR
To perform collision checks may occur, we usually divide the model into groups of objects and generate a collision
boundary around the groups. This reduces the detailed geometric features of the objects for collision checking,
significantly improving the computation efficiency. However, it is not trivial to generate ideal boundaries that can
represent the physical structure of a group of objects appropriately.
In this component, we utilized Propagation Clustering Method (PCM) developed in the previous research (Hung
and Kang, 2010) to automatically group the elements in the site model. Figure 3 shows the workflow of the PCM
method. PCM is an iterative process, where each iteration executes both model clustering and quality evaluation. It
keeps grouping the elements according to their dimension and position using the k-means clustering method in
each interaction (Kanungo et al., 2002). By this method, the component can efficiently generate appropriate
collision boundaries from a construction site model with thousands of elements.
246

Model
Clustering Models
Object
Groups
Quality Evaluation
(are all groups
satisfy the
criteria)
Boundary
Generation
yes
Re-cluster the
unsatisfied groups
no
i = i + 1
i = 0
Hierarchical
Boundary Tree
Append i-depth Nodes

Fig. 3: The workflow of the PCM.
6. EFFICIENT ERECTION PATH PLANNING
In order to provide feasible erection paths to assist engineers in micro planning, a fast construction method for
configuration space (C-space) is required. Configuration space (C-space) is a procedure to simplify the path and
motion planning. It represents each configuration of an articulated machine by a point; therefore we can easily find
a collision free path by connecting the points in C-space. In this component, four reduction methods are proposed
to reduce the processing time of C-space.
Pre-consideration of lifting capacity and the lifting object: Lifting capacities of a crane is a map that defines that
loading capacities at certain configuration. The capacities are majorly related to the radius of the load, which is the
distance from the axis of horizontal rotation of booms to the center vertical hoist line (Peurifoy and Schexnayder,
2002). To reduce the processing time in C-space construction, we pre-considered the lifting capacities and the
lifting object in the C-space construction procedure. This can significantly reduce the configuration range to
perform collision detection. By applying the available luffing range in C-space construction can reduce the process
time about 30 to 40 percentages according to the type and capacity of the applied crane. In additions, we also
pre-considered the lifting objects weight before C-space construction. For example if we are planning an erection
task with a ten tons element. According to the lifting capacity map, there are some configurations not available for
this load. The extent of the reduction by this way is related to the weight of lifting object, the more heavier of the
lifting object is, the more range in boom luffing will be reduced.
Ignorance of the cable hoisting degree: According to the crane instructions of Peurifoy and Schexnayder (2002) in
crane operations, the cable hoisting can greatly affect the crane capacity during the erection process. It is also the
most inefficient operation compared with the boom luffing and slewing operations. Frequent cable hoisting is
useful to repetitive lifting tasks by decreasing the operations in luffing and slewing (Nunnally, 1999). However for
critical erection tasks, the operations needs to be avoided to reduce the possible capacity lost.
In the collision detection of each configuration during C-space construction, we calculate the minimum hoisting
length of the attached lifted-object as shown in Figure 5. The minimum hoisting length is the cable length when a
lifted-object contacts the crane booms. It is computed according to the luffing angle and the lifted-object radius.
After we acquired the corresponding minimum hoisting length and the lifted-object radius. We used a cylinder
boundary to encapsulate the lifted-object and its upper area to the top of the boom. The booms are encapsulated by
a single orientated bounding. By this way, we can guarantee that the erection path found by the generated C-space
can allow the lifted-object pass through without collisions.
247

B
o
o
m
l
e
n
g
t
h
Luffingangle
Li ed-objectradius
Minimumhois nglength
(a) (b)
Boundingcylinder
Orientated-Boundingbox

Fig. 4: Ignore the degree-of-freedom of cable hoisting: (a) illustration of the minimum hoisting length; (b)
boundaries for collision detection in C-space construction.
Establishment of collision index for environment objects: The third strategy to reduce the processing time of the
C-space construction is to establish index to the environment objects for collision detection due to the maximum
working range of configuration. We classify the object in the environment by several bounding domes with
different length of radius and give them the corresponding index. Then in the collision detection process to each
crane configuration at certain luffing boom angle, slewing angle, and boom length, we sequentially check the
collision between the crane and the environment objects in each bounding dome from the smallest dome to the
maximum dome that the current boom length can reach. In addition, when a crane configuration is been detected
that there are collisions on booms, It is not necessary to check the configuration which with the same luffing and
slewing angle however with a longer boom length due to that a boom is a solid and continuous object, which can
also reduce additional configurations for collision detection.
Bounding dome 1
Bounding dome 2
Object A
Object B

Fig. 5: Bounding domes for different extended length of the booms.
Reduction of the slewing range: The final strategy to improve the construction of the C-space is to reduce the range
of the slewing angle in the C-space. In an erection task, there are two directions to lift an object to the destination,
which are slewing clockwise or counterclockwise. In common situation, engineers would choose the clockwise
path that is more efficient and less cost in crane operations. No matter where the lifting object and its destination
are located, the maximum boom-slewing angle would be 180 degrees, which means there are at least 180 degrees
do not need to be considered in the C-space for common erection task. By this method, after the initial location and
the destination are specified, users can decided how much ranges will be reduced due to working space of the crane.
This method can save about half process time of the C-space Construction. It can also be used for planning to the
sequential erection tasks when the materials are centralized and located together.
7. SPACE-EMPHASIZED HUMAN MACHINE INTERFACE
To allow engineers clearly understand the space relationship during erection process, this component integrated
the stereoscopic and kinesthetic visions to increase the comprehensibility of the spatial information (Juang et al.,
2011). These two visualization methods can support and implement intuitive behaviors of human in spatial
information perception, which are binocular visions and motion parallax (Cutting, 1986). Stereoscopic vision
generates binocular images to human eyes. Wheatstone (1838) first described the perception mechanism of
248

stereopsis, which is generated by binocular vision3. In this component, we adapt the method for use in our
visualization pipeline and provide realistic and accurate depth perception for users.
Kinesthetic vision considers the users head position of a user during rendering, enabling realistic and accurate
space perception. It simulates the geometry of the users frustum through a virtual eye frustum, so that users can
see the appropriate virtual scene according to their head position. It calibrates the two important matrixes in the
rendering pipeline, the modelview matrix and the projection, from the operator's perspective.
(a)
(b)
(c) (d)

Fig. 6: An example of the perspective rendering: (a) normal perspective rendering; (b) expected perspective
rendering; (c) incorrect rendering without changing the view; and (d) incorrect perspective frustum.
Figure 7 shows an example of kinesthetic vision, where Figure 7(a) is the normal rendering result when the
operator's head is at the center of the screen, and Figure 7(b) is the expected rendering result using kinesthetic
vision when the operator moves his head away from the center of the screen. However, when the operator moves
his head, the rendering result on the screen will be deformed from the operator's point of view because the system
does not update the perspective according to the head position. Figure 7(c) is the rendering result when we do not
consider the position of the head, and Figure 7(d) is the result that accounts for head position but does not use the
correct perspective frustum.
By continuously providing accurate images from different angles on the screen, the system can implement the
visual cue, motion parallax, to engineers and crane operators, which is able to increase the comprehensibility of
spatial relationship.
8. IMPLEMENTATION AND RESULTS
To implement the proposed planner, two critical program libraries are integrated in the system, which are the
Microsoft XNA and NVIDIA PhysX (Microsoft XNA, 2008, NVIDIA PhysX, 2012). XNA is used to generate a
series of images of a virtual scene. It performs the process of transferring the geometry of a large number of virtual
objects, setting the viewpoint, assigning textures to objects, handling lighting and shading information, and
producing visual output. It supports a shader programming language through the Microsoft High Level Shading
Language (HSLS) as standard. Since a GPU shader takes advantage of the GPU for hardware acceleration,
Microsoft XNA can process a complicated 3D scene of a virtual construction site efficiently and can render a
realistic virtual world in real time; PhysX is a physics engine that has potential for use in real-time simulation due
to its efficient solver and high stability derived from PBD methods. The controllability of PBD allows the
simulation of dynamic joint attachment and detachment without error propagation. This allows erection activities
to be simulated continuously. We used the .NET version of the PhysX to integrate with XNA framework.
In kinesthetic and stereoscopic visions, we utilized the Microsoft Kinect sensor (Shotton et al., 2011) to capture
users' head positions and provide the data to the game framework. Two frameworks, NITE and OpenNI, were
249

integrated to retrieve the depth data and image from the Kinect sensor and convert the data into human skeleton
information (OpenNI, 2011; PrimeSense, 2011); we also integrated an active stereoscopic technology called
NVIDIA 3D Vision (NVIDIA, 2010) into the system for generating binocular images.

Fig. 7: The rendering results of an erection simulation applied kinesthetic vision, which are captured from different
heads positions.
Figure 7 shows the rendering results of an erection simulation generated the proposed micro planner. The
screenshots are captured from different heads positions and distance to the screen. The simulation was performed
at 40 frames per second running on a system equipped with an Intel Core2 Duo 2.13GHz processor with 3GB of
RAM and a Geforce 7950 display card. The result shows that the developed crane model is able to support smooth
and realistic simulation of erection. The space-emphasized interface also delivered realistic and accurate views to
users from different point-of-views.
This research developed and implemented the required methods and technologies to support real-time crane
erection simulation and automatic erection path planner in the micro planning stage. These methods include
modeling method for crane; automatic generation of appropriate collision boundaries; fast generation of C-space
for feasible erection path finding; and an effective human-machine interface to improve the comprehensibility of
space relationship of erection environment. These methods allow efficiently prototype and setup simulation
environment for erection task in a short time. The proposed micro planner is expected to effectively help micro
construction planning for crane operations and reduce the potential problems and fatal accidents. Future work will
focus on validation of the proposed planner in micro planning for crane activities.
250

10. REFERENCES
Al-Hussein, M., Alkass, S., and Moselhi, O. (2001). An Algorithm for Mobile Crane Selection and Location on
Construction Sites. Construction Innovation Journal, UK, Vol. 1, 91-105.
Birkfellner, W., Figl, M., Matula, C., Hummel, J., Hanel, R., Imhof, H., Wanschitz, F., Wagner, A., Watzinger, F.,
and Bergmann, H. (2003). Computer-enhanced Stereoscopic Vision in a Head-mounted Operating Binocular,
Physics in Medicine and Biology, Vol. 48, No. 3, N49-57.
Chang, Y. C., Hung, W. H. and Kang, S. C. (2012). "A Fast Path Planning Method for Single and Dual Crane
Erections." Automation in Construction, Vol. 22, 468-480.
Chi, H. L., Hung, W. H., and Kang, S. C. (2007). A Physics Based Simulation for Crane Manipulation and
Cooperation, Proceedings of Computing in Civil Engineering Conference, Pittsburgh, U.S. July 24-27.
Cutting, J. E. (1986). Perception with An Eye for Motion, MIT Press.
Erleben K., Sporring J., Henriksen K., and Dohlmann H., (2005). Physics Based Animation. Charles River
Media.
Garca-Fernndez, I., Pla-Castells, M., and Martnez-Dur, R. J. (2008). Elevation cable modeling for interactive
simulation of cranes, Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer
Animation, 173-181.
Hung, W. H. and Kang, S. C. (2010). Boundary Generating Method for Collision Detection in Virtual
Construction, Proceedings of 10th International Conference on Construction Applications of Virtual Reality
Conference, Sendai, Japan. November 4-5.
Kang, S. C. and Miranda, E. (2006). Planning and Visualization for Automated Robotic Crane Erection Processes
in Construction, Automation in Construction, Vol. 15, No. 4, 398-414.
Jourdan, I. C., Dutson, E., Garcia, A., Vleugels, T., Leroy, J., Mutter, D., and Marescaux, J. (2004). Stereoscopic
Vision Provides a Signicant Advantage for Precision Robotic Laparoscopy, British Journal of Surgery Society
Ltd, Vol. 91, No. 7, 879-885.
Juang, J. R., Hung, W. H. and Kang, S. C. (2011). Kinesthetic And Stereoscopic Vision For Crane Training
Systems, Proceedings of 11th International Conference on Construction Applications of Virtual Reality, Weimar,
Germany. November 4-5.
Kang, S. C. and Miranda, E. (2009). Numerical Methods to Simulate and Visualize Detailed Crane Activities.
Computer-aided Civil and Infrastructure Engineering, Vol. 24, No. 3, 169-185.
Kanungo, T., Mount, D. M., Netanyahu, N. S., Piatko, C. D., and Silverman, R. (2002). An Efficient k-Means
Clustering Algorithm: Analysis and Implementation. IEEE Transactions on Pattern Analysis and Machine
Intelligence.
Liebherr. (2012). LTM 1040-2.1. Retrieved May 13, 2012, from
http://www.liebherr.com/AT/en-GB/products_at.wfw/id-7981-0/measure-nonMetric.
Lozano-Perez, T. (1983). Spatial Planning: A Configuration Space Approach, IEEE Transactions on Computers,
C-32(2), 108-120.
Microsoft XNA. (2008). XNA, Retrieved 7 14, 2009, from MSDN:
http://msdn.microsoft.com/en-us/xna/default.aspx
Mller, M., Heidelberger B., Hennix M., and Ratcliff J. (2007). Position Based Dynamics, Journal of Visual
Communication and Image Representation, Vol. 18, No. 2, 109-118.
Nunnally, S. W. (1999). Managing Construction Equipment. Prentice Hall.
NVIDIA PhysX. (2012). PhysX. Retrieved May 13, 2012, from
http://www.nvidia.com.tw/object/physx_new_tw.html.
251

NVIDIA. (2010). NVIDIA 3D Vision Pro and Stereoscopic 3D, Retrieved May 13, 2012, from
http://www.nvidia.pl/docs/IO/40505/WP-05482-001_v01-final.pdf.
OpenNI. (2011). OpenNI User Guide, Retrieved May 13, 2012, from
http://openni.org/Documentation/ProgrammerGuide.html.
Peurifoy, R. L. and Schexnayder, C. J. (2002). Construction Planning, Equipment and Methods. Mc Graw Hill.
PrimeSense. (2011). NITE Middleware: PrimeSense, Retrieved May 13, 2012, from
http://www.primesense.com/.
Rppel, U. and Schatz, K. (2011). Designing a BIM-based Serious Game for Fire Safety Evacuation
Simulations, Advanced Engineering Informatics, Vol. 25, No. 4, 600-611.
Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., and Blake, A. (2011).
Real-Time Human Pose Recognition in Parts from Single Depth Images, IEEE Computer Vision and Pattern
Recognition (CVPR) 2011.
Wheatstone C. (1838). Contributions to the Physiology of Vision-Part the First. On Some Remarkable and
Hitherto Unobserved Phenomena of Binocular Vision, Philosophical Transactions of the Royal Society, Vol. 128,
371-394.
Xu, J. and Li, Z. (2012). Multi-Objective Dynamic Construction Site Layout Planning in Fuzzy Random
Environment, Automation in Construction, Vol. 27, 155-169.
Yang, J., Arif, O., Vela, P. A., Teizer, J., and Shi, Z. (2010). Tracking Multiple Workers on Construction Sites
using Video Cameras, Advanced Engineering Informatics, Vol. 24, No. 4, 428-434.
Yoo, W. S., Lee, H. J., Kim, D. I., Kang, K. I., and Cho, H. (2012). Genetic Algorithm-based Steel Erection
Planning Model for a Construction Automation System, Automation in Construction, 24, 30-39.
Zhang, C. and Hammad, A. (2012). Improving Lifting Motion Planning and Re-planning of Cranes with
Consideration for Safety and Efficiency, Advanced Engineering Informatics, Vol. 26, No. 2, 396-410.
252

ENHANCED HD
4
AR (HYBRID 4-DIMENSIONAL AUGMENTED
REALITY) FOR UBIQUITOUS CONTEXT-AWARE AEC/FM
APPLICATIONS
Hyojoon Bae, Mani Golparvar-Fard & Jules White
Virginia Tech, United States
ABSTRACT: Construction site activities require access to large amounts of cyber-information, such as Building
Information Models (BIM) and project specifications. Field personnel carry around large stacks of project
documents, or frequently travel to a trailer to lookup this information. Recently, several context-aware
techniques have been proposed to deliver relevant information to on-site users by intelligent interpretation of
their environment. These techniques primarily rely on GPS and/or wireless localization, which typically does not
provide sufficient precision in congested construction sites. To address these limitations, this paper extends our
work on Hybrid 4-Dimensional Augmented Reality (HD
4
AR), a high-precision mobile augmented reality system
that allows field personnel to query and access semantically-rich 3D cyber-information and see it overlaid on
top of real-world imagery. With our proposed method, field personnel can use mobile devices to take pictures of
building elements and be presented, on-site, with a detailed list of related cyber-information in an augmented
reality (AR) format. In contrast to previous techniques, our proposed vision-based method localizes the user
purely based on image matching and provides more accurate positioning. With HD
4
AR, the image captured by
the field personnel using a mobile device is sent to a server to conduct GPU-based feature extraction and
matching against pre-collected images from the jobsite. The mobile devices 3D position and orientation is then
accurately derived by solving for the Direct Linear Transform followed by a Levenberg-Marquardt optimization
against an underlying Structure-from-Motion 3D point cloud model. The paper further validates the HD
4
AR
localization method for several practical metrics. Particularly it presents the accuracy of GPU-based
localization and further reduction in 3D reconstruction and localization time from our previous CPU-based
method. The perceived benefits and limitations of the HD
4
AR system for on-site context-aware applications are
discussed in detail.
KEYWORDS: Augmented Reality, Context-awareness, 3D Reconstruction, Computer Vision, Structure from
Motion
1. INTRODUCTION
Automated, on-demand, and inexpensive access to project information on-site has significant potential to
improve important decision-making tasks during on-site construction activities. Information access helps with
prompt identification, processing, and communication of discrepancies between actual and expected construction
performance. Fast access to this information also enables project managers to proactively decide on corrective
actions and minimize the cost and delays due to performance discrepancies. (Golparvar-Fard et al. 2012).
Despite the importance of on-site information access, current approaches to jobsite progress monitoring include
manual and time consuming data collection, non-systematic analysis and visually/spatially complex reporting
(Golparvar-Fard et al. 2009, Navon and Sacks 2007). As part of data collection and analysis activities, field
personnel have to carry large stacks of paperwork and drawings on jobsites and spend significant amount of time
accessing and retrieving relevant project information (Khoury and Kamat 2009). The quality and timing of
accessing relevant site information can either facilitate or delay successful execution of onsite activities (Chen
and Kamara 2011). Such inefficiencies in site analysis and information gathering can cause downtime or rework
and ultimately lead to schedule delays or cost overruns.
Over the past decade, several research projects have focused on providing site information to field personnel
through mobile devices and/or augmented reality systems (Akula et al. 2011, Behzadan et al. 2008, Khoury and
Kamat 2009). These works have primarily focused on using Global Positioning Systems (GPS), Wireless Local
Area Networks (WLAN), Ultra Wide Band (UWB), and Indoor GPS for accurately positioning the user within
congested construction environments. The main drawback of these location tracking technologies is their high
degree of dependency on preinstalled infrastructure, which makes their application either difficult or impractical
for construction sites. In recent years, research has focused on developing location tracking technologies that are
infrastructure-independent. These systems are typically based on inertial measurements and make use of highly
accurate accelerometers and gyroscopes. Given their independence from an existing infrastructure, their
253

application may result in accumulated drift error which grows with the distance traveled by the users (Akula et al.
2011). More recently, (Akula et al. 2011) has proposed a Real Time Kinematic-Global Positioning System
(RTK-GPS), to ubiquitously track a mobile user, in dynamic construction environments where GPS coverage
may be uncertain. Nonetheless accuracy of this system is in centimeter level and requires the environment to be
retrofitted with a correction marker grid so that the system is recalibrated and the accumulated drift errors are
minimized.
Instead of adding a new technological component to provide accurate location and orientation information, this
paper builds on our previous work on Hybrid 4-Dimensional Augmented Reality (HD
4
AR) system (Miranda et al.
2012) and provides reliable identification of the location and orientation of the user based on existing and
already available mobile devices. The proposed vision-based system not only provides the location and
orientation of the user, but it also provides high-precision visualization of semantically-rich 3D
cyber-information over real-world imagery in an augmented reality (AR) format. Rather than using imprecise
mobile GPS and/or wireless sensors, as in existing mobile AR approaches, the HD
4
AR system allows
construction field personnel to use mobile devices, such as smartphones or tablets, to take pictures for accurate
localization. Given the popularity of smartphones and tablets and particularly their rising availability on
construction sites (ENR 2010), a solution with mobile devices which does not require precise location tracking
technology can lead to many promising mobile augmented reality approaches for context-aware
Architecture/Engineering/Construction and Facility Management (AEC/FM) applications.
The remainder of this paper first presents the process of integrating images with semantically-rich 3D models.
Next, the new vision-based method for localization and AR visualization is introduced and improvements in
searching and matching with physical model are discussed. It also demonstrates our new enhancement on system
which significantly reduces 3D reconstruction and localization time with multi-core CPU and GPU based
implementation. Finally the tremendous increase in localization success ratio is presented to prove that our
system enables user to get consistent localization experience at random locations. The perceived benefits and
limitations of the proposed method in the form of open research challenges are presented. Video demo, detailed
performance data, along with additional supplementary material for the proposed vision-based method can be
found at http://www.raamac.cee.vt.edu/hd4ar.
2. BACKGROUND AND RELATED WORK
In the past decade, advances in digital photography and image processing have led to new research on the
application of image-based reasoning for various construction management tasks and techniques that can
manually, semi-automatically, and automatically interpret them (Cheng and Chen 2002, Golparvar-Fard et al.
2009, Golparvar-Fard et al. 2011, Golparvar-Fard et al. 2010, Kiziltas et al. 2008). This research has shown that
a set of overlapping images can be used to extract accurate 3D geometry of stationary objects such as buildings
under construction. After the cyber models (e.g., BIM) and the physical models (e.g., the extracted 3D geometry
models) are aligned, they can be compared to determine the actual state of the physical elements on construction
site versus the expected state. Researchers have shown that the fused cyber-physical model is accurate to within
millimeters (Golparvar-Fard et al. 2011) and can be used to predict the actual construction progress versus the
planned cyber-model with high accuracy, even when visual obstructions are present (Golparvar-Fard et al. 2010).
As shown in Fig. 1, the physical models extracted from computer vision approach can successfully localize the
users photograph with high accuracy.
Fig. 1: The example of physical model and cyber model (adopted from (Golparvar-Fard et al. 2012).)
254

Generating a physical model from construction site imagery is computationally expensive and can take hours or
more. Producing a physical model from a set of construction photographs requires non-linear multi-dimensional
optimization as well as exhaustive matching of the photographs in the data set. A specific aim of our proposed
work is to overcome these challenges, speeding up overall time of 3D reconstruction and localization by
developing and optimizing enhanced Structure from Motion (SfM) algorithms.
The remainder of this paper is organized as follows: Section 3 presents the technical solutions of HD
4
AR
localization and the HD
4
AR augmented reality system. Section 3 discusses the details of the 3D reconstruction
algorithm and its related parameters. Also the bootstrapping, localization and augmentation process of our
system are explained in Section 3. Section 4 presents empirical results from experiments with HD
4
AR and also
compares the performance to another SfM implementation, called Bundler package (Snavely et al. 2008), which
takes unordered photographs and using the principles of SfM, automatically generates a 3D point cloud model.
Finally, conclusions and recommendations are described in Section 5 and 6.
3. HD
4
AR: HYBRID 4-DIMENSIONAL AUGMENTED REALITY
3.1 Overview
In order to augment a given photograph with cyber-information, HD
4
AR requires that construction field
personnel first take overlapping photos of the construction site to produce an initial physical model, i.e., 3D
point cloud. The initial 3D point cloud generation is based on the SfM algorithm that rapidly estimates the 3D
position of the image features through feature extraction, matching, initial 3D reconstruction, and an
optimization process called Bundle Adjustment. After the initial bootstrapping is complete, the actual user can
take a new photo at a random location and his/her location is determined by comparing this new image to the
initial 3D point cloud. Once the user is localized, the fused cyber-physical model can be used to predict where
cyber-information, such as BIM elements, should appear in the original photograph and where physical objects
actually are in the cyber 3D space. This ability to track where BIM elements should appear in photographs serves
as the foundation of HD
4
AR. Moreover, after a field engineer has an augmented photograph, he or she is able to
use a multi-touch interface to select physical objects in view to retrieve more information associated to that
object. Fig. 2 summarizes overall procedures of HD
4
AR system.

Fig. 2: The overall procedures of HD
4
AR system
3.2 Bootstrapping Process
As discussed in Section 3.1, an initial physical model of target construction site must be created to serve as a
reference model for the localization process. Creating this 3D point cloud, which we call the bootstrapping process,
requires collection of an initial set of base images of the construction site and processing these images using the
SfM algorithm that estimates the 3D positions of 2D image feature points.
In our initial work (Miranda et al. 2012), we used the Bundler package (Snavely et al. 2008), a widely-used
software package that implements SfM on an unordered set of photographs. However, this package takes from
255

hours to days to generate a 3D point cloud even for small numbers of images, due to its use of exhaustive
computations of non-linear multi-dimensional optimization processes on a single-thread CPU. In addition, it uses
SIFT (Scale Invariant Feature Transformation) descriptors (Lowe 2004) for feature extraction, which have
reasonable invariance properties but require multiple layer computations for each spatial scale, and thus is time
consuming.
For these reasons, we designed and implemented a new parallelized 3D reconstruction modeling system that
operates across cores in a multi-core CPU and GPU. Most engineering workstations today have a multi-core CPU
with 2-16 cores and a GPU with anywhere from 4 to 128 cores. Exploiting this hardware parallelism is a key to the
performance and scalability of HD
4
AR system. We parallelize all the steps for 3D point cloud generation to obtain
performance gains of 2,587% and implement GPU-based SURF (Speeded-Up Robust Features) descriptors (Bay
et al. 2006) for fast feature detection. The algorithms in HD
4
AR mostly follows original steps in SfM algorithm
of the Bundler package except 1) type of feature descriptors, 2) new optimization parameters for reducing noise
of 3D point cloud to get accurate localization results, and 3) exploiting multi-core CPU and GPU hardware for
speedup. All the steps in our system can be summarized as follows:
1) Feature Detection: A feature detection algorithm is run on each image to create a set of feature
descriptors to be used as the basis for matching image to one another. GPU-based SURF descriptors are
implemented and used in HD
4
AR for feature extraction. The extracted descriptors are stored as binary
files for faster Input/Output (I/O) tasks.
2) Feature Matching: The set of features for each image are iteratively compared against each other. For
initial descriptors matching, HD
4
AR creates k-d tree of feature descriptors and runs Approximate
Nearest Neighbors (ANN) searching algorithm (Arya et al. 1998). Then it performs the ratio test as
described in (Lowe 2004) to reduce the percentage of false matches. Finally, it robustly estimates a
Fundamental matrix and removes outliers for every image pair using the RANSAC (RANdom SAmple
Consensus) algorithm (Fischler and Bolles 1981) with the eight-point algorithm (Hartley 1997). To
shorten the overall execution time, each image pairs is processed on different CPUs with parallelized
I/O tasks.
3) Structure from Motion:
a) Estimation of Camera Parameters: The SfM algorithm begins by estimating the camera
parameters, such as focal length, rotation matrix and translation vector, of an initial image pair
using Nistrs five-point algorithm (Nistr 2004), and the 3D positions of their feature points are
triangulated. As discussed in Snavely et al. 2008, the initial pair should have a large number of
matched feature points, but also have a long separation distance between the cameras, so that the
Bundle Adjustment algorithm will not get stuck in a local minimum. After estimating the camera
parameters of initial image pair, HD
4
AR system attempts to calibrate camera parameters of each
additional base image from already triangulated 3D points and to register new points seen by these
new cameras. This registration fails in the event that an additional image does not have matched
feature points against previously registered images. These camera calibration and triangulation
steps are well-parallelized to exploit multi-core CPUs.
b) Incremental Bundle Adjustment: While the images are being added (registered), the 3D
reconstruction pipeline is run through a GPU-based sparse bundle adjustment module to minimize
the overall re-projection error, i.e. the difference between predicted 2D positions of the feature
points in the photographs given their triangulated 3D positions and the locations of where the
feature points actually appear in the images. The HD
4
AR adopts Parallel Bundle Adjustment (Wu
et al. 2011) to significantly enhance the speed of this optimization.
c) Noise Reduction: Bundle Adjustment is an optimization process that tries to minimize the overall
re-projection error of all 3D points at the same time. It is possible that some 3D points have high
re-projection error while other 3D points have very small re-projection error, resulting in an overall
small Minimum Mean-Square Error (MMSE). Since the ultimate purpose of 3D point cloud
generation is the user localization, not the visual representation of physical model, it is very
important to reduce the noise of 3D point cloud by removing these 3D points with high
re-projection errors. To achieve this, HD
4
AR uses double-threshold scheme. The first threshold is
for controlling the target MMSE of Bundle Adjustment. We set this threshold as 1 pixel
2
so that the
average re-projection error of entire 3D point cloud is not greater than 1 pixel. Another threshold,
256

which we called as an absolute re-projection threshold, is for removing individual 3D points from
point cloud. This threshold is set to be 1.5 pixels so that no 3D points in final point cloud have a
re-projection error greater than 1.5 pixels.
Due to our algorithmic and parallelization improvements, 3D reconstruction with HD
4
AR is ~26 times faster
than our previous work based on the Bundler package (Miranda et al. 2012). Experimental results and
validations are discussed in detail in Section 4. Fig. 3 shows some examples of physical models generated by
HD
4
AR using photos from a real-world construction project in Blacksburg, VA.
Fig. 3: Result of 3D point cloud reconstruction from base images. (b) and (c) represent the outcome of the SfM
algorithm implemented in the HD
4
AR system and Bundler package respectively.

After generating a 3D point cloud, the physical model must be aligned with the cyber model so that they share
the same coordinate system. This step is require for the system to be able to determine what BIM elements are
visible in photographs and determine where they should be rendered in the photographs as part of the
localization process. Similar to (Golparvar-Fard et al. 2012), HD
4
AR uses closed-form solution (Horn 1987) to
achieve this alignment. The detail results will be discussed further in Section 4.
3.3 LOCALIZATION AND AUGMENTATION PROCESS
Once the point cloud is generated, the system can augment photos sent from the HD
4
AR client running on a users
mobile device. From a high-level perspective, this process operates as shown in Fig. 2. Step 1, the field personnel,
upon finding a section of the worksite he/she wishes to query, takes a picture of the area using a mobile device.
Step 2, the device uploads the captured image to the HD
4
AR server. Step 3, the server runs a feature detection,
feature matching, and camera calibration to identify relative location of camera against the base point cloud.
Step 4, using the relative rotation and translation information of image as input, the server determines what BIM
objects are within the images field of view, and where they appear. Step 5, the objects are sent back to the user
device with positional information and semantic information. Step 6, the user device renders the captured image,
overlaid with the returned objects.
A key improvement over our previous work is that we removed the reduced single-image SfM procedure form
the HD
4
AR localization process. HD
4
AR uses only the RANSAC algorithm for camera pose recovery, which is
faster and still provides sufficient accuracy for localization. Another improvement over our prior work is that the
previous reduced SfM algorithm would change the position of 3D points or location of registered cameras. These
changes are undesirable and require a computationally expensive re-transformation of the cyber models
coordinates to align it to changed physical model. It also prevents user from getting consistent localization
results since the physical model keeps changing its coordinates. Therefore, we eliminated reduced single-image
SfM procedure from HD
4
AR, which we found speeds up the localization by approximately 600% and provides
more reliable localization results.
Fig. 4 shows screenshots from the Android HD
4
AR client running on a smart phone with Android version 4.1. In
the screenshots the 3D BIM information is precisely aligned with the real-world imagery despite significant
changes in the position and orientation of the users device. As shown in the top-right screenshot, clicking on
elements can show additional cyber-information related to that element, such as an Android Toast notification, or
open more detailed list views showing element specifications, code numbers, or cost. Moreover, HD
4
AR
successfully recognizes target buildings from the photograph and overlays the BIM information precisely, as
257

shown in the bottom-right screenshot. These examples imply that HD
4
AR remains stable under different
illumination and weather condition as well as totally different viewpoints of a users device.
Fig. 4: Screenshots of the Android HD
4
AR client overlaying 3D BIM information precisely on photos from
different viewpoints.
This section describes experiments we conducted to assess the ability of HD
4
AR to produce initial 3D point
clouds, localize new photographs taken at random locations, and displaying appropriate cyber-information on top
of the photographs.
4.1 Platform Specification
The server-side components of the experiments were conducted on a workstation with 12 gigabytes of 667 MHz
DDR3 1333 RAM, and a 4-core Intel i7 CPU 980 (@ 3.33 GHz) processor running Ubuntu version 11.14. The
workstation graphic card was an NVIDIA GeForce GTX 480. The images used to create the 3D point cloud
came from the construction site for a new Virginia Tech parking garage and other static buildings on campus.
The photographs were taken using Samsung Nexus S and Samsung Galaxy Nexus smartphones. Both of these
smartphone models were co-developed by Google and Samsung and have 5-Mega pixel cameras. Since the CCD
widths of these phones were unknown, we calibrated these cameras to obtain focal lengths. This calibration
process helps to properly choose the initial image pair and minimizes the chances that Bundle Adjustment will
terminate in a local minimum.
The same smartphone models were used to run the HD
4
AR client for localization tests. The test images were
taken at random locations and tested for on-site localization correctness. For fast data transfer, the client-side
communication was based on Wi-Fi 802.11g connection rather than using the cellular network.
4.2 HD
4
AR Performance on 3D Reconstruction
The first experiment was performed to generate the initial 3D point cloud by using total of 143 images from the
experimental construction site. To show the computational speed of the proposed approach, the same data set
was run on Bundler package. The overall computation time is compared in Table 1 and it shows that HD
4
AR
reduces 3D reconstruction time by 2,587%. Though HD
4
AR has smaller number of registered images, it still
provides same level of localization success-ratio. The reason for this is that HD
4
AR introduces absolute
threshold to remove several 3D points which have high re-projection errors. The resulting 3D point-clouds from
each system are shown in Fig. 3.
4.3 HD
4
AR Performance on Localization
In order to measure the reliability of localization, we investigated how the number of initial base images affects
the localization success-ratio of system. For this experiment, four subsets of initial 143 base images were
selected. First, a set consisting of 25 images, attempting to cover the entire scene with a minimal number of
258

photographs were selected. Then, 25 additional images were added to make a set of 50 with more feature points
for matching. The process was repeated for 75 and 100 image sets. Each of these subsets was run through our
system and Bundler package to generate the initial 3D point cloud of the scene. Finally, total of 112 images
different from initial base images were tested against each 3D point-cloud.
Table 1: Elapsed time of 3D point cloud reconstruction
HD
4
AR system Bundler package
No. of registered images 138/143 143/143
Elapsed Time 25.708 mins 665 mins
Performance Gain 25.87 1
Localization success-ratio 100% (112/112) 100% (112/112)
In addition to how reliable the system is, we were interested in the time required to localize the user with
HD4AR system. Therefore we also measured localization/augmentation time for each case and compared them
to our previous work. These experiment results are shown in Table 2.
Table 2: Localization success-ratio and elapsed time for different number of base images
Number of base images Localization Success-ratio Average Localization Time
HD
4
AR
Bundler package
100
100
100% (112/112)
100% (112/112)
23.21% (26/112)*
13.04 sec (5.91)
77.094 sec (1)
HD
4
AR
Bundler package
75
75
98.21% (110/112)
96.43% (108/112)
23.21% (26/112)*
9.871 sec (6.13)
60.543 sec (1)
HD
4
AR
Bundler package
50
50
97.32% (109/112)
96.43% (108/112)
21.43% (24/112)*
8.871 sec (4.83)
42.812 sec (1)
HD
4
AR
Bundler package
25
25
72.32% (81/112)
72.32% (81/112)
18.75% (21/112)*
4.116 sec (5.78)
23.814 sec (1)
* As in our previous work, this result came from Bundler package without accounting radial distortion factor of camera.
Fig. 5: Example of localization results: Three photos of a construction site augmented by information about
safety guidelines using the HD
4
AR application
As observed, the result proves that HD
4
AR can localize a user solely based on an image and within few seconds.
With HD
4
AR, field engineer has to wait only 4-10 seconds after he or she takes a photo of target objects to
259

retrieve related cyber-information. This is very promising for the use of HD
4
AR in practice. Information retrieval
using HD
4
AR now can be done in much less time than our previous work or the traditional means (traveling
back to trailer to lookup cyber information or carrying large stacks of drawings on site and looking up for
information on demand). The system increases the speed of localization by an average factor of 5.66. Another
interesting thing is that a large amount of base images beyond certain number, 50 images for this case, produces
redundant information which just increases the localization time without any gain for localization reliability. In
future work, there is a need to design a means of quantifying the quality of 3D point cloud which will guide user
to take a minimal number of images for 3D reconstruction.
Finally, we applied HD
4
AR system to several construction sites and static buildings for investigating ability of
the system on on-site information management and facility management applications. For these experiments, the
images taken from various sites were used to generate 3D point cloud models. Then the testers are encouraged to
take test photos at random locations and test localization on-site. As shown in Figs. 5-7, HD
4
AR localizes the
user with high success-ratio and overlays cyber-information precisely. Figs. 5 and 6 show the application of the
HD
4
AR system on two different construction projects. Fig. 5 in particular shows three snapshots captured on the
smartphone device. These photos which relatively show the same part of the site are all augmented with
information about onsite safety guidelines. Fig. 6 also shows a scenario for owner representatives. In this case,
these three sections of a concrete wall are augmented by progress reports that were documented previously by
the contractor/construction management team. The results of HD
4
AR localization on different static buildings are
represented and summarized in Fig. 7.
Fig. 6: Examples of HD
4
AR applications on a real-world construction site.

Fig. 7: Examples of HD
4
AR applications on different static buildings
260

5. DISCUSSION ON THE PROPOSED METHOD AND RESEARCH CHALLENGES
This study presented a robust localization and mobile augmented reality solution for providing context-aware
AEC/FM information. The localization success-ratio of 98.21% given 50 base images promises the applicability
of the proposed method. Such results also indicate the robustness of the method to dynamic changes of
illumination, viewpoint, camera resolution, and scale which is typical for both unordered construction image
collections and also the ability of the user taking photos from all viewpoints. While this paper presented the
initial steps towards localization and AR visualization for the purpose of context aware applications, several
critical challenges remain. Some of the open research problems include:
Quantify the quality of 3D point cloud which will guide user to take a minimal number of images from
various construction sites for bootstrapping.
Cluster the 3D point cloud using supplemental information such as mobile GPS information available
in smartphones to reduce data set to be matched, increasing speed of localization.
Make automated alignment algorithm among physical models and between physical model and cyber
model to make system more efficient.
6. CONCLUDING REMARK AND FUTURE WORK
The current practice of construction progress monitoring still has a lot of opportunities for improvement through
the integration of cyber-information into regular site operations. The HD
4
AR system was designed with the
intent of bringing augmented reality to construction sites, to better facilitate the use of cyber-information around
worksites. HD
4
AR takes vital project information, such as location of elements, project schedule and
specifications, and cost information, which traditionally has been difficult to access on a jobsite, and makes it
mobile, accessible, and possible to visualize over real-world imagery. HD
4
AR works using visual features as the
basis for user localization, using SfM techniques to build and match a 3D geometric model from regular
smartphone camera images. Users can use a smartphone outfitted with a camera, screen, and wireless
communication to upload a captured image, localize it, determine relevant cyber information, and then overlay
the returned cyber information on the physical objects to which it pertains. The performance of HD
4
AR with
localization success-ratio of 98.21% also implies that it is possible to develop a near real-time augmented reality
systems with only ~50 base images. It takes 4-10 seconds for localization and ~30 minutes for point cloud
generations. With everyday data collection and application of HD
4
AR, related point clouds can be produced very
quickly and it means that AEC/FM practitioners could easily monitor construction progress by quickly accessing
relevant information. In future work, we are planning to use full IFC-based (Industry Foundation Class) BIM
rather than using manually created elements. Reducing search space with the aid of GPS information available in
smartphone to further enhance the performance is also in our focus.
7. ACKNOWLEDGEMENTS
This material is based upon work supported by the National Science Foundation under NSF CMMI-1200374
award. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of
the authors and do not necessarily reflect the views of the National Science Foundation.
8. REFERENCES
Akula M., Dong S., Kamat V. R., Ojeda L., Borrell A. and Borenstein J. (2011). Integration of Infrastructure
Based Positioning Systems and Inertial Navigation for Ubiquitous Context-Aware Engineering Applications,
Advanced Engineering Informatics, Vol. 25, No. 4, 640-655.
Arya S., Mount D. M., Netanyahu N. S., Silverman R. and Wu A. Y. (1998). An Optimal Algorithm for
Approximate Nearest Neighbor Searching Fixed Dimensions, Journal of the ACM, Vol. 45, No. 6, 891-923.
Bay H., Tuytelaars T. and Gool L. V. (2006). SURF: Speeded Up Robust Features, Lecture Notes in Computer
Science, Vol. 3951, No. 404-417.
Behzadan A. H., Aziz Z., Anumba C. J. and Kamat V. R. (2008). Ubiquitous Location Tracking for Context
Specific Information Delivery on Construction Sites, Automation in Construction, Vol. 17, No. 6, 737-748.
261

Chen Y. and Kamara J. M. (2011). A framework for using mobile computing for information management on
construction sites, Journal of Automation in Construction, Vol. 20, No. 7, 776-788.
Cheng M.-Y. and Chen J.-C. (2002). Integrating barcode and GIS for monitoring construction progress, Journal
of Automation in Construction, Vol. 11, No. 1, 23-33.
ENR (Engineering News-Record) (2010). Gadgets are hot and more are at hand, McGraw-Hill.
Fischler M. A. and Bolles R. C. (1981). Random sample consensus: a paradigm for model fitting with
applications to image analysis and automated cartography, Communications of the ACM, Vol. 24, No. 6,
381-395.
Golparvar-Fard M., Pea-Mora F. and Savarese S. (2009). Application of D4AR - A 4-Dimensional augmented
reality model for automating construction progress monitoring data collection, processing and communication,
Journal of Information Technology in Construction, Special Issue Next Generation Construction IT: Technology
Foresight, Future Studies, Roadmapping, and Scenario Planning, Vol. 14, No. 129-153.
Golparvar-Fard M., Savarese S. and Pea-Mora F. (2010). Automated Model-Based Recognition of Progress
Using Daily Construction Photographs and IFC-Based 4D Models, Proceedings of 2010 Construction Research
Congress, Banff Alberta, Canada, 51-60.
Golparvar-Fard M., Pea-Mora F. and Savarese S. (2011). Integrated Sequential As-Built and As-Planned
Representation with D4AR Tools in Support of Decision-Making Tasks in the AEC/FM Industry, ASCE
Journal of Construction Engineering and Management, Vol. 137, No. 12, 1099-1116.
Golparvar-Fard M., Pea-Mora F. and Savarese S. (2012). Automated Model-Based Progress Monitoring Using
Unordered Daily Construction Photographs and IFC As-planned Models, ASCE Journal of Computing in Civil
Engineering, Vol. No.
Hartley R. I. (1997). In defense of the eight-point algorithm, IEEE Transactions on Pattern Analysis and
Machine Intelligence, Vol. 19, No. 6, 580-593.
Horn B. K. P. (1987). Closed-form solution of absolute orientation using unit quaternions, Journal of the Optical
Society of America (JOSA) A, Vol. 4, No. 4, 629-642.
Khoury H. M. and Kamat V. R. (2009). Indoor User Localization for Context-Aware Information Retrieval in
Construction Projects, Automation in Construction, Vol. 18, No. 4, 444-457.
Kiziltas S., Burcu A., Ergen E. and Pingbo T. (2008). Technological assessment and process implications of field
data capture technologies for construction and facility/infrastructure, Journal of Information Technology in
Construction, Special Issue Sensors in Construction and Infrastructure Management, Vol. 13, No. 134-154.
Lowe D. G. (2004). Distinctive Image Features from Scale-Invariant Keypoints, International Journal of
Computer Vision, Vol. 60, No. 2, 91-110.
Miranda P., Sharakhov N., White J., Golparvar-Fard M. and Dougherty B. (2012). Hybrid 4-Dimensional
Augmented Reality - A High-precision Approach to Mobile Augmented Reality, Proceedings of 2012 Pervasive
and Embedded Computing and Communication Systems (PECCS) Conference, Rome, Italay, 156-161.
Navon R. and Sacks R. (2007). Assessing research issues in Automated Project Performance Control (APPC),
Journal of Automation in Construction, Vol. 16, No. 4, 474-484.
Nistr D. (2004). An efficient solution to the five-point relative pose problem, IEEE Transactions on Pattern
Analysis and Machine Intelligence, Vol. 26, No. 6, 756-770.
Snavely N., Seitz S. M. and Szeliski R. (2008). Modeling the World from Internet Photo Collections,
International Journal of Computer Vision, Vol. 80, No. 2, 189-210.
Wu C., Agarwal S., Curless B. and Seitz S. M. (2011). Multicore Bundle Adjustment, IEEE Conference on
Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA: 3057-3064.

262

COMMUNICATION SUPPORT SYSTEM FOR WORKERS AND
SUPERVISORS BY MULTIPLE-CAMERA HANDLING AND
AUGMENTED REALITY TECHNIQUES
Takehiro Urano, Tsuneya Kurihara & Yuki Watanabe
Central Research Laboratory, Hitachi Ltd., Japan
Tatsuhiko Kagehiro
Research & Development Group, Hitachi Ltd., Japan
Tetsuya Tsubokura, Fumio Hatori & Yuichi Yashiro
Matsudo Research Laboratory, Hitachi plant Technologies, Ltd. Japan
ABSTRACT: A large-scale video-monitoring system for monitoring the progress of a construction site and
assessing safety and security at the site was developed. The system is based on a technology for handling a large
number of IP-cameras fitted with a function for controlling their pan-tilt-zoom (PTZ) parameters. The PTZ
parameters are controlled in order to capture images in various directions and zoom factor. When the cameras
catch significant events (such as peoples motions and faces), those events are shown on a head-office supervisors
PC in high definition at high frame rate. In addition, a function called space tag for supporting information
sharing was developed. Camera-ID, PTZ parameters, and information such as text messages are registered in a
database. When an image taken by a camera is viewed, the text in the field of view of the camera is obtained from
the database and displayed overlapping the image. The developed system thereby acts as a platform for efficient
monitoring of work status at construction sites. Moreover, it supports communication with workers at overseas
construction sites and supervisors at head offices. In a field demonstration of the system at an actual construction
site of a thermal power plant in Japan, seven cameras were installed around the plant, and the system has been
successfully operating for 24 hours a day for the last six months
KEYWORDS: Remote Monitoring, Communication Support System
1. INTRODUCTION
1.1 Background of system development
In the construction industry, ICT (Information and Communication Technology)-based technology is being
increasingly introduced. In many cases, it is used by project supervisors for monitoring the progress of a
construction. While the number of plant constructions and the amount of accompanying maintenance work in
overseas countries is increasing, it is necessary to perform the required tasks by a small number of head-office
staff and overseas staff. It is therefore required to teach operational procedures and to determine the status at a
construction site efficiently from the head-office side. On the other hand, understanding the status of a
construction site is a task for skilled supervisors. To help these staff understand the construction status and
perform maintenance activities effectively, it is therefore expected to utilize images, sensor data, and 3D-CAD
drawings taken during construction.
With this expectation in mind, the authors have developed a large-scale video- monitoring system that uses
multiple IP-cameras to support a supervisors tasks for managing materials, personnel, and work progress. It also
supports a supervisors duties concerning safety monitoring through assessment of current site conditions,
issuing work instructions remotely, safety and security monitoring in managed areas, and monitoring health of
operators.

1.2 Challenges and solutions
The first challenge in developing the large-scale video-monitoring system was to solve the problem concerning a
conventional system, namely, viewing multiple images simultaneously. This problem is caused by the
communication bandwidth being congested by the vast amount of data traffic between the construction site and
the monitoring base station when communication devices such as cameras are installed at a construction site. If
JPEG-images are sent at frame rate of 30 fps and resolution of QVGA (320x240) from 100 cameras, the required
263

bandwidth is 150 MBps. For instance, the bandwidth of satellite communication (which must be used in many
construction-site cases) is limited to about 2 Mbps. In addition, if all images are sent successively, the amount of
information would exceed a persons recognition capacity. As for the developed communication-support system,
these problems are solved in the two ways. First, the importance of each image is judged by using face- and
motion-detection technologies. Second, only the images judged to be important are transmitted at high frame rate
and high definition and presented to the users (e.g., supervisors and managers at the head office). Transmission
data volume can be reduced by installing an image-processing PC at the construction site and using it to control
image transmission according to the image-recognition results so that only the required images are sent.
The second challenge concerns communication between managers at the head office and workers at the overseas
construction site. For example, when a manager wants to give an instruction related to the position of a piece of
equipment at the construction site, the worker will understand the instruction more easily if it is attached to an
image of that piece of equipment. This concept is therefore adopted by the proposed system as a feature whereby
the users (i.e., both head-office supervisors and on-site workers) can easily attach text/image data to a specific
part of an image. Accordingly, image-based communication becomes possible.

2. REMOTE LARGE-SCALE VIDEO MONITORING SYSTEM FOR
CONSTRUCTION SITE
The large-scale video-monitoring system is overviewed in Fig. 1. It uses a set of standard IP-cameras and
provides a connection between the cameras and an image processor. The images acquired by the cameras are
transmitted via a local area network (LAN). By providing the images through a LAN in the head office in Japan
or a LAN in the overseas office at the construction site, it is possible to support a supervisors tasks. In addition,
by providing messages or CAD images according to the location of workers, it is possible to give instructions or
follow up the status of construction. This system has two key features. The first is real-time image processing. In
particular, image processing units are installed at the overseas construction sites, and the importance of
individual images is judged by motion detection and face detection. To reduce the amount of communication
traffic, the units compress the image to JPEG format or resize the resolution on demand from users. The second
feature is dynamic display control showing important views from a particular camera at high frame rates and
other views from other cameras at low frame rates, the system can reduce the cost of the supervisors visual
check. To transmit only the important views at high frame rate and with high definition, the communication rate
from the construction site to the monitoring base station can be reduced.

CloudServer
ForeignConstructionSite1
Cameras
PoE Hub
WiFi
ImageProcessor
DataCoder
EncryptionNetwork
CellularNetwork
MovementDetection
ImageCompression
BaseStation1
DomesticBaseStation
ImageProcessor
ImageProcessor
DataCoder
DataCoder
ConstructionSite1
ConstructionSite1
BaseStation2

Fig. 1: Overview of the large-scale video surveillance system
264

2.1 System architecture
The system architecture of the large-scale video-monitoring system(Y. Yashiro, 2012) is shown in Fig. 2. The
system consists of the following six parts. (1)IP-cameras have a function for controlling their pan-tilt-zoom
(PTZ) parameters. These cameras can capture an image in various directions and zoom factor by controlling the
PTZ parameters. The system can manage both wired IP-cameras and wireless IP-cameras. (2)Image-processing
servers, namely, PCs, communicate with the cameras and determine the importance of images on the basis of
motion detection by using background subtraction (E. Komagal, 2012) and face detection (P. Viola and M. J.
Jones, 2004). Each server handles 10 cameras and gathers images at 10 fps. If an event of possible interest is
detected, the server calculates feature vectors from the image and sends them to the image-retrieval server.
(3)The image-retrieval server receives the data concerning the event detected by the image-processing servers
and stores the feature values and images. When a user requests a search for similar images to the image that the
user already has, the image-retrieval server searches for images according to a statistical approach (D. Matsubara
and A. Hiroike, 2009). In other words, it behaves like an event-management server. (4)An access control server
then controls transmissions of images dynamically according to the event-detection results. (5)Two kinds of data
archiving server are then utilized. A temporary-archiving server (a kind of an image-archiving server) archives
all the images captured by cameras in daytime, encodes into as MPEG videos at nighttime, and transmits the
video data to a video archiving server. (6)A monitor application displays the images captured by the
higher-priority cameras with high resolution and thumbnail images from all the other cameras. Since the monitor
application displays images that need extra attention at high frame rate, it reduces the cost of the supervisors
visual check.

(1)Cameras
(4)AccessControl
Server
Decidepriority
(2)Image
ProcessingServer
(5)Temporary
ArchivingServer
(5)Video
ArchivingServer
(3)Retrievalserver
Adaptive
transmission
Narrowband
Network
C onstruction Site
C loud Server
Highspeed
Search
(6)Monitor Application
(4)AccessControl
Server

Fig. 2: System architecture

2.2 Experiment
2.2.1 Communication-traffic data rate
To experimentally investigate the performance of the large-scale video-monitoring system, we installed 40
cameras in a test office environment and measured the traffic data rate through the camera servers. Each camera
sends JPEG images at a rate of 10 frames per second. When an event (such as a detected motion) is detected, one
of the camera servers sends images at a frame rate of 10 fps for 5 s to the monitor application; at other times, it
sends images at 1 fps.
The sum of all measured traffic data rates (input and output) through both camera servers is shown in Fig. 3.
According to the measurement results for one hour, the data rate of input traffic is 14.16 MBps on average, and
that of output traffic is 2.26 MBps on average. The system therefore reduces the traffic rate by 84%. In the office
environment used in the test, few events occur while workers are doing desk work. When workers left their
desks, however, certain motions were detected, and the output traffic increased to 3.69 MBps, that is, 26.1% of
the input traffic. The percentage reduction of output traffic depends on scene and its contents taken by the
camera. Accordingly, developing an event-detection method optimized according to image contents is an
important future works.
In the case of construction site monitoring, it is necessary to transmit images that supervisors want to see. So, it
is future works to evaluate whether all important events are taken.
265

0
2
4
6
8
10
12
14
16
0 10 20 30 40 50 60
Input
Output
[MBps]
[minutes]
14.46
13.80
14.16
3.69
1.22
2.26

Fig. 3 Measured input and output traffic data rate through camera servers with time

2.2.2 Field demonstration at construction site of thermal power plant
As a field demonstration, the remote monitoring system was tested at the construction site of the Hitachinaka
Thermal Power Plant Generator No. 2 (in Japan). The configuration of the system is shown in Fig. 4. An
encrypted communication device and an image processor (connected to a cloud server) were installed at the
construction office of the site. An RF relay capable of performing step-stage data relay was additionally
connected to the boiler facilities at the construction site (approximately 300 m from the office). Eight cameras
were set around the construction site: two connected to the office via a wired LAN, two connected via a wireless
LAN, and four installed in the boiler facilities to communicate with the image processor at the office via the RF
relay. The images captured by these cameras have been archived in a video database for one and a half years.
This system serves three purposes: providing an understanding of the progress of construction, communicating
with workers on site and with designers at the architects offices (especially with those involved in the
construction), and monitoring the weather, especially on windy days.
266

Office
Construction
Site
Camera1
Wired
LAN
Encrypted
Communications
device
Image
ProcessingUnit
Camera2
Camera4
RFLAN
Relay
Camera3
Camera5 Camera6
RFLAN
RFLAN
Relay
RFLAN
Relay
N
a
r
r
o
w
b
a
n
d
N
e
t
w
o
r
k
Cloud Server
Monitoring
Application

Fig. 4 Configuration of the remote monitoring system installed at a construction site

3. COMMUNICATION-SUPPORT SYSTEM BY AUGMENTED-REALITY
TECHNOLOGY
To support information sharing with workers and supervisors, the developed large-scale video-monitoring
system provides functions named space tag and PTZ AR. The system can handle IP-cameras with a function
for controlling pan-tilt-zoom (PTZ) parameters. Controlling the PTZ parameters makes it possible to capture
images in various directions and various zoom factors. When an image taken by a camera is viewed, the system
searches for information of interest in the field of view of the camera and displays it on the image in an
overlapping format.

3.1 Text-based information sharing
Using augmented reality (AR) for information sharing has been proposed as a development target for mobile
devices. For example, Sekai Camera (Sekai Camera) is a social-tagging service, which displays information
on a video image of the users surroundings. AR4BC (C. Woodward, 2010) is a system for displaying
BIM(Building Information Modeling) information according to the progress of a construction site. The
developed large-scale video-monitoring system utilizes an AR technology for the PTZ cameras. Space tag is a
application that associates text data with PTZ parameters. An image of space tag in operation is shown in Fig. 5.
When the user (supervisor or on-site worker) inputs a text message into this application, the text data and PTZ
parameters are sent to the retrieval server. When an image captured by a camera is viewed, the text in the field of
view of the camera is obtained by searching the retrieval server, and it is displayed by overlapping it on the
image as tags.
Locations of the tags to be displayed are calculated by the following formulas.
267

(1)
where Rp is the pan parameter registered in the retrieval server, p is the pan parameter obtained from a camera,
Ah is the horizontal angle of view calculated from a zoom value and the specification of the camera, Rt is the tilt
parameter registered in the retrieval server, t is the tilt parameter obtained from the camera, and Av it the vertical
angle of view calculated from the zoom value and the specification of the camera. A message list can thus be
created from the retrieval server.
The position (x,y) of the message to be displayed on the viewed image is calculated from Eq. (2), where w is the
width of the image, h is the height, and p and t are the pan and tilt parameters associated with the message.

(2)

Fig. 5 Image of Space Tag application in operation
If the user wants to view a real-time image at the pan and tilt angles associated with a message, the camera turns
in that pan-and-tilt direction by selecting the text on the list.
The space-tag function is an effective way to share information through the large-scale monitoring system. For
example, it makes it possible to share information concerning progress of a construction project between the
construction site and the architectural design office. Moreover, utilizing the system as a means for providing
technical guidance is expected to contribute to improving workers safety, construction quality, and work
efficiency.
2 2
h
p
h
A
p R
A
p + < <
2 2
v
t
v
A
t R
A
t + < <
268

3.2 CG-based information sharing
A function named PTZ AR, which superimposes a virtual object (such as computer graphics) on the
PTZ-camera images, was developed and incorporated into the large-scale remote monitoring system. Some
examples of PTZ AR in action are shown in Fig. 6. 3D-CAD drawings of a seawater-desalination plant
overlapping on real images are shown in Figs. 6(a) and (b), and 3D measurement data superimposed on images
of a laboratory are shown in Figs. 6(c) and (d).
PTZ AR shows images as virtual objects on the images obtained from the large-scale monitoring system. This
function makes it possible to display a plant model of the planned construction at the construction site. By
superimposing a CAD image extracted from the construction plan onto an actual camera image, it is possible to
verify that the construction is done as planned or there are no differences between the actual construction and the
CAD model.
In the latest implementation of PTZ AR, the 3D coordinates and directions in CAD space of the PTZ cameras are
given by the user. When users change a PTZ value, the virtual object is superimposed on the camera image
according to the PTZ value and the coordinates. For this reason, the two following problems arise, and their
solutions must be devised in future works. First, because the CG object is superimposed on the camera image,
the camera image is obscured by the CG image. As a result, even if something (like a person) exists in front of
the CG object, that object is obscured by the CG object. Second, misregistration between the camera image
and the CG image can occur because the initial position and direction of camera are given by users manually.

(a)

(b)

(c)

(d)
Fig. 6 PTZ AR: (a) and (b) 3D-CAD drawings overlapping real images; (c) and (d) point-cloud data
overlapping images of a laboratory.

269

4. CONCLUDING REMARKS
A large-scale video-surveillance system for monitoring the progress of construction and assessing safety and
security was developed. This system can manage a large number of IP-cameras via a image-processing PC set at
a construction site and control data transmission dynamically on the basis of motion detection by the cameras. It
was experimentally demonstrated that the system can reduce output traffic data rate by 84% compared to input
traffic. In addition, the system supports communication with workers at overseas construction sites and
supervisors at head offices by displaying text/image information overlapping camera images. A field
demonstration was conducted at an actual construction site of a thermal power plant in Japan. Seven cameras
were installed in the plant, and the system has been successfully operating for 24 hours a day for the past six
months.

5. REFERENCES
Y. Yashiro and T. Tsubokura, Large-Scale Monitoring System Using Cloud Computing Technology for
Construction Sites, 14
th
International Conference on Computing in Civil and Building Engineering, 2012
E. Komagal, A. Vinodhini, Archana and Bricilla, Real time Background Subtraction Techniques for Detection of
Moving Objects in Video Surveillance System, 2012 IEEE International Conference on Computing,
Communication and Applications (ICCCA)
P. Viola nad M. J. Jones, Robust Real-Time Face Detection, International Jurnal of Computer Vision Vol.57 No 2,
137-154, 2004
D. Matsubara and A. Hiroike (2009), High-Speed Similarity-Based Image Retrieval with Data-Alignment
Optimization Using Self-Organization Algorithm, 11
th
IEEE International Symposium on Multimedia, pp.
312-317
Sekai Camera: < http://www.tonchidot.com/>
C. Woodward et al. Mixed Reality for Mobile Construction Site Visualization and Communication, 10
th

International Conference on Construction Application of Virtual Reality, 2010

270

IMPROVING THE BENEFIT OF VIRTUAL REALITY SESSION
DOCUMENTATION THROUGH AUGMENTED REALITY
Martin Heinig, Axel Friedewald & Hermann Ldding
Hamburg University of Technology
ABSTRACT: The increasing complexity in manufacturing is one of the biggest challenges for todays production.
Especially the assembly planning for one-of-a-kind products causes the necessity for an efficient information
support of the involved persons. Virtual tools such as Virtual Reality (VR) partly support the product development
process with applications, e.g. design reviews, the animation of assembly steps or necessary changes in the
assembly planning due to delayed parts. Nevertheless the operation of such a VR session is often not trivial and
usually requires a lot of time, expert knowledge and customized methods.
The concept of the Virtual Production Composer (VR Composer) is focused on the session preparation and the
session handling. An automatic approach based on a filter mechanism reduces the preparation time and errors
caused by manual work steps. For the session handling, several customized methods exist for specific assembly
investigations and a new interaction device was tethered for better usability.
However, the documentation of the session results still is a common problem. Present approaches use screen shots,
virtual markers, written notes or audio records. Although the quintessence of the result may be maintained, the
documentation form implies suboptimal characteristics. In addition to the discontinuity of the information flow,
there is a loss of information if the result is converted to 2D screen shots or written notes. The formerly dynamic
session result becomes static and cannot be adapted to a new usage situation.
Therefore the paper proposes to visualize the VR session results with Augmented Reality (AR). The approach
extends the VR Composer to provide the same mechanisms of session preparation to an AR device. It avoids a
manual authoring of the provided information because the VR session result is stored as meta data which is used
for the AR content preparation.
KEYWORDS: Virtual Reality, Augmented Reality, Process Integration, Documentation, Information Visualization
1. INTRODUCTION
The planning of complex products is a demanding task. In the one-of-a-kind like production, where every
product is unique in planning and manufacturing, the processes of optimization and validation have to be carried
out respectively. This applies to construction processes, which are very intricate and of different variety
(Bargstdt 2010). Many restrictions have to be taken into account in every step of the development progress. For
this reason, the planning of products is nowadays often supported with virtual methods. Virtual models are
constructed in 3D CAD environments and enriched with information. This includes functional constraints like a
door radius or organizational considerations like scheduled dates. These so called 4D models are dynamic and
therefore able to show the progress of the 3D models during the planning process. They improve the
communication between planning and design phases and are used to resolve constructability problems, conduct
interference analysis, and to perform scheduling and hazard analysis (Goedert and Meadati 2008).
The current challenge is to provide a system with the relevant data at the right time. Whereas complex planning
tools have been developed in the past and are still improving, appropriate tools for the construction industry still
lack the ability to be incorporated within a larger data management frame and share data between two systems
automatically (Bargstdt 2010). The handling of the large amount of data requires appropriate concepts. In the
construction industry, the Building Information Model (BIM) is often in use. Goedert and Meadati state that it
has the potential of being a database with all necessary information (Goedert and Meadati 2008). In addition to
geometry details, other information can be attached to these objects including manufacturers, fire rating,
schedule and cost estimates.
The handling of the enriched digital representation of the product needs suitable visualization and interaction
methods. Therefore, digital tools such as Virtual Reality (VR) or Augmented Reality (AR) are available and
assist the user during the planning or construction phase. VR is a technology for reviewing digital models in an
immersive environment and offers the possibility to discuss them interdisciplinary. AR on the contrary offers the
271

possibility of a location-independent visualization, for example at the construction site. The real view of the user
is augmented with virtual objects. Although these technologies serve different purposes, it does not exclude that
they complement each other. The planning of an assembly sequence could be done in VR and later be shown
with AR at the construction site. However, easily and automatically providing and authoring augmented content
from VR scenarios to AR applications remains a challenge (Xu et al. 2008). Despite the wide spread adoption of
computing by architectural, engineering and construction (AEC) industries over the last thirty years, this digital
data continues to be abstracted to paper drawings for information transfer (McMeel and Armor 2011). In
consequence to this media disruption, the former digital data becomes static and not reusable.
The idea of this paper is to improve the VR session documentation by providing the results of the session to an
AR application. For this purpose, the VR Composer, an extension of a VR software, is adapted to offer the
existing functionality of VR session preparation to automatically prepare the AR content without explicit
authoring.

2. VIRTUAL TECHNOLOGIES
2.1 Virtual Reality Process
Virtual Reality assists the product development of complex products. It is used for design reviews and the
analysis of assembly feasibility or ergonomics. Every VR session requires a preparation of the 3D models as well
as the functions to be used. The effort can increase significantly, if the products are unique like in the
one-of-a-kind like industry. Each session needs a custom preparation, even if the kind of investigation remains
the same.
The process of a VR session can be divided into six phases (Nede et al. 2009-a). It starts with the definition of
objectives. After that, the user imports and prepares the necessary 3D data, prepares the VR scene and performs
the session. The last phase documents the results. Many authors state that the most important prerequisite of VR is
to integreate the VR process into the business process but that industry is facing various problems with it (Amditis
2006). A seamless integration of VR is still not achieved (Whler et al. 2010). The requirement is an explicit
definition of necessary input data for the VR process from the previous process and the return of the data as output
(Matysczok et al. 2008). Thus, for a seamless integration into the business process, the VR process needs two
interfaces, one at the first phase for input, one at the last phase for output. Fig. 1 shows the circumstances.
The input must contain the objective of the VR sub process. It defines the desired result of the session and can
include the necessary steps to achieve it. Friedewald introduced an approach to give this kind of information
directly to the VR software (Friedewald et al. 2012). Another important aspect is data. It is divided into geometry
and additional data, in the following called meta data. Meta data contains functional or organizational
information about the 3D model, e. g. assembly dates, construction states or weights. This kind of data is often
available from a PDM system or BIM. The necessary geometry data is converted from 3D CAD construction
data into a VR format. Overall, the input data needs to be correct and up-to-date with the construction process.
272

Fig. 1: The connection between the business process and the VR reference process
The output of the VR process contains the documentation of the session result. Possible data are the description
of the session result, additional or changed meta data or even changed geometry data. Although session
documentation exists, it is a problem to return the data from VR to the business process for further interactive
usage (Whler et al. 2010). So far, the efforts on how documentation can be conditioned for later digital
processing are not feasible in the authors opinion. This is explained in detail in the next section.

2.2 Virtual Reality session documentation
Despite the fact that VR session documentation is a crucial step in the VR-Process, it is often neglected.
Dangelmaier states that the problem is the support for the entire process of handling virtual prototypes because
support for documentation may exist as annotation or screenshot functions but is not systematic (Dangelmaier and
Haselberger 2008). Therefore, the generated VR based documentation is often static and in parallel administrated
text, which is not connected to PDM (Whler et al. 2010). In addition, a static documentation necessitates that the
author has precise knowledge about its future usage. To improve this, manipulated data from a source system and
derived results must be reusable for further processing (Symietz 2006).
There are three types of recipients of VR session results. The first one is the VR environment itself. If the
documentation stays within the VR environment, it is simply saved in a session file and can be easily reloaded
for further usage. In the other two cases, the results leave the VR environment and require a suitable
transportation form. One possibility is to give them back to the previous phase of the business process, where it
originally came from. Here, the new findings from the session must be saved properly and transported to the
planner or planning system. The second type of recipient is the next phase of the business process. In this case,
the results must be accessible for the following steps in the product life cycle, e. g. the workmen at the
construction site.
For understanding the manner of documenting a VR session, it is necessary to differentiate between the content
and the methods to share it. The content describes which information is represented. Depending on the kind of
session, this varies from a simple choice of the better variant in a design review to the planning of an assembly
sequence with animations. The methods to share the session results vary depending on the recipient. Popular
choices are screenshots, videos, lists of model parts as well as printed hints and annotations. The following table
shows the relation between the functional content and the methods to share it depending on the recipient.
273

Fig. 2: Documentation methods of VR session results
Figure 2 illustrates, that the documentation methods for making results available outside of VR converts
dynamic information into a static representation. This can be sufficient in many cases but it might be problematic
if a further processing or adaption of data is required. These methods only save the representation of the result;
they do not store any meta data necessary to change the representation of the result afterwards.
Several ideas exist to improve the session documentation. Hosseini and Georganas describe a flexible recording
of animations in VR (Hosseini and Georganas 2002). However these records can only be played in the specific
system they were made in and therefore lack portability. Another example is software from EsZett which
focusses on managing several media of the documentation like screenshots or videos and prepare them for return
into the planning systems of the company (Eszett 2012).
Although there are some approaches for improved session documentation, they do not form a dynamic
representation for later personalization and digital use. The approach in this paper shows how VR session results
can be stored independent of the media for later use, e.g. for visualization within Augmented Reality.

2.3 Augmented Reality
Augmented Reality enriches the real view of the user with virtual objects. The development of this technology
started many years ago and several research projects such as ARVIKA or AVILUS have shown the potential as
well as the technological feasibility for industrial use (Friedrich 2004), (Schreiber and Zimmermann 2011). AR
can visualize planning data such as assembly sequences or other meta information according to the current
vision of the user. The data can be static like annotations and videos, or dynamic if user interaction is possible.
Mobility and easy handling are major aspects for todays technology usage. AR fulfills these requirements
because AR capable mobile devices like smartphones or tablet computers are widely available providing an
intuitive handling via touch screen.
An AR system consists of a camera, a tracking system, a display device and a software which connects the
components to generate the AR scene. This paper focuses on the data to run an AR session. We will investigate
the necessary data flows into the AR process for the proposed concept. Similar to the input of a VR session, an
AR session also requires the geometries of the model parts. They are converted from their native CAD format
into a suitable AR format. To place model data correctly into the view of the user, a tracking mechanism is
needed.
In contrast to a VR session, data about the sequence of events as well as user interaction possibilities is necessary.
This kind of preparation is known as authoring. The author defines the visible content by assigning geometries,
meta data and functionality to a trackable target. In the moment the camera captures the target, the view of the
user is augmented. For dynamic contents, the required interaction options for the user must be implemented.
274

Several authoring approaches exist to generate AR content. A common method of AR software companies is to
provide an editor to specify the session configuration. In contrast, the framework Instant Reality offers to create
the authoring in a VR environment (Behr et al. 2008). Regardless of the approach used, authoring remains a
manual and time consuming process (Whler et al. 2010). Therefore, the concept of this paper suggests to
automatically derive the authoring from the VR session documentation.

3. SITUATION
Virtual tools can play an important role in the planning process of complex buildings. The digital data from such
tools is often converted to analog information. 2d construction drawings contain lots of different building
information and serve as a traditional mobile information carrier. They can be enriched with annotated 3D screen
shots. The assembly operator at the construction site usually utilizes a combination of a 3D overall plan and 2D
construction drawings. Peansupap and Walker describe that the availability and identification of information at
the point of activity (POA) on site is cited as key to improved problem-solving (Peansupap and Walker 2005),
yet drawings are primarily generalized and fragmented abstractions (McMeel and Armor 2011). Therefore, the
efficient and effective retrieval of onsite construction information from 2D drawings and their interpretation is
one of the major challenges faced by construction engineers in the exploration of building information (Yeh et al.
2010).
In practice, often many plans exist on the construction site at the same time, each containing information of
different levels of detail and serving different purposes. There is a general or coordination plan with all data
integrated and specific plans such as architecture floor plans, structural floor plans, plans of structural
reinforcements, piping plans and electrical plans each with just one key aspect and in general made for just one
category of worker. For example, general arrangement (GA) drawings lack assembly detail (AA). Furthermore,
scale and temporality play a part so that GA drawings often dominate the early construction programme, where
AA drawings have more relevance in the later stages of construction (McMeel and Armor 2011). Especially for
the general plan, many engineers have highlighted the cumbersome and inefficient nature of the exploration and
selection of building information from the large format 2D drawings at the construction site (Yeh et al. 2010).
The problem is that either too much information is in the available plan or necessary information is missing.
Also data inconsistency is a challenge because different plans with redundant information or different revisions
of the same plan exist at the same time. The principle Single Source of Truth is violated. Yeh sees three main
problems with current practices of using 2D drawings to explore building information (Yeh et al. 2010):
- 2D drawings have poor portability and maneuverability and a large amount and range of information.
- the dimensional limitation of representation on 2D drawings presents difficulties for representing spatial
relationships between building elements in a 3D environment.
- users encounter browsing and readability problems.
An essential benefit of virtual planning are interdisciplinary reviews. People discuss the state of planning,
annotate necessary information and will probably change the planning if problems occur. This additional
information needs to be available at the construction site as well. Virtual Reality is a good reviewing tool in this
context. As it is common to do the planning of construction in 3D CAD, 3D geometries are available for all parts
and only need to be converted to a VR format. Possible documentation forms are screenshots, paper based lists,
audio recordings, videos or notes (see chapter 2.2). Although some of the documentation can be included directly
in the 2D plans, the representation is suboptimal and in any way, the former digital content becomes analog.
Another problem is that documentation is mostly generated according to the push principle. The author
decides which information is needed at the construction site and correspondingly provides them in a static way.
He guesses the necessary information for the assembly operator in advance and generates the plans. If the
content is not sufficient, he generates new plans later on which causes additional effort and time delays. The risk
is that unnecessary information is included while crucial information is not. Too much information reduces the
clarity of the documentation. The case that important information is missing is even more worse. This happens if
necessary perspectives are not present in the 3D screen shots. Besides that, documentation can target different
types of receivers which could be unknown at the time of creation. This makes it difficult to choose and prepare
the right content.
275

This situation illustrates the drawbacks of conventional documentation techniques for building sites. Therefore
the following requirements arise: Content should stay digital. The information should be provided via pull
principle so that the workman can select the information he needs. This requires suitable interaction
possibilities with the displaying device. The device must be portable for mobile usage at the construction site and
be able to show 3D geometries.
A technology like AR can fulfill these requirements. It offers an extension to the paper plans and can be used on
the construction site. If a paper plan does not contain all necessary information, the missing information can be
chosen by the workman instead. Extra information in the 2D drawings could be annotations in AR. A benefit is
created if the 3D model representation is available in addition to 2D drawings. Knowingly omitted information,
e.g. ignored parts in a 3D plan could be provided later on.
The session result, if taken directly from VR without media disruption is already in a digital and dynamic format.
Nevertheless, this solution will only be suitable if there is no or only little additional effort for authoring.
Because the required information cannot be foreseen, an automatic authoring procedure is the preferred solution.
The idea is to derive the authoring directly from the VR content. The following chapter explains the
implementation of this approach.

4. CONCEPT
4.1 VR Composer
The VR Composer is a utility for a VR software. According to the VR process in chapter 2.1, the VR Composer is
involved in the session preparation, session execution and session documentation. It provides easy access to
necessary functions of the software (Nede et al. 2009-b) and offers an automatic preparation of the VR scene as
well as manipulation possibilities on the VR model during the session (Ldding et al. 2011-a). Therefore, the VR
Composer imports and attaches meta data at run time to the geometry objects. The meta data can be provided from
a PDM system or a BIM-database and requires a suitable interface and data access concepts. However, the efficient
information flow between such systems is still an issue (Whler et al. 2010).
For the handling of the model parts, the user can select geometric objects in the scene via filter mechanism. The
Composer offers different actions for the computation of a filter result. For example, the action load imports an
object into the scene and the action collision detection prepares the object for collision detection. The
combination of filters and actions is a template and defines the action to be applied on a filter result. Figure 3
illustrates the functionality of the VR Composer.

Fig. 3: The VR Composer in the VR reference process
276

The VR Composer automatically prepares the VR session. It uses filters to determine the necessary scope of the
model and loads it into the scene. Possible parameters include the model section, part categories or assembly dates.
During the session, the VR Composer assists the user. Predefined execution steps can be selected one after another
which is useful for validation tasks (Friedewald et al. 2012). A session description as well as instructions can be
included and later displayed to the user in the immersive environment. In addition, the parts for investigation can
be colored or preselected. It is also possible to define camera views or cut planes and set them automatically.
At the end of the session the results are documented. If a change is made or additional information is added into the
scene, the VR Composer saves them back as meta data. This includes annotations or a change in the assembly
sequence. In consequence, the documentation is not directly available as a screenshot or a list but it stays in a
dynamic form. The aspect of how the documentation from the VR session can be further processed is explained in
the next chapter.

4.2 AR Extension
The mechanisms of the VR Composer are modified to show the VR session result in AR. The VR documentation
replaces the AR authoring because the necessary configuration is automatically derived from meta data. Since
VR scenarios are not designed for AR environments, some relevant parameters have to be adjusted for AR
applications (Xu et al. 2008). The extension to the VR Composer solves this issue and provides the VR session
results in AR. The interface for the AR software allows to use the existing mechanisms of the Composer for
session preparation (Ldding et al. 2011-b). This enables to prepare the AR content automatically without
additional authoring.
As VR and AR session preparation are slightly different, the VR-Composer is adapted to convert the description
of a VR scene to an AR scene. The prerequisites of the VR usage also apply for AR. The 3D CAD data has to be
converted to a suitable format and meta data must be available through an interface. The following figure
illustrates the concept:

Fig. 4: Concept of using VR session results for AR
For the preparation of the VR scene, the composer needs to know about the session details such as necessary
model parts or camera positions. This is defined via filters and position parameters of the camera. For the session
execution, the purpose of the session, the parts in scope and annotations are needed.
Contrary to VR sessions, the tracking gives the information of the scenes perspective in AR. Therefore,
information about the position and view direction of the user is required. AR also impacts the filter mechanism.
The planning of an assembly sequence in VR necessitates for example, that the current state of the model is
277

filtered on the basis of building state information and that the parts to assemble are selected by their assembly
dates. As AR is primarily used on the construction site, usually the real model state is displayed and the first
filter step is not necessary. The other filter step must be carried out to select the geometries and load them into
the AR scene.
It is often helpful to add additional information to an AR session. While it is no problem to add augmented
content such as annotations, animations and videos to VR scenarios, it is difficult to directly employ these
enriched VR scenarios into AR (Xu et al. 2008). The key is to provide the necessary data. The VR Composer
offers to show animations similar to VR, because the animation paths are saved as meta data and therefore
connected to the geometry parts. If a change of planning is made in VR, for example a change in the assembly
sequence, this is updated in the meta data and therefore immediately available in AR. Audio records or videos
can be linked in the same way into the AR scene, if the AR device supports this kind of media.
Additional content and dynamic model behavior require suitable interaction possibilities. They play an important
role during the AR session to enable information access via pull principle. The interaction depends on the
capabilities of the AR device. In this concept, the assumed device is a tablet computer, but it is also possible to
use AR glasses with a separate controlling device. To achieve interaction without respectively changing the code
of the AR application, the functions are parameterized and filled with data during run time. For example a touch
listener for displayed geometries can show the meta data of the part if activated. The user interface for
interaction is realized on a tablet computer. It is convenient to use a similar tablet control as for the VR
Composer (Ldding et al. 2011-a).

5. PROTOTYPICAL IMPLEMENTATION
5.1 Results
This chapter describes the prototypical implementation of using VR session results with AR. The scenario
chosen to demonstrate the use of the concept is that the assembly planning for a complex room is not longer
feasible due to delayed parts. Therefore the assembly sequence was changed and enriched with annotations in the
VR session. Instead of printing out screen shots, paper based lists or text documents, the results were written
back as meta data and the session configuration was saved. The VR software used is VDP from ICIDO (Icido
2012).
These session results are made available at the construction site with AR. For the implementation of the AR
application, the mobile SDK from Metaio (Metaio 2012) is used and the application is deployed on a tablet
computer. The geometries are converted from a 3D-CAD model into an interpretable format of the AR software.
It is necessary to define the tracking data in the Metaio SDK. For the prototype, the easiest method, simple
marker tracking, was chosen. Thus, a marker can be printed out and attached to a wall at the construction site.
The marker is related to a specific session documentation.
Once the AR software is started, the desired session is selected. The configuration file is read, the tablet control
module interprets it and executes the specified filters and actions. The filter configuration defines which
geometries are needed and the VR-Composer sends the filter result back to the tablet. Then, the geometries are
loaded into the scene. This means, that for an assembly sequence, all parts of the sequence are loaded, but only
the first one becomes visible, once the marker is tracked. After pressing a button, the next part is shown. The
configuration file also indicates the available functions of the control module. To review an assembly sequence,
back and forward buttons are needed. The controls for the filter mechanism are always available. Furthermore,
additional information or annotations for the geometry objects are loaded to allow that touching a geometry on
the screen shows annotations and other meta data about the object. Fig. 5 shows the AR prototype using the filter
mechanism to load user selected geometry into the scene.

278

Fig. 5: AR prototype using VR Composer functions on the building site
The implementation points out that the defined requirements from the third chapter are fulfilled: The session
result remains digital. Instead of paper based 2D drawings, 3D visualization is used. The information is
presented on a portable tablet computer usable on the construction site. The content is provided indirectly
because the filter mechanism selects the necessary information and loads it from the server. This is advantageous
as all information is available and can be personalized and provided on demand. Further additional information
or documents can be linked and loaded afterwards. Therefore, the pull principle is fulfilled and the approach is
much more flexible. The AR device offers the required interaction possibilities through the user interface of the
VR-Composer control application.

5.2 Limitations
The presented approach underlies some limitations, which are explained in the following. The proceeding can
only be used, if VR is already part of the business process and additional information is created during a VR
session. If session results are needed at the construction site, they can be visualized with AR. This is possible, if
model data is available and convertible to an AR format. If the model data is very large, limitations of the AR
device must be taken into account. Considerations apply if different hardware platforms are used. Diverse
display sizes may require an adaption of the representation so that the desired information fits on a small screen.
Another important aspect is the tracking to trigger the content visualization. Later on, markerless tracking may
be more convenient instead of marker based tracking which was used for simplicity reasons. Irrespective of the
tracking method the desired accuracy must be assured and can be critical, e.g. for the positioning of assembly
parts. Meta data such as the current state of work or assembly dates must be available and digitally accessible.
Such data is often included in the BIM. The technical details of AR for industrial use are targeted by many other
research projects and publications. The focus of this paper was to show how content can be automatically
transferred from VR to AR.

6. CONCLUSION
The introduced concept showed the potential of improving the VR session documentation through AR. The
current situation contains deficits, if information should be brought outside of the VR environment, e.g. at the
construction site. The reason is that due to media disruption, digital content gets analog and therefore becomes
static. A possible solution to overcome this situation is the use of AR to show the results and provide them via
pull principle. Instead of composing the content in advance by guessing the relevant information, the person at
the construction site can choose the necessary data later on. To avoid manual authoring, the concept proposes to
derive the AR authoring automatically from the VR session result. This is achieved by using the existing VR
Composer functions. The tablet control of the VR Composer is embedded into the AR application. There is a
tendency of fusion between 3D-CAD, VR and PDM software, which may be capable of solving some problems
stated in this paper. The current development of new AR devices may further enhance the possibilities of this
technology and could lead to a closer embedment in production environments.
279

7. Acknowledgements
The results presented in this paper were founded by the German Government Federal Ministry of Economics and
Technology due to a decision of the German Bundestag.

8. REFERENCE
Amditis A. (2006). VR and Virtual Environments: Bringing Together Research, Industry and End Users The
INTUITION Vision, Virtual Reality und Augmented Reality zum Planen, Testen und Betreiben technischer
Systeme, Proceedings of the 9th IFF-Wissenschaftstage, Magdeburg, 21-27.
Bargstdt H. (2010). How Virtual Reality for Construction Processes advances in Germany, Proceedings of the
10
th
International Conference on Construction Applications of Virtual Reality (CONVR 2010), Japan, 25-32.
Behr J., Bockholt U., Stricker D. (2008). Instant-Reality Mixed Reality Technologien im Produktlebenszyklus,
Augmented und Virtual Reality in der Produktentstehung, Proceedings of the 7th Paderborner Workshop,
Paderborn, 2008, S. 17 27.
Dangelmaier M. and Haselberger F. (2008). Immersive virtuelle Begehung und Inspektion, Augmented und
Virtual Reality in der Produktentstehung, Proceedings of the 7th Paderborner Workshop, Paderborn, 127-140.
Eszett (2012). VR meets Engineering, eSZett GmbH & Co. KG, http://www.eszett-vr.de/, Duisburg
Friedewald A., Heinig M., Schfer C., Ldding H. (2012). Produktivittssteigerung von virtuellen
Absicherungsauftrgen, Digitales Engineering zum Planen, Testen und Betreiben technischer Systeme,
Proceedings of the 15th IFF-Wissenschaftstage, Magdeburg
Friedrich, W. (2004). ARVIKA, Augmented Reality fr Entwicklung, Produktion und Service, Erlangen.
Goedert J.D. and Meadati P. (2008). Integrating construction process documentation into building information
modeling, Journal of Construction Engineering and Management, Vol. 134, No. 7, 509-516.
Hosseini M. and Georganas N. (2002). MPEG-4 Based Recording and Replay of Collaborative Virtual Reality
Sessions, Proceedings of the IEEE Virtual Reality Conference, Orlando, 271.
Icido (2012). IC.IDO The Visual Decision Company, ICIDO GmbH, http://www.icido.de/, Stuttgart.
Ldding H., Friedewald A., Heinig M. and Schleusener S. (2011-a). Virtual Reality supported Assembly
Planning in the Shipbuilding Industry, Journal of Ship Production and Design, Vol. 27, 146-152.
Ldding H.; Friedewald A.; Heinig M. (2011-b). Untersttzung von Montageaufgaben in der Unikatproduktion
mit dem ViP-Toolset, Proceedings of the 3th Interdisciplinary Workshop Maritime Systeme, Hamburg, 1-11.
Matysczok C., Frnd J. and Meyer D. (2008). VR, AR Im Spannungsfeld zwischen Forschung, Markt und
Kundenanforderungen, Augmented und Virtual Reality in der Produktentstehung, Proceedings of the 7th
Paderborner Workshop, Paderborn, 275-288.
McMeel D. and Armor R. (2011). Construction Aids: Augmented Information Delivery, Proceedings of the 11
th

International Conference on Construction Applications of Virtual Reality 2011 (CONVR 2011), Weimar,
392-402.
Metaio (2012). Metaio GmbH, http://www.metaio.com, Mnchen.
Nede C., Friedewald A., Schfer C., Schleusener S. (2009-a). Deploying Virtual Reality (VR) Technology in
Shipbuilding by using a Construction Kit for VR Applications Focused on the Business Process, 8
th
Conf. on
Computing and IT Applications in the Maritime Industries (COMPIT 2009), Budapest, 271-286.
Nede C., Friedewald A., Schfer C., Schleusener S. (2009-b). Virtual Production Workflow Composer - ein
Werkzeug zur Integration von Virtual Reality in betriebliche Prozesse, Proceedings of Go-3D, Go for
Innovations, Rostock, 27-38.
280

Peansupap D. and Walker P. (2005). Factors Enabling Information and Communication Technology, Diffusion
and Actual Implementation in Construction Organisations, ITcon, Vol. 10, 193-218.
Schreiber, W. and Zimmermann, P. (2011). Virtuelle Techniken im industriellen Umfeld, Das AVILUS-Projekt
Technologien und Anwendungen, Heidelberg.
Symietz M. (2006). Virtuelle Technologien im Produktprozess in Virtual Reality und Augmented Reality zum
Planen, Testen und Betreiben technischer Systeme, Proceedings of the 9th IFF-Wissenschaftstage, Magdeburg,
43-46.
Whler T., Rothenburg U., Strauchmann M., Mader S. and Haupt M. (2010). Konzepte und Implikationen zur
Integration von VR-, CAX- und PDM-Systemen fr die funktionale Absicherung von Montage-/
Demontage-Operationen, Proceedings of the 13. IFF-Wissenschaftstage, Magdeburg, 238-243.
Xu J., Tmer J., Mecke R. (2008). A Concept for Virtual Reality Based Authoring of Augmented Reality
Content, Augmented und Virtual Reality in der Produktentstehung, Proceedings of the 7th Paderborner
Workshop, Paderborn, 95-105.
Yeh K., Tsai M. and Shih K. (2010). A projective mobile device for presenting location-based building
information in a construction site, Proceedings of the 10
th
International Conference on Construction
Applications of Virtual Reality (CONVR 2010), Japan, 45-54.

281

MOBILE AUGMENTED REALITY: BRIDGING THE GAP BETWEEN REAL
AND VIRTUAL IN SUSTANINABLITY EDUCATION
Katsuhiko Muramoto & Vui Huynh
Pennsylvania State University, USA
ABSTRACT: As opposed to a purely synthetic computer generated world, Augmented Reality (AR), according to Wikipedia, is
a term for a live direct or indirect view of a physical real-world environment whose elements are augmented by virtual
computer-generated imagery. Being partly virtual and partly real, AR technology is capable of displaying relevant
information that is otherwise not available by simply observing the environment at the appropriate time and location, thus
affording the demonstration of a new kind of spatial relationships and the interactions of elements within a real-world
experience. This provides the potential for seamless interaction between the real and virtual world, creating media rich in-situ
learning environments in a 3D space. The main objective of this research is to investigate new opportunities for teaching
sustainable design brought about by mobile AR and to understand the extent to which it can enhance learning. This paper
describes the proof of concept stage of an AR application for teaching daylighting and reports lessons learned during the
design and development of the application, and the results of the user evaluation.
KEYWORDS: Augmented Reality, Visualization, Smartphones, Sustainability, Design Education
1. INTRODUCATION
The digital age has radically reconfigured the relationship between conception and production, creating a direct digital link
between what can be conceived and what can be built through file-to-factory processes of computer numerically controlled
(CNC) fabrication.
Branko Kolarevic (2003)
While the level of implementation varies among architectural schools, digital technology now pervades the education of
architects and has become an integrally embedded part of curriculum. The domination of paper based design (sketch and
drafting) and physical models has now been over taken by digital technologies. With sophisticated 3d modeling software,
students are now producing designs they could not have imagined before. Furthermore, as Branko Kolarevic argued, with aid
of the latest digital fabrication machines, they can even realize their complex, blobby forms. Indeed, the recent advances in
both digital and fabrication technology have allowed many architects to realize highly experimental designs which were
unimaginable and unobtainable a decade ago. Due to this seemingly smooth transition from virtual to real, some theorists
claim that digital technology is increasingly blurring the difference between them. Lured into visual simulations and believing
that the computer simulated environment is a faithful representation of how the material appears and functions in the real world,
many students lost touch with the very materials out of which their designs would have been built. They quite often fail to
recognize the close relationship between design and construction by confusing the virtual realities and their seductive
appearances with the real thing. Consequently, students seemed to have removed themselves from the vivid experience of the
lived world and the process of traditional making. Thus, paradoxically, one can argue such distance between real and virtual
is getting wider.
While the importance of the immediacy of the world-as-lived, and general problem of embodiment, corporeality and
concomitant situatedness discussed by the philosophical and phenomenological writings of Heidegger and Merleau-Ponty has
been acknowledged for some time, there has been minimal attempt to integrate this understanding into architectural education.
The two worlds do not necessarily have to be antagonistic, but can be complementary or bi-directional: as experience from the
real world can be used to enrich the purely synthetic virtual world, so too can abstract information/data from the virtual world
be used to enrich an existing real world experience. This research attempts to address this chasm by exploring the pedagogical
potential of augmented reality (AR) to enable interactions between real and virtual worlds for architectural design education
within the aforementioned theoretical context.
2. AUGMENTED REALITY
As opposed to a purely synthetic, computer generated virtual world, AR, like ancient mnemonic techniques, adds information
and meaning to a real object or place. Unlike virtual reality, augmented reality does not create a simulation of reality. Instead, it
takes a real object or space as the foundation and incorporates technologies that add contextual data to deepen a persons
282

understanding of it (Educause, 2005). Being partly virtual and partly real, AR technology is capable of displaying relevant
information at the appropriate time and location, thus affording the demonstration of spatial relationships and the interactions
of elements within a 3D space. This contextrelevant information that is otherwise not available by observing the environment,
offers many potential applications for the fields including aviation, education (Kerawalla et al., 2006), educational
exhibit/museum (de Feitas & Levene, 2005), entertainment, emergency management, tourism, manufacturing, military (Julier,
et al., 2000), and medicine (Henderson & Feiner, 2009). Numerous researchers have shown AR to be particularly useful for
teaching subject matter that students could not possibly experience first hand in the real world (Shelton & Hedley, 2002).
AR has also been studied in several different applications in the AEC industry (Behzadan & Kamat, 2007; Dunston and Wang,
2005; Dunston et al., 2003; Liarokapis et al., 2004; Schall et al., 2008; Shin et al., 2007 & 2009). Previous research on AR,
however, required a highly specialized, expensive and cumbersome head mounted display with other assorted equipment,
including GPS, laptop computer, video camera, etc. (Donathnet al., 2001; Feiner, MacIntyre, and Seligmann, 1993; Feiner et
al., 1997; Gervautz, 1997; Kuo, 2004). Much of the prior research identified major challenges related to inadequate computing
power and hardware, but was generally positive about the potential benefits AR might bring.
Steadily improving smartphones with significant processing power and graphics abilities with cameras, sophisticated GPS
systems (which use satellites to locate the phones), compasses (which tell the direction the phone faces), and accelerometers
(which relay the device's tilt) provide an inexpensive and versatile platform for AR applications. AR environments provide
significant benefits in the field of design education:
i. Visualization of abstract or invisible elements Site boundaries, structures, airflow diagrams, etc.
ii. Visualization of past and future environments Demolished buildings, proposed buildings, landscape in different seasons,
etc.
AR can overlay more than one piece of information (text, photos, graphs, etc.) and allow the relationship between objects and
their environment to be examined and studied in a more interactive manner (Azuma 1997; Furmanski et al. 2002). By creating
learning environments that are richly populated with relevant information, AR also provides the potential for producing vivid
experiences and meaningful interaction between the real and virtual world. It also realizes a fundamental concept of situated
learning scenario (Alessi & Trollip, 2001) that all learning is highly specific and best be studied in real world context.
Constructivism learning theorists, Duffy and Cunningham (1996) argued that when learning is removed from its context and
taught in an abstract and de-contextualized manner, the value and the relevance of the knowledge diminish drastically. Thus, by
turning place into an educational medium with an active field of information where a tactile and multimodal approach to
learning is encouraged, AR technology will provide powerful educational opportunities in architectural design. Furthermore,
an AR environment can make this place available for end user controls with anyone equipped with a smartphone. By being
involved in this learning environment, students take responsibility for thinking for and learning by themselves. As a result of
this independent learning process, they start to establish meaningful connections between various informations in both virtual
and real. However, according to the 2011 TechWatch Report on Augmented Reality for Smartphones, uptake of smartphone
based AR in education has been very modest so far and there are no examples of channels being created in existing AR
browsers such as Layar and Junaio that are specifically geared towards learning and teaching (Butchart, B., 2011). Thus, our
aim is to take advantage of already popular smartphones and turn them into learning devices.
2.1 AR Platforms
Based on our previous literature reviews of AR technology, we first experimented with two different kinds of smartphone AR
technologies in this proposal: GPS/compass based and Marker based (black and white squares that trigger augmented reality
elements). In the content of this research, however, a marker-based solution is not considered ideal due to the potential
difficulty of creating large markers, or difficulties of mobile device recognizing markers within a space. Thus, we first
examined two marker-less AR applications: Layar (http://www.layar.com/) and Junaio (www.junaio.com). Each application
has its own set of capabilities and requires different levels of technical knowledge to publish content and customize the
interface. Although Junaio appears to be the most advanced browser in terms of functionality, we eventually choose Layar for
its ease and slightly more sophisticated interface. Layar is an open source platform AR app developed by the Amsterdam based
SPRXMobile. It allows developers to build their own layers (layers are similar to plug-ins that allow you to customize your
computer) and run them in the Layar AR browser.
3. Implementation of the AR Application
For the proof of concept stage of this research in evaluating the potential of AR in architecture education and the feasibility of
current mobile tablet-based AR technology, we focused on how sustainability education for beginning design students can be
283

enhanced by this technology. This stage included the implementation of an AR application (app) and a survey study to obtain
user perceptions on the app and the potential of using AR in architecture learning. This topic of sustainability was chosen to
make students aware of the importance of the relationship between building and environment, especially less visible climatic
elements such as sun radiant energy, sun heat, etc. Additionally, we consider this to be a topic where we can take advantage of
visual and media content. We used the Stuckeman Family Building (SFB) at Penn State University, as the test case, and
developed several preliminary designs for the application along with initial storyboards. The current app consists of two parts:
concept and principle lessons on daylighting at the SFB Library and an interactive design game to show how design
modification affects performance.
3.1 Preliminary Design
We developed preliminary designs for the application along with several storyboards. The images below describe the working
of the first Layer application.

Fig. 1: SALA floaticon; Fig. 2: SALA Daylighting icon with options below; Fig. 3: App used with iPad
Once you start up the Layar app, the SALA icon will float over the camera image. When you tap the SALA Building icon,
options - Web, Video, Audio, Sustainable Features and Take Me There - will appear (Fig. 1). Select Sustainable Features, you
have an option of choosing Daylighting or Energy. The Daylighting (Fig.2) screen captures are shown here. You have
the option of going to the website, watching a video clip or listening to an audio clip describing more about your chosen topic.
Initially, we had hoped that the GPS device would be accurate enough to register and align virtual information with the location
for lessons, but after some testing, we discover that the margin of error in location (GPS/compass) based mobile AR technology
is about 10-30 feet. As a result, any AR application can at best indicate a point of interest (POI) in a general direction, and show
an approximate location. We also experienced that depth perception of the POI icons (icons located further from you should
look smaller) is hard achieve. As seen in Fig. 3, one sometimes has to deal with glare spots on the screen during outdoor use.
3.2 Current Application
Based on the above described implementation of the AR environment, we refined the storyboards and implemented it into our
new Layar application, this time focusing on the topic of daylight at the SFB Library. We also have structured the process of
learning into two steps: lessons on sustainable design principles for daylighting (daylight factor and light rays) and a design
simulation game where students will explore modifications to the existing window design. According to Bransford (2000), the
use of simulations may engage learners as active participants in their learning by focusing their attention on critical elements,
encouraging abstraction of common themes or principles, and evaluating their own progress toward understanding. In
addition to active learning (Prince, 2004), this simulation game helps students connect an isolated understanding of
fundamental concepts they learned in previous lessons, and allows them to make predictions on their design choice within a
real context. By implementing this selected scenario, not only we can demonstrate the feasibility of the proposed ideas of using
AR in sustainable design education, but also explore current limitations of the tablet-based AR technology.
The following describes the workflow of how this application may be used. When the user uses a smartphone to point to the
window, the SFB floaticon appears, indicating that there is information overlaid on that window (Fig. 4). When the user
284

touches on the icon, several options appear: General Information, Daylighting, Modify Design, and Window Design Strategies.
Below are the results of the choices.
3.2.1 General Information

Fig. 4: Screenshot with the SFB app floaticon; Fig. 5: General Information of the window over camera image.
When the user chooses the General Information option, the floaticon - the plan of the building with the library location as well
as the window dimensions - will be overlaid on the screen (Fig. 5). Since all of the information was linked to a remote data
server, additional options such as further specifications or a link to the manufacturers website can easily be added.
3.2.2 Daylight Factor Lesson
When the user selects the Learn about Daylighting option, both a Daylight Factor and a Light Rays floaticon appear (Fig.
6). By choosing Daylight Factor, a short video describing a concept of daylight factor, followed by daylight factor analysis
simulation will play. Ecotects simulation of a false color picture of the distribution of the daylight factor will be overlaid (Fig.
7). This simulation allows students to understand and estimate the physical amount of daylight available in the room, and to see
the relationship between window size and daylight factor. As an option the user can bring up a chart of recommended daylight
factors (upper left in Fig.7) for typical buildings so that they can convert the simulated result into daylight performance
measure.

Fig. 6: Screenshot with Daylight Factor and Light Rays floaticons; Fig. 7: Daylight simulation overlay
3.2.3 Light Rays Lesson
When the user chooses Light Rays, a short movie plays, showing how deep light rays reach into the room at the summer
solstice and winter solstice. Light rays in space are simulated and visualized in order to facilitate understanding of path and
performance (Fig. 8). There is also some explanation about the significance of light rays in designing windows and building
orientation. Following these demonstrations, a short lesson on the daylighting design strategy will be delivered. This will cover
285

basic rules of thumb in daylight design such as the relationship between window heights to the depth of daylight penetration in
room (Fig. 9). As a way of controlling the amount of sunlight admitted, a lesson on shading devices and design of effective
shading devices is given (Fig.10). Students also can change the existing shading device to a different type, such as vertical
shading, and then check how it can affect daylight performance. Through this process, students can learn the advantages and
disadvantages of each type of shading device as well as the best way to utilize them.

Fig. 8: Light Rays on summer solstice; Fig. 9: Daylight design strategies

Fig. 10: Building sections with shading devices; Fig. 11: Three Modify Design floaticons
3.2.4 Modify Design: Simulated Game
After lessons on daylight factors and light rays are completed, the user can play a simulated design game by choosing the
Modify Design option. The intention of this game is to offer the player opportunities to actively engage in learning through
doing. They first make a design decision and try to interpret the performance by relying on the knowledge they just acquired.
This learning-by-doing model using simulations allows students to test knowledge they gained in a context and helps students
make predictions on their decisions (Schank et al., 1994).
Three modification options are given in the game (Fig. 11); smaller window with same height, smaller window with lower
placement, and narrower window. Upon selection (Modify Design Option 1 was illustrated below), one has an option of
displaying daylight factor analysis of the chosen design (Fig.12). A video of the selected design on daylight analysis plays. The
user has to speculate if the daylight factor for the new design is suitable a library. The last video is on a comparison between the
original design and the new design given with some explanation for the differences. Fig. 14 shows a diagram comparing
illuminance in three different window locations.
286

Fig. 12: Rendering of the selected design (option 1); Fig. 13: Posing a question and option (yes or no); Fig. 14: Daylight
simulation overlay for the new design; Fig. 15: Comparisons between Three Modify Design options
4. Survey experiment and analysis of results
The last goal of this pilot study was to evaluate how effectively students can learn some basic concepts and principles of
daylighting design with the assistance of AR. First, a pre-survey was administered, then the app was introduced. Each student,
accompanied by the evaluator, spent approximately 15-20 minutes using the application in the Library and then completed the
post-survey. We used a typical five-level Likert-type item for both of our surveys.
4.1 The Participants and Procedure
The participants were twenty-four students; seventeen 5th year undergraduate and seven graduate architecture students.
Although this application eventually targets primarily entry-level design students, the upper-class students were intentionally
selected for the first survey because of the fact that they have studied sustainability in a more traditional classroom setting and
we considered this to be useful in evaluating the effectiveness of this app. The study was conducted in three parts: pre-survey,
observation of the usage of the application, and post-survey.
4.2 Data analysis and Observations
We are currently working on a qualitative analysis of the interview data collected from the study. Thus, the following is a very
brief and simple observation gained from the data collected. The pre-survey included 9 questions. Nineteen students had
experience with energy simulation software. Seventeen students were owners of smartphones, but only three had some
knowledge of and experience with mobile AR applications. Popular usages for smartphones were social networking and
playing games. Twenty out of 24 indicated that they had taken some kind of unconventional classes, mostly on-line classes.
Twenty-three of them answered that they are open to unconventional teaching methods, and we consider this group of students
to be a good sample for our research question on the potential application of AR in architecture education. Table 1-5 below
shows the results of the post-survey, which was administered immediately after the experiment was completed.
4.2.1 General Questions
As indicated in percentage in the charts below, the feedback from the participants has been generally positive. Q1 to Q3 show
the participants' responses related to overall impression of the application. They found the AR app to be very interesting
(average=4.4), helpful (average=4.1) and educational (average=4.1), and the potential of AR in architectural education to be
promising (average=4.4). No response to the general questions was lower than 4 on the 5-point scale.
Q1: Through your experiences with the AR application from this experiment, what do you think about the potential of AR in
architectural education?
Q2: Do you find this AR application interesting?
Q3: Do you find this AR application helpful?
287

Q4: Do you find this AR application educational? (Graph not shown here)
4.2.2 General Information about the Window
Responses to the general information about the window questions received the lowest scores (GIW1 average =3.9 and GIW2
average =3.7). Judging from some students comment that the overlay effect was not attractive enough, we speculate that the
interface did not meet satisfactory levels of usability by the time of the experiment. Additionally, we need to examine the kind
of information necessary for this section and how they can be delivered most effectively, i.e. text, icons, images, or 3D objects,
etc.

GIW1: This application was helpful in providing the general information about the window.
GIW2: The general information was described effectively by images and audios.
DF1: This application was helpful in explaining the concept of daylight factor.
DF2: Seeing daylight factor analysis in reality helped to understand more about related concepts.
4.2.3 Daylight Factor
Students found the daylight factor and light rays lessons to be useful (DF1 average=4.0, DF2 average=4.5, LR1 average=3,8
and LR2 average=3.9). As indicated in the pre-survey, 19 students had experience in several simulation programs such as
Climate Consultant or Ecotect Weather Tool, but their usages were limited to studio with a desktop computer, or at best limited
mobility with a laptop. These responses support our thesis that AR with its advantage in mobility and ability to bring
information to real sites can help students to access design information easily and learn it effectively. To confirm our intention,
one student noted that it was very helpful to learn about design and technical issues through real examples, rather than a
traditional slide show. He continued on to comment that with more development, this application could be very beneficial to
further knowledge for students.
4.2.4 Modify Design
The results for the modify design simulation game were good to fairly neutral (MD1 average =3.7, MD2 average =4.0 and
MD3 average=3.1). These relatively low scores were surprising because the extensive prior research on the merits of
simulation games in design field were very positive. We speculate that this was partially due to the time limitation we were able
288

to devote to developing this section. Due to our development time constraint, the interface was not refined enough. In addition,
the students were given only three design change options to explore, and then to visualize their choice. Furthermore, the
feedback on their design may not have been thorough, sophisticated and complex enough to meet the students expectations,
since many were accustomed to playing very complex video games. The MD3s lowest score (3.1) indicates that we failed to
illustrate the tradeoff between daylighting and thermal comfort. The level of comfort is hard to measure and evaluate
scientifically, and we need to investigate how this can be effectively measured and illustrated.

LR1: This application was helpful in showing different direct light rays in different times of the year.
LR2: This application was helpful in showing the importance of light rays in designing windows and shading devices.
MD1: This application was helpful in illustrating the relation between window dimensions and energy usage.
MD2: This application was helpful in illustrating the relation between window location and daylight factor.
MD3: This application was helpful in illustrating the tradeoff between daylighting and thermal comfort. (Graph not shown
here)
5. Conclusions
Overall, the application successfully demonstrates the potential of AR. It is especially encouraging that our experiment proved
that AR can convey dynamic spatiotemporal information related to sustainability education in a more effective way than static
resources such as drawings or text books. It also supported the situated cognition theory that all learning is highly specific and
best be studied in real world context (Lave & Wenger, 1991). Considering the positive feedback, future work could be
undertaken to improve the proof of concept application used here. However, the biggest hurdle seems to be the overlaying of
media content. Through our previous experiment, we found out that the margin of error in location (GPS/compass) based
mobile AR technology is about 30 feet. An additional problem we faced, as we started working inside, was that indoor spaces
became more problematic and challenging since it required more precise alignment and the GPS signal weakens drastically. In
order to accomplish what we had originally imagined in our initial storyboards, AR requires extremely accurate position and
orientation tracking in order to register and align virtual information with the physical objects (i.e. existing window with a
luminance factor overlay). Or, even better, information needs to be registered onto the physical object, instead of a
geographical location. Such an operation is very hard to achieve with this level of accuracy using current location based
tracking systems. There is some research on indoor tracking systems such as Wireless Local Area Networks (WLAN),
Ultra-Wide Band (UWB), and Indoor GPS based position tracking system (Khoury, 2009). We investigated the possibility of
utilizing the PlaceEngine (http://www.placeengine.com/en/), which uses WiFi information sent from WiFi access points.
According to their website, the PlaceEngine can improve the accuracy to 10 feet. However, at the time when we were
conducting the experiment the service at this point was limited to within Japan.
As this technology is in its infancy, it is important to understand its limitations and we had to work within these limitations. It is
also important to acknowledge that the tracking and registration accuracies required for mobile AR depend very much on the
purpose of information delivery. There are several instances within our experiment that are technically defined as true AR as
defined by Milgram and Kishino (1994): when the live camera image was augmented by graphic image of General
Information, Daylight Factor or Light Rays icons. Those POIs are describing something general and do not require
accurate tracking and registration, and worked fine. The rest is defined as a variation of mixed reality, or better described as
situated simulation - a non-mixed clean screen solution, as defined by Gunner Liestl. It is a scenario when the users visual
perspective in the real physical environment is almost identical to the visual perspective of the virtual content displayed on the
289

screen (Liestl, G., 2011). Thus, this technique of situated simulation, as indicated by the survey, is capable of delivering
multimodal information of a specific place as effective as true AR.
In relation to further work, there are some obvious ways in which this application can be improved. The first is to refine the
interface design. It may require several iterations to refine the graphics, especially of floaticons, and adjust the sizes within the
screen in order to create the desired effect. No doubt there will be further technology developments that can allow exact
alignment of the physical objects and virtual information, hence the possibility that more complex and interactive information
delivery may become possible soon. However, instead of relying solely on technology, we have to further experiment with how
to render different combinations of text, icons, images, and 3D objects so that we can deliver relevant information in several
different modal forms to the learner in a productive and meaningful manner. Additionally, what remains crucial is reflective
examination of the relationship between technologies and the body, beyond the effects upon the learner. As media theorist
Roger Silverstone suggests, we must recognize that our experience as embodied subjects is in part a condition of the
technologies we use and the spaces they help to configure (Silverstone, R, 1992). Any technology is non-neutral, as it always
transforms our experience. Technology also is both context dependent and medium specific. As we begin to understand how
different kinds of AR might produce different user experiences that differ from other kinds of technological mediation, we need
to systematically examine AR from this theoretical perspective.
One student responded in the post-survey stating that it is an excellent application for basic knowledge of learning, but when it
comes to actual concepts of items, a classroom is better suited. AR in education cannot answer basic questions a student may
have like a classroom setting. This comment reminded us that we are not seeking to replace the traditional mode of education
with this AR application, but to rather to augment it. Thus, it is important that we need to develop it further in concert with other
traditional classes students take on sustainable design education.
6. Acknowledgement
This research was partially funded by the Raymond A. Bowers Program for Excellence in Design and Construction of the Built
Environment, Pennsylvania State University (PSU). Credit should be given to the other original team members of the proposal,
John Messiner and Steve Ayer at the PSU Architectural Engineering Department, and Tim Murtha at Department of Landscape
Architecture, the Stuckeman School, PSU.
7. References
Alessi S. M. & Trollip, S.R. (2001). Multimedia for learning: Methods and development, Boston: Allyn & Bacon.
Behzadan, A., and Kamat, V. R. (2007). Georeferenced registration of construction graphics in mobile outdoor augmented
reality. Journal of Computing in Civil Engineering, 21(4), 247 - 258
Butchart, B., (2011). Augmented Reality for Smartphones: A Guide for developers and content publishers, TechWatch
(Technology and Standards Watch) report, JISC Observatory, http://observatory.jisc.ac.uk/docs/AR_Smartphones.pdf
Donath, D., Beetz, J., Grether, K., Petzold, F. and Seichter, H. (2001). Augmented Reality Techniques for Design and
Revitalisation in Existing Built Environments, Proceedings of the ACCADIA 2001, pp 322-399.
de Feitas S, Levene M (2005). Wearable and mobile devices for informal learning, Encyclopedia of human computer
interaction. Information Science Publishing. 12
Duffy, T. M., & Cunningham, D. J. (1996). Constructivism: Implications for the design and delivery of instruction. In D. H.
Jonassen (Ed.), Handbook of research for educational communications and technology, New York: Macmillan. pp. 170-198.
Dunston, P., and Wang, X. (2005). Mixed Reality-Based Visualization Interfaces for Architecture, Engineering, and
Construction Industry. Journal of Construction Engineering and Management, 131(12), 1301-1309.
Dunston, P., Wang, X., Billinghurst, M., and Hampson, B. (2003). Mixed Reality benefits for design perception. NIST
Special Publication SP, pp. 191-196.
Feiner, S., MacIntyre, B. and Seligmann, D. (2003). Knowledge-based augmented reality. Communications of the ACM, 36(7):
pp.5262.
Feiner, S., MacIntyre, B., Ho llerer, T., and Webster,A. (1997). A touring machine: Prototyping 3D mobile augmented reality
290

systems for exploring the urban environment. In Proc. ISWC 97 (First Int. Symposium on Wearable Computers), pp.7481.
Gervautz, M. and Szalavari, Z. (1997). The personal interaction panel - a two-handed interface for augmented reality.
Computer Graphics Forum, 16(3), pp. 335-346.
Henderson S & Feiner S (2009). Evaluating the Benefits of Augmented Reality for Task Localization in Maintenance of an
Armored Personnel Carrier Turret, Proceeding of IEEE International Symposium on Mixed and Augmented Reality (ISMAR
'09), pp. 135 144
Julier, S., et al. (2000). BARS: Battlefield Augmented Reality System, NATO Symposium on Information Processing
Techniques for Military Systems
Kerawalla et al. (2006). Making it Real: Exploring the potential of augmented reality for teaching primary school science,
Virtual Reality. pp. 163174
Kolarevic, Branko. (2003). Digital Production/Fabrication Digital Technology & Architecture White Paper. Ed. Bermudez,
Julio and Klinger, Kevin. Indianapolis, IN: Association for Computer Aided Design in Architecture, 2003: 4
Kuo C., L. H., Yang-Ting Shen, Jeng, T. (2004). Mobile Augmented Reality for Spatial Information Exploration, Proceedings
of the CAADRIA'04, pp 891-900.
Khoury, K. M. (2009). Context-Aware Information Access and Retrieval for Rapid On-Site Decision Making in Construction,
Inspection and Maintenance of Constructed Facilities, Ph.D. Dissertation.
Lave, L. and Wenger, E. (1991). Situated Learning: Legitimate peripheral participation. Cambridge, UK: Press Syndicate of
the University of Cambridge
Liarokapis, F., Mourkoussis, N., White, M., Darcy, J., Sifniotis, M., Panos, P., Basu, A., and Lister, P. F. (2004). Web3D and
augmented reality to support engineering education. World Transactions on Engineering and Technology Education, 3(1), 11
- 14.
Liestl, G., (2011). Chapter 14: Situated Simulations Between Virtual Reality and Mobile Augmented Reality: Designing a
Narrative Space, Handbook of Augmented Reality, Borko Furht (Ed), New York: Springer-Verlag, pp. 309-319
Milgram P. and Kishino F., (1994). A taxonomy of mixed reality visual displays, IEICE Trans. Inf. & Sys., vol. E77-D, no.12,
pp.1321- 1329
Prince, M. (2004). Does active learning work? A review of the research. Journal of Engineering Education, 93(3), pp.
223231.
Schank, R. C. (1994). Goal-based scenarios: A radical look at education. Journal of the Learning Sciences, 3, pp. 429453.
Schall, G., Mendez, E., Kruijff, E., Veas, E., Junghanns, S., Reitinger, B., and Schmalstieg, D. (2009). Handheld augmented
reality for underground infrastructure visualization. Personal Ubiquitous Computing, 13(4), pp. 281291
Seely Brown, J., Collins, A., & Duguid, P. (1989). Situated Cognition and the Culture of Learning Institute for Research on
Learning, Technical Report 88-0008, Palo Alto, California
Shelton B, Hedley N (2002). Exploring a cognitive basis for learning spatial relationship with augmented reality, Tech Inst
Cogn Learning, pp. 323357
Silverstone, R. and Hirsch, E. (Eds.) (1992). Consuming Technologies: media and information in domestic spaces, London:
Routledge. p79
Shin, D. H., Jung, W., and Dunston, P. S. (2007). Large Scale Calibration For Augmented Reality On Construction Sites.
Tai, Nan-Ching and Mehlika Inanici (2009). Depth Perception in Real and Pictorial Spaces: A Computational Framework to
Represent and Simulate Built Environments, Proceedings of the 14th International Conference on Computer Aided
Architectural Design Research in Asia/Yunlin (Taiwan), pp 543-552

291

A FRAMEWORK FOR UTILIZING CONTEXT-AWARE AUGMENTED
REALITY VISUALIZATION IN ENGINEERING EDUCATION
Amir H. Behzadan
University of Central Florida, Orlando, FL, USA
Vineet R. Kamat
University of Michigan, Ann Arbor, MI, USA
ABSTRACT: A growing number of construction engineering students complain about the lack of engagement and
interaction with the learning environment. A major factor contributing to this issue is that instructors in most
construction and civil engineering programs still rely on traditional teaching methods including the use of
chalkboard, printed handouts, and computer presentations that are often verbose but contain little visual
information. This is despite the fact that according to a number of recent research studies these teaching
techniques are considered outdated by a large group of new generation students who are visual learners or team
workers. At the same time, the influence of interactive social media in every aspect of daily life has changed
student perceptions and how they expect the instructional materials to be presented in a classroom setting. This
paper presents an innovative learning tool that uses remote videotaping, augmented reality (AR), and ultra-wide
band (UWB) localization to bring live videos of a real construction jobsite to the classroom, create an interactive
interface for students to watch the streaming video and select objects in the scenes, and visually deliver
location-aware instructional materials to students in real time. The objective of this research is to explore the
extent to which these technologies can be used to provide students with a learning environment that enables them
to develop an in-depth understanding of and gain hands-on experience with construction equipment and
operational issues, while experiencing teamwork and critical thinking in a collaborative setting.
KEYWORDS: Augmented Reality, Visualization, Construction, Education, Collaborative Learning.
1. INTRODUCTION
Engineering systems are rapidly growing in complexity, size, uncertainty, and interdisciplinary characteristics.
Construction and infrastructure systems are not exceptions as they mostly involve dynamic processes that occur in
constantly evolving environments and continuously interact with one another. The circumstances in which these
processes take place often become more complicated due to factors such as unforeseen site conditions, change
orders, and project-related disputes. Despite this, figures show that many construction and civil engineering
students have historically lacked a comprehensive knowledge of onsite construction tasks, and the dynamics and
complexities involved in a typical construction project (Arditi and Polat 2010). Yet, the curricula of most
construction and civil engineering programs do not fully convey the necessary knowledge and skills required to
effectively face and resolve these challenges. Arguably, the major emphasis on traditional information delivery
methods which include the use of chalkboard, handouts, and lecture-style presentations, coupled with focusing
mainly on simplistic approaches and unrealistic assumptions to formulate and solve complicated engineering
problems can potentially result in construction and civil engineering students falling behind in applying what they
learn in the classroom to the practical scenarios in the field (Tener 1996). In particular, a major concern shared by
many students is that little effort is made to educate them with the latest trends of emerging technologies and
advanced problem-solving tools. Figure 1 shows the results of a recent survey of 63 junior-level students of civil
and construction engineering. The results clearly indicated that a solid majority of students believed that compared
to students in other engineering disciplines, they were exposed to less technology advancements in the classroom.
While engineering students need to pick up the social and technical skills (e.g. critical thinking, decision-making,
collaboration, and leadership) they need to be competent in the digital age (Bowie 2010). Mills and Treagust
(2003) discussed that most students are graduating with descent knowledge of fundamental engineering science,
but they dont know how to apply that knowledge in practice. One of the fastest emerging technologies in
engineering education is visualization. According to the same student survey (Figure 1), more than 90% of those
responded indicated that they learn better when the instructor uses 3D representations or visualization to teach
engineering concepts and theories. Although instructional methods that take advantage of visualization techniques
have been around for several decades, many still rely on traditional media and tools. For example, students who
take a course in construction planning may use drawings, scheduling bar charts, sand table models, and more
recently, 3D CAD models. However, none of these techniques are capable of effectively conveying information on
292

every aspect of a project. For instance, 2D or 3D models do not reflect temporal progress, while scheduling bar
charts do not demonstrate corresponding spatial layout.

Fig. 1: A survey of undergraduate students revealed that a large percentage support the prospect of reforming
current instructional methods.
At a broader level, previous work has highlighted the benefits of technological advances to the learning process.
For example, Stratham and Torell (1996) reviewed the effectiveness of technology in the classroom. Their findings
indicated that when properly implemented, computer technology has a significant effect on student achievement,
stimulated increased instructor-student interaction, encouraged cooperative learning, collaboration,
problem-solving, and student inquiry skills. More recently, the introduction of computer technologies such as
computer-aided design (CAD) and building information modeling (BIM) has aimed to improve the quality of
learning in construction education. However, many students and new project personnel still fail to relate this
abstract knowledge to real problems in the field. While the construction industry lacks a steady supply of skilled
and experienced workforce, accumulating adequate skills and training to operate equipment and conduct
engineering tasks through traditional training methods takes significant time and has proven to be costly and
inefficient (AbouRizk and Sawhney 1994). Thus, a main challenge is to provide a timely and effective education to
the future workforce through integrating technology into core curricula and implementing it in a classroom setting,
rather than only providing devices and software (Ash 2011). At the same time, professional development and
collaboration between students and instructors needs to be encouraged and new forms of pedagogy and assessment
293

must be accordingly created. What is essential is to make technology a ubiquitous resource in the learning process,
personalize it based on students individual needs and learning styles, and then ask instructors to mentor and
facilitate the use of technology while students learn, direct, and collaborate in a technology-rich environment.
Integrating technology into the curriculum in todays schools should not mean finding ways that computers can
help instructors teach the same old topics in the same old ways. Instead, instructors must have the opportunity to
combine technology with emerging models of teaching and learning to transform education.
To address the abovementioned challenges, this paper reports on the latest findings of an ongoing research being
conducted by authors that aims to explore an innovative approach to integrate advanced visualization and location
tracking technologies into the teaching and learning experience of engineering students. In particular, the authors
have investigated the applicability of augmented reality (AR) visualization and sensing techniques to study, design,
implement, and evaluate a potentially transformative pedagogical paradigm for engineering process education to
impart the required training while providing flexibility, mobility, and ease of use. These enhancements will also
provide a more exciting and vivid experience for students and instructors while increasing the quality of learning
through hands-on interaction with construction equipment, tools, and processes.
2. AUGMENTED REALITY-BASED INFORMATION DELIVERY FRAMEWORK
In the past, several researchers have explored the feasibility of advanced visualization technologies in
architecture, engineering, and construction (AEC) (Nikolic et al. 2010, Messner et al. 2003, Psotka 1995) as well
as other scientific and engineering disciplines (Yair et al. 2001, Kaufmann et al. 2000, Ota et al. 1995). Most of
the previous work in this area has used the capabilities of virtual reality (VR) visualization paradigm. In order to
have realistic and convincing virtual scenes, it is necessary to first create 3D models of the facilities and
equipment. These models will then need to be properly rendered inside a virtual environment and displayed to
the user. The process of creating, rendering, and updating the computer generated graphics, which is often
referred to as CAD model engineering (Brooks 1999) may turn into a tedious if not an impossible task especially
when the size and complexity of the scene increase. An alternate approach to VR, is the AR visualization
paradigm in which views of the real world objects are used as readymade backdrops of the visualized scenes and
the modeler needs to only create and render those information that are necessary and of relevance to the user
(Behzadan and Kamat 2005). In addition, since real and virtual worlds coexist in AR visualization, human users
(from the real world) can more effectively interact with computer generated graphics (of the virtual world) on a
1:1 scale. Hence, in this research, AR was used as the main visualization technology to develop an interactive
learning environment for engineering students. Figure 1 shows the components of the developed AR-based
instructional information delivery framework.

Fig. 2: Components of the augmented reality-based information delivery system developed in this research.
In this framework, each student wears an AR head-mounted display (HMD) which enables viewing of
augmented information and graphics overlaid on the fiducial markers inside an AR Book. A fiducial marker is
simply a logo bounded by a thick black frame that is visually detectable by the camera mounted on the HMD and
294

can be mapped to certain 2D/3D graphics. When a marker is visible through the HMD, the corresponding
information is shown to the student. When the application is launched, real time video streams of a remote
construction jobsite captured by an IP-addressable camera are transmitted via the internet to the classroom, and
displayed on a large projection screen. The camera is equipped with a global positioning system (GPS) that
continuously transmits the position of the camera (i.e. longitude, latitude, and altitude). To track objects (e.g.
crane, excavator, hauler) in the video, they must be as well geo-referenced. This is done by capturing each
objects global position using a GPS device. Most modern construction equipment such as graders or dozers
takes advantage of onboard GPS units which can be used for this purpose. However, if necessary, additional GPS
units can be mounted by site personnel on any object of interest the position of which needs to be geo-referenced
in the video. The positional information is constantly sent to a computer. Knowing the global position of the
camera (viewpoint) and any object in the video, local position of that object inside the coordinate frame of the
projection screen (with the camera located at the center point of the screen) is then calculated using existing
geo-referencing methods such as the algorithm introduced by Vincenty (1975) and used by Behzadan and Kamat
(2007). For example, if the camera located at 81 20 59 W and 28 27 57 N (elevation 28 meters above mean
sea level), is capturing views of an object (e.g. construction equipment) located at 81 21 00 W and 28 28 00
N (elevation 26 meters above mean sea level), the planar distance and azimuth between the camera and the
object will be 96 meters and 343.84, respectively. Hence, assuming that X values indicate points to the right (+)
or to the left (-) of the cameras lens, Y values show elevation difference (positive if the object is located above
the cameras lens elevation, and negative otherwise), and Z axis runs from the cameras lens into the depth of the
field, the local position of the object in the cameras coordinate frame can be calculated as follows,

The perspective viewing properties of the camera (e.g. horizontal and vertical fields of view, near and far planes)
will then be used to construct a transformation matrix that converts the calculated coordinate values to
orthogonal coordinates inside the 2D frame of the projection screen. This contextual knowledge is essential for
real time detection and further interaction with the objects inside the live video streams. As this information is
constantly updated as new GPS data is received by the system, students are able to walk up to the projection
screen while carrying their AR Books and watch the video stream. As shown in Figure 2, each student wears a
smart tag on his or her index finger to have the ability to interact with the video scene and retrieve information
about the objects of interest. As this tagged finger moves across the screen, its local position inside the 2D
coordinate frame of the projection screen is captured by a network of data receivers installed in the room at
known positions. As shown in Figure 3, this calculation is based on the time-of-flight (TOF) concept. At least
three data receivers are needed in order to precisely calculate the tagged finger position through 3D triangulation
in the local coordinate of the projection screen. An additional (fourth) data receiver is also used to increase
accuracy and eliminate any potential errors in locating the position of the tag on the projection screen.

Fig. 3: 3D triangulation is used to determine the exact position of a mobile tag in real time.
meters 00 . 2 00 . 26 00 . 28
meters 72 . 26 ) 84 . 343 360 sin( 96
meters 21 . 92 ) 84 . 343 360 cos( 96
= =
= =
= =
Y
X
Z
295

When the students tagged finger moves close to an object in the video (i.e. linear distance between the tagged
finger and the position of a video object is less than a certain threshold), relevant visual information (e.g. 2D or
3D models, manufacturers data, loading charts, work schedule) will be displayed to the student through the
HMD on AR Book markers. Students can also move their AR Books around the room to form groups, virtually
manage a project, discuss a certain scenario, and explore alternative solutions in a collaborative setting, while
learning basic concepts such as equipment operations, jobsite safety, resource utilization, work sequencing, and
site layout.
3. IMPLEMENTATION AND PRELIMINARY RESULTS
The authors have successfully created a first generation AR Book in order to test if contextual graphical
information can be effectively presented to students in real time. The first generation AR Book developed in this
research, GEN-1, is a prototype of an AR-enhanced book that was implemented in Visual Studio .NET, using the
ARToolkit library. ARToolkit is one of the earliest object-oriented programming libraries that provide
functionalities and methods to track fiducial markers. Using the functionalities provided by ARToolkit, GEN-1
provides a fiducial marker-based AR interface for students and helps them gain a better understanding of
construction equipment by overlaying 3D models of construction machinery on AR markers. Figure 4 shows a
sample GEN-1 lesson on excavators. As shown in this Figure, each left hand page contains informative details
and illustrations about a certain piece of construction equipment (e.g. excavator), which can include a wide range
of information such as details about various parts of the equipment, history of the equipment, major components,
functions, and also its current manufacturers. Each left hand page is coupled with a corresponding right hand
page that contains several marker patterns.

Fig. 4: Samples pages from GEN-1 AR Book. Each left hand page is accompanied by a corresponding right hand
page which contains several marker patterns.
Figure 5 shows snapshots of two validation experiments. As shown in this Figure, GEN-1 uses a normal
textbook as the main interface. Students can turn the pages of the book, look at the pictures, and read the text
without any additional technology. However, when looking at the same pages through an AR display, students
will see 3D virtual models of the equipment discussed on the left hand page on top of the markers depicted on
the right hand page. The marker patterns are detectable by the designed AR visualization platform. Once a
marker is detected, a virtual model of construction equipment (previously assigned to that marker inside the AR
application) is displayed on the marker. The output can be either seen on a handheld display or a HMD or even
on the computer screen. The models appear attached to the real page so students can see the AR scene from any
perspective by moving themselves or the book. As stated earlier, the virtual content displayed to students can be
static or animated. This interface design supports collaborative learning as several students can look at the same
book through their AR displays and see the virtual models superimposed over the book pages from their own
viewpoint. Since they can see each other, the real world, and the virtual models all at the same time they can
easily communicate using normal face-to-face communication cues. All of the students using the AR Book
interface have their own independent view of the content so any number of people can view and interact with a
virtual model as easily as they could with a real object.
296

Fig. 5: Snapshots of proof-of-concept experiments conducted using GEN-1 AR Book. 3D virtual models of
construction equipment are displayed on right hand side pages.
Currently, the authors are working on the design and implementation of full-scale usability experiments in
classroom settings. Table 1 lists the major hardware components used to develop this large-scale test
environment.
Table 1: Major components used to develop a large-scale test environment.
Component Item Purpose
IP-accessible Camera StarDot NetCam XL (wireless) Capture and transmit real time videos of a
remote location.
Locationing System Ubisense Ultra-Wide Band (UWB)
platform (receivers and tags)
Locate the position of student tagged finger
on the projection screen.
Head Mounted Display (HMD) eMagin Z800 3DVisor Display graphical information to the student
through a pair of AR goggles.
HMD-Mounted Video Camera Microsoft LifeCam VX-5000 Capture views of student surroundings to be
used as the backdrop of the AR visualization.
The experiments will be conducted to evaluate student learning in the context of the developed methodology. As
part of these experiments, several direct and indirect measures such as performance on assignments,
end-of-course student assessments, pre- and post- surveys, and interviews will be deployed to collect and analyze
data about the impact of the newly developed tool on students perception and quality of learning. In order to
track the position of tagged fingers on the projection screen, an UWB real time locationing system (RTLS) is
used. The system includes 4 Series 7000 IP30 sensors (i.e. data receivers) and a number of smart tags that can be
attached to student fingers. Table 2 shows manufacturers specification of the UWB tracking system used in this
research.
Table 2: Manufacturers specifications of the UWB tracking system used in the large-scale test environment.
Accuracy 3D accuracy up to 6 inches at 95% confidence level (depending on specific
environment and system configuration)
Tag Update Rate Variable from 10 per second to 1 every 10 minutes
Aggregate Cell Update Rate 40 updates per second (maximum)
UWB Radio Transmission 6.0 GHz 8.5 GHz, -41.3 dBm/MHz, Center frequency: 7.02 GHz
Telemetry Radio Channel 2.4 GHz ISM band, Ubisense control protocol
Tag-Sensor Maximum Range > 160 meters at 40 updates per second in Open Field Measurement (optimally
aligned Slim Tag)
Suggested Sensor Spacing 100 feet to 200 feet at 40 updates per second in an indoor open environment (e.g.
warehouse)
297

4. SUMMARY AND CONCLUSIONS
The main motivation behind this research was the need to fundamentally revive the existing construction and
civil engineering curricula that still heavily rely on traditional instructional methods and mostly fall behind in
terms of integrating state-of-the-art information delivery technologies into the classroom. This paper reported on
the latest results of an ongoing project aimed to investigate the requirements and develop a real time interactive
visual information delivery framework for construction and civil engineering education. In this framework, real
time video streams of a remote construction jobsite are captured and transmitted via the internet to the classroom,
and displayed on a large projection screen. Each student can walk up to the screen while carrying an AR Book
and watch the video stream. Students have the ability to interact with the scene and retrieve information about
any object in the video by pointing directly to that object. A network of wireless receivers captures the position
of the students finger on the screen and maps that position to the locations of objects in the video. When the
students finger moves close to an object in the video, relevant visual information are augmented on the AR
Book and displayed to the student. Students can also move their AR Books around the room to form groups,
virtually manage a project, discuss specific planning scenarios, and explore alternative solutions in a
collaborative setting. Preliminary results and proof-of-concept experiments have illustrated the feasibility of this
approach in visually imparting basic engineering knowledge about construction equipment to students. The
authors are currently working on the design and implementation of classroom-scale experiments where multiple
users can simultaneously interact with a live video stream and learn the concepts in a collaborative environment.
5. REFERENCES
AbouRizk S. M. and Sawhney A. (1994). Simulation and gaming in construction engineering education,
Proceedings of the ASEE Conference, Edmonton, Canada.
Arditi D. and Polat G. (2010). Graduate education in construction management, ASCE Journal of Professional
Issues in Engineering Education and Practice, Vol. 136, No. 3, 175179.
Ash K. (2011). Effective use of digital tools seen lacking in most tech.-rich schools [online]. Education Week.
Available at: http://www.edweek.org/ew/articles/2011/02/23/21computing.h30.html [accessed 26 May 2011].
Behzadan A. H. and Kamat V. R. (2007). Georeferenced registration of construction graphics in mobile outdoor
augmented reality, ASCE Journal of Computing in Civil Engineering, Volume 21, No. 4, 247258.
Behzadan A. H. and Kamat V. R. (2005). Visualization of construction graphics in outdoor augmented reality,
Proceedings of the Winter Simulation Conference, Orlando, FL
Bowie J. (2010). Enhancing classroom instruction with collaborative technologies [online]. Available at:
http://www.eschoolnews.com/2010/12/20/enhancing-classroom-instruction-with-collaborative-technologies/
[Accessed 26 May 2011].
Brooks Jr. F. P. (1999). Whats real about virtual reality, IEEE Journal of Computer Graphics and Applications,
Volume 19, No. 6, 1627.
Kaufmann H., Schmalstieg D., and Wagner M. (2000). Construct3D: A virtual reality application for mathematics
and geometry education, Journal of education and Information Technologies, Volume 5, No. 4, 263276.
Messner J. I., Yerrapathruni S. C. M., and Baratta A. J. (2003). Using virtual reality to improve construction
engineering education, Proceedings of the ASEE Annual Conference and Exposition, Nashville, TN.
Mills J. E. and Treagust D.F. (2003). Engineering education Is problem-based or project-based learning the
answer [online]. Available at: http://www.aaee.com.au/journal/2003/mills_treagust03.pdf [Accessed 21 December
2011].
Nikolic D., Messner J. I., Lee S., and Anumba C. (2010). The virtual construction simulator development of an
educational simulation game, Proceedings of the International Conference on Computing and Building
Engineering (ICCCBE), Nottingham, UK.
Ota D., Loftina B., Saitoa T., Leaa R., and Kellera J. (1995). Virtual reality in surgical education, Journal of
Computers in Biology and Medicine, Volume 25, No. 2, 127137.
298

Psotka J. (1995). Immersive training systems: Virtual reality and education and training, Journal of Instructional
Science, Volume 23, No. 56, 405431.
Stratham D.S. and Torell C. R. (1996). Computers in the classroom: The impact of technology on student learning
[online]. Army Research Institute, Boise State University. Available at: http://www.temple.edu/LSS/spot206.htm
[Accessed 26 May 2011].
Tener R. K. (1996). Industry-university partnerships for construction engineering education, ASCE Journal of
Professional Issues in Engineering Education and Practice, Vol. 122, No. 4, 156162.
Vincenty T. (1975). Direct and inverse solutions of geodesics on the ellipsoid with application of nested equations,
Survey Review, Volume 33, No. 176, 8893.
Yair Y., Mintz R., and Litvak S. (2001). 3D-virtual reality in science education: An implication for astronomy
teaching, Journal of Computers in Mathematics and Science Teaching, Volume 20, No. 3, 293305.
299

2C - MOBILE COLLABORATIVE FIRE HAZARD DETECTION
SYSTEM
Hugo M. da Silva, Joo P. Jacob, Antnio F. Coelho & Rui Rodrigues
INESC TEC, (formerly INSEC Porto) and Faculdade de Engenharia da Universidade do Porto, Portugal
ABSTRACT: In the year 2005 alone, 35,824 forest fires occurred in Portugal, resulting in the loss of a total of
more than 339,000ha (PORDATA, 2012). There is a direct relation between the danger of a fire spreading and the
reaction time (time it takes firefighters to arrive). This reaction time depends on when the fire was reported and
how long it took firefighters to arrive at the location. With this in mind we have developed 2C (TwoSee), a Mobile
Collaborative Fire Hazard Detection System that is capable of quickly collecting and providing relevant and
accurate information that can be useful in the initial stages of a fire. The system uses multiple reports and
cross-references them in order to pinpoint the position of the incident and can be fed with information from anyone.
The more reports received, the more reliable the computed result becomes. These reports consist of a photo,
position and orientation data which is then sent to a server that uses this information to automatically calculate the
target's position. Furthermore, affected area and fire progression can be calculated. This paper focuses on the
collaborative and mobile components of the system, describing how data from multiple mobile devices is filtered
and processed in order to pinpoint a fire's location with reliability.
KEYWORDS: Mobile Computing, Collaborative, Crowdsourcing, Fire Hazard, Sensors
1. INTRODUCTION
Forest fires have been reported to affect the rate and direction of growth of plants, potentially significantly altering
the structure of the surrounding ecosystems (Ahlgren, 1960).
Despite the continuous efforts of firefighters and military forces, Portugal is very prone to forest fires during the
summer. In the year 1980, 2,349 fires were registered, covering a total area of 44,251ha. These numbers have
grown substantially, with 2,026 fires registered in 2010, representing 133,091ha of the territory (PORDATA,
2012).
According to AFOCELCA (an association for forest fire prevention and combat), all types of smoke and fire must
be verified, monitored and handled in the fastest possible way (AFOCELCA, 2012). It is therefore important to
have a system that can be used in the whole territory which can trigger the fastest possible action.
One of the measures taken to detect fires in their beginning is the creation of Fire Lookout Towers. Unfortunately
these towers are located at specific places leaving a large area of the territory uncovered. Furthermore, each tower
has both a fixed initial price and a continuous cost and some of them do not even have a watch guard, rendering
them useless.
Whenever a fire is noticed, the watch guard reports the fire to the command post, as well as its bearing relative to
the watch tower. If two towers can spot the fire, two bearings are traced, their intersection is calculated and the
location of the fire is calculated.
We propose a solution based on mobile devices which allows any citizen who owns a smartphone to communicate
a fire with a touch on the screen. Data from the magnetometer inside most mobile smartphones informs the server
of the direction at which the device is pointing. The location of the fire is calculated and improved with newer
samples, providing a cheap and viable collaborative way of reporting fires anywhere in the world.
Modern subsidized smartphones are usually complemented with software by service providers. A fire reporting
application could be available with every new smartphone, as well as downloadable from the internet, providing
each citizen with an easy, fast and accurate way to report fire hazards. Ideally, like emergency phone numbers, eg.
911, this service would be free of charge and data would not be deducted from the users account.
This work is part of two ongoing PhD thesis with the collaboration of GIPS - Grupo de Interveno, Proteo e
Socorro of GNR (intervention, protection and rescue group of the Portuguese national guard).
300

2. RELATED WORK
Many studies focus regarding firefighting focus on presuppression, or the rapid discovery of fires by optimizing
the placement of lookout towers and improvement of response time, in order to minimize damage caused by
flames (Soto, 2008)(Pedernera, 1995).
In order to detect fires, different approaches are taken, which differ in terms of cost and resources, as well as
results (Proteco Civil):
a) Fixed Terrestrial Vigilance, where an official uses binoculars to spot the fire and traces the azimuth with
the aid of an alidade. The fire can be located by triangulation if it falls within the visual range of two
towers.
b) Mobile Terrestrial Vigilance, performed by land patrols that usually rely on motorized vehicles and may
start fire fighting at the time of detection.
c) Aerial Vigilance uses helicopters and small planes, usually backed by nearby aerodromes.
Julio (Julio, 1994) adds a fourth process based on indirect observers, based on remote sensors like infrared
radiometers which can depend on terrestrial or aerial media.
Modern smartphones often contain several sensors that may contribute to data collection, analysis and processing.
Several interesting applications are available for the different platforms that calculate distances, sizes and sound
levels among others. These applications are often capable of detecting a users current context and presenting
context-sensitive information (Han, 2008).
Smart Tools (Smart Tools, 2012) is an example of an application which retrieves real measurements from a smart
use of sensors. It can measure distances, for example, by asking the users height and measuring the angle at
which the smartphone is held when pointing to the base of the object to which distance is required. By a simple
application of the Pythagoras theorem it calculates the distance to the target. Although precision is not very high,
combining this information with the yaw from the magnetometer could result in an approximate location of a
visual target. Similarly, the height may be computed after calculating the distance, by providing an angle to the
top of the object as well.
Several other applications rely on the same principles to deduce measures based on computations on sensor data,
like Advanced Ruler Pro (Advanced Ruler Pro 2012) or Partometer (Partometer, 2012), which calculates
measures based on comparisons using the camera.
Crowdsourcing is based on the distribution of parts of a task to different people who work for a common goal.
People may or may not be aware that they are collaborating. SETI@Home (Anderson, 2002) and
Folding@Home (Beberg,2009) are examples of distribution of a task over several users in order to achieve a
solution to a problem. The first one focuses on the Search for Extraterrestrial Intelligence, while the latter intends
to simulate protein folding and other biomolecular phenomena. Human effort is kept to a minimum and
computers do all the work, in many cases exploring the time the computer is idle, resulting in the equivalent of a
supercomputer working for a cause.
Collaborative Social platforms provide relevant data based on inputs from different people, many of which are
just playing a game and remain unaware that they are contributing for World Wide data collection. Foursquare
(Foursquare, 2012) is a well-known example of an application that provides specific data based on the reports of
many. Users can create new sites on specific geographic locations they visit, check in when they arrive there and
comment on one site. Other users can receive a list of places in their neighborhood, along with ideas, critics and
reviews. Furthermore, data collected can be used to different ends, from publicity to statistics. In (Cranshaw,
2010), the authors analyze the social context of a geographic region, including location entropy, or the diversity
of unique visitors per location based on Foursquare data.
3. METHODOLOGY
The major limitations of current fire spotting techniques are coverage and cost. Crowdsourcing the task of
locating fires to the general public may be a solution for the first problem. It would be impractical to provide
every citizen with dedicated equipment, so the solution is to explore equipment that is already available. With the
proliferation of smartphones, more and more of the population is equipped with devices that can be used to
collect data and send it to a central server in real time.
301

Presently, report of fires by the population relies on a voice call to an emergency number. Our solution improves
on this approach by increasing report speed, since only two screen touches are necessary as opposed to a full
conversation, lower costs by cutting phone operators, and improved accuracy by replacing subjective comments
with precise data. Furthermore, one report is enough to communicate fire within visible range. Two reports
provide a comparatively precise location, and further reports improve precision while simultaneously reducing
the significance of false alarms. Users should be driven by civil duty alone, since rewarding a user for reporting a
fire would inevitably give rise to false reports or, in a worst case scenario, might lead some people to start fires
with the objective of receiving a reward.
The development of this work consists of three main parts: the mobile client that sends the devices sensor
readings to be processed, the remote server, capable of collecting the information and making it available, and
the control application that is responsible for processing and representing visually the information stored in the
server. This chapter covers the development of all these components.
The above image (Fig. 1) shows that the server receives information from different mobiles devices. That
information is stored in a database to be accessed whenever needed. The information sent by the mobile devices
consists of, most notably, the devices yaw, latitude and longitude. The small amount of information makes it
possible to use SMS instead of a data connection - when the latter is unavailable - with the exception of the photo,
which can be posted at a later time when data becomes available. The control application retrieves this stored data,
processes it and displays the results.
3.1 Remote Server
The remote server consists of a PHP + MySQL webservice. Its purpose is the storage of the information retrieved
by the mobile devices client application.
Fig. 1: Physical architecture of the proposed solution.
Fig. 2: UML Diagram of the database
302

The above image (Fig. 2) shows the structure of the database and the data that it stores. The four tables hold the
following data:
Context - Holds the information of the context to which data that will be gathered shall be added. An
example of a context would be Fire spotted near Oporto. The information related to this context would only be
relevant for it. This information is essentially a picture (taken by the user that created the context), a location (of
the user that created this context) and time.
Line - Each line represents a measurement taken with the mobile application for a specific context. This
measurement is a representation of the users line-of-sight that will be used in future calculations.
Point - Points are the result of intersecting Lines with each other. They represent points where different
lines of sight meet.
Centroid - A centroid is the computed point that represents the common point that all lines of sight in a
context were most likely looking at. It is intended to represent the contexts event location. In the previous example
of a fire spotted near Oporto, the centroid would represent the location (latitude and longitude) of that fire.

The webservice acts as a middleware between the Mobile Application and the Control Application with the
database. The services it provides are typical database operations for each table (add, remove and update). It does
not tamper with the information it receives, it simply stores it in the database.
3.2 Mobile Application
An mobile application for Android smartphones was developed with the purpose of providing context-related
information to the server. This application requires the following features to be available on the device:
Internet Connection - For accessing Google services and communicating with the applications
webservice.
Magnetic-Field Sensor - This sensors reading are used for determining the users current bearing.
GPS Sensor - The Global Positioning System sensor provides the users current global location
Camera Sensor - This sensor is used for helping the user in the task of aiming the device and taking a
picture that represents the context.
Upon opening the application, the user is presented with the following options (Fig. 3):
Choose an existing context - a user that identifies a context in the list that represents his target event,
may join that context and provide multiple measurements. The contexts are listed by proximity to the user,
so the closer a context is to the top of the list, the more likely it is to be the one the user is addressing.
Create a new context - If a user is unable to identify the current context in the context list, he may choose
to create a new one. The first measurement the user takes for that context will record his location (and the
address of that location via Google Geocoding API) and a picture that will be used to represent the newly
created context.

Fig. 3: Mobile application's main menu
303

After joining a context (when creating a new context the user joins it automatically) the user is presented with a
video feed from the devices back camera and a crosshair to help aim at the target. Note that the screenshot in (Fig.
4) also displays additional information simply for debug purposes. To add a measurement to the current context
one must align the crosshair with the target and touch the screen. An unlimited number of measurements can be
taken by each user in every context the same way. It is worth mentioning that the user is only required one tap to
join/create a context and one additional tap per measurement.

Each measurement is comprised of:
Context identification - linking the current measurement to presently joined context
Location - GPS sensors reported latitude and longitude
Accuracy - GPS sensors indicated precision, in meters
Yaw - Reported compass bearing in degrees.
These elements constitute a line-of-sight, starting from the users current location following the reported bearing.
The yaw information extracted from the compass is pre-processed in the device in two ways:
Geomagnetic declination - Since the devices compass points to the magnetic north instead of the
geographic north, it is necessary to apply this correction. This value is dependent on the users current
location and is always applied in order to ensure correct measurements.
Sample Averaging - As the magnetic field sensor is prone to fluctuations, an optional filter that averages
a variable number of last measurements is used. This feature is further explained in the Results section.

After taking any measurement the mobile application automatically sends the values to the database via the
aforementioned webservice or an SMS (not implemented). This data is then used to extract further information.
3.3 Control Client Application
The
Fig. 5: Screenshot of the Control client application
Fig. 4: Mobile applications measurement view
304

Control client (Fig. 5) is a multifaceted application that is responsible for extracting additional information from
the multiple mobile clients readings and for displaying those results.
This application allows for viewing the contexts stored in the solutions database and their related information
(lines-of-sight, points, centroids and metadata). It also allows the deletion of lines or contexts. All this
information is overlaid on satellite images provided by the Google Maps Static API, making them easier to
interpret. It is also responsible for extracting additional information for each context whenever it is required.
This computed information consists of:
Points - Intercepting each line of sight with another one results in the creation of a point where those
two lines intersect. In order to calculate it we considered the earth as a sphere, and the intersection of
the lines to be that of great circles. This approximation leads to an error of 0.3% in a worst case scenario
(Veness, 2012) and as such we consider it to be adequate. However, as the geomagnetic sensors are
prone to fluctuations intercepting two nearly parallel lines can result in erroneous values to be
determined as a simple one degree variation can make a point of intersection to be incorrectly placed.
These variations may also be introduced by other sources of error, such as a displacement between the
center of the camera image and the screen, or even shaking hands. As such a formula was used for
calculating the weight (or confidence) of each point.

In formula 1 the resulting point of intersection between two lines is considered to be precise if the two
lines are perpendicular. Inversely, the point is determined to be unreliable if the two lines are almost
parallel.

In (Fig. 6) the points represent different information: the white dot represents the point where all the lines
should be converging and was added manually for testing, the yellow point represents the centroid, which
was automatically determined and that ideally should be coincident with the white one. Finally the dots
whose colors vary from red to green represent the intersection points with their confidence color-coded
(red being minimum confidence and green maximum).
Centroid - The centroid represents the point in the map that most likely represents the location where all
the lines of sight, from all users that took measurements for a context, meet.

(2)

| Sin (bearing1 - bearing2) | (1)
Fig. 6: Lines and points drawn in the control application.
305

It results from the weighted average formula 2 of all the points in that context; the weight being the confidence
interval (from 0 to 1) determined previously using formula 1. As such, it is possible that this point is not contained
in any line-of-sight.
4. RESULTS
The application was tested in three different contexts with multiple measurements per context.

Table 1: Context Statistics and Precision Analysis

Context 1 Context 2 Context 3
Target A A B
Filtering Level 1 200 1
Samples per location 3 1 3
Number of lines 9 3 9
Centroid displacement error (m) 64 62 47
Average Distance to Target(m) 792.3333 685 326.6667
Maximum error in line (%) 9.411765 9.553159 18.14672
Minimum error in line (%) 6.484296 8.659218 11.66253
Average error(%) 8.303382 9.065961 14.78678
Table 1 contains the following relevant statistics:
Target The place the user is pointing at, represented by the white dot.
Filtering level - how many bearing samples were used to determine the average bearing per
measurement.
Samples per location - the number of measurements taken per location by each user. In contexts 1 and 3
only one sample per measurement was taken but three measurements were made. In context 2, however,
only one measurement was taken, but that measurement is the result of averaging the last two hundred
samples that are continuously polled from the sensor.
Number of lines - total number of lines to intersect among each other per context
Centroid displacement error Distance between the calculated centroid (yellow dot) and the true target
(white dot, manually added for testing purposes).
Average distance to target The arithmetic mean of the distances between the position of the user and
the target.
Minimum and maximum error in line - The error given by dividing the centroid displacement error by
the length of the shortest / longest line.
Average error - The error given by the average of all Errors in line

The collection of multiple samples per measurement reduces human and sensor errors. In order to improve
usability, automatic multi-sampling was introduced. This approach provides more accurate results without relying
on the user to take several measurements per location.
The following images show the actual measurements and data that were used to build this table. Note that very
distinct locations were used, as well as devices and users in order to guarantee a controlled yet plausible scenario.
Additionally, each user aimed at the center of the targeted structure (these structures simulated active fires).
306

The results show an approximate average error of about 10% (average between all readings). Context 3 shows a
relatively abnormal Maximum Error in line considering the short average distance to target. The other two contexts
show an average error that we consider acceptable in the scenario of forest fires (as they can be spotted from a
distance, so the approximate location is all that is needed).
We presented 2C (TwoSee), a mobile collaborative fire hazard detection system. The system is based on
line-of-sight intersections, from registrations by any bystanding person with a smartphone. The presented solution
consists of a webservice and database, a mobile application and a control application. The current results show an
average of about 10% error in the measurements, sometimes lower, in some cases with absolute errors under 70m
in urban contexts. Considering that an urban environment is prone to create magnetic interference with a mobile
devices magnetic-field sensor and that a user may inadvertently move the device when taking the measurement
(leading to erroneous bearing calculation) we can conclude that the presented methodology is adequate, yet limited
by these sensors sensitivity, even when filtering. GPS accuracy error was discarded, as although significant in
short distances, in the scenario of usage for this application (fire hazards) it can be ignored. Testing in non-urban
scenarios is still needed, but was not done due to the lack of common landmarks for testing and we predict it will
lead to even better results as the magnetic interference of power lines and other electronic devices is minimized. In
the future we intend to explore additional features and functionalities that enable the user to take measurements at
the edges of a structure or fire instead of the center so that the points of intersection may be used to determine a
potential enclosing area. Furthermore, wind direction and speed can possibly be inferred, as multiple pictures of
the same fire taken at a known distance (determined by the presented method) can be used to calculate the winds
speed and direction, possibly helping in the prediction of the fires progression.
The presented approach replaces a call to an emergency number, with additional benefits. Although two screen
touches are necessary in order to identify and report an occurrence, in many cases the first point may be calculated
automatically, reducing the number of screen presses to one. As soon as one intersection is calculated, further lines
may be automatically considered to belong to an existing context or a new context based on the distance between
the intersection between the newly created line and the context centroid and considering the timestamp of those
measurements. This solution would improve even further on user experience, since reporting a fire would
represent pointing the phone to it and touching the screen, much like taking a photo of the incident.

Fig. 7: Images for the contexts (top to bottom) 1,2 and 3
307

6. REFERENCES
Advanced Ruler Pro, Google Play - https://play.google.com/store/apps/details?id=com.yingwen.rulerpro, last
accessed on 15/6/2012
AFOCELCA, Preveno e Combate a Incndios Florestais,
http://www.afocelca.com/?mid=3&zona=Estrat%E9gia, last accessed on 15/6/2012
Ahlgren I. F., Ahlgren C. E.(1960), Ecological effects of forest fires, The Botanical Review, Volume 26, Number 4
(1960), 483-533, DOI: 10.1007/BF02940573
Anderson D. P., Cobb J., Korpela E., Lebofsky M., Werthimer D.. (2002). SETI@home: an experiment in
public-resource computing. Commun. ACM 45, 11 (November 2002), 56-61.
DOI=10.1145/581571.581573http://doi.acm.org/10.1145/581571.581573
Beberg, A.L.; Ensign, D.L.; Jayachandran, G.; Khaliq, S.; Pande, V.S. (2009) , "Folding@home: Lessons from
eight years of volunteer distributed computing," Parallel & Distributed Processing, 2009. IPDPS 2009. pp.1-8,
23-29 May 2009 doi: 10.1109/IPDPS.2009.5160922
Cranshaw J., Toch, E., Hong, J., Kittur, A., Sadeh, N. (2010). Bridging the gap between physical location and
online social networks. In Proceedings of the 12th ACM international conference on Ubiquitous computing
(Ubicomp '10). ACM, New York, NY, USA, 119-128. DOI=10.1145/1864349.1864380
http://doi.acm.org/10.1145/1864349.1864380
Foursquare, Google Play - https://play.google.com/store/apps/details?id=com.joelapenna.foursquared, last
accessed on 14/6/2012
Han, L., Jyri, S., Ma, J., & Yu, K. (2008). Research on Context-Aware Mobile Computing. 22nd International
Conference on Advanced Information Networking and Applications - Workshops (aina workshops 2008), 24-30.
Ieee. doi:10.1109/WAINA.2008.115
Julio, G. (1994). Fundamentos de Manejo del Fuego. Apuntes del Curso Control de Incendios Florestales. Escuela
de Ciencias Florestales. Universidad de Chile. Santiago. 329p.
Partometer, Google Play - https://play.google.com/store/apps/details?id=com.vistechprojects.partometer, last
accessed 15/6/2012
Pedernera P., Julio G., Castillo E. (1995). Modelo de Optimizacion para el Diseo de Sistemas de Torres de
Deteccion de Incendios Florestales. Universidad de Chile, Santiago.
PORDATA 2012, Incndios florestais e rea ardida (R) -
http://www.pordata.pt/Portugal/Incendios+florestais+e+area+ardida+%20+Continente+(R)-1192 , last accessed
on 19/6/2012
Proteco Civil - Escola Superior Agrria IPCB, Incndios Florestais, (internal report)
Smart Tools, Google Play - https://play.google.com/store/apps/details?id=kr.aboy.tools, last accessed on
15/6/2012
Soto, C., Koch, M., Esteban, G. (2008). Diseo de un Sistema de Torres de Deteccin de Incendios Forestales para
la Regin Metropolitana. Master Thesis. Universidad de Chile.
Veness C. (2012), Movable Type Scripts 2012 http://www.movable-type.co.uk/scripts/latlong.html, last accessed
on 14/6/2012
308

VIRTUAL PROTOTYPING FOR ROBOTIC FABRICATION OF REBAR
CAGES IN MANUFACTURED CONCRETE CONSTRUCTION
Akash Garg & Vineet R. Kamat
University of Michigan
ABSTRACT: With the advent of precast construction technology, the construction industry has seen a rapid
change in the way structures are built. Precast construction has brought down project costs and durations,
increased the predictability and improved the quality of construction. It has also resulted in a shift in construction
tasks from the construction site to the factory floor. Precast elements are manufactured and cured in controlled
factory environments and are then transported to the construction site for assembly. This shift into the factory
makes the process a viable candidate for automation. However due to the nature of the construction industry, it is
important that the used automation technology be reconfigurable in a way that accommodates frequent changes in
design and demand. The majority of precast elements currently used are made of reinforced concrete. This paper
considers the unique constraints inherent in precast construction, and proposes a method to automate the
production of precast concrete elements. Assembly and placement of rebar cages is an important step during the
precast segment manufacturing process. An automated procedure for assembling the steel reinforcement cage for
precast elements is described. This includes the placement of longitudinal reinforcement bars, followed by the
placement and bending of transverse reinforcement. The entire design and analysis is conducted using Virtual
Reality-assisted Virtual Prototyping to reach a favorable design for the proposed mechanism, without having to
construct an actual prototype.
KEYWORDS: Automation; Design; Precast Construction; Robotics; Virtual Prototyping.
1. INTRODUCTION
A Virtual Prototype, or digital mock-up, can be defined as a computer simulation of a physical product that can be
observed, analyzed, and tested from product life-cycle perspectives such as design, manufacturing, service, and
recycling as if it were a real physical model. The creation and evaluation of such a Virtual Prototype is known as
Virtual Prototyping (VP) (Wang 2002). By using a digital model instead of a physical prototype, VP can alleviate
several shortcomings of the product design, evaluation, and manufacturing process.
For example, using VP, defects concerning fabrication, product design and production planning can be quickly
identified before major physical resources are committed. This can significantly reduce the number of physical
iterations needed during product design and manufacturing, and thus reduces the associated costs involved in
product development, evaluation, and manufacturing (Choi and Chan 2004).
Virtual Reality (VR) is defined as a computer simulation of a real or imaginary system that enables a user to
perform operations on the simulated system and shows the effects in real time (AHP 2009). VR is thus of
significant value to VP as illustrated by the following examples:
VR can facilitate an immersive understanding of a virtual product, especially for ergonomic and aesthetic
design, as well as end-user participation in the design and evaluation process (e.g., Jayaram et al. 1998).
VR can provide advanced user interfaces for product design, and can facilitate collaborative work-flow (e.g.,
Wang et al. 1999).
VR can enhance the interface for finite element analysis, as well as kinematic and dynamic analyses
associated with product design and evaluation (e.g., Haase and Preb 1997).
VR can support the analysis required for evaluating the assembly, manufacturability and maintainability of a
proposed design by offering the possibility of immersing a end-user in the virtual environment to perform
specific tasks (Bauer et al. 1998)
VR driven VP has most extensively been deployed in the automobile and aerospace industries. The focus of these
efforts has primary been two folded: product design and process simulation. In the product design phase, analyses
such as functional testing, form-and-fit testing, ergonomic testing, assembly testing and disassembly testing, can
be conducted on a virtual prototype using VR for design validation (Choi and Chan 2004). On the other hand, for
309

process simulation, a manufacturing process can be simulated using VR to identify possible manufacturing
problems or bottlenecks in production without having to use expensive physical mock-ups. Choi and Chan (2004)
provide a detailed review of such specific VP implementations that have exploited VR.
The objective of the presented research was to design, evaluate, and demonstrate a new robotic system capable of
fabricating precast concrete segments, entirely using VR and VP techniques. Automation and robotics is an
emerging field within construction engineering (Zavadskas 2010). The successful implementation of a new robot
on real construction jobsites requires the buy-in of several stakeholders including owners, contractors, equipment
manufacturers, and workers. The costs of building and evaluating a real physical prototype of a new robot are often
prohibitive both from monetary as well as safety perspectives. VP using VR principles is thus a clear choice for
creating, communicating, evaluating, and improving the design and operation of a new proposed machine.
The remainder of this paper is organized as follows: First, we provide a literature review of automation in
construction, making the case for rapid, low-cost design and evaluation of new machines. The remainder of the
paper then describes how VR was used to systematically design, evaluate, and demonstrate a virtual prototype of a
new robotic mechanism for rebar cage fabrication in precast concrete construction.
Since the early 1980s there has been rapid development on the construction automation front. However most of
the advances that have been made are in the steel construction sector and not much work has been done in the
prefab construction sector. Moreover most of these developments have been concentrated in Japan and parts of
Europe (Shinko 2007; Neelamkavil 2008; Wing and Atkin 2002). Among the primary construction automation
systems in the world are the SMART system by Shimizu Corporation, Shuttle Rise by Kajima Corporation and
ABCS and Big Canopy by Obayashi Corporation (Shinko 2007). Other efforts include the Future Build Project
(Wing and Atkin 2002) and the ManuBuild Project, both of which are initiatives taken by the European Union
towards the advancement of prefab construction (Neelamkavil 2008).
A relatively new system for automated steel erection is presented in (Hong et al. 2007; Jung et al. 2008). It
involves a rail and boom based transportation system. A wire suspended positioning system to bring the girders
and beams into position is also employed. The paper identifies the reasons for automation as the lack of skilled
laborers, difficult work terrains and work too dangerous to be done by human workers. Another similar material
handling system is given in (Abderrahim et al. 2001; Leyh 1995).
To the best of the authors knowledge, not much literature is available on the automation of the prefab element
manufacturing process. Needless to say automation of the prefab element manufacturing process would have
tremendous advantages including reduction in material wastage, improved safety of construction workers,
increased accuracy in the placement of the reinforcement, reduced cost for casting difficult shapes, reduced site
disturbance and easy pre-stressing. These points are further emphasized by (Acker 1996). In addition, support for
utilities such as electric cables and water pipelines can be easily built into the prefab element before the concrete
casting step. Once a robust method for manufacture of precast concrete beams, columns and other members is
developed, even automated steel erection techniques can be used for the construction process.
Though there are a few industrial products available (PCC 2007; NBS 2008; PCP 2008; Bock 2007), they suffer
from deficiencies such as the inability to manufacture a variety of shapes and put in utilities such as electric
connections and pipelines before the concrete casting step. These manufacturing lines are used primarily for
building wall panels and other similar elements. The assembly of the rebar cage is an important part of the
automated concrete precast element fabrication. Bock (2007) provides description of one such machinery. It uses
longitudinal reinforcement bars and partially open stirrups. The procedure followed involves first placing the
longitudinal rebars at appropriate locations on the support, followed by placing the partially open stirrups in
place. The stirrups are then bent in to place holding the longitudinal bars at its location. The stirrups and bars are
then welded into their correct position. The parts of the automated plant are the bar supporting mechanisms, the
bending robot, robots for moving the rebars, movable storage table and a welding table.
However the proposed system does have some limitations. The mechanism is not suitable for more than two
layers of reinforcement and circular reinforcement cages which are often used in pile foundations. No
mechanism has also been proposed to lay the shuttering and place the concrete. Furthermore, the number of
robots used in the process can be reduced by introducing the concept of tool changing, thereby drastically
bringing down the cost of the project.
310

The design investigated in this research attempts to overcome these limitations, and is robust enough to
manufacture reinforced concrete precast segments of an array of shapes and sizes in an economical fashion. The
proposed mechanism is created and evaluated entirely using a virtual prototype without having to build a
physical mock-up.
3. DESCRIPTION OF THE PROPOSED SYSTEM
This section describes the design of the entire proposed mechanism. Before going into a detailed description of
the components of the mechanism, it is imperative to understand the sequence of steps involved in the reinforced
concrete construction process. Figure 1 shows the steps typically followed. The first two steps involved i.e. the
placement of the transverse reinforcement and welding appropriate stirrups to the reinforcement constitute the
steps involved in putting together the reinforcement cage. During the course of this research, it was identified
that these are the most labor intensive and repetitive tasks, making them perfect candidates for automation. The
designed mechanism primarily focuses on these aspects.
The following two steps, which are the placement of the shuttering in place followed by the placement of the
rebar cage into the mould, prepare the specimen for casting. Though these steps can be automated, it might not
be cost effective. Once the mould is prepared and the reinforcement is in place, the concrete can be poured in
and cured. Finally once the concrete has dried the formwork can be removed. It should be noted that after the
concrete has been poured in the specimen, it can be taken to an alternate location as curing is a slow process.
Fig. 1: Sequence of steps involved in prefab reinforced concrete construction.
The major advantage of the proposed system over existing ones is its ability to manufacture reinforcement
frames of varying shapes. Reinforcement cages may be square, rectangular, circular or other arbitrary shapes.
Some examples of these shapes are illustrated in Figure 2. The reconfigurability of the system is achieved with
the help of the template plates which may be drilled for the shape required. It should also be noted that not only
may the configuration of the longitudinal reinforcement change but the transverse reinforcement may also be
attached in different ways such as a continuous spiral or separated segments at defined gaps.
Fig. 2: Typical reinforcement cage shapes and sizes.

311

The transverse reinforcement can be attached to the longitudinal reinforcement using a variety of techniques
ranging from tying to different types of welding. End effecters appropriate for the desired task may be attached to
the robotic arm, as and when required. Welding is a process easier to automate as compared to tying, owing to the
dexterous capabilities required for the tying process. Owing to the extent to which it is used in the automation
sector, the framework for automated welding is well developed and is thus easier to implement.
For the proposed system it may be noted that sensors are not needed for feedback control of the arms due to the
structured nature of the task. Simple open loop control implementation is sufficient. This can drastically reduce
cost. The overall design for the proposed system is shown in Figure 3. The system primarily consists of the
supporting frame, robotic arms mounted on guide rails, and the end template plates. The following subsections
give the key design principles behind each component of the mechanism.
Fig. 3: The overall design of the mechanism.
3.1 Robotic Arm
In this section, the design of the two mechanical arms is outlined. The robotic arms are capable of changing their
end effecters, making the system cost effective. The arm has six degrees of freedom as illustrated in Figure 4. Out
of these degrees of freedom only the first one is translation. The remaining five degrees are rotational in nature.
The links of the mechanism are designed in a manner such that they allow maximum possible rotation. The control
scheme used to control the arm is discussed in section 5.
Fig. 4: Degrees of freedom for the robotic arm.

312

The robotic arm contains a servo motor at its end, the rotation of which can be used by the end effecter attached.
Some of the different end effecters that can be attached to the robotic arm are described in the subsequent
subsections. It was of prime importance to work out the proper kinematics and inverse kinematics for the gantry
based robotic mechanism in order to ensure appropriate control. This was achieved using the Denavit-Hartenberg
(D-H) convention (Craig 2005). While working out the kinematics, six degrees of freedom including the
translation of the top gantry mechanism are considered. The six joints ensure that the end effector has the
capability to reach any general location in the dexterous workspace with all possible orientations.
The inverse kinematic problem involves finding a transformation between the Cartesian space to the joint space
and eventually to the actuator space. It is usually beneficial to work out a closed form solution for the inverse
kinematics in comparison to a numerical solution to obtain optimum performance. The inverse kinematic
problem can be formulated by the following equation which needs to be solved for d1, 1, 2, 3, 4 and 5. The
equation was solved by geometric methods to work out the trajectories in joint space.

3.1.1 Drilling tool
The drilling tool is used to drill holes in the template plates, the end plates that support the longitudinal
reinforcement. Figure 5 shows a screenshot from the visual simulation of the drilling tool in action.
Fig. 5: Drilling tool drilling the template plate.
313

The drilling tool attaches to the robotic arm. The drill is driven by the motor in the robotic arm which connects to
the drill effecter through a shaft. The speed and torque of the motor are then appropriately adjusted to finally
rotate the drill bit. The drill bit can be changed either manually or automatically. Based on the CAD drawing
given as an input to the system, the drilling tool can drill the template plate at the given locations. The control of
the robotic arm during this time period is not a very difficult task due to the absence of reinforcement bars in the
area during this time. To ensure that the drilling tool reaches its intended location it is important to work out the
inverse kinematics with utmost precision. Further to ensure appropriate performance the drill must remain
orthogonal to the template plate.
3.1.2 Gripping tool
The gripping tool is used to grip the longitudinal bars and bring them to their appropriate location. Figure 6 is a
screenshot from the VR simulation showing the gripping of a reinforcement bar. The gripping tool must be
capable of gripping bars of varying diameters. The pressure applied by the grippers should be sufficient to hold
the bar in place, but at the same time should not be so high that it damages the gripper or the reinforcement bar.
Fig. 6: Gripping tool gripping the reinforcement bar.
3.1.3 Stirrup bending tool
As the name suggests the stirrup bending tool is used to bend the stirrups and put them into place. While
designing this component of the system it was important to take into consideration an important constraint.
While reaching the stirrup or bending it the longitudinal reinforcement might prove to be an obstruction, and was
particularly considered in the design iterations. The source of the stirrup reinforcement is assumed to be a roll of
steel rod of appropriate dimension.
The above listed tools are the most crucial ones required for manufacture of the reinforcement cage. In addition,
other end effecters catering to tasks such as pouring in the concrete, painting the manufactured job or coating it
with diverse materials can also be developed.
3.2 Template Plates
The template plates are a very important feature of this particular research as they play the lead role in providing
reconfigurability to the system. Owing to the design of the template plates, rebar cages in a large variety of
shapes can be manufactured. The role of the template plates is to hold the reinforcement in place while other
operations such as attachment of reinforcement are performed. The first time a new design is to be built the
template plates are drilled at the required locations. For subsequent production of the same design there is a need
to drill the plates again. One may argue that drilling new plates for every new project may not be cost effective.
However it was determined during the course of this work that the advantages of using these plates far outweigh
this disadvantage due to the typical nature of civil engineering projects.

314

It is more likely that cages of a certain shape would be required in a large number as opposed to the probability
of requiring a larger variety of cages with a smaller quantity for each. Further the template plates are equipped
with full 360 degree rotation motors that are particularly important for the stirrup attachment step. These motors
allow rotation of the entire rebar cage, making it easy for the robotic arm to reach its different parts.
The motors rotate the template plate through the use of a belt-pulley system. There are two primary reasons for
the use of such a mechanism. The first is that it is important that there must be no obstruction behind the
template plate, to ensure sufficient space for the reinforcement to be placed. The second is that it helps reduce
the speed of rotation and increase the torque to a level that can drive the rotation of the template plate.
4. ASSEMBLY PROCEDURE
This section describes the steps involved in the production of the reinforcement cage using the described
automated system. There are three primary steps involved. First is the drilling of the template plates. Second is
the placement of transverse reinforcement. Finally, third is the placement and welding of stirrups at their
appropriate locations. The details for each of these steps are provided in the following subsections.
4.1 Drilling of the template plates
The end plate drilling is the first step of the fabrication procedure. An input drawing is given to the system in
order to drill the plates. The input drawing has points marking the locations that need to be drilled. This is then
read by a program which calculates the joint angle trajectories for the robotic arms taking the inverse kinematics
into consideration and sends the signal to the mechanism. The appropriate size drill bit must be chosen. This may
either be done manually or can also be automated, though that would not really offer any significant advantages.
4.2 Placement of transverse reinforcement
This is the first and one of the most critical stages involved in the precast segment process. Aberrations from the
given design can lead to catastrophic failure of the structure using these precast segment blocks. To ensure that
the transverse reinforcement is at the appropriate location at all times it is important that it is gripped by multiple
grippers to take its deflection into account. The reinforcement bar of appropriate size from a specially designed
stock pile is first picked up by the robotic arm on the outer side of the template plates.
This reinforcement is then carefully guided into the template plate hole that it corresponds to. The other robotic
arm then assists the first one to guide the bar to the hole in the template plate on the opposite side. This
procedure is carried out for all the longitudinal reinforcement bars. Once this step is complete the robotic arm
positioned on the outer side of the template plate shifts to the other side of it. Figure 7 illustrates this process as a
VR simulation screenshot.
Fig. 7: Placement of the transverse reinforcement.
315

Fig. 8: Flow of Control.

316

4.3 Bending of the stirrups
The next step is the bending of stirrups. The rotation provided by the template plates play a major role in the
positioning of the longitudinal bars in a manner that they can be easily manipulated by the stirrup bending tool.
5. CONTROL SCHEME
This section presents the overall control scheme for the mechanism. Figure 8 illustrates the flow of control,
starting from the input CAD model to the final motor control. The flowchart is largely self-explanatory and is not
discussed further due to space restrictions.
6. DISCUSSION AND CONCLUSIONS
VR assisted Virtual Prototyping is a cost-effective, safe, and practical method for creating, communicating,
evaluating, and improving the design of new industrial machines, particularly in situations where the cost and
safety aspects of building and testing a physical prototype are prohibitive. The major improvement that the
proposed rebar fabricating robot design has over its predecessors is the degree of reconfigurability that it
provides.
The presented technique is suitable for production of precast products of varying shapes and sizes such as beams,
panels or bridge deck elements. VR facilitates the effective communication and demonstration of these features.
The other advantage is in terms of cost reduction over conventional automation techniques. The end effecter
proposed with tool changing capabilities makes is economically feasible to use a large variety of tools using the
same robotic arm. In this case also, VR facilitates the evaluation of the proposed designs reconfigurability
without having to create physical prototypes of all possible end effecters.
The design for the proposed mechanism has been completed followed by several simulations to ensure optimum
performance. As described, the kinematic design of the mechanism has also been evaluated. Simulation results
generated are being used to further study and optimize the joint trajectories to ensure no collisions take place and
a given task can be completed in the least possible time. Future work in this direction will involve building a
physical scaled prototype of the system, which finally would be followed by a full scale assembly for use by the
precast construction industry.
Though it might seem futuristic, colonization of other planets and the moon by humans may very well happen
within the next few decades. In such situations, in order to make human habitation feasible it would be necessary
to first construct appropriate shelters, with minimum involvement of human labor. It is in such situations also,
that technologies of automated construction such as those discussed in this paper may be particularly useful.
Since creating physical mockups may be unrealistic in such scenarios, it is clear that VR will continue to play a
dominant role in the design and evaluation of new machines and methods created as virtual prototypes.
7. REFERENCES
Abderrahim, M., A. Gimnez, A. Nombela, S. Garrido, R. Diez, V., M. Padrn and C. Balaguer (2001), The Design
and Development of an Automatic Construction Crane, Proceedings of the 18th ISARC, Krakow, Poland.
Acker, A. V. (1996). Automated Production of Precast Architectural Elements, Structural Engineering
International, International Association for Bridge and Structural Engineering.
American Heritage Publishing (AHP) (2009). The American Heritage Dictionary of the English Language, Fourth
Edition, Houghton Mifflin Company, Boston, MA.
Bauer, M. D., Siddique, Z., and Rosen, D. W. (1998). A Virtual Prototyping System for Design for Assembly,
Disassembly, and Service, Proceedings of the 1998 ASME Design Technical Conference and Computers in
Engineering Conference, DETC98/CIE-5539, Atlanta, Georgia.
Bock, T. (2007) Construction Robots, Autonomous Robots, Vol 22, Number 3, 201-209.
Choi, S.H., and Chan A.M.M. (2004). A Virtual Prototyping System for Rapid Product Development,
Computer-Aided Design, 36 (2004), Elsevier Science, New York, NY, 401412.
317

Craig , J.J. (2005). Introduction to Robotics: Mechanics and Control, 2nd edition, Pearson/Prentice Hall.
Haase, H. and Preb, T. (1997). Improved Interaction and Visualization of Finite Element Data for Virtual
Prototyping, Proceedings of the 1997 ASME Design Technical Conference and Computers in Engineering
Conference, DETC97/CIE-4305, Sacramento, California.
Hong, D., S. Park, U.K. Lee, T. Kang, S. Lee, N. L. Doh, Gwi-Tae Park, Kyung-In Kang, Myo-Taek Lim (2007),
Robotic Technologies for the Automatic Assemble of Massive Beams in High-Rise Building International
Conference on Control, Automation and Systems.
Jayaram, S., Angster, S. R., Gowda, S., Jayaram, U., and Kreitzer, R. R. (1998). An Architecture for VR based
Virtual Prototyping of Human-Operated Systems, Proceedings of the 1998 ASME Design Technical Conference
and Computer in Engineering Conference, ETC98/CIE-5542, Atlanta, Georgia.
Jung, K., B.Chu, K. Bae, Y. Lee, D. Hong, S. Park, and M. T. Lim (2008), Development of Automation System for
Steel Construction Based on Robotic Crane, International Conference on Smart Manufacturing Application.
Leyh, W. (1995). Experiences with the construction of a building assembly robot, Automation in Construction,
Volume 4, Issue 1, Elsevier, 45-60.
NBS (2008). Potentials for automation in the precast plant, Available:
www.new-building-system.com/downloads/080400_CPI_Elektrodosen_e.pdf
Neelamkavil J. (2008), Automation in modular and prefab industry, Report # RR261, National Research Council,
Canada.
PCC (2007). High-tech production of precast concrete elements, Available:
www.pc-control.net/pdf/032007/pcc_0307_katzenberger_e.pdf
PCP (2008). Potentials for automation in the precast plant Part 2, Available:
precastconcreteplants.com/downloads/080600_CPI_Insertroboter_e.pdf
Shinko Research Co. Ltd (2007), Automation of building construction and building products industry - state of art
in Japan, Available: www.tekes.fi/fi/gateway/PTARGS_0.../japanreport.pdf
Wang, Y., Jayaram, S., Jayaram, U., Lyons, K., and Hart, P. (1999). Representation of Swept Volumes in a
Parametric CAD System Using Trajectory Information from Virtual Environments, Proceedings of the 1999
ASME Design Technical Conference and Computers in Engineering Conference, DETC99/CIE-9109, Las Vegas,
Nevada.
Wang, G.G. (2002). Definition and Review of Virtual Prototyping, Journal of Computing and Information
Science in Engineering, Volume 2, Issue 3, American Society of Mechanical Engineers, New York, NY, 232-236.
Wing, R. and Atkin, B. (2002). FutureHome: A Prototype for Factory Housing. In Proceedings of International
Symposium on Automation and Robotics in Construction, 19th (ISARC). National Institute of Standards and
Technology, Gaithersburg, Maryland. 173-178.
Zavadskas, E.K. (2010). "Automation and robotics in construction: International research and achievements",
Automation in Construction, 19(3), Elsevier Science, New York, NY, 286-290.

318

PERFORMANCE BETWEEN GENDERS IN ASSEMBLY TASK
RELEVANT TO MEMORIZATION AFTER TRAINING USING AR AND
MANUAL PRINTS
Lei Hou & Xiangyu Wang
School of Built Environment, Curtin University, Australia
ABSTRACT: Past studies suggest that Augmented Reality (AR) training schema is more effective for training
assembly task with complexity that requires cognitive load. Gender has not been investigated as an independent
factor which might cause the performance difference between training schemas and sexes. It is therefore
hypothesized gender is an independent factor impacting over post-training performance after both training
schemas. This paper presents an experiment-based evaluation, which indicates that: 1) AR training is able to
generate paralleled learning curves in between the male and female assemblers, 2) AR training is able to generate
shorter learning curves to the both male and female assemblers compared with training of manual prints and 3)
manual prints is more effective to train the male assemblers.
KEYWORDS: AR, gender difference, post-training performance, learning curve.
1. REVIEW OF PREVIOUS RESEARCH
The previous research used a small scale LEGO model as test bed and conducted one experiment to compare the
task performance, cognitive workload and learning curves of novice assemblers between using an animated AR
system and manual prints to assemble the LEGO model. Findings revealed that the animated AR visualization
yielded shorter task completion time, less assembly errors and lower total task load (Hou and Wang 2010). The
experiments further investigated the learning curves of assembly novices. The learning curves established for the
two assembly guidance (3D manual prints and AR) in the previous research revealed a significant difference in
between two groups of trainees under different training schemes (manual prints training schema and AR training
schema) (Hou et al. 2012). This paper further investigates the difference when gender is considered as an
independent variable against the training schema. Experimentation was the major method accompanied with the
quantitative evaluation method (SAS was used as statistical tool). The qualitative performance information
through observation or monitoring of the subjects task performance during experiment was gathered from
different training schemas.
2. EXPERIMENTAL DESIGN
AR System prototype: A prototype named the animated AR system which utilizes the marker tracking technology
(Kato and Billinghurst 1999) based on computer visualization was developed, as presented in Figure 1. This
prototyped AR system for assembly provides the information about the positions of to-be-mounted components
and outputs a step-by-step animation for guiding assembly. The constitution of the prototyped AR system
involves the traditional establishment of AR devices, a computer monitor, a webcam, pre-defined paper-based
markers, computer modeling and rendering software (3DSMAX), ARToolkit (Kato, Billinghurst and Poupyrev
2000) and attached OpenGL. Via ARToolkit, the virtual images of product components can be registered onto the
pre-defined markers and captured by the webcam, which is typically connected with a computer monitor or head
mounted display.

319

Figure 1. the Animated AR System Designed for Step-by-step Assembly Contents

Subjects: Twenty eight (28) graduate student participants (divided into 2 groups with 7 males and 7 females in each
group) were recruited to participate in the study. All of them had never used AR before. Firstly, each group
implemented assembly training under a specific training schema (3D manual prints vs. AR), where the trainees
were encouraged to remember the assembly sequence and component fixation/installation from a chosen model.
Then, trainees began to assemble the same model without being allowed to seek help from the manual prints or the
animated AR system.
Measurement: Task performance was videotaped and measured in terms of three types of metrics for evaluating the
trainees task performance: number of assembly trials, time consumed to complete a trial and number of errors
committed during each trial. The number of trials indicates how many trials a trainee needed to complete assembly
without an error. After each unsuccessful trial, the number of errors was added and told to each trainee, thus each
trainee was able to check the steps where the errors had occurred. Subjects were videotaped during their task
assignment so that potential errors could be identified. Since there was no guidance or information available,
trainees had to mentally retrieve information and recall the assembly steps from their memorization that had been
developed in the training sessions.
Experimental Variables: the following independent variables involved in the experiment were identified and
determined: training schemas (3D manual prints vs. AR) and gender difference (male vs. female).
Procedure: The two groups of test trainees were encouraged to remember the assembly sequence and component
fixation/installation within specified training schema but limited to one single LEGO model assembly cycle
(LEGO MINDSTORMS NXT 2.0). Before the start of the actual experiment, all the trainees had 5 minutes of
reading material irrelevant to the experiment (e.g., newspaper). The two test groups of 28 students were then
starting the first trial, where neither manual prints nor AR was provided. More trials were conducted when
erroneous assembly was conducted.
3. RESULTS AND DISCUSSIONS
In the first trial, the formal assembly task did not reflect an evident difference between two genders among the AR
trainees in terms of time consumption. An average of 13.14 minutes was needed to complete the assembly for the
male assemblers while just a bit faster, only 13 minutes was needed for the females. The p value of 1 further denied
the significance of this difference. It was however evident to observe a significant difference of time indicator
between two genders of manual trainees, as an average number of 17.14 minutes was spent for the male assemblers
whilst 20.1 minutes for the females. The error indicator showed that for both male and female AR trainees, they
320

had committed the same number of errors (3.6 errors) during the first assembly trial. By comparison, the female
trainees committed an average of 6.7 errors after training in manual prints, which were 1.1 errors more than the
male trainees.
The parallel findings were manifested in the second trial. The performances of both males and females after AR
training did not significantly vary in terms of time and number of errors (8.57 vs. 9.1 minutes; 1 vs. 1.1 errors),
whereas significantly varied for those who did not use AR (11.14 vs. 14.3 minutes; 2.7 vs. 4 errors). After training
in manual prints, the average time consumed for the female assemblers was 3 minutes more than the male
assemblers, while the number of errors was 1.6. The p values of 0.77 and 0.87 for time and error did not present the
significant correlation of performance disparity in the AR group, whereas the p values of 0.04 and 0.05 supported
the significant disparity for the time and error in between two genders in the manual prints group. To complete the
second trial, the manual trainees averagely needed 11.14 and 14.3 minutes, and committed 2.4 and 4 errors, better
than the first trial but still worse than the AR trainees.
For AR group, only 3 males and 3 females stepped into the third trial, where an average time of 7.67 minutes was
spent but no error was made. As only 6 trainees (3 males and 3 females) in AR group had entered the third trial, the
significance of performance difference for males and females between AR and manual group was not discovered.
Since all the manual trainees had erred in the second trial, they thus entered the third trial. Male and female
assemblers averagely spent 8.57 and 10 minutes and committed 0.6 and 1.1 errors. Based on the statistical data
collected, the performance curve of males and females conducting certain times of trails in the formal assembly is
given in Figure 2.

Figure 2. Performance Curve of Conducting Formal Assembly between Training Schemas and Genders (Hou et al
2012)

321

4. CONCLUSIONS
The AR training is able to generate paralleled learning curves in between male and female assemblers. In other
words, AR training is equally effective to both male and female. Assemblers after AR training are able to achieve
better performance compared with those who do not use AR. However, training using manual prints is more
effective to male assemblers, as the female assemblers typically spent more time completing each formal assembly
trial and committed more error within each trial. It is also concluded that the training combined AR visualization
means can be applied in training assembly novices, as against to manual prints the effectiveness and efficiency of
AR have been scientifically supported based on the analysis of post-training performance, as the better
performance the AR trainees can achieve, the better effect of information recall and the shorter learning curve can
be generated. As the previous finding has indicated a higher level of cognitive workload using manual prints as
training schema, it is proved that the training scheme which induces a higher level of cognitive workload could
more impact females task performance relevant to memorizing effect. However, this performance disparity
between two genders derived from manual prints training schema can be curtained or eliminated when AR is
introduced.
5. REFERENCES
Hou, L, and X. Wang. 2010. A Novel Application in Guiding Assembly Task: Augmented Reality Animation.
Proceedings of The 2010 International Conference on Computing in Civil and Building Engineering (ISCCBE)
and The XVII EG-ICE Workshop on Intelligent Computing in Engineering, June 30-July 2, Nottingham, UK.
Hou, L., X. Wang., L. Bernold, and P. Love. 2012. Using Animated Augmented Reality to Cognitively Guide
Assembly: an Experimental Perspective, ASCE Journal of Computing in Civil Engineering (in press).
Kato, H., and M. Billinghurst. 1999. Marker Tracking and Hmd Calibration for a Video-Based Augmented Reality
Conferencing System: IEEE.
Kato, I.P.H., M. Billinghurst, and I. Poupyrev. 2000. "Artoolkit User Manual, Version 2.33." Human Interface
Technology Lab, University of Washington.
322

EFFECTIVENESS OF VIRTUAL REALITY ON PRESENTING FIELD
MONITORING DATA
Hsieh, Yo-Ming
National Taiwan University of Science and Technology, Taiwan
Lu, Ya-Sue
ABSTRACT: Field data collected by automated instrumentations provide valuable information regarding the
state of the surrounding in construction projects. Monitoring data are mostly used to infer the safety of
construction projects in-progress. They are also used to safeguard nearby facilities. Todays practice on
presenting them is mostly based on conventional two-dimensional (2D) maps and charts. Such practice is not
optimal. In this paper, authors propose using three-dimensional (3D) virtual reality approach for presenting
monitoring data. The system was used to evaluate its effectiveness against 2D charts, which are common practice.
Our evaluation shows the interactive 3D-VR approach developed helps greatly users to interpret monitoring data.
Both correctness and effectiveness of data interpretation are improved by more than 50%.
KEYWORDS: field monitoring, virtual reality, effectiveness, monitoring information system
1. INTRODUCTION
Modern construction projects often use automated monitoring to collect monitoring data related to safety.
Some examples are measurements on building tilt, retaining wall deflection, and water pressure. These
measurements record true site response to engineering activities. They are also used to understand safety
conditions of the construction site and its nearby facilities. Unfortunately, todays practice on presenting these
monitoring data is rather primitive. As a result, valuable information that can be obtained from these data
cannot be fully appreciated. Furthermore, basing on the result of this study, the efficiency of interpreting data is
not optimal.
To fully appreciate monitoring data, three pieces of information must be fused: 1) environment, 2) deployment of
sensors, and 3) monitoring data (Hsieh and Lu, 2012). In the past, the environment and deployment of sensors
are presented using 2D maps. Monitoring data is presented using separated charts. To fully understand data,
one must fuse all these information their mind, resulting in different interpretation of monitoring from different
viewers. In extreme events that remedial reactions must be formulated according to monitoring data, such
inconsistency will hinder the final decision.
In this work, a 3D-VR approach for presenting monitoring data is introduced. To realize such presentation, a
visualization system was developed. In this paper, the visualization system for conducting 3D-VR
representation of monitoring data is first introduced. Then, how to present scalar monitoring data in 3D space
will be discussed. Finally, an evaluation for assessing the effectiveness of the 3D-VR approach is presented.
The evaluation suggests 3D-VR approach can significantly improve users ability or efficiency on interpreting
monitoring data.
2. VISUALIZATION SYSTEM ARCHITECTURE
In this study, a visualization system was built 1) to demonstrate how data can be fused in a virtual reality or
simply 3D environment, and 2) to evaluate effective of such approach. The system is composed of three
essential elements. The first element is an information system for collecting and managing monitoring data.
The second element is the way for building 3D field models. The last and the most crucial element in this study
is the visualization system. The following three sections describe these elements. Figure 1 illustrates the
interaction of these three elements.
2.1 Information system
The information system (IS) is the first important element for managing field monitoring data. It enables data
323

retrieval, data-storage, and backup. Effective management of monitoring data enables: 1) quick assessment of
service conditions of built structures (Wong 2004, Li et al. 2006), 2) sculpturing better computer models (Finni
and Calvello 2005, Hashash et al. 2006), 3) building credible decision support systems (Cheng and Ko 2002,
Cheng et al. 2002), and 4) accumulation of knowledge through data-mining and regression (ORourke and
McGinn 2006). Conventionally, such information system is designed and implemented using the client-server
architecture. Such architecture is mature and well understood. However, such architecture may not be
suitable for future generations of automated monitoring. Thus, the information system used in this research
was designed with distributed computing and service-orientation (Hsieh and Hung 2009, Hsieh and Chen 2012).
The information system is composed of five discrete services: field service, coordination service, data service,
alert service, and project service. The visualization system created in this study mainly interacts with the data
service and project service to retrieve monitoring data and project field model.
2.2 3D field model
Field 3D models in this study play important roles. They help viewers understand spatial distributions of
instrumentations, monitoring data, and nearby pre-existing structures. Therefore they assist viewers gain insight
to the system behavior based on monitoring. It is well known that creating 3D models can be tedious,
labor-intensive, and expensive due to both software and labor costs. To overcome these obstacles, Google
SketchUp is used as the modeling tool for creating field 3D models. There are several advantages of using this
tool: 1) it is free, 2) it can integrate with Google Earth to get satellite imagery of the site to facilitate model creation,
3) it is easy to use, and 4) it can access Google 3D warehouse to download community-made building models and
other model components to ease the work of modeling. Thus, using Google SketchUp can alleviate the cost for
creating field models.
Once users have built the desired field 3D model, it can then be exported using COLLADA file format, which is a
specialized XML-based file format for data exchange for interactive 3D applications (Wikimedia 2009). The file
can then be read by the developed visualization system using COLLADA-DOM.
It should be noted the main purpose of the field model is to give viewers an overview to the monitored site.
They serve as the context or the background for the monitoring data. Therefore, detailed building information
such as those provided by BIM (building information models) is not necessary.
2.3 Visualization system
Once monitoring data (provided by IS introduced in 2.1) and field model (prepared by SketchUp and stored in IS)
are available, they are then processed and presented by the visualization system. The toolkit used for
developing the visualization system is VTK (Schroeder 1997). It is an extensive and well-developed class
library for scientific visualization. The toolkit was used to facilitate interactive visualization based on
high-level programming concepts such as pipelines and design patterns (Gamma et al. 1994) rather than
low-level APIs such as OpenGL and callback functions. Furthermore, using VTK enables future extensions of
the system using advanced visualization techniques such as contouring and iso-surfaces with ease.
Information System
Data Service
Alert Service
Field Service
Project Service
Coordin
ation
service
[VTK]
[VTK]
Monitoring Data
[Web service API]
3D Field Model
[COLLADA]
Visualization System

Fig. 1: Visualization system architecture
3. Visualization
The visualization system designed fuses display of 3D field model and monitoring data. This section describes
324

some core functions regarding visualizing monitoring data in the developed visualization system. These core
functions include: 1) representing monitoring data, and 2) filtering. These are discussed subsequently.
3.1 Scalar representation in 3D
In this study, the field monitoring data concerned are scalar data. Neither descriptive data collected by
engineers nor vector data such as wind velocity and 3D displacements are concerned in this work.
To fuse scalar data in 3D environments, virtual geometrical entities are used. These virtual entities do not
present actual objects in the field. Their purpose is to give scalar data presences in the 3D environment.
Although one may use text labels such as 13.45 or 3.14 to show measured quantities, this is only suitable to tiny
amount of measured data. It is easy to imagine that if there were thousands of sensors collecting data, the
whole display would be cluttered with text labels everywhere and it would be impossible to read any of them.
Therefore, virtual geometrical entities placed at measured locations are used to present scalar data collected.
Three strategies, illustrated in Fig-2, are used in this study to present scalar data with virtual geometrical entities
in the 3D environment. These strategies are orthogonal and can be used simultaneously. These three
strategies are described subsequently.
The first strategy is mapping scalars to geometrical properties such as radius and height, i.e. Fig-2a. In the
developed system, four geometrical shapes were tried: sphere, hexagon, cone, and cylinder. When using
spherical entities, data affect their radius, and in turn determine the volume or size of the spherical entity. The
larger the sphere is, the larger the data it presents. One nice thing using spheres is they are invariant to
viewpoints. No matter which viewpoint the viewer is taking, these data-carrying spheres look exactly the same.
The main disadvantage using spheres is that it cannot represent negative values. Other three shapes (hexagon,
cone and cylinder,) all map quantities to be represented to their heights. These three representations are quite
opposite to sphere. They look differently when viewer takes different viewpoints, and they can represent
negative values.
The second strategy is mapping scalars to color. Such strategy needs to define a color transfer function that
maps scalars with RGB values of color. Often, red corresponds to the maximum value while blue corresponds
to the minimum value, which can be negative. Defining such color transfer function requires some thought.
Some colors have implications that are culture dependent. For example, red often implies danger or warning to
engineers, and they become alert and worry when they see such color. On the other hand, in Taiwans stock
market, stock price shown in red means the price is raising and green means the price is dropping. As a result,
investors become happy when they see lots of red numbers and become sad when they see lots of green numbers.
It should be noted that it is necessary to show the color-mapping legend in order to allow users understand how
values are mapped to colors, as shown in Fig-2b.
The last strategy enhances the former two. Former two strategies have one major disadvantage; they disallow
exact values to be observed directly. Thus, the last strategy uses the balloon widget found in VTK. The
balloon widget allows users to move their mouse cursor onto interested geometrical entities, and after a
programmer-specified period, additional information balloon shows up. The information balloon would show
the sensor id, sensor data, and sensor type in text. Such approach avoids cluttering the display with too many
texts. Texts only show up when necessary.
3.2 Filtering
Although human vision is capable of filtering data very quickly by identifying different colors, shapes, etc., it
becomes difficult when the number of readings increases. Furthermore, presenting data in 3D space using
geometrical properties or colors does not allow users read exact values. These exact quantities are important to
identify existing structures that needs immediate attention based on certain rules (e.g. government regulations,
handbooks). Therefore, it is necessary to help users filter monitoring data based on their specifications in the
system.
To present data that satisfies the user-specified criteria, two techniques are used: 1) adding a symbolic icon on
the data point, and 2) removing data that fail to meet the user-specified criteria. These two techniques are
illustrated in Fig-3 and discussed in the next two paragraphs.
Adding symbolic icons around data points meeting user-specified criteria helps viewers find the data of interest.
Meanwhile, data that do not meet the criteria are also displayed. This technique is suitable when there are only
325

few data points displayed on screen, and that adding icons do not introduce much clutter and confusion. This
technique gives viewers more information since the unqualified data may contain information that is only
revealed after deep inspection of data.
The other technique only displays data points that meet user-specified criteria. Unqualified data are hidden
from the user. This technique is helpful when there are too many data to be displayed on screen at once, and
previous technique may introduce confusion that hinders viewers comprehension of the displayed information.

Spher e
R
Dat a=R

Cube
H
Dat a=H

Cone
H
Dat a=H

Cyl i nder
H
Dat a=H

(a) Mapping scalars to geometrical properties

(b) Mappingscalarstocolor (c) Balloon widget shows the exact quantity
Fig 2: Fusing scalar representation in the 3D environment with the field model
4. EFFECTIVENESS OF VISUALIZATION
To understand the effectiveness of VR representation of monitoring data, user testing was conducted. The user
testing compares users performance, including response time and correctness, using two different approaches to
interpret monitoring data. The first approach is the 3D-VR approach. This approach visualizes monitoring
data fused with sensor deployment and environment information in 3D. This is done using the developed
visualization system previously described. The second approach is the 2D-MC approach. This approach uses
2D maps to present sensor deployment in the field and charts (e.g. bar chart or scatter plot) to present monitoring
data. This 2D-MC approach represents the current industry practice that authors have observed. In practice,
most of the time the sensor deployment map are prepared using CAD drawings, and sometimes they are
managed by with geographical information systems (GIS). Nevertheless, their final presentations are almost
always bar charts, scatter plots and planar maps. In subsequent sections, user testing conducted in this research
is described. First, the test participants and the testing method are described. Then, using the results of user
testing, the effectiveness of 3D-VR approach can then be revealed.
326

4.1 Test Participants
The test participants in this evaluation were 30 sophomore and junior year undergraduate students major in
construction engineering. This group of students had some understanding of civil and construction engineering
but un-experienced in reading field monitoring data. In other words, participants had similar level of
acquaintance for both 3D-VR and 2D-MC.

(a) Add extra icons/symbols for data points meeting criteria

(b) Hide filtered data
Fig 3: Two approaches to filter monitoring data
4.2 Testing method
Each participant was tested individually in our laboratory. Before the test was started, each participant was
given an introductory session explaining questions listed in Table 1. These questions were formulated with
three considerations. First, these questions are easy to understand and answer with considerations of
educational background of participants. Second, these questions have practical engineering values. Finally,
answers to most of these questions depend on digesting multiple monitoring data, understanding their spatial
distribution, and performing spatial searching. These activities are common tasks for interpreting monitoring
data. It is hoped by evaluating measuring participants performance on these questions can prove the proposed
visualization system can indeed benefit these tasks, and therefore provide aid to help engineers understand
327

monitoring data better.
Once participants had agreed that they understood the introduction and instructions, they would then do two case
studies using two different approaches to interpret the settlement data and answer questions listed in Table 1.
Reasons of doing two cases are explained in the next paragraph. The questions were given to participants one
at a time by our assistant, and time needed to answer each of the questions was recorded. For questions in
Table 1 that requires identifying buildings, participants were required to point out buildings instead of reporting
building numbers. After tests were finished, their answers would then be checked to see if their answers were
accurate according to our instructions. Therefore, for each participant, both time and accuracy on answering
each question was collected. These data were then used to evaluate the effectiveness of the 3D-VR approach
compared to the 2D-MC approach.
Table 1: List of questions given to participants for effectiveness evaluation
Q1: Please identify the building with the largest settlement.
Q2: Identify buildings with the top five largest settlements.
Q2-1. Are these buildings close to each other
1
?
Q3: Building number X has shown visible cracks on walls, and building residents are asking for your help.
After a site visit, there are two engineering activities nearby identified: 1) a deep excavation three blocks
away, and 2) a subway tunnel construction that is also about three blocks away. Which construction is
responsible for this accident
2
?
Q4: Are buildings in the specified area (on the map) safe
3
?
Q4-1: If there are unsafe buildings in the specified area, identify three most dangerous buildings.
1
We define close to each other as within one street block.
2
We instruct students to look at the variations of the measured settlement between the constructions sites
and the buildings.
3
Safe is defined as having settlement less than 2.3cm.
p.s. The above definitions are given both during introductory sessions and in their questionnaires.

After the introductory session, the designed questions listed in Table 1 were then answered by participants using
3D-VR or 2D-MC approach. 3D-VR approach uses visualization similar to ones shown in Fig. 3 generated by
our visualization system. For 2D-MC approach, a 2D map as shown in Fig. 4 is prepared for participants.
Additionally, participants are given two sets of charts, one set having settlement data ordered by sensor ID, and
the other having data ordered by their magnitude. Sensor IDs were designed to be building numbers on which
they are installed followed by a serial number. For example, sensor ID 54-1 shows this sensor is the first sensor
installed on building number 54. It should be noted in this study, usability of the developed visualization
system was not evaluated. We only concern whether 3D-VR approach is more effective than the 2D-MC
approach. Therefore, participants were not given instructions on how to use our software. They were only
instructed on what was shown on screen, and our assistant would help them operate the software if they wanted
to change the viewpoint of visualization, or navigate in the 3D models, etc.
The reason why each participant needs to do two test cases, case 1 and case 2, was that we wanted to be able to
compare each users performance on using both 3D-VR and 2D-MC approaches. Doing so enabled us to
compare ones performance using both methods. If we allowed one user do the same case using both methods,
results of the former method will likely affect results of the later. Therefore, each participant was asked to work
on two similar cases; one case used 3D-VR approach, and the other case used 2D-MC approach. These two
cases shared the same plane view but had different settlement data. Furthermore, results of these 30
participants were grouped into group A and group B according to the testing sequence they performed. Group
A first used 2D-MC approach on case 2 and then used 3D-VR approach on case 1; group B first used 3D-VR
approach on case 2 and then used 2D-MC approach on case 1. Doing so should allow us to eliminate
discrepancies caused by using one method first and the other method second, giving us a fair comparison of the
two methods.
Once tests are finished, results from 30 different participants were collected. These results are discussed in the
next section.
4.3 Effectiveness of visualization
Table 2 summarizes performances of 30 participants, showing benefits of using 3D-VR approach over 2D-MC
328

approach. Due to the designed questions listed in Table 1 and the dataset, there was no need to answer Q4-1 for
case 2, and therefore no data was collected for Q4-1 in case 2. Table 2 can be divided into four sectors: 1) case
1 using 2D-MC approach, 2) case 2 using 2D-MC approach, 3) case 1 using 3D-VR approach, and 4) case 2
using 3D-VR approach. Sector 1 and 4 list performance by 15 participants; sector 2 and 3 list performances of
the other 15 participants. By comparing sector 1 to 3 or sector 2 to 4, one can answer whether the 3D-VR
approach is better than the 2D-MC approach.

Fig. 4: 2D map prepared for 2D-MC approach
We first focus on the accuracy summarized in Table 2. No conclusive advantages or disadvantages can be seen
using either method. Most questions were answered correctly by more than 85% of the participants, except
Q2-1 using 3D-VR approach in case 1. Reason for this anomaly requires more investigation to be performed.
Table 2: Participants performance using two approaches for interpreting monitoring data
Case 1 Case 2
Response time (sec.) Response time (sec.)
avg. stdev max. min.
Accuracy
avg. stdev max. min.
Accuracy
Q1 17.1 8.7 37.9 8.0 1.00 21.7 15.2 53.9 3.2 0.93
Q2 30.9 16.0 68.0 15.2 0.87 37.6 20.3 72.3 10.8 0.93
Q2-1 14.6 9.1 37.5 2.9 0.87 12.2 6.9 25.0 1.0 1.00
Q3 83.0 51.2 216.9 26.5 1.00 75.9 32.1 130.8 13.7 0.93
Q4 44.4 34.2 124.0 3.0 0.80 60.9 26.9 97.9 16.2 0.93
2
D
-
M
C

Q4-1 6.7 6.5 22.1 0.0 0.91
Q1 14.5 4.5 23.3 4.0 1.00 5.6 1.8 8.5 2.4 1.00
Q2 16.4 9.2 33.3 5.2 0.80 23.0 7.8 38.6 12.6 0.93
Q2-1 7.3 8.3 22.6 1.0 0.33 3.8 4.5 12.3 0.4 1.00
Q3 23.7 11.6 45.6 8.1 0.87 29.2 11.7 50.5 9.9 1.00
Q4 23.1 27.4 120.0 5.3 0.93 14.4 8.8 30.8 2.5 0.87
3
D
-
V
R

Q4-1 4.8 4.8 16.2 1.0 0.93
Accuracy is defined as (number of correctly answered questions)/(total number of
questions).
Sector 1: case 1 interpreted using method 2D-MC
Sector 2: case 2 interpreted using method 2D-MC
Sector 3: case 1 interpreted using method 3D-VR
329

Sector 4: case 2 interpreted using method 3D-VR
Sector 1 and 4 are performed by 15 participants.
Sector 2 and 3 are performed by the other 15 participants.
Then, we focus on the time needed to answer the designed questions in Table 1. Assuming 3D-VR approach
improves the efficiency of interpreting data, an improvement ratio (IR) is defined as:

2 3
2
D MC D VR
D MC
time time
IR
time

=
(1)
The ratio is computed on both the average and the standard deviation of the recorded response time. A positive
IR on average time suggests 3D-VR approach reduces time to interpret data; a positive IR on standard deviation
suggests 3D-VR approach reduces differences of performance of different participants. In other words, positive
IR on standard deviation meaning improved uniformity of group performance using 3D-VR approach over
2D-MC approach.
Fig. 5 shows the computed IR from Table 2. The horizontal axis is the IR of average response time and the
vertical axis is the IR of the standard deviation. It is seen 3D-VR approach does indeed show improvements
over 2D-MC approach. 3D-VR approach on average reduces users response time by over 50%, implying the
proposed visualization reduces time for engineers to interpret monitoring data by 50%. Therefore, engineers
can use less time to understand the monitored system performance, and reach decisions and devise solutions to
unfavorable engineering events more promptly by using 3D-VR approach.
Similarly, the improvement of standard deviation seen in Fig. 5 suggests using 3D-VR approach yields more
consistent performance of participants over 2D-MC approach. In other words, the improvement of using
3D-VR approach over 2D-MC approach is universal and consistent.
In summary, the evaluation conducted in this study suggests the proposed 3D-VR approach for visualizing
monitoring data fusing with the environment and the deployment configuration is very effective.

0%
20%
40%
60%
80%
100%
0% 20% 40% 60% 80% 100%
I
m
p
r
o
v
e
m
e
n
t
o
f
p
e
r
f
o
r
m
a
n
c
e
u
n
i
f
o
r
m
i
t
y
Improvementofaverageresponsetime
Case1
Case2

Fig. 5: Improvement due to using 3D-VR approach over 2D-MC approach
5. SUMMARY
3D-VR approach introduced in this work for presenting monitoring data is proven to be more effective than
conventional 2D-MC approach. It is seen that using the 3D-VR approach to present monitoring data can
improve the efficiency of interpreting monitoring data by more than 50%. Furthermore, this improvement is
observed uniformly across all participants. This suggests most people can benefit from using 3D-VR approach
to help them interpret monitoring data.
330

6. ACKNOWLEDGEMENT
Authors would like to thank funding support from National Science Council of Taiwan under grant number: NSC
95-2221-E-011-184.
7. REFERENCES
Cheng M.-Y., Ko C.-H., Chang C.-H. (2002), Computer-aided DSS for safety monitoring of geotechnical
construction, Automation in Construction, Vol. 11, 375390.
Cheng M.-Y., Ko C.-H. (2002), Computer-aided decision support system for hillside safety monitoring,
Automation in Construction, Vol. 11, 453466.
Finno R.J., Calvello M. (2005), Supported excavations: observational method and inverse modeling, Journal
of Geotechnical and Geoenvironmental Engineering, Vol. 131, 826836.
Hashash Y.M.A., Marulanda C., Ghaboussi J., Jung S. (2006), Novel approach to integration of numerical
modeling and field observations for deep excavations, Journal of Geotechnical and Geoenvironmental
Engineering, Vol. 132, 10191031.
Gamma E., Helm R., Johnson R., Vlissides J.M. (1994), Design Patterns: Elements of Reusable
Object-Oriented Software, Addison-Wesley Professional.
Hsieh Y.M. and Chen Y.H. (2012), Critical reliability assessments of distributed field-monitoring information
systems, Automation in Construction, Vol. 26, 21-31.
Hsieh Y.M. and Hung Y.C. (2009), A scalable IT infrastructure for automated monitoring systems based on the
distributed computing technique using simple object access protocol Web-services, Automation in Construction,
Vol. 18(4), 424-433.
Hsieh Y.M. and Lu Y.S. (2012), Visualization system for field monitoring data and its effectiveness,
Automation in Construction, Vol. 26, 54 68.
Li H., Ou J., Zhao X., Zhou W., Li H., Zhou Z, Yang Y. (2006), Structural health monitoring system for the
Shandong Binzhou Yellow River Highway Bridge, Computer-Aided Civil and Infrastructure Engineering, Vol.
21, 306317.
O Rourke T.D., McGinn A.J. (2006), Lessons learned for ground movements and soil stabilization from the
Boston Central Artery, Journal of Geotechnical and Geoenvironmental Engineering, Vol. 132, 966989.
Schroeder W., Martin K., Lorensen B. (1997), The Visualization Toolkit: An Object-Oriented Approach to 3D
Graphics, Kitware, Inc.
Wikipedia (2009), COLLADA, http://en.wikipedia.org/wiki/COLLADA
Wong K.Y. (2004), Instrumentation and health monitoring of cable-supported bridges, Structural Control and
Health Monitoring, Vol. 11, 91124.

331

SOLAR RADIATION SURVEY METHOD WITH IMAGE STITCHING
Yoshihiro Yasumuro, Takashi Shimomukai, Hiroshige Dan & Masahiko Fuyuki
Faculty of Environmental and Urban Engineering, Kansai University
ABSTRACT: Coupled with higher population density and urbanization, verticalization of buildings has become
popular in many cities. From the point of view of energy consumption, residential building rooftops also receive
attentions by playing a significant role in the deployment of solar technology, especially after the disastrous
nuclear accidents lately. However, shading from surrounding buildings can limit the access of solar radiation to
the rooftops. Therefore, it is desirable to examine the quantitative accesses of solar radiation to each building
rooftop and the efficiency before solar unit installation. Existing major estimation methods for the solar radiation
are based on the averaged value for each regions; e.g. prefectures or city scales. We focus on more personal,
easy-to-use approach to cope with local conditions including solar interferences such as adjacent higher buildings
and trees around. Existing techniques using vision sensors show good possibilities to take the local conditions into
account for solar estimation, however, the equipments used for them might be too specific for general users; e.g.
stereo cameras, fish-eye cameras and so on. We propose a simple image-based method using panoramic imaging.
Image stitching technique allows to create a large view-angle image by a normal perspective camera. Further-
more, mobile smart devices in late years are equipped with not only a camera but also a digital gyro and a
compass, which are able to attach azimuth and elevation angles into the camera images technically. We utilize the
combination of these sensors. Multiple camera images are combined by feature points correspondences extracted
and selected by SURF and RANSAC algorithms. Since giving the on-site latitude value allows to calculate the
solar trajectory for any dates, solar visibility simulation is possible in a way that the solar trajectory is observed
through the only open-sky regions in the superimposed panoramic image. The visible part of the trajectory can be
virtually integrated to calculate the radiation as amount of heat. Some experimental results showed the effec-
tiveness by clearly differentiating the locational conditions.
KEYWORDS: Image stitching, Panoramic image, Insolation survey, Solar radiation, Mobile computing
1. INTRODUCTION
Coupled with higher population density and urbanization, verticalization of residential buildings has become
popular in many major cities. The sudden appearances of high-rise apartment buildings in existing residential areas
are often problematic in terms of the impairment of scenery, perspective occlusions, and insulation matters. From
the point of view of energy consumption, while solar energy is one of the major alternative power resources,
residential building rooftops also receive more and more attention by playing a significant role in the deployment
of solar technology, especially when considering the threats of nuclear accidents. However, it is clear that the
high-rise buildings can be inevitable obstacles that interrupt the sunshine from many other lower rooftops in
dense residential area. Therefore, it is desirable to examine the quantitative accesses of solar radiation to each
building rooftop and the efficiency before solar unit installation. In essence, the insolation conditions are differ-
ent for every point on an each roof surface. The sun changes its position over short and long scale of time period.
Therefore insulation survey must cover the spatio-temporal distribution of solar radiation on the surface within
the target surfaces. This paper proposes an obstacle measurement scheme for insulation survey by employing
simple image processing with a mobile camera with a gyro sensor to effectively capture surrounding's panoramic
shade distribution. Generally speaking, any kinds of terrestrial objects could be possible shading obstacles. Con-
sidering the complex-shaped objects such as trees nearby, image-based measurement has a good potential for
capturing the geometrical conditions of them, comparing ranging method using point-by-point targeting.
2. RELATED WORK
Tomori et al. (2000) conducted triangulation based on two images taken from two different ground positions.
Triangulation provides 3D surrounding information that allows one to survey multiple interest points in the 3D
space. Tomori uses the same scheme as normal stereo-camera triangulation to calculate the nearby buildings'
height and distance, which can be used for estimating the shaded area on the ground. One of the key aspects for
accuracy is the setup of two different camera positions. Normally, the stereo-image measurement quality signifi-
cantly depends on the camera setup, including initializing the relative camera positions and orientations. Espe-
cially in outdoor situations, ground height and inclination vary over the positions; therefore, adjusting and meas-
332

uring relative camera setups are important and troublesome in on-site work.
Using a camera with a fish-eye conversion lens, wide-angle views can be captured as spherical images. Therefore,
fish-eye images have been used often for recording sky-dome images to investigate the solar radiation situation in
forest ecosystems. Schwalbe uses a hemispheric image modeling technique to determine the solar radiation dis-
tribution on the ground level (Schwalbe 2009). Fish-eye images are used for segmenting the open-sky regions and
the occluded regions with tree shades to estimate solar radiation. Yamashita also uses hemisphere photos taken in a
temporal series to record the cloud distribution on the sky-dome and its solar radiation (Yamashita 2004). To
simplify the camera setup and on-sight work, a vertical triangulation method using fish-eye camera is then pro-
posed (Yasumuro 2010). This method requires a small space to setup and a simple calibration procedure, howev-
er, the specification of the camera resolution and a fish-eye lens are not so common.
3. PROPOSED APPROACH
3.1 Overview
We focus on a easy-to-use system not for specialized
users such as forest ecosystem biologists but more gen-
eral users like solar power customers. We propose a
simple on-site image sensing method that can be used for
estimating solar radiation spot by spot, e.g. when you
figure out which roof of your house is effective for in-
stalling solar power units. A hand-held camera with a
gyro sensor is a popular combination quipped for many
smart-phones and tablet PCs, lately. Using this combina-
tion, taking multiple photos allows you to get a pano-
ramic imagery accompanied with ray direction infor-
mation associated with each pixel on it.
This paper also provides an image-based processing scheme required for computing the global amount of solar
radiation. Finally, this paper shows the feasibility of the proposed scheme, which uses the following steps:
1) Take multiple photos
2) Construct a panoramic images by image stitching technique based image feature correspondences.
3) Convert panoramic image and images to correct the principle axis and angle of view
4) calculating the heights of the surroundings by triangulation with corresponding points between 2 images
The details of the each step is described in the following subsections.

Fig. 1: Basic idea: A hand-held camera with a gyro
sensor can capture the surrounding scenery as a
panoramic image by image stitching technique, that
shows opened or shaded sky regions.

Fig. 2: Process flow of the proposed method
333

3.2 Image Mosaicing for Panoramic View
3.2.1 Cylindrical Projection
The images needed to be projected onto a cylinder from planar projections of the scenes in order to fit them
together as a panorama. The computation can be performed in the inverse form of the equation for cylindrical
projection:
x = f tan (1)
y = h / cos (2)
This converts cylindrical coordinates , h to planar coordinates x, y. The variable f is the focal length, the radius of
the cylinder. We also calibrate the intrinsic camera parameters, including image center, focal length and field of
view range, which are necessary for associating each pixel of the photo to an individual ray direction. I our
method, taking photos is performed together with a gyro sensor whose directional data is associated with the
image center.
3.2.2 Feature-based Correspondences
In order to create a panorama image, the general idea consists in identifying common points between a pair of
images and then projecting one of the images on top of the other so as to match those points. In order to identify
those points, which can be called as interest points, we use an interest point detector SURF (Speed Up Robust
Features). SURF method is a quick scale invariant feature detection method (H. Bay 2006) and speeds up the
feature detection several times than the SIFT (Scale Invariant Feature Transform) (D. Lowe 2004). It computes
horizontal and vertical pixel differences, dx and dy, in local coordinate system for rotation and scale invariance.
And then it accumulates dx, dy, and |dx|, |dy| over 4x4 pixels sub-regions or 3x3 pixels sub-regions. It then nor-
malized for gain invariance, but distinguishes bright blobs and dark blobs based on the sign of Laplacian.
3.2.3 Image Matching

Fig. 3: Image Stitching Example: For successive images ((a) and (b)), SURF feature key-points are extracted ((c) is
from (a)) , RANSAC algorithm selects the "inliers", correct key-point pairs to match (b) to (a) as shown in (e). The
line segments in (e) shows the key-points pairs and too long ones are "outliers" in this case.
334

From the detected features, correspondences between successive images need to be found. We achieve this by
using the nearest distance search over the entire features detected. Among all the correspondence, we need the
build the Homography model of the projective transform, i.e. we need to find the homography matrix as below:
The homography matrix describes the projective transform, and h
00
, through h
22
, h
01
are the transform parame-
ters. For each pair of potentially matching images we have a set of feature matches that are geometrically con-
sistent and a set of features that are inside the area of overlap but not consistent.
RANSAC is an iterative method for robust parameter estimation to fit mathematical models from sets of ob-
served data points which may contain outliers. The algorithm is non-deterministic in the sense that it produces a
reasonable result only with a certain probability, with this probability increasing as more iterations are performed.
The basic assumption of the method is that the data consists of "inliers", i.e., data whose distribution can be ex-
plained by some mathematical model, and "outliers" which are data that do not fit the model. Outliers could be
considered points which come from noise, erroneous measurements, or simply incorrect data selection. For the
problem of homography estimation, RANSAC works by trying to fit several models using some of the point
pairs and then checking if the models were able to relate most of the points. The best model, i.e., the homogra-
phy which produces the highest number of correct matches, is then chosen as the answer for the problem as
shown in Fig. 3.

3.3 Astronomical Solar Trajectory
In our method, the sun trajectory needs to be described in an angular space equivalent to cylindrical coordinate,
which is expressed by elevation and azimuth relative to the measurement point as shown by following
equations:
= sin
1
{sin () sin () + cos () cos () cos (h)} (3)
= tan
1
{cos () cos () sin (h) / sin () sin () sin () } (4)
where, is declination of the sun, h is our angle and is latitude.
3.3.1 Declination of the Sun
The earth moves around the sun in a year, rotating once a day. The axis of rotation of the earth has inclination
of 23.27 degrees. Since this inclination is constant, the incident angle of the ray from the Sun relative to the
plane of the equator is determined by the yearly round of the earth. This is the declination of the sun, whose
value becomes -23.27 [deg] on the winter solstice, 23.27 [deg] on the summer solstice, and 0 [deg] on the
spring and autumn equinox. The value of can be calculated equation (5) and (6), where d
n
[day] is the number
of days from New Year's day.

Fig. 4: Solar Trajectory: Arbitrary date can be applied to plot the sun position in angular space.
1 1
22 20
13 12 10
03 02 01
y
x
h h
h h h
h h h
w
y
x
335

0 = 2 (dn 1) / 365 (5)
= 0.006918 0.399912 cos (0) + 0.070257 sin (0)0.006758 cos (20)
+ 0.000907 sin (20) 0.002697 cos (30) + 0.001480 sin (30) (6)

3.3.2 Geocentric Distance of the Sun
The distance between the earth and the sun is the geocentric distance. Since the orbit of the earth is not a circle
but a ellipse, the geodesic distance (r/r') is determined by the position of the earth, or ecliptic longitude, as
shown in equation (8).
r/r' = 1 / {1.000110 + 0.034221 cos (0) + 0.001280 sin (0) ()
+ 0.000719 cos (20) + 0.000077 sin (20)}
0.5
(8)

3.3.3 Equation of Time
Just as the orbit of the earth is an ellipse whose focus point is located on the sun. Therefore the revolution speed
of the earth changes according to the distance from the sun; the faster the near from the sun. In fact, the inclina-
tion between the rotation axis and the revolution axis of the earth causes differences in length of each day. Equa-
tion of time is derived by calculating the difference between the actual length of a day and that of the case as-
suming that the rotation axis and the revolution axis are parallel and the evolution velocity is constant on a circle
orbit, on the same celestial longitude, as follows:
Eq = 0.000075 + 0.001868 cos (0) 0.032077 sin (0) ()
0.014615 cos (20) 0.040849 sin (20) (9)
s
3.3.4 Hour Angle of the Sun
Hour angle is time expressed by angle at a rate of 360 degrees per 24 hours. Hour angle is useful for expressing
the path of a planet on the celestial sphere. The arc passing through true north, vertex, true south and nadir is
meridian. The hour angle is supposed to be zero when a planet reaches to culmination meridian, and counted up
along the west direction. Applying this to the sun, hour angle h can be expressed as follows:
h = (JST 12) /12 + (difference of longitude from the standard meridian) + Eq , (10)
where, JST is Japan standard time [h].

3.4 Amount of Solar Radiation
Overlaying the generated panoramic image onto the solar trajectory in the identical angular space, the amount of
solar radiation can be estimated. (see Fig. 4)Solar trajectory expressed with the elevation and the azimuth
derived by Equation (4) and (5) rasterized on the panoramic image plane whose pixels are also associated to the
ray directions in a cylindrical coordinate. The open-sky regions can be extracted by choosing the sky color with
some similarity range after converting the RGB pixel values to HSV color values. The unchosen regions are used
for masking the solar trajectory for considering the losses of radiation caused by physical shading. This process
can easily performed by pixel-by-pixel logical AND operation. In general, the amount of global solar radiation q
[W/m2] is used for solar power estimation. For this calculation, we use equation (11) as follows;
q = 1367 (r/r')
2
sin (), (11)
where, r/r' [rad] is the geocentric distance from Equation (8) and 1367 [W/m2] is a solar constant. We can ac-
336

quire the amount of global solar radiation q for each points of the "visible" solar trajectory. The total amount Q is
a temporal integration of the global solar radiation q. We can compute this integration numerically as sum of
products pixel-by-pixel along the "visible" solar trajectory as Equation (12).
Q' = 1367 (r/r')
2
sin ((t)) dt (12)
We use OpenCV library plotting the solar trajectory and masking it by logical AND operation on the panoramic
image.

4. EXPERIMENTS
We chose 3 location in our university campus for
applying the proposed method as shown in Fig. 5.
Point #1 is supposed to be comparatively open
space with grass. This place has open sky on the
north side and three are some trees and buildings
on the south side. Point #2 is also an open square
surrounded by some buildings and some trees on
southeast. Point #3 is on a path leading down to
southeast, sandwiched by some buildings and
trees nearby. The date and time we took the pho-
tos at Point #1, #2 and #3 are 2012.1.16 (11:55),
2012.2.09 (13:00) and 2012.2.05 (15:00), respec-
tively.
We used a laptop PC (OS: Microsoft Windows
XP Professional with CPU: Intel Core 2.33 GHz
and 2GB Memory), 3D motion sensor
(MDP-A3U9S by NEC/TOKIN inc. :Euler angle
output: X-axis:0-180 deg, Y-axis: 0-90 deg,
Z-axis:180 0-deg) and a USB Camera (QuickCam
by Logicool inc., 640x480 pixel, fov:48x42 deg,
CMOS sensor). The generated panoramic images and overlaid solar trajectories are shown in Fig. 6. The esti-
mated values for amount of the global solar radiation are shown in Table 1.

Fig. 5: Schematic "visible" solar trajectory for computing the amount of solar radiation for the measurement
point.

Fig. 6 3 Locations for the experiments: The point #1 and
#2 are open spaces with farther buildings in different direc-
tions and point #3 is on a street surrounded by buildings .
337

The results show that the clear differences
of visibilities of solar trajectories, depend-
ing on the local surroundings. These dif-
ferences also causes relative differences of
solar radiation. Both of Point #1 and Point
#2 are located at the center of open spaces.
Trees and Some buildings are further from
the measurement point in Point #2 than in
Point #1. Since the date to the experiments
were in winter season, the distribution of
the elevation of the obstacles around the
south have profound effect on total amount
of solar radiation. In fact, the square space
of Point #1 is larger than that of Point #2,
but the amount of the radiation is higher in
Point #2, because of the configuration of
the distance and height of the buildings
relative to the solar trajectory of that date.
As for the Point #3, which is on a narrow
path surrounded by some buildings on ei-
ther sides comparatively catch a good
amount of radiation. As shown in Equation
(11) and (12), integrated radiation amount
highly depends of the elevation value of the
visible segments of the trajectory. It is im-
portant that the more portions of the trajec-
tory with the higher elevation are visible.
We have shown that the proposed method
have a useful potential for variance analysis
of radiation estimation, considering the
very local surrounding situations.
We have not conducted quantitative analysis of the proposed system, including the precision of the direction
measurement of pixel by pixel and absolute illuminance measurement. We are planning to validating the geo-
metrical and radiation measurement error, comparing to the reference from 3D laser scanning data and an illu-
minance meter data for the same spot.
The implementation so far is based on assembly of a gyro sensor and a USB camera connected to a normal lap-
top configuration. One of our major step is to implement more all-in-one style of system. employing a tablet type
PC or a smart phone type platform.

Fig. 7: Visible solar trajectories for computing the integration of the total amount of radiation, depicted on azi-
muth-elevation space for Point #1(left), Point #2(middle) and Point #3 (right)

Fig. 7: Visible solar trajectories (yellow lines) for Point #1 (a),
Point #2 (b) and Point #3 (c).
338

REFERENCES
Ellen Schwalbe, Hans-Gerd Maas, Manuela Kenter, and Sven Wangner (2009). Hemispheric Image Modeling
and Analysis Techniques for Solar Radiation Determination in Forest Ecosystems, Photogrametric Engineering
and Remote Sensing, Vol. 75, No. 4, 375-384.
Juho Kannala and Sami Brandt (2006). A Generic Camera Model and Calibration Method for Conventional,
Wide-Angle, and Fish-Eye Lenses, IEEE Trans. on PAMI, vol.28, 1335-1340.
Li Shigang, Nakano Masao, Chiba Norishige (2004). Acquisition of Spherical Image by Fish-eye Conversion
Lens, Proceedings of the 2004 IEEE Virtual Reality, 235-236.
Toshiyuki Tomori, Kenji Otani, Koichi Sakuta, and Kosuke Kurokawa (2000). On-site BIPV array shading eval-
uation tool using stereo-fisheye photographs, 28
th
IEEE Photovoltaic Specialists Conference, 1599-1602.
Yamashita M., Yoshimura M., Nakashizuka T. (2004). Cloud Cover Estimation Using Multitemporal Hemisphere
Imageries, Geo-Imagery Bridging Continents XXth ISPRS Congress, 826-890.
Yoshihiro Yasumuro, Takuya Goto, Hiroshige Dan, Masahiko Fuyuki (2011). Vertical Triangulation with Fish-eye
Camera for Insolation Survey, CONVR2011, pp. 332-340, 2011.
H. Bay, T. Tuytelaars, and L. Van Gool (2006). Surf: Speeded up robust features. ECCV 2006, 404-417.
M. Brown and D. Lowe. Automatic panoramic image stitching using invariant features (2007). International
Journal of Computer Vision, 74(1), 59-73.
M. Fischler and R. Bolles (1981). Random sample consensus A paradigm for model fitting with applications to
image analysis and automated cartography. Communications of the ACM, 24(6), 381-395.
D. Lowe (2004). Distinctive image features from scale-invariant keypoints, International journal of computer
vision, 60(2), 91-110.
Table 1: Estimated Radiation Values
Location Point #1 Point #2 Point #3
Sampling Date & time 2012/1/17 11:55 2012/2/9 13:45 2012/2/9 15:20
Latitude [deg] 135.51 135.51 135.51
Longitude [deg] 34.72 34.72 34.72
Geocentric distance [rad] 0.9834 0.98631 0.9856
Declination of the sun [rad] -0.3648 -0.3648 -0.2822
Equation of time [min] -0.0406 -0.061 -0.0599
Hour angle [rad] -0.0625 0.3966 0.8118
Elevation [deg] 34.2643 35.8482 22.6136
Azimuth [deg] -4.0587 31.2506 49.5896
Sunup-Sundown time 7:08-17:05 6:54-17:29 6:58-17:25
Global solar radiation
(full direct light) [W/m2]
4334 6940 5587
Global solar radiation
(our method) [W/m2]
2053 5205 2449
339

PHOTOGRAMMETRIC MODELING AND SLING LENGTH
MEASUREMENTS OF A RIGGING SYSTEM ENGINEERED FOR
INDUSTRIAL MODULE ERECTION
Ming-Fung Francis SIU, Sheng MAO, Ming LU & Simaan ABOURIZK
Department of Civil & Environmental Engineering, University of Alberta, Edmonton, Alberta, Canada
ABSTRACT: An industrial construction/turnaround project consists of existing site facilities and temporary
facilities (scaffolds, rigging, and cranes). Cost-effective modeling and visualization of site operations can be
instrumental in analyzing and revealing potential workface space conflicts and refining the estimate of man-hours
and required task time. In previous research, the Photo-AR requires at least two control points with known
coordinates (stationary points fixed in the site) to be explicitly included and visible in each photo in order to enable
photogrammetry processing. This research enhances Photo-AR by synchronizing two robotic total stations to track
two dynamic points on the moving object. The resulting methodology would better cater to the application needs of
4D industrial schedule visualization in terms of modeling dynamic temporary facilities critical to construction
efficiency and safety performances (such as rigging, scaffolding, carnage). In this paper, a mock experiment using
a moving small box is described to illustrate the hybrid technology, followed by field implementation of the
proposed methodology to check sling lengths of the engineered lift frame in a module assembly yard. Conclusions
are drawn based on the findings from lab and field experiments.
KEYWORDS: Photogrammetry, robotic total station, surveying integration, real-time tracking, synchronization,
rigging system, lifting frame.
1. INTRODUCTION
As Canadas leading producer of oil, gas, and petrochemicals, Alberta is home to four ethane-cracking plants,
including two of the worlds largest, with combined annual capacity to produce 8.6 billion pounds of ethylene; in
the foreseeable future, new refining capacity will be added to produce ethane from bitumen upgrading which will
directly source feedstock from downstream oil sands mining (Government of Alberta 2010). New construction and
turnaround activities at industrial process plants consume enormous amounts of equipment and manpower
resources and involve dozens of diverse stakeholders who work closely towards delivering a project under a finite
time window and a tight cost budget. In general, a work item (a pipe spool, a valve, or a storage tank) undergoes a
sequence of tasks in a fabrication shop, at a module yard, or on an industrial site. Each task is conducted by
specialist manpower resources in a confined work space with the assistance of temporary facilities (e.g., rigging
system, scaffolds, and cranes). A newly engineered rigging system by a major industrial contractor is designed to
handle super modules with a maximum 160-ton lift capacity. There are length-adjustable slings connecting the
frame and overhead plate of the rigging system to facilitate the lift. The rigging frame system is made of steel and
subjected to bending under loadings (Westover et al. 2012). Therefore, the real-time sling length measurements are
critical to balance the frame during its own assembly and heavy lift processes.
Traditional surveying, by using total station, is common practice to determine the position in the site space based
on traverse and leveling algorithms. Mainstream surveying research focuses on improving the accuracy of data
being collected, such as the least square adjustment algorithm (King 1997). Similarly, the state-of-the-art robotic
total station adds automation capability to total station survey, finding construction engineering applications in
monitoring such as building settlement monitoring, bridge deflection, and tunnel boring machine guidance (Shen
et al. 2011). At the current moment, the drawbacks of robotic total stations include the high investment and
application costs and its limited capability to fix only one point at one time.
Photogrammetric surveying has been widely applied in medical surgery, 3D modeling, engineering, manufacturing,
and map production. By regarding photos as the most straightforward, cost-effective means of collection of data in
the field, virtual and augment reality research efforts based on photos are predominant in the construction
management domain (Kamat et al. 2011). Physical dimensions of construction resources were collected for quality
control purposes (Dai and Lu 2010). Image processing required significant efforts to build 3D models based on
photos. Its realism requires solid surveying and mathematical foundations. The conjugated point pairs are
identified in photos. Collinearity and coplanarity equations are solved for determining relative orientation
followed by the absolute orientation; alternatively, bundle adjustment algorithms can be applied to simultaneously
340

calculate all parameters including the object coordinates. Interested readers may refer to McGlone (2004) and
Mikhail et al. (2001). Due to its complexity, construction engineering and management researchers attempted to
reduce the photos used by imposing geometric constraints and automate the modeling process based on pattern
recognition and feature detection (Golparvar-Fard et al. 2009). A photogrammetry-enabled AR approach
(Photo-AR) was applied to assist in crane allocation and sequencing of bored pile foundation construction on a
congested building site (Siu and Lu 2010, 2011). The depths for various geological layers are color-coded on a pile
to be constructed while a completed concrete pile is rendered in solid grey. Its mathematical foundation may refer
to Siu and Lu (2009).
A Photo-AR scene, linked with a scheduled event on an industrial construction/turnaround project, will consist of
existing site facilities and virtual models of temporary facilities (scaffolds or cranes). The AR scene will be
instrumental in analyzing and revealing potential workface space conflicts and refining the estimate of man-hours
and required task time. In previous research, the Photo-AR requires at least two control points having known
coordinates (stationary points fixed in the site) to be explicitly included and visible in each photo. This research
enhances Photo-AR by synchronizing two robotic total stations to track two dynamic points on the moving object.
This would better cater to the application needs of 4D industrial schedule visualization in terms of modeling
dynamic temporary facilities which are critical to construction efficiency and safety performances (such as rigging,
scaffolding, carnage). The hybrid use of photogrammetry and two synchronized robotic total stations is the most
cost-effective dynamic approach to Photo-AR: multiple control points on the object being modeled (end points of
slings or beams) can be fixed in 3D simply by processing photos taken from different angles at the same time,
while the two robotic total stations will be synchronized to survey two control points on the object to ensure the
accuracy of all the measurements resulting from Photo-AR. In this paper, a mock experiment using a moving small
box is first given to illustrate the hybrid technology, followed by field implementation of the proposed
methodology to check sling lengths of the engineered lift frame in a module assembly yard. Conclusions are drawn
based on the findings from the lab and field experiments.
2. RESEARCH METHODOLOGY: MOCK TESTING ILLUSTRATION
In this section, the overall research framework is presented. Both photogrammetric and robotic total station (RTS)
surveying techniques are firstly introduced in terms of their respective role in the hybrid technology applied,
followed by the lab mock testing.
2.1 Leading-Role Technology: Photogrammetric Surveying
The proposed technique to survey multiple points on a moving object is called close range photogrammetry, as
mentioned in the previous section. The advantages in applying photogrammetric techniques compared to
traditional surveying include (1) non-contact measurements, (2) cost-effective field processing, and (3)
measuring multiple points at one particular time. Photo-based 3D models can be built for a given state of the
object by using two to three photos taken from different angles simultaneously. The coordinates of two control
points at that particular time point are prerequisite for photogrammetric surveying to fix the absolute dimensions
of the photo-based model and take geometric measurements from the model.
Three Canon T3i cameras (CAD$699.98 each in May 2012) fixed at 18mm focal length are each set up (Fig. 1)
for photogrammetric surveying in lab experiments. The cameras internal clocks were synchronized before
laboratory and site experiments. The time stamps of photos in each dataset were examined and compared, so as
to guarantee the synchronization of cameras.
In order to reduce the turnaround time for actual site testing, Eye-Fi cards (Fig. 2) are used to facilitate the
wireless photo transfer from the cameras to a laptop. The models are Connect2 4GB (CAD$39.99) and Pro2
8GB (CAD$99.99). The card functions as a normal SD card for camera image storage (Eye-Fi 2012). As Wi-Fi
signal receiver and transmitter are embedded in the Eye-Fi card, the time of transferring three photos based on
resolution (5184 3456; 4.58 MB) is within 30 seconds with the Internet or within 60 seconds without the
Internet. According to the specifications, the typical working range of the wireless network is 27m outdoors and
13m indoors.
Camera calibration is essential to determine the parameters for interior orientation and lens distortions. The
interior orientation configures the geometric dimensions of a camera, which include the principle distance and
location of principle point. The lens distortion inevitably results from the processes of lens manufacturing and
assembling. Thus, the parameters from the calibration can be used to regulate the image coordinates and hence
341

increase the quality of the surveying results. Multiple-sheet calibration method supplied by PhotoModeler (Eos
System 2012) was used for high quality calibration (Fig. 3). There were 125 coded targets (5 coded targets on
each calibration sheet, and 25 sheets were printed) for automated coded targets detection. Three cameras were
successfully calibrated (Fig. 4) and the parameters were stored in photogrammetric software for modeling
adjustments.

Fig. 1: Canon T3i cameras Fig. 2: Eye-Fi card

Last Processing Attempt: Fri Jun 08 23:44:33 2012
Camera1: Canon EOS REBEL T3i [18.00]

Focal Length: Value: 18.768697 mm
Xp - principal point x: Value: 11.315538 mm
Yp - principal point y: Value: 7.708430 mm
Fw - format width: Value: 22.686819 mm
Fh - format height: Value: 15.113000 mm
K1 - radial distortion 1: Value: 5.103e-004
K2 - radial distortion 2: Value: -1.221e-006
K3 - radial distortion 3: Value: 0.000e+000
P1 - decentering distortion 1: Value: 1.100e-005
P2 - decentering distortion 2: Value: 1.284e-004

Average Photo Point Coverage: 87%
Point Marking Residuals: Overall RMS: 0.204 pixels
Maximum: 0.676 pixels

Fig. 3: Camera calibration Fig. 4: Internal parameter configurations
2.2 Supplementary Technology: Robotic Total Station Surveying
Robotic total station (RTS) is the most accurate on-site survey instrument and is capable of providing
millimeter-level accuracy for coordinate measurement of target points (Leica Geosystems 2012). As each total
station can only measure one target at a time, two available RTS units are synchronized to simultaneously
measure two points through wireless automation command control. An in-house developed tablet program has
been built so as to facilitate remote control and real-time synchronization of RTS units during site experiments.
The robotic total station models used in field testing are TS15I and TCRP1203+.
To set up the coordinate system, RTS
1
is assigned as the origin of the coordinate system, and the direction,
pointing from RTS
1
to Circular Prism
1
, is defined as north. Thus, the east direction can be established as
cross product of the directions of north and zenith (perpendicular to the ground) being defined. A built-in
RTS computer program is used to automatically initiate the coordinate system set up. RTS
1
and RTS
2

automatically lock on, track, and measure one reflector (glass) prism, each which is discussed in later sections.
Two measurements are simultaneously taken by operating the developed automation control program. The two
point measurements with the same time stamp are used as inputs for photogrammetric surveying.
2.3 Moving Object Tracking Procedures
In the laboratory testing, two cameras, synchronized by cable communication, were set up to capture the moving
beam, fixed on cross support, at one particular instant (Fig. 5). The configurations of lines and points are shown
graphically in Fig. 6. The objective of this testing is to introduce the hybrid techniques. The measurement
accuracy is also examined against manual measurements. Two photos were taken to build one photo-based 3D
model. Simultaneously, the length of line #10 was determined by using two synchronized robotic total stations.
Eq. (1) was employed to calculate the sling length based on the coordinates of two end points. The length
(393.718mm) was used as input to set the scale of the 3D model as shown in Fig. 7. The length between two
markers on the small box was then measured in PhotoModeler against tape measurements. It was found that the
difference between two sets of results is within 0.4mm on average with the standard deviation of 0.1mm (Table
342

1). The results prove the feasibility of taking measurements and positioning on a moving object at one particular
time by the proposed hybrid technique.
2
2 1
2
2 1
2
2 1
) ( ) ( ) ( Z Z Y Y X X L Eq. (1)

Fig. 5: Equipment setup Fig. 6: Locations of prisms and markers
Fig. 7: Photo-based 3D model built in PhotoModeler
Table 1: Comparisons between photogrammetric surveying and hand measurements
From To Actual (mm) Measured (mm) Difference (mm)
Point 2 Point 3 100.000 99.708 0.292
Point 3 Point 4 99.800 99.491 0.309
Point 4 Point 5 100.000 99.856 0.144
Point 5 Point 6 100.200 100.070 0.130
Point 6 Point 7 100.400 100.475 0.075
Standard deviation 0.093
3. RIGGING SYSTEM ASSEMBLY: YARD TESTING
Construction site experiments using the proposed hybrid methodology in the rigging assembly yard are described
in this section. The entire structure consists of slings connecting the horizontal beam of the rigging frame and the
overhead connection plate (Y-plate). The plate is linked to the mobile crane during lifting. The aim of the testing
was to check the sling lengths on the back side of the lifted rigging frame without loading to ensure the
343

as-designed geometric configuration.
Fig. 8 shows the locations of installed prisms (for automatic robotic total station tracking) and markers (for
photogrammetric surveying) on the entire rigging system. The movable prism was directly placed on the
non-movable markers (Fig. 9). When surveying the beam, the two prisms are moved manually to particular pin
locations. The robotic total station survey results will be compared to photogrammetric data for validation and
verification. Figs. 10 and 11 depict the rigging system and measuring instrument setup on site, respectively. A
total of eight reflector prisms are available. Six prisms were fixed on Y-plate while the remaining two serve as
movable prisms to be manually placed on the beam (Fig. 12). After the measurements are obtained, the lengths
of the slings (Fig. 13) were adjusted to ensure the beam is leveled during lifting.

Fig. 8: Prism and marker positions Fig. 9: Markers for photogrammetric surveying
Fig. 10: Rigging system Fig. 11: Equipment setup
Fig. 12: Marker and prism positions Fig. 13: Sling length adjustments
3.1 Site Constraints
The weather during field experiments was cloudy and windy. Testing was conducted to check sling lengths after
removing the loading from the frame during lifting.
344

Owing to the large rigging system being measured, the rigging frame and cameras are significantly separated in
order to capture the entire structure. The cameras are therefore separated to a length which exceeds the length of
cable for synchronization, to ensure the geometric strength of the photo-based model. The camera
synchronizations were currently done manually. People are involved to control the cameras. Photos are collected
using wireless transfer and selected based on checking the closest time stamp data recorded in the photos. The
procedure to build the photo-based 3D model is semi-automated by picking the corresponding centre point of
markers.
The rigging frame swung significantly due to the wind, making it difficult for robotic total stations to track the
position of the markers manually. Currently, the synchronized total stations are the only solution which
automatically provides two points on a moving object at one particular time point. The robotic total station
surveyed two targets: one (Model: TS15I) was responsible for surveying the prism on the Y-plate, and the other
(Model: TCRP1203+) was to lock on the prism on the beam simultaneously (Fig. 14). The results of the
surveying at one particular time point were converted into sling lengths by Eq. (1).
The equipment, including (1) the trash bin, and (2) the mobile crane, blocked the line-of sight when surveying
the back side of the frame using both cameras and robotic total stations. The trash bin was removed, but the
crane was present throughout lift testing. Thus, 115 feet must be available between the cameras and the frame
and clear line-of-sight must be available, allowing full coverage of the whole frame structure in all the photos.

Fig. 14: Synchronized robotic total station surveying
3.2 Field implementation and testing results
The interface and the procedure for photo-based modeling and photogrammetric surveying on site are shown in
Fig. 15. Two point-coordinates obtained from synchronized RTS (reflector-less laser mode), which were used to
fix the length of Sling B (L.), refer to Fig. 8, informed photo modeling. This would fix the scale of the model and
obtain the sling lengths as shown in the laboratory experiment.
The photogrammetric results were cross-checked with the ground truths provided by robotic total station to
guarantee the data quality. Table 2 shows the results obtained from photogrammetric surveying. Three sets of
data were compared. Table 3 depicts that the standard deviation between average values of the two data sets is
11mm. The data were verified and therefore utilized for sling length adjustments.
Note that the B (R.) sling measurement data was initially misleading from RTS, owing to the failure in tracking
the correct prism on the Y-plate on site. The end pin of the B (R.) sling on the Y plate was mistaken by the end of
A (R.) sling due to a small angle between the prisms on Y-plate and robotic total station. The photogrammetric
surveying data caught this mistake and could potentially compensate the RTS missing measurements.
345

Fig. 15: Photo-based modeling for site testing
Table 2: Surveying data
Slings
Photogrammetry (m) Ground truths (m)
[10.18.37am] [10.19.42am] [10.18.45am] [Trial 1] [Trial 2] [Trial 3]
A (L.) 21.451 21.463 21.454 21.482 21.478 21.483
B (L.) 18.188 18.188 18.188 18.197 18.189 18.178
C (L.) 16.265 16.268 16.257 16.253 16.26 16.256
C (R.) 16.257 16.268 16.260 16.266 16.261 16.263
B (R.) 18.188 18.233 18.183 18.212 18.215 18.218
A (R.) 21.498 21.571 21.434 21.491 21.507 21.500
Table 3: Photogrammetric and robotic total station data comparisons
Slings
Loading removed (Data averaged)
Photogrammetry (m) Ground truths (m) Difference (mm)
A (L.) 21.456 21.481 25
B (L.) 18.188 18.188 -
C (L.) 16.263 16.256 -7
C (R.) 16.262 16.263 1
B (R.) 18.201 18.215 14
A (R.) 21.501 21.500 -1
Standard deviation 11
To guarantee that sling lengths are consistent with the design and that the main beam of the frame is leveled
during lifting and assembly, the theoretical lengths (as-designed value) of slings were computed based on
geometrical configurations. After obtaining the coordinates of two end points of each sling from hybrid
technology, the length adjustment of each adjustable sling is determine using Eq. (2). A spreadsheet is developed
to automatically read the coordinate inputs and produce the adjustment values. Fig. 16 shows the graphical
presentation of sling adjustment. Further research could be done in (1) analyzing the structural loading based on
the sagging magnitude provided by tracing the three segments of the entire sling (Fig. 17), (2) automating the
modeling by using image detection methods such as pattern recognition and feature detections.
Surveyed Design As
L L L

Eq. (2)
346

Fig. 16: Graphical presentation of sling length adjustment
Fig. 17: Sling sagging for structural analysis
4. CONCLUSIONS
This research study proved the potential and feasibility of photogrammetric surveying supplemented by robotic
total stations in checking moving rigging frame dimensions during the dynamic assembly process. The automatic
target tracking and point survey synchronization enabled by robotic total stations were used to provide two-point
measurement data for photogrammetry in Photo-AR, as demonstrated in laboratory and construction site
experiments. The photogrammetric surveying technique was introduced to survey multiple points at one
particular time. Through integration and automation, the hybrid photogrammetric surveying technology and the
robotic total station complement one another to guarantee accuracy, reliability, and completeness in the
measurement data obtained in real time, cost-effectively.
347

5. ACKNOWLEDGEMENT
The authors would like to acknowledge the following for their contributions in site testing. Ulrich Hermann and
Jacek Olearczyk (manager and construction coordinator of construction engineering, PCL Industrial
Management Inc.) for providing site access and facilitating the mobilization of site equipment; Xuesong Johnson
Shen (postdoctoral fellow of University of Alberta) for helping develop the robotic total station synchronization
technique; Hexu Liu (PhD student of University of Alberta) for assisting in developing the algorithm for sling
length adjustments. Special thanks to Xiaodong Wu (MSc student of University of Alberta) for equipment
transportation; Brenda Penner and Duojia Helen Chen (administrative assistants of University of Alberta) for
equipment purchase; Amy Carter and Celine LePage (technical writers of University of Alberta) for streamlining
the paper in terms of technical writing; Bruce King (associate professor at Hong Kong Polytechnic University)
for his educational efforts towards Francis Siu and his valuable comments in photogrammetric surveying.
6. REFERENCES
Bethel J. S., Mikhail E. M., McGlone J. C., Mullen R. (2004). Manual of photogrammetry, 5
th
Edition, American
society for photogrammetry and remote sensing.
Dai F. and Lu M. (2010). Assessing the accuracy of applying photogrammetry to take geometric measurements
on building products, Journal of construction engineering and management, 136(2), 242-250.
Eos Systems (2011). PhotoModeler Quick start guide, Eos Systems Inc.
Eye-Fi (2012). Products Compare cards, Eye-Fi Inc.
Government of Alberta. (2010). Albertas energy industry: an overview 2009, published by Government of
Alberta: http://www.energy.alberta.ca/Org/pdfs/Alberta_Energy_Overview.pdf.
Golparvar-Fard M., Pea-Mora F., and Savarese S. (2009). D
4
AR- A 4-dimensional augmented reality model for
automating construction progress data collection, processing and communication, Journal of information
technology in construction, 14, 129-153.
Kamat V. R., Martinez J. C., Fischer M., Golparvar-Fard M., Pea-Mora F. and Savarese S. (2011). Research in
visualization techniques for field construction, Journal of construction engineering and management, 137(10),
853-862.
King B. A. (1997). Some considerations for the statistical testing of least squares adjustments of
photogrammetric bundles, The photogrammetric record, 15(90), 929935.
Leica Geosystems (2012). Product overview Total stations, Leica Geosystems AG.
Mikhail E. M., Bethel J. S., McGlone J. C. (2001). Introduction to modern photogrammetry, Wiley.
Shen X. S., Lu M., and Chen W. (2011). Tunnel-boring machine positioning during microtunneling operations
through integrating automated data collection with real time computing, Journal of construction engineering and
management, 137(1), 72-85.
Siu M. F. and Lu M. (2011). Augmented-reality visualizations of bored pile construction, Proceedings of the
2011 CSCE Annual General Meeting & Conference (CSCE2011), 10 pages, Jun. 14-17, 2011, Ottawa, Canada.
Siu M. F. and Lu M. (2010). Bored pile construction visualization by enhanced production-line chart and
augmented-reality photos, Proceedings of 10th International Conference on Construction Applications of Virtual
Reality (CONVR 2010), 165-174, Nov. 4-5, 2010, Sendai, Miyagi, Japan.
Siu M. F. and Lu M. (2009). Augmenting site photos with 3D as-built tunnel models for construction progress
visualization, Proceedings of 9th International Conference on Construction Applications of Virtual Reality
(CONVR 2009), 187-196, Nov. 5-6, 2009, Sydney, Australia.
348

Westover L., Olearczyk J., Hermann U., Adeeb S. and Mohamed Y. (2012). Module rigging assembly dynamic
finite element analysis, Proceedings of Canadian society of civil engineering annual conference, June 6-9,
Edmonton, Alberta, Canada.
349

NAVIGATION IN VIRTUAL REALITY USING MICROSOFT KINECT
Florian Kammergruber, Andreas Ebner, Prof. Willibald A. Gnthner
Technische Universitaet Muenchen, Institute for Materials Handling, Material Flow, Logistic, Germany
ABSTRACT: Currently interaction in a virtual reality system is performed by the help of a tracking system, or with
some special input devices. Some examples of these well-known devices are motion platforms, haptic interaction
devices with force feedback, data gloves, and navigation devices like a flystick or a game controller. Not all of them
can be used in the construction industry regarding their sturdiness and ease of use. There are continuously new
developments, especially in the area of mobile input devices. These are driven by the game console market, which
satisfies the requirements of the users for wireless input devices and for innovative approaches which integrate the
player interactively with his body movements in the game. The latest of such devices was developed by Microsoft
for the Xbox game console: "Kinect". This device processes video and infrared information of a camera system to
enable markerless tracking, which is based on certain geometries and reference images of the user. The recorded
three-dimensional body movements are used to create a skeletal model of the user and his entire body can be
tracked.
The use of these properties allows an officially released SDK from Microsoft. During the development of new
interaction concepts for highly mobile virtual reality systems, the Kinect has been integrated into a virtual reality
system as an input device for navigation. With a mobile Virtual Reality system in a building container, the system
can be used for planning purposes and navigation through the construction site. Predefined gestures and
movements of the user enable a totally free and intuitive navigation and control of the system in the immersive
environment. The implemented approach at the Technischen Universitaet Muenchen is characterized by extremely
low-cost hardware, and coded as a plug-in for the RTT DeltaGen software.
KEYWORDS: navigation, interaction, Microsoft Kinect, gesture tracking
1. 3D NAVIGATION IN VIRTUAL REALITY
An essential property of Virtual Reality (VR) is its interactivity, which allows the user to interact with the virtual
world. Numerous input devices that serve as a man-machine interface within the VR system were developed for it.
Possible characteristics of the interaction are navigation in the virtual world, tampering with objects and feedback
via a force feedback. The requirements for the interaction are speed, purposefulness and especially intuitive
handling, which may not have a recognizable delay between input and output for real time suitability and has a
high positional accuracy in space. Using suitable and intuitive operating metaphors, the conducted actions and
gestures of the input devices are converted into movements and actions in the virtual world. This is necessary due
to the abstraction of the model character, as the 3D model is only displayed but does not exist in reality. The
operating metaphor therefore is based on the logic of how human body movements can be used for a control to
steer something which does not exist in reality.
For most interactions in VR or input devices, a continually tracking or a spatial determination is necessary (Ong
and Nee, 2004). Spatial movements in the real world are transferred to the digital and virtual world with the help of
the calculated coordinates. According to the specific application and purpose of usage, input devices can be used
without a tracking system.
1.1 Optical Tracking Systems
Due to their basic physical principle tracking systems, they can be divided into acoustic, electromagnetic, inertial,
mechanical and optical tracking as well as hybrid systems. In the last few years, optical tracking systems have been
the most prevalent. An optical tracking system operates with one or more digital cameras to record objects that will
be followed in the tracking area. The tracking area can be extended scalable by using additional cameras. Video or
infrared cameras are used for orientation within the tracking area using reference geometries like marker, targets,
shapes or edges (Wagner, 2005).
The optical tracking using infrared cameras is especially popular and excels in its high precision and sturdiness by
the exact calculation of position even in the case of rapid movements. A flashing infrared light sent out by the
cameras is reflected by the special markers. Using the known constellations and dimensioning of the single targets,
350

the position of the target recorded out of different perspectives can be calculated using triangulation. These
infrared systems with cameras, software and special markers are very expensive to purchase. The reference
geometries can be of passive or active form whereby passive targets are recognized by the cameras from their
pattern or their solid geometries. In contrast, active targets send out their own signals by being equipped with
LEDs, which send out infrared light (see Fig. 1).

Fig. 1: Infrared tracking camera as well as a passive and active target (ART, 2012)
Aside from the infrared systems, optical image processing systems are increasingly used in which a video camera
(in the simplest case a webcam) evaluates the picture of the surroundings with image-processing software. This
technology is most frequently used in the area of augmented reality. In this case, the system orients itself from the
targets, which are mostly made up of 2D matrix codes and have a black/white pattern that can easily be processed.
Such video systems cost less than infrared systems, as the camera and the software are significantly cheaper. In
return, the tracking is less sturdy and depends a lot on the lighting conditions as well as the angle between the
camera and the 2D matrix codes.
1.2 Input Devices
For the complete depiction, an overview is given of common - and in some cases - special input devices as a
man-machine interface to VR systems, which are especially intended for navigation in the virtual world. In the area
of mobile input devices in VR, there are constantly new developments or prototypes designed for specific
applications (Bowmann, 2005). On the other hand, there is the market for gaming consoles which creates an
increasing variety of mobile interaction devices. This is based on the market demands of the users for wireless
interactive devices and on the demands of the providers who, by using innovative approaches, try to interactively
integrate the gamer into the game.
1.2.1 Motion platforms
To depict the natural walking movement of a person in VR, different technical concepts were developed to enable
a movement close to reality. The problem is that the user can move unrestrictedly in a virtual and
three-dimensional world, but is limited by the size of the room in the real world. Therefore, the developments aim
to find a solution where the user moves on the spot, while the floor moves under him. The technical solutions range
from the Virtusphere, a pivoted hollow sphere (Medina et al., 2008), the CyperCarpet a turning table set up
with many spheres and a treadmill in front of a power wall (Wulz, 2008). Fig. 2 shows the mentioned solutions.

Fig. 2: Motion platforms Virtusphere, CyperCarpet and treadmill
These input devices are technically very sophisticated, but inflexible in their assembly position and very expensive
to build.
351

1.2.2 Navigation devices
Aside from the motion platforms, there are navigation devices that allow navigation in six degrees of freedom. One
of these navigation devices is the game controller (gamepad) from the area of gaming consoles and computers
(Riva et al., 2007). It has an ergonomic design and is used by gamers to move within the game world and carry out
various actions.

Fig. 3: RumblePad 2 made by the Logitech Co.
This wireless device has a range of up to 10 meters. As seen in Fig. 3, there are two analogue steering elements
(control sticks) for command input as well as an 8-way direction block and ten freely programmable buttons.
A further input device usable for VR is the Wii controller distributed by the Nintendo Co. for their gaming console,
which was already modified for a use in the area of VR (Lee, 2008). The ergonomically-shaped device can be
controlled simply by pressing a few buttons and is connected by Bluetooth. As a new and popular feature, the Wii
controller has three integrated acceleration sensors which recognize the position of the controller in the users hand
and its acceleration.

Fig. 4: Nintendo Wii controller with Nunchuck extension
The device, which can be obtained for almost $30, can be used individually or with further accessories as the
Nunchuck, for example. Fig. 4 shows the Wii controller and the separately available Nunchuck extension, which
has a control stick and more buttons.
Aside from the Wii Controller, the Nintendo Co. has also developed a balance board for the Wii gaming console
for interaction with the feet (see Fig. 5). It is equipped with four corner pressure sensors that record the applied
forces and thus the exertion of force by the feet or the body (center of gravity). The balance board is
battery-powered and connected by Bluetooth.

Fig. 5: Nintendo Wii balance board
The balance board was already successfully applied in another VR project of Haan et al. (2008) and also by
Hilsendeger et al. (2009) as an affordable interactive medium. They used the possibility of calculating the body
center of gravity to navigate dynamically in the VR using the users weight relocation.
352

However, input devices - which imitate a pointer and can thus portray a beam metaphor - are the industrys
most-used solution for interacting and navigating in VR. For these devices, the term flystick has been
commercially established, as it is very suitable for the flying through virtual scenes. Fig. 6 shows two different
flysticks that are controlled like a pistol and can be bought for approx. $1,000.

Fig. 6: Flystick models made by the A.R.T. Co. (ART, 2012)
Usually, the flysticks are connected with the VR control computer by wireless and allow navigation in all six
degrees of freedom. With their beam gesture, they are very suitable to operate virtual menus in three-dimensional
space (Bormann 1994). For navigation and interaction, the flysticks must be equipped with passive or active
tracking targets so their position can be determined by optic tracking systems. Additionally, the flysticks are
equipped with several control buttons where defined actions can be controlled.
1.3 Summary
In tracking procedures, it is shown that the optical infrared systems dominate the current VR equipment. Acoustic,
electromagnetic, inertial and mechanical ones are only rarely used for commercial purposes. The purchase and
usage of professional optical infrared tracking systems must still be assessed for approx. $15,000 (ART tracking).
This high purchase price is an obstacle for many companies using VR systems. Furthermore, the pursuit of objects
and people demands equipping them with suitable tracking targets.
One can see that for VR input devices, the solutions and products shown are also partially very well engineered and
can be used in the industry without a problem. These devices can, however, be very expensive and unwieldy,
especially if they are motion platforms or haptic input devices with force feedback. This paper describes the
development and adaptation of new intuitive operating metaphors and new interactive concepts that can be
implemented with the help of Microsoft Kinect. In the foreground is the very mobile usage of the input device as
well as its very low purchase price. The device is able to unite the tracking system and the input device into one
single system unit. The input has to be done as intuitively as possible and adjusted to the needs of navigation in
VR.
2. MICROSOFT KINECT
The interactive device Kinect (see Fig. 7) developed by the Microsoft Co. for the Xbox gaming console was
launched into the market in November of 2010. Initially, it was intended as an extension of the gaming console
to create a more interactive and natural gaming experience. Using gestures and movements, the user is able to
interact with the game world. Suitable games were developed for the specific use of this type of game navigation.

Fig. 7: Microsoft Kinect camera
However, it did not take a long for the first computer experts to dare to modify the camera. The Kinect was
353

connected to a PC and drivers for it were developed by them. Originally, this was not officially supported by
Microsoft. The first attempts of the software developers ranged from a three-dimensional recording of rooms and
objects, to a multitouch interface where photographs could be rotated and enlarged with two hands.
Non-experienced developers who used this technology that had been developed for years obtained fantastic results
from Kinect within a few days. Early in 2012, the first Kinect camera intended for use with a PC was released. It
featured an improvement in the close range of the tracking: Kinect for Windows 7. An official SDK was released
for this. The new Kinect was used for the functions and interactions described in this paper.
The Kinect is an input device for the detection of movements. The camera system enables the recording of
three-dimensional body movements of a person using the collaboration of a 3D camera which has optical and
infrared sensors and a stereophonic microphone. It combines both video and infrared information. From the
recording of the RGB camera and the depth information, the software calculates a skeleton model of the user with
the corresponding nodal points. Arms, legs, head and torso of the skeleton model move with only slight delay
synchronous to the user. Even in the case of changing users with different body sizes and measurements, the
camera recognizes their contours without problems. Astonishingly, neither calibration for a user nor special
markers on the body are necessary. The Kinect combines both the functioning of a tracking system and the
properties of an input device in a system environment. It enables the control and interaction with a PC through a
Natural User Interface (NUI) using gestures. The skeletal information has spatial coordinates of joints in a human
body. The x,y,z coordinates of joints are referenced to the Kinect hardware. The following joints are utilized for the
recognition of gestures in our project: head, left elbow, right elbow, hip center, hip left, hip right, shoulder center,
shoulder left, shoulder right, left wrist, right wrist, left hand and right hand. Fig. 8 shows all the joints and the RGB
picture.

Fig. 8: Skeletal View by the Kinect camera und RGB picture by the camera
With the skeleton model supplied by Kinect with its joint points, navigation in VR can be enabled using defined
gestures. For this, it is necessary to define the single gestures for the different movements and determine them
according to the joint points. In this paper, Version 10.0 of the RTT Deltagen VR Visualization software is used. As
a research partner of RTT, it is possible to obtain an SDK for RTT Deltagen, which enables it to access the position
and orientation of the camera and manipulate it. With the SDK by RTT a plug-in was developed that reverts to the
data of the Kinect camera and implements a corresponding translation and rotation of the observer camera using
the different gestures. Thus, the position of the observer in the digital scene can be controlled and navigation in VR
using gestures is enabled. The preceding papers allowed a navigation using different methods, all of them
demanding an input device on the user. For the methods developed so far, expensive equipment is necessary. In this
project, Kinect is suggested as an inexpensive tool for control and navigating. The advantages of NUI are: low
learning effort for using and controlling, learning by doing, and securing a direct interaction.
2.1 Definition of Gestures and Implementation
A small software was developed to obtain and gather the data from the Kinect sensor and to detect the gestures.
With the help of information on joints, many gestures are recognized. The coordinates are obtained and drawn in
an application. The red circles represent the right hand and the green circles the left hand. The blue circles have
head, spine and hip joints. The speed is shown in the scroll bar and a text box. The detected gesture is shown in a
text box. A provision is made to adjust the angle of the Kinect camera as well. The following Fig 9 shows the
described software window in the idle position.
354

Fig. 9: Software window depicting the idle state
In the next step, it was necessary to determine solid borders or areas for the recorded joint points. From these, the
software can recognize which gestures are made by the user at the current point in time. The first column of Table
1 shows a few of the defined and combined gestures that the user can make in front of the VR system in the
recording area of the Kinect camera. Of course, the corresponding opposites of the gestures shown such as turn
right, tilt down, pan right, etc also exist. Column 2 shows the recorded picture of the developed software and the
recognition of the gesture by the software. Column 3 describes the characteristic properties of the gestures, so that
they can be recognized as well as the resulting navigation movement.

Table 1: Gestures with Software Recognition and Characteristics

Gesture Software Recognition Characteristic

Bend backwards with hands down
Tilt Up

Raise the left hand above shoulder
sideways with the right hand down
Turn Left
355

Move the left hand across body
with right hand down
Pan Left

Raise both arms
Fly, move forward

Raise both arms and move
right hand across head
Fly and Pan Right
The source code necessary for development is displayed exemplarily in Figure 10. In it, the single rules for the
software to recognize and differentiate the gestures are stored. The x,y,z coordinates of the different joints are
taken as a basis and the gesture is defined according to their relative position to one another. If, for example, both
wrists are above the hip and the head, the gesture Fly is recognized. If, furthermore, the x coordinate of the elbow
is smaller than that of the shoulder joint, the gesture fly pan right is recognized. In this case, the origin of the
coordinate systems lies on the middle hip joint point.
356

Fig. 10: Pseudo code explaining the function determining the gestures with comments
In investigations, these gestures have been proven to be resilient and intuitive and can be used for the navigation in
six degrees of freedom in the VR. For that, the developed gestures are sorted out from the software and recognized
as described. Then they are transferred to the RTT plug-in, which implements these gestures into a translation
and/or rotation of the observing camera.

2.2 Possibilities and Limitations
One of the greatest obstacles for the application of the Kinect camera was the SDK of the Kinect, which was
written in C#. However, the programming language C++ was necessary for an integration and application in the
RTT SDK. Therefore, it was necessary to rewrite the Kinect SDK into C++. Unfortunately, the support by
Microsoft and the templates for the SDK in Kinect in C++ were very rudimentary.
Basically, you can conclude that the application of Kinect in VR offers new cheap possibilities in man-machine
communication. Each user has the possibility of independently defining new or different gestures and using them
for the interaction. The approaches analyzed in this paper only sufficed for the suitability of Kinect as a navigation
interface. A direct manipulation of objects in the virtual world and an interaction with them was not implemented.
For this operation, new operating metaphors would be necessary, as the Kinect camera cannot interpret and
evaluate the hands grabbing motions, for example. The problem of the somewhat coarse recording and tracking of
the user indicates that a head tracking is currently not possible. Thus, one cannot deliver a user-specific perspective
of the standard view in VR systems. Further developments concerning accuracy of resolution of the Kinect must be
awaited. Moreover, the condition of light and the surroundings also present a challenge. The room cannot be too
dark, as no clean tracking of the user via the camera can be obtained. Often this is exactly the case in VR systems
and could be a problem area in the application of the camera in CAVE facilities.
Aside from recording limitations, ergonomic aspects also play a role in evaluating the Kinect. Human beings are
not made for moving a long time with outstretched arms, as they become heavy and start hurting. This especially
happens during longer navigation and application sessions. For short VR sessions, this problem is not a limitation.
However, this problem also occurs in existing interactive devices. Regarding ergonometrics, it can also be
determined that the Kinect as such does not supply haptic feedback to the user, neither for the activation of a
function (e.g. pressing of a button) nor for possible collisions within the virtual world.
357

3. USE IN THE CONSTRUCTION INDUSTRY
In construction planning, it is necessary to create the possibility of analyzing selected conditions in a spatial model.
In the area of VR visualization, there are many previous research papers that show how construction sites can be
depicted in VR (Doulis et al., 2007) for detecting collisions between static elements of the construction site and
observing the current level of planning and progressed. Generally, VR is thus a useful instrument to create an
improved vision and a common view with reference to future intentions.
Due to external influences at construction sites, there are often unexpected delays that, in turn, change the
construction process. The detailed plans at the beginning of the construction thus become obsolete and another
plan must be created at once and all construction site workers must be informed about the newly planned processes.
With the help of VR, possible planning mistakes can be easily recognized and all participants can be given a clear
understanding of the planned alterations. Current planning can be visualized graphically and all people
participating actively at the construction site can discuss the corresponding problems together. For use in
construction sites, it is necessary for VR, to employ highly mobile VR systems. So construction site workers
accept the new technology, a simple intuitive and especially resilient solution for the interaction with the system is
necessary. The navigation via the Kinect camera offers these properties. Fig. 11 shows, as an example, how a
mobile VR-system could be integrated into a construction site container. A large visualization screen enables a
detailed and comprehensive observation of the construction site. The size of the screen also permits mutual
observation by several participants affected by the planning. The Kinect camera set above the visualization enables
a free and intuitive navigation in the immersive environment. Using the explained gestures, the user can navigate
freely through the VR visualization and emphasize the observation of selected details. The navigation can be done
by any user who stands in the center of the observation area of the Kinect camera.

Fig. 11: Construction trailer with integrated mobile VR-System and Kinect navigation
Using the research approach explained here, a prototypical implementation of control by gestures for mobile VR
systems was created that also enables a sensible usage of VR in the construction industry at the construction site.
Construction managers and planners can react to altered fringe conditions during construction and better assess the
consequences of their actions. Due to the transparency obtained by VR, planning during and before
implementation can be improved and costly mistakes in the construction process avoided. The intuitive control by
hand gestures lowers the inhibition level of users concerning the usage of the system and simplifies the interaction
with the VR compared with current interactive devices. Furthermore, the Kinect camera significantly lowers the
investment costs for a mobile VR system as no expensive infrared tracking systems is necessary. The investment
for the hardware is thus reduced to a visualization display (flat screen), a powerful graphics computer and the
Kinect camera. This system and the developed interaction techniques to visualize accompanying construction
should be implemented and tested in a pilot construction site at the Technische Universitaet Muenchen to evaluate
the potential of the new concept.
4. ACKNOWLEDGMENTS
The author gratefully acknowledges the support of the TUM Graduate Schools Faculty Graduate Center
Mechanical Engineering at the Technical University of Munich.
This research project was performed on behalf of the Federal Logistics Association (BVL) and funded by the
Federal Ministry of Economics and Technology via the Federation of Industrial Cooperative Research
Associations "Otto von Guericke" (AIF) by a resolution of the German Bundestag.
358

5. REFERENCES
ART - Advanced Realtime Tracking GmbH (2012): Company website, Retrieved January 27, 2012
http://www.ar-tracking.de/.
Bormann S. (1994): Virtuelle Realitt Genese und Evaluation, Bonn: Addison-Wesley.
Bowman D.A., Kruijff E., LaViola J., Poupyrev I.: 3D User Interfaces. Boston: Addison-Wesley, 2005.
Doulis M., Vogel M., Pflger J., Rietmann M., Raps M. (2007): 4Dive - A 4D Interface for the Visualization of
Construction Processes in a Virtual Environment. Proc. of the Conference on Construction Applications of Virtual
Reality, 2007.
Haan de G., Griffith E., Post F. (2008): Using the Wii Balance Board as a low-cost VR interaction device, In:
Proceedings of the 2008 ACM symposium on Virtual reality software and technology, Bordeaux, 289-290.
Hilsendeger A., Brandauer S., Tolksdorf J., Frhlich C. (2009): Navigation in Virtual Reality with the Wii Balance
Board, In: Virtuelle und Erweiterte Realitt, 6. GI-Workshop VR/AR, Aachen: Shaker, 269-280.
Lee J. (2008): Hacking the Nintendo Wii Remote, In: IEEE Pervasive Computing, Vol. 7, 39-45.
Medina E., Fruland R., Weghorst S. (2008): Virtusphere: Walking in a Human Size VR Hamster Ball, In:
Proceedings of the Human Factors and Ergonomics, Vol. 52, 21022106.
Ong S. K. and Nee A. Y. C. (2004): Virtual and Augmented Reality Applications in Manufacturing, London:
Springer.
Riva G., Mantovani F., Capideville C., Preziosa A., Morganti F., Villani D., Gaggioli A., Botella C., Alcaniz M.
(2007): Affective Interactions Using Virtual Reality: The Link between Presence and Emotions, In:
CyberPsychology & Behavior, Vol.10, Nr. 1, 45-56.
Wagner M. (2005): Tracking with multiple sensors, Dissertation, Munich: Technische Universitaet Muenchen.
Wulz J. (2008): Menschintegrierte Simulation in der Logistik mit Hilfe der Virtuellen Realitt, Dissertation,
Munich: Technische Universitaet Muenchen.
359

EMOTIONAL 3D EXPERIENCE IN AUGMENTED HERITAGE
REALITY THROUGH ADVANCED TECHNOLOGY FUSION
Matteo Fabbri
Tryeco 2.0 srl, Italy
Emanuele Borasio
G-maps, Italy
Daniel Blersch & Christoph Froehlich
Zoller + Froehlich GmbH (Z+F), Germany
ABSTRACT: The objectives of this work are the creation and the exploitation of synergies between applied new
technologies for the high density documentation of archaeological and artistic heritage as terrestrial laser
scanning, rapid prototyping and the use of ICT channels, based on Smartphone and tablet applets.
Recently, the cultural heritage sector has evolved a pioneer platform for new media applications, especially for
Augmented Reality. AR technology and its characteristics of transposition of virtual contents on real objects, allow
to illustrate with video sequences and digital 3D models which are adaptable and customizable on-the-fly to the
users preferences with language or level of background information, e.g. in combination with museum info panels,
catalogues or posters. This allows immersive free 3D experience of arts and heritage within the
location-independent environment. Cultural Heritage meets users at their homes!
This case study shows the workflow of a Lidar survey of an artistic artifact through advanced modeling to rapid
prototyping and the museumization through mobile device applets for iOS and Android operating systems, based
on location intelligence.
The visitor lives an emotional experience by holding the artifact directly in his hands. The applet for museum
exploration is installed on any NFC equipped device and enables to access additional contents. The user just
physically approaches the Smartphone to the pre-located tag at the artifact. The personal experience can be
directly socialized through sharing personal images and comments on social networks as Twitter or Facebook.
This induces a contagious network for promotional purpose of the event.
KEYWORDS: Augmented Reality, Heritage, Laser Scanner, Rapid Prototyping, Advanced modeling, Applet,
Smartphone
1. INTRODUCTION
Since some years the cultural heritage sector is changing significantly due to the introduction of new
methodologies of preservation, restoration and divulgation of artistic heritage. New professional figures have
appeared on the horizon opening additional and creative work opportunities.
Laser scanning has altered the methodologies of analysis and survey of the artifacts. Rapid prototyping did it with
cataloging methods. But mainly the efficiency of divulgation has been enforced with the creative use of
information and communication technologies ICT. Often, the simple and direct use of ICT is most efficient for
improving the knowledge and divulgation, because based on the already known. This familiarity refers to the
cultural operators on the territory, to the conscious tourists, and anyone who is accessing directly at complex digital
data through web pages, social networks weblogs and online performing Smartphone/tablet devices. These
technologies are used to improve the management in many natural, environmental, demo-ethnoantropological, and
historic-artistic fields like archaeology, archives, libraries, theatre, music and film making. At the same time,
thanks to their intelligent divulgation, they help the collectivity and territory to close up with the artistic heritage.
360

1.1 Research Background
In the common mind the cultural heritage sector is perceived opposite to cutting-edge technologies. But in recent
time there has been a massive invasion of new Information and Communication Technology ICT in the use for
exploitation of exhibits. Enhanced fruition and a highly emotive relation between artifact and visitor are among the
attested benefits.
A few early and significant examples:
At the Capitoline Museums in Rome, which is considered the first public museum in history, for the first time in
Italy is used Near Field Communication NFC technology for illustrating the exhibits with Samsung Star NFC
smartphones.
At the J. Paul Getty Museum in Los Angeles the 17
th
century Augsburg Display Cabinet can be interactively
visited through an AR marker giving great engagement to the visitor by controlling the cabinets movement:
opening, spinning, and tilting it like its original owner could.
The Museum of London has created an app for iPhone. Visiting selected places in London historic photographs can
be projected in realtime on smartphone and iPhone which give a strong emotional experience to the visitor.
As well, there are important contributions to geomarketing and the heritage of the cities which are creating new
tools for exploitation and valorization of the territory. Thematic routes are articulated in emotional experiences and
invite the tourist to live the place, beyond the physical visit, as for example the Ferrara Typical Emotions
Guaranteed (Ferrara ETG) Project in Italy. Technically this can be done with applications like Wiki-tude or Layar.
2. REALISTIC DIGITAL AS-BUILT DATA
2.1 Survey: laser scanning
The beautiful and locally well known statue The Reader is situated in front of the city library in Wangen, South
Germany. The 1.7m artwork was donated by the Frhlich family. A high density documentation survey was carried
out using nine scans at high resolution and color implementation using Z+Fs M-Cam solution. The 3D solid mesh
was created using Geomagic 2012. The results produced a high level of detail. This example provided evidence of
the capabilities and possibilities of the phase based scan system, even performing well at extreme short range,
outside its design capabilities and normal work environment.
2.2 Analisis. 3D modeling
The metric digital model can be used for various functions: as a base for measurements and which are used for
conservation and preservation the original artifact, or as a medium to communicate the artifact and its history. The
transformation of triangular high resolution models to usable less complex communication and reproduction
models is a challenging and not immediate procedure. Iterative down sampling techniques used by Tryeco 2.0
permit case sensitive model adaption and quality preservation.
Fig. 1: Location of the artifact The Reader in front of the city library of Wangen.
361

In the case of The Reader sculpture the directly from scan derived model consists of 8 million points. The point
cloud model was massively reduced and meshed, which is mandatory for prototyping and 3D visualization
systems. The model with about 2.5 million triangles was generated with a texture map deriving from scanning and
then was used for prototyping of two solid printouts at reduced scale.
In a later step this model was further processed to an AR application. The technical limitations of the current
devices (Smartphone and tablet) impose a massive simplification of the model. Through various iterations the
number of polygons was reduced to about 350 000 triangles without the loss of the polygons color information.
The reduction in this phase brought some light alteration to the surface mostly by unordered detachment of surface
normals which is read as wholes from the AR software. The localization and correction of these triangles was the
main issue in this modeling step.
On the other hand, for processing the video sequence a 50% reduced model performed with high visual quality and
limited calculation time.
The versatility of the digital model resulting from High Density Documentation HDD shown by the possibility to
perform under so different requests is a central advantage of the methodology.
3. MASS ACCESS TO THE ARTWORK
3.1 Conservation: Rapid Prototyping

Fig. 3: The 3D printout in full color during processing. Right: one single layer of the model profile
Fig. 2: The digital polygon model
362

The true-scale digital model allows processing physical copies at any scale, through rapid prototyping technology.
This extends the usability of the artwork, e.g. haptic visit to museums for visually disabled. In the case of The
Reader a sculpture of human scale located in the exterior has been reproduced in two downscaled copies that
allowed divulgation to a larger audience.
The prototyping technology by plaster layers was adapted with the print system ZCORP 650, currently unique to
reproduce 390k colors at 600 x 540 dpi resolution. The mesh surface of the digital model underwent a 5mm
extrusion for sufficient print out stability. The solid print is processed by successive profiled layers of 0.089
0.102 mm thickness. After the printing the maquette gets a treatment with bi-component epoxy resins which give
brilliance to the colors and physical solidity.
At Salone del Restauro 2012 (Italy) and SPAR International 2012 (USA) have been exposed the mono color and
the multicolor copies of the artwork which originally is located in Wangen, Germany. The haptic property of the
prototype extended the emotional experience in addition to the integrated Augmented Reality (AR) and Near Field
Communication (NFC) technologies which allowed further interactivity on personal tablet and / or Smartphone.

3.2 Diffusion: Mobile Augmented Reality as new media
The recent Smartphones and tablets support complex contents, as interactive 3D models or geospatial data. The
diffusion of these devices is breathtaking and already outperforms non-Smartphones. Mobile internet connections
outperform static connections.
Recently, the new devices contain hardware components that interact with reality: GPS, electronic bubble,
accelerometer, gyroscopes (MEMS). The combination with interactive digital cartography and with the possibility
of online query mediation makes these devices an universal auxiliary as assistant to visits to museums and events.
Augmented reality on mobile platform is an instrument able to enhance the perception of the observed reality by
the user, through the overlay with virtual contents generated by the device with the video stream of the camera.
With the resulted mix the user can extend his personal knowledge about the surrounding real objects by
interrogating the associated multimedia content.
Fig. 4: Exposition of the mono color printout with Near Field Communication (NFC) tags
363

There are different accesses to AR. Or through a dedicated application (app) or by AR browsers (Layar, Aurasma,
Junaio, BlippAR) which are freely downloadable at the virtual stores like Google Play Store (Android Market) and
Apple App Store.
The cultural heritage sector is considered a pioneer in using augmented reality, which today sometimes is named
the eighth mass media. Thanks to AR technology it is possible to illustrate multimedia contents which adapt
themselves to the users language and level of interest. Starting from an existing image - like an exhibition panel, a
catalogue or a poster - it is possible to visualize enriched and augmented contents. An application may initially
behave as a traditional audio guide within an exhibition area and then guide the user directly out to further events in
the neighborhood.
By keeping this idea in mind, in the collaboration between the SMEs G-maps and Tryeco 2.0, has been developed
the Z+F The Reader AR application. The intention is the interactive viewing of a 3D model of a statue which has
been documented by means of laser scanner technology. The application is adapted to the need of using the 3D
model independently wherever and whenever. The application Z+F The Reader for Smartphone and iPad allows
interactivity to the scene with the digital 3D model of the statue. On the other hand the physical model has been
produced with the 3D printing technology.
Fig.5: Augmented multimedia content and impact on the public
Fig. 6: AR Postcard with QR-code and marker
364

The mobile application is extremely easy to use and does not require expertise: once freely downloaded at Apple
App Store or Google Play Store you need just to point the Smartphones/tablets camera on the prior distributed
postcard. In this case the ground plan of the statue. Right away the user can show the 3D digital model is and access
to a video clip with background information about the rapid prototyping process of the object (from registration of
the single scans, the advanced modeling for the hardcopy of the statue used for exhibitions).
This kind of exploitation of an artwork is extremely emotional. The user holds a virtual copy of the artwork in his
own hands, independently from any geographic location.
4. CONCLUSIONS
The emotional and surprising experience which links the user directly with the exploitation of the artwork by using
his personal mobile device is a new way of earning loyalty and customization of a user. Wherever in the world, the
user or visitor can touch with his personal emotions his favorite pieces of art, by the AR filtered vision. Just a
postcard, a journal or a paper are sufficient to act as a marker. If the AR app is combined with Near Field
Communication NFC sensors it is possible to access to the additional contents simply by passing the Smartphone
(with NFC support) close by a dedicated tag. Finally, users may socialize their personal experience of the visit by
publishing their comments and snapshots on social network sites as Twitter or Facebook. In this way a contagious
network promoting the event or product will spread out.
In a near future technological standard in the cultural heritage valorization will offer the possibility to enjoy
downloadable cultural tours on mobile devices which are adapted to personal profiles (appassionate, scholar,
tourist, educational, childhood ), multilanguage and immediate availability. Museums, without distinction, are
able to reach and inspire to visit millions of potential visitors all over the world through dedicated AR publications
on their exhibits, due to over a billion smartphones and tablets out there.
Soon there will appear new tools for environment fruition as AR lenses (AR Goggles) which some prototypes have
already been seen (Project Glass by Google). The lenses will be a technical breakthrough permitting the user to
keep hands free while moving in a physical internal or external space (See-Through technology) accessing
virtually augmented contents overlaid to the real environment.
5. REFERENCES
Fabbri M. and Gorgoglione M. (2010). Innovazione nella comunicazione museale: strumenti e metodi per la
protezione e la valorizzazione di un sito archeologico. La realizzazione del prototipo del Sito Archeologico di
Scalo di Furno, Restauro 2010, 85-92.
Fabbri M. (2009). Realizzazione di una copia del ritratto di Matidia Minore per sostituire l'originale
temporaneamente in prestito al British Museum di Londra per una mostra figurata su Adriano, Restauro 2009,
91-93.
Fig. 7: AR digital model and emotional experience of the personal contact with the artwork
365

Balzani M., Fabbri M., Persiani F., Seccia L., De Crescenzio F., Fantini M., Virgili V. (2007). L'Ebe di Antonio
Canova nella Pinacoteca Civica di Forl Studio di fattibilit per l'applicazione di tecnologie avanzate di rilievo,
prototipazione e realt virtuale , Restauro 2007, 222-226.
Balzani M., Fabbri M., Maietti F. (2006). Tecniche di prototipazione e tonalizzazione di copie in scala:
metodologie di prototipazione e studi sulla caratterizzazione superficiale dei modelli,Rilievo 3D e studi per la
riproduzione dei bassorilievi provenienti dal Museo dellErmitage e che ornavano i Camerini di Alabastro del
Castello Estense di Ferrara, Restauro 2006, 201-207.
Balzani M. and Fabbri M. (2006). Un lapsus tra lapide e gesso, Architetti Idee cultura e progetto, No. 1/2, 8-9.
Balzani M., Fabbri M., Maietti F., Santopuoli N. (2005). Survey, Modeling and Scientific Integrated Researches
for Restoration and Enhancement of Cultural Heritage Study of Bas-Relieves of the Camerino dei Marmi di
Alfonso I for the Estense Castle, Proceedings of 6th International Symposium on Virtual Reality, Archaeology and
Cultural Heritage VAST.
366

III. 3D/4D MODELING

367
368

4D VISUALIZATION OF RAILROAD TRANSOPORT OPERATIONS OF
EQUIPMENT TO A POWER GENERATION FACILITY
Brandon S. Reisser & John C. Hildreth
University of North Carolina at Charlotte
ABSTRACT: Visualizations in 4D were used to facilitate and improve the planning of railroad transport
operations of oversized and heavy equipment to a power generation facility. Critical planning for the delivery of
a new power generator via rail required special consideration of the initial sequence of rail cars, movements of
the train to allow rail cars to be re-sequenced, and the final position and configuration of the train to allow final
transport with a mobile gantry crane. The planned rail operations were modeled and simulated to enhance
communication among the operational planning team, which included representatives from the power plant
operator, rail road, and generator manufacturer.
The modeled elements included the surrounding terrain and infrastructure, rail bed configurations, rolling stock,
and mobile gantry crane. The models were developed using AutoCAD and SketchUp. Simulation was achieved
using the Synchro software, which allows the integration and synchronization of 3D models with a CPM
schedule. A CPM schedule was developed to model the planned operational tasks, sequence, and task durations.
The resulting visualizations could be viewed using the Synchro software or output into a standard digital video
format.
The visualizations were used to quickly and efficiently communicate the operational plan to all parties
represented on the project team and reviewed to identify improvements to the transport operations to achieve the
project objectives.
KEYWORDS: Visualization, Planning, Operations
1. INTRODUCTION
Designers and contractors expend a substantial amount of effort towards planning construction projects. This
effort is warranted, in that planning plays a pivotal role toward the success of a project, from the conceptual
phases through project completion. This paper will focus on planning from a preconstruction perspective, and
how 4D planning was implemented to improve this process.
This paper will demonstrate how 4D planning was implemented to improve the coordination and communication
amongst the operational planning team of a project involving the delivery of a 1.2 million pound stator to a
nuclear power generation facility. The operational planning team includes representatives from the power plant,
rail road, and generator manufacturer. The execution of this task requires careful planning due to the various
constraints present on the project, and when working around a nuclear facility. The major constraints include:
The sequence of rail cars during transport is dictated by the constraint that the caboose is not
structurally sufficient for load to be pulled through it.
The rail car carrying the stator is a Schnabel car, which can only be pulled, due to the nature of
the car and the track curvature around the power station.
Spatial limitations present within the facility, dictate that the rail car configuration be modified
prior to approaching the power station.
The train cannot restrict ingress or egress to the facility.
The track crosses a critical water impoundment structure, over which the train can only
traverse once per day (crossing once to enter the power station and again to exit the station in a
single 24 hour period).
The project constraints and the critical nature of nuclear power generation infrastructure led the operational
planning team to pursue 4D modeling and simulation as a tool for improving the planning process of the
transport operations and increasing the overall success of the project.
2. BACKGROUND
2.1 Planning
Pre-construction planning involves design review, site investigation, determining task sequence, and
369

optimization to improve the execution of the project (Waly and Thabet, 2002). Each party involved in the project
team will rely upon his or her experience when generating ideas to tackle the challenges associated with todays
more complex construction projects. During this stage, the means of methods of planning are developed. This
includes: the selection of information to be gathered and how it will be distributed to the project team, the
scheduling techniques to be used, how often the schedule will be updated, and other project parameters that will
establish the framework for the rest of the project (Laufer and Tucker, 1987).
Once an initial plan is developed, the project team reviews the plan to ensure that it is in accordance with project
requirements and constraints such as, cost, duration, quality, and constructability. Constructability can be more
difficult to review than cost and duration, because it requires extensive experience interpreting plans and
coupling them with a schedule. After the project team identifies all potential conflicts, the team works to develop
solutions that will satisfy these conflicts within the constraints of the project. The project team will find some
issues that they do not possess the ability to solve; these issues will have to be alleviated through modifying the
constraints present on the project. Due to the fact that planning is an iterative process that cannot occur without
highly experienced personnel, it requires an extensive amount of resources to carry out. The problem then
becomes making a decision on the amount of resources that adds the most value, in a cost effective manner
(Allen, 2007).
Throughout the planning process each individual on the project team will use the resources available on the
project (e.g., specifications, drawings, environment, etc.), to make decisions. The information obtained from
these resources will be analyzed, leading to decisions that reflect the project plan. The traditional approach to
this process involves communication between the owner, designer, and contractor through paper drawings and
bar charts. All parties involved are required to visualize the physical relationships, means and methods, resources,
and site conditions. Because this is an extensive amount of information to process, the traditional approach is
difficult for the project team to undertake (Waly and Thabet, 2002). The burden placed on the project team can
lead to less effective overall plan, and ultimately decrease the success of the project.
2.2 4D Planning
The combination of the graphical potential of 3D CAD, with the construction project schedule, is commonly
known as 4D CAD (Webb et al., 2003). 4D planning involves implementing this practice during the planning
phase of a construction project. When deriving a plan, informed decisions have to be made based on an extensive
amount of information, as previously discussed. One has to process the effects of many variables simultaneously,
in order to develop a comprehensive plan. Compounding the difficulty of this task is the fact that, construction
projects have increased in complexity in terms of owner, environmental regulations, and societal expectations.
This complexity has led to partially informed decision making, which can result in missed opportunities, and
unaccountable, undesired effects (Pappamichael, 1998).
The 4D CAD approach to planning presents several opportunities for use as a tool for project management, with
respect to the way that it graphically represents the relationship between space and project schedule through the
actual transformation of that space over time during the construction phase of a project (Webb et al., 2003). By
simulating and visualizing the construction phase of a project, the project team can exert less effort towards
visualizing manually and independently, and expend those efforts on developing more comprehensive planning
alternatives and ultimately, solutions.
2.3 Advantages of 4D Visualizations for Planning
The utilization of 4D CAD to enhance the planning process of construction projects has been discussed in the
literature. The potential impacts of visualization were presented by Koo and Fischer (2000), and later quantified
by Songer et al. (2001). The study quantified the advantages of using visualizations to review and analyze a
construction schedule. Using the traditional approach, paper contract documents and CPM schedule, participants
had a difficult time locating conflicts within the schedule provided. When the participants utilized animation,
they were able to identify more conflicts, and addressed problems with safety and over-crowding.
Communication interference diminishes greatly when employing visualizations during the planning process (Issa
et al., 2003). The level and quality of communication between all members of the project team, during all phases
of construction, usually determines if a project will be delivered on time and under budget with a limited number
of change orders. Of all the parties involved, the owner has the greatest difficulty visualizing the schedule as it
relates to the physical elements (Issa et al., 2003). The use of visualization to convey information to the owner
increases the speed at which the owner is able to process the information and provide approval of the proposed
project plan.
370

Utilizing 4D visualizations also allows the project team to play what if scenarios. The use of 4D modeling to
simulate a project allows for several options to be generated and simulated in model space. Because the
scenarios are carried out in model space, they can be completed relatively quickly and inexpensively, then a
comprehensive evaluation can be carried out to determine the best option.
As discussed by Webb et al. (2003), the implementation of 4D visualizations provides the entire project team
(owner, designer, and contractor) with a more effective approach for the:
improvement of communication between each other
evaluation, implementation, and monitoring of design changes
evaluation of alternative materials and installation practices
evaluation of site logistics (i.e. material staging, field office locations)
identification and development of alternatives when the original plan is disrupted
training of construction crews and visitors of the site of potential hazards
monitoring of progress by comparing as-planned with as-built
overcoming of language barriers on international construction projects
3. 3D MODEL DEVELOPMENT
As previously stated, 4D visualizations and planning, employ the use of 4D CAD, which consists of a 3D model
that is linked to a CPM schedule. The components of the model include:
the terrain and infrastructure
the rolling stock
3D paths of movement
Information made available by the operational planning team was used to develop the 3D models. Pre developed
models were also used if applicable. It is important to utilize all available resources effectively to minimize cost
and increase the value of the visualization. The ultimate goal is to improve the planning process while
minimizing the resources required. The modeling took approximately 120 man hours to complete. A large
portion of this time was spent learning the software as the project developed. If another project of similar size
and scope were undertaken, it could be completed in a time frame of approximately 80 man hours.
3.1 Terrain and Infrastructure
The terrain and infrastructure model was developed using Autodesk AutoCAD and Google SketchUp. Drawings,
depicting the site and surrounding areas with the location of roadways, railroads, parking lots and major
buildings were provided by the power company to aid in the development of the model.
Pictures of the facility and surrounding area were taken to provide information on the appearance of major
landmarks to be included in the model. Also used in the development of the model of the terrain, were 4 ft
contours, obtained from the North Carolina Department of Transportation (NCDOT). The NCDOT utilizes Light
Detection and Ranging (LIDAR) technology to produce contours across the entire state. 4 ft contours were the
best available from this source and were readily available; this made them an attractive option.
The first step in developing the model included adjusting the contours provided to accommodate for recent
changes to the site and to increase the accuracy of the terrain near and adjacent to the railroad track. For typical
construction projects, a more accurate surface would need to be developed to properly analyze and evaluate
constructability issues, but because this project is focused on the visualization of sequencing, they were
sufficient.
Upon completion of the alteration of the contours in AutoCAD, the file was then imported into Google SketchUp,
where a surface was created from the contours using the sandbox tool. The sandbox tool can be utilized to
develop a TIN surface from a set of contours imported from another 3D modeling software, or to create a surface
from scratch in SketchUp. For this project, both methods were used. The contours that were modified in
AutoCAD, were imported and converted to a TIN surface, and then a second surface was created after the
infrastructure was in place to address minor grading issues (this is what is meant by adjusting the surface to
provide additional accuracy of the terrain near and adjacent to the railroad track). The entire surface was
371

assigned a grassy texture, and then all bodies of water, parking lots, and gravel areas were assigned their
respective textures.
Once development of the surface was complete, the roadways, railroads, bridges, buildings, parking lots and
major landmarks were developed. This was all completed in SketchUp, on the TIN surface developed from the 4
ft contours. A lesser level of detail was required for these items, due to the fact that they were included to give
the viewer a sense of location and orientation within the model, and would not be examined from close proximity.
At this point, the terrain and infrastructure model was near completion. The major infrastructure provided a
means for the user to locate and orient themselves. The minor infrastructure was then developed to increase the
realism of the model. Included in the minor infrastructure are transmission lines, power poles, street lights, trees,
and vehicles. These items were all acquired from the Google SketchUp 3D Warehouse. The assortment of
models available for use in the 3D Warehouse, was one of the major reasons SketchUp was the modeling
software of choice on this project. Figure 1 depicts the terrain and infrastructure model in its entirety. The model
space includes 8592 linear feet of track on 988 acres.
Figure 1: Terrain and Infrastructure Model
3.2 Rolling Stock
The rolling stock required for the transportation of the stator from the manufacturing facility to the power
generation plant included:
2 GP-40 4 axle locomotives- required to provide the necessary force for transportation
2 flat bed idle cars- required to space the load applied to the bridge over the Catawba River
Caboose- required to transport and store the tools for the Schnabel car
Schnabel car- required to transport the stator
Shown in Figure 2 is the entire ensemble of rolling stock that was developed in Google SketchUp. The overall
dimensions of each car and a detail of both the locomotives and Schnabel car were provided, but details of the
caboose and idle cars were not available. For those cars lacking a detail, a generic design was selected from the
3D Warehouse. Each car, with the exception of the Schnabel car, was available in the 3D Warehouse. Once
loaded from the 3D Warehouse, the component attributes of each car were modified to the appropriate length,
width, and height to reflect the planned resources and to increase the accuracy of the model/simulation. It was
important to ensure that the lengths of each car in the model would reflect the actual lengths, so that the
visualization would help identify any potential conflicts related to the constraints previously discussed. One of
the biggest reasons 4D simulation was utilized on this project, was to identify potential conflicts with ingress and
372

egress to the facility, and to ensure that there was enough room available in the facility to maneuver the stator to
the position where it could be unloaded.

Figure 2: Rolling Stock
An acceptable Schnabel car was not available in the 3D Warehouse, so an original model was created to
complete the train assembly delivering the stator. The body of the Schnabel car with the stator was developed,
using SketchUp. The trucks located beneath the body of the car were retrieved from the 3D Warehouse (the
Schnabel car was built by developing the body and incorporating trucks from an object readily available in the
warehouse). Utilizing the trucks from the warehouse reduced the time required to develop the model
substantially. Figure 3 depicts the Schnabel car body integrated with the wheelsets. The Schnabel car is 180 ft in
length by 12 ft wide at its widest point. The car utilizes 11 axles on each end to distribute the 1.2 million pound
load.

Figure 3: Schnabel Car Model
3.3 3D Paths of Movement
The software employed to produce the 4D visualization was Synchro, which allows the movement of resources
(the rolling stock) throughout model space by assigning the resources, to a 3D path. The paths can be developed
in Synchro, or in disassociated software. For this project, the paths were created in AutoCAD and imported.
In order to ensure that each 3D path would follow the railroad developed in the terrain model, the SketchUp file
was imported into AutoCAD as a DWG file. AutoCAD was used to create 3D polylines defining the paths each
model component was to travel. This was completed to make assigning the paths more manageable in Synchro.
Because Synchro only allows a resource to be fixed to the 3D path about one point, each component of the
373

rolling stock had to have its own separate path (a separate path was required for each movement of each
component). The geometric center of each car at the minimum Z value was used as the fixed point throughout
the entirety of this project. This is illustrated in Figure 4. Using the center and minimum Z value ensured that
each piece of the train would follow the path appropriately. Because each component required a separate path,
each path would have to be the same length and begin and end at a specific distance apart. This would ensure
that the train would remain connected during movement. As previously mentioned, each path is fixed to the
center of each car. This allowed the spacing between each car to be calculated by dividing the length of each car
in half then adding them together.

Figure 4: Geometric Center and Minimum Z Illustration
Once the distances between the beginning and end of each path were calculated, the paths were drawn over top
of the 3D polyline previously developed. A separate path for each movement of each component of the train was
established. The number of movements for each resource is tabulated in Table 1.
Table 1: Resource Movements
Resource No. of Movements
Locomotive 1 10
Idle Car 1 1
Schnabel Car 5
Caboose 7
Idle Car 2 8
Locomotive 2 8
4. CPM SCHEDULE DEVELOPMENT
The development of the schedule to produce the visualization of this project was a reasonably straightforward
process. Due to the fact that the exact speed of the train is not known at this stage of the project, the schedule
was developed to reflect only the sequence of movements and the approximate speed. The proposed sequencing
was provided by the representative of the power company (the preliminary sequence was developed based on the
outcomes of a preliminary meeting attended by the involved parties). That sequence was then translated to a
CPM format, containing 17 tasks linked together in a chain (these are activities linked by finish-to-start logic
links). Due to the relative simplicity of the schedule, it was created directly in Synchro. However, Synchro
provides the capability to import schedules created in Microsoft Project as well as Primavera.
5. INTEGRATING THE 3D MODELS WITH THE CPM SCHEDULE
The first step in integrating the 3D models with the schedule was to develop a custom resource use profile.
374

Synchro allows the user to assign different use profiles to different resources within the 3D model. This allows
the user to highlight work in progress and assign different animations to components of the model for
visualization purposes. For this project, a resource profile that did not change during the duration of the task was
utilized.
Each 3D model was imported into Synchro assigned as a separate model resource. Resources could then be
assigned to the schedule tasks and 3D paths to define their movements. The forward direction of the resources on
the 3D paths associated with the first task had to be modified so that the rolling stock was tangent with the track.
This was completed for each path until the rolling stock remained tangent to the path throughout the entirety of
the visualization.
As previously stated, Synchro only allows a resource to be attached to a 3D path at one point; because a train car
should be fixed to the path at each of its axles, each car appears to veer off of the track when making turns.
Figure 5 illustrates the each cars deviation from the assigned path. This occurs because the resource is assigned
to remain tangent to the path about its center. This is more evident with the Schnabel car, because the magnitude
of deviation is a function of the objects length. It is also more evident on the curves with smaller radii. A path
could have been defined to alleviate this problem, but it would have required a substantial amount of work that
was deemed unnecessary for all intents and purposes of this project. When developing a visualization, it is
important to keep the final objective in mind. Decisions must be made regarding the level of detail required to
fulfill the purpose of the visualization. For this project, it was decided that the cars appearing to deviate from the
track, would not diminish the end result of the visualization. This issue would not affect the demonstration of the
sequence of train movements.

Figure 5: Deviation From the Track
6. VISUALIZATION/ ANIMATION OUTPUT (RESULTS/ DISCUSSION)
For this project, the terrain, rolling stock, CPM schedule, and 3D paths have all been developed and then
integrated in Synchro. If an alteration to the schedule is to be made, the changes can be made, in Synchro, in an
abbreviated amount of time. The animations produced, depict each movement of the train, and allow the viewer
to develop a clear understanding of the trains sequence. Figure 6 demonstrates a portion of the sequence,
through a series of screen shots, taken during the animation.

375

Figure 6: Reconfiguration of the Car Sequence Prior to Entering the Power Generation Plant
The terrain and infrastructure model initially contained an elevated level of detail over the model used for the
visualization depicted above. Throughout the development of the animation, ways to simplify the model were
explored in attempt to speed up the rendering process. It is important to recognize the intent of the visualization,
and produce an appropriate level of detail that achieves the project goal within a reasonable amount of time. The
model utilized in this case, contained textures and individual components, which significantly increased the time
required to render the animation. The grass, gravel, water, and asphalt textures, initially applied to the terrain
were substituted for solid colors. The model proved to be substantially easier to manage in terms of rendering.
The visualization of the entire project was demonstrated through the use of four individual animations that would
later be aggregated into a single animation video. This allowed the production of the output animation to be
broken into manageable sections. It reduced the burden placed on the rendering computer, and also allowed for
each segment of the video to be reviewed, to ensure a desired output was being produced. If a portion of the
animation did not meet expectations, it could be adjusted without having to export an entire animation. This was
significant, because the time required to produce the entire animation was approximately 6 hours.
The machine employed to produce the animations comprises an Intel(R) Core i7 CPU X 980 @3.33GHz
processor with 12.0 GB of RAM. The hardware installed on the machine includes a 64-bit operating system and
ATI FirePro V8800 (FireGL) 2 GB graphics card. Each animation output was in AVI format, with a resolution of
1920 x 1080 and a frame rate of 20 frames per second. The Lagarith Lossless Codec was utilized for
compression to reduce the overall file size of the video. The length of each animation and their respective times
required for export, are included in tabular format in Table 2 below.
Table 2: Animation Export Times
Animation Length of
Animation
File Size Export Time Animation Time/ Export
Time
1 00:01:02 331 MB 01:15:00 01:12:35
2 00:02:02 550 MB 02:00:00 01:59:01
3 00:01:19 555 MB 01:25:00 01:04:33
4 00:02:00 932 MB 02:00:00 01:00:00
The animations were assembled into a single video, and then shared with the power company representatives,
railroad company, and stator manufacturer. The animation provided a tool that the parties could use to visualize
the sequence, giving them the ability to work from a common ground, provide feedback, and modify the plan of
execution if necessary.
376

7. CONCLUSION
During the planning phase of a construction project, informed decisions must be made based on a vast array of
information. The decisions required on these projects have increased in complexity as owners needs, regulations,
and societys expectations have risen. The utilization of 4D visualizations facilitates and enhances the planning
and coordination efforts of the project team. The level and quality of communication between all members of the
project team, during all phases of construction, usually determines the overall success of the project.
When developing a visualization, it is important to keep the final objective in mind. Decisions must be made
regarding the level of detail required to fulfill the purpose of the visualization. This was evident when deciding
what components to include on the terrain and infrastructure model, the level of detail present on the components
included, and if the 3D paths should be modified to accommodate and correct the train cars appearing to deviate
from the track. Developing the visualization requires additional time and resources. It is important to optimize
the time and resources required while fulfilling the purpose of the visualization.
Four foot contours were obtained to develop the TIN surface of this model. After considering the additional time
required to adjust the contours to fit the requirements of this project, it may have been beneficial to have spent
the additional time required to obtain a set of contours with better accuracy.
Overall, SketchUp was an effective tool to develop the models required for this project. SketchUp interfaced
well with other 3D CAD software, and allowed the development of the 3D paths to be carried out with accuracy.
It is user-friendly and requires minimal training to become proficient. SketchUp also provides an invaluable
amount of components through its 3D Warehouse. The 3D Warehouse reduced the amount of modeling time
substantially.
Synchro provided an exemplary platform to integrate the 3D models with the CPM schedule. The schedule can
be generated in Synchro and modified with relative ease. Synchro also interfaces with an assortment of 3D CAD
software. Synchro allows the user to assign resources to tasks and provides a variety of options regarding their
movements and display. Once the 3D models have been integrated with the schedule, Synchro is capable of
producing high quality animations to distribute amongst members of the project team. In addition to the benefits
discussed, the learning curve for Synchro is steep, meaning it can be learned relatively quickly.
When working with a large model, it is important to reduce the level of detail in terms of texture to allow the
rendering process in Synchro to run smoothly. There is room for improvement in terms of the export process in
Synchro, to allow for the use of models with increased complexity and ultimately increase the appearance of the
visualization. The maneuverability could also be improved within the model space of Synchro. It can be difficult
to pan and zoom when trying to set up the camera locations when developing the animation sequence.
Synchro was chosen to produce the visualization because it allows resources to be easily assigned to a 3D path
and animated. Synchro is also an effective tool for visualizing the construction of a commercial building or any
other structure. It is a platform that allows the user to assign resources to a schedule and provides several options
when animating those resources.
Google SketchUp is a relatively affordable tool that is easy to learn. The visualizations produced for the typical
commercial building projects usually contain a high level of detail and require a substantial investment of time
and money to produce. SketchUp is not the tool to produce a high detail model to address constructability issues
between two systems in a building, but there is potential for use in evaluating site logistics issues on the
construction site.
The operational planning team was very pleased with the quality of the animation produced. Upon receipt of the
visualization, the team decided to incorporate some additional train car movements to better illustrate the plan.
The team also identified some changes to the initial sequence that would improve the execution of the project.
Moving forward, some additional pieces will be included in the model to increase the realism and provide more
information for the team to make better decisions.
8. REFERENCES
Allen, C. (2007). Improving construction planning through 4D planning. MSc. Thesis. Department of Built
Environment and Information Technology, Nelson Mandela Metropolitan University.
Issa, R., I. Flood and W. OBrien (editors) (2003). Application of 4D CAD in the construction workplace. 4D
CAD and Visualization in Construction. A.A. Balkema, Lisse, The Netherlands, 195-210.
Koo, B., Fischer, M. (2000). Feasibility Study of 4-D CAD in Commercial Construction. Journal of
377

Construction Engineering and Management 126(4): 251-260.
Laufer, A., Tucker, R. (1987). Is construction project planning really doing its job? A critical examination of
focus, role and process. Construction Management and Economics 5(1): 243-266.
Papamichael, K. (1998), Application of information technologies in building design decisions. Building Research
& Information 27(1): 20-34.
Songer, A., Diekmann, J., Hays, B., Al-Rasheed, K. (2001). Construction scheduling using 3-D CAD and walk-
thru. Construction Innovation 1(3): 191-207.
Waly, A., Thabet, W. (2003). A Virtual Construction Environment for preconstruction planning. Automation in
Construction 12(2): 139-154.
Webb, R., Smallwood, J., Haupt, T., (2004). The potential of 4D CAD as a tool for construction management.
Journal of Construction Research 5(1): 43-60.

378

A SPACE MODEL FOR PERSONAL LIGHTING CONTROL
Georg Suter, Filip Petrushevski & Milos Sipetic
Design Computing Group, Vienna University of Technology
ABSTRACT: We describe a space model which supports spatial data needs of a personal lighting control (PLC)
system. The potential benefits of such a system include improved individual control of the visual environment, user
satisfaction, and energy effectiveness. In our system, lighting agents dynamically query the space model for
luminaires and windows that surround a given user location. Agents use that data to adjust luminaire outputs and
window shades with respect to user preferences. We introduce a user scenario and the architecture of a PLC system
prototype which we have developed. Spatial data requirements of lighting agents are described next. These depend
on the lighting agents mode. Supported modes include artificial lighting, artificial and natural lighting, and
artificial and natural lighting and shading. The space model consists of a lighting layout. Layout elements include
whole spaces, partial spaces or subspaces, and space elements that are relevant for lighting, such as luminaires,
windows, and furnishing elements. Selected spatial relations between layout elements are modeled explicitly as a
geometric network. The network supports efficient graph queries, such as nearest neighbor or shortest path
queries. In our system, good query performance is necessary to ensure short system response times. The network is
derived automatically from space and space element data defined in a commercial BIM system for architectural
design. Query requests of lighting agents are processed by a space model server. The server decomposes a query
request into a subquery sequence. Each subquery involves tracing of the lighting layouts network, or a subnetwork
extracted from it. Subqueries are illustrated with the example of a test space where we have deployed our system
prototype. We describe the implementation of the lighting agent and the space model server and conclude with a
discussion of open issues.
KEYWORDS: Building information modeling, Space models, Building automation systems
1. INTRODUCTION
Personal control of indoor lighting is increasingly feasible due to advances mobile computing and building
automation systems. Studies suggest that personal control of luminaires improves user satisfaction and, possibly,
productivity (see, for example, Newsham 2004). It may further mitigate the over-illumination problem in facilities,
which adversely affects human health and energy use (see, for example, Galasiu 2009). Existing personal lighting
control (PLC) systems are typically deployed in offices. Users may adjust dimming levels of luminaires that are
near their workplaces via remote control devices or GUIs on PCs or smartphones (see, for example, Shin 2009,
Krioukov 2011). However, in order to minimize system complexity and maintenance, luminaires tend to be
aggregated into large, static (that is, hard-wired and/or hard-coded) control groups. This may negatively affect the
effectiveness of PLC systems. First, existing systems are not flexible enough to respond to changing user needs as
control of lighting conditions is limited by static control groups. For example, when a user moves to a different
location, existing systems must typically be reconfigured. Second, the need for shared control of luminaires by
multiple users is more likely in large control groups than in small ones. Third, large control groups may result in
over-illumination.
To address these issues, we have designed and implemented a prototype of a PLC system (Suter et al. 2011). The
systems core modules are:
a lighting control agent (LCA) which adjusts dynamically derived luminaire control groups based on
lighting preferences, location and illuminance data retrieved from a users smartphone,
a space model which addresses LCAs spatial data needs, and
a space model server (SMS) which provides a query interface for LCAs and supports high concurrency
and fast response times.

The focus of this paper is on the description of the space model structure and query processing of SMS for the
envisioned PLC system. In section 2, we give an overview of a system prototype which we have designed and
implemented. Spatial data needs of LCA are identified in section 3. The description of the space model structure in
379

section 4 is divided into two parts. Space model concepts which we have developed in previous work are reviewed,
and the space model for PLC is introduced. In section 5, we describe and illustrate the processing of LCA query
requests by SMS. LCA and SMS implementations are discussed in Section 6. Open issues are identified in Section
7.
2. PERSONAL LIGHTING CONTROL SYSTEM
2.1 User scenario
The concept of personal lighting control, as we envision it, is illustrated with a user scenario (Suter et al. 2011).
Start situation: User N. enters his office at 7 a.m. on a winter day. Ceiling luminaires in the office are turned on to
provide basic visual comfort. No other users are present yet in the office, where three workplaces are shared on a
first-come-first-serve basis. A PLC application is installed on N.s smartphone.
Flow of events: N. approaches an available workplace and puts his smartphone face up on the work desk. The
smartphone detects N.s location. The PLC application is triggered. The application uses N.s location, his lighting
preferences, and data from an illuminance sensor on the smartphone to adjust output levels of luminaires that
surround N.s desk. While the desk luminaire and ceiling luminaires that are near the desk are adjusted to high
output, distant ceiling luminaires are adjusted to low output as they have less influence on illumination of the work
area. Luminaire outputs are set according to N.'s preferences for computer work, which is the assumed default
activity. Resulting desk illuminance is 495 lx, which is within N.s preferred illuminance interval of 480 520 lx
for computer work.
End situation: N. sets up his laptop and starts working. Lighting conditions correspond to his preferences for
computer work.
2.2 System architecture
We have developed requirements for a PLC system prototype from the scenario. The system adopts a client/server
architecture (Figure 1, Suter et al. 2011). Compared with a service-oriented architecture, the benefit of a
client/server architecture is its relative ease of implementation. On the client side (that is, on the users
smartphone), the main module is the Personal Lighting Manager (PLM), which reads illuminance and symbolic
location data from, respectively, Illuminance Sensor (IS) and Near Field Communication (NFC) modules. NFC is
chosen for symbolic location sensing because the technology is increasingly supported on smartphones (Juniper
2011). NFC tags are mounted on users desks.
Both IS and NFC process sensor data streams and generate events due to changes in illuminance levels or location.
PLM receives these events, interprets them and, if necessary, sends sensor data together with the users lighting
preferences to the Lighting Control Agent (LCA) module. Presently implemented preferences include illuminance
levels and size of the region managed by LCA. In a large, open plan office spaces, for example, regions would
typically be smaller than these spaces. IS sends updates only when there are significant changes in lighting
conditions. LCA is responsible for translating lighting requests into luminaire and window shade control actions. It
interfaces with the lighting automation system (LAS) module to execute these actions. LCA queries the space
model server (SMS) module to obtain luminaires and windows that are near the user location. LCA uses a
closed-loop control algorithm to iteratively adjust luminaire outputs until a targeted illuminance at the users
location is reached. Moreover, it deploys window shades if necessary. At present, LCA deals only with single user
situations.
380

Fig. 1: Architecture of the personal lighting control system.
3. SPATIAL DATA REQUIREMENTS
LCA operates in three modes which determine what luminaire and window data it needs from the space model. The
first mode is artificial lighting. In this mode, LCA needs luminaires that are near a user location. Luminaires and
the user location must be contained in the same space. Each luminaire must include its distance (according to a
distance measure) relative to the luminaire that is nearest to the users location. LCA computes luminaire output
based on that distance. Luminaires that are near the user location have higher output than distant ones. The
maximum distance (that is, the size of the region managed by LCA) is defined in user preferences.
The second LCA mode is artificial and natural lighting under diffuse sky conditions. In this mode, natural light is
used to illuminate a space. Natural light is complemented by artificial light if necessary to achieve minimum
illuminances. Similar to artificial lighting mode, LCA needs luminaires that are near a user location. In addition to
the distances of luminaires to the luminaire that is nearest to the user location, distances of luminaires to the nearest
window must be returned as well. LCA considers both distances when it computes a luminaires output. Assuming
that they have the same distance to the user location, the output of a luminaire that is near windows is lower than
the output of a distant luminaire.
The third LCA mode is artificial and natural lighting and shading. This mode is similar to artificial and natural
lighting insofar as natural lighting is complemented by artificial lighting, if necessary. In addition to luminaires
that are near a user location, LCA needs to know about nearby windows with a line-of-sight to the sun. Buildings,
trees, or clouds may obstruct line-of-sight. LCA shades these windows. A shade may be either open or closed.
Distances of luminaires with respect to nearest non-shaded windows must be returned together with distances to
the luminaire that is nearest to a user location.
4. SPACE MODEL STRUCTURE
4.1 Overview
The space model for PLC consists of a network-based space layout. The layout supports high spatial granularity,
rich spatial relations, and efficient local queries. Concepts of network-based space layouts are reviewed first. A
layout that specifically meets spatial data needs for PLC is described subsequently.
4.2 Space layout concepts
Network-based space layouts incorporate aspects of existing architectural space models (Bjoerk, 1992; Eastman
and Siabiris, 1995; Ekholm, 2000; BuildingSmart, 2010). For a detailed description, the reader is referred to Suter
(2010a and 2010b). A layout includes spaces and space elements that model a particular spatial view of a building.
For example, a set of layouts may model a buildings circulation system, including wings, floors, circulation
spaces, doors, and furnishing elements. Spaces in other layouts may model heating, ventilation, and air
conditioning zones. Space elements in these layouts would include air inlets and outlets as well as temperature
Space Model Server
(SMS)
Near Field
Communication
(NFC)
Lighting Control Agent
(LCA)
Lighting Automation
System
(LAS)
Personal Lighting
Manager
(PLM)
Illuminance Sensor
(IS)
Server modules
Client modules
reused module
core module
Legend
invokes
381

sensors. Selected spatial relations between layout elements are modeled explicitly as a geometric network for
efficient local querying with graph algorithms.
Figure 2 shows four partial views of an example network-based space layout. Each view features different spatial
relations between layout elements. There are four types of layout elements: whole spaces, subspaces, space
boundary elements, and space elements. A whole space is a space which is bounded on all sides by space boundary
elements. A space boundary element is part of an immaterial layer with zero thickness that bounds a whole space.
A partial space or subspace is a space which is contained in a whole space. It may or may not be bounded on all
sides by space boundary elements and may surround space elements. Different types of subspace volumes are
supported. In the example layout in Figure 2, subspace volumes correspond to geodesic Voronoi cells
(Aurenhammer and Klein, 2001) that are derived from whole space boundaries (used as obstacles) and subspace
positions (used as sites). Other types of subspaces such as spherical subspaces are supported as well. Space
elements are (physical) objects, including windows, desks, or luminaires that are either contained in or enclose a
whole space. Space elements have attributes that indicate if they are contained in or enclose a whole space. A desk
is a space element that is contained in a whole space, and a door one that encloses a whole space. This distinction of
space elements matters because they participate in different spatial relations. A layout has a layout element
network, which is a directed, weighted graph with layout elements as nodes and spatial relation elements as edges.
Layout and spatial relation elements have weights, which facilitates layout queries with graph algorithms.
Support for the generation of space layouts is desirable to minimize manual modeling effort. In particular, the
generation of a layout element network and subspaces are time consuming and error-prone tasks. A conversion
routine is described in Suter (2010a) which automatically derives the layout element network and subspaces. The
routine evaluates spatial constraints on layout elements and spatial relations to identify and resolve spatial
inconsistencies. Figure 3 illustrates the conversion routine with the floor layout of an existing office building in
Vienna where our test space for validating the PLC system is located. For improved visualization, the figure only
includes a subset of spatial relations in the layout element network. Rooms, luminaires, and desks are modeled as,
respectively, whole spaces and space elements in a BIM system for architectural design (Autodesk 2011). The
geometric network and subspaces are derived using a solid modeling engine (Spatial 2011). Subspaces are derived
from luminaire and desk templates in which default subspaces are defined. Subspace volumes are derived from
two-dimensional Voronoi diagrams (Aurenhammer 2000). These volumes are not shown in Figure 3.
4.3 Lighting layout
The space layout for PLC models elements and spatial relations that are relevant for artificial and natural lighting.
On the one hand, it models physical elements, that is, luminaires, windows, and desks. On the other hand, it
includes related whole spaces and subspaces. The adjacency relation between subspaces (A
SS
) and the surround
relation between subspaces and space elements (S
SS,SE
) constitute a layout element network together with
subspaces, luminaires, windows, and desks. The layout in Figure 3 shows these relations. The corner office, which
is enlarged in Figure 3, is our test space. The office includes five desks and six luminaires that are arranged in an
irregular 2x3 grid. For simplicity, task lights, chairs, cabinets, and doors are not included in the layout. Five
windows admit natural light from two sides into the office. Each window may be shaded individually. The lighting
layout also includes an external whole space, which models a buildings site. This whole space is useful to
determine which windows are exposed to direct sunlight (Section 5.6). For clarity, it is not shown in Figure 3.

382

Fig. 2: Layout elements and spatial relations in a network-based space layout. a. Adjacency relation between whole
spaces. b. Boundary relation between space boundary elements and whole spaces, proximity relation between
space elements and space boundary elements. c. Adjacency relation between subspaces, surround relation between
subspaces and space elements. d. Boundary relation between space boundary elements and subspaces, touch
relation between space boundary elements.
Fig. 3: Derivation of a layout element network and subspaces. a. Whole spaces and space elements created in a
BIM system for architectural design. b. Converted layout with automatically derived layout element network and
subspaces.
bounds
b.
is adjacent to
whole space
whole space
enclosing
space element
(ese)
is near
bounds
touches
surrounds
subspace
is adjacent to
a.
c. d.
whole space
contained
space element
(cse)
383

5. QUERY PROCESSING
5.1 Overview
LCA queries the lighting layout for the set of luminaires and windows that are near a given user location. LCA
sends high-level query requests to SMS to minimize data communication between these modules. A query request
is divided by SMS into a sequence of sub-queries. An LCA query request has two parameters. The first parameter
is the lighting mode, which is either artificial lighting (AL), artificial and natural lighting (AL-NL), or artificial and
natural lighting and shading (AL-NL-S). The second parameter is the user location, which corresponds to the
unique identifier of the desk where a user has checked in (Section 2.1). Figure 4 shows how SMS processes LCA
query requests. Most subqueries use graph algorithms to trace the lighting layouts layout element network or a
subnetwork retrieved from it. Graph algorithms include connected component, nearest neighbor, and shortest path
queries (Bondy and Murty 2010). In the following, each subquery is described in more detail.

Fig. 4: Processing of an LCA query request by SMS for artificial lighting (AL), artificial and natural lighting
(AL-NL), and artificial and natural lighting and shading (AL-NL-S) modes.
5.2 Get lighting element subnetwork
The first subquery in the processing of a LCA query request retrieves a connected component of the lighting
layouts layout element network. In AL mode, a connected component consists of desks, luminaires and A
SS
and
S
SS,SE
relations in the whole space that contains the user location (that is, the users desk). The latter is the start node
in the connected component query. In order to limit tracing to the whole space that contains the user location, a
filter predicate is used that designates windows as barriers (note that walls and doors are not modeled in the
lighting layout). Without windows acting as barriers, the connected component query would proceed to external
subspaces (not shown in Figure 3), which are modeled explicitly in the lighting layout to support the derivation of
shaded windows (Section 5.6). Barrier elements may or may not be included in the result network of a connected
component query. In AL-NL and AL-NL-S modes, windows are included, whereas they are excluded in AL mode.
The network of the corner office in Figure 3 is an example of the subnetworks that are retrieved from a lighting
layout in AL-NL and AL-NL-S modes.
5.3 Get nearest luminaire
The luminaire that is nearest to a given user location is determined by a nearest neighbor query. With the user
location as a start node, the lighting element subnetwork obtained in the previous subquery is traced. Path weight is
used as a distance measure in this query. Path weight refers to the sum of edge weights in a path, where a weight
corresponds to the Euclidean distance between incident nodes. Nearest neighbor candidates must meet a filter
predicate. Multiple nearest luminaires are feasible. Figure 5 illustrates the retrieval of nearest luminaires for three
user locations.
Get Lighting
Element Subnetwork
Get Nearest
Luminaire
Derive Luminaire
Weights for Nearest
Luminaire
AL
AL-NL
AL-NL-S
Derive Luminaire
Weights for
Nearest Window
Derive Shaded
Windows
Derive Luminaire
Weights for Nearest
Non-Shaded Window
Start End Mode?
User
location
Lighting
layout
Connected
Component Query
Nearest Neighbor
Query
Shortest Path
Query
Shortest Path
Query
Shortest Path
Query
Lighting
mode
Get Result
Luminaires and
Shaded Windows
384

Fig. 5: Illustration of Get nearest luminaire subquery.
5.4 Derive luminaire weights for nearest luminaire
According to the user scenario (Section 2.1), luminaire outputs are set relative to their distance to the user location.
In this subquery, the distance of each (source) luminaire in the lighting element subnetwork with respect to the
(target) luminaire that is nearest to the user location is derived. This is done with a shortest path query. Figure 6
illustrates weight derivation of luminaires for three nearest luminaires. Path length (that is, the number of edges in
the path) is used as distance measure to derive weights. Two filter predicates are used. The first filter selects source
luminaires for which weights are derived. In each lighting mode, all luminaires in the lighting element subnetwork
are selected as source luminaires. The second filter selects target luminaires, which corresponds to the luminaire
that is nearest to the user location.

Fig. 6: Illustration of Derive luminaire weights for nearest luminaire subquery.
5.5 Derive luminaire weights for nearest window
In AL-NL mode, output levels of luminaires that are near windows are reduced more than output levels of distant
luminaires. Respective luminaire weights relative to nearest windows are derived in this subquery. Figure 7a
illustrates the subquery. Again, path length is used as a distance measure. All luminaires in the lighting element
subnetwork are selected as sources for which weights are derived, and all windows are selected as targets.
385

Fig. 7: Illustration of Derive luminaire weights for nearest (non-shaded) window subquery. a. All windows admit
natural light. b. Two shaded windows. c. Three shaded windows.
5.6 Derive shaded windows
Shaded windows need to be derived in AL-NL-S mode. For that purpose, the lighting layout includes an external
whole space which models a buildings site. The volume of the external whole space is derived from the volumes
of the building in question as well as nearby buildings and other obstructing objects, such as trees. Each window
that encloses an internal and the external whole space (and is thus part of the building facade) is surrounded by two
subspaces that are contained in, respectively, an internal and the external whole space. The suns position at a given
time is modeled as a solar subspace in the external whole space. A window is exposed to direct sunlight:
1. if it is surrounded by an external subspace that is adjacent to the solar subspace (that is, an A
SS
edge
connects the two subspaces in the layout element network of the lighting layout), and
2. if there is a line-of-sight between the two subspaces.
An A
SS
edge has a Boolean attribute that indicates if line-of-sight exists between its subspaces. Attribute values are
derived during the layout conversion routine (Section 4.2). It is thus feasible for SMS to derive the set of shaded
windows by tracing the lighting layouts layout element network. Figure 8 illustrates the derivation of shaded
windows for a floor in the example building. A winter day with low solar elevation angles is assumed. In the
morning, windows on the facade facing Southeast receive direct sunlight. At noon, this is true for windows on
Southeast and Southwest facades, and in the afternoon for windows on the Southwest facade. Windows facing the
courtyard do not receive direct sunlight on that day.
A benefit of the described method to derive shaded windows is that SMS does not need to perform computation
intensive spatial reasoning during query processing. However, we have not yet resolved the issue of how to best
model dynamically changing solar subspaces and incident A
SS
edges. One approach is to add timestamps to solar
subspaces. Queries could use a timestamp filter to identify the solar subspace and incident A
SS
edges that match a
given timestamp. Another unresolved issue is how to consider indirect sunlight reflecting e.g. from nearby
buildings.
5.7 Derive luminaire weights for nearest non-shaded window
When shaded and non-shaded windows are known in AL-NL-S mode, the next subquery derives luminaire
weights with respect to non-shaded windows. This subquery is similar to the derivation of luminaire weights with
respect to all windows in AL-NL mode (Section 5.5). The only difference concerns the selection of non-shaded
windows as destination nodes. Figures 7b and 7c illustrate the effects of shaded windows on luminaire weights.
When all windows are shaded, then the space does not receive natural light and this subquery is skipped.

386

Fig. 8: Illustration of the Derive shaded windows subquery.
5.8 Get result luminaires and shaded windows
The last subquery in the processing of an LCA query request involves the retrieval of luminaire and shaded
window result sets from the lighting element subnetwork. Each luminaire in the result set includes a weight for the
distance to the luminaire that is nearest to the user location, and a weight for the distance to the nearest non-shaded
window. In AL and AL-NL modes, the shaded window result set is always empty.
6. IMPLEMENTATION
We have implemented a prototype of the described PLC system. The prototype has been deployed in our test space
and supports the user scenario (Section 2.1). SMS processes LCA query requests by accessing a lighting layout of
the office building in which the test space is located (Figure 3). At present, the system prototype supports AL mode.
The lighting layout in SMS is implemented using Boost Graph Library (BGL, Boost 2011). BGL is a compact,
efficient, and flexible graph framework that lets clients customize graph representation methods and algorithms.
Communication between LCA and SMS is implemented using the Boost.Asio, which is a C++ library for
asynchronous networking and low level input/output programming (Kohlhoff 2008). Asynchronous network
programming supports a large number of concurrent network connections.
7. CONCLUSION
We are currently testing and evaluating the prototype system. Lighting quality and response times for the whole
system and SMS will be measured in the test space. Load and stress tests will be performed in virtual environments.
A direction for future work concerns the development and evaluation of alternative luminaire weight derivation
methods. For example, the weight of a luminaire with respect to windows may be derived by addition of weights or
lengths of shortest paths from the luminaire to each window. Resulting weights would better approximate natural
lighting distribution in spaces than luminaire weights in Figure 7.
8. ACKNOWLEDGEMENT
The work presented in this paper is supported by grant Austrian Science Fund (FWF): P22208.
387

9. REFERENCES
Aurenhammer, F. & Klein, R. (2000), Voronoi diagrams, in J. Sack & G. Urrutia, ed., 'Handbook of Computational
Geometry, Chapter V', Elsevier Science Publishing, , pp. 201--290.
Autodesk (2011), 'AutoCAD Architecture 2012 user's guide'.
Bjoerk, B.-C. (1992), 'A conceptual model of spaces, space boundaries and enclosing structures', Automation in
Construction 1(1), 193-214.
Bondy, A. & Murty, U. (2010), Graph Theory, Springer.
Boost (2011), 'Boost Graph Library', Boost, Technical report, Boost.
BuildingSmart (2010), 'Industry Foundation Classes IFC2x4', BuildingSmart, Technical report, BuildingSmart.
Eastman, C. & Siabiris, A. (1995), 'A generic building product model incorporating building type information',
Automation in Construction 3 (1), 283-304.
Ekholm, A. & Fridqvist, S. (2000), 'A concept of space for building classification, product modelling and design',
Automation in Construction 9 (3), 315-328.
Galasiu, A. D. & Newsham, G. R. (2009), Energy savings due to occupancy sensors and personal controls: a pilot
field study, in 'Proceedings of Lux Europa'.
Hillier, B. & Hanson, J. (1989), The Social Logic of Space, Cambridge University Press.
ISO (2004), 'Standard for the exchange of product model data (ISO 10303)'(ISO 10303), TC184/SC4, Technical
report, ISO.
ISO/IEC (2008), 'SQL:2008 (ISO/IEC 9075)'(ISO/IEC 9075), Technical report, ISO/IEC.
Kohlhoff, C. M. (2008), 'Boost.Asio Library', Boost, Technical report, Boost.
Krioukov, A.; Dawson-Haggerty, S.; Lee, L.; Rehmane, O. & Culler, D. (2011), 'A Living Laboratory Study in
Personalized Automated Lighting Controls''BuildSys: 3rd ACM Workshop On Embedded Sensing Systems For
Energy-Efficiency In Buildings, Seattle WA'.
Liggett, R. S. (2000), 'Automated facilities layout: past, present and future', Automation in Construction 9(2), 197
- 215.
Newsham, G.; Veitch, J.; Arsenault, C. & Duval, C. (2004), Effect of dimming control on office worker satisfaction
and performance, in 'IESNA Annual Conference Proceedings, Tampa, Florida', pp. 19-41.
Shin, C. & Woo, W. (2009), 'Service Conflict Management Framework for Multi-user Inhabited Smart Home',
Journal of Universal Computer Science 15(12), 2330-2352.
Spatial (2011), '3D ACIS Modeler'.
Suter, G. (2010a), Outline of a schema for network-based space layouts, in K. Menzel & R. Scherer, ed., '8th
European Conference on Product and Process Modelling, European Group for Intelligent Computing in
Engineering (EG-ICE)', Taylor & Francis, Cork, Ireland.
Suter, G. (2010b), Topological constraints for consistency checking of network-based space layouts, in W. Thabet,
ed., '27th International Conference on Applications of IT in the AEC Industry'.
Suter, G.; Petrushevski, F. & Sipetic, M. (2011), Design of a personalized lighting control system enabled by a
space model, in Jeff Haberl, ed., '11th International Conference for Enhanced Building Operations'.
388

AUTOMATIC GENERATION OF 3D BUILDING MODELS BY
STRAIGHT SKELETON COMPUTATION
Kenichi Sugihara
Gifu Keizai University, Japan
Zhen-jiang Shen
Kanazawa University, Japan
ABSTRACT: 3D city models are important in several fields, such as urban planning and gaming industries.
However, enormous time and labor has to be consumed to create these 3D models, using a 3D modeling software
such as 3ds Max or SketchUp. In order to automate laborious steps, we are proposing a GIS and CG integrated
system for automatically generating 3D building models, based on building polygons (building footprints) on
digital maps. Digital maps shows most building polygons' edges meet at right angles (orthogonal polygon). In the
digital map, however, not all building polygons are orthogonal. In either orthogonal or non-orthogonal polygons,
we propose the new system for automatically generating 3D building models with general shaped roofs by straight
skeleton computation defined by a continuous shrinking process. In this paper, the algorithm for shrinking a
polygon and forming a straight skeleton are clarified and we propose the new methodology for constructing roof
models by assuming the third event: simultaneous event in addition to edge event and split event and, at the
end of the shrinking process, some polygons are converged to a line of convergence.
KEYWORDS: 3D urban model, automatic generation, GIS, CG (Computer Graphics), 3D building model,
polygon partitioning, straight skeleton.
1. INTRODUCTION
3D urban models are important in urban planning and in facilitating public involvement. To facilitate public
involvement, 3D models simulating a real or near future cities by a 3D CG can be of great use. However, enormous
time and labour has to be consumed to create these 3D models, using 3D modeling softwares such as 3ds Max or
SketchUp. For example, when manually modeling a house with roofs by Constructive Solid Geometry (CSG), one
must use the following laborious steps:
(1) Generation of primitives of appropriate size, such as box, prism or polyhedron that will form parts of a house
(2) Boolean operations are applied to these primitives to form the shapes of parts of a house such as making holes
in a building body for doors and windows (3) Rotation of parts of a house (4) Positioning of parts of a house (5)
Texture mapping onto these parts.
In order to automate these laborious steps, we are proposing a GIS and CG integrated system that automatically
generates 3D building models, based on building polygons or building footprints on a digital map shown in Fig.1
left, which shows most building polygons edges meet at right angles (orthogonal polygon). A complicated
orthogonal polygon can be partitioned into a set of rectangles. The proposed integrated system partitions
orthogonal building polygons into a set of rectangles and places rectangular roofs and box-shaped building bodies
on these rectangles. In order to partition complicated orthogonal building polygons, a useful polygon expression
(RL expression: edges Right & Left turns expression) and a partitioning scheme was proposed for deciding from
which vertex a dividing line (DL) is drawn (Sugihara 2006, 2009).
In the digital map, however, not all building polygons are orthogonal. In either orthogonal or non-orthogonal
polygons, we propose the new system for automatically generating 3D building models with general shaped roofs
by straight skeleton computation defined by a continuous shrinking process (Aichholzer 1995). In this paper, the
algorithm for shrinking a polygon and forming a straight skeleton are clarified and we propose the new
methodology for constructing roof models by assuming the third event: simultaneous event in addition to edge
event and split event and, at the end of the shrinking process, some polygons are converged to a line of
convergence.
389

2. RELATED WORK
Since 3D urban models are important information infrastructure that can be utilized in several fields, the researches
on creations of 3D urban models are in full swing. Various types of technologies, ranging from computer vision,
computer graphics, photogrammetry, and remote sensing, have been proposed and developed for creating 3D
urban models. Using photogrammetry, Gruen and et al. (1998, 2002) introduced a semi-automated topology
generator for 3D building models: CC-Modeler. Feature identification and measurement with aerial stereo images
is implemented in manual mode.
During feature measurement, measured 3D points belonging to a single object should be coded into two different
types according to their functionality and structure: boundary points and interior points. After these manual
operations, the faces are defined and the related points are determined. Then the CC-Modeler fits the faces jointly
to the given measurements in order to form a 3D building model. Suveg and Vosselman (2002) presented a
knowledge-based system for automatic 3D building reconstruction from aerial images. The reconstruction process
starts with the partitioning of a building into simple building parts based on the building polygon provided by 2D
GIS map. If the building polygon is not a rectangle, then it can be divided into rectangles. A building can have
multiple partitioning schemes. To avoid a blind search for optimal partitioning schemes, the minimum description
length principle is used. This principle provides a means of giving higher priority to the partitioning schemes with
a smaller number of rectangles. Among these schemes, optimal partitioning is manually selected. Then, the
building primitives of CSG representation are placed on the rectangles partitioned.
These proposals and systems, using photogrammetry, will provide us with a primitive 3D building model with
accurate height, length and width, but without details such as windows, eaves or doors. The research on 3D
reconstruction is concentrated on reconstructing the rough shape of the buildings, neglecting details on the faades
such as windows, etc (Zlatanova 2002). On the other hand, there are some application areas such as urban planning
and game industries where the immediate creation and modification of many plausible building models is
requested to present the alternative 3D urban models. Procedural modeling is an effective technique to create 3D
models from sets of rules such as L-systems, fractals, and generative modeling language (Parish et al. 2001).
Mller et al. (2006) have created an archaeological site of Pompeii and a suburbia model of Beverly Hills by using
a shape grammar that provides a computational approach to the generation of designs. They import data from a
GIS database and try to classify imported mass models as basic shapes in their shape vocabulary. If this is not
possible, they use a general extruded footprint together with a general roof obtained by a straight skeleton
computation defined by a continuous shrinking process (Aichholzer et al. 1995).
Fig.1: Pipeline of Automatic Generation for 3D Building Models
GIS
Application( ArcGIS )
*Building Polygons on 2D
Digital Map
*Attributes (right below) such
as the number of stories
linked to a building polygon
GIS Module
(Visual Basic &
MapObjects)

*Partitioning
orthogonal
polygons into
rectangles

*Contour
Generation

*Filtering out
noise edges,
unnecessary
vertices
CG Module
(MaxScript)

*Generating 3D
models &
Boolean
operation

*Rotating and
positioning 3D
models

*Automatic
texture mapping
onto 3D models
Automatically generated 3D urban model
390

The straight skeleton is the set of lines traced out by the moving vertices in this shrinking process and can be used
as the set of ridge lines of a building roof (Aichholzer et al. 1996). However, the roofs created by the straight
skeleton are limited to hipped roofs or gable roofs with their ridges parallel to long edges of the rectangle into
which a building polygon is partitioned. There are many roofs whose ridges are perpendicular to a long edge of the
rectangle, and these roofs cannot be created by the straight skeleton. Since the straight skeleton treats a building
polygon as a whole, it forms a seamless roof so that it cannot place roofs independently on partitioned polygons.
To create the various shapes of 3D roofs, we proposed the system that has an option to select partitioning scheme;
separation prioritizing or shorter DL (dividing line) prioritizing, which is decided by an attribute data manually
stored beforehand (Sugihara 2009). The proposed system also tries to select a suitable DL for partitioning or a
separation, depending on the RL expression of a polygon, the length of DLs and the edges of a polygon. But, the
system was not able to create 3D building models with roofs from non-orthogonal building polygons. In this
paper, we propose the system for creating 3D building models with general shaped roofs by straight skeleton.
More recently, image-based capturing and rendering techniques, together with procedural modeling approaches,
have been developed that allow buildings to be quickly generated and rendered realistically at interactive rates.
Bekins et al. (2005) exploit building features taken from real-world capture scenes. Their interactive system
subdivides and groups the features into feature regions that can be rearranged to texture a new model in the style of
the original. The redundancy found in architecture is used to derive procedural rules describing the organization of
the original building, which can then be used to automate the subdivision and texturing of a new building. This
redundancy can also be used to automatically fill occluded and poorly sampled areas of the image set.
Aliaga et al. (2007) extend the technique to inverse procedural modeling of buildings and they describe how to use
an extracted repertoire of building grammars to facilitate the visualization and modification of architectural
structures. They present an interactive system that enables both creating new buildings in the style of others and
modifying existing buildings in a quick manner.
Vanega et al. (2010) interactively reconstruct 3D building models with the grammar for representing changes in
building geometry that approximately follow the Manhattan-world (MW) assumption which states there is a
predominance of three mutually orthogonal directions in the scene. They say automatic approaches using
laser-scans or LIDAR data, combined with aerial imagery or ground-level images, suffering from one or all of
low-resolution sampling, robustness, and missing surfaces. One way to improve quality or automation is to
incorporate assumptions about the buildings such as MW assumption. However, there are lots of buildings that
have cylindrical or general curved surfaces, based on non-orthogonal building polygons.
By these interactive modeling, 3D building model with plausible detailed faade can be achieved. However, the
limitation of these modeling is the large amount of user interaction involved (Nianjuan et al. 2009). When creating
3D urban model for urban planning or facilitating public involvement, 3D urban models should cover lots of
citizens and stakeholders buildings involved. This means that it will take an enormous time and labour to model
3D urban model with hundreds or thousands of building.
Thus, the GIS and CG integrated system that automatically generates 3D urban models immediately is proposed,
and the generated 3D building models that constitute 3D urban models are approximate geometric 3D building
models that citizens and stakeholder can recognize as their future house or real-world buildings.
3. PROPOSED SYSTEM FOR AUTOMATIC BUILDING GENERATION
As shown in Fig.1, the proposed automatic building generation system consists of GIS application (ArcGIS, ESRI
Inc.), GIS module and CG module. The source of the 3D urban model is a digital residential map that contains
building polygons linked with attributes data such as the number of storeys and the type of roof. The GIS module
pre-processes building polygons on the digital map. pre-process includes filtering out an unnecessary vertex
whose internal angle is almost 180 degrees, partitioning orthogonal building polygons into sets of rectangles,
generating inside contours by straight skeleton computation for positioning doors, windows, fences and shop
faades which are setback from the original building polygon, forming the straight skeleton for the general
shaped roof and exporting the coordinates of polygons vertices and attributes of buildings. The attributes of
buildings consist of the number of storeys, the image code of roof, wall and the type of roof (flat, gable roof,
hipped roof, oblong gable roof, gambrel roof, mansard roof, temple roof and so forth). The GIS module has been
developed using 2D GIS software components (MapObjects, ESRI).
391

The CG module receives the pre-processed data that the GIS module exports, generating3D building models. CG
module has been developed using Maxscript that controls 3D CG software (3ds MAX, Autodesk Inc). In case of
modeling a building with roofs, the CG module follows these steps:
(1) generation of primitives of appropriate size, such as boxes, prisms or polyhedra that will form the various parts
of the house (2) Boolean operations are applied to these primitives to form the shapes of parts of the house, for
examples, making holes in a building body for doors and windows, making trapezoidal roof boards for a hipped
roof and a temple roof (3) rotation of parts of the house (4) positioning of parts of the house (5) texture mapping
onto these parts according to the attribute received.
4. STRAIGHT SKELETON COMPUTATION FOR ROOF GENERATION
4.1 PROCESS of STRAIGHT SKELETON COMPUTATION
As shown Fig.2(f), Aichholzer et al.(1995,1996) introduced the straight skeleton defined as the union of the pieces
of angular bisectors traced out by polygon vertices during a continuous shrinking process in which edges of the
polygon move inward, parallel to themselves at a constant speed. The straight skeleton is unexpectedly applied to
constructing general shaped roofs based on any simple building polygon, regardless of their being rectilinear or not.
The straight skeleton computation is applied for creating roof models based on either orthogonal or non-orthogonal
polygons (Sugihara 2011). In this paper, we propose the new methodology for constructing roof models by
assuming the third event: simultaneous event in addition to edge event and split event and, at the end of the
shrinking process, some polygons are converged to a line of convergence.
According to Aichholzer et al.(1995,1996), in shrinking process as shown in Fig.2, each vertex of the polygon
moves along the angular bisector of its incident edges. This situation continues until the boundary change
topologically. There are two possible types of changes:
(1) Edge event: An edge shrinks to zero, making its neighboring edges adjacent now.
(2) Split event: An edge is split, i.e., a reflex vertex runs into this edge, thus splitting the whole polygon. New
adjacencies occur between the split edge and each of the two edges incident to the reflex vertex. A reflex vertex is
a vertex whose internal angle is greater than 180 degrees.

(b) Split event happens and the
polygon is split into two
polygons.
(a) Shrinking polygon just
before a split event
(c) Shrinking polygon split
just before another split event
(d) Split event happens and
the polygon is split into two
triangles.
Fig.2: Shrinking process and a straight skeleton, a roof model automatically generated
(f) The straight skeleton defined as
the union of the pieces of angular
bisectors traced out by polygon
vertices during the shrinking process
(g) A roof model automatically generated: each roof
board is based on an interior monotone polygon
partitioned by straight skeleton
(e) Polygons shrinking at a constant
interval: nodes by an edge event:
nodes by a split event: nodes by a
collapse of a triangle to a point.
ed2
ed1
392

All edge lengths of the polygon do not always decrease during the shrinking process. Some edge lengths of a
concave polygon will increase. For example, as shown by ed1 and ed2 in Fig.2 (a), the edges incident to one
reflex vertex grow in length. If the sum of the internal angles of two vertices incident to an edge is more than 360
degrees, then the length of the edge increases, otherwise the edge will be shrunk to a point. During the shrinking
process, when the edge is shrunk to a point, we call this event an edge event.
Shrinking procedure is uniquely determined by the distance d
shri
between the two edges of before & after
shrinking procedure. The distance e_d
shri
is the d
shri
when an edge event happens in the shrinking process. e_d
shri

for edge ( ed
i
) is calculated as follows:

where L
i
is the length of ed
i
, and
i
&
i+1
are internal angles of vertices incident to ed
i
. The edge event will happen
when 0.5*
i
0.5*
i+1
180 degrees, i.e., the sum of the internal angles of two vertices incident to an edge is less
than 360 degrees.
Fig.2 from (a) to (d) show a shrinking process for a non-orthogonal concave polygon: the polygon just before a
split event: the polygon being split into two polygons after the split event happens. Fig.2 (e) shows a set of
polygons shrinking at the constant interval and nodes by an edge event and a split event, and nodes by a collapse
of a triangle into a point. Fig.2 (f) shows the straight skeleton defined as the pieces of angular bisectors traced
out by polygon vertices during the shrinking process. Fig. 2(g) shows the roof model automatically generated.
Since the straight skeleton partitions the interior of a polygon with n vertices (n-gon) into n monotone polygons,
each roof board that constitutes the roof model is formed based on these partitioned interior monotone
polygons.
For some polygons such as shown below, the event different from above two events will happen. In our research,
we propose to add the third event: the simultaneous one in which above two events happen simultaneously.
This event happens at an orthogonal part of the polygon as shown in Fig.3, in which event a reflex vertex runs
into the edge, but the other split polygon is collapsed into a node since an edge event happens in the split
polygon at the same time.

Fig.3: Shrinking process and a straight skeleton for simultaneous events
(f) The straight skeleton defined as
the union of the pieces of angular
bisectors traced out by polygon
vertices during the shrinking process
(g) A roof model automatically generated: each roof
board is based on an interior monotone polygon
partitioned by straight skeleton
(e) Polygons shrinking at a constant
interval: nodes by an edge event & a
simultaneous event: nodes by a
collapse of a rectangle to a line.
(b) A simultaneous event happens
and the polygon is split into one
node and one polygon.
(a) Shrinking polygon just
before a simultaneous event
(c) Shrinking polygon split
just before another
simultaneous event
(d) A simultaneous event happens
again and the polygon is split into
one node and one polygon
pt
ia

ed
ib

pt
ib

ed
ia

393

Fig.3 shows a shrinking process of an orthogonal polygon. In the process, as shown Fig.3 (b) & (d), the system
detects the simultaneous event by checking if pt
ia
(vertex) is on ed
ib
(edge) or pt
ib
is on ed
ia
where pt
ia
& pt
ib
are
the vertices next to two vertices coherent by the edge event, and ed
ia
& ed
ib
are the edges adjacent to these two
coherent vertices.
Aichholzer et al.(1996) demonstrated three edge events let a triangle collapse to a point in the last stage of each
split polygon as shown in Fig.2 (e). In this paper, we propose to add the case in which two edge events let a
rectangle collapse to a line segment (a line of convergence) in the last stage, a rectangle whose opposite sides
have the same and the shortest e_d
shri
.
Since a line segment does not have area, it is not shrunk anymore. The central area of an orthogonal polygon in
Fig.3 (e) shows a line of convergence to which the shrinking polygon (rectangle) are converged.

4.2 ALGORITHM for STRAIGHT SKELETON
Building parts, such as doors, windows, fences and shop faades are placed along the inner contour which is
setback by the constant distance from the original building polygon (P). The inner contour setback by the
constant distance can be accurately computed by the well-known medial axis, which consists of all interior
points whose closest point on P's boundary is at least two. However, the inner contour by the medial axis
computation contains parabolic arcs in the neighborhood of reflex vertices, which arises a difficulty of creating
and placing windows, fences and shop faades. Indeed, it is quite difficult to create curved windows and shop
faades. While on the contrary, the inner contour by the straight skeleton computation contains only straight
segments, and it excludes the difficulty of creating and placing curved building parts including roof boards.
The straight skeleton is defined as the pieces of angular bisectors traced out by polygon vertices during the
shrinking process. The angular bisectors are given by the sum of two unit edge vectors whose origins are the
same vertex of P. However, the sum of two vectors does not always lie inside P. The system uses the cross
product of the two vectors so that angular bisectors is inside of P.
Inner polygon shrunk by straight skeleton computation are generated by the following algorithm in Fig.4.

Algorithm: Shrink Polygon ( P , dshri )
Input: n vertices of the polygon P , dshri
Output: polygon Ps shrunk by dshri
The vertices of P are numbered clockwise.
1) Let the previous vertex of pi (the i-th vertex) to be pb and the next vertex of the pi to be pa.
2) Normalize the vector from pi to pb and get uib. Normalize the vector from pi to pa and get uia.
// (uib + uia ) will be the angular bisector of i-th vertex pi.
// However, (uib + uia ) does not always lie inside the polygon.
3) If the cross product of uib and uia > 0
4) then the inside bisector will be - (uib + uia ). // The edge turns to the left.
5) Else the inside bisector will be (uib + uia ). // The edge turns to the right.
6) End if
7) Each vertex of Ps is positioned along the angular bisector of its incident edges.
The offset of each vertex from original vertex is calculated as d_bisect = dshri / sin(0.5 * angle(i))
where angle(i)is the internal angle of pi.
8) Check if any edge shrinks to zero. If so, then make its neighbouring edges adjacent now and recalculate
the length of each edge and internal angles of each vertex.
Fig.4: Algorithm for shrinking a polygon by straight skeleton
394

Fig.5 shows the overall outline pseudo-code for
the straight skeleton computation by split & edge
event and collapse of a triangle to a node. At first,
one simple polygon (P) is given such as shown in
Fig.2. If there is any reflex vertex in the P, then it
can be divided into two or more polygons.
At five lines from the top of the code, the
system calculates e_d
shri
for all edges and finds
the shortest of them. Then, the system checks if
split event occurs by increasing d
shri
by (e_d
shri

/n_step). In this way, the shrinking process may
proceed until d
shri
reaches the shortest e_d
shri

found. In the process, a split event may happen
and the polygon will be divided.
To be concrete, during shrinking to the shortest
e_d
shri
, the system checks if a line segment of an
angular bisector from a reflex vertex intersects
another edge of the polygon or not. If an edge is
found intersected, the system calculates the node
position by the split event. However, one edge
will be intersected by several angular bisectors
from several reflex vertices. Among the several
reflex vertices, the reflex vertex that gives the
shortest d
shri
will be selected for calculating the
position.
After any type of event happens and the polygon
changes topologically, there remains one or more
new split polygons which are shrunk recursively
if they have non-zero area. At that moment, the system recalculates the length of each edge and internal angles of
each vertex in order to find the shortest d
shri
for next events.
The P has a member: split_event_finish_flag (sp_ev_fin_fl) and edge_event_finish_flag (ed_ev_fin_fl) which
indicate whether or not the P can be processed by split event or edge event respectively, during the shrinking
process. If sp_ev_fin_fl is set for the P, then the P is finished with split event checking. If sp_ev_fin_fl is reset,
then the P will be checked whether split event is happened or not.
In the upper half of the algorithm, if at least one possibly divided P remains unchecked for split event, then
SplitEventLoopFinish_flag will be reset and the system cannot get out of the while loop. After all Ps have been
checked for split event, then all Ps are checked only for edge event and then triangle procedure for nodes
generation as shown in the lower half of the algorithm.
The generated nodes will be associated with the edges of original P (original edge: o-edge), since at least three
original edges sweep to form the node. Therefore, at each event when the node is generated, at least three o-edges
will be linked to the node. When a square or a regular hexagon collapses to a node, four or six o-edges will sweep
into a node. This is the case of degeneration.
The third event as the simultaneous one is processed at edge event, since the other split polygon disappears into
a node in this event. After detecting the split event and edge event have occurred simultaneously, the system deals
with the event and links the generated node to three o-edges.
5. APPLICATION
Here are the examples of 3D building models automatically generated by the integrated system. Fig.6 shows
the examples of 3D building models automatically generated by straight skeleton computation from
non-orthogonal building polygons. To ease the discussion, Aichholzer et al.(1996) exclude degeneracies caused by
special shapes of polygon, e.g., a regular polygon. In this paper, we deal with the degenerate cases in which more
While (Event procedure is not finished for all one or more split P) {
While (SplitEventLoopFinish_flag == reset) {
For all one or more split P: { If (P. sp_ev_fin_fl == reset) {
Find the shortest e_d
shri
of the P.
Check if Split Event occurs by increasing d
shri
by (e_d
shri

/n_step).
If (Angular bisector from a reflex vertex intersects another edge) {
Calculate the node position by Split Event.
}
}

For all one or more split P:{ If (P. sp_ev_fin_fl == reset)
{ Reset SplitEventLoopFinish_flag }
}
}

For all one or more split P:{ If (P. ed_ev_fin_fl == reset) {
Find the shortest e_d
shri
of the P; Shrink P by e_d
shri
;
Calculate the node position by Edge Event.}
}

For all one or more split P: { If (P is a triangle) {
Calculate the node position of the triangle.
Associate the node with the original edge. }
}
}
Fig.5: Algorithm for forming straight skeleton by Split &
Edge event and Collapse of a triangle to a node
395

than three edges are shrunk to a point. Ideally, simultaneous n edge events cause a regular n-gon to collapse to a
point but it is difficult to draw such a perfect regular n-gon. Accordingly, the system rectifies the shape of the
regular n-gon so as to let n edge events at the same time. Fig.6 center shows the 3D dodecagon building model
automatically generated based on the degeneracy of 12 edges being shrunk to only one node.

Based on digital maps, we propose the GIS and CG integrated system for automatically generating 3D building
models. To generate real-world 3D urban models, the 3D shapes and material attributes of buildings and other
objects need to be reconstructed. In the reconstructing process, the image data will be acquired by taking
photographs of the objects in the city. But, when thinking of the city layout for the future or the ancient city
models, we cannot take photos of the future of the city, planning road, or the cities of ancient times. Usually and
traditionally, urban planners design the town layout for the future by drawing the maps, using GIS or CAD
packages. There may be several plans (digital maps) for urban planning. Similarly excavation and investigation
companies also submit digital maps as the result of excavation. Usually there are several plans for restored
architectural heritage in the estimation process. There are several restoration maps estimated and proposed by
several archaeological researchers. If the integrated system immediately converts these maps into 3D city models,
the system surely supports the researchers and urban planners investigating the alternative idea.
Fig.7 shows the digital map: the city layout proposed by an urban designer and a 3D urban model automatically
generated. In the 3D model, terrace houses sharing green spaces in large courtyard are laid out for viewing
Nagoya castle. Fig.7 also shows buildings are increasing in height away from castle for each buildings top floor
to see Nagoya castle.
Nagoya Castle
Fig.7: Proposed layout for the town and an automatically generated 3D urban model: terrace houses sharing
green spaces in large courtyard, laid out for viewing Nagoya castle
Fig.6: Non-orthogonal building footprints and 3D building models automatically generated by straight skeleton computation
from two angles
396

6. CONCLUSION
For everyone, a 3D urban model is quite effective in understanding what if this alternative plan is realized, what
image of the town were or what has been built. Traditionally, urban planners design the city layout for the future
by drawing building polygons on a digital map. Depending on the building polygons, the integrated system
automatically generates a 3D urban model so instantly that it meets the urgent demand to realize another
alternative urban planning.
If given digital maps with attributes being inputted, as shown in Application section, the system automatically
generates two hundreds 3D building models within less than 30 minutes.
In either orthogonal or non-orthogonal polygons, we propose the new system for automatically generating general
shaped roof models by straight skeleton computation. In this paper, the algorithm for shrinking a polygon and
forming a straight skeleton are clarified and we propose the new methodology for constructing roof models by
assuming the third event: simultaneous event in addition to two events and, at the end of the shrinking process,
some rectangles are converged to a line of convergence.
Thus, the proposed integrated system succeeds in automatically generating alternative city plans.
The limitation of the system is that automatic generation is executed based only on ground plans or top views.
There are some complicated shapes of buildings whose outlines are curved or even crooked. To create these
curved buildings, the system needs side views and front views for curved outlines information.
Future work will be directed towards the development of methods for the automatic generation algorithm to
model curved buildings by using side views and front views.
7. REFERENCES
Aliaga G. Daniel, Rosen A. Paul ,and Bekins R. Daniel (2007). Style Grammars for interactive Visualization of
Architecture, Visualization and Computer Graphics, IEEE Transactions on Volume:13, 786 - 797
Armin Gruen ,and Xinhua Wang (1998). CC Modeler: A topology generator for 3D urban models, ISPRS J. of
Photogrammetry and Remote Sensing, 53, 286-295.
Armin Gruen ,and et al., (2002). Generation and visualization of 3D-city and facility models using CyberCity
Modeler, MapAsia, 8, CD-ROM.
Bekins R. Daniel ,and Aliaga G. Daniel (2005). Build-by-number: rearranging the real world to visualize novel
architectural spaces, Visualization, 2005. VIS 05. IEEE, 143 150.
Nianjuan, Jiang, Ping, Tan, and Loong-Fah, Cheong (2009). Symmetric architecture modeling with a single
image, ACM Transactions on Graphics - TOG, vol. 28, no. 5
O. Aichholzer, F. Aurenhammer ,and D. Alberts, B. Grtner (1995). A novel type of skeleton for polygons,
Journal of Universal Computer Science, 1 (12): 752761.
O. Aichholzer ,and F. Aurenhammer (1996). Straight skeletons for general polygonal figures in the plane, Proc.
2nd Ann. Int. Conf. Computing and Combinatorics (COCOON '96): 117126.
Pascal Mueller, Peter Wonka, Simon Haegler, Andreas Ulmer, Luc Van Gool. (2006). Procedural modeling of
buildings, ACM Transactions on Graphics 25, 3, 614623.
Sugihara Kenichi (2005). Automatic Generation of 3D Building Model from Divided Building Polygon, ACM
SIGGRAPH 2005, Posters Session, Geometry & Modeling, CD-ROM.
Sugihara Kenichi (2006). Generalized Building Polygon Partitioning for Automatic Generation of 3D Building
Models, ACM SIGGRAPH 2006, Posters Session Virtual & Augmented & Mixed Reality & Environments,
CD-ROM.
Sugihara Kenichi (2008). Automatic Generation of 3-D Building Models with Multiple Roofs, Tsinghua
Science and Technology, Vol.13 No.S1 pp.368-374
397

Suveg, I. ,and Vosselman, G. (2002). Automatic 3D Building Reconstruction, Proceedings of SPIE, 4661, 59-69.
Vanegas A. Carlos, Aliaga G. Daniel ,and Bene Bedich (2010). Building reconstruction using Manhattan-world
grammars, Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on: 358 - 365
Yoav I. H. Parish, and Pascal Mller (2001). Procedural modeling of cities, Proceedings of ACM SIGGRAPH
2001, ACM Press, E. Fiume, Ed., New York, 301308.
Zlatanova, S. ,and Heuvel Van Den, F.A. (2002). Knowledge-based automatic 3D line extraction from close range
images, International Archives of Photogrammetry and Remote Sensing, 34, 233 238.
398

A 3D CAD ENGINE WITH DATA MODELS BASED ON
INTERNATIONAL STANDARDS
Satoshi Kubota
Iwate Prefectural University, Japan
Etsuji Kitagawa
Hannan University, Japan
Kantaro Monobe
Miyagi University, Japan
Kenji Nakamura
Osaka University of Economics, Japan
Shigenori Tanaka
Kansai University, Japan
ABSTRACT: Civil infrastructure is developed through an ongoing process of planning, surveying, design,
construction, and maintenance, and drawings are created in each phase. Drawing data are normally generated
by computer-aided design (CAD) software in the design phase, and are also frequently needed during the
construction and maintenance phases. An environment should be provided in which three-dimensional (3D)
structure information can be efficiently and smoothly used throughout the project lifecycle. Primarily
two-dimensional drawing data are now used in the design and construction stages with the use of 3D
information during these stages having only recently started. In this paper, a 3D CAD engine using a data model
based on international standards is proposed for creating and modifying 3D CAD data. There are three design
concepts required for applying the information to international construction projects: data models based on
international standards, parametric modeling, and the consideration of temporal attributes. The data model is
constructed according to International Organization for Standardization 10303 Standard for the exchange of
product model data. The proposed 3D CAD engine is composed of a data model that manages data for each
function and the application program interface. The 3D CAD engines data model is composed of 3D geometric
shape data, and manages 3D model data, sketches, modeling operational histories, attributes with temporal
elements, and assembly data (used to manage assembly model data). For a case study, 3D models of various
concrete structures are generated with the proposed 3D CAD engine and the software functionality is tested.
KEYWORDS: 3D CAD, International standards, Visualization of design, Product data model, ISO, STEP.
1. INTRODUCTION
Civil infrastructure is developed through an ongoing process of planning, surveying, design, construction, and
maintenance. A wide variety of construction projects, for example, those involving roads, buildings, and rivers,
generate information on geometric shape and geometric behavior (Fischer and Kam, 2002). In construction
projects, generated data should be exchanged and shared between project participants on an ongoing basis. In
standardizing three-dimensional (3D) data, a product data model must be constructed to define and organize the
lifecycle information (Karim and Adeli, 1999; Fischer and Kam, 2002). Generating structure drawing data by
using computer-aided design (CAD) software during the design phase is one effective means of achieving that
objective. The use of CAD software functions and model-generating procedures and operations helps to preserve
the design intent of construction engineers. An environment should be provided in which 3D structure
information can be efficiently and smoothly used throughout the project lifecycle (Kamat and Martinez, 2001;
Aspin, 2007; Staub-French and Khanzode, 2007). Primarily two-dimensional (2D) drawing data are now used in
the design and construction stages, with the use of 3D information in these stages having only recently started. A
3D CAD engine should therefore be developed and operated to create and modify 3D shape information.
Interoperability and compatibility are important when sharing and exchanging civil infrastructure information,
and international standards should be used for international construction projects. In Japan, there are no 3D CAD
engines for civil infrastructure domain. The existing geometry modeling engines such as Parasolid or Acis are
used. However, it is expensive to use the existing kernel in the CAD software.
This paper proposes a 3D CAD engine for creating and modifying 3D CAD data by using data models based on
399

international standards. In this study, International Organization for Standardization 10303 Standard for the
exchange of product model data (ISO 10303, commonly known as STEP) is used to construct the data models.
The data model is represented, and the exchanged 3D information is generated by the proposed 3D CAD engine.
For a case study, 3D models of various concrete structures are generated to evaluate the software functionality.
2. OUTLINE OF 3D CAD ENGINE
A 3D CAD engine is proposed to create and modify 3D data by using data models based on international
standards.
2.1 Outline
The proposed 3D CAD engine is composed of two parts, as seen in Fig. 1: a data model that manages the data
for each function, and an application program interface (API). A final geometric shape is generated by
assembling the sketch and modeling operational histories. The sketch operation defines 2D geometric shapes on
a plane, and the modeling operational histories keep a record of modeling operations in the 3D CAD software. In
the Parts Model section, Geometric Shapes and Attributes are connected. Attributes contains display
and annotation information, as well as information about other attributes. APIs are utilized for CAD software
domains such as roads, bridges, and dams. The 3D CAD engines data model is composed of 3D geometric
shape data, and manages 3D model data, sketches, modeling operational histories, attributes with temporal
elements, and assembly data (used to manage assembly model data).
Fig. 1: Outline of the 3D CAD engine.
2.2 Design Concepts
There are three design concepts required for complying with international standards and applying the 3D CAD
engine to international construction projects: data models based on international standards, the parametric
modeling approach, and the consideration of temporal attributes.
2.2.1 Data Model Based on International Standards
Interoperability and compatibility are important when sharing civil infrastructure information. Project
participants, stakeholders, and computer systems can share and utilize the standardized information by means of
a product data model, which is a method for representing the product and shape information over the lifecycle of
the designed structures. A product data model contains product attributes taken from documents, CAD data,
drawings, design calculations, and other resources so that construction project participants can access the
information. The 3D information should therefore be standardized based on accepted international standards. In
international construction projects, ISO or similar standards should be used. ISO 10303 (STEP) is an
Data model
Assembly drawings
Geometric shape
Attributes
Sketch
Modeling operational
histories Final geometric
shape
Run
Display Annotation Attributes
Connection
Parts model
Applicationprogram interfaces(API)
Operational run
Parts drawings
3
3
Parts model
Store
Store 3
3 Assembly
model
400

international standard pertaining to the exchange and representation of all information about a product over its
lifecycle.
2.2.2 Application of Parametric Modeling
The 3D CAD engine is developed based on the parametric modeling approach. The 3D models created by
existing model functions have no means for design, and do not retain the designers intent. Parametric modeling
considers both the final geometric shapes and constraint conditions on those shapes, and can recreate 3D
geometric shapes by processing the constraint conditions and modeling the operational histories. The proposed
method adopts such features, storing procedural modeling operations and automatically generating final
geometric structure shapes by reprocessing them. Shape definitions are also retained, and the modeling
operational histories and constraint conditions are applied to them.
Designs often change during civil infrastructure design and construction phases. In adopting parametric
modeling, the 3D model data incorporate modeling operational histories and thus the design intent of the
construction engineers who generated the data. The 3D model data can be regenerated by modifying the 3D
model parameters.
2.2.3 Considering Temporal Attributes
Information produced over the entire civil infrastructure lifecycle must be accumulated to analyze problems and
solutions within a temporal sequence and to maintain the infrastructure strategically and effectively (Fisher and
Kunz, 1995; Koo and Fisher, 2000; Chau et al., 2004). Spatial and temporal information must be used to perform
effective structure management. Information is generated over the civil infrastructure lifecycle, including spatial
information such as geographical and positional attributes, and temporal information such as the time that a
service opens and closes to the public (Hartmann et al., 2008; Park et al., 2011). The proposed 3D CAD engine
considers temporal attributes in addition to geometric information.
3. UTILIZATION OF INTERNATIONAL STANDARDS
3.1 Application of ISO 10303 (STEP)
ISO 10303 aims to facilitate the exchange and sharing of all product information across all industries, as well as
to standardize all construction information. ISO10303 specifies the method to represent and exchange the
products informational and drawing data, which pertain to the lifecycle of products in all industries. In this
paper, the data models of the 3D CAD engine are constructed on the basis of international standards; the World
Trade Organization (WTO) established the Agreement on Technical Barriers to Trade, which mandates the use of
international standards. 3D CAD data are defined as being distributed in accordance with ISO standards and
produced by 3D CAD software based on the concept of product data models. ISO 10303 Part 203
(Configuration controlled design) standardizes the exchange and sharing of 3D information (ISO 10303-203,
1994), and is used to represent and exchange data generated by the proposed 3D CAD engine.
Sketch and constraint conditions in parametric modeling are based on ISO 10303 Part 108 (Parameterization
and constraints for explicit geometric product models; ISO 10303-108, 2005), modeling operational histories
are based on Part 55 (Procedural and hybrid representation; ISO 10303-55, 2005) and Part 111 (Elements for
the procedural modeling of solid shapes; ISO 10303-109, 2004), and assembly modeling is based on Part 109
(Kinematic and geometric constraints for assembly models; ISO 10303-111, 2007).
3.2 Data model construction
The data model used in the 3D CAD engine is constructed on the basis of ISO 10303.
3.2.1 Parametric modeling
The 3D data modeling operations are stored in a database based on ISO 10303; Fig. 2 shows the exchanged data.
The data model is based on Part 55 of ISO 10303. The Procedural_shape_representation entity of Part 55 is
used to manage modeling operational histories, and the Procedural_shape_representation_ sequence entity is
used to represent each operation. Fig. 3 shows the proposed data structure for parametric modeling.
3.2.2 Sketches
401

The sketch data model includes characteristics of arbitrary planes, arranged geometric shapes, and geometric
constraints. Arbitrary planes and geometric shapes are represented per Part 42 of ISO 10303, and geometric
constraints are represented per Part 108.
Fig. 2: 3D CAD engine data exchange format.
Fig. 3: 3D CAD engines data structure.
3.2.3 Geometric shapes
The 3D CAD engine uses OpenGL to display the 3D model data. OpenGL cannot describe circles or curved
surfaces, and must represent them with a triangulated irregular network (TIN). The geometric shape data model
holds the 3D data represented by the TIN, as well as previous 3D model data. The 3D geometric shapes are
constructed according to Part 203 of ISO 10303. ISO 10303 prepares application protocols (APs) in Part 200
series to represent product data models for the various application fields. AP203 defines 3D configuration
management, and is called configuration controlled design. This AP can represent the 3D shape of products as
well as handle configuration management information.
3.2.4 Assembly modeling
The assembly modeling data model is similar to the parametric modeling in Part 55 of ISO 10303; the operations
are based on Part 109.
The data model of the proposed 3D CAD engine is shown in Fig. 4, based on the discussion above. All
information in the 3D CAD engine is managed by using the Kaiser package, which contains two classes for
managing system information: a Utility package composed of the classes that define common data structures
for the 3D CAD engine, and a Draw package that holds model data. The Draw package is composed of
Procedure, Sketch, Shape, Geometry, Constraint, Attribute, Annotation, and Presentation
packages. Figure 5 shows the data model of Procedure package; Fig. 6 shows the Geometry package.
Exchanged data
Assembly data (Part109)
3D model
3D geometric
shape data
AP203, Part42
Attributes (Part41)
Connection
Sketch and geometric
constraintsPart108
Constructional operations
Part55, Part111
3D geometric
shapes
Modeling
operational histories
Operation 1
Operation
Creation
Modification
Chamfer the corners
Extrusion
Sweep
Sketch
Operation 2
Operation 3
Operation 4
402

Fig. 4: Parametric modeling data structure.
Fig. 5: Data model of procedure package.
4. SYSTEM FUNCTIONS
4.1 Modeling functions
The 3D CAD engine defines modeling operations for basic shape creation and modification. Construction
engineers can represent 3D shapes by using sketches that describe the cross-sectional shape of the 3D models,
sketch constraint conditions, and modeling operational histories. The modeling methods of extrusion, rotation,
sweep, ruled, mesh, and Boolean are available for basic shape creation. Commands for corner chamfering,
filleting, cutting, translation, rotation, and zoom are available to modify basic shapes. The 3D CAD engine
records modeling operations in sequential order as modeling operational histories. Parameter values can be
modified by construction engineers, and the modeling operational histories refer to the sketch shapes.
Procedure Sketc Constraint
Geometry
Shape
Attribute
Annotation
Presentatio
Utillity
Draw
Kaiser
Utility
Procedure Sketch Constraint
Geometry Attribute
Shape
Anno-
tation
Presen-
tation

403

Sketch functions define 2D geometric shapes on a plane. 2D geometric shapes such as lines, curves, circles,
circular arcs, ellipses, parabolas, hyperbolas, non-uniform rational basis spline (NURBS) curves, and clothoid
curves are available for shape creation. Sketches can define geometric constraint conditions in the horizontal,
perpendicular, and angular directions. Assembly modeling functions allow the assembly of multiple 3D
geometric shapes into parts.
Fig. 6: Data model of geometry package.
4.2 Attribute functions
Attribute functions add attributes to the 3D model data that are originally defined by the 3D CAD engine users.
The 3D model data and attributes have a many-to-many relation. As mentioned above, the 3D CAD engine
records temporal attributes. The 3D CAD engine also allows text annotation, markup, and the modification of
layer, color, line type, and line width display attributes. The engine includes the civil infrastructure specific data
in the attribute data.
4.3 Constraints evaluation algorithm
Construction engineers can create sketches and input geometric constraint conditions by using the 3D CAD
software based on the proposed engine. The function that determines the final geometric shapes by calculating
the geometric constraints on the shapes could be subject to problems, but Fig. 7 shows how such problems are
resolved. The proposed method adopts a graph-solving algorithm for triangular shapes that transforms line
segments and circular arcs created by the user into lines and circles per the constraint conditions. The solver
judges whether the constraints are appropriate, and notifies users of underconstraining or overconstraining. Fig. 8
shows the elements for solving geometric problem. The elements are consists of point, line, and circle.
5. PROTOTYPE EXPERIMENTS
APIs in the 3D CAD engine and its prototype system were evaluated by constructing 3D models of a retaining
wall, a pre-stressed concrete girder, and a gravity retaining wall, as shown in Fig. 9. The prototype system,
programmed in C++ and OpenGL 1.1 under the Visual Studio 2008 Service Pack 1 development environment,
included the functional requirements for sketching, model extrusion, sweeping, corner chamfering, filleting, and
creating offsets. Experiment results indicate that the modeling functions could create 3D model data with the
proposed 3D CAD engine.
6. CONCLUSIONS
In this paper, a 3D CAD engine was proposed to create civil infrastructure project data that comply with ISO

404

10303. There were three design concepts required for applying that information to international construction
projects: data models based on international standards, parametric modeling, and consideration of temporal
attributes. The use of the proposed 3D CAD engine should benefit infrastructure projects through the adoption of
parametric modeling, the consideration of temporal information, and the ability to conform to international
standards. In a case study, 3D models of various concrete structures were generated to functionally evaluate the
software.
Fig. 7: Solver flow for constraints evaluation.
Fig. 8: Elements for solving geometric problem.

Fig. 9: Retaining wall, pre-stressed concrete girder, and gravity retaining wall models.
7. REFERENCES
Aspin R. (2007). Supporting collaboration, in colocated 3D visualization, through the use of remote personal
interfaces, Journal of Computing in Civil Engineering, Vol. 21, No. 6, 393-401.

Solve the equations
Constraints solver
Transformation to line and circle
Transformation to point and line
Transformation to graph
solver
Creation of cluster
Display the geometric shapes
Judgment
of
constraints
Correct
Feedback to the users
Under or over-constraints
Element{
int Type;
double Parameter[3];
};
Type:
Point
Parameter[0];Parameter[1];
Line
Parameter[0];Parameter[1];Parameter[2];
Circle
Parameter[0];Parameter[1];Parameter[2];
405

Chau K., Anson M., and Zhang J. (2004). Four-dimensional visualization of construction scheduling and site
utilization, Journal of Construction Engineering and Management, Vol. 130, No. 4, 598-606.
Fisher M. and Kunz J. (1995). Architecture for integrating software, Journal of Computing in Civil Engineering,
Vol. 9, No. 2, 122-133.
Fischer M. and Kam C. (2002). Product Model and 4D CAD Final Report, Center for Integrated Facility
Engineering Technical Report, Stanford University, No. 143.
Hartmann T., Gao J., and Fischer M. (2008). Areas of application for 3D and 4D models on construction projects,
Journal of Construction Engineering and Management, Vol. 134, No. 10, 776-785.
ISO 10303-203 (1994). Industrial automation system and integration Product data representation and
exchange Part203Application protocol Configuration con-trolled design, International Organization for
Standardization.
ISO 10303-108 (2005). Industrial automation system and integration Product data representation and exchange
Part108: Integrated application resource: Parameterization and constraints for explicit geometric product
models, International Organization for Standardization.
Part55: Integrated generic resource: Procedural and hybrid representation, International Organization for
Standardization.
Part111: Integrated application resource: Elements for the procedural modeling of solid shapes, International
Organization for Standardization.
Part109: Integrated application resource: Kinematic and geometric constraints for assembly models,
International Organization for Standardization.
Kamat V. R. and Martinez J.C. (2001). Visualizing simulated construction operations in 3D, Journal of
Computing in Civil Engineering, Vol. 15, No. 4, 329-337.
Karim A. and Adeli H. (1999). OO information model for construction project management, Journal of
Koo B. and Fischer M. (2000). Feasibility study of 4D CAD in commercial construction, Journal of
Park J., Kim B., Kim C., and Kim H. (2011). 3D/4D CAD applicability for life-cycle facility management,
Journal of Computing in Civil Engineering, Vol. 25, No. 2, 129-138.
Staub-French S. and Khanzode A. (2007). 3D and 4D modelling for design and construction coordination: issues
and lessons learned, Journal of Information Technology in Construction (ITCon), Vol. 12, 381-407.

406

Emergency Medical Dispatch: A Case Study of New Taipei City
Hsiao-Hsuan Liu & Albert Y. Chen
Department of Civil Engineering, National Taiwan University
ABSTRACT: Emergency medical dispatch is a challenging task due to the dense population and unique geospatial
layout in urban areas such as New Taipei City. The allocation and distribution of Emergency Medical Service
(EMS) units and hospitals should be assessed to provide an efficient service. The objective of this research is to
assess the current emergency medical dispatch of New Taipei City using geospatial tools for analysis such as a
facilitys service area. In this study, spatial statistics for EMS units and hospitals are conducted. In addition,
regions that cannot be reached within 10 minutes by EMS units are discovered. The geospatial arrangement of
EMS units and upgrade and new construction of hospitals are suggested to strengthen the efficiency of emergency
medical response. Additional data is required to perform more advanced and complex spatial and temporal
analysis to better prepare the city to future emergency medical demand.
KEYWORDS: Emergency Medical Service (EMS), Service Area, Geographic Information Systems (GIS).
1. INTRODUCTION
Emergency Medical Service (EMS) includes on-site medical treatment of injured or ill patients and the transport of
patients from the incident to hospital in emergency or mass casualties (Laws and Regulations DB of ROC 2012).
High quality EMSs secure life and health of the injured and ill patients in emergency situations. As a result, the
efficiency of EMS arrangement is critical to the survival rate for severe cases. New Taipei City, the most populous
city in Taiwan, has great demand for EMS (Sun et al. 2012). However, due to the vast extent of territory, its unique
geospatial layout, and its uneven distribution of population and medical resources, it is challenging to efficiently
dispatch EMS in New Taipei City. There are also challenges in the present Emergency Medical Service System
(EMSS) of New Taipei City, such as the allocation of its resources and the systematic planning and execution of its
services. These factors all suggest seeking of a more effective way to integrate the EMSS of New Taipei City.
In this paper, the Geographic Information System (GIS) is used as the tool for spatial analysis. A GIS is a piece of
computer software that captures, analyzes, and visualizes geospatial information. GIS can be used for various
scientific investigations including, but not limited to, data visualization, resource management, and development
planning (USGS 2012). Nowadays, with the improvement of technology, GIS has been massively applied to
different domains, such as environmental science, hydrology, health care, and transportation.
The current emergency medical dispatch system of New Taipei City is called Interactive Voice Response (IVR)
(Sun et al. 2012). When an emergency medical dispatcher (EMD) receives an emergency call, the EMD will follow
the standard operation procedures to ask the caller some pre-designed questions and lead the caller to provide
critical information. With the information, the EMD can understand the type, scale, and location of the emergency.
At the same time, the computer uses GIS to find the closest emergency medical unit. The EMD then dispatch that
unit which also needs to have enough human resources. After the mission is completed, the information recorded
by the EMD is combined with the on-the-scene recording sheet, forming up a transaction in the EMS database.
In New Taipei City, the Fire Department plays the role as the emergency medical unit. Under the Fire Department
of the New Taipei City Government, there are six battalions including sixty-seven units which are in charge of
different areas in New Taipei City. In the EMS database, each emergency incident had been recorded. Users can
easily find the unit dispatched and the hospital sent to for each transaction. There are fifty-seven hospital
candidates locating in New Taipei City and the other three cities Taipei City, Taoyuan County, and Keelung City
near New Taipei City. If the patients are sent to the hospitals other than these fifty-seven hospitals, they will be
recorded as being sent to other hospitals.
Throughout an emergency medical action, spatial factors play important roles in its success (Ko 2008). For major
trauma patients, the travel time between the emergency medical unit and the scene, and the travel time between the
scene and the hospital are both decisive of the mortality. As a result, we should take the spatial factors of
emergency medical actions into consideration by using GIS to assess the allocation of the emergency medical units
and whether the service area of each unit is enough or proper in New Taipei City.
407

2. OBJECTIVE
The objective of this research can be separated into several parts. The first is to find out whether the current
allocation and distribution of emergency medical units and the hospitals are proper. The second is to assess the
rationale for the selection criteria of hospitals. Most important of all is to analyze if the service area of both
emergency medical units and hospitals are enough or proper. The authors aim to discover the black regions - places
non-reachable in a reasonable amount of time for emergency transport - of EMS in New Taipei City.
In recent years, GIS has been applied to assist the EMS research in many ways. Ong et al. (2008) have used GIS
technology with a national cardiac arrest database to found the geographical distribution pattern of pre-hospital
cardiac arrest in Singapore. The choice of the traffic mode in EMS can also be simulated by GIS. Lerner et al.
(1999) applied GIS and historical transport data to create a reference model that helps to outline the region where
air or ground transport can minimize out-of-hospital time in emergency medical incidents. Peleg and Pliskin
(2004) present a GIS simulation model suggesting that a dynamic load-responsive ambulance deployment can
make EMS more effective. Eight minutes response time, at most, was set to analyze the allocation patterns of
ambulance in different time of a day.
For the GIS analysis in Taiwan EMS, Ko (2008) applied GIS to analyze the distribution and characters of
cross-district EMS transport for major trauma patients in northern Taiwan. He pointed out that utilizing GIS may
show the spatial characters of these cross-district transports. Furthermore, spatial analysis is a good way to search
the region of concern and the priority for EMS reconstruction.
Previous research show that GIS can help analyze and identify the problem in the existing EMS (Warden et al.
2007; Henderson and Mason 2004; Higgs 2004; Peters and Hall 1999). However, service areas of both emergency
medical units and hospitals have not been thoroughly considered using the transportation network.
4. APPROACH
In order to analyze the spatial factors of EMS in New Taipei City, GIS has been used to integrate all the related
data.
4.1 GIS Visualization
The Institute of Transportation, Minister of Transportation and Communications provided the GIS data for this
work. Data include the outlines for all the cities, districts, and towns in Taiwan. Fig. 1 displays the map of the
districts and the population distribution in New Taipei City.

Fig. 1 Map of New Taipei City with Population
408

In the EMS database provided by Fire Department of New Taipei City Government, there are 1,335,534
emergency medical incidents in total for the year 2010. The emergency medical unit dispatched and the hospital
selection of each incident has been recorded. There are 67 candidate units to be dispatched and 57 candidate
hospitals to be selected. Aside from these 57 hospitals, the patients may be sent to other hospitals. There are
also cases when the patients were not delivered to any hospitals. Although there are 57 hospital candidates within
the EMS database, hospital No. 6 and No. 7 are the same hospital, which is China Medical University Hospital
Taipei Branch. Hospital No. 41 and No. 44 are also the same hospital, Taipei Tzu Chi General Hospital. As a
result, there are only 55 unique hospital candidates. The analysis and results are not significantly affected by
these two inconsistencies in the database, as each of the hospital No. 6, No.7, and No. 41 had been selected for
only once. In addition, there are 50 incidents with incorrect hospital selection. They have been treated as invalid
data in this work.
From the EMS database, the number of total counts each emergency medical unit being dispatched in 2010 is
obtained, as well as the number of total counts each hospital being selected. Those numbers have been input into
the GIS software, ArcGIS Desktop 10. In order to better visualize the data, the authors input the x-y coordinate
of each emergency medical unit and hospital. The total counts of each emergency medical unit being dispatched
in 2010 are displayed in Fig. 2. On the other hand, the total counts of each hospital being selected in 2010 are
displayed in Fig. 3.

Fig. 2 Counts of each EMS unit being dispatched

Fig. 3 Counts of each hospital being selected

409

In 2010, the EMS Unit - Chongyang Unit - had been dispatched for 6687 times, which is the EMS unit with the
most dispatch count among all the 67 EMS units. Near 47% of the incidents for which Chongyang Unit were
dispatched selected New Taipei City Hospital Sanchong Branch, which is the nearest hospital for Chongyang
Unit. 27% of the incidents for which Chongyang Unit were dispatched, the patients were not sent to any
hospitals. About 13% of the incidents selected Mackay Memorial Hospital Taipei Branch. The counts of each
hospitals being selected by Chongyang Unit is shown in Fig. 4. It is easy to discover from Fig. 4 that more
patients were sent to National Taiwan University Hospital and Shin Kong Wu Ho-Su Memorial Hospital while
Taipei City Hospital Zhongxing Branch is much nearer.

Fig. 4 Count of each hospitals being selected by Chongyang Unit

Far Eastern Memorial Hospital had been selected by all the emergency medical units of New Taipei City for
14,612 times in 2011, which is the only hospital that had been selected for more than 10,000 times. Fig. 5 depicts
the counts of each EMS unit selecting Far Eastern Memorial Hospital. Most of the units selecting Far Eastern
Memorial Hospital are located at the districts around it, including Banqiao District, Zhonghe District, Tucheng
District, Shulin District, and Xinzhuang District.

Fig. 5 Count of each EMS unit selecting F. E. Memorial Hospital

410

4.2 Service Area
Service area is a network-based analysis to investigate the region where a certain facility provides its service.
With the streets network, the service area of the facility can be obtained. Rather than a circle area with the same
radius, the service area is shaped according to the transportation network and its length from the facility to the
boundary. The result may look like an odd-shaped polygon.
The Institute of Transportation provided the road network of Taiwan, including those for New Taipei City. The
authors applied these data into GIS and used the network analyst tool in ArcGIS to analyze the service area of
both the EMS units and the hospitals. The analysis has been set to limit the network length to 10,000 meters
since we assume that the average travel speed of ambulances is 60 kilometers per hour and that both the travel
time between the units to the scene and the scene to the hospitals should not be greater than 10 minutes. Fig. 6
displays the 10 km service area of all the 67 emergency medical units in New Taipei City. On the other hand, the
10 km service area of all the 55 hospitals is shown in Fig. 7. To better understand the service area of New Taipei
City EMS, these two GIS layers have been superimposed, as in Fig. 8.

Fig. 6 Service area of EMS units

Fig. 7 Service area of hospitals

The 10 minutes travel time is still dangerous for major trauma patients. These patients need to be stabilized
within the golden period of 4 to 6 minutes. In certain cases without having enough Oxygen, the brain cells of the
patient will be damaged and those cells cannot be recovered. As a result, the study of a smaller service area is
needed.
411

Fig. 8 Superimposed service areas of units and hospitals

The authors choose Chongyang Unit for a deeper investigation. The 6 minutes and 3 minutes travel time is
selected to be included into the GIS representation. In Fig. 9, three layers are superimposed in GIS, including the
service area layers of 10 km, 6km, and 3 km for the Chongyang EMS Unit. In addition, Far Eastern Memorial
Hospital is also checked through the same procedures for 10 km, 6 km, and 3 km service area. The resulting
analysis is displayed in Fig. 10.

Fig. 9 Service area of Chongyang Unit

Fig. 10 Service area of Far Eastern Memorial Hospital
412

5. DISCUSSION & FUTURE WORK
5.1 Allocation & Distribution
Fig. 6 shows that the allocation and distribution of the EMS units in New Taipei City are proper. Most of the
units locate near the residential and industrial areas. The density of the units is in accordance with the population.
The area that is not covered by the 10 km service distance is mostly in the mountains where the population
density is low.
Fig. 7 indicates the allocation and distribution problem of the hospitals. Due to the limitation of medical
resources, the hospitals and the EMS units tend to be located at areas with larger populations. However, there are
6 districts in New Taipei City that are almost not covered by the 10 km service distance. For Wulai, Shiding,
Pinglin, and Pingxi Districts, most of the territory are mountains and the populations are all below 8,000. These
might be reasons why these districts are not in the service area of the hospitals. However, there are some famous
tourist spots in these 4 districts. Lacking the hospitals that can be reached in 10 minutes may seriously threaten
the survival rate of the major trauma patients. Furthermore, the populations of Shimen District and Sanzhi
District are 12,700 and 23,263. It is improper that they are not included in the service area of any hospital.
From Fig. 8, we can see that almost all areas in New Taipei City covered by the service area of hospitals are also
covered by the service area of EMS units. However, there are several areas that are only covered by the service
area of EMS units. Most of these areas are located in the six districts aforementioned in the previous paragraph.
The areas that are not covered by the service areas of both emergency medical units and hospitals cannot be
reached in 10 minutes by both the units and the hospitals. It may take more than 20 minutes for the unit to get to
the scene and then deliver the patient to a hospital. Some reallocation actions or construction of new facilities for
both the EMS units and the hospital should be considered near these areas.
5.2 Dispatch Criterion
Comparing Fig. 10 to Fig. 5, the results are generally consistent. The closer the EMS unit is to the Far Eastern
Memorial Hospital, the more patients will be sent to Far Eastern Memorial Hospital by the unit. There is only a
small contradiction. Yumin Unit and Fuying Unit are on the same direction away from Far Eastern Memorial
Hospital. Yumin Unit is in the 6 minutes service area while Fuying Unit is in the 10 minutes service area.
However, Yumin Unit sent more patients to Far Eastern Memorial Hospital than Fuying Unit did. After checking
the EMS database, the authors realize that Yumin Unit was set up on August, 2010. As a result, Yumin Unit
would only be dispatched after then. It accounts the reason why it sent much fewer patients to Far Eastern
Memorial Hospital than Fuying Unit did in 2010.
Comparing Fig. 9 to Fig. 4, it shows that aside from distance, there may be some other factors that affect the
choice of the hospitals. After reviewing the EMS database, the authors find that those factors include the
condition of the patients and the request from the family member. The requirement of family is the human factor
that is hard to change. However, the other factor reveals that the medical resource and capability of a certain
hospital will affect the choice. Taipei City Hospital Zhongxing Branch is in the 3 minutes service area of
Chongyang Unit , but few patients had been sent there. Shin Kong Wu Ho-Su Memorial Hospital and Taipei
Veterans General Hospital receives much more patients from Chongyang Unit while they are only in the 10
minutes service area. Since Taipei City Hospital Zhongxing Branch is close to populous districts of New Taipei
City, such as Banqiao, and Sanchong Districts, its medical capabilityand capacity should be improved.
5.3 Future Work
The authors have applied GIS to analyze the current EMS in New Taipei City. One EMS unit and one hospital
have been checked to assess the dispatch criterion. The same method can be applied to all other units and
hospitals in New Taipei City to have a deeper understanding of the current condition.
The EMS database of New Taipei City is incomplete. The locations of each emergency medical incident have not
been clearly recorded. To retrieve this data, access to several databases needs to granted and merging of the
transactions needs to be carried out. In addition, geocoding of these incidents form its address to latitude and
longitude coordinates will also need to be done to port the data in to GIS. With the current limited data, the
authors can only discuss the unit dispatched and the hospital selected separately in each incident. If the complete
dataset is acquired including the location of the incidents, the authors can better analyze each emergency medical
413

performance in both dispatch and hospital selection more completely.
EMS is important to everyone in the city. The allocation and the distribution of the EMS units and the hospitals
greatly affect the EMS services. The majority of the population should be covered in both service areas of EMS
units and hospitals. Furthermore, a good emergency medical dispatch pattern may enable a more efficient EMS
spatial arrangement. As a result, more efforts should be made to ensure a better system.
6. ACKNOWLEDGEMENT
The authors would like to thank the Institute of Transportation for the GIS data provided, and the EMS data from
the Fire Department of the New Taipei City Government. In addition, this work has been supported by the
National Science Council of Taiwan under grant number 101-2218-E-002-003, and the New Taipei City Fire
Department.
7. REFERENCES
Henderson S. G., and Mason A. J. (2004) Ambulance service planning: simulation and data visualization
Operations Research and Health Care, vol. 70, pp. 77102.
Higgs G. A. (2004) A literature review of the use of GIS-based measures of access to health care services, Health
Services & Outcomes Research Methodology, vol. 5, pp. 119134.
Ko C. I. (2008) Spatial Distribution and Influence Factors of Cross-District Transports among major trauma in
emergency medical services system, Master Thesis, National Taiwan University, 2008.
Laws & Regulation Database of The Republic of China (Laws and Regulations DB of ROC) (2012) Emergency
medical services act, [Online], Available:
<http://law.moj.gov.tw/Eng/LawClass/LawAll.aspx?PCode=L0020045>.
Lerner E. B., Billitier A. J. IV, Sikora J., and Moscati R. M. (1999) Use of a geographic information system to
determine appropriate means of trauma patient transport, Academic Emergency Medicine, vol. 6, pp. 11271133.
Ong M. E., Tan E. H., Yan X., Anushia P., Lim S. H., Leong B. S., Ong V. Y., Tiah L., Yap S., Overton J., and
Anantharaman V. (2008) An observational study describing the geographic-time distribution of cardiac arrests in
Singapore: What is the utility of geographic information systems for planning public access defibrillation? (PADS
Phase I), Resuscitation, vol. 76, pp. 388396.
Peleg K. and Pliskin J. S. (2004) A geographic information system simulation model of EMS: reducing ambulance
response time, The American Journal of Emergency Medicine, vol. 22, pp. 164170.
Peters J. and Hall G. B. (1999) Assessment of ambulance response performance using a geographic information
system, Social Science & Medicine, vol. 49, pp. 15511566.
Sun W. J., Ma H. M., Yu J. Y., and Chen A. Y. (2012) Applying enterprise management method using balanced
scorecard for emergency medical management, Fire Department of New Taipei City Government.
USGS (2012), Geographic information systems, [Online], Available: http://egsc.usgs.gov/isb/pubs/gis_poster/.
Warden C. R., Daya M., and LeGrady L. A. (2007) Using geographic information systems to evaluate cardiac
arrest survival, Prehospital Emergency Care, vol. 11, pp. 1924.
414

IV. REAL TIME VISUALIZATION

415
416

FROM PHYSICAL TO VIRTUAL: REAL-TIME IMMERSIVE
VISUALISATIONS FROM AN ARCHITECTS WORKING MODEL
Gerhard Schubert
Technische Universitt Mnchen, Chair for Architectural Informatics, Germany
Christoph Anthes
Leibniz Supercomputing Centre (LRZ), Germany
Dieter Kranzlmller
MNM-Team, Ludwig-Maximilians-Universitt (LMU) Mnchen & Leibniz Supercomputing Centre (LRZ)
Frank Petzold
Technische Universitt Mnchen, Chair for Architectural Informatics, Germany
ABSTRACT: Presentations and discussions between architects and clients during the early stages of design
usually involve sketches, paper and models, with digital information in the form of simulations and analyses
used to assess variants and underpin arguments. Laypeople, however, are not used to reading plans or models
and find it difficult to relate digital representations to the real world. Immersive environments represent an
alternative approach but are laborious and costly to produce, particularly in the early design phases where
information and ideas are still vague. This paper presents how linking analogue design tools and digital VR
representation has given rise to a new interactive presentation platform that bridges the gap between analogue
design methods and digital architectural presentation. The test platform creates a direct connection between a
physical volumetric model and interactive digital content using a large-format multi-touch table as a work
surface combined with real-time 3D scanning. Coupling the 3D data from the scanned model with the 3D digital
environment model makes it possible to compute design relevant simulations and analyses. These are displayed
in real-time on the working model to help architects assess and substantiate their design decisions. Combining
this with a 5-sided CAVE offers an entirely new means of presentation and interaction. The design (physical
working model), the surroundings (GIS data) and the simulations and analyses are presented stereoscopically in
real-time in the CAVE. While the architect can work as usual, the observer is presented with an entirely new
mode of viewing. Different ideas and scenarios can be tried out spontaneously and new ideas can be developed
and viewed directly in three dimensions. The client is involved more directly in the process and can contribute
own ideas and changes, and then see these in 3D. By varying system parameters, the model can be walked
through at life size.
KEYWORDS: early design stages, urban design, immersive environment, simulations, HCI.
1. INTRODUCTION
New ways of using computers in the early stages of the architectural design process is the focus of an
interdisciplinary research project in the fields of architecture and informatics. The research intention is to
connect traditional analogue design tools with digital content to support the design process. In this approach,
established design tools are not entirely replaced by digital methods but instead combined with a view to making
the most of both worlds: by combining the advantages of each realm, we expand the possibilities of designing in
real and virtual environments. Building on a pre-existing platform (Schubert et al., 2011a), (Schubert et al.,
2011b), we examined aspects of human-computer interaction and the use of computers in the early design
phases.
The sub-project described in this article focuses on conceptual research and development on the creation of an
interactive, immersive presentation environment for use in the early creative stages of the architectural design
process. Examples of possible uses include presentations to clients or public participation procedures. Building
on the concept of a digital toolkit, the aim is to facilitate a simple and direct transition between design
exploration and design presentation that obviates the need for time-consuming and labour-intensive model
creation, which for architects is of limited use during the conceptual phase where ideas and deliberations are still
vague and ill-defined. This is achieved by coupling a physical 1:500 scale volumetric model with an immersive
VR presentation environment in real time. Here, the actual volumetric model serves as a direct tangible interface
that is familiar and intuitive to use. The real-time immersive representation on the other hand makes it possible
to present and discuss the design not just using the model but actually from within the model. This integrated
417

presentation system offers an entirely new interactive mode of presentation, discussion and exploration of ideas.
Interactive calculations, simulations and analyses augment the model and provide an additional means of
objectively evaluating design variants.
2. THE TASK
Plans and models have traditionally been used not just for planning architecture but also for presenting
architectural ideas for discussion with colleagues, with clients and other participants in the building process.
While architects and planners are schooled in reading plans and understanding, visualising and interpreting the
information they contain, many laypeople find them challenging. To the untrained eye, classical architectural
presentations such as floor plans, sections and elevations are generally hard to understand and to interpret. The
cognitive difficulty lies in translating and interpreting the coded representations in two-dimensional drawings
into imaginable spatial situations.
As a consequence, three-dimensional approaches, such as perspective drawings, have often been used as an
alternative means of presentation. The increased use of computers and advances in hardware and software
performance have given rise to the widespread use of computer renderings and animations, which have now all
but replaced traditional hand-drawings and sketches. Here we should distinguish between two different modes of
presentation: animations and interactive presentations:
Animations can be used to present pre-defined content without user interaction. Generally the
movement follows a pre-defined camera path or flight. The user can pause, stop or start the
animation.
Interactive modes of presentation, on the other hand, allow the user to interact with what is happening.
In many cases, interaction is restricted to certain pre-defined sequences or alternatives.
A further difficulty is the increasing amount of information that needs to be presented. Computers are now used
during the design process to compute analyses and simulations that provide supplementary information about the
design variant and can help in making or justifying design decisions. Some of these, such as general indicators of
building costs or energy efficiency, are now standard tools for assessing design variants. As a consequence,
two-dimensional representations are in many cases no longer an adequate means of communicating information.
Instead such information is communicated in animations and interactive representations as, unlike static
visualisations, they are able to communicate processes and temporal progression.
As building tasks become more extensive, the amount of information increases and the relationships between
them more complex, it becomes increasingly important to employ digital media to present, communicate and
discuss ideas. A key area of application is to communicate the impact of complex architectural projects in cities
and their relationship to the urban environment to the general public, but also in terms of costs and ecological
aspects, to mention just possible scenarios. As a consequence simple three-dimensional visualisations, as well as
animations and in some cases interactive three-dimensional presentations, are increasingly becoming part of an
architects repertoire of presentation tools.
A significant problem, however, is the creation and preparation of data. Currently the aforementioned scenarios
are both costly and labour-intensive. A complete three-dimensional model has to be manually constructed using
3D-modelling software. As only pre-defined scenes and ideas can be presented, it is not possible to make
spontaneous changes to the design or to experiment with the design idea.
3. DESIGN TOOLS IN A COLLABORATIVE ENVIRONMENT
The process of designing is hard to define and describe. While it is, of course, possible to identify several
fundamental principles, a single, clear and universally applicable definition is not possible. This is due not least
to the fact that it is not a linear process. Rather it is an iterative process based on the generation of variants and
the decision processes that follow from these. It is simpler, by contrast, to identify and classify the tools and
media we use to resolve particular design tasks. (Gnshirt, 1999) divides design tools into six main groups:
observation, sketch, design drawing, model, calculation and verbalisation.
All of these tools can be understood as a form of sketching. Sketches represent a draft or design idea: they are
tentative, not fully thought-through [] ideas, thoughts and visions that need further development and
elaboration (Nalbach and Figa, 2003). They are short, concise representations of an idea that focus on an
418

essential aspect, and a means of what (Arnheim, 1972) has described as Visual Thinking. Through the direct
expression of a thought in sketch form, thoughts, ideas and potential solutions are made visible. During the act of
sketching, the resulting sketch itself is simultaneously received as a new impression, assessed and responded to.
Designing can be understood as a kind of dialogue, a conversation, usually held via a medium such as paper and
pencil, with an other (either an actual other or oneself acting as an other) as the conversational partner
(Glanville, 1999). The sketch (whether it is an actual sketch, a model or another medium) becomes ones
conversational partner, firing the doodlers enthusiasm, personal research, and commitment (ibid.), although
the choice of tool depends on the respective design task, the design idea, the designers experience and the
design stage.
Design tools are, therefore, nothing more than a means of exploring ones ideas, and of visualising them in a
concrete form for oneself and for others. Design tools are therefore applicable both for use when exploring an
idea oneself as well as when developing or discussing ideas with others. Those discussing an idea use established
tools such as models or freehand sketches to present ideas or respond to wishes or suggestions from others in an
intuitive way, communicating them simply, clearly and quickly to others involved in the process.
Here we should differentiate between two different kinds of collaboration: the closed coupled design process
and the loosely coupled design process. While with the closed coupled design process, all collaborators
work on a joint design product, in the loosely coupled design process, different designers work on different
sub-aspects. The individual sub-aspects can in turn be the product of a team or of an individual. (Kvan, 2000)
Experience tells us that much design is in fact loose-coupled, with each participant contributing what they can
in different domains of expertise at moments when they have the knowledge appropriate to the situation. (Kvan,
2000)
But what implications does this knowledge have for the way design tools are used? Generally speaking one can
say that, a loose-coupled design process requires a quite different set of tools and conditions to be successful
than a close-coupled one. Collaboration requires more than machinery and systems to occur. (Kvan, 2000). This
applies particularly to the use of purely digital tools. Experience has shown that, in contrast to digital tools,
established design tools such as freehand sketches and models can be used in almost any kind of collaborative
work whether loosely or closed coupled design processes. Alternatively it is possible to use established design
tools for collaborative work where the users may not be at the same physical location. Interaction can also be
performed at one site where the results of the design can be distributed to a single remote site or even streamed
to several participants.
4. SUPPORTING THE DESIGN PROCESS
During the design process, architects repeatedly come up against situations that they cannot immediately resolve.
Various attempts are made to develop alternative solutions, which are then made more precise, modified, worked
up in greater detail or, alternatively, discarded. In assessing which variant is appropriate, architects draw on their
own experience and knowledge they have gained as well as refer to calculations, simulations and other sources
of information. This information is often used in presentations and in discussions with clients to justify certain
design variants. They illustrate the reasoning behind decisions and can serve as criteria for discussing design
ideas.
As part of a semester course together with the chair of urbanism and urban development, students devised and
implemented a set of context-related and human-related tools in prototypical form. The analysis and simulation
tools, which are integrated in the design environment and function in real-time, create decision spaces that assist
the designer in assessing design variants. The use of such tools helps avoid structural weaknesses in the planning
process. The aim is to simulate tendencies during the early design phases, where the data available is often vague
and incomplete, and to display design-relevant parameters with a view to making the spatial quality and
functional aspects of a design more legible and the decision-making process more transparent, effective and clear.
The students realized six interactive real-time simulation tools in prototypical form:
Light and shadow
Legal regulations
Energy-efficiency
Access and distances
Visual spatial relationships
Specific building information (costs, volume)
419

With the help of these simple digital tools, simulations that are normally undertaken at the end of the design
phase can be applied continually while actually in the process of designing. For example, statutory planning
constraints such as building regulations can be incorporated into the design process at an early stage. As a means
of optimising the design, they save time and provide objective assistance that can have a direct effect on the
quality of the design. One can imagine this as a creative cycle in which the computer provides real-time
objective feedback on a variety of relevant issues, which can in turn inform the direction of the architects design
decisions. The boundary between sketch, simulation and analysis blurs into a continuous, creative design
process.

Fig. 1: Four prototypes of simulation and analysis tools: Legal regulations, Light and Shadow, Access and
Distances, Visual spatial relationships.
5. USER SCENARIO
Our user scenario is a client presentation with public participation for a large building project. Aside from the
need to minimise costs, the key criteria are to reduce noise pollution, overshadowing and improve access routes.
Instead of using conventional presentation media such as 2D plans, renderings and models, the presenters have a
large-format multi-touch table and an immersive presentation screen in the form of a Powerwall.
The projection surface of the multi-touch table shows the site as a figure-ground plan augmented with details of
noise levels that updates interactively. A volumetric massing model of the planned project has been placed on the
table. Parallel to the real model and digital representations on the multi-touch table, a perspective view of the
urban situation is shown on the immersive presentation screen from a first-person viewpoint. Everyone watching
is wearing VR-glasses so that they have an immersive experience of the perspective view.
The architect describes the project to the people in the room using the haptic model on the multi-touch table and
the presentation on the Powerwall screen, covering key aspects such as its impact on the urban situation, the
access routes and so on. While explaining, the architect switches on relevant analytical data on the surface of the
table or in the perspective view. From time to time he shifts parts of the model on the table and the analytical
data and simulation results update to reflect the new situation, showing on the multi-touch table and the
Powerwall in real time.
The client asks a few questions and proposes trying out an alternative arrangement for the orientation of the
buildings. To illustrate his point, he re-arranges parts of the model on the table. The 3D perspective view as well
as the calculation of noise levels update immediately to reflect the new urban situation. How this changes the
feeling of the space, the routes into the space and the noise levels visualised based on the analytic data and
simulations is immediately visible for everyone to see immersively, i.e. as if they were in the space itself. The
new situation can be seen and evaluated in the model on the touch-table as well as in the perspective view.
420

6. RELATED WORK
In the past, different concepts for immersive representations of architectural designs and ideas have been
developed and implemented, both in the field of Augmented Reality as well as Virtual Reality. The connection
between the real world and the digital world is a central theme of almost all the approaches.
An example from the realm of Augmented Reality is TAP The Architectural Playground or the tool
SKETCHAND+ (Seichter et al., 2003). The SKETCHAND+ prototype, for example, deals with the connection
between digital and real worlds and makes it possible to present an augmented representation of a digital model
in a real model environment. The project uses AR glasses to project digital and interactive content such as
3D models, images or text onto placeholders in the form of markers in the original model.
The main problem with the above example is the need for a pre-existing 3D model and the effort required to
create them. While it is possible to connect real and digital worlds in different ways, this only works for
pre-defined models and scenarios. In this context, it is not possible to interact entirely freely with it, for example
to amend the geometry. Other ways can be seen at the following projects:
In the field of virtual reality a very early approach for modelling 3D environments while being immersed inside
a VR environment was 3DM (Butterworth, 92), which made use of an HMD. It combined techniques from CAD
as well as drawing programs. Since then the idea of virtual design studios has been pursued by many researchers.
For example, VeDS looked at collaborative design with the help of interconnected VEs (Schnabel, 01).
DesignStation is another tool which makes use of VR technology to ease the design process in an architectural
context (Anderseon, 03). 3D modelling and the use of existing 3D objects and other media is supported. The
environments that are created can be exported into traditional CAD tools.
7. CONCEPT
The project shown here builds on work undertaken last year on a developed design environment. By directly
coupling a real, volumetric model (a working model as traditionally used for urban design projects) with
interactive digital content, the gap between the real model and interactive content is bridged, expanding the
information that is stored in the otherwise purely physical model (Schubert et al., 2011a), (Schubert et al.,
2011b).
A special aspect of our Tangible User Interface (TUI) is the ability to automatically capture 3D objects. This is
what facilitates the seamless interface between the digital tool and the architects familiar way of working by
making it possible for a physical working model, as commonly used by architects, to interact directly with
interactive design, supporting simulations and analyses in real time. In contrast to typical Tangible User
Interfaces as introduced by Hinckley in the context of Visualisation (Hinckley, 1994), the objects are not solely
used as an adaptation of the control system (whereby the geometry of the object can usually be ignored see
also Urp (Underkoffler and Ishii, 1999). Through the markerless, direct connection between the physical and
digital worlds, the analogue objects are connected to the simulation not just in two dimensions but also as whole
volumes, and as such become direct participants in the digital design scenario.
The coupling of the input in combination with the output establishes a straightforward link between the real
existing model and the interactive content. Simulations, analyses and calculations (see Chapter 4) are not only
calculated based on the real model but also projected onto and from beneath the model.
The system employs a semi-modular setup that permits additional tools to be added to the existing platform.
This can be hardware or software tools. In the sub-project presented in this article, we have developed a system
based on the needs of collaborative creative conceptual work that permits the immersive presentation of design
ideas and concepts in the early design stages. The existing scenario is extended through the provision of an
additional mode of presentation which makes it possible to understand architectural environments more naturally
from the perspective of the viewer (the first-person perspective). At the same time the direct and real-time
connection to the real model makes it possible to explore ideas simply and intuitively. The first-person
perspective responds in real time to changes made to the real volumetric model on the multi-touch table. That
means that if a block is shifted in the real model, the three-dimensional representation in the perspective view
updates accordingly, along with any simulations, analytical data or calculations displayed in the perspective.

421

Fig. 2: Concept setup: The architectural design consists of a real, volumetric architectural model, semantic GIS
data and interactive analyses, simulations and calculations. This constellation is displayed as an immersive
presentation in a 5-sided cave or a Powerwall.
7.1 Technical setup of the Design Environment
The technical basis for the design environment is a large-format multi-touch table. Coupled with real-time 3D
object recognition, the table acts as a tangible interface that, in addition to reacting to touches and markers, can
sense objects on the table. The data used is semantic GIS data (Level of Detail 1) in City-GML format.
Alternatively online data, such as from OpenStreetMap 3D can be used. This makes it possible to incorporate not
only purely geometric data such a building envelopes and topographic information but also to use semantic
geographic data, for example roads, streets, public transport networks, building functions, year of construction
etc. The three-dimensional data from the model on the multi-touch table in combination with the GIS data can
serve as the basis for simulations, analyses and rough calculations. The designers are provided with an additional
information layer containing simulation data and analyses that are displayed directly on the model and help them
evaluate the consequences of design decisions. The physical model and the computer, previously two separate
realms, are now united resulting in a continuous design process without interruptions when switching between
media.
Based on the aforementioned predefined requirements, a concept for middleware was developed and
programmed as part of an interdisciplinary student project. The middleware serves as an interface between the
design (in the form of the working model) and the supporting design tools (simulations and analyses) and at
present provides the following components:
Semantic 3D environment based GIS data (City GML) and OpenStreetmap 3D
3D object capture (Kinect)
Gesture recognition as an input device (Firewire Camera)
Marker Tracking as an input device
422

Interface Framework (QT)
Render engine (Open TK)
Based on a modular building block principle, different design-support tools can dock onto the middleware.
This makes it possible for architecture students in future student projects to be able to develop and program their
own tools. For the moment, the scenarios described in section 3 were ported to the system in C# and trials were
undertaken. To make handling as easy as possible, a plugin architecture has been devised. The data exchange
retrieval of the base data on the one hand and communication of the results for display on the other happens in
the middleware. A uniform user interface, and therefore a consistent operating paradigm, is ensured with the help
of a central interface library in the middleware.

Fig 3: Structure of the software concept: middleware, hardware and plugin architecture.
7.2 Technical setup of the immersive displays
In this sub-project, the table setup has been extended to connect it to two different immersive displays supporting
a variety of usage scenarios. These displays act not only as simple projection planes in which the camera
perspective can be adjusted based on the input generated by the tracking systems. While the table makes use of
tangible interaction and provides a very intuitive way of scene manipulation, the VR installations make use of
optical tracking systems and also offer basic interaction possibilities in the form of navigation through the scene.
For the project discussed here we explored immersive environments using a large-format Powerwall (6 3 m) as
well as a 5-sided CAVE
1
(Cruz-Neira, 1992). It is possible to switch between hardware environments depending
on the requirements of the respective application scenario. For technical reasons, with both of these systems, the
perspective representation can only be displayed correctly for a single user. All other users see a slightly
off-perspective view. For lots of viewers, multi-user systems would be needed, such as the C1X6 System. (Kulik
et al., 2011)
7.2.1 Setup of the 5-sided projection installation
The 5-sided projection installation is based on the concepts of the original CAVE and offers additional perceptual

1
CAVE is a registered trademark of the University of Illinois Board of Trustees. The term is used in the context
of this paper to generically refer to CAVEs and CAVE-like displays.
423

qualities and interaction possibilities through the provision of a floor and ceiling. Each of the 5 sides is projected
with two projectors whose projections overlap. The overlap is visually hidden through edge blending technology,
which results in a per plane resolution of nearly 2k. While in this scenario, the additional value of a ceiling is not
as relevant as it is for experiencing architectural interiors, the floor of the installation is especially useful in the
context of collaborative use.
In the CAVE-like installation we have two options for rendering the environment. A first-person view can be
generated as well as a visualisation of the actual table. The CAVE-like installation can simply be used as an
interactive gateway inside a life-size representation of the virtual world. The setup arranged and simulated on the
table is displayed in the CAVE from a users perspective. The integration of collision detection with the help of a
physics engine will allow for realistic walkthrough scenarios. This option is ideal for collaboration and
presentation using a table and display setup in the same location. Where the display shows a first-person view
including the ability to navigate through the scene, a link back to the table could be used to display the users
position in the virtual world as well as a viewing cone on the table to illustrate the direction of view and the
perceived scene of the VR user for the table user.
The second option is a classic tele-presence scenario. Here, we display the table inside the installation as a
representation of the real world table so that users in another location can see what is happening on the
multi-touch remotely. Using the ground plane of the 5-sided projection installation it is possible to walk around
the table as if one were standing in front of it.
7.2.2 Setup of the Powerwall
The Powerwall installation used offers the ability to view stereoscopic content on a 6 3 m projection plane in a
resolution of 4k without using any blending technology. It is driven by a server with 2 QuadroPlex graphics
cards and 2 Sony 4k projectors. While the Powerwall is in general designed for presentations it can also be used
as an interactive display by using its tracking system.
The Powerwall can be used for life-size visualisation as well as a downscaled visualisation. In the first case users
are able to navigate through the scene with a large field of view when they are actually interacting in front of the
display. The downscaled version, on the other hand, offers an overview of the table setup and the simulation data,
which makes it applicable for presentation purposes where larger audiences are involved, which is appropriate
for the client presentation and public participation scenario described in section 2. In these examples two types
of interaction are possible, the use of tangible interfaces and the table on the one hand for scene manipulation
and simulation control and the navigation and viewpoint control for the display on the Powerwall.
8. NETWORK COMMUNICATION AND PROTOTYPICAL IMPLEMENTATION
If we take a look at the implementation of the actual prototype it is interesting to understand the concepts of the
network protocol and the software stack used.
8.1 Network communication
Communication channels between the table and the immersed users have to be bi-directional to allow for
collaboration, irrespective of whether the table and the projection installations are adjacent to one another or in
two separate locations. It has to be separated into different channels.
Audio communication can be established with the help of conventional tools and is not required when
the users are at the same location.
Data transfer from the interactive table to the VR installations: all updates to the physical scene
resulting from user interaction have to be transmitted to the immersive installation. It is reasonable to
calculate the simulation results locally in order to save bandwidth and reduce latencies.
Interaction and navigation attributes are communicated from the VR installations to the table setup.
This aspect is particularly relevant if the users are geographically separate from one another.
If we take a look at the two communication channels, data transfer and interaction and navigation attributes we
424

can define fine-grained protocols for communication at different levels of detail. The table setup generates larger
amounts of data which have to be transmitted reliably. The gathered data is post-processed before sending and
3D-meshes are generated. This geometric data is then transmitted to the VR installation. The transfer of
geometries and of textures has to be performed in a reliable way, typically via a TCP stream. In order to save
bandwidth the whole scene is not constantly distributed. Instead, updates take place dynamically and only what
has changed in the scene is transmitted.
8.2 Prototypical implementation
The prototypical implementation for the immersive installations uses the following software stack:
On the lowest level, graphics drivers are used that include the low-level graphics API OpenGL for rendering and
VRPN (Taylor II, 2001), a common library for gathering input from the tracking devices. TCP and UDP
protocols are used to communicate with the table. To allow for more abstract, high-level, and comfortable
application development, the scene graph OpenSG (Reiners, 2002) is incorporated. Since OpenSG supports a
well-designed concept for rendering with multiple render servers, different display variants can easily be
supported without code alteration, for example a cluster setup such as the current CAVE or a more traditional
setup with nVidia QuadroPlex boards on a server that drives the Powerwall.

Fig 4: Concept for implementation
9. CONCLUSION
The concept and prototypical sub-elements presented here for an immersive presentation environment coupled
with a real model offers an entirely new means of working collaboratively and discussing and presenting ideas. It
makes it possible for laypeople and untrained observers to interact with architectural concepts and ideas, and to
ascertain their impact in a simple and intuitive manner. The viewer no longer considers the model from above
but can move around inside the model, obviating the need to cognitively translate abstract representations into
imagined environments. The result is a natural three-dimensional way of looking at architectural concepts.
Presentations can occur in the same location with all participants present or can be streamed to remote desktops
or immersive visualisation environments for interested viewers to interact with from afar.
A real working model at a scale of 1:500 serves as a Tangible User Interface and provides a way of working
directly and intuitively with no need for special prior knowledge or skills. Conceptual ideas and changes can be
realised simply and intuitively using the real model and viewed simultaneously, without the need for laborious
modelling work, on a computer where it can explored intuitively and immersively in real time. By coupling the
design ideas with interactive simulations, the system provides direct feedback on architectural ideas, concepts
and proposals and their impact on the environment and urban surroundings.
425

9.1 Future Work
There are many possibilities for extending this project. The first stage will be to complete full implementation of
the prototype and to evaluate it through user studies. A number of different scenarios will be explored (working
together in one room or remotely) as will the reactions of different users (e.g. trained users and laypeople). An
additional idea would be the creation of a web-client, which would make it possible to follow an open discussion
from an arbitrary location.
10. REFERENCES
Anderson, L.; Esser, J. & Interrante, V. A Virtual Environment for Conceptual Design in Architecture
International Immersive Projection Technologies Workshop (IPT '03), 2003
Arnheim, R. (1972), Anschauliches Denken: Zur Einheit von Bild und Begriff, DuMont Dokumente Reihe
Kunstgeschichte, Wissenschaft, DuMont Schauberg, Kln.
Butterworth, J.; Davidson, A.; Hench, S. & Olano, T. M. 3DM: A Three Dimensional Modeler Using a
Head-Mounted Display Symposium on Interactive 3D graphics (I3D '92), 1992, 135-138
Cruz-Neira, C.; Sandin, D. J.; Defanti, T. A.; Kenyon, R. V. and Hart, J. C. (1992) The cave: Audio visual
experience automatic virtual environment Communications of the ACM, 1992, 35, 64-72
Gnshirt, C. (1999). Sechs Werkzeuge des Entwerfens, Wolkenkuckucksheim, Vol. 4 No. 1.
Glanville, R. (1999). Researching Design and Designing Research, Design Issues, Vol. 15 No. 2, pp. 8091.
Hinckley, K.; Pausch, R.; Goble, J. C. & Kassell, N. F. Passive Real-World Interface Props for Neurosurgical
Visualization Conference on Human factors in computing systems (SIGCHI '94), 1994, 452-458
Kulik, A., Kunert, A., Beck, S.R.R., Blach, R., Zink, A. and Froehlich, B. (2011). C1x6. A Stereoscopic Six-User
Display for Co-located Collaboration in Shared Virtual Environments, ACM Transactions on Graphics, Vol. 30
No. 6.
Kvan, T. (2000). Collaborative design: what is it?, Automation in Construction No. 9, pp. 409415.
Nalbach, G. and Figa, D. (2003), Die erste Skizze: The first sketch, Frderkreis Dortmunder Modell Bauwesen,
Lehrstuhl fr Entwerfen und Innenraum, [Dortmund].
Reiners, D. OpenSG: A Scene Graph System for Flexible and Efficient Realtime Rendering for Virtual and
Augmented Reality Applications Technische Universitt Darmstadt, 2002
Schnabel, M. A.; Kvan, T.; Kruijff, E. & Donath, D. The First Virtual Environment Design Studio 19th
Conference on Education in Computer Aided Architectural Design in Europe, 2001, 394-399
Schubert, G., Artinger, E., Petzold, F. and Klinker, G. (2011a). Bridging the Gap. A (Collaborative) Design
Platform for early design stages., Proceedings of the eCAADe (Zupani-Strojan, T., Juvani, M., Verovek, .
and Jutra, A., editors), eCAADe; Faculty of Architecture, Brussels, Ljubljana, 187193.
Schubert, G., Artinger, E., Petzold, F. and Klinker, G. (2011b). Tangible tools for architectural design. seamless
integration into the architectural workflow, Proceedings of the ACADIA 2011 (Taron, J.M., editor), Association
for Computer Aided Design in Architecture, Stoughton, WI, 252259.
Seichter, H., Donath, D. and Petzold, F. (2003). TAP. The Architectural Playground - C++ framework for
scalable distributed collaborative architectural virtual environments., IJAC - international journal of architectural
computing No. 01, pp. 2433.
Taylor II, R. M.; Hudson, T. C.; Seeger, A.; Weber, H.; Juliano, J. & Helser, A. T. VRPN: A Device-Independent,
Network-Transparent VR Peripheral System ACM Symposium on Virtual Reality Software and Technology
(VRST '01), ACM Press, 2001, 55-6
Underkoffler, J. and Ishii, H. (1999). Urp: A Luminous-Tangible Workbench for Urban Planning and Design,
CHI 99, pp. 386393.
426

3D MODEL VR APPLIED IN TOURISM PLANNING-TAIWAN
TAOYUAN HAKKA FOLK TOURISM PLANNING FOR EXAMPLE
Ren-Jwo Tsay & Jian-Yuan Chen
Department of Civil Engineering, Vanung University, Chungli City, Towyang County, Taiwan

ABSTRACT: Traditional tourism planning introduction maybe just include 2D pictures and text. Visitors must use
graph and text data to understand the transport and tourist attractions points before tourism. Sometime will low
down the tourism quality for not familiar with planning transportation paths or the characteristics of the view
points.
Hakka culture is quite distinctive characteristics of cultural tourism in Taiwan. The unique architecture and
culture style attracted many domestic and foreign tourists. But the characteristic Hakka buildings sometimes hide
in the countryside are difficult to approach without navigation. . If we can develop a tourism planning system to
introduce the special view points and travel information by 3D system will enhance the visibility and tourism
value.
This paper we hopes to develop tourism planning system of Taiwan Taoyuan Hakka community building 3D
building model by SketchUp program and Google Earth travel guide GIS system. To provide the government and
personal tourism planning reference.
KEYWORDS: 3D model, Hakka folk tourism, SketchUp, Google Earth
1. INTRODUCTION
With the improvement of cross-strait relations and industrial transformation, Taiwan has been actively
supporting the tourism industry. Governments develop many new tourist attractions points and establishment of
tourism related hardware to attract visitors. If we can support tourist attractions environment and tourism content
that can enhance tourism will. Traditionally travel information through leaflets publicity. However, in recent
years with the advances in network and communications technology, many travel information and active news
through the electronic device to provide travelers the latest information. Some device also can provide the
function of the navigation and real-time dissemination of information through the 3D display. So user can easy
to understand the tourism information before travel.
Previous studies about 3D skill applied in tourism planning were discussed as follow. Williams et al. (Williams
et al. 1995) point out virtual reality (VR) potential and some weak point of the tool. Almer et al. (Almer, A. et al.
2002) studied the tourism information system combined with GPS and GIS how to effect the tourism industries.
Laakso et al. (Laakso , A. et al. 2003) studied the 3D maps effect leisure boat tourists found 3D maps have
advantages over 2D maps. Guttentag (Guttentag , D. A., 2010) point out the acceptance of VR tourism will be
determined by a tourists attitudes toward authenticity and his or her motivations and constraints. Fontana et al.
(Fontana, R. et al. 2002) used laser scanner to record the statues 3D digital model.
Hakka ethnic have very large proportion of the population in Taiwan. The special habits and customs have great
value of tourist and retained. If we can apply the VR skill to travel information introduction then travelers can
understand the environment and important view points or special cultures they can visit before journey. In this
paper we applied SketchUp program to setup 3D model in traditional Hakka building and 3D tour planning.
2. 3D MODEL VR APPLY IN TOURISM PLANNING
2.1 SketchUp 3D model setup
(a) SketchUp software
SketchUp program was developed in 1999. This program is easy to operate and visual modeling capabilities.
Google acquired the program in 2006 and combined 3D model with Google Earth program. User can build 3D
model in SketchUp program and translate the model in Google Earth. SketchUp program include drawing,
427

shadow, material input, sandbox and component import functions. Developer can very easy to build 3D model in
short time. The program can connect to other software such as AutoCAD and 3DsMax and Soildwork so is very
popular in school and engineers.

(b) Google Earth and kmz translation file
Google earth was opened in 2005. The system include aerial photo, geographical Information system (GIS),
digital terrain model (DTM), navigation and 3D building function. From the free software user can check the
whole world digital information on the web. The kmz file is translation file between SketchUp and Google Earth
program. From the function we can easy to translate development model to set in the global location.

2.2 Hakka traditional building and culture
Hakka people traditional lived in central China area then migration from central China to southeast China coastal
areas for war or politics. Then for the purpose of business or migration move to Taiwan. Taoyuan County,
Hsinchu and Miaoli region is main Hakka people distribution in north Taiwan. The Hakka traditional building is
call group house. The building setup like shape C, worship ancestral hall is in the middle of square. Traditional
Hakka building also plant tree in the north of the house to protect the north wind in the winter like Fig. 1 shown.
Hakka culture heritages were also very different to other ethnic. Every family has their private earth god or
tiger god to protect their family as Fig. 2 shown. The interesting culture sometime is very important assets or
attractive tourist destination for the tourism.

Fig. 1 :Traditional Hakka building
428

Fig. 2 Hakka culture heritage
3. APPLICATION
3.1 Implementation process
First of all we collect the basic map of tourism area. Then we build local important building 3D model like
traditional Hakka building or transport facilities by SketchUp program. Planning travel path through the
attractions give some special Hakka culture tourism recommended. The flow chart was presented as Fig. 3.
Begin
Basic map
collection
Hakka building and heritage 3D model setup
Tourism planning in 3D model
3D tourism guild animation output
End

Fig. 3 Flow chart for 3D tourism planning
429

3.2 3D model setup
We visited some important traditional Hakka buildings in Taoyang County. Then we setup 3D building model
and internal decoration by SketchUp program as Fig. 4 to Fig. 6 shown.

Fig. 4 3D model in traditional Hakka building

Fig. 5 Hakka 3D tradition building outlook

Fig. 6 Hakka 3D tradition building internal furnishings

430

3.3 Tourism planning in Google Earth
In order to promote green travel concept we plan bike tourism path to connect different attractions and setup 3D
path suggestion as Fig. 8 and Fig. 9. Getting Google Earth kmz file by translation SketchUp function then we
can find tourism planning system on Google Earth as Fig. 10 shown.

Fig. 7 Bike tourism path plan near Taoyang international baseball field

Fig. 8: 3D Bike route planning example
431

Fig. 9: Tourism planning of bike path near Taoyang baseball field

Fig. 10: 3D model of nearby Toyang MRT bule line system in Google Earth

4. CONCLUSIONS
From above discuss and application we can found the conclusion as follows.
1. SketchUp program is very convenience program to build special 3D model such as ancient or new building
model.
2. The translation function of SketchUp can easy to setup 3D reality result in Google Erath.
3. The government or travel agencies can enhance the travel guide by 3D VR navigation system through the
internet in computer or smart phone system to improve the quality of tourism of Taiwan.

432

5. REFERENCES
Williams, P., and Hobson, J. S. P., (1995), Virtual reality and tourism: fact or fantasy?, Tourism Management, Vol.
16, Issue 6, 423427
Almer, A., and Stelzl, H., (2002), Multimedia Visualisation of Geoinformation for Tourism Regions Base on
Remote Sensing Data, Symposium on Geospatial Theory.
Laakso , A., Gjesdal , O., and Sulebak , J. R., (2003), Tourist information and navigation support by using 3D maps
displayed on mobile devices, Workshop "HCI in mobile Guides", Udine (Italy).
Guttentag , D. A., (2010), Virtual reality: Applications and implications for tourism?, Tourism Management , Vol.
31, 637651
Fontana, R., Greco, M., Materazzi, M., Pampaloni, E., Pezzati, L., Rocchini, C., (2002), Three-dimensional
modeling of statues: the Minerva of Arezzo. Journal of Cultural Heritage, Vol. 3, 325331.
433

INCREASED INTERACTION WITH MULTI-USER VIRTUAL
REALITY IN CONSTRUCTION PROJECTS
Janne Porkka, Nusrat Jung
Technical Research Centre of Finland, Espoo, Finland
Sunil Suwal, Pivi Jvj
Metropolia University of Applied Sciences, Helsinki, Finland
Anssi Savisalo
FCG Finnish Consulting Group Ltd., Helsinki, Finland
Jani Pivnen
WSP Finland Ltd., Helsinki, Finland
Jarkko Sireeni
Vianova Systems Finland Ltd., Espoo, Finland
ABSTRACT: During last decades the rapid development of ICT has strongly reformed architecture, engineering
and the construction industry. Model-based applications, such as building information modelling (BIM) and urban
planning tools, are being used by architects and engineers for increasing their productivity. These tools are a
promising platform for planning, but more user-friendly interaction is needed for communicating plans to other
stakeholders and end-users.
This paper introduces an approach where virtual reality applications, together with participatory design and
collaborative design approaches, are used in order to improve interaction and communication between various
stakeholders in urban planning projects. The approach deploys experimental multi-user virtual reality
environment in design review meetings carried out in two urban planning projects in Finland. The environment
consists of three adjacent large wall displays positioned at 45 degrees angle between each other. Navigation in the
model is performed by multi-touch gestures on a large tabletop interface. The first case study, an area development
in Inkilnportti retail area harnessed virtual reality for collaborative design review meeting between the client and
multiple design fields. In the second case study, virtual reality was utilised for a participatory concept development
of Finnoo-Kaitaa city district.
Based on the feedback from participants, visual communication has its benefits when compared to traditional
meetings. The empirical findings collected through self-completion questionnaires from the participants of
meetings indicate a positive attitude towards the new technology uptake. Virtual reality applications are a very
promising channel for client and user participation, and as well, help to bridge the social interaction gap between
industry professionals and non-experts, whose participation is also crucial to successful urban planning. Urban
plans are usually large and complicated. Therefore, effective visualisation is significant for successfully
understanding the spaces, dimensions and associated atmosphere indicated by design solutions.
KEYWORDS: Construction industry, Model-based design, Virtual Reality, Design review, Interaction.
1. INTRODUCTION
During last decades the rapid development of ICT has strongly reformed architecture, engineering and the
construction industry. Ground rules for planning processes are generally described in national land use and
building acts. Major development projects within the industry usually take long time to proceed, even up to 25
years or more (Porkka et al. 2012). Moreover, the industry is constantly struggling to unite fragmented building
and infrastructure data sources and databases. In relation to ICT in construction industry, model-based applications,
such as building information modelling (BIM) and urban planning tools, are being used by architects, engineers,
and planners for increasing their productivity. Eastman et al. (2008) stated that BIM in particular is an emerging
technology for building modelling, collaborative design, and integrated project deliveries.
These model-based tools are a promising platform for planning, but more user-friendly interaction is needed for
communicating plans to other stakeholders and end-users. A participatory approach requires interaction with users,
making it even more complicated to convey the design proposal in a simplified representation. The users may lack
434

skills, knowledge and understanding to evaluate the substance. A traditional design communication at an early
project phase is often built on a paper-based graphical representation, consuming time and limiting the
opportunities for teamwork. Virtual Reality (VR), in particular, has been said to offer promise for design
visualisation to convey messages without communication difficulties (Reich et al., 1996). Visualisation is the
easiest common language for all stakeholders, where professionals and non-specialists are able to relate and
understand the content of design proposals.
Virtual Reality applications have been harnessed to production by many industries. New product development and
communication in automotive, aviation, maritime and military industries have benefitted from them (Brooks,
1999; Gould, 2009) and interaction in projects is more effective (Ganah et al., 2005). The construction industry,
however, has been slow in adopting new virtual reality applications to processes. The implementers have reported
great benefits (Dawood, 2009), for example to help in bridging the communication gap between planning and
design professionals and non-experts. Since large development projects usually involve a great number of
stakeholders, the success in communication is a necessity for stakeholder contribution. Processes are urgently in
the need of easily understandable material for decision makers, who because of distinctive backgrounds might be
inexperienced over the domain specific details.
To enhance interaction, design reviews are a promising playground for virtual reality implementations. The
development process in project includes various design field collaboration through multiple phases. A planning
coordination approach in relation to a land use, infrastructures, buildings, energy, and environment needs to be
thoroughly considered (Porkka et al. 2012). The goal of design review is to spot, as soon as possible in the
production chain, product and process weaknesses, errors and potential problems (Uva et al., 2010), to help in
finding opportunities for creating new alternatives. Further, a design review improves stakeholder communication,
saves costs and prevents possible delays (Carlin, 2010). Since the renewal of Land Use and Building Act back in
1999, participation has widely been encouraged and actively discussed in Finland (Olsson et. al., 2012).
Participatory design approach is an emerging process where users feedback is considered and their opinions have
an influence on the planning (Jvj et al., 2012). Virtual reality applications have recently started to be included in
participatory design methodology (Mobach, 2008) and they increasingly provide opportunities also for
collaborative design. Thorough implementing both approaches, the collaborative and participatory methodologies,
the projects are more likely to succeed.
This paper introduces an approach where virtual reality applications, together with participatory design and
collaborative design approaches, are used in order to improve interaction and communication between various
stakeholders in urban planning projects. The approach deploys experimental multi-user virtual reality environment
in design review meetings carried out in two construction projects in Finland with positive results on increased
interaction amongst the stakeholders.
2. VIRTUAL REALITY ENVIRONMENT UTILIZED IN CASE STUDIES
The use of virtual models is becoming more stimulating among the municipalities and various developers in
Finland. Within the last few years, a term city model has been discussed in the planning and built environment
curriculum. The term was introduced for a process to maintain an up-to-date building and infrastructure model
revision for the city area. Later, the model is being utilised efficiently for land use planning and construction
projects (Vianova Systems Finland, 2012). The city model is an immediate tool between multiple stakeholders,
especially during planning and design phases. Besides, the model highlights the communication between the city
officials, design consultants and contractors, enabling the municipalities to provide better solutions and services.
In future, new building permit procedures, where one can extract specific site data as per ones need, may be
submitted to the authorities and managed digitally.
One of the cities employing the city model in practice is Tampere, which is the third largest city in Finland
located 200 km north from the capital (Tampere, 2012). The city is located between two lakes, a river and a
railroad dividing the centre. An example of a city model is presented in Figure 1.

435

Fig. 1: Two screenshots from Vianova Novapoint Virtual Map application; 3D view on the left and street view on
the right (Images courtesy of Vianova Systems Finland Ltd.).

This paper draws from the case study findings. A virtual reality environment called Innovation kitchen at VTT
Espoo has been utilized to increase interaction between the stakeholders. The multi-user virtual environment,
shown in Figure 2, consists of three adjacent large back-illuminated projection walls adjusted at an angle of 45
degrees. There are two projectors and a dedicated PC operating each of the 3x2 metre display walls. The control
of graphical outputs has also a connection matrix. The matrix enables connecting the laptops of meeting
participant to the displays. To avoid potential problems in the meeting contribution, stereoscopic visualisation
produced with polarisation filters and glasses was not used in case studies. Large projection-based immersive
environments are potentially ideal to experience a shared virtual environment, collaborate and exchange ideas
(Simon and Scholz, 2005).
The navigation in the environment is performed with multi-touch gestures on a large custom-built tabletop user
interface (Harviainen et al., 2011). The solution consists of an ordinary LCD display, 4 cameras, illumination
bands, and an in-house VTT developed software to compute the finger location from shadows. The multi-touch
gestures enable users to move in the model environment with the help of an intuitive map interface. Location and
orientation messages are sent wirelessly as UDP (User Datagram Protocol) messages from the tabletop to the
backend PCs. The terrain model is visualized in the development version of Vianovas Novapoint Virtual Map
viewer.

Fig. 2: Virtual space at VTT Espoo in Finland, set up presented on the left and navigation by tabletop interface
on the right (Photos by Janne Porkka).
Vianovas Novapoint Virtual Map is an add-on tool on top of AutoCAD. It creates a visually rich real-time
virtual model automatically from the input data. The application has two modules; a model builder running in
AutoCAD for generating models, and a standalone viewer for visualising and distributing the models freely. The
model builder produces a complex 3d model from an input data, with parametric rules containing modelling
436

layers, materials, geometries and object relations. Materials and textures are attached to objects automatically.
Currently, the supported data exchange formats in Virtual Map application are DWG, DWG+XREF, LandXML
and Civil3D DWG.
3. INTERACTION IN TWO CASE STUDIES
This article builds on empirical findings from two Finnish cases. The first, an area development in Inkilnportti
retail area, which harnessed virtual reality for collaborative design review meeting between the client and
multiple design fields. In the second case study, virtual reality was utilised for an interaction in the concept
development of Finnoo-Kaitaa city district.
3.1 Inkilnportti retail area development in Kirkkonummi
The future Inkilnportti retail area is located in Kirkkonummi municipality about 25 kilometres west from the
centre of Helsinki city (Kirkkonummi, 2012). The planning area covers 50 hectares of land, which at present is
predominantly forest with sparse settlement. The development is located at a strategic point, where the main road
artery (Highway 51) westwards from Helsinki city meet the end point of the outer Ring Road III. This gives a
lucrative position to commercial development. Total floor area of the development will be approximately 100 000
square metre. The development will be connected to the surrounding future housing areas, as well as a nearby
railway station, with a network of bicycle and pedestrian routes. A direct bus connection towards Helsinki city and
Kirkkonummi centre connects the development area to public transport.
FCG Finnish Consulting Group Ltd. has been commissioned by landowners to prepare a detailed city plan
(asemakaava) and preliminary plans for infrastructure and public areas to the area (Jvj et al. 2012). Planning
is based on the Greater Helsinki regional plan (maakuntakaava) from Uusimaa regional council in 2006 and
general plan (yleiskaava) from Kirkkonummi municipality published in the year 2006. The general plan is being
revised by the municipality in a separate simultaneous process, which will end in a detailed general plan
(osayleiskaava) in the surrounding Inkil area. The detailed city plan (asemakaava) will be subjected to political
decision making by the end of 2012. At an early stage of planning, it became apparent that the complexity of the
planning object and the on-going simultaneous processes pose an exceptional challenge for project management.
An innovative use of BIM in project management, collaboration and communication within the planning team
and towards the other stakeholders (planners, land owners, citizens, municipality and regional authorities) is an
effective tool for the project (Jvj et al., 2012). Screenshots from the Inkilnportti retail area project are shown
in Figure 3. The general Landscape Information Model (LIM) overview is presented on the right, and left section
shows details from an adjacent existing housing area.

Fig. 3: Screenshots from Inkilnportti project (photos on the left by Janne Porkka, model image on the right
courtesy of Skanska CDF Ltd., SRV Group Ltd., Vianova Systems Finland Ltd. and FCG Finnish Consulting
Group Ltd.).
437

3.2 Finnoo-Kaitaa district concept development in Espoo
Espoo, with the population of 250,000, is the second largest city in Finland. The special feature of city is an
urban structure that relies on five different centres combining urban, suburban and rural living (Espoo, 2012).
The city includes also wide-reaching natural areas, international companies and Aalto university campus.
Finnoo-Kaitaa district is situated on the coastline, fifteen kilometres from the Helsinki city centre. The
forty-seven hectare area is at present largely unconstructed, and projected at nearly 20 000 inhabitants and
several thousand jobs in future (WSP Finland, 2012).
The collaborative land use planning for new area vision led by multidisciplinary consulting firm WSP Finland
Ltd. started in the year 2009. The starting point for the development process was from an open ideas evening,
and since then, workshops on different themes were arranged to complete a shared vision. Themes discussed at
workshops were city structure, energy, nature and recreation, architecture and housing, and finally livelihood and
services. For the final evaluation, a synthesis was drawn from freely presented and developed ideas. New Finnoo
underground metro station is the focal point of several on-going development plans. Many participatory methods
activating residents, non-governmental organisations, various professionals, and city representatives are being
used within the development.
The target is to create an urban maritime district, which some participants coined as the Venice of Espoo.
However, the area aims at being recognized regionally for a wide range of ecological built environment
including a water research centre, carbon neutral housing, business and leisure. Architecture in blocks near the
metro station is of a high quality and social places are easily accessible for residents. The area provides a variety
of pedestrian routes and public spaces spiced with exciting artworks, light installations and greeneries. The
preliminary plan of metro station with surroundings is illustrated in the Figure 3. The plan was drafted by
Helsinki-Zrich architects Ltd, and is at the time being further developed by WSP.

Fig. 4: Images from Finnoo-Kaitaa project; photo renderings from model on the left (images courtesy of WSP
Finland Ltd.) and the development plan on the right (image courtesy of Helsinki-Zrich Architects Ltd.).
4. RESULTS FROM UTILIZING VIRTUAL REALITY IN PRACTICE
The project management of both case studies perceived the coordination benefits by use of model-based tools
since the early planning. These applications appeared to support various stakeholder viewpoint considerations in
relation to design content. The model-based tools tend to give an important, easily communicable data for
process analyses. The Novapoint Virtual Map viewer application was used in both case studies for design review
meetings. The research setting used the participatory methods in order to assist conversations between the
stakeholders, including clients, multiple design fields, architects, construction companies and other interested
participants. The sample size for a self-completion questionnaire was entirely 24 professionals, 14 men and 10
women, with ages ranging from 23 to 60 years. The meetings clarified that the industry needs common
438

agreements and data exchange standardisation for presenting correspondingly land use, infrastructures and
building data. We used the tailor-made viewer, to present a myriad of design content.
A traditional design review meeting was the research setting in the Inkilnportti retail area development project.
The virtual model produced by Vianova Systems Finland Ltd. combined various design field 3D plans, such as
ground model, source data for landscape, buildings from two architects, motorway, roads, streets, intersections,
storm water and pedestrian routes. The integration model was an addition to a traditional work meeting, where
each member of the design team usually introduces individual models. The virtual model approach helped to
identify integration difficulties, such as a large elevation difference between two lots, need to redesign an
intersection and to reconsider few pedestrian routes. The navigation was performed by a facilitator, mostly from
flyover perspective, and when challenges where positioned pedestrian perspective was also used. The input and
comments where documented to a meeting memo like in a traditional meeting, but also comments were attached
directly to the Virtual Reality model. Later, the model coordinator delivered the commented Virtual Reality
model to all participants.
Within the other Finnoo-Kaitaa case study, virtual reality application was used to enable better communication
between stakeholders at early district concept development workshop, where an increased interaction was highly
appreciated. The virtual model was developed by WSP and the workshop focused on the livelihood and services
within the area. The use situation at second case study utilised virtual model to motivate workshop participants
to have dialogue and discuss about the plan characteristics. The workshop organisers collected feedback and
reviewed comments. Relevant issues where later changed at the development plan. The city officials noticed
during the meeting that flat land causes sight problems to the area. The intention is to develop a new maritime
district, but buildings and trees planted in pedestrian walkways block the sight to the shoreline. The elevation
problem needs further improvements, like location and height considerations. Photos in Figure 5 elucidate the
research setting in both case studies.

Fig. 5: Photos from the case study meetings; a work meeting in Inkilnportti project on left and interaction at
Finnoo-Kaitaa concept development workshop on right (photos by Janne Porkka).

The stakeholder feedback from the meetings was collected with similar self-completion questionnaires. The
inquiry targeted on to evaluate the acceptance of technology and to capture what kinds of influences the use of
virtual model has on understanding the design content. The questionnaire included nine fixed questions, to be
answered on a scale of 1 to 7, spanning from totally disagree to totally agree. Empirical results are summarised
in Table 1.
439

Based on the feedback, 88 percent of the participants reported clear benefits from the model use, and likewise,
are interested in utilising a similar technology in future. Additionally, the multi-projection wall solution supports
comprehensibly the increased design content understanding (67% agreed) and is considered as a suitable
presentation format for the meeting (75% agreed). The realism in virtual models was reflected neutral by 58
percent of the observers. We also had a facilitator assisting in the movements within the environment. All
respondents were neutral or liked the model navigation, although nearly all were first-timers in experiencing the
virtual environment, and none felt the environment physically unpleasant either.

Table 1: Summary of empirical results as seven-scale percentage values (%) in Inkilnportti and Finnoo-Kaitaa
case studies (sample size 24 respondents).
Tot.
disagree
Neutral Tot.
agree
Q1. Meeting was successful 0 0 0 0 33 46 21
Q2. Meeting benefitted from virtual model 0 0 0 0 13 25 63
Q3. Helps to understand design content 0 0 4 4 25 29 38
Q4. Suitable presentation format for meeting 0 0 0 4 21 29 46
Q5. Virtual model felt realistic 0 0 8 8 42 38 4
Q6. Movement in model was normal 0 0 8 29 38 25 0
Q7. Physically unpleasant experience 58 25 0 4 13 0 0
Q8. Interest to use technology in future 0 0 4 4 4 17 71
Q9. Interest to test model via Internet in personal computer 0 4 4 0 4 25 63

The two case studies have differences in a project size, the restrictions by surrounding area plans and structures,
which also reflects partially on interaction. The scale of design options was more compact in Inkilnportti case
study. Therefore, the discussions were also focused and issues like the elevations and levels of existing ground
with respect to the plan were considered. Whereas, in Finnoo-Kaitaa the concept development of unconstructed
area covered many viewpoints, abstract thinking being at the preliminary level for the district development.
The quality of virtual model is an interesting, and partially biased topic. For expert communication, where
participants are likely to understand the substance precisely, a realistic visualisation seems to be an interaction
enabler. We tried to minimise the amount of manual work at the virtual model creation and visual quality was not
a priority. Instead, the priority was to have a virtual model helping the planning coordination. A well-managed
navigation deemed an essential part of a meeting, since the designs usually are reviewed in the patterns of
solutions. Virtual environment enables easier interaction and helped to understand complex projects through
visual aids. Urban planning projects are complex in nature, the size of project and the participants have a straight
impact on how one can quickly and easily perceive the project. As identified within both cases, going through
the key stakeholder perspectives at the early stage help to avoid flawed decisions. These flawed decisions may
result later in compromises and potentially unsustainable development (Isaacs et al., 2011).
For efficient communication, land use development and construction projects are a challenging domain. The
empirical findings collected through questionnaires from two case studies, a retail area development and the
concept development of city district, indicate a positive attitude towards the virtual reality technology uptake. The
two case studies are different in the project size and restrictions, reflecting partially on interaction. In the retail
area development project, the scale of design options was more compact with focused discussions. Whereas, in
the other case concept development called for relatively abstract viewpoint considerations.
440

Multi-user virtual reality applications were seen as a very promising channel for collaboration and participation.
A vast number of respondents were able to realise clear benefits from virtual reality. An immersive wall solution
increases the understanding of design content and fits well into the meeting presentations. On the other hand, the
novice virtual reality utilizers may be overwhelmed about the immersing technology, which surrounds the
observers. Still, we argue that virtual reality applications are a very promising channel for client and user
participation, and as well, help to bridge the social interaction gap between industry professionals and non-experts,
whose participation is also crucial to successful urban planning.
An effective visualisation is significant for successfully understanding the spaces, dimensions and associated
atmosphere indicated by design solutions. This is similar to collective intelligence, which tries to understand the
relations of various objectives, agree on common targets and detect promising opportunities (Pivnen, 2005).
The urban development is currently aiming at understanding the components of functional areas and how social
problems are avoided with the help of detecting the neighbourhood signals.
The rapid pace of technology advancement has influences on virtual technologies. Recently, the applications have
been overlapping video game industry (Smith & Trenholme 2009; Kaplan & Haenlein, 2010; Yan, Culp & Graf
2011). Serious gaming applications complement the existing applications to a better visual performance and an
enhanced user-friendliness. The gaming is also a path opening towards a multi-platform support. We asked in the
questionnaire opinions on the web models, and interestingly, nearly all respondents were in favour of using
Internet as a distribution channel. What if the models in future are reviewed on a web-based multi-user
environment?
6. ACKNOWLEDGEMENTS
This research has been supported by the Finnish Funding Agency for Technology and Innovation (TEKES) under
the project PRE/BIMCity (dnr 1954/31/2010). Authors want to express their gratitude to stakeholders in two case
studies, especially project development director Kari Hovi in Skanska CDF Ltd. and Finnoo-Kaitaa project
manager Torsti Hokkanen in the city of Espoo. Main and the second author are doctoral candidates in Finnish
universities. Janne Porkka in the Faculty of Built Environment at Tampere University of Technology and Nusrat
Jung in the Department of Energy Technology at Aalto University. Special thanks for our colleague Kari Rainio
at VTT for assisting in virtual space arrangements and helping in technical description.
7. REFERENCES
Brooks, F.P. (1999). What's Real About Virtual Reality?, IEEE Computer Graphics Applications, Vol. 19, No. 6,
16-27.
Carlin, B., (2010). The Design Review Process, ASHRAE Journal, Vol. 52, No. 6, 64-66.
Dawood, N. (2009). VR-roadmap: a vision for 2030 in the built environment, Information Technology in
construction ITcon, Vol. 14, 489-506.
Eastman, C.M., Teicholz, P., Sacks, R. and Liston, K. (2008). BIM Handbook: A Guide to Building Information
Modeling for Owners, Managers, Architects, Engineers, Contractors, and Fabricators, John Wiley and Sons,
Hoboken, NJ.
Espoo (2012). The city of Espoo web pages, available at: http://www.espoo.fi (accessed 15
th
June 2012).
Ganah, A., Bouchlaghem, N. and Anumba, C. (2005). VISCON: Computer visualisation support for
constructability, Journal of Information Technology in Construction (ITcon), Vol. 10, pp. 69-83.
Gould, L. (2009). Simulation makes vehicles real, Automotive Design & Production, Vol. 121, No. 7, 22-23.
Harviainen, T., Koskinen, H. and Laarni, J. (2011). Tabletop user interface for navigation in virtual environments.
Proceedings of IADIS International Conference in Interfaces and Human Computer Interaction, Italy, Rome, July
24-26th, 8 pg.
Isaacs, J., Gilmour, D., Blackwood, D. AND Falconer, R. (2011). Immersive and Non-immersive 3D virtual city:
Decision support tool for urban sustainability. Journal of Information Technology in Construction (ITcon), Vol. 16,
151 - 162.
441

Jvj, P., Suwal, S., Porkka, J., Savisalo, A. and Liukas, J. (2012). Social interaction in urban planning projects,
Proceedings of CIMW78 2012, Beirut, Lebanon, October 17-19th. 10 pg., to be published.
Kaplan, A. M. and Haenlein, M. (2010). Users of the world, unite! The challenges and opportunities of Social
Media, Business Horizons, Vol. 53, Iss. 1, 59-68.
Kirkkonummi (2012). The municipality of Kirkkonummi web pages, available at: http://www.kirkkonummi.fi
(accessed 15
th
June 2012).
Mobach, M. P. (2008). Do virtual worlds create better real worlds? Virtual Reality journal, Vol. 12, Iss. 3, 163179.
Olsson, T., Savisalo, A, Hakkarainen, M. and Woodward, C. (2012). User Evaluation of Mobile Augmented
Reality in Architectural Planning, Proceedings of ECPPM 2012, Iceland, Reykjavk, July 25-27th. 8 pg., to be
published.
Pa iva nen, J. (2005). Collective Intelligence, Creativity and Trust (in Finnish). Creating Partnership in City
Districts (Kurki, H. and Tolvanen, A., editors). City of Helsinki and Uusimaa regional council, Helsinki.
Porkka, J., Jung, N., Pivnen, J., Jvj, P. and Suwal, S. (2012). Role of social media in the development of land
use and building projects, Proceedings of ECPPM 2012, Iceland, Reykjavik, July 25-27th. 8 pg., to be published.
Reich, Y., Konda, S.L., Levy, S.N., Monarch, I.A. and Subrahmanian, E. (1996). Varieties and issues of
participation and design, Design Studies, Vol. 17, No. 2, 165180.
Simon, A., and Scholz, S. (2005). Multi-viewpoint images for multi-user interaction, Proceedings of Virtual
Reality05, IEEE, Germany, Bonn, March 12-16th, 7pg.
Smith, S.P. and Trenholme, D. (2009). Rapid prototyping a virtual fire drill environment using computer game
technology, Fire Safety Journal, Vol. 44, 559569.
Uva, A., Cristiano, S., Fiorentino, M. and Monno, G., (2010). Distributed design review using tangible augmented
technical drawings, Computer-Aided Design, Vol. 42, 364372.
Vianova Systems Finland (2012). Web pages for virtual city model (in Finnish), available at:
http://www.kaupunkimalli.fi (accessed 15
th
June 2012).
WSP Finland (2012). Finnoo-Kaitaa vision - Finnoo an Urban Marina, online development document published at:
http://www.finnoovisio.info (accessed 15
th
June 2012).
Yan, W., Culp, C. and Graf, R. (2011). Integrating BIM and gaming for real-time interactive architectural
visualization, Automation in Construction, Vol. 20, 446458.
442

A HEATSTROKE PREVENTION SYSTEM FOR CONSTRUCTION
WORKERS USING SIMULATION AND VR
Nobuyoshi Yabuki, Takuya Onoue & Tomohiro Fukuda
Osaka University, Japan
Shinji Yoshida
University of Fukui, Japan
ABSTRACT: Recently, the number of heatstroke sufferers is increasing among construction workers. To prevent
heatstroke at construction sites, it is necessary to predict accurately both thermal environment of construction sites
and physiological condition of construction workers, which is difficult to achieve now. In this research, therefore,
Construction Workers Heatstroke Prevention (CWHP) System was developed. The CWHP System consists of
Thermal Environment Prediction (TEP) System, which predicts the changes of thermal environment with thermal
sensors at construction sites, the forecast values of meteorological institute and results of CFD and shadow
simulation, and Internal Body Temperature Prediction (IBTP) System, which predicts the changes of internal body
temperature of construction workers with the result of TEP System and physiological sensors attached to
construction workers. The CWHP System was applied at Osaka University campus with a hypothetical situation in
summer, 2011. As the result, the time history of the internal body temperature of the construction worker could be
predicted and the system informed the time of the occurrence of heatstroke.
KEYWORDS: heatstroke, CFD, construction worker, VR, thermal environment.
1. INTRODUCTION
In recent years, heatstroke has become a serious problem in summer in Japan. The number of heatstroke
sufferers per year is over 20,000 after 2006 and is increasing year by year. Particularly in 2010 by smothering
heat wave, 56,064 people were reportedly transported to hospital by ambulance and 171 were killed by
heatstroke. Heatstroke occurs when the body temperature rises abnormally because of high temperature and
humidity. About 70% of fatal occupational decease due to heatstroke is from construction industry. As
construction site is a severe thermal environment in summer, risks of heat stroke of construction workers should
be evaluated considering the thermal environment and human physiological amount.
The risk of heatstroke occurrence becomes high when the internal body temperature rises to about 39 degrees
Celsius (Ko 1966, Sakurai 2010). Equations for estimating the internal body temperature have been proposed by
Kubota et al (1999). However, no methods have been developed to predict heatstroke occurrence at construction
site by integrating technologies of sensing and predicting thermal environment conditions, monitoring and
evaluating construction workers physiological conditions. Therefore, the objective of this research is to develop
a heatstroke prevention system for construction workers by using and integrating Information and
Communication Technology (ICT) and simulation technologies.
2. OVERVIEW OF THE CWHP SYSTEM
The developed system in this research is named Construction Workers Heatstroke Prevention (CWHP) System.
This system is composed of the following three sub systems, i.e., the Thermal Environment Prediction (TEP)
System, the Internal Body Temperature Prediction (IBTP) System, and the Navigation System as shown in
Figure 1.
In the TEP System, firstly, thermal environmental data at the construction site is obtained from sensors installed
at the site. Then, climate forecasting information of the pressure pattern, maximum and minimum temperatures
of the day issued by the Meteorological Office of Japan is obtained via the Internet. As the thermal environment
change is strongly related to the pressure pattern, the TEP System predicts the change of the thermal
environment of the site of the day on the basis of the statistical data pattern of the monitoring data of Automated
Meteorological Data Acquisition System (AMEDAS) of the Meteorological Office near the site for each pressure
pattern.
Next, the IBTP System predicts the heat balance of each construction worker, based on his/her monitoring data
443

and predicted thermal environment data by the TEP System and then predicts change of the internal body
temperature. The risk of heatstroke is evaluated by the internal body temperature.
Finally, the Navigation System sends adequate information such as alarm, caution, advice, etc., to each
construction worker, and based the predicted heatstroke risk, the worker can avoid heatstroke through following
the received information.

Fig. 1: Overview of the CWHP System
3. THE TEP SYSTEM
In order to estimate the risk of heatstroke, it is necessary to forecast the change of the thermal environment
including temperature, humidity, wind direction, wind velocity, insolation, etc., in several hours from the present
time and to calculate the heat load to the body, based on the predicted thermal environmental data. However,
since Computational Fluid Dynamics (CFD) analysis usually takes from hours to days for computation, a much
faster system should be developed to predict the thermal environment change.
Thus, in the TEP System, using statistical analysis, pressure patterns are classified as 1) summer type, 2) front
type, 3) unsettled weather type, and 4) typhoon type, and for each pressure pattern, typical charts of temperature,
humidity, wind directions, wind velocity, and insolation are made. The temperature patterns are shown in Figure
2. Deviations are also considered for each chart. Based on the Meteorological Offices forecast of the pressure
pattern and the maximum and minimum temperature of the day, the thermal environment at the construction site
can be predicted with the results of the number of CFD analyses which have already been done. Based on the
CFD analyses, each cell of the gridded area has its own coefficient for temperature and wind velocity compared
with the benchmark point at the site, which is indicated as a star in Figure 3. By the coefficients, each places
prediction can be done very quickly. Figure 4 shows a general flow of the TEP System. At the construction site, a
number of sensors are installed and environment data is recorded in real time. The predicted data and sensing
data are compared and if they are not in good agreement, prediction should be modified, otherwise, the predicted
data is employed by the IBTP System.
4. THE IBTP SYSTEM
The IBTP System predicts the change of the internal body temperature of each construction worker, based on the
predicted environment data by the TEP System, the workers physiological data and planned place information.
Then, the system predicts the probable time of heatstroke occurrence if anticipated. Figure 5 shows the flow of
the system. The internal body temperature for each worker can be estimated by the Equation (1) in the following.
Sensing Data Simulation Data
Meteorological Office
Climate Forecasting
Information
Construction Site
Thermal
Environmental Data
Construction
Workers
Physiological Data
Construction Workers
Heatstroke Prevention
(CWHP) System
Thermal Environment
Prediction (TEP) System
Internal Body
Temperature Prediction
(IBTP) System
Navigation System
Environmental Data
Analysis According to
Pressure Patterns
CFD Analysis
Sun and Shade Simulation
444

3
5 1
( )
4 3.47 10 4
D
re res res sk
t
A
T M W C E C R E dt T
W

(1)
where
T
re
: change of body temperature during the day [], A
D
: total surface of a human [m
2
],
W
t
: weight of a human [kg], M: metabolism [W/m
2
], W: external work [W/m
2
],
C
res
: heat by respiratory convection flow [W/m
2
],
res
E : respiratory heat of vaporization [W/m
2
]
,
C: heat by convection [W/m
2
], R: heat by radiation [W/m
2
], E: heat by vaporization [W/m
2
],
T
sk
: change of average skin temperature []

If the heatstroke is anticipated, the IBTP System sends the data to the Navigation System which gives the
construction workers and managers alarm, caution, suggestion, or advice based on the seriousness and
emergency.

Fig. 2: Average temperature charts for each pressure pattern

Fig. 3: Coefficients for each cell of the gridded area for predicting temperature based on the benchmark place
(star)
445

Fig. 4: Flow of the TEP System

Fig. 5: Flow of the IBTP System
446

5. THE VERIFICATION OF THE METHODOLOGY AND THE SYSTEM
In order to verify the proposed methodology we developed a prototype system and executed an experiment at the
Techno Alliance Building of Osaka University in summer of 2011. We used 3ds Max of Autodesk for 3D
modeling and 3DVIA Virtools 5.0 of Dassalt Systems for developing the prototype system. The building was
constructed in 2010 and already completed. However, it was hypothetically assumed under construction for the
experiment. Although the experiment should be executed by real construction workers at a real site on a hot
summer day, it would be dangerous for the workers and such an experiment ethically cannot be done. Thus, the
experiment was performed in a cyber-world, in which a realistic construction site on a hot summer day was
replicated based on the in-situ monitored data. To replicate such an environment, the VR Avatar Thermal
Environment (VRATE) System was used. The VRATE System has been previously developed by the authors
(Onoue et al 2010). Figure 6 shows a screenshot of the VRATE System. In this research, actual observed data on
a hot summer day, i.e., August 17, 2011 was employed (Figure 7) and hypothetical construction work was
assigned to the Avatar (Figure 8) and his internal body temperature was computed by the CWHP System (Figure
9). Since the internal body temperature was predicted to be close to 39.0 degrees Celsius, the initial work order
was evaluated inappropriate. Thus, the work order was changed so that he could work at cooler place. Then, the
plan what-if constructing a sunshade tent was evaluated for comparison. Figure 10 shows the three patterns of
the Avatars internal body temperature. Figure 11 shows the difference depending on the amount of perspiration.
The experiment showed that the change in work order and using a tent were obviously effective. The experiment
showed that the system enables its users to evaluate the effectiveness of countermeasures and the system would
be useful for selecting appropriate countermeasures.

Fig. 6: A screen shot of the VRATE System and annotations

VR Avatar and Visualization of thermal
environment of VR Avatar
Position and orientation of VR Avatar
Information
of wind
Analytical date, temperature and humidity
Information
of thermal
index
Menu of Visualization
447

Fig. 7: Observed temperature and relative humidity on August 17, 2011 near the Techno Alliance Building

Fig. 8: Hypothetical work order to an Avatar construction worker

Fig. 9. Predicted internal body temperature of the Avatar construction worker.
35.0
36.0
37.0
38.0
39.0
8 9 10 11 12 13 14 15 16 17 18
I
n
t
e
r
n
a
l

b
o
d
y

t
e
m
p
e
r
a
t
u
r
e

[
]
Time of day [hr]
40
45
50
55
60
65
70
75
80
85
90
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
H
u
m
i
d
i
t
y

[
%
]
Time of day [hr]
448

Fig. 10: Predicted internal body temperature of the three difference cases

Fig. 11: Predicted internal body temperature of different amount of perspiration cases.
6. CONCLUSIONS
In this research, the Construction Workers Heatstroke Prevention (CWHP) System was developed by integrating
sensing and simulation technologies. The developed methodology and the system were verified by executing an
experiment using the VR Avatar Thermal Environment (VRATE) System, and the following items were found.
The system could predict the temporal change of the internal body temperature of the construction
worker at a hypothetical construction site, based on the environment data and workers
physiological data. The system could predict the time when heatstroke was highly occur during the
day based on the internal body temperature.
The system could predict the risk of heatstroke, corresponding to the changes of the site
environment and change of workers amount of perspiration flexibly.
Future work includes the reduction of much work in CFD analysis and the sun and shade simulation, verification
of the system at a more realistic environment. Further, where on a human body to place thermal sensors, how
comfortable workers are if they wear such sensors and the privacy issue of workers should be investigated. This
research will be useful not only for construction industry but also for elderly citizens who live remotely, alone to
reduce their death and disease caused by heatstroke in summer.
7. REFERENCES
Ko I. (1966). Studies on Heavy Muscle Work under the Environments of High Temperature and Humidity. II.
About Physilogical Thresholds under the Labouring Conditions and the Adequate Rest Afterwards, Journal of
36.0
36.5
37.0
37.5
38.0
38.5
39.0
39.5
8 9 10 11 12 13 14 15 16 17 18
Before change of a
work area
After change of a work
area
After a sunshade tent
constructed
36.0
36.5
37.0
37.5
38.0
38.5
39.0
39.5
40.0
8 9 10 11 12 13 14 15 16 17 18
I
n
t
e
r
n
a
l

b
o
d
y

t
e
m
p
e
r
a
t
u
r
e

[
]
Time of day [hr]
80% of the amount of
perspiration
perspiration
perspiration
449

Okayama Medical Association, Vol. 78, No. 1, 159-170. (In Japanese)
Kubota H., Asahina T., Matsuo T., Hamada H., and Yamagoshi T. (1999). Prediction of Human Internal Body
Temperature under the Environment of High Temperature, Proceedings of Annual Academic Conference of
Architectural Institute of Japan, 367-368, 1999. (In Japanese)
Onoue T., Yabuki N., Yoshida S., and Fukuda T. (2010). Visualization Technique of Outdoor Thermal
Environment Using a VR Avatar, Proceedings of the 10
th
International Conference on Construction Applications
of Virtual Reality (CONVR), 493-502.
Sakurai O. (2010). Significance of Heatstroke Occurrence Conditions and Countermeasures, Safety and Health,
Vol. 11, No. 5, 433-435. (In Japanese)
450

VIRTUAL REALITY: FACTORS DETERMINING SPATIAL PRESENCE,
COMPREHENSION AND MEMORY
Bimal Balakrishnan, Danielle Oprean, Brad Martin & Melina Smith
University of Missouri, U.S.A.
ABSTRACT: New advances in 3D hardware technology have given a boost to the desktop-based approach to
building stereoscopic virtual reality (VR) labs such as the iCon lab and the IEL at Penn State and the iLab at
University of Missouri. These labs showcase how these new advances can be implemented into an accessible and
affordable solution, but still lack evidence on the impact such technology has on spatial understanding. This can
be attributed to two aspects: 1) the complexity of VR systems given the number of structural features that
collectively comprise a given VR system, and 2) the box-centered approach used by early studies focused on
comparing VR to non-VR systems using various media. Our current research builds on a media effects approach to
architectural visualization (Balakrishnan & Kalisperis, 2009). Here we take a variable-centered approach to VR
technology, identifying its structural aspects (screen size, field of view, stereoscopy etc.) and content parameters
(level of detail, level of realism etc.). The advantage is by using a variable-centered approach proposed by Nass
and Mason (1990), we can systematically vary these parameters and measure the relative impact on spatial
presence and spatial comprehension. Then findings from a particular parameter become generalizable to another
technology that has the same value for the given variable. In our current study, we systematically manipulated field
of view (wide/ narrow), stereoscopy (stereo/ non-stereo) and photorealism (photorealistic / hidden line rendering)
through a controlled experiment (N=80). Results indicate significant main effects for these technology attributes.
The results also reveal trade-offs between parameters, which indicate a unique relationship between spatial
understanding, and each of the variables. These findings are more or less consistent with the Kalisperis et al
(2006) study. Theoretical and practical implications are discussed.
KEYWORDS: Virtual reality, VR system variables, spatial presence, spatial comprehension, stereoscopy, field of
view, photorealism
1. VIRTUAL REALITY AND VISUALIZATION
Virtual reality (VR) in architectural design and visualization is not a new phenomenon (Campbell & Wells, 1994).
The rationale for incorporating virtual reality into architectural design and visualization comes from the ability to
experience simulated spaces without having to physically build them. Many of the early explorations of VR
systems being incorporated as computing tools for architectural design and visualization were impeded by the
technology itself (i.e. slow frame rates, narrow fields of view, etc.). In many cases, the technology was found to
inadequately handle the situation necessary for design to occur (Campbell & Wells, 1994). Advances in
technology have aided in making VR technology more robust and therefore better able to handle information
specific to architectural design and visualization. With the technology improvements over the years, the aspect of
the influences of that technology on the context of architectural design and visualization comes into question.
Though not specifically focused on architectural design and visualization, Wann and Mon-Williams (1996)
explored questions of how VR technology influenced sense of space and spatial perception through specific
linkages like depth cues. This leads into the focus of this study, to identify what role newer more affordable virtual
reality systems play in architectural design and visualization.
As architectural design and visualization center on the communication of designed space, a better understanding of
the influence of VR technology on spatial comprehension and experience measures can provide practical
implications on design computing tool development. For instance, one such influence of VR technology could be
that of immersion and spatial presence. Campbell and Wells (1994) discussed that immersion worked well in terms
of facilitating an understanding of spatial qualities of spaces designed during different stages of the design process.
This indicated that immersion could play a role in the understanding of virtual spaces. Following the definition
provided by Slater (1999), and later built on by Bowman and McMahan (2007), immersion could be thought of as
a direct connection to VR technology, specifically visual immersion. Gaining insights into how the relationship
between visual immersion components of virtual reality technology with spatial comprehension and user
experience measures could help to establish a better idea of how technology portrays depth cues for spatial
perception. This study focused primarily on three areas of visual immersion relating to VR technology:
stereoscopy, field of view and, photorealism.
451

2. A VARIABLE-CENTERED APPROACH TO STUDYING VIRTUAL REALITY
TECHNOLOGY
Balakrishnan and Kalisperis (2009) proposed a media effects approach for research on architectural visualization
and evaluation, building on Nass and Masons (1990) variable centered approach to studying technology. In this
approach, visualization technologies were conceptualized in terms of their structural and content variables and
their impact on cognitive, affective and behavioral responses systematically measured. The variable-centered
approach was useful to identify what attributes of a given technology influence the experience and assess their
relative contributions. Building on Nass and Mason (1990), we broke down VR into its component variables and
the potential values for that variable. Please refer to Balakrishnan and Kalisperis (2009) for a more detailed
introduction to media effects approach to assessing architectural visualization technology, and Kalisperis et al.
(2006) for an illustration of this approach.
Taking a variable-centered approach, our primary research question investigated the influence of visual immersive
technology attributes of a VR system on spatial presence, comprehension and memory. Specifically, the study
investigated the question; how do individual technology attributes influence spatial presence, comprehension and
memory. In addition to the main question, the study also investigated interactive effects between different
technology attributes to identify their relative contributions and identify any potential trade-offs between these
attributes. The technology variables of interest primarily were focused on the visual aspects of a VR system. These
included: stereoscopy which portrays depth through the overlap of dual projected imagery, field of view which
encompasses the area of space the human eye can view at any given time during a simulation, and photorealism
which presents simulated material in a manner considered closer to that of reality. Often times, stereoscopy was
found and considered important to a VR system and therefore including it as one of the technology variables made
sense. Field of view also was a logical choice as all simulations provide a space to view using technology such as
screens or head-mounted display (HMD). Lastly, photorealism was a software aspect that had been considered an
important goal for visualization since its inception. The inclusion of photorealism would provide insight into how
necessary photorealism actually was for visualization.
3. RESEARCH METHODS
3.1 Operationalization of technology variables
Several technology variables were utilized as conditions of the study: stereoscopy, field of view, and
photorealism. Each technology manipulation related to a different virtual reality system aspect. All the
manipulations were limited to a dichotomous (high-low) presentation to better explore each variable.
Stereoscopy was defined as a technique for creating the illusion of depth and 3D imaging while presenting a
different image to each eye (Baos R. M., Botella, Rubi, Quero, Garca-Palacios, & Alcaiz, 2008). Stereoscopy,
a binary dimension, was commonly cited as either being on or off (Schuchardt & Bowman, 2007). Likewise, this
study also treated stereoscopy as either on (enabled) or off (disabled).
Stereoscopy in the literature had given mixed perspectives on its importance towards spatial comprehension and
user experience measures. In many cases, stereoscopy was found significant when grouped with other
technology manipulations. Bennett, Coxon and Mania (2010) found a positive influence of stereoscopy on
memory of objects in a space, but cautioned it was only when the objects were contextually consistent with the
space. IJsselsteijn, de Ridder, Freeman, Avons and Bouwhuis (2001) found a significant positive effect on
presence especially when the displayed information was in motion. Similarly, Hendrix and Barfield (1995)
established a relationship between stereoscopy and presence, determining that added illusion of depth increased a
participants sense of presence. Taking these studies into consideration, stereoscopy was suggested to not
necessarily have a direct main effect on the outcome variables. Instead the interaction of stereoscopy with the
other manipulations was more preferable for finding significance.
Field of view (FOV) had several operational definitions in the literature, often pertaining to different
technologies. As such, this study had chosen to focus on one definition, which combined the aspects of screen
size and geometric field of view (GFOV). Throughout this study, the two manipulations will be referred to as
narrow FOV (defined as a small screen size and a narrow GFOV) and wide FOV (defined as a large screen size
and wide GFOV).
Field of view generally refers to the physical field in which a user can see a simulated space. In other words, the
452

screen or monitor and how much of a users direct and peripheral vision was filled. Humans generally have a
specified field of view of 200 degrees horizontally and 170 degrees vertically (Zikic, 2007). Based on the normal
field of view for an average human, a virtual reality system should try to match those values using technology
through two aspects: 1) adjusting the physical peripheral view of the individual and 2) adjusting the simulated
peripheral of the camera. Due to the two needs of simulating field of view in a virtual environment, FOV consists
of two mechanisms: display (DFOV) and geometric (GFOV) (Banton, Thompson, & Quinlan, 2001). The
differences in the two types lie in what the focus is on. For DFOV, the focus was on the hardware (i.e. the display
screen) enabling a virtual environment to encompass a certain portion of a users field of vision whereas a GFOV
represented the software aspect (i.e. the camera view) which allowed the user to see inside the virtual environment.
Lastly, it was important to note that when field of view was increased both vertically and horizontally; the amount
of screen space enlarges, indicating a relationship with display size.
Hendrix and Barfield (1995) found the GFOV did increase sense of presence as the field of view increased. It
was interesting the authors mentioned both the 50 and 90 manipulation together in comparison with the 10
manipulation. This could have been due to the small variation in findings between the 50 and 90 results. This
provided some insight into using only a wide-narrow manipulation of FOV as opposed to a series of varying
manipulations. Ragan et al (2010) found a wide FOV did significantly improve recall accuracy when dealing
with procedural memorization. The authors did note they were surprised to find their hypothesis about wide FOV
reducing recall time not being significant. This finding suggested that wide FOV could increase accuracy but
possibly not speed of memory.
Lastly, photorealism was also sorted into two manipulations to reduce redundancy in operationalizations:
photorealistic and hidden line. The photorealistic manipulation was defined as the render of the 3D space using
high quality textures, lighting, and shadows. This manipulation was consistent with various studies also focused on
level of realism (Meijer, Geudeke, Egon & Broek, 2009). Hidden line was also the render of the 3D space but
without textures and only solid black lines outlining the contours of the shapes within the space. The hidden line
manipulation was selected as a definite contrast to the photorealistic render. Meijer et al (2009) found photorealism
useful for increasing spatial knowledge within the virtual environment. Slater, Khanna, Mortensen and Yu (2009)
found ray-traced rendered environments, a higher level of realism, produced higher ratings of subjective presence.
Little to no research was found on the influences of photorealism on spatial memory as related to the
operationalizations used in this study.

3.2 Measures for spatial presence, spatial comprehension and memory
Spatial presence was a variable highly linked to user experience and immersion in VR. Several self-report
questionnaires existed for capturing spatial presence in virtual environments. In the case of this study, the
MEC-Spatial Presence Questionnaire by Vorderer et al (2004) was adapted. Two aspects of the MEC-SPQ were
specifically used, self-location and attention allocation, due to a perceived relationship with VR technology.
Self-location was defined as the feeling of being located in a simulated space (Wirth, et al., 2007). From this idea
came the relationship of self-location with embodiment where, embodiment occured in virtual environments due
to sensory engagement on different channels (i.e. vision, hearing, etc.) (Biocca, 1997). Through this connection to
visual aspects of technology, the measure of self-location could be explored as a measure of spatial presence while
manipulating visual immersion factors of a VR system. Similarly, attention allocation related to voluntary and
involuntary attention being given towards some aspect of a virtual environment, usually dealing with spatial cues
provided by some aspect of the virtual environment (Wirth et al, 2007). In addition to the spatial presence
measures, reality judgment was adapted from Banos et al (2000). Reality judgment dealt with how well the VR
technology portrayed the virtual environment to each individual.
Spatial comprehension stemmed from the larger construct of spatial cognition, which spanned several concepts
including spatial knowledge. Comprehension could be rationalized to deal with understanding and in the case of
the concept of spatial comprehension; an understanding of spatial aspects. The measurement of comprehension
through a self-report method focused on accuracy of reporting dimensions of objects in relation to one another
was a common practice in the literature (Zikic, 2007). For this study, the width, depth and height of the simulated
room were used.
We used recall measures to capture the extent to which participants could remember elements from a given
architectural space. A free recall task requested the participants to list all items they were exposed to in the given
scene. They were also administered a cued recall task, aimed at measuring their recognition of elements in the
453

scene. Here they would indicate yes or no against a series of images to identify if those elements were
indeed in the scene.

3.3 Participants and procedure
Participants for this research (N = 85) were recruited from among undergraduate students at the same
mid-western university. The average age of the participants was 20.67 years (S.D. = 1.72) and 64 percent were
females. The participants were randomly assigned to one of the eight experimental conditions. They were first
briefed about the research procedures and their informed consent obtained. Next, the participants completed a
brief questionnaire which captured: demographic information, their computer usage for academic and leisure
activities, their familiarity and expertise with various 2D and 3D software, and 11 items measuring their
immersive tendencies from the Witmer and Singer (1998) Immersive Tendency Questionnaire (e.g. How often
do you become so involved in what you are doing that you lose all track of time?). The stimulus based on the
experimental condition was then presented to the subjects who were allowed to focus on the image for 3 minutes.
The participants then completed the spatial presence instrument. Subsequent to this, the participants estimated
the width, depth and height of the simulated room for spatial comprehension. The spatial memory of the
participants was assessed through both free recall and cued recall tasks. In the free recall task, the subjects listed
the items they had seen in the room. For the cued recall task, the participants identified items recalled in the
room from a set of images of objects. In both recall tasks, the number of correct responses was used as a measure
of their memory.

3.4 Stimulus material
The interior of a residential room was used as the stimulus material for this study. Depending on the experimental
condition, half of the subjects were presented with a photorealistic rendering and the other half with a hidden line
rendering of this room. Please see Figure 1 below for the photorealistic and hidden line renderings. Depending on
the experimental condition, the images were presented either in stereo 3D or non-stereo and either in a wide field
of view or a narrow field of view. The subjects were seated approximately six feet away from the screen and in the
wide field of view, the image extended to their peripheral vision whereas in the narrow condition, it did not.

Fig. 1: Operationalization of high (top) and low (bottom) photorealistic condition.

3.5 Index construction and preparation for data analysis
Immersive tendencies of the participants were measured using an 11-item scale. An immersive tendency index
was constructed by averaging the items and showed good reliability (Chronbachs = 0.78). Principal
component analysis was conducted to analyze the dimensionality of the 19 items measuring different dimensions
454

of spatial presence. Using the scree plot, three factors accounting for 65.79% of the variance were identified. On
rotation using the Varimax procedure, 15 of the 19 items loaded clearly on to the three factors with their highest
loading exceeding 0.6 and the other two loadings less than 0.4. Four items that cross loaded on multiple factors
were excluded from further analysis. The rotated solution captured three factors and three corresponding indices:
spatial presence (Chronbachs = 0.94), reality judgment (Chronbachs = 0.91) and attention (Chronbachs =
0.81) were created by averaging the respective items. Before proceeding with statistical analysis exploring the
primary research questions, all assumptions, including normality, of the data were verified and confirmed.
4. RESULTS
Analysis of variance (ANOVA) was carried out to assess the impact of the three VR system variables on the
dependent variables of interest in this study. Data pertaining to a number of potential confounding variables
including gender, age, academic standing, computer use and experience with 3D modeling tools and gaming
were collected among others and used where appropriate as covariates to improve accuracy. The following
sub-sections describe the findings with respect to the three dependent variables of interest.
4.1 Spatial Presence
The factorial analysis of variance conducted on presence indicated a significant main effect for photorealism,
F(1,76) = 4.75, p < 0.05. Participants who were presented with the photorealistic image reported a greater sense
of presence (M = 3.17) than those who were presented with the hidden line rendering (M = 2.51). The ANOVA
also indicated a significant main effect for field of view, F(1,76) = 18.28, p <0.001. Participants in the wider field
of view reported a greater sense of presence (M = 3.75) than subjects in the narrow field of view (M = 2.12).
Results did not indicate a significant main effect for stereoscopy on presence, F(1,76) = 2.12, p = 0.15. Also, the
results did not show any two-way or three-way interactions.
4.2 Spatial Dimensions
Participants were asked to estimate the width, depth and height of the simulated living room space. The width,
depth and height responses were converted to accuracy scores by dividing with the corresponding correct response.
A score of 1.0 indicated accurate perception of that dimension. Scores above 1.0 indicated the participant
over-estimated the distance and scores below 1.0 indicated under-estimation. When estimating the width, results
showed a significant main effect for photorealism, F(1,76) = 5.91, p < 0.05. The results were counter-intuitive.
Participants in both photorealistic and hidden line conditions under-estimated the width of the room. However,
those in the hidden line condition were more accurate in estimating the width of the room (M = 0.80, with 1.0 being
100% accurate) when compared to those in the photorealistic condition (M = 0.63). When estimating the depth,
results indicated there were no significant differences between the photorealistic and the hidden line conditions,
F(1,76) = 0.04, p = 0.85. Similarly, there were no significant differences between the different levels for field of
view, F(1,76) = 1.70, p = 0.20 or stereoscopy, F(1,76) = 0.23, p = 0.63. While estimating height, there were no
significant main effects, but there were significant interactions between photorealism and field of view, F(1,76) =
8.81, p < 0.01, as well as between photorealism and stereoscopy, F(1,76) = 6.05, p < 0.05. For narrow field of view,
participants estimated the height more accurately in the photorealistic condition (M = 1.16) compared to the
hidden line condition (M = 1.56). For the wider field of view, the opposite held true. Participants estimated the
height more accurately in the hidden line condition (M = 1.13) compared to the photorealistic condition (M = 1.39).
Please see Figure 2 below for the interactive effect between realism and field of view on height estimation.

Fig. 2: Interactive effect between realism and field of view on height estimation.
455

For stereoscopic display, participants in the photorealistic condition were more accurate in estimating the height
(M = 1.19) when compared to those in the hidden line condition (M = 1.53), who tended to over-estimate much
more. For non-stereoscopic display, the opposite results held true. In that case, participants in the hidden line
condition tended to be more accurate in estimating height (M = 1.16) when compared to those in photorealistic
condition (M = 1.37), who tended to over-estimate. Please see Figure 3 below for the interactive effect between
photorealism and stereoscopy on height estimation.
Fig. 3: Interactive effect between realism and stereoscopy on height estimation.
4.3 Memory
For the free recall variable, the three way interaction between the independent variables approached significance,
F(1,76) = 2.86, p < 0.1 indicating the collective influence of those variables. Results also indicated significant main
effects for all three independent variables. For free recall scores, there was a significant main effect for
photorealism, F(1,76) = 7.35, p < 0.01, field of view, F(1,76) = 18.55, p < 0.0001, and stereoscopy, F(1,76) = 4.24,
p < 0.05. Participants in the photorealistic condition recalled more objects from the room (M = 13.70) than those in
the hidden line condition (M = 11.62). Similarly, participants in the wide field of view condition recalled more
objects (M = 14.84) than subjects in the narrow field of view condition (M = 11). The result for stereoscopy was
counter intuitive. Participants in the non-stereo condition recalled more objects (M = 13.93) than in the stereo
condition (M = 11.70).
These main effects should be interpreted in light of significant interactions between photorealism and field of view,
F(1,76) = 4.05, p < 0.05. For narrow field of view, there was little difference between those in photorealistic
condition (M = 11.18) and those in the hidden line condition (M = 10.54). For wide field of view, participants in
photorealistic condition recalled more objects (M = 17.00) than those in the hidden line condition (M = 12.65).
Please see Figure 4 below for the interaction between photorealism and field of view on free recall.
Fig. 4: Interactive effect between photorealism and field of view on free recall.
456

For cued recall, the results were similar to those for free recall. A significant three-way interaction between
independent variables, F(1,76) = 4.97, p < 0.05, indicated the combined effect of the three independent variables
on cued recall. There was a significant main effect for photorealism, F(1, 76) = 14.82, p < 0.001. Participants in the
photorealistic condition recalled more objects (M = 24.60) from the image list than those in the hidden line
condition (M = 22.14). There was a significant main effect for field of view, F(1,76) = 33.94, p < 0.0001 and
participants in the wide field of view condition recalled more objects (M = 25.87) compared to those in the (M =
21.47). The results also showed a significant main effect for stereoscopy, F(1,76) = 6.88, p < 0.05, similar to free
recall. Participants in the non-stereo condition recalled more objects (M = 24.63) than those in the stereo condition
(M = 22.44). The above main effects should be interpreted in light of the significant two-way interaction between
photorealism and field of view, F(1,76) = 12.88, p < 0.001. For narrow field of view, there was little difference in
cued recall scores between photorealistic (M = 21.50) and hidden line (M = 21.3) conditions. However, for wide
field of view, participants in the photorealistic condition recalled more objects (M = 28.7) than those in the hidden
line (M = 23.01) condition. Please see Figure 5 below for the interaction effect between photorealism and field of
view on cued recall.
Fig.5: Interactive effect between photorealism and field of view on cued recall.
5. IMPLICATIONS
The results for the three dependent variables provided insight into the influences of each technology variable on
individual perceptions. Spatial presence proved greater when both photorealism was high and a wider field of view
was utilized. For spatial dimensions, few of the estimations proved any significance with the exception of height
where photorealism versus hidden line did distinguish the degree of accuracy. Interactions with field of view and
stereoscopy also provided influences on height estimation. Lastly, spatial memory measured using free and cued
recall tasks showed a three-way interaction between the independent variables. In particular, the interaction
between field of view and photorealism showed significant differences in spatial memory for wide field of view
with photorealistic imagery.
Results for spatial presence were consistent with the previous results from Kalisperis et al. (2006). Presence is an
important aspect of simulating spatial experience and one can conceptualize virtual reality systems as presence
enabling technologies. The findings for presence validate current emphasis placed by CAD software developers
on photorealistic real-time renderings relying on improved hardware capabilities of the video cards. Earlier,
given limited capabilities of graphics cards, improved photorealism often had to be at the expense of interactivity.
With ever improving capabilities of graphics cards, highly photorealistic, yet interactive VR environments are
possible using desktop based VR systems.
Results for field of view were consistent with results reported by Zikic (2007). Most desktop computers have a
narrow field of view and the results from this study indicated the importance of wide field of view in enhancing
presence. Designers of virtual reality systems for training and simulation where an increased sense of presence is
important must take this into consideration. In cases where wider fields of view are important; head-mounted
displays (HMDs) have an advantage over large screen displays in terms of physical space limitations as well as
wider degrees of viewing available.
457

Looking at the results of spatial dimension estimation, the findings for height estimation, particularly the
interactive effects involving photorealism provided some interesting clues into the impact of these system
variables on spatial comprehension. When the FOV was narrow, (i.e. when the scale was small), realism improved
accuracy, perhaps by providing more spatial cues through texture and detail. However, when the FOV was large,
there seemed to be an additive effect of bottom up cues resulting in over-estimation. This suggests the need for
elements in VR environments like avatars that will help participants relate to spatial dimensions more easily than
relying on more difficult bottom-up cues like that of texture and detail. The interactive effect between stereoscopy
and realism indicated that realism was helpful in improving accuracy for stereoscopy; as it resulted in
over-estimation of height for the non-stereo condition. This indicated the importance of perspectival depth for
accurate height perception. Findings for dimension estimations needs further analysis and follow up research
before one can meaningfully discuss implications in depth. We hope to achieve this in our follow up research.
In the case of spatial memory, the results, particularly those of the interactive effects, indicated the importance of
photorealism in improving memory. While a wider field of view improved memory for objects, photorealism was
found to significantly improve memory. Higher realism may have helped participants to easily distinguish between
various objects and read them as independent items when compared to the hidden line rendering. This has
implications for development of VR environments for simulation and training, particularly those that involve
visual search tasks and memorization.
6. LIMITATIONS AND DIRECTIONS FOR FUTURE RESEARCH
The majority of the participants in the study was drawn from design disciplines and as such limits the
generalizability of the findings beyond that of the design professions. Also, selection of all undergraduate
students from the same university was used with the understanding of the issue of external validity in mind.
Findings for stereoscopy maybe have been limited by the use of a still image for the stimulus. Since architectural
space is best experienced through exploration, we are planning follow up studies which allow participants to
interactively explore spaces. We hypothesize this will also enhance self-location and possibilities for action, two
dimensions of spatial presence (Wirth et al., 2007).
Since many of the findings from Kalisperis et al. (2006) were replicated in this study, independent variables for
future research can be operationalized at more than two levels. This would confirm if some of the main effects
detected in this research, are indeed linear in nature. This study detected some counter intuitive results,
particularly in the case of stereoscopy for recall measures. Future research should further investigate this issue by
taking into careful consideration potential explanatory mechanisms and the research design should strive to rule
out alternative explanations. Another limitation of this study was the conceptualization and operationalization of
photorealism in this study. While the Kalisperis et al. (2006) study looked at realism and level of detail as two
separate variables, in this study we focused primarily on realism. It may be more beneficial to operationalize
these concepts as the number and type of cues provided. This will allow us to explain and the results building on
the vast body of relevant literature related to visual cognition in psychology. Our current research projects are
exploring interactive effects of navigational affordances of virtual reality system variables for spatial presence
and spatial comprehension.
7. REFERENCES
Balakrishnan, B., and Kalisperis, L.N. (2009). Design visualization: A media effects approach, International
Journal of Architectural Computing, Vol. 7, No. 3, 415-427.
Baos, R. M., Botella, C., Garca-Palacios, A., Villa, H., Perpina, C., & Alcaiz, M. (2000). Presence and reality
judgment in virtual environments: A unitary construct?, CyberPsychology & Behavior, Vol. 3, No. 3, 327-335.
Baos, R. M., Botella, C., Rubi, I., Quero, S., Garca-Palacios, A., & Alcaiz, M. (2008). Presence and emotions
in virtual environments: The influence of stereoscopy, CyberPsychology & Behavior, Vol. 11, No. 1, 1-8.
Banton, P., Thompson, P., & Quinlan, P. T. (2001). Effect of geometric field of view on stereoscopic spatial
judgments, Human Factors: The Journal of the Human Factors and Ergonomics Society, Vol. 43, No. 1, 405-414.
Bennett, A., Coxon, M., & Mania, K. (2010). The effect of stereo and context on memory and awareness states in
immersive virtual environments, APGV '10 Proceedings of the 7th Symposium on Applied Perception in Graphics
and Visualization, New York, NY: ACM, 135-140.
458

Biocca, F. (1997). The cyborg's dilemma: Embodiment in virtual environments. Proceedings Second International
Conference on Cognitive Technology Humanizing the Information Age. IEEE, 12-26.
Bowman, D. A., & McMahan, R. P. (2007). Virtual reality: How much immersion is enough?, Computer, Vol. 40,
No. 7, 36-43.
Campbell, D. A., & Wells, M. (1994). A critique of virtual reality in the architectural design process. Report No.
4-94-3: Human Interface Technology Laboratory (HITL).
Hendrix, C., & Barfield, W. (1995). Presence in virtual environments as a function of visual and auditory cues,
Proceedings of the Virtual Reality Annual International Symposium (VRAIS '95), IEEE, 74-82.
IJsselsteijn, W., de Ridder, H., Freeman, J., Avons, S. E., & Bouwhuis, D. (2001). Effects of stereoscopic
presentation, image motion, and screen size on subjective and objective corroborative measures of presence,
Presence, Vol. 10, No. 3, 298 311.
Kalisperis, L.N., Muramoto, K., Balakrishnan, B., Nikolic, D. and Zikic, N. (2006). Evaluating Relative Impact of
Virtual Reality System Variables on Architectural Design Comprehension, Communicating Space(s) 06 eCAADe
(Bourdakis,V. and Charitos, D., eds.), 66-73.
Meijer, F., Geudeke, B. L., & van den Broek, E. L. (2009). Navigating through virtual environments: Visual
realism improves spatial cognition. Cyberpsychology & Behavior, Vol. 12, No. 5, 517-521.
Nass, C. and Mason, L. (1990). On the Study of Technology and Task: A Variable-Based Approach, Organizations
and Communication Technology (Fulk, J., editor), Charles Steinfield, Newbury Park, CA, 46-67.
Ragan, E. D., Sowndararajan, A., Kopper, R., & Bowman, D. A. (2010). The effects of higher levels of immersion
on procedure memorization performance and implications for educational virtual environments, Presence:
Teleoperators & Virtual Environments, Vol. 19, No. 6, 527-543.
Schuchardt, P., & Bowman, D. A. (2007). The benefits of immersion for spatial understanding of complex
underground cave systems, Proceedings from VRST 2007, Newport Beach, CA: ACM, 121-124.
Slater, M. (1999). Questionnaire, measuring presence: A response to the witmer and singer presence, Presence, Vol.
8, No. 5, 560-565.
Slater, M., Khanna, P., Mortensen, J., Insu, Y. (2009). Visual realism enhances realistic response in an immersive
virtual environment. Computer Graphics and Applications, IEEE, Vol. 29, No. 3, 76-84.
Vorderer, P., Wirth, W., Gouveia, F. R., Biocca, F., Saari, T., Jncke, F., et al. (2004). MEC spatial presence
questionnaire (MEC-SPQ): Short documentation and instructions for application. Report to the European
community, Project Presence: MEC (No. IST-2001-37661).
Wann, J., & Mon-Williams, M. (1996). What does virtual reality NEED? : Human factors issues in the design of
three-dimensional computer environments, Internation Journal of Human-Computer Studies, Vol. 44, 829 847.
Wirth, W., Hartmann, T., Bcking, S., Vorderer, P., Klimmt, C., Schramm, H., et al. (2007). A process model of
the formation of spatial presence experiences. Journal of Media Psychology, Vol. 9, 493525.
Witmer, B. G., & Singer, M. J. (1998). Measuring presence in virtual environments: A presence questionnaire,
Presence: Teleoperators and Virtual Environments, Vol. 7, 225-240.
Zikic, N. (2007). Evaluating relative impact of VR components screen size, stereoscopy and field of view on spatial
comprehension and presence in architecture, Technical Report #53, Penn State, Computer Integrated Construction
Research Program.

459

DIGITAL ECOLOGIES: A SHIFTING PARADIGM FOR
CONSTRUCTION
Robert Amor
University of Auckland
ABSTRACT: Many industries are currently being transformed, through phenomenon driven by what is being
described as cloud computing or the emergence of new digital ecologies. Here we explore this concepts
potential within the construction industry through the creation of a Revit Add-on created with its proprietary API
and an iPad application, which facilitates a direct and immediate exchange of information between people
gathering data at the point of work (POW) on site and the central Revit BIM. The paper reflects on some known
obstacles to this approach, particularly the latency and inaccuracies that exist between the model and the actual
building as well as complexities in transferring the model from a desktop computer to low powered, usable and site
friendly devices. We focus on a test case in defect management and expose emergent possibilities for improvement
through techniques that are driving digital ecology; we appropriate the Revit API, deposit data on a cloud service
and develop a single serving iPad application. We have hypothesised elsewhere, that construction operates
through negotiating tensions, frictions and potentials created within its inherent complexity of people, materials
and disciplines. In this paper we go further and suggest that construction exhibits the behaviour of an ecology.
There is a chequered history of the construction industry gaining advantage from appropriating organisational
methods from other disciplines. However, having suggested construction exhibits the behaviour of an ecology, our
test case points to opportunities where adopting the digital ecology, making easy to use mobile applications and
leveraging localised delivery of site data could potentially improve existing construction and facility management
processes.
KEYWORDS: Revit, API, Digital Ecology, Locative Media, iPad, Application
1. INTRODUCTION
We are becoming evermore attuned to the notion of digital ecologies or the Internet of things. Where a multitude of
objects are interconnected and communicating, much like a biological ecology. This communication is, or appears
close to, real-time and in popular computing it is changing the dynamics of many industries and driving new
innovations; the most interesting and influential of which are unexpected. The popular photo-hosting site Flickr
was an unexpected success originally only a part of Game Neverending a massive multiplayer online game
(MMOG). The mobile application Waze, which appropriates crowd-sourced information about location and traffic
then redeploys it through map and location aware services to report real-time traffic conditions, is finding popular
usage specifically to subvert police traps and speed cameras. We can observewith some regularitythrough
innovations such as Dropbox, Foursquare and Yelp, when ecologies of data sources and services are made
available, new and unexpected innovations emerge that reshape industries and activities.
The New York Metropolitan Transport Authority (MTA) has capitalised on this phenomenon (Press 2010) by
making their real-time transportation data available to developers. Within hours mobile phone applications and
websites emerged appropriating that data and combining it with other information and services, such as Google
Maps, to provide a wide range of utility and functionality. Traditionally such a company might use customer
surveys to elicit functionality for new services, then take the most popular ones and undertake an extensive
software development process to realise them. However, making the data available has resulted in considerably
more niche software and applications tailored to specific needs and much more easy to use. A caf developed a live
notice board that told people exactly how far away their bus was from the bus stop directly outside. Increasing the
convenience and likelihood of individuals using this caf, as well as increasing usage of the MTA services.
Hacking and mashups are increasingly important in contemporary innovation, the authors have previously
interrogated this notion within design and construction (McMeel et al. 2011a; McMeel 2010). In this paper we
advance the discussion asking if benefits from open and available data and services might be possible within
construction? We begin to unpick this question by asking how might we implement a digital ecology to test this
supposition? Then ask where might it be implemented?
In terms of how, the introduction of new technology has a long and tumultuous history in construction, with high
failure rates (Peansupap et al. 2005, 193). This failure is not unique to construction, Kling has conducted many
studies on a variety of organisations uncovering motivations and results of technological introduction (Kling 1980;
460

Kling 1996; Kling et al. 1984). He identifies a recurring problem where change is required in one area of an
organisation to benefit another area; the operation being disrupted by change does not gain advantage or benefits
from it. Another issue reveal by Peansupap et al is technologys failure to meet expectations. This is also not
unique to construction, the digital designer and innovator Jeffery Veen has identified this as a cause of poor uptake
of technology in other industries (Veen 2004). Both Kling and Veen point to a lack of consideration for nuances
within existing processes; an affected process should be easier, or at the very least, not more difficult. Within this
context the authors made a number of presentationsof mobile technologiesto architects, stimulating
discussion. This has pointed to frustration with the snagging process, which we will discuss in greater detail in
section 2. The question of how is thus driven by a problem sedimented within the context of design and
construction, not by available or novel technology. According to Kling and Veen attending closely to this defined
group and the targeted snagging process should provide the necessary guidance when faced with the question
how?
In terms of where, snagging is perhaps a suitable place for intervention. Throughout construction there is a
substantial difference between the project documentationbe that printed schematics or digital building
information models (BIM)and the site condition. This difference reduces as the project continues although some
might argue the documentation never actually catches up with the site until the preparation of the as built
documentation. That notwithstanding, during snagging this difference is at an absolute minimum and there is a
reasonable similarity between the documentation, model and the physical building. At the very least this minimises
disparity, latency or inconsistency that has been identified as limiting practice-led research on actual construction
documents and materials (Plume et al. 2005).
2. CONSTRUCTION
Snagging processes can be frustrating, but are a key stage in construction. They occurs just prior to building hand
over and is the process where any minor defects in the construction, such as a cracked ceiling tile or damaged paint
or plaster, are identified and remedied so a completed building can be handed over to the client. Predominantly a
paper-based process where defects are identified by the architect or project manger on-site; they are usually
reformatted and logged off-site before being passed to the contractor for remedial action. As groups of snags are
rectified they will be inspected by the architect or manager and logged as complete.
On commercial projects defect lists can extend to hundreds of individual items distributed throughout an entire
building. A particular point-in-case is current processes being executed in the aftermath of the Christchurch
earthquake. The defect logging processes is considerable, our informal discussions with architects suggest for
every hour spent on-site recording building defects, they spend at least another hour rewriting and reformatting the
documentation for the numerous stakeholders, such as engineers and insurance companies. In both these examples,
to some extent, what is considered suboptimal is the rewriting and reformatting of a singular source of information,
as well as problems and delays that arise from documenting and finding the exact location of defects.
It is within this context that we conceived of an application for managing defects with superior location awareness
and cloud or online-based storage to facilitate a more efficient repurposing of data. Informed by the Metropolitan
Transport Authority example discussed in the introduction we focused on a strategy that would present data in an
open and available format and location, as well as the development of an application for recording snags. The
application should facilitate note taking and reading/writing data to an online source. The note should have
location information to help with locating the snag precisely within a building. From our discussions with several
architectural firms we arrived at a general briefing document that included a 3D interface on a tablet device where
snags could be recorded at exact locations. This application should also assist with the repurposing and rewriting
of repetitive information, making the processes more efficient. This focused the development on three areas:
1. Strategy for moving a BIM model to an iPad.
2. Online storage, visualisation and manipulation in a resource constrained device.
3. Making information on the iPad available for appropriation by a nascent digital ecology of systems.
Cheshire Architects made available to us the Revit model of their Q Theatre building in Auckland city centre. The
model provided us with an archetype of realistic detail, quality and information, as it exists within a real
construction project. This is also a fairly typical size of commercial construction project, a three story existing
building converted to provide three main performance spaces; the main auditorium can hold a maximum audience
of 450 people. The construction cost was approximately NZ$21 million (12.8 million euro), however most
461

importantly this was not a demonstration exemplar for the purpose of research, it was an actual industry created
BIM.
3. CONVERSION CONTROL
The desire to have a 3D interface with some interaction capability to locate snags in 3D space directed us to
investigate games engines. Some recent work using Unity3D (Boeykens 2011) revealed the interactive
possibilities as well as some strategies for moving from a BIM model to a mobile device. It also has an integrated
workflow for publishing to iOS devices like the iPhone and iPad, which made it an attractive option.

Figure 1. Conversion pathways explored by Jae Shin
Advancing Boeykens work we explored the conversion potential. What was revealed through conversion was the
propencity of certain pathways illustrated in Figure 1 to drop location information. Retaining this information is of
critical importance within our context, the highlighted pathway of Revit to 3DS Max via dwg format and from
3DS Max to Unity3D via fbx formatalthough suboptimal for the translation of textureswas highly reliable for
transfering object location and coordinate information between the BIM environment of Revit and the Unity3D
environment.
Figure 2. iPad application screenshot showing highlighted snags and the interface for adding information.
The size of the model was another known issue, where the processing power available and required for BIM on
desktop computers is not availabe on mobile devices; causing slow performance and rendering any innovation
inneffectual. With Unity3D there is a standard game methodology of incremental loading, whereby as a viewers
proximity to an object increases, the object will load into the environment; as the viewer moves further away from
an object it is purged from the system. This ensures a smaller number of polygons are being handled than if the
entire model was loaded. We experiemented with the useful DPR model slicer (http://modelslicer.dpr.com/), a
Revit add-on for slicing a model into specific sized segments. These were then exported through the identified
workflow in Figure 1, thus retaining their relative position to each other within Unity3D. Although this worked
462

very well, through experimentation we found with a second generation iPad it was possible to load entire floors of
the Q theatre building and still retain smooth movement (Figure 2)
The main limitation we discovered through this processes was an unsatisfactory translation of the material quality
of the building, which can be seen in Figure 2. Although Boeykens claims Unity3D has better material possibilites
than alternatives like the Unreal Editor, it remained problematic to get both textures and acurate location
information translated from Revit into Unity3D.
4. ONLINE STORAGE
The availablty of information for other applications within our nascent digital ecology is important. Web server
access is no longer complex or costly, the interested amateur can obtain it through free sources on the internet, for
the professional Amazons highly reliable web services are availble. Even Autodesk has its own proprietry cloud
offering for Autodesk users in the form of Autodesk 360 (https://360.autodesk.com/Landing/Index). The very
popular Dropbox (https://www.dropbox.com/home) has its own proprietry API and enables powerful sharing
possibilities between users and application. With these services the potential to create a distributed network and
filing system for projects using public and reliable services is not unrealistic. Thus the possibilityas seen in the
MTA example in our introductionto create a framework for open and available data to stimulate developer
innovation is a highly realistic proposition.
We created a free web server account at Free Web Hosting (http://www.000webhost.com/) and established a
simple POST and GET protocol using the php language for exchanging information between the web server and
the iPad application. Initially this accessed data from a comma separated variable (CSV) file to prove proof of
concept. The iPad application was handling information quite efficiently, firstly an XYZ location coordinate and
the name of the object as defined by Revit was passed to Unity3D, then the annotation, author information, time etc.
Essentially just standard information that is recorded as part of existing snagging processes. Using the Revit object
identifier to coordinate the location information between Revit and Unity3D our framework was freed from the
need to continually transfer 3D model data. Thus once the model was encoded on the iPad, using standard web
protocols, providing there was reliable WiFi or 3G connectivity, we could send and receive data with considerable
speed and reliability.
Returning to our earlier analysis we are careful to consider where change is being implemented. Being a standard
format the CSV file can be opened in Microsoft Excel, where we would begin formatting the information.
Continuing research is developing a MySQL database to replace the CSV file as the underlying data resource for
the application. Supporting this digital ecology with a database provides the potential for users to manage snags
centrally and export them as standard formats (.xls, .doc, .rtf) where they can be passed onto contractors, thus not
imposing change on other processes or people. Moreover there are many sophisticated issue tracking systems
developed for IT support (http://bestpractical.com/rt/) that could potentially be tailored to meet very sophisticated
managerial needs within this context. For now we continue to focus on the management of snags, particularly to
investigage if we could facilitate a two way data exchange between the mobile model and the BIM. Existing
mobile BIM offering, such as Graphisofts BIMx, have a unidirectional flow. From the BIM software onto a mobile
device. If we are to buy into the digital ecology we want to have a multidirectional exchange, the possibility of data
flowing and being exchanged within the ecology on multiple levels. Revit comes bundled with a powerful API,
and in the next section we will discuss our investigation into creating a Revit add-on to pull the
infromationcreated on the iPad and stored on our web serverback into Revit.
5. API
As the name suggests, building information models (BIMs) are no longer just geometrical representations they are
information rich descriptions of many aspects of a building, from geometry to engineering schematics and object
specifications. Following the earthquake in Christchurch there is an increased onus being placed on documenting
construction process particularly where this involves deviations from specification. We speculate the proposed
digital ecology concept provides a frameworkif desirableto draw from a manifold of processes, procedures
and decisions, which could be encoded in the BIM description.
The key functionalities we require of the API here are reading a CSV file, drawing the location information from
set fields in the file and creating markers related to each defect at its respective location within the BIM model.
There are some tutorials for working with the API provided by Autodesk
463

(http://usa.autodesk.com/adsk/servlet/index?siteID=123112&id=2484975), as well as a comprehensive
developers guide and software development kit (SDK). Add-ons can be written in both the Visual Basic (VB) and
C# programming languages.
Figure 3. Visual C# Studio environment with some of our API add-on code.
Early development work (Figure 3) shows the basic functionality, reading a CSV file and adding markers in the
Revit environment at points contained within the CSV file (https://vimeo.com/36167862). By using a CSV file we
could test suppositions regarding information interchange by accessing the data with traditional spreadsheet
software such as Microsoft Excel or OpenOffice. With our current work the CSV file will give way to a MySQL
database as the information repository, which will in turn generate the necessary file types for additional
applications within the digital ecology. The scale, rotation and position of the snags were proportional however
they did not match the master BIM model. We were able to execute manual translations on the BIM file so the
snags matched the building. This was of course not optimal, an automatic translation and one-off configuration
procedure would be desirable in this type of context. Nevertheless the final process of automatically creating
markers in Revit at the same location as snags were recorded in the iPad application (Figure 4) can be seen online
at https://vimeo.com/40420799.
Figure 4. The Revit model before adding snag markers (left) and after (right).
It is reasonable to ask how might such an ecology grow? Subsequent to this project we have had discussions with
pipe manufacturers that are looking for ways to provide total lifecycle management of their assets; from recording
data of pipe manufacture, testing, installation to ongoing maintenance until replacement. Managing defects and
maintenance within facility management is not unlike snagging, only more difficult as buildings are fully
operational, furnished and occupied. These examples suffer from data input at a variety of times, locations and
mediums, which results in fragmented data repositories that complicates timely and meaningful access. Already in
popular culture we are seeing the notion of the digital ecology reduce this type of problem; predicaments caused by
temporal and spatial disparity within data. Might it reveal a means to identify successful and unsuccessful
464

combinations of consultants, contractors and sub-contractors? Would having access to lifecycle information on a
building reduce the unknown factors for contractors or sub-contactors making tenders? Sophisticated project
management software creates dynamic Gantt charts where elements are contingent on each other, how far are we
away from an iPhone variant that tracks them and informs key stakeholders of milestones that are approaching or
milestones that have shifted because of decisions being made elsewhere? We have discussed elsewhere the
importance of physical meeting within the dynamics of construction processes (McMeel et al. 2011b), any
suggestion it can be automated or purged from construction is overly simplistic. Thus this research is not to replace
humans with some Marxist dystopian, rather it is to investigate where and how digital technology might be
leveraged to augment natural and already effective behaviours.
6. SUMMARY
Given the closeness of the model to the site condition at the snagging stage of construction, it is perhaps more
useful to have a location and spatial interface at this stage than any other. The Unity3D game engine provides
powerful loading methodologies that render the problems associated with large files on mobile devices almost
completely void. Although the translation of texturing and materials requires further investigation, by establishing
a relationship between the location information within Unity3D and the Revit BIM model we were able to
exchange modestly sized data packets containing coordinate information rather than large proprietary 3D object
files leaving the data more accessible for other applications.
Perhaps most importantly it points to the potential of open information and standard protocols for communication;
the same trends that can be observed in consumer computing. This one test case does not provide immediate
solutions for innovating with mobile technology but it demonstrates how quickly it can be conceived, developed
and tested within a digital ecology using services already available to the consumer. It is a demonstration of the
possibilities of current innovations for design and construction and an exhortation to consider software and
services as part of the complex ecology of construction and enable freedom for them to be meaningfully
appropriated by developers and innovators beyond their immediate corporate family.
7. ACKNOWLEDGMENTS
The authors would like to thank Remy Lim who created the iPad application and Tommy Shin who worked on
import/export pathways for the Revit model; Cheshire Architects for the use of their Q Theatre BIM model. We
would also like to thank Rowan Walker who did some early development work on the potential of mobile snagging
applications for construction processes.
8. REFERENCES
Boeykens, Stefan (2011). Using 3D Design Software, BIM and Game Engines for Architectural Historical
Reconstruction, in Pierre Leclercq, Ann Heylighen & Genevive Martin (Eds.) CAAD Futures: Designing
Together. Belgium, Les ditions de l'Universit de Liege.
Kling, Rob. 1980. "Social Analyses of Computing: Theoretical Perspectives in Recent Empirical Research,"
Computing Surveys, 12.
Kling, Rob. 1996. Computerisation and Controversy: Value Conflicts and Social Choices, San Diego: Academic
Press.
Kling, Rob & S Iacono. 1984. "The Control of Information Systems Developments After Implementation,"
Communications of the ACM, 27: 1218-1266.
McMeel, Dermott. 2010. "System mash-ups: emergent crafts and rule-based approaches to design and
construction." Series. Accessed 03/08, 2010. doi:
http://www.nomads.usp.br/virus/virus03/submitted/layout.php?item=3&lang=en
McMeel, Dermott & Robert Amor (2011a). Construction AIDs: Augmented Information Delivery, in
Hans-Joachim Bargstdt & Karin Ailland (Eds.) Proceedings of ConVR the 11th International Conference on
Construction Applications for Virtual Reality. Weimar, Germany.
McMeel, Dermott & Robert Amor. 2011b. "I think therefore i-phone." CAAD Futures 2011: Designing together
465

edited by Pierre Leclercq, Ann Heylighen & Genevive Martin, Lige, Beligum: Les Editions de l'Universit de
Lige.
Peansupap, Dr. Vachara & Professor Derek H. T. Walker. 2005. "Factors Enabling Information and
Communication Technology Diffusion and Actual Implementation in Construction Organisations," ITcon, 10:
193-218.
Plume, J. & J. Mitchell. 2005. "A Multi-Disciplinary Design Studio using a Shared IFC Building Model."
Proceedings of 11th international CAADFutures conference edited by Bob Martens & Andr Brown, 445-464.
Vienna: Springer.
Press, Elizabeth. 2010. "A Case for Open Data in Transit." Series Street Films. Accessed 15/05, 2012. doi:
http://www.streetfilms.org/a-case-for-open-data-in-transit/
Veen, Jeffrey. 2004. "Why Content Management Fails." Series Adaptive Path. Accessed 01.04.04, doi:
http://www.adaptivepath.com/publications/essays/archives/000315.php

466

SERIOUS PLAY: INTUITIVE ARCHITECTURAL CONCEPTUAL DE-
SIGN WITH IMMEDIATE STRUCTURAL FEEDBACK AND ECO-
NOMICAL AND ECOLOGICAL PERFORMANCE PREDICTIONS
Manfred Breit
Institute for 4D Technologies, University of Applied Sciences Northwestern, Switzerland
Li Huang
Chair for Metal Structures, Technische Universitt Mnchen, Germany
Frank Lang
Architecture, Entwerfen und Baugestaltung, Technische Universitt Darmstadt, Germany
Fabian Ritter & Andr Borrmann
Computational Modeling and Simulation Group, Technische Universitt Mnchen, Germany
ABSTRACT: "Serious Play" (Schraege, M. 1999) is about experimenting with computer models, prototypes and
simulations that may lead to innovations. "Play" is related to create alternative design models in a playful way.
While exploring, comparing and evaluating the design options, computerized performance predictions can help to
"seriously" select and further develop design candidates towards better qualities and performances. We report
about an ongoing research project regarding sustainable office design for composite steel-concrete structures. A
team at Technische Universitt Mnchen works on IT-based design aids for the conceptual design phase. We
observed that architects traditionally use building blocks for conceptual design of office facilities. Their design
also includes a playful process to create, modify and compare competing options. 2D Sketching on paper is their
primary choice for creating and communicating design ideas. Later physical models are used to check and verify
the design. We want to lever this playful sketching-based approach from the paper drawings and physical models
to computer supported volumetric modeling with an implementation in SketchUp that support Serious Play.
Based on initial 2D sketching of foot prints, the architect can create volumetric building blocks, which follow a set
of rules implemented in the software from building typology knowledge. While creating and modifying the volu-
metric architectural model a functional structural design development model is generated in parallel, which
serves as input for an automatic structure generator. It will provide instant feedback to architectural design
variants with ecologically optimized steel composite systems for the building blocks. The system uses an abstract
structure representation that informs about necessary spaces for structural systems and elements. Key perfor-
mance indicators will be displayed to support performance oriented design. A first prototype is ready for test runs
of Serious play for conceptual architectural office building design.
KEYWORDS: conceptual design, structural feedback, performance predictions
1. INTRODUCTION
Architectural design procedures need to address environmental sustainability issues early in the design processes.
The decisions taken in the conceptual phases have a great impact on how the building will perform in respect to the
stakeholder needs. Performance oriented conceptual architectural design however needs an engineering, con-
struction and operation feedback. In this paper we present an IT-based approach that supports volumetric modeling
of building blocks for office buildings by design sketching in SketchUP. We propose to support the process in
providing generated structural systems proposals and display them appropriately in the sketching environment
together with a set of performance predictions. For the subsequent design iteration steps (try more options, refine a
design or compose different variants) changes in the structural response and the related performance indicators can
provide a better informed decision making process. The work is based on two hypotheses:
1. The approach of Serious Play in conceptual office building design can be supported by a sketching tool for
volumetric modeling of building blocks that creates automatic structural systems responses and related
performance predictions
2. Composite steel concrete structural systems for office buildings are scientifically understood sufficiently well
that we can implement the corresponding construction and design methods for conceptual design in algo-
rithms executable by computers. The goal is to quantify the necessary material quantities of standardized
composite building systems in conceptual design sufficiently accurate and to specify the space requirements
467

for the structural system.
The second hypothesis is based on Donald Knuths statement about science and art (Knuth, D. 1974):Science is
knowledge which we understand so well that we can teach it to a computer; and if we don't fully understand
something, it is an art to deal with it. Since the notion of an algorithm or a computer program provides us with an
extremely useful test for the depth of our knowledge about any given subject, the process of going from an art to a
science means that we learn how to automate something
For the concept and the tool implementation, we named the sketching-based volumetric modeler for building
blocks Sustainable Office Designer (SOD) and the structure generator as Office Structure Generator (OSG). Both
applications were developed separately, the SOD as a plug-in to SketchUP and the OSG as stand-alone application.
Currently the OSG has been implemented in SketchUp and can provide a structural response to a volumetric
architectural model in SOD. Thus the architects can use one application, which we refer as SOD.
We will start our paper with the reflection of the architectural design process, which serves to understand the
context in which our sketching volumetric modeler has to work and the necessary requirements for SOD and OSG
to support that a playful design sketching and a better informed decision-making process. Next we introduce the
concept of the SOD followed by a description of the envisioned workflow for the architectural conceptual design.
We introduce the concept of the structure generator OSG and how both applications work together. We conclude
with an outlook of the next steps.
2. ARCHITECTURAL CONCEPTUAL DESIGN FOR OFFICE BUILDINGS
2.1 About the design of Buildings
The design process is an iterative approach and a collective process that involves a variety of actors. The archi-
tect has to take this into account with an appropriate design. He must be able to recognize and order the mixture
of individual and collective needs and the resulting constraints and develop a tailored integrative idea of space.
This is a complex process that cannot be described as a linear sequence of individual steps, but must be present-
ed as an iterative process.
2.2 Involved Actors and the Constraint of the Buildings Location and Function
To illustrate the complexity, the following is a brief description of the stakeholders and the general conditions to
be considered. Who are the actors in this process? First, the builder or the builder community is to be named who
will own the facility finally. Particularly for larger construction projects, this group is not always congruent with
the group of users who work, live or use the building in another way in the future. There is the large group of
indirect stakeholders who nevertheless want to see their interests taken into account: from the neighbors, to the
citizens of a community, to our society as a whole. It is responsible for the safeguarding of public welfare and
the promotion of a culture of build environment and tries to influence this by norms and rules. All these groups
of actors have wishes regarding the future building.
The conditions of the location and function are the constraints every project is subject to:
1. Categorized under the term location are all factors that deal with the location of a construction project: the
prevailing climate, the orientation of the site, the topography, the site's history, its social context, the existing
legislation (blueprints etc).
2. Summarized by the term function are all the functional aspects of a specific building, namely: the space
program, the operational procedures, the energy budget, requirements for indoor climate and acoustics,
statics, etc.
2.3 The playful Approach in Design
As already described, it is the responsibility of the architect to develop an "idea of space", which can integrate
this mixture of different constraints and requirements in itself. As designing is a creative process, it always needs
a playful free space. To maintain this free space, despite the described complex starting position, the architect
can use different techniques:
1. He can try - as a result of the analysis - to define system boundaries within which he blithely play
2. Complicating matters are first simplified to enable a more playful approach with individual building
468

components
3. The architect in a first step can work with alternative design ideas to explore the range of possibilities
and spatial relations of individual aspects
4. In Iterative steps the initially set of simplifications are gradually revised and differentiated.
As mentioned above, it is important for the architect to abstract complex relationships in order to preserve the
ability to act and play in design. The concept of building typology plays an important role in this respect. The
term building type summarizes specific knowledge about functional relationships and the necessary spatial se-
quences. This knowledge is based more on a mixture of exemplary buildings and experiences than on sound
scientific methods. Nor does it represent all the possibilities of design. Nevertheless, it helps the architect for
example, in the office building design at a very early stage to link the geometrical dimensions of his Play
Blocks with the assignment of specific office organizations types.
To integrate the necessary specific discipline knowledge, the designer binds specialists in the process. Since this
happens at a relatively late stage, he must be able to imagine and to consider the overall design in advance, in a
literal sense to provide space for them. For this he uses his knowledge about building typologies.
With the increasing demands on our buildings, the complexities of their functional conditions rise as well. The
demand for ecological performance optimization, structural systems optimization and related the minimization of
the fabrication and construction energy, as well as a growing economic pressures require the integration of high-
ly specialized knowledge at a very early stage of the design process. The traditional knowledge about building
typologies is not sufficient.
3. CONCEPT OF THE DESIGN SKETCHING TOOL
According to Eisele 2011, office building can be considered as an assembly of building blocks. For the given
conditions of the location and the functions required for an office building, architects use their knowledge about
building typologies to create spatial building block sequences that provide the needed functional relationships (see
Fig. 1). For different office lay-outs like single offices, group offices, combination of single and group offices,
business club and open space offices rules have been derived regarding comfort of use, flexibility for change of
office use requirements (i.e. still office use) without the necessity to modify the structure. Among others, the
flexibility rules specify: clearance height ranges, widths to depths ratios, distance ranges for internal column rows,
etc.
Fig. 1: Office buildings as parametric models in a system of assembled building blocks (from Eisele, 2011)
To support the design processes described above, we developed a Plug-In tool for Google SketchUp that we
named Sustainable Office Designer (SOD) We chose SketchUp, because it is a freely available sketching tool,
with intuitive features like easy drawing and loading the geo-position from Google Maps. Furthermore, the in-
cluded functionalities can be extended easily by scripts written in the programming language Ruby. The exten-
sions have been developed, because the original SketchUp program provides mere geometry interaction and does
not support automated approaches to geometry generation. To build a tool that complies with the conventional
way of working in early design phases and, most importantly, does not hinder but support the creative process, we
implement some design rules defined by the architects involved in the project. These design rules are derived
from architectural knowledge about building typology and experience gained from different projects and design
studies. They consider different design aspects like the room depth to provide optimal natural lighting or column
free spaces to support a minimal amount of flexibility. Furthermore, a number of German design standards for
working spaces are considered. As another way to keep things simple, we decided to use as less different func-
tionalities without restraining the freedom of the architect to design what he is thinking of. The functionalities
have to be intuitive. Accordingly, the use of numbers and formulas has to be avoided, and therefore we do not
469

attempt the architect to use numbers and formulas. The architect can just focus on the visual appearance of the
design.
3.1 Sketching-based Design from 2D Footprint to volumetric Building Blocks Models
The main design constituents are:
a site plan or a preloaded image of the surrounding environment (e.g. from Google Maps)
a 2D footprint design sketch
generated volumetric architectural model of multi-floor building blocks as a vertical extrusion of the
footprint
subtractive volumes for openings and accesses to the building, like entrances and passages
volume elements for elevators and staircases
volumes for column free spaces
In the following, we illustrate the prospective workflow for the conceptual design. In a first step, the architect
sketches different shapes for the footprint in a preloaded site plan (Fig. 2) or alternatively in a representation of the
surrounding environment (Fig. 4), which was uploaded from Google maps. The following hand drawn architec-
tural sketches illustrate both the design workflow by an example of an office building placed next to an existing
L-shaped building and as a visual means to define the requirements for the SOD. In a second step a volumetric
architectural model of the building block will be generated as a vertical extrusion of the selected footprint using the
provided parameters e.g. number of floors, office types and flexibility class. The SOD will generate the volumetric
model by deploying the implemented rules from the architectural building typology knowledge. In parallel a
functional structural design model (FSDM) is created, which serves as input for the structure generator OSG.
Depending on the chosen office type the FSDM e.g. has a requirement for allowable distance ranges for internal
column rows.
Fig. 2: Variants of 2D footprint design sketches and the generation of volumetric building block models
The architect can now explore the volumetric model in the 3D visualization and compare it visually with other
alternatives, which have been designed based on the different foot prints in Fig. 2, left. We are interested to learn in
the next research steps, if the computer visualization of the 3D models is sufficient for decision making and has the
potential to reduce the need for physical architectural models.
Fig. 3: Sketching and modifying entrances, passages, openings and core elements for vertical circulation
470

Following the volumetric design of the building block the architect can design the access to the building and the
vertical circulation. Again the sketching approach is used to insert subtractive volumes for entrances, passages and
openings (see Fig. 3, left) and staircase and elevator volume elements, which can be placed into the building block
model (see Fig. 3, right). The designer can add and modify the circulation volumes. Especially the vertical cir-
culation elements in most cases are not part of the steel construction due to the fire safety reasons. In the FSDM
these volumes are specified as concrete cores containing the staircases, elevators and sanitary installations.
3.2 The playful Approach - conceptual Design Iteration Loops
As already mentioned in Chapter 2, architectural design is a creative, iterative process. The architect can apply
different strategies to playful develop conceptual designs. Fig. 4 show typical design loops that involves
sketch-based modifications, adjustments etc. and show an example of one design variant in SOD.
Fig. 4: Iterative design loops and the prospective work flow from a footprint sketch to a volumetric architectural
building block model.
3.3 The structural Response to the volumetric architectural Design
How to communicate the Essence?
In addition to visually comparing and evaluating the design options we suggest providing instant feedback to
architectural design variants with automatically generated, ecologically optimized steel composite systems for the
designed building blocks. To communicate the required spaces for the structure an abstract representation is
proposed to keep the design model free from congestions. In Fig. 5 top left, an idea of a mixed 3D volume for the
architectural design and a 2D grid representation for beams and columns is shown and in Fig. 5 top right the
actual implementation in SOD. Fig. 5 bottom left displays the transparent architectural building block model and
471

necessary construction height for each floor is displayed inside by a grey volume extruded from the footprint of
the slabs. Beams and columns are displayed as grids on the slabs. The last variant shown in Fig. 5 bottom right
will be displayed in SOD next to the volumetric architectural model. The linear structural elements like beams
and columns are shown with their 3D center lines; the slabs are shown as an extruded foot prints with the neces-
sary construction heights.
Fig. 5: Structural response to volumetric architectural design. Abstract representations of the structure
From the generated structural system for an architectural volumetric design variant, the necessary material con-
sumption, the ecological footprint to be expected and further performance indicators will be calculated. In a
performance view (Fig. 6) a selectable number of design variants can be quantitatively compared to support a
better informed decision making process. The necessary structural materials, like concrete, reinforcement steel,
Fig. 6: Design performance view to compare design variants.
structural steel and sheet metal decking are currently the input parameter of for the calculation of the ecological
footprint. We plan to add, the coating of the structural steel and the faade system in the future. Thus only a part
on the buildings impact on the global warming potentials is covered. The intention of this approach is, to give
instant feedback about the structure and faade system related consequences for architectural design variants.
472

4. FROM ARCHITECTURAL DESIGN TO STRUCTURAL DESIGN
The main objective to create a structural systems for architectural conceptual design variants is to assess the
necessary materials quantities sufficiently enough that we can create reliable performance predictions. Office
buildings with steel concrete composite structure in this respect can be considered as an assembly of several
building blocks of office floors as shown in Fig. 1. The footprint of the building block is rectangular or trapezoid.
For the same architectural design different composite floor systems (Fig. 7 shows some examples) can be applied.
We refer primarily to the primary-secondary composite beam and slab floor system to explain the concept (Fig. 7
in the middle). The other systems have been implemented in the OSG as well.

Fig. 7: Composite steel concrete floor system (SZS 2006)
4.1 Parametric structure Definition and design Rules
Design rules from Eurocode 4 (2004) are used for the dimensioning and the verification of building elements.
Considering a preliminary structural design, only composite beams, composite slabs and steel/composite columns
are verified. The focus is on the vertical load transfer, while bracings and connections between the building blocks
are neglected in the approach. To find an appropriate structure for a given volumetric building block design,
engineering knowledge has to be implemented to create parametric structural system models that handle the
composite systems characteristics, the structural variation possibilities and the structural restrictions well and in a
balanced way. Different strategies have been developed for automating the distribution of loads, structural analysis
and design verification, reducing the number of different profile sizes or handling specialties like propped con-
structions. As an example, one characteristic of the primary-secondary composite beam and slab system is to
maintain load transfer during the optimization steps. The primary beam serves as continuous multi-span system in
one direction and secondary beams are single span beams in the other direction. To cope with the architectural
flexibility requirements the parametric model definition was extended such that systems can use either the lon-
gitudinal or the transverse direction for the primary beams and apply different span widths for both the secondary
and the primary beam.
4.2 Finding an appropriate structural System
Our goal is to find an appropriate structural solution for a given volumetric architectural office building block that
has an optimized sustainability performance. For the search we use a Genetic Algorithm (GA) approach that
encodes the parametric structural model and uses a single fitness function which estimates the sustainability
performance of the designed structure. In the artificial evolution process candidate structural design solution with
low fitness values die out and the candidate with the best fitness is chosen.
Fig. 8: Functional structural design model (FSDM) left, generated structural model right
473

As mentioned in chapter 3.1 every time the architect creates or modifies the volumetric model, the SOD will create
or update a functional structural design model (FSDM) see Fig. 8 left. It describes the requirements, specifications
and constraints for the structural system to be designed. The FSDM serves as an input to the automated structure
generator. The structure generator initially applies a parametric composite structural system model with column
positions and their possible variation ranges are defined according to the FSDM but the composite floor system
type, beam layouts and dimensioning are subjected to the evolutionary search process. The resulting structural
design model and the corresponding performance indicators can be inspected by the architect on demand (see Fig.
5 + Fig. 6). The architect may modify the model and possibly introduce additional requirements and restrictions on
the structural system, e.g. modify the allowable column locations, demands additional clearance spaces or make
justification to the previous designs. In Fig. 8 left, we see the red range as possible location for internal columns,
which comply with the rules for flexibility to use different office type lay-outs; a clearance space in green, concrete
cores for stairs, elevators and vertical installation spaces for building services and two restricted areas in the left
wall and rear area. With the updated FSDM the architect can invoke a modified structure generation (e.g. Fig. 8
right). These architectural-structural design iterations can be continued until the architect finds a satisfied design,
which is compliant with the clients needs.
5. CONCLUSIONS AND FURTHER WORK
So far, we have the following preliminary results:

With the first implementation of the SOD in SketchUp, we could prove our concept of levering the pa-
per-based playful conceptual office design to computer supported volumetric modeling that deploys
rules from architectural knowledge of building topologies.
For multi-floor office building blocks with rectangular foot print, we could automatically generate pri-
mary-secondary composite beam and slab floor systems with OSG, which have optimized sustainability
performances. The structure generator was checked against an existing structural office building design
and showed acceptable results. The estimated material quantities were sufficiently accurate for concep-
tual design
The structure generator has seamlessly been integrated in the SOD and communicates via a functional
structural design model
The structural systems for volumetric architectural building bloc models can be communicated in dif-
ferent level of abstraction
A performance design view provides to compare the quantitative key performance indicators of differ-
ent design variants
The architects of the team will start using the SOD and OSG prototype and use it on three existing office build-
ings. Their feed-back especially regarding the playful design and the structural response are used to stabilize
and further develop the application. Hopefully we soon might reach the point, when we can systematically
test-run "Serious Play" in sustainable office design with larger user groups.
6. ACKNOWLEDGEMENTS
The authors wish to thank the Forschungsvereinigung Stahlanwendung e.V. (FOSTA) and the AiF
Arbeitsgemeinschaft industrieller Forschungsvereinigungen for supporting and financing the research. The
research project "Nachhaltige Bro- und Verwaltungsgebude in Stahl- und Stahlverbundbauweise is a
collaboration of the following universities: Lehrstuhl fr Stahl und Leichtmetallbau, RWTH Aachen, Fachgebiet
Entwerfen- und Baugestaltung, TU Darmstadt, Institut fr Stahl- und Holzbau, TU Dresden, Institut fr Technik
und Arbeit e.V., TU Kaiserslautern, Lehrstuhl fr Unternehmensrechnung und Controlling, TU Kaiserslautern and
the Computational Modeling and Simulation Group, Technische Universitt Mnchen, with the lead from
Lehrstuhl fr Metallbau, TU Mnchen.

474

7. REFERENCES
British Standards Institution. (2004). Eurocode 4: design of composite steel and concrete structures. London, BSI.
Eisele, J. (2011). Nachhaltige Gebudetypologien im Hochbau. In: Nachhaltig Planen, Bauen und Betreiben -
Chancen fr den Stahl(leicht)bau, 2011, Berlin, Germany.
Forschungsinstitut der Zementindustrie (FIZ), Forschungsgemeinschaft Transportbeton e.V. (FTB). (2010).
kobilanzielle Profile fr Bauteile aus Transportbeton.
Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning. Reading, Mass: Ad-
dison-Wesley Pub. Co.
Grierson, D. E. and Khajehpour, S. (2002). Method for conceptual design applied to office buildings. J. Comput.
Civ. Eng. 16(2), pp. 83-103.
Kaveh, A. and V. Kalatjari (2003). Topology optimization of trusses using genetic algorithm, force method and
graph theory. International Journal for Numerical Methods in Engineering 58(5), pp. 771-791.
Knuth, D., Computer Programming as an Art, CACM, Dec. 1974
Mensinger, M., Baudach, T., Breit, M., Eisele, J., Feldmann, M., Franz, C., Hogger, H., Kokot, K., Lang, F.,
Lingnau, V., Pyschny, D., Stroetmann, R. and Zink, K.J. (2011). Nachhaltige Brogebude mit Stahl. In:
STAHLBAU 80 (10), pp. 740749.
Nanakorn, P. and Meesomklin, K. (2001). An adaptive penalty function in genetic algorithms for structural design
optimization. Comput. Struct. 79(29-30), pp. 2527-2539.
Nimtawat, A. and P. Nanakorn (2009). Automated layout design of beam-slab floors using a genetic algorithm.
Comput. Struct. 87(21-22), pp. 1308-1330.
Nimtawat, A. and P. Nanakorn (2010). A genetic algorithm for beam-slab layout design of rectilinear floors. Eng.
Struct. 32(11), pp. 3488-3500.
Rafiq, M. Y., Mathews, J. D. and Bullock, G. N. (2003). Conceptual building design - Evolutionary approach. J.
Comput. Civ. Eng. 17(3), pp. 150-158.
Report of the Conference of the Parties on its third session, held at Kyoto from 1 to 11 December 1997. Addendum.
Part two: Action taken by the Conference of the Parties at its third session.
Shrestha S.M. and Ghaboussi, J. (1998). Evolution of Optimum Structural Shapes Using Genetic Algorithm. J.
Struct. Eng. 124(11), pp. 1331-1338.
SZS STAHLBAU ZENTRUM SCHWEIZ. (2006). Bauen in Stahl: Brandschutz im Stahlbau. Zu rich: SZS
Stahlbau Zentrum Schweiz.
Turrin, M., von Buelow, P. and Stouffs, R. (2011). Design explorations of performance driven geometry in archi-
tectural design using parametric modeling and genetic algorithms. Adv. Eng. Inf. 25(4), pp. 656-675.
Umwelt-Produktdeklaration (EPD). (2011). Bausthle: Offene Walzprofile und Grobbleche. (EPD-BFS 2010111
nach ISO 14025) der Hersteller ArcelorMittal, Dillinger Htte, Ilsenburger Grobblech, Peiner Trger und
Stahlwerk Thringen.
Breit, M., Mrki, F., and Vogel, M. (2008). iDecision - About Decision Modeling and Decision Thread Optimi-
zation, In: ICCCBE-XII & INCITE 2008: 12th International Conferences on Computing in Civil and Building
Engineering & 2008 International Conference on Information Technology in Construction. 2008. Beijing, China.
Schraege, M. (1999): Serious Play: How the World's Best Companies Simulate to Innovate, Harvard Business
Review Press
475

VIRTUAL REALITY AND SCIENTIFIC VISUALIZATION APPLIED IN
TECHNICAL TRAINING IN BRAZILIAN CONSTRUCTION
Fabiano Paulico Stange & Sergio Scheer
Federal University of Parana, Brazil
ABSTRACT: Currently the Brazilian construction industry is undergoing to economic process which results in an
attractive pole for young people. They are striving to learn professions like Architecture, Engineering and
Construction (AEC). They belong to a generation where computer is present in their lives to access information
and knowledge. This generation is no longer a passive receiver of content transmitted, but a producer of his
knowledge through interactive media. AEC Education in Brazil is still based on a paradigm where the knowledge
is centered on the figure of the educator. In general, the Brazilian Educator does not exploit the existing resources
that are familiar in this new generation. This underscores the need to change this paradigm, addressing the
Information Technology in learning. Researching new contexts for learning in AEC can be noted that Virtual
Reality (VR) presents itself as a powerful teaching tool. The aim of this paper is to describe the development of a
tool to complement the teaching and learning process. It is presented a prototype of a virtual environment (VE)
based on VR-desktop and it is oriented to construction tasks. It is chosen to simulate a task that is frequently found
in Brazilian construction which is the assemblage of a wood mold for shaping a concrete pillar. Geometric
modelers like Autodesk 3ds Max were used to create a virtual construction site and export it in an appropriate
format ready to apply interactivity using toolkits like Kitware VTK. It is hoped that through the experience in VE
the apprentice can understand task simulated in an intuitive and less oppressive than the old methodologies in the
classroom. It also aims to encourage educators to seek more innovative and motivating pedagogies.
KEYWORDS: Virtual Reality, Scientific Visualization, Construction, Task training.
1. INTRODUCTION
1.1 Motivation
The Brazilian civil construction and real estate are at an economic boom thanks to the actions of the Brazilian
Federal Government through strategies of social interest such as Growth Acceleration Program (Programa de
Acelerao do Crescimento - PAC) and the Program Minha Casa, Minha Vida (PMCMV). One effect of this
phenomenon can be noticed as the professions which involve Architecture, Engineering and Construction (AEC)
are currently the target of young people who aspire to enter in Brazilian labor market. This can be confirmed, for
example, when observing the increased demand for the civil engineering course to the point to be the more
competitive course in certain educational institutions (CEF 2012, Correio Braziliense 2012, Tchne 2012,
Tribuna do Norte 2012, Veja 2011).
These apprentices belong to a generation where the computer is present in their lives by facilitating access to
information and knowledge. The individual of this generation is no longer a passive receiver of the transmitted
content, but a producer of his own knowledge through interactive media that were once exclusive access and now
are becoming commonplace. They are more ready to be influenced by the new technologies because each day they
receive a considerable amount of new electronic devices and latest applications.
But the Brazilian education in AEC continues based on a paradigm where knowledge is centered exclusively on
the figure of the educator/trainer/teacher and usually this one does not explore the current features that this
generation is familiar. Still, the fields of AEC are not the same as 30 years ago, highlighting the need for a more
interactive learning, the building of critical thinking, the motivation of creativity to solve problems in exchange for
an apprenticeship with a passive perspective. Such requirements or skills are not always acquired in university but
when the learner is already inserted in the labor market. Then there is a need to change this paradigm, and a
possible approach is the computer as a learning support. The educational process must be updated to address the
discrepancy between teaching and learning and should be wary of the differences between interactive man/man
and man/machine and dialogue between educator/trainer/teacher and the learner through the proper use of
technology, aiming for higher quality to the professional himself, the institution and the course (Carvalho 2003,
Hutchinson et al. 2003, Pompeu 1999).
Parallel to this context, the Virtual Environments (VE) are using the resources of the Virtual Reality (VR) and are
476

becoming more accessible thanks to advances in the computer industry and making possible its use in teaching
aimed at AEC. The VE has all that other graphic programs offer, but with more motivation through the pursuit of
the stimulus to the learner, providing this one a closer examination of the object contained in the VE, the
participation of disabled people, a bigger liberty of time to the learner in exchange for fixed hours in classes, active
participation instead of the passive and allows a multiperspectival learning experience. Its important to remember
that the VR will not solve all the problems of education but it is possible to show the educator/trainer/teacher that
there are options beyond the classroom (Pasqualotti 2000, Seabra and Santos 2005).
1.2 Aim of the research
This paper studies the development of prototype of a VE as a complementary instrument learning which is adapted
to this new generation. The prototype was developed like a VR world-on-window or desktop (via a notebook)
where one user at a time can experience a virtual training of a typical activity in Brazilian construction: the
sequence of assembly of a mold to shape a concrete column. The choice of this type of VR is the fact that the high
cost of building an immersive VE, provide some facility for educator/trainer/teacher and apprentice to program
only the activity simulated rather than of spending hours in programming/synchronization physical meshes with
advanced devices for interactivity. However, the prototyping VE may be adapted to a semi-immersive VR
(coupling the computer to a 3D projector, for example). The use of a Scientific Visualization (SciVis) library as
VTK can provide the visualization of quantities related to the AEC theme further than the simulation itself. The
validation of the prototype VE will occur through exposure it to the students in the early years of AEC graduation
who will evaluate the VE.
1.3 Organization of the paper
The first section presented the justification of the research. In Section 2 will be presented the related concepts to
the learning in AEC and its relationship with VR, Section 3 discusses how it was developed the VE prototype and
the methodology that will be used for validation in learning and a conclusion will be presented in Section 4.
Acknowledge and references were left to the final sections, Section 5 and Section 6 respectively.
2. APPRENTICESHIP AND TEACHING IN AEC
2.1 Constructivist perspective in using VE as a learning instrument
Constructivism has its basis reasoned in researches done by the Swiss biologist Jean W. F. Piaget and it is a part of
the set of theoretical trends committed to illustrate how human intelligence develops. It assumes that the increase
of human intelligence is a result of reciprocal action between individual and environment. Considers the
hypothesis that man does not born intelligent and is not a passive individual in front the influence from the medium,
that is, he responds to external stimuli which act on him to build his own knowledge, increasingly organized and
elaborated (Lopes 1996). In constructivism, the educator/trainer/teacher is not seen as a conductor of learning, but
a facilitator who can provoke situations that will contribute to the enrichment of the medium which the apprentice
will interact to develop his knowledge (Weiss and Cruz 2001).
According to this theory, the individual can build his knowledge through physical or mental actions on the objects
that are contained in the medium and this causes an imbalance in this individual. As a result, it happens the process
of assimilation of these actions or, processes of accommodation and assimilation for the individual to build his
mental structures called schemes. In other words, since the individual is unable to capture and organize
(cognitively) the object of the medium to expand his schemes, that is, to proceed with the assimilation, he tries to
make a modification or accommodation of a scheme because of the peculiarities of the object and then proceeding
with the process of assimilation and finally reach equilibrium. The individual builds and rebuilds successively his
cognitive schemes to get more balance (Lopes 1996, Moraes 2003).
McLellan (1996) comments that the VE are a powerful educational instrument for learning when using the
constructivist perspective, because the data are not perceived as a numerical list coming from a simulation, but as
an environment surrounding the individual or apprentice for contribute to the process of building his knowledge
through intuitive exploration. In a VE the individual makes the world and cares for it, he can interact and learn by
trial and safely and also, try out the possible consequences for then build his knowledge, both individually and
collectively. This knowledge can gain more potential when the learners strive to create VE by themselves, because
as Krauchunas and Flannery (2000) learners probably create new ideas when they are actively engaged in creating
new projects and share with others, with considerable experience in the towards they can confirm or refute their
477

hypotheses.
The use of a VE as a learning instrument is suggested in the following situations: a) when the education or training
in the real world offers a danger both to the learner as to the real environment, there is inconvenience to teach or
train in real world, or when it is impossible to put into practice; b) a model will train or teach as effectively as in
the real world; c) the interactivity offered by VE can be considered motivating; d) develop an VE is an object of
importance for the learning construction; e) overcome some barriers such as introspection or provide
opportunities for learners who have disability to do experiments and tasks to which they could not perform or
participate (Pantelidis,1998).
2.2 Related works
Haque (2001) at the University of Texas A & M, USA has developed a VE non-immersive for teaching concrete
structures to be applied to the students of Civil Engineering using the resources of the internet combined with the
JAVA language and HTML and VRML. The VE was structured and modulated according to content and level of
knowledge of the user about the theme and divided into five concepts of visualization which are: presentation of
didactic material in HTML, visualization and animation of 3D structures using GIF format, manipulated images
with indications and information, interactive simulation provided to the user when it can change the intensity of
certain actions on a virtual structure, navigation in environments that represent some structural parts.
Santos and Duarte (2005) at the University of Sao Paulo, Brazil researched the impact of VR technologies, in
particular Augmented Reality, interactivity with datagloves and stereoscopic visualization to develop an VE where
students of the first year of courses such as Engineering and Architecture, especially for those who study the
discipline of Descriptive Geometry could develop the ability to visualize three-dimensional elements and do not
leave such courses.
At the University of California, USA was developed a conceptual class called Vizclass which used some of VR
technologies to be applied to the process of learning the discipline of the finite element method to civil engineering
students, between 2003 and 2005. This environment learning had a software simulation of earthquakes in
engineering systems, a software for visualization the results of the finite element method, three 2D whiteboard
touch screens of 72 inches, a stereoscopic projector, a cluster of 8 cores, a trackball, a keyboard and a wireless
surround sound system. It was adopted the following methodology to evaluate pedagogy: students and teacher
were submitted to a questionnaire evaluating the learning process. The results show a better performance by
students, creating better works, more interest and commitment to the responsibilities proposed by teachers. The
teacher reported an improvement in the approach to discipline, increasing the ability to expose the contents,
additional stimulus to the discussions inside and outside the classroom and an enthusiasm to use the whiteboard
and discomfort when using the trackball (Grimes et al. 2006).
The School of Construction at the University of Southern Mississippi, USA used VR tools for 3D games and
educational aid in the teaching of architecture by introducing students to two disciplines: 3D modeling and
animation and VR applied. The goal was to evolve the skills of students in three-dimensional visualization with the
use of photorealism in real-time navigation through teaching tools such as engine C4Engine (Theraton Software),
SketchUp (Google) for geometric modeling and Artlantis (Abvent Group) to generate the photorealism and
animation and a tool for creating VE developed on the C4Engine (Theraton Software) called BuildIT4. The study
focused on feedback from students after coursing the disciplines already mentioned, and in response to students'
motivation and ability to use examples of VR in architecture and also the need for better hardware and some
discomfort in using the tools because they are familiar with software CAD (Shiratuddin and Fletcher 2007).
Sampaio et al. (2008) indicates the need for new ways of education for civil engineering courses in order to provide
students with a better way of understanding the relationship between steps, tasks, deadlines and construction
process involved in a real construction. Through two virtual models, a typical wall of a building and the
construction of a bridge, experts in these areas were consulted to assist in the development of models searching the
accuracy and efficiency to the didactic. The research applied to VR software (EON Studio) and three-dimensional
modeling by computer (AutoCad) to explain the evolution of the models using VR. Students of technical design,
construction processes and bridges could interact with the models by selecting quantities, changing dimensions,
controlling schedules and deadlines and the way of construction.
The power distribution company COPEL, Parana, Brazil, in partnership with the Federal University of Parana,
Brazil and the Institute of Technology for Development (LACTEC), Brazil developed a VE whose goal was to
develop and modeling a training of live line power network maintenance using VR techniques, establishing a new
478

training model to complement the existing process. The VE was developed using the open source toolkit called
OpenSceneGraph. The physics simulation was implemented using Bullet Physics, two TV Mitsubishi 72 ",
geometric modeling using 3D Studio Max (Autodesk) and interactive devices through the Wiimote (Nintendo)
(Buriol et al. 2009).
At Pennsylvania State University, USA, was developed an experience with VE around a case study, the MGM
Grand hotel in Las Vegas, Nevada. By developing a software called 4D Virtual Construction Simulator (VCS4D)
based on engine Deep Creator (now known as Esperient Creator), ten groups of engineering students were asked to
develop a critical path schedule from the production of a floor from the hotel and its components, including walls,
floors, structural parts and connections. In conclusion the interactivity added in VCS4D helped students to develop
higher-quality schedules and provided an enjoyable learning experience. Running VCS4D proved also encourage
collaborative work in groups and promoting greater generation of solutions with the best visualization of
construction processes (Nikolic, Messner and Anumba 2010).
The architecture course and the course of computer science at the Federal University of Ceara, Brazil are
developing an education project called the image space - image object. Through immersive VR, it wants to involve
the student in the exercise of exploring design solutions for architectural spaces. The project is set up for research
into new strategies for the perception of immersive spaces, the use of three-dimensional modeling, BIM
methodology and workshop for prototyping building (Cardoso et al. 2010).
A system is being developed as a resource for learning and teaching with a large group of students in construction
management from the University of New South Wales, Australia. The analysis is used to investigate a specific
design and construction options. By modifying the engine CryENGINE 2, students explore the possibility of
several tools for building homes (Newton and Lowe 2011).
Lin et al (2011) proposes an AV comprehensive safety training for construction sites where AEC students assume
the role of inspectors of safety and walk the venue to identify potential hazards. The game was designed with
features such as realism, self-learning, non-linearity, interactivity through the use of the Torque 3D engine to run
the game system, geometric modeling using 3DS MAX (Autodesk) as well as MilkShape 3D (Soft Chumbalum)
was used to create 3D objects are not available. Results of tests indicated that students with gambling interests
increased learning, appreciated the learning itself and were motivated to update their knowledge in safety.
3. IMPLEMENTATION OF VE PROTOTYPE
3.1 Initial considerations
For the implementation of VE prototype was used the Microsoft Visual C + + 2008 Express Edition as an IDE and
the Visualization Toolkit version 5.4 (VTK - Kitware Inc.) to generate both visualization and interactivity to create
the scene in the VE prototype. The VTK is an object oriented library with more than 700 pre-compiled classes in
C + + and has been highlighted in the area of scientific visualization. The geometric modeling of the construction
site was developed using 3DS Max (Autodesk) that allows the creation of virtual scenes quite easily and provides
tools to export the mesh of geometric objects in a compatible format with VTK. Finally, the VE prototype was
tested in a notebook equipped with an Intel Core i7-2630 2 GHz, 6GB RAM, operating system Microsoft Windows
7 64-bit video card NVIDIA GeForce GT 540M.
The VE prototype starts with a presentation screen and other screens appear in sequence to inform the user about
the characteristics of the activity through illustrations and informative text (Figure 1). Finally, the scene is loaded
with the virtual construction site. In the upper right of the viewing window of the scene there is a legend that
informs the student what he should do during the activity, indicating which part of the mold in the sequence and its
location in the virtual construction site (Figure 2). During the simulation, the mouse must be used to navigate and
choose a part of the mold related to the assembly sequence. For each selected part, there is another part in the same
scene, in magenta, which tells the user to position that chosen piece must be leased. It happens with these steps
until the end of the simulation of the activity of the mold assembly.
3.2 Description of the virtual scene
The scene consists of a virtual construction site surrounded by buildings and streets. This is enclosed by a fence.
On the construction site has an accommodation, a warehouse and a concrete structure that is executing. The
structure is composed of eight pillars and seven are already ready. The simulation is to assembly a mold to shape
the final pillar of concrete as construction techniques. The mold parts are on the virtual construction site.
479

Figure 1: Example of the presentation screen (left) and informational screen (right).

3.3 Simulation of the activity in construction
A project related to civil construction lists a set of activities according to the purpose of the project, the team
responsible for project and construction techniques which will be effective to result in a product with quality. Even
when trying to standardize the construction activities according to of standards scientifically conceptualized, it is
difficult to verify a single approach to certain activities. During the implementation of VE prototype were found at
least six ways to execute the wooden molds according to the recommendations of the technical bulletin no.
50/1943 of the Brazilian Association of Portland Concrete (ABCP) and their guidelines are: a) implementation of
molds depending on the dimensions of the parts described in structural design; b) the mold must have sufficient
rigidity to prevent defects in the structure; c) prevent loss of cement during the molding, observing the sealing of
the mold; d) the molds must be designed to allow the removal of parts with ease, must be performed allowing the
greatest number of reuses (Junior et al. 2000).
The methodology used to simulate the constructive procedure was based on the GETHAL system (Junior et al.
2000, Yazig 2009), which are used in plywood sheets with thickness of 18 mm for straight panels straight and 12
mm for curved panels, with some procedures implementation: a) panels stiffened by double battens (dimensions:
1" x 4") or rafters (dimensions: 3 "x 3") and locked by anchor bars (rods); b) ties modulated by lath and metal
fittings; c) plates covered with metal parts and with plywood. It was adopted for the simulation the execution mode
described in (a). The sequence of assembly is implemented in the following: positioning gastalho (gauge
according to dimensions of the pillar), positioning a panel, lock this panel with latches of wood, placing the two
panels that are perpendicular to the first panel, position the reinforcement steel bars, close with the last panel, lock
the mold with the remaining panels, lock the panels with tensioners (metal parts to give stiffness will mold when
occurs the launch of the concrete).
3.4 Implementing the virtual environment
3.4.1 Geometric modeling
The geometric modeling of objects that belong to the scene employs the use of mesh with triangular elements for
curved surfaces and mesh with quadrilateral elements for objects with surfaces are not curved.
480

Figure 2: View of the virtual construction site and the legend with information about the sequence of activity.

It was used the OBJ format (Wavefront Technology) to export the meshes of the objects. The vtkOBJReader class
allows importing geometric objects in this format but was not found a method to import the material relating to this
object. This class is a source object whose output is a polygonal data that will be mapped through the
vtkPolyDataMapper which transforms the data into graphics primitives. The vtkActor class represents the graphics
primitives in the rendered scene. Although VTK supports VRML format (Virtual Reality Modeling Language is a
format typically used in RV) through the vtkVRMLImporter class there is no support for all of the VRML nodes,
especially the ImageTexture node responsible for applying the texture. The VTK also offers vtk3DSImporter class
that inherits methods for vtkImporter class and it imports files in formats like 3DS (geometry and material). But
since is an instance of the vtkRender class it directs the dataset directly to renderer and makes it difficult the use of
classes to manage and synchronize the interactivity.
3.4.2 Textures applied to the objects
The textures are represented by 2D images in JPEG (Joint Photographic Experts Group) and are imported through
the vtkJPEGReader class who is also a source object. The image mapping for the geometric model is made through
the vtkTexture class. Some properties of the texture can be adjusted using: SetInterpolateOn( ) for interpolation of
the texture during the rendering, EdgeClampOn( ) to adjust the image when outside the parametric interval [0,1],
SetQualityTo32Bit( ) to the texture quality. Currently, VTK supports only 2D textures. The instances of VtkTexture
are associated with actors by SetTexture( ).
The virtual camera and lighting use vtkCamera class and vtkLight class respectively. To manipulate the virtual
camera through the mouse (rotation, zoom, pan) was used the vtkInteractorStyleTrackballCamera class. During
the synchronization between modeled objects and routines for interface and interactivity it was tested features
provided by VTK to manipulate of the virtual camera. It was decided the fly navigation because it offers to the
learner greater freedom to explore the scene and facility to perform the movement of parts through the possible
positions that the user can enjoy his field of vision.
481

3.4.3 Interactivity through VTK
The objects in the scene that require interactivity, such as parts of the mold use the classes of 3DWidget. These
classes are subclasses of vtkInteractor, an abstract class responsible for watching the events invoked in the window
of vtkRenderWindow class. The widgets have their own representation in the rendered scene and offer various
types of interactivity to their respective actors in the scene, synchronized by the method SetProp3D( ). Each widget
must use SetInteractor( ) to specify what widget is rendered.
In VE prototype was created a class that inherits vtkBoxWidget called vtkBoxWidget1 to handle the parts used to
assemble the mold of the concrete pillar. Some characteristics of the class: the widget is not visible in the scene to
not disturb the identification of parts, not visible and not turned on the handles and outlinecursors to control scale
and rotations, the class initializes the position of the widget in the scene.
The movement of the parts happens through the interactivity provided by vtkInteractor, vtkCommand, and
3Dwidgets. A 3Dwidgets through vtkObject class can observe any event through one of its instances. When it
observes an event that is related invokes an associated command. And unlike 3DWidgets, vtkInteractor does not
have any graphic representation. This class is the basis for the support of actors control events and virtual cameras
in a scene, having the most control routines, such as interaction with mouse, joystick, and trackball.
The class created for interactive in VE prototype is called vtkMyCallback class and it inherits vtkCommand class.
It has pointers to the actors and widgets that represent parts of the mold, for the actors that indicate where the piece
should be positioned in the scene called for indicator actor and legend, updating the information. When a part has
been selected occurs a comparison between widget of the scene and the pointer that points to assembly sequence. If
the comparison results in true the interactivity starts and movement happens. The pointer returns false if there is no
interactivity with the part. The coordinates are saved by an array as a linear array of processing the selected widget.
By positioning the part from the wood mold near the actor indicator, the adjusting of the final position occurs
automatically. The interactivity is ended and a new indicator appears the scene and so is released an interactivity to
the next piece of the assembly sequence. An example of this process can be seen in Figures 3 and 4.
The legend by vtkLegendBoxActor provides information about which parts should be selected and positioned as
the actor indicator. The information is updated as the user will position parts, through the class pointer
vtkMyCallback (Figure 2 and Figure 4).
3.5 Methodology to validate the VE
To validate the use of AV prototype will be exposed in a lecture to a group of students in the first year of AEC on
the subject, divided into small groups (5-6 students) who will have the opportunity to experience the simulation.
At the end of the section will be submitted to a questionnaire to assess the applicability of AV prototype. The
questionnaire is presented in Table 1.
Table 1: Questionnaire to be used with students to evaluate the VE prototype.
Questions Rating
Perform the task is simple 1 2 3 4 5 6 7 8 9 10
Organization of information 1 2 3 4 5 6 7 8 9 10
Reliability of the VE 1 2 3 4 5 6 7 8 9 10
Motivation to learn 1 2 3 4 5 6 7 8 9 10
The course should invest in technologies like this to teach the classes YES NO
What is the sequence to assemble a mold

482

Figure 3: Scene from the construction site and moving a part of the wood mold.

Figure 4: Position of the mold part and an indicator actor of the assembly sequence appears in the scene.
483

4. CONCLUSIONS
This paper presented the concepts of Virtual Environment and the performance in the learning process within the
educational context of Architecture, Engineering and Construction. Although it was proposed and developed a
technical training environment in the construction that can serve both to help in the classroom for students to
understand the task simulated as well as can be seen as a template for other implementations of techniques and
activities related to construction. But it needs to be tested by students in civil engineering and architecture to
validate its potential use in learning and curriculum of Universities.
5. ACKNOWLEDGMENTS
The authors would like to thank the Graduation Program in Numerical Methods in Engineering of the Federal
University of Parana and the financial support from CAPES. Also like to thank Alias/Wavefront, Silicon
Graphics, Kitware, Microsoft, GETHAL, Autodesk, ABNT and ABCP.
6. REFERENCES
ABNT (2007). NBR 6118/2007 Projetos de estruturas de concreto Procedimentos, Associao Brasileira de
Normas tcnicas (ABNT), 154 p.
ABNT (1997). NBR 7190/1997 Projetos de estruturas de concreto Procedimentos, Associao Brasileira de
Normas tcnicas (ABNT), 107 p.
CEF (2012). Caixa Econmica Federal, Available at: http://www1.caixa.gov.br/gov/gov_social/municipal/
programas_habitao/ pcmcmv/saiba_mais.asp. Accessed: 15/01/2012.
Correio Braziliense (2012). Correio Braziliense, Available at: http://www.correiobraziliense.com.br/app/noticia
/cidades/2011/08/14/interna_cidadesdf,265443/maior-parte-dos-cursos-procurados-no-df-sao-para-areas-saturadas.shtml.
Accessed: 15/01/2012.
Buriol, T. M., Rosendo, M., Scheer, S., de Geus, K., Felsky, C., and Goulart, J. C. (2009). Proposta de plataforma
baseada em realidade virtual para treinamento de atividades em linha viva, XXX CILAMCE, 113.
Cardoso, D., Bonfim, C., Barros, A., Souza, A. (2010). Imagens espao imagens objeto: o recurso da imerso nos
processos de ensino-aprendizagem em arquitetura, XIV SIGRADI 2010, 85-87.
Carvalho R. G. (2003). Mudanas promovidas pela aprendizagem colaborativa e tecnologia da informao em sala
de aula na disciplina Sistemas Estruturais, Thesis (Master), Federal University of Parana, Technology Sector,
Graduate Program in Civil Engineering, 112p.
Flannery, K. A. and Krauchunas, S.(2000). Virtual cognition lab: a constructivist approach toward learning, 3rd
Workshop on Virtual Reality - Gramado, Rio Grande do Sul.
Grimes, D., Warschauer, M., Hutchinson, T. C., and Kuester, F. (2006). Civil engineering education in a
visualization environment: Experiences with vizclass. J. Engineering Education (JEE), 95, 3, 249-254.
Haque, M.E. (2001). 3-D Visualization and Animation Techniques in Structural Design Education, the
International Conference on IT in Construction in Africa, CIBW78 Conference Proceedings, South Africa, 2.1-2.9.
Hutchinson T., Kuester F., Kim S., Lu R. (2003). Developing an advanced it-based visualization classroom for
enhanced engineering learning, International Conference on Engineering Education- Spain, 1-9.
Junior C. C., Okimoto F., Stamato G. C. and Pfister G, (2000). SET 613 - Formas de madeira para concreto armado,
Booklet (USP editor), University of So Paulo, School of Engineer of So Carlos.
Katheleen A., Flannery K. A., Krauchunas S. (2000). Virtual cognition lab: a constructivist approach toward
learning. 3rd Workshop on Virtual Reality in RGS -Brazil, Federal University of Rio Grande do Sul.
484

Kitware (2004). The VTK users guide: Updated for VTK Version 4.4, Kitware Inc., 332 p.
Lin, K., Son, J. and Rojas, E. (2011). A pilot study of a 3d game environment for construction safety education,
Journal of Information Technology in Construction - ITcon Vol. 16, 69 84.
Lopes, J. (1996). Jean Piaget, Magazine Revista Nova Escola, ano XI ,n. 95.
McLellan H. (1996). Virtual realities, Handbook of research for educational communications and technology (D,
Jonassen, editor), 457487.
Moraes, R. (2003). possvel ser construtivista no ensino de cincias? Construtivismo e ensino de cincias
reflexes epistemolgicas e metodolgicas, 2 edio, (EDIPUCRS editor), Pontifical University of Rio Grande do
Sul, Brazil, 103 130.
Newton, S. Lowe, R. Using an analytics engine to understand the design and construction of domestic buildings,
Proceedings of RICS - Construction and Property Conference - COBRA 2011, Manchester: RICS, 410-419.
Nikolic, D. Lee, S. Messner, J. I. Anumba, C. (2010). The virtual construction simulator: evaluating an educational
simulation application for teaching construction management concepts, Proceedings of the CIB W78 2010: 27th
International Conference in Egypt,1-10.
Pantelidis, V. (1998). Reasons to use virtual reality in education. Available at: http://eastnet.educ.ecu.edu/ vr/reas.html,
Accessed: 15/01/2011.
Pasqualotti A. (2000). Ambientes VRML para o ensino-aprendizagem de matemtica: modelo conceitual e estudo
de caso, Thesis (Master), Federal University of Rio Grande do Sul, Institute of Informatics, 99p.
Pompeu R. C. (1999). Um estudo sobre ambientes virtuais de apoio ao ensino e aprendizagem de resistncia dos
materiais, Thesis (Master), Federal University of Paran, Technology Sector, Graduate Program in Numerical
Methods in Engineering, 90p.
Rezende, R. B. (2010) Uma viso sobre o uso de frmas e escoramentos utilizados em cidades de grande, mdio e
pequeno porte do Brasil Central e as novas diretrizes normativas, Thesis (Master), Federal University of
Uberlandia, 164 p.
Sampaio, A.Z., Henriques, P.G. and Martins, O.P. (2008). Virtual Reality Technology Used in Civil Engineering
Education, The Open Virtual Reality Journal 2,18-25.
Seabra, R. D. and Santos, E. T. (2005). Utilizao de tcnicas de realidade virtual no projeto de uma ferramenta 3d
para desenvolvimento da habilidade de visualizao espacial, Revista Educao Grfica n. 9, Brazil, 111 122.
Schroeder W., Martin K. and Lorensen B. (2004). The Visualization Toolkit: an object-oriented approach to 3D
graphics 3rd. edition, Kitware Inc.
Shiratuddin, M. F. and Fletcher, D. (2007). Utilizing 3d games development tool for architectural design in a
virtual environment, 7th International Conference on Construction Applications of Virtual Reality, 20 - 27.
Tchne (2012). Available at: http://www.revistatechne.com.br/engenharia-civil/134/artigo89320-1.asp, Accessed:
25/03/2012.
Tribuna do Norte (2012). E-Journal Tribuna do Norte. Available at: http://tribunadonorte.com.br/noticia/
engenharia-civil-e-a-mais-procurada-por-alunos-de-ct/215237, Accessed in: 19/03/2012.
UNISANTA(2012). E-Journal of University of Santa Cecilia - UNISANTA online. Available at:http://www.online.
unisanta.br/2012/03-24/campus-4.htm, Accessed: 25/03/2012.
Veja(2011).Available at: http://veja.abril.com.br/noticia/economia/brasil-produz-19-milionarios-por-dia- aponta
-revista -forbes, Accessed: 15/01/2012.
Weiss, A. and Cruz, M.(2001). A informtica e os problemas escolares de aprendizagem, 3 edio (DP&A editor),
Rio de Janeiro, Brazil.
Yazig W. (2009). A tcnica de edificar - revisada e atualizada, 10ed., Editora Pini (editor), 770 p.
485

SURFING IN THE VIRTUAL DIGITAL LIBRARY A LINKED
DATABASE STRUCTURE FOR INTUITIVE INFORMATION-SEEKING
BEHAVIOR OF CHILDREN
Ko-Chiu Wu
National Taipei University of Technology, Taiwan
ABSTRACT: The information seeking behaviors are quite different between children and adults. Children are
more likely to use picture than text for their information-seeking. This research develops an icon-searching
interface and a spatial-linked structure for Children On-line Databases of National Tai-Chung Library. 33
databases in National Tai-Chung Library have been linked to the suggested database frame. The database frame is
classified in 5 major subject categories including Natural Sciences, Fine Arts and Music, Athletic, History and
Culture, and Treasure of Knowledge. The Natural Sciences consists of Science Knowledge, Exploring Ecology,
and Experiments of Natural Sciences. The Fine Arts and Music consists of Creation of Fine Arts, and Enjoying
Fine Arts. Athletic consists of only one sub-category, Athletic Competition. History and Culture consists of Native
Inhabitants, Historical Buildings, and Searching Your Origin. Treasure of Knowledge consists of Encyclopedia,
Literary Allusion, Multi-media Resource, Reading Collection, and E-learning. The web interface is designed
according to the way-finding concept and map-structure hierarchy. When children searches information in the
main frame designed, they move courser (the little red spaceman) right or left in the map to choose the icon which
represents different category, and then click to enter next frame. Child may surf through different icons which
represent sub-frames and linked database. When reach the final target, a small frame of chosen database will
jumped out in order to describe the detail database content. Statistical analysis shows that children prefer the
interface than the text-lined interface. This research develops an on-line interface and a database structure to
allow children searching information by maneuver in a virtual environment to support their studies. While spatial
memory and icon representation are helpful for children to recognize different categories of knowledge content, it
is important to further research the factors influence on childrens information-seeking patterns.
KEYWORDS: Information seeking behavior; Emotional informational navigation; Human-computer interaction.
1. INTRODUCTION
Although highly curious, children lack information search skills and require the customized interface of a digital
library to facilitate online searches. With many digital libraries established for children worldwide, the National
Taichung Library (NTL) also set up a digital library for children in Taiwan. The text-based interface for NTL
digital library for children has been transformed into a visualized interface, providing a virtual space integrated
with different icons connected to different databases. This study elucidates information seeking behavior of
children and the differences in navigational experiences between the two interfaces. While the children successful
passing rates and operating time are examined with the statistical analysis, the statistical results significantly
contribute to efforts to develop a better interface for emotional information retrieval.
2. RELEVANT LITERATURE
2.1 Children information-seeking behaviors and interfaces for children digital library
Solomon (1993) examined the factors affecting information retrieval behavior of children while using OPAC,
including user characteristics, school setting, interface usability, and information access feature for children. While
comparing the web search patterns of children and graduate students, Bilal and Kirby (2002) found that the latter
scored markedly higher than the former in searching effectiveness, efficiency, and quality of their Web
navigation. Graduate students were also more proficient than children in recovering from search related
breakdowns. Hutchinson, Druin, Bederson, Reuter, Rose and Weeks (2005) examined the errors and frustrations
of young digital library users. Hutchinson, Rose, Bederson, Weeks and Druin (2005)

discussed the limitations of
childrens information seeking skills, in which they occasionally could not spell, type, read, navigate, compose
queries, and/or select small objects. Children could thus not search for books using the same criteria as adults.
Actually, children have difficulty in manipulating and understanding conventional interface tools, and often adopt
various searching and browsing strategies from adults. Hutchinson, Druin and Bederson (2007) then designed a
category browser with Boolean queries, as well as a flat, simultaneous interface to help children in seeking
486

information in the International Children Digital Library (ICDL). Bilal (2002) suggested that emotions can
profoundly impact the overall effectiveness of children who used a Web search engine to perform fully
self-generated search tasks, when calling for teachers of information literacy skills, in order to incorporate the
affective aspects of information retrieval in their instruction model. Blom and Monk (2003) suggested that users
personalize their computers for emotional reasons, including improved sense of control, ownership, fun, and relief
from boredom. Zhang (2008)

illustrated students mental models of search on the Web from three perspectives:
acquiring information, understanding the working mechanisms of search engines, and devising search tactics.
Lopatovska and Arapakis (2011) developed affective systems, capable of recognizing and appropriately
responding to human emotions and, ultimately, making humancomputer interaction (HCI) experiences more
effective and pleasurable.

2.2 Information visualization and context-related HCI interface
As literacy skills of children advance gradually up to adolescence, children of varying ages adopt different
strategies of text combined with images to help them retrieve information. Interestingly, children play at a
fun-oriented, information visualized interface, while searching through the icons in a context-related Human
Computer Interface to retrieve accurate information. Shneiderman (1997) identified information visualization and
user interface development as two important factors for developing window management. Kulper, Ute, Schulz and
Ursula (1997) developed BUCHERSCHATZ, an OPAC for children with a graphical user interface, and a
metaphor to raise childrens curiosity. Bates (2002)

addressed context issues related to interface design. Adequate
design requires sufficiently understood interactions and proper design for a specific information system. However,
Bilal and Bachir (2007)

stressed the importance of representations and meaning of information search for
non-native English speaking children while using ICDL, in which older children perform better than younger
ones. Their work focused on incorporating meaning and representation recognized by children to allow them to
search for information in a visualized interface. Hu, Ma and Chau (1999) suggested that interface design may
significantly affect systemuser concept communication, regardless of user familiarity with the search task; in
addition, a graphical user interface may be more effective in facilitating such communication than a list-based
design. The interfaces incorporating multiple visual properties may be more effective in supporting concept
communication than those relying exclusively on a visual property. To use of graphical interfaces may lead to a
higher level of user satisfaction than text-based linear lists, although both may be comparable in cognitive load
requirements. Interface-induced user satisfaction may become increasingly prominent when the design includes
multiple visual properties. The improvements in interface-induced system-user concept communication may
increase with the number of system-suggested items, yet decrease with user task familiarity. Furthermore,
although size and distance may provide many effective and essential visual cues in systemuser concept
communication, color may be less effective for such communication support. Several studies have assessed
systems that communicate effect by incorporating emotionally expressive computer agents. Leide, Large,
Beheshti, Brook and Cole (2003) mapped out the encounter of domain novice user with an IR system, from
beginning to end, to incorporate adequate classification-based visualization schemes. Tzeng (2004) examined
effects of apologies for the failed computer game expressed through text and emoticons on user. Apologetic
(sad) emoticons were found to communicate emotions more effectively than pure text. Brave, Nass and
Hutchinson (2005) examined how emotionally expressive computer agents affect user perceptions of computer
game. Empathetic agents were perceived as likeable, carrying, trustworthy, supportive, and beneficial to
participants. In addition to reviewing some of the human perceptual theories behind the graphical interface of
information visualization systems, Koshman (2006) analyzed iconic representations and information density on
visualization displays, as well as examined information retrieval tasks performed in visualization system user
research. Schmidt and Stock (2009) utilized a similar method for tagging images using tags for basic emotions:
anger, disgust, fear, happiness and sadness. Hornbk and Hertzum (2011) examined the major concepts involving
the overview and provided an overview of information visualization research. While above studies suggest
collective emotion tagging, and information visualized operations to describe and retrieve information, related
factors must be identified to understand the affective state of children for emotional behaviors associated with
information retrieval.

487

3. MEASUREMENT
A visualized interface prototype was designed, and revised by interviewing children, with those results evaluated
by experts. The study was performed at the National Taichung Library (NTL) for 104 children to compare how the
NTL original text-based interface and the visualized interface prototype differ from each other, in order to
understand their experiences with interface usability.

3.1 Visualized Interface Design
NTL has 33 databases linked together in a new visualized interface, including Copernicus 21 Sciences Knowledge
database, Taiwan Amphibious Reptile video database, Taiwan Ecology Notebook database, Taiwan Fish Learning
Knowledge database, Taiwan Bird video database, Newton Textbook-Nature science Learning Treasure -
Experimental Part, Insect Picture Encyclopedia, The Scientist Magazine Chinese-English Knowledge database,
Paper Festival Series - Interactive Multi-media Net database, Tsai Chih-Chung Caricature and Animation database,
Chinese Poem database, Collection of Historical Calligraphy, Cultural Sound-picture Digital Database, World Fine
arts database, The Palace Museum On-line database, The International Athletic and Competition Animation
Special Reports, Taiwan Native Learning database, Taiwan Historical Building Learning database, Taiwan
Origin-seeking database, Taiwan Hundreds of Events database, Chinese Historical Building database, British
Encyclopedia On-line Chinese version, Chinese Encyclopedia database, Old Teacher E-learning Idiom- caricature
database, The Sound-picture On-line for K-9 Digital Learning curriculum, Charng-Jinn Multi-media Digital book,
Little Kiss Digital book, Discovery Digital Curriculum Teaching Video database, English DNA Bi-linguistic
-teaching Post, and Tumble Talking Books.

The database structure is classified into 5 major subject categories: Natural Sciences, Fine Arts and Music,
Athletics, History and Culture, and Treasure of Knowledge. The Natural Sciences category links to lower-level
icons of Science Knowledge, Exploring Ecology, and Experiments of Natural Sciences. The Fine Arts and Music
links to lower-level icons of Performance Arts, Visual Arts, and Music. Athletics links to the lower-level icons of
Human and Food, and Athletic Competition. History and Culture links to the lower-level icon of Taiwan History,
Native Inhabitants, Historical Buildings, and Social Knowledge. Treasure of Knowledge links to lower-level icons
of Encyclopedia, Chinese Literary Allusion, Multi-media Resource, English Reading Collection, Children Story
Collection, and Native Language E-learning. The visualized interface is designed according to the way-finding
concept and map-structure hierarchy. Figure 1 illustrates the main frame designed in this study. When searching
for information, a child may move the cursor (i.e. a little red spaceman) right or left on the map to select the icon
that represents a different category, and then click to enter the next frame. A child may surf through different
icons that represent sub-frames and linked database. While the final target is reached, a small frame of the chosen
database appears to describe details of the database content.

Fig. 1: Visualized Interface for Childrens On-line Database of National Tai-Chung Library.

488

3.2 Hypotheses for Interface Comparison
Whether the visualized interface performs better than the original text-based interface for children searching for
information must be determined. We thus hypothesize the following:
H1: Children, regardless of different grades, operate two interfaces with different success rates.
H2: Children, regardless of different grades, who succeed in the information search, operate two interfaces with
different times.
H3: Children, regardless of different computer usage experiences, operate two interfaces with different success
rates.
H4: Children, regardless of different computer usage experiences, who succeed in the information search, operate
two interfaces with different times.
Two missions were set for children searching for information. The first mission is as follows:
You encounter a problem while doing homework, and teacher told you, you can see "online Taiwan Historical
Dictionary database". Now, you are trying to find "online Taiwan Historical Dictionary of databases".
The second mission is as follows:
The teacher instructed the children to look at the "green sour bear picture books database" after school, which
may offer many children books for leisure reading.

4. ANALYSI S
4.1 Comparison of visualized and text-based linear lists in interface
In Table 1, mission 1 contains 1-2 grade (**p=.006<.05), 3-4 grade (**p=.000<.05), and 5-6 grade (**p=.003<.05),
which displays a statistical significance difference, regardless of children in different grades, between the
original text-based interface and the visualized interface. The success rates for mission 1 of children of different
grades use the visualized interface, which are superior to the original text-based interface. Mission 2 contains 1-2
grade (**p=.000<.05), 3-4 grade (**p=.000<.05), and 5-6 grade (**p=.000<.05), which shows a statistically
significant difference, regardless of children in different grades, between the original text-based interface and the
visualized interface. The success rates for mission 2 of children of different grades use the visualized interface,
which are also superior to the original text-based interface. Hypothesis H1 is thus validated.
Table 1. Comparison of successful rates between visualized interface and text-based linear lists interface
Mission 1 N=104
Visualized interface Text-based linear lists interface Significance

Failure Success Successful rate Failure Success Successful rate

1-2 grade 6 6 50.00% 11 0 0.00% 0.006 **
3-4 grade 6 19 76.00% 16 5 23.81% 0.000 **
5-6 grade 2 11 84.62% 15 7 31.82% 0.003 **
Mission 2 N=104
1-2 grade 2 10 83.33% 11 0 0.00% 0.000 **
3-4 grade 3 22 88.00% 18 3 14.29% 0.000 **
5-6 grade 0 13 100.00% 17 5 22.73% 0.000 **
489

For children passing the information search successfully, they vary in the time they required to pass the two
interfaces. In Table 2, mission 1 contains (**p=.005<.05), which displays a statistically significant difference,
regardless of children in different grades, between the original text-based interface and the visualized interface.
Mission 2 contains (**p=.000<.05), which displays a statistically significant difference, regardless of children in
different grades, between the original text-based interface and the visualized interface. The average times that
children in different grades require to pass the visualized interface are less than those for the original text-based
interface. Hypothesis H2 is thus validated.

Table 2. Comparison of searching time between visualized interface and text-based linear lists interface for
children
Mission 1 N=48

Mean Std. Dev. Number Mean Std. Dev. Number

1-2 grade 59.33 15.397 6 - - -
3-4 grade 40.11 20.981 19 61.00 11.640 5
5-6 grade 38.64 18.559 11 64.71 13.425 7
average 42.86 20.370 36 63.17 12.298 12 .005**
Mission 2 N=53
1-2 grade 44.50 20.239 10 - - -
3-4 grade 37.36 19.817 22 79.67 9.609 3
5-6 grade 35.08 17.090 13 67.20 13.255 5
38.29 19.056 45 71.88 12.977 8 .000**

Table 3. Comparison of successful rates between visualized interface and text-based linear lists interface for
children with different usage experiences
Mission 1 N=104

Failure Success Successful rate Failure Success Successful rate

Low-experience 14 28 66.67% 35 6 14.63% 0.000 **
High-experience 0 8 100.00% 7 6 46.15% 0.011 *
Mission 2 N=104
Low-experience 5 37 88.10% 38 3 7.32% 0.000 **
High-experience 0 8 100.00% 8 5 38.46% 0.005 **

In Table 3, mission 1 contains low experience (**p=.000<.05) and high experience (*p=.011<.05), which
displays a statistically significant difference, regardless of children with different usage experiences, between the
original text-based interface and the visualized interface. The success rates for mission 1 of children of different
usage experiences for the visualized interface are superior to those for the original text-based interface. Mission 2
490

contains low experience (**p=.000<.05) and high experience (**p=.005<.05), which displays a statistical
significant difference, regardless of children with different using experiences, exists between the original
text-based interface and the visualized interface. The success rates for mission 2 of the children of different usage
experiences that use the visualized interface are superior to those for the original text-based interface. Hypothesis
H3 is thus validated.

Table 4. Comparison of searching time between visualized interface and text-based linear lists interface for
children with different usage experiences
Mission 1 N=48

Mean Std. Dev. Number Mean Std. Dev. Number

Low-experience 43.57 21.923 28 66.50 13.939 6
High-experience 40.38 14.550 8 59.83 10.572 6
average 42.86 20.370 36 63.17 12.298 12 .021*
Mission 2 N=53
Low-experience 40.19 19.379 37 58.33 7.095 3
High-experience 29.50 15.611 8 80.00 7.036 5
average 38.29 19.056 45 71.88 12.977 8 .000**
For children passing the information search successfully, they vary in the time they required to pass the two
interfaces. In Table 4, mission 1 contains (*p=.021<.05), which displays a statistically significance difference,
regardless of children with different using experiences, between the original text-based interface and the
visualized interface. Mission 2 contains (**p=.000<.05) which shows a statistically significant difference,
regardless of children with different usage experiences, between the original text-based interface and the
visualized interface. The average times that children with different usage experiences require for passing the
visualized interface are less than those for the original text-based interface. Hypothesis H4 is thus validated.

5. CONCLUSIONS AND RECOMMENDATIONS
Children exhibit different information seeking behaviors through different media-type interfaces, allowing them to
perform in different affected states. This study developed a customized interface of digital library for NTL to help
children overcome obstacles impeding their information seeking behavior. Comparing the existing text-based
interface and the new visualized interface in terms of child usage behavior reveals that the latter is preferred to the
former. However, further researches for children information-seeking behaviors are need for enhancing the
interface usability related to the presentation of visualized information. The statistical confirmatory analysis shall
be conducted to elucidate the relationship among latent variables as children's information seeking for changing
moods, children seeking information to solve problems, text-sound media-type interface, graphic-icon media-type
interface, affective state-Uncertainty, and affective state-Easiness in use. While children enjoy using the NTL
visualized interface for fun, statistical results provide a valuable reference for efforts to develop an improved
interface for emotional information retrieval.
6. ACKNOWLEDGMENTS
The authors would like to thank the National Science Council of the Republic of China, Taiwan for financially
supporting this research under Contract No. NSC 99-2515-S-027 -004.

491

7. REFERENCES (STYLE HEADING 2)
Bates, M. J. (2002). The cascade of interactions in the digital library interface. Information Processing and
Management, 38(3), 381-400.
Bilal, D. (2002). Childrens use of the Yahooligans! Web search engine. III. Cognitive and physical behaviors on
fully self-generated search tasks. Journal of the American Society for Information Science and Technology, 53(13),
11701183. doi: 10.1002/asi.10145
Bilal, D., & Bachir, I. (2007). Childrens interaction with cross-cultural and multilingual digital libraries: I.
Understanding interface design representations. Information Processing and Management, 43 (1), 4764.
Bilal, D., & Kirby, J. (2002). Differences and similarities in information seeking: children and adults as Web users.
Information Processing & Management, 38(5), 649670. doi:10.1016/S0306-4573(01)00057-7
Blom, J. O., & Monk, A. F. (2003). Theory of personalization of appearance: Why users personalize their PCs and
mobile phones. HumanComputer Interaction, 18(3), 193228.
Brave, S., Nass, C., & Hutchinson, K. (2005). Computers that care: Investigating the effects of orientation of
emotion exhibited by an embodied computer agent. International Journal of HumanComputer Studies, 62 (2),
161178.
Hornbk, K., & Hertzum, M. (2011). The notion of overview in information visualization. International Journal
of Human-Computer Studies, 69(7-8), 509525.
Hu, P.J., Ma, P., & Chau, P. Y. (1999). Evaluation of user interface designs for information retrieval systems: a
computer-based experiment. Decision Support Systems, 27 (1-2), 125143.
Hutchinson, H., Druin, A., Bederson, B. B., Reuter, K., Rose, A., & Weeks, A.C., (2005). How do I find blue books
about dogs? The errors and frustrations of young digital library users. In Proceedings of the 11th International
Conference on Human-Computer Interaction (HCII 2005) (CD-ROM). Mahwah, NJ: Lawrence Erlbaum
Associates. (Tech Report HCIL-2005-27). Retrieved from University of Maryland, Human-Computer
Interaction Lab website: http://hcil.cs.umd.edu/trs/2005-27/2005-27.pdf
Hutchinson, H., Rose, A., Bederson, B. B., Weeks, A.C., & Druin, A. (2005). International childrens digital
library: a case study in designing for a multilingual, multicultural, multigenerational audience. Information
Technology and Libraries, 24(1), 4-12.
Hutchinson, H.B., Druin, A., & Bederson, B. B. (2007). Supporting Elementary-Age Childrens Searching and
Browsing: Design and Evaluation Using the International Childrens Digital Library. Journal of the American
Society for Information Science and Technology, 58(11), 16181630.
Leide, J. E., Large, A., Beheshti, J., Brooks, M., & Cole, C., (2003). Visualization schemes for domain novices
exploring a topic space: the navigation classification scheme. Information Processing and Management, 39(6),
923940.
Lopatovska, I., & Arapakis, I. (2011). Theories, methods and current research on emotions in library and
information science, information retrieval and humancomputer interaction. Information Processing and
Management, 47(4), 575592.
Koshman, S. (2006). Visualization-based information retrieval on the Web. Library & Information Science
Research, 28(2), 192207.
Kulper, U., Schulz, U., & Will G. (1997). Bucherschatz - A prototype of a childrens OPAC. Information Services
& Use, 01675265, 17(2/3), 201-214.
Schmidt, S., & Stock, W. G. (2009). Collective indexing of emotions in images. A study in emotional information
retrieval. Journal of the American Society for Information Science and Technology, 60(5), 863876.
doi:10.1002/asi.21043
Shneiderman, B. (1997). The next generation of graphical user interfaces: information visualization and better
window management. Displays, 17(3-4), 125-129.
492

Solomon, P. (1993). Children's Information Retrieval Behavior: A Case Analysis of an OPAC. Journal of the
American Society for Information Science, 44(5), 245-264.
Tzeng, J.I. (2004). Toward a more civilized design: Studying the effects of computers that apologize. International
Journal of HumanComputer Studies, 61(3), 319345.
Zhang Y. (2008). Undergraduate Students Mental Models of the Web as an Information Retrieval System. Journal
of the American Society for Information Science and Technology, 59(13), 20872098.

493
AUTOMATED 3D HUMAN SKELETON EXTRACTION USING RANGE
CAMERAS FOR SAFETY ACTION SAMPLING
SangUk Han
University of Illinois at Urbana-Champaign, USA
Madhav Achar & SangHyun Lee
University of Michigan, USA
Feniosky Pea-Mora
Columbia University, USA
ABSTRACT: About 80%-90% of construction incidents are caused by workers unsafe actions. For measuring
and understanding workers behavior, a vision-based motion capture technique has recently been applied in
construction. The technique uses video cameras to record workers, extracts 3D human skeletons from the videos,
and detects similar motions with prior motion models that represent pre-defined unsafe actions. In this process,
our previous study revealed that the accuracy of the detection highly relied on the quality of prior models; thus,
we suggested building prior models through the thorough observation of actual workers actions. In the previous
study, however, we tested the proposed approach using prior models collected from a laboratory where multiple
cameras (e.g., 6 or 8) were required to track a performers movement, and the performer wore a special suit with
reflective markers attached to his/her body joints. This of course brings up practical issues for the approachs
application to construction workers on a real jobsite. To address the issues, range cameras (i.e., Kinect) are used
in this paper to extract 3D skeleton models with a smaller number of cameras (i.e., 1 or 2), and no marker
attachments are needed. The range cameras can provide depth information to efficiently create 3D skeleton
models. In our experiment, prior models for actions during ladder climbing are collected with 1 and 2 range
cameras. Finally, the results of motion capture and analysis are compared to the ground truth (i.e., the result
from a commercial motion capture system) to evaluate the performances. The results show that using either 1 or
2 range cameras offers a good performance in terms of detecting similar actions in motion data from videos.
Range cameras thus have great potential to be used for gathering actual workers motion data for prior models.
KEYWORDS: Safety, worker behavior, vision-based motion tracking, range camera, machine learning, motion
recognition
1. INTRODUCTION
The construction industry is one of the most at-risk fields; employees are consistently and greatly exposed to
work-related injuries. In the United States in 2005, the fatality and non-fatality rates in construction were 11.1
and 239.5 per 100,000 full-time workersthe fourth highest rates after agriculture, mining, and transportation;
and the second after transportation, respectively (CPWR 2008). Other countriesfor instance, the United
Kingdom, Hong Kong (China), Taiwan, and Japanalso had similar accident trends in 2005; fatality rates for
the four countries in the construction industry were 3.5, 42.2, 17.2, and 11.0, respectively (Chen et al. 2011). In
particular, about half of the workplace accidents in Taiwan occurred in construction between 2000 and 2006
(Chen et al. 2011). Although construction accidents are associated with various safety elements (e.g.,
management procedures, safety culture, worker attitude), it is obvious that workers unsafe actions eventually
lead to accident occurrences, considering that about 80%-90% of accidents are caused by the unsafe actions and
behaviors of construction workers (Heinrich et al. 1980; Helen and Rowlinson 2005). In this respect, worker
monitoring can play a key role in measuring workers behavior and in understanding which factors affect
behavior in terms of safety. According to Han et al. (2012b), however, too much time and effort is required to
collect consistent and reliable behavior data from a jobsite, and this makes it difficult to observe workers
behavior in practice.
For measuring and understanding workers behavior, a vision-based motion capture technique has recently been
applied in construction (Li and Lee 2012; Han et al. 2012b). The technique uses video cameras to record workers,
extracts 3D human poses from the videos, and re-produces human skeleton models in a 3D virtual environment.
In order to detect the same actions, the skeleton models are then compared with motion priors, which are pre-
defined unsafe actions, and which have been built in advance through experiments. In this process, our previous
study (Han et al. 2012a) revealed that the accuracy of any given detection highly relied on the quality of the prior
494
models; this is because detection is carried out by measuring the similarity between motion priors and recovered
3D pose data. To improve accuracy, we thus suggested building prior models through the thorough observation
of actual workers actions. In the previous study (Han et al. 2012a), however, we tested the proposed motion
recognition approach using prior models collected from a laboratory where multiple cameras (e.g., 6 or 8) were
required to track a performers movement, and the performer wore a special suit with reflective markers attached
to his/her body joints. This brings up practical issues for collecting motion priors from a construction jobsite, as
the motion capture system used in the experiment requires a controlled environment.
As an alternative means to build motion priors, this paper proposes the use of recently released Microsoft Kinect
motion sensors, and evaluates their performance with respect to different configurations (i.e., a single Kinect and
dual Kinects). The Kinect-based motion capture solution does not track reflective markers attached to a human
body, but does automatically estimate the 3D locations of body joints using a depth sensor. In this way, workers
do not need to wear any special suit or markers, and hence the proposed approach facilitates the acquisition of
motion data from a jobsite without disturbing workers movements. Moreover, the solution supports utilizing
multiple sensors, as well as a single sensor, for motion capture. The use of multiple sensors helps address issues
of occlusion by construction tools (e.g., ladder) and self-occlusions during an activity, while fewer constraints
and less cost in an experiment setting are demanded when using a monocular sensor. In this paper, the 3D human
skeleton models that result from two configurations (i.e., a single Kinect and two Kinects) are compared with the
results obtained by a commercial motion capture system, Vicon, well known to be very accurate and used as the
ground truth in previous studies (Dutta 2011; Stone and Skubic 2011; Fernndez-Baena et al. 2012). In addition,
motion detection results for the two configurations are evaluated to validate the proposed approach to motion
data sampling for unsafe action detection.
The rest of this paper is organized as follows. Section 2 provides background on an application of vision-based
motion tracking and analysis for construction safety. Section 3 demonstrates a research methodology used to
evaluate and compare the Kinect-based motion capture between two configurations. Section 4 describes the
experiment conducted to collect motion capture data. Results, including 3D pose estimation and motion analysis,
are presented and discussed in Section 5. Finally, Section 6 summarizes the findings of this study and suggests
the direction of future research.
2. BACKGROUND
In this section, the vision-based approach to safety behavior analysis, proposed in our previous studies (Han et al.
2012a; Han et al. 2012b; Li and Lee 2012), is summarized, and the issue involved in motion data sampling is
demonstrated.
The approach is motivated by behavior-based safety processes (Krause 1990; McSween 2003), which have been
found to improve workers behavior and dramatically reduce incident rates. In the processes, critical unsafe
behaviors are identified based on accident records, workers are trained to observe the identified behaviors, and
their behavior is positively modified by providing feedback based on observations (Krause 1990; McSween
2003). Similarly, the proposed vision-based approach aims to detect pre-defined critical unsafe actions from site
video recordings of construction workers. For the detection, critical unsafe actions involved in construction
activities are formalized, and motion priors associated with these unsafe actions are collected and built through
experiments. The resulting priors take the form of motion capture data (e.g., BVH format), which contain
angular information between body joints. Motion data, which take the same format as the priors, are also
extracted from site videos by estimating the locations of body joints on sequential 2D images and recovering the
3D poses using images from two different views. Then, the motion prior that represents an unsafe action is used
as a template to measure the distance (i.e., similarity) between the template and the motion data from a jobsite
using Dynamic Time Warping (DTW) (Okada and Hasegawa 2008). The distance decreases when similar actions
occur, and increases when the motion variation is larger; hence, theoretically, the distance becomes zero when
the exact same action as the prior model (i.e., the same variation in joint angles over time) occurs. This implies
that the accuracy of the detection can be significantly improved by using motion priors close to the actual actions
that most likely happen on a jobsite.
In this regard, an effective way to build robust motion priors can be to collect the priors from a jobsite rather
than a laboratory. Particularly, motion priors from a worker, who will be observed, may potentially perform well
at detecting the workers actions, in that the effect of movement patterns varying from individual to individual
can be minimized. This approach is feasible as our application is confined to a construction site, where people to
be monitored are not arbitrary but already known; that is, the number of employees on a site is limited and
495
traceable (Han et al. 2012b). Accordingly, it is realizable and ideal to take motion samples from a jobsite, and to
use them as priors for further motion analysis.
For data acquisition, a motion capture system mainly needs to meet the following requirements: (1) it has no
influence on workers movements, (2) it ensures sufficient portability to adjust experimental setup to a
construction site, and (3) it provides accurate outputs of motion tracking. Among currently available motion
capture systems (e.g., mechanical, magnetic, optical systems), a Kinect sensor-based system is regarded as the
most suitable for this application. Unlike other systems, the Kinect motion capture does not require any marker
or wearing of devices to track the locations of body joints. Combined with motion capture solutions (e.g., iPi
Motion Capture, Brekel Kinect), the Kinect system automatically generates human skeleton models, which is the
same type of output that other systems produce. Moreover, a Kinect depth sensor allows the creation of a 3D
skeleton model with only a single camera. Using a structured infrared laser (i.e. a grid), the Kinect sensor
computes the distortion of the scene on an infrared image and measures the depth through disparity retrieval
(Khoshelhan 2011). Compared to a commercial optical system (e.g., Vicon) that uses multiple cameras (e.g., 6 or
8), a Kinect thus has fewer constraints on experimental conditions, thus making it easy to conduct an experiment
on a site. Lastly, a Kinect sensor accurately measures the depth of the scene at its designed operating ranges (e.g.,
0.8-4m). Khoshelhan (2011) compared the point cloud generated by a Kinect with the one from a laser scanner,
and found that a Kinect depth sensor performs well within the designed distance. This paper thus tests motion
datasets collected using Kinect sensors, and evaluates the potential for motion data sampling on a site.
The main objective of this paper is to assess the accuracy of the Kinect for motion analysis, and to develop
experiment suggestions that can be implemented on a jobsite in order to build strong motion priors. To fulfill the
objective, an experiment that tests one-Kinect and two-Kinect systems is conducted in the University of
Michigan 3D lab.
An available motion capture solution such as iPi Desktop Motion Capture (www.ipisoft.com) supports the use of
both one and two Kinect settings for recording a performer. On a jobsite, a cameras line of sight is often not
secured by construction tools and devices, and hence one can take advantage of multiple cameras for occlusions.
In this experiment, two Kinects at various angles are tested to find the optimal installation arrangements of
Kinects in order to improve accuracy.
As a test case, actions involved in ladder climbingascending and descendingare simultaneously recorded
and captured with a single Kinect, two Kinects, and a Vicon. Then, two different structures of skeleton models
provided by the Kinect and Vicon are incorporated into one single model by computing the locations of body
joints and re-calculating the angles of the corresponding joints. The resulting datasets are compared in terms of
angles between body joints in order to evaluate performances regarding the settings. Joint angles are effective
parameters to describe human motions, and the locations of body joints can be easily computed using the angles,
given the size of bones. For motion analysis, in particular, the issue of body size can be avoided by using the
angular information.
To validate the reliability of the sample priors for the motion analysis proposed in our previous study (Han et al.
2012a), we perform dimension reduction for the high-dimensional datasets, using Kernel Principal Component
Analysis (Kernel PCA) (Schlkopf et al. 1998), and compare distributions of the transformed datasets in a 3D
latent space. In the low-dimensional coordinate, the distance between a dataset from a Vicon and the ones from a
single or dual Kinects is also measured with the DTW (Okada and Hasegawa 2008) to evaluate the similarity of
the datasets, which determines the accuracy of motion detection. Eventually, datasets from the three different
systems are used as priors for motion detection, and the detection results are compared.
4. EXPERIMENT
In the experiment, the iPi Motion Capture solution was used with Kinect sensors to capture human motions. The
resulting motion capture data was the BVH format that defined a hierarchical structure of body parts. It
contained the positions and postures of the performer changing over. Human motion could thus be reproduced
and animated in a 3D virtual environment, as shown in Figure 1. A Vicon, used as the ground truth, also yielded
the BVH file; thus, the results from both systems could be easily compared to evaluate their performances.
496

Figure 1: A screenshot of motion capture data in a 3D virtual environment (iPi Desktop Motion Capture).
To determine the positions of two Kinects, the following three configurations were tested to investigate their
impact on the motion capture accuracy: (1) Kinects positioned between 30-90 degrees, facing the back of a
performer; (2) Kinects positioned between 90-135 degrees, facing the side-back of a performer; and (3) Kinects
positioned between 135-180 degrees, facing the side of a performer. On a construction site, a ladder is often put
against a wall, so front views were not considered in this experiment. Videos for each configuration were
recorded and visually analyzed with the motion capture solution. As an example, the snapshots of each
configuration are presented in Figure 2.

Figure 2: Snapshots for the three configurations: (a) and (b) 30-90 degrees, (c) and (d) 90-135 degrees, and (e)
and (f) 135-180 degrees.
497
In the experiment, the dual Kinect setup that was between 135 and 180 degrees facing the ladder from the side
was found to be more likely to avoid self-occlusions by the performer. The increase in accuracy could be visually
seen when the video was processed in the iPi solution. This was expected since the side view provided additional
depth information concerning the arms, which were occluded in the other two dual Kinect setups. Accordingly,
motion data were collected using two Kinects positioned at 170 degrees, as well as a Vicon and a single Kinect,
as illustrated in Figure 3.

Figure 3: Figure 3: Camera configurations: (a) a Vicon, (b) a single Kinect, and (c) dual Kinects
To evaluate the accuracy, actions during ladder climbing were simultaneously recorded with a single Kinect, dual
Kinects, and a Vicon. The actions consisted of ascending and descending, and each action was repeated 25 times.
In total, the Kinects yielded 3,136 frames, while the Vicon produced 12,544 frames for the same scene; the frame
rates of the Kinects and Vicon were 120 and 30 frames per second, respectively. The datasets were thus
synchronized so as to have the same number of frames (i.e., 3,136) for the performance comparison.
The motion capture data that were extracted from the Vicon and Kinect systems, however, had different
hierarchical structures of a human body (i.e., the number and locations of body joints). For example, the Kinect
system estimated the locations of 22 body joints (Figure 4.a), while the Vicon system tracked 25 reflective
markers attached to body joints (Figure 4.b). Furthermore, the base poses of two systems did not correspond.
Rotation angles stored in motion capture data were based on the base pose; the angle between the base pose and
the recorded pose is computed and stored in the file. To address these issues, 3D locations of body joints for both
systems were computed using the rotation angles (Meredith and Maddock 2001), and the rotation angles were
calculated with respect to a newly created body skeleton model (Figure 4.c). These converted datasets were
eventually used to compare the accuracy of the systems.

Figure 4: Human skeleton models of (a) a Kinect and (b) a Vicon, and (c) a converted skeleton model.
5. RESULTS
The motion capture datasets from the three systems are compared in terms of the accuracy of motion capture and
the quality of the data for motion analysis. For the former, this paper compares the angles stored in the datasets;
for the latter, detection errors when using each of the datasets as a prior model are computed and evaluated.
498
5.1 Angular Validation
The converted datasets based on the new skeleton model (Figure 4.c) have 3D rotation angles for 12 body joints
(i.e., hip, left and right up-legs, legs, shoulders, etc.), and those angles in Kinect datasets are compared with the
ones in a Vicon dataset; a hip angle is excluded from the comparison since the angle refers to the orientation of
the performer and may differ from system to system in defining a global coordinate. For the accuracy
comparison, the Root Mean Square Errors (RMSE) are computed as follows;
RMSE =
where x
v
is an angle in Vicon data, x
k
is an angle in Kinect data, and the total number of frames, n, is 3,136. The
results are summarized in Table 1.
Table 1: Joint angle comparison (degree) of a single Kinect and dual Kinects with a Vicon
Kinect
Left
up-leg
Left
leg
Right
up-leg
Right
leg
Neck
Left
shoulder
Left
arm
Left
forearm
Right
shoulder
Right
arm
Right
forearm
Mean
Single 5.14 5.59 6.17 8.18 4.41 13.68 13.94 34.22 15.84 18.94 48.98 15.92
Dual 5.71 6.89 6.72 7.94 2.81 13.91 14.68 49.72 13.10 25.45 57.93 18.62

Compared to the Vicon, which was used as the ground truth, the average differences of 15.92 and 18.62 degrees
were found for the single and dual Kinect settings, respectively. In particular, the results show that the errors
were caused mostly by forearms and handsthe angle in forearms refers to the location of hands. In the
experiment, Kinects were positioned at the rear side of the performer, and hence the forearms and hands were
frequently occluded by the torso. Consequently, this occlusion negatively affected the overall accuracy of the
motion capture. Interestingly, the results also reveal that, in our experiment, the single Kinect slightly
outperformed the dual Kinects even though two Kinects certainly secure a clearer view. This implies that the
self-occlusion was not fully recovered from both side views; for example, arms bent inwardly are not seen from
either side. We also presume that the body joint locations that the iPi solution estimated from the rear view (or
the front view) may be closer to the locations of the reflective markers for the Vicon. Basically, a Vicon tracks
the marker and computes the 3D locations of the markers, while an iPi solution estimates the locations of body
joints and directly measures the depth. Thus in the case of the Kinects, the estimated locations may vary slightly
based on the viewpoints of the sensors, and this small discrepancy may lead to the inaccuracy of the dual-Kinect
setting. To confirm the cause, though, further investigation is needed. Overall, the errors are less than 10 degrees,
except the case of occlusions, and the impact of this occlusion on motion analysis is examined in the next section.
5.2 Performance Evaluation for Motion Analysis
For motion analysis, the high-dimensional motion datasets that had 33 variables were mapped onto a 3-
dimensional space using the kPCA, and the data similarity between Vicon and Kinect settings was quantitatively
measured using the DTW. Figure 5 illustrates the distribution of the transformed datasets in the space, and the
similarity measurements are presented in Table 2.

Figure 5: Mapping of motion datasets onto a 3D space: (a) a Vicon, (b) a single Kinect, and (c) dual Kinects.
499
Table 2: Similarity measurements between Vicon and two Kinect settings, using DTW
Similarity Single Kinect Dual Kinects
Vicon 0.96 (10
6
) 1.07 (10
6
)

As shown in Figure 5, the mapped datasets of the three settings have a similar distribution in the latent space,
and this indicates that posture information in motion data is well preserved throughout the mapping. Indicated, as
well, is that the transformed Kinect datasets have similar posture information as the Vicon. In addition, the
similarity measures (Table 2), which determine how similarly two datasets are distributed over time, reveal that
the data from a single Kinect has a smaller distance (i.e., is more similar) to the one from a Vicon; thus, a single
Kinect is regarded as providing a more accurate result, closer to a Vicons one as in angular validation.
To validate the use of Kinect datasets for motion analysis, a prior from each setting was used for motion
detection. The distance between the prior (i.e., ascending and descending action) and a subset of a testing dataset
(i.e., Vicon data) was measured over time (Figure 6), and the same actions could be recognized by searching for
the ones with a small distance. Each of the ascending and descending actions occurred 25 times in the testing
dataset, and the distance decreased when the action occurred. The results show that the distance in the three
settings similarly fluctuates over time, and from this we can infer that the Kinect provides reliable data to be
used for the proposed motion analysis. The detection errors, shown in Table 3, also support the high quality of
Kinect data as a prior model. Compared to the Vicon, only one and two not-detected cases were found when
using single and dual Kinect data as a prior, respectively.

Figure 6: Measured distance comparisons among Vicon, single Kinect, and dual Kinect settings: (a) ascending,
and (b) descending actions.
500
Table 3: Detection error comparison.
Template
source
Action type
# of actions
in data
# of correctly
detected actions
# of incorrectly detected actions
Not detected Mis-detected
Vicon
Ascending 25 25 (100%) 0 0
Descending 25 25 (100%) 0 0
Single
Kinect
Ascending 25 24 (96%) 1 0
Descending 25 25 (100%) 0 0
Dual
Kinects
Ascending 25 23 (92%) 2 0
Descending 25 25 (100%) 0 0

On the other hand, the results indicate that the occlusions did not significantly influence the accuracy in this
experiment. Despite the low accuracy of motion capture regarding hands, the predefined actions could be
correctly recognized using the data from a Kinect. In this respect, there may be no need of highly expensive and
accurate motion capture systems such as a Vicon for the proposed motion analysis. Kinects or any sensors that
provide an acceptable level of accuracy can be used to collect motion data on a jobsite.
6. CONCLUSION
In this paper, the accuracy of Kinect motion capture data is evaluated in terms of rotation angles by comparing
the resultant data to the ground truth obtained with a Vicon system. The reliability of the data as prior models is
also investigated through the proposed motion detection analysis. The results show that both 1 and 2 range
cameras perform well at capturing human motions and detecting similar actions in motion data from videos.
Range cameras thus have great potential to be used for gathering actual workers motion data from a jobsite.
For motion data collection, the results indicate that the positioning of sensing devices is critical to minimizing
the impact of occlusions and hence improving accuracy. In our experiment, a single Kinect produced slightly
more accurate data than dual Kinects. Although further investigation is needed to find out the fundamental cause
for this difference, it is obvious that the position of cameras is highly related to motion capture performance. In
this experiment, the rear side position was better for installing a camera to record a ladder climbing activity than
a side view position was. In our future study, the effect of occlusions will be systematically and quantitatively
analyzed to identify optimal camera positions for motion sampling.
In addition, this paper proposes a Kinect motion capture system as a means to collect motion priors for behavior
analysis. Despite the sound performance of a Kinect system, its short operating ranges and unsuitability in an
outdoor environmentdue to its sensitivity to external infrared lightmay bring up limitations for the constant
monitoring of construction workers on a site. To address the issue, our ongoing research includes 3D skeleton
extraction from videos recorded by regular video cameras, and further research efforts need to be focused on the
accurate estimation of 3D human poses from a monocular video.
7. ACKNOWLEDGEMENT
We would like to thank Chunxia Li, a graduate student at the University Michigan (UM), as well as the staff of
the UM 3D lab, for their help in motion data collection. The work presented in this paper was supported
financially by two National Science Foundation Awards (No. CMMI-1161123 and No. CMMI-1200120), as well
as CPWR through the NIOSH cooperative agreement OH009762. Any opinions, findings, and conclusions or
recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the
National Science Foundation, CPWR, or NIOSH.
8. REFERENCES
501
Chen, W. T., Lu, C. S., and Huang, Y. (2011). Investigating the safety cognition of Taiwans construction
personnel. Journal of Marine Science and Technology, Vol. 19, No. 4, 398-408.
CPWR, The Center for Construction Research and Training (2008). The construction chart book: The U.S.
construction industry and its workers. The Center for Construction Research and Training, Silver Spring, MD.
Dutta, T. (2012). Evaluation of the Kinect sensor for 3-D kinematic measurement in the workplace. Applied
Ergonomics, 43, 645-649.
Fernndez-Baena, A., Susn, A., and Lligadas, X. (2012). Biomechanical validation of upper-body and lower-
body joint movements of kinect motion capture data. Universitat Politcnica de Catalunya.
<http://www.pagines.ma1.upc.edu/~susin/files/KinectvsMocap_final.pdf> (June 2012)
Heinrich, H. W., Petersen, D., and Roos, N. (1980). Industrial accident prevention. McGraw-Hill, Inc., New
York.
Han, S., Lee, S., and Pea-Mora, F. (2012a). A machine-learning classification approach to automatic detection
of workers actions for behavior-based safety analysis. Proceeding of 2012 ASCE International Workshop on
Computing in Civil Engineering, Clearwater Beach, FL.
Han, S., Lee, S., and Pea-Mora, F. (2012b). Vision-based motion detection for safety behavior analysis in
construction. Proceeding of 2012 Construction Research Congress (CRC), West Lafayette, IN, 1032-1041.
Helen, L., and Rowlinson, S. (2005). Occupational health and safety in construction project management. Spon
Press, London, pp. 157-158.
Khoshelhan, K. (2011). Accuracy analysis of Kinect depth data. Proceeding of ISPRS Workshop Laser Scanning
2011, Eds, Lichti, D. D. and Habib, A. F., Volume XXXVIII-5/W12, Calgary, Canada, Aug. 29-31, 2011,
<http://www.isprs.org/proceedings/XXXVIII/5-W12/Papers/ls2011_submission_40.pdf> (June 2012)
Krause, T. R. (1990). The behavior-based safety process Managing involvement for an injury-free culture.
John Wiley & Sons, Inc., New York, NY.
Li, C. and Lee, S. (2011). Computer vision techniques for worker motion analysis to reduce musculoskeletal
disorders in construction. Proceeding of 2011 ASCE International Workshop on Computing in Civil Engineering,
Miami, FL, 380-387.
McSween, T. E. (2003). Value-based safety process Improving your safety culture with behavior-based safety.
A John Wiley & Sons, Inc., Hoboken, NJ.
Meredith, M. and Maddock,S. (2001). Motion capture file formats explained. Production, 1-36.
Okada, S. and Hasegawa, O. (2008). Motion recognition based on Dynamic-Time Warping method with self-
organizing incremental neural network. Proceeding of 19th International Conference on Pattern Recognition,
ICPR 2008.
Stone, E. and Skubic, M. (2011). Evaluation of an inexpensive depth camera for in-home gait assessment.
Journal of Ambient Intelligence and Smart Environments, 3, 349-361.
Schlkopf, B , Smola, A., Mller, K. (1998). Nonlinear component analysis as a kernel eigenvalue problem.
Neural Computation, 10, 12991319.

502

V. SIMULATION AND PLANNING

503
504

SIMULATION MODELLING FOR ORGANIZING THE BUILDING
MACHINERY WORK AT THE CONSTRUCTION SITE
Alexander Ginzburg
Moscow State University of Civil Engineering, Russia
ABSTRACT: The organization of the building manufacture is the complex problem which can be solved in the
course of organization and technological design (OTD), i.e. designing the process of the future building
construction. OTD deals with the issues of time organization of the construction process including logistical
support, well-timed delivery of construction material, carcasses and goods to warehouses and construction sites.
One of vital issues is the organization of the building machinery work. As a rule, it is a limited resource which can
go out of service from time to time. Accordingly there is a problem of designing the way the broken-down machines
are repaired while managing the limited resources such as maintenance gangs with different qualification and
execution speed. In practice one can often deal with the situation when the analytical dependence linking the
system input and the system output, and therefore, the system analytical model cannot be created because of the
random nature of the building machinery breakdown and repair. So the classical mathematical methods are
inappropriate to analyze the situation at the construction site. Simulation modelling approaches are applied in this
case. The analytic models neither research the internal structure nor investigate the system behavior. In fact only
the connection between input and output is taken into consideration, whereas the system itself remains the black
box. The simulation models are characterized with the absolutely different approach. The system behavior is
described in detail in the course of time. The state transition processes are simulated in the way as if they took
place in reality. Then the system behavior is observed for the indicated time period and in accordance with the
results one can make a conclusion as to the obtained factors characterizing the output. The expanded features of
simulation models concerning the obtained results visualization allow to effectively analyze the systems that have
not practically taken into account before because of their formalization complexity. The research contains the
review of the general imitation model connected with functioning and repairing the building machinery complex
which allows to effectively plan and organize the machinery work at the construction site taking into account the
maintenance gangs load.
KEYWORDS: Computer modelling, imitating modelling, simulation modelling, organizational and technological
design, organization of the building machinery work, stochastic models.

The organization of the building manufacture is the complex problem which can be solved in the course of
organization and technological design (OTD), i.e. designing the process of the future building construction. OTD
deals with the issues of time organization of the construction process including logistical support, well-timed
delivery of construction material, carcasses and goods to the warehouses and construction sites. Thus, the
modeling of the process in OTD includes various problems based on the mathematical processes investigation.
One of the vital issues is the organization of the construction machinery work. As a rule, it is a limited resource
which can go out of service from time to time. Accordingly there is a problem of designing the way the
broken-down machines are repaired while managing the limited resources such as repair gangs with different
qualification and execution speed. The sequences of the breakdowns and the following repairs can be considered
as the elements of mathematical processes.
The arrival process of requirements to the construction organization technological system is characterized with
the number of requirements n(t), arriving at the system per time slot [0;t), where n(t) is a non-decreasing,
non-negative, integer-valued function;
the time slots between the adjacent requirements, where are the random quantities.
The function n(t) can be set by the joint distribution probability:
P
n
(t) = P{n(t
1
)=k
1
; n(t
2
)=k
2
; } (1)
The probability is set the most simply for the simplest process which simultaneously meets the following
requirements:
505

stationarity, that is the probability referred to the arrival of the definite number of requirements P
n
(t) on
the time slot [0;t) depends only on the interval range and does not depend on its location within the time
line;
the absence of aftereffects (prehistory), that is n(t) and P
n
(t) do not depend on the events taking place
before the time point t
0
=0;
ordinariness, it implies a time slot t (t0) = dt, which is characterized with the arrival of the single
requirement.
For this process is typical, that
P
1
(dt) = dt, (2)
i.e. the probability of the single requirement arrival is proportional to the time slot and the intensity of the
requirements process .
The interval length between the adjacent requirements in the simplest process is defined by the exponential
distribution law: F() = 1 - e
-
with the expectation value M=1/ and the dispersion D=1/
2
.
The analysis of the process organizational and technological systems in construction proves that most of them
come to the queuing systems (QS), that is the systems designated for requirement stream-processing for queuing.
Thus, solving the organizational and technological design issues resulted in pointing out the main QS
characteristics:
input process characterization;
estimating a queue length and a delay period;
identifying a number of channels necessary for the effective processing service (identifying the service
time);
output process characterization.
The main factors characterizing QS are as follows:
the number of unserved requirements (estimated by the loss probability);
the number of busy channels (estimated by the probability, that k channels are busy while servicing);
the downtime factor (or the utilization factor) QS (estimated as a part of used channels in general).
The functioning of the simplest QSs (single-channel, open-loop, with unlimited expectation and the simplest
requirements processes) is formalized in the most common way. The conduct of such a system can be represented
as the process of state transitions: X
0
- the system includes no requirements, X
1
- the system includes 1 requirement,
, X
n
- the system includes n requirements.
The probability of the arrival (service) of the next requirement (the probability of the transition in the next or the
previous state X) for the period of time t for the stationary process is proportional to the time and the process
intensity ( - for input and - for output). If the processes, acting in the QS, are ordinariness, that is for the period
of time t exactly one requirement enters or it is serviced, skipping through the adjacent state is impossible. The
probability to keep the same state can be defined on the assumption of the probability of the divisible group of the
disjoint events: the integrated probability for all events must be equal to 1.
For the QS satisfying the limitations of Markov chains (the process probabilistic characteristics in the future
depend only on the current state and do not depend on the pre-history) Markov theorem can be applied. It shows
that with a sufficiently large n the system tends to the limiting state, which is described by the vector P
n
.
To describe the transition in the modeling one can apply Chapman- Kolmogorov differential equation illustrating
the time history of the transitional probabilities. For the stationary process the system degenerates into the
algebraic equations solved as Cauchy problem.
506

However due to the random character of the machinery breakdown and the further repair one can often deal with
the situation when the analytical dependence linking the system input and the system output, and therefore, the
system analytical model cannot be created. Accordingly the classical mathematical methods are inappropriate to
analyze the situation at the construction site. Simulation modeling approaches are applied in this case.
The simulation models applied in organizational and technological design have the following peculiarities:
the models always depend on time;
the results are always identified with the random character of investigated processes.
Depending on the initial data on the object under investigation one can apply two methods of simulation modeling:
the method t;
the singular conditions method.
The method t implies step-by-step modeling of the system state transition within the equal time intervals t. It is
evident that the higher the modeling accuracy, the less is the time step under consideration. However if t
decreases, the quantity of interactions increases and the labour intensity of the model realization grows.
Within the framework of the singular conditions method not the state transition, but the length of the state stay is
modeled. The labour intensity of the method is much lower, but its realization requires the availability of the raw
data that characterize not the probability of the state transition, like in the method t, but the lead time in the
system.
The obvious challenge for the simulation model realization is the collecting of valid statistical data in order to get
stochastic characteristics describing the investigated object. However the organizational and technological design
can solve the similar problems.
Let us consider the generalized simulation model of functioning and repairing the complex of construction
machinery, this model allows to effectively plan the machinery work at the construction site taking the load of
repair gangs into account.
Let N repair gangs repair K types of the construction machinery. The period of repair is the random quality which is
distributed randomly f
nk
(t), that is the period of repair is defined by the staff of the repair gang and the type of the
accessory. In general a repair gang can service not all the machinery, so the functions f
nk
(t) cannot be given for all
combinations n and k.
The duration of the overhaul period (the duration of accident-free operation of construction machinery) is also a
random quality, distributed by the random law g
k
(t), which depends on the type of machinery.
The modeling of the functioning of the construction machinery complex comes to the definition of value of the
respective random qualities. In the general case one can transform the random quality x, distributed by the standard
uniform law of distribution R[0;1]:
dy y f x ) ( , (3)
where
f(y) - the distribution law (density) of the random quality;
- modeled duration - the quality, distributed by the law f(y).
After solving the equation (3) in relation to , we will get the sought duration of this or that process. The initial
random quality x is generated by the standard random-number generators, characteristic of all the high-level
programming languages.
One can use different approximate computing methods for nonintegrable functions which describe the distribution
law of the random quality. For instance, the central limit theorem is applied for the normal law, according to this
507

theorem the sum of many random qualities (in this case one can use the sequence of random qualities from the
standard random-number generator) is distributed by the normal law.
While modeling one working day of machinery one can suppose that all the machines begin working in the
operation condition. In this case it is necessary to obtain the work duration of all the machines
work
k
and to define
which of them will be out of order first and when it (the breakdown period) will occur:
work
k
fail
j
t
k
min = , (4)
where
j - the index of the machine which is prepared for repair work.
One defines for the chosen machine if free repair gangs are available (those repair gangs for which the random
distributional law concerning the duration of repairing type j machines and which are free of work at the present
moment). In the case of the absence of free repair gangs one defines the standing period of the machine - the period
between the moment of the breakdown and the moment
rep fin
t
.
, when the first repair gang prepared to repair the
given type of machinery will be free (will finish the repair of some other machine):
fail
j
rep fin
n
n
wait
j
t t =
.
min . (5)
If several repair gangs, that is the gangs which got free before
fail
j
t are available, the choice of the repair gang
must be realized according to the different criteria:
the minimum expectation of the duration of repair works for the given type of machinery;
the minimum value of the estimated repair;
the specified scale of priority depending on the skills of the repair gang etc.
The repair duration is generated according to the number of the chosen repair gang and the type of the machinery
according to the same rules as the work duration (3).
The described process of choosing the machine with the minimum breakdown period
fail
j
t and searching for a
suitable repair gang continues until the running time exceeds the fixed time of the day work completion.
In the general case different variants of multi-shift work at the construction site can be chosen. For instance the
machines can work in two shifts and the repair gangs can work in three shifts. In practice it means that the modeled
duration of the machinery work should be limited by the frames of the working day, and the duration of repair in
the third shift is not changed. Otherwise at the beginning of the modeling one should take into account that a part of
the machinery is repaired and a part of gangs is engaged in repairing it.
In the course of the simulation modeling one can store the information concerning
the work duration of each machine;
the repair duration of each machine;
the awaiting repair time duration of each machine;
the work duration of each repair gang;
the idle time of each repair gang.
As a result one should take into account that every particular realization of the machine working day is considered
as the element of the random sample. The conclusions concerning the conduct of the modeled object are made after
the statistical manipulation of many tests.
The given time characteristics of system machinery work can be interpreted from the position of the economy if
one gives the cost of repair gangs` work, defining the downtime costs of the machines.
508

The main advantage of simulation models is the possibility to modify the settings characterizing the system
behavior. One can control time decelerating it in the case of fast processes and accelerating it for modeling the
systems with slow mobility.
When controlling the model one can design the effective organization of the machinery work at the construction
site with the high degree of reliability taking into account the variety of the tasks the system deals with.

REFERENCES
Averill M., Low V., Kelton D. (2004). Simulation modeling. Classic Computer Science, 848 p.
Berezhnaya E.V., Berezhnoy V.I. (2006). Mathematical methods of economic systems modeling. .: Finances and
statistics, 432 p.
Emeliyanov A.A., Vlasova E.A., Duma R.V. (2002). Simulation modeling of economic processes. .: Finances
and statistics, 368 p.
Ginzburg A.V. (1999) The automation of the organizational technological construction reliability designing. .:
SEP RIA.
Ginzburg A.V., Kagan P.B. (1999). CAD of construction organization. The journal "CAD and graphics", 9,
32-34.
A.Ginzburg. Computer modeling in organizational and technological design. Proceedings of the 11th International
Conference on Construction Applications of Virtual Reality 2011. -Weimar, Germany: Bauhaus-Universitat, 2011.
- pages 29-30
Gusakov A.A. (1983). System engineering in construction. .: Stroyizdat, 440 p.
Horafas D.N. (1967). Systems and modeling. .: The World, 420 p.
Investment construction engineering. The reference book. (2007). Edited by professors I.I. Mazur and V.D.
Shapiro. .: Elima, 1216 p.
Karpov Y. (2006). System simulation modeling. Introduction to modeling with AnyLogic 5. BHV-Petersburg, 400
p.
Kolemaev V.A. (2005). Econometric modeling: Modeling of macroeconomic processes and systems: The manual.
UNITY-DANA, 295 p.
Moiseev N.N. (1981). Mathematical problems of system analysis. .: Science.
Shennon R. (1978). Simulation modeling of systems - art and science. .: The World, 424 p.
Sinenko S.A. and others (2002). Automation of organizational technological design in construction: The manual.
.: Publishing company SV, 240 p.
Snetkov N.N. (2008). Simulation modeling of economic processes. .: Publishing company EAOI, 228 p.
Vainshtein M.A., Ginzburg V.M. (1996). Virtual reality in automation system of architectural construction design.
The journal "Informatization problems", 2, 32-37.

509

SIMULATION OF PROCESS INTERACTION FOR QUALITY
ASSURANCE DURING CONSTRUCTION
Vitali Kochkine & Jrgen Schweitzer
University of Stuttgart, Germany
Ilka von Gsseln & Nils Rinke
Leibniz Universitt Hannover, Germany
ABSTRACT: The time dedicated to erection of high-rise buildings decreases progressively. Nowadays, less than
a week is required to accomplish the concrete structure of a single story implementing new techniques such as
climbing formwork. However, the faster execution of construction works and the increasing requirements request
essential improvement of the quality assurance methods applied directly and continuously on construction site.
Quality assurance systems gain growing significance in construction industry and have already become a part of
project management. This contribution focuses on the modeling of interaction between the geodetic measurement
process for quality assurance and the construction process itself. By ensuring the quality of these particular
processes the geometrical accuracy of main building elements and therefore of the whole building can be
considerably improved without reducing the pace of construction. Prerequisite, measurement and evaluation
processes of engineering geodesy have to be integrated into the construction process seamlessly. This
challenging goal can be achieved by means of process simulation and optimization, which requires a highly
detailed modeling of both processes and their interaction. The modeling and simulation are realized with
high-level Petri nets. Thus it is possible to take conditions and dependencies into account. In order to simulate
the required processes, duration has to be assigned to each transition that takes time in reality. These durations
can be modeled deterministic or stochastic. In this case stochastic distributions for durations were selected.
During the construction work, data about the quality of the product and process is directly available. If some
processes cannot be carried out as planned, alternative paths have to be provided in Petri nets and can be
executed. This contribution illustrates a collaborative work between civil and geodetic engineers, and
emphasizes especially the interface between construction and engineering geodesy processes.
KEYWORDS: Simulation in construction, surveying, high-level Petri nets, quality assurance
1. INTRODUCTION
Construction of high-rise buildings is highly optimized and involves many different disciplines and parties. The
understanding of the term quality within these different parties is not equal. For example, constantly maintaining
acceptable tolerances required for the lift shafts is often an underestimated challenge during the execution of
concrete works.
This paper is based on a scientific research project in which a geodetic method of quality assurance in construction
is developed. This method is theoretically applied to a model of core construction of a characteristic high-rise
structure. For the concrete works the typical system of climbing formwork is chosen. The modeling and simulation
of the construction and geodetic processes is carried out by means of high-level Petri nets.
2. CONSTRUCTION AND GEODETIC PROCESSES
2.1 Construction processes
The systems of climbing formwork are usually used if horizontal supporting structural components, such as slabs
or beams, are not accessible. This is typical, for example, for cores of high-rise buildings, bridge pylons, stairwells,
cooling towers etc. Climbing formworks can only be supported by the structural elements they produce and has
proven to be sufficient due to high grade of repetition of routine working processes. The whole method is based on
iterating procedures and is fixed to an exactly planned working cycle virtually regardless of the height the works
are carried out in.
Figure 1 displays the details of a simple climbing formwork. The characteristic climbing formwork contains
mainly standard large wall formwork mounted on climbing carriages which are supported by the concrete of the
section already produced.
510

The most climbing formwork systems can be split into self-climbing and crane-depending. The advantages of a
crane independent formwork are the crane capacity relief and a nearly wind-independent construction (Schmitt
2001).

Figure 1: Climbing formwork.
In the presented research project the self-climbing system is considered. Only the working platform inside of the
lift shaft is shifted by crane. The dimensions of the representative typical high-rise building are chosen to be
23 x 20 m with a lift shaft of 8 x 5 m. The formwork elements are placed on eight climbing rail-suspended
carriages K1-K8 (Figure 2), which are displaced sequentially by means of hydraulic jacks. These carriages serve as
operational platforms as well and can bear wind protecting scaffolding.

Figure 2: Layout of a typical story.
After putting concrete in place, the formwork stays in position until the hydration of the concrete is finished to a
certain grade and provides enough support for climbing. After that the hydraulic climbing can progress to the next
511

segment. For the process model used in this project a representative working cycle of one week per story for the
climbing formwork system is chosen. The climbing formwork has normally a progress interval of one or two levels
relating to the nearest slab, which represents certain difficulties to the quality assurance based on the geodetic
surveying on site.
2.2 Geodetic processes and their interaction with construction processes
An integration of geodetic measurement processes into construction processes is useful to increase the quality of
the skeleton. This integration ensures that the accuracy requirements are met. Focusing on the measuring processes
that are carried out during the creation of the skeleton, the main task of the surveyor is staking out the climbing
formwork. After concreting the core, a control measurement is carried out. For all measurements performed at the
working level, fiducial points are required. With these fiducial points the connection to the coordinate system is
established. Generally, these points must be transferred to the working level, because often there is not sufficient
visibility to points outside of the building. Commonly, the transfer of the horizontal and height network is carried
out separately because different measurement methods are used. Plumbing is a common method for the transfer of
the horizontal position while leveling is used for a precise height transfer. Furthermore, the transfer can be
performed with a joint measurement method, the tacheometric measurement. However, this measurement method
provides a slightly worse but often acceptable accuracy for the height component than leveling.
The main construction and related measuring processes for the erection of one story are shown in Figure 3. The
construction of a core by using a climbing formwork is shown in the left part of the figure and the construction of
the slab is shown in the right part. In most cases the construction of the slab is a few stories behind the construction
of the core. Therefore, both of the construction processes can be regarded as almost independent. But there are
exceptions: joint staff, concrete delivery or measuring processes. For example, the transfer of the horizontal and
height network can only be performed, if the slab can be accessed. These fiducial points on the currently top level
of the building are absolutely necessary for staking out the climbing formwork. During the alignment of the
climbing formwork an interaction between the construction and measurement process is required. After moving up
the climbing formwork it is aligned roughly. The precise alignment is performed with the assistance of the
surveyor. Therefore reflectors which are fixed to the formwork have to be measured. Using these measurements,
the deviation of the current position of the formwork to its target position can be calculated. Then the formwork
has to be realigned based on the determined deviations. After this, the measurement to the reflectors on the
formwork is carried out again. This is an iterative process which is repeated until the formwork element is in the
correct position.

Figure 3: Main processes for the construction of one story i and the related measuring tasks.
All measurement processes can be divided into modules that repeat themselves in the creation of each story. Even
in a completely different structure the process flow remains the same. Often, the only difference is the number and
location of the points to be measured.
512

2.3 Disturbances and delays
Every construction project is highly complex due to numerous links and dependencies between different processes.
This complexity is one of the reasons for different disruptions, which occur during the execution of works. In the
research project described herein, disturbances and delays during both construction and geodetic processes are
considered and taken into account.
Disturbance is considered to influence one certain construction or measuring process within the model.
Disturbance causes an alteration of the process sequence, which compensates the extension of time and can be
reflected in the process model and the Petri net as an alternative path. Disturbance does not necessarily lead to a
delay of the whole project or certain kind of works.
Delay is considered to affect one certain process as well, but cannot be compensated by any reasonable activity
within the project and causes an immediate extension of the project or works duration.
Within the research project following disturbances and delays are assumed and reproduced with Petri nets.
The disturbance during the proceeding of a working platform on the climbing carriage on the outer side of the lift
shaft should represent a case of minor failure during the concrete works. It can be normally compensated by an
alteration in the process sequence. The works can proceed with the next carriage and then come back to the failed
one.
The disturbance during the displacement of the working platform inside of the lift shaft can be compensated by
changing the sequential arrangement of the processes. The outer formwork should be put in place first before the
lift shaft working platform can be displaced.
A failure with the slab formwork cannot be compensated by any alteration within the process model. This failure
could lead to unavoidable delay of the whole project and can only be counterbalanced by increasing the manpower
or extending the working time. In this research project the latter approach was chosen due to the necessity of
adherence to the working cycle of one week per story.
During a measurement process various problems may appear. Prior to the precise alignment of the climbing
formwork elements, the problem may arise that a stationing of the measuring instrument is not possible. The
reason might be that one of the provided fiducial points is not visible or accessible. Thus a connection to the
coordinate system is initially not possible and the subsequent precise alignment of the formwork cannot be done.
As a reaction to this disturbance a different fiducial point can be selected. If this is not possible, a different
stationing position might be preferred, or the alignment of the formwork could be done with a different
measurement method (e.g. plumbing).
If the required quality is not maintained during the alignment of the formwork, this is regarded as a disturbance,
too. If this occurs, it needs to be checked, how the discrepancy can be explained. It may be sufficient to repeat the
stationing and alignment with the same measurement method. If it is found that this is not sufficient or possible, a
different measurement method (e.g. plumbing) has to be selected.
In section 4 will be shown how alternatives to a regular measurement process are integrated into the process
model.
3. HIGH-LEVEL PETRI NETS
Petri nets are mathematical representations of dynamic systems, which allow a high flexibility for modeling
process hierarchies and durations. These nets are an expansion of classical graph theory by the dynamic transport
of information through the net. Due to the mathematical formulation Petri nets can be adapted easily to the
requirements of the working process.
Petri nets are often used for modeling and simulating of discrete systems in different areas of application also in
construction. Petri nets can be used for the simulation of earthmoving operations (Cheng et al. 2011). Different
variations like coloured Petri nets (Chung 2011) or Petri nets in combination with multi-agent systems (Molinero,
Nez 2011) can be used for modeling of construction schedules. Furthermore, Petri nets can involve e.g. Fuzzy
set theory for detailed analysis of resource allocation (Subulan et al. 2011).
In this scientific research project high-level Petri nets are used to model and simulate the construction and
513

measurement processes and their interaction, which has not been considered before in this constellation.
3.1 Formulation
Petri nets are bipartite graphs with two disjoint vertex sets called places and transitions. In the following we
describe the formalism for a place/transition net (P/T net), which is most common used as the basis for high-level
Petri nets. A P/T net can be described by a 6-tupel (P, T, F, C, W, M
0
), where
P is a finite set of places. A place represents a condition or a state in a process,
T is a finite set of transitions. A transition represents an activity in a process,
F is a set of relations F (P T) (T P), which connects places with transitions and vice versa,
C is a map C : P N
+
, which indicates the capacity of every place,
W is a map W : F N
+
, which assigns a weight to every edge.
M
0
= {m(p
0
), m(p
1
), , m(p
n
)}, is the initial marking of the Petri net, which assigns a number of token to
every place p
i
. A token on a place describes either the resources, like staff and equipment, or the actual
progress (Baumgarten 1996).
The set of places which are predecessors of a transition t are called input places of transition t or the preset of t:

{ } | ( , ) t p p t F =
(1)
Accordingly the set of successor places of a transition t are called output places of t or the postset of t:

{ } | ( , ) t p t p F =
(2)
A transition is called active, if all input places contain enough token and the capacity of all output places will not be
exceeded by the outgoing token. A process is modeled by the transport of token through a transition which is called
firing. The result of the firing of a transition t is a new marking M = {m(p
0
), m(p
1
), , m(p
n
)} with:

( )
( ) ( )
( ) ( )
( ) ( ) ( )
( )
,
,
'
, ,
m p W p t if p t and p t
m p W t p if p t and p t
m p
m p W p t W t p if p t and p t
m p otherwise

=

+
(3)
During the simulation of a Petri net several transitions can be activated based on same tokens in the preset. In order
to decide automatically which transition will be fired, each transition is prioritized with a positive integer.
3.2 Time dependent Petri nets
For a detailed simulation of a process each transition has to consume time, due to the duration of the representing
process. A global timer can achieve the integration of time to Petri nets. To model the duration of a process, each
transition gains a local timer. If a transition fires, all involved tokens are blocked instead of transporting the token
from the preset to the postset immediately and the local timer is activated. All active local timers are decremented
constantly subject to the global timer. The transition finishes and releases the token to the postset if the local timer
goes down to zero (see Ajmone Marsan et al. 1994).
The timer of a transition can be modeled deterministically or stochastically to depict all influencing constraints of
a process. A process, whose duration differs very small, can be described by a deterministic value. The duration of
many processes (both construction and measurement processes) are influenced by many outer constraints like
weather or the layout of the building. The timer of these transitions can become more realistic by using a random
value that follows a particular probability density function (PDF).
It is obvious, that in a high level of detail different processes have to be approximated by different PDFs. Recurring
processes with a short duration can be approximated best by a rectangular distribution function. The more time a
514

recurring process consumes, the more a normal distribution function is a better description. Processes, which have
many constraints or will be performed seldom, lasts usually longer than the expected value. They might be
approximated by a gamma distribution or other unbalanced PDFs. Further information for a stochastic modeling of
geodetic processes can be found in von Gsseln and Kutterer (2012).
3.3 Hierarchy and Modularization
The workflow of construction processes contains a variety of processes, which can be ordered hierarchical. Due to
the design of a Petri net, it is possible to represent a process, consisting of sub-processes, by one transition
including a subnet. When this transition is fired, all incoming tokens are transferred to the start of the subnet. This
leads to a clearly arrangement of the whole workflow in any number of hierarchical levels. The duration of a
transition, consisting a subnet, can either be computed by simulating the deeper levels or can be estimated, which
means a reduction of the computation time.
Due to the hierarchical arrangement, periodically recurring processes like building of the walls can be modeled by
a module once only. Every time one of these processes occurs, only a reference to the module needs to be inserted.
The main advantage is that the editor can use these modules as often in one or any similar project. For further
information see Rinke et al. (2012).
4. MODELING OF CONSTRUCTION AND GEODETIC PROCESSES
4.1 Petri net of the example construction process

Figure 4: Top level of the Petri net for modeling the example construction process.
In Rinke et al. (2012) and Schweitzer et al. (2012) the Petri net of the exemplary high-rise building (see Figure 2)
is presented. At the top hierarchical level of the Petri net (Figure 4) all important construction and measurement
processes for the construction of one story are modeled. Every story is constructed in a week-cycle. Using Petri
nets as a process model, the dependencies and interactions of individual processes mentioned in section 2.2 and
Figure 1 can be modeled very well.
515

Figure 5: Alternative path for a disturbance during the moving of a climbing bracket.
The process model built up in different levels (levels 0 to 2) with increasing degree of detail as shown in the
Figure 5. The top level (level 0) represents a summarizing process overview, level 1 represents single activities
within a process and the third hierarchical level (level 2) describes particular actions within activities. This detailed
study of both construction and geodetic processes makes a precise interaction and efficient quality control
possible.
4.2 Integration of alternative paths into the Petri net
For the delays and disturbances described in section 2.3 appropriate alternative paths in the Petri net model are
described. An alternative path for the disturbance within the construction process during the climbing of a bracket
is shown in the Figure 5 exemplary for the bracket K1.
The process Form/Reinforcement Wall has already been described in Rinke et al. (2012) in more detail. The
modeling of the working steps is up to the third hierarchical level (level 2). An alternative path for the disturbance
within the geodetic measurement process is shown in the Figure 6.
516

Figure 6: Alternative path for a disturbance during the precise alignment of the climbing formwork.
In the modeling level 2, the Precise Alignment of the Climbing Formwork is shown. First, the current position of
the formwork and the deviation from the nominal position are determined. Then, the formwork moves accordingly
and the position will be reviewed. If the specified tolerance has been met, the precise alignment is complete. If it
has not been met, the loop is executed again. However, during the Nominal-Actual Comparison it may also be
noted that the required quality cannot be met. Then, the measurement process will be aborted and in the above
hierarchy level (level 1) it is modeled, how to react to it. In level 1, a token at the place Quality Not Met is
created when this disturbance occurs. Subsequently the Evaluation is carried out. It will assess whether it is
sufficient to repeat the stationing of the total station and the precise alignment or whether an alternative
measurement method must be used for the precise alignment of the formwork.
Another disturbance is modeled on the level 1. It occurs when a stationing of the total station is not possible. In
section 2.3, the causes and reactions have been reported. In level 1, only one alternative is presented. Namely, that
an alternative measurement method has to be chosen. The other alternatives have to be modeled in the hierarchical
level 2.
4.3 Simulation
The Petri net for the week-cycle shown in Figure 4 can be simulated for the time estimation of the process and the
quality assurance. The simulation of a Petri net is as follows. Starting at a point in time t
0
with the initial marking
M
0
(in Figure 4 only one token is set in the left place) all transitions are checked for their activity. All active
transitions are fired in order of their priority. If a module is fired, all incoming tokens are transferred to their subnet
(see examples in Figure 5 and Figure 6). All other transitions receive an actual duration described in section 3.3.
When no active transition is left, the simulation shifts to the point of time t
i
, at that the first transition finishes. Here,
the working hours of staff and the interruptions by breaks or non-labor days are considered. At transitions, in
which a quality validation is done, it is decided by their stochastic parameters which path the outgoing token have
to follow. The simulation ends when at a point in time no active transition is left and no transition is currently
firing.
In the post processing it is possible to analyze the simulated process flow e.g. the latency of token or the deviation
of the total working time. It is obvious, that a delayed transfer of the geodetic networks causes the token
representing the builder has to wait to form the core walls. By simulating the Petri net sufficient number of times,
517

a simple bottleneck analysis is carried out so that arrangements to prevent this delay can be made already in the
planning phase.
4.4 Quality assurance
The quality assurance concept is based on a quality model. It is a conceptual framework in which the abstract term
of quality is gradually resolved into individual aspects. It typically consists of characteristics and parameters. A
quality characteristic is an inherent feature of a product or process related to a requirement. Each characteristic
may be described by a number of parameters as shown in Table 1.
Table 1: Characteristics and parameters of the quality model
Parameters Characteristics
Standard deviation Accuracy
Tolerance correctness Correctness
Topological correctness
Number of missing/ odd elements Completeness
Adherence to the plan
Condition density Reliability
Minimal detectable error (mde)
Impact of mde on parameters
Vulnerability to failures
Time delay Timeliness
The quality model consists of classical geodetic accuracy and reliability parameters but also other parameters like
correctness and completeness. The aim is to have a complete description of the quality related to the building
geometry. Additionally, besides product orientated parameters also process oriented parameters like Adherence to
the plan and Time delay are available. Details about the derivation and the development of this specific quality
model can be found in Schweitzer and Schwieger (2011).
Based on the values of the quality parameter (e.g. standard deviation =5mm) an evaluation can be done.
Afterwards, the evaluation is used to undertake quality assurance measures (e.g. additional measurements). This
process (get parameter values, evaluation and quality assurance measures) can be seen as a closed loop system. In
Mhlenbrink and Schwieger (2007) there are two different closed loop systems regarding the construction control.
The outer loop, which deals with the construction control, and the inner loop, which deals with the technical
control of special sub-processes like manufacturing machines. Applied to our exemplary process model (Figure 3),
the outer control loop contains the construction of the building core and the corresponding floor and the inner loop
contains the alignment of the climbing formwork (Alignment of climbing formwork).
The inner loop has to act in real time (within few seconds or minutes). Here the parameter Tolerance correctness
is evaluated (Tolerance complied) and a correctness information (yes/no) is given back to the construction
process. The outer loop cannot be act in real time. The measured error can only be used as knowledge for the
further processes or additional, unplanned measures to correct the abuses at the building object. Here additional
quality parameters like Adherence to the plan, Number of missing elements etc. can be returned to the
construction process.
518

Figure 7: Inner and outer closed loop system for climbing formwork (modified from Mhlenbrink and Schwieger
(2007))
This closed loop system can be seen as basis for the development of automatic measurement systems in civil
engineering. An example in the area of machine guidance is presented in Beetz and Schwieger (2010).
This quality assurance concept can be applied during the design or construction phase. The differences between
both phases are shown in Table 2.
Table 2: quality assurance concept in the design phase and during the construction phase
Design phase Construction phase
Show impact of different input quantities (e.g.
measurement configuration) on the results
Quality assurance measures depends on the
improvement of the measurements and
construction processes
parameter values are derived from
the measurements
A quality assurance measure can
be the use of alternative paths
In the design phase the Monte Carlo Method can be used (e.g. for the parameter standard deviation) to simulate
measurement processes to find out the impact of different input quantities on the results. (see Schweitzer et al.,
2012). Therefore the process model simulation based on petri nets (see section 4.3) is used. Furthermore, the
effects of disturbances and delays on the parameters can be analyzed. During the construction phase parameter
values are used to make the decision in real time. Here alternative paths are used (see section 4.2).
5. CONCLUSION
This article gives an overview of the concepts and developments in the presented research project, which deals
with quality assurance and efficiency optimization in construction. The idea of the project is based on the
simulation of the processes of engineering geodesy and construction. It is a collaborative work between civil and
geodetic engineers and therefore the interface between construction and geodetic processes is placed in the focus
of the project. As a result, the quality assurance concept for geometry of high-rise buildings is developed. Further
research activities regarding a deeper integration of the quality parameters into the construction process are
proposed.
The modeling and simulation of the presented construction processes with high-level Petri nets offer the use as a
planning tool. Yet only a few simulations of the presented process flow are performed with a part of the Petri net
for the whole construction processes (see section 4.1). By the integration of more levels and the simulation with
different stochastic parameters it is possible to analyze the process flow before execution to detect crucial
processes. Through the integration of alternative paths the process model can automatically react if delays or
disturbances occur. Combined with the presented quality assurance concept an adequate solution can be computed.
Thus it is possible to take measures for a smooth process flow in the planning phase as well as the use on site.
519

6. ACKNOWLEDGEMENT
The scientific research published in this article is granted by the German Research Foundation (DFG) under the
sign BE 2159/11-1/2, BE 3873/1-1/2, KU 1250/10-1/2 and SCHW 828/3-1/2. The authors cordially thank the
funding agency.
7. REFERENCES
Ajmone Marsan, M., Balbo, G., Conte, G., Donatelli, S., Franceschinis, G. (1994). Modelling with Generalized
Stochastic Petri Nets. New York, NY, USA: John Wiley and Sons.
Baumgarten, B. (1996). Petri-Netze: Grundlagen und Anwendungen. 2. Ed. Heidelberg: Spektrum Akademischer
Verlag.
Beetz, A. and Schwieger, V. (2010). Enhancement of the Control Quality by an Automatic Calibration Procedure
using the Example of a Construction Machine Simulator. Proceedings on 2nd International Conference on
Machine Control and Guidance, Bonn, 9.-11.03.2010.
Cheng, F.F., Wang, Y.W., Ling, X.Z., Bai, Y. (2011). A Petri net simulation model for virtual construction of
earthmoving operations. In: Automation in Construction, Vol. 20 (2), 2011, pp. 181-188.
Chung, T.H., (2011). Modeling of Construction Scheduling with Coloured Petri Nets Process Automation,
International Conference on Process Automation Control and Computing (PACC), 2011 , IEEE, pp. 1-6
von Gsseln, I. and Kutterer H. (2012). Efficiency Optimization of Surveying Processes. Proceedings of the 1st
International Workshop on the Quality of Geodetic Observation and Monitoring Systems, Mnchen (accepted).
Mhlenbrink, W. and Schwieger, V. (2007). Zur Prozessintegration der geodtischen Messtechnik. Allgemeine
Vermessungsnachrichten, Heft 11-12.
Molinero, C. and Nez, M. (2011). Planning of work schedules through the use of a hierarchical multi-agent
system, Automation in Construction, Elsevier, 2011.
Rinke, N., von Gsseln, I., Berkhahn, V. (2012). High-level Petri nets for modeling of geodetic processes and their
integration into construction processes, ECPPM 2012, Reykjavik, Iceland, 25.-27.07.2012.
Schweitzer, J., Kochkine, V., Schwieger, V., Berner, F (2012). Quality Assurance in Building Construction, based
on Engineering Geodesy Processes, FIG Working Week, Rome, 06.-10.05.2012.
Schweitzer J. and Schwieger V. (2011). Modeling of quality for engineering geodesy processes in civil engineering.
In: Journal of Applied Geodesy 1/2011: 1322.
Schmitt, R.(2001). Die Schalungstechnik: Systeme, Einsatz und Logistik. Berlin: Ernst & Sohn Verlag fr
Architektur und technische Wissenschaften GmbH.
Subulan K., Saltabas, A. and Tasan, A.S. and Girgin, S.C., (2011). Modeling and Analyzing of a Construction
Project Considering Resource Allocation Through a Hybrid Methodology: Petri Nets and Fuzzy Rule Based
System, Proceedings of the 41st International Conference on Computers & Industrial Engineering, 2011.
520
GENERATING WORKSPACE REQUIREMENTS IN A FINISHING
EXCUTION PHASE
Trang Dang, M.Sc., Amir Elmahdi, M.Sc. & Hans-Joachim Bargstdt, Prof. Dr-Ing.
Bauhaus Universitt Weimar
ABSTRACT: Detailed planning plays an important role in the success of a project in an execution phase.
Mallasi and Dawood indicated that 30% non-productive time on site due to the lack of detailed and space
planning, which results in workspace conflicts and inefficient utilisation of time (Mallasi and Dawood, 2001).
However, it is a fact that planning considering workspace requirements has not succeed in practice up to now,
despite receiving a lot of interest from researches for years. One of the reasons is the huge amount of input data
for workspace requirements. In order to overcome this problem, an automated generation of workspace
requirements should be considered. This paper will analyze advantages and disadvantages of previous
researches relating to generating workspace requirements. Based on that, a model for automating process of
generating workspace requirements in a finishing execution phase will be proposed.
KEYWORDS: workspace, BIM
1. INTRODUCTION

Workspace allocation has been a big challenge to managers on site. Kaming et al. investigated about the factors
influencing productivities; from the survey they showed that overcrowding of crews has been one of frequent
problems occurring and it could make crew productivities lose up to 40% (Kaming et al., 1998). Mallasi and
Dawood also indicated that 30% non-productive time on site due to the lack of detailed and space planning,
which results in workspace conflicts and inefficient utilisation of time (Mallasi and Dawood, 2001). In order to
better understand the issues related to stacking of trades, Hana et al. also conducted a survey from electrical
contractors (Hana et al., 2008). According to the respondents, poor planning is one of important reasons causing
stacking of trades, especially probabilities of its occurrence at the third and fourth quartiles of a projects
duration are up to about 30% and 50% respectively. This means stacking of trades is a big deal at the finishing
phase.
Table 1. Generating workspace requirements for activities in the previous researches
WORKSPAC
E
RESEARCH
ES OF
INPUT DATA
WORKSPACE POSITION &
SIZE METHOD FOR
GENERATING
WORKSPACE
changeable
with products
position
changeable
with products
size
(1) Bargstdt,
Elmahdi
workspace size of resources
relative positions of resource
spaces to respective tasks
products
products positions
resources required by tasks
YES YES manually
(2) Akinci et
al.
template of
activities workspaces
type of activities
YES YES
automatically
generating based on
available templates
(3) Dawood,
Mallasi,
Winch, North
(VIRCON)
coordinators of potential
workspace area
the size of workspace
NO NO manually
(4) Bansal coordinators of workspace NO NO manually

521
In the past twenty years workspace planning has already received a lot of interest from researchers. Bargstdt,
Elmahdi et al. presented a model based on discrete event simulation to allocate workspaces and resolve time-
space conflicts (Bargstdt and Elmahdi, 2010b; Bargstdt and Elmahdi, 2010a; Elmahdi et al., 2011). They
undertook site interviews and observations of the interior trades and their work procedure to capture common
characteristics. They examined practical execution strategies for individual trades at the interior construction
work phase and their workspace arrangement strategies such as required workspace types and their occupation
behaviour over time. The aim was to be able to automate the generation of the labourer workspace size for
multiple trades in a simulation model based on common characteristics. The analysis has shown that workspace
size can be automated based on the building object size and the nature of the activities including the work
procedures. Furthermore, they proposed human-oriented and -engineering labourer workspace, where the size of
the building element is smaller than the human operation labourer size so that boundaries between workable and
passable labourer workspace can be determined. Riley et al. suggested a method based on various execution
patterns for space planning (Riley and Sanvido, 1997; Riley and Sanvido, 1995). Akinci et al. built a 4D-CAD
based model to generate workspace requirements, detect and prioritize potential time-space conflicts (Akinci et
al., 2002a; Akinci et al., 2002b; Akinci et al., 2002c). Dawood and Mallasi developed a model to investigate and
measure the severity of spatial congestions and then searched for a strategy to minimize this problem (Mallasi,
2006; Dawood and Mallasi, 2006). Winch et al. proposed the concept of critical space analysis (CSA) (Winch
and North, 2006), Zhang et al. suggested a cell-based model in a discrete-event simulation (Zhang et al., 2007)
and Bansal applied GIS and Topology to identify and overcome this issue (Bansal, 2011).
Despite receiving a lot of interest from researchers for a long time, space planning, due to a huge amount of
input data for workspaces, is still not available in practice until now. Although the research of Akinci et al.
(Akinci et al., 2002b) generates workspaces automatically, which can reduce the time for creating workspace
input data compared to the manually methods, they still contain disadvantages for an automatic scheduling
process at a finishing phase. The advantage of this method is that workspace sizes and positions can be
automatically changed depending on products geometry. However, it just can be applied for a single product
(not for a group of products) and cannot automatically recognize which side of the objects, on which one-surface
trades, such as plastering, painting.., are carried out. Hence, it cannot know exact places where corresponding
workspaces should be located without the orientation information defined in the templates. This drawback makes
such the method for generating workspace bring not so much benefit in the finishing phase.

Previous researches did not consider room as a unit to
group products and locate workspaces for them
Previous researches could not automatically know whether
right or left side the workspace for tiling the wall must be
located.
Figure 1. Previous researches disadvantages for automated generation of workspace requirements
As already mentioned above, one of the reasons given for workspace conflicts and over-manning was lack of
detailed planning among crews (Rivas et al., 2011; Mallasi and Dawood, 2001). In order to support for an
automatic process of making a detailed schedule in a finishing phase, the paper proposes a model to generate
workspace requirements for activities in this period based on the type of trades, size and shape of products.
522
2. MOTIVATION

In order to create a template of workspace requirements for activities in a finishing phase, a template like the
model of Akinci et al. has been considered. However, several reasons given following make such a model not
suitable to several trades in this period.
1. Several trades relating to interior finishing are carried out in closed areas like a room, such as plastering,
sanitary installing, flooring and wiring (figure 7). This property leads allocating workspace for activities not only
depending on the size of product and type of trade, but also being limited by the size of room.
2. Not like other components in a structural phase such as columns and slabs which are conducted separately,
several products in an interior finishing phase are implemented in a group, e.g. plastering all the faces of walls,
or wiring inside walls and a ceiling in a room. They are normally considered as a component and completed
successively. Therefore, workspace for each single segment of walls is not suitable in this case.
With all of the above reasons, a template which is able to provide workspaces varying with size of products and
also able to recognize groups of them automatically is required beside of the basic model of Akinci et al. In this
paper, a framework of a model for generating workspace requirements, which is suitable to activities in a
finishing phase, will be presented in detail. The approach considers workspace in a macro level. It does not
differentiate where is a place for materials, where is for labourers.., it just focuses on a general area which an
activity requires to be able to take place.
3. GENERATING WORKSPACE REQUIREMENTS IN FINISHING PHASE

Workspace is a location where a construction activity is carried out. It can be a space for labourers working, a
place on which equipment such as scaffolds is set up, or a specific area for storing material. Bargstdt and
Elmahdi have classified it into 13 types and assigned them into three groups (figure 2) (Bargstdt and Elmahdi,
2010b). In this approach, however, just the type of workspace related to a process area is regarded. It does also
not take into account the difference between the place for labourers, for equipment or material. A workspace, in
this case, is considered a sum of area which an activity required to be able to take place; it includes spaces for
labourers, in-place material, equipment, hazard spaces and area for debris as well.
A workspace can be defined in various ways with different levels of complexity. It can be identified with fixed
or variable volumes, located at a static position during whole the time an activity is carried out, or at a dynamic
position which varies along a products length over an activitys duration. Besides, some activities may have
more than one possibility of workspace position. In this research, it should be also mentioned that, no matter how
the workspace is formulated, its position varies with corresponding products positions. The following presents
the workspace formulation and the workspace allocation in order from simple to complex.
523

Figure 2. Classification of workspaces (Bargstdt and Elmahdi, 2010b)

Figure 3. Generating workspace requirements
Construction
Activity
process
fabricate
disasssemble
supply
transport
move
deliver
long time store
short time store
1) Labourer
2) In-Place Storage
3) Tools & Equipment
4) Hazard
Process Area Supply Area
8) Staging Area
9) Prefabrication
Area
6) Initial Delivery
7) Storage Area
Transport Path
10) Material
11) Debris
13) Labourer
12) Tools & Equipment
requires
requires
requires
6) Debris
524
3.1. Workspace Formulation

3.1.1. Fixed Area

In a finishing phase, several products have standardized sizes such as windows, doors. In order to install same
products with a same size, workspaces required have same area. In this case, a fixed area with a specific length
and width is suitable to identify a working area.

Figure 4. Generating a fixed area
3.1.2. Variable Area

Trades in a finishing execution phase are related much with products of walls, floors, and etc., which have
different sizes even in a same project. Workspaces required for conducting these trades, therefore, have different
dimensions depending on size of products. In this case, variable area which can vary with length and height of a
wall, or length and width of a floor is more convenient for identifying their working areas (figure 5 & 6).
Figure 5. Variable area for products lying in vertical
planes
Figure 6. Variable are for products lying in
horizontal or leaning planes

3.1.3. Room

According to observing construction sites, it is recognized that various trades like painting, flooring, ceiling,
installing sanitary, etc., once take place in a small room, usually take whole the room during their executing
duration, due to the separation of working area (figure 7). Although, in some cases, not whole the area of a room
is used totally at a point of time such as in trades like painting, plastering; but in other cases, a room is too small
525
if compared to a comfortable area which a trade should require such as in trades related to sanitary. It is
beneficial, therefore, to collect a room to identify a workspace.

(*)
(*) Source: Internet

Figure 7. Using a room as a workspace

However a question might be raised here is that how is considered a room small enough to take whole a room to
be a workspace for an activity. This question will be answered in the next section of workspace allocation.
Besides, identifying a room, which depends on position of products and type of trades, is also regarded in that
part.
3.2. Workspace Allocation

Like other resources, workspace has a property of temporary. A given workspace does not exist in whole the
duration of a project, but it is just required in a short time for a task, even for a part of a task.
Before going to the detail of workspace allocation, it should be mentioned that the terms object and product are
used in different meaning in this paper. Object refers to a general element such as wall, floor, ceil, etc. In order
to create an object, various trades must be required, e.g. to produce a wall, the trades of masonry, plastering,
painting, etc. must be carried out. Which a trade creates is called a product. That means, for example, a product
of the trade of masonry is a brick wall, a product of the trade of plastering is a plastered face of a wall.
3.2.1. Static Position

In the approach, allocating a workspace with a static position means that the position and dimension of the
workspace is not changed in whole the duration of the task. Such a kind of workspace allocating is suitable in the
case that products of a task stand not far away from each other.
a. One-surface Trade
Trades like plastering, painting, flooring, ceiling, etc., whose products just can be carried out from one side of
their objects, are called one-surface trades in this approach. The workspace required by them is just located at the
position corresponding to the side of products. Allocating a workspace for a task of a single segment belonged to
such the trades is quite simple (figure 8).
526

Figure 8. Allocating a workspace for a single segment
However, if a trade is always decomposed into detailed tasks, each of which takes place just to create a product
as a single segment, then various disadvantages following should be considered.
1. A large number of detailed tasks are created as consequences. With such a structure of schedule, the number
of possibilities of manipulating the schedule is numerous. This takes a lot of time to searching for a suitable
solution.
2. It does not take the relationship of products positions into account. The fact that when a group of given
product segments stand not so far away from each other, they are considered a product of a task which is carried
out with a same crew and a same workspace, instead of separating them apart for different activities. This help to
save the time of labourers traveling and equipment relocating.
With the above weak points, a method to allocate workspaces for a group of product segments is proposed based
on the fact that a group of products, for trades in an interior finishing phase, is created when they stand in a same
room; for example several faces of wall in a same room are collected for an activity of painting. And the
workspace for them is the room itself. However, a question raised here is that whether or not all of product
segments in a same room should be always assigned to a unique task? Just take a look in the figure 9.

Figure 9. Allocating workspace for a group of segments with a room
527
The trade of painting of four faces of walls around should occupy whole the room number one (1) during its
duration. The room is small and if another trade locates its workspace there, it may disturb one another and cause
loss of productivity. However for the room number two (2), it is large enough so that more than one trade can
work together there without disturbing. If all faces of walls are assigned into a unique task, then whole the area
of the room must be locked during the tasks duration, no any trade else could take place at that time there. The
occupation of a large workspace in a long time in this case makes the schedule not flexible and efficient.
So how to recognize a room small or large, or in other word, when should we collect all the elements around a
room for a unique task and when should we assign them into multiple tasks? In order to solve this problem, a
variable of the maximal distance from two objects will be considered (figure 10). If the maximal distance
between objects in a room is greater than the permitted value, then products are considered single segments.

Figure 10. The factor determines whether or not a room is considered as a workspace
b. Two-surface trade
Trades like masonry, installing windows, etc., whose products can be carried out from both sides of their objects,
are called two-surface trades in this approach. Taking masonry like an example, this kind of trade can be take
place in one of two sides of objects or in both of them as well (figure 11).

Figure 11. Building a brick wall (source: internet)
528

Figure 12. Possibilities of allocating workspace for two-surface trades
Allocating workspaces for such these trades includes multiple possibilities (figure 12 and 13). In some cases, two
sides of objects play equal roles in identifying workspaces; that means no matter which side of an object a
workspace is located, the amount of its area is the same; for instance, workspaces for installing inside windows.
In the other cases, however, they take different parts. Workspace for building a brick wall from inside and
outside of a building is an example for this case. When a wall is built from outside, the width of workspace
required just is the width of scaffolds, about 1.25 meters. However if it is carried out from inside, it requires,
maybe, up to 4 meters of width.
Therefore, for two-surface trades, two kinds of workspace formulations must be defined when the role of object
sides is not equal. It is also require a factor to distinguish them from one another. In the approach, a vector
defined by the start and end points of an object is used to identify the position of two different workspaces
(figure 13).

Figure 13. Identifying type of workspace
allocating

Figure 14. Identifying the positions of workspace 1 and
workspace 2
3.2.2. Dynamic Position

Allocating workspace with dynamic position, in the approach, means that the position of workspace can be
changed during a tasks duration (figure 15 and 16). In order to reduce the complexity, the interval time (t) to
change the position of workspace should be an integer number and its unit should be day or week.
(1)
(2)
(1)
(2)
product
Legends
workspace
Workspace is just located in one side Workspace is located in both sides
529

Figure 15. Sliding workspace
This kind of allocating workspaces is suitable to tasks which are carried out to produce continuous products
which lay in a large area, and, very importantly, their equipment and material are moveable over time. The trades
of plastering, painting, flooring, wiring in a large room are a good example for this situation. Compared to the
method of decomposing products and using static positions to allocate workspaces, using dynamic position has
some following advantages.

Figure 16. Two direction moving workspace
1. It creates fewer tasks in detailed schedule, so it is efficient to control.
2. It guaranties the continuous property of trades. For example, if a very long wall is decomposed into several
segments, it is not sure that the wall is built continuously because the independence of tasks in a schedule.
However, if it still exists as a product and using method of dynamic position for allocating workspaces, it is built
continuously and the number of interruption for the trade is controllable, the time for traveling of labourers and
relocating equipment therefore is reduced.
However, it also contains several below disadvantages compared to the method of decomposing products and
using static positions.
1. The structure of workspace data is more complex.
2. The production rate must be defined at the planning period via functions f(t), f1(t) and f2(t); therefore it cannot
be changed during manipulating process of a schedule. In contrast, in the case of using decomposing method,
two or more segments can be carried out at the same time or different durations depending on the availability of
crew. It means that the production rate can be changed during manipulating a schedule.
In summary, there is not any method which totally dominates the other. Allocating workspaces with static or
dynamic positions should be based on the algorithm, the capability of input data of the model, and the type of
products in generating workspace requirements.
530
4. FUTURE WORKS

Besides workspace allocation, detailed planning also much involves crew distributions. Allocating crew on site
will define production rate and then determine the duration a workspace exists. In return, the number of
labourers, those can be distributed to an activity, depends on the amount of its available workspace. In order to
support for automating of generating detailed planning, in the future, a model for allocating crew based on
available workspaces and products shape will be conducted. Such the model of generating workspace
requirements and crew allocations, after that, will be implemented in a BIM based drawing, in which types, sizes,
positions and shapes of objects can be automatically recognized. So that an input data of workspaces and crews
will be automatically create. This overcomes the current obstacles and promises of bringing a detailed schedule
into practice.
5. REFERENCES
Akinci B, Fischen M, Levitt R, et al. (2002a) Formalization and Automation of Time-Space Conflict Analysis.
Journal of Computing in Civil Engineering 16: 124-134.
Akinci B, Fischer M and Kunz J. (2002b) Automated Generation of Work Spaces Required by Construction
Activities. Journal of Construction Engineering and Management 128: 306-315.
Akinci B, Fischer M, Kunz J, et al. (2002c) Representing Work Spaces Generically in Construction Method
Models. Journal of Construction Engineering and Management 128: 296-305.
Bansal VK. (2011) Use of GIS and Topology in the Identification and Resolution of Space Conflicts. Journal of
Computing in Civil Engineering 25: 159-171.
Bargstdt H-J and Elmahdi A. (2010a) Automatic Generation of workspace Requirements Using Qualitative and
Quantitative Description. 10th International Conference on Construction Applications of Virtual Reality. Japan:
Sendai CONVR2010 Organizing Committee, Japan, 131-137.
Bargstdt H-J and Elmahdi A. (2010b) Simulation von Bauprozessen - ein Qualittssprung in der
Arbeitsvorbereitung. 8. Grazer Baubetriebs- und Bauwirtschaftssymposium Graz Tecnhnische Universitt Graz,
Germany, 131-146.
Dawood N and Mallasi Z. (2006) Construction workspace planning : Assignment and analysis utilizing 4D
visualization technologies. Computer-Aided Civil and Infrastructure Engineering 21: 498-513.
Elmahdi A, Wu I-C and Bargstdt H-J. (2011) 4D Grid-based simulation framework for facilitating workspace
management. 11th International Conference on Construction Applications of Virtual Reality. Weimar Germany:
Bauhaus Universitt Weimar, 403-412.
Hana AS, Russell JS and Emerson EO. (2008) Stacking of Trades. In: Rojas EM (ed) Construction Productivity:
A Practical Guide for Building and Electrical Contractors. J Ross Publishing, 75-110.
Kaming PF, Holt GD, Kometa ST, et al. (1998) Severity diagnosis of productivity problemsa reliability
analysis. International Journal of Project Management 16: 107-113.
Mallasi Z. (2006) Dynamic Quantification and Analysis of the Construction Workspace Congestion Utilising 4D
Visualisation. Automation in Construction 15: 640-655.
Mallasi Z and Dawood N. (2001) Assessing space criticality in sequencing and identifying execution patterns for
construction activities using VR visualisations. ARCOM doctoral research workshop: Simulation and modelling
in construction. Edinburgh University, UK, 22-27.
Riley DR and Sanvido VE. (1995) Patterns of Construction-Space Use in Multistory Buildings. Journal of
Construction Engineering and Management 121: 464-473.
Riley DR and Sanvido VE. (1997) Space Planning Method for Multistory Building Construction. Journal of
Construction Engineering and Management 123: 171-180.
Rivas RA, Borcherding JD, Gonzalez V, et al. (2011) Analysis of Factors Influencing Productivity Using
Craftsmen Questionnaires: Case Study in a Chilean Construction Company. Journal of Construction Engineering
and Management 137: 312-320.
Winch GM and North S. (2006) Critical Space Analysis. Journal of Construction Engineering and Management
132: 473-481.
Zhang C, Hammad A, Zayed TM, et al. (2007) Cell-based representation and analysis of spatial resources in
construction simulation. Automation in Construction 16: 436-448.
531

ONSITE PLANNING OF 3D SCANNING FOR OUTDOOR
CONSTRUCTIONS WITH MATHEMATICAL PROGRAMMING
Hiroshige Dan, Yoshihiro Yasumuro, Taisuke Ishigaki, Tatsuaki Nishigata
Kansai University
ABSTRACT: A 3D scanner is capable of capturing surface shapes of the objects as a set of "point cloud" and is
extending its applicability toward examining, re-designing and preserving the existing constructions and cultural
heritages. One of the key issues to collect overall surface data of outdoor constructions is a trade-off between cost
and scan quality. Multiple measurements from different viewpoints decrease miss-scanned surfaces by occlusion.
However, increase of viewpoints easily produces huge redundant point cloud, which costs so much computation
and memory consumption as well as scanning time and labor. The authors have been developed a mathematical
programming approach to optimize the view planning, using a 2D diagram. The view planning is modeled as a
pair of integer programming problems to find the least number of viewpoints for sweeping all-around surfaces,
and to find the arrangement of the viewpoints for maximizing scan density. This paper focuses on bringing this
approach to an onsite work application, which allows users to perform try-and-error tasks according to the
conditions and the examination on the spot. The basic idea is to provide an interactive graphical user interface
(GUI) to reflect the onsite conditions in solving the view planning problems. This framework enables a
field-oriented feedback to acquire a practical view plan, taking into account the information which is impossible
to know in advance. The GUI provides a capability to allow the user to edit the 2D diagram intuitively, and then
interpret the edited diagram into the parameters of the mathematical programming models to find the newly
optimized scanning plan. We also enhance the visualizing functionality to estimate the data quality by simulating
the plan on the 2D diagram in order to decrease the chances of miss-measurement before actual measurement.
This paper also shows and discusses the effectiveness of our approach by experiments along a scenario from 2D
diagram assessment to actual scanning through onsite examinations.
KEYWORDS: View Planning, 3D scanner, Mathematical Programming, Graphical User Interface (GUI), Onsite
Planning.
1. INTRODUCTION
A 3D scanner is a surface imaging system which is based on accurate distance measurement by electro-optical
distance measurement. We can obtain the surface data of measurement objects by performing a number of
independent measurements, and a 3D image emerges by merging them. In this paper, we assume that
measurement objects are buildings and a ground plan around them is available in advance.
One of the most difficult problems to collect the complete surface data of measurement objects is to avoid
occlusions. For collecting the complete data, we have to measure the objects from multiple viewpoints usually.
However, examining multiple surface visibilities relative to variable multiple viewpoints is a complicated
problem. Moreover, multiple measurements require plenty of time and labor, and each measurement gives a data
set consisting of hundreds of millions of 3D points to be processed for further computations. Hence, it is very
important to make an effective measurement plan a priori for avoiding redundancy for both labor and
computational costs.
View planning with a laser scanner has been developed as trial-based schemes, which segment scanned and
unscanned regions in a target area to find the next best scanning viewpoint for minimizing the unscanned regions.
This approach includes the methods for promoting efficiency of sequential scanning (Asai et al. 2007, Pitto 1999,
Pulli 1999) and three-dimensional environmental map generation by autonomous mobile robots (Blaer and Allen
2009, Grabowski et al. 2003, Surmann et al. 2003).
For this problem, the authors have proposed the method for making a measurement plan by using a ground plan
of a target area (Dan et al. 2010). This method generates an optimized initial view plan beforehand by using
mathematical programming. Estimating the minimum scale and the complexity of the whole scanning task
before starting the scanning process is important and useful, especially for large scale outdoor measurement.
However, this method is based only on the data which is obtained from a ground plan. Hence, it is vulnerable to
the change of the onsite conditions: it is difficult to reflect the difference between the ground plan and the real
532

environment. Therefore, this paper focuses on bringing this approach to an onsite work application, which allows
users to perform try-and-error tasks, according to the conditions and the examination on the spot. Figure 1 shows
a measurement work flow which is proposed in this paper.
To enable this work flow, we provide an interactive graphical user interface (GUI) to reflect the onsite conditions
in solving the view planning problems. Figure 2 shows the system functionality for the objective work flow.
This framework enables a field-oriented feedback to acquire a practical view plan, taking into account the
information which is impossible to know in advance. The GUI provides a capability to allow the user to edit the
2D diagram intuitively, and then interpret the edited diagram into the parameters of the mathematical
programming models to find the newly optimized scanning plan. We also enhance the visualizing functionality to
estimate the data quality by simulating the plan on the 2D diagram in order to decrease the chances of
miss-measurement before actual measurement. This paper also shows and discusses the effectiveness of our
approach by experiments along a scenario from 2D diagram assessment to actual scanning through onsite
examinations. We can expect that this GUI helps the people who are non-professional on mathematical
programming to make a view plan which reflects the onsite conditions and has the good effectiveness.
2. VIEW PLANNING BY MATHEMATICAL PROGRAMMING
In this section, we explain the view planning method by using mathematical programming. This method has
been proposed in (Dan et al. 2010). For completeness of this paper, we restate the method in the following.
2.1 Definition of Measurement Problems
As we wrote in Section 1, a view plan deeply depends on shapes of structures, self and mutual occlusions, and
existence of obstacles. The visibility and occluding property of walls vary with the change of viewpoints
nonlinearly and discontinuously, so it is very difficult to determine the least number of viewpoints to measure all
the walls.
Accordingly, we define one of the measurement problems as follows:
[Problem 1] How many viewpoints do we need to measure all the walls of constructions in the target area?
Figure 1: Objective human work flow for executing 3D scanning measurement
Figure 2: System functionality for the objective work flow
533

On the other hand, too few viewpoints may cause deterioration of quality of measurement. The amount of data
for a wall is proportionate to the scanning density, and the scanning density is proportionate to an angular
distance from a viewpoint to a wall.

From this fact, we have to consider the following problem:
[Problem 2] How we can determine an optimal layout of viewpoints under
the limitation of the number of measurements?
In this paper, we propose a method for solving Problems 1 and 2 with
mathematical programming.
We assume that a 2D diagram will be available for making a view plan, as
a ground plan, a geological plan or an aerial photograph of the target area. Moreover, we suppose that candidate
points of measurement would be placed in the target area beforehand. One of layout examples is to place
candidate points on a grid in the target area. We can expect that the smaller interval of grid reduces the
probability of existence of unmeasurable walls. In addition to this, we assume that candidate points are not on
outlines or inside of constructions.
2.2 Mathematical Programming Models
In this section, we formulate Problems 1 and 2 as 0-1 integer programming problems.
We will use the following symbols in mathematical programming models.
[Sets and Indexes]
- : candidate points for measurement,
- : walls of constructions in the target area.
[Variables]

[Parameters]

We calculate the value of parameters from a 2D diagram of the target area. The calculation method will be
introduced in Section 2.4.
In this paper, we propose these two mathematical programming models:
(1)
(2)
The objective function of (1) is to minimize the number of viewpoints. Also, the term in the first constraint
of (1) means as follows:

Figure 1: Angular distance
534

Therefore, the first constraint of (1) means that all the walls are measured from one viewpoint at least.
The term of the objective function of (2) means as follows:

Therefore, the objective function of (2) is to maximize the sum of angular distances, that is, the density of
scanning. In addition, the first and third constraints of (2) are the same as that of (1). Moreover, the second
constraint of (2) is to restrict the number of measurements less than or equal to , that is, the optimal objective
value of (1).
As we see above, the mathematical programming problems (1) and (2) are the counterparts of Problems 1 and 2,
respectively.
To solve mathematical programming problems (1) and (2), we have to prepare the following parameters:
- positional information of walls
- visibility of walls from candidate points ( )
- angular distances from candidate points to walls ( )
In this following, we describe a method to calculate the value of these parameters.
2.3 Positional Information of Walls
We will obtain positional information of walls from a 2D diagram of the target area. In this paper, we will
approximate a wall by a line segment, and the position of a wall is determined by two endpoints of a line
segment. The algorithm which we propose to calculate positional information of walls is as follows:
Algorithm 1: Calculation of Positional Information of Walls
Step 1: Extract constructions from a 2D diagram of the target area by an image manipulation software, and make
a binary image.
Step 2: Find outlines of constructions by the binary image.
Step 3: Approximate the outlines by line segments.
In this paper, OpenCV is employed in Algorithm 1. Concretely, we use the functions cvFindContours (Suzuki
and Abe 1985) and cvApproxPoly (Douglas and Peucker 1973) of OpenCV in Step 2 and 3, respectively. In the
rest of this paper, we assume that walls which are extracted by Algorithm 1 are not overlapped or intersected.
2.4 Visibility and Angular Distance
From the positional information of the walls obtained by Algorithm 1, we check the visibility of the walls, that is,
, and calculate the angular distance if a wall is measurable from a candidate point ( ).
First, we define the state of measurable. In this paper, a wall is measurable from a candidate point if there is no
occlusion by the other walls. Namely, we call it measurable that an entire wall is visible from the candidate point,
and unmeasurable that any part of the wall is invisible.
Now we set that is a candidate point in a 2D diagram. Moreover, and
are endpoints of a line segment which stands for a wall , where

and holds (Figure 3). Note that we can exclude a wall when holds, because this
means that an angular distance is 0 when a wall is measured from a candidate point , so a wall is
unmeasurable in a case , or a wall does not occlude from in a case .

535

We assume that there is no wall in a direction of from . In
(Dan et al. 2010), we have proposed some methods for dealing with
such walls.
denotes two angles which are made by a candidate point and two
endpoints of a wall, such that

Note that does not have a direction, then holds.
We can classify the relationship between a wall and the other
wall as follows:
(a) Case that and share an endpoint
(a-1) (Figure 4, left), (a-2) (Figure 4, right), (a-3) , (a-4)
(b) Case that and does not share an endpoint
(b-1) (Figure 5, left), (b-2) (Figure 5, right), (b-3)
(b-4) .

Figure 4: Examples of (a-1) and (a-2) Figure 5: Examples of (b-1) and (b-2)
In addition, denotes the distance from to a wall in a direction (Figure 6).
Now we describe an algorithm to judge whether a wall occludes a measurement of a wall from a
candidate point or not.
Algorithm 2: Judgment of Visibilities of Walls
(a-1) If holds, then is not occluded by (from , is
nearer than ). Otherwise, is unmeasurable ( is nearer than , so
disturbs measurement of ). Note that does not occur because
we assume that walls are not overlapped.
(a-2) is not occluded by ( does not exist in a direction which
exists).
(a-3) The same as (a-2).
(a-4) If holds, then is not occluded by . Otherwise, is
unmeasurable.
(b-1) is not occluded by ( does not exist in a direction which
exists).
(b-2) If holds, then is not occluded by ( is nearer than ). Otherwise, is
unmeasurable ( is nearer than , so disturbs measurement of ). Note that does not
occur because we assume that walls are not intersected.
(b-3) If holds, then is not occluded by . Otherwise, is unmeasurable.
(b-4) The same as (b-1).
Applying this algorithm for all the walls , we can check the visibility of a wall from a candidate
point . Moreover, we can set by applying this algorithm for all the combinations of candidate points and
Figure 3: An example of a candidate point
and endpoints of a wall
Figure 6: Distance from a candidate
point to a wall in a direction
536

walls.
Also, we can easily calculate an angular distance . It is enough to calculate
for all combinations which holds.
2.5 Minimal Angular Distance
In the previous section, we treat a wall is measurable from a candidate point when there is no obstacle
between and . However, if an angular distance is smaller than expected, it should be treated as
unmesurable. Therefore, in the following of this paper, we consider the minimal angular distance, which is a
threshold of visibility. If an angular distance is less than the minimal angular distance, then a wall is
treated as unmeasurable from a candidate point even if there is no obstacle between and .
3. USER INTERFACE FOR ONSITE CHECK
As we wrote in Sections 1 and 2, our mathematical programming approach uses the data which is extracted from
a ground plan of the target area. Therefore, when the ground plan does not express the environmental conditions
on the spot, our approach would calculate an infeasible view plan. For example, vulnerability of footholds and
plants around the buildings, which are not recorded on a ground plan usually, may spoil the measurement
feasibility. Also, the feature of the shape of the target buildings makes it difficult to scan them completely. In this
research, we propose a GUI to modify an image of a ground plan to reflect such onsite conditions. Also, this GUI
has a function to subdivide a wall on the ground plan into some small parts for improving the measurement
quality. Using this GUI, we can recalculate the optimal view plan which reflects the practical conditions on the
spot. In the following, we explain the functions of our proposed GUI.
3.1 Correction of Area for Measurement
As we said, we can calculate the optimal view plan in advance by using the method in Section 2, but usually it
would not reflect the onsite conditions exactly. For such cases, our proposed GUI has a function to edit an image
of a ground plan on the spot. Especially, we can add/delete an area in which we can/cannot measure the target
objects. After that, we can recalculate the optimal view plan under the new conditions.
Figure 7: Candidate viewpoints limitation by editing map:
(The grid interval is 10 pixels and the minimum angular distance is 30 degrees.)
(a-1) (b-1)
(a-2)
(b-2)
537

Figure 7 is an example to show this function: green regions show target buildings, and white/gray regions are
available/not available for measurement. The image (a-1) in Figure 7 is a sample image which is extracted from
an original ground plan, and (a-2) is the optimal view plan for (a-1). It is obtained by the method in Section 2.
On the other hand, (b-1) is an example image which reflects the onsite conditions: the off-limits areas for
measurement are added to (a-1). Moreover, (b-2) is the optimal view for (b-1). Comparing (a-2) with (b-2), the
number of the measurement points on (b-2) is greater than (a-2) because of the increase of the infeasible area.
3.2 Subdivision of Walls
Figure 8: Scan quality control by subdividing wall segments
Wall contours approximated by the initial line segments (a-1) are subdivided to satisfy the length limitation of 100 pixels
(b-1) and 50 pixels (c-1). As the line segments increase, the initial viewpoints (a-2) are also rearranged to (b-2) and (c-2),
respectively.
When a wall is long to a certain extent, the measurement density may vary widely in a wall because of the
positional relationship between the wall and the measurement points. In such a case, our proposed GUI can
subdivide a wall into some parts virtually and improve the measurement quality on the whole wall. So, if we
measure and find walls whose measurement quality are lower than expected, then we can subdivide them
virtually by using this function and recalculate the optimal view plan under the new settings. This function
makes it possible to reflect the users demand for measurement quality into the view plan flexibly.
Figure 8 is an example of this function. The image (a-1) in Figure 8 is an example image which has no limit of
the length of wall, and (a-2) is the optimal view plan for (a-1). Also, the images (b-1) and (c-1) are the examples
whose limitation of the wall length are 100 and 50 pixels, and (b-2) and (c-2) are the optimal view plans for (b-1)
and (c-1), respectively. As we can see, the number of viewpoints increases as the limitation of the wall length
tightens.
4. EXPERIMENTS
In this section, we report some practical experiments by using our proposed GUI, whose functions are introduced
in the previous section. We would like to show the usability of our research through these experiments. The
experiments were executed with the following scenario:
Step 1: Making the optimal view plan without the knowledge of the onsite conditions,
Step 2: Making the optimal view plan reflecting the onsite conditions,
Step 3: Making the optimal view plan with subdividing some walls.
(a-1) (b-1) (c-1)
(a-2) (b-2) (b-3)
538

In these experiments, we use the 3D scanner LMS-Z420i (RIEGL). Moreover, we used reflectors appropriately
and merged the data obtained by multiple scans into one image.
4.1 Step 1: Making the optimal view plan without the knowledge of the onsite
conditions

First, we prepared the ground plan of the campus of Kansai University (Osaka, Japan) and extracted the target
building and its neighbor from the ground plan (Figure 9, 640 480 pixels). The green region is the target
building, and the white/gray region is the feasible/infeasible area for measurement.
Here, we calculated the optimal view plan by using the method in Section 2. As a result, we obtained the optimal
view plan, Figure 10, and found that the number of minimum viewpoints to scan the whole of the target
buildings walls is two.
4.2 Step 2: Making the optimal view plan reflecting
the onsite conditions
Next, we investigated the conditions around the target building, and
we found that there are some regions in which are inappropriate to
measure the walls: vulnerability of foothold, traffic of people, and so
on. Such places were treated as infeasible regions. Also, there are
trees and roofs along some walls and they could interrupt the
measurement of walls from some areas, so we treated such regions as
infeasible. On the other hand, we found some feasible regions which
were judged as infeasible areas from the ground plan. We treated
them as feasible areas on the GUI.
We recalculate the optimal view plan under this modification of
conditions, and obtained the optimal viewpoints (Figure 11), which are
almost the same as the result in the previous section. Moreover, we
measured the walls by the 3D scanner practically with this view plan, and the result is Figure 14, (a-1) and (a-2).
4.3 Step 3: Making the optimal view plan with subdividing some walls
From the result in the previous section (Figure 14, (a-1) and (a-2)), there are some parts whose measurement
quality is very poor. Then, we subdivided the walls by using the function in Section 3.2. We tried two patterns:
(i) the long sides of the building are divided into two parts, and (ii) the long sides are divided into three parts and
the narrow sides are divided into two parts (Figure 12). Also, we adopted the onsite environment which is used
in the previous section.
Figure 9: Ground plan of Kansai University (Osaka, Japan)
(Left: the ground plan of the whole campus, right: the ground plan around the
target building)
Figure 10: Optimal Viewpoints
in Step 1
in Step 2
539

In the case (i), we obtained the optimal view plan (Figure 13). Also, we measured the walls by a 3D scanner with
this view plan practically (Figure 14, (b-1) and (b-2)). Comparing (a-1), (a-2) and (b-1), (b-2) in Figure 14, we
found that the measurement result with the optimal view plan under the case (i) is better than that of the previous
section. On the other hand, we could not obtain the optimal view plan in the case (ii). The reason of this
infeasibility is the minimal angular distance: there is no candidate point which ensures the minimal angular
distance for some subdivided walls.
(a-1) (b-1)
(a-2) (b-2)
Figure 14: Density of the scanned data ((a): Step 2, (b): Step 3)
Figure 12: Subdivision of walls (Left: (i), right (ii))
in Step 3, case (i)
540

5. CONCLUSIONS
In this research, we proposed and implemented the GUI which can reflect the onsite conditions and calculate the
optimal view plan under it. Also, the experiments in the previous section show that the people who are
non-professional on mathematical programming can make the optimal and practical view plan which reflects the
onsite environment. Our implementation is immature now, but this research shows that our approach has much
prospect of making the onsite view plan adaptively.
6. ACKNOLEDGEMENTS
This work was supported by JSPS Grant-in-Aid for Scientific Research (C) (24510239).
7. REFERENCES
Asai T., Kanbara M. and Yokoya N. (2007). Data acquiring support system using recommendation degree map
for 3D outdoor modeling, Proceedings of SPIE, Vol. 6491 (doi: 10.1117/12.704292).
Blaer P. S. and Allen P. K. (2009). View planning and automated data acquisition for three-dimensional
modelingof complex sites, Journal of Field Robotics, Vol. 26, No. 11-12, 865-891.
Dan H., Yasumuro Y., Ishigaki T. and Nishigata T. (2010). Shape Measurement Planning of Outdoor
Constructions with Mathematical Programming and Its Applications, Proceedings of the 10th International
Conference on Construction Applications of Virtual Reality (CONVR2010), 319-328.
Douglas D. and Peucker T. (1973). Algorithms for the reduction of the number of points required to represent a
digitized line or its caricature, The Canadian Catographer, Vol. 10, No. 2, 112-122.
Grabowski R., Khosla P. and Choset H. (2003). Autonomous Exploration via Regions of Interest, Proceedings of
the IEEE/RSJ International Conference on Intelligent Robots and Systems, Vol. 2, 1691-1696(doi:
10.1109/IROS.2003.1248887).
Pitto R. (1999). A Solution to the Next Best View Problem for Automated Surface Acquisition, IEEE
Transactions on Pattern Analysis and Machine Intelligence, Vol 21. No. 10, 1060-1030.
Pulli K. (1999). Multiview Registration for Large Data Sets, Proceedings of Second International Conference on
3-D Imaging and Modeling, 160-168 (doi: 10.1109/IM.1999.805346).
Surmann H., Nuchter A. and Hertzberg J. (2003). An autonomous mobile robot with a 3D laser range finder for
3D exploration and digitalization of indoor environments, Robotics and Autonomous Systems, Vol. 45, No.3-4,
181-198.
Suzuki S. and Abe K. (1985). Topological structural analysis of digitized binary images by border following,
Computer Vision, Graphics, and Image Processing, Vol. 30, No. 1, 32-46.
541

GLOBAL PATH PLANNING IN 4D ENVIRONMENTS USING
TOPOLOGICAL MAPPING
Vitaly Semenov, Vladislav Zolotov & Constantine Kazakov
Institute for system programming RAS, Moscow
ABSTRACT: Global path planning is a challenging problem raised in many fields of research. It is of particular
interest to construction planning community facing the requirements of trustworthiness and feasibility of the
project schedules. Correct construction schedules must exclude conflicting situations at project sites, thereby
improving project productivity and reducing risks and waste. Some conflicts can be foreseen and avoided using
emerging 4D modeling and planning technologies. Modern 4D modeling systems enable to identify and report
simple clashes among construction elements, equipment units, workspaces occupying the same place at the same
time. However, they are not capable of revealing more sophisticated conflicts caused by lack of collision-free paths
to deliver the elements to final destination locations. Global path planning methods look promising to assure the
existence of collision-free paths for displaced elements. In particular, topological mapping methods have proved to
be well suited for single path planning requests in static 3D scenes. However, they could not be directly applied to
complex pseudo-dynamic scenes originating from 4D modeling and planning applications. In this paper a new
efficient method intended for global path planning in dynamic environments is presented. It generalizes existing
mapping techniques by means of incremental and concordant updates of all the deployed metric and topological
structures as the explored environment is being evolved under events appearing in discrete time moments. The
obtained results, both theoretical and experimental, showed that the proposed method gives a significant
performance gain for multiple path planning requests in complex pseudo-dynamic environments and can be
successfully applied for validation of construction project schedules against path conflicts.
KEYWORDS: 4D modeling, project planning and scheduling, validation, path planning.
1. INTRODUCTION
Global path planning is a challenging problem raised in many fields of research. It is of particular interest to
construction planning community facing the requirements of trustworthiness and feasibility of project schedules.
Correct construction schedules must exclude conflicting situations at project sites, thereby improving project
productivity and reducing risks and waste. Some conflicts can be foreseen and avoided using emerging 4D
modeling and planning technologies. These technologies are becoming important on complex construction
programmes facing the problems of advanced communication among stakeholders, better utilization of critical
resources and effective coordination of works taking into account both space and time factors.
Modern 4D modeling systems like Synchro, Autodesk Navisworks, Bentley Schedule Simulator, Intergraph
Schedule Review are used for basic functionalities to consolidate both 3D CAD models and scheduling
information delivered from project management systems, thereby allowing simulation of project activities in
space dimensions and across time. They are also capable of identifying and reporting simple clashes induced by
collisions or interferences of construction elements, equipment units and workspaces occupying the same place
at the same time. Nevertheless, more sophisticated conflicts caused by lack of collision-free paths to deliver the
elements to final destination locations can not be identified by these tools.
This sort of conflicts is especially crucial for large-scale pseudo-dynamic environments prevailing in 4D
modeling and planning applications. Indeed, most complex projects are usually simulated in pseudo-dynamic
mode under common suggestion that most, if not all, objects appear, disappear or move strongly in discrete time
moments in which the project activities usually start or finish. It is explained by the intensive computations
needed to carry out such analysis, as well as by enormous efforts to specify all the trajectories and kinematic
rules, the elements can move accordingly (Semenov et al. 2011c). But such a simplification may lead to
ambiguous situations when an object placed in its final destination location does not clash other objects, but
could not be physically delivered to this location along collision-free paths.
To reveal such spatio-temporal conflicts in construction schedules, an advanced validation has to be performed.
If an object appears in the scene at a fixed moment of time (or removed from it), the existence of a collision-free
path from a predefined outdoor location to a destination location (or from an original location to a final outdoor
542

location) is required. If an object is displaced in the scene, then we require the existence of a collision-free path
between original and destination locations.

(a) (b)
Fig. 1: An example of the path conflict in a pseudo-dynamic environment.
An example of the path conflict in a pseudo-dynamic environment is shown in Figure 1. The original placement
of an equipment unit (green ball) and materials (blue balls) is presented in Figure 1a. The project schedule
assumes the displacement of materials to the corridor and then, moving the equipment from one room to another.
The final placement of the objects is presented in Figure 1b. It can be seen that the environment is correct in
terms of the lack of explicit clashes, but the schedule could be not correctly performed as it contains a path
conflict caused by the lack of collision-free paths to deliver the equipment to the required destination because of
the already moved materials. It is noteworthy that if the displacement works are scheduled in the reverse order,
the project plan will be feasible.
Thus, the validation of construction schedules against path conflicts can be reinterpreted as a multiple path
planning problem. Extensive research efforts have been directed towards this problem in various statements
(Lamarche 2009). Most reports have concluded that the existing algorithms work well in simple 2D
environments, but have relatively high computational complexity that grows considerably with the input data
volume. The transition to the third and fourth dimensions makes the highlighted problem highly intractable.
Being targeted at exact or approximate metric representations of scenes, traditional local path planning methods
have significant limitations in case of large-scale environments. Their inability to use overall priory information
on the whole environment creates serious shortcoming in the path planning.
Topological schemes try to avoid these drawbacks by representing the original scene environment by means of
route graphs. Typically, the vertices of such graphs are associated with identifiable locations and edges with
possible routes between them. Topological schemas scale better than the metric ones, but being resistant to
geometric representation errors may yield incorrect or suboptimal solutions (Lavalle 2006). Nevertheless, they
look most promising for the discussed validation purposes. In particular, a method based on space-gate
extraction exhibits satisfactory scalability and performance for multiple path planning requests in static 3D
scenes. Details can be found in our work (Semenov et al. 2011a,b).
This paper introduces a new efficient method intended for multiple path planning in pseudo-dynamic 3D
environments. It generalizes the method mentioned above by means of incremental and concordant updates of all
the deployed metric and topological structures, as the explored environment is being evolved under discrete
appearing events. The obtained theoretical and experimental results show that the proposed incremental method
gives a significant performance gain for multiple path planning requests in large-scale pseudo-dynamic 3D
environments and can be successfully applied for validation of construction project schedules against path
conflicts.
The rest of the paper is organized as follows: in Section 2 we shortly introduce main principles and algorithmic
components of the space-gate extraction method in conformity with global path planning in static 3D
environments. Section 3 is devoted to a more detailed consideration of the proposed generalized method suitable
for dynamic environments. Particular emphasis is put on how the deployed metric and topological structures are
updated on discrete events to support the valid representation allowing efficient multiple path planning. In
Conclusions we present some theoretical estimates of the method complexity and preliminary experimental
results of its usage for validation of the real construction schedules against path conflicts.
543

2. PATH PLANNING IN STATIC 3D SCENES
2.1 Environment specification
We suggest the environment is composed of objects which may be geometric primitives, algebraic implicit and
parametric surfaces like quadrics, NURBS and Bezier patches, convex and non-convex polyhedrons, solid
bodies given by constructive solid geometry (CSG) or boundary representation (BREP). In contrast to many
works addressing to path planning (Thrun 2008, Fabrizi & Saffiotti 2000), no specific restrictions are imposed
on the geometric representation of both simulated environment and moved objects considered as its intrinsic
parts. It is only assumed that there is a common function to detect collisions between pairs of environmental
objects.
The behavior of all the environmental objects is well known and it corresponds to the pseudo-dynamic
simulation mode defined above. According to this mode, the environment can be considered as static except for a
finite set of discrete time moments when new objects may suddenly appear and existing objects may disappear or
move to new locations. In fact, due to inevitable discretization of time any dynamic environment is simulated as
pseudo-dynamic one with essentially shorter time intervals at which latent changes appear.
It is also suggested that at any time moment only one object may change its own state and the environment as
whole. Such approach assumes that the only active object is considered as movable to be subjected to path
planning and other passive objects as the environment obstacles. Next time moment, the objects may
exchange roles so that another object may become active as the previously active object is interpreted as a
passive obstacle.
Thus, we suggest that only one event may occur at one point at a time and the event is either object appearance
or object removal. Admitted events of the object displacement will be treated as pairs of corresponding events:
removal of the object in an original location and further its appearance in a final location. Such interpretation
remains valid for all the considered algorithms except for local planning techniques applied at final steps of the
method.
2.2 Main principles and underlying algorithms
The discussed method leverages local and global path planning techniques by combining both metric and
topological schemas. Topological map is extracted for the explored static environment and it is used for making
high-level decisions about perspective global routes over the scene. Then local path planner is used to perform
exact validation of the selected route using original metric representation of the scene and, if necessary, to
correct it in the immediate vicinity. Generally, the method assumes the following five phases:
At the first phase an occupancy octree is generated as a simplified metric representation of the whole
environment. Using the occupancy octree, a path planning problem can be solved more efficiently by successive
finding neighboring free cells, navigating over them and spatial reasoning when deciding how to move the object
from a starting point to its destination location. Importantly, it can be done under very general assumptions about
the simulated environment.
The occupancy octrees reflects the so-called adaptive space decomposition approach. It assumes subdividing the
environment space into cells and determining the occupancy status for each localized cell. The cells of the
deployed octree are then marked as grey, black or white depending on their occupancy status that points out
whether the cell is partially occupied, entirely full, or entirely empty. Usually, it is used to reduce the number of
cells suffered to the analysis and to waste less memory storage space and computation time. The regular shape of
the cells is maintained, but the cells are recursively reduced in size in order to cover the space more efficiently
and to maintain as much details as possible. It is a good adaptive tactic for the complex indoor/outdoor
environments simulating construction sites and containing large regions with the same traversability. An example
of the octree representation is shown in Figure 2b. It was generated for a simple building model presented in
Figure 2a.
To make the octree more constructive for spatial reasoning and topology extracting, it is enriched by distance
field values. Distance field computations correspond to the second phase of the method.

544

(a) (b)
(c)
(d) (e) (f)
Fig. 2: Illustrations to the global path planning method.
For this purpose empty cells store distances to the nearby obstacles. The Euclidian distance provides high
estimate for the radius of a solid ball which can be safely placed in the center of the cell without any risk to
collide with the environment obstacles. To simplify computations, the distance estimates can be obtained using
already deployed metric schema and avoiding consumable analysis of the original geometric representation. In
the presented algorithm the distances are computed from the center of each empty cell to the nearest faces, edges
or corners of the cells occupied, either entirely or partially, by the environment objects. The algorithm tends to
minimize the number of cells exposed to the analysis. It runs by traversing all empty octants, collecting
neighboring cells for each such octant and determining the distance to the nearest occupied cell or cells.
Neighboring cells are determined by finding common ancestor and by descending from it to adjacent cells of the
explored octant, thereby it proceeds in a way similar to the famous backtracking method (Kim & Lee 2009).
Figure 2c illustrates the distance field distribution computed for the building model and the occupancy octree
presented above.
Next, or the third phase, consists in determination of free regions and their identification as spaces or gates.
Indeed, the occupancy octree is a useful metric representation to navigate over empty octants, but being
represented by huge number of cells, it does not allow efficient coverage in the large-scale environments. The
observation that indoor environments consist of rooms sequentially connected via doors, corridors, stair wells
inspired us to follow space-gate reasoning paradigm and to support corresponding topological schema. Both
spaces and gates are considered as non-overlapping, simply connected subsets of empty cells of the occupancy
octree. The main difference between the introduced categories is that the spaces approximate large free regions
of the environment, whereas the gates small narrow regions.
The Euclidian distance field computed at the previous phase is utilized for detecting free regions of the
environment. Underlying principle is to find the cells where the distance field reaches local maxima. Each of
such a maximum originates a subset of empty, simply connected cells surrounding it and having distance values
not exceeding the local maximum. All cells connected in this way form a single space. Cells, which may be
assigned to different spaces, form gates. An example of space and gate identification is shown in Figure 2d.
The centers of extracted spaces and gates should be good initial approximation for the waypoints on paths
crossing corresponding spaces and gates. Indeed, the computed centers never leave free region of the
environment, they end to remain in the middle being situated as far from the obstacle boundaries as possible.
This makes the choice of such waypoints quite motivated.
All the recognized spaces and gates match corresponding vertices of the bipartite topological graph generated at
the fourth phase. Space vertices are connected by edges with incident gate vertices, but not with other space
vertices. In a similar way, gate vertices are connected to incident space vertices, rather than to other gate vertices.
Thus, spaces and gates are alternated when navigating over environment and traversing the topological graph. A
545

bipartite topological graph generated for the presented building model is shown in Figure 2e.
Finally, the fifths phase corresponds to search of the final routes in the 3D environment. It seems to be a right
idea to use a global algorithm only on a rough, larger scale, topological schema and to apply a local algorithm
for a detailed metric or geometric representation of the moved object and its immediate surroundings.
It looks just as humans do when determining how to get from one point to another in a building: initially they
only decide which rooms and doors to pass through and then they move carefully bypassing corners and small
obstacles. Therefore, the principle of combining global and local planners suit our purposes best because none of
the strategies, being applied separately or discordantly, gives good results.
Using the space-gate topological schema and graph search algorithms, we can determine at any moment
what the possible path to get to a certain destination is. Then the path is checked for potential collisions and, if
necessary, it is corrected using popular local planning algorithms like visibility graphs, Voronoi diagrams,
probabilistic roadmaps (PRM), rapidly exploring random trees (RRT), and potential fields. For more details see
the monograph (Lavalle 2006).
In our method we prefer a well-known modification of the RRT algorithm that assumes growing trees from
neighboring waypoints in opposite directions. In such way, the waypoints are linked and the resulting path is
formed as a collision-free polyline close to the original path. It can be suffered to additional post-processing to
reduce the number of waypoints and to make the path more convenient for practical navigation purposes. The
algorithm parameters like the penetration step and the allowable size of expanded trees help to control the
accuracy and to interrupt the process if the initial approximation turned up imperfect and the final result cannot
be obtained in reasonable time. In such critical situations the alternative routes must be determined by global
strategies and be carefully explored by local strategies to complete the problem solving. Figure 2f provides an
example of the resulting path in the presented building model obtained using both global and local planning
strategies.
3. PATH PLANNING IN PSEUDO-DYNAMIC 3D SCENES
As explained above, the global path planning in static 3D environments can be successfully accomplished using
adaptive cell decomposition, topological mapping and local planners. The method assumes preliminary
computing and deployment of auxiliary metric and topological data structures before solving the path planning
problem. It imposes serious problems on the discussed pseudo-dynamic environments originating from the 4D
modeling and planning applications. When changes are happening, the entire data structures of the metric and
topological schemas should be completely revamped or correctly updated.
Fortunately, the method allows to update the data structures on typical events like object appearance, object
removal and object displacement for reasonable CPU time. It makes possible to maintain the entire data
structures in the representation necessary for multiple path planning and validation of project schedules in spite
of path conflicts.
Lets suggest that all the metric and topological structures have been already deployed for 4D environment at a
fixed time moment and a new event connected with the appearance of the new environmental object has been
produced. For brevity, we omit here other types of events which can be considered and processed in a similar
way.
First of all, the occupancy octree should be updated. The traditional algorithm of the octree generation begins
with imposing a large size cell over the entire planning space. If a grid cell is partially occupied, it is sub-divided
into eight equal subparts or octants, which are then reapplied to the planning space. These octants are then
recursively subdivided again and again until each of the cells is either entirely full or entirely empty. The
subdivision process is also interrupted for refined cells if their size becomes equal or smaller than a given
tolerance of the generated metric representation . The resulting octree has grid cells of varying size and
concentration, but the cell boundaries coincide very closely with the obstacle boundaries (not further than ).
Note that sometimes it is difficult to identify entirely full cells for the environments consisted of the so-called
polygon soups rather than solid primitives or assemblies. In such situations the subdivision process has to be
recursively continued under suggestion that the cells are partially occupied regions. By construction, white and
black cells are always leaves, but grey cells may be intermediate octants too. Grey leaf cells have a size equal to
546

the given tolerance. Other cells may have a larger size.
The proposed incremental algorithm implies local updating of the octee structure on the object appearance. If an
object is located inside the octree space, then the lowest octant covering the whole object is determined. If the
object is located outside the octree space, then the octree is expanded by adding top level octants and then the
lowest covering octant is determined. It reveals that only those cells that belong to the covering octant, have to
be updated. The status of other cells and their internal structure remain valid, which eliminates the need to fully
reconstruct the octree representation. Updating costs depend on the object sizes and the prescribed tolerance .
However, if the scene contains n identical objects isolated one from other, then the average updating costs are
smaller than complete deployment costs in n times, which gives reason to expect a proper scalability of the
incremental method at this phase.
The second phase implies updating distance field values assigned to the empty cells of the octree and/or
computing such values for newly created cells. An important advantage of the proposed algorithm is that the
necessary computations can be localized in the immediate vicinity of the octants that suffered the changes.

(a) (b)
Fig. 3: (a) Estimated search area for a fast determination of the distance field, (b) Localised octants suffered to
recalculation of distance field.
To compute distances to the nearby obstacles, the algorithm collects direct and indirect neighbors which are
located not farther than the search radius r from the explored octant. It enables to avoid the redundant analysis of
remote cells and to minimize total computations if some occupied neighbors have already been found. An initial
estimation for the search radius is given by the expression )
2
3
( 3 d a , where a is the linear size of the
explored octant and d is the size of the smallest cell coincident with the given metric tolerance. Indeed, by
construction the parent of the explored empty octant may be only a grey cell and it must contain at least one
occupied child cell. Its evident that it is located not farther than ) ( 3 d a from the center of the parent cell
and not farther than )
2
3
( 3 d a from the center of the explored octant. As new neighbors are found and
the distance value is decreased, the estimated search radius r can be refined and replaced by the current value of
the distance. This leads to additional savings on traversing and processing cells. The estimated search area and
the occupied cell nearby to the central empty octant are shown in Figure 3a as a circle and a bold arrow
correspondingly.
To determine cells which must be undergone to distance field updates, the radius estimate obtained above can be
directly utilized. When updating the octree structure, descending process starts from the top and continues until
the lowest covering octant is found. At every step empty octants are checked to see whether the appeared object
547

occupies them or not. It is evident that if the empty octants are located farther than )
2
3
( 3 d a from the
object, they should be eliminated from the analysis and the updates as the object could not affect the assigned
distance values. Indeed, by the occupancy octree construction, the nearest obstacle is located not farther than the
given radius estimate from the center of the empty cell. All the children cells of the covering octant must be
recursively updated by means of the recalculation of distance values. Figure 3b presents an example of such a
recalculation with the object appearance. The appeared corner object is drawn black, the octants suffered from
recalculation are shown as shaded squares.
Lets consider how the space-gate topological map can be modified to correspond to the updated representation
of the environment. Remember that according to the original method, the spaces and gates are defined and
identified using the following definitions of the adjacency and dominancy relations.
The cells C c c , are adjacent (or c c ~ ) if and only if there is a common face belonging to boundaries
of both cells. The cell C c dominates over the cell C c (or c c > ) if and only if the cells are
adjacent and estimated distance value for the cell c is larger than the corresponding value for the cell c .
Regions C R R R ,... , ,
3 2 1
are subsets of cells obtained by transitive closure of the dominancy relation on a
set of all empty cells.
Spaces C S S S ,... , ,
3 2 1
are non-intersecting subsets of regions ...} { \
3 2 1 1
= R R R S ,
...} { \
3 1 2 2
= R R R S , ...} { \
2 1 3 3
= R R R S .
Gates C G G G ,... , ,
3 2 1
are subsets of cells obtained by transitive closure on set of adjacent cells owned by
different spaces }... { } { } {
3 2 3 1 2 1
R R R R R R G = .
The presented definitions are quite constructive as they give a discipline how to spaces and gates can be
identified based on the occupancy octree with the supplied distance field. First of all, the dominancy relation
must be established among the cells and then the transitive closure of this relation on a set of the cells must be
determined. It enables to extract cells originating spaces as well as to prescribe all other cells to them. The
adjacent cells prescribed to different spaces form gates. All the recognized spaces and gates match the
corresponding vertices of the bipartite topological graph. The adjacent spaces and gates are connected by edges
in the graph. Figure 4 illustrates this algorithmic phase providing both metric and topological representations of
the 4D indoor environment at two consecutive time moments. A new wall is installed in the center of a large
room which causes subdivision of the space
0
S into two spaces
5 4
, S S as well as emergency of two new
gates
3 2
,G G . The original and updated octree structures deployed for the environment are presented in Figure
548

4a,c. The corresponding space-gate graphs are shown in Figure 4b,d.

(a)

(b)

(c)

(d)
Fig. 4: (a),(b) Original metric and topological representations of the 4D environment, (c),(d) Updated
representations of the environment.
Under the updates carried out at the considered algorithmic phases, new cells can be created in the octree
representation, some cells can be removed from it and some cells can change their distance values. Then the
previously extracted spaces and gates must be modified and topological map must be corrected in accordance
with the changed dominancy relations. For this purpose, a local propagation algorithm can be applied. It starts
from the cells that underwent changes and captures more and more neighboring cells until their dominancy is
changed. The algorithm stops if all such cells have passed and dominancy status is correctly recalculated. For all
the passed cells the transitive closure is determined and newly recognized spaces and gates are matched to the
graph.
Its of great importance that due to the propagation algorithm and the restraining criteria, all the performed
computations are localized by octants immediately intersected by the appeared object and their neighbors that
suffered the needed additional recalculations of the distance field and the dominancy. Figure 4 illustrates this fact
clearly.
At final phase, the perspective routes are found using well-known graph search algorithms, particularly, the
classical Dijkstras algorithm that solves the single-source shortest path problem for arbitrary directed graphs
with unbounded nonnegative weights (Dijkstra 1959). Then the found routes are validated and, if necessary, are
corrected using RRT-based local planners. Thus, no incremental versions of the method are used for solving
target problems.
4. EXPERIMENTS
To prove the efficiency, feasibility and suitability of the proposed incremental method for the highlighted
problems of validation of construction schedules, lets consider a typical 4D environment originating from a
large-scale building project. Imagine that the current status of the project corresponds to a multi-storey building
with a relatively large number of rooms, doors, corridors, and stairs and the working schedule assumes the
installation of a large-size equipment unit or an engineering facility of a complicated form in one of the rooms.
549

The use of the global path planning method for this validation purpose would require a complete deployment of
all the metric and topological structures, search of routes in the space-gate graph and then, most likely, their final
correction by local planners. Therefore, the cost of a single path planning request includes significant CPU
expenses on the preliminary deployment of the necessary structures.
The incremental method gives an opportunity to avoid the repeated and expensive deployment of the necessary
structures. Instead of the deployment the structures are updated being maintained in the representation allowing
immediate and fast search of perspective global routes. For the considered 4D environment, all updates are
localized by the octants of the room where the new element is installed. As the number of octants belonging to
the room is fewer than the total number of the octants in the building octree in a fixed significant factor, the costs
on preliminary deployment of all the metric and topological structures become comparable with the costs on
their incremental updates during the whole validation process.
The additional costs necessary for searching routes in the graph and final correction of the selected routes take
relatively low CPU resources and, therefore, slightly change the comparative performance estimates.
To validate the proposed incremental method and to assess its practical benefits, we have implemented a
program and conducted timing experiments, in which middle-size building models were applied. They were
models of a hospital represented by 1319 construction elements with total number of triangles 62268 in their 3D
boundary representations, as well as a steel construction, a bridge, a warehouse with total numbers of elements
and triangles 595-14416, 811-61524, 1452-259226 correspondingly. The tolerance of the metric representations
for different models was limited by the regular grid sizes 100 100 100. The experiments and timing analysis
carried out on a typical computer configuration Core 2 Duo E8600 processor (2.13 GHz), 2GB of RAM (800
MHz).
The first series of experiments corresponded to the static simulation mode assuming global path planning in the
constructed buildings. The results presented in Table 1 show the total of CPU seconds spent for the topology
extraction and the possible further path planning. In the next columns relative CPU resources are given for
particular algorithmic phases, namely: octree deployment, distance computations, dominancy relations
determination and space-gate identification.
Table 1: Total of CPU seconds spent for the topology extraction.
Building model Total
CPU
Octree deployment
(CPU)
Distance computations
(CPU)
Dominancy
determination (CPU)
Space-gate
identification (CPU)
Hospital 10608 5585 1903 1248 1872
Steel construction 320118 17843 8939 21653 271683
Bridge 21505 14095 2433 874 4103
Warehouse 287243 14773 12543 20904 239029

It can be seen that in all the cases the method solves the topology extraction problem for reasonable time not
exceeding a few minutes, and the main computational expenses fall to the space-gate identification phase. The
obtained topology representation is well suited for multiple path planning requests strongly related to the given
final point of time, but not applicable to validate the project schedule throughout whole construction period as it
would re-quire repeated computations and deployment of all the metric and topological structures.
Table 2: Total CPU expenses required for incremental topology extraction and validation of the project plan.
Building model Incremental method (CPU) Usual method (CPU) Performance gain factor
Hospital 59873 27168523 453
550

Steel construction 1693584 63111483 37
Bridge 149584 16341413 109
Warehouse 4603657 129034623 28

The second experiment series reproduced pseudo-dynamic simulation mode assuming the validation of
construction plans for the same building models using the proposed incremental method. The plans were
scheduled in such a way that all the construction elements were installed consequentially without any continual
motions. For every installed element the existence of the collision-free paths was proved. Table 2 provides some
results of the conducted experiments related to total CPU expenses required for incremental topology extraction
and validation of the project plan for each scheduled event. The results are given against CPU time taken for
solving the same problem using usual method as well as reached performance gain factor. The table illustrates
the significant performance of the incremental method and its applicability to the discussed computationally hard
problems.
5. CONCLUSIONS
The obtained theoretical and experimental results showed that the proposed method gives a significant
performance gain for multiple path planning requests in complex pseudo-dynamic environments and can be
successfully applied for the validation of construction project schedules against path conflicts.
6. REFERENCES
Dijkstra E. W. (1959). A note on two problems in connexion with graphs. Numerische Mathematik, 1 (1959), pp.
269271.
Fabrizi E. and Saffiotti A. (2000). Extracting Topology-Based Maps from Gridmaps. In International Conference
on Ro-botics and Automation, 2972-2978.
Kim J. and Lee S. (2009). Fast neighbor cells finding method for multiple octree representation. In IEEE Int.
Symposium on Computational Intelligence in Robotics and Automation. 540-545.
Lamarche F. (2009). TopoPlan: a topological path planner for real time human navigation under floor and ceiling
con-straints. Computer Graphics Forum. 28 (2). 649-658.
Lavalle S. M. (2006). Planning algorithms. Cambridge University Press, Cambridge, UK.
Semenov V. A., Kazakov K. A, Zolotov V. A. (2011a). Topological Mapping Complex 3D Environments Using
Occupancy Octrees. Proceedings of the XXI Inter-national Conference on Computer Graphics and Vi-sion, 26-30
Sep-tember, 2011, Moscow, Russia, p.111-114, ISBN 978-5-317-03808-3.
Semenov V. A., Tarlapan O. A., Morozov S. V., Kazakov K. A., Zolotov V. A. (2011b). Global path planning in
complex en-vironments using metric and topological schemes. Proceed-ings of the International CIB W078-W102
Conferences on Information Technology for Construction and Information and Know-ledge Management in
Building, 26-28 October, 2011, Sophia-Antipolic, France, p.87-95.
Semenov V. A, Kazakov, K. A., V.A, Zolotov, V. A. (2011c). Virtual Construction: 4D Planning and Validation.
Proceedings of the XI International Conference on Construction Applica-tions of Virtual Reality, 3-4 November,
2011, Weimar, Germany, p.135-142.
Thrun S. (1998). Leaning Metric-Topological Maps for Indoor Mobile Robot Navigation. Artificial Intelligence.
99. 21-71.
551

LOGISTICS AND LAYOUT PLANNING OF CONSTRUCTION
EQUIPMENT ON A VR-MULTI-TOUCH-TABLET
Andreas Ebner, Florian Kammergruber, Tim Horenburg & Willibald Gnthner
Technische Universitaet Muenchen, Germany
ABSTRACT: Significantly more complex building projects and strong international competition create a
significant cost pressure in the industry. To improve the efficiency of planning and implementation of construction
work, digital methods from the stationary industry shall be transferred, adjusted and extended to the construction
industry. Many construction sites are currently being planned in 2D and by experience. With 2D it is nearly
impossible or really difficult to visualize 3D collisions. In order to withstand the pressure of costs, planning only by
experience is not enough. The logistics planning on the construction site and in particular the material flow
planning is often determined at the start of construction. The very late planning of the logistics leads to mistakes
and increased costs. Many unnecessary rearrangements or the search of already supplied materials, are the result.
This increases the time needed and leads to higher costs. A integrated logistics planning on the site will be
developed in this project through the use of digital methods. Several planners arrange the logistics of the
construction site with the various phases using a multi-touch table. The screen visualizes a 2D top view. All
necessary items are selected from an object library and placed on the layout with the touch of a finger.
Subsequently, there objects are validated by planning algorithms. The objects obtain source-sink and time/date
information. This allows to show and evaluate the material flow. Bottlenecks can be made visible. On another
screen the construction with all his objects are shown in 3D. This tool will bring everyone who is involved in the
logistical planning to one table. With the digital tools it is possible to evaluate and show errors or information in
real time.
KEYWORDS: layout planning, virtual reality, collision, multi-touch table
1. PRESENTATION OF THE PROBLEM
Nowadays the construction process is subject to enormous demands concerning costs and completion times.
Significantly more complex building projects and strong international competition create a significant cost
pressure in the industry. The construction industry can only fulfill these demands by improving the quality of
processes regarding the keeping of deadlines and cost security.
To improve the efficiency of planning and implementation of construction work, digital methods from the
stationary industry shall be transferred, adjusted and extended to the construction industry. The applied methods in
stationary industry of the digital factory (Khn, 2006) shall serve as a model to overcome the complex basic
conditions of current construction projects in future. The latter are basically made up of
The construction and creation of unique items,
The dependency on climatic conditions,
The strong fragmentation of the industry during the completion of construction,
The distinct segmentation along the process chain as well as
The reluctant usage of modern information and communication technologies (Gnthner, 2011).
The use of innovative digital technologies supports the planning of logistical processes. By considering these
fringe conditions, it is necessary to determine the layout from aspects of material flow and thus in future conduct
construction processes more effectively and cost-efficiently.
Usually each construction site is subject to different conditions, the production sequences vary and the local
conditions are different from project to project (unique construction). This demands an intensive, detailed planning.
However, in reality the logistics and more specifically the planning of material flow is poorly done and determined
close to the beginning of construction (Bauer, 2007). Due to the immediateness and poor methods this 2D plan
based approach tends to lead to non-transparency and mistakes in the later process. There is no digital validation or
verification of planning process taking into consideration the specific fringe conditions at the construction site.
Digital methods based on intelligent, three-dimensional objects can support those responsible for planning and
increase the planning performance. Furthermore, it is necessary to portray the different construction phases and
their dynamic fringe conditions and deduce corresponding suitable measures for the planning of supply and
552

material flow.
Compared with the stationary industry the construction industry is very fragmented, a significant majority of the
companies are in the group of SMEs and in this case small enterprises (Bauer, 2007). As a consequence, all
processes are conducted by different companies, especially in the case of large construction sites. The lack of
consistency leads to insufficient transparency in the process and in the worst of cases to different levels of planning.
A central model for the planning of supply and material flow ensures a consistent communication of all
participating away from a plan on paper to a digital 3D model.
Up to now digital methods are only used very restrictedly in the construction process, a completely digital
construction site is far off from standard practice. A prescient planning of the logistical process chain is only very
rarely applied (Langhammer, 2009). Even though the potential of a construction logistic encompassing the entire
construction process was already recognized, there is a lack of methods and instruments of how the construction
logistics can be integrated in the inter-company execution process of orders according to current knowledge
(Krau, 2005). Nevertheless single tools are capturing the market, as 2D CAD or model building no longer suffices
for the demands of current construction processes and the necessary flexibility.
By the aggregation of the logistics planning and a centralized supply of information of the participants in a digital
3D model, different scenarios can be compared and a corresponding layout can be constructed for every step in
construction. Despite constricted space conditions in inner-city areas, the project costs can thus be significantly
lowered by an efficient material flow based on a well-planned layout (Hofstadler, 2007). Mutual dependencies and
influences of the single elements of material flow become understandable and an integral planning is enabled.
! shows possible reduction of costs resulting from consequently planned and executed
construction logistics

Fig. 1: Potential reductions of costs by construction logistics (Gnthner, 2008)
The main goal is a practice-oriented planning instrument which allows the determination of layout and the
equipment of the construction site for the entire running time of the project and is only changed if needed. A
suitable approach for cross-company processes supports the preparation of work and enables a cross-company
planning during the execution. Intelligent development algorithms guarantee a qualified logistics planning by
evaluating equipment-related parameters with reference to the current planning step and logistical weak points are
revealed prematurely. The different phases of construction can be portrayed clearly and single supply processes
can also be analyzed during the execution. Thus the dynamic surrounding of the construction site and the
principles construction means transport and construction means communication (Deml, 2008) can be taken
into consideration.
553

2. CONCEPT
Within the research project LogPlan Bau an approach is developed for the transparent planning of different objects
on a construction site. The topic of the paper is to have a collision-free and lean logistics for the supply of
construction sites. Function units such as storage areas or transport routes, large appliances and their working
rooms as well as static elements (container) shall be planned cheaply. The subgoals of LogPlan Bau can be
deduced:

Development of planning algorithms to support the planner during the interpretation and conception of
the corresponding construction steps
Creation of a general 4D model of the object of planning objective during all phases of construction.
Use of an intuitively usable planning tablet combined with the spatial representation of a VR system.
Fig. 2: Different object on site with dependencies

The result should be a full-functioning demonstrator, which creates an innovative possibility for all phases of
construction to equip construction sites plan-based and depicts the dynamics of construction sites spatially and
temporally.
Beside the development and the build-up of a function pattern, the creation of a consistent planning method is
decisive. A central approach is to be developed which anticipates the material flow processes at the construction
site as well as logistical processes for supply and disposal cross-company (see Fig. 2Fig. 3: ). Computer-aided
planning algorithms additionally guarantee a sensible planning of the single resources.

554

Fig. 3: Priority and flexibility as possible influence quantities of planning methods
The approach and development algorithms help the person responsible for planning by suggesting a sequence for
the shaping of the material flow processes and possible mistakes in planning are caught.
The single elements are aggregated into coarse classes and should be modeled as parametric objects which are
summarized in a common library. The parameters (e.g. height, radius) of the general class (in the example: crane)
are only determined during use and then checked for validity by the planning algorithms.
The function pattern for the planning and the layout of large construction sites is basically made up of a planning
tablet and a VR system. Fig. 4 illustrates the intended setup.
constructionphase 3
constructionphase 1
constructionphase 2
VR-system
multi-touchtable
for user interaction

Fig. 4: Functional schematic of the demonstrator
The equipping of the construction site is done on a dynamic 2D plan which is portrayed on the to be developed
surface of the planning tablet. Corresponding to the properties of the construction site, the layout can be changed
over time so that the different construction phases can be planned using digital models. The blending in of specific
resource parameters as well as their dependency chains on the planning tablet enables a safe and efficient
construction process. The VR system shows a stereoscopic view of the current planning state, so that the total
model for all phases of the project can be evaluated. This way the different phases of construction can be depicted
clearly and single supply processes can be checked even during the execution. Mistakes in planning and supply can
be identified and spatial geometric collisions can be avoided.
The result of the project is a central tool for the mutual planning of logistic actions and the communication of these
between those participating in the project.
555

Following partial results are amongst them:
By the transfer into the model area object-orientated components result for the planning of equipment as
well as their time-dependent parameters.
Different deployment scenarios are developed for a systematic evaluation of these components.
Those responsible are supported by implemented planning algorithms which check the planning effort with
reference to logistic and geometrical demands.
A consistent approach is developed to plan the layout as well as the material flow of large construction sites.
A model with several scales allows both rough and detailed planning.
The equipment elements are assembled to a component library and a parametric, resource specific 3D model
is deposited for each.
A virtual spatial model is created which unites static and temporally dynamic parts. The latter enables the
portrayal and planning of different phases of construction.
For demonstrative purposes as well as for the validation and improvement of the planning system, a function
pattern with different configuration levels is put into operation.
On the basis of the results a new method to plan cross-company logistic processes for the construction site are
defined and provided. This is based on a universal innovative planning method whose processes small
companies can also conduct with regard to a reduction of complexity. For this the logistical supply of
construction work is planned at a joint model taking into consideration the mutual influences.
The planning algorithms to be worked out in this research project both for the integrated planning process as
well as for the specific (material flow) objects are independent of the system. Therefore it is possible to transfer
the newly developed algorithms to other potentially interesting systems. The requirements for such a planning
tool are developed systematically and available solutions are investigated for the compliance of these demands.
Therefore it is also possible in future to investigate new systems for their usability for the construction industry.
The research project is the first to apply the use of innovative VR technologies in combination with methods
from the digital factory to plan the supply of construction sites. By the development or implementation of an
innovative man-machine interface, the user shall be supported with information as good as possible at the
specific planning action without expecting too much of him with reference to his technical knowledge. Different
scenarios can be analyzed realistically and logistical key figures such as e.g. use of area can be gathered. Aside
from the feasibility of such a system, especially the high potentials shall be verified.
Users can yield advantages in the communication over different hierarchical levels and specifically over the
limits of companies. The mutual work in a virtual environment which enables an intuitive system command and
visualizes necessary information sensitizes people of different areas for the task of others or promotes the
understanding of problems and creates transparency. The worked out interdisciplinary results of planning can be
accessed from the central model at all times and independent of location.
Based on the construction site model the planner at the construction site can access important information based
on the already planned construction site model by using augmented reality.

556

3. CONCEPTS PLANNING TABLE
The planners at the planning table use a software onto which the layout of the construction site can be imported.
After the layout is imported, the planer must identify the areas for the building to be constructed as well as border
areas of the construction site. This is necessary so that the system knows the areas which can be used for the logical
planning of the construction site. In the rough planning, the planer can mark this onto the plan using the touch of
his finger.
Next he can determine the construction phases and for each phase of construction chose a different 3D model
which depicts the buildings to be constructed in each respective phase of construction. In this step he shall also be
able to determine pits and other information on heights.

Fig. 6: Distance of a machine to a pit
After the basic information was determined, the actual planning can begin. From an object library sorted according
to topics and significance, the single planner or several planners can then select the object, which is necessary,
somewhere on the construction layout. When placing the object other data is needed such as measurements, weight
or other resource specific parameters, which has to be added by the planner. After inputting the data, it is checked
for validity by the planning algorithms. Additionally the system provides information if a device / machine can
stand at the spot where the planner has placed it. A simple example of this is if a machine is not placed in a building.
The distance of a machine to a pit is also controlled (see Fig. 6). In the case of a distance-pit problem, the system
informs the planner that the machine could be too close to a slope and thus must be placed further away. However,
it can also provide recommendations that the slope should be enforced. The system only informs the planner, he
must still make important decisions himself.

Fig. 7: Field of work of different machines (crane and deep drill) on computer and paper

557

When placing a machine which has a specific field of work (e.g. cranes) the user can display this field of work. Fig.
7 illustrates this example. However current practice is that a model is used on a plan (see Fig. 7) and this model is
then moved and turned manually. It is more precise if this happens digitally like shown in Fig. 7: The cranes have
different support structure areas. The visualization of the top view by circles of different colors reveals if a crane
can take the load at an important spot (position of material supply, low ground) or if either the location of the low
ground must be positioned better or the crane must be set up differently. Furthermore, the tool allows to see the
collision of cranes with other cranes and buildings (see Fig. 8).

Fig. 8: collisions between different cranes and building

In the course of the project time lots of such information will be collected and visualized through the software.
558

4. SELECTED HARDWARE
For the concept, different devices were researched and evaluated which could come into question for planning
tables. The same was done for the VR system. Due to the demands of a construction site and the desire to not only
set up such a planning instrument in an office, but also in a container at a construction site, the following system
was acquired:
- Planning table: 46 multitouch frame for a screen which is set up horizontally as a table. It is large enough
so that several planners can all work at a table but sufficiently small so that it can be stored in a
construction trailer (size x m times xm). The full functionality of the multitouch frame is independent of
lighting influences.

Fig. 10: Example of the selected planning tablet

- VR system: 3D television set. Normally large screens or even caves are known as VR systems. However,
these are hard to transport or cannot fit into a construction trailer. For this reason, a 3D television was
selected for this subsystem. During planning however, the 3D television set is operated in 2D so that the
planers must not put on or take off the glasses according to if they look at the 3D television or at the
planning table. If the construction site is evaluated in 3D, the planner can decide if he wants to activate 3D
mode so he can immerse himself into the 3D world.
Both subsystems each have a computer that communicates with each other via Ethernet. Tablets are used for the
AR system. During the project both systems (Android as well as iOS) are tested. Furthermore, the system has a
connection to the internet which gives the AR system access to the information of the construction site model
which should be blended in by augmented reality.

559

5. REFERENCES
Bauer H. (2007). Baubetrieb, Springer-Verlag, Berlin, Heidelberg.
Deml A. (2008). Entwicklung und Gestaltung der Baulogistik im Tiefbau, Kovac, Hamburg.
Gnthner W. A. and Zimmermann J. and Eber W. and Haas B. and Lgger M. and Sanladerer S. and Schorr M.
(2008). Logistik in der Bauwirtschaft, Bayern Innovativ, Nrnberg, Bayern Innovativ - Cluster Logistik.
Gnthner W. A. and Borrmann A (2011). Digitale Baustelle - innovativer Planen, effizienter Ausfhren,
Springer-Verlag, Berlin, Heidelberg.
Hofstadler C. (2007). Bauablaufplanung und Logistik im Baubetrieb, Springer-Verlag, Berlin, Heidelberg.
Krau S. (2005). Die Baulogistik in der schlsselfertigen Ausfhrung, Bauwerk, Berlin.
Khn W. (2006). Digitale Fabrik, Mnchen, Carl Hanser
Langhammer M. (2009). Simulation baulogistischer Entscheidungsprozesse, VDM Verlag Dr. Mller.
560

EXPLORING USER EXPERIENCE OF WAYFINDING IN A LARGE
AND COMPLEX HOSPITAL
Mi Jeong Kim & Sooyeon Han
Kyung Hee University, Republic of Korea
Xiangyu Wang
Curtin University, Australia
ABSTRACT: Wayfinding is a very complicated perceptual and cognitive process, especially in large and
complicated environments such as hospitals and big shopping centers. Many studies show inhabitants of large
environments can become lost very easily if they are unfamiliar with the environment. Although they may
eventually be able to find a target location, it is very time-consuming to consult a map, understand the signs, and
ask people for directions. This research identified factors that influence wayfinding and developed a customized
questionnaire with a focus on user experience of wayfinding: generating an environmental image of the lobby,
identifying information factors, understanding environmental factors, and linking environmental factors with
information factors. An explorative study was conducted to investigate user experience of wayfinding in a large
hospital, where the application of the questionnaire was validated in a real-life situation. The results show that
participants experienced difficulties in generating a clear image of the lobby, identifying wayfinding aids, and
understanding some of the information provided by the wayfinding aids. Further, there was no portable and
continuous wayfinding service, which caused problems in recalling the information learned from the wayfinding
aids, especially when attempting to return to the lobby. The potential of mobile AR for wayfinding aids is
presented as an implication from the results of the explorative study, and the technological framework of the
mobile AR wayfinding aid and its implementation in hospitals is proposed for future study.
KEYWORDS: WAYFINDING, HOSPITAL, METRICS, AUGMENTED REALITY
1. INTRODUCTION
Wayfinding is a very complex perceptual and cognitive process, especially in large and complicated environments
such as hospitals and shopping centers. Wayfinding aids commonly used include maps, landmarks and signs on
walls, but frequently they are misleading and confusing because of poor design or varied appearance. Additionally,
signage was sometimes discontinuous, with signs missing in vital sections along the route. Inhabitants in large
environments can very easily become lost if they are unfamiliar with the environment. Hospital inhabitants
normally include patients, doctors, nurses, staff, visitors, etc. Even when it is possible to find a target location, it is
often very time-consuming due to the necessity to consult a map, understand the signs, and ask people for
directions. Simply providing signs only for inhabitants' use failed to overcome the problem of wayfinding
(Miller and Lewis 1990; Rooke et al. 2010); thus many researches focused on the support provided by
wayfinding aids and technologies (Chen and Truc 2008; Chumkamon and Keeratiwintakorn 2008; Wright et al.
2010). Numerous interface issues such as modality choices and unobtrusive supports have been proposed for
developing wayfinding aids. The ultimate aim of this research is to propose efficient wayfinding techniques and
the implementation of wayfinding aids that use the proposed techniques. In order to further the development of
appropriate variations of wayfinding aids, this paper first identifies influencing factors on wayfinding, based on
the literature review and the investigated human experience of wayfinding in the natural setting of a large
hospital. To explore wayfinding experiences, this paper developed and conducted a customized questionnaire
survey in addition to the observation during the wayfinding experiment. Contrary to other studies into wayfinding,
this study is oriented towards user experience (UX) rather than technical aspects of the wayfinding aids. In the
following sections, an overview of related works is presented, and a customized questionnaire focusing on UX is
proposed with several influencing factors, which is explored in a case study. Finally, our conclusion and plans for
future study are presented.
2. RELATED WORKS
The term 'wayfinding' was coined by Lynch (Lynch 1960) to introduce the concept of urban legibility and
finding one's way around unfamiliar environments. Many of the people researching wayfinding focused on
outdoor open spaces or horizontal traveling; however, some researchers emphasized inter-story and vertical
561

traveling in complex multi-story buildings. The process of wayfinding has been conceptualized in a variety of
ways. Arthur and Passini (Arthur and Passini 1992) defined the key elements of the wayfinding process as
decision making, decision executing, and information processing. Carpman and Grant (Carpman and Grant
1993) explained the wayfinding process as knowing where you are, knowing your destination, knowing and
following the best route to your destination, recognizing your destination upon arrival, and finding your way
back out. Mollerup (Mollerup 2005) described wayfinding as search, decision, and motion. In general, spatial
planners and information designers are responsible for wayfinding; thus, many studies have focused on how to
design the layout of a physical setting and its circulation system, and how to improve wayfinding aids. Some
research has attempted to discover critical factors that affect the performance of wayfinding. Weisman (Weisman
1981) described three major environmental factors that affect the ease of orientation and wayfinding:
differentiation, visual access, and layout complexity. Carpman and Grant (Carpman and Grant 1993) developed
environmental wayfinding checklists for spatial planners. Miller and Lewis (Miller and Lewis 1990) established
effective wayfinding guidance for healthcare facilities that categorized the factors affecting wayfinding into
people, environment, and information factors. Pearson (Pearson et al. 1999) emphasized factors that influence
the accessibility of information systems: ease of use, location, interface and information.
Some works have promoted adopting new technologies for wayfinding aids. Chumkamon (Chumkamon and
Keeratiwintakorn 2008) proposed a navigation method using Radio Frequency Identification (RFID) triggers that
activate an audio message when a user walks past an RFID tag embedded in a footpath block, and argued its
potential to support blind people and tourists under normal conditions, and firefighters in a smoke-filled building.
Wright (Wright et al. 2010) highlighted the need for a print option in interactive kiosks based on a case study
showing difficulties in recalling information, and also suggested the potential for mobile telephony to enable
web access from anywhere without maintaining touch-sensitive displays. Advocating more advanced approaches,
Chen and Truc (Chen and Truc 2008) proposed the potential of Virtual Reality (VR) and Augmented Reality
(AR) techniques for evacuations in emergency situations, since the display can be switched from a real scene to a
virtual scene under poor visibility conditions caused by dense smoke, etc. It is clear from what has been
highlighted so far in most literature that wayfinding concerns more than just signs, and it should be approached
from the integrated views of the users, spatial planners and information designers. In particular, the user is the
ultimate reference for wayfinding; all factors should therefore be judged by UX. Little research has been
undertaken to evaluate wayfinding performance from a UX viewpoint. Lawton (Lawton 1994) asked participants
of a case study to rate their level of anxiety with wayfinding tasks, using a self-report with a spatial anxiety scale.
Li (Li 2007) conducted a study on cognitive map activity by requiring users to perform a route-drawing task in
wayfinding, and analyzed turn errors, distance errors and end-point errors. Rook et al. (Rooke et al. 2009)
adopted an ethnomethodologically informed approach that employed observations and interviews with staff.
3. A WAYFINDING QUESTIONNAIRE USER EXPERIENCE
Wayfinding can be affected by three major types of factors: human factors, environmental factors and
information factors. Human factors are related to psychological and physical ability, such as peoples familiarity
with the environment, sensory acuity, emotional state, and so forth. Environmental factors include issues
concerning the layout of a setting and the environmental communication for wayfinding. The information factors
are associated with the quality of wayfinding aids. As explained in section 2, much research categorized
environmental and information factors from the viewpoint of space designers, which resulted in wayfinding
checklists focusing on how the environment and information should be designed. Little research has studied how
those factors are experienced by people wayfinding in the real environment. In this study, we focused on UX of
wayfinding, and developed a customized wayfinding questionnaire to thoroughly understand the reasons people
become lost in a large and complex building. From the viewpoint of UX, the wayfinding process can be
categorized as follows: understanding environmental factors, identifying information factors, and linking the
environmental factors with the information factors.
3.1 Understanding Environmental Factors
Wayfinding problems are intimately linked to the configuration of the environmental layout, including its form
and organization of architectural elements, the circulation it provides, and its content such as building signs and
room numbers. People finding their way in complex settings go through perception and cognition processes to
identify what the setting contains, to understand how it is organized, and to form a mental map of the setting. In
order to develop a UX-based questionnaire, we extracted the environmental indicators based on the literature
review, and connected each indicator to a UX indicator as shown in Table 1. Above all, imaginability of a
562

location, in this case a lobby, is significantly important to the wayfinding. As the lobby gives visitors the first
impression of the building, a failure to generate a clear environmental image of the lobby would make it difficult
for people to find clues to navigate properly in the building. Identifiability of key decision sites such as building
entrances, pathways and vertical accesses, and the identifiability of graphic expressions such as department signs
and room numbers are other indicators for wayfinding. Many studies suggested the details of legible physical
settings are important (Weisman 1981; Miller and Lewis 1990; Arthur and Passini 1992; Carpman and Grant
1993); identifiability of physical settings can only be sustained if all architectural elements and graphic
expressions have features that can be distinguished from others. Accessibility of the pathway circulation is a
crucial point to be considered to allow easy wayfinding, thus ensuring a good visitor flow. Additionally,
provision of comprehensive graphic expressions is a significant factor as graphic expressions are easy to design.
The last indicator is the reliability of graphic expressions, emphasized by Arthur and Passini (Arthur and Passini
1992), who argue that wayfinders must be able to follow a series of signs located at reassuring intervals.
Table 1. User Experience of Environmental Factors
Category
Environmental
Indicator
UX UX questionnaire
Architectural
elements
Clarity and legibility of
lobby
Imaginability
1. How did you feel about the environmental image when you
entered the lobby?
2. How did the environmental image of the lobby affect your
confidence in wayfinding?
3. To what extent did the environmental image of the lobby help
you to make a decision about how to find your destination?
building entrances
Identifiability
4. How much effort did you make to identify the locations of the
building entrances?
pathways?
elevators?
escalators?
stairs?
pathways
vertical accesses
(elevators/escalator/stair
s)
Organization of pathway
circulation
Accessibility
9. To what extent were you interrupted by passers-by while
wayfinding?
Graphic
expressions
Location of signs and
room numbers from a
distance
Identifiability
10. How much effort did you make to find the signs and room
numbers from a distance?
11. How much effort did you make to find the signs and room
numbers within eye level zone?
Placement of signs and
room numbers within
eye level zone
Level of complexity of
terminologies on signs
Comprehensivity
12. To what extent did you understand the terminologies used on
signs?
13. To what extent did you understand the pictograms used on
signs?
14. To what extent did you understand where the arrow was
pointing?
pictograms on signs
arrows pointing on signs
Continuity and
consistency of signs at
reassuring intervals
Reliability
15. To what extent did you feel confident about being able to
follow a series of signs to your destination?
3.2 Identifying Information Factors
The second category, information factors, deals with the issues around providing information to assist with
wayfinding such as maps, directories, interactive kiosks and so forth. We do not categorize signs and room
numbers as information factors, but instead as environmental factors, as shown in Table 1, because of their
limited role in identifying what it is, rather than how to reach it. We categorize wayfinding aids, including
detailed wayfinding information, as information factors. The important issues posed by the researches for
wayfinding aids and technologies were the matter of design and operation style, and directional system of
wayfinding aids. Four indicators stood out from a UX perspective: identifiability, accessibility, comprehensivity,
and interactivity (Table 2). Although people generally use wayfinding aids, the information provided often
cannot be easily and quickly identified and understood. If people experience problems in identifying and
understanding directional information provided by the wayfinding aids, it is possible that they might make
incorrect decisions.

563

Table 2. User Experience of Information Factors
Category
Information
indicator
UX indicator UX questionnaire
Wayfinding
aids
Clarity of
wayfinding aids
Identifiability 1. How much effort did you make to indentify wayfinding aids?
Location of
wayfinding aids
Accessibility
2. Which wayfinding aid did you access first?
3. To what extent could the wayfinding aids be accessed from anywhere?
4. To what extent were you interrupted by other people while you were
using wayfinding aids?
Number of
wayfinding aids
Information
overload/deficiency
Comprehensivity
5. To what extent did you receive satisfactory information to assist your
understanding?
Flexibility of view
point of map
Interactivity
6. To what extent did you interact with wayfinding aids to move the
viewpoint of maps?
Complexity of
terminology on
wayfinding aids
Comprehensivity
7. To what extent did you understand the terminologies used on
wayfinding aids?
Directional
system
Indication of current
location
Identifiability
8. How much effort did it take to identify your current location on your
wayfinding aids?
9. How much effort did it take to identify your destination on your
wayfinding aids?
10. How much effort did it take to identify your orientation on your
wayfinding aids?
Indication of
destination
Indication of
orientation
Indication of route Comprehensivity
11. To what extent did you understand the horizontal route on your
wayfinding aids?
12. To what extent did you understand the vertical route on your
wayfinding aids?
3.3 Linking Environmental Factors with Information Factors
The third category focused on how people link the environmental factors with the information factors through
wayfinding aids. Passini (Arthur and Passini 1992) defined wayfinding as a continuous problem-solving exercise,
and demonstrated basic wayfinding tasks and the corresponding spatio-cognitive manipulations. Considering
how complex the process is, the confliction of the information offered by wayfinding aids with the actual
environment can be inferred. First, factors influencing the spatio-cognitive operations were extracted from
environmental and information factors, i.e., portable and continuous services of wayfinding aids, consistent use
of terms, spatial accuracy between the wayfinding aids and the real environment, and the continuity of
directional information. Then, questions were developed to measure users reliability, memorability and
comprehensivity in linking the environmental factors with the information factors (Table 3).
Table 3. User Experience of Linking Environmental Factors and Information Factors
4 EXPERIMENT
It was important to be sensitive to ethical issues and the physical and psychological burdens that visitors to
hospitals have, which placed limits on the actual investigation of visitors who find their way around a hospital.
We decided to conduct an experiment on wayfinding in a university hospital consisting of four interconnected
buildings, each with 8 to 17 floors. In order to determine the experiment set-up, we visited the hospital prior to
the experiment, and thoroughly studied the layout and environmental settings. There are four kinds of
wayfinding aids, namely an information desk, y-a-h (you are here) maps, directories, and interactive kiosks in
Category
Environmental/
information
indicator
UX indicator UX questionnaire
Linking
environmental
factors with
information
factors
Portable and
continuous services
of wayfinding aids
Reliability
1. To what extent did the wayfinding aids provide portability,
allowing them to be continually referred to during wayfinding?
2. How many wayfinding aids did you use?
Memorability
3. To what extent did you recall the information you learned from
wayfinding aids?
4. To what extent did you stop to refer to wayfinding aids?
Consistent use of
terms between labels
on wayfinding aids
and building signs in
the real environment
Comprehensivity
5. To what extent were you confused by the inconsistent use of terms
between labels on wayfinding aids and building signs in the real
environment?
Spatial accuracy Comprehensivity
6. What extent of spatial accuracy did you feel between wayfinding
aids and the real environment?
Continuity of
directional
information
Reliability
7. To what extent did your wayfinding aids provide continuous
directional information about your route?
564

the hospital (Figure 1). People at the information desk only provide spoken instructions, while the interactive
kiosks show animated horizontal and vertical routes. Although no hand-held map was provided by the hospital,
we could envisage the possibility that people would take photos of the y-a-h maps or directories using their own
mobile phones and utilize them as hand-held maps. For the experiment, we recruited 10 university students as
subjects, 5 male and 5 female, who were unfamiliar with the hospital layout. Our focus was on indoor
wayfinding, where the participants journey was supposed to start from the main entrance of the building, not the
outdoor area. The task given was to find the department of internal medicine, which is located on the third floor,
starting at the main gate, and then return to the original position. Avoiding choosing a department on the first
floor and selecting instead a department on the third floor was intentional in order to provide several options for
vertical access, i.e. stairs, elevators, and escalators, as most people would be able to walk up to the third floor via
the stairs. We expected that different vertical accesses would also contribute to the generation of different
horizontal routes. The experiments were conducted in ten separate visits over two days. While participants were
undertaking this wayfinding journey in the hospital, we followed and observed their wayfinding behaviors. All
behaviors and verbal accounts were recorded by a video camera, and we made field notes as a supplementary
tool. After completing the task, participants were required to fill in the questionnaires.

\

Figure 1. Wayfinding Aids: Information Desk, Directory, Map and Kiosk,
5 RESULTS
5.1 Questionnaire
The questionnaire consist of four parts: generating environmental images of the lobby (3 questions), identifying
information factors (12 questions), understanding environmental factors (12 questions), and linking the
environmental factors with the information factors (6 questions). Participants responses were rated along the
five point Likert scale. The result of Cronbach's alpha test for internal consistency was 0.866, which indicates
high reliability of the questionnaire. Firstly, we investigated the imaginability of the lobby environment. As
shown in Table 4, participants encountered problems in generating a clear environmental image of the lobby in
the large and complex hospital, which might impact psychologically on their wayfinding. The complexity of the
lobby environment yielded some stress, and the lobby image was not very useful when making a wayfinding
decision.
Table 4. Generating Environmental Image of the Lobby
Question Mean (Standard deviation)
Clarity and legibility of the lobby image 2.40 (.966)
Level of stress related to the problems caused by the lobby image when making a wayfinding
decision
3.70 (.823)
Usefulness of the lobby image when making a wayfinding decision 2.40 (.966)
In identifying information factors, there were several negative results regarding identifiability, accessibility,
comprehensivity, and interactivity of the wayfinding aids, as shown in Table 5. Participants found it difficult to
identify the location of wayfinding aids. Although some participants successfully accessed the wayfinding aids,
many of them faced trouble in identifying the current location, destination, orientation, and horizontal route on
the wayfinding aids. In addition, satisfaction was low regarding whether the quality of the information for the
wayfinding aids is sufficient to increase participants understandability.

565

Table 5. Identifying Information Factors
Difficulty in identifying location of wayfinding aids 3.20 (1.229)
Difficulty in accessing wayfinding aids 2.50 (1.179)
Difficulty in understanding terminologies on wayfinding aids 2.80 (1.398))
Difficulty in identifying current location on wayfinding aids 3.10 (1.287)
Difficulty in identifying destination on wayfinding aids 3.20 (1.476)
Difficulty in identifying orientation on wayfinding aids 3.00 (1.247)
Difficulty in understanding horizontal route on wayfinding aids 3.00 (1.333)
Difficulty in understanding vertical route on wayfinding aids 2.80 (1.317)
Sufficient information to increase understandability 2.90 (1.595)
Interactivity with wayfinding aids 2.60 (1.350)
Interruption by other people while using wayfinding aids 1.50 (0.972)
In understanding environmental factors, participants had difficulties not only in identifying the location of
elevators and stairs, but also in identifying signs and room numbers from a distance. Moreover, it was found that
understanding terminologies and pictograms on signs was not easy. The result of the question regarding
assurance of the destination while following a series of signs was low, which implies that it is important to
convince people by providing signs containing visible and easy-to-understand terminologies and pictograms
frequently enough to reassure people that they are still on the correct route to their destination.
Table 6. Understanding Environmental Factors
Difficulty in identifying building entrances 1.20 (0.422)
Difficulty in identifying pathways 2.70 (1.059)
Difficulty in identifying the location of elevators 3.40 (1.075)
Difficulty in identifying the location of escalators 2.70 (1.337)
Difficulty in identifying the location of stairs 3.50 (1.080)
Difficulty in identifying signs and room numbers from a distance 3.90 (.994)
Difficulty in identifying signs and room numbers within eye level zone 2.70 (.949)
Difficulty in understanding terminologies on signs 3.40 (1.174)
Difficulty in understanding pictograms on signs 3.10 (.738)
Difficulty in understanding arrows pointing on signs 2.60 (.966)
Assurance of destination while following a series of signs 2.90 (1.370)
Interruption by other people during wayfinding 1.80 (1.135)
Regarding linking the environmental factors with the information factors, the portability and continuity of
wayfinding services is very low. Participants had difficulties in recalling the information learned from the
wayfinding aids, thereby causing them further difficulties in linking information with the real environment. In
particular, participants were confused by the inconsistent use of terms between labels on wayfinding aids and
building signs in the environment, and also felt that there was lack of spatial accuracy between wayfinding aids
and the environment. However, despite several negative aspects, overall satisfaction of wayfinding supported by
wayfinding aids was not rated low, which suggests the significance of wayfinding aids in large and complex
buildings.
Table 7. Linking Environmental Factors with Information Factors
Portability and continuity of wayfinding services 1.20 (.632)
Difficulty in recalling the information learned from wayfinding aids 3.90 (.994)
Confusion caused by inconsistent use of terms between labels on wayfinding aids and building signs in
real environment
3.50 (1.080)
Spatial accuracy between wayfinding aids and real environment 2.70 (1.337)
Continuity of directional information 2.60 (1.265)
The number of pauses to refer to wayfinding aids 2.90 (1.287)
Overall satisfaction of wayfinding supported by wayfinding aids 3.40 (1.430)
Participants were asked to provide their opinions in written open notes. Regarding the environmental image of
the lobby, several participants pointed out an overload of information in the lobby. Too much information spread
566

all around the lobby confused participants, and disrupted the identification and access of the wayfinding aids. On
the other hand, some participants complained about information deficiency at key locations such as reception
areas or around elevators, where people expect to find helpful information for wayfinding. In identifying
information factors, participants indicated that because most wayfinding aids were immovable and located on the
first floor, they were not able to provide continuous information to people. Among the wayfinding aids,
participants were more satisfied with the interactive kiosks that provide animated routes with arrows pointing the
way. However, it was noted that the use of interactive kiosks is very complicated due to too much information on
the screen of the kiosks. In understanding environmental factors, participants found that identifying the location
of stairs, elevators or pathways was very difficult because their views were obscured by columns and banners.
Moreover, participants expressed the inconvenience caused by building signs and room numbers being mounted
at the top of walls and hung from ceilings, which made it difficult for them to keep walking and look at the signs
at the same time. In particular, people with limited mobility using a wheel chair would find it difficult to keep
looking up at the top of the wall. Regarding linking the information factors with the environmental factors,
participants had problems in recalling the information learned from wayfinding aids, and suggested that more
wayfinding aids should be available for access in appropriate positions throughout the building, and thus should
be located where flows of circulation are activated.
5.2 Observation
While the questionnaire survey provides quantitative results that can be subjected to statistical analysis,
observation of wayfinding enables researchers to interpret aspects of participants behaviors that might not be
captured by the standardized questionnaire.The data from the observation were analyzed both quantitatively and
qualitatively. The focuses of the observation were participants behavior types, location, duration, circulation,
vertical access, travel route, and overall use pattern of environmental factors and information factors, as shown
in Table 8. Above all, counting the number of participants circulation errors was very useful for quantitatively
presenting their off-route travels. The shortest time for the wayfinding task was 5 minutes. One participant went
directly to a destination without any off-route travels. Meanwhile, another participant, who took 13 minutes to
complete the task, repeated his circulation four times in attempts to correct his direction. Particularly,
participants who relied merely on y-a-h maps or directories took longer to complete the wayfinding task, and had
more circulation errors than those who used interactive kiosks. This result indicates the usefulness of direct and
intuitive directional information such as animated routes and arrows compared to relying on participants
arbitrary decisions and interpretation of maps. However, we observed that some participants overlooked the
interactive kiosks while searching for wayfinding aids, until they noticed someone else using them, and only
then finally accessed and used the interactive kiosks. This illustrates a lack of clarity in the identification of
wayfinding aids, so it is important to provide clues to attract people to the wayfinding aids. Interestingly, after
getting to the destination, returning to the starting point was also a considerable challenge for the participants.
Although many participants took the same routes back as they had taken on the way there, some participants
took a different route when they returned from the destination to the original position. Returning to the lobby
was not an easy journey for most of them. This seems to be closely related to the recall problem. Currently no
wayfinding aid exists that provides portable and continuous services in the hospital; thus, people often faced
difficulties in recalling the information they had earlier learned from the wayfinding aids. In a real-life context in
a large and complex hospital, there might be not only many different routes, but also quite different reasons to go
to various destinations. Therefore, it would be necessary for environment designers and information designers to
consider how to cover all possibilities in an integrated way together.
Table 8: Quantitative Data from the Observation
Total run-time Wayfinding aid Number of
circulation errors
Vertical access Return route
ID 1 8mins (5mins+3mins) Info desk Directory 2 Stairs Different route
ID 2 5mins (4mins+1min) Info desk 0 Escalator Same route
ID 3 6mins (4mins+2mins) Y-A-H map Interactive kiosk 1 Escalator Same route
ID 4 9mins (7mins+2mins) Y-A-H map 2 Stairs Same route
ID 5 6mins (4mins+2mins) Directory 1 Stairs Different route
ID 6 5mins (3mins+2mins) Interactive kiosk 0 Elevator Different route
ID 7 9mins (7mins+2mins) Directory Interactive kiosk 3 Escalator Same route
ID 8 13mins (10mins+3mins) Directory 4 Escalator Same route
ID 9 6mins (2mins+4mins) Directory 2 Stairs Same route
ID10 13mins (12mins+1min) Y-A-H map Interactive kiosk 3 Escalator Same route
567

6 IMPLICATIONS AND CONCLUSION
This study developed a customized wayfinding questionnaire with a focus on UX, and presented the results of a
wayfinding experiment that was undertaken in a large hospital to obtain a better understanding of the UX of
wayfinding. This study identified factors that influence the UX of wayfinding, and are significant for future
development of wayfinding aids. To determine the essential components of wayfinding aids, the difficulties and
problems in the UX of wayfinding should be identified first. Through the questionnaire survey in the experiment,
it was noted that participants had difficulties in the follow situations:
Attempting to generate a clear environmental image of the lobby, causing some stress and confusion
Identifying the location of wayfinding aids. and further, the current location, destination, orientation and
horizontal route on the wayfinding aids
Identifying the locations of elevators and stairs, signs and room numbers from a distance, understanding
terminologies and pictograms on signs, and being assured of the destination while following a series of
signs
Securing continuous wayfinding services, recalling the information learned from wayfinding aids, and
matching the spatial accuracy between wayfinding aids and the real-world environment.
Through the observation, the following user behaviors were noticed:
Participants who relied on y-a-h maps or directories took longer to perform the wayfinding and made
more circulation errors compared to those who used interactive kiosks.
Participants faced difficulties in recalling the information they had learned from the wayfinding aids;
thus, the return journey to the lobby was not an easy one for most.
Participants spent significant time finding, accessing, and figuring out wayfinding aids.
The findings raise several UX issues to be considered for wayfinding services and devices. Some cutting-edge
technologies have the potential to offer helpful information and support to those who have trouble wayfinding in
large and complex buildings. Mobile devices such as iPhones and iPads, which have recently become popular,
enable the mobility and ubiquity of wayfinding services, thereby making a large building as a pervasive space. To
overcome the limitations of the conventional wayfinding aids, mobile devices with a real-time feed into
Augmented Reality (AR) can be considered for the purpose of achieving continuous and portable wayfinding
services. AR augments a human being's perception of the real space with digital information, thus it can provide
augmented visualization and intuitive interaction techniques that influence a users memorability and reliability
with wayfinding information. Although AR cannot modify environmental factors directly, it can provide
augmented information for wayfinding such as augmented routes and color notes to distinguish destination or key
points. When these features work with mobile devices, they can supplement environmental factors such as signs
and room numbers off the eye level zone, and difficult medical terminologies on signs. Moreover, AR techniques
superimpose augmented information over the real-world environment, and thus can mitigate many problems in
identifying and understanding spatial accuracy between the real-world environment and wayfinding aids. It can
even delete unnecessary environmental elements in a virtual setting to provide a simplified circulation scene,
thereby alleviating the complexity caused by information overload. Further, the combination of AR and tracking
and sensing technologies can bring about a much richer sensory experience of digital information, off-loading a
users cognition. LLA-based AR is Longitude, Latitude and Altitude AR where all the individual points of interest
(POI) can be pre-stored in a database geographically. The current position and orientation of the user can be
tracked by an iPhone or iPad, which also functions as the viewing device for AR. When a POI (e.g., cafe,
pathology lab) is input into the system, the iPad/iPhone can indicate a direction and route (represented as a virtual
directional arrow and also a text label indicating what the target POI is) to guide the user. The digital information is
changed and updated as the location and orientation of the user changes. The case example below demonstrates
how mobile AR can help with wayfinding to a destination. The green arrow in Figure 2 explicitly shows the
direction the user should take to move from one milestone to the next. Each time a new tracking marker is
approached, the user just needs to scan the tracking marker with the iPad and retrieve the directional arrow to the
next milestone. After a few wayfinding steps, the user has reached the desired destination.
568

Figure 2. Wayfinding with Augmented Reality Smart Service
With the shift to an Information Technology age, smart services for wayfinding are becoming indispensable.
Technological support with mobile AR could play an important role in assisting people to navigate in large and
complicated buildings. The empirical study using the questionnaire has provided an insight into what aspects of the
wayfinding aids should be considered for enhancing UX of wayfinding. It was noted that more consideration of
portable and continuous services is required in wayfinding, to avoid the recall problem. Accordingly, mobile smart
services with AR are currently emphasized for the development of wayfinding aids. This research can lay the
groundwork for evolving knowledge about wayfinding aids in order to provide an understanding of UX of
wayfinding. The knowledge and lessons that we have learned from the experiment can be sources of inspiration for
future studies on smart wayfinding devices. Further, influencing factors on the UX of wayfinding can contribute to
the development of wayfinding aids that ultimately support users satisfaction and sensibility. The technological
framework of the mobile AR wayfinding aid and its implementation in hospitals will be proposed for the next step
of this study.
7 ACKNOWLEDGEMENTS
This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea
government (MEST) (No. 2012-0000609).
8 REFERENCES
Arthur, P. and Passini, R. (1992). Wayfinding: People, Signs, and Architecture, Ontario. McGraw-Hill Book Co.
Carpman, J. R. and Grant, M. A. (1993). Design That Cares : Planning Health Facilities for Patients and Visitors.
Chicago, Ill., American Hospital Pub.
Chen, P. and Truc, N. (2008). Computer-Aided Visual Communication for Way-Finding in Emergency in Indoor
Environment. Proceedings of The 25th International Symposium on Automation and Robotics in Construction.:
440-446.
Chumkamon, S. and Keeratiwintakorn, P. (2008). A Blind Navigation System Using Rfid for Indoor Environments.
Proceedings of 5th International Conference on Electrical Engineering/Electronics, Computer,
Telecommunications and Information Technology, ECTI-CON 765-768.
Lawton, C. A. (1994). "Gender Differences in Way-Finding Strateties: Relationship to Spatial Ability and Spatial
Ansiety." Sex Roles, 30.
Li, R. (2007). Human Wayfinding and Navigation in a Large-Scale Environment: Cognitive Map Development
and Wayfinding Strategies. Academic Department, University of Saskatchewan, Master of Arts.
Lynch, K. (1960). The Image of the City. Cambridge [Mass.], Technology Press.
Miller, C. and Lewis, D. (1990). Wayfinding: Effective Wayfinding and Signing Systems, Guidance for Healthcare
Facilities. UK National Health Service(NHS) Estates.
Mollerup, P. (2005). Wayshowing : A Guide to Environmental Signage : Principles & Practices. Baden, Lars
Mller.
Pearson, J., et al. (1999). The Accessibility of Information Systems for Patients: Use of Touchscreen Information
569

Systems by 345 Patients with Cancer in Scotland. Proceedings of AMIA(American Medical Informatics
Association), Washington, DC: 594-598.
Rooke, C. N., et al. (2010). Achieving a Lean Wayfinding System in Complex Hospital Environment: Design and
through-Life Management. Proceedings of 18th Annual Conference of the International Group for Lean
Construction, Haifa, Israel: 233-242.
Rooke, C. N., et al. (2009). Wayfinding: Embedding Knowledge in Hospital Environments. HaCIRIC
2009:Improving Healthcare Infrastructures Through Innovation. Brighton, UK.
Weisman, J. (1981). "Evaluating Architectural Legibility: Way-Finding in the Built Environment." Environment
and Behavior 13(2).
Wright, P., et al. (2010). "Using Audio to Support Animated Route Information in a Hospital Touch-Screen Kiosk."
Computers in Human Behavior 26(4): 753-759.
Wright, P., et al. (2010). "Using Audio to Support Animated Route Information in a Hospital Touch-Screen Kiosk."
Computers in Human Behavior 26(4): 753-759.

570
A SIMULATION DRIVEN VISUALIZATION APPROACH FOR
CONSTRUCTION MOBILE RESOURCES PLANNING
Amr A. ElNimr
Suncor Energy, Canada
Yasser Mohamed & Simaan M. AbouRizk
University of Alberta, Canada
ABSTRACT: Simulation modeling is an effective approach for analyzing construction operations, yet it is not widely
used by construction practitioners because of its inability to represent construction sites spatial data in an intuitive
way. This problem may be overcome by developing a simulation driven visualization (SDV) framework, which helps
model changes in site space geometry and site layout, throughout a simulated construction project. This framework
makes use of the advantages of simulation techniques and 3D modeling. The frameworks path-finding mechanism
extension (PME) builds on the ability to model changes in space by performing tempo-spatial planning for resources
mobilization, in a dynamically changing site layout throughout a construction project. To achieve this, the site mesh
generation mechanism and the A* search algorithm are implemented inside the framework. Simulation driven
visualization is a relatively new area of research in the construction management field. This research integrates
Computer-Aided Design (CAD) modeling with animation and simulation techniques to create SDV frameworks or
mechanisms that build on the strengths of each of the individual components. To achieve the aforementioned aims,
this developed framework had to address some of the technical limitations that exist in the current construction
research state of the art SDV mechanisms. This research presents a new SDV framework which uses distributed
simulation as its foundation. The research also makes use of visual analytics and display design principles in the
frameworks visualization component conceptual design and the transformation of the simulations object classes
statuses into visualization behaviors. The result is a framework that can intuitively depict changes in site space,
validate construction logic, and validate simulation models.
KEYWORDS: Simulation, 3D modeling, industrial construction, path-finding.
1. INTRODUCTION
Simulation modelling is an effective approach to analyze construction operations, yet it is not widely used by
construction practitioners because simulation models are unable to represent site space in an intuitive way. This
research addresses the problem through simulation driven visualization. Visualization in general, and particularly in
construction projects, is a convenient and intuitive way to convey project information among various project parties.
Recently, construction management researchers have been investigating adding a visualization component to
construction simulation models in order to make these models more intuitive and appealing to decision-makers.
These researchers argue that enhancing visualization and spatial representation of construction operations in a
simulation environment can improve the adoption of simulation techniques by the industry.
3D modeling has been utilized in the construction industry for its ability to represent prototypes of construction
processes and the changes in site geometry. This research suggests a new and intuitive way to model sites spatial
data and then builds on the space modeling ability to solve the shortest safe travel paths on a dynamically changing
site layout with changing geometries. This is done through a concurrent Simulation Driven Visualization (SDV)
framework in which a 3D modeling component is integrated with a discrete event simulation (DES) engine. The
framework is based on distributed simulation High Level Architecture (HLA) Institute of Electrical and Electronics
Engineers (IEEE) standards and allows addition of different visualization components to the simulation environment
with a great deal of flexibility. The framework provides two-way data communication between the simulation and
visualization components for data transformation.
The HLA-based framework for construction operations simulation visualization (HSV framework) utilizes 3D
modeling to depict future changes in both the site layout geometry and the site space that take place throughout the
projects simulation. The framework has a path-finding mechanism extension which evaluates various site layout
571
geometries in terms of the resources mobilization durations throughout the projects lifecycle simulation. This path-
finding mechanism automates the costly and time consuming planning process of the mobilization (movement) of
onsite resources. Both the frameworks ability to model changes in site space that couldnt be addressed by
simulation models alone, and the frameworks path-finding mechanism extension interoperate to aid decision-makers
in the following planning activities, at any stage of the projects lifecycle:
Modeling and experimenting with various site layout scenarios.
Testing existing resource paths, clearing obstacles from the predefined resources paths.
Minimizing the expensive heavy-lift resource mobilizations time.
Accurately calculating the expected heavy-lift resource mobilization durations to incorporate them in the
projects schedule.
An HLA-based SDV framework is employed to model changes in site space, with time. Before exploring the
development of this framework and its path-finding mechanism, it is appropriate to look at the background of SDV in
construction research. A critical literature review of SDV research in construction and the limitations of existing
simulation visualization packages, frameworks and mechanisms (both commercial and research-oriented) are
discussed below.
2. BACKGROUND
Simulation is defined as the process of designing a model of a real system and conducting experiments with this
model for the purpose of either understanding the behavior of the system or evaluating various strategies for the
operation of the system (Shannon, 1975). The strength of simulation approaches stems from their ability to examine
various scenarios of the simulated system, rather than simply giving a mathematically optimum result, as is the case
in mathematical optimizations. Most researchers stress the fact that simulation is not used to its maximum potential
in the construction industry (Ioannou and Martinez, 1996; Kamat and Martinez, 2001; Huang and Halpin, 1994).
There are two major reasons for this lack of use:
Most simulation models show results in statistical and text formats, which causes simulation to be
perceived by the construction industry as a black box.
Current simulation models do not consider the transformations of space that result from the evolution
of the constructed product (Kamat and Martinez, 2001).
There are commercial, off the shelf (COTS) SDV packages on the market. These include Delmias Quest and
Brooks Software AutoMod. However, these COTS generally concentrate on manufacturing operations. They are
unable to effectively handle the additional complications introduced by changes in the geometry of the construction
site as work progresses, and their use to model and animate construction operations requires a radical change in the
model conceptualization and thought process followed by construction model developers (Oloufa, 1993 and Kamat
and Martinez, 2008).
The first construction simulation tools to adopt graphics did so in the early 1980s (Rohrer, 2002). This was done
through attempts to link construction simulation packages to animation to achieve a post-process playback of the
simulation (Halpin and Woodhead, 1976; Kalk and Douglas, 1980; Paulson et al., 1983). The Utopian Framework
for earth moving operations is an example (Oloufa and Crandall, 1992). Another attempt to extract product
information from a CAD design into a simulation was done through the PSE (Product-oriented Simulation
Environment) (Xu and AbouRizk, 1999). Recent research in construction operations SDV involves the use of
simulation trace files to invoke post-simulation animation (replay). This includes the Dynamic Construction
Visualizer (DCV) and Visualization of Simulated Construction Operations (VITASCOPE) by Kamat and Martinez,
2001, Kamat and Martinez, 2004 and Kamat and Martinez, 2008, respectively.
The above simulation visualization packages (both commercial and research oriented) provide advanced
visualization capabilities. However, they incorporate some characteristics that limit their use in visualizing
simulation behaviors of construction operations. Some of these characteristics include:
572
Post-processing visualization. After a simulation is complete, visualization is produced as a replayed
record of what took place during the simulation run. It does not allow the decision-maker to interact
with the simulation. This causes the visualization component to rely on simulation trace statements
produced by the simulation engine exclusively for visualization purposes. Recent efforts by Rekapalli
and Martinez, 2009 and Rekapalli and Martinez, 2011 has focused on achieving two-way
communication and user interaction with the visualization component of a DES visualization.
Tight coupling between simulation and visualization engines. Existing SDV mechanisms and COTS
packages are usually tightly coupled with whatever simulation engine they serve. A particular
visualization component will usually work perfectly with a particular simulation engine, but it is very
time consuming or demands expertise to tailor it to work with other simulation engines. This
compromises the strength of both the simulation and visualization components. Also, this makes the
two tightly coupled components strong in depicting certain applications and weak in depicting others.
Finally, this decreases the reusability of tightly coupled frameworks or COTS packages and limits their
use in day-to-day construction operations.
Compromising the power of the visualization components. Graphics usually demand specialized and
high-powered computer processing. Running simulation models is also a demanding task for computer
processors. Running the two in parallel significantly increases the demand on computer hardware and
may be deemed infeasible in cases of highly detailed simulation and visualization models. In addition,
there has been significant advancement and optimization of software specially developed for handling
graphics and visualization tasks. Capitalizing on these advancements requires the development of a
mechanism to allow different specialized simulation and visualization components to interoperate
without limiting or compromising the strengths of either.
3. HSV FRAMEWORK DEVELOPMENT
Simulation models usually cannot model geometry and site spaces in an intuitive way. On the other hand, 3D
modeling has proven a valuable tool for modeling the geometry of shapes. In order to build on simulations modeling
capabilities and 3D modelings geometry depiction capabilities, researchers turned to SDV as a tool that could better
model the changes in site space. To overcome the limitations in state of the art construction operations SDV
mechanisms that were developed through recent construction research, a loosely coupled simulation visualization
framework was developed (ElNimr and Mohamed, 2010; 2011). The HSV framework (ElNimr and Mohamed, 2010;
2011) is employed to model changes in site space with time and study their effects on resource paths onsite using a
path-finding mechanism implemented inside the visualization component.
The HSV framework can be a useful tool in modeling site space geometry. An application of modeling changes in
site space geometry that result from dynamic changes in site layout is modeling the changes in the geometry of
mobile resource paths. The developed HSV framework provides an intuitive depiction of changes in site space
throughout the entire project lifecycle. The shortest, safest path for heavy lift resources is found based on the changes
in site space, through the frameworks visualization component, connected to the simulation model, and the
mechanism developed inside this component for mobile resources path finding. The HSV framework also takes the
safety factor into consideration when planning resource mobilizations given the changes in site space.
The representation of the sites geometry is a prerequisite to the operation of the path-finding mechanism. The site
geometry changes are based on the projects progress. There is always a change in the site geometry due to (1)
permanent structures monotonic addition to the site area, and (2) locations and geometry of objects representing
temporary site facilities. A projects progress is simulated through simulation based modeling; however, this type of
modeling does not have the ability to depict/model changes in site geometry, so SDV is used to enhance the site
spatial data and geometry modeling.
To effectively model the changes in site space that result from the simulation, this research proposes a loosely
coupled simulation visualization framework. In this framework, the execution of the visualization components is
parallel to and independent from the simulation engine. Execution of these components can also be carried out by
completely separate and dedicated computer systems, allowing the utilization of the strength of each component
573
independently and exclusively. In addition, the framework capitalizes on existing high-level 3D graphic engines to
reduce the effort required to customize and develop the visualization components. For an effective method to
visualize simulated construction operations and model both the changes in site geometry and site space, the proposed
framework capitalizes on gaming technologies.
The following sections will discuss further extension of the framework to (1) solve the resource path-finding problem,
and (2) plan onsite resource mobilization, based on the frameworks ability to model changes in both sites geometry
and sites spatial data. This initially develops the tool (framework) that:
Depicts simulated construction processes, the final construction product development and the construction
resources interactions onsite together with their respective positions based on the projects simulation
scenario.
Depicts changes in site layout and site geometry based on the projects simulation. The current site
geometry (spatial data and layout) is then used by the frameworks PME to solve the path-finding problem.
This section will explain the development of this new HSV framework.
3.1 Distributed Simulation Standards
The proposed Simulation Visualization framework is based on the distributed simulation standard known as High
Level Architecture (HLA) (IEEE 1516.2000), and is referred to as HSV. Distributed simulation is a technology that
enables models to be linked over computer networks so that they work together (or interoperate) during a simulation
run. HLA is a set of standards that regulates distributed simulation development (Taylor et al., 2003). In HLA, each
simulation is connected with the others through Runtime Infrastructure (RTI) software. The simulations or models
connected to the RTI are called Federates and a collection of them working together to simulate the same system is
known as a Federation. Federates are not restricted to simulation models, but can be any piece of software or
hardware, or even an interface for a human user (user-in-the-loop), as long as each Federate complies with HLA
standards (Zhen et al., 2010). RTI facilitates communication between Federates and performs time management of
the simulation. A key component of a Federation is the Federation Object Model (FOM) which defines all the entities
produced by the various Federates during the simulation run, states the attributes of those entities and information
about which Federates will update those entities' attributes (i.e. publish them), and which Federates will read them
(i.e. subscribe to them). For example, in the proposed HSV framework, Federates responsible for the visualization
may only subscribe to some attributes, while Federates responsible for simulation behavior generation can write and
read (publish and subscribe) these attributes.
Figure 1 shows the HSV concept, typical components and communications between the visualization Federate(s) on
one hand and other functional/simulation Federates on the other hand.
Fig. 1: HSV framework concept and communication
As shown in Figure 1, the following are typical components that are part of the framework or can be connected to it:
Visualization Federates: communicate with the source simulation or any other
Federate. They depict the construction operation simulation concurrently as the simulation time advances
574
to reflect what is going on in the simulation instantly. Visualization Federates have a 3D engine such as
Blender Game Engine (BGE) or TrueVision3D at their core.
Source Simulation Federates: are where the simulation of the various construction processes takes place.
Also, simulation time is usually advanced by these Federates. They connect to their respective Viewer
Federate through the RTI.
Other Functional Federates: include any other Federates that may provide supporting services to the
Federation. For example, a database Federate or a CAD Model Federate used to extract simulated objects'
attributes values from external information systems or CAD systems to update them during the simulation
run.
In the following sections we describe the application of this framework to a case of industrial construction operations.
3.2 Framework Application to Industrial Construction Site Installations
The integrated HSV framework was implemented in a real industrial construction project (Constructing Crude Oil
Upgrader in Alberta) to demonstrate how the framework can contribute to planning heavy-lift resource mobilizations
in a dynamically changing site layout. The site involved in this case study is shown in Figure 2.
The project represents the construction of an upgrader for crude oil. It is built using preassembled pipe spool modules.
The pipe spools are manufactured off-site in a fabrication shop and completed spools are then passed to a module
assembly yard. In the module yard, each module is allocated a space (bay) for completing its assembly. The pipe
spools that comprise a module are moved to the bay where the assembly will take place. Once modules are
assembled, they are shipped to the site and placed into position using mobile cranes. The site construction sequence
depends on the module installation schedule and whether its predecessor has already been installed in its final
location onsite. The scheduled heavy-lift crane mobilization events take place when a heavy-lift crane is scheduled to
perform a module lift at a certain location onsite. Each lift is preceded by a heavy-lift crane mobilization process.
Four different heavy-lift mobile cranes (two 880-ton, one 660-ton, and one 400-ton) were used to lift the modules to
their final set points. There are over 100 heavy lifts to take place, which require more than 50 crane mobilization
events throughout the project. These mobilization events are expensive and have to be accounted for in the overall
project schedule towards the overall project duration. It is a difficult task to ascertain the shortest and most obstacle-
free mobilization route for each of these cranes in a continually changing site layout. It is also hard to estimate the
mobilization duration required for each of the modules lifting activities in the changing site layout.
The specific location of any of the heavy-lift cranes or modules onsite is depicted through the Visualization Federate
of the framework. The lifting event involves the mobilization of a crane from an initial to a final point. The lift
locations are predefined onsite for each set of modules. Figure 2 shows the predefined lift locations for the cranes.
Once the Discrete Event Simulation (DES) component of the framework triggers a lifting event and chooses a crane,
the crane moves on the visualization screen from its initial location to its lifting location (final location), where it will
install the modules. The path-finding mechanism will find the shortest, safest path for each crane, and the expected
duration of this mobilization. The path for each lifting process will vary depending on the current site layout during
the projects lifecycle.
Fig. 2: PCL-Shell Fort Saskatchewan site and its representation in the BGE
575
HLA Based Simulation Visualization (HSV) Framework

3.3 Communication between Simulation and Visualization Components
The communications between the simulation and visualization components of the framework are done through
several layers, as shown in Figure 3. The visualization Federates reflect the simulations object classes attribute
values as they are updated by the simulation models. These reflections are communicated between the two Federates
through the RTI of the HLA configuration.
On the other hand, the viewer Federates and the 3D visualization engine(s) communicate through a network User
Datagram Protocol (UDP), which provides language-independent communications between the .NET-based HLA
Federation and 3D engine.

Fig. 3: Framework used in simulation and visualization of pipe modules assembly and site installation
3.4 Implementation and Operation of Pathfinding Mechanism Extension (PME)
Path finding inside the Visualization Federate can be divided into two phases:
The mesh generation mechanism phase. The mesh generation mechanism categorizes the modeled site
spaces into Resources Admissible Site Space (RASS), and Resources Forbidden Site Space (RFSS). The
RASS is the site space that is not occupied by objects representing temporary facilities or permanent
2
nd
Layer of
HSV
Framework
Replaceable
Components
3D
Gaming
Engine
Layer
1
st
Layer
HLA
Communication
Spool Fabrication
Simulation Federate
Site Construction
Simulation Federate
Resource
Allocation
Federate
RuntimeInfrastructure(RTI)
Module Yard
Simulation Federate
Site Visualization
Federate
Yard Visualization
Federate
UDP Messages Layer
Python UDP Network Client
Blender Game Engine (BGE) logic bricks
+
Path-finding Mechanism (A* Algorithm &
Mesh Generation Mechanism)
3D Object Behaviors (Site and Yard
Visualization Screens)
HLA Messages
Windows Sockets/Ports
576
structures and can be used by mobile objects (depending on the objects geometry and the space geometry)
for manoeuvring and/or forms a part of the objects road geometry. On the other hand RFSS is the site space
that is occupied by objects representing temporary facilities or permanent structures and therefore cannot be
used by mobile objects for manoeuvring, and does not form a part of the objects road geometry.
The A* algorithm phase. The algorithm uses the mesh generation mechanisms output. The algorithm finds
the shortest, most obstacle-free path.
Both the mesh generation mechanism and the A* algorithm are implemented inside the 3D engine of the
visualization component. Figure 4 illustrates the generated grid that mark the RASS at time 0, and two snap
shots of path selection between lift points at two different stages of project progress where additional
structures are added to the site (red wireframes).
Fig. 4: Mesh covering the construction sites RASS at simulation time zero and path selection at different progress
stages
4. CONCLUSION
This research presents a new way to address the integration of simulation models and spatial data representation. It
presents a framework that uses distributed simulation HLA standards to build on the inherent strengths of both
simulation and visualization and facilitates two way communication between the discrete event models and
visualization components, which become, in this case, contributers to the simulation behaviours. The framework is
demonstrated through a case for simulating heavy crane movements on an industrial construction site. The
visualization component contributes to the simulation be providing estimates of the shortest paths between crane lift
locations, that avoid obstacles onsite, and are dynamically updated with time. Furture work can include extensions to
the proposed approach to analyse and optimize worker movements onsite and placement of temporary site facilities
at different time phases of project progress.

5. REFERENCES
ElNimr A. and Mohamed Y. (2010). A simulation driven visualization framework for construction operations:
Development and operation, Proceedings of the Construction Research Congress, Banff, AB, Canada, May 2010.

ElNimr A. and Mohamed Y. (2011). Application of gaming engines in simulation driven visualization of
construction operations, Journal of Information Technology in Construction, Vol.16, January 2011.

Halpin D. and Woodhead R. (1976). Design of construction and process operations, John Wiley and Sons, New York,
N.Y., USA.

577
Huang R. and Halpin D. (1994). Simulation of cable stayed bridges using DISCO, Proceedings of the Winter
Simulation conference, Buena Vista, USA, December 1994.

Ioannu P. and Martinez J. (1996). Animation of complex construction simulation models, Proceedings of the
Congress of Computer in Civil Engineering, New York, USA, 1996.

Kalk A. and Douglas S. (1980). Insight: Interactive simulation of construction operations using graphical techniques,
Technical Report No. 283, Construction Institution, Department of Civil Engineering, Stanford Univ., Stanford,
California, USA.

Kamat V. and Matrinez J. (2001). Visualizing simulated construction operations in 3D, Journal of Computing in
Civil Engineering, Vol. 15, No.4, October 2001.

Kamat V. and Matrinez J. (2004). General Purpose 3D-Animation with VITASCOPE, Proceedings of the Winter
Simulation Conference, Washington, DC, USA, December 2004.

Kamat V. and Matrinez J. (2008). Software mechanisms for extensible and scalable 3D visualization of construction
operations, Journal of Advances in Engineering Software, Vol. 39, No.8, August 2008.

Oloufa A. (1993). Modelling and simulation of construction operations, Journal of Automation in Construction, Vol.
1, No.4, March 1993.

Oloufa A.A. and Crandall K.C. (1992). Feedback mechanism for operational simulation, Journal of Computing in
Civil Engineering, Vol. 6, No. 2 , April 1992.

Paulson B.C., Douglas S.A., Kalk A., Touran A. and Victor G. (1983). Simulation and analysis of construction
operations, Journal of Technical Topics in Civil Engineering, Vol. 109, No. 2 , August 1983.

Rohrer M. and McGregor I. (2002). Simulating reality using Auto-Mod, Proceedings of the Winter Simulation
Conference, San Diego, USA, December 2002.

Shannon R.E. (1975). System Simulation: The Art and Science, Prentice-Hall, Englewood Cliffs, N.J., USA.

Taylor SJE, Gan BP, Stra burger S and Verbraeck A (2003). HLACSPIF panel on commercial off-the-shelf
distributed simulation. Proceedings of the Winter Simulation Conference. Chick S, Sanchez PJ, Ferrin D and
Morrice DJ (eds). Association for Computing Machinery Press: New Orleans, LA, USA, pp 881887.

Xu J. and Abourizk S. (1999). Product-based model representation for integrating 3D CAD with computer simulation,
Proceedings of the Winter Simulation Conference, Phoenix, USA, December 1999.

Zhen, X.-J., Wu, D.-L. , Hu, Y. , and Fan, X.-M. 2010. A real-time simulation grid for collaborative virtual assembly
of complex products. International Journal of Computer Integrated Manufacturing, 23(6): 500514.
578

DEVELOPMENT OF GRID-BASED NETWORK MODEL FOR PATH
FINDING IN REAL DYNAMIC ENVIRONMENTS
Tatsuru Tomii
Kokusai Kogyo Co., Ltd., Japan
Koji Makanae
Miyagi University, Japan
ABSTRACT: With increasingly widespread use of mobile terminals and continuing improvements in their
positioning capability, pedestrian navigation systems are entering the stage of practical use. Many of the existing
pedestrian navigation systems perform route search based on a network consisting of node links similar to those
used for car navigation. This makes it difficult to express a two-dimensional movement area and dynamic
environmental changes. In order to solve these difficulties, a path finding method based on grid network models
and a path finding system is developed in this study. By dividing the movement area for pedestrians into a uniform
grid of cells and connecting each cell with adjacent cells, the grid network allows setting the environment in the
movement area in detail. The algorithm for used for path finding is A* which performs path calculation efficiently
if given the distance from the start point to the goal point as an estimated value. As the calculated path may include
nodes that do not need to be passed through, a path smoothing algorithm which enables to skip such unnecessary
nodes is developed in this study. For the validation of this study, a prototype of path finding system using a grid
network model was developed. The system performs path finding every time an obstacle is added and indicates
how the calculated path changes because of the obstacle. The result means that the proposed method can be
applied to route findings in real dynamic environments.
KEYWORDS: pedestrian navigation system, grid network model, path finding, A* algorithm
1. INTRODUCTION
With increasingly widespread use of mobile terminals and continuing improvements in their positioning
capability, pedestrian navigation systems are entering the stage of practical use. Many of the existing navigation
systems perform route search based on a network consisting of node links similar to those used for car
navigation. This makes it difficult to express a two-dimensional movement area for pedestrians. It is also
difficult to express dynamic environmental changes in the movement area that can affect pedestrian movement.
This study aims to develop a path finding system using with A* and path smoothing algorithm based on grid
network model so that pedestrians and robots can search routes to their destinations, avoiding dynamic obstacles
on their way, and cope with two-dimensional movement needs associated with indoor environments and such
features as underpasses and pedestrian decks in real dynamic environment, and verify the usefulness of the
newly developed system.
2. PREVIOUS STUDIES ON NETWORK MODELS
In a network used for pedestrian route search, the movement area consists of planar elements instead of linear
elements. This means that there are an unlimited number of possible pedestrian routes, and it is difficult to
perform route search if nothing is done to address that problem. One method of abstraction from a planar
movement area to a linear network is to generate skeletons from the shapes of the areas of interest and use the
skeletons thus obtained as topological maps. Skeleton generation methods include the method of deriving
skeletons from Voronoi diagrams (Roberts et al., 2004), the method of obtaining skeletons by Denaulay
triangulation (Chithambaram et al., 1991) and the method of deriving skeletons from the figures obtained by a
continuous shrinking process in which the edges of each polygon are moved inwards in parallel with themselves
(Aichholzer et al., 1995). As an approach to be used when dealing with raster data such as images, a study
(Walter et al., 2006) focused on the method of deriving a skeleton by continuing the process of shrinking the
movement area by morphological operation until the width of the area becomes one cell.
In a plane, the shortest path can be determined by topological mapping and smoothing. It is difficult, however, to
perform path finding taking into consideration such factors as topographic relief, small steps and the ground
surface environment. One way to perform optimum path finding in a plane taking cost into consideration is to
579

divide the plane into grid cells (tiles) and assign a cost to each tile. A plane can be divided into uniform cells by
dividing it into regular triangle, square or regular hexagon cells (Smed and Hakonen, 2006). If the sizes of
obstacles existing on the grid and the largeness of area to be related with cost are to be taken into consideration,
grid cell size is also an important consideration. For example, if the distance between two obstacles is smaller
than grid cell size, the link of that route is cut off so that the route becomes impassable. A grid that makes all
routes passable can be generated by setting cell size according to the narrowest route. Doing so, however, would
increase the number of nodes so as to increase the time required for path finding. As a solution to this problem, a
study (Ericson, 2005) looked at the method of using a non-uniform-cell grid (hierarchical grid) in which cell size
can be hierarchically changed. As a study on the use of uniform grids, Masaki et al. (2002) developed an
application schema taking human activities into consideration and constructed a model capable of linking a grid
network to a representative line network. Kawabata et al. (2005), in their study, assigned spatial attributes to each
cell and performed path finding taking into consideration the cost at each location. Behzadi et al. (2008) used a
genetic algorithm for shortest path computation.
If shortest path finding is performed in a uniform grid network, node-to-node links can only connect adjacent
cells. Consequently, links are made only for predetermined directions so that more than one shortest path
candidate exists. Depending on the algorithm, therefore, the computed path is not necessarily the shortest path in
reality. One network approach that makes it possible to find the actual shortest path is the visibility graph method
(Ota et al., 2001). In the visibility graph method, the start and goal points and the vertices of obstacles are
regarded as nodes. If the line segment between two nodes does not pass through any obstacle, a link is drawn
between the nodes. A visibility graph is a graph thus drawn.
3. NETWORK MODELING
3.1 Grid network model
The shortest distance in a plane can be determined by using a visibility graph showing the links connecting the
vertices of obstacles. In the visibility graph method, however, it is difficult to express the environmental
conditions such as rises and falls in the movement area that can affect the movement of pedestrians.
Environmental details can be reflected in the plane by dividing the area into small cells and assigning local
environmental attributes to each cell. This chapter explains a grid network model as a network model that can be
used for the purpose of pedestrian navigation.
(1) Generating a grid network
A network in which the movement area is divided into an array of uniform square cells and adjacent cells are
interconnected (hereafter referred to as a "grid network") is explained here. As the first step, the movement area
is divided uniformly according to cell size to generate nodes. Nodes are stored in a list containing an index of
coordinates. Data can be expressed in a two-dimensional array, and it is advantageous in terms of computation
efficiency to express data in the form of a hash table keyed to cell coordinates (Ericson, 2005). If not only nodes
but also other values including links and lists used for shortest path finding are unique to the key, associative
array objects are used for management. Nodes are managed by using their coordinates as keys, and links are
managed by using the coordinates of the two end points of each link as keys. Generated nodes are connected to
adjacent nodes. In the case of four-way connection, a great number of shortest paths would be indicated as a
result of shortest path finding, while in the case of eight-way connection, a smaller number of shortest paths are
indicated so that a path close to the actual shortest path can be selected (Figure 1). A link is expressed as a
directed link from a node to another node, and a non-directed link is expressed with a pair of opposed directed
links. A different cost dependent on the direction of movement can be expressed by using two differently
directed links.

Fig. 1: Eight-way connection
580

(2) Adding obstacles
A network changes with the addition of obstacles. Figure 2 shows a polygonal obstacle added to a network.
When adding an obstacle to an area, calculation needs to be done to determine which nodes are included in the
obstacle area and which links are disconnected. Because scanning all nodes is inefficient, a bounding box is
defined so as to enclose the obstacle, and the boxed area is examined to determine which nodes are included in
the obstacle. If a node is located inside the obstacle, the node is deleted from the list. When a node is deleted
from the network, all links connecting to that node are also deleted.
If an obstacle collides with any of the links a node has, the path becomes impassable even when the node itself is
not included in the obstacle. If such collision is found, therefore, it is necessary to judge which link collides with
the edge of the obstacle and delete all crossing links. Figure 3 shows a linear obstacle added to a network. As the
first step, the nodes in the vicinity of the line segment representing the obstacle are identified. If any of the nodes
thus identified exists on the line segment, the node is deleted as in the case where a point is included in an
obstacle. After adjacent nodes are identified, checks are made for collision between the links each node has and
the obstacle. If a link is found to be crossing, that link is deleted. For the edges of a polygonal obstacle, too,
similar processing is performed to alter the network.

Fig. 2: Polygonal obstacle Fig. 3: Linear obstacle

(3) Adding nodes and links
In the case of a layered structure like a building, path finding cannot be made unless different levels are linked.
An area obstructed by a roadway in which pedestrians are not allowed can be made crossable by pedestrians by
providing a link connection. A link can be generated by selecting the grid node closest to the location in need of
a link. If you want to place a node at an arbitrary location and generate a link from that node, you can do so by
creating an extension node list, adding a node to be placed and linking the node to adjacent nodes.
(4) Making judgment as to node inclusion and link collision
In order to disconnect a link in the grid network, smooth a computed path or generate a link in a visibility graph,
it is necessary to make judgment as to whether the link collides with an obstacle. The nodes existing in obstacles
are deleted from the network in order to reduce the computation workload required for path finding. This study
deals with obstacles that can be expressed as rectangles or polygons.
3.2 Path finding
(1) Path generation
This section explains the path finding algorithm used for the grid network and the visibility graph method. The
algorithm used for path finding is A*. In the case of path finding in a plane, in the grid network, a path of a
distance close to that of the actual shortest path can be generated by smoothing the calculated path. In the grid
network in a plane, path smoothing is also performed in this study.

linear obstacle
crossing links
crossing node
Proximity nodes
Bounding box
581

(2) Path finding by use of A*
The A* algorithm performs path calculation efficiently if given the distance from the start point to the goal point
as an estimated value (Hart et al., 1968). The A* algorithm is briefly explained below.

The first step is to consider the shortest path from start point S through node N to goal point G. Let f(N) be the
cost of the path from S to G; g(N), the minimum cost of the path from S to N; and h(N), the minimum cost of the
path from N to G. The relationship among these costs can expressed as
(1)
Since, however, the minimum costs cannot be given in advance, estimates of those costs are used.
(2)
If the shortest path passes through node M linked to N, then f*(M) can be expressed as
(3)
where COST(N, M) is the cost of the path from N to M, which is given in advance. Let h*(N) be a heuristic
function that returns a linear distance value. Then,
(4)
(5)
(6)
If N = S, then g*(S) = 0. Hence,
(7)
From Eq. (6), calculation can be made as follows:
(8)
Since f(N) and f(M) are of the same path, f(N) = f(M), and the following relation holds true:
(9)
Hence,

(10)
Because the shortest path does not become shorter than f*(S) regardless of the nodes that are passed through,
f*(M) = DISTANCE(S, G). From Eq. (2), the value of g*(M) can be estimated as follows:
(11)
If the shortest path passes through the node L linked to M, from Eq. (4.2.3), we have
(12)
and its value can be estimated from Eq. (11). Thus, the shortest path can be found by updating the values of
g*(N) and f*(N) while performing search from the start point.
582

In node search, the nodes linked from the start point are added to the list, and their values are estimated. Then,
the nodes with the smallest value of f*(N) in the list are followed sequentially, and the linked nodes are added to
the list.
The flow of the process resulting from the implementation of A* is described below.
1) Add S to the Open list and calculate the value (estimate) from Eq. (7).
2) If the Open list is empty, then the search fails and the process ends.
3) Extract the node N that minimizes f*(N).
4) If N = G, the search ends.
5) Transfer N to the Close list.
6) Calculate g*(N) from Eq. (2).
7) Perform the following processing for each node, M, linked to N.
Calculate f'(M) from Eq. (6).
In cases where M is included in the Open list, if f'(M) < f*(M), then update f*(M) to f'(M) and make M's
parent node N.
In cases where M is included in the Close list, if f'(M) < f*(M), update f*(M) to f'(M) and make M's parent
node N. Also, transfer M to the Open list.
In cases where neither the Open list nor the Close list includes M, update f*(M) to f'(M) and make M's
parent node N. Also, transfer M to the Open list.
8) Return to Step 2.
(3) Path smoothing
The calculated path may include nodes that do not need to be passed through. In such cases, the path can be
made shorter by skipping such unnecessary nodes. So that a path smoothing algorithm is developed and applied
in this study.
There are two methods of smoothing when making judgment as to skipping from the start point. One (A) is to
make judgment by making pairs with nodes closer to the start point, and the other (B) is to make judgment by
making pairs with nodes closer to the goal point. Figure 4 shows smoothed passes using these methods. Both
methods can find a same shortest pass when the intervals of grid points is wider than the width of corridors.
However the method(B) can find a shorter pass when a loophole which is narrower than the interval of grids
exists.
(A) Making pairs with nodes closer to the start point

(B) Making pairs with nodes closer to the goal point

Fig.4: Comparison of path smoothing methods
583

Figure 5 shows smoothed paths derived by two methods in an environment in which multiple obstacles exist. As
the figure shows, the method (B) can find a shorter pass in such an environment.

Fig. 5: Path smoothing
4. PATH FINDING SYSTEM USING GRID NETWORK MODEL
4.1 Constructing a path finding system
For the validation of this study, a path finding system using a network model was developed. The system is
capable of performing path finding by use of a grid network and a visibility-graph-based network model and
comparing two or more paths by drawing them on the same canvas. Path finding is performed when an obstacle
is placed in the space, and changes in path calculation results resulting from the obstacle can be observed. For
the purpose of the system development, HTML5 Canvas elements were used to draw graphics, and the programs
were written in JavaScript. Execution checks were made by using Firefox and Google Chrome supporting
Canvas elements.
Figure 6 shows path finding results obtained by using the newly developed system by means of the grid network
model and the visibility graph network model. The figure on the left shows the paths indicated before an obstacle
is added, and the figure on the right shows the paths indicated after the obstacle is added. As shown, the path
indicated by the grid network model has become closer to the path indicated by the visibility graph network
model. Thus, the system performs path finding every time an obstacle is added and indicates how the calculated
path changes because of the obstacle. The result means that the proposed method can be applied to route findings
in real dynamic environments.

A
B
grid
584

Fig.6: Route finding in dynamic environments

4.2 Application of the path finding system
If there are any surface elements that can affect the movement of pedestrians, such as rises and falls, in the
movement area, it is necessary to reflect the surface conditions in the movement area in path (route) finding
results. Since a grid network allows different costs to be assigned to uniformly spaced nodes and links, the
environmental conditions in the movement area can be reflected in route search results by using a grid network.
Figure 7 shows route search results obtained by assigning an elevation value to each node. White cells represent
high elevation areas, and black cells represent low elevation areas. Lower costs are assigned to downslope links,
and higher costs to upslope links. As a result of path calculation, paths have been selected so as to avoid upslopes
as much as possible.

(using Digital Map Grid (Elevation), GSI, Japan)
Fig.7: Route search using digital elevation models

added obstacle
grid (smoothing) visibility
585

5. CONCLUSION
This paper has reported on the development of a path finding system using a grid network model as a network
model for pedestrian and robot navigation in real dynamic environments and the validation of the newly
developed system.
By dividing the movement area for pedestrians into a uniform grid of cells and connecting each cell with
adjacent cells, the grid network allows setting the environment in the movement area in detail. Also, the
algorithm for pass smoothing is developed and added to the process of route findings in this study. The system is
capable of performing path smoothing by skipping nodes that do not need to be passed through and calculating
shorter paths.
One problem is that data on obstacles in the pedestrian space are not necessarily available. In view of the extent
of the area in which pedestrians can cover, optimum paths are thought to be dependent on whether or not moving
obstacles are reflected. There are similar problems with the environment of the area covered by the grid network.
The environment affecting the movement of pedestrians and robots consists of various elements. It is necessary,
therefore, to find ways to identify those elements from real dynamic environments and reflect the information
thus acquired in the network.
6. REFERENCES
Aichholzer O., Alberts, D., Aurenhammer, F. and Grtner, B. (1995). A Novel Type of Skeleton for Polygons,
Journal of Universal Computer Science, 1(12), 752-761.
Behzadi, S. , Alesheikh, A.A., and Poorazizi, E. (2008). Developing a genetic algorithm to solve shortest path
problem on a raster data model. Journal of Applied Sciences, Volume 8, pp. 3289-3293.
Chithambaram, R., Beard, K., and Barrera, R. (1991). Skeletonizing polygons for map generalization. in
Technical papers, ACSM-ASPRS Convention, Cartography and GIS/LIS, Vol. 2.
Ericson, C. (2005). Real-Time Collision Detection (The Morgan Kaufmann Series in Interactive 3-D Technology).
Morgan Kaufmann.
Hart, P.E., Nilsson, N.J., Raphael, B. (1968). A Formal Basis for the Heuristic Determination of Minimum Cost
Paths. IEEE Transactions on Systems Science and Cybernetics, SSC4 (2), pp. 100107.
Roberts, S.A., Hall, G.B. and Boots, B. (2004). Street centerline generation with an approximated area Voronoi
diagram. In P, Fisher (ed.), Developments in Spatial Data Handling, Berlin, Springer, 435-446.
Smed, J. , Hakonen, H. (2006). Algorithms and Networking for Computer Games, Wiley.
Walter, V., Kada, M. and Chen, H. (2006), Shortest path analyses in raster maps for pedestrian navigation in
location based systems. International Symposium on Geospatial Databases for Sustainable Development, Goa,
India, ISPRS Technical Commission IV (on CDROM).
586
Author Index
A
Abourizk, Simaan 340, 571
Achar, Madhav 494
Amor, Robert 460
Anderson, Anne 54
Anthes, Christoph 417
B
Bae, Hyojoon 253
Balakrishnan, Bimal 451
Bargstdt, Hans-Joachim
29, 64, 162, 521
Behzadan, Amir H. 292
Blersch, Daniel 360
Borasio, Emanuele 360
Borrmann, Andr 467
Breit, Manfred 467
C
Chan, Long 107
Chen, Albert Y. 407
Chen, Cheng-Yi 142
Chen, Huan-Ting 90
Chen, Jian-Yuan 427
Chen, Po-Han 11, 107
Cheng, Jack Chin Pang 187
Cheng, Yuan-Yuan 142
Chou, Chien-Cheng 73
Chou, S.A. 198
Chuang, Po-Chuan 11
Coelho, Antnio F. 300
Crosbie, T. 80

D
da Silva, Hugo M. 300
Dan, Hiroshige 332, 532
Dang, Trang 521
Dawood, N. N. 80, 99
Deng, Yichuan 187
Di Mascio, Danilo 37
Dimitrov, Andrey 152
Dossick, Carrie Sturts 54
E
Ebner, Andreas 350, 552
Elmahdi, Amir 521
El-Nimr, Amr 571
F
Fabbri, Matteo 360
Feng, Chen 235
Friedewald, Axel 271
Froehlich, Christoph 360
Fukuda, Tomohiro 443
Fuyuki, Masahiko 332
G
Ganapathi, Adithya 183
Garg, Akash 309
Ginzburg, Alexander 505
Golparvar-Fard, Mani 152, 253
Gu, Ning 212
Gnthner, Willibald 350, 552
H
Hakkarainen, Mika 203
Han, SangUk 494
587
Han, Sooyeon 561
Hatori, Fumio 263
Heinig, Martin 271
Hildreth, John C. 369
Hodds, Michael 99
Hollermann, Sebastian 64
Horenburg, Tim 552
Hou, Lei 319
Hsieh, Cheng-Yuan 47
Hsieh, Shang-Hsien 90, 113
Hsieh, Yo-Ming 323
Huang, Li 467
Huang, Po-Ming 132
Hung, C.L. 198
Hung, Wei Han 243
Huynh, Vui 282
I
Ishigaki, Taisuke 532
J
Jacob, Joo P. 300
Jvj, Pivi 434
Jung, Nusrat 434
K
Kagehiro, Tatsuhiko 263
Kamat, Vineet R. 235, 292, 309
Kammergruber, Florian 350, 552
Kan, Meng-Shen 11
Kang, Julian 183
Kang, S.M. 113
Kang, Shih Chung 243
Kazakov, Konstantine 542
Kim, Mi Jeong 561
Kitagawa, Etsuji 399
Kochkine, Vitali 510
Kranzlmller, Dieter 417
Kubota, Satoshi 399
Kung, Cheng-Ju 142
Kurihara, Tsuneya 263
Kuula, Timo 203
L
Lang, Frank 467
Lee, Jeong-Shing 107
Lee, Meng-Chung 19
Lee, Ming-Tsung 142
Lee, SangHyun 494
Lee, W.L. 113
Lee, Wan-Li 132
Leite, Fernanda 122
Li, Chih-Yuan 19
Lin, Bing-Yi 142
Lin, Chia-Ying 73
Lin, Chih-Chuan 132
Lin, Ken-Yu 54
Lin, Yu-Cheng 19
Liu, Hsiao-Hsuan 407
Liu, Jang-Jeng 142
Liu, Ya-Sue 323
Ldding, Hermann 271
Lu, Hsiang-Wei 132
Lu, Ming 340
M
Makanae, Koji 579
Mao, Sheng 340
Martin, Brad 451
McMeel, Dermott 460
Melzner, Jrgen 162
Mhalas, A. 80
Moh, R. 113
Mohamed, Yasser 571
588
Monobe, Kantaro 399
Muramoto, Katsuhiko 282
N
Nakamura, Kenji 399
Nishigata, Tatsuaki 532
Nseir, Hussam 183
O
Onoue, Takuya 443
Oprean, Danielle 451
Ostwald, Michael 212
P
Pivnen, Jani 434
Pei, T.J. 113
Pea-Mora, Feniosky 152, 494
Petrushevski, Filip 379
Petzold, Frank 417
Piira, Kalevi 203
Porkka, Janne 434
R
Rankouhi, Sarah 225
Rasys, Edvinas 99
Reisser, Brandon S. 369
Rinke, Nils 510
Ritter, Fabian 467
Rodrigues, Rui 300
S
Savisalo, Anssi 434
Scheer, Sergio 476
Schubert, Gerhard 417
Schweitzer, Jrgen 510
Seisto, Anu 203
Semenov, Vitaly 542
Shen , Zhen-jiang 389
Shimomukai, Takashi 332
Sipetic, Milos 379
Sireeni, Jarkko 434
Siu, Ming-Fung Francis 340
Smith, Melina 451
Stange, Fabiano 476
Su, Yu-Chih 19
Sugihapa, Kenichi 389
Suter, Georg 379
Suwal, Sunil 434
T
Tanaka, Shigenori 399
Tomii, Tatsuru 579
Tsay, Ren-Jwo 427
Tsubokura, Tetsuya 263
U
Urano, Takehiro 263
V
von Gsseln, Ilka 510
W
Wang, C.H. 113
Wang, Li 122
Wang, Shih-Hsu 142
Wang, Wei-Chih 142
Wang, Xiangyu 319, 561
Watanabe, Yuki 263
Waugh, Lloyd M 225
Weng, Shao-Wei 142
White, Jules 253
Woodward, Charles 203
589
Wu, I-Chen 47
Wu, Ko-Chiu 486
Wu, Si-Wei 90
Y
Yabuki, Nobuyoshi 443
Yashiro, Yuichi 263
Yasumuro, Yoshihiro 332, 532
Yeh, Ming-Feng 142
Yi, Junlin 122
Yoshida, Shinji 443
Yu, Rongrong 212
Z
Zhu, Zhenhua 172
Zolotov, Vladislav 542

590

CONVR 2012 Proceedings

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

CONVR 2012 Proceedings

Uploaded by

Copyright:

Available Formats

CONVR 2012

Author Index ....................................................................................................... 587

Inteiioi Aiea of 0ne Layei Builuing = (a - 1u) (b - 1u) (3.1)

the best value for criteria , respectively(also referred to as positive ideals)

You might also like