EVALUATING THE IMPACT OF BUILDING MODELING (BIM) ON CONSTRUCTION

Patrick C. Suermann, Maj, USAF, P.E. PhD Candidate, The University of Florida suermann@ufl.edu Raja R.A. Issa, Ph.D., J.D., P.E. Professor, The University of Florida raymond-issa@ufl.edu ABSTRACT:

INFORMATION

This research assessed perceptions about the impact of the implementation of Building Information Modeling (BIM) on construction projects. Survey questions centered on impact with respect to six primary construction key performance indicators (KPIs) commonly used in the construction industry as accepted metrics for assessing job performance. These include: quality control (rework), on-time completion, cost, safety (lost manhours), dollars/unit (square feet) performed, and units (square feet) per man hour. Qualitative data was collected through a survey instrument intended to assess practitioners’ perceptions about BIM impacts on the six Key Performance Indicators. The survey was targeted at National Institute of Building Sciences (NIBS) Facility Information Council (FIC) National BIM Standard (NBIMS) committee members. The survey, with a response rate of 50 completed surveys, showed preliminary results indicating that the NIBS FIC NBIMS members felt that a BIMbased approach improves construction metrics compared to construction without BIM. Specifically, the highest three ranking KPIs in order of most favorable responses were quality, on time completion, and units per man hour. The second tier of favorable responses included overall cost and cost per unit. Finally, only 46%, or less than half, of the respondents thought that construction safety was improved through BIM. KEYWORDS: BIM, Construction, NBIMS, Metrics, KPI

1. INTRODUCTION
In 2004, the National Institute of Standards and Technology (NIST) published a report stating that poor interoperability and data management costs the construction industry, approximately $15.8 billion a year, or approximately 3-4% of the total industry. Since this report, many have labeled Building Information Modeling (BIM), an emerging technological information management process and product, as the answer to this problem. From the pending release in July 2007 of the National BIM Standard (NBIMS), a BIM (i.e. a single Building Information Model) is defined as “a digital representation of physical and functional characteristics of a facility.” Furthermore, a BIM represents a shared knowledge resource, or process for sharing information about a facility, forming a reliable basis for decisions during a facility’s life-cycle from inception onward. In the words of the NBIMS Executive Committee Leader and former Chief Architect of the Department of Defense, Dana K. “Deke” Smith, R.A., “A basic premise of BIM is collaboration by different stakeholders at different phases of the life cycle of a facility to insert, extract, update or modify information in the BIM to support and reflect the roles of that stakeholder” (NBIMS 2007).

2. METHODOLOGY 2.1 Overview
This paper focuses on only the first phase of our research, which proposes to accomplish data collection and analysis in four phases. These four phases will be aligned with a process originally created by United States Air Force Colonel John Boyd (1927-1997) and shown in Figure 1. Information Management (IM) professionals have often used Boyd’s model, which is widely known as the “OODA Loop” (Observe, Orient, Decide, and Act), to

7th International Conference on Construction Applications of Virtual Reality: October 22-23, 2007

206

and the state where the respondent resided. in this research.2 Survey Specifics • • • • The survey was divided into four sections: Part I: Basic Demographic Information Part II: BIM Effects on KPIs Part III: Ranking KPIs Part IV: Free Answer Part I was intended to find descriptive information about the respondents.demonstrate the continual improvement process of strategic decision making. On time Completion. a subset of the NIBS-FIC. Cost. Boyd said that this cycle of decision-making could operate at different speeds for different organizations but the goal is to complete the OODA Loop process at the fastest tempo possible. then the decision makers will orient their thinking in erroneous directions and eventually make incorrect decisions. under their “NEWS” portion. there is little empirical data regarding its application and use. However.2 Phase I: Observation Because BIM is so new in the US Architecture. Through Boyd’s OODA Loop. he expanded his OODA Loop theory into a grand strategy with benefits to anyone who needs to pragmatically and quickly process information. This survey data was used to determine current BIM practices and perceptions to formulate additional research hypotheses for use in Phase II. it will be used to make the best. and Operations (AECO) industry. Phase I included publishing a web-based survey with the sole purpose of garnering industry stakeholders’ impressions of BIM’s effect on construction through specific construction metrics based on six (6) primary. In this way. people will observe unfolding circumstances and gather outside information in order to orient their decision making system to “perceived threats.facilityinformationcouncil. a reminder email was sent to the listserv asking for more people to complete the survey or for those who had started the survey to complete the survey.2. The OODA Loop will be used here to structure the research to ensure that each phase builds on the one before it and in this way the conclusions will be logically valid. Colonel Boyd’s philosophy dictated that individually.1 Survey After receiving University of Florida Institutional Review Board (UFIRB) authority. the possibility of tainting the data was considered negligible.2. and action (Boyd 2007). http://www. quantitative construction KPIs: Quality Control. Halfway through the month-long survey availability. and to group answers from similar respondents together across the data pool. qualitative industry perceptions were quantified. The second method of garnering qualified respondents was to advertise the survey on the NIBS FIC website. this research will be structured in four phases aligned with the ideas of observation. this data was shared for their own empirical research.com through an account login funded by the National Institute of Building Sciences. 2007 until April 5. and this website is only “advertised” in the AECO community. First. Units/Manhour as determined in a 2003 study by Cox et al. a qualitative survey was administered to garner initial data about practitioners’ perceptions about the effects of BIM on construction key performance indicators (KPIs). This listserv had 104 members from across the AECO industry at the time of the survey’s launch. not necessarily the fastest. orientation. Questions 7th International Conference on Construction Applications of Virtual Reality: October 22-23.zoomerang. age. Engineering. $/Unit. Since most people only happen upon this website when signing up to join the NIBS-FIC NBIMS committee. Most questions were standard for surveys such as gender. 2. or misunderstand what is happening in the environment. 2. the first iteration of the survey was available from March 5. 2007 and was advertised to a select group in two different ways: direct email through a distribution list and a website advertisement. But in the later years of his career. The survey was hosted on http://www. decision. Facility Information Council (NIBS-FIC). an email was sent to the FIC listserv distribution list. 2007 207 . (2003). Therefore. to ensure that they were qualified to answer the questions. in addition to the typical review of literature in the field.org/bim.” Boyd states that the orientation phase of the loop is the most important step. Boyd developed the theory based on his earlier experience as a fighter pilot and he initially used it to explain victory in air-to-air combat. choices about the proper items to collect and investigate. In concert with the National BIM Standard (NBIMS) Committee testing team. because if decision makers perceive the wrong threats. Construction. Safety. 2.

question #14 of the survey addressed BIM’s impact on units per man hour. Cost was defined as “cost variance in actual costs to budgeted costs. asked about safety. Regarding organizational role. #15. as well as the additional choice of Not Applicable or “N/A. At the beginning of Part II.” The respondents’ choices of answers ranged on a 5-point Likert scale from least favorable to most favorable with the following possible choices: Severely Inhibits Lessens No Effect Improves Maximizes 1 2 3 4 5 The next question.” The response options were similar to those for question #17 with the exception of variance equating to a “late” project on the unfavorable side of the scale to “max variance – early” on the favorable side of the scale. Respondents were asked to rank the KPIs on a Likert scale from 1-10. respondents were asked to “read the following statements and choose the one that most closely matches your view of BIM’s effect on safety. respondents were asked to rate their perception of BIM’s impact on the list of six construction key performance indicators.” The 5-point Likert scale had the following choices: Max Variance: ($ Lost) Worsens No Effect Improves Max Variance ($ Saved) 1 2 3 4 5 Question #18 focused on “on time completion. annual company revenue. Regarding safety. Here. Finishes.” The answers. the possibility of errant responses from people just putting the maximum answer down for every question was avoided.especially germane to the research were the following which were targeted at collecting the respondent’s educational level. This question prepared the respondent for answering by saying. #17. The final question in Part II. with regard to lost man-hours. Mechanical. and Overall. so that it could be investigated more thoroughly in Phase II of the research while collecting case study data. and give respondents a chance to express themselves if they felt the survey stifled their responses in any way. were again arranged on a 5-point Likert scale: Eliminates Lessens No Effect Increases Greatly Increases 1 2 3 4 5 The next question. and people’s organizational role. and then the survey skipped to the question that addressed the proper organizational role with a follow-up question formulated to find out the specific role the respondent filled on a daily basis. First. 2007 208 . had to do with cost. In this way. “quality control can be defined as percent (%) of rework in ($) compared to overall cost in ($). 7th International Conference on Construction Applications of Virtual Reality: October 22-23. respondents were asked to make a selection from a list based on the organizational roles listed in Table 32 of the Construction Specifications Institute (CSI 2007). Electrical. and Plumbing (MEP). This means that 1 would be a score showing that BIM inhibited construction to 5 equalling no effect to 10 showing the most improvement. respondents could choose from a 5-point Likert scale. Units per man hour were defined for respondents as “measure of completed units (typically square footage) put in place per individual man hour of work. #19. respondents were asked to select their overarching organizational role. Specifically. Question #16. receive contact information if people wanted specific follow-up information. asked respondents what they thought about BIM’s impact on quality control/rework. Part II of the survey served as the beginning of the primary data collection instrument. These choices also came from the CSI’s (2007) Omniclass Table 32 for organizational roles. Structural. This part asked questions on each of the six construction KPIs in various formats with varying scales of favorable to unfavorable perceptions regarding the impact of BIM on construction. asked for the same perception about BIM’s impact on “dollars per unit” or cost per square foot ($/SF) with the same choices on the 5-point Likert scale.” Here there were five sub-questions under this one question that centered on different types of costs including: General Conditions.” The choices were: Increases Rework Worsens No Effect Improves Nearly Eliminates Rework 1 2 3 4 5 Part III of the survey was structured to determine whether there was any one construction KPI which BIM impacted more than any other in a logical ranking fashion. Part IV of the survey was intended to gather open ended responses from respondents that could help identify problems with the current survey. necessary points to investigate in future surveys.

Army Corps of Engineers (USACE).3 Future Research: Phases II-IV. the lessons learned from conducting the embedded research will be applied to a further revised methodology recommended for future case study data collection. In Phase IV. In addition. a wider cross section of construction projects will be studied including commercial and industrial projects. The other choices included: • BIM is 3D CAD • BIM is a tool for visualizing and coordinating A/E/C work and avoiding errors and omissions • BIM is an open standards based info repository for facilities lifecycles 2.The Summary portion of the survey was intended to determine respondents’ personal definition of Building Information Modeling. and Action Phase II will include building on the survey data garnered in Phase I and will test the primary research hypothesis and possible follow on hypotheses by conducting embedded research on federal construction projects. noted trends will be discussed in the research analysis portion of this document and recommended for consumption and implementation by federal entities and construction firms for recommended best business practices that yield the most productivity improvements. there is little documented evidence on BIM’s impact on the construction phase of the facility lifecycle. The specific locations where the embedded research will be conducted due to their considerable experience at managing projects through BIM methodologies are: • U. 2007 until April 5. Phase III entails comparing the data collected in Phases I and II to longitudinal data. “Don’t Know” which was a response intended to eliminate unqualified respondents from tainting the data pool. South Carolina • General Services Administration (GSA) After assessing and analyzing the data and comparing it to longitudinal data from construction projects similar in size and scope to those studied in Phase II. Coast Guard Naval Engineering Support Unit (NESU). Possible additional case study data will include trend analysis on commercial and industrial projects and comparison to the federal construction projects case study data 3. Decision. including one response. As far as education 7th International Conference on Construction Applications of Virtual Reality: October 22-23. Regarding gender. However. despite recent promulgation of BIM procedures in documents like the GSA BIM Guide and USACE BIM Roadmap.1 Part I: Basic Demographic Information Figures 2-5 show the data gathered through the Zoomerang online survey or data analysis derived from the data in the survey. Lastly. the 45-54 year olds with an overall normal distribution of respondents. the data will be analyzed to determine if trends exist that demonstrate significant differences in productivity or performance according to the construction KPI metrics. after the bulk of the data is collected. fulfilling the OODA Loop. Orientation. There were four choices. This would be accomplished by using data collected and maintained by research bodies such as the USACE Civil Engineering Research Laboratory (CERL) or the Construction Industry Institute (CII) and comparing their baseline construction data to BIM case study data from Phase II. There was only one respondent under 25 years old. future research proposes to evaluate BIM effects on federal construction projects according to the KPI metrics listed in the survey. recent strides in the General Services Administration (GSA).S. While federal work has not always led the way on implementing new technological initiatives. In this way. 50 respondents fully completed the survey for a highly successful 48% response rate. Louisville District • U. Therefore. 2007. Charleston. 3. Phase III will include revisions and changes to the data collection model. Army Corps of Engineers (USACE). The age data of the respondents shows that the mode response was also the median age group. RESULTS The first iteration of the survey was sent out to the NBIMS committee of the NIBS-FIC and was available from March 5. 86% (43/50) of the respondents were male and 14% (7/50) female. The information below represents a summary of the results from the survey. Seattle District • U. and United States Coast Guard (USCG) demonstrate that they are exceeding typical industry rate of BIM adoption. Additionally.S. 2007 209 .S. the research will act on the lessons learned. The rationale behind this research is that federal entities have provided testbeds for implementing new ideas and new technologies in the past in the field of construction. Army Corps of Engineers (USACE). Of the 105 people on the committee when the survey was closed out.

9 Million with 24% (12/50) of the respondents choosing this answer. but are either American citizens or are liaisons for wider interests such as the North American BIM buildingSmart Initiative (sic). and Finishes (58%. with 56% (28/50) of them holding graduate or professional degrees. Respondent #8: The way you ask your questions. It will make it easier to keep the data current and to verify it. Cost (480). as well as 11/50 respondents opined that BIM.S. with 10 showing the most improvement. etc. Structural (76%). Respondents’ geographic locations were varied with 47/50 respondents living in the U. “Nearly Eliminates Rework” for a total rating of 90% (45/50). 7th International Conference on Construction Applications of Virtual Reality: October 22-23. Management. The following list is organized in order of the highest rated to the lowest rated of the six KPIs: Quality Control/Rework (90%).2 Part II: BIM Effects on Construction KPIs Respondents were asked to rate their perception of BIM’s impact on six KPIs. The organizational role data results showed that the two most frequent responses were from those with a Design Role with 44% (22/50) respondents and from those with a Management role. etc. making it the KPI that in their perception is the least impacted by BIM. misconceptions and the net effect may be additive (but save the contractor the time. Cost (80%). Units/Man-hour (516). with 18% or nine of the 50 respondents living there. and 1 showing that BIM inhibits the given KPIs. For example. and three from outside the U. Dollars/Unit (70%). I have quite often seen that BIM corrects errors. General Conditions (70%). Design Role. 5 showing no effect. so it is possible for respondents on the U. Organizing the construction KPIs according to merely adding positive response frequency percentages (anything over a score of 5).e.4 Part IV: Comments • • • A few of the most representative comments made by the respondents were: Respondent # 3: A BIM will likely affect KPIs rather than the other way around. scheduling. For the second most frequent response. money and the embarrassment of a mistake). (Note: despite being the U. A good. several members live and work outside the U. estimating. the KPIs score the same: Quality (564). You may want to look at asking that questions that a builder. or trade contractor would ask. structured source of accurate data that all the stakeholders can access will reduce stove pipes. 47% (7/15) were Vice Presidents in their organization and 40% (6/15) respondents were the Chief Executives of their organization. On-time Completion (90%). Units/Man-hour (86%). 3.S. 2007 210 . more accurate. Dollars/Unit (80%).) The most frequent response by state was from Maryland. assigned a value greater than 3 on the Likert scale) by the respondents: Overall (84%). Dollars/Unit (480).S. it seems as if you assume that BIM should save time and money. Cost-Overall (84%). and Plumbing (78%). NBIMS listserv to live outside the U. On-time Completion (528). comprehensive. redundant data and inaccurate information.3 Part III: Ranking Construction KPIs Respondents were asked to rank the construction KPIs according to their perceptions of how well BIM improved the given KPIs on a scale of 1-10. 3. I believe that the BIM makes your planning. In reality. NBIMS committee. There was no definite trend indicated on the organizational revenue question. 86% (43/50) of the respondents had college degrees. If there was inadequate time or more planned for a given scope. and Safety (54%. Units/Man hour (76%). Respondent #7: The questions that are being asked are of the type that an A/E would ask. Mechanical.S. On-time Completion (88%).) It is important to note that 46% or 23/50 respondents also felt that BIM has “No Effect” on safety or lost man-hours in construction projects. which accounted for 30% (15/50) respondents.S.S. the KPIs score the following in order from most to least favorable: Quality (94%). Electrical. vendor. Cost was similarly broken down and the following list organized in the order of highest to lowest rated favorable opinion (i. 34/50 respondents opined that BIM “Improved” the Quality Control/Rework KPI. This was calculated by evaluating responses that exceeded the neutral Likert value of 3 and comparing that to the total number of responses.) This information is graphically illustrated in Figures 4 and 5. 73% (16/22) of the respondents were architects and 27% (6/22) respondents were engineers. Full data on the responses can be seen in Figures 2 and 3.) When weighting the answers for the degree of favorability (Σ(%*100*Score)). Of the first most frequent response. 3. and Safety (324.level. although the most frequent response was $1-$9. than it may it may be just as likely to add time or money as save (sic).. and Safety (46%).

which means that 15% of the respondents from the NBIMS committee have a personal definition of BIM that is different than the committee’s formal definition. 6. No respondents chose the answers “Don’t Know” or “BIM is 3D CAD. the definition of BIM according to the NIBS-FIC NBIMS Committee received the most responses.” As shown in Figure 6. ACKNOWLEDGEMENT This study was partially supported by the National Institute of Building Sciences.• • • Respondent #13: More KPIs: Reduction in Claims. 5. The other response was. SUMMARY The summary question in this survey asked respondents which definition of BIM most closely matched their own. Respondent #17: While BIM a goal to strive for and is relevant to certain projects . and will also reduce the initial project cost. their affiliation could have also biased their results. the number of unknowns and field coordination efforts are reduced. More research needs to be conducted in order to corroborate the “BIMfavorable” results here. While this response is not necessarily incorrect. quantifying the impact of a BIM approach through real world construction case studies will offer a more compelling argument for BIM adoption by AEC firms. More sustainable structures Respondent #16: BIM will minimize change orders. Facility Information Council (NIBS-FIC.” received 30% or 15/50 responses.the fractured nature of the A/E/C (sic) industry means that it will be a long time before BIM has a significant overall effect on the industry 4. “BIM is a tool for visualizing and coordinating AEC work and avoiding errors and omissions. While the respondents are certainly knowledgeable about BIM because of the demographics shown herein and membership on the NBIMS listerv. Thus. Contractors will sharpen their pencils and will provide pricing per known factors. there is still some work to be done for the NBIMS Committee to educate and inform the AECO community. 2007 211 . Additionally.” with 70% or 35/50 respondents making this selection.) 7th International Conference on Construction Applications of Virtual Reality: October 22-23. CONCLUSION As the results suggest. the respondents felt that BIM is most likely to positively impact the construction KPIs of quality and on time completion. “BIM is an open standards based information repository for facilities’ lifecycles. Improved public outreach/agency coordination. it does not align with the NBIMS’ view of the definition. even within its own organization.

Action) (Col John Boyd. USAF (Ret. Decision.) ) FIG. 2007 212 . FIGURES FIG.7. Orientation. 2: Part II Responses about BIM’s impact on construction KPIs (raw data from zoomerang) 7th International Conference on Construction Applications of Virtual Reality: October 22-23. 1: “OODA Loop” (Observation.

FIG 3. 2007 213 . 4: Overall Favorable Responses when ranking KPIs with respect to impact on BIM (unweighted) 7th International Conference on Construction Applications of Virtual Reality: October 22-23. Part II Responses about BIM’s impact on construction KPIs (raw data from zoomerang) FIG.

6: Answers to definition of BIM Question 7th International Conference on Construction Applications of Virtual Reality: October 22-23. 2007 214 .FIG. 5: Overall Favorable Responses when ranking KPIs with respect to impact on BIM (weighted) 30% 70% BIM is a tool for visualizing and coordinating AEC work to avoid errors and omissions BIM is an open standards based information repository for the facility lifecycle FIG.

3 (2). 2007).S. Building Sciences A Publication of the National Institute of Building Sciences. “Management's Perception of Key Performance Indicators for Construction. Kosiak. “02 – GSA BIM Guide for Spatial Program Validation. The Executive Office of the President of the United States. CA.org/AboutBOMA/BOMAMissionStatement.gov/oae/publications/gcrs/04867. http://www. University of California. and Ahrens.Applying Information Modeling to Buildings.edu/online. (2006). Kam.gsa. (2007) “ http://www. Boyd.S. Constr. A.” Lawrence Berkeley National Laboratory.R. J. (2007).. Illinois. V. 142-151.pdf> (June 15. (1995). Lee.gsa. B. (2004) “Cost Analysis of Inadequate Interoperability in the U. E. (2004) “Virtual Building Environments (VBE) . (2005) “nD Modeling – A Driver or Enabler for Construction Improvement?” RICS Research Paper Series. C. MS.” <http://www.S. (2003). NBIMS website. “GSA Mandates Building Information Modeling” Federal Computer Week.F. “BOMA Mission Statement.boma. Stipes Publishing. Issa. Bazjanac.htm> (June 25. Kennett.pdf (July 20.” <http://www.wes. BIM Standard. Office of Management and Budget. https://tsc. 30.A.” Military Geospatial Edition.” USACE. US Department of Commerce. And Mgmt. (2007) “Information Warfare. Vol. R.gov/bim> (June 15. ER 1110-1-12. U. Technology Online Hardy.publications/WP103.mil. D. Case Studies and Implementation Suggestions. (WES) Vicksburg. (2006).J. M.nsf/PDAWebToday/09CFE365DC995AA185257233003D8B33?OpenDocument > (June 15.0. National Institute of Standards and Technology. S.” Stanford Center for Integrated Facility Engineering <http://cife.nist. Cullis. 2007). BOMA International (2007). J. CADD/GIS Technology Center. 7th International Conference on Construction Applications of Virtual Reality: October 22-23. Waterways Experimentation Station.A.html> (July 20.org/bim/pdfs/NBIMS_Awareness_Handout.12manage. Center Headlines. (2004) Office of the Deputy Undersecretary of Defense (Installations & Environment) Business Enterprise Integration Directorate (2006) “Real Property Acceptance Requirements Document” Draft V5. March.gsa..S. “3D-4D Building Information Modeling” GSA webpage. Berkeley. <http://www. REFERENCES Adrian.8.army. (2007).gov/gsa/cm_attachments/GSA_DOCUMENT/GSA_BIM_02_Appendix_v09_R2C-a3l_0Z5RDZ-i34K-pR. 2007). 129(2).. R. Explanation of OODA Loop of John Boyd.pdf> (June 14. “Engineering and Design – Quality Management. 2007). Kam. 2007) Cox. Construction Productivity: Measurement and Improvement. C.facilityinformationcouncil.” GSA Building Information Modeling Guide Series. (NIST GCR 04-867. Center for Strategic and Budgetary Assessments.pdf. 5 (6).” J. Champaign. Capital Facilities Industry”. USACE (U. Engrg. M.stanford. 2007). 2007). M. and Fischer. (2006). < http://w3. (2005) “Geospatial Mandates Pave Way for DISDI. Kunz. (2004) Analysis of the FY 2005 Defense Budget Request. <http://www. J.gov/ClipsMgt. August 2004).bfrl. 2007 215 . Army Coprs of Engineers). “Virtual Design and Construction: Themes. (2006) New NIBS Group to Create U.com/methods_boyd_ooda_loop.