You are on page 1of 2

An AR-System for Visualizing Deviations between the Planned and Real Shape

of a Car Body

Martin Wagner1,2 , Gudrun Klinker1,2 , Mustafa Isik2 ,
Kay Kindermann3 , Jörn Trilk3 and Henning Müller-Vogelmann3
1 peyclonintegration 2 Technische Universität München 3 BMW AG
Moos 2 Fakultät für Informatik Forschungs und Innovationszentrum (FIZ)
Kirchseeon, Germany Garching b. München, Germany München, Germany

Abstract tion which resulted in building an Intelligent Welding Gun,
a tool using Augmented Reality to weld large numbers of
An essential part of experimental vehicle construction con- studs with high precision into a car frame [2].
sists of obtaining and analyzing measurements describing
deviations between an actual car and the CAD-model. This 2 Requirements
paper presents ongoing work towards presenting such mea-
surement data directly on a car frame using Augmented Re-
ality. In a first prototype, the augmentations can be seen by During a number of conversations with the end users of the
a large number of individuals in HMDs or jointly in video- system, we established the following key requirements:
see-through mode on a monitor. A moderator can select Joint presentation for a moderator and many spectators:
subsets of measurement data with a pointing device. A moderator must be able to interact with the system,
changing viewing styles and selecting measurements.
Other spectators are passive, wearing a head-mounted
1 Introduction display (HMD) or jointly following a visualization
on a screen, as seen by a virtual spectator, i.e. a
In the automotive industry, experimental vehicle construc- video-based augmentation from a mobile camera
tion and technical integration includes building a limited which the moderator holds in his hand.
number of prototypes of a new model in order to validate
manufacturing processes and product functionalities. As Presentation: Presentation styles that car builders are ac-
part of the evaluation process, first prototypes are com- customed to from their current presentation practice
pared with the CAD-model to assure that deviations be- must also be available in the AR-system. Furthermore,
tween the built car and the CAD-model are within specified not only deviation measurements but also photogram-
tolerances. When significant deviations are detected, they metric reference points must be shown.
are related back to the manufacturing process, resulting in Interaction: The moderator must be able to select subsets
changes of machine settings to ensure that subquent cars are of measurements via 3D gestures while viewing the
built within tolerance levels. data. Other interactions can be provided via a tradi-
The process of analyzing significant deviations and of tional WIMP interface.
communicating them to the people building the cars has
proven to be challenging because it is not easy to visualize Integration into the work environment: The AR-system
systematic properties of large numbers of small deviations, must be robust to operate within an everyday automo-
such as a systematic misalignment (wrong orientation) of tive work environment. Furthermore, the software sys-
one component with respect to another one. Current ap- tem needs to be integrated with the database holding
proaches have relied on 3D drawings, accompanied by ta- the measurement data. Eventually, HMDs should op-
bles of measurement data in a document and on slides. erate wirelessly.
This paper presents ongoing work towards presenting
measurement data directly on the car frame using Aug- Performance: The system must provide close to real-time
mented Reality. It is a continuation of a previous coopera- performance.

Server Client posed on the right back side of a car as a video-based aug-
mentation. In the selected visualization style, each mea-
Tracking surement is presented as a small coordinate system to in-
dicate deviations separately with respect to each major car
UDP axis. The visualizations are significantly enlarged in order
Calibration to be visible. Other visualization styles show directional
vectors and textual annotations.
Visualization The current prototype is a proof-of-concept. Next
TCP steps involve integrating the system with the measurement
Scene database, providing more complex visualization schemes
Graph that also include time series of measurements, handling
Database occlusions, and providing high precision calibrations of
HMDs and the screen-based visualization, both for close
and far distances.
Figure 1. System Architecture

3 System Architecture

For a first prototype, the subset of the most critical require-
ments was implemented to provide a proof-of-concept.

Physical setup. The AR-based presentation system is set
up around the car. In the current prototype, computers are
placed on a mobile cart next to the car. Attached are an
HMD, a video camera and a trackable 3D interaction device
which the moderator can use to point at positions on the
car and to select spots or areas by pressing a button and
dragging the pointer. We use an outside-in optical tracking
system [1] to track the HMD, camera, pointer and car [2].

Software system. The software system consists of a
server and an arbitrary number of presentation clients. The Figure 2. Enlarged deviation measurements
server accesses the measurement data, generates visualiza- superimposed on a real car
tion descriptions according to the moderator’s current selec-
tion and viewing preference, and forwards them to all pre-
sentation clients. It also continually receives current pose
information for all tracked objects from the external track- Acknowledgments
ing system and forwards it to the clients after a calibration
step. We are greatful to Friedrich Wolf, Klemens Obermeier and
There is one client for every display. For the screen- Georg Klinker for many discussions about visualizing de-
based visualization, a camera is attached to the client and viation data for car builders and planning the AR-system.
a video-based augmentation style is used. For HMDs, Stephan Huber helped with the setup at BMW and provided
augmentations are generated in optical-see-through mode. us with the photo showing the system in operation.
Each client continually receives pose data for its associated
display from the server. It also sporadically receives new vi- References
sualization descriptions when the moderator alters the view-
ing style or the selected data set. The client renders the vi- [1] ART G MB H, ARTrack1. On-line documentation at
sualization from the current viewing pose in real-time., 2003.
J. S TILLA , J. T RILK , and H. NAJAFI, The Intelligent
4 Results and Future Work Welding Gun: Augmented Reality for Experimental Vehicle
Construction, in Virtual and Augmented Reality Applications
A first system prototype has been installed and tested at in Manufacturing, Chapter 17, S. Ong and A. Nee, eds.,
BMW. Figure 2 shows deviation measurements superim- Springer Verlag, 2003.