An AR-System for Visualizing Deviations between the Planned and Real Shape of a Car Body

Martin Wagner1,2 , Gudrun Klinker1,2 , Mustafa Isik2 , Kay Kindermann3 , J¨ rn Trilk3 and Henning M¨ ller-Vogelmann3 o u
1 peyclon

integration Moos 2 Kirchseeon, Germany

2 Technische

Universit¨ t M¨ nchen a u Fakult¨ t f¨ r Informatik a u Garching b. M¨ nchen, Germany u


AG Forschungs und Innovationszentrum (FIZ) M¨ nchen, Germany u

An essential part of experimental vehicle construction consists of obtaining and analyzing measurements describing deviations between an actual car and the CAD-model. This paper presents ongoing work towards presenting such measurement data directly on a car frame using Augmented Reality. In a first prototype, the augmentations can be seen by a large number of individuals in HMDs or jointly in videosee-through mode on a monitor. A moderator can select subsets of measurement data with a pointing device.

tion which resulted in building an Intelligent Welding Gun, a tool using Augmented Reality to weld large numbers of studs with high precision into a car frame [2].



During a number of conversations with the end users of the system, we established the following key requirements: Joint presentation for a moderator and many spectators: A moderator must be able to interact with the system, changing viewing styles and selecting measurements. Other spectators are passive, wearing a head-mounted display (HMD) or jointly following a visualization on a screen, as seen by a virtual spectator, i.e. a video-based augmentation from a mobile camera which the moderator holds in his hand. Presentation: Presentation styles that car builders are accustomed to from their current presentation practice must also be available in the AR-system. Furthermore, not only deviation measurements but also photogrammetric reference points must be shown. Interaction: The moderator must be able to select subsets of measurements via 3D gestures while viewing the data. Other interactions can be provided via a traditional WIMP interface. Integration into the work environment: The AR-system must be robust to operate within an everyday automotive work environment. Furthermore, the software system needs to be integrated with the database holding the measurement data. Eventually, HMDs should operate wirelessly. Performance: The system must provide close to real-time performance. 1



In the automotive industry, experimental vehicle construction and technical integration includes building a limited number of prototypes of a new model in order to validate manufacturing processes and product functionalities. As part of the evaluation process, first prototypes are compared with the CAD-model to assure that deviations between the built car and the CAD-model are within specified tolerances. When significant deviations are detected, they are related back to the manufacturing process, resulting in changes of machine settings to ensure that subquent cars are built within tolerance levels. The process of analyzing significant deviations and of communicating them to the people building the cars has proven to be challenging because it is not easy to visualize systematic properties of large numbers of small deviations, such as a systematic misalignment (wrong orientation) of one component with respect to another one. Current approaches have relied on 3D drawings, accompanied by tables of measurement data in a document and on slides. This paper presents ongoing work towards presenting measurement data directly on the car frame using Augmented Reality. It is a continuation of a previous coopera-

Tracking UDP


Calibration Visualization




Scene Graph


Figure 1. System Architecture

posed on the right back side of a car as a video-based augmentation. In the selected visualization style, each measurement is presented as a small coordinate system to indicate deviations separately with respect to each major car axis. The visualizations are significantly enlarged in order to be visible. Other visualization styles show directional vectors and textual annotations. The current prototype is a proof-of-concept. Next steps involve integrating the system with the measurement database, providing more complex visualization schemes that also include time series of measurements, handling occlusions, and providing high precision calibrations of HMDs and the screen-based visualization, both for close and far distances.


System Architecture

For a first prototype, the subset of the most critical requirements was implemented to provide a proof-of-concept. Physical setup. The AR-based presentation system is set up around the car. In the current prototype, computers are placed on a mobile cart next to the car. Attached are an HMD, a video camera and a trackable 3D interaction device which the moderator can use to point at positions on the car and to select spots or areas by pressing a button and dragging the pointer. We use an outside-in optical tracking system [1] to track the HMD, camera, pointer and car [2]. Software system. The software system consists of a server and an arbitrary number of presentation clients. The server accesses the measurement data, generates visualization descriptions according to the moderator’s current selection and viewing preference, and forwards them to all presentation clients. It also continually receives current pose information for all tracked objects from the external tracking system and forwards it to the clients after a calibration step. There is one client for every display. For the screenbased visualization, a camera is attached to the client and a video-based augmentation style is used. For HMDs, augmentations are generated in optical-see-through mode. Each client continually receives pose data for its associated display from the server. It also sporadically receives new visualization descriptions when the moderator alters the viewing style or the selected data set. The client renders the visualization from the current viewing pose in real-time.

Figure 2. Enlarged deviation measurements superimposed on a real car

We are greatful to Friedrich Wolf, Klemens Obermeier and Georg Klinker for many discussions about visualizing deviation data for car builders and planning the AR-system. Stephan Huber helped with the setup at BMW and provided us with the photo showing the system in operation.

[1] ART G MB H, ARTrack1. On-line documentation at, 2003. [2] F. E CHTLER , F. S TURM , K. K INDERMANN , G. K LINKER , J. S TILLA , J. T RILK , and H. NAJAFI, The Intelligent Welding Gun: Augmented Reality for Experimental Vehicle Construction, in Virtual and Augmented Reality Applications in Manufacturing, Chapter 17, S. Ong and A. Nee, eds., Springer Verlag, 2003.


Results and Future Work

A first system prototype has been installed and tested at BMW. Figure 2 shows deviation measurements superim2

Sign up to vote on this title
UsefulNot useful

Master Your Semester with Scribd & The New York Times

Special offer for students: Only $4.99/month.

Master Your Semester with a Special Offer from Scribd & The New York Times

Cancel anytime.