Journal of Materials Processing Technology 120 (2002) 458±465

An optical sensing system for seam tracking and weld pool control in gas metal arc welding of steel pipe
K.-Y. Baea,Ã, T.-H. Leea, K.-C. Ahnb

Department of Industrial Automation Engineering, Chinju National University, Chinju, South Korea b Department of Mechanical Design Engineering, Chinju National University, Chinju, South Korea Accepted 8 November 2001

Abstract A visual sensing system was developed for automatic gas metal arc welding (GMAW) of the root pass of steel pipe. The system consisted of a vision sensor that consisted of a charge-coupled device (CCD) camera and lenses, a frame grabber, image processing algorithms, and a computer controller. A specially designed ®ve-axis manipulator was used to position the welding torch and to provide the vision sensor with automatic access to view the welding position. During the root pass welding, an image of the weld pool and its vicinity was captured using the camera without interference of the intensive arc light by viewing at the instance of a short-circuit of the welding power. The captured image was then processed to recognize the weld pool shape. For seam tracking, the manipulator was used to adjust the torch position based upon the pool image to the groove center. The measured gap size was used to determine the appropriate welding conditions to obtain sound penetration. The welding speed was chosen using fuzzy logic with the knowledge of a skilled welder and measured gap. The automatic welding equipment demonstrated that both welding conditions and torch position could be appropriately controlled to obtain a sound weldment and a good seam tracking capability. # 2002 Elsevier Science B.V. All rights reserved.
Keywords: Visual sensing; Weld pool control; Seam tracking; Root pass welding

1. Introduction As demands on pipe structures for ocean exploration, steel towers, water and oil pipe lines, etc. are increasing, the requirements for greater productivity and accuracy in manufacturing these structures are also increasing. For joining one pipe to the other, girth arc welding, which consists of the root pass and the ®ll pass procedures, is prevalently performed. The welding process for manufacturing pipe structures can be accomplished with automatic equipments that have been partially mechanized only for ®ll pass welding and operated by a welding operator. Manufacturing tolerances at the preparation stage cause unavoidable amounts of distortion, eccentricity, and mismatch that produce a variation in gap size and also in seam location. The irregularities in the root shape of a groove make it impossible to set welding conditions and torch location to be constant during welding. Therefore, the root pass procedure has been very dif®cult to automate and is generally performed by a skilled welder. To achieve full automation of steel pipe welding, the root pass weld should ®rst be automated. The automation of
* Corresponding author. E-mail address: (K.-Y. Bae).

the root pass requires the ability to adjust torch position and welding conditions, dealing with a variety of root states. There have been many studies on visual sensing techniques for observing weld pool images during welding. A weld pool control technique with the vision sensor system consisting of a charge-coupled device (CCD) camera and a long wave pass ®lter for lowering the arc intensity in bead-on-plate welding has been investigated [1,2]. Control of the weld pool width in pulsed metal inert gas (MIG) welding was also studied using measurement of the weld pool with a vision sensor during the base current period of the pulse [3,4]. Moreover, the relationship between the observed bead width and the penetration depth in gas metal arc welding (GMAW) process has also been investigated [5]. However, because most of these studies observed the weld pool with a top-side view, only the width of the weld pool could be investigated, and information on penetration through the joint could not be determined. Although an automatic pipe welder with a vision sensor that could give a front view of the weld has previously been reported with GMAW [6,7], only a limited information is available on this equipment. In this study, experimental results from testing of a seam tracking and weld pool control system for the root pass welding of steel pipe is presented. The system consisted of a

0924-0136/02/$ ± see front matter # 2002 Elsevier Science B.V. All rights reserved. PII: S 0 9 2 4 - 0 1 3 6 ( 0 1 ) 0 1 2 1 6 - X

K.-Y. Bae et al. / Journal of Materials Processing Technology 120 (2002) 458±465


®ve-axis pipe welding manipulator with its controller, the hardware logic for detecting the short-circuit and the visual sensing system. The manipulator was constructed to permit the welding torch and the vision sensor to have automatic access to a welding position with predetermined con®gurations when the pipe to be welded was selected. The visual sensing system was composed of a CCD camera, lenses and ®lters, a frame grabber and image processing algorithms. When low current GMAW, that is used for root pass welding, is applied, the intensive arc light momentarily extinguishes periodically with a short-circuit. In this study, the short-circuit phenomena was utilized in order to acquire an image of the weld pool and its vicinity using the vision sensor. The sensor was located almost tangent to the surface at welding position. The acquired image was then processed to detect the shape and size of the weld pool, from which the relative distance between the torch position and the weld seam was determined and the gap size of groove measured. The information made it possible for the system controller to determine a new welding position, permitting the torch to advance to that position, apply new welding conditions all in real time. For the determination of welding conditions, the relation between the gap size and the welding speed was chosen using fuzzy logic and incorporating the knowledge of a skilled expert. A series of experiments was conducted to evaluate the performance of the seam tracking and weld pool control of the system for the root pass welding of a steel pipe. 2. Pipe welding system 2.1. Welding manipulator The pipe welding system developed in this study consists of a positioner for turning the pipe, a ®ve-axis welding

manipulator, a welding power source, a control panel and a control computer. Fig. 1 shows a schematic drawing of the pipe welding manipulator. The base table of the manipulator is composed of three Cartesian axes, where the X- and Y-axes cooperate to travel the curved racks grasping the welding torch and the CCD camera to a predetermined welding position when a pipe diameter is given. The Z-axis is used for seam tracking. By rotating one curved rack grasping the torch holder, the torch can rotate around thewelding position to a suitablewelding con®guration angle (A-axis). Similarly, the other curved rack can also rotate the camera to another con®guration (B-axis) corresponding with the torch con®guration to observe the weld pool. The manipulator ®nds the welding position (x, y) using the following coordinate transformation with given information on a pipe such as diameter (D) and welding start position (c). x ˆ L1 À L2 À 1 D cos…c†; 2 y ˆ H À 1 D sin…c† 2 (1) where L1 is the distance from the origin of X-axis to the center of the pipe along the X-direction, L2 the distance from the x-position to the center of the curved rack along Xdirection, and H is the distance from the origin of the Y-axis to the center of the pipe along the Y-direction. 2.2. Controller Fig. 2 shows a block diagram of the controller of the pipe welding system that is based on an IBM compatible personal computer. The computer acquires images of the weld pool through the CCD camera and the frame grabber, sets the welding current and voltage through the digital±analog (DA) converter board, monitors the welding current through the analog±digital (AD) converter board, commands the arc to be on or off, controls the wire, checks the gas, etc. through the digital input and output (DIO) board, sets torch position through a motion control board.

Fig. 1. Pipe welding system.


K.-Y. Bae et al. / Journal of Materials Processing Technology 120 (2002) 458±465

was additionally equipped with a Pyrex window, an ultrared ®lter, and a neural density ®lter. The vision sensor was set to be located at a 300 mm distance away from the weld pool along the tangential direction on the pipe surface to observe the weld pool with a front view. Then, with the camera which had a size of 2/3 in. CCD array where 768 pixels were in column and 493 pixels in row, the sensor resulted in a resolution of 0.04557 mm per pixel in the transverse direction and 0.05355 mm per pixel in the longitudinal direction in viewing the ®eld of view of a 41 mm …vertical†  30:8 mm (horizontal) of rectangular area around the weld pool. 3.2. Short-circuit detection The root pass welding necessarily uses low current and low voltage, so there would be periodic contacts between the wire and the substrate, showing the short-circuit phenomena in which the arc extinguishes and the current abruptly increases [9]. Thus, it was considered to be possible to observe only the weld pool during the period of the arc extinction without any interference of the arc. With this basic concept of acquiring the weld pool image during the short-circuit period, a short-circuit detection was constructed to detect the uprising edge of the current pulse. The current signal was measured using a 10À5 O shunt resistor connected in series with the power cable, then ampli®ed by 100 times to be the raw current signal of 0±5 V. Fig. 3 shows a raw current signal having periodic analog pulses and a digital pulse train of the short-detection signal extracted from the raw analog signal. The short-detection signal was used to trigger the camera to take an image of the weld pool and also to trigger the frame grabber to acquire an image of the weld pool in synchronization with the camera. 3.3. Pool image searching The image searching algorithm to detect boundary of the weld pool was based on the technique that searched the

Fig. 2. Control system of torch position and weld pool.

The computer also controls the rotational speed of the pipe positioner by voltage variation (0±10 V) through the DA board. In addition to the computer, the controller has the short-circuit detection board that generates the triggering signal for an imaging of the camera as soon as it detects a short-circuit between the wire and the pipe. 3. Weld pool monitoring 3.1. Vision sensor A vision sensor for monitoring the weld pool requires a special optical system to reduce the intensity of the arc [8]. In this study, for this purpose, the vision sensor which basically consisted of a CCD camera and a close-up ring,

Fig. 3. Short detection from raw current signal: (a) current signal; (b) short signal.

K.-Y. Bae et al. / Journal of Materials Processing Technology 120 (2002) 458±465


pixels along the counterclockwise (CCW) direction around the presently detected position for a next boundary position [3]. In the proposed algorithm in this study, a next searching direction for the present boundary position was ®rstly determined to be located at 458 in the CCW direction from the presently searched direction. The angle would be then increased by a step of 458 until to ®nd the ®rst pixel having a gray level above the threshold, and the found pixel became a boundary pixel of the weld pool. This algorithm can be expressed as follows:  H    u cos q sin q u ˆ (2) vH Àsin q cos q v where uH , vH are the components of searching direction vector to find a next boundary pixel, q ˆ np=4 (n ˆ 1; . . . ; 7), and u and v are the components of present searching direction vector derived from the following equation:       u i i ˆ B À P v jB jP Here i and j are the pixel coordinates, subscript `P' the present boundary pixel, and subscript `B' is the previous boundary pixel. 4. Seam tracking and weld pool control 4.1. Seam tracking As shown in Fig. 4, an image of the weld pool necessarily contains the shadow of the wire indicating current position of the torch, and the center of the weld pool presenting the center of groove. In order to search the center of the wire with the features from an image of the weld pool, the left and the right extreme columns of the weld pool were at ®rst identi®ed, respectively. From the mid column between the left extreme column and the groove center to the mid one between the groove center and the right extreme column, the gray level of each pixel located between a pool start pixel

and the pool center in the vertical direction for each column were then added. And then, the differentiation for the summation along the horizontal direction was carried out to obtain minimal and maximal values, and their column positions, respectively. Consequently, the center of the torch could be then determined to be located at the center of the two columns. An offset of the welding wire from the center of a groove makes shapes of the weld pool at both sides around the wire to be of asymmetry. In this study, for tracking a groove center, i.e. a weld seam, the welding torch was controlled to move to the direction along which the asymmetry of the weld pool around the wire diminished. 4.2. Weld pool control The control technique of a skilled welder was implemented as the control algorithm of the weld pool to obtain a sound penetration in welding of the pipes which had a variation of gap. For the purpose of this, the gap size should be ®rstly measured from an image of the weld pool as done by a welder in a manual welding. In this study, widths of a partial region of the weld pool from the lowest row to 10 pixels above corresponding a 0.5 mm height were averaged, and then the result was de®ned to be a measured gap size. Meanwhile, because the weld pool should be appropriately penetrated through the gap of pipe to obtain a sound penetration, there may be a close relation between the real gap and the measured gap for the sound weldment. For identifying the relationship, the temperature distributions around the groove were considered to be of valuable information and were therefore predicted with a welding current of 180 A, a voltage of 20 V, and a speed of 2.1 mm/s by using the FEM program developed for a previous study [10]. Fig. 5 shows the predicted temperature distribution of a half section around the center of groove. The region heated above 1500 8C can be considered to be a part of the weld pool, and it penetrates through the non-melted zones to be a back bead. The penetrated region below the bottom of the gap would be observed as a part of the bright weld pool by the camera, and the width of the region is considered to be a measured gap in this study. With the results, the relation between the actual

Fig. 4. Features extraction in weld pool image.

Fig. 5. Temperature distributions at sections in circumferential direction under arc for pipes with different gap sizes.


K.-Y. Bae et al. / Journal of Materials Processing Technology 120 (2002) 458±465

Table 1 Suggested welding conditions for producing sound penetrations Gap size (mm) 2.0 2.5 3.0 3.5 4.0 Speed (mm/s) 2.4 2.2 2.1 1.8 1.7 Current (A) 190 185 180 175 170 Voltage (V) 20 20 20 19 19

3. If gap is big …e…k† < 0† and becomes smaller …De…k† > 0†, then set lower speed …u < 0†. 4. If gap is small …e…k† > 0† and becomes bigger …De…k† < 0†, then set higher speed …u > 0†. Based on the knowledge, a fuzzy rule base was constructed. Each fuzzy subset for an error was set to correspond to an error of 0, Æ0.5 and Æ1.0 mm, respectively; each fuzzy subset for the change of an error to correspond to the change of an error of 0, Æ0.125 and Æ0.25 mm, respectively; and each fuzzy subset for the change of a speed to correspond to the change of a speed of 0, Æ0.2 and Æ0.4 mm/s, respectively. Then, the fuzzy inference to relate the process input with the two input variables was performed with the min± max algorithm [11]. The process input that is a change of speed could be then obtained by defuzzifying the fuzzy value using the center of gravity method. Thus, the welding speed would be updated for the next control step, whereas the current and the voltage corresponding to the newly determined speed were obtained by the linear interpolation of the relation shown in Table 1. The processing of an image of the weld pool and the determination of a control variable at every control step could be carried out within a time period of 0.3 s, and it was then set to be the control cycle time for the visual sensing system. 5. Experiment Welding experiments were conducted with GMAW for the proposed pipe welding system to evaluate the feasibility in automatic control of the root pass welding process of a mild steel pipe workpiece that had a diameter of 192 mm and a thickness of 15 mm and was prepared as a butt joint with a single V-groove of 608. To make a weld joint, two pipes, each of which had a length of 150 mm and was prepared at one end, were tacked at four quadratic positions with a 10 mm length of welding for each position. An SM70 welding wire having a diameter of 1.2 mm and a shielding gas of 100% CO2 were used for the welding. To begin with,

gap and the measured gap can be therefore estimated. The relation was also con®rmed and quanti®ed through a series of welding experiments, and the result will be shown in Fig. 9. A real gap can be therefore obtained from the measured gap by using the relationship. In this study, to construct the knowledge-based model, to begin with, proper welding conditions producing the sound penetration for a given gap size were determined through actual pipe welding experiments performed with assistance of a skilled welder for each gap size of 2.0, 2.5, 3.0, 3.5, and 4.0 mm which could stand for available gaps during welding of the pipes speci®ed in this study. The results of the experiments are shown in Table 1. This was stored in the computer as a database for this speci®ed welding, and then used as basic information for the knowledge base to control the weld pool for a gap variation during actual pipe welding. For further developing the relation and thus realizing the heuristic rules obtained from experiences of skilled worker corresponding to a gap variation in real time, especially for the relation between gap size and welding speed, the fuzzy logic controller as shown in Fig. 6 was proposed. The proposed fuzzy logic took two variables to be fuzzi®ed. One was an error (e(k)), which was the difference of a gap (G(k)) at the concerned time step (k) from the reference gap …G…k À 1†† at the previous one …k À 1†, and the other was the change of an error, de…k†=dt …De…k††. The following knowledge was utilized to determine a new welding speed. 1. If gap is small …e…k† > 0† and becomes smaller …De…k† > 0†, then set much higher speed …u @ 0†. 2. If gap is big …e…k† < 0† and becomes bigger …De…k† < 0†, then set much lower speed …u ! 0†.

Fig. 6. Block diagram of fuzzy controller.

K.-Y. Bae et al. / Journal of Materials Processing Technology 120 (2002) 458±465


the torch was moved to a welding start point located on the joint at 458 around the center of the pipe toward CCW from the horizon. The torch angle was then set to be at 508 toward CCW from the horizon, and the camera angle to be at À158. Finally, the manipulator was ®xed not to move all but in the Z-direction which was dedicated for a seam tracking. The direction of the pipe rotation was set to be the one so that the torch relatively seemed to travel downward along a ®xed slope. 5.1. Seam tracking For simulating an actual weld seam, an initial linear offset from 0 to 2 or À2 mm was preset from the straight seam along a welding distance of 140 mm located between two adjacent tacks. The welding experiment was then carried out for each specimen to verify a capability of the proposed system in the seam tracking with a visual sensing of the weld pool. 5.2. Weld pool control In order to obtain the relation between measured and actual gaps, welding experiments were carried out to measure the gap sizes with the visual sensor during welding of each pipe having an actual gap of 2.0, 3.0 and 4.5 mm, respectively, with the conditions for a gap of 3.0 mm. To verify the capability of the proposed sensing system in adjusting welding conditions to control the weld for obtaining a sound penetration in real time, corresponding to a gap variation, the root pass welding experiment was performed for each pipe having a gap of 2.5 and 4.0 mm, respectively, with the welding conditions for a standard gap of 3.0 mm. In addition, for a comparison of weldability between with control and without control of the weld pool, another experiment was performed without the pool control for the pipe having a gap of 3.5 mm. 6. Results and discussion During the experiment, the pipe welding system with the proposed manipulator exactly performed the decided sequence for making con®gurations of the torch and the camera on the surface of a pipe, which shows a very ¯exible aspect of the system and superiority to its human competitor. With the short-detection logic, the image of the weld pool could be acquired during the period of a short-circuit when the arc extinguished. Fig. 7 shows a captured image of the weld pool, presenting clearly the boundary between the weld pool and the background and also the shadow image of the wire. At every observation time, based on the features from the image of the weld pool, the relative distance between the two centers of the wire and the weld pool was calculated and diminished by using the seam tracking algorithm. The result

Fig. 7. Weld pool image.

of the test of a weld seam tracking for the root pass was shown in Fig. 8. In this ®gure, each seam tracking for an offset error of 2 or À2 mm at the end of a 140 mm weld seam along the circumferential direction, which is presented with a change degree, shows a favorable result when it is compared with the desired seam. Accordingly, the system is considered to have the capability to track a weld seam which has up to 18 offset error. In fact, under the conditions that the image processing time is 0.3 s and the welding speed for a 3.0 mm gap is 2.1 mm/s, a correction for the torch position is performed at every 0.63 mm welding, and the ®eld of view of the camera along the transverse direction is as large as

Fig. 8. Torch trace during seam tracking for offsets in weld joint location: (a) negative offset; (b) positive offset.


K.-Y. Bae et al. / Journal of Materials Processing Technology 120 (2002) 458±465

Fig. 9. Comparison between actual and measured gap sizes.

30 mm, any kind of the actual seam can be therefore theoretically traced. However, to cope with any instantaneous disturbance, the maximum correction data for the weld seam were limited to 0.5 mm in each control step. Fig. 9 shows the comparison between measured gaps obtained from the images of the weld pool and actual gaps preset to the preparation of the groove. In this ®gure, as the preset actual gap goes small, the difference from the measured gap becomes large. The difference is considered to be caused by the fact that the smaller an actual gap size is, the more is a root edge melted, as previously predicted with the FEM analysis in Section 4.2. On the other hand, for a larger actual gap, the same size of gap was measured. For the larger gap, the weld metal penetrated between the two prepared edges which had a small melted region and therefore added a small shinny region, which resulted in only a little difference from the actual gap. When the root pass welding was performed with the conditions for a 3.0 mm gap on the pipe which had the gap changing from 2.0 to 2.5 mm over a whole welding distance of 140 mm, the welding conditions could be appropriately controlled with the algorithm for the weld pool control in real time along the welding length as shown in Fig. 10 to obtain the weld pool for producing a sound penetration through the actual gap. Especially, as soon as the welding started, the initial speed and current for a 3.0 mm gap were rapidly changed at the same time to be the ones for a smaller gap. When the root pass welding with initial conditions for a 3.0 mm gap was applied for the pipe having an actual gap of 4.0 mm, the welding conditions were changed as shown in Fig. 11. Both the initial speed and current were rapidly decreased to be suitable for the larger new gap. Welding conditions could be also sustained fairly well for the gap along the welding distance. According to these results, it could be assured for the proposed visual sensing system to be very feasible to control the weld pool for obtaining a sound penetration through the root gap even when there was a variation of gap size.

Fig. 10. Response of welding conditions to a gap size of 2.0±2.5 mm: (a) welding speed; (b) welding current.

Fig. 11. Response of welding conditions to a gap of 4.0 mm: (a) welding speed; (b) welding current.

Fig. 12 shows two pictures illustrating each result of the welding experiment with or without the weld pool control, respectively. Fig. 12(a) shows the result of the welding for a gap of 4.0 mm with the weld pool control, and it presents a sound weldment. On the contrary, Fig. 12(b) shows the result of the welding for a gap of 3.5 mm without the weld pool control. In the case when the gap was 3.5 mm, although there

K.-Y. Bae et al. / Journal of Materials Processing Technology 120 (2002) 458±465


Fig. 12. Result of weld pool control: (a) with control; (b) without control.

was only a small increment in the gap size compared with the 3.0 mm gap for which initial welding conditions were decided, no good weldment was produced by occurrence of a penetration of the wire into the gap. Accordingly, these results meant that the weld pool control should be always required to deal with any gap variation in the root pass welding of a pipe. 7. Conclusion A visual sensing system was proposed for the seam tracking and the pool control to automate the root pass welding of steel pipe. As the result of the investigation of the feasibility and weldability through a series of welding experiments for the proposed system and its algorithms, following conclusions could be derived: 1. The manipulator with the torch handling mechanism worked exactly in locating the torch at a start position of welding for any specified diameter of pipe. 2. The short-detection logic captured very well the period of an arc extinction during welding process and correctly issued a trigger pulse for the camera to acquire an image of the weld pool without interference of the arc. 3. The edge-detection technique could quickly and easily find the boundary of the weld pool from its background, which shortened the processing time with considerable exactness in searching an image of the weld pool. 4. The position of wire, the center of the weld pool, and the gap size of the weld pool could be calculated with the features extracted from an image of the weld pool. 5. The seam tracking logic for sustaining a symmetry of the weld pool around the wire was appropriate enough to obtain a suitable result in the seam tracking.

6. The fuzzy logic of which rule was based on the knowledge of a welding expert could determine in real time an optimal welding speed corresponding to a gap variation. References
[1] D.-C. Kim, S.-H. Rhee, K.-W. Um, A study of weld pool width control in gas tungsten arc welding using the digital image processing, J. KSME(A) 20 (9) (1996) 2760±2769. [2] Stenning Vision Ltd., N.N. Reference Manual, Stenning Vision Ltd., Canada. [3] K. Ohshima, M. Morita, K. Fujii, M. Yamamoto, T. Kubota, Sensing and digital control of weld pool in pulsed MIG welding, Trans. Jpn. Weld. Soc. 23 (1) (1992) 36±42. [4] K. Ohshima, M. Yamamoto, T. Tanshi, S. Yamane, Digital control of torch position and weld pool in MIG welding using image processing device, IEEE Trans. Ind. Appl. 28 (3) (1992) 607±612. [5] R. Kovacevic, Y.M. Zhang, L. Li, Monitoring of weld joint penetration based on weld pool geometrical appearance, Weld. J. 74 (10) (1996) 317s±329s. [6] Tecnar Automation Ltd., N.N. Reference Manual, Rotoweld 2000 Pipe Welding Workstation, Tecnar Automation Ltd., Canada. [7] F. Nadeau, P. Fafard, G. Patenaude, J. Tremblay, Method and apparatus for controlling root pass weld penetration in open butt joints, US Patent 4,733,051 (1988). [8] H. Wata, Y. Manabe, Y. Hiromoto, H. Yoritaka, Group-control system for narrow-gap MIG welding, in: H. Namura (Ed.), Sensors and Control System in Arc Welding, Chapman & Hall, London, 1994, pp. 176±181. [9] J. Weston, Advanced Welding Systems, Vol. 2, IFS Ltd., UK (Chapter 6). [10] K.-Y. Bae, S.-J. Na, D.-H. Park, A study of mechanical stress relief treatment of residual stresses for one-pass submerged arc welding of V-grooved mild steel plate, Proc. Inst. Mech. Eng., Part B: J. Eng. Manufac. 208 (1994) 217±227. [11] J.-W. Kim, S.-J. Na, A self-organizing fuzzy control approach to arc sensor for weld joint tracking in gas metal arc welding of butt joints, Weld. J. 71 (2) (1993) 60s±66s.