1640

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 59, NO. 3, MARCH 2012

Real-Time Visual Tracking and Measurement to Control Fast Dynamics of Overhead Cranes
Cheng-Yuan Chang and Handra Wijaya Lie
AbstractApplying visual tracking technology to dynamic systems is important to many industrial applications. However, image sensing is difcult to implement in a fast dynamic plant without high-speed computational resources. This work presents a simple yet effective method of image processing to capture the dynamic movement of an overhead crane, enabling it to be controlled in real time. Visual tracking based on color histograms involves comparison of the color in a model image with the color in a sequence of images to track a dynamic object. The best match is identied using Bhattacharyya similarity measurement. Upon tracking, the sensing data are sent to an adaptive fuzzy sliding mode controller (AFSMC). The advantages of this approach include robustness and the model-free properties of the sliding mode and the fuzzy logic controllers; adaptable slopes of the sliding surface of AFSMC are also presented to improve control performance. Index TermsAdaptive fuzzy sliding mode controller (AFSMC), color histogram, measurement, overhead crane, visual tracking.

I. I NTRODUCTION HE overhead crane is an important industrial vehicle for transporting heavy loads. In general, one operator moves the cart slowly to avoid damaging the loads, and another operator monitors the motion of the crane system on the ground when driving the crane. However, an overhead crane system should be automatic, since it is one of the most dangerous pieces of equipment to operate at an industrial site. Many researches have developed controllers to operate the crane. In general, crane controllers use encoders as sensors for cart position and load swing. Swinging of the load when a crane moves is inevitable. However, vibration of the cart induced by the movement of a heavy load can cause sensor displacement and breakdown of the sensor. This work proposes the use of visual tracking technology rather than physical encoders to control an overhead crane, enabling it to transport the load rapidly and smoothly. This technology provides remote and contactless sensing, and is free of the sensor allocation problem, electrical noise, and other problems [1]. Some works on visual tracking have been published and their proposals implemented in some applications. Some researchers

Manuscript received February 7, 2011; revised April 23, 2011; accepted July 12, 2011. Date of publication August 8, 2011; date of current version October 25, 2011. This work was supported by the National Science Council of Taiwan under Contract NSC-99-2628-E-033-003. The authors are with the Department of Electrical Engineering, Chung Yuan Christian University, Jhongli 320, Taiwan (e-mail: ccy@cycu.edu.tw). Color versions of one or more of the gures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identier 10.1109/TIE.2011.2163916

have attempted to implement image sensing to measure the swing angle of an overhead crane or used visual tracking technology to track an object, but they have not used it in realtime visual feedback control [2][7]. Kawai et al. [8] installed a camera in the cart of a crane to measure the swing angle. In this kind of installation, the camera suffers from displacement that is caused by vibration of the cart. Osumi et al. [9] utilized two charge coupled devices (CCDs) to measure the angle of the swing, but the required calculations were burdensome. Miyamoto et al. [10] used a laser to detect a cross mark on the cranes hook. Matsuo et al. [11] implemented visual feedback using a PID+Q controller as the crane controller, using a builtin video tracker. Yoshida and Tsuzuki [12] made an anti-swing controller with visual feedback using a stereovision camera that was mounted on a robot. Several applications, such as tracking of spacecraft, vehicle, and robot, have also been addressed by visual-based approaches [7], [13][16]. These visual tracking methods are complex, and the sensing instruments are costly. Moreover, various investigations of the crane controller have also been published. Lee [17] proposed a classic model and an anti-swing control method for a 3-D crane, but the design of the controller requires that the complex dynamic equations of the crane system be taken into account. Several works have adopted intelligent control schemes, such as fuzzy logic control (FLC) [18][21], which is model-free and exhibits universal approximation properties. However, FLC always takes time to achieve stability, and the need to determine some of the parameters in its rules and the system variables may degrade control performance. Accordingly, the adaptive sliding model control (SMC) method [22][25] has the potential to compensate for the shortcomings of the FLC in designing the crane controller. The SMC, belonging to a kind of variable structure control system, is a nonlinear feedback control. Its advantages include order reduction, disturbance rejection, uncertainty insensitivity, and fast response. Because SMC is less sensitive to system parameter variations and noise disturbance, it is proper using SMC to control underactuated systems, such as the overhead cranes. This work presents an adaptive fuzzy sliding mode controller (AFSMC) based on visual feedback technology for position and anti-swing control of an overhead crane. Using a standard home video CCD camera, the proposed visual tracking method can simplify the image processing procedures and obtain the position of cart and load swing angle information in time to control the crane effectively. The performance to drive the crane by a speed prole is also examined. The proposed method has two major features. First, the main constraint on a visual feedback control system is

0278-0046/$26.00 2011 IEEE

CHANG AND LIE: VISUAL TRACKING AND MEASUREMENT TO CONTROL DYNAMICS OF OVERHEAD CRANES

1641

Fig. 1.

Conguration of visual control-based crane model.

computing time [26]. Some researchers have used expensive high-end computational technologies to improve processing time. A high-speed camera can also be used to perform realtime tracking. In contrast, in this work, a visual feedback control system is constructed from a 30 fps CCD camera and a home PC without any extra hardware. Nevertheless, it does require a fast algorithm to process real-time data for the feedback control algorithm. The system applies visual tracking based on color histograms. The system benets from fast recognition and localization of the object. It matches color histograms of the tracking data with the color histograms in a sequence of images. To optimize the performance achieved using the CCD camera, an algorithm that does not process a whole image is developed. It operates only in the tracking area, moving and following the tracking point. Second, the controller is an AFSMC. The SMC is a nonlinear feedback control system whose structure is purposely altered. The SMC can be applied to a nonlinear system with parameter uncertainties and external disturbances. The FLC, which is independent of any system model, will be employed to analyze system control characteristics under varying system parameters, such as mass of the load and wire length [22]. Therefore, the proposed method involves no system model and its other advantages include disturbance rejection, insensitivity to uncertainty, and speed of response. The slopes of the sliding surface of SMC can be adapted to suit the controlled system. This paper is organized as follows. Section II reviews the concepts on which the proposed visual tracking methodology is based. The steps in obtaining crane information using the tracking area and color histogram are presented. Section III presents the AFSMC design technology. The procedures for designing an anti-swing controller for positioning are described in detail. Section IV shows the motion planning approach for the crane system by a speed prole. The acceleration and deceleration of cart are derived for smooth and rapid transportation of crane. Section V presents experimental results concerning the performance of the crane control system to verify its effectiveness. Finally, Section VI draws conclusions. II. S IMPLIFIED V ISUAL T RACKING M ETHOD Fig. 1 displays the conguration of the visually based crane control system. A 2-D crane was built with a CCD camera perpendicular to the central part of it. A red wire was installed

Fig. 2. Tracking area.

in the crane model to tie the load. All of the images captured by the CCD camera were processed in red-green-blue (RGB) color space. A color contains information that is very useful in object tracking. Since a histogram of color is invariant under scaling and rotation of the object [4]; therefore, visual tracking based on color histograms is utilized to track the red wire and measure the swing angle. It is also used to measure the position of the cart and the load. To track the position of the wire, the algorithm must match the color of the object in the model image to that in the sequence of images. Accordingly, to nd the best match between the colors in the model image and those of other images in the sequence, a Bhattacharyya similarity measurement will be adopted [27]. The applications of real-time image processing that involve fast movement are difcult. Therefore, the algorithm that is proposed here is not applied to each whole image, but only to the tracking area to increase computational speed. Since the processing area is small, the calculation time will be sufciently short to optimize the performance achieved using the CCD. The designing steps of the proposed visual tracking method are depicted as follows. Step 1Set the Tracking Area: Fig. 2 shows the way in which two points on the wire (x1, y 1) and (x2, y 2) are chosen as the initial tracking points in the RGB model image. The tracking area covers only a length of Rx pixels and a width of Ry pixels. The length and the width of the tracking area must be equal to or larger than Hx and Hy , which are the length and the width of the color histogram patch, respectively. Hence, the two constraints of image processing are that the color histogram patch has a size of no more than (Hx , Hy ) and that the tracking area has a size of up to (Rx , Ry ). The tracking area that contains the tracking point (x1, y 1) only follows the wire and moves along the X-axis, so the Y-axis value does not change. Accordingly, Ry can be set to its lowest possible value, Hy . The second tracking point (x2, y 2) follows the end point of the wire because the initial length of the wire is known. In addition, since the load will be hoisted or lowered during the movement of the crane, the second tracking point moves along the X- and Y-axes; therefore, the size of the second (lower) tracking area, must be larger than the rst (upper) tracking area. The tracking area must be as small as

1642

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 59, NO. 3, MARCH 2012

possible to minimize the processing time, while large enough to catch the tracking point. Step 2Compute the Color Histogram: To make useful measurements, the target is modeled as a color distribution. The rst point (x1, y 1) can be any point on the wire, but the second point (x2, y 2) must be at the end of the wire to enable the position of the load to be tracked. The distance between the rst and second points determines the resolution of the swing angle calculation. Three 1-D color histograms of the wire for R, G, and B around both of the points are [2]:
R : r = 1, . . . R} q R = {qr G q G = {qg : g = 1, . . . G} B : b = 1, . . . B } q B = {qb

Fig. 3.

Example of the tracking area method.

(1)

R G B where qr , qg , and qb are the number of pixels in the patch (Hx , Hy ); r denotes the red value in the red bin; g is the green value in the green bin, and b is the blue value in the blue bin. The constants R, G, and B represent the number of red, green, and blue bins, respectively. Step 3Undertake Moving Tracking Area and Similarity Measurement: The method of moving tracking areas that follow the respective moving tracking point (x1, y 1) and (x2, y 2) is proposed. Because of the uncertainties in tracking point movement that are associated with the speed of the cart, the speed of the camera, the position of the camera, and other factors, determining the optimal size of the tracking area is difcult. Additionally Rx and Ry can be determined only by considering the speed of the motion of the tracking point from the sequence of images. The tracking area simply follows the tracking point. The algorithm that describes the movement of the tracking area is as follows:

x1 (k ), y 1 (k )) , c1 (k + 1) = (

c2 (k + 1) = ( x2 (k ), y 2 (k )) (2)

where c1 and c2 are the centers of mass of both tracking areas; ( x1 , y 1 ) and ( x2 , y 2 ) are the positions of the tracking points in the sequences of images and k is the number of time sequences, k = 0, 1, 2, 3, . . .. The initial centers of mass, c1 (1) and c2 (1), are chosen at positions (x1, y 1) and (x2, y 2) in the image model, respectively. Hence, the center of the tracking area moves with the tracking point. Color histograms pR , pG , and pB are generated within a region of size (Hx , Hy ) for each point inside the tracking area, and then the color histograms of the model are compared with the color histograms in image sequences. The Bhattacharyya similarity measurement is used to measure the matching values of the color histograms [2]: = p (x, y )q (3)

where {R, G, B }, R , G , and B are Bhattacharyya similarity coefcients. The three similarities thus obtained are blended to yield a unied similarity as, (x, y ) = R (x, y ) + G (x, y ) + B (x, y ) (4)

where , , and are the weighting coefcients of red, green, and blue, and satisfy + + = 1 and 0 , , 1. The values of , , and depend on the characteristic color of the object. For example, the red wire has a larger value of to capture its redness and smaller values of and , which are less affected by changes in the illumination of the wire when it moves. Thus, set , , and to 0.8, 0.8, and 0.1, respectively; the wire has a solid red color, with R = 255, G = B = 0. As the wire moves, the red color changes as it enters shadows, and captured color values are R = 240, G = 15, B = 20. The composition 0.8 R + 0.1 G + 0.1 B will not result in a large difference in (x, y ). However, this color composition still can be distinguished from other colors. Each value (x, y ) at every position inside the tracking area will be compared to that of the model. The point at which (x, y ) has the closest value to that of the model is the position of the target, which is called the tracking point. The same procedures are applied 1 (k )) and ( x2 (k ), y 2 (k )) to both tracking areas. Then, ( x1 (k ), y which are the positions of the tracking points in the k th image sequences, are determined. Fig. 3 displays an example of the tracking area method. The model is a red ball at (x1, y 1) = (8, 4). A color histogram of the model patch (q R , q G , q B ) can be generated with Hx = Hy = 3. The assumption is that the ball does not move up or down, but only along the X-axis, and the distance of the movement in each image sequence is uncertain. Hence, to track the ball to the next image in the sequence, a tracking area of the appropriate size with a center of mass c1 (1) = (x1, y 1) = (8, 4) is required. In this example, Rx = 25 is set, and because the target does not move up or down in this case, Ry is set to Hy to minimize the processing time. Then, a color histogram (R , G , B ) is generated and the Bhattacharyya similarity coefcient (x, y ) inside the tracking area associated with each position is calculated; each (x, y ) is compared to the determined by the model. The position at which the values are closest is the position of the target. 1 (1)) = Now, suppose that the position of the target ( x1 (1), y (20, 4). To track the next sequence, a tracking area with the same shape, but whose center of mass has already been shifted x1 (1), y 1 (1)) = to the rst sequence tracking point c1 (2) = ( (20, 4), is created, and then the same procedure as was used before is adopted to track the target for c2 (k ). Furthermore, the movement along both X- and Y-axes is tracked using the same procedure, except that Ry must have a value that covers the Y-axis movement of the target. Step 4Perform the Correction: Generally, the load swing angle can be determined after the positions of the tracking points are known, but the problem of distortion of the image must be considered. Major external distortion is brought by the rotation matrix dening the camera orientation. In other words, it arises when a 3-D object is projected on a plane.

CHANG AND LIE: VISUAL TRACKING AND MEASUREMENT TO CONTROL DYNAMICS OF OVERHEAD CRANES

1643

Fig. 5. Designing steps of the proposed visual tracking method.

Fig. 4.

Keystone distortion at the acquired image. TABLE I R ELATIONSHIP B ETWEEN P OSITION AND D ISTORTION

Therefore, a useful calibration method is necessary for the image correction. By conducting steps 14, we can implement the position tracking and calculate the load swing of an overhead crane by the proposed simplied tracking method, shown in Fig. 5. III. D ESIGN A NTI -S WING AND P OSITION C ONTROLLERS A. Method of Traveling the Cart [22] The AFSMC is derived in this paper to drive the trolley according to the position and swing angle information by visual tracking. Suppose the desired coordinate of the cart position in the crane control system is rp and the desired angle of the load swing is r . The present cart position is P , and the present angle of the load swing is . The denitions of the sliding functions s1 and s2 s1 = e p + C1 ep s2 = e + C2 e (6) (7)

This distortion transforms a square into a keystone (trapezoid). Fig. 4 presents the keystone distortion in the image. An ideal line is added on the left and right sides in the image, and the red wire in the image is thus determined to be somewhat distorted. Therefore, to determine the distortion coefcient, Table I describes the results. The term xprac denotes the position of the real distorted red wire in terms of pixels and xideal represents the position of the ideal black line. In this image, the line at xprac = 203 is approximately at the position perpendicular to the CCD. When the positions of the tracking points are known, the angle of the wire can be corrected by compensating for the image distortion, as follows: = tan
1

where ep = rp P is the position error; e = r is the load swing error, and both C1 and C2 are real positive numbers, are dened as the slopes of the sliding surface [23]. However, C1 and C2 inuence the rising time, overshoot, and settling time of the total system response; therefore, the determined C1 and C2 may contradict with each other because the speed of the cart may cause additional load swing. The regulation of load swing is a more important issue than the control of speed of the cart in a real crane. Therefore, an adaptive method for tuning C2 to enhance the performance is utilized:
b C2 = C2 + B2 C2

( x1 + round( y1 yp ) tan( x1 ) x 2 ) ( y1 y 2 )

(8)

(5)
b where C2 is the initial matrix of C2 ; B2 is the weighting matrix, and C2 is the tuning term, which is given by m Rm : If |e | is Am then C2 is C2

where 1/( y1 y 2 ) is the limit of resolution of the sensed 2 ) yields a better angular image. A larger value of ( y1 y resolution. By setting = 0 , we can obtain the correction factor = 0.00157 deg/pixel. Remark 1: Deciding the size of tracking area is important to the proposed approach. Fast movement of cart requires a larger tracking area. Moreover, the tracking area must be kept small to minimize the processing time, while being large enough to catch the tracking point. Remark 2: The keystone distortion is large enough to have serious effect on stability. Since an accurate measurement of the load swing angle is crucial for stabilization, the deviation of wrong angle will cause a destructive effect in the experiment.

(9)

Am is the mth fuzzy input linguistic term with function m m Am (|e |) = e(|e |C / )2 ; C m is the mean; m is the m variance, and C2 is the mth fuzzy output singleton. Consequently, applying M fuzzy rules yields the output
M m C2 Am (|e |)

C2 =

m=1 M

. Am (|e |)

(10)

m=1

1644

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 59, NO. 3, MARCH 2012

After C2 is normalized to [0, 1], the dynamic range of C2 b b is [C2 , C2 + B2 ]. Thus, when the load swing becomes severe, the slope C2 increases, reducing the load swing. Combining s1 and s2 yields the composite sliding function, s = s1 + Cs2 (11)

where C is a real negative-denite matrix. The matrix C is used to determine the weight between s1 and s2 . A lower C means that the AFSMC is focused on position control whereas a higher C means that the AFSMC is focused on the load swing. Therefore, an adaptive method for tuning C is proposed here to enhance performance. C = C b + B C (12)

Fig. 6.

Block diagram of the proposed method.

where C b is the initial C ; B is the weighting matrix, and C is the tuning term, obtained from Rn : If |s2 | is An then C is C n (13)

where An is the nth fuzzy input linguistic term with the n n function An (|s2 |) = e(|s2 |C / )2 ; C n is the mean; n is n the variance, and C is the nth fuzzy output singleton. Hence, applying N fuzzy rules yields the output:
N

C n An (|s2 |) . An (|s2 |) (14)

Fig. 7. Speed prole.

C =

n=1 N n=1

B. Method of Hoisting and Lowering Load Hoisting and lowering the load affect the acceleration of the swinging load [17]. Therefore, the hoisting and lowering of the load must be considered in designing a crane controller. The proposed AFSMC method utilizes an adaptive algorithm for tuning the corresponding weighting matrices of sliding slopes; therefore, the effect of hoisting or lowering the load on its swinging can be accounted for. Accordingly, to ensure a rapid and precise response of hoisting and lowering load, a simple PID controller is proposed
t

After the fuzzy output C is normalized to [0, 1], the dynamic range of C is [C b , C b + B ]. Accordingly, if the load swing subsystem is far from the sliding surface, s2 = 0, then the adaptive tuning method increases C to limit the load swing directly. When the load swings close to the sliding surface, C declines and the AFSMC switches focus to position control. Therefore, |e | and |s2 | are chosen as the fuzzy antecedents in (9) and (13), helping the proposed AFSMC method to control the crane and ensure smooth and safe transportation, as required in a real case of control. The power to the motor is controlled as follows. The sliding surface function s is treated as the fuzzy antecedent, and the power to the motor, fx , is used as the fuzzy consequent. Hence, the fuzzy rule is Rj : If s is B j then fx is F j (15)

fl = kpl el + kil
0

el dt + kdl

del dt

(17)

where B j is the input linguistic term with membership function j j 2 B j (s) = e(sC / ) ; C j and j are the mean and variance, and F j is the output fuzzy singleton of the j th rule. Therefore, applying J rules yields
J

where fl is the power to the hoisting motor; el = rl l is the error between the set wire length rl and the actual, present wire length l, and kpl , kdl , and kil are the gains for proportional, integral, and derivative values. The block diagram of the proposed control method is shown in Fig. 6. IV. M OTION P LANNING Rapid and smooth transportation are the main goals of crane control. Hence, Fig. 7 presents a reference speed prole for planning crane motion [28]. In this case, r is the reference speed, vmax is the highest speed of the crane; ta is the duration of acceleration, and td is the duration of deceleration. Hence,

F j B j (s)
J

fx =

j =1

. B j (s)

(16)

j =1

CHANG AND LIE: VISUAL TRACKING AND MEASUREMENT TO CONTROL DYNAMICS OF OVERHEAD CRANES

1645

the speed during 0 < t ta and ta < t td are denoted by r a and r d , respectively. Thus, we have r a (t) = r d (t) = vmax 2 vmax 2 1 cos 1 + cos t ta (t ta ) (td ta ) (ta < t td ) and the corresponding accelerations are, r a (t) = r d (t) = vmax sin t 2ta ta vmax sin 2(td ta ) (0 < t ta ) (t ta ) (td ta ) (ta < t td ). (21)
Fig. 8. Encoder-Visual tracking comparison, blue: encoder result, red: visual tracking result.

(18)

(19)

(20)

From (20) and (21), the maximum acceleration and deceleration are r a max = vmax /2ta and r d max = vmax /2(td ta ) at t = ta /2 and t = ta + td ta /2, respectively. Therefore, the minimum durations of acceleration and deceleration time are (ta )min = (td ta )min vmax 2r a max vmax = . 2r d max (22) (23)

Thus, the tracking distance d of the prole can be determined by integration

ta

d=
0

vmax 2
td

1 cos

t dt ta (t ta ) (td ta ) dt. (24)

+
ta

vmax 2

1 + cos

Hence, the motion of the crane can be planned using the speed prole in Fig. 7. V. E XPERIMENTAL R ESULTS A crane model with a base area of 1.2 m 1.1 m was built. The initial wire length was 0.8 m and the mass of the load was 1 kg. The image sensing performance is compared with that of a real physical sensoran encoder with 2500 pulse/ rotation. For visual tracking, a CCD camera was installed. It was a Sony DCR-PC330 with an IEEE 1394 interface. The CCD camera had a maximum video rate of up to 30 fps and a resolution of 720 480 pixels. The PC that is used in image processing and in AFSMC had Pentium 4 Core2Duo 2 GHz processors with 2 GB of memory. To maintain a constant speed of image processing, the controller herein is implemented on another PC. The image processing data are sent using an RS232 with a 19 200 Bd rate connection; Fig. 1 shows the system conguration. MATLAB 7.0 with RT Workshop software is applied to develop the crane controller.

In this paper, the length of the tracking area is Rx , which is 50 pixels for the rst and second tracking areas. The width of the tracking area Ry that is associated with the second point is set to 15 pixels. The dimensions of the patch, Hx and Hy , are three pixels and one pixel, respectively. The initial length ( y1 y 2 ) is set to 350 pixels, and so the minimum angular resolution is = tan1 (1/350) = 0.1637 . To conrm the visual tracking performance, the load is forced to swing without movement of the card. Fig. 8 compares the encoder with visual tracking with calibration coefcient = 0.00157 deg/pixel. Visual tracking of wire movement based on color histograms is successful according to Fig. 8. The outputs of visual tracking (red line) and the encoder (blue line) are almost identical. Also, the processing time for calculating the swing angle is only around 1 or 2 ms. Therefore, the proposed visual tracking method can optimize the performance of the CCD, with 30 fps. The rst experiment reveals the performance of the AFSMC as an anti-swing controller using an encoder. Visual tracking of the wire does not provide an input, but is compared with the use of the encoder. The parameters of the proposed AFSMC are b = 8, B2 = 10, C b = set to rp = 0.7 m, r = 0 , C1 = 5, C2 2, and B = 2 [23]. The maximum acceleration and velocity of the cart are 1.5625 m/s2 and 0.225 m/s, respectively. Fig. 9 plots the result of the visual tracking in image sequences; the red dot indicates the tracking point. Fig. 10 depicts the results concerning swing of the load. The AFSMC has successfully suppressed the load swing. The maximum load swing is around 2.965 . However, the load swing is measured as 0 deg by the encoder. The results of visual tracking indicate that the load has swung through a small angle of approximately 0.2 . In fact, a CCD can capture the true movement of the load. The small swing is caused by the traveling of the wire through a wide aperture, but it cannot be detected by encoder. The diagram and practical situation are shown in Fig. 11(a) and (b). Hence, wrong information from the encoder can stop the controller; meanwhile, the visually based tracking sensor provides the true swing angle data, and so this work proposes the use of a visually based method for cranes.

1646

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 59, NO. 3, MARCH 2012

Fig. 11.

Loose at the hole (a) diagram and (b) practical situation.

Fig. 9. Visual tracking result in image sequences (red dot is the tracking point).

Fig. 10. Anti-swing with encoder on AFSMC without hoist the load, blue: encoder result, red: visual tracking result.

The performance of the motion planning method is also experimentally examined. The cart travels 0.7 m, and the load is hoisted 0.2 m. The results obtained using the AFSMC with and without motion planning using an encoder are compared and analyzed, shown in Fig. 12. The blue and green lines present the performance with and without the motion planning method, respectively. The payload is 1 kg. The gures clearly

demonstrate that the AFSMC with the motion planning method suppressed the load swing much more effectively than that without motion planning method, as displayed in Fig. 12(a), at the cost a slightly later arrival at the target position, as presented in Fig. 12(b). The input power to the cart motor with motion planning follows the velocity prole, slowly increasing to the maximum power as presented in Fig. 12(c). This situation differs from that with the AFSMC without motion planning, in which the power is at its maximal value from the start. Fig. 13 shows the performance obtained by replacing the payload with a 2 kg payload. Experiments that were carried out with various load masses have established that the stability does not vary with the load mass, proving the insensitivity of AFSMC. Next, an experiment demonstrates the performance of the AFSMC using visual tracking with motion planning method in terms of the motion and hoisting of the load. The results obtained using the encoder are considered for comparison. Fig. 14 clearly reveals that both the AFSMC using the encoder and that using the visual tracking successfully suppressed the load swing during the movement and hoisting of the load. The maximum load swing that occurred when the encoder was used was 1.73 ; that obtained using visual tracking was 1.728 , as displayed in Fig. 14(a). The residual swing angles obtained using encoder and the image sensor at the target position are approximately 0 . The results in Fig. 14(b) and (c) also reveal that visual tracking as the sensor in the AFSMC effectively replaces a physical sensor, and is even sensitive to small angles that cannot be detected by the physical sensor, as displayed in Fig. 10. Previous experiments established that the visual tracking method in place of an encoder can effectively control the crane. The distortion was compensated for in those experiments by the calibration coefcient method, improving measurement. The nal experiment yielded results concerning distortion of the keystone in image sensing the crane. A crane can be moved using the AFSMC with visual tracking method. In Fig. 15(a), the cart stops around 7 s after it began to move, because the movement is unstable. The false data obtained by visual tracking caused the crane controller to input the wrong power to the motor. Therefore, the motor moves unstably. Although the distortion coefcient is very small, only 0.00157 deg/pixel, the instability has a very destructive effect on the crane.

CHANG AND LIE: VISUAL TRACKING AND MEASUREMENT TO CONTROL DYNAMICS OF OVERHEAD CRANES

1647

Fig. 12. Comparison on AFSMC with and without motion planning, payload = 1 kg. (a) Swing angle, (b) cart position, and (c) input power.

Fig. 13. Performance of AFSMC with motion planning, payload = 2 kg. (a) Swing angle, (b) cart position, and (c) input power.

The experimental results above demonstrate that the AFSMC with the visual tracking method worked very well as an antiswing controller of crane systems. The AFSMC is robust

against varying parameters, such as load mass and wire length. The motion planning method also minimizes the load swing.

1648

IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS, VOL. 59, NO. 3, MARCH 2012

Fig. 14. Transporting and hoisting the load using AFSMC with motion planning method, blue: encoder, red: visual tracking. (a) Load swing, (b) cart and load position, and (c) power input.

VI. C ONCLUSION A visual tracking method that is based on a color histogram is used in place of a physical sensor. The sliding mode controller is combined with a fuzzy logic controller, which does not require a system model to analyze system characteristics and adjusts the slopes of the sliding mode function automatically. The combine system is designed to achieve fast positioning and minimize the swing in a nite time. A motion planning method was developed to ensure smooth movement of the load and reduce the swing. Experimental results reveal that

Fig. 15. Visual tracking without distortion compensating on AFSMC. (a) Swing angle, (b) position, and (c) input power.

the proposed visual tracking method can effectively replace a physical sensor and is sensitive to small angles that cannot be detected using an encoder as the physical sensor. The proposed

CHANG AND LIE: VISUAL TRACKING AND MEASUREMENT TO CONTROL DYNAMICS OF OVERHEAD CRANES

1649

method successfully suppresses the load swing and positions the target in a nite time. ACKNOWLEDGMENT T. Knoy is appreciated for his editorial assistance. R EFERENCES
[1] K. Hirata, Y. Kimura, and K. Sugimoto, Visual feedback control of cart-pendulum systems with webcam, in Proc. Int. Conf. Mechatron., May 2007, pp. 16. [2] D. Xu, L. W. Han, M. Tan, and Y. F. Li, Ceiling-based visual positioning for an indoor mobile robot with monocular vision, IEEE Trans. Ind. Electron., vol. 56, no. 5, pp. 16171628, May 2009. [3] C. Wook, M. Akbarian, V. Rubtsov, and C. J. Kim, Microhand with internal visual system, IEEE Trans. Ind. Electron., vol. 56, no. 4, pp. 10051011, Apr. 2009. [4] P. Y. Hsiao, C. L. Lu, and L. C. Fu, Multilayered image processing for multiscale harris corner detection in digital realization, IEEE Trans. Ind. Electron., vol. 57, no. 5, pp. 17991805, May 2010. [5] T. Celik and H. Kusetogullari, Solar-powered automated road surveillance system for speed violation detection, IEEE Trans. Ind. Electron., vol. 57, no. 9, pp. 32163227, Sep. 2010. [6] H. C. Liaw and B. Shirinzadeh, Robust adaptive constrained motion tracking control of piezo-actuated exure-based mechanisms for Micro/ Nano manipulation, IEEE Trans. Ind. Electron., vol. 58, no. 4, pp. 1406 1415, Apr. 2011. [7] N. Sudha and A. R. Mohan, Hardware-efcient image-based robotic path planning in a dynamic environment and its FPGA implementation, IEEE Trans. Ind. Electron., vol. 58, no. 5, pp. 19071920, May 2011. [8] H. Kawai, Y. Choi, Y. B. Kim, and Y. Kubota, Measurement system design for sway motion based on image sensor, in Proc. IEEE Int. Conf. Netw. Sens. Control, Mar. 2009, pp. 185188. [9] H. Osumi, A. Miura, and S. Eiraku, Positioning of wire suspension system using CCD cameras, in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., Aug. 2005, pp. 258263. [10] D. Miyamoto, S. Nara, and S. Takahashi, Visual feedback control with laser for the position detection of crane hook, in Proc. SICE-ICASE Int. Joint Conf., Oct. 2006, pp. 20792083. [11] T. Matsuo, R. Yoshino, H. Suemitsu, and K. Nakano, Nominal performance recovery by PID+Q controller and its application to antisway control of crane lifter with visual feedback, IEEE Trans. Control Syst. Technol., vol. 12, no. 1, pp. 156166, Jan. 2004. [12] Y. Yoshida and K. Tsuzuki, Visual tracking and control of a moving overhead crane load, in Proc. IEEE 9th Int. Workshop Adv. Motion Control, May 2006, pp. 630635. [13] Y. Xia, Z. Zhu, M. Fu, and S. Wang, Attitude tracking of rigid spacecraft with bounded disturbances, IEEE Trans. Ind. Electron., vol. 58, no. 2, pp. 647659, Feb. 2011. [14] Y. L. Chen, B. F. Wu, H. Y. Huang, and C. J. Fan, A real-time vision system for nighttime vehicle detection and trafc surveillance, IEEE Trans. Ind. Electron., vol. 58, no. 5, pp. 20302044, May 2011. [15] M. Chacon and S. Gonzalez, An adaptive neural-fuzzy approach for object detection in dynamic backgrounds for surveillance systems, IEEE Trans. Ind. Electron., 2011, to be published. [16] T. Sasaki, D. Brscic, and H. Hashimoto, Human-observation-based extraction of path patterns for mobile robot navigation, IEEE Trans. Ind. Electron., vol. 57, no. 4, pp. 14011410, Apr. 2010. [17] H. H. Lee, Modeling and control of a three-dimensional overhead cranes, ASME J. Dyn. Syst. Meas. Control, vol. 120, no. 4, pp. 471476, Dec. 1998. [18] C. Y. Chang, Adaptive fuzzy controller of the overhead cranes with nonlinear disturbance, IEEE Trans. Ind. Informat., vol. 3, no. 2, pp. 164 173, May 2007.

[19] M. Biglarbegian, W. W. Melek, and J. M. Mendel, Design of novel interval type-2 fuzzy controllers for modular and recongurable robots: Theory and experiments, IEEE Trans. Ind. Electron., vol. 58, no. 4, pp. 13711384, Apr. 2011. [20] C. Y. Chang and K. H. Chiang, Fuzzy projection control law and its application to the overhead crane, Mechatronics, vol. 18, no. 10, pp. 607 615, Dec. 2008. [21] Y. J. Chen, W. J. Wang, and C. L. Chang, Guaranteed cost control for an overhead crane with practical constraints: Fuzzy descriptor system approach, Eng. Appl. Artif. Intell., vol. 22, no. 4/5, pp. 639645, Jun. 2009. [22] D. Liu, J. Yi, D. Zhao, and W. Wang, Adaptive sliding mode fuzzy control for a two-dimensional overhead crane, Mechatronics, vol. 15, no. 5, pp. 505522, Jun. 2005. [23] C. Y. Chang, K. C. Hsu, and K. H. Chiang, Anti-sway and motion planning control of overhead cranes, Int. J. Innov. Comput. Inf. Control, vol. 6, no. 7, pp. 33133328, Jul. 2010. [24] A. G. Loukianov, J. M. Canedo, L. M. Fridman, and A. Soto-Cota, Highorder block sliding-mode controller for a synchronous generator with an exciter system, IEEE Trans. Ind. Electron., vol. 58, no. 1, pp. 337347, Jan. 2011. [25] K. K. Shyu, C. L. Jen, and L. J. Shang, Sliding-mode control for an underactuated overhead crane system, in Proc. IEEE IECON , Paris, France, Apr. 2006, pp. 412417. [26] A. Arsenio and J. Santos-Victor, Robust visual tracking by an active observer, in Proc. IEEE Int. Conf. Intell. Robots Syst., Sep. 1997, vol. 3, pp. 13421347. [27] J. Ning, L. Zhang, D. Zhang, and C. Wu, Interactive image segmentation by maximal similarity based region merging, Pattern Recognit., vol. 43, no. 2, pp. 445456, Feb. 2010. [28] H. H. Lee, Motion planning for three-dimensional overhead cranes with high-speed load hoisting, Int. J. Control, vol. 78, no. 12, pp. 875886, Aug. 2005.

Cheng-Yuan Chang was born in Taiwan, in 1968. He received the B.S. and M.S. degrees in control engineering from National Chiao-Tung University, Hsin-Chu, Taiwan, in 1990 and 1994, and the Ph.D. degree in electrical engineering from National Central University, Jhongli, Taiwan in 2000. He was with the Department of Electronic Engineering, Ching Yun University, Jhongli, Taiwan, from 1994 to 2007. From 2007, he joined the Department of Electrical Engineering, Chung Yuan Christian University, Jhongli, Taiwan. He is currently a Professor. His research interests are in the area of visual servo control, active noise control, fuzzy neural controller, GA algorithm, and their applications design.

Handra Wijaya Lie was born in Balikpapan, Indonesia in 1985. He received the B.S. degree from Petra Christian University, Surabaya, Indonesia in 2007 and the M.S degree from Chung Yuan Christian University, Jhongli, Taiwan, in 2009, both in electrical engineering. His current research interests are motion planning and visual tracking system.