Industrial Robot: An International Journal

Emerald Article: A robotic system based on fuzzy visual servoing for handling flexible sheets lying on a table Paraskevi Zacharia, Nikos Aspragathos, Ioannis Mariolis, Evaggelos Dermatas

Article information:
To cite this document: Paraskevi Zacharia, Nikos Aspragathos, Ioannis Mariolis, Evaggelos Dermatas, (2009),"A robotic system based on fuzzy visual servoing for handling flexible sheets lying on a table", Industrial Robot: An International Journal, Vol. 36 Iss: 5 pp. 489 - 496 Permanent link to this document: http://dx.doi.org/10.1108/01439910910980213 Downloaded on: 08-01-2013 References: This document contains references to 22 other documents Citations: This document has been cited by 1 other documents To copy this document: permissions@emeraldinsight.com This document has been downloaded 286 times since 2009. *

Users who downloaded this Article also downloaded: *

Paraskevi Zacharia, Nikos Aspragathos, Ioannis Mariolis, Evaggelos Dermatas, (2009),"A robotic system based on fuzzy visual servoing for handling flexible sheets lying on a table", Industrial Robot: An International Journal, Vol. 36 Iss: 5 pp. 489 - 496 http://dx.doi.org/10.1108/01439910910980213 Paraskevi Zacharia, Nikos Aspragathos, Ioannis Mariolis, Evaggelos Dermatas, (2009),"A robotic system based on fuzzy visual servoing for handling flexible sheets lying on a table", Industrial Robot: An International Journal, Vol. 36 Iss: 5 pp. 489 - 496 http://dx.doi.org/10.1108/01439910910980213 Paraskevi Zacharia, Nikos Aspragathos, Ioannis Mariolis, Evaggelos Dermatas, (2009),"A robotic system based on fuzzy visual servoing for handling flexible sheets lying on a table", Industrial Robot: An International Journal, Vol. 36 Iss: 5 pp. 489 - 496 http://dx.doi.org/10.1108/01439910910980213

Access to this document was granted through an Emerald subscription provided by TECHNICAL UNIVERSITY OF SOFIA For Authors: If you would like to write for this, or any other Emerald publication, then please use our Emerald for Authors service. Information about how to choose which publication to write for and submission guidelines are available for all. Please visit www.emeraldinsight.com/authors for more information. About Emerald www.emeraldinsight.com With over forty years' experience, Emerald Group Publishing is a leading independent publisher of global research with impact in business, society, public policy and education. In total, Emerald publishes over 275 journals and more than 130 book series, as well as an extensive range of online products and services. Emerald is both COUNTER 3 and TRANSFER compliant. The organization is a partner of the Committee on Publication Ethics (COPE) and also works with Portico and the LOCKSS initiative for digital archive preservation.
*Related content and download information correct at time of download.

Research article

A robotic system based on fuzzy visual servoing for handling exible sheets lying on a table
Paraskevi Zacharia, Nikos Aspragathos, Ioannis Mariolis and Evaggelos Dermatas
Department of Mechanical and Aeronautics Engineering, University of Patras, Patras, Greece
Abstract Purpose The purpose of this paper is to present a exible automation system for the manipulation of fabrics lying on a work table and focuses on the design of a robot control system based on visual servoing and fuzzy logic for handling exible sheets lying on a table. The main contribution of this paper is that the developed system tolerates deformations that may appear during robot handling of fabrics due to buckling without the need for fabric rigidization. Design/methodology/approach The vision system, consisting of two cameras, extracts the features that are necessary for handling the fabric despite possible deformations or occlusion from the robotic arm. An intelligent controller based on visual servoing is implemented enabling the robot to handle a variety of fabrics without the need for a mathematical model or complex mathematical/geometrical computations. To enhance its performance, the conventional fuzzy logic controller is tuned through genetic algorithms and an adaptation mechanism and the respective performance is evaluated. The experiments show that the proposed robotic system is exible enough to handle various fabrics and robust in handling deformations that may change fabrics shape due to buckling. Findings The experiments show that the proposed robotic system is exible enough to handle various fabrics and robust in handling deformations that may change fabrics shape due to buckling. Research limitations/implications It is not possible to cover all the aspects of robot handling of exible materials in this paper, since there are still several related issues requiring solutions. Considering the future research work, the proposed approach can be extended to sew fabrics with curved edges and correcting the distortions presented during robot handling of fabrics. Practical implications The paper includes implications for robot handling a variety of fabrics with low and medium bending rigidity on a working table. The intent of this paper deals with buckling in context of achieving a successful seam tracking and not the correction strategy against folding or wrinkling problems. Originality/value This paper fulls an identied need to study the fabrics behavior towards robot handling on a working table. Keywords Fuzzy control, Robotics, Materials handling equipment, Textiles, Intelligent agents Paper type Research paper

1. Introduction
The development of intelligent control systems with vision feedback enables robots to perform skillful tasks in more realistic environments. Flexible materials pose additional problems compared to rigid bodies due to low bending rigidity, non-linearity, anisotropy and structure complexity. Thus, their mechanical behavior is rather difcult to be modeled. A method was introduced in Torgerson and Paul (1988) for the manipulation of fabrics towards the needle employing a geometric approach for path-point computation related to robot motion. The deviation errors lie in the range of 3-5 mm, while visual feedback was not used. The deformation problems that arise towards handling of fabrics were not discussed.
The current issue and full text archive of this journal is available at www.emeraldinsight.com/0143-991X.htm

The FIGARO system (Gershon and Porat, 1988, 1986) performed handling and assembling operations using two superimposed servo control systems. The rst system maintains a small tension moving the robot forward with the cloth and the second one rotates the cloth about the sewing needle to produce a seam parallel to the edges. The system performed best with shirting and worsted woven fabrics, which have a reasonable resistance in buckling. Robotic manipulation strategies were investigated in Gershon (1993) for handling limp materials proposing a parallel process decomposition of the robot sewing task (Gershon, 1990). A robot arm manipulates the fabric, modeled as a mass-spring-damper system, to modify its orientation and control the fabric tension during the sewing process, which was decomposed into four concurrent processes within a superposition parallel architecture.
This paper is nanced by the General Secretariat for Research and Technology of Greek Government as part of the project XROMAHandling of non-rigid materials with robots: application in robotic sewing PENED 01. This project is integrated to EU-funded FP6 Innovative Production Machines and Systems Network of Excellence.

Industrial Robot: An International Journal 36/5 (2009) 489 496 q Emerald Group Publishing Limited [ISSN 0143-991X] [DOI 10.1108/01439910910980213]

489

A robotic system based on fuzzy visual servoing Paraskevi Zacharia et al.

Industrial Robot: An International Journal Volume 36 Number 5 2009 489 496

An automated sewing system was developed (Kudo et al., 2000) including two coordinated robots handling the fabric on the table. The programming included hands coordination during task execution by applying hybrid force/position control and synchronization with the sewing machine speed. Visual feedback was used to control the seam path and the deviation from the desired trajectory and the trajectory error was within ^0.5 mm. The present work is part of a project for the development of a robotic workcell for sewing fabrics. The project includes handling tasks (ply separation, translation, placement, folding, feeding and orientation), tension control and quality control of fabrics and stitches (Moulianitis et al., 1999; Koustoumpardis and Aspragathos, 2003; Koustoumpardis et al., 2006; Zoumponos and Aspragathos, 2008). The innovation of this work is the design of a control system that is capable of reliably handling exible materials towards sewing. The main contribution of this paper is the design of a robot fuzzy visual servoing controller for handling fabrics on a work table, which is enhanced using genetic-based and adaptive control. The system works without the need for special mathematical computations and can tolerate deformations on condition that they do not appear in the seam segment to be sewed.

2. Robot handling of exible sheets towards sewing

The developed vision system is a combination of image- and position-based control system (Hutchinson et al., 1996; Corke, 1996). The image-based analysis is used for the identication of the fabrics shape, the needle-point and the sewing lines orientation. The needle position is also known in the robot coordinate system. The end-effectors position is unknown in the image coordinate system; however, the robot system gives feedback to the control system of the current end-effectors position in the robot base coordinate system. Moreover, the relation between the robot- and the imagecoordinate system is known from the calibration of the camera. This approach makes the system more exible and limits the calibration errors: . The movement towards a point (e.g. a needle). First, the cameras image extracts the vertices of the fabric. After the starting seam edge is identied, the distance (r) between the starting seam vertex and the sewing needle and the orientation angle (u) formed by the starting seam segment and the sewing line (Figure 1(a)) are computed. The linear Figure 1 Scene of the fabric lying on the work table

(u) and angular end-effectors velocity (v) are derived through the designed fuzzy decision system (Section 4). The velocity (u) is the resultant of the velocities in x- and y-direction. The angular velocity (v) is referred to the z-direction, since the x- and y-component are zero; otherwise, collision between the gripper and the worktable would occur. Given the time step Dt and the angle f, i.e. the orientation of r in the image coordinate system, the new position and orientation of the end-effector is computed. Therefore, the fabric is transferred to a new position with new orientation by moving end-effector along the direction of r (dened by f) rotated around z-axis until the angle becomes u0 . The fabric is stuck on the gripper to avoid slipping; however, the system can cope with slipping, but it is more probable to make some oscillations before the fabric reaches the needle within an acceptable tolerance. The sewing process. During sewing, the fabric is guided along the sewing line with a constant velocity, which should be the same with the velocity of the sewing machine, so that good seam quality is ensured. At each time step, the current image of the fabric is captured in order that the orientation error is determined and then, fed back to be corrected by rotating the fabric around the needle. The rotation around the needle. The robot rotates the fabric around the needle until the next seam segment is aligned to the sewing line. The end-effector is enforced to make a circular motion around the needle, since it has penetrated into the fabric. The fuzzy system outputs the endeffectors angular velocity (v ) around an axis perpendicular to the table at the needle-point.

3. Image-based estimation of fabric location

In real-time systems, the estimation of the fabric position and seam line trajectory must be fast and accurate. In this direction, a novel estimator of seam lines trajectory uses a system of two cameras. The rst camera detects the fabric location during robotic handling towards the needle and its view covers the worktable, while the high-resolution images of the second camera are used to detect the exact position of the fabric edges in the area near the needle. 3.1 Detection during robot handling The detection of the fabric position is affected by the transportation of the fabric, the partial occlusion caused by

Needle

Needle

Sewing line y-axis r Sewing line r y-axis

x-axis (a) Without deformations

x-axis (b) With deformations

490

A robotic system based on fuzzy visual servoing Paraskevi Zacharia et al.

Industrial Robot: An International Journal Volume 36 Number 5 2009 489 496

the robotic arm and possible deformation of the fabric (Koch and Kashyap, 1987; Ayach and Faugeras, 1986). A model and scene description (SD) method has been evaluated on the detection of rigid industrial parts (Ayach and Faugeras, 1986), while several tasks involving fabric handling aided by visual information are presented in Fahantidis et al. (1997), addressing the problem of fabric deformation, in occlusion absence. In the proposed three-stage fabric detection method, efcient solutions presented in Ayach and Faugeras (1986) are modied to face both partial occlusion and deformation problems. In the rst stage, the fabric is placed into the working area in the absence of any occlusion or deformation. The points of high curvature, called fabric-corners, are detected by applying Harris corner detector (Deriche and Giraudon, 1990; Vernon, 1991) on the acquired grayscale image. The fabrics shape is then approximated by the planar polygon having these fabric-corners as vertices and a model description (MD) of the fabric is generated using the image coordinates of the fabric corner, the polygon sides lengths and angles. During robot handling, grayscale-images of the fabric are acquired (Figure 2(a)) and thresholded using the ISODATA method (Ridler and Calvard, 1978) (Figure 2(b)) segmenting the binary image into objects using the eight-connected components criterion (Haralick and Linda, 1992), and keeping the largest object (LO) (Figure 2(c)). Each pair of gray-scale and binary images is used to generate a SD containing all corners detected in the scene image. The novel method of detecting potential fabric-corners is accomplished by applying the Harris algorithm to both images and selecting only the common corners. In the last stage, multiple hypotheses for all triples of SD corners are created by selecting the MD elements with error in polygon angle and sides length less than the corresponding threshold that are experimentally derived. For each hypothesis, the corresponding fabric corners are used to dene the optimum rigid transformation of the fabric with respect to the least square error. The rigid transformation of the fabric with maximum common area to LO results to the estimation of the actual fabric position. In the proposed method, since only segments of the fabric are compared, apart from coping with partial occlusion, a certain amount of deformation is tolerable as well. Actually, if at least two successive vertices of the fabric are not deformed or occluded, the method can generate a valid hypothesis. Figure 2

3.2 Detection of fabric edges near needle In the case of the second camera, only a part of the fabric is contained in the image and no occlusion occurs. Therefore, after applying the Harris corner detector, all the detected corners belong to the fabric and the one closest to the needle is dened as the rst sewing corner. The rst sewing corner and the two corners that appear at the image borders dene two sides. The side with minimum absolute difference between its slope and the slope of the sewing line is selected as the desired sewing side of the fabric and the border corner of that side is dened as the second sewing corner. In the rest video images, the Harris corner-detection process is repeated and the sewing corners of the new image are the detected corners closest to the sewing-corners of the previous image.

4. The fuzzy control system

Since modeling the mechanical behavior of fabrics for realtime applications is rather difcult due to their low bending rigidity, an approach based on a fuzzy logic controller (FLC) is developed considering the robustness and the fast response requirements. The block diagram for the control system is shown in Figure 3, where the symbols in parentheses stand for the fuzzy system that controls the orientation of the end-effector. The controller inputs are the position error (er) and the orientation error (eu), which are computed on the 0 0 image, and their time rates (er ) and (eu ). The linear (u) and angular velocity (v) of the end-effector around z-axis are the outputs. The membership functions for the two inputs (Figure 4(a) and (b)) and the output of the system that controls the translation of the fabric towards the needle (Figure 4(c)) are experimentally tuned with various fabrics. Expert knowledge, often aficted with uncertainty, is summarized in the proposition:
The larger the distance/angle and the smaller its change rate is, the faster the fabric should move/rotate.

The fuzzy associative memory (FAM) of the system that controls the translation of the fabric (Table I) is derived after studying the behavior of the human workers towards sewing. For the rule evaluation, the min operator is used and for the aggregation mechanism the max operation is used. The centroid defuzzication method is used to determine the linear and angular velocity. Owing to space limitation, only

(a)

(b)

(c)

Notes: (a) Original grayscale image acquired by the first camera; (b) binary image after thresholding; and (c) binary image after selecting the largest eight-connected object

491

A robotic system based on fuzzy visual servoing Paraskevi Zacharia et al.

Industrial Robot: An International Journal Volume 36 Number 5 2009 489 496

Figure 3 The block diagram of the fuzzy logic control

for handling the fabric and can cope with the uncertainties that arise due to deformations. The proposed controller can handle the possible deformations without changes in its structure achieving the desired accuracy on condition that the seam segment targeted for sewing is undistorted. Figure 1(b) shows the fabric of Figure 1(a), which has been deformed except for the seam segment targeted for sewing. It is clear from Figure 1(b) that the presence of deformations does not affect the control system, despite the fact that the shape of the fabric has signicantly changed. It is worth noting that the intent of this paper deals with buckling in context of achieving a successful seam tracking and not the correction strategy against folding or wrinkling problems. 4.1 The proposed genetic fuzzy logic controller The parameters of each membership function are determined using genetic algorithms (Michalewitz, 1996) as tuning mode (Figure 3) to improve the efciency of the system, to overcome the subjectivity and extensive manual experimentation. The optimum values for the parameters of

the study for the system that controls the translation is presented. An analytical model of the robot and the fabric is not necessary, since no geometrical characteristics are taken into account and there is no need for special mathematical computations for different fabrics. The proposed technique is feature-based, since only the features, r and u, are necessary Figure 4 The membership functions
1 0.8 0.6 0.4 0.2 0 0 50 100 150 200 250 Position error (pixels) (a) Very small Small Medium Large Very large 300 Small Medium Large

1 0.8 0.6 0.4 0.2 0 0

Small

Medium

Large

Degree of membership

Degree of membership

50

100 150 200 250 Position error diagram (pixels/s) (b)

300

1 0.8 0.6 0.4 0.2 0

1 0.8 0.6 0.4 0.2 0

Very small Small

Medium

LargeVery large

Degree of membership

10

20

30 40 50 60 70 Error velocity (pixels/s)

80

90

100

Degree of membership

10

20

30 40 50 60 70 Error velocity (pixels/s)

80

90

100

(c) (d) Notes: (a) The position error; (b) the position error rate; (c) the linear velocity after manual tuning; and (d) the linear velocity after optimization with GAs

492

A robotic system based on fuzzy visual servoing Paraskevi Zacharia et al.

Industrial Robot: An International Journal Volume 36 Number 5 2009 489 496

Table I FAM of the fuzzy system

Position error rate Small Medium Large Small Very small Very small Very small Position error Medium Medium Medium Small Large Very large Very large Large

the output membership functions are determined, so that the total time required for the fabric to reach the needle is minimum and the deviations of the seam segment from the needle and the seam line are less than the predened acceptable limits. The evaluation mechanism. The total time ttotal required for the robot to move the fabric until reaching the needle is given by the summary of the total steps satisfying the constraints: jrj # e1 pixels and juj # e2 , where e1 and e2 are the maximum acceptable error limits for the desired seam quality. Encoding. The problem variables are the parameters dening the shapes of the membership functions of the output of each fuzzy system. The triangular fuzzy sets are dened through three parameters and the trapezoidal fuzzy sets are dened through four parameters. Therefore, the chromosome can be generated by concatenating the parameters of the membership functions using the binary alphabet. For example:
1 11..0 001..0 100..1 000..1 a1 b1 c1 d1 1st fuzzy set ..... ..... ..... a4 b4 c4 4th fuzzy set ..... ..... ..... a2 b2 c2 2nd fuzzy set ..... ..... ..... a3 b3 c3 3rd fuzzy set

two parts: the fuzzy knowledge-based controller (Section 3.1) and an adaptation mechanism, which tunes the output (Figure 3) through a supervisory fuzzy system. The adaptation mechanism resembles to the fuzzy knowledge-based controller in the sense that it has the same inputs (Figure 4(a) and (b)) and the same rule base, but the output is a scaling factor (Figure 5) lying inside an experimentally derived interval [1, 2.5]. The parameters dening the membership functions of the output have been optimized using the genetic algorithm presented in Section 4.1. The scaling factor changes the universe of discourse of the output of the fuzzy knowledge base controller and consequently, the support of each fuzzy set is changed by the same ratio according to:
The larger the distance from the needle and the smaller its change rate is, the greater the increase of the scaling factor.

The alteration of the universe of discourse by a ratio greater than one results in the acceleration of the sewing process. In addition, the AFLC has the advantage of the online tuning.

5. Experimental results
The experiments were carried out using a robotic manipulator with six rotational degrees of freedom (RV4A) and controlled by a PC. The robot is programmed in Melfa-Basic language in Cosirop environment, while the analysis of the visual information is performed with Matlab 7.1. The vision system consists of a Pulnix analog video camera at 768 576 pixels resolution RGB and an analog camera of the same resolution, which are xed above the working table in a vertical position (Figure 6). The vision is transferred to the second camera when the position error becomes less than ten pixels (< 12.3 mm) and the orientation error is less than 58. Using the second camera, the accepted position and orientation error are set equal to ten pixels (<1.389 mm) and 18, respectively. A simple gripper has been designed, so that the arm of the robot is distant from the fabric, as shown in Figure 6. In the experiments, four fabrics with different shape, color and bending rigidity have been used: a red-eece trousers leg, a green-baize sleeve, a blue-jean pocket and a tweed shorts Figure 5 The membership functions of the scaling factor
1 Very small Small Medium Very large Large Degree of membership 0.8

..... ..... 000..0 001..1 a5 b5 c5 d5 5th fuzzy set

The genetic fuzzy logic controller (GFLC) was simulated and tested for various combinations of the genetic algorithm control parameters. The whole procedure is off-line, since the required time for convergence is prohibitive for an online application, and the resulted fuzzy sets of the output is shown in Figure 4(d). Comparing Figure 4(d) to Figure 4(c), the support of the membership function Very Small is broadened in order to resolve the oscillations that appear when the vertex is very close to the needle. The support of the fuzzy set Very Large is narrowed and the membership function becomes more abrupt, which urges the fabric to make greater movements when it is distant from the needle. The supports of the fuzzy sets, Small, Medium and Large have also changed, so that the minimum time is achieved. 4.2 The proposed adaptive fuzzy logic controller Adaptive fuzzy logic controllers (AFLCs) (Passino and Yurkovich, 1998; Patcharaprakiti et al., 2005) provide fast response and can change the fuzzy parameters for improving the control systems performance. An AFLC is also designed for robot handling fabrics. Again, the goal is to alter the supports of the fuzzy sets of the outputs, so that the performance of the system is improved. The proposed AFLC uses only qualitative knowledge of the plant and consists of 493

0.6

0.4

0.2 0 1 1.5 Scaling factor 2 2.5

A robotic system based on fuzzy visual servoing Paraskevi Zacharia et al.

Industrial Robot: An International Journal Volume 36 Number 5 2009 489 496

Figure 6 The experimental stage

forming a loop at one side of the fabric. In some experiments, the fabric presented simultaneously both wrinkles, and folds. First, the FLC, the GFLC and the AFLC are experimentally tested for several times changing the initial arbitrary location of the cloth part on the table. In some experiments, the fabric undergoes wrinkling or partially folding due to buckling and its shape deforms. Table II presents the mean absolute values for the steps required to reach the needle and the position and orientation error for the four different fabrics with and without deformations when reaching the needle, which are derived from 15 iterations for each tested case starting from a different location on the table. The total steps needed for the robot to move the fabric until its edge reaches the needle are decreased using the GFLC and the AFLC compared to the simple FLC. For example, the total steps for the case of the sleeve have been decreased using the GFLC by: 14:5842 2 11:6667 100% < 20% 14:5842 while they have been decreased by: 14:5842 2 10:0000 100% < 31:4% 14:5842 using the AFLC. However, no direct comparison between GFLC and AFLC concerning the performance is possible due to the fact that the GFLC has been tuned off-line through simulation tests, where some factors affecting the system, such as the camera calibration and the robot accuracy, are ignored. Comparing the experiments carried out using fabrics with and without deformations, only slight changes are observed in the mean total steps, the mean absolute position and orientation error. Consequently, the existence of deformations on the fabrics does not degrade the controller performance and does not decelerate the sewing process. This is the experimental proof that the FLC exhibits exibility and robustness against deformation, since the performance of the controller is not affected by possible deformations. In the following, some indicative diagrams are presented for the total sewing process of the pocket using the GFLC. In Figure 8(a), the position error is presented while the robot moves from the random initial location towards the needle.

leg. The curved edges of the cloth parts are approximated with straight lines as shown in Figure 7, since this paper focuses on the handling of fabrics with straight edges. The fabrics that are selected for the handling experiments are prone to small or relatively large deformations due to buckling, such as wrinkling or even folding. In our work, buckling modes during robot handling are supportable on the condition that it does not appear in the segment to be sewed. However, it still remains a problem that should be avoided. Thus, the position where the gripper touches the fabric is estimated in terms of reducing the possibilities for deformation appearance, taking into account the properties of the fabric (Koustoumpardis and Aspragathos, 2003). It is worth noting that the intent of this paper deals with buckling in context of achieving a successful seam tracking and not the correction strategy against folding or wrinkling problems. The experimental tests also showed that deformations are likely to appear close to the gripper position on the fabric, and not on the edge. Since there is no standardization for deformations, it is difcult to be quantied. However, the greater deformation observed in buckling is about 30 mm, which is the height between the highest and the lowest point of the fabric. In some cases, fabrics were partially folded. In other cases, the wrinkles induced the fabric to fold due to its own weight Figure 7

(a)

(b)

(c)

(d)

Notes: (a) Sleeve; (b) trousers leg; (c) pocket; and (d) shorts leg

494

A robotic system based on fuzzy visual servoing Paraskevi Zacharia et al.

Industrial Robot: An International Journal Volume 36 Number 5 2009 489 496

Table II Experimental results

Controller/condition FLC/No deformations FLC/Deformations GFLC/No deformations GFLC/Deformations AFLC/No deformations AFLC/Deformations FLC/No deformations FLC/Deformations GFLC/No deformations GFLC/Deformations AFLC/No deformations AFLC/Deformations FLC/No deformations FLC/Deformations GFLC/No deformations GFLC/Deformations AFLC/No deformations AFLC/Deformations FLC/No deformations FLC/Deformations GFLC/No deformations GFLC/Deformations AFLC/No deformations AFLC/Deformations Fabric type Sleeve Sleeve Sleeve Sleeve Sleeve Sleeve Trousers leg Trousers leg Trousers leg Trousers leg Trousers leg Trousers leg Pocket Pocket Pocket Pocket Pocket Pocket Shorts leg Shorts leg Shorts leg Shorts leg Shorts leg Shorts leg Mean total steps 14.5842 14.2277 11.6667 11.6667 10.0000 8.6667 14.0867 14.8333 10.8333 11.3333 12.1667 11.8666 13.9733 13.9584 10.5000 10.0000 11.0000 10.6667 12.7582 12.3948 11.6667 11.6667 8.6667 9.0000 Mean absolute position error (pixels) 4.8333 5.6667 5.3333 4.6667 7.3333 5.3333 6.6667 8.0000 6.0000 7.0000 8.8333 8.6667 3.3333 7.0000 5.6667 6.0000 6.1667 8.8333 5.3333 6.8333 6.5000 6.0000 6.5000 6.8333 Mean absolute orientation error (degrees) 0.2924 0.2809 0.3210 0.3038 0.5274 0.5389 0.5790 0.1777 0.5446 0.2121 0.2350 0.2064 0.3898 0.2112 0.1949 0.5159 0.2293 0.3210 0.2006 0.2178 0.3497 0.2809 0.3325 0.3841

Figure 8 (a) Position error and (b) orientation error versus steps for the movement towards the needle
400 350 Position error (mm) 300 250 200 150 100 50 0 1 2 3 4 5 Steps (a) 45 40 35 30 25 20 15 10 5 0 1 2 3 4 5 Steps (b) 6 7 8 9 10 6 7 8 9 10

The change between the cameras happens at the seventh step, where a slight increase in the position error is observed due to the higher resolution of the second cameras image. The position error in the last step is three pixels (< 0.4167 mm), which is less than the accepted position error of ten pixels. In Figure 8(b), the orientation error decreases gradually until it reaches 0.00578, which is a really satisfactory error. The sewing process is accomplished in eight steps, shown in Figure 9. The absolute orientation error varies between 0.0229 and 1.14658 and it is corrected at each step. In the nal phase, the fabric is rotated around the needle until the orientation error becomes 0.75678, which is lower than the predened acceptable limit of 18. As a concluding remark, the achieved errors are very satisfactory bearing in mind that the robot accuracy is within ^ 0.03 mm. Figure 9 Orientation error versus steps for the sewing process
1.2 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0.1 1

Orientation error ()

Orientation error ()

4 Steps

495

A robotic system based on fuzzy visual servoing Paraskevi Zacharia et al.

Industrial Robot: An International Journal Volume 36 Number 5 2009 489 496

6. Conclusions
The main focus of this work lies on the design of an innovative visual servoing manipulator controller based on fuzzy logic that aims to enable robots to handle exible fabric sheets lying on a work table. The designed visual servoing control system can deal with a variety of fabrics that are likely to deform and can cope with possible deformations due to buckling (wrinkling and folding) towards handling without degrading its performance, on condition that the seam segment to be sewed is undistorted. The desired accuracy is achieved in approaching the needle even in cases where deformations appeared. Besides the conventional fuzzy controller, which is manually tuned, a GFLC and an AFLC are proposed and tested for different fabrics. The experimental results show that the proposed approach is effective and efcient in guiding the fabric towards the sewing needle, sewing it and rotating it around the needle. After extensive experimentation, it has been proved to be rather simple, exible and robust due to its capability to respond to any position and orientation error for a variety of fabrics that are likely to present deformations. Since it is not possible to cover all the aspects of robot handling of exible materials in this paper, there are still several related issues requiring solutions. Considering our future research work, the proposed approach can be extended to sew fabrics with curved edges and correcting the distortions presented during robot handling of fabrics.

References
Ayach, N. and Faugeras, O. (1986), HYPER: a new approach for the recognition and positioning of two dimensional objects, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 8 No. 1, pp. 44-54. Corke, P.I. (1996), Visual Control of Robots High Performance Visual Servoing, Research Studies Press, Taunton. Deriche, R. and Giraudon, G. (1990), Accurate corner detection: an analytical study, Proceedings of 3rd International Conference on Computer Vision, Osaka, pp. 66-70. Fahantidis, N., Paraschidis, K., Petridis, V., Doulgeri, Z., Petrou, L. and Hasapis, G. (1997), Robot handling of at textile materials, IEEE Robotics & Automation Magazine, Vol. 4 No. 1, pp. 34-41. Gershon, D. (1990), Parallel process decomposition of a dynamic manipulation task: robotic sewing, IEEE Transactions on Robotics and Automation, Vol. 6 No. 3, pp. 357-66. Gershon, D. (1993), Strategies for robotic handling of exible sheet material, Mechatronics, Vol. 3 No. 5, pp. 611-23. Gershon, D. and Porat, I. (1986), Robotic sewing using multi-sensory feedback, Proceedings of the 16th International Symposium of Industrial Robots, Brussels, pp. 823-34. Gershon, D. and Porat, I. (1988), Vision servo control of a robotic sewing system, Proceedings of the IEEE International

Conference on Robotics and Automation, Philadelphia, PA, pp. 1830-5. Haralick, R.M. and Linda, G.S. (1992), Computer and Robot Vision, Vol. I, Addison-Wesley, Reading, MA, pp. 28-48. Hutchinson, S., Hager, G.D. and Corke, P.I. (1996), A tutorial on visual servo control, IEEE Transactions on Robotics and Automation, Vol. 12 No. 5, pp. 651-70. Koch, M.W. and Kashyap, R.L. (1987), Using polygons to recognize and locate partially occluded objects, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 9 No. 4, pp. 483-94. Koustoumpardis, P. and Aspragathos, N. (2003), Fuzzy logic decision mechanism combined with a neuro-controller for fabric tension in robotized sewing process, Journal of Intelligent and Robotic Systems, Vol. 36 No. 1, pp. 65-88. Koustoumpardis, P., Zoumponos, G., Zacharia, P., Mariolis, I., Chatzis, I., Evangelidis, G. and Zampetakis, A. (2006), Handling of non-rigid materials (XROMA), application in robotic sewing, paper presented at 37th International Symposium on novelties in Textiles, Ljubljana. Kudo, M., Nasu, Y., Mitobe, K. and Borovac, B. (2000), Multi-arm robot control system for manipulation of exible materials in sewing operation, Mechatronics, Vol. 10 No. 8, pp. 371-402. Michalewitz, Z. (1996), Genetic AlgorithmsData Structures Evolution Programs, 3rd ed., Springer, New York, NY. Moulianitis, V., Dentsoras, A. and Aspragathos, N. (1999), A knowledge-based system for the conceptual design of grippers for handling robots, Articial Intelligence for Engineering Design, Analysis and Manufacturing, Vol. 13 No. 1, pp. 3-25. Passino, K.M. and Yurkovich, S. (1998), Fuzzy Control, Addison-Wesley, Reading, MA. Patcharaprakiti, N., Premrudeepreechacharn, S. and Sriuthaisiriwong, Y. (2005), Maximum power point tracking using adaptive fuzzy logic control for grid-connected photovoltaic system, Renewable Energy, Vol. 30 No. 11, pp. 1771-88. Ridler, T.W. and Calvard, S. (1978), Picture thresholding using an iterative selection method, IEEE Transactions on System, Man and Cybernetics SMC-8, pp. 630-2. Torgerson, E. and Paul, F.W. (1988), Vision-guided robotic fabric manipulation for apparel manufacturing, IEEE Control Systems Magazine, Vol. 8 No. 1, pp. 14-20. Vernon, D. (1991), Machine Vision, Prentice-Hall, New York, NY, pp. 34-6. Zoumponos, G. and Aspragathos, N. (2008), Fuzzy logic path planning for the robotic placement of fabrics on a work table, Robotics & Computer-integrated Manufacturing, Vol. 24 No. 2, pp. 174-86.

Corresponding author
Paraskevi Zacharia can be contacted at: zacharia@mech. upatras.gr

To purchase reprints of this article please e-mail: reprints@emeraldinsight.com Or visit our web site for further details: www.emeraldinsight.com/reprints

496