2011 International Conference on Pattern Analysis and Intelligent Robotics 28-29 June 2011, Putrajaya, Malaysia

Robot Soccer Vision: An Overview for New Learner

Najmi Hafizi Bin Zabawi1, Khairuddin Omar2
1 2

Electrical Engineering Department, Politeknik Sultan Haji Ahmad Shah, Pahang, Malaysia Faculty of Information Science And Technology, National University Of Malaysia, Malaysia
1

hafizi@polisas.edu.my 2 ko@ftsm.ukm.my

Abstract Vision is a part of a robot soccer system. It completes the system by providing the eye to see what occurs inside the pitch during the game. Similarly as in humans, the sight is then sent to the brain. The control system needs to be processed before making any decision to be sent back to robots on the field to react. The process will be repeated over and over again. This paper discusses the robot soccer vision system which deals with surface theory and some examples of applications used. Suggestions for future improvement are included at the end of this paper. Keywords image calibration, image processing, vision system, image filter, object recognition

II. CURRENT DEVELOPMENT A. Colour calibration Colour calibration is important as a robot soccer game is most dependent on the colour recognition application in determining a robots team members. Every robot is mounted with a colour tag designed to distinguish each of them by way of their roles; i.e. those of goalkeeper, striker and defender. The colour tag may be designed in various formats such as quadrilateral, oblique and round colour patches [3][4][16]. Although they may be different in design, they have one thing in common - the colour of patches used. Every colour tag must consist of that teams identifying colour such as blue or yellow. This is to avoid confusion and tracking mistakes to each teams vision system. In this case, colour space plays the important role in this colour determination task. Colour space comes from different colour models such as RGB, YUV and HSV. Researchers commonly use these colour models according to their research requirement. Some of them prefer to use RGB, some YUV and the rest HSV. All these models are used by the computer to process images, whereby each of them has their own advantages as compared to each other. B. Image processing Image processing is the step where images captured by the video camera are processed by computer. The objective is to enable the vision system to recognize target objects such as the ball and robots on the game field. In this step, the image frames usually undergo several filtering processes to distinguish between desired and undesired object images. The common image filtering process terms include RaoBlackwellised particle filters [5][6], Kalman filter [7][8], Gaussian filter, Sobel filter [9], Laplacian [10], morphological filters [11], Bayesian [12], Monte Carlo filter [13], and Canny [14]. C. Object recognition Object recognition refers to the task of the vision system in identifying the object which has been obtained from the previous filtering task. The techniques that the system uses in object recognition process include colour based identification

I. INTRODUCTION One of the main elements in robot soccer is the vision system. Robot soccer uses a visual imaging method to track objects in order to play the soccer game. In the vision system, the components involved are colour calibration, image processing and object recognition. Colour calibration is about setting up the camera and colour elements to perform the best image capturing quality to feed to the image processing stage. Image processing is the task that is carried out by the robot soccer system to process the images that are obtained from a CCD camera. A CCD (Charge Coupled Device) is chosen for robot soccer as it is stable, portable and accurate besides other advantages [1]. Object recognition is the next stage performed on what has been yielded by the image processing task. During the image processing stage, the image is filtered using algorithms and the result is channelled as an output to the object recognition process in the decision-making stage.

Fig. 1 Role of the vision system in the robot soccer system as carried out by the overhead camera [2]

978-1-61284-406-0/11/$26.00 2011 IEEE IR-06 125

[10][15], environment based [10], behaviour based [17] and pattern based recognition [18]. III. GENERAL ARCHITECTURE A. Colour models Colour calibration depends much on colour space analysis. Colour spaces that are popular in robot soccer games come from different colour models such as RGB, YUV and HSV. The RGB (red, green, blue) model consists of three basic colours which exist in the natural world. The RGB colour format is usually used by a computers graphic card to capture images from a camera but the biggest disadvantage of this model is that it is not suitable to the human eye [19]. This is because the values of the components R, G and B are highly inter-related and cannot be understood by bare eyes. Furthermore, the components are very sensitive to environmental conditions such as illumination and light intensity which will affect the stability of the components values [8]. That is why researchers tend to use other colour models in performing their research. The YUV colour model on the other hand comes from the RGB colour model, in which component Y represents brightness or luminosity, while the colour information is represented by U for hue and V for saturation [20][21][22]. HSV (hue, saturation, value) is the most popular colour model used by programmers to represent the colour data in computers as the components are more familiar to humans than the previous colour model components [23]. Most of RGB data that is collected from the camera is converted first into the HSV format before other processes are executed [24]. Hue has a magnitude such that HSV colour space is sometimes called as HSI where I represent intensity replacing the V for the value component.

jerseys which use yellow and red respectively. The left image shows the jersey with the red almost similar to the right one, but looking at their HSV values, the left image has a higher Hue value compared to the right one, while its Saturation value is lower. However, both patches preserve the same value for V or the Value component.

Fig. 3 Robot soccer patch sample colour comparison; Left (Reds HSV : 238, 205, 125) Right (Reds HSV : 2, 232, 125)

B. Image filters

Among the purposes of image processing practice are cleaning noise from the concerned image, improving contrast between neighbouring regions or features, applying smoothing or elimination of features at certain scales and keeping only features at a wanted scale [27]. A filter is a program dedicated to process the raw information as the input and the information of interest as the output [28]. In short, the main purpose of a filter is to purify an image of the unwanted noise that may affect the scene originality of the image. There are many image filtering techniques available and they are extensively used by scholars to perform their research. Among the filters is the Bayes filter which refers to a method of applying prediction cycle to forecast the state of a method of using a time-dependent system from sensor measurement. Two types of Bayes filters are the Kalman filter and the particle filter [29]. A Kalman filter is an algorithm of data processing that performs recursively and which predicts the condition of a noisy linear dynamic system. The filter was found by Richard Kalman in 1960 and some believe that the discovery is one of the greatest discoveries in statistical estimation theory in the twentieth century [30][31]. This most widely used filter is basically chosen by many researchers for its ability to predict the future state of a dynamic system variable which is difficult for people to control [30]. In image processing practice, the Kalman filter works efficiently with image data that is represented by the Gaussian function a function that is used to reduce image noise and detail [30]. However, for Fig. 2 HSV colour space model [19] information that contains non-linear orientation data, the The HSV model can be represented as an inverted cone as Kalman filter needs to be modified to cater for the nonshown in Fig. 2 above. In the cone, the 360 circle denotes hue, linearity attributes, which becomes known as the Extended radius from 0 to 1 denotes saturation; purity being the most at Kalman filter (EKF). Even so, the EKF still has two 0 and least at 1, and the cones vertical axis represents the significant drawbacks, which are implementation difficulty dark and the bright sides of the model. Hue is a property that and instability of the filter. This situation has led to the relates to the dominant wavelength in the light spectrum [26]. invention of the Unscented Kalman filter (UKF) [31]. The Particle filter basically is a Bayesian based filter that Hue represents the type of colour that a human can understand. For example Fig. 3 shows two colour patches of robot soccer processes a sequence of action and perception as the input and outputs the belief distribution of the subject that acts and

126

perceives [28]. It works by representing the required posterior probability density function by a set of random samples with associated weights and to compute estimates based on these samples and weights [32].

two groups, which are gradient-based and Laplacian based. Under the gradient based group, Sobel, Canny, Roberts and Prewitt are the examples [35] while under the Laplacian based group, Marr-Hildreth, Laplacian of Gaussian etc. are placed. The first group focuses on finding the minimum and maximum values on an image to detect edges, during the first derivative of the image. The Laplacian group on other hand detects edges by finding zero crossings in an image during the second derivative of the image [36]. The derivatives are represented by some equation algorithms that are not discussed in this paper. Fig. 5 demonstrates the examples that result from applying the two edge detection methods that are discussed above.

Fig. 5 Left: Input image, Middle: Gradient method, Right: Laplacian method [36]

(a)

(b)

Fig. 4 Comparison of algorithms for nonlinear image filtering [33]

(c)

(d)

The Morphological filter is an evolution of mathematical morphology which focuses on identifying the geometrical structure of image objects. One of the advantages of using a morphological filter in image processing is it can solve a nonlinear task such as a non-Gaussian noise function [27]. It smooth out object outlines by filling small holes which eliminates small projections [25]. Another popular image processing technique is the edge detection method. Edge detection is a process of extracting edges from an image. Researchers have found that the most important data lies in its edges. The edges will form the object boundaries on an image. Edges are identified as the changes of gray value from a pixel to the next pixel of an image. By only focusing on edges, other redundant pixel information on an image can be ignored and this will save time while preserving the significant structural attributes of the image [34]. There are several methods of performing edge detection, with the majority of them being able to be categorized into

Fig. 6 Significance of morphological filter on noisy image [25]

Some researchers apply both morphological and edge detection methods to filter out their images. As demonstrated by Mohamed Roushdy (2006) in his journal, the noisy image that is shown in Fig. 6 (a) will be as in (c) after the Sobel filter is applied while on the other hand it will be as in (d) if the image is applied with a morphological filter first as shown in (b) [25]. The Sobel filter is used to perform the edge detection process. This method works by assuming that an edge happens when there is a sudden intensity gradient in the image [34]. In the Sobel filtering procedure, the first step is to measure the edge tendency value of each pixel in vertical and horizontal axes. With a threshold value being applied, the pixel that is above the threshold value will be set to an edge value. Then the process will be repeated for the rest of the unset pixels until no pixels are left unchecked [36].

127

C. Object recognition

IV. FUTURE DIRECTION To improve the performance e in the vision system of robot i the use of soccer, several things can be do Colour-based identification techniques includes one. First, we can use the face various colour patch designs such as quadril lateral, oblique and recognition technique in identif fying and locating the robot on round colour patches [3][4][16]. The colour r model approach is the pitch. The robots are no longer using colour patches, but sometimes used in colour based recognition tasks [15]. For use face pictures instead. A sample program has been en chosen by Ren developed using OpenCV whic example, the HSV colour space has bee ch is able to track faces using Honge et al. to perform object recognition work using colour Haar-like features and it has proven p its ability to track both ction technique [23] human faces anda comical face as well as shown in Fig. 7 [43]. segmentation combined with the edge detec while YUV colour space is preferred by Genichi Yasuda The advantage of this tracking g method is that the complex and Bin Ge for the same purpose [38]. Fa anny F. L. Tong et filtering process that is comm monly carried out on a colour al. proved that YUV colour space is better compared to RGB image can be avoided since th his method can not only track n when the targeted colour images, but also black and colour space in terms of object recognition a white ones. Hence, in the object is exposed to different light condition ns [22]. future we can implement a new w kind of robot jersey that no The Hough transform is a method which h can also be used. longer uses colour patches, but only face similar pictures to The Hough transform (HT) is a technique which can be used perform robot localization. to separate features of a particular shape in an image [39]. It is ms associated with an algorithm that is able to solve problem line detection and definition. It was introduc ced by Paul Hough in 1962 and patented by IBM [40]. Origina ally, its application was in particle physics, in which it was use ed for detecting the lines and arcs in the pictures captured in clo oud chambers [41]. Basically, Hough transform is used as preprocessing method before an image is further processed [39]. The Hough transform algorithm is best if used to detect a curve or a circle. eir research to track Hui Zhang et al. has proven its ability in the a robot soccer ball even without botheri ing about the ball colour. They used the Sobel filter first to detect edge points plying the Hough and the direction of gradient before app transform algorithm [9]. Besides the colou ur space technique, the combination of neural network and fuzz zy logic techniques can also be applied in this recognition task [42]. [ Fig. 6 below is the summary chart of al ll filtering types or methods that have been discussed above. Each technique is ithout emphasizing placed according to its type and group wi much on their specific function and charac cteristic; enough to give a new learner an overview of the tech hniques available in the image processing field that are related to the robot soccer world.
Image Filtering Methods

Bayesian

Morphology

Hough Transform (for line detection)

Edge Detection

Fig. 7 Face tracking program trac cks face-like features. Not only can it track a real face image, but the comica al hand- drawn face as well. The facelike pictures can be mounted on socce er robots, replacing the normal colour patch je erseys.

Linear / Gaussian

Non-linear

Laplacianbased

Gradientbased

Kalman Filter (KF)

Unscented KF, Extended KF

Particle Filter

MarrHildreth, Laplacian of Gaussian

Sobel, Roberts, Prewitt

Canny (for gray scale)

Monte Carlo

Rao Blackwellised

Markov

Fig. 6 Placement of the mentioned image filtering ap pproaches in the image processing field of study

Secondly, the ability of th he system to track objects in different light conditions needs s to be enhanced. This means that the vision system should be able to recognize the field hout much human calibration pitch, robots and the ball with intervention. This can be realize ed by applying fuzzy logic and a neural network algorithm tha at enable the vision system to learn and identify the desir red objects time after time, regardless how the light conditions are. It means the learning process is not only subjected to being confined within one t games it experiences in the game, but can be based on all the larger scale. The recognition data d of the objects should be stored to be used in the next ga ame and eventually the system

128

will adapt to different challenging conditions to yield a smart robot soccer system. V. CONCLUSION In this paper, the non-physical elements that are related to the robot soccer vision system have been discussed. The elements are important as without them, it is impossible for the robot soccer system to operate properly. The elements are colour calibration, image processing and object recognition. Every element has been discussed along with its theory and application.
REFERENCES [1] Nascimento, T.P.; Lima, R.S.; Ludwig, O.; Brito, L.; , "Supervisory Control System for Cooperative Agents," Electronics, Robotics and Automotive Mechanics Conference, 2006 , vol.2, no., pp.182-187, Sept. 2006. Yen-Hsun Wu; Han-Pang Huang; , "Robust color classification for global soccer vision," Mechatronics, 2005. ICM '05. IEEE International Conference on , vol., no., pp.439-444, 10-12 July 2005. Lee, D.; DoEun Kim; KyungHun Hwang; ChaeWook Chung; TaeYong Kuc; , "A Novel Color patch System for the Large League MIROSOT," SICE-ICASE, 2006. International Joint Conference, vol., no., pp.576-580, 18-21 Oct. 2006. Jun Zhou; Kui Li; Ting-qi Wang; , "Algorithm of compensating loss of round color mark based on circle fitting," Mechatronics and Automation, 2005 IEEE International Conference , vol.3, no., pp.13971401 Vol. 3, 2005. Messom, C.H.; Sen Gupta, G.; Demidenko, S.; Lim Yuen Siong; , "Improving predictive control of a mobile robot: Application of image processing and Kalman filtering," Instrumentation and Measurement Technology Conference, 2003. IMTC '03. Proceedings of the 20th IEEE , vol.2, no., pp. 1492- 1496 vol.2, 20-22 May 2003. Schulz, D.; Fox, D.; , "Bayesian color estimation for adaptive visionbased robot localization," Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings. 2004 IEEE/RSJ International Conference on , vol.2, no., pp. 1884- 1889 vol.2, 28 Sept.-2 Oct. 2004. Coltin, B.; Liemhetcharat, S.; Merili, C.; Junyun Tay; Veloso, M.; , "Multi-humanoid world modeling in Standard Platform robot soccer," Humanoid Robots (Humanoids), 2010 10th IEEE-RAS International Conference on , vol., no., pp.424-429, 6-8 Dec. 2010. Ma Gang, Liu Tian-shi, Han Jia-xin, Wang Xiao-xiao, "The Color Tag Design and Color Model Study in Mirosot," aici, vol. 1, pp.72-76, 2010 International Conference on Artificial Intelligence and Computational Intelligence, 2010. Huimin Lu; Hui Zhang; Shaowu Yang; Zhiqiang Zheng; , "Visionbased ball recognition for soccer robots without color classification," Information and Automation, 2009. ICIA '09. International Conference on , vol., no., pp.916-921, 22-24 June 2009 Yasuda, G.; Ge, B.; , "Vision based environment recognition and movement control for autonomous wheeled mobile robots," Mechatronics and Automation, 2009. ICMA 2009. International Conference on , vol., no., pp.1083-1088, 9-12 Aug. 2009 Figueroa, P.; Leite, N.; Barros, R.M.L.; Cohen, I.; Medioni, G.; , "Tracking soccer players using the graph representation," Pattern Recognition, 2004. ICPR 2004. Proceedings of the 17th International Conference on , vol.4, no., pp. 787- 790 Vol.4, 23-26 Aug. 2004. Nascimento, T.P.; Lima, R.S.; Ludwig, O.; Brito, L.; , "Supervisory Control System for Cooperative Agents," Electronics, Robotics and Automotive Mechanics Conference, 2006 , vol.2, no., pp.182-187, Sept. 2006. Wei Li; Yannan Zhao; Yixu Song; Zehong Yang; , "A Monte-Carlo based stochastic approach of soccer robot self-localization," Human System Interactions, 2008 Conference on , vol., no., pp.915-920, 25-27 May 2008. Maini, R. & Aggarwal, H., Study and Comparison of Various Image Edge Detection Techniques, International Journal of Image Processing (IJIP), Volume (3) : Issue (1).

[15]

[16]

[17]

[18]

[19]

[2] [3]

[20]

[21]

[4]

[22]

[5]

[23]

[6]

[24]

[7]

[25] [26]

[8]

[27] [28]

[9]

[10]

[29]

[11]

[30] [31] [32]

[12]

[13]

[33] [34]

[14]

Da-Lei Song; Lei-Hua Ge; Wei-Wei Qi; Ming Chen; , "Illumination invariant color model selection based on genetic algorithm in robot soccer," Information Science and Engineering (ICISE), 2010 2nd International Conference on , vol., no., pp.1245-1248, 4-6 Dec. 2010. Ching-Chang Wong; Chun-Po Huang; , "Role construction and recognition in robot soccer games," Networking, Sensing and Control, 2004 IEEE International Conference on , vol.2, no., pp. 1115- 1120 Vol.2, 2004. Takahashi, Y.; Tamura, Y.; Asada, M.; , "Behavior development through interaction between acquisition and recognition of observed behaviors," Fuzzy Systems, 2008. FUZZ-IEEE 2008. (IEEE World Congress on Computational Intelligence). IEEE International Conference on , vol., no., pp.1518-1523, 1-6 June 2008. Fathzadeh, Ramin; Mokhtari, Vahid; Mousakhani, Morteza; Mahmoudi, Fariborz; , "Mining Opponent Behavior: A Champion of RoboCup Coach Competition," Robotics Symposium, 2006. LARS '06. IEEE 3rd Latin American , vol., no., pp.80-83, 26-27 Oct. 2006 Ren Honge; Zhong Qiubo; Kang Junfeng; , "Object recognition algorithm research based on variable illumination," Automation and Logistics, 2009. ICAL '09. IEEE International Conference on, vol., no., pp.1609-1613, 5-7 Aug. 2009. Bin Ge; Gen'ichi Yasuda; Fuliang Yin; Hongwei Zhao; , "Object Recognition and Self-Localization for Interactive Soccer Robots," Intelligent Control and Automation, 2006. WCICA 2006. The Sixth World Congress on , vol.2, no., pp.10245-10250, 0-0 0. Chung-Hsien Kuo; Chun-Ming Yang; Fang-Chung Yang; , "Development of intelligent vision fusion based autonomous soccer robot," Mechatronics, 2005. ICM '05. IEEE International Conference on , vol., no., pp.124-129, 10-12 July 2005. Tong, F.F.L.; Chong, Z.Y.M.; Meng, M.Q.-H.; , "Sensor Fusion and Play Strategy Programming for Micro Soccer Robots," Robotics and Biomimetics, 2004. ROBIO 2004. IEEE International Conference on, vol., no., pp.833-837, 22-26 Aug. 2004 Ren Honge; Zhong Qiubo; Kang Junfeng; , "Object recognition algorithm research based on variable illumination," Automation and Logistics, 2009. ICAL '09. IEEE International Conference on , vol., no., pp.1609-1613, 5-7 Aug. 2009. Xiaovvei Han; Junsheng Li; Yanping Li; Xinhe Xu; , "An Approach of Color Object Searching for Vision System of Soccer Robot," Robotics and Biomimetics, 2004. ROBIO 2004. IEEE International Conference on , vol., no., pp.535-539, 22-26 Aug. 2004 Roushdy, M. Comparative Study of Edge Detection Algorithms Applying on the Grayscale Noisy Image Using Morphological Filter, GVIP Journal, Volume 6, Issue 4, December, 2006. C. Amoroso, E. Ardizzone, V. Morreale, P. Storniolo, "A New Technique for Color Image Segmentation," iciap, pp.352, 10th International Conference on Image Analysis and Processing (ICIAP'99), 1999. P. Maragos & L. Pessoa: Chapter for The Image and Video Processing Handbook, Acad. Press. Yi Fen, Particle Filter and Monte Carlo Localization, Department of Computer Science, The Graduate Center, CUNY. (http://www-ee.ccny.cuny.edu/www/web/jxiao/particle-filters-Yi1.pdf) Kaijen Hsiao, Henry de Plinval-Salgues, Jason Miller. Particle Filters and Their Applications. Cognitive Robotics, April 11, 2005. (http://web.mit.edu/16.412j/www/html/Advanced%20lectures/Slides/H saio_plinval_miller_ParticleFiltersPrint.pdf) Negenborn R. 2003. Robot Localization and Kalman Filters On finding your position in a noisy world. Masters Thesis, Utrecht University. S. G. Mohinder, P. A. Angus, 2001. Kalman Filtering: Theory and Practice Using MATLAB, Second Edition. John Wiley & Sons. Wenyan Guo; Chongzhao Han; Ming Lei; , "Improved unscented particle filter for nonlinear bayesian estimation," Information Fusion, 2007 10th International Conference on , vol., no., pp.1-6, 9-12 July 2007. Daum, F.; , "Nonlinear filters: beyond the Kalman filter," Aerospace and Electronic Systems Magazine, IEEE , vol.20, no.8, pp.57-69, Aug. 2005. O. R. Vincent, O. Folorunso, A Descriptive Algorithm for Sobel Image Edge Detection, Proceedings of Informing Science & IT Education Conference (InSITE) 2009.

129

[35]

[36] [37]

[38]

[39]

[40] [41] [42]

[43]

Barkhoda, W.; Tab, F.A.; Shahryari, O.-K.; , "Fuzzy edge detection based on pixel's gradient and standard deviation values," Computer Science and Information Technology, 2009. IMCSIT '09. International Multiconference on , vol., no., pp.7-10, 12-14 Oct. 2009. Zrimec T., Toward Automatic Colour Calibration Using Machine Learning, IEEE International Conference on Industrial Technology, 2003. LaViola, J.J., Jr.; , "A comparison of unscented and extended Kalman filtering for estimating quaternion motion," American Control Conference, 2003. Proceedings of the 2003, vol.3, no., pp. 2435- 2440 vol.3, 4-6 June 2003. Bin Ge; Gen'ichi Yasuda; Fuliang Yin; Hongwei Zhao; , "Object Recognition and Self-Localization for Interactive Soccer Robots," Intelligent Control and Automation, 2006. WCICA 2006. The Sixth World Congress on , vol.2, no., pp.10245-10250, 0-0 0. Brahim Nini , Brahim Mehelain , Bilel Flifel . Agent-based Hough Transform: A Way to the Improvement of the Execution Time in the Detection of the Dominant Straight Line in an Image. International Journal of Computer Applications (0975 8887)Volume 11 No.2, December 2010. Kyewook Lee, Application Of The Hough Transform, University of Massachusetts, Lowell. January 2006. Danko Antolovic, Review of the Hough Transform Method, With an Implementation of the Fast Hough Variant for Line Detection Hung-Ching Lu; Cheng-Hung Tsai; , "Image Recognition Study via the Neural Fuzzy System," Intelligent Engineering Systems, 2006. INES '06. Proceedings. International Conference on , vol., no., pp.222-226, 0-0 0. Bradski, G. & Kaehler A. (2008). Learning OpenCV Computer Vision with OpenCV Library. O Reilly.

130