Journal of Information, Control and Management Systems, Vol.11, (2013), No.

35

LASER SPOT DETECTION

Matej MEKO, tefan TOTH University of ilina, Faculty of Management Science and Informatics, Slovak Republic e-mail: mesko@kst.uniza.sk, stefan.toth@fri.uniza.sk Abstract The paper proposes method for detection of laser spot emitted by a laser pointer. A camera captures projection screen on which is displayed image from a digital projector and also laser spot from a laser pointer. Our developed algorithm can effectively detect red and green laser spot in real-time using image processing and by setting up camera to be captured high intensity of laser beam. To demonstrate the workings of our method we developed a simple game similar to Duck Hunt game in which main goal is to shoot a duck by laser pointers. Keywords: laser pointer, laser pen, detection, duck hunt game, image processing 1 INTRODUCTION Laser pointer (or laser pen) has become a common tool used during a presentation. It is a small device that emits an intense beam of light over a long or narrow distance. One can use it to highlight something for example to point out important parts on a slide of presentation. In [1] authors used it for guidance using librarian robot. The robot points out position or direction to a target. Other usage presents [2] shooting simulator using photodiode sensors to detect presence of laser. Detection and localization of the laser spot on a projection screen using a camera can be an interesting task. We could use laser pointer as a mouse [3] or use it to execute some actions based on gesture recognition (run an application, move to next/previous slide, etc.). Methods for detection of laser spots can be various. In [4] authors used template matching and genetic algorithms. Other approach uses frequency-demodulation CMOS sensor [3]. Authors in [5] even implemented algorithms on hardware using FPGA. In this paper we would like to propose our method for laser spot detection using proper camera settings on the projection screen. The following sections describe issues and restrictions produced by a camera. Then we present the detection algorithm of laser spot and a game that demonstrates the workings of the implemented algorithm.

36

Laser Spot Detection

LASER POINTER DETECTION Laser pointers may have various colors of the laser beam (the most common is red or green color). Therefore a detector of laser spot can focus on color values of pixels. It searches pixels in input image that contains color in specified range. The main issues of laser spot detection on projection screen bring about a change of light condition during application execution, projection of animations, projector power, laser pointer power, and quality of camera (image sensor, image resolution and camera FPS). However the main problem causes the camera chip. The chip is not as sensitive as the human eye. For example a laser spot is seen by human eye as red dot, while the camera see it as white. This is due to the camera chip, which cannot process higher light intensity because signal is truncated and turned into highest possible value i.e. white color. This problem can be partially reduced by adjusting of the camera exposition. Reducing exposition of the camera reduces time during scene is captured. So only the strongest sources of light will be captured. 2.1 Laser color Correct exposure configuration of the camera can improve color detection of laser spot. Figure 1b shows a laser spot captured by a camera at correct exposure settings. As you can see only the edge of the laser spot contains information about color since middle of laser spot is too much brightness. So to determine color of laser spot we can use pixels near to the brightness middle (Figure 1c gray area). Size of this area depends on the ambient light conditions, exposition configuration and power of laser source.

Figure 1 Appearance of red laser spot a) at incorrect exposure settings b) at correct exposure settings c) zoomed in at correct exposure settings (white color represents pixels with high value in HSV value channel, gray area is used for recognition of laser color) 2.2 EmguCV and OpenCV We chose the .NET framework and C# programing language to create a test application since allow easy and rapid development any application. In order to image processing we used EmguCV library. It wraps OpenCV library written in C/C++, so it can be used in various .NET programing languages.

Journal of Information, Control and Management Systems, Vol.11, (2013), No. 1

37

OpenCV is a huge open-source library oriented for image processing. It contains many high optimized functions for real-time computer vision oriented applications. This library is multi-platform and it can be used in such operating systems as Windows, iOS, Linux, Android etc. 2.3 Camera settings At first we researched how to programmatically adjust the camera settings, because automatic adjusting is defiantly turned on which is not good for image processing (detection algorithms need to have stable focus or exposure, which can be ruined using automatic settings). Creating library for wide camera configuration was first step. We used the library DirectShowLib successfully which is wrapper for key COM objects IAMCameraControl or ICameraControl. They provide rich camera control. However, how much settings can be configured depends on the camera quality. We tested 6 different kinds of web cameras from different vendors. Cheaper cameras had fewer options to adjust. Finally we decided to use Logitech HD Pro Webcam C920 since it has many options for settings. 2.4 RGB and HSV model The most commonly used color model in computers is RGB color model. In this model each color is presented by values of three basic colors red, green and blue. In computer graphics and image processing it is mostly used HSV color model since it can be easier to pick out range of colors than in RGB by human [6]. It consists of three components hue, saturation and value. Due to reasons we have used HSV. 2.5 Detection algorithm Before describing the algorithm we would like to note, that values of thresholds was drawn and designed for laser pointers with laser power below 1mW. Our testing devices were laser pointers Logitech Professional Presenter R800 (green laser) and Legamaster LX-1 (red laser). At beginning an input image from camera is captured and converted from RGB to HSV color model. Our interest is focused on value channel from HSV of the input image. If the camera exposure is correctly configured, color of laser spot will display brightest color in the input image. Then average value of value channel pixel is acquired (in OpenCV is this value in interval 0-255). It is necessary to configure camera exposure if average value is too high because image is too bright and laser spot may not to be recognized. The average value is compared to AIV1 threshold. If it is bigger than the threshold then exposition is adjusted. The AIV threshold is set to 30% of value channel maximum value (255*0.3 ~= 77). Big value of AIV threshold can
1

AIV threshold average input value threshold

38

Laser Spot Detection

cause many of false detections or unwanted merging of different groups with high brightness into big ones. On the other side, very small value causes unfulfilled requirements on exposure or loss of demanded brightness sources. After the camera exposure correction is carried out the highest value from value channel is acquired and DV2 threshold is determined. The DV threshold is dynamically computed for each input image. Value of DV is set to 95% of highest value of value channel of current image so the rest in 5% contains bigger group of valid pixels, which can be possibly our laser spot. Number of pixels that fit DV threshold is found. If the number is bigger than 100, the algorithm skips further recognition steps to maintain real-time computing. This also may mean a change of light condition during detection and it is necessary to correct the exposure again. We found out during testing, that average size of one laser spot, is presented by group of pixels with high value 36 (6x6) in value channel. If the number is below limit, pixels selected by DV threshold are grouped. Pixels in group are explicit neighbors; also group may consist of one pixel. Then for each group centroid and also diameter is computed as follows:

where n is number of pixels in a group. Result of root is ceil-rounded to nearest high integer number, so diameter value is always at least 1. The reason for using the square root is assumption that a shape of laser spot is circle (only nearest surrounding is selected) and cropped area is square. For further calculation is provided on square cropped area with length of side 2x diameter + 13. Next step is obtaining the highest value from value channel of cropped area. New threshold is constructed and its value is set to 90% of this highest value. To determine whether the cropped area contains laser spot is realized by the proportion of area size and count of pixels from area that fits the threshold (value may not be bigger than 0.3). Color recognition is last step of the algorithm and this process searches for pixels with color within defined intervals in cropped area (it is possible do detect multiple colors and detect continually multiple laser spots). The resulting color is the one that has highest number of pixels as first. The algorithm can now be described as follows: 1. Capture image from the camera. 2. Convert image from RGB to HSV color model. 3. Check the exposition of camera. If it is not suitable make corrections and go to the step 2.
2 3

Dynamic value threshold The 1 is width of centroid.

Journal of Information, Control and Management Systems, Vol.11, (2013), No. 1

39

4. Obtain maximum value of value channel and calculate DV threshold. 5. Obtain number of pixels that fit DV threshold. If the count > 100 go to the step 2. 6. Group pixels that fit DV threshold. 7. For each group do: a. Compute diameter and centroid. b. If the proportion of cropped area size and cropped area threshold > 0.3 get next group. c. Recognize laser spot color and add data to the local list. 8. Update the global laser spot list from the local list. 9. Go to the step 1. 2.6 Localize the position of laser spot We can detect a position of laser spot in an image captured by a camera. Since a camera can be located in different positions in the scene the captured image can be warped. In order to use laser pointer as a mouse we need to transform coordinates from surface of projection screen in image captured by a camera to coordinates on screen in computer. Therefore we create a transformation matrix using a calibration image of chessboard. The transformation matrix is computed by known points in tiles on screen of computer and same detected points from image captured by a camera. For this purpose finding homography is used with Least-Median robust method. 2.7 Complexity proposed algorithm Average processing time takes 62.5ms (16 frames per second) that means it could be used in real-time processing. The computing time of the laser detection algorithm can be improved by rewriting some parts of code to C programming language. 3 DEMONSTRATION GAME To demonstrate the functionality of our proposed method we have developed a simple game inspired by the Duck Hunt game. The main goal of the game is to shoot a flying duck using laser pointers. The game works as follows (Fig. 2). A computer displays the image from desktop screen of the game through a projector on a projection screen. The projection screen is captured by a camera which is adjusted to proper exposition setting so the detection algorithm can recognize laser spot if one use laser pointer. At first we have to establish coordinates of projection screen using calibration image of chessboard. If laser spot appears on the projection screen, the algorithm detects it and transforms the coordinate of it to computer coordinates. If the coordinate is overlapped with a duck, the duck is hit. In addition the sound of shot will be played.

40

Laser Spot Detection

The game can be playable by two players, since our detector can detect two colors of laser beam red and green. Every shot is counted for red or green player, so player with highest score is winner. The Duck Hunt game was presented at Open Day of Faculty of Management Science in ilina, Prievidza, and at Prien 2013 organized by Bilingual Grammar School in order to demonstrate it and stimulate potential students for research area. Everyone could play the game right after take a laser pointer into hands.

Projected image on projection screen

Projector

Source image

Camera Computer Laser pointer Captured image

Figure 2 Scheme of laser spot detection for the demonstration game 4 CONCLUSION This paper proposed the method for detection of the laser spot emitting by a laser pointer on the projection screen. The algorithm is based on the camera settings and value channel of HSV color model. It can recognize more laser spot colors at once. The algorithm works very well. The developed game proves useful of the algorithm as it has been used for several times to demonstrate of computer vision area. Usage of laser detection algorithm is not only for games but for example to control of presentation through laser pointer (drawing with pointer on a projection screen), computer control or commercial use in entertainment industry. Therefore our future work will be focused to optimize the recognition algorithm and then to create a realtime application for computer control by gesture.

Journal of Information, Control and Management Systems, Vol.11, (2013), No. 1

41

Acknowledgement This contribution/publication is the result of the project implementation: Centre of excellence for systems and services of intelligent transport II., ITMS 26220120050 supported by the Research & Development Operational Programme funded by the ERDF.

"Podporujeme vskumn aktivity na Slovensku/Projekt je spolufinancovan zo zdrojov E"

REFERENCES [1] [2] MIKAWA, M., MORIMOTO, Y., TANAKA, K.: Guidance method using laser pointer and gestures for librarian robot, IEEE RO-MAN, Sept. 2010 SOETEDJO, A., NURCAHYO, E., NAKHODA, Y. I.: Development of a costeffective shooting simulator using laser pointer, International Conference on Electrical Engineering and Informatics (ICEEI), July 2011 WADA, T., TAKAHASHI, M., KAGAWA, K., OHTA, J.: Laser pointer as a mouse, Annual Conference SICE, Sept. 2007 CHAVEZ, F., FERNANDEZ, F., GACTO, M. J., ALCALA, R.: Automatic Laser Pointer Detection Algorithm for Environment Control Device Systems Based on Template Matching and Genetic Tuning of Fuzzy Rule-Based Systems, International Journal of Computational Intelligence Systems, Vol. 5, No. 2 (April 2012), 368-386 BYUNG, M. J., NGYUEN, D. D., SANG J. L., JUNG D. J., JAE W. J.: Hardware architecture for detecting laser point using FPGA, 12th International Conference on Control, Automation and Systems (ICCAS), Oct. 2012 WEN, C., SHI, Y.Q., GUORONG X.: Identifying Computer Graphics using HSV Color Model and Statistical Moments of Characteristic Functions, IEEE International Conference on Multimedia and Expo, July 2007

[3] [4]

[5]

[6]

42

Laser Spot Detection