You are on page 1of 34

A Seminar Report on

BAR CODE RECOGNITION IN COMPLEX SCENES BY  CAMERA PHONES
Submitted in partial fulfillment of the requirements for the award of B.Tech Degree in Electronics And Communication Dept. by Jinesh R S

AUGUST 2010

     

ADI SHANKARA INSTITUTE OF ENGINEERING  &TECHNOLOGY
 (An ISO 9001 Certified Institution) VIDYA BHARATHI NAGAR, KALADY, KERALA

1

ADI SHANKARA INSTITUTE OF ENGINEERING  &TECHNOLOGY
 (An ISO 9001 Certified Institution) VIDYA BHARATHI NAGAR, KALADY, KERALA

Certified that this is the Bona fide Record of the SEMINAR entitled “BAR CODE RECOGNITION IN COMPLEX SCENES BY CAMERA PHONES” submitted during the year 2010 in partial fulfillment for the award of Bachelor of   Technology in Electronics & Communication engineering by the candidate with      Name: Jinesh R S            Register No: 56028  Branch: Electronics and Communication Staff In­charge
2

Certificate

      H.O.D

Date…………....

                             ACKNOWLEDGEMENT

       It is a great pleasure to express my sincere gratitude to Dr.S.G.IYER,  Principal, ASIET, for his guidance advice and encouragement throughout this  endeavor. I'm also grateful for granting me all the facilities for the seminar  presentation.        I'm greatly indebted to HOD, Electronics & Communication Dept., for the  help and guidance at stages of my work.                     I take this opportunity to extent my sincere gratitude and thanks to  Ms. Anju,  seminar guide for her guidance & help in connection with this seminar.                    I wish to express my gratitude and sincere thanks to all those who helped  me by giving their valuable advice and guidance.
3

4 .

 In deal with the infuence by the different light conditions and noise a recognition  algorithm is used. we propose an accurate location and orientation.  Barcode  is  a   fast.   Mobile  phone  can use for capturing barcode with  their  cameras  and decoding them  with  software   running   on  the   phone. rough location and binarization to erase noises and  reduse computation. skew correction with  the help of alignment pattern algorithm to build a bar code grid. Quick response code is a 2D bar code which has been widely used  in industrial information tagging applications where high data capacity and robust error corrections  are required. First. Bar code enables product to be tracked efficiently and acurately at speeds not possible  using manual data entry system. Finally. we use error correction and  decoding process to generate the result. 5 . we combine filter.ABSTRACT Mobile phone with camera functions are capable of capturing image and processing  task. Then.  easy  and   accurate  automatic   data   collection  method.

6 .

..........19 CHAPTER..………......................10 5......................................................................................3 BINARIZATION..............……………..........II LIST OF FIGURES…….........TABLE OF CONTENTS ACKNOWLEDGMENT..............................4 3..........……………..................1 CHAPTER...1 FEATURES OF QR CODE………………........………….......................................13 5.......……..........................................6 ERROR CORRECTION AND DECODE........2........................2 FILTER AND LOCATION..............5 CORRECTION THE DEFORMED SHAPE............................................... 2 WHY USE MOBILE PHONES……………………………….................................8 CONCLUSION AND FUTURE WORK.1 GRAY CONVERSION………..1 DATA MATRIX………………………………….......7 CHAPTER........…………...............IV CHAPTER............17 CHAPTER.......................... 3 BAR CODE.............……….....16 CHAPTER...…………............................................................…………………………………………………....6 EXPERIMENTAL RESULTS......12 5................ 4 VARIOUS ALGORITHMS………….....2 QR CODE…………………………………………..........8 CHAPTER.11 5....20 REFERENCES...3 CHAPTER..........................………......I ABSTRACT.......................21 7 .......6 3. 1 INTRODUCTION...........7 RECOGNITION TEST.............. 5 RECOGNITION ALGORITHM………….................4 ACCURATE LOCATION AND ORIENTATION..……………4 3...................10 5....................................14 5.....................…........

6 FIGURE 3……………………………………………………….14 FIGURE 4………………………………………………………..15 8 ..LIST OF FIGURES FIGURE 1……………………………………………………….4 FIGURE 2……………………………………………………….

 and movie shooting by using embedded camera devices.   Although   still  primarily used for voice communication. skew distortion. with the inclusion of digital cameras these devices have  become a potential platform for machine vision application such as bar code recognition.CHAPTER 1 INTRODUCTION Modern mobile phones are high resolution colour displays.4   percent   of   the   respondents   whohad   camera   phones   with   QR   Code  (Quick Response Code) readers used camera phones with QR Code. But previous research work has  shown that recognition of 2D barcode in mobile phone is very difficult because of the high noise. So an interesting  approach is capturing bar code with their cameras and decoding them withsoftware running on the  phoneIn   August   2006. It is very difficult to  robustly extract accurate features such as edges and peaks of the bars and spaces from the barcode  9 .   advanced   technology   has   succeeded   in   continuously   producing  smaller yet smarter devices. they support different  standards   of   wireless   networking.   and   they   have   reasonable   processing   power.   In   recent   years.Now mobile phones can implement many new kinds of applications  such as taking photos.   82.  non­uniform illumination. low resolution and optical blur.

Ohbuchi et al.   In   this  method. 10 . Many   new   algorithms   are   presented   for   dealing   with   1D   bar   code   in   complex  situation.  external   contours   finding.  Here we describe a new image recognition algorithm which applied to capture image in various  light condition.presented an algorithm capable of the real­time recognition of barcodes on a  mobile phone.   inverse   perspective   transformation   and   cell   grids   generating.introduce an algorithm to analyze and correct the distorted image of QR  Code.Sun et al. The algorithm includes gray­scale image transformation. binary image and edge detection is very important for following decode. But it is very  difficult to binaries image and extract accurate edge in the blurred and damaged bar code images. binary image.images taken by a camera phone. canny edge detection.

Have No Other Use. Too Bulky. 11 .CHAPTER 2 WHY USE MOBILE PHONE BAR CODE READERS • • • • Too Expensive Scanners. Workers Have To Be Trained.

MOBILE PHONES • • • • Affordable Prices And Common. Have Various Uses.  12 . Too Handy. No Training. CHAPTER 3 BAR CODE There are two types of barcodes are present ­One dimensional and two dimensional.

 a patterned­line locator( the two  alternating dark and light patterned lines). 13 . and aquiet  zone( a blank area) surrounding the whole tag.   aerospace   and   computer   manufacturing   industries.1 DATA MATRIX Data matrix is one of the most well known 2D bar code standards.   for   large   data   capacity  labelling. such as direct part marking and package marking. It is widely used  in   the   automotive.1D barcodes have low data capacity. the inside area with encoded data in binary. so we commonly use 2D bar codes.    Figure 1                      It consist of a solid­ line locator( the two solid line). 3. There ar two standardised  2D barcodes­Data matrix and quick response(QR) code.

 It employs the Read­ Solomon error correction to enable accurate reads even  when substential pats of the code are distorted. 14 .  encoding any data.Its   capacity   is   2334   alphanumeric   characters   or   1556   8­bit   ASCII   characters.

 and the dark­ light ratio is 1:1:3:1:1. decide the  rotation direction of symbol and the width of module. alignment pattern. size and inclination.2 QR CODE It   comprised   of   the   following   patterns:   finder   pattern. The finder patterns located at the three corners of the  symbol intended to assist in easy location of its position.The location of alignment pattern center is  important for correcting the deformed shape. Three dark­light­dark  squers are overlapped in every finder pattern. and data cell. The timing  patterns provide the secondary information which can help us to locate the symbol.3. 15 .   timing   pattern   format  information.

2) High spead reading Adapted with ccd reading. it can recognize more qr code symbol per second than pdf417 symbol for  encoding same data capacity.2.    Figure 2 3. its maximum symbol can encode7089 numeric data or  4296 alphanumeric data. 3) Chinese encoding capability Chinese   and   Japanese   characters   are   represented   by   a   two   byte   compination   in   other   two­  16 .1 Features of QR code 1) High capacity encoding of data QR code has high capacity encoding of data.

dimensional barcode. 4) Readable from any direction from 360 degree CHAPTER 4 VARIOUS ALGORITHMS Many new algorithms are presented for dealing with 1D bar code in complex situation. 17 . it can use 13 bits encoding a  chinese character. But in QR code there is a specific chinese mode.

The first step detects the code region using the Otsu algorithm and the Least Square Method. • An algorithm capable of recognizing the PDF 417 barcode is proposed in real time with a mobile phone. finally psf estimation and line de-blurring is adopted. presented an algorithm capable of the real-time recognition of barcodes on a mobile phone. Besides that. This method includes three steps.• InImage Pre-processing for Bar Code Detection in Mobile Devices . the above algorithm used in 2D bar code is not well. • Ohbuchi et al. wavelet-base and knowledge-base approach is used to locate the bar code area and segment the bar code character Because of the feature of 2D bar code is different from 1D bar code. The second method searches for the cutoff rules with a scanning approach. The most important drawback of the method is the fact that it has been hand tailored for one certain hardware device. It relies on access to a powerful but also very specific hardware element. which is not accessible by normal application developers. the algorithm relies on two specific conditions: First is prior to the code’s decoding. So the algorithm is difficult to be used in 2D bar code decoding. followed by the start and the ending points detection. This is mean that the recognition result will rely on the ef18 . a de-noising and vignetting elimination is performed first. the barcode’s position is detected using a spiral scanning algorithm that runs on the device’s signal processor. • A statistical method is proposed in which. Thisassumption is not realistic in usually environment. Second is this algorithm makes the assumption that the point in the middle of the screen is located in the code. In third step symbol characters are segmented from the original image. So the successful binarization of the code areas during the first step is the most key step in this algorithm.

In this method. inverse perspective transformation and cell grids generating. which specific for low quality cameras.fect of the Otsu method. low contradistinction. • Sun et al. grid vibration (Bayesian decision). canny edge detection. But it is very difficult to binaries image and extract accurate edge in the blurred and damaged bar code images. But the Otsu method could be failed in complex lighting conditions. The algorithm includes Hough transformation. CHAPTER 5 RECOGNITION ALGORITHM 19 . • An efficient scheme for 2D bar codes decoding is proposed in [9]. and various mixed conditions . and deformable model (psf estimation) and decode. introduce an algorithm to analyze and correct the distorted image of QR Code. The algorithm includes gray-scale image transformation. binary image and edge detection is very important for following decode. binary image. In this method. so the application scope is decreased. including highlight spots. BCRL(location with horizontal and vertical waveforms). Additionally. many constraints are added in this method. nonhomogeneous lighting. the recognition time will be cost more than above because there is no binarization operation. external contours finding.

It   consists   of   several   steps.  5. error correction and decoding. It is  needless to deal with color information and the gray image calculated quickly with little space.  binarization. and images are captured  in RGB 24bit format by most phones. so  gray conversion is needed to do firstly. and normalization the skew shape.  20 .1 GRAY CONVERSION  QR Code symbol is captured by mobile phone with camera. but QR Code symbol is a set of dark and light pixels. The input is a color  image which is captured by mobile phone and the output is a decoding result.   filter   and   location.   gray   scale   image   conversion.

  wavelet analysis  etc.  the   gray  change frequency of the code area is quite high.2  FILTER AND LOCATION  Perform morphological closing on the gray­scale image from above step.   such   as   pattern  searching. Before binarizing the image.  21 .5.   texture segmentation.  Compared  with  the  background. so we can easily get the rough bar code area in  frequency domain. This will  smooth the bar code and reduce the influence of noise. a barcode area  in image is extracted that our approach can reduce computational power and increase the accurate  rate   of     binarizing   bar   code.   There   are   several   methods   to   detect   code   area.

 Selection of a proper binarization method is critical to the performance of  barcode recognition system. and the gray value was sorted. Then the gray histogram is calculated for each block. This method is existed two drawbacks: (1) the  barcode symbol must be in the center of the captured image.   and   it   is   used   in   most  recognition algorithm .   the   resulting  binary image will be very bad. The  middle value was choosing as the threshold of each block. if an image was in variable lighting conditions. First the image was divided into  nine  blocks. The main problems with a local threshold method are hard to set  22 .3 BINARISATION                                                                         Binarization   is   a   necessary   step   in   recognition   bar   code. In this case. (2)  This   method   results   in   excessive   segmentation. Finally the smallest value of these middle  values is the global threshold of this whole image. a simple and popular method is threshold.5.   Using   a   global   threshold   method. a local  threshold method performs better. otherwise the method will failed.  Ohbuchi  propose an improving global threshold method. In binarizing an image.

 and reduce the execution time.  There are three identical position detection patterns located at three of the four corners of QR Code.  Three  dark­light­dark squares are overlapped in every finder pattern.  The bar code area is getting in previous step.4 ACCURATE LOCATION AND ORIENTATION  After binarizing the code area. The timing patterns provide the secondary information which can help us to  23 . But when one of the finder patterns is partially dirty or damaged. it is concluded that Otsu’s method is the best. So the ratio area  should be quickly find. There is the off chance that similar graphic existed in barcode symbol. the accurate location and orientation is implemented. and the dark­light ratio   is  1:1:3:1:1. which chooses the threshold that minimizes  within­group variance. Among more than 20 threshold  methods. So we used the Otsu’s technique as the threshold algorithm. 5. we can use  the timing patterns. so the non­uniform illumination has  little effect on bar code area comparing with the whole image. eliminate the block effect.a right window size.

 This method avoids the  rotation and interpolation. decide the rotation direction of symbol and the width of module.   The   amount   of   calculation   is   great   and   is   not   accurately.5 CORRECTION THE DEFORMED SHAPE                       The input image has a deformed shape because of being captured from the embedded  24 .   After   located   the   finder  patterns.   But   the   interpolation   operation   is   used   in   rotating  operation. We get the   grid  moving along the line which connects the central of three finder patterns.  modular   distance   offset   algorithm   was   used   without   rotating   symbol.  5.   So   we  can   skip   this   step. increases the computing speed.locate the symbol.   the   module  width and height and  the angle of rotation were known. Rotated images  are   handled   after   orientation   in   common.

 An inverse projection transformation is used to normalize the code sharp. With the version increased. the other 30 percent is  25 . so the computation complex degree  algorithm is high. Link the  central point of the alignment patterns and three position detected patterns.      Figure 3                The location of alignment patterns center is important for correcting the deformed shape. in order to correct the contorted QR Code  symbol.camera device. Different from the other 2D barcode. distortion is little.  In international standard. the number of alignment patterns is 6. the number of  alignment patterns is 1. In a small sampling grid. the location algorithm of alignment pattern center will be failed when the  bar code symbol is deformed. because the 70 percent of  estimate location of alignment pattern center is inside the alignment pattern. When the version is 3. the small sampling  grid  is formed. there are many alignment patterns in QR Code symbol. the  number of alignment patterns was added.. But the  specific feature of  QR Code symbol is not used in this method. as shown in Fig. but when the version is 7. so we combine the affine transformation and  the central coordinates of alignment pattern to correct the deformed shape.

               Assume the point A is the estimation location. we can search the eight  direction of point A in order to find the area of approximately dark­ light ratio is 1:1:1:1:1. and 96 percent of alignment pattern center was successful location. we can get the point E is the alignment pattern center.                                                      Figure 4     26 . so the  point C is omitted.   So   we   proposed   an   eight­direction­   estimation   algorithm. as shown in Fig. An  additional condition is that the eight neighbor points of alignment pattern center are all black.outside   of   alignment   pattern. Finally.   it   can  improve the accurate rate.

  27 .   the  segment  is easy to do.6 ERROR CORRECTION AND DECODE                                                          When   the   finder   patterns   and   alignment   patterns   were   located   successfully.  The error correction and decoding process is the last step of recognition barcode.5. Decoding is just the reverse of the encoding procedure and the decoding  steps can reference to international standard of QR Code. Then the pixels were getting to ready  decoding. and the grid is easy to generate. It  employs the Reed­ Solomon error correction to enable accurate reads even when substantial parts of  the code are distorted.

CHAPTER 6 EXPERIMENTAL RESULTS This application is implemented in our test hardware architecture. 28 . and camera interface (twenty thousand pixel CMOS sensors) and LCD controller etc.0 and embedded in this mobile. including additional 512MB Micro-SD. In addition to the processor core.. An Intel® XScale processor of Motorola. The barcode image processing algorithm is developed with windows mobile 5. its associated caches and memory management units. Motorola A1200r is run at up to 312MHz.

5% 95% 91% 87% 29 . In order to confirm the effectiveness of the proposed recognition method. the experiment verified the proposed method is efficient. In a whole. With the lighting conditions became weak or uneven. alignment patterns location from eight directions. and higher recognition rate was obtained. the barcode image was captured and decoded in mobile phone. binarization. so the recognition rate is the slowest. The recognition rate and executing time were compared in Table . these images were divided into four groups: the first group barcode image was captured in normal lighting conditions. accurate location and orientation of finder pattern. the fourth group image was captured with deforming barcode. which including filter location. For the evaluation. The experimental results have shown that the barcode recognition rate is higher (the first group image). about average 400ms. Compare of different group Group The first group The second grop The third group The four group The average of total recognition  time(ms) 380 430 470 540 Recognition rate 97. In order to evaluate the validity of our method. The images captured have a resolution of 640*480 pixels. The experiments were done in real conditions. We evaluated it with whole recognition performance. The four group is the deformed barcode image.A sample set of 100 unconstrained QR Code barcode image were tested to verify the proposed recognition methods. decode an image will cost average more 500ms. the second group image was captured with highlighted spots and background noise. and the recognition time is litter in the normal lighting conditions. the captured barcode image can be recognized slower and the recognition rate decreased. average 97%. and decoding. All original images are acquired by camera phone. the third group image was captured in low contrast or non-homogeneous lighting conditions.

CHAPTER 7 RECOGNITION TEST                                  In order to conform the effectiveness of the proposed recognition method. determination of the location and orientation of the  finder pattern. located alignment patterns from eight directions. which  including multilevel thresholding algorithm. and decoing.we evaluate it with the  30 .

 we captured more than 70 barcode image and  run recognition algorithm taken by our test hardware.whole recognition performance. Experiment have shown that over 98% of the captured  image can be successfully recognized with the automatic recognition method. Binarization time Reference method of  standard Ohbuch’s method Our method 40 20 30 Total recognition  time(ms) 90 60 70 Recognition rate 75% 83% 98%    CHAPTER 8 CONCLUSIONS AND FUTURE WORK 31 . Higher recognition  rate was obtained with our algorithmas compared with other methods.   and   Ohbuchi’s   method.   The  experimental result are summarised in table.   the   reference   algorithm   of   QRcode   standard. For the evaluation.The recognition result were compare with two  other   techniques.

 We will enhance our work to find the best way to  solve the problem of highly distorted and low resolution bar code images by improving the work .             . Usability and  user­testing is to be carried out in the later stage. In the future we will enhance our system to include more applications  that are related to consumer industry. This application facilitates consumers with quick  and easy way to verify product’s status by enabling them to capture the barcode images rather than  typing in the barcode number. 32 .                   The barcode recognition system discussed a simple but unique mobile application system  that has a huge potential to be widely used due. not necessarily related with Halal products.

Computer Vision.8 pages. Wong.REFERENCE • H. 41. and Image Processing. Volume 02. et al. Sahoo. ICIP06.. 783­788.  Proceedings of the Sixth International Conference on Intelligent   Systems Design and Applications(ISDA’06). and A.2006. March. Soltani.597–600 • Hao Wang. Feb. Tan. Yanming Zou. • “Decoding with Camera­Equipped Mobile Phones”.2007.. 2005. “ Automatic Recognition Algorithm of Quick Response Code Based  on Embedded System”. 2005  2nd  International Conference on   Mobile Technology. S. 2006.pp.2005.C.8-11 Oct. issue 2. Nov.K. 33 . vol. 19-23.pp. “Camera Readable 2D Bar Codes Design and Decoding for Mobile Phones”. • Yue Liu.pp. Kato. • P.K. 15­17 Nov.233-260. 15­17 Nov. Applications and Systems. T. “ Barcode reading from images capture by camera phone”. “A survey of thresholding techniques”. 1998. Mingjun Liu.pp. 2D Barcodes for mobile phones”. Applications and Systems.469-472. • Wang.Proceedings of the Fifth Annual IEEE International Conference on Pervasive Computing and Communications Workshops (PerComW'07). IEEE International Conference on Image Processing. Graphics. Kongqiao. 2005 2nd  International Conference on Mobile Technology.

34 .