You are on page 1of 10
10 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 44, NO, 4, NOVEMBER 1995, Optical Recognition of Motor Vehicle License Plates Paolo Comelli, Paolo Ferragina, Mario Notturno Granieri, and Flavio Stabile Abstract—In this paper a system for the recognition of car license plates is presented. The alm of the system is to read ‘automatically the Italian license number of a car passing through a toligate. ATV camera and a frame grabber card are used to ‘acquire a rearview image of the vehicle. The recognition process consists of three main phases. First, a segmentation phase locates the license plate within the image. Then, a procedure based upon feature projection estimates some image parameters needed to ‘normalize the license plate characters. Finally, the character rec- ‘ognizer extracts some feature points and uses template matching operators to get a robust solution under multiple acquisition conditions. A test has been done on more than three thousand real images acquired under different weather and iMlumination conditions, thus obtaining a recognition rate close to 91%. 1. IntRopucTion UTOMATIC recognition of car license plates is a neces- sary capability for the realization of unattended tolgates. ‘A vision system for the car identification can also help a hhuman operator and improve the overall quality of a service. Highways, parking areas, bridges or tunnels are places where such a system can be applied. Any situation requiring the automatic control of the presence and identification of a motor vehicle provided with a license number, may represent a potential application. If a payment operation is also expected, then the use of a credit card or of an automatic cashier might bbe a solution, but a number of different situations have to bbe considered, namely, the machine is out of order oF the customer has not proper money. In such a case, a rearview picture of the car can be taken using a TV camera, and then analyzed by a human operator or by a vision system to send an invoice to the car owner ‘An interesting application of machine vision to traffic con- trol can be found in [1] where an algorithm for vehicle detection and also for vehicle tracking is presented. Using this technique, the monitoring of vehicle traffic is possible, although no car identification is expected. In this paper a vision system forthe recognition of the Italian license number is presented. The paper mainly addresses the issue of reading the license number within an acquired image. Even if the system has been designed for the specific case of the Italian license plate, it can be adapted to different models (e., foreign license plates). This system has been designed to be applied on a real service, called TELEPASS", offered Manuscript revived October 4, 199%; revised Api 1, 1995 P.Comei is with Oservatro Geofsico Speimentle of Teste, Test, ly Fegan i with Dipartimento Informatica, Universit di Pia, Piss, tly MN. Grane and Sub ar with he IBM SEMEA, ScienteSoation Unit Rome, aly TEEE Log Number 9413251 TELEPASS® is «registered matk of Autosade SpA by the main company that manages the Italian highways. TELEPASS® is a new and efficient way to pay the imposed toll on the Italian highways, by means of a radio link between the ‘motor vehicle and the tollstation. Only the vehicles equipped with suitable radio transmitter may use the TELEPASS® tollgates. When the vehicle passes through the gate, it transmits its identification number through a coded signal to the toll- station and an automatic payment is performed. The main ‘goal of this service is to pay without stopping at the gate. A TELEPASS® (ollgate is unattended, and thus there are no Ihuman operators to solve the problems that might possible arise ‘on line’ (ie. improper payment). A TV camera is used to frame the cars, while they are exiting the tollgate and all the unsuccessful transits are recorded. A vision system can be successfully applied to process these images in order to show to the human operator the recognized characters and more information retrieved from a license numbers database. This approach leads toa faster and more accurate data analysis and therefore improves the overall quality of the service. ‘The paper is organized as follows. Section II describes the main modules and their goals. In Section III the Italian license plate model is presented and the problem of its location within the image is described. Section IV deals with the normalization ‘process needed to prepare the data for the recognition phase. In Section V the character recognition problem is treated. Finally, ‘experimental results and conclusions are discussed in Sections VI and VIL TL System OvERVIEW ‘The presented system, unofficially called RITA, RIconose- ltore di Targhe Automobilistiche (license plate recognizer), has been designed to recognize automatically the characters written on the license plate placed on the rear-side of motor vehicles. The goal is to read only the Italian license plates and reject all the others. ‘The image acquisition is performed by a CCD TV camera ‘mounted on the framework of a tollgate. In such a way it is possible to frame a rearview of a passing vehicle, Whose presence is reported by an ending-race sensor. Image acquisition is made while the vehicle is passing through the tollgate (ie, the vehicle is moving). In these conditions, even if a limit on the vehicle speed exists, to obtain good results it is necessary to store only half a frame (the odd rows or the even ones) because of the acquisition time delay between ‘wo adjacent image rows. A quasi-static image is so obtained, ‘The analog image is then converted into a digital signal to be processed later by a computer. A frame grabber is used to obtain the proper data. The working area for the TV camera 0018-9548/95805.00 © 1995 TREE ‘COMELLL «al: OPTICAL RECOGNITION OF MOTOR VENICLE LICENSE PLATES. a | tieogan Fig. 1. RITA architerue: The system is composed of tee modules ‘comesponding othe tree min comyultionl phases: The license plate aea Tocaion module tbe pre-processing mle and the recognition module. An hanced image is generaed to be presented 0 a human operator. is about 180 x 180 cm?. The acquired image size is 256 x 512 using 256 gray levels (8 biv/pixel). ‘The variety of acquisition conditions makes the recognition task not easy to solve. Indeed, the examined images are very different depending upon the illumination, perspective and ‘weather conditions which were present during the acquisition, Stil, the TV camera can be mounted in different positions, depending on the examined tollgate. Moreover, even if an illumination system is available, the scene can be acquired at any time, thus generating images with different contrast solutions. Finally, another difficulty might arise due to the vehicle speed, which could allow the framed vehicle to be not always at the same distance from the TV camera. AS a result, the main defects in the images can be summarized as follows, out of focus, geometric distortion and noise presence. Furthermore, the resulting license plate characters are even very small (about 15 pixels high and 10 pixels large) and noisy. ‘Therefore, in order to manage such a difficult recognition task, RITA is composed by three modules corresponding 10 the three main computational phases: the license-plate area location module, the preprocessing module and the recognition module (see Fig. 1). Phase I: Mt tries to locate the position of the license plate ‘within the acquired image. The estimation ofthe image portion containing the license plate is not yet very accurate, because the utilized algorithm does not segment the single characters ‘but the whole license plate area. Nonetheless the error rate of locating the license plate area is close to zero. Phase 2: The preprocessing phase removes the noise, en- hhances the image, and moreover, it detects, centers and normal- izes the license plate image. Within the cut area itis possible to find all the information necessary for the recognition. It is important to have the characters of constant size because of the chosen recognition technique. The output of this module is. still an image representing the isolated and normalized whole license plate. Phase 3: The recognition phase employs a template match- ing technique in order to find a match for the characters into the license plate. If it fails, a reject message is output. Even though a “simple” template matching technique on gray level mages has been used, the achieved results are satisfactory, as shown in Section VI. Ml, PHASE 1: LICENSE PLATE AREA LOCATION In this section a brief description of the Italian license plates js given and the technique to locate it within the input image is also presented. ‘An Italian car license number consists of two main fields: ‘The first, on the left, contains the province initials: the second, fon the right, contains a six character string. The initials refer to one out of the 95 main Italian provinces and the six characters represent an identifier for each registered vehicle in that province. Because of the large number of registered cars in the big provinces, digits and letters appear in the identifier. For all the provinces, except the case of the province of Rome, the initials are represented by two letters. For the province of Rome, all the four characters in the name are used. ‘The license plate, placed on the rear-side of a vehicle consists of two separate metal plates: one for the initials (on the left) and one for the identifier (on the right). These metal plates are usually mounted as shown in Fig. 2, placed side by side to form a single row. The characters used either for the initials ‘or for the numbers are black on a reflecting white background, and the province initials are thicker than the string characters. Since foreign cars might also go through the unattended tollgate, their license plates may be acquired and presented to the system, too, However, the algorithm designed to locate the area containing the license plate (Phase 1), is not able t0 distinguish among the various license plates of other countries. In fact, the task of detecting such a situation and rejecting the license plates out of interest is solved later by the recognition module (Phase 3). This is not the only problem for the recognition. In fact, sometimes (.e., less than 1% of the total ‘number of the Italian cars) the metal plates of the Italian cars ‘are mounted stacking the province field on top of the six characters, giving to the license plate a square form. Even these Italian license plates are assumed to be rejected. License plate location is based upon the observation of some peculiar characteristics presented by the image function. Indeed, such a computation hinges on the structure of the Italian license plate, which is rectangular and contains black characters over a white background. Thus, the algorithm picks within the picture, the area presenting the maximum local ‘contrast, that (possibly) corresponds to the rectangle containing. the license plate. A gradient analysis on the whole image leads ™ IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY, VOL. 44, NO, 4, NOVEMBER 1995 Fig. 2. (@) An example of input image acquired under normal illumination conditions (6) The license plate area automatically Toca, to estimate such a rectangle, which is then isolated. The main information, obtained by Phase I, is the knowledge about the coordinates of the center of the window containing the license plate. It is fundamental to compute this information in order to execute correctly the following phases. TV. Puase 2: IMAGE NORMALIZATION ‘The preprocessing phase consists of conventional techniques like image filtering, enhancement and resampling, as well as specific processes based on the character size and the license plate baseline orientation. Therefore, we may distinguish three ‘main steps: 1) Filtering operation and histogram stretching, in order to produce an enhanced image; 2) character size estimation and evaluation of the license plate tilt; 3) bilinear resampling of the image, to obtain standard dimension charac- ters. Note that those three steps are applied only to the license plate area in order to evaluate the parameters related to the character size and to the license plate orientation. The role of these parameters will be to improve the recognition rate (see Section 4.2). We remark that the algorithms employed to perform the steps above are chosen according to the char- acter recognition technique used in Phase 3 (ie., a template ‘matching algorithm). A, Filtering and Enhancement ‘The original images, acquired by black and white commer- cial TV camera, are gray level images. In order to facilitate the location of the characters into the license plate, a threshold technique could be applied. Unfortunately, a great variety of illumination conditions are present in the set of examined images. Obviously, when the images are uniformly illuminated and not too noisy, it is not difficult to find automatically good threshold that allows the various characters 10 be isolated from the background. Such good images. present a bimodal gray level histogram, so that a good threshold might be selected by finding first the two local maxima of the histogram (provided that they are some minimum distance apart), and then evaluating the valley between these two peaks (2), [3]. However, the idea of thresholding has been discarded, since such “good” images are only a small percentage of the otal number of acquired images. Moreover, ‘ther binarization techniques have been considered. Namely, iterative methods founded on homomorphic filtering (4], 5], adaptive thresholding as in chromosome images [6], or edge enhancement as in cardioangiograms thresholding (7) However, even if each one of these interesting techniques provided satisfactory results for a good amount of images, our final choice has been to avoid any thresholding in order 10 ‘manage as much as possible the great variety of acquisition conditions. For this reason, our Phase 2 has been designed as follows. The first operation (Step 1) is represented by a classical Gaussian filtering along the rows of the original image. with the convolution kernel: L aie The contrast of the sub-image, roughly (and possibly) containing the license plate, at this point is the same of the original image. Now, in order to improve the visualization and the recognition rate ofthe characters into the license plate, an enhancement operation is needed. Among all the enhancement operations presented in the literature (3) [8], 19}, we adopt a gray level transformation that changes the image gray scale, in order to inerease the contrast and make as visible as possible the details of the picture (2). Such a transformation is defined a follows. Let ¢ be the mapping from the given gray scale i into a transformed gray scale i = ¢(i), where the co-range of gray level values is equal to [4, B] If f(2,y) is an image with values fimited into [0,6] < [A,B], linear gray seale wansformation, which stretches and shifts the gray scale to span over the full range [A, B], may be defined as B-A bop li-a+aq 0.508 10.8 0.5 0.25} B-A boa Ab Ba ba it In the same way, a piecewise linear transformation may be defined as es asish ay ice o B ino when most of the gray levels of the given image lie in {a,b RITA's transformation function is based on (I) with A = 0 and B = 255. Let us assume that the image is n x m pixels. Now, we ‘compute the histogram function h(i) and two particular gray level values. The first, a, corresponds to the smallest integer such that: ye a0 @