Contents

Introduction A brief background

p.3 p.4

Performance and comparison with other forms of biometrics p.14 Implementing a facial recognition system
p.16

- The origins - Facial recognition with good quality portraits - General facial recognition

The applications
- Criminal Justice

p.6

-  Integrating facial recognition in the existing technical environment -   Managing the expectations of customers and operators -  Developing the expertise of operators
- Expertise in data acquisition - Expertise in checking searches

- Identification and maintenance of a portrait reference database - Identity checks in the field - Criminal investigations and information - Prevention

-  Helping the end users

-  Civil applications, access control and border controls

- Issue of identity cards - Control of identity cards - Access control

The market associated with automatic facial recognition

 The market for facial recognition -  Driving forces and obstacles

p.18

-  Applications for the general public

- Access to computerized services - Photo album management

The experience of Morpho in the field of facial recognition p.20

p.10

The Technology
-  The different steps of automatic facial recognition

Glossary and acronyms Bibliography

p.22 p.23

- Step 1 - image acquisition - Step 2 - face searches with scaling and alignment - Step 3 - enhancement of face images - Step 4 - extraction of characteristics - Step 5 - representation as a template and comparison - Step 6 - thresholding and decision-making

-  The facial recognition algorithms

- Procedural algorithms - Training algorithms

 Special features of video processing -  Special features of 3D - Light sources

Introduction
For obvious reasons, the human brain and visual organs can be considered the best existing
face recognition machine ever and forever. A specific area of the human brain called fusiform face area (FFA) has been proven to be totally dedicated to this task. Because face recognition is the most natural thing for any human being, facial recognition would appear to be the most natural of biometric techniques. Human face recognition is the most widely used way of identification or authentication of identity, and accounts for the presence of portraits on most of the identification documents that we carry in our purses and wallets, be it an ID card, driver license, credit card, library card or gym club card. This apparent ease of use is the cause for many fantasies, inconsistencies and difficulties while implementing automatic face recognition systems in the field. Because the technology has an outstanding competitor : the human brain, trained from birth to somehow do the exact same thing There are many applications. Automatic facial recognition is actually used in civil spheres in order to guarantee the unique nature of identity documents and in military or law enforcement applications, since the human face easily leaves traces when crimes are recorded by CCTV cameras or the cameras of witnesses. The acquisition of portrait images is simple, contactless and does not require any highly specific equipment; a fact that facilitates the implementation of automatic facial recognition. Rapid advances are being made in facial recognition technology, and it thus has every trait expected of a major biometric technique, up to the point where it is on the verge of taking on its greatest challenge: beat the human brain, best face recognition machine ever, but not forever. Still, in order to deliver its full efficiency, a number of caveats have to be taken into account during implementation. In an effort to answer the increasing number of questions being put forward, Morpho has taken stock of facial recognition in general, its use, the state of the art of the technology and its technical and commercial potential.

A brief background
The origins
From the very advent of photography, both government agencies and private organizations have kept collections of portraits and ID photos have gradually made their way onto all personal identification documents, from the most official passports to informal membership cards issued by sports clubs. Before the use of computers to recognize faces was even considered a possibility, facial recognition was already the subject of a great deal of research. Examples include:  the development of identification parade or lineup(1) techniques in the United Kingdom, in which a witness is confronted with a group of physically similar people, one of whom is a suspect. The witness must decide whether one of the persons in the group was present at the scene of the crime.  the work done by Bertillon on face classification. In order to recognize delinquents who are repeatedly arrested, without having to resort to large collections of portraits, Bertillon suggested that the portraits be sorted by common morphological characteristics, i.e. the specific shapes of the different parts of the face. This classification is known as the spoken portrait.

Facial recognition with good quality portraits

The first attempts to automate facial recognition started in the 1960s in semi-automatic mode. They essentially consisted in checking the coherence of measurements between different characteristic points of the face (e.g. the corners of the eyes, the hairline, etc.). They were not very successful, because faces are by nature very mobile and measurements between characteristic points are affected by orientation, to the extent that speciallydeveloped models quickly proved to be necessary. At the end of the 1980s, the development of the eigenfaces(2) technique prompted a more intense research effort. This technique is used to find a face in a photo and to compare images of faces. Researchers quickly found that the overall issue of facial recognition was complex, but could be simplified by only taking into consideration portraits that are coherent in terms of orientation, lighting, expression and image quality. Research focused on this problem, the ICAO* defined criteria to obtain controlled portraits and meaningful test sets were created. At the start of 2007, the NIST* published the results of its FRVT 2006* test. Its conclusions were quite clear. Research had reached a point where the operational use of facial recognition on high-resolution frontal images taken in a controlled environment was now feasible. But this event obviously did not put an end to work on the recognition of controlled portraits. More improvements are expected, but facial recognition has thus become a biometric technique in its own right.

(1)

Alphonse Bertillon, 1853-1914, criminologist who developed

judicial anthropometry in France.

(2)

Eigenfaces: a facial recognition technique that consists in learning

the distinctive characteristics of faces from a broad sample of portraits using each complete image rather than local characteristics (e.g. the eyes, nose or mouth).

Figure 1: Portrait parl class. Source Library of Congress, USA.

General facial recognition

Since 2007, research has been looking into significantly more difficult problems, in which faces are not viewed frontally, resolution is low or the image quality is sometimes poor. With the MBGC*, the NIST is again seeking to assess performance and has provided researchers with representative data (images and videos of faces under non-controlled conditions). It is the start of a new era and we can expect to see significant progress over the coming years.
Figure 2: Eigenfaces, courtesy of Santiago Serrano Drexel University, USA.

Figure 3: Facial recognition history.

The Applications
Automatic facial recognition is a form of biometrics. It is used for authentication (checking that a person really is who they say they are) and identification (finding out who someone is from a group of known persons). Like most biometric techniques, facial recognition has applications in the policing and civil fields and for access control. Facial recognition is special due to the portraits themselves, which are widely available and easy to acquire. Their use is acceptable to the general public. Criminal Justice
Identification and maintenance of a portrait reference database
Just like automatic fingerprint recognition, facial recognition allows police forces to manage the files of people of interest by making sure that there are not several different records for a single person. While this task is already performed using fingerprints, facial recognition provides more benefits:  it increases population coverage of the identification scheme, enabling identification of individuals whose fingerprints cannot be acquired for various reasons  by combining the two biometric modalities, superior identification performance can be achieved, therefore reducing the workload involved in the verification process The Pierce County Sheriffs Office in Washington, USA, demonstrated the high precision of the automatic facial identification of suspects and that identification is possible without calling on fingerprint experts.

Identity checks in the field

With just a camera and suitable means of transmission, it is possible to check the identity of a person in the field using a photograph of their face. Police officers equipped with PDAs can submit search requests to remote facial recognition systems and quickly determine whether an individual is already known to the forces of law and order. ID checks can be carried out on just the face or both the fingerprints and the face, if the officer has the equipment required to take fingerprints. The combination of the two biometric techniques increases the precision of searches and allows reliable, automatic decisions to be sent to the field, without the officer requiring any expertise in fingerprints.

Figure 4: mobile facial recognition.

Criminal investigations and information

Images are often made available for inquiries. They may come from surveillance videos, a witnesss camera, Internet sites or copies of identity papers. These images may show the face of a suspect. To begin with, the portraits must be extracted from the available evidence. In some investigations, hundreds of hours of video footage are analyzed and the manual search for excerpts in which faces are visible is a long and painstaking job. It is the reason why automatic assistance is necessary. Current automatic face extraction techniques work well with almost full frontal views of faces and when the quality of the video is good enough. Research is currently being made into the extraction of side and three-quarter views of faces. Even if the quality of the extracted portraits is highly variable, it is still possible to compare them with portraits of persons who are known to the police. Morphos experience in this field shows that these searches can already solve and correlate crimes. Our French, American and Australian customers have scored numerous hits with high-quality images, such as authentic or false ID documents or images posted on the Internet. It is also interesting to note that certain criminal cases have been solved using low quality images. Operational examples of the use of surveillance videos do exist, but they are rarer. By way of example, images of fraudulent use of ATMs or assaults close to an ATM can be used to solve crimes if the camera obtains well-lit, facial images. However, they cannot be used to

successfully close investigations if the film only shows the top of the suspects head or if the images are blurred. The combined advances of video surveillance systems and facial recognition technology should enable more crimes to be solved using video data in the next few years. Another source of facial images is the facial composite picture. If the recollections of the victim or the witnesses are precise enough to make a facial composite picture resembling the offender, then investigation by facial recognition may lead to success.

Prevention
Facial recognition can also be used for preventive purposes. It can be used to search for precedents. By way of example, if a file of pedophiles is available, then ID photos can be used to check whether people who work with children are in the file. In some cases, facial recognition can also be used to interactively locate persons wanted by the police in video footage. This application is subject to controversy, since it is often considered to infringe civil liberties. In any case, it is not currently suited to cases in which a very small number of persons need to be identified in a crowd. Even if this technique were to reach the excellent accuracy level of 90% of persons actually found with just 0.1% false alarms, looking for one person amongst a crowd of 100,000 passing people per day would operationally generate 100 false alarms per day. This would have a negative impact on the vigilance of control operators.

Figure 5: facial recognition in a crowd.

On the other hand, interactive facial recognition is already possible in controlled passages. By way of example, when travelers approach the border police for a travel document check using facial biometry, they must be cooperative and show their face. In this case, it is quite easy to check the travel documents and make a comparison with the lists of wanted persons. The operators who check passport control processes can also check any alerts received in response to these searches.

As a consequence, facial recognition applications can be used to guarantee that a single motorist cannot possess several driving licenses. Morpho developed a solution for this very purpose for the state of New South Wales in Australia. With regard to travel documents, the ICAO has recommended that the portrait should be the only compulsory biometric record.

Civil applications, access controls and border controls

Issuance of identity documents
Facial recognition is particularly well suited to checks of the uniqueness of application for identity papers. In a non-criminal context, it is quite normal to provide a photo, while fingerprints will always have criminal connotations and it is more difficult to acquire an image of an iris than of a face. By way of example, driving licenses include a photo of the holder, but rarely include any other biometric data.

Control of identity documents

Once the documents have been issued, facial recognition can be used to check that they are indeed being used by their legitimate holders. This check can be made by simply filming the holders when they present their documents. By way of example, the SmartGates* for automatic passport checks deployed by Morpho have accelerated border formalities in Australian airports. Today, document holders are required to stand still in front of the camera, but in the near future the check will be made as they pass through the checkpoint. Morphos rapid and robust Face on the Fly* technology is capable of acquiring faces in three dimensions, without requiring the subject to stand still.
Figure 7: biometric passport.

Figure 6: facial recognition at airport.

This application is so easy to use that a broad range of usages is possible. For example, it would be easy for universities to check the identity of students when they arrive to take an exam (authentication checks).

Applications for the general public

Access to computerized services
Biometric facial logins are already possible on certain computers. But the system has come in for some criticism, since logins are possible if a photo of the user is shown instead of the users actual face. Recent algorithms are capable of detecting whether the face is indeed three-dimensional and mobile, and future generations of biometric facial login systems will not be fooled by photos.

Access control
The purpose of access control is to check that anyone attempting to access a secure zone is entitled to do so. Access controls are made in the same way as ID checks. They are very easy for the user if facial recognition is used. The main advantage of portraits is that checks can still be made once the person has passed through the access barrier. If the access control gates are unmanned, then staff members could easily allow strangers to enter limited-access zones. But thanks to portraits, which can be acquired without any special cooperation, it is possible to check permanently that the people in protected zones are indeed entitled to be there. Facial recognition can therefore be used to extend access control by checking presence in particularly sensitive environments.

Photo album management

Facial recognition applications are now availableto manage personal collections of photographs by showing the names of persons in photos, if they already appear in older pictures in the collection. Products include iPhoto from Apple and Picasa from Google. While this application may appear trivial, it shows the full potential of facial recognition, whose limits are still far from known.

Figure 8: facial recognition manages personal collections of photographs.

The Technology
This chapter describes the different steps in the facial recognition process and describes the main technologies that are available today.
Step 3: enhancement of face images
Once the faces have been found and calibrated, they need to be enhanced. By way of example, the effects of compression can be minimized, inconsistent lighting can be corrected or unusable zones (masked by a veil, for example) can be detected and excluded. In this step, models can be applied to correct the orientation of the face, the effects of ageing and expressions. While some enhancements can be made automatically, the assistance of an operator may prove to be very useful when working on difficult images.

Step 4: extraction of characteristics

Most facial recognition algorithms use mathematical transformations in order to compare images. These transformations can highlight the distinctive specific features of an image: frequencies, directions, contours, etc. Transformed images can not usually be used by the operators naked eye.

Figure 9: Portrait acquisition.

The different steps of the automatic facial recognition process

Step 1: image acquisition
This step is decisive, because the precision of facial recognition hinges on the quality of the images acquired. In this step, automatic facial recognition systems may assess the quality of the acquired images. In interactive acquisition, the portrait can be re-acquired in order to obtain a better image that meets the criteria of the image assessment process.

Step 5: representation as a template* and comparison

A binary record, or template, is extracted from the transformed image. The comparator then compares this template with those of the images in the reference database and scores each image. The higher the score, the higher the similarity with the image of the wanted face.
Figure 10: portrait comparison.

Step 2: face localization, scaling and alignment

Before comparing faces, it is first necessary to find them in images that may contain all sorts of other information and adjust them to the same scale, with the head positioned vertically. This step is quite simple when working on controlled portraits, because each image only contains one face. But it is much more difficult to extract a multitude of faces from a video taken outdoors.

10

Step 6: thresholding and decision-making

If the search is simple, i.e. when the quality of the query image and the images in the reference database are good, the operator should only receive a small number of images that stand a high chance of matching the wanted person. The ideal case is when the operator only receives a candidate list when theres a hit in the database, and nothing otherwise, thus preserving the operators resources and attention on relevant cases. This operation is called thresholding. It requires a similarity function that makes a clear distinction between hits and false alarms. The operator can then make a decision and make changes to the systems reference database* according to the requirements of the job in hand.

learning algorithms, which apply a mathematical logic in order to define and use the criteria that an operator may not be capable of interpreting. Both categories of algorithms can be used for different types of face data: fixed images, videos, or 3D acquisitions.

Procedural algorithms
The main procedural algorithms use the visible facial landmarks, such as the corner of the eye, the middle of the upper lip, the lowest point of the chin or the details and color of the skin. After detecting the landmarks of the face - a process that may be manually assisted - the procedural algorithms attempt to measure the coherence between the parts of the two faces. They do this by using models designed to demonstrate how a face is distorted by its expression, age, orientation and lighting. The most commonly used algorithms in this category are Elastic Bunch Graph Matching (EBGM) and the comparison of facial texture. Facial texture analysis is used in particular to distinguish twins. These algorithms are used (at step 5 above) to convert images into templates in order to compare them.

The facial recognition algorithms

This chapter contains an overview of the best known algorithms. For more details, visit http://www.face-rec.org/ algorithms/, which is an excellent source of information. There are two prominent categories of algorithms when it comes to facial recognition: procedural algorithms, which imitate the analysis made by an operator, and machine

Figure 11: bunch graph matching.

11

Training algorithms
Training methods rely on an abstract process in order to find independently an optimal organization based on examples. There is a great number of training algorithms. Examples include:  Vector projections. The input used by these methods is a large vector - the characteristics extracted in step4 - which is then projected in a smaller space. If the two original images match the same person, they must have close projected vectors. If they represent two different persons, then the projected vectors are more distant. It is the definition of the projection that is complex, and leads to the training process. The most common methods are: Principal component analysis (PCA), which extracts the most distinctive vectors from a space. Eigenfaces apply this principle.
Y Y

Independent component analysis (ICA), which keeps the axes as independent from one another as possible. Non-linear methods, including Kernels and SVMs* (Support Vector Machines).
Separation may be easier in higher dimensions

Feature map

separating hyperplane complex in low dimensions simple in higher dimensions

Figure 14: Space transformation.

N  eural networks. These networks use a set of cells that transform the information that they exchange with one another. They resemble neurons, synapses and nervous influx. Neural networks are defined by training. In the comparison phase, the characteristics extracted from the portrait are entered for input into the network. The network output is used to decide whether the face resembles the different faces in the reference database.
DB
I1 H1 25 x 25 Original image I3 O40 H50 I2 H2 O2 O1

Figure 12: Axes created with principle component analysis.

Linear discriminant analysis (LDA), which separates different objects.

0.0498 0.0496 0.0494 = type A image = type B image

Detection boundary

I625

0.0492 0.0490 0.0488 0.0486 0.0484 0.0482 1.058

Figure 15: Sample neural network.

1.060

1.062

1.064

1.066

1;068

1.070

1.072

1.074

Statistical methods. These methods seek to measure the probability that a photo matches a statistical model of the face. Each face is represented in the comparator by its statistical model, e.g. by a number of states, their respective probability and the probability of transition from one state to another. New facial images are represented as a sequence of successive

Figure 13: Boundary detection with linear discriminant analysis.

12

states. For each face in the reference database, the level of probability of the sequence can be determined in order to decide whether the resemblance between the two images is high or not. Training algorithms are obviously used in step 5 (representation as templates and comparison), but they can also be used to locate faces in images (step 2) and to extract visible facial landmarks.

face or by stereoscopy*. Morpho has developed an innovative 3D acquisition concept based on stereoscopy that is capable of acquiring a face on the fly when a person passes through a control gate without stopping. It is called Face on the Fly* technology. 3D technology can be used, even without any 3D sensors. 3D morphable models can be used to take one or more images of the same face and associate a 3D shape that matches the face as closely as possible. This association significantly improves the robustness of the comparison of oriented faces.

Special features of video processing

A persons face appears several times in a video. It can be viewed from one video frame* to the next. These multiple views are useful for facial recognition purposes, because they can be used to obtain more information about the face than a single view. A range of tracking techniques has therefore been developed. The most robust techniques use movement statistics models. At each step, they generate probabilized detection and tracking hypotheses, which are consolidated in order to make a decision. The actual comparison process uses a series of images of the same person that are sorted according to the quality of the views and the different positions that they represent. Using these different views rather than a single view even if its quality is superior will always improve the precision of the search.

Light sources
The images used for facial recognition are usually taken in visible light. But infrared images can also be used and research is currently looking into other data types, such as terahertz*.

Figure 16: examples of pose angles.

Special features of 3D
Faces are naturally three-dimensional. If a face is to be completely represented, then its shape and the color or texture of every part of the face must be known. Photos only contain the color of part of the face. Therefore they only contain partial, or 2D, information. Traditional face comparison techniques rely on these incomplete data. Using all the information of the face can only serve to improve the precision of facial recognition. The European 3D Face project (http://www.3dface.org/) has demonstrated how the association of 2D data (texture) and 3D data (shape) improves precision compared with the use of only the texture or the shape. The acquisition of three-dimensional images requires special sensors that are not yet widely available. These sensors work by projecting structured light onto the

13

Performance and comparison with other forms of biometrics

Two figures are required to measure the performance of a biometric technique: the false acceptance rate (FAR*), which measures the percentage of fraudsters who are mistakenly accepted, the false rejection rate (FRR*), which measures the percentage of persons who are not accepted, whereas they are not fraudsters.
Ideally, the false acceptance and rejection rates should be zero. But in reality, biometric systems can be characterized by the graphs on the model below.
FALSE ACCEPTANCES

with 0.1% of false acceptances. This means that, for example, in the ideal passport control application, out of 1,000 authentic passport holders, only 10 would not get through the automatic check and would have to call on an operator, while out of 1,000 fraudsters carrying passports that do not belong to them, 999 would be detected by the automatic control system. But we all know that the ideal conditions implemented by the NIST for FRVT 2006 are difficult to achieve operationally. In addition to orientation, resolution and lighting, facial recognition is also confronted with problems due to physical changes and changes in appearance: expression, changes of hair, changes in weight, spectacles, hats, ageing, injuries, illness, etc. The current facial recognition algorithms can only tolerate limited variations in the portrait. For example, if a person allows his beard to grow, then the automatic portrait recognition system will recognize him with almost the same reliability, as if his beard had not changed. If the same person allows his beard to grow and pulls a face, the probability that the system will recognize him drops a little, but still remains high. But if a number of small changes are accumulated (by way of example, if the person allows his beard to grow, pulls a face, does not face the camera, remains a long way away from the camera and conceals a large part of one of the sides of his face with his hand), then the

Very low false rejections: comfort for users, weak controls

Very low false acceptance: security FALSE REJECTIONS

Figure 17: False acceptance false rejection graph.

The lower the rate of false acceptances, the more secure is the application. The lower the rate of false rejections, the greater the comfort for users and the more limited the work done by operators. The tests in NISTs FRVT 2006 demonstrated that, with frontal portraits taken in a strictly controlled environment, with high resolution and only slight differences in age, facial recognition is a very precise biometric technique. False rejections totaled just 1%,

Figure 18: Variations with age and orientation.

Figure 19: Variations with accessories.

14

probability that he will be recognized is much lower. Since there are no test samples that can be used to measure the impact, with the appropriate statistical validity, of the different criteria that minimize the precision of facial recognition, and since these criteria are not independent, the drop in the precision of facial recognition is not predictable in absolute terms. Consequently, Morpho advises every potential user to proceed with tests and measurements of their own data, depending on their own operational needs. But these tests must not simply take account of the purely algorithmic precision. They must also take the speed of

execution of the algorithms into consideration. A very slow system will necessarily find fewer hits than a fast system, with the same power and precision, and will be more difficult to adapt to the working procedures of the operators. If one compares facial recognition with fingerprinting and iris recognition, then it becomes clear that facial recognition is intrinsically trickier than other biometric techniques. Facial recognition already works well and still has plenty of potential for improvement. Nevertheless, it is quite improbable that it will achieve the same levels of precision as iris or fingerprint recognition in the short term.

Comparison of biometric methods

Iris Fingerprints Face Two persons may resemble one another very closely One face per person

Uniqueness Every iris is unique Every fingerprint is unique Number of images Two irises per person to be acquired 10 fingerprints per person, plus other parts of the body where friction ridges are located

Stability over time Invariable from birth Invariable from childhood Representation in the dimensions Easy to represent in 2D and few problems due to orientation

Changes with age, state of health, etc.

Easy to represent in 2D

Intrinsically 3D

Distortions Pupil dilation Resolution Requires high resolution

Distortion limited by the elasticity of the skin Usually standard 500 dpi Strong command of the identification process by generations of fingerprinting experts

Highly variable, according to expressions Acquired at any scale Underdeveloped expertise

Maturity of the processes Underdeveloped expertise associated with the technology

15

Implementing a facial recognition system

The deployment of a facial recognition system must take both technical and human factors into consideration. Some of these factors, which are specific to facial recognition, are quite important. Integrating facial recognition in the existing technical environment
Photos are a very common form of data. Most potential users of facial recognition therefore already possess a number of information systems that can be interfaced with a facial recognition system:  databases of individuals (with portraits) or criminal cases other systems designed to compare biometric data biometric data acquisition systems There is almost always a business interest to be gained by integrating a facial recognition system with the existing environment.
Figure 20: identification verification screen within the verification application.

processes and the benefits of using a number of methods that increase the performance of searches and require operators to verify only the most difficult cases. Even if biometric searches are not natively consolidated, multi-biometrics does increase performance:  it allows cases to be processed, when one modality is absent or is of poor quality,  it can make more extensive, and consequently more efficient links. For example, in a criminal police system, if it is known that a first offense of bank card fraud and a second offense of shoplifting were committed by the same person, because the images show the same face, and if the bank card fraud is solved using fingerprints, then it is highly likely that the shoplifting offense will also be solved. Two isolated systems would not come to the same conclusion. In an effort to facilitate the integration of facial recognition with other information systems, Morpho has developed generic interfaces that meet the ANSI/NISTITL 1-2007 standard for the data format for fingerprints, faces and other biometric data.

Managing the expectations of customers and operators

Facial recognition seems to be so simple and intuitive that the expectations relative to this technique may be out of all proportion. The unsuccessful experiment in Tampa, Florida, USA in 2003 is one notable example. A facial recognition system was deployed in order to recognize wanted persons in a crowd. But the operators only received false alerts. Preliminary tests in a controlled environment and an elementary probability calculation could have concluded that the technology of the time was not suited to facial recognition in crowds. Therefore, it is advisable that every potential user of facial recognition conducts tests in order to assess the suitability of the technology to their operational

Integration with databases allows for a simplified recovery of existing data and allows legacy data acquisition processes to be used. By way of example, in order to add a facial recognition function to a civil status register in order to detect identity fraud, it is always preferable to keep the same civil status register and operate the facial recognition system in back office mode, without impacting the potentially complex processes that are used with the civil status register. Integration with other biometric systems provides for both optimized, redundancy-free data acquisition

16

Figure 21: on-the-fly portrait acquisition for traveller screening in airports.

applications before proceeding with deployment. The independent tests conducted in 2006 by the BKA(1) and also the NPIA(2) serve as an example. Research reports explain perfectly the state of the art, the tests conducted, the conclusions and the outlook. In order to conduct these tests, Morpho has developed a very simple pilot system containing the most recent advances in algorithms that can be installed and programmed in less than one day. In this way, it is possible to test facial recognition usage scenarios, check the results that may be obtained for a given target application and measure the workload required to obtain these results. Morpho can provide support to its potential customers in this assessment process.

Expertise in checking searches

Visually recognizing people may appear to be simple. When working on subjects that one knows well, and that are seen face to face rather than on a photo, even little children are capable of recognizing a face. But even known persons can be misidentified if they appear on blurred or old photos, or if they have an unusual expression. And when the person is unknown, the quality of the photos is variable and the angles and lighting differ, visual recognition becomes a tricky task, resulting in many errors detected by academic studies. A number of methods have been developed to improve the visual recognition of persons, but they are not as robust as the methods used to check fingerprints. The relevance of recognition criteria (e.g. the stability of wrinkles) remains to be scientifically established. Therefore, in order to avoid making mistakes and wasting time, it is necessary to develop methods and training for operators in the use of the facial recognition system. It is also important to cooperate with academic researchers and facial recognition technology vendors in order to make progress in terms of both the practices and the tools in this field.

Developing the expertise of operators

Acquiring facial images and recognizing people on photos is not easy and requires the development of specific expertise. Morpho can provide training in its products.

Expertise in data acquisition

If facial recognition is to be efficient, then the quality of the images in the reference database must be satisfactory. The criteria that measure this quality are defined by the ISO* standard, and the automatic acquisition systems are capable of verifying most of these criteria. But the judgment of an operator remains the best guarantee of good image acquisition. Automatic checks cannot be 100% reliable and the subject that is filmed may have particular physical properties that prevent certain criteria from being reached. Similarly, the operator is the person who is best placed to judge the suitability of an image for facial recognition searches. When acquiring an image in the field for an identity check, filming people behaving violently in a demonstration or choosing which images from many images of the same face should be used for search purposes, the operators expertise is decisive in making optimal use of facial recognition.

Helping the end users

End users react very differently to biometric systems. Reactions range from total hostility (against a society that some people feel is obsessed with security) to a certain amusement at being a pioneer in the use of new technology. Whether they be cooperative or hostile, they are all novices, and it is essential to give them clear and concise instructions on the behavior to be adopted. In applications for the general public, such as passport controls, it is impossible to support the users one by one. This is the reason why close attention must be paid to the ease of use of biometric tools for end users; they must be as enjoyable to operate as possible. The SmartGate* passport control system deployed in Australia meets this need. The end users are happy with the system and prefer automatic passport controls using facial recognition to conventional control gates.

(1)

BKA, Bundeskriminalamt, the German police authorities. NPIA: National Police Improvement Agency. An organization tasked with making technological recommendations to the British police force in order to improve efficiency.

(2) 

17

The market associated with automatic facial recognition

The market for facial recognition
Compared with other biometric techniques (fingerprinting, iris recognition, etc.), facial recognition accounts for 16% of the market (source: Frost & Sullivan, 2007, 2008). Also according to Frost & Sullivan, the value of this market is growing constantly, with an average annual growth rate of 54%. Totaling 72.7 million in 2006 and about 250 million in 2009, the facial recognition market should exceed 622 million by 2011 and 1 billion in 2013. These estimates correspond to all the links in the value chain of developments in the field of facial recognition.

Driving forces and obstacles

A number of different factors drive or hinder the development of facial recognition, ranging from technology to politics, applications and even standards.

The driving forces

16%

84%

 The creation of an international standard (ICAO) for travel documents. This standard specifies three possible forms of biometrics: iris recognition, fingerprinting and facial recognition. It has resulted in the creation of a reference database of high quality portraits.

Figure 22: Face recognition share in the biometric market.

Facial Recognition: value chain

Components / Imaging sensors

Face Tech. Development

SW Development

System Integration

1- 2D/3D cameras 2- Photo-video

Development of high accuracy face recognition algorithms, in 2D and 3D, for static images and video

Development and integration of FR utilization in vertical markets

Deployement or upgrade of integrated FR oriented solutions in public security market

Figure 23: Facial recognition: value chain.

18

 The highly favorable conclusions of FRVT 2006. Facial recognition of good quality images is a mature biometric technique and the algorithms continue to progress, opening the way for the combination of 2D and 3D facial recognition techniques.  Existing implementations. Notably, governmental or inter-governmental identity programs (electronic passports, ID cards, driving licenses, etc.) and automatic border passport control programs. These implementations have been a real success and the expectations of potential users are becoming more precise and realistic.  The proven benefits of multi-biometrics. Multibiometrics cut staffing costs in biometric research, can solve difficult cases in which the data from one of the biometric techniques is of poor quality and can

correlate the connections made by different biometric techniques.  T he availability of video cameras with improved resolution. These cameras take better pictures, which allow for more precise facial recognition.

The obstacles
 Facial recognition is intrinsically more difficult than other major biometric techniques.  Facial recognition is often quoted as an infringement of civil liberties.  A pplications are broader than for other biometric techniques and the potential new users must be introduced to the field of biometrics and understand its benefits.

19

The experience of Morpho in the field of facial recognition

Morpho has been active in the field of facial recognition since 2002. Some of the most significant milestones in our activity include:

 2002 assessment of the algorithms on the market: we opted to start with technology from Cognitec, which was found to be the best on the market at the time by the NISTs FRVT 2002. Since then, Morpho has significantly improved the technology.  Morpho deployed a pilot tool designed to assess the technology with simple and efficient user interfaces. The tool allowed numerous police agencies to test the possible applications of facial recognition. This pilot allowed a number of conclusions to be drawn: The Morpho subsidiary MorphoTrak received the Best Biometric Identification Technology award for its facial recognition pilot system when it was exhibited at the Global Border Security Conference and Expo in Austin, Texas in May 2008. The Australian police force concluded that facial recognition offered decisive benefits for the correlation of criminal cases and now uses Morpho technology in operational applications.

The University of Lausanne, in cooperation with the Romande regional police force in Switzerland, successfully developed a strategy for the use of Morphos facial recognition for demonstrations. The Pierce County Sheriffs Office in Washington, USA, demonstrated how Morphos facial recognition reached a level of precision higher than 94% when identifying individuals in its collections, even with significant differences in age. It also surmised that the combined use of Morphos facial recognition and fingerprinting technology could allow subjects to be identified without calling on fingerprinting experts. In 2008, the Mexican police confirmed that Morphos technology was the fastest and the most accurate on the market in operational tests. In 2009, the Paris police authorities helped investigators by piloting a Morpho facial recognition system on a reference database of 470,000 faces. Morpho developed the SmartGate border control system that is now used in all international airports in Australia and will soon be deployed in New Zealand. This project uses the photos on biometric passports to speed up and facilitate passport controls. By the end of 2008, 150,000 people had passed through a SmartGate.

20

 Morpho also developed MorphoFace Investigate* on the basis of recommendations made by our users working in dedicated Focus Groups. This product is designed to perform identification tasks, to help the police to solve criminal cases or simply for identification purposes in civil applications.  Morphos research laboratories continuously develop and improve the basic technology. Research focuses on facial recognition algorithms, but also the optimization of portrait acquisition and the extraction of portraits from surveillance video footage. As a result of this research work, Morpho finished first in the NISTs portal challenge in March 2009. In May 2009, Morpho received the HCFDC (French High Committee for Civil Defense) innovation trophy for its Face on the fly face acquisition technology.

Morpho takes part in cooperative research projects in the United States and Europe, including the 3D Face project, which has concluded that the use of both 2D and 3D facial data improves the precision of facial recognition for travel document applications (http://www.3dface. org/). Morpho is making significant investments in innovation and the development of its facial recognition technology in order to consolidate its position as leader in biometrics. Major advances are continuously being made in terms of both the quality and diversity of the algorithms and the development of dedicated products adapted to the needs of the market.

Websites:
http://face.nist.gov/mbgc/mbgc_presentations.htm http://www.hcfdc.org/trophees2009/palmares.php http://www.3dface.org/ NB : On May 27 2010, Sagem Scurit changed its name to Morpho. For events, benchmarks having taken place before this date, we are listed or quoted under the name of Sagem Scurit.
21

Glossary and acronyms

A
Alert - An alert is an automatic and interactive event that is generated when analyzing video streams. Alerts are generated when the recognition algorithms conclude that a person seen in the video is very probably contained in the reference database. Alerts appear in the user interface by displaying the image of the person spotted in the video next to the image in the reference database.

I
ICAO - International Civil Aviation Organization. An international organization that is part of the United Nations. Its mission is to contribute to the development of standards used to standardize international air transport ISO - International Organization for Standardization. An international organization made up of the national standardization institutes from more than 100 countries.

N
NIST - National Institute of Standards a n d Te c h n o l o g y. T h e A m e r i c a n standardization organization and a member of ISO.

R
Reference database - A database of persons of interest to be identified in the images and videos to be processed.

F
FA - False Acceptance. False acceptances and false rejections are used to measure the performance of a biometric system. False acceptances correspond to fraudsters that the biometric control fails to detect. Face on the Fly - Face On The Fly is an innovative technology developed by Morpho. The purpose of this technology is to acquire facial images when a person passes through a control gate, without stopping and without having to look at a particular camera. Several images are acquired by a series of cameras to create a three dimensional view of the face. A frontal projection of this image can then be compared and used for authentication or identification purposes. Face On The Fly technology won the French High Committee for Civil Defenses award for technological innovation in May 2009. FR - False Rejection. False rejections correspond to authorized users that the biometric control fails to recognize. Frame - An image extracted from a video recording or stream. FRVT - Face Recognition Vendor Test.

S
SmartGate - SmartGate is a project run by Australian customs. The purpose of the project is to speed up customs clearance in Australias international airports using the portraits on electronic passports and facial recognition. Morpho deploys a system in Australian airports as part of this project. Stereoscopy - Stereoscopy refers to all the techniques used to reproduce a perception of a contour from several flat images. SVM - Support Vector Machine. A prominent technique used to solve classification problems.

L
Line-up - Line-up techniques are used by the police to determine whether a witness has spotted a suspect. The suspect is presented to the witness amongst a group of physically similar people. Witnesses must then decide whether they recognize one of the members of the group.

M
MBGC - Multi Biometric Grand Challenge (http://face.nist.gov/mbgc/). A test organized by the NIST to make advances in research into the recognition of persons from a distance using facial recognition and iris recognition. MFI - MorphoFace Investigate. A facial recognition system developed by Morpho. This system is essentially designed for use in police investigations. It can be used to solve cases from portrait traces left on the scene of the crime. Modality - A type of biometrics.

T
Template - The code extracted from an image of a face by image processing. Facial comparison is carried out via extracted templates. Terahertz - An electromagnetic wave in the electromagnetic spectrum between infrared (the optical domain) and microwaves (the electronic domain).

22

Bibliography
ForshungsProjekt, Gesichtserkennung als Fahndungshilfsmittel, Foto-Fahndung, Abschlussbericht, BKA, 2007, www.bka.de/kriminalwissenschaften/fotofahndung/pdf/ fotofahndung_abschlussbericht.pdf - See also: http://www.bka. de/kriminalwissenschaften/fotofahndung/faq.html Automatic Face Recognition, applications within law enforcement, Market and technology review, Ambika Suman, October 2006, www.npia.police.uk/en/docs/Face_Recognition_ Report.pdf CCTV on trials, Josh P Davis and Tim Valentine, Goldsmiths, University of London, London, UK, 2007. Morphological Classification of Facial Features in Adult Caucasian Males Vanezis, et al. - Journal of Forensic Sciences, 1996. Low-dimensional procedure for the characterization of human Limitations in Facial Identification: The Evidence , R. Jenkins, M. Burton, Department of Psychology, University of Glasgow, Glasgow, UK, Justice Of The Peace Volume 172, January 2008. Quality Standards for Forensic Opinions on the Identity of Living Offenders in Pictures, 9th Biennial Scientific Meeting of the International Association for Craniofacial Identification, F.W. Rosing, Institut fur Humangenetik und Anthropologie Universitatsklinikum, Ulm, Germany. Facial Comparisons by Subject Matter Experts: Their Role in Failure of Anthropometry as a Facial Identification Technique Using High-Quality Photographs, Kleinber, Pharm, Vanezis, and Burton - Journal of Forensic Science, July 2007. Photo identification: facial metrical and morphological features in South African males, Roelofse, Steyn, Becker, - Forensic Science International, 2008. Facial comparison area of expertise: formulation of the issues, and international inventory, Arnout C. Ruifrok, Ph.D, Laan van Ypenburg 6, 2497 GB Den Haag. Biometrics and Their Training, Federal Bureau of Investigation Forensic Audio, Video and Image Analysis Unit, Nicole A. Spaun, Advances in Biometrics, June 2009. World Biometrics Market, Frost & Sullivan (2007). Privacy and technologies of identity: a cross-disciplinary conversation, Katherine Jo Strandburg, Daniela Stan Raicu, Springer, 2005, ISBN 0387260501, 9780387260501, page 146. Prfecture de Police de Paris, Identit judiciaire, Etude du signalement descriptif, Portrait parl. faces. L Sirovich and M. Kirby, Journal of the Optical Society of America A 4: 519524, 1987. Face recognition using eigenfaces, M. Turk and A. Pentland, Proc. IEEE Conference on Computer Vision and Pattern Recognition: 586591, 1991. Eigenfaces for recognition, M. Turk, A. Pentland, Journal of Cognitive Neuroscience, 3(1), 1991. ISO/IEC 19794-5, Biometric Data Interchange Format - Part 5: Face Image Data. Forensic Art and Illustration Karen T. Taylor, CRC Press, 2000, ISBN 0849381185, 9780849381188. ANSI / NIST: Data Format for the Interchange of Fingerprint, Facial, & Other Biometric Information.

Polyvalence.com - Photos : Getty Images / Fotolia / PhotoAlto. Printed on paper made from responsibly managed forests - D-10156

Phone: +33 (0) 58 11 88 76 - Fax: +33 (0) 58 11 87 81 www.morpho.com Registered Office: Le Ponant de Paris - 27, rue Leblanc - F-75512 PARIS CEDEX 15 - France Socit anonyme au capital de 159.876.075 e 440 305 282 RCS PARIS