A Project Report On College Attendance System based on Facial Recognition Submitted to Amity University, Uttar Pradesh

In partial fulfilment of the requirements for the award of the degree of Bachelor of Technology In Computer Science & Engineering By Rohan Khurana Under the guidance of Ms. Divya Sharma Department of Computer Science & Engineering Amity School of Engineering & Technology Amity University, Noida Uttar Pradesh April, 2012

DECLARATION I, Rohan Khurana, student of Bachelor of Technology (Computer Science & Engineering) hereby declare that the project titled “College Attendance System based on Facial Recognition” which is submitted by me to Department of Computer Science & Engineering, Amity School of Engineering and Technology, Amity University Uttar Pradesh, Noida, in partial fulfilment of the requirement for the award of the degree of Bachelor of Technology in Computer Science & Engineering, has not been previously formed the basis for the award of any degree, diploma or other similar title or recognition.

Place: Noida Date: 30th April 2012 Rohan Khurana

ii

CERTIFICATE

On the basis of declaration submitted by Rohan Khurana, student of B. Tech (Computer Science & Engineering), I hereby certify that the project titled “College Attendance System based on Facial Recognition” which is submitted to Department of Computer Science & Engineering, Amity School of Engineering and Technology, Amity University Uttar Pradesh, Noida, in partial fulfilment of the requirement for the award of the degree of Bachelor of Technology in Computer Science & Engineering, is an original contribution with existing knowledge and faithful record of work carried out by him/them under my guidance and supervision.

To the best of my knowledge this work has not been submitted in part or full for any Degree or Diploma to this University or elsewhere.

Noida Date

(Ms. Divya Sharma) Department of Computer Science and Engineering Amity School of Engineering and Technology Amity University Uttar Pradesh, Noida

iii

ACKNOWLEDGEMENT “College Attendance System based on Facial Recognition” is a biometrics oriented project aimed at presenting some concepts from various computer science disciplines with a programming approach. An effort has been made to present this material in a very lucid manner followed by scrutinized observation to every step of the source code so implemented. I take this opportunity to express my deep regards to respected Ms. Divya Sharma, Dept. of Computer Science & Engineering, A.S.E.T, AUUP. I am indebted to her for providing me the necessary counselling and support. I am deeply indebted to our mentor Miss. Divya Sharma for encroachment upon her time. I would like to express a heartfelt gratitude for her insight, expertise and energy contributed greatly to this project. In addition to this I acknowledge my debt of gratitude to the numerous authors whose great masterly work I have consulted during the preparation of this project. Last but not the least; I would like to thank my parents for their sound counselling and cheerful support. I hope that this project on “College Attendance System based on Facial Recognition” in the present form caters to the needs of all concerned.

Rohan Khurana

iv

ABSTRACT

This research aims at providing a system to automatically record the students’ attendance during lecture hours in a hall or room using facial recognition technology instead of the traditional manual methods. The objective behind this research is to thoroughly study the field if pattern recognition (facial recognition) which is very important and is used in various applications like identification and detection.

v

1 Logical Data Flow (DFD) 3.3 Proposed Scope and Enhancement CHAPTER 2 Literature Review and Prior Research 2.3 Internet Connection 3.1 System Development Requirements 3.2.1.2.1 Biometric Recognition 2.1 Hardware 3.1.3 Types 2.1.2 Facial Recognition Techniques 2.2 Problem Definition 1.2 Data Structures 3.2 Software 3.1.2 Advantages 2.1.2 System Analysis and Design 3.2.1 Automatic Facial Recognition Process Flow 2.1 Background of the Study 1.4 Modules of a Biometric System 2.1.2.1.2.3 Limitations of Face Recognition Technologies CHAPTER 3 Materials and Methodology 3.CONTENTS Declaration Certificate Acknowledgements Abstract Contents CHAPTER 1 Introduction 1.3 Database Design(ERD) vi iii iv v vi vii 1 1 1 2 3 3 3 4 5 8 10 10 12 14 18 18 18 18 18 19 19 23 24 .1 Emergence 2.2 Facial Recognition 2.2.

3 Principal Component Analysis 4.6 MATLAB Implementation 4.2.2 PCA based Face Recognition Algorithm 4.5 Flow Chart 4.1 Description 4.2.CHAPTER 4 Implementation 4.3 Calculation of Eigenfaces with PCA 4.4 Recommendations 4.2 Average Face Output 6.1 The Basic User Interface 6.3.2.2 Colour Based Techniques 4.2 Problems Faced 4.3.3 Recognition Output CHAPTER 7 Conclusions CHAPTER 8 Future Prospects CHAPTER 9 References 25 25 25 25 26 26 26 27 28 28 29 35 39 40 41 41 42 43 43 44 45 46 47 48 vii .4 Classifying Images 4.1 Introduction 4.2.3.2.3.5 Input and Output 4.7 Database Implementation CHAPTER 5 Discussion of Result CHAPTER 6 Presentation of Result 6.1 Description 4.3.3.3 Limitations 4.3.

Biometric recognition has the potential to become an irreplaceable part of many identification systems used for evaluating the performance of those people working within the organization. the lecturer delays the lecture to record students’ attendance. the lecturers tend to call a couple of student names at random which is not a fair student evaluation process either. This is mainly because these applications help the top-management take decisions that improve the performance and effectiveness of the organization. It also causes a lot of disturbance and interruption when an exam is held. these attendance records are used by the staff to monitor the students’ attendance rates. 1 . (signature recognition. facial recognition).1 Background of the study The rapid development in the field of pattern recognition and its uses in different areas e. it needs accurate and fast means of recording the performance of the people inside this organization. Moreover the attendance sheet is subjected to damage and loss while being passed on between different students or teaching staff. arises the importance of the utilization of this technology in different areas in large organizations. for an organization to be effective. On the other hand. Although biometric technologies are being applied in many fields it has not yet delivered its promise of guaranteeing automatic human recognition. This research is the first of its kind to attempt to provide an automated attendance system that recognizes students using face recognition technology through an image/video stream to record their attendance in lectures or sections and evaluating their performance accordingly. This is a lengthy process and takes a lot of time and effort. 1. especially if it is a lecture with a huge number of students. This process could be easy and effective with a small number of students but on the other hand.1.g. Introduction 1. Finally.2 Problem definition Every time a lecture or laboratory starts. dealing with the records of a large number of students often leads to human errors. And when the number of students enrolled in a certain course is huge.

laboratory or exam at its specific time. thus saving time. In addition.1. effort and reducing distractions and disturbance.3 Proposed scope and enhancement This project proposes solutions to all the above mentioned problems by providing an automated attendance system for all the students that attend a certain lecture. Another advantage concerning exams. is when the invigilator or the advisor accidentally loses an exam paper or the student lies about attending the exam. there will be a record of the students’ attendance for the exam at that time. thus protecting both lecturer’s and student’s rights. an automated performance evaluation would provide more accurate and reliable results avoiding human error. 2 .

the human mind automatically recognizes different people depending on their external physical and behavioural features. 2. such as badges or cards. Generally speaking. whether it is in identification mode where one-to-many comparisons take place. Each of these three authentication methods has its advantages and disadvantages. involving the statistical analysis of biological characteristics”. and finally something you are. eye and hair colour to identify individuals for business transactions.1. 3 . ranging from simple business interactions to criminal investigations. an in-depth study on facial recognition is conducted for it is the technology to be used by our proposed project. 2. The literature is divided into two main parts.2. a study of previous attendance systems and its atomization attempts was conducted to examine the similar previous systems. While the ancient Jordanian have commonly used physical traits such as height. scientists and researchers consider biometric recognition systems as high-level security systems. Literature review and prior research I have studied the literature available in the fields of pattern recognition and biometric systems with a focus on facial recognition. Biometric recognition generally matches a live digital image of a portion from a certain physical body part with a previously recorded image of that same portion.1 Emergence For thousands of years.1 Biometric recognition: There are three different types of authentication: something you know. which mainly depends on biometrics and physical traits. In the second part. humans have used biometrics and physical characteristics such as face and voice to identify other human beings for a wide variety of purposes. scars. In addition. and each is considered appropriate for certain types of application. Among there three types. The study of fingerprint dates back to ancient China. or verification (authentication) mode where one-to-one comparisons occur. They define the biometric term as the “science. something you have. the first part examines the different biometric characteristics and systems. such as passwords.

For example.2. in a way imitating the human capabilities. authentication of a regular user with iris recognition would take two to three seconds. which are almost impossible to change. While some others object and point out that they are not a secret and could be in many cases falsified or stolen from computer systems. and using real human physical characteristics. it is important to point out that those technologies that are able to support biometric based applications are becoming more and more available. And finally. Another advantage of using biometrics would be speed in doing work. Moreover the normal human perception system has an impressive capability of recognizing and distinguishing different faces.2 Advantages The main advantage of biometric systems over normal automated systems is that they really do what they are supposed to do. In addition. loss or passing to anyone else like what is done with cards or passwords. some researchers proposed that biometrics are not subjected to theft. which make biometric based systems more available to users. which is authenticating the user. This limitation is overcome by the computerized facial recognition systems that can mimic the human mind capability in addition to storing and processing as much people as necessary. while using a key to open the door would need five to ten seconds.1. but this capability is limited by the number and type of faces that can be easily processed. 4 .

Different tools maybe used in these systems such as standard video. almost permanent. 2. facial hair. glasses.1.  Iris: It is the coloured tissue representing the ring surrounding the eye pupil. sophisticated and complex artificial intelligence and machine learning is needed to be able to compare and match human faces with different poses. software. or still imaging or thermal imaging.1 The Eye Analysing the eye is generally thought to present the highest levels of accuracy and uniqueness.2. biometric systems now come in all different shapes and sizes. certain modules or complete solutions. therefore it is unlikely that a dead iris could be used to by-pass a biometric system.2 Face For a computerized system to mimic the human ability in recognizing faces. OEMs. Each person’s iris has a unique structure and a complex pattern. collectable and clear enough not to fool the capturing system. we will quickly go through the different biometric types used for person detection. it is believed that artificially duplicating an iris is virtually impossible. In this section of our literature review.1.  Retina: It is the layer of blood vessels situated at the back of the eye.3.3. the retina forms a unique pattern and decays quickly after death. It is also known that the iris is from the first body parts decaying after death.3 Types Any human body characteristic that is considered universal. It ranges from hardware. They can be divided into two different technologies: iris biometrics and retina biometrics. Just as the iris. 5 . In addition. can be used as a biometric feature if it is accepted to be used by humans. distinctive. Retina recognition systems are complex but at the same time regarded as the most secure biometric system. etc. Besides that. That is why these systems depend greatly on the extraction and comparison engines. 2.1.

1. The shape and size of the vocal tract. where the print and the patterns of ridges. That is why it is important to distinguish between this technology and those technologies recognizing words and commands. 2. Voice recognition techniques may either use text-dependent or text-independent methods.  Hand geometry: Just as with the finger geometry. voice may be captured by speaking out any phrase.5 Voice Voice biometrics focus on the sound of the voice. Although it is not very accurate. However this biometric system may be 6 . in addition.4 Hand There are mainly three biometric systems using the characteristics of the human hand. in addition to finger geometry which concentrates on the shape of the finger rather than the print itself.3. This technique is called the “vein check” which examines the unique pattern of the blood vessels or what is called the “vein tree” which can be captured by the infra-red light. and Palm identification.  Vein pattern analysis: The past number of years has witnessed much development in the area of analysing the patterns of the veins in the back of the human hand.  Palm identification: Palm biometrics is very similar to the study of finger print. valleys and minutiae found on the palm are analysed and studied. the hand geometry analysing system captures a three-dimensional image of the hand and measures the shape and length of fingers and knuckles. Hand geometry was from those first biometric systems used for access control applications. In other words. not on what is being said. Vein pattern analysis.1. word or number (text-independent) or by specifically saying a certain password combining phrases.3 Finger In this type of study.2. the shape of the mouth and the nasal cavities all contribute in the way a voice sounds. it is thought to be very convenient and large amounts can be processed quickly. These are: Hand geometry.3.3. words or numbers (text-dependent).1. 2. the finger tips are analysed for its unique pattern and print produced by the finger minutiae. The sound of the human voice is caused by vibration in the vocal chords.

7 .challenged by the background noise that reduces the quality of the data and the system’s performance.

1.4. for example. 2.4.1. voice recorder. video camera.1. 2. infrared… etc. 8 . Different devices are used for different biometric system according to the feature that is being captured.2 Feature Extraction Module This module processes the captured biometric data to extract a set of discriminatory features forming a template.1. the number of matching minutiae points determine the matching score and consequently the match or not results.1 Sensor or Capturing Module This module captures the biometric data.4 Modules of a Biometric System Biometric systems are designed using the following four modules as explained in figure 1: Figure 1: Typical biometric recognition system 2. thermal camera.3 Matcher Module In this module.4. For example: in a fingerprint based biometric system.2. For example: the position and orientation of minutiae points in a fingerprint image are extracted in the feature extraction module of a fingerprint-based biometric system. the features extracted from the previous module are compared to those stored and decisions regarding matching score are made.

1. The enrolment module is responsible for enrolling individuals into the biometric system database.4 System Database Module It is used by the biometric system to store the templates of the enrolled users. 9 .2.4. where the biometric characteristic of an individual is first scanned by the sensor to produce a digital representation (initial stored template) of the characteristic.

alignment. In the case of video streams.2 Face recognition processing flow The facial recognition process can be divided into two main stages: processing before detection where face detection and alignment take place (localization and normalization). 2. feature extraction. ears and mouth. Afterwards normalization with respect to geometrical transforms such as size and pose.1 Automatic Facial Recognition Process Flow Generally any biometric system goes through the same processes of the four modules explained earlier.5. where the detection step roughly estimates the position of the face.2. and afterwards recognition occur through feature extraction and matching steps. eyes. nose. The facial recognition process is similar to the general biometric recognition process.  Face Alignment: This process focus on finding the best localization and normalization of the face.  Face Detection: This process separates the facial area from the rest of the background image. 10 . this step outlines the facial components. In the face-base biometric systems detection. in addition to photometrical properties such as illumination and grey scale take place. feature extraction. and comparison with templates available in the database as explained in figure 1. biometric capture. faces can be tracked using a face tracking component.2 Facial Recognition Face recognition is considered to be one of the most successful applications of image analysis and processing.2. and matching take place. such as face outline. That is the main reason behind the great attention it has been given in the past several years. Figure 2.

 Feature Extraction: After the previous two steps.  Face Matching: The extracted features are compared to those stored in the database. 11 . and decisions are made according to the sufficient confidence in the match score. feature extraction is performed resulting in effective information that is useful for distinguishing between faces of different persons and stable with respect to the geometrical and photometrical variations.

5 Geometrical Feature Matching Geometrical feature matching techniques are based on the computation of a set of geometrical features from the picture of a face. 2.2. which can be associated with the states of a hidden Markov model.. 12 .e. individuals) increases.. This method would be less sensitive to appearance changes than the standard Eigen face method. 2.3 Graph Matching Graph matching is another approach to face recognition presented a dynamic link structure.2.1 Eigen faces Eigen face is one of the most thoroughly investigated approaches to face recognition. Current automated face feature location algorithms do not provide a high degree of accuracy and require considerable computational time.2.2.2. nose. 2. In general. Faces were intuitively divided into regions such as the eyes. dynamic link architecture is superior to other face recognition techniques in terms of rotation invariance. In general. 2. however. the matching process is computationally expensive.2.2.4 Hidden Markov Models (HMMs) Stochastic modelling of non-stationary vector time series based on (HMM) has been very successful for speech applications applied this method to human face recognition.2 Neural Networks The attractiveness of using neural networks could be due to its non-linearity in the network.2. 2. etc. neural network approaches encounter problems when the number of classes (i. mouth.2.2.2. Facial Recognition Techniques This section gives an overview on the major human face recognition techniques that apply mostly to frontal faces.2.2.

2.2.7 3D Morphable Model The morphable face model is based on a vector space representation of faces that is constructed such that any convex combination of shape and texture vectors of a set of examples describes a realistic human face. such as the Euclidean distance.6 Template Matching A simple version of template matching is that a test image represented as a twodimensional array of intensity values is compared using a suitable metric. 13 . In general.2. 2.2. template-based approaches compared to feature matching are a more logical approach. with a single template representing the whole face.2.

face recognition technology. it is still quite limited in its applied scope. As explained in figure 3. they can be categorized in four main challenges: 2.2. has not yet delivered it promise. Many researchers have identified different problems for the biometric system.1 Accuracy Two biometric samples collected from the same person are not exactly the same due to the imperfect imaging conditions. Figure 3: Biometrics System Errors 14 . the face recognition technology is not robust enough to handle uncontrolled and unconstrained environments. In consequence. In addition.3 Limitations and Challenges of Face Recognition Technologies As mentioned earlier. In spite of all its potentials.3. inaccuracy can occur in two different forms. the results accuracy is not acceptable. just as any other biometric technology. either False Non-Match (false reject / Type 1 error) in which the system falsely declares the failure of match between the instance and the correct stored template.2. or False Match (false accept / Type 2 error) in which the system incorrectly declares a successful match between the instance and one of the templates in the database.2.

figures 4. aging. pose. lighting. hairstyle and accessories. Figure 4: Position. 5 and 6 present examples for some types of variations. expression. This challenge is reduced as more training data is available to enhance the performance of the biometric system.These errors are mainly caused by the complexity and difficulties of the recognition process because of the uncontrollable variables such as lighting. weight gain or loss. expression Figure 5: Occlusion/blockage/hiding of some features Figure 6: Ageing 15 .

such as the American Civil Liberties Union (ACLU) which opposes the use of face recognition software at airports due to privacy concern. according to some studies conducted. This is caused by two main reasons:  Biometrics is not a secret: This means that anyone including the attacker knows exactly the biometric features of the targeted user.2. a facial authentication takes about 90 msec. an age range can be determined… etc.4 Privacy The issue of using recognition based systems has raised many concerns of possible privacy violations.2.2. The database of a biometric system hold irrefutable proof of one’s identity. This can be reached by performing course pattern classification.2. is very questionable. accuracy and performance in case of identification systems. In this way. it is absolutely necessary to find more efficient ways for biometric comparison. For example: the captured face is firstly classified into male or female.  Biometrics is not recoverable: This means that one cannot change his face in case it became compromised. while a large scale ID throughput needs about 0. which is a major concern in many locations. Unlike the verification where one to one comparison takes place.3. then.2.3.3.66 min. in biometric identification systems. The large intra-class variation still represents a great challenge in the field of pattern recognition. But in fact. security of biometrics (especially face). where each new instance is compared to all those in the database to find a match. Scale The number of entities enrolled in the database of the biometric system greatly affects the speed. 2. where many studies are being conducted to find the best way for indexing patterns similar to the ways used in conventional databases. As mentioned in this article. Security Facial recognition and other biometric systems are used for many security applications. and there are no regulations or guarantees on how these 16 . 2. claiming that biometrics is a secure way of authenticating access. the search in the system’s database can be narrowed down in an efficient way.3. That is why.

information might be used or what it could be used for. This would be achieved through managing data protection and encryption along the biometric system. 17 . These privacy issues mostly result in the reluctance of users to use these biometric systems. On the other hand some people propose that biometrics is a privacy protection tool rather than intrusion to civil rights.

3.1.1.4 MHz or higher clock rate and 2 MB or higher cache   3GB RAM or higher 250GB storage space 3.1 System Development Requirements 3.3 Materials and Methodology This chapter enlists the system requirements for the development of this project as well as the system designs implemented for the project.1.3 Internet Connection 1) Any way of connecting to the internet (For confirmation emails) 18 .2 Software 1) Operating system (Windows 7) 2) MATLAB 3) Database (MySQL) 4) Video for Windows (VFW) 5) Camera Driver 3.1 Hardware 1) A camera 2) A desktop or laptop computer with the following specifications:  Intel core 2 duo processor with 2.

processes that are done by the system and the dataflow between them. and under what circumstances. Low diagram (level 2) ( Figures 9 and 10) has a purpose of breaking down the small. main processes into even smaller processes (children processes) to simplify their work to become clearer to the reader. It has no further details.1 Logical Data Flow (DFD) The data flow diagram is a diagram that shows how the data manipulated by a system flows through the various processes. System with proposed automated attendance module: In the figures below is the proposed system data flow that will be operating on automated facial recognition attendance recording methods. Together they wholly and completely describe the parent process. nor where the data will be stored (all of which are shown on a DFD). but not what kinds of data will be input to and output from the system. and combined must perform its full capacity.2. allowing a reader to determine what operations will be performed. The third level of DFD.2 System Analysis and Design 3. one main process only and the data flow between them. which shows the flow of control through an algorithm. It provides no information about the timing or ordering of processes. It is therefore quite different from a flowchart. nor where the data will come from and go to. 19 .3. Then is drawn the next level of DFD which is High level diagram (level 1) ( Figure 8) in which the data stores are shown and the main single process is broken down into the major highlevel. or about whether processes will operate in sequence or in parallel. DFD consists of four main symbols which are:    External Entities Data stores Processes Dataflow lines The system will be illustrated on three different levels of DFD which are Context diagram (level 0)( Figure 7) which consists of all the external entities. in what order. This decomposition of the parent process is called explosion of the process.

level 1) 20 .Context Diagram (Level 0): Figure 7: (Context Diagram.Level 0) High Diagram (Level 1): Figure 8: (High Diagram.

We will only examine the segmentation fig and the face detection fig 3. Figure 9: (Low Level Diagram.4 processes because these are the processes where the automation and most of the project work will be take place.Low level Diagram (Level 2): This level is the level where we explode or break down the process into smaller ones.Level 2 (Process 2)) 21 .

Figure 10: (Low Level Diagram.Level 2 (Process 3)) 22 .

3.2.2 Data Structure Course DS = Course ID + Course Name Student DS = Student ID + First Name + Last Name + Email Attendance = Attendance ID + Attendance Date Attendance Date = Day + Month + Year 23 .

2.3.3 Database Design (ERD) A diagram that pictorially represents entities. data types and relations between the tables. Figure 11: (ERD) 24 . The figure below illustrates the attributes of each entity and the relationships between them. the relationships between them and the attributes used to describe them in a database. This figure shows the types of the entities and their lengths.

4 Implementation 4. 25 .1 The new image is scanned pixel by pixel and if each pixel satisfies the condition stated in Equation 4. which depends on the detection of the human skin colour with all its different variations in the image.1 Introduction In this project different techniques have being studied like colour based detection and Principle Component Analysis (PCA) for face detection and for feature extraction.071 * B) + 128 Equation 4.(0. PCA can also be used for detection. HCbCr where H is the Hue component of the HIS colour space and the Cb and Cr components are calculated by the formulas stated in Equation 4. 4. Cb= (0. which is the range of the human skin.1 Description In this technique the image is converted into a new colour space.439 * B) + 128 Cr= (0.368 * G) .439 * R) .2.148 * R) .(0.2 Colour based technique 4. PCA and Linear Discriminate Analysis (LDA). PCA technique has been implemented which a statistical approach that deals with pure mathematical matrixes not image processing like the colour based technique used for detection. For detection.291*G) + (0. its value is set to white else it is set to black. The skin area of the image is then segmented and passed to the recognition process.1. For recognition.2.(0. Colour based technique was implemented.

26 . which also supports multiple face detection. But it reduces the face area a little. It is tremulously affected by the environment’s lighting conditions.3 Limitations 1.2. 4.2 The resultant image is converted into black white colour space in which pixels have values of 0 and 1 only (black and white).1 ) && (Cr(x.If (H(x. A better technique to solve these problems is to use PCA in the detection of the face. It doesn’t support the detection of multiple faces.y)<=195) Set pixel value to white Else Set pixel value to black Equation 4. which is acceptable.01) && (H(x. This algorithm should be applied on one face at a time. To use this technique the lighting conditions should be well controlled. 4. 4.y)>=140) && (Cb(x. 2.2 Problems faced After applying this technique there was a problem that there was noise that tremendously affected the segmentation process so a noise removal filter was implemented to successfully eliminate this problem.y)>=.4 Recommendations 1. 2. The black white image is sent to the segmenting function which gets the x axis and y axis coordinates of the face area to crop it out of the original image to remove the rest of the data in the image that is not important like the background.y)>=140) && (Cr(x.y)<=0.2.2.y)<=165) && (Cb(x.

2.5 Flow chart Figure 12: Colour based technique 27 .4.

the extractions of the most applicable information in a face image is desired. to what degree the specific features (eigenface) is present in the original image. which may or may not be present in the original image. Hence. Therefore.4. one has to build a kind of weighted sum of all the eigenfaces. and then encoded as effectively as possible. losses due to omitting some of the eigenfaces can be minimized. the share of the corresponding eigenface in the sum of the eigenfaces should be greater.1 Description Based on an information theory approach. On the other hand if the particular feature is not present in the original image.3. with each eigenface having a certain weight. 28 . Eigenfaces are the characteristic features of the faces. Each eigenface represents only certain features if the face. the corresponding eigenface should contribute a smaller part to the sum of eigenfaces. However. the reconstructed original image is equal to a sum of all eigenfaces. Indeed.3 Principal Component Analysis 4. That is. In PCA. If all the eigenfaces extracted from the original images are used. Therefore one could say that the original face image can be reconstructed form eigenfaces if one adds up all the eigenfaces (features) in the right proportion. the reconstructed image is an approximation of the original image. in order to reconstruct the original image form the eigenfaces. But using only a part of the eigenfaces is applicable. which is achieved by selecting only the most important features (eigenfaces). In the language of information theory. If the feature is present in the original image to a higher degree. This weight specifies. Then it compares one face encoding with a database of models encoded similarly. one can transform each original image of the training set into a corresponding eigenface. PCA decomposes face images into a small set of characteristic features images called “Eigenfaces” that can be described as the principal components of the initial training set of images. one important feature of PCA is that the reconstruction of any original image from the training set by combining the eigenfaces is possible. one can reconstruct the original images from the eigenfaces exactly.

by projecting their face images onto the face space. Similar faces (images) possess similar features (eigenfaces) to similar degrees (weights).3. 4. Calculate the eigenfaces from the training set. The coming sections will elaborate the steps needed to perform the PCA using eigenfaces on a set of images in detail. 29 . Thus. the following steps are used to recognize new faces: Calculate a set of weights based on the input image and the M eigenfaces by projecting the input image onto each of the eigenfaces. it is possible not only to extract the face from eigenfaces given a set of weights. These M images define the face space.2 PCA-based face recognition algorithm The approach to PCA-based face recognition involved the following initialization operations: 4. the image probably not a face. 2. the eigenvalues can be updated or recalculated. keeping only the M images that correspond to the highest eigenvalues. In the case the weights of the image differ too much from the weights of face images. Having initialized the system. the images could be grouped to clusters. but to the extract the weights from eigenfaces and the face to be recognized. Determine if the image is a face at all (whether known or unknown) by checking to see if the image is sufficiently close to the face space.3. all images having similar weights are likely to be a similar face.1 Initialization Acquire an initial set of face images (the training set). Calculate the corresponding distribution in M-dimensional weight space for each known individual. Therefore. Check whether the image is a face.Moreover. using these weights one can determine two important things: 1. These weights act as the amount by which the face differs from the “typical” face represented by the eigenfaces. If weights from all images available are extracted. classify the weight pattern as either a known person or as unknown. If it is a face. As new faces are experienced.2.

the face library has to be filled with face images.2.3. it is compared with the weight vector of every face library member with a user defined “threshold”. Otherwise. These M eigenfaces define the M-dimensional face space. 4.2. After choosing the training set. In order to start the face recognition process.3.2. When the new faces are acknowledged. The characteristics of these phases are described below: 4.3. one seeks to recognize). Eigenfaces are calculated from the training set.2.2.2. the face image is classified as known”. now the system is ready to perform the recognition process. 30 .2. Weight vectors of the face library members are empty until a training set is chosen and eigenvectors are formed. Face images are stored in a face library (file system) in the system. If there exists at least one face library member that is similar to the acquired image within that threshold then.2 Main Algorithm phases The previous initialization processes can be summed up into three main phases. The corresponding weight vector of each face library member has now been updated. keeping only the M images that correspond to the highest eigenvalues. the gathering and the pre-processing of the face images that are going to be added to the face database are performed.1 Face database formation phase During this phase. The recognition process in initialized by choosing the input image (the image. the eigenfaces can be updated or recalculated.3. eigenfaces are formed and stored for later calculations. Every action such as training set or eigenface formation is performed on this face library. This process is called learning to recognize. The weight vector is constructed with the aid of the eigenfaces that were already stored during the training phase. After obtaining the weight vector.4. a miss has occurred and the face image with its corresponding weight vector for later use.3 Recognition and learning phase After choosing a training set and constructing the weight vectors of face library members.2 Training phase Images that are going to be in the training set are chosen from the entire face library. 4. Three main functional units are involved in these phases.

where u is an eigenvector of the matrix M and v is the corresponding eigenvalues. 31 .2.2. This relationship can be described by the equation M * u =v * u.3 The Use of Eigenfaces for Recognition 4.3. the unknown X is considered not a face.One way to do it would be to regard each weight vector as a point in space and calculate an average distance D between the weight vectors from Wx and the weight vector of the unknown image Wx (the Euclidean distance described in the appendix B would be a measure for that). Wx is compared with the weights of images. Its direction is not changed by that transformation.2.3.2 Eigenvectors and eigenvalues definitions An eigenvector of a matrix is a vector such that. 4. an eigenvalue of 2 states that the length of the eigenvector has been doubled. Otherwise (if X is actually a face). When observing an unknown image X. This means. Overview of the algorithm using eigenfaces The algorithm for the facial recognition system using eigenfaces is basically described as follows: First. Afterwards. if multiplied with the matrix. This integer value is the corresponding eigenvalue of the eigenvector.3. the weights are calculated for each image of the training set and stored in the set W. Afterwards. In this case. the weights are calculated for that particular image and stored in the vector Wx.1. the original images of the training set are transformed into a set of eigenfaces E.3. The corresponding eigenvalue is the proportion by which an eigenvector’s magnitude is changed.3. its weight vector Wx is stored for later classification.4. then the weight vector of the unknown image Wx lays too far apart form the weights of the faces. of which one knows for certain that they are faces (the weights of the training set W). An eigenvalue of 1 means that the length of the eigenvector stays the same. If this average distance exceeds some threshold value. the result is always an integer multiple of that vector.

Use sufficient light to illuminate the scene 32 .e.3.3.3. 4.4 Limitations Non-uniform backgrounds and lighting conditions affect the recognition process.Eigenvectors possess following properties:   They can be determined only for square matrices.2.3.3. All the eigenvectors are perpendicular. i. There are n eigenvectors (and corresponding eigenvalues) in an n * n matrix.3 Problems faced Calculating the average and weights of the faces takes so much processing and time when run for the first time to run the program.5 Recommendation   Using black or uniform background used in the image.3. 4.2. 4. at right angle with each other.2.

3.3.2.6 Flow chart Figure 13: PCA Training Phase 33 .4.

Figure 14: PCA Recognition Phase 34 .

As a matter of fact.3 Calculation of Eigenfaces with PCA 4.3. Some examples of eigenfaces are shown in Figure 15. The basic idea of the principal component analysis is to find the vectors that best account for the distribution of face images within the entire image space.4. these vectors are the eigenvectors of the covariance matrix corresponding to the original face images. which is known as “face space”. These vectors define the subspace of face images.536 dimensional space.3.536 or equivalently a point in 65. An image can also be considered as a vector of dimension N2. Figure 15: sample eigenfaces form the sample training set 35 .y) be a two-dimensional N * N array of 8-bit intensity values. and is a linear combination of the original face images. that describes an N * N image. so that a typical image of 256 * 256 becomes a vector if dimension 65. we refer to them as “eigenfaces”.1 Basic Definitions Let a face image I(x. Each vector is of length N2. and because they are face-like in appearance.3.

3. then subtracted from the original faces (Ti) and the result is stored in the variable  i.4. Let the training set of face images be T1.4 36 . refer to Appendices A and B. The average matrix  has to be calculated.3 Each face differs from the average by the vector Equation 4. Step 1: Gather and prepare the data. The average of the set is defined by Equation 4.2 Computation steps For being acquainted with the terms and notations that are going to be used during the eigenfaces algorithm evaluation. T2… TM Step 2: Subtract the mean.3.

which seeks a set of M orthonormal vectors. Uk. subject to Equation 4. This set of very large vectors is then subject to principal component analysis.7 Where the matrix A= [ 1. The Kth vector. is chosen such that Equation 4.6 The vectors uk and λk are the eigenvectors and eigenvalues.Step 3: Calculate the eigenvectors and eigenvalues of the covariance matrix. 2. 3… M] 37 . The covariance matrix C is calculated according to Equation 4.5 is a maximum. Step 4: Calculate the covariance matrix. Um which best describes the distribution of the data.

the “training” phase of the algorithm is accomplished. V1 of L is computed. 38 .8 The success of this algorithm is based on the evaluation of the eigenvalues and the eigenvectors of the real symmetric matrix L that is composed from the training set of images. and the M eigenvectors. After this step. Equation 4. where Lmn= Tm n . the M * M matrix L= A TA is constructed. These vectors determine linear combinations of the M training set face images to form the eigenfaces U1.Following the matrix analysis.

9 The weights form a feature vector. otherwise it is classified as “unknown” and can be added to face library with its feature vector for later use. The face classes Wi can be calculated by taking the average of the results of the eigenface representation over a small number of face images (as few as one) of each individual.3. then face image is classified as “known”.10 describes the share of each eigenface in representing the input face image. treating the eigenfaces as a foundation set for image images.4. Equation 4. If the comparison falls within the user defined threshold.4 Classifying Images Classifying an image with eigenfaces A new face image (Γ) is transformed into its eigenface e components (projected onto “face space”) Equation 4. thus making the system learning to recognize new face images. Classification is performed by comparing the feature vectors of the face library members with the feature vector of the input face image. 39 . This comparison is based on the Euclidean distance between the two members to be smaller than a user defined threshold k.

The output of our system will consist of the face ID with the closest match. the Eigenvector matrix will be of size 10304 x M. an Eigenvector matrix.4. Finally. It will always call a stored procedure in the database with image ID as its parameter to record the student’s attendance in the database. will consist of 122 x 92 =10304 pixels. as well as the image from the database with which the closest match is found. All this information will be displayed on the GUI of the PC.5 Input and Output The inputs of our system consist of bit streams representing the image to be analysed. these values were used successfully by many research groups. Furthermore. the set of projections will consist of M vectors. In a database with M faces. These figures were chosen because they provide a good balance between size and accuracy. an email sent to the student’s registered email address confirming his attendance for the lecture. Additionally. as well as the average face. each being 8-bit grayscale (28=256 levels ranging from 0 to 255). The image to be analysed. an average face. 40 . each having M values.3. and a set of projections.

The actual database will be the university’s database which contains real data of the students. where these images represent 40 individuals with 10 different images for each individual.7 Database Implementation A dummy database was implemented on MySQL workbench to support recording the attendance of the students. The remaining 6 images of the individual were used for the testing of the recognition phase. which was chosen as an intermediate number to properly evaluate the performance of the algorithm. 4.3. 41 . timetables and attendances records. representing various poses and facial expressions.4. courses. This database is composed of 400 images of 112 x 92 pixels each. A set of 4 images out of each individual’s 10 images was taken as the training set.3.6 Matlab Implementation The PCA algorithm was implemented on MATLAB and tested using AT&T ORL face database.

Integration of these sub modules was done basically by providing calls to and from various method. Each task was then handled with a different script. The constructed GUI also included the ways to access the information stored in the database at real time along with calling functions for updating the database. 9. It was noted that when a legal image was introduced to the system. the project cannot implement its tasks on an image with multiple faces. The final GUI was so constructed that it handled the final integration of both of the most important tasks. Building the project step-by-step or unitby-unit created concerns regarding the division of whole project into smaller modules and the way of integrating them into a one complete working project. At present.5 out of 10 times it responded with the right match. the student’s email address was retrieved from the database and an email was sent to his address confirming his attendance in the particular lecture. Division of the entire project into various sub-modules was done by recognising various tasks needed to be performed.5 Discussion of Result The initial phase of the project was faced by many challenges concerning the materials and the methodology to be used. It was noted that if a match was found. 42 . Separating the design from its implementation was yet another challenge. namely. Average Face Calculation and Recognition.

Here the blank area is to present the result upon clicking of any of the 2 push buttons provided.6 Presentation of Results 6. 43 .1 The basic user interface Figure 16: Basic GUI This is the basic Graphical User Interface of the project for the user.

6.2 Average face output Figure 17: Average face output The use of push button with text “Press to find the average face” gives the above output. This is the calculation of average face from all the images present in the test data. 44 .

When the user enters an image. 45 . its difference with the average calculated in the previous step is calculated and checked for an image with the minimum variation of this difference from the test data.6.3 Recognition output Figure 18: Recognition output Use of “Press for Recognition” push button prompts the user to enter the image.

7 Conclusions The working model of this project can perform all the intended tasks on the standard face image database provided by the AT&T ORL. Though it is not yet ready for the real time. The project is able to perform all the tasks described in the problem description with sustained accuracy. online processing of class attendances. 46 . it is one of the optimal basic systems to provide a path for such a project.

scheduling of classes and marking of attendance. 47 .e. online data with capturing live images.  Integrating it with faculty attendance system by providing 2 different databases.8 Future Prospects Some of the future prospects for this system are:   Processing real time. Integrating time table processing with this system to provide a single system for a college for its 2 major functions i.

It’s All About Authentication. Option 1 pp. In: IEEE Transactions on Circuits and Systems for Video Technology. F. Pankanti. Spooner R. An Introduction to Biometric Recognition. [4] Chalker. Feature Based Techniques. Face Recognition . (2004). S. US Patent. S. Riha. Vol 5.J. Handbook of face recognition. S. 48 . A. Time and Attedance System and Method therefor. Malassiotis . [2] Anhonen. UK. In: InterNational Committee for Information Technology Standards. “A piece of yourself”: Ethical issues in biometric identification. Option 1 pp. Ross. [8] Jain. USA. R.L. Wayman J. In: InfoSec Reading Room. [5] Cherry. In: International Conference on Pattern Recognition. W. S. Jain...Z. (2004). Face Recognition: A Literature Survey. Z. Faculty of Informatics. K (2003).org/articles/pages/6741/Face-Recognition.J(1982).K. Biometrics: A Grand Challenge.9 References [1] Alterman. 3-23 [6] Graham.Face Detection. New York. M. Phillips.html [14] Zhao. [12] Roethenbaugh. Problems and Considerations. Vol. Hong. Masaryk University Brno. pp: 139–150. Biometrics: An In Depth Examination. Global Approaches for. A. Face Description with Local Binary Patterns: Application to Face Recognition. Springer Science and Business Media. T. A (2003).jrank. SANS Institure. Version 1. In: InfoSec Reading Room. Jr. 14 No. G (2005). Cambridge.K. Rosenfeld. 1 pp 1-29 [9] Li. Chellappa. M (2006).J (1996). Pietikainen.G. pp 1-10. SANS Institure.. S (2004). http://encyclopedia. L. Biometric Authentication. US Patent. Special Issue on Image. A. [11] Maytas V. Strintzis. M. Czech Republic. 2-12 [7]Jain. Hadid. K. Conclusions and Future Developments. Prabhakar. D (2003). Version 1.4b. IEEE members [3] Bennett. [13] Tsalakanidou. In: ACM Computing Surveys.and Video-Based Biometrics.Security and Usability. Prabhakar. A. In: Ethics and Information Technology. Ross. Biometrics Explained. A. A (2003). P. Inc.4b. Automated time and attendance systems.

Sign up to vote on this title
UsefulNot useful