A Project Phase -1 Report

On
ONLINE EXAM AUTOMATION SYSTEM BASED ON DEEP LEARNING
TECHNIQUES THROUGH FACIAL RECOGNITION
Submitted by

19F61A05G1 - M.VENKATA MURALI

19F61A05G5 - M.VENKATESU
19F61A05F2 - C.TEJASHWI
19F61A05C3 - D.SAMJEESH

Under the esteemed guidance of

Mrs.G.INDIRAVATHI, M.Tech.,(Ph.D)
Assistant Professor, Department of CSE

Department of Computer Science and Engineering

SIDDHARTH INSTITUTE OF ENGINEERING & TECHNOLOGY
(AUTONOMOUS)
(Approved by AICTE & Affiliated to JNTU, Anantapuram)
(Accredited by NBA for Civil, EEE, ECE, MECH & CSE)
(Accredited by NAAC with ‘A+’ Grade)
Siddharth Nagar, Narayanavanam Road, PUTTUR-517583, A.P

2022

i
 SIDDHARTH INSTITUTE OF ENGINEERING & TECHNOLOGY
(AUTONOMOUS)
(Approved by AICTE & Affiliated to JNTU, Anantapuram)
(Accredited by NBA for Civil, EEE, ECE, MECH & CSE)
(Accredited by NAAC with ‘A+’ Grade)
Siddharth Nagar, Narayanavanam road, PUTTUR-517583, A.P

Department of Computer Science and Engineering

BONAFIDE CERTIFICATE

This is to Certify that the project entitled “Online Exam Automation System Based on
Deep Learning Techniques Through Facial Recognition” is being partially done as
Project phase 1 & submitted by

19F61A05G1 - M.VENKATA MURALI

19F61A05G5 - M.VENKATESU
19F61A05F2 - C.TEJASHWI
19F61A05C3 - D.SAMJEESH

Internal Guide Head of the Department

Mrs.G.Indiravathi, M.Tech.,(Ph.D) Dr.B.Geethavani, M.Tech., Ph.D.
Assistant Professor, HOD and Professor,
Department of CSE, Department of CSE,
SIETK. SIETK.

Project Phase-I viva-voice examination held on

INTERNAL EXAMINER EXTERNAL EXAMINER

ii
 ACKNOWLEDGEMENT

We take this opportunity to acknowledge all the people who helping me to do our project a
successful one.

We greatly convey my sincere thanks to our beloved chairman Dr. K. Ashok Raju, Ph.D.,
Vice chairperson Dr. K. Indiraveni, M. Tech., Ph.D. for providing us the facilities and time for
accomplishing the Project Phase-I.

We wish to express our profound gratitude to one of our all-time well-wisher and Principal
of our college Dr. K. Chandrasekhar Reddy, M.Tech., Ph.D., for constant encouragement for
completing the Project Phase-I successfully.

We wish to thank our Head of the Department, Dr. B. Geethavani, M.Tech., Ph.D., for
her guidance in accomplishing the Project Phase-I.

We wish to convey our heartfelt thanks to Mrs.G.Indiravathi, M.Tech., (Ph.D.),

Supervisor of this project and Dr.P.M.S.S.Chandu, M.E., Ph.D., Project Co-Ordinator, who
supported and helped to make this Project Phase-I as successful one.

We extend our thanks to all the staff members of the CSE Department, who gave us the
moral support for the completion of the Project Phase-I.

We also extend our thanks to our parents and our friends for the encouragement
inproceeding the Project Phase-I in right way and for the completion of the Project Phase-I in
successful way.

iii
 LIST OF CONTENTS

S.NO. CONTENT PAGE NO.

ACKNOWLEDGEMENT iii

LIST OF FIGURES vi
ABSTRACT vii

CHAPTER 1 INTRODUCTION 1-2

1.1 Introduction 1

1.2 Problem Statement 2

CHAPTER 2 LITERATURE SURVEY 3-44

2.1 Deep Learning 3

2.1.1 Deep Learning Uses 3

2.1.2 Examples of Deep Learning 4

2.1.3 Deep Learning Works 5

2.1.4 Difference Between Machine 5

Learning And Deep Learning

2.1.5 Choosing Between Machine 6
Learning and Deep Learning

2.1.6 Introduction to Python 7-21

2.2 Facial Recognition 22-29

2.2.1 Steps in Facial Identification 22

2.2.2 Working of Facial Recognition 22

2.2.3 Basic Operations 23

2.2.4 Facial Detection Techniques 24

2.2.5 Facial Recognition Techniques 25

2.2.6 Advantages 27

2.2.7 Applications 29

2.3 Reference Research Papers 30-50

iv
CHAPTER 3 PROPOSED METHODOLOGY 51-58

3.1 Existing System 51

3.1.1 Disadvantages 51

3.2 Proposed System 52

3.2.1 Advantages 52

3.2.2 Project Algorithm 55

3.3 System Requirements 58

REFERENCES 59-60

v
 LIST OF FIGURES

FIGURE NO. FIGURE NAME PAGE NO.

3.1 Block Diagram of Proposed Method 53

3.2 Architecture of The Project 54
3.3 Convolution Operation 55
3.4 Convolutional Neural Network Scan Images 56
3.5 Convolutional Neural Network 57

v
 ABSTRACT
.
One of the most significant challenges to online learning today is identity verification and
proctoring of online students. Specifically for online certification and accreditation, training
organization must ensure that the online students who completed the learning process and
received academic credits are the same people who registered for the courses. In addition, we
must ensure that these students complete all of the online training activities without cheating or
inappropriate behaviour. The COVID-19 pandemic has hastened (in some cases, hastened) the
migration and implementation of online education strategies, necessitating the need for secure
mechanisms to authenticate and proctor online students. Currently there are several
technologies with different degrees of automation. This document details specific solutions
based on authentication and automated monitoring systems for various biometric technologies
system (system workflows and AI algorithms) including features to solve key market problems
highly scalable, automatic, affordable, with few user hardware and software requirements,
highlyreliable, acceptable for students. Finally, technical performance tests of large-scale
systems, describes a user privacy perception survey and its results. This system offers identity
authentication service of online learns through a constant face recognition technology.

v
 CHAPTER -1

INTRODUCTION

1.1 INTRODUCTION

Nowadays, there are several technologies with different grades of automation. we

deeply describe a specific solution based on the authentication of different biometric
technologies and an automatic proctoring system (system workflow as well as AI algorithms),
which incorporates features to solve the main concerns in the market: highly scalable,
automatic, affordable, with few hardware and software requirements for the user, reliable and
passive for the student. Finally, the technological performance test of the large-scale system,
the usability privacy perception survey of the user and their results are discussed in this work.

E-learning has a serious deficiency, which is the lack of efficient mechanisms that assure
user authentication, in the system login as well as throughout the session. Especially for
online certification and accreditation, the training organizations need to verify that the online
learners who completed the learning process and received the academic credits are precisely
those who registered for the courses. Inadequate methods of identity verification affect the
reliability of credentials and certification earned online. Without certainty of the authenticity
of the online learner’s identity, the aspiration towards fully online education is stymied and
the evaluation of the knowledge and skills obtained by the online learner is unreliable. In
order to prevent compromising the credibility of online accreditation, validation must be
carried out in a constant or continuous manner. At the same time, validation should be non-
invasive and non- disruptive, and does not distract the learning process. Online proctoring,
generally refers to proctors (humans) monitoring an exam over the internet through a webcam.
It includes as wellthe processes, occurring at a distance, for authenticating the examinee as
the person who shouldbe taking the exam. Online proctoring was first introduced by Kryterion
in 2006, marketing it as a technological solution in 2008. Since then, several other
organizations have followed Kryterion’s lead creating more capable technology-based
alternatives, which are gaining attention, such as online proctoring.

1
1.2 PROBLEM STATEMENT

To overcome these issues, there is a specific solution based on an automatic proctoring system.
This solution consists of the features that help to resolve the primary concerns at hand. The
system allows the student to log in and authenticate him/her with the image verification during
the examination. It allows the student to take the exam, check the results, and report the
problems if any occurred.

2
 CHAPTER - 2

LITERATURE SURVEY

2.1 DEEP LEARNING

Deep learning is a type of machine learning and artificial intelligence (AI) that imitates the
way humans gain certain types of knowledge.
Deep learning is a machine learning technique that teaches computers to do what comes
naturally to humans: learn by example. Deep learning is a key technology behind driverless
cars, enabling them to recognize a stop sign, or to distinguish a pedestrian from a lamppost. It
is the key to voice control in consumer devices like phones, tablets, TVs, and hands-free
speakers. Deep learning is getting lots of attention lately and for good reason. It’s achieving
results that were not possible before.
In deep learning, a computer model learns to perform classification tasks directly from
images, text, or sound. Deep learning models can achieve state-of-the-art accuracy, sometimes
exceeding human-level performance. Models are trained by using a large set of labeled data
and neural network architectures that contain many layers.

2.1.1 Deep Learning Uses

In a word, accuracy, Deep learning achieves recognition accuracy at higher levels than ever
before. This helps consumer electronics meet user expectations, and it is crucial for safety-
critical applications like driverless cars. Recent advances in deep learning have improved to
the point where deep learning out performs humans in some tasks like classifying objects in
images. While deep learning was first theorized in the 1980s, there are two main reasons it has
only recently become useful:

3
  Deep learning requires large amounts of labeled data. For example, driverless car
development requires millions of images and thousands of hours of video.

 Deep learning requires substantial computing power. High-performance GPUs have a

parallel architecture that is efficient for deep learning. When combined with clusters or
cloud computing, this enables developmentteams to reduce training time for a deep
learning network from weeks to hours or less.

2.1.2 Examples of Deep Learning

Deep learning applications are used in industries from automated driving to medical devices.

Automated Driving: Automotive researchers are using deep learning to automatically detect
objects such as stop signs and traffic lights. In addition, deep learning is used to detect
pedestrians, which helps decrease accidents.

Aerospace and Defense: Deep learning is used to identify objects from satellites that locate
areas of interest, and identify safe or unsafe zones for troops.

Medical Research: Cancer researchers are using deep learning to automatically detect cancer
cells. Teams at UCLA built an advanced microscope that yields a high-dimensional data set
used to train a deep learning application to accurately identify cancer cells.

Industrial Automation: Deep learning is helping to improve worker safety around heavy
machinery by automatically detecting when people or objects are within an unsafe distance of
machines.

Electronics: Deep learning is being used in automated hearing and speech translation. For
example, home assistance devices that respond to your voice and know your preferences are
powered by deep learning applications.

4
2.1.3 Deep Learning Works

Most deep learning methods use neural network architectures, which is why deep learning
models are often referred to as deep neural networks.
The term “deep” usually refers to the number of hidden layers in the neural network.
Traditional neural networks only contain 2-3 hidden layers, while deep networks can have as
many as 150. Deep learning models are trained by using large sets of labeled data and neural
network architectures that learn features directly from the data without the need for manual
feature extraction.
One of the most popular types of deep neural networks is known as convolutional neural
networks (CNN or ConvNet). A CNN convolves learned features with input data, and uses 2D
convolutional layers, making this architecture well suited to processing 2D data, such as
images. CNNs eliminate the need for manual feature extraction, so you do not need to identify
features used to classify images. The CNN works by extracting features directly from images.
The relevant features are not pretrained; they are learned while the network trains on a
collection of images. This automated feature extraction makes deep learning models highly
accurate for computer vision tasks such as object classification.

CNNs learn to detect different features of an image using tens or hundreds of hidden layers.
Every hidden layer increases the complexity of the learned image features. For example, the
first hidden layer could learn how to detect edges, and the last learns how to detect more
complex shapes specifically catered to the shape of the object we are trying to recognize.

2.1.4 Difference Between Machine Learning and Deep Learning

Deep learning is a specialized form of machine learning. A machine learning workflow starts
with relevant features being manually extracted from images. The features are then used to
create a model that categorizes the objects in the image. With a deep learning workflow,
relevant features are automatically extracted from images. In addition, deep learning performs

5
“end-to-end learning” – where a network is given raw data and a task to perform, such as
classification, and it learns how to do this automatically.
Another key difference is deep learning algorithms scale with data, whereas shallow learning
converges. Shallow learning refers to machine learning methods that plateau at a certain level
of performance when you add more examples and training data to the network.
A key advantage of deep learning networks is that they often continue to improve as the size
of your data increases.

2.1.5 Choosing Between Machine Learning and Deep Learning

Machine learning offers a variety of techniques and models you can choose based on your
application, the size of data you're processing, and the type of problem you want to solve. A
successful deep learning application requires a very large amount of data (thousands of
images) to train the model, as well as GPUs, or graphics processing units, to rapidly process
your data. When choosing between machine learning and deep learning, consider whether
you have a high-performance GPU and lots of labeled data. If you don’t have either of those
things, it may make more sense to use machine learning instead of deep learning. Deep
learning is generally more complex, so you’ll need at least a few thousand images to get
reliable results. Having a high-performance GPU means the model will take less time to
analyze all those images. Most deep learning methods use neural network architectures,
which is why deep learning models are often referred to as deep neural networks. Deep
learning is a specialized form of machine learning. A machine learning workflow starts with
relevant features being manually extracted from images. The features are then used to create a
model that categorizes the objects in the image. Another key difference is deep learning
algorithms scale with data, whereas shallow learning converges. Shallow learning refers to
machine learning methods that plateau at a certain level of performance when you add more
examples and training data to the network.
A key advantage of deep learning networks is that they often continue to improve as the size
of your data increases.

6
2.2 FACIAL RECOGNITION

Face recognition is a method of identifying or verifying the identity of an individual using their
face. Face recognition systems can be used to identify people in photos, video, or in real-time.
Law enforcement may also use mobile devices to identify people during police stops.

But face recognition data can be prone to error, which can implicate people for crimes they
haven’t committed. Facial recognition software is particularly bad at recognizing African
Americans and other ethnic minorities, women, and young people, often misidentifying or
failing to identify them, disparately impacting certain groups.
It is faster and more convenient compared to other biometric technologies like fingerprints or
retina scans. There are also fewer touchpoints in facial recognition compared to entering
passwords or PINs. It supports multifactor authentication for additional security verification.

2.2.1 Steps in Facial Identification

1. Image is captured.
2. Eye locations are determined.
3. Image is converted to grayscale and cropped.
4. Image is converted to a template used by the search engine for facial comparison results.
5.Image is searched and matched using a sophisticated algorithm to compare the template to
other on file.
6.Duplicate licenses are investigated for fraud.

2.2.2 Working

Face recognition systems use computer algorithms to pick out specific, distinctive details about
a person’s face. These details, such as distance between the eyes or shape of the chin, are then
converted into a mathematical representation and compared to data on other faces collected in a
face recognition database. The data about a particular face is often called a face template and is
distinct from a photograph because it’s designed to only include certain details that can be used
to distinguish one face from another.

Some face recognition systems, instead of positively identifying an unknown person, are
designed to calculate a probability match score between the unknown person and specific face

2
templates stored in the database. These systems will offer up several potential matches, ranked
in order of likelihood of correct identification, instead of just returning a single result.

Face recognition systems vary in their ability to identify people under challenging conditions
such as poor lighting, low quality image resolution, and suboptimal angle of view (such as in a
photograph taken from above looking down on an unknown person).

When it comes to errors, there are two key concepts to understand:

A “false negative” is when the face recognition system fails to match a person’s face to an
image that is, in fact, contained in a database. In other words, the system will erroneously
return zero results in response to a query.

A “false positive” is when the face recognition system does match a person’s face to an image
in a database, but that match is actually incorrect. This is when a police officer submits an
imageof “Joe,” but the system erroneously tells the officer that the photo is of “Jack.”

When researching a face recognition system, it is important to look closely at the “false
positive” rate and the “false negative” rate, since there is almost always a trade-off. For
example, if you are using face recognition to unlock your phone, it is better if the system fails
to identify you a few times (false negative) than it is for the system to misidentify other people
as you and lets those people unlock your phone (false positive). If the result of a
misidentification is that an innocent person goes to jail (like a misidentification in a mugshot
database), then the system should be designed to have as few false positives as possible.

2.2.3 Basic Operations

These are the basic operations involved in Face Recognition:
 Face Detection: Its the first and most essential step in face recognition.
 Features Segmentation: Its a simultaneous process, sometimes face detection suit
comparatively difficult and requires 3D Head Pose, facial expression, face relighting,
Gender, age and lots of other features.
 Face Recognition: It is less reliable and the accuracy rate is still not up to the mark.
Extensive work on Face Recognition have been done, but still it is not up to the mark for
implementation point of view.

2
  More techniques are being invented each year to get better and realistic results.

2.2.4 Face Detection Techniques

Viola-Jones Algorithm
 Its an efficient algorithm for face detection.

 Developers of this algo showed faces being detected in real time on a webcam feed.

 It was the most stunning demonstration of computer vision and its potential at the time.

 Soon, it was implemented in OpenCV & face detection became synonymous with Viola
and Jones algorithm.

Basic Idea:
1. It takes a bunch of faces as data.

2. We hard-code the features of a face.

3. Train a SVM(Classifier) on the feature set of the faces.

4. Use this Classifier to detect faces!

Histogram of Oriented Gradients

Basic Idea:
1. For an Image I, analyze each pixel say P(i) of the image I for the relative dark pixels
directly surrounding it.

2. Then add an arrow pointing in the direction of the flow of darkness relative to P(i).

3. This process of assigning an oriented gradient to a pixel P(i) by analyzing it's surrounding
pixels is performed for every pixel in the image.

4. Assuming HOG(I) as a function that takes an input as an Image I, what it does is replaces
every pixel with an arrow. Arrows = Gradients. Gradients show the flow from light to dark
across an entire image.

2
 5. Complex features like eyes may give too many gradients, so we need to aggregate the
whole HOG(I) in order to make a 'global representation'. We break up the image into
squares of 16 x 16 and assign an aggregate gradient G′ to each square, where the function
could be max(), min(),etc.

R-C NN

Basic Idea:
 R-CNN creates bounding boxes, or regions, using selective search.

 Selective search looks at the image through windows of different sizes, and for each size
it tries to group together adjacent pixels by texture, color, or intensity to identify objects.

1. Generate a set of regions for bounding boxes.

2. Run the images in the bounding boxes through a pre-trained neural network and finally
an SVM to see what object the image in the box is.

3. Run the box through a linear regression model to output tighter coordinates for the box
once the object has been classified.

2.2.5 Face Recognition Techniques

Face recognition is a challenging yet interesting problem that it has attracted researchers who
have different backgrounds like psychology, pattern recognition, neural networks, computer
vision, and computer graphics.

The following methods are used to face recognition:

 Holistic Matching

 Feature Based (structural)

 Model Based

 Hybrid Methods

2
Holistic Matching

In this approach, complete face region is taken into account as input data into face catching
system. One of the best example of holistic methods are Eigenfaces, PCA, Linear Discriminant
Analysis and independent component analysis etc.

Let's see the steps of Eigenfaces Method:

This approach covers face recognition as a two-dimensional recognition problem.
1. Insert a set of images into a database, these images are named as the training set because
they will be used when we compare images and create the eigenfaces.
2. Eigenfaces are made by extracting characteristic features from the faces. The input
images are normalized to line up the eyes and mouths. Then they are resized so that they
have the same size. Eigenfaces can now be extracted from the image data by using a
mathematical tool called PCA.
3. Now each image will be represented as a vector of weights. System is now ready to
accept queries. The weight of the incoming unknown image is found and then compared
to the weights of already present images in the system.
4. If the input image's weight is over a given threshold it is considered to be unidentified.
The identification of the input image is done by finding the image in the database whose
weights are the closest to the weights of the input image.
5. The image in the database with the closest weight will be returned as a hit to the user.

Feature-Based
Here local features such as eyes, nose, mouth are first of all extracted and their locations,
geometry and appearance are fed into a structural classifier. A challenge for feature extraction
methods is feature "restoration", this is when the system tries to retrieve features that are
invisible due to large variations, e.g. head Pose while matching a frontal image with a profile
image.

Different extraction methods:

 Generic methods based on edges, lines, and curves

 Feature-template-based methods

 Structural matching methods

2
Model Based
The model-based approach tries to model a face. The new sample is introduced to the model
and the parameters of the model are used to recognize the image. Model based method can
beclassified as 2D or 3D.

Hybrid Methods
This uses a combination of both holistic and feature extraction methods. Generally 3D Images
are used in these methods. The image of a face is caught in 3D, to note the curves of the eye
sockets, or the shapes of the chin or forehead. Even a face in profile would serve because the
system uses depth, and an axis of measurement, which gives it enough information to construct
a full face. The 3D system includes Detection, Position, Measurement, Representation and
Matching.

1. Detection - Capturing a face by scanning a photograph or photographing a person's face

in real time.
2. Position - Determining the location, size and angle of the head. Measurement - Assigning
measurements to each curve of the face to make a template.
3. Representation - Converting the template into a numerical representation of the face.
4. Matching - Comparing the received data with faces in the database. The 3D image which
is to be compared with an existing 3D image, needs to have no alterations.

2.2.6 Advantages

Improving security systems and identifying criminals are often cited when arguing in favour
of facial recognition, as well as getting rid of unnecessary labour or human interaction.
However, there are also plenty of other examples.

1. Finding missing people and identifying perpetrators

Facial recognition technology is used by law enforcement agencies to find missing people or
identify criminals by using camera feeds to compare faces with those on watch lists.

The technology has also been used to locate missing children. Sometimes it is combined with
advanced ageing software to predict what children might look like based on photos taken when

2
they disappeared. Law enforcement uses facial recognition with live alerts which can help them
track potential matches after being pinged by the system.

2. Protecting businesses against theft

Facial recognition software has been used as a preemptive measure against shoplifting.
Business owners use the software and security cameras to identify suspects against a database
of known thieves, and it has been argued that the mere presence of facial recognition cameras
has an effect as a deterrent for would-be offenders.

If something is stolen from the business, the software can also be used to catalogue the thieves
for future reference.

3. Better security measures in banks and airports

Facial recognition has also come to be used as a preventative security measure in sensitive
locations such as banks and airports. Similar to identifying criminals that come into shops, the
software has helped identify criminals and passengers that pose a potential risk to airlines and
passengers.

The US Customs and Border Protection (CBP) has dedicated itself to using facial recognition
on 97% of international passengers by 2023.

Border checks have also been sped up at some airports through the use of facial recognition
cameras at passport-check gates.

Institutions like banks use the software in the same way to prevent fraud, identifying those
previously charged with crimes and alerting the bank to watch specific individuals more
carefully.

4. Shopping is far more efficient

While identifying and finding missing persons and criminals are arguably the most important
benefits of facial recognition, they extend beyond security to convenience.

Instead of making cash or credit purchases at stores, facial recognition technology can
recognize your face and charge the goods to your account.

Use of this increased during the pandemic to serve both convenience and security purposes, as
well as help manage the smaller ratio of staff to customers, but retailers also see the tech being
used in the future to recognize and advertise to loyalty club members and clock employees in

2
and out.

5. Drastically reduces human touchpoints

Facial recognition requires fewer human resources than other types of security measures, such
as fingerprinting. It also doesn’t require physical contact or direct human interaction. Instead, it
uses artificial intelligence (AI) to make it an automatic and seamless process.

It also limits touchpoints when unlocking doors and smartphones, getting cash from the ATM
or performing any other task that generally requires a PIN, password or key.

6. Better tools for organizing photos

Facial recognition can also be used to tag photos in your cloud storage through iCloud or
Google Photos. Users who wish can enable facial recognition in their respective photo app’s
settings, resulting in named folders for regular photo subjects. Facebook also used facial
recognition to suggest people to tag within a photo.

7. Better medical treatment

One surprising use of facial recognition technology is the detection of genetic disorders.

By examining subtle facial traits, facial recognition software can, in some cases, determine how
specific genetic mutations caused a particular syndrome. The technology may be faster and less
expensive than traditional genetic testing.

2.2.7 Applications

 Security companies are using facial recognition to secure their premises.

 Immigration check points use facial recognition to enforce smarter border control.
 Fleet management companies can use facial recognition to secure their vehicles.
 Ride sharing companies can use facial recognition to ensure the right passengers are
picked up by the right drivers.
 IOT benefits from facial recognition by allowing enhanced security measures and
automatic access control at home.
 Law enforcement can use facial recognition technologies as one part of AI-driven
surveillance systems.

2
2.3 REFERENCE RESEARCH PAPERS

Ref 1:
Title: A Design of Continuous User Verification For Online Exam Proctoring On M-
Learning

Authors: H. S. G. Asep and Y. Bandung

The answer script evaluation is an important part of assessing students' performance.

Typically, an answer script evaluation is done manually that sometimes can be biased. The
evaluation depends on various factors like mood swing of the evaluator, the interrelation
between the student and evaluator. Additionally, evaluation is a very tedious and time-
consuming task. In this paper, a natural language processing-based method is shown for
automatic answer script evaluation. Our experiment consists of text extraction from answer
script, measuring various similarities between summarized extracted text and stored correct
answers, and then assign a weight value to each calculated parameters to score the answer
script. For summary generation from the extracted text, we have used keyword-based
summarization techniques. Here four similarity measures (Cosine, Jaccard, Bigram, and
Synonym) are used as parameters for generating the final mark. Automatic evaluation of
answer scripts has been found very useful from our experiments, and often the assigned marks
is the same as manually scored marks. The use of m-learning or other remote education
continue to increase due to its ability to reach people who don't have access to campus. Exams
are important components of educational programs as well as on an online learning program.
In an exam, a proctoring method to detect and reduce the cheating possibility is very
important to ensure that the students have learned the material given. Various methods had
been proposed to provide an efficient, comfortable online exam proctoring. Start with
implementing an exam design with hard constraints in a no proctoring exam, a remote
proctoring using a webcam, a machine based proctoring and finally research on automated
online proctoring. A visual verification for the whole exam session is needed in an online
exam, therefore a face verification is needed. A remaining problem in face recognition area is
the system robustness for pose and lighting variations. In this paper, we proposed a method to
enhance the robustness for pose and lighting variations by doing an incremental training
process using the training data set obtained from m- learning online lecture sessions. As a
result, the design of the proposed method is presentedin this paper.

3
Ref 2:

Title: Student Facial Authentication Model Based on OpenCV’s Object Detection

Method And QR Code For Zambian Higher Institutions Of Learning

Authors: L. K. Musambo and J. Phiri

Facial biometrics captures human facial physiological data, converts it into a data item
variable so that this stored variable may be used to provide information security services, such
as authentication, integrity management or identification that grants privileged access or
control to the owner of that data variable. In this paper, we propose a model for student
authentication based on facial biometrics. We recommend a secure model that can be used in
the authentication and management of student information in the registration and access of
resources, such as bursaries, student accommodation and library facilities at the University of
Zambia. Since the model is based on biometrics, a baseline study was carried out to collect
data from the general public, government entities, commercial banks, students, ICT regulators
and schools on their understanding, use and acceptance of biometrics as an authentication
tool. Factor analysis has been used to analyze the findings. The study establishes that
performance expectancy, effort expectancy, social influence and user privacy are key
determinants for application of a biometric multimode authentication. The study further
demonstrates that education and work experience are regulating factors on acceptance and
expectancy of a biometric authentication system. Based on these results, we then developed a
biometric model that can be used to perform authentication for students in higher learning
institutions in Zambia. The results of our proposed model show 66% acceptance rate using
Open CV. Applying a secure biometric infrastructure is a key in ensuring that organizational
and or private data is well managed and accessed only by the intended party. It is important
that a possibility to authenticate only those individuals that are registered as students of a high
institution exists. This study focuses on the University of Zambia (UNZA), which is Zambia’s
biggest institution of higher learning. The findings can be generalized to cover the rest of
Zambia’s higher institutions of learning. The current authentication processes for UNZA are
paper- based systems installed by the management of the university. Though these
authentication processes which are prone to data redundancy are implemented; issues of over
payment to ghost students on bursaries always arise, issues of illegal residents at the
university arise from time to time, issues of non-availability of student records or non-
available student recordswhich were earlier created and filed with the office of Dean of
Students arise.

3
Ref 3:

Title: A Virtual Laboratory System With Biometric Authentication And Remote Proctoring
Based on Facial Recognition

Authors: Z. Zhang, M. Zhang, Y. Chang, S. Esche, and C. Chassapis

Virtual laboratories are used in online education, corporate training and professional
skill development. There are several aspects that determine the value and effectiveness of
virtual laboratories, namely the cost of development which includes the cost of creating the
virtual environment and designing and implementing the laboratory exercises, the benefits
broughtto the trainees compared with those provided by traditional physical laboratories, and
the operation which includes the communication between trainers and trainees, the
authentication and remote proctoring of the trainees, etc. In this paper, a virtual laboratory
system with biometric authentication and remote proctoring by employing facial recognition
techniques is introduced. The general logic and basic algorithms used to enable biometric
authentication and remote proctoring are described. When using this virtual laboratory
system, the students log in by scanning their faces with a camera. While performing a
laboratory exercise, the students sit in front of the camera and the virtual laboratory system
monitors their facial expressions and

head motions in order to identify suspicious behaviors. Upon detection of such suspicious
behaviors, the system records a video for further analysis by the laboratory administrator. An
evaluation of the feasibility of this approach is presented. As one of the most important
implementations of virtual reality (VR), virtual laboratories (VLs) are becoming more and
more popular at various levels of education and in various fields of training. There are several
factors that speed up the development of VL systems. The first factor is the wide-spread
adoption of the Internet which provides the possibility of remote access to VLs (including the
remote control of physical devices and the communication between distance users). The
second factor is the development of advanced computer graphics which enables the
visualization of the real word in virtual environments with CAD software or real-time
rendering tools. The third factor is the appearance of artificial intelligence which enables
developers to implement virtual activities. Other research in materials, electronic components
and devices, etc. also boosts the development of VL systems. VL systems are inherently safer
and less failure prone than their physical equivalents. This renders them an alternative
solution for dangerous and costly training programs. VL systems can be accessed remotely by

3
the learners.

3
Ref 4:

Title: Generalized End-to-End Loss for Speaker Verification

Authors: L. Wan, Q. Wang, A. Papir, and I. L. Moreno

we propose a new loss function called generalized end-to-end (GE2E) loss, which
makes the training of speaker verification models more efficient than our previous tuple-based
end-to-end (TE2E) loss function. Unlike TE2E, the GE2E loss function updates the network
in a way that emphasizes examples that are difficult to verify at each step of the training
process. Additionally, the GE2E loss does not require an initial stage of example selection.
With these properties, the model with new loss function learns a better model, by decreasing
EER by more than 10%, in shorter period of time, by reducing the training time by > 60%.
We also introduce the Multi Reader technique, which allow us do domain adaptation-training
more accurate model that supports multiple keywords (i.e., "OK Google" and "Hey Google")
as well as multiple dialects. Speaker verification (SV) is the process of verifying whether an
utterance belongs to a specific speaker, based on that speaker’s known utterances , with
applications such as Voice Match. Depending on the restrictions of the utterances used for
enrollment and verification, speaker verification models usually fall into one of two
categories: text-dependent speaker verification (TD-SV) and text-independent speaker
verification (TI-SV). In TD-SV, the transcript of both enrollment and verification utterances is
phonetially constrained, while in TI- SV, there are no lexicon constraints on the transcript of
the enrollment or verification utterances, exposing a larger variability of phonemes and
utterance durations.In this work, we focus on TI- SV and a particular subtask of TD-SV
known as global password TD-SV, where the verification is based on a detected keyword, e.g.
“OK Google”.In previous studies, I-vectorbased systems have been the dominating approach
for both TD-SV and TI-SV applications . In recent years, more efforts have been focusing on
using neural networks for speaker verification, while the most successful systems use end-to-
end training. In such systems, the neural network output vectors are usually referred to as
embedding vectors(also known as d- vectors). Similarly to as in the case of I-vectors,such
embedding can then be used to represent utterances in a fix dimensional space, in which other,
typically simpler, methods can be used todisambiguate among speakers.

3
Ref 5:

Title: FaceBoxes: A CPU Real Time Face Detector with High Accuracy

Authors: S. Zhang, X. Zhu, Z. Lei, H. Shi, X. Wang, and S. Z. Li,

Although tremendous strides have been made in face detection, one of the remaining
open challenges is to achieve real-time speed on the CPU as well as maintain high
performance, since effective models for face detection tend to be computationally prohibitive.
To address this challenge, we propose a novel face detector, named FaceBoxes, with superior
performance on both speed and accuracy. Specifically, our method has a lightweight yet
powerful network structure that consists of the Rapidly Digested Convolutional Layers
(RDCL) and the Multiple Scale Convolutional Layers (MSCL). The RDCL is designed to
enable FaceBoxes to achieve real-time speed on the CPU. The MSCL aims at enriching the
receptive fields and discretizing anchors over different layers to handle faces of various
scales. Besides, we propose a new anchor densification strategy to make different types of
anchors have the same density on the image, which significantly improves the recall rate of
small faces. As a consequence, the proposed detector runs at 20 FPS on a single CPU core and
125 FPS using a GPU for VGA- resolution images. Moreover, the speed of FaceBoxes is
invariant to the number of faces. We comprehensively evaluate this method and present state-
of-the-art detection performance on several face detection benchmark datasets, including the
AFW, PASCAL face, and FDDB. Face detection is one of the fundamental problems in
computer vision and pattern recognition. It plays an important role in many subsequent face-
related applications, such as face alignment, face recognition and face tracking. With the great
progress over the past few decades, especially the breakthrough of convolutional neural
network, face detection has been successfully applied in our daily life under various scenarios.
However, there are still some tough challenges in uncontrolled face detection problem,
especially for the CPU Corresponding author devices. The challenges mainly come from two
requirements for face detectors: The large visual variation of faces in the cluttered
backgrounds requires face detectors to accurately address a complicated face and non-face
classification problem; The large search space of possible face positions and face sizes further
imposes a time efficiency requirement. These two requirements are conflicting, since high-
accuracy face detectors tend to be computationally expensive. Therefore, it is one of the
remaining open issues for practical face detectors on the CPU devices to achieve real-time
speed as well as maintain high performance.
3
Ref 6:

Title: A Virtual Proctor with Biometric Authentication for Facilitating Distance Education

Authors: Z. Zhang, E.-S. Aziz, S. Esche, and C. Chassapis

The lack of efficient and reliable proctoring for tests, examinations and laboratory
exercises is slowing down the adoption of distance education. At present, the most popular
solution is to arrange for proctors to supervise the students through a surveillance camera
system. This method exhibits two shortcomings. The cost for setting up the surveillance
system is high and the proctoring process is laborious and tedious. In order to overcome these
shortcomings, some proctoring software that identifies and monitors student behavior during
educational activities has been developed. However, these software solutions exhibit certain
limitations: They impose more severe restrictions on the students than a human proctor would.
The students have to sit upright and remain directly in front of their webcams at all times. The
reliability of these software systems highly depends on the initial conditions under which the
educational activity is started. For example, changes in the lighting conditions can cause
erroneous results. In order to improve the usability and to overcome the shortcomingsof the
existing remote proctoring methods, a virtual proctor (VP) with biometric authenticationand
facial tracking functionality is proposed here. In this paper, a two-stage approach (facial
detection and facial recognition) for designing the VP is introduced. Then, an innovative
method to crop out the face region from images based on facial detection is presented. After
that, in order to render the usage of the VP more comfortable to the students, in addition to an
eigen face-based facial recognition algorithm, a modified facial recognition method based on
a real-time stereo matching algorithm is employed to track the students’ movements. Then,
the VP identifies suspicious student behaviors that may represent cheating attempts. By
employing a combination of eigen face- based facial recognition and real-time stereo
matching,the students can move forward, backward, left, right and can rotate their head in a
larger range.In addition, the modified algorithm used here is reliable to changes of lighting,
thus decreasingthe possibility of false identification of suspicious behaviors.

3
Ref 7:
Title: Comparative Analysis Between Different Facial Authentication Tools for Assessing
Their Integration in M-Health Mobile Applications

Authors: F. Guillen-Gamez, I. García-Magariño, and G. Palacios

The security and privacy in the access to mobile health applications are a challenge
that any company or health organization has to take into consideration to develop reliable and
robust applications. In this line, the facial authentication becomes a key piece to improve the
access to users and that they do not lose their privacy of their data due to cyber-attacks or
fraudulent users. The purpose of the current framework is to compare the two relevant facial-
based mechanisms to select the most appropriate one for the authentication security in our
under-development Framework for developing M-health apps (FAMAP).Nowadays, the
advance of the technological devices has allowed us to improve and to automate the biometric
recognition systems. This type of system is mainly used for security purpose in mobile
applications, database security, Internet security, medical records, safe trade terminals,
automated teller machines (ATMs), among other uses. These biometric methods provide a
way of maintaining privacy of users regarding the access to their data. Jain et al.claim that
biometrics seek to find reliable, safe and fast security systems through the identification of the
personal characteristic of users, preventing the possibility of a fake user could usurp their
passwords or access cards, achieving that the identity impersonation of a user is almost
impossible. The biometric systems are based on the comparison of users from the analysis of
their physiological or behavioral characteristics. Depending on the type of security that an
organism or company wants to have, different biometric methods are used. Prabhakaret
classify them in fingerprint, iris, the geometry of the hand, the voice or the facial
authentication. From these parameters, a unique pattern is obtained for each person which it
will be used in later comparisons. The operation of a biometric system, indistinctly from the
method used, is the following: capture, where the user’s biometric data are collected through
an electronic device; process, in whichthe user’s personal characteristics are extracted and a
model is created with them; inscription, where the pattern that has been created, itis stored in a
database so the biometric system could authenticate the user; and authentication, where the
system captures a biometric sample of the user and it compares it with the data recorded
previously. Nowadays, the recognition through fingerprints is the most used biometric form,
Although thanks to the technological advances and the desire to improve the security in
networks and mobile.

3
Ref 8:
Title: Continuous Biometric User Authentication in Online Examinations

Authors: E. Flior and K. Kowalski

Online examinations pose a unique problem for distance-based education, in that it can
be very difficult to provide true user authentication. Due to the inherent anonymity of being
online, compared to taking an examination in a classroom environment, students may attempt
to artificially boost their scores in online examinations by having another individual take the
exam for them, which a typical user/password authentication scheme cannot detect. This
paper discusses and presents a method for providing continuous biometric user authentication
in online examinations via keystroke dynamics. The last decade has witnessed a growing
interest in the area of continuous authentication, with several publications being produced by
the research community and diverse products being released by the industry. Continuous
authentication consists of verifying repeatedly the identity of a user throughout computing or
online session, with the purpose of preventing identity fraud (Traore and Ahmed
2012).Identity fraud can broadly be categorized in three classes: identity theft, identity
sharing, and identity denial. Identity theft occurs when the identity of an unsuspected user is
hijacked by a fraudster and used to conduct malicious activity pretending to be the legitimate
user. Vehicles for conducting such attacks include phishing, social engineering, and password
cracking .Denial of identity occurs when an authorized individual conducts illegal actions and
repudiates such actions when caught. Typically, this would consist of a malicious insider who
repudiates malicious actions associated with their identity. Identity sharing, also referred to as
identity gift, occurs when an authorized individual willingly share their credentials with other
users, in violation of established policies and regulations. Illegal password sharing can
happen, for instance, in the nancial industry to circumvent two-man rules, or for paid
subscription services such as Netflix. In the situation of giving an online examination, there
are security factors to consider beyond simple password authentication for access to the
examination. It is not unreasonable to assume that a student may willingly give their password
to someone else, with the intent that the other person will take the examination for the student.
With this in mind,a system must be developed in order to determine that the person taking the
examination is, in fact, the student registered to take the examination. While it may be
infeasible to guarantee with 100% confidence that the person taking the examination is the
student.

3
Ref 9:
Title: Automated Online Exam Proctoring

Authors: Y. Atoum, L. Chen, A. X. Liu, S. D. H. Hsu, and X. Liu

Massive open online courses and other forms of remote education continue to increase in
popularity and reach. The ability to efficiently proctor remote online examinations is an
important limiting factor to the scalability of this next stage in education. Presently, human
proctoring is the most common approach of evaluation, by either requiring the test taker to
visit an examination center, or by monitoring them visually and acoustically during exams via
a webcam. However, such methods are labor intensive and costly. In this paper, we present a
multimedia analytics system that performs automatic online exam proctoring. The system
hardware includes one webcam, one wear cam, and a microphone for the purpose of
monitoring the visual and acoustic environment of the testing location. The system includes
six basic components that continuously estimate the key behavior cues: user verification, text
detection, voice detection, active window detection, gaze estimation, and phone detection. By
combining the continuous estimation components, and applying a temporal sliding window,
we design higher level features to classify whether the test taker is cheating at any moment
during the exam. To evaluate our proposed system, we collect multimedia (audio and visual)
data from subjects performing various types of cheating while taking online exams. Extensive
experimental results demonstrate the accuracy, robustness, and efficiency of our online exam
proctoring system. Massive open online courses (MOOCs) offer the potential to significantly
expand the reach of today’s educational institutions, both by providing a wider range of
educational resources to enrolled students and by making educational resources available to
people who cannot access a campus due to location or schedule constraints. Insteadof taking
courses in a typical classroom on campus, now students can take courses anywhere inthe
world using a computer, where educators deliver knowledge via various types of
multimediacontent. According to a recent survey, more than 7.1 million students are taking, at
least, oneonline course in 2013 in America. It also states that 70% of higher education
institutions believethat online education is a critical component of their long-term strategy.
Exams are a critical component of any educational program, and online educational programs
are no exception. In any exam, there is a possibility of cheating, and therefore, its detection
and prevention are important.

3
Ref 10:

Title: Student’s Perception Of The Importance Of Facial Authentication Software in

Moodle Tools
Authors: F. Guillen-Gamez, J. Bravo, and I. Garcia-Magarino

Nowadays, the use of facial authentication software in distance education could help to
guarantee the proper login of students in the Learning Management Systems. Also, facial
software utilization in online activities will probably identify students and prevent cheating.
Thus, the current research analyzes the suitability of introducing a facial authentication
software application in different LMS tools. In order to reach this goal, the perceptions of 67
master students were analyzed. Students were divided into two groups. The facial software
was applied to the first group at the same time when this group did their coursework using
Tests, Glossaries, and Lessons Moodle tools. The second group did their course work using
Forums, Task mailboxes, and Databases Moodle tools without the surveillance of the facial
software. Then, both groups were surveyed by a questionnaire related to the students'
perception about using facial authentication software in Moodle tools. The data analysis
revealed that the Moodle tools in which the implantation of facial authentication software
would be important are Tests, Glossaries and Lessons. Nowadays a common feature of higher
education is e- Learning. Moreover, new technologies of education and communication are
being used to improve or even replace some aspects of face-to-face learning. The Learning
Management Systems(LMSs) are becoming an integral part of the open universities.
Nevertheless, current LMSs only offer a user identification mechanism, which is a username
and password at the beginning of the virtual session. The user´s identify should play a leading
role in these LMSs, since these only identify and verify the user at the time of session
initiation, like in the Moodleplatform, but not all the time that students are within the platform
carrying out its tasks. In fact,these systems do not currently allow a real and ongoing
assessment for students, since there isa lack of methods to prevent students from cheating
while they do their learning activities. In order to provide a solution for this need of
authentication for online students, biometric can be used as an authentication tool, and more
specifically a facial authentication application. For example, one can use commercial products
like ProctorU, Secure Exam Remote Proctor,or Kryterion Web assessor. Using an appropriate
procedure, it is possible to verify the identity ofthe user not only at the beginning of the
session, but more important, during the entire online interaction. A facial recognition system
includes three steps. The first step is facial detection through a photograph of the user.

4
Ref 11:
Title: A Dynamic Profile Questions Approach to Mitigate Impersonation in Online Examinations
Authors: A. Ullah, H. Xiao, and T. Barker

Online examinations are an integral component of many online learning

environments, which face many security challenges. Collusion is seen as a major security
threat to such examinations, when a student invites a third party to impersonate or abet in a
test. This work aims to strengthen the authentication of students via the use of dynamic profile
questions. The study reported in this paper involved 31 online participants from five countries
over a five- week period. The results of usability and security analysis are reported. The
dynamic profile questions were more usable than both the text-based and image-based
questions. An impersonation abuse scenario was simulated using email and mobile phone.
The impersonation attack via email was not successful, however, students were able to share
answers to dynamic profile questions with a third party impersonator in real time, which
resulted in 93% correct answers. The sharing of information via phone took place in real time
during an online test and the response time of an impersonator was significantly different than
a student. The study also revealed that a response time factor may be implemented to identify
and report impersonation attacks. In typical online learning environments, students are
assessed from remote locations, which raise the security concerns of stakeholders about the
integrity of online examinations. Cheating is one of the major threats due to vulnerable
authentication approaches and the degree of difficulty to verify the identity of remote users.
Face-to-face invigilation can be expensive and logistically challenging in dispersed
geographical locations. However, many educational institutions prefer supervised
examinations to the use of high stake online examinations largely because of the difficulty in
the authentication of a remote user with no face-to-face interaction. Student cheats in online
examinations using a number of methods. The work presented in this paper investigates
collusion attacks i.e. impersonation, when a student invites a third party to take the test on
his/her behalf. The student shares their access credentials via two methods: email, and instant
messaging using mobile phone. Collusion is a challenging threat which is difficult to detect
and report after the completion of an online examination. This paper presents the findings of
an empirical study conducted in a real online course with remote international participants.
The work focuses on research that aims to strengthen the authentication of examinees via the
use of a challenge questions approach. The traditional text-based challenge questions
approach requires students to register their answers before authentication.

4
Ref 12:
Title: FaceNet: A Unified Embedding for Face Recognition And Clustering
Authors: F. Schroff, D. Kalenichenko and J. Philibin

Despite significant recent advances in the field of face recognition , implementing

face verification and recognition efficiently at scale presents serious challenges to current
approaches. In this paper we present a system, called Face Net, that directly learns amapping
from face images to a compact Euclidean space where distances directly correspond to a
measure of face similarity. Once this space has been produced, tasks such as face recognition,
verification and clustering can be easily implemented using standard techniques with Face Net
embeddings as feature vectors. Our method uses a deep convolutional network trained to
directly optimize the embedding itself, rather than an intermediate bottleneck layer as in
previous deep learning approaches. To train, we use triplets of roughly aligned matching /
non- matching face patches generated using a novel online triplet mining method. The
benefitof our approach is much greater representational efficiency: we achieve state of the art
face recognition performance using only 128-bytes perface. On the widely used Labeled
Faces in the Wild (LFW) dataset, our system achieves a new record accuracy of 99.63%. On
YouTube Faces DB it achieves 95.12%. Our system cuts the error rate in comparison to the
best publishedresult by 30% on both datasets. In this paper we present a unified system for
face verification (is this the same person), recognition (who is this person) and clustering (find
common peopleamong these faces). Our method is based on learning a Euclidean embedding
per image using a deep convolutional network. The network is trained such that the squared
L2 distances in the embedding space directly correspond to face similarity: faces of the same
person have small distances and faces of distinct people have large distances. Once this
embedding has been produced, then the afore-mentioned tasks become straight forward: face
verification simply involves thresholding the distance between the two embeddings;
recognition becomes a k-NN classification problem; and clustering can be achieved using off-
the-shelf techniques such as k- means or agglomerative clustering. Previous face recognition
approaches based on deep networks use a classification layer trained over a set of known face
identities and then take an intermediate bottle neck layer as a representation used to generalize
recognition beyond the set of identities used in training. The downsides of this approach are
its indirectness and its inefficiency.

4
Ref 13:

Title: Semantic Image Inpainting with Deep Generative Models

Authors: R.A.Yeh, C.Chen, T.Y.Lim, A.G.Schwing, M.Hasegawa Jhonson and M.N.Do

Semantic image inpainting is a challenging task where large missing regions have to
be filled based on the available visual data. Existing methods which extract information from
only a single image generally produce unsatisfactory results due to the lack of high level
context. In this paper, we propose a novel method for semantic image inpainting, which
generates the missing content by conditioning on the available data. Given a trained
generative model, we search for the closest encoding of the corrupted image in the latent
image manifold using our context and prior losses. This encoding is then passed through the
generative model to infer the missing content. In our method, inference is possible irrespective
of how the missing content is structured, while the state-of-the-art learning based method
requires specific information about the holes in the training phase. Experiments on three
datasets show that our method successfully predicts information in large missing regions and
achieves pixel-level photorealism, significantly outperforming the state-of-the-art methods.
Semantic inpainting refers to the task of inferring arbitrary large missing regions in images
based on image semantics. Since prediction of high-level context is required this task is
significantly more difficult than classical inpainting or image completion which is often more
concerned with correcting spurious data corruption or removing entire objects. Numerous
applications such as restoration of damaged paintings or image editing benefit from accurate
semantic inpainting methods if large regions are missing. However, inpainting becomes
increasingly more difficult if large regions are missing or if scenes are complex. Classical
inpainting methods are often based on either local or non-local information to recover the
image. Most existing methods are designed for single image in painting. Hence they are based
on the information available in the input image, and exploit image priors to address the
illposedness. For example, total variation (TV) based approaches take into account the
smoothness property of natural images, which is useful to fill small missing regions or remove
spurious noise. Holes in textured images can be filled by finding a similar texture from the
same image. Prior knowledge, such as statistics of patch off- sets, planarity or low rank (LR)
can greatly improve the result as well. Patch Match (PM) searches for similar patches in the
available part of the image and quickly became one of the most successful inpainting methods
due to its high quality and efficiency.

4
Ref 14:
Title: Behavioural Biometric Verification of Student Identity in Online Course
Assessment and Authentication Of Authors In Literacy Works

Authors: J. V. Monaco, J. C. Stewart, S.-H. Cha, and C. C. Tappert

Keystroke and stylometry behavioral biometrics were investigated with the objective
of developing a robust system to authenticate students taking online examinations. This work
responds to the 2008 U.S. Higher Education Opportunity Act that requires institutions of
higher
learning undertake greater access control efforts, by adopting identification technologies as
they become available, to assure that students of record are those actually accessing the
systems and taking the exams in online courses. Performance statistics on keystroke,
stylometry, and combined keystroke-stylometry systems were obtained on data from 30
students taking examinations in a university course. The performance of the keystroke system
was 99.96% and 100.00%, while that of the stylometry system was considerably weaker at
74% and 78%, on test input of 500 and 1000 words, respectively. To further investigate the
stylometry system, a separate study on 30 book authors achieved performance of 88.2% and
91.5% on samples of 5000 and 10000 words, respectively, and the varied performance over
the population of authors was analyzed. The main application of interest in this study is
verifying the identity of students in online examination environments, an application that is
becoming more important with the student enrollment of online classes increasing, and
instructors and administrations becoming concerned about evaluation security and academic
integrity. The 2008 federal Higher Education Opportunity Act (HEOA) requires institutions of
higher learning to make greater access control efforts for the purposes of assuring that
students of record are those actually accessing the systems and taking online exams by
adopting identification technologies as they become more ubiquitous. To meet the needs of
this act, the keystroke biometric seems appropriate for the student authentication process.
Stylometry appears to be a useful addition to the process because the correct student may be
keying in the test answers while a coach provides the answers with the student merely typing
the coach’s words without bothering to convert the linguistic style into his own. Keystroke
biometric systems measure typing characteristics believed to be unique to an individual and
difficult to duplicate. The keystroke biometric is abehavioral biometric, and most of the
systems developed previously have been experimental in nature.

4
Ref 15:
Title: A Multi-Biometric System for Continuous Student Authentication in E-Learning
Platforms
Authors: G. Fenu, M. M arras, and L. Boratto

In recent years, online courses have emerged as a new way to educate students in distance
learning settings. However, as the demand increases, educational institutions are facing the
challenge of how to prove that online students are who they claim to be during e-learning
activities, especially exams. Human proctoring is a non-scalable approach which requires a
person to monitor each student remotely. On the other hand, automated proctors tend to target
a specific type of device and verify the students’ presence without considering their
interaction with the e-learning platform. In this paper, we propose a device/interaction-
agnostic multi- biometric system aimed at continuously and transparently verifying both the
presence and the interaction. By performing a score-level fusion of different biometric
responses (face, voice, touch, mouse, keystroke) based on the device used and the interaction
carried out with it, the system is able to attest the student's identity throughout the learning
experience. In preliminary comparison with the existing approaches, our contribution has a
good potential to provide a flexible and reliable support on a larger set of online experiences.
In recent years, online courses have emerged as a flexible way to educate large groups of
students in distance learning settings. However, as the demand increases, educational
institutions are facing the challenge of assessing the integrity of their initiatives, without
classroom attendance. In this context, possible threats include impostors taking an online
examination as well as the dissimulation of the identity credentials to other users. In order to
minimize such threats, institutions should verify the identities of online students, guarantee
they take exams without cheating, and assess their performance adequately. In this paper, we
focus on the first task aimed to prove that the student who is certified is the one who performs
the e-learning activities. In general, the existingsolutions are usually designed to proctor
students only during online exams. They do not tend to consider the rest of the course
participation. The common procedure allows students to perform exams online while a person
monitors them remotely. It is non-scalable and depends on the ability of the human proctor.
Other systems replace human with algorithms to achieve thesame goal via post or real-time
analysis, but at lower costs, as they eliminate the human effort.

4
Ref 16:
Title: A Review of Face Recognition Technology
Authors: L. Li, X. Mu, S. Li, and H. Peng

Face recognition technology is a biometric technology, which is based on the

identification of facial features of a person. People collect the face images, and the recognition
equipment automatically processes the images. The paper introduces the related researches of
face recognition from different perspectives. The paper describes the development stages and
the related technologies of face recognition. We introduce the research of face recognition for
real conditions, and we introduce the general evaluation standards and the general databases
of face recognition. We give a forward-looking view of face recognition. Face recognition has
become the future development direction and has many potential application prospects. Face
recognition is a subdivision problem of visual pattern recognition. Humans are recognizing
visual patterns all the time, and we obtain visual information through our eyes. This
information is recognized by the brain as meaningful concepts. For a computer, whether it is a
picture or a video, it is a matrix of many pixels. The machine should find out what concept a
certain part of the data represents in the data. This is a rough classification problem in visual
model recognition. For face recognition, it is necessary to distinguish who the face belongs to
in the part of the data that all machines think of the face. This is a subdivision problem. Face
recognition in a broad sense includes related technologies for building a face recognition
system. It includes face detection, face position, identity recognition, image preprocessing,
etc. Face detection algorithm is to find out the coordinate system of all faces in one image.
This is the process of scanning the entire image to determine whether the candidate area is a
face. The output of the face coordinate system can be square, rectangular, etc. The face
position is the coordinate position of the face feature in the face detection coordinate system.
The deep learning framework basically implements some current good positioning
technologies. Compared with face detection, the calculation time of face positioning algorithm
is much shorter. An artificial intelligence (AI) product called AlphaGo which was developed
by a team led by Deep Minda’s Demis Hassabis came out. And it beat Ke Jie who was the No.
1 player in Go level in May 2017. In October 2017, the DeepMind team announced the
strongest version of AlphaGo, named AlphaGo Zero. The essence of chess playing and face
recognition is to find suitable transform function. Although their principles are the same, the
complexity of face recognition transformation is far greater than the complexity of finding the
optimal solution in the chessboard.

4
Ref 17:

Title: Efficient Generic Face Model Fitting to Images and Videos

Authors: L. Unzueta, W. Pimenta, J. Goenetxea, L. P. Santos, and F. Dornaika

we present a robust and lightweight method for the automatic fitting of deformable 3D
face models on facial images. Popular fitting techniques such as those based on statistical
models of shape and appearance require a training stage based on a set of facial images and
their corresponding facial landmarks, which have to be manually labeled. Therefore, new
images in which to fit the model cannot differ too much in shape and appearance (including
illumination variation, facial hair, wrinkles, etc) from those used for training. By contrast, our
approach can fit a generic face model in two steps: the detection of facial features based on
local image gradient analysis and the back projection of a deformable 3D face model through
the optimization of its deformation parameters. The proposed approach can retain the
advantages of both learning-free and learning-based approaches. Thus, we can estimate the
position, orientation, shape and actions of faces, and initialize user-specific face tracking
approaches, such as Online Appearance Models (OAM), which have shown to be more robust
than generic user tracking approaches. Experimental results show that our method
outperforms other fitting alternatives under challenging illumination conditions and with a
computational cost that allows its implementation in devices with low hardware
specifications, such as smartphones and tablets. Our proposed approach lends itself nicely to
many frameworks addressing semantic inference in face images and videos. Generic face
model fitting has received much attention in the last decade. Face model fitting can be seen as
a basic component in many Human–Computer Interaction applications since it enables facial
feature detection, head pose estimation, face tracking, face recognition, and facial expression
recognition. In general terms, two main kinds of approaches have been proposed: learning-
free and learning-based. The latter need a training stage with several images to build the
model, and therefore depend on the selection ofimages for a good fitting in unseen images.
Learning-free approaches rely heavily on some radiometric and geometric properties
associated with face images. These approaches exploit generic knowledge about faces, which
often includes the position, symmetry, and edge shape of facial organs. They can locate facial
features through low-level techniques (e.g. gradients, filtering), usually focusing on detecting
individual face features (irises, nose, lips, …). Most ofthe learning-free approaches do not
provide a full set of extracted face features, in contrast with learning-based techniques.

4
Ref 18:

Title: SSD: Single Shot MultiBox Detector

Authors: W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu and
A.C.Berg

We present a method for detecting objects in images using a single deep neural
network. Our approach, named SSD, discretizes the output space of bounding boxes into a set
of default boxes over different aspect ratios and scales per feature map location. At prediction
time, the network generates scores for the presence of each object category in each default box
and produces adjustments to the box to better match the object shape. Additionally, the
network combines predictions from multiple feature maps with different resolutions to
naturally handle objects of various sizes. SSD is simple relative to methods that require object
proposals because it completely eliminates proposal generation and subsequent pixel or
feature resampling stages and encapsulates all computation in a single network. This makes
SSD easy to train and straightforward to integrate into systems that require a detection
component. Experimental results on the PASCAL VOC, COCO, and ILSVRC datasets
confirm that SSD has competitive accuracy to methods that utilize an additional object
proposal step and is much faster, while providing a unified framework for both training and
inference. For 300×300 input, SSD achieves 74.3 % map on VOC2007 test at 59 FPS on a
Nvidia Titan X and for 512×512 input, SSD achieves 76.9 % map, outperforming a
comparable state of the art Faster R-CNN model. Compared to other single stage methods,
SSD has much better accuracy even with a smaller input image size. Current state-of-the-art
object detection systems are variants of the following approach: hypothesize bounding boxes,
resample pixels or features for each box, and apply a high-quality classifier. This pipeline has
prevailed on detection bench marks since the Selective Search work through the current
leading results on PASCAL VOC, COCO, and ILSVRC detection all based on Faster R-CNN
albeit with deeper features such as 3. While accurate, these approaches have been too
computationally intensive for embedded systems and, even with high- end hardware, too slow
for real-time applications. Often detection speed for these approaches is measured in frames
per second, and even the fastest high-accuracy detector, Faster R-CNN, operates at only 7
frames per second (FPS). There have been many attempts tobuild faster detectors by attacking
each stage of the detection pipeline (see related work in Sect.4), but so far, significantly
increased speed comes only at the cost of significantly decreased detection accuracy.

4
Ref 19:
Title: Keystroke Dynamics for User Authentication
Authors: Y.Zhong, Y.Deng and A.K.Jain

we investigate the problem of user authentication using keystroke biometrics. A new

distance metric that is effective in dealing with the challenges intrinsic to keystroke dynamics
data, i.e., scale variations, feature interactions and redundancies, and outliers is proposed. Our
keystroke biometrics algorithms based on this new distance metric are evaluated on the CMU
keystroke dynamics benchmark dataset and are shown to be superior to algorithms using
traditional distance metrics. With the ever increasing demand for more secure access control
in many of today’s security applications, traditional methods such as PINs, tokens, or
passwords fail to keep up with the challenges presented because they can be lost or stolen,
which compromises the system security. On the other hand, biometrics based on “who” is the
person or “how” the person behaves present a significant security advancement to meet these
new challenges. Among them, keystroke dynamics provides a natural choice for secure
“password- free” computer access. Keystroke dynamics refers to the habitual patterns or
rhythms an individual exhibits while typing on a keyboard input device. These rhythms and
patterns of tapping are idiosyncratic , in the same way as handwritings or signatures, due to
their similar governing neurophysiological mechanisms. As early as in the 19th century,
telegraph operators could recognize each other based on one’s specific tapping style. This
suggests that keystroke dynamics contain sufficient information to serve as a potential
biometric identifier to ascertain a specific keyboard user. Compared to other biometrics,
keystroke biometrics has additional desirable properties due to its user-friendliness and non-
intrusiveness. Keystroke dynamics data can be collected without a user’s cooperation or even
awareness. Continuous authentication is possible using keystroke dynamics just as a mere
consequence of people’s use of computers. Unlike many other biometrics, the temporal
information of keystrokes can be collected to ascertain a user using only software and no
additional hardware. In summary, keystroke dynamics biometrics enables a cost effective,
user friendly, and continuous user authentication with potential for high accuracy. Although
keystroke dynamics is governed by a person’s neurophysiological pathway to be highly
individualistic, it can also be influenced by his or her psychological state. As a “behavioral”
biometrics, keystroke dynamics exhibits instabilities due to transient factors such as emotions.

4
Ref 20:

Title: Generative Adversarial Nets

Authors: Goodfellow, J.Pouget Abadie, M.Mirza, B.Xu, D.Warde-Farely, S.Ozair,
A.Courville, and Y. Bengio

We propose a new framework for estimating generative models via adversarial nets, in
which we simultaneously train two models: a generative model G that captures the data
distribution, and a discriminative model D that estimates the probability that a sample came
from the training data rather than G. The training procedure for G is to maximize the
probability of D making a mistake. This framework corresponds to a minimax two-player
game. In the space of arbitrary functions G and D, a unique solution exists, with G recovering
the training data distribution and D equal to 1/2 everywhere. In the case where G and D are
defined by multilayer perceptrons, the entire system can be trained with back propagation.
There is no need for any Markov chains or unrolled approximate inference networks during
either training or generation of samples. Experiments demonstrate the potential of the
framework through qualitative and quantitatively evaluation of the generated samples. The
promise of deeplearning is to discover rich, hierarchical models that represent probability
distributions over thekinds of data encountered in artificial intelligence applications, such as
natural images, audio waveforms containing speech, and symbols in natural language corpora.
So far, the most striking successes in deep learning have involved discriminative models,
usually those that mapa high-dimensional, rich sensory input to a class label. These striking
successes have primarilybeen based on the back propagation and dropout algorithms, using
piecewise linear units whichhave a particularly well-behaved gradient. Deep generative
models have had less of an impact,due to the difficulty of approximating many intractable
probabilistic computations that arise inmaximum likelihood estimation and related strategies,
and due to difficulty of leveraging the benefits of piecewise linear units in the generative
context. We propose a new generative modelestimation procedure that sidesteps these
difficulties. In the proposed adversarial nets framework, the generative model is pitted against
an adversary: a discriminative model that learns to determine whether a sample is from the
model distribution or the data distribution. The generative model can be thought of as
analogous to a team of counterfeiters, trying to produce fake currency and use it without
detection, while the discriminative model is analogousto the police, trying to detect the
counterfeit currency. Competition in this game drives both teams to improve their methods
until the counterfeits are in distinguishable from the genuine

5
articles. This framework can yield specific training algorithms for many kinds of model and
optimization algorithm. In this article, we explore the special case when the generative model
generates samples by passing random noise through a multilayer perceptron, and the
discriminative model is also a multilayer perceptron. We refer to this special case as
adversarial nets. In this case, we can train both models using only the highly successful back
propagation and dropout algorithms and sample from the generative model using only forward
propagation. No approximate inference or Markov chains are necessary.

5
 CHAPTER - 3

PROPOSED METHODOLGY

3.1 EXISTING SYSTEM

Under the current system, online certification and accreditation, training institutes require
verification that only online students who have completed the learning process and obtained
academic credentials are enrolled for the courses. Furthermore, they need to make sure that
these students complete all the activities of online training without cheating or misbehaving.
The COVID-19 pandemic has accelerated the migration and implementation of online
educational strategies (suddenly in some cases) and consequently the need for safer
mechanisms to standardize and promote online students.

3.1.1 Disadvantages:

• Have fake students attending to classes and exams

• Less secure

5
3.2 PROPOSED SYSTEM

In proposed system, we will describe in depth the specific solution based on the authentication
of various biometric technologies and the automatic procuring system, it has the features to
solve the major problems in the market: highly scalable, automatic, flexible, user-friendly
with some hardware, user-friendly and passive.

3.2.1 Advantages:

• Identifying the fake students

• Allows only real users

5
Block Diagram:

Fig 3.1: Block Diagram of Proposed Method

5
 ARCHITECTURE

Fig 3.2: Architecture of the Project

5
3.2.2 PROJECT ALGORITHM

CONVOLUTIONAL NEURAL NETWORK

Step(1a): convolutional operation

The first building block in our plan of attack is convolution operation. In this step, we will
touch on feature detectors, which basically serve as the neural network's filters. We will also
discuss feature maps, learning the parameters of such maps, how patterns are detected, the
layers of detection, and how the findings are mapped out.

Fig 3.3: Convolution Operation

5
Step (1b): Relu Layer

The second part of this step will involve the Rectified Linear Unit or Relook. We will cover
Relook layers and explore how linearity functions in the context of Convolutional Neural
Networks.

Not necessary for understanding CNN's, but there's no harm in a quick lesson to improve your
skills.

Fig 3.4: Convolutional Neural Network Scan Images

Step 2: Pooling Layer

In this part, we'll cover pooling and will get to understand exactly how it generally works. Our
nexus here, however, will be a specific type of pooling; max pooling. We'll cover various
approaches, though, including mean (or sum) pooling. This part will end with a demonstration
made using a visual interactive tool that will definitely sort the whole concept out for you.

Step 3: Flattening

This will be a brief breakdown of the flattening process and how we move from pooled to
flattened layers when working with Convolutional Neural Networks.

5
Step 4: Full Connection

In this part, everything that we covered throughout the section will be merged together. By
learning this, you'll get to envision a fuller picture of how Convolutional Neural Networks
operate and how the "neurons" that are finally produced learn the classification of images.

Summary:
In the end, we'll wrap everything up and give a quick recap of the concept covered in the
section. If you feel like it will do you any benefit (and it probably will), you should check out
the extra tutorial in which Soft ax and Cross-Entropy are covered. It's not mandatory for the
course, but you will likely come across these concepts when working with Convolutional
Neural Networks and it will do you a lot of good to be familiar with them.

Fig 3.5: Convolutional Neural Network

5
3.3 SYSTEM REQUIREMENTS

HARDWARE REQUIREMENTS:

Operating system : Windows

RAM : 4GB (min)
Hard disc or SSD : More than 250 GB (min)
Processor : 13/intel processor (min)

SOFTWARE REQUIREMENTS:

Application Server : Xampp

Front End : HTML, JSP
Scripts : JavaScript
Technology : Python
Database : My SQL 6.0
IDE : Pycharm

5
 REFERENCES
[1] H. S. G. Asep and Y. Bandung, ‘‘A design of continuous user verification for online
exam proctoring on M-learning,’’ in Proc. Int. Conf. Electr. Eng. Informat. (ICEEI), Jul.
2019, pp. 284–289.

[2] L. K. Musambo and J. Phiri, ‘‘Student facial authentication model based on OpenCV’s
object detection method and QR code for Zambian higher institutions of learning,’’ Int. J.
Adv. Comput.Sci. Appl., vol. 9, no.5, Jan. 2018.

[3] Z. Zhang, M. Zhang, Y. Chang, S. Esche, and C. Chassapis, ‘‘A virtual laboratory
system with biometric authentication and remote proctoring based on facial
recognition,’’ Comput. Educ. J., vol. 7, no. 4, pp. 78-84, 2016.

[4] L. Wan, Q. Wang, A. Papir, and I. L. Moreno, ‘‘Generalized end-to-end loss for speaker
verification,’’ in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), Apr.
2018, pp. 4879-4883.

[5] S. Zhang, X. Zhu, Z. Lei, H. Shi, X. Wang, and S. Z. Li, ‘‘FaceBoxes: A CPU real-time
face detector with high accuracy,’’ in Proc. IEEE Int. Joint Conf. Biometrics (IJCB), Oct.
2017, pp. 297–309

[6] Z. Zhang, E.-S. Aziz, S. Esche, and C. Chassapis, ‘‘A virtual proctor with biometric
authentication for facilitating distance education,’’ in Online Engineering & Internet of
Things, M. E. Auer and D. G. Zutin, Eds. Cham, Switzerland: Springer,2018, pp.110-
124.

[7] F. Guillen-Gamez, I. García-Magarino, and G. Palacios, Comparative Analysis Between

Different Facial Authentication Tools for Assessing Their Integration in m-Health
Mobile Applications. Cham, Switzerland: Springer, Mar. 2018, pp. 1153–1161.

[8] E. Flior and K. Kowalski, ‘‘Continuous biometric user authentication in online

examinations,’’ in Proc. Int. Conf. Inf. Technol., Jan. 2010, pp. 488–492.

[9] Y. Atoum, L. Chen, A. X. Liu, S. D. H. Hsu, and X. Liu, ‘‘Automated online exam
proctoring,’’ IEEE Trans. Multimedia, vol. 19, no. 7, pp. 1609–1624, Jul. 2017.

[10] F. Guillen-Gamez, J. Bravo, and I. García-Magarino, ‘‘Students’ perception of the

importance of facial authentication software in moodle tools,’’ Int. J. Eng. Educ., vol. 33,
pp. 84–90, Jan. 2017.

6
[11] A. Ullah, H. Xiao, and T. Barker, ‘‘A dynamic profile questions approach to mitigate
impersonation in online examinations,’’ J. Grid Comput., vol. 17, no. 2, pp. 209–223,
Jun. 2019,doi: 10.1007/s10723-018-9442-6.

[12] F. Schroff, D. Kalenichenko, and J. Philbin, ‘‘FaceNet: A unified embedding for face
recognition and clustering,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.
(CVPR), Jun.2015, pp. 815–823.

[13] R. A. Yeh, C. Chen, T. Y. Lim, A. G. Schwing, M. Hasegawa-Johnson, and M. N. Do,

‘‘Semantic image inpainting with deep generative models,’’ in Proc. IEEE Conf.
Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 6882-6890.

[14] J. V. Monaco, J. C. Stewart, S.-H. Cha, and C. C. Tappert, ‘‘Behavioural biometric

verification of student identity in online course assessment and authentication of authors
in literary works,’’ in Proc. IEEE 6th Int. Conf. Biometrics, Appl. Syst. (BTAS), Sep.
2013, pp. 1-8.

[15] G. Fenu, M. Marras, and L. Boratto, ‘‘A multi-biometric system for continuous student
authentication in e-learning platforms,’’ Pattern Recognit. Lett., vol. 113, pp. 83-92, Oct.
2018.

[16] L. Li, X. Mu, S. Li, and H. Peng, ‘‘A review of face recognition technology,’’ IEEE
Access, vol. 8, pp. 139110-139120, 2020.

[17] L. Unzueta, W. Pimenta, J. Goenetxea, L. P. Santos, and F. Dornaika, ‘‘Efficient generic

face model fiiting to images and videos,” Images Vis. Comput., vol. 32, no. 5, pp. 321-
334, May 2014.

[18] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, SSD:
Single Shot Multibox Detector (Lecture Notes in Computer Science: Lecture Notes in
Artificial Intelligence and Lecture Notes in Bioinformatics). Springer, 2016.

[19] Y. Zhong, Y. Deng, and A. K. Jain, ‘‘Keystroke dynamics for user authentication,’’ in
Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. Workshops, Jun. 2012,
pp.117- 123.

[20] Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A.

Courville, and Y. Bengio, ‘‘Generative adversarial nets,’’ in Proc. 27th Int. Conf. Neural
Inf. Process. Syst.(NIPS), vol. 2. Cambridge, MA, USA: MIT Press, 2014, pp. 2672-
2680.
6