Person-specific Precision Analysis

of Gaze Estimation Models on

PARKS-Gaze Dataset
Human machine interaction | Assignment

Submitted by,
Gara Shashi Kumar (M.Tech (SM)) | Annu Niraj (M.Tech (SM))
Pravinkumar Vasveliya (M.Des) | Anand Kumar (Phd. – CPDM)
Under the guidance of LRD Murthy, Professor Pradipta Biswas
 Agenda
Introduction Work Code logic Lessons

Task
Approach
description

Output

03/10/2022 HMI Assignment 1 2

 Task Description

Train 2 gaze estimation models (AGE-Net & I2D-Net) which involves both fully
connected layers and convolutional layers on the PARKS-Gaze dataset which involves
large number of samples per participant recorded in the wild conditions with laptop web
cameras. This dataset is intended to build models that estimate human eye gaze using RGB
images. The task involves to pick each participant (a total of 18 participants) and train a
person-specific model. Once you have a person-specific model, in addition to compute
gaze angle error, you are expected to compute precision of gaze estimates corresponding to
a single PoG.
03/10/2022 HMI Assignment 1 3
Expected Work
The following things are expected:
1. Split the data into train and validation
splits.
2. Train the model using training data and
save the best performing models based on
the validation loss.
3. Report the metrics - test loss, gaze angle
error on test data and average precision for
each participant for both models.
4. Provide insights/comments on the
correlation of precision w.r.t head pose
variation for each fixation.
 Brief Description of the project
• Person-specific Precision Analysis of Gaze Estimation Models on PARKS-
Gaze Dataset
• I2D (I-Gaze estimation using dilated and differential layer network) model
was built using python.
• Then after, this I2D net model was trained using the datasets provided to us.
• Loss graph was plotted against 35 epochs to get the accuracy of the model.
• Code was written to train and split data manually.
• Person specific model was built and trained.

03/10/2022 HMI Assignment 1 5

 Approach

03/10/2022 HMI Assignment 1 6

 Code logic
Defining Blank arrays Using ‘for’ loop to read Loading data into train,
for storing train and test the data for each test and validation Reshape the input Model formulation
data session arrays

CNN with VGG output Combining model for

CNN for eye channel VGG for face Dense the model
for face both eyes

Architecture for angle

error computation and Plotting the graph of
Fitting the model Angle validation
unit gaze vector epoch vs validation loss
calculation

03/10/2022 HMI Assignment 1 7

 Output

mean angle error test:    8.11979845100328

 std dev error test:  4.363696565758978 
Model loss graph after training the model Model Loss graph after training the model using person
using PARKS-GAZE Dataset specific data

03/10/2022 HMI Assignment 1 8

 Final Output

03/10/2022 HMI Assignment 1 9

 Process
Sr. No. Stages of Project Process and Outcomes

AGE- NET model was based on the attention- mechanism approach.

Understanding the
1. CNN- Backbone code for both eyes and face channel was studied.
AGE - NET model
Basic model architecture and convolutional layers arrangement was derived.

The inferences derived from AGE- NET code was used to design the model for I2D NET
Formulation of I2D -
2. model.
NET model
Model summary was derived to obtain the parameters.

Training the I2D - NET The model was trained using sample data.
3.
model Train – Test – split was utilized.

Splitting the data into

Code was written for session wise splitting data.
4. validation, train and
Out of 8 sessions, 5 was used for training the model, 1 for validation and 2 for testing.
test.

Retraining the model

5.
03/10/2022 Error
HMI was obtained
Assignment 1 for data cardinality ambiguous. 10
using updated code
 Lessons
• Concepts of Machine Learning, Deep Learning and Data
Science.
• Python programming language.
• Using other softwares such as Jupyter, google colab,
anaconda etc.
• Using libraries such as Tensorflow, Keras, Matplotlib
etc.
• Learnt to handle arrays using numpy, and Random.

03/10/2022 HMI Assignment 1 11

Thank you.