Facial & Emotion

Recognition
What’s the difference?

By Matt Celuszak, CEO

Crowd Emotion Limited & Element Human Limited
 CONTENT

Content
Facial & Emotion
About the author

Introduction Recognition
Why we want it

What are they? What’s the difference?

What’s the difference?

Uses Cases By Matt Celuszak, CEO

Crowd Emotion Limited & Element Human Limited
Risk & Regulation

Contact

Matt Celuszak
matt@elementhuman.com
 ABOUT THE AUTHOR

!e better we understand each other, the

better our world will be.
Content Disclaimer: I am not a god at machine learning, nor am I a guru psychologist. I'm not
going to sit here and tell you that facial recognition is the bee's knees and you should
About the author
throw away your privacy to protect your freedom.
Introduction
Nope, I'm just the guy who couldn't read body language to save his life. Too many
Why we want it awkward, face-palm moments is a testament to that. I am a glutton for punishment. So, I
started CrowdEmotion to see if technology could help me understand people better.
What are they?
Along the journey, I ended up challenging how facial recognition was built, applied and
What’s the difference?
deployed after living first hand many of the issues we see today. Here’s what I learned.
Uses Cases

Risk & Regulation Matt Celuszak is the Founder and CEO of CrowdEmotion and Element Human. He has
spent the last 16 years building human measurement systems and founded the
Contact
companies to unlock our understanding of humanity through technology.

Matt and the Element Human team are on a mission to enable everyday devices to
observe, monitor and feedback human interaction in a safe way that promotes the well
being of the individual, monitors the health of the population, and furthers the evolution
of humanity.

Matt Celuszak
matt@elementhuman.com
 INTRODUCTION

Face technologies are here to stay. !ey

need work, clarity and regulation.
Content
Background
About the author
We spent 7 years curating over 2.5 Billion facial attention and emotion data points with
Introduction consent from 450,000 people in 89 countries to inform 40 businesses to understand how
body language relates to human behaviour and business performance. We built, and
Why we want it
rebuilt, emotion recognition technology 5 times to solve for a number of both technical
What are they? and systematic bias challenges prevalent within the technologies on the market.

What’s the difference? In short, there are technical and societal problems that need to be solved, but after 7
years of seeing the benefits of face detection technologies, they are worth solving.
Uses Cases
Carefully.
Risk & Regulation
Below we explore what the different technologies are, their use cases and the
Contact considerations needed for policy and decision making.

Matt Celuszak
matt@elementhuman.com
 WHY WE WANT IT

Why we want face

Content
detection technology
About the author
For me, it was about understanding people.
Introduction
!e better we understand each other, the better our world will be.
Why we want it
Idealistic, but true. !en I realised the true power of accessibility.
What are they?
But the real reason is that... You are lazy.
What’s the difference?
Yes, YOU dear reader. Let's be really honest here.
Uses Cases !
You love no-pin access to your iPhone.! Facial Verification.
Risk & Regulation You want to date a real person online. Facial Recognition.
!
You want digital banking, but secure fraud!detection. Facial Recognition.
Contact
You want to be entertained with animojis. Emotion Recognition.

This extends into a number of other applications. Some much higher brow, but mostly,
you are lazy and demand that the world recognise you, verify your access and
personalise with empathy.

And why shouldn't you? We have the means, you demand the best.

Matt Celuszak
matt@elementhuman.com
 WHY WE WANT IT

Content What do we need to consider?

About the author

• Exposure - How private is it, full identity or only a bit?
Introduction
• Control - Who controls my data, me or the open market?
Why we want it
• Culpability - Who is responsible for malpractice, me or the machine?
What are they?
• Power - How unbalanced is the power for those holding the information?
What’s the difference?

Uses Cases

Risk & Regulation

Contact

Matt Celuszak
matt@elementhuman.com
 WHAT ARE THEY

What are they

Content

About the author

Facial Verification
Introduction
Typically used to identify an individual in realtime at the point of interaction, it uses the
Why we want it device webcam to take a picture of your face and match it to a photo of your face from
an accepted ID (like a passport, drivers license, residence permit etc). The photo can
What are they?
often be uploaded or can be simply displayed to the device webcam as well.
What’s the difference?

Uses Cases

Risk & Regulation Facial Verification

Contact
Pros Cons
• Directly identifies an individual • Ability to use can be limited by poor quality
webcams, low lighting and racial bias
• Protects consumers from fraudulent access to
their accounts and transactions • Data is determinative

• Can be done in real time with no need to store

identity data long term

Matt Celuszak
matt@elementhuman.com
 WHAT ARE THEY

Content

About the author

Facial Recognition
Introduction
Like a fingerprint, facial recognition is typically used to identify individuals and link
Why we want it information to their digitally stored profiles often associated with government ID. It works
by using cameras to capture images of your face and match it to a database of profiles
What are they?
to start building a deep characteristic of you as an individual.
What’s the difference?
While it can be used to verify the identity of individuals like Facial Verification, it’s primary
Uses Cases use is within security and law enforcement to identify, track and source criminals through
camera networks.
Risk & Regulation
Given the law enforcement use and the technological bias prevalent within computer
Contact
recognition systems today, this is where the distinct issues arise as citizens are not
empowered to dispute the veracity of the technology with their tax funded law
enforcement agencies putting a relatively large power imbalance in the hands of
authorities.

Matt Celuszak
matt@elementhuman.com
 WHAT ARE THEY

Facial Recognition

Content Pros Cons

About the author • Directly Identifies an individual • Stores directly identifiable data on a server

Introduction • Can track criminals through identity • Power imbalance - technology only

Why we want it accessible by big corps and government

• Fast track check-in processes
What are they? • Systemic bias determines enforcement
• Create personalised experiences
What’s the difference? • Data is determinative

Uses Cases

Risk & Regulation

Contact

Matt Celuszak
matt@elementhuman.com
 WHAT ARE THEY

Content

About the author

Facial Biometrics
Introduction
These are secondary features extracted from the face like head movement, eye-
Why we want it movement, heart rate, muscle movement, expressions, assumed emotions, predicted
gender, predicted age etc. Like the others, facial biometrics typically uses a webcam to
What are they?
find a face within an image or video, identify the facial features, and track how they
What’s the difference? move.

Uses Cases You see this capability in everyday devices and photo editing tools that map and move
to a face.
Risk & Regulation
Facial biometrics are typically, but not always, conducted with explicit consent from the
Contact
individual and do not require identity to be useful.

Matt Celuszak
matt@elementhuman.com
 WHAT ARE THEY

Facial Biometrics

Content Pros Cons

About the author • Unlocks contextual information - attention, • Processing power required means videos are
expression, emotion stored on servers
Introduction

Why we want it • No direct identity needed

What are they? • Often used in aggregate

What’s the difference? • Data is informative

Uses Cases • Often requires explicit consent

Risk & Regulation

Contact

Matt Celuszak
matt@elementhuman.com
 WHAT ARE THEY

Content Explanation of Terms

About the author
Determinative vs Informative Data
Introduction
When looking at automating information, the question comes down to the role and
Why we want it influence a human has to play in the final decision.

What are they? Determinative:

What’s the difference? The machine will make a decision without a human verification. Facial verification and
Facial Recognition technology fall into this category.
Uses Cases
On the plus side, determinative data can be scaled up and left to run on its own.
Risk & Regulation
Conversely, this means any systematic bias will impact many more people than
Contact anticipated.

Informative:

The machine will inform a human who will make a final decision. This information is rarely
binary and needs to be interpreted by an individual before invoking action.

On the plus side, informative data can rarely be used in isolation from human judgement.
Conversely, it has limits to scalable applications and needs to be embedded within a
human workflow.

Matt Celuszak
matt@elementhuman.com
 WHAT’S THE DIFFERENCE

What’s the difference

Content

About the author Facial Verification Facial Recognition Facial Biometrics

Type
Introduction
Human Influence Determinative Determinative Informative
Why we want it

What are they?

Explicit Consent Req’d Yes No Yes
What’s the difference?

Uses Cases Anonymised Yes No Yes

Risk & Regulation

Contact Storage Device Server Both

Matt Celuszak
matt@elementhuman.com
 USES CASES

Uses Cases
Facial Verification Facial Recognition Facial Biometrics
Content Uses Cases*

About the author

Law Enforcement Primary
- -
Introduction - ID Criminals Criminal ID

Why we want it
Health Care
What are they? Primary
- Depression detection - -
Mood and health detection

What’s the difference? - Autism aid

- Alzheimer's aid
Uses Cases
- Psychologist aid

Risk & Regulation

Retail
Contact Primary Primary Primary
- ID & Track shoppers
Payments ID & Track Shoppers Customer Satisfaction
- Payments Payments

- Customer Satisfaction

Hospitality
Primary Primary
- Check-in rooms -
Check-in ID Check-in ID
- Loyalty Profiles Loyalty Tracking

* NB: Use Cases organised and described by CB Insight:

Matt Celuszak
matt@elementhuman.com https://www.cbinsights.com/research/facial-recognition-disrupting-industries/
 USES CASES

Facial Verification Facial Recognition Facial Biometrics

Uses Cases*

Marketing
Content Secondary Primary
- Customer experience -
Customer ID Customer feedback
About the author feedback
Scaled MR
- Scaled market research
Introduction

Why we want it Banking

Primary Primary Secondary
- ID detection
What are they? ID detection ID detection Unbanked Credit
- Credit Application Confidence
What’s the difference? for unbanked

Uses Cases Events

Primary Primary Secondary
Risk & Regulation - Ticket ID
Ticket ID Ticket ID Crowd Mood
- Crowd Mood
Contact

Social Media
Primary Primary
- Search -
Search Filters/Lenses
- Filters/Lenses Animojis

- Animoji

* NB: Use Cases organised and described by CB Insight:

Matt Celuszak
matt@elementhuman.com https://www.cbinsights.com/research/facial-recognition-disrupting-industries/
 USES CASES

Facial Verification Facial Recognition Facial Biometrics

Uses Cases*

Air Travel
Content Primary Primary Secondary
- Check-In
Passport scan Check-in Customer feedback
About the author - Passport scan Check-in Passport scan

- Customer feedback
Introduction

Why we want it
Automobiles
Primary Primary Primary
What are they? - Vehicle access
Vehicle access Vehicle access Driver safety
- Driver safety Cabin experience
What’s the difference?
- Cabin experience

Uses Cases
Gambling
Risk & Regulation Primary
- ID gamblers - -
ID gamblers

Contact - Maximise profit Maximise profit

Voting Primary
- -
- Voter verification Voter verification

* NB: Use Cases organised and described by CB Insight:

Matt Celuszak
matt@elementhuman.com https://www.cbinsights.com/research/facial-recognition-disrupting-industries/
 USES CASES

Facial Verification Facial Recognition Facial Biometrics

Uses Cases*

Education
Content Primary Primary Primary
- Roll call
Visitor access Roll call Teacher’s Aid
About the author - Visitor access Visitor access

- Teacher’s aid
Introduction

Why we want it Ride Hailing Secondary

Primary
-
- Passenger verification Passenger verification Passenger verification
What are they?

What’s the difference?

Food & Beverage
Primary Primary
Uses Cases - Diner experience -
Loyalty ID Diner experience
- Loyalty ID Payment verification
Risk & Regulation
- Payment verification

Contact
Consumer Tech
Primary Primary Primary
- Access
Access Access User experience
- User experience Personalisation

- Personalisation

* NB: Use Cases organised and described by CB Insight:

Matt Celuszak
matt@elementhuman.com https://www.cbinsights.com/research/facial-recognition-disrupting-industries/
 RISK & REGULATION

Risk & Regulation

Content
Considerations
About the author Like most technology, it depends on the use case and the user, but most of the risk
comes down to access and control: access to the technology and control of the decision
Introduction
stemming from the technology.
Why we want it
All of the technologies use artificial intelligence and are subject to systemic bias resulting
What are they? from training data and access to real world use during trial periods.

What’s the difference? This is the first risk that should have regulation across the board, requiring a reliability test
across two factors:
Uses Cases

Risk & Regulation

1. Trainability - the number of use cases required to achieve acceptable accuracy as
Contact set out by the use case experts. Ideally, better than a human practitioner, unless
where highly specialised, in which case more scalable than a human practitioner.

2. Transferability - Ability to apply an algorithm trained in one use case to another with
acceptable transfer rates based on similarity of use case.

Matt Celuszak
matt@elementhuman.com
 RISK & REGULATION

Content

About the author

Facial Recognition
Introduction
Facial Recognition holds the most amount of use cases and the highest risk. At the core,
Why we want it it is used to control access and keep a history of individuals which catalyses a power
imbalance between the public and law enforcement agencies. The public have no
What are they?
control or little recompense against a judgement made by facial recognition that can
What’s the difference? only really be accessed and funded by the state.

Uses Cases Technically, Facial Recognition holds the highest security risk for three main reasons:

Risk & Regulation 1. It is determinative, removing any human ownership

Contact 2. It controls access to necessary products and services

3. It cannot be anonymised by its very nature

Socially, Facial Recognition also holds the highest social risk: It can be systematically
biased creating unequal law enforcement.

Matt Celuszak
matt@elementhuman.com
 RISK & REGULATION

Content Regulation Considerations for Facial Recognition

About the author • Watchdog - For these technologies, regulation will need to account for the security of
the hosted database and abuse of power by those who have access to it
Introduction
• Algorithm Auditors - Suggest implementing AI and cyber security auditors for
Why we want it
systematic bias and algorithmic overfitting
What are they?
• Penalties for abuse of power - also severe penalties towards agencies for misuse
What’s the difference?

Uses Cases

Risk & Regulation

Contact

Matt Celuszak
matt@elementhuman.com
 RISK & REGULATION

Content

About the author

Facial Verification
Introduction
Facial Verification is a benign technology. While it does identify an individual, it is only to
Why we want it match it to a government issue ID. Like snapchat, it does not need to track a history, but
can still provide tighter security access controls than keys, passwords and even
What are they?
fingerprints.
What’s the difference?
Regulation Considerations for Facial Verification
Uses Cases
For Facial Verification, the appetite to migrate commercially into Facial Recognition is
Risk & Regulation
high.
Contact
• Realtime use only -restricted use cases in verification defined by matching
capabilities that do not reference a stored database

• Device only - restricted to device only use cases

Otherwise, it is Facial Recognition.

Matt Celuszak
matt@elementhuman.com
 RISK & REGULATION

Content

About the author

Facial Biometrics
Introduction
Facial Biometrics holds less direct risk because it is informative to a human, meaning the
Why we want it human makes the final decision acting upon all information including Facial Biometrics.
Medical use cases can pose a higher risk if positioned as determinative, but this is not
What are they?
recommended. Rather they should inform a doctor to make a final decision.
What’s the difference?
Facial Biometrics risks vary by type, but can include the following systematic bias:
Uses Cases
• Facial coverings - beards, glasses, burkas, niqabs may prevent access
Risk & Regulation
• Dark skin - low contrast can be difficult for cameras to read and many algorithms do
Contact no work well on dark skin - this is improving

• Low Quality Images - prevents those with low quality devices to participate

• Expertise Required - emotion interpretation often needs a psychology degree

Matt Celuszak
matt@elementhuman.com
 RISK & REGULATION

Content Regulation Considerations for Facial Biometrics

About the author In addition to the ICO practices already established, suggest the following regulation
requirements for providers of Facial Biometrics:
Introduction
• Certification Qualification - create a Ground Truth data source with the Turing
Why we want it
Institute that is government approved for companies to certify against
What are they?
• Ethical Code - similar to the ICC/ESOMAR Ethics Code & Guidelines (https://
What’s the difference? iccwbo.org/publication/iccesomar-international-code-on-market-and-social-
research)
Uses Cases
• Explicit Consent - Clear consent and frequent consent checks for ongoing use cases
Risk & Regulation
• Training data separation - meaning that market data cannot be used for training
Contact
data except where explicitly identified and consented by the participant along with
medical use case approvals

Matt Celuszak
matt@elementhuman.com
 CONTACT

Got any questions?

We’d love to hear from you!

Content

About the author

Introduction Matt Celuszak

matt@elementhuman.com
Why we want it

What are they? www.elementhuman.com

What’s the difference?

Uses Cases

Risk & Regulation

Contact

Matt Celuszak
matt@elementhuman.com