Emotion Recognition in Images

Emotion recognition in images
From idea to a model in production

Hagay Lupesko
Sandeep Krishnamurthy
©2018 Amazon Web Services, Inc. or its affiliates, All rights reserved
© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Brief Intro to Deep Learning
Can machines think? AI

Can machines do what we can?
(Turing, 1950) Machine
Learning
Data
Machine
Traditional
Answers
Rules Rules
Answers
Learning
Programming Deep
Learning
Brief Intro to Deep Learning – Artificial Neurons
Inspired by the brain’s Neurons

We have ~100B of them, and ~1Q Synapses
Inputs Weights Non-Linear

Activation
𝑦 =𝜑 ¿
x1 w1
x2 w2 Σ φ 𝑦
…
xn wn
Brief Intro to Deep Learning – Neural Networks
• Non linear
More…
Many
• Hierarchical
feature learning
• Scalable architecture
• Computationally
intensive Input Hidden Output
Layer Layers Layer
Training Neural Networks
Neural
Forward Pass Input Data
Network
Inference
Update Back
Backwards Pass Weights Propagate
Loss
Backwards Pass is where the magic of learning happens, leveraging

Gradient Descent.
Deep Learning is a Big Deal
It has a growing impact on our lives
Autonomous
Personalization Logistics Voice
Vehicles
Deep Learning is a Big Deal
It’s able to do better than humans (and ML)
Facial Emotion Recognition
Misconception - AI First Strategy
• Start with the problem statement!

• Problem -> Is AI necessary -> Use AI
• I want to use AI -> Let me solve this problem with AI -> How
can I solve it now?
Problem Statement
XYZ bank wants to improve the customer experience rating. They recognize
bank customer care representative is one of the main factor to improve the
experience for customers visiting the bank branch offices. XYZ bank decides
to analyze the emotion {Happy, Stressed, Tired …} of its customer facing
employees in different settings {Time of the day, location, before and after
event …} and strategize on –
Happy employees, happy customers!
Stage 1 - Problem Formulation
• Scalable solution is to automate.

• Capture photos of the customer care representatives and analyze the
emotions.
• We want a machine to be able to See, learn and classify.
• A good problem to be solved with AI using Computer Vision (CV)
techniques.
“We want to build a Facial Emotion Recognition model”
Stage 2 – Do I need to build my model?
• State of the Art Research (SOTA)

SOTA
• Is the pre-trained model available in Model Zoo - https://github.com/onnx/models,

Model https://gluon-cv.mxnet.io/model_zoo/index.html, http://gluon-nlp.mxnet.io/scripts/index.html
Zoo
Transf
er
• Extend the pre-trained model – Transfer Learning, Fine Tuning?
Learni
ng /
Fine
Tuning
Build
• Build and Train the model for your problem.
the
Model
SOTA – FER+
• FER+ – Training Deep Networks for Facial Expression Recognition with

Crowd-Sourced Label Distribution by Barsoum et. al.
https://arxiv.org/pdf/1608.01041.pdf
• 8 emotion types: {neutral, happiness, surprise, sadness, anger, disgust,
fear, and contempt}
• Pre-trained model in ONNX model zoo -
https://github.com/onnx/models/tree/master/emotion_ferplus
For this talk, let us implement the FER+ in Apache MXNet GLUON
Apache MXNet - Background
• Apache (incubating) open source project

• Framework for building and training DNNs
• Created by academia (CMU and UW)
• Adopted by AWS as DNN framework of http://mxnet.io
choice, Nov 2016
Apache MXNet - Highlights
• Imperative, Symbolic and Dynamic APIs
Ease of Use • APIs in Python, Scala, C, C++, R (and more)
• Examples and tutorials
• Optimized for CPU, GPU, ARM (and more)

Performance • Highly scalable distributed training
• Quantization, Sparse, NCCL, and more…
• Train on cloud, predict on edge

Portability • Model serving framework
• ONNX support
Model Training Lab
https://github.com/TalkAI/facial-emotion-recognition-gluon
So what does a deployed model looks like?
Model
Mobile
Model Server Internet Desktop
IoT
Performance
Availability
Networking
The Undifferentiated
Monitoring
Heavy Lifting of
Model Serving Model Decoupling
Model Server for
Cross Framework
MXNet
Cross Platform
MXNet Model Server
• Machine learning model server

• Serves MXNet and ONNX models
• Automated HTTP endpoints setup
• Auto-scales to all available CPUs and GPUs
• Pre-built and configured containers
http://modelserver.io
• CLI to package model artifacts for serving
• Open source project under AWS Labs
Model Archive
Trained
Network
Model Model Archive

Signature
Model Export CLI
Custom
Code
Auxiliary
Assets
Back
Serving Our Model
Containerization
• Production-ready orchestration tools: ECS, Docker, Kubernetes

• Easy to scale out
• Robust and scalable images
• Automatically leverages all GPUs and CPUs on host
• Pre-configured images on DockerHub: awsdeeplearningteam/mms_cpu
Back
Containerization
MXNet Model Server
Balancer
Load
MXNet NGINX
MMS Container Pull or Build
MMS
MMSContainer
Container Run MMS
Docker
Container Cluster Image
Deploying Models with Containers
Demo - http://bit.ly/mxnet-fer
Apache MXNet Resources
• http://mxnet.incubator.apache.org/
• Learn Deep Learning with Gluon - https://gluon.mxnet.io/
• GitHub Repo - https://github.com/apache/incubator-mxnet
• Medium: https://medium.com/apache-mxnet
• Twitter: @ApacheMXNet
• Wiki: https://cwiki.apache.org/confluence/display/MXNET
• Subscribe to dev list – dev@mxnet.incubator.apache.org
Apache MXNet Evaluation
• Want to learn more and start using MXNet?

• Our experts can review your use cases
• We will help you with an evaluation of MXNet
• We can walk you through the steps to get to a production ready state
• Partner with AWS!
• Become a customer reference
• Write a blog and receive market and peer recognition
• Customer case study to promote what you’re doing in Deep Learning
• Contact Marcelo Bossi to get started
• Email: mbbossi@amazon.com
• Cell: (650) 796-1781
Try it out, Star and Contribute!
http://mxnet.io
http://modelserver.io

Emotion Recognition in Images - MXNet Meetup SF - Aug 2018

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Emotion Recognition in Images - MXNet Meetup SF - Aug 2018

Uploaded by

Copyright:

Available Formats

From idea to a model in production

Can machines think? AI

Inspired by the brain’s Neurons

Inputs Weights Non-Linear

Backwards Pass is where the magic of learning happens, leveraging

• Start with the problem statement!

• Scalable solution is to automate.

“We want to build a Facial Emotion Recognition model”

• State of the Art Research (SOTA)

• Is the pre-trained model available in Model Zoo - https://github.com/onnx/models,

• FER+ – Training Deep Networks for Facial Expression Recognition with

• Apache (incubating) open source project

choice, Nov 2016

• Optimized for CPU, GPU, ARM (and more)

• Train on cloud, predict on edge

Model Server Internet Desktop

• Machine learning model server

Model Model Archive

• Production-ready orchestration tools: ECS, Docker, Kubernetes

MXNet Model Server

• Want to learn more and start using MXNet?

You might also like