Product Recommendation Beyond Collaborative

Filtering - welcome to the Twilight Zone!

Olivier Koch, Machine Learning Lead

o.koch@criteo.com
@olivkoch
RecSys Meetup London - Sept 20, 2017
Copyright © 2015 Criteo
Outline

• What we do

• Lessons learned from building collaborative filtering at scale

• Now what?

Copyright © 2015 Criteo

What we do

• Buy ad space on publishers’ websites.

• Build banners showing products that users will like / want to buy.

• Get paid if users click / buy the product.

Copyright © 2015 Criteo

 3 billion ads/day
5 billion products
100 ms
 15 data centers
24 000 servers
2000-node hadoop cluster
Copyright © 2015 Criteo
Machine learning is at the core of Criteo

• How much should we bid?

• What look & feel should we use?

• Which products should we recommend?

Copyright © 2015 Criteo

Recommending products from user timelines

User
= ?

Clicks / Sales / Views / Baskets events (on advertisers’ websites)

Click / no-click / attributed sales events (on publishers’ websites)

… for billions of users and products

Copyright © 2015 Criteo

Bob saw orange shoes

Some candidate products

Historical

Most viewed

Similar

Complementary
Lessons learned from building
a CF system at scale
Start simple (counters)

Expect a long tail

Randomize

Yes, your Spark jobs will break

Log everything (and more)

Revisit your evaluation metrics

Revisit your features

Check your attribution model

Beating CF is really hard

CF is great…

intuitive

simple to implement

scales relatively well

captures many implicit signals via wisdom of the crowd

But CF has limitations too…

does not scale that well actually (quadratic in user timelines)

does not capture temporal signals

does not solve cold start

does not address exploration in the long tail

Deep learning to the rescue

Fusing image, text and CF (Content2vec)

Specializing Joint Representations for the task of Product Recommendation, Thomas Nedelec,
Elena Smirnova, Flavian Vasile, RecSys 2017 DL Workshop, arXiv:1706.07625

Contextual RNNs

Contextual Sequence Modeling for Recommendation with Recurrent Neural Networks, Elena
Smirnova, Flavian Vasile, RecSys 2017 DL Workshop, arXiv:1706.07684
Deep learning to the rescue

Hierarchical recurrent neural networks

Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks by

Massimo Quadrana, Alexandros Karatzoglou, Balázs Hidasi, Paolo Cremonesi, RecSys 2017

Compressed embeddings

Getting Deep Recommenders Fit: Bloom Embeddings for Sparse Binary Input/Output Networks
by Joan Serrà and Alexandros Karatzoglou, RecSys 2017

3D Convolutional Networks for Session-based Recommendation with Content Features, Trinh

Xuan Tuan and Tu Minh Phuong, RecSys 2017
Deep learning to the rescue

Can we build neural network architectures that will make our recommendations
more relevant?

Can we leverage temporal information and product metadata?

At scale.
Attribution and incrementality

The true objective of recommendation is to predict and show ads that cause new
(incremental) sales

Large-scale Validation of Counterfactual Learning Methods: A Test-Bed. Damien Lefortier,

Xiaotao Gu, Adith Swaminathan, Thorsten Joachims and Maarten de Rijke, arXiv:1612.00367,
NIPS What If 2016

http://research.criteo.com/dataset-release-evaluation-counterfactual-algorithms/
 Join us!

We are reverse-engineering the brain

of 1B+ shoppers worldwide!

http://labs.criteo.com/rd-jobs/

Copyright © 2014 Criteo