Personalized marketing using recommendation systems Peter A.

Csikos
10 February 2009

SOME FACTS ABOUT THE HUNGARIAN E-COMMERCE SPACE

Although the total population is 10 million the number of people actually doing anything on the net is low

Population over 14 years or older: 8.515.000

Based on 2005 Microcensus

Number of regular internet users (14 years or older): 3.596.000

Number of people actually using the internet for at least one hour per month (2010 Q1)

Number of people actually doing ecommerce 1.075.000

People who have bought anything over the internet in 2009

Source: GKINET / Hungarian Association for e-commerce

ONLINE SHOPPING AND E-COMMERCE BETWEEN 2001 AND 2010

Ratio of people actually doing e-commerce and those who are planning to do so and those who are even not planning to do it
100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%
2001. III. 2002. III. 2003. III. 2004. III. 2005. III. 2006. III. 2007. III. 2001. IV. 2002. IV. 2003. IV. 2004. IV. 2005. IV. 2006. IV. 2007. IV. 2008. IV. 2009. III. 2002. II. 2003. II. 2004. II. 2005. II. 2006. II. 2007. II. 2008. II. 2009. II. 2002. I. 2003. I. 2004. I. 2005. I. 2006. I. 2007. I. 2009.IV. 2010.I.

1.075.000

Not even planning to Nem tervezi use e-commerce

Tervezi commerce in the

future

Planning to do e-

Vsrolte-commerce Did

Source: Hungarian Association of e-commerce

WHAT ARE THESE PEOPLE ACTUALLY DOING ON THE INTERNET?

Number of people actually doing online shopping last year

online banking Online Banking Doing administrative things gyintzs Gathering information vsrls eltti tjkozds before regular shopping

General Information ltalnos informcik 0% 20% 40% 60% 80% 100%

Gyakran veszi ignybe Regularly

GKINET / Association of e-commerce

Ignybe veszi sometimes

TURNOVER OF ONLINE SHOPS IN HUNGARY BETWEEN 2001 AND 2010

EUR 500million
EUR 350 million

Planned
GKINET / Association of e-commerce

Actual

Key players on the Hungarian ecommerce market

CEMP
Index Velvet Totalcar Inda Portfli Etarget InfoRdi BOOKLINE DVD Rent C-TRAVEL OnGo Honfoglal Mobile24 Burek ArtOffice

Sanoma

[origo]
Origo Freemail IWIW [ORIGO] TKA Videa Meglla-podsok ms szolgltatkkal ORIGOVSRLS

Startlap Hotdog PROFESSION JOBMONITOR Vezess.hu Figyelnet Citromail Videoplayer STARTAPRO Mobilport Wellnesscafe Hrstart

Nk Lapja Caf Geographic Cosmopolitan Storyonline HZIPATIKA.COM csoport TECHNET SPEEDSHOP.HU

HVG (WAZ)
Jobline Ingatlamenedzser Autmenedzser Adzna Ecoline Techline Medizna Eduline URLguru Travelline Aprnet Life & Style Zldtr Hrszerz

Holtzbrinck Networks
Randivonal.hu Brands.hu Habostorta.hu Funpic.hu Gyaloglo.hu Gumicsizma Iz.hu Lira.hu Metnet.hu NB1.hu Minimax

Allegro Hungary
Vatera.hu TeszVesz.hu Arukereso.hu Otomoto.hu

General Media
MyVIP.hu MYCLUB.HU Donna.hu DonnaLight myMAP.hu SportHRAD Olimpia.hu Nascar.hu Data.hu INGATLANOK.HU Hirposta.hu Love.hu TALALKA.HU JOJATEK.HU MultiPlay Chat.hu Mellesleg.hu Farm.Hu Postafiok.hu Fashion.hu Divat.hu BOOK.HU Freeweb.hu

Hrek Mdia
DEPO.HU Egyperces.hu Terminl CTnetwork Hrek.hu JOBINFO.HU Kpzsinfo.hu KutatCentrum Matton Images Maxima Monddmeg.hu Terminal.hu

Viala Kft.
Netpincer.hu Sportolj.hu Klikkmnia.hu Nethaj.hu

Lapcom
HASZNALTAUTO.HU Garancilis.hu SZALONAUT.HU BAZR.HU AUTBAZAR.HU INGATLANBAZAR.HU Workania.hu Delmagyar.hu Kisalfold.hu Hahu.hu

Arkon Zrt.
Ingatlan.com (B2B) Koponyeg.hu Utcakereso.hu Alaprajz.hu Linkcenter.hu Blogol.hu Telepiuleskereso.hu

Extreme Digital
EXTREMEDIGITAL.HU

Ebolt Kft.
EBOLT.HU

GRoby
Groby.hu

TYPICAL HUNGARIAN PROBLEM FOR E-COMMERCE STARTUPS:

GETTING THE DEVELOPERS CLOSE TO THE INDUSTRY

Focus, Focus, Focus Differences in motivation

IT excellence
Market share, market size

freedom of development

fame; IT cups, contests

Publication, open source

New Products

official regulations

ROI, NPV, IRR

Norbert, Buzas, ValDeal / University of Szeged

Business Card Gravity R&D

Domain: Personalized Recommendation Solutions for ECommerce, Digital Media and IPTV/Digital Cable Service Providers Our offering: Delivering real time, personalized context-based product recommendations to each and every user, the Gravity solution produces measurable increase in revenues from existing customer bases and improves customer satisfaction A fast-growing company: significant number of customers and several pilot projects running including ALEXA top35 video live streaming site, 5 to 30 employees within 12 months Investors: US and Hungarian investors, offices in Budapest, London, Utrecht and Berlin, sales representatives in Romania, Russia and Israel and China
8

E-commerce market trends

The problem

10

Market trends
Personal product recommendations are one of the TOP5 most important emerging technology priorities in e-commerce (Internet Retailer, Emerging Technology conducted by Vovici Corp, Sept 2008)
50% of online retail stores want to introduce personal recommendations (Internet Retailer, Emerging Technology conducted by Vovici Corp, Sept 2008)

40% of US e-commerce sites plan to adopt customer reviews and ratings (The Kelsey Group and ConStat Inc, Local commerce monitor, Aug 2008)
77% of retail executives said that online behavior tracking is the most promising website technology for online customer engagement (Retail Systems Research, Playing Well with Others: eCommerces Evolving Role in the Customer Experience, Aug 2008) 76% of US e-commerce executives said the personalized recommendation may or definitely increase loyalty (The Executive Guide to Captivating Customers, June 2008)

Social shopping sites and collaborative technologies are deemed currently as leading shopping innovations and expected to be widespread by 2015 (US household shoppers prediction) (TNS Retail Forward, New Future in Store, May 2008)

11

THE EMERGENCE OF RECOMMENDATIONS

Internet-based shopping and media consumption differs from the classical one
In-store, personal shopping experience Customers know how to find the desired products Assistances help the customer: show them the desired products & recommend Internet shopping experience Web portals offers much more products, customers get lost easily Recommendation systems helps the customer to find & discover products

The key is to keep the customers active & increase conversion

20% of the customers is accountable for 80% of the profits To gain new customers, companies have to spend 3-4X more than to keep the existing ones

Solution: Change the way you personalize

13

WHAT IS A RECOMMENDATION ENGINE?

Recommender systems or recommendation engines form a specific type of information filtering system technique that attempts to present information items (films, television, video on demand, music, books, news, images, web pages, etc.) that are likely of interest to the user.

14

HOW DOES IT WORK?

User registers or starts using services

User browses, watches and optionally rates content

System builds a preference profile

System provides presonalized recommendations

15

WHAT IT DOES?

Recommender logic

Data collection and processing Relevance & preference ordering Display recommendations Self-learning & improving capabilities

Mathematical models Information systematization

16

THE RECOMMENDATIONS
Customer is looking for a product

Receive tips:

Receive personal offerings:

17

SHORT SCIENCE: RECOMMENDATION ALGORITHMS

Recommendation in general: Gravity is using a wide palette of recommendation algorithms The best fitting algorithms are selected after careful analysis of the data to the given recommendation problem and the corresponding optimization task.
Overview of recommendation algorithms: 1. Collaborative filtering (CF): Based on events generated in your service (Vod purchase, Live channel watching event), finds similar behavior on users, and similarity on items (VoD content, live schedule, etc.) 2. Content based-filtering (CBF): Using only user/item metadata. Recommendations are based on matching keywords. Measuring Recommendation Quality: Average Relative Position (ARP): The distance between the prediction and the users choice Top 10 Recall: the probability of hitting the chosen item from the top 10 items of the personalized list

18

Early generation recommendation solutions

Did not offer really personalized recommendations for each and every user
Not personalized

Only based on part of the available information Low customer retention (if any)

Minimal revenue increase Lower conversion rate Increase of customer satisfaction is questionnable 3

THE NETFLIX COMPETITION RECOMMENDATION QUALITY MATTERS

Largest DVD rental company in the US Oct 2006: Netflix Prize Award competition starts
Goal: Improve the recommendation performance of Netflixs in-house developed Cinematch system by 10% Prize: 1.000.000 US$

The benefits of quality recommendations:

Increase customer life-time value 1 months increase results in 50-60 million USD/year average extra income Customers upgrade their plan 5 USD/month/customer extra income USD 80 Million + additional revenue due to quality recommendation

Gravity & The Ensemble Team has been able to improve Netflix recommendation system by 10.1%
20

NEW GENERATIONAL RECOMMENDATION ENGINES: RELEVANT

RECOMMENDATION BASED ON THE ANALYSIS OF ALL SOURCES
Duration of Page Views Order of Page Views Clickstream Searches etc. Product Viewed Product in Catalog Location Brand & Manufacturer Descriptions Ratings etc.

VISITOR BEHAVIOR

PRODUCT DETAILS

HISTORICAL DATA
Geography Visitorss Past Searches Past Shopping Behavior Aggregated Past Behavior etc.

SESSION STATS
Type of URL Page Refer URL Broadband Speed IP Address etc.
21

THE PERFORMANCE BENEFIT OF NEW GENERATIONAL RECOMMENDATION SYSTEMS

*Based on Netflix competition results Even a slight change in recommendation accuracy results a significant increase in business potential!

Increase in revenues +$80,000,000

Increase in accuracy +10%

22

BENEFITS

User

Service provider

Receives personalized services Faster product search Easier product and content discovery, find such products he/she wasnt aware

Higher sales per visit Increased time spent on site value More non-best-selling (long tail) products sold Strenghtened loyalty and stickyness Decreased churn rate

GRAVITY RECO
Gravity Reco is a presonalization solution that continuously tracks and learns user behavior and recommends the most relevant items to the client at any given time. Gravitys recommendations dont just pull matches expected by anyone but our highly sophisticated algorithms dig deeper to delight the customer and to offer products that are more lucrative for the provider.
Gravity Reco
Key features:
Custom tailored to fit your exact needs Real-time No limitations on collecting data Proactive & accurate Fully customizable business & merchandising rules

Benefits:
boosted sales more cross- and up-selling more returning customers better conversion

COMPTETITIVE ADVANTAGES OF GRAVITY

Seamless learning from all available user actions, and information sources

Tailored and personalized recommendations for each individual user real-time reconfiguration & adaptation to user action trends
manages various item classes and provides cross-class recommendations (optimal for companies with various portfolios) recommendations are optimized to meet real customers goals (not optimized to hypothetic behavioral models) best-in-class mixed strategy based collaborative filtering based, and context aware recommendation techniques advanced reporting capabilities
25

CASE STUDY - VATERA.COM

Vatera.com most dynamically growing auction portal in CEE region, member of
Naspers Group (about 10+ auction and media sites in the portfolio) 16 million page impressions of front page per week (in average) 500.000+ actively listed items for bidding & sale

Key objectives of recommendation solution:

increasing the bid value of goods easing the discovery and findings of goods

Results of the pilot:

better banner conversion: 217+ % more clicks on the recommendation banner better product discovery: 293+ % more bids higher hammer price: 524+ % increase in bidding values
26

CASE STUDY - GROBY

Groby.hu the largest online grocery shop in Hungary
~5000 item/day sold ~15000 different items available ~45000 registered users Personalize the customer experience during the browsing session Facilitate discovery of the usually bought items using recommendations Increase cart value Implemented during the re-design of the Groby website, hosted at Gravity HQ Displays 4-8 personalized recommendations on each page ~12000 recommended items/day displayed

Key objective personalize the shop

The recommendation solution provided by Gravity

Results of the pilot:

Over 12% of the income is generated by the recommendation widgets Daily ~7% of the displayed recommendations is successfully converted into a purchase Daily ~50% of the customers purchase a recommended item
27

CASE STUDY - RANDIVONAL PILOT

Randivonal.hu Hungarys leading online dating portal
1.000.000+ registered users (freemium model) 800.000 daily pageviews

Pilot environment
All users are divided into two statistically significant and equal groups (A and B) Group A receives Randivonals own recommendation, Group B receives Gravitys one Goal: improve the discovery, increase the conversion of Personal tips box

200+ % improvement on conversion to control group

30000 25000 20000 15000 10000 5000 0 28

Weekly conversion on "Personal Tips"

Gravity free account Randivonal free account Gravity premium account Randivonal premium account

CASE STUDY DIGITAL MEDIA PROVIDER

Our client has been the 26th most visited website of the world in January, 2010. It is a
constant Alexa TOP 50 website. It is is an adult video chat site with thousands of chat hosts
online in several categories.

Key objectives of the pilot

Increase per show length Increase number of performances Increase conversion rate

The recommendation solution provided by Gravity

24 200 000 recommendations generated /day System has been configured to generate up to 4000 recommendations/sec

Results of the pilot:

Income uplift generated by Gravitys recommendation engine: +9% Registration rate increase: +5%
29

CASE STUDY - INTERACTIVE FOOTBALL PORTAL FOR VODAFONE

Company: Beeweeb is an Italian software design company focused on convergent and mobile solutions. Beeweeb partners with leading technology and system integrators companies and handset manufacturers (RIM, Samsung, NokiaSiemensNetworks, Ericsson, PacketVideo, Vidiator) to offer end-to-end solutions and to distribute its applications worldwide.

Customer: Vodafone Italia

Requirement: in collaboration with Beeweeb, Gravity developed the recommendation engine for a thematic portal (Vodafone calcio). Vodafone Italia wanted to increase customer engagement and provide a unique mobile experience with the personalization feature. Solution: Gravitys recommendation engine provided personalized, football related content that is relevant to the user. It facilitates advanced content discovery of news, media and other content. Recommendations are pulled from different content types and are based on the personal preferences.

30

GRAVITARG CASE STUDY - TARGETED

ADVERTISEMENTS
GraviTarg for E-commerce: Personalized
& targeted marketing campaings of Internet companies to drive sales
using user behavior actions and previous purchase information provides targeted ad messages and combines with personal offerings seamlessly integrates into your ad display delivery platform and into 3rd party systems Before After

Business benefits
Increase ad click-thru rates & conversions Increase your sales and average order size Maximize your media placements with personalized ads Increase the ROI of your ad campaigns

31

CASE STUDY - VELTI PRODUCT DESCRIPTION

Company: Velti is a leading global provider of mobile marketing and advertising technology. Requirement: Behaviour and taste based advertisement targeting Problem: Velti wanted to introduce an updated service, with increasing the average ad impression value. They chose Gravity as the partner for providing personalized advertisements. Solution: Displays only relevant ads Achieve same click-thru & conversion performance with less impression Better monatization of ad display assets Recommends other relevant ads
using user behavior actions and previous purchase information on portfolio sites builds user preference and ad relevance profiles for targeting displays ads with highest user preference relevancy to drive successful ad conversions integrates into existing ad campaign management & reporting environment efficiently serves ad displays on pages with long tail contents ultra-scalable, high performance architecture, easy to integrate APIs
32

CASE STUDY IPTV / CABLE TV

Based on a real-life customer
1M active subscribers 400K active VOD users, purchasing 3,5M VODs per month Revenue generated from VOD purchases: 63,000,000 / year Gross margin on VOD sales: 35%

Gravity IMPRESS (IPTV recommender) increased both the VOD customer base and

the purchase activity, resulting in an additional direct revenue of 2,600,000+ /

year

Offered business models:

a) Revenue share based on income uplift Click here to download the product brochure b) Volume based licensing, based on the number of active subscribers
33

COOPERATION PROCESS

Business aspect

NDA
1 week

Business goals & targets

2 weeks

Agreement
2 weeks

Pilot
1 months

Launch
2 months

Technical aspects

Data gathering

Data analysis

Integration

Fine-tuning

Pilot

34

CASE STUDY INTEGRATION TIMELINE

Business aspects
defining the scope of the pilot project (1 weeks) deciding the business drivers, success factors, and key performance indicators (KPIs) (2 weeks) integrating a recommendation solution into the customers system (2-3 weeks) running the pilot and evaluating the KPIs (4-8 weeks based on the scope of the pilot)

Information requirements for the pilot

detailed technical description of partners system environment detailed description of the use cases how the end user accessing the application meta information about the end users involved in the pilot if available

35

RISK FREE BUSINESS MODEL

Gravity Reco Software as a service: Success fee based on the sales increase generated by the Gravity engine / monthly fee Appliance One-time set up fee (covers system integration) Monthly licensing fee based on performance
no costly upfront license fee only recovery of integration cost then monthly success fee that is proportional to the extra value generated

36

SUMMARY - VALUE PROPOSITION OF

RECOMMENDATION ENGINES
Increased customer lifetime Drive revenue , new users to premium services

Significant ARPU increase

Maximize user engagement, time spent on site, CTR Recommendation system fine-tuned to your business exact needs Customizable business rules Earn incremental revenues from relevant and targeted advertisements and And change the way your customers search
37

38

Can you guarantee increase in your sales? Because Gravity can.

For more information contact our business development team at sales@gravityrd.com

10 February 2009

39

Backup Slides

10 February 2009

22/03/2012

Content based filtering (CBF)

Content based recommendations: recommendations based on metadata catalogues uses EPG, VOD catalog metadata weak personal taste / preference prediction power only those items can be recommended that has a contextual similarity (e.g. titles are similar, actors are same, genre or category is same, etc.) Solaris by Trakovski from 1972 and Solaris by Sorderbergh from 2002 would be linked incorrectly Monster Inc and Shrek would likely not linked, because no overlapping metadata only the genre does not follow trends - users watching behavior changes continuously, especially with live recommendations when habits changes easily (e.g. effect of reality shows / new series) greatly depends on the quality of the metadata (metadata is hard to maintain and expensive) popularity factors, ratings cannot be considered

40

Collaborative filtering (CF)

Collaborative filtering based recommendations: recommendation based on usage activities accurate precision technique users actions do not lie, users preferences are naturally encoded in their actions recommendations are done by clustering people into interest groups based on their consumption items are recommended based on the consumption trends of the clusters no metadata info is needed for recommendation considers popularity factors, ratings, and accommodate to trends the importance of VOD rental and live watching events: Recommending New Movies: Even a Few Ratings Are More Valuable Than Metadata, a scientific paper by Gravity chief data miners (can be downloaded from http://www.gravityrd.com/gravity/download/recsys2009pila_draft.pdf)

41

Recall precision of VOD

Recall precision of VOD tells the likeliness of VOD purchase from the recommended items

42

Scenario & Business Rule UI Demo

43

Scenario & Business Rule UI Demo

44

CUSTOMIZATION OF RECOMMENDATION
IMPRESS contains an advanced business rule & scenario editor
Complex templates can be composed from the following general building blocks:
filter block union block recommender block emphasize block order block blender block

Internal A/B testing capability Time-based recommendation capability Version control & test before launch capability Currently we provide scenario templates on operators request Jan 2011: user-friendly UI

45

GRAVITARG EXAMPLE
Normal ad serving approach
Display ads without relevance Wasted ad impressions Inefficient click-thru & conversion performance Ad impressions: 16, Relevant ads: 8, CTR: 3

GraviTarg approach
Display only relevant ads Achieve same click-thru & conversion performance with fewer impressions Better monatization of ad display assets Recommends other relevant ads Ad impressions: 8, Relevant ads: 8, CTR: 3 Displayed ad: brown Relevant ad: Irrelevant ad: Clicked ad:
46

GRAVITARG - EASY INTEGRATION TO EXISTING

AD MANAGERS

More value for the money

AD request
47

Targeted AD