Artificial Intelligence and Marketing

End Term Examination

Revised AI Canvas

Instructor: Prof. Anuj Kapoor

Academic Associate: Ms. Binal Joshi

Submitted on: 25th July, 2023

Submitted by: Anubhav Mehra (PGPGC202200313)

Objec&ve: Introduce a revised AI Canvas Framework

Benefits of AI Canvas by Ajay Agarwal

Before we look at what could be improved in the AI Canvas suggested by Mr. Ajay Agarwal and
his colleagues, it is essenBal to point out what the framework helps with. It provides a well-
organized structure to help businesses assess predicBon machines and integrate them into their
decision-making procedures. It provides a good high-level overview to companies and
individuals who haven’t been exposed to the AI world.

Mo&va&on behind tweaking the model

However, I believe that when a business tries to inculcate arBficial intelligence in its processes, it
has to go beyond just the high-level overview. The above-menBoned seven aspects (which
themselves need more intricacies than those menBoned in the AI Canvas) focus on just the
processes of the AI model and not on the business-related decisions taken before and aLer the
model-related decisions. The AI Canvas should help align the technical resources with the
business objecBves. Hence, a more holisBc overview is needed.

Thought process behind the revised framework

The revised canvas focuses on both the business and the technical aspects. It divides the enBre
process into three phases:
 1) Pre-Implementa&on Phase: It focuses on the iniBal decisions the business has to take
before jumping into the AI Model. Hence, this involves all the preparatory work.
2) During-Implementa&on Phase: The business has done the iniBal setup by this phase.
This is the phase where the actual development and implementaBon of the model take
place. This is where the AI Canvas is situated.
3) Post-Implementa&on Phase: This phase begins aLer the AI model is deployed and
operaBonal.
The revised framework is an extension of the AI Canvas. It adds new secBons with Pre and Post
ImplementaBon Phase. Further, it adds new secBons within the tradiBonal AI Canvas to make it
more detailed.
Value Proposi-on What value does your firm provide to the customers?

Business Driver How will this decision help the overall business?
Pre ImplementaBon Phase

Infrastructure What type of infrastructure will this model be hosted on?

Key Stakeholders Which stakeholders will be affected the most?

Handling Cold Start Problem How will you handle the cold start problem?

What will be the significant cost heads with the new

Costs Categories
system?
Revenue Categories What will be the new streams of revenue from the system?

Predic'on What are you trying to predict with the model?

Judgement: How are different results evaluated?

Judgement & Type of Customer Type of customer: Do these results vary for different types
During ImplementaBon Phase

of customers?
Considera-ons Is Accuracy the sole criteria for the company’s AI model?

Ac'on What steps will be taken aDer the predicEon is generated?

Outcome What will be the metrics used to assess the model?

Input: What data needs to be input into the model?

Input & Data Ownership Data Ownership: Who owns the data? The users, the
government, the company, or any other stakeholder?
Training: What labels will be used to train the model?
Training & Algorithm
Algorithm: What algorithms will be used to train the model?
Feedback How can you use the results to improve the model?
 Post
Imp. Iden-fica-on of Moat In what ways will your AI Model differ from compeEtors?
Phase

ExplanaBon of new aspects in the revised AI Canvas:

Pre-Implementa&on Phase
1) Value Proposi&on: MenBoning a company's value proposiBon is essenBal as the
objecBve the company is trying to achieve through the AI Model should Be back to the
core value that the company provides.
2) Business Driver: The business objecBves the company tries to achieve through the
model. It could be either reducing the churn rate or increasing the LTV of a customer.
3) Infrastructure: Knowing the tech stack helps a firm opBmize hiring, resource planning,
and integraBon with the current systems.
4) Key Stakeholders: IdenBfying the stakeholders helps in requirement gathering, resource
allocaBon, and project buy-in and support.
5) Handling Cold Start Problem: Most firms face this problem with new users, so it’s
essenBal to know how the firm will tackle it.
6) Costs Categories: This refers to the various costs heads that the business will incur with
introducing the new model.
7) Revenue Categories: This AI model's potenBal for generaBng new forms of income.

During Implementa&on Phase

1) Type of Customer: The cost of outcomes can vary with the type of customer. For
example, a loyal customer who has been on the pla\orm for many years is more flexible
to minor errors in predicBon than a new customer who has just been on the pla\orm for
a few months.
2) Considera&ons: What all factors does the company consider important for the system?
Is it just accuracy, or are there other factors involved?
 3) Data Ownership: It is vital to clearly understand who will own the data used to train the
system. It helps the business in addressing the privacy concerns that might arise in the
future.
4) Algorithms: IdenBfying the algorithms that will be used to train the model helps
opBmize the model's performance, its suitability, and the limitaBons associated with the
algorithm.
Post Implementa&on Phase
1) Iden&fica&on of Moat: This is essenBal to idenBfy as most of the compeBtors will
eventually use a similar AI model, and hence, it becomes necessary to look at aspects
that will differenBate your model from the rest.

The "pre, during, and post" framework provides a structured approach to AI project
management, ensuring that the enBre lifecycle of the AI project is well-planned, executed, and
evaluated for conBnuous improvement.

DemonstraBng usage of the revised AI Canvas by using the Recommenda&on System of NeNlix

Ne\lix’s recommendaBon system, known as Ne\lix RecommendaBon Engine (NRE) (Exhibit 2),
is the major source of revenue for the OTT pla\orm. More than 80% of Ne\lix users' content
discoveries are based on the service's recommendaBons. Over 3,000 Btles are sorted through at
once by the engine, which uses 1,300 recommendaBon clusters to determine a user's tastes.
Even the thumbnails and the teasers are customized to the individual’s tastes. So if an individual
likes “strong female leads,” then the teaser of House of Cards will show more of Claire
Underwood rather than Kevin Spacey. Ne\lix generates thumbnails using sBll images taken from
thousands of TV episodes and films. Images are labeled and ranked according to their predicted
click-through rates. Hence, Ne\lix’s recommendaBon system presents a good opportunity to
use a framework that not gives an overview but also goes into intricate details.
 Revised AI Canvas for NETFLIX

Value ProposiBon Providing entertainment in the most convenient way

Business Driver To reduce churn and thereby increase the LTV of the consumer.
Cloud Services: Amazon AWS
Infrastructure DevOps Technology: Jenkins and Github.
Programming Language: Python, NodeJS. Other tools (Exhibit 3)
Pre Implementa,on Phase

Customers
Content Providers: movie studios, prod. Companies etc.
Regulatory AuthoriBes and Data protecBon agencies
Key Stakeholders Employees:
( Data PlaUorm Team, Product team, Content team &
Studio and MarkeEng teams )
Cold Start Problem Initial survey to help determine a user's tastes and preferences.
Personnel costs: Data scienEsts and engineers
Costs Categories Data acquisiBon and storage costs
Infrastructure and cloud services costs
Recurring revenue from a loyal customer
Revenue Categories Reduced customer acquisiBon cost by up to $1B/year.
Reduced costs for opBmized content generaBon.
Predict the content the customer is most likely to be interested in
PredicBon watching to increase user engagement
Judgement: compare the cost of retenEon of a user when not offered
Judgement & Type recommended content with the cost of retenEon when offered
of Customer recommendaEons.
During Implementa,on Phase

Type of customer: ExisEng and New Customers.

It’s not just about accuracy, it’s also about the diversity of content,
ConsideraBons accessibility, and hardware capabiliEes of the subscriber’s device.
AcBon Recommending personalized content to the user
Measuring the effect of recommendaBons viz. EffecBve Catalog Size
Outcome (ECS) and Take-Rate (Appendix 1) (Exhibit 4)
Input: Labels (Appendix 2). Input data is divided into Implicit (behavioral
Input & Data data) and Explicit (say, giving thumbs up).
Ownership Data Ownership: Subscriber owns the data and can also have the right to
delete their data from the plaUorm in the EU.
Uses the quesBons from the survey to determine the content for new
users. Uses the following algorithms for exisEng users:
Training & Algorithm Reinforcement Learning, SARSA(State acBon reward state acBon), User
CollaboraBve, Item CollaboraBve, and Content-based filtering.
Uses click-through rate and user engagement to see if the user was
Feedback saEsfied with the recommended content or not and accordingly updates
the algorithm
Post Extensive content library
IdenBficaBon of Original Content
Imp.
Moat Strong emphasis on A/B tesBng
Phase
Appendices

Appendix 1: Effec&ve Catalog Size(ECS) and Take Rate(TR)

Effec&ve Catalog Size: ECS measures how evenly viewers sample content from Ne\lix's library. It
will be close to 1 if most views come from a single video. If all videos anract the same number
of views, that would be close to the total number of videos in the catalog. Without this, all the
subscribers would be recommended the same content.
Take Rate(TR): the fracBon of recommendaBons offered resulBng in a playing that content by
the user.

Appendix 2:

Exhibits

Exhibit 1: EvoluBon of the personalizaBon approach of Ne\lix

Exhibit 2: Ne\lix RecommendaBon System

Exhibit 3: Tech stack used for the recommendaBon system by Ne\lix
Exhibit 4: The metrics used by Ne\lix for analyzing the outcome

References:

• Szwec, A. (2019, October 20). ML/AI model canvas. Medium.

https://medium.com/@RoboAndy/ml-ai-model-canvas-59ac4c03c0ea
• Maillet, A. (2019, March 12). Introducing the human-centered AI Canvas. Medium.
https://medium.com/@albmllt/introducing-the-human-centered-ai-canvas-a4c9d2fc127e
• Zawadzki, J. (2020, January 6). Introducing the AI project canvas. Medium.
https://towardsdatascience.com/introducing-the-ai-project-canvas-e88e29eb7024
• How Netflix’s Recommendations System Works. Help Center. (n.d.).
https://help.netflix.com/en/node/100639
• Privacy statement. Netflix Help Center. (n.d.). https://help.netflix.com/legal/privacy
• Plummer, L. (2017, August 22). This is how Netflix’s top-secret recommendation system
works. WIRED UK. https://www.wired.co.uk/article/how-do-netflixs-algorithms-work-
machine-learning-helps-to-predict-what-viewers-will-like
• Gomez-Uribe, C. A., & Hunt, N. (2015). The Netflix Recommender System. ACM
Transactions on Management Information Systems, 6(4), 1–19.
https://doi.org/10.1145/2843948
• India, S. (2019, November 5). How does Netflix’s recommendation Engine Work?
Medium. https://medium.com/@springboard_ind/how-netflixs-recommendation-engine-
works-bd1ee381bf81
• Zaichenko, M., & Sozinova, S. (2023, May 26). Tech stacks: What are industry giants
using to power their applications? Custom Software Development Company.
https://maddevs.io/blog/tech-stack-of-prominent-companies/