ChatGPT and Generative AI

Landscape (UW Guest Lecture)

Sanjeev Jagtap
Contents
 Demo
Harry Potter and the Goblet of Fire Chapter Seven
Harry Potter and The Goblet of Fire: Chapter Seven
When asked,
ChatGPT gets
imaginative,
incorrectly making
up the book and
school names aka
“hallucinates”.
We help by providing
the page text and it
correctly picks out
the correct answers.

(Can also be search

results, documents,
web pages, …)
Another
example,
similar mistake
to begin with…
but this time,
we ask it to
“think it
through”….
Still stubbornly
incorrect,
but we persist in
nudging and
instructing…
It finally
gets the
school's
name
correct.
Takeaways
• Semantically, ChatGPT has no idea. It’s just doing Math.
• Factually, it can occasionally make errors.
• It has awesome natural language understanding and generation skills.
• Prompting and grounding with missing knowledge closes the gaps.
• Generative AI models are “jack of all trades”.
• “Collaborate” with them via prompting, finetuning, humans, models.
• Think “Copilots”, “Assistants”, “Wizards”, “Suggestions”…
The Tech
The Problem of Language modeling
Pre-2017, while CNNs (convolutional neural
networks) worked great for images, RNNs
(recurrent neural networks) for language did not.

RNNs were sequential, error-prone, and did not

capture the language model insights and were
therefore did not output natural looking text.
“[Self] Attention is all you need”

• 2017 breakthrough paper titled “Attention is

all you need” introduced the Transformer
model architecture.
• Key idea: Focus on “positional encodings”,
“attention” and “self attention” for the tokens
(integers) representing the words/sub-words.

https://arxiv.org/abs/1706.03762
In plain English

Capture a sense of “nearness” by similarity of

use, numerical distance, etc. and relate to
other words (aka tokens) for ordering and
deep understanding of the language model to
compute the probability of the next token in
sequence. (Its just Math)
Over-simplified How-it-works
• Assign to each unique word a unique identifier, a number that will serve as a
token to represent that word.
• Note the location of every token relative to every other token.
• Using just token and location—determine the probability of it being adjacent to,
or in the vicinity of, every other word.
• Feed these probabilities into a neural network to build a map of relationships.
• Given any string of words as a prompt, use the neural network to predict the
next word (just like AutoCorrect).
• Based on feedback, adjust the internal parameters of the neural network to
improve its performance.
• Extend the prediction to the next phrase, the next clause, the next sentence, the
next paragraph, and so on, by using feedback to further adjust its internal
parameters.
• Based on the above, generate text responses to user questions and prompts
that reviewers agree are appropriate and useful.
Summarizing
• Generative AI language models fundamentally predict
the next words (represented as numerical tokens) in
sequence like auto-complete.
• Words are tokenized into numbers and the model
processes these “tokens” while dealing with context,
nearness, and weights in the neural network and
iterating on them as the model is trained.
 Adoption landscape and trade-offs

Usage
Prompted: Zero or One-shot Prompted: Multiple-shot Finetuned DIY

Scope General/Broad task Narrow task

Composition LLM LLM

+ Specialized Specialized

Lifecycle Synthetic data AI Quality Inferencing in

generation Evaluation production

Iterative benchmarking and regression testing become critical.

The Race
 AI models were getting larger quickly…

AS AI MODELS HAVE GOTTEN

PROGRESSIVELY LARGER THEY
HAVE BEGUN TO SURPASS
MAJOR HUMAN PERFORMANCE
BENCHMARKS. SOURCES: © THE
ECONOMIST NEWSPAPER
LIMITED, LONDON, JUNE 11TH
2022. ALL RIGHTS RESERVED;
SCIENCE.ORG/CONTENT/ARTICLE
/COMPUTERS-ACE-IQ-TESTS-
STILL-MAKE-DUMB-MISTAKES-
CAN-DIFFERENT-TESTS-HELP
https://www.sequoiacap.com/article/
generative-ai-a-creative-new-world/
Recall the Tech Adoption ‘S’ Curves
Transitioning from one S-curve to
another is the tricky part for companies.
The bridges go over choppy waters.

The transitions between innovations can

span months and years. Therefore, you
need bridges in form of hybrid systems,
handovers of technology and
customers, and managing of resources
and investment as the landscape shifts.
The race in a nutshell
Era of ferment Learn (fast)
Dominant Chosen
design dominant Adopt (fast)
Technological emerges design
discontinuity Dominate (fast)

Era of incremental change

Size of
installed
base Cost
Performance
decreasing
Availability of increasing with
with usage
complimentary usage
goods
 First mover(s) pros & cons

Winner(s) take(s) all

1 Winner(s) must deal with:
Brand amplification and loyalty • Research and development expenses
2
Technological leadership • Undeveloped supply and distribution channels
• Immature enabling technologies and complements
3 Preemption of scarce assets
• Uncertainty of customer requirements
Exploitation of buyer switching costs

There is no free lunch. Just strategic tradeoffs.

Which CEO said what (paraphrasing)
“The one that I think is going to have the fastest direct business loop is going to be around Amazon
helping people interact with businesses. I mean, you can imagine a world where over
time every business has an AI agent that basically people can message and interact with
them. … It’s quite human labor intensive for a person to be on the other side of that
interaction.”
Microsoft
“Some of our most successful products were not first to market. They gained momentum
because they solved important user needs and were built on deep technical insights.”

“The most important thing we are doing in AI is trusted and responsible AI. Customers, Facebook
particularly large enterprises, are equal parts intrigued and concerned by the
technology’s potential. We’ve all seen the movies and where this can go. We have all
these crazy ideas in our head of what can happen.”

“Many of today’s A.I. chatbots and other generative A.I. tools are part of the “hype cycle,” Google
while we are focused on the “substance cycle…This is a 10K race while everyone is just 3
steps in.”

“We are making the first moves and if our friends (competition) respond and they will, we Salesforce
want everyone to know we made them move.”
Customers
Most practical customer scenarios
• Written content augmentation and creation: Producing a “draft” output of text
in a desired style and length
• Question answering and discovery: Enabling users to locate answers to input,
based on data and prompt information
• Tone: Text manipulation, to soften language or professionalize text
• Summarization: Offering shortened versions of conversations, articles,
emails and webpages
• Simplification: Breaking down titles, creating outlines and extracting key
content
• Classification of content for specific use cases: Sorting by sentiment, topic,
etc.
• Software coding: Code generation, translation, explanation and verification

https://www.gartner.com/en/topics/generative-ai
Generative AI
Application
Landscape

https://www.sequoiacap.com/article/generative-ai-a-creative-new-world/
 The
Generative AI
Landscape
Generative AI projected timeline view

https://www.sequoiacap.com/article/generative-ai-a-creative-new-world/
Advanced Use Cases by Industry
“Generative AI has
already been used
to design drugs for
various uses within
months, offering
pharma significant
opportunities to
reduce both the
costs and timeline of
drug discovery.”

https://www.gartner.com/en/articles/beyond-chatgpt-the-future-of-generative-ai-for-enterprises
In a recent Gartner webinar
poll of more than 2,500
executives, 38% indicated
that customer experience
and retention is the primary
purpose of their generative
AI investments. This was
followed by revenue growth
(26%), cost optimization
(17%) and business
continuity (7%).

https://www.gartner.com/en/articles/beyond-chatgpt-the-future-of-generative-ai-for-enterprises
Costs: From negligible to many millions
• Free versions of public, openly hosted applications, such as
ChatGPT, or by paying low subscription fees. However, free and
low-cost options come with minimal protection of enterprise data
and associated output risks.

• Larger enterprises and those that desire greater analysis or use of

their own enterprise data with higher levels of security and IP and
privacy. In this instance, costs can be in the millions of dollars.

• Generative AI capabilities will increasingly be built into the

software products you likely use everyday, like Bing, Office 365,
Microsoft 365 Copilot and Google Workspace. Vendors will pass
on costs to customers as part of bundled incremental price
increases. Microsoft recently announced their per-user pricing for
Office 365 CoPilot pricing.
Careers
Product (Managers)
• Learn up the what/how(AI level) to
update product (what/why)
• More OOB model power => emphasis
on user value and experience
• User generated vs. AI generated content
trade-offs
• More collaborative with higher
accountability bar
• Career opportunity to get into AI (Tech)
Engineering professionals
• Vocabulary – Prompting, Finetuning, Turns, …
• Engineering – Systems, frameworks, tooling, metrics
• Frameworks- LangChain, Semantic Kernel
• Prompting – knowledge, tooling (PromptFlow)
• Research – Augment, enhance, build on top
• Open source – Alternatives, crowd-sourced
Business planning
• New underlying economics
• Willingness to pay (WTP) questions
• Pricing models – new units (e.g., tokens)
• Stand-alone vs. bundling with existing
• New and more stakeholders to manage
• Business projections vs. realization
User experience design impact*
Collaborative
AI notices ala Reverse-
not
disclaimers prompt users
command

Address
Citations and Feedback for
costs and
references improvement
speed
https://www.linkedin.com/learning/ux-for-ai-design-practices-for-ai-developers/designing-for-ai
 • Learn about AI quality metrics and lifecycle
• Word Error Rate (WER), FactX, ROUGE-L,
Human evaluations
• Precision, Recall, F1-Score, Ground truth
(reference) data
Software • Learn about development and evaluation
data set creation
Testing/QA • Learn about AI benchmarking and regression
best practices
• Create a GitHub repo, download a public
dataset as your ground truth (reference),
create a test dataset from the internet, GPT,
or your own and publish your findings
Recap career tips
Generative AI levels the playing field between tech and non-tech

Opportunity for entry into the tech and AI space

New disciplines like Prompt Engineering are hot

Responsible AI is an example of new careers to learn up and enter the AI industry

Use experience design principles are evolving to meet challenges

For newbies, learn it up while prioritizing your target niche

For experienced ones, decide how and where you want to play
Responsible AI
Why it is more important today
With Generative AI power also come significantly increased
challenges related to harmful content, manipulation, human-like
behavior, privacy, and more.
The industry is unanimous and for good reasons that responsible
use and AI need to be tied to the hip.
Example planning framework
Pillars Principles Practices
• Policy • Fairness • Operationalization
• Research • Reliability and safety • Advocacy
• Engineering • Privacy and Security • Alliances
• Inclusiveness • Positioning
• Transparency
• Accountability

https://www.microsoft.com/en-us/ai/responsible-ai
Responsible Use Product Guidance (example)
• Harm mitigation • Transparency note
• Introduction
• Identify • Concepts
• Measure • Intended use cases
• Mitigate • Considerations when choosing a use case
• Capabilities
• Operation • Limitations

• Limited access aka “gating” • Code of Conduct

• Application • Access requirements
• Review of use case
• Content requirements
• Other criteria
• Approvals • Mitigations requirements

Azure OpenAI Responsible Use Guidance (example)

Data, privacy, and security
• What data is processed?
• How does the service process data?
• Cover over the wire and in the cloud infrastructure
• Which secure networking options are available
• Regulatory compliance certifications
• Government clouds, Public clouds

Microsoft Azure Compliance Offerings (example)

Content safety as a service

https://azure.microsoft.com/en-us/products/ai-services/ai-content-safety
Takeaways
• Learn up on the latest and what companies and agencies are up to
• Excellent way to get into AI as a career
• Critical regulatory and compliance space that’s poised to grow
• Solving hard problems while working with AI researchers
 More references
• https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/
• https://www.deeplearning.ai/short-courses/building-systems-with-chatgpt/
• https://build.microsoft.com/en-US/sessions/db3f4859-cd30-4445-a0cd-553c3304f8e2
• https://karpathy.medium.com/software-2-0-a64152b37c35
• https://www.linkedin.com/pulse/understanding-chatgpt-triumph-rhetoric-geoffrey-moore
• https://developer.nvidia.com/blog/deep-learning-nutshell-core-concepts/
• https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/
• https://daleonai.com/transformers-explained
• https://a16z.com/2023/05/25/ai-canon/
Thank You!