Generative AI at SAP

Unit 1: Approaches to Artificial Intelligence

Generative AI at SAP
Course Overview and Learning Objectives

How it works:
▪ Course consists of 5 units
▪ Course assignment
▪ Discussion forum

What you will learn:

1. What the different approaches to artificial intelligence are,
and why probabilistic models extend the reach of machines
2. What makes generative AI different and what its capabilities
are
3. What the limitations of generative AI are, and what methods
there are to adapt generative AI to business context
4. How to build and extend SAP applications with generative AI
5. What some of the generative AI business use cases are,
and SAP’s product portfolio

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

Approaches to artificial intelligence
Objectives of Unit 1

In Unit 1, you will learn

▪ Definitions of “intelligence” and “artificial intelligence”
▪ Humanity’s approaches to artificial intelligence
– Symbolic AI
– Statistical AI
▪ The relation between AI, machine learning, deep learning,
and generative AI

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3

Approaches to artificial intelligence
What is “intelligence”?

Intelligence: the ability to accomplish complex goals.

Max Tegmark, Life 3.0: Being Human in the Age of Artificial Intelligence

The ability to see patterns and predict outcomes based on

previous experiences.
Dr. Jeff Hawkins, Neuroscientist

That quality that enables an entity to function appropriately

and with foresight in its environment.
Intelligence allows humans to understand and generate language,
perceive and respond to sensory outputs, play challenging games,
synthesize and summarize information, and create (art, music.
theorems…).
Source: Wikimedia Commons. Dr. Nils J. Nilsson, Stanford, one of founders of AI research
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4
Approaches to artificial intelligence
What is “artificial intelligence”?

Artificial intelligence (AI): intelligence exhibited by non-biological systems.

Neuron on substrate of biological Digital neural network on substrate of

tissue. silicon chips.

Narrow AI: ability to accomplish a narrow set of goals, aka

“weak AI”. e.g. a model that excels at classifying cat
pictures cannot play online chess, and vice-versa.
Artificial general intelligence (AGI): ability to understand
and learn any intellectual task that a human being can, aka
“strong AI”.*
* recent approaches show “sparks of AGI”* by generalizing across goals, but experts
disagree when or even if AGI will ever be achieved.
Source: Photo by David Hanson. Wikimedia Commons. * [2303.12712] Sparks of Artificial General Intelligence: Early experiments with GPT-4 (arxiv.org)

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5

Approaches to artificial intelligence
Approaches to AI

Symbolic AI: explicitly represent knowledge e.g. through rules, expert systems, and deterministic
programming

false
Condition

true

Code 1 Code 2

If (and/or expression) {[Then]}

Else { } ...

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 6

Approaches to artificial intelligence
Approaches to AI

Thought experiment: how many rules would you need to describe something as simple as
identifying a picture of a cat vs. a picture of a dog in a computer?

“I shall reconsider human knowledge by starting

from the fact that we can know more than we
can tell.”

Michael Polanyi, on the nature of explicit knowledge vs. tacit

knowledge

We quickly run into limits describing the complex rules and

relationships between inputs and outcomes, i.e. trying to
codify knowledge explicitly based on our tacit knowledge.
Source: Wikimedia Commons.

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 7

Approaches to artificial intelligence
Approaches to AI

Symbolic AI: explicitly represent Probabilistic machine learning: learn from

knowledge e.g. through rules, expert examples in the data without being explicitly
systems, and deterministic programming programmed

Training = “cat”
Data
false
Condition (examples) = “dog”

Training
true

= 𝜃𝑗
Machine 1
𝑚

Code 1 Code 2 Learning − 𝛼 ෍൫ℎ𝜃 𝑥 𝑖

−𝑦 𝑖 )
𝑚
𝑖=1
∙ 𝑥 (𝑖) ൯

If (and/or expression) New = cat!

{[Then]} Model Prediction
Data (conf: 0.972)
Else { } ...
Inference
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 8
Approaches to artificial intelligence
Statistical approaches to AI

Supervised learning Unsupervised learning Reinforcement learning

▪ Train the model with labeled examples ▪ Data set is unlabeled – model ▪ Agents observe the environment,
(input) and the solution (output) finds patterns/structures perform actions, and are rewarded or
▪ Output: regression, classification ▪ Output: clustering, dimensionality penalized
▪ Example: ticket classification reduction ▪ Do not confuse this with updating a
▪ Example: fraud detection supervised learning model!
▪ Output: agent learns a policy (what
𝑥2 𝑥2 actions to best take given a situation)
▪ Example: AlphaGo
Reward
k=2
Action
𝑥1 𝑥1 Agent Environment

Observation (state)
Self-supervised learning
▪ Neural networks where the “label examples” are given in the structure
of the data itself, e.g. large language models predict the next word
▪ Basis of foundation models and generative AI. We will explore more in Unit 2.
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 9
Approaches to artificial intelligence
A note on statistical approaches to AI, probability, and uncertainty

Although all knowledge has some

uncertainty, many business people are
not comfortable with the probabilistic
nature of AI.

What did you see? And how certain

are you on a scale of 1-10?

Concert… or cotton harvest?

As amazing as human intelligence is,
people are far from 100% accurate.
A computer can learn to execute tasks…
often faster and more accurately than
people, but usually not with 100%
accuracy.
Humans + AI working together Source: BBC. 9. November 2017. “This picture has gone viral, but it’s not what you think it is.”

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 10

Approaches to artificial intelligence
Summary of approaches to AI

Intelligence: the ability to achieve complex goals

Artificial Intelligence (AI)

▪ Intelligence exhibited by machines
▪ Includes many approaches, like symbolic AI all the way to neuromorphic computing

Machine Learning (ML)

▪ Computers learn from examples in the data without being explicitly programmed
▪ Numerical and statistical approaches to train a model, including many kinds of algorithms
▪ Supervised learning (including self-supervised learning), unsupervised learning, reinforcement learning

Deep Learning
▪ A subfield of machine learning that uses specialized computational techniques, namely various multi-
layer artificial neural network architectures (e.g. RNNs, CNNs, transformers…)

Foundation models and generative AI

▪ Neural networks, typically using the transformer architecture,
trained via a self-supervised learning objective
▪ Exhibit emergent properties, generalize across tasks, and can
generate new content
Focus of this course

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 11

Thank you.
Contact information:

open@sap.com
Follow all of SAP

www.sap.com/contactsap

© 2023 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of
SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its
distributors contain proprietary software components of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or
warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials.
The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty
statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional
warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or
any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation,
and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platforms, directions, and
functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason
without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or
functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ
materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and they
should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names
mentioned are the trademarks of their respective companies.
See www.sap.com/trademark for additional trademark information and notices.
Generative AI at SAP
Unit 2: Introduction to Generative AI
Introduction to generative AI
Objectives of Unit 2

In Unit 2, you will learn

▪ What’s behind all the generative AI news and its
impact on the economy
▪ Definitions of foundation models, large language
models, and generative AI
▪ Why generative AI is a new approach to artificial
intelligence
▪ Examples of generative AI

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

Introduction to generative AI
Generative AI is all over the news

ChatGPT Is a Tipping Point for AI

Source: Harvard Business Review, 14. December 2022

Congress to hold new AI hearings

as it works to craft safeguards
Source: Reuters, 8. September 2023

‘Video games are in for quite a trip’: How

generative AI could radically reshape gaming
Source: CNN, 23. October 2023

The Coming AI Economic Revolution

Source: Foreign Affairs, November/December 2023

How generative AI could add trillions to SAP prepares to add Joule

the global economy generative AI copilot across its apps
Source: World Economic Forum, 14. July 2023 Source: CIO, 26. September 2023

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3

Introduction to generative AI
Generative AI will profoundly impact the economy and increase productivity

Generative AI could add

$2.6 to $4.4 trillion

incremental value annually to
the global economy Already widespread adoption of generative AI

of companies globally are using

33% generative AI regularly in at
least one business function

of organizations will increase their

$11-$17tn 40% investment in AI overall because
of advances in generative AI
productivity value from existing
generation of narrow AI by 2040
of organizations adopting AI have
60% seen some revenue increase

Source: McKinsey, State of AI in 2023, 1,684 participants globally

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC
Source: McKinsey, The economic potential of generative AI: 4
The next productivity frontier (June 2023)
Introduction to generative AI
Definitions

Foundation models are neural networks

Foundation trained on large volumes of data using self-
LLM learning objective:
Models supervised learning that can be applied to
Predict the next word in text.
many tasks.

Large England
London is in _______
Large language models (LLMs)
Language
are a subcategory of foundation Two hours south of Toronto, London is
Models (LLMs)
models for text incl. computer code. Ontario
in _______

Generative AI can create novel output in … trained on billions of words, images,

OpenAI’s text, images, sound, or video based on lines of code, etc.
ChatGPT simple user input (called “prompts”).

ChatGPT is an application built on a large

language model created by the vendor
OpenAI. The model has been optimized for
dialog.
Generative AI
Multi-modal models can process prompts and
generate output in various formats, including text,
images, video, and speech.

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5

Introduction to generative AI
Capabilities of foundation models

Example
Capabilities
Characteristics
Data ▪ Huge models, often measured in billions of
Classification,
parameters and trained on billions/trillions of tokens
entity extraction
▪ Stateless: respond only to data sent to the model,
Text but do not “remember” or “learn” from previous
interactions
Summarization
▪ Performance and capabilities, in general, scale as
Image models increase in size

TRAINING Search and Q&A

New approach to AI
Speech
▪ From supervised to self-supervised learning
Foundation Content creation
models ▪ From predefined to emerging capabilities
(images, music,
… ▪ From single-purpose to multi-scenario AI video…)
▪ From classifying AI to generative AI
Users can easily interact with the model without Code generation
requiring prior technical knowledge.

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 6

Introduction to generative AI
Foundation models come in all sizes and generally scale with more data and parameters

Trainable Parameters (in billions): Example LLMs by parameter size

1,200

175 1,000

800

Attention-based transformer
600
as neural network
architecture (2017)
400

200
1.6
0.1
0
GPT (2018) GPT-2 XL GPT-3 (2020) Mistral Llama Falcon Llama GPT-3 BLOOM Lum- PaLM Mega- GPT-
(2019) 7B 2 13B 40B 2 70B inous tron 4?

Foundation models scale with more compute, more data, and more parameters (model size)
→ Higher accuracy, adaptability, and generative performance.

But also a trend towards smaller models benchmarking on certain tasks as well as much larger models due
to better training data and methods.
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 7
Introduction to generative AI
Examples of generative AI

Generative AI processes prompts Stable Diffusion 2 (text-to-image multimodal model)

Prompt: explicit instructions or cues to guide
“Hasso Plattner, the founder of SAP,
generative AI models in producing desired
giving a high five to a robot.”
outputs Prompt

Output:

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 8

Introduction to generative AI
Examples of generative AI

Large Language Models: create text. Large Language Models: summarize text.

Write an email introducing our logistics Summarize key benefits from the following
company, Treasure Island Transports Ltd, to 1000-word document in less than 100
a retail prospect. Do it the style of a pirate. Prompt words. Use bullet points.
Prompt

Ahoy there, ▪ SAP Business AI offers transformative intelligence to

businesses through ready-to-use AI capabilities in all
We be Treasure Island Transports Ltd, masters of the seven
business processes such as lead-to-cash, design-to-
seas! Our fleet be as swift as the wind, and our crew as
operate, recruit-to-retire, and source-to-pay.
reliable as the North Star. We be ready to set sail with your
goods, ensuring they reach port safe and sound. So, hoist ▪ SAP Business AI allows businesses to extend AI
the anchor and join us for a voyage of smooth logistics and capabilities to meet their specific needs using the SAP
swift deliveries. Business Technology Platform.
Yarrrs sincerely, ▪ SAP Business AI provides trust and reliability through AI
capabilities built on stringent ethics policy and data privacy
[Your Name]
standards that enable responsible use of AI with full
Treasure Island Transports Ltd. transparency and compliance.

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 9

Introduction to generative AI
Examples of generative AI

Large Language Models: classify and extract text. Write working code based on a text description

Classify the sentence as negative, neutral,

or positive sentiment and extract the object.
Prompt “I loved your course on openSAP!”

Positive sentiment
The object being referred to in the sentence is “your course
on openSAP”.
Prompt
Code

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 10

Thank you.
Contact information:

open@sap.com
Follow all of SAP

www.sap.com/contactsap

© 2023 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of
SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its
distributors contain proprietary software components of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or
warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials.
The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty
statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional
warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or
any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation,
and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platforms, directions, and
functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason
without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or
functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ
materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and they
should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names
mentioned are the trademarks of their respective companies.
See www.sap.com/trademark for additional trademark information and notices.
Generative AI at SAP
Unit 3: Adapting Generative AI to Business Context
Adapting generative AI to business context
Objectives of Unit 3

In Unit 3, you will learn about

▪ The limitations of generative AI models
▪ Methods to address limitations by grounding
and adapting generative AI to business context
▪ Best practices when adapting generative AI

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

Adapting generative AI to business context
Limitations of generative AI

Generative AI’s ability to answer broad, general knowledge problems is remarkable!

Such successes may give the impression that generative AI can tackle any problem out of the box.
Users must be aware of current limitations, keep expectations in check, and understand how to make
generative AI enterprise-ready to handle business problems.
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3
Adapting generative AI to business context
Example limitations of generative AI

Hallucination. Large language models can generate plausible-sounding yet false answers.
Related: LLMs can provide different “facts” depending on the language in which the prompt was given.

“… six legal cases listed to support their March 1

written arguments did not exist.”
“… we made a good faith mistake in failing to
believe that a piece of technology could be
making up cases out of whole cloth.”

Source: arXiv 2307.10169. Challenges and Applications of Large Language Models.

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4

Adapting generative AI to business context
Example limitations of generative AI

Up-to-date and specific knowledge. The knowledge of a generative AI model is frozen in time from when it
was trained. In contrast to world general knowledge, business information changes quickly.

Who won the NBA preseason game last What libraries and SDKs are available
night between the LA Lakers and Golden with SAP AI Core?
Prompt State Warriors? Prompt
As of my training data until September
I’m sorry for any inconvenience, but as an 2021, SAP AI Core does not provide any
AI last trained on data up to September specific libraries or SDKs. Instead, it Output
2021, I don’t have the ability to access Output integrates with various Python libraries
real-time information or updates, (model 1) and ML frameworks used for machine
including recent sports scores. learning and data science, such as
TensorFlow, PyTorch, Scikit-Learn, etc.
The Golden State Warriors defeated the Ground truth:
Los Angeles Lakers 114-101 in their SAP AI Core, launched October 12th 2021,
preseason game last night. Output provides SAP AI Core SDK, AP API Client
(model 2 – SDK, and Metaflow Python Library (SAP AI
hallucination) Core Product Documentation, 2023-09-01)
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5
Adapting generative AI to business context
Example limitations of generative AI

Inconsistent math abilities and limited notion of time. Although improving, large language models are not
calculators. Other kinds of foundation models could conceivably complete forecasts and math.

Jane was older when she was elected

Jane was elected class president in 1973 class president. In 1973, Jane was 12
when she was 12 years old. years old, while in 2012, Jill was 13 years
Prompt Output
Her daughter, Jill, was elected class president old. (model 1)
in 2012 when she was 13 years old.
Jane was elected class president in 1973
Who was older when she was elected class
when she was 12 years old, and her
president, Jane or Jill?
daughter, Jill, was elected class president Output
in 2012 when she was 13 years old. (model 2)
Correct: When elected, Jill was 13 and Jane was 12. Therefore, Jane was older when she
Therefore, Jill was older. was elected class president.
* Experiment conducted October 2023 using three different
leading LLMs. Same result when the years were removed. Jane was older when she was elected
class president, because she was 12
Note: Various prompt engineering techniques could guide
years old in 1973, and Jill was 13 years Output
the model to provide the correct answer.
old in 2012. (model 3)
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 6
Adapting generative AI to business context
Example limitations of generative AI

… even though the capabilities of generative AI to solve and explain some math problems are impressive!
Techniques like calling functions and chain-of-thought prompting can increase performance on math problems.

Solve for X, show your work, and list the This is a quadratic equation, and we can solve
answer: it by using the quadratic formula which is
Prompt x = [-b ± sqrt(b^2 - 4ac)] / (2a).
X^2 + 3x -5 = 0 Output
Here, a = 1, b = 3, and c = -5. (GPT-4)
Let’s plug these values into the formula:
x = [-3 ± sqrt((3)^2 - 4*1*(-5))] / (2*1)
Note: x = [-3 ± sqrt(9 + 20)] / 2
Checking the x = [-3 ± sqrt(29)] / 2
output, the LLM got
So, the solutions are:
the correct answer!
x = (-3 + sqrt(29)) / 2
x = (-3 - sqrt(29)) / 2

These are the exact solutions. If you want to

approximate to decimal places:
x ≈ 1.1925824
x ≈ -4.1925824
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 7
Adapting generative AI to business context
Methods to ground and adapt generative AI to business context

Grounding & Adaptation

Data
Task-specific instructions Re-training
Prompt Engineering Fine-Tuning
Text
Provide more information to describe the Improve performance for
task. Zero-shot and few-shot learning. domain-specific tasks with
labeled data.
Image Retrieval Augmented Generation Includes supervised task-
(RAG) specific and instruction tuning.
TRAINING
Extend to external domain knowledge by
Speech retrieving and injecting information via
Foundation models embeddings, knowledge graph, and Reinforcement Learning with
search. Can provide references. Human Feedback (RLHF)
General purpose.
… Update model by incorporating
With limitations. Orchestration Tools human preference.
Agents, functions, plug-ins, prompt and Improve the quality and adjust
model chaining, memory. weights after initial training.

Processes & Governance

Continuous
Human-in-the-loop Output validation
AI ethics Testing, red-teaming feedback and …
Humans + AI and cross-checks
monitoring

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 8

Adapting generative AI to business context
Prompt engineering example to create a job description

Prompting: Give the model detailed information to reliably produce the desired output.

Zero-shot learning Instruction following In-context learning including examples

Prompt 1: Prompt 2: Prompt 3:
Write a job description for a Write a job description for a Create engaging job
support engineer. support engineer. Keep it less descriptions for company
than 300 words. ACME. Location is Chicago User input
and start date is January.
Locate the job in Chicago.
Include the following skills: { “job_title”: “support engineer”, System prompt
1) … “required_skills”: “...”} with content

Ensure diversity and fairness.

Use subtitles for required
skills and for competencies. System prompt
List skills and competencies with instructions
as bulleted lists.

Number of tokens in the context window sent to the LLM increases

Use the same style based on
(and along with it cost and response time). examples of ACME job postings:
System prompt
… with examples
More information and consistent input result in more precise output.
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 9
Adapting generative AI to business context
Prompt engineering example with SAP business documents

Extract information from business documents with large language models

Text with preserved 2D structure Structured result

{
"DeliveryNoteNo": "70500",
"DocumentDate": "2023-03-07",
Original input image "PurchaseDoc": "G5BPLGK",
"SupplierID": "770570-3O",
”Items": [
{
"ItemNo": "2",
"ProductId": "7303500",
"Quantity": "4",
OCR LLM "UnitOfMeasure": "Stück"
},
{
"ItemNo": "3",
+ Append instructions/examples to "ProductId": "7303506",
the prompt (“meta prompt”) "Quantity": "1",
Extract the following fields as
"UnitOfMeasure": "Stück"
JSON: }]
- DeliveryNoteNo: ID for the delivery note }
- DocumentDate: Date of the document
- PurchaseDoc: Reference to purchase doc
- SupplierID: Supplier's unique ID Output validation
- Items: List of products
Supplier ID XXXXXX-XX incorrect.
- ItemNo: Number for the item in the list
Review proposed supplier ID match
- ...:
from master data: XXXXXY-XX
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 10
Adapting generative AI to business context
Retrieval augmented generation (RAG) and embeddings example

First: what are embeddings? Business Knowledge

Configure business
process by opening…
Embeddings: numerical representations of information that retain
semantic, contextual meaning (text, images, etc.)
LLM
Example: embed

▪ Encode product documentation as embeddings using machine

learning (Word2Vec, SBERT…). Store in vector database. [0.934, 0.351, 0.003, 0.116, …]

▪ “Configure business process by opening… ” is represented as a Embeddings

vector [0.934, 0.351, 0.003, 0.116, …]

The vector captures the semantic meaning in the text, e.g. “Apple”
has a different vector in “Apple makes phones” vs. “I have an
Apple iPhone 15” vs. “Apple is an ingredient in pies”.

Business data represented as embeddings can be easily searched

and retrieved using techniques like vector similarity scoring.

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 11

Adapting generative AI to business context
Retrieval augmented generation (RAG) and embeddings example

You have a set of business documents that should be used

to answer user questions Business Knowledge

Answering business-specific questions requires two steps:

1. Embed your business knowledge to retrieve relevant items
LLM
given a question, using LLMs and other foundation models. embed
2. Generate the answer from the best results using LLMs. Use
prompt engineering as needed.

Embeddings

“How can I adapt my business process for VAT LLM Retrieve

calculation?” embed relevant results*
[0.934, 0353, 0.906, 0.163, …] Inject into prompt.
“Create response
“You need to configure your business based on this text:”
process and add a step by…” LLM
generate answer
based on search
results
Using techniques like embeddings 1) grounds the prompt with relevant
information and 2) references source information to generate the response.
* Possible to use classical keyword-
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 12
based search as an alternative
Adapting generative AI to business context
Orchestration tools (“Agents”)

Orchestration example (experimental): Give an LLM access to tools. An agent retrieves API specifications from SAP
systems, develops plan, and extracts data via API calls to formulate a response.
Example

Task/Question: User Inputs a Task

LLM 1. Retrieve
SAP systems and their API specifications are retrieved

Autonomous Agent
based on the input (powered by knowledge graphs)
2. Plan
Task or
Answer The LLM generates a plan based on the API retrieved
question
specifications
Tools
3. Execute
The LLM follows the plan and generates API calls and
Autonomous agent reacts to the system responses

Answer: The final result is returned

▪ Use existing systems via API calls.

▪ LLM has no prior knowledge of the system. Knowledge is
provided as part of the prompt.
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 13
Adapting generative AI to business context
Fine-tuning

Fine-tuning: adjusting an existing foundation model’s parameters (i.e. weights) to perform better at a
specific task by retraining it on a new data set.

Data

Text
New Data
Improved
Curated
INITIAL domain- and/or
performance on
Image
TRAINING task-specific domain- and task-
data. Labeled specific problems
Pretrained with examples Fine-Tuned
Speech Foundation Model of desired input Foundation
and output Model

…
Fine-tuning process

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 14

Adapting generative AI to business context
Fine-tuning

Considerations for fine-tuning foundation models

Opportunities Challenges
▪ Strong potential to create task-specific models with ▪ May reduce but will not eliminate problems like
significant boost in accuracy. hallucination.
▪ Continued training on domain- or industry-specific data ▪ May not “forget” existing biases and information.
improves quality for downstream tasks in an area.
▪ Does not reliably learn new factual information.
▪ Smaller to medium-sized (~5-70 billion parameter)
▪ Requires sufficient (curated) training data.
LLMs can reach or exceed performance of more costly
massive models. ▪ Can be computationally expensive and costly.
▪ Currently difficult to generalize the volume of
training data needed to achieve meaningful results.

Consider and test task-specific instructions (prompt engineering) first before investing in fine-tuning models.

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 15

Adapting generative AI to business context
Recommendations for adaptation of foundation models

DO NOT DO
 Trust generic generative AI ✓ Be aware of limitations. Ground
models to answer factual and adapt generative AI models on
questions correctly, esp. in business data to ensure output is
a business context. relevant and reliable.

 Start with fine-tuning. ✓ Start with task-specific instructions

like prompt engineering and RAG.

 Use the biggest model by ✓ Test and adapt different generative

default. AI models and optimize based on
price and performance.

 Productize a technically ✓ Ensure proper governance and

functional generative AI use design for generative AI (e.g.
case without established human in the loop, output
supporting processes. validation, monitoring…).

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 16

Thank you.
Contact information:

open@sap.com
Follow all of SAP

www.sap.com/contactsap

© 2023 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of
SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its
distributors contain proprietary software components of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or
warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials.
The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty
statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional
warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or
any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation,
and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platforms, directions, and
functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason
without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or
functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ
materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and they
should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names
mentioned are the trademarks of their respective companies.
See www.sap.com/trademark for additional trademark information and notices.
Generative AI at SAP
Unit 4: Extending SAP Applications with
Generative AI
Extending SAP applications with generative AI
Objectives of Unit 4

In Unit 4, you will learn about

▪ The SAP Business AI approach
▪ Generative AI Hub and its capabilities
▪ Example of Generative AI Hub
▪ SAP BTP reference architecture for generative AI

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

Extending SAP applications with generative AI
SAP Business AI approach

Finance | Supply Chain | HR | Procurement | Marketing and Commerce | Sales and Services | IT

Human capital Spend management Customer relationship Business Technology

Cloud ERP
management and business network management Platform

AI ecosystem partnerships and investments

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3

Extending SAP applications with generative AI
AI foundation

Applications & Extensions

AI Foundation
on Business Technology Platform

AI Services

Foundation Models

SAP built Partner built

Lifecycle Management
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4
Extending SAP applications with generative AI
AI Foundation

AI Foundation
on Business Technology Platform

AI Services
Document Processing Recommendation Machine Translation

Toolset Trust & Control Access Training Inference

Vector Engine

Data Management

Foundation Models

SAP built Hosted Remote Fine-tuned

Lifecycle Management
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5
Extending SAP applications with generative AI
Generative AI Hub

Toolset Access Trust & Control

Streamline generative AI Accelerate outcomes with instant Supercharge mission-critical
development with purpose-built access to top-rated foundation processes while preserving security
tools and services models from multiple providers and privacy

Unlock generative AI for your business

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 6
Extending SAP applications with generative AI
Generative AI Hub

Joule Applications & Extensions

Generative AI Hub

SAP AI Playground Libraries & SDKs

Toolset
Launchpad Prompt Engineering Prompt Management Prompt Registry*

Inference Engine Agents* Multi-tenancy Security

Content Moderation* Grounding Plug-ins* Data Privacy

Foundation AVAILABLE UPCOMING

• Microsoft Azure OpenAI • Aleph Alpha
Model SAP built* Custom Partner built • Falcon • Meta Lama2
Access • and more…

• Embeddings
Vector Engine Application data
• Similarity functions

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC * coming soon 7
Extending SAP applications with generative AI
Generative AI Hub: Toolset

Playground
Generative AI Hub

SAP built*

Client
API Custom

Partner built
Libraries & SDK

*planned

Explore Experiment Enhance

different models and different with a prompt engineering your applications with
capabilities of generative AI playground generative AI
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 8
Extending SAP applications with generative AI
Generative AI Hub: Access

Generative AI Hub SAP built*

Custom

AVAILABLE UPCOMING
• Microsoft Azure • Aleph Alpha
Partner built OpenAI • Meta Llama2
• Falcon • and more…

*planned

Connect Accelerate Switch

easily and seamlessly to any adoption thanks to SAP-managed easily between models to find and upgrade
supported LLM model legal and commercial framework to the best suited technology for your needs
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 9
Extending SAP applications with generative AI
Generative AI Hub: Trust & Control

Generative AI Hub

*planned

Secure Control Extend

your applications with native models response with your solutions with advanced
and embedded SAP security grounding capabilities
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 10
Extending SAP applications with generative AI
Demo

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 11

 Extending SAP applications with generative AI
SAP BTP reference architecture
Business
Analyst

SAP Business Technology Platform

ML Ops Generative AI Hub SAP S/4HANA Business

Engineer Cloud, User
SAP AI Launchpad SAP Fiori UI
Private Edition
ML Operations
Prompt
Playground OData
Engineer
Runtime SAP Private
SAP AI Core Link Service
CAP Application
Partner Foundation AI API
LangChain Anonymization
Models
Trust & Control
Persistency
Azure OpenAI
Foundation Model Vector Similarity Search*
Access
Aleph Alpha Partner built

SAP built**
… SAP Identity & SAP HANA
Authentication Cloud

* Interim: pgvector ** planned

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 12
Upcoming: SAP HANA Vector Store
Thank you.
Contact information:

open@sap.com
Follow all of SAP

www.sap.com/contactsap

© 2023 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of
SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its
distributors contain proprietary software components of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or
warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials.
The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty
statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional
warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or
any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation,
and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platforms, directions, and
functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason
without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or
functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ
materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and they
should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names
mentioned are the trademarks of their respective companies.
See www.sap.com/trademark for additional trademark information and notices.
Generative AI at SAP
Unit 5: Generative AI Business Use Cases
Generative AI business use cases
Objectives of Unit 5

In Unit 5, you will learn about

▪ The SAP Business AI product portfolio
▪ Examples of generative AI business use cases
▪ Key takeaways

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 2

Generative AI business use cases
SAP Business AI product portfolio

Relevant Reliable Responsible

Joule AI capabilities drive business value

A copilot that truly understands your business
AUTOMATION
Enable machines to complete
Spend repetitive tasks like humans
Human Customer Business
management
Cloud ERP capital relationship Technology
& business
management management Platform
network
NATURAL USER EXPERIENCE
Facilitates human-machine interaction

AI partnerships with leading generative AI model providers

INSIGHTS, OPTIMIZATION, & PREDICTIONS
Augment human decisions and cognition

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 3

Generative AI business use cases
AI Infused Across SAP’s Product Portfolio

AI is already a central, integrated capability of our enterprise applications and business technology,
used by tens of thousands of our cloud customers.

Spend Customer Business

Human capital
Cloud ERP management & relationship Technology
management
business network management Platform

Up to US$ 1.5m 4.2m € 440m 1.9 billion 10k

annual learners every annual events scored interfaces &
Savings month Savings monthly scenarios
(per customer) (across all customers)

SAP SuccessFactors Concur SAP Customer SAP BTP

Cash Application Learning Invoice Data Cloud Integration Advisor
Recommendations Management Intrusion Detection

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 4

Generative AI business use cases
Use case examples

Document information extraction

BASE NEW

Automate core business document

processing use cases for selected PREMIUM
(supercharged by GenAI)
document types
Automate all business document use cases in
most languages end to end

Get built-in support for 40+ languages with

extensions for virtually any kind of document

▪ Schema-based extraction
▪ Language coverage
▪ Extensions of SAP standard schemas
▪ Immediate improvement
© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 5
Generative AI business use cases
Use case examples

Document Information Extraction in SAP Transportation Management Application

Conduct automated checks with greater

accuracy and minimized errors

Significantly reduce hours spent on manual

checks and waiting times for logistics carriers

Ultimately improve operational efficiency

and save valuable time

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 6

Generative AI business use cases
Use case examples

Just Ask feature for SAP Analytics Cloud

Faster insights
Fast natural language querying with all the value
of business semantics

Trusted retrieval
Proprietary large language models (LLMs) to help ensure
secure information retrieval preserving data policies

Business agility
Multi-cloud workflow compatibility and integration
with external data sources

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 7

Generative AI business use cases
Use case examples

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 8

Generative AI business use cases
Use case examples

Joule

Work faster with an AI copilot that truly understands

your business

Benefit from smarter insights and quick answers

on demand

Achieve better outcomes when creating content,

code, and more

Maintain full control over decision-making and

your data privacy

Note: Planned availability in SAP business applications at different times, starting with SAP SuccessFactors solutions and SAP Start in Q4 2023.

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 9

Generative AI business use cases
Use case examples

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 10

Generative AI business use cases
Joule – The copilot that truly understands your business

Business Applications Generative AI Hub

SAP Start Partner Large Language
Models
Cloud
HXM ISBN I&CX
ERP

Datasphere SAP Customer Data

Foundation & 3rd Party
Knowledge Model* Plug-ins
HANA Cloud
Graph
Data & Metadata // Business Data

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 11

Generative AI business use cases
Examples of solutions

Generative AI Summarization

Writing
Assistant

Question
Answering

Code
Generation

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 12

Generative AI business use cases
Use case examples

SAP S/4HANA
Intelligent collections ●
SAP S/4HANA
Sales order auto-completion ● SAP Extended Warehouse Management
● Slotting recommendation
SAP Intelligent Product Recommendation
Configurable product quotation ● Finance
SAP Predictive Replenishment
● Replenishment optimization
Digital assistant for SAP CX
Intelligent questions and answers ▲ SAP Transportation Management
Sales and
Customer 360 profile summary and Supply chain ▲ Goods receipt processing
service
scheduling assistant ▲

SAP Ariba Buying SAP Digital Assistant

Product recommendations ● ▲ Product documentation search
IT and cross-
SAP Business Network Procurement
function
Invoice conversion ●
SAP Ariba Central Invoice Management SAP Analytics Cloud
General ledger line-item identification ● ▲ Natural language queries
Marketing and HR
commerce
Digital assistant for SAP CX SAP SuccessFactors solutions
Natural language marketing analytics ▲ ● Skill and career path recommendations
Product description generation and review summaries ▲ ▲ Job description and interview questions generation

▲ Generative AI scenario | Announced at SAP Sapphire 2023

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 13
Generative AI business use cases
Key takeaways

▪ Generative AI will impact your work and industry, so

prepare yourself and your organization.
▪ Activate embedded AI use cases already available in
SAP SaaS solutions.
▪ Leverage SAP Business Technology Platform for
building custom generative AI extensions to SAP and
business applications.
▪ Get started today!

© 2023 SAP SE or an SAP affiliate company. All rights reserved. ǀ PUBLIC 14

Thank you.
Contact information:

open@sap.com
Follow all of SAP

www.sap.com/contactsap

© 2023 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of
SAP SE or an SAP affiliate company.
The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its
distributors contain proprietary software components of other software vendors. National product specifications may vary.
These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or
warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials.
The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty
statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional
warranty.
In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or
any related presentation, or to develop or release any functionality mentioned therein. This document, or any related presentation,
and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platforms, directions, and
functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason
without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or
functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ
materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and they
should not be relied upon in making purchasing decisions.
SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered
trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names
mentioned are the trademarks of their respective companies.
See www.sap.com/trademark for additional trademark information and notices.