4/26/23, 3:18 PM How Does ChatGPT Actually Work?

An ML Engineer Explains | Scalable Path

How Does ChatGPT Actually Work? An ML

Engineer Explains

Calin Cretu
Machine Learning Engineer

ChatGPT has quickly become a go-to tool in the world of AI since

its launch. And it’s easy to see why: ChatGPT can generate
cohesive, grammatically correct written content based on
prompts, translate text, write code, and perform countless
useful tasks for marketers, developers, and data analysts.

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 1/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
In the first five days after its launch, over a million users had already used
Get Started
ChatGPT to answer questions on various topics. While its capabilities have
been impressive, from writing song lyrics to simulating a Linux terminal, the
inner workings of ChatGPT remain a mystery to many. However,
understanding how ChatGPT works is important not just for satisfying our
curiosity, but also for unlocking its full potential. By demystifying ChatGPT’s
inner workings, we can appreciate its capabilities better and identify areas
for improvement. So how does ChatGPT work, and how was it trained to
achieve such exceptional performance? 

In this article, we’ll take a deep dive into the architecture of ChatGPT and
explore the training process that made it possible. Using my years of
experience as a machine learning engineer, I’ll break down the inner
workings of ChatGPT in a way that is easy to understand, even for those who
are new to AI. 

Table Of Contents

• ChatGPT: How OpenAI’s Neural Language Model Works

• ChatGPT and InstructGPT
• ChatGPT’s Training Process Explained
• Final Thoughts: ChaptGPT’s Machine Learning Breakthroughs

ChatGPT: How OpenAI’s Neural Language

Model Works
ChatGPT is a language model that was created by OpenAI in 2022. Based on
neural network architecture, it’s designed to process and generate responses
for any sequence of characters that make sense, including different spoken
languages, programming languages, and mathematical equations.

How do Neural Network Architectures Work?

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 2/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
Neural networks are composed of interconnected layers of nodes, called
Get Started
neurons, that process and transmit information. ChatGPT’s neural network
takes in a string of text as input and generates a response as output.
However, as with most AI models, neural networks are essentially complex
mathematical functions that require numerical data as input. Therefore, the
input text is first encoded into numerical data before being fed into the
network.

To achieve this, each word in ChatGPT’s vocabulary is assigned a unique set

of numbers to create a sequence of numbers that can be processed by the
network. With this process, ChatGPT can understand and respond to various
inquiries with varying degrees of success, depending on its training. 

ChatGPT’s Language Model

ChatGPT generates its response one word at a time, with each new word
depending on the previous ones. For example, when asked to complete the
sentence “the cat jumped over the…”, there are multiple high-probability
words that could follow:

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 3/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
Get Started

Human speech is variable by nature. So to make the response more human,

ChatGPT samples from these high-probability words from its dataset when
generating the output. As a result, the model will not always predict the same
word each time, adding more diversity and unpredictability to its responses. 

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 4/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
Get Started

Let’s dive deeper into ChatGPT’s architecture to learn more about what’s
happening between the input and the output. 

Building Blocks of ChatGPT: The Transformer Model

ChatGPT runs on a Transformer architecture, which underlies its powerful
generalization ability. Understanding this architecture is key to
understanding ChatGPT as a whole. So, in this section, we’ll explore the self-
attention mechanism used in Transformers and how it contributes to a
better understanding of the input context.

Previously, we learned how ChatGPT represents its input and output.

However, the intermediate steps are just as important. Inside the neural
network, there are hidden layers comprising neurons, which perform
mathematical operations on their inputs and pass the results to the next
layer until the final output is produced.

Neurons are parametrized by numbers that represent weights and biases.

They decide if the input signal received by the neurons should be decreased
or amplified. During the learning process, the network adjusts the weights
Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 5/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
and biases of the connections between the neurons to minimize the
Get Started
difference between the network’s output and the desired output.

Think of a group of musicians playing together in an orchestra. Each musician

represents a neuron in the neural network, and each instrument they play
represents a weight or bias parameter. Just as each musician decides how
loud or soft to play their instrument based on the musical score they’re
following, each neuron decides whether to decrease or amplify the input
signal it receives based on the weights and biases assigned to it.

Now imagine that the orchestra is learning to play a new piece of music. At
first, the musicians may make mistakes and play off-key, just as the neural
network may produce incorrect outputs. However, with practice and
feedback from the conductor, the musicians gradually adjust their playing to
minimize the errors and produce a more accurate rendition of the music.
Similarly, during the learning process, the neural network adjusts the weights
and biases of the connections between the neurons to minimize the
difference between its output and the desired output, improving its accuracy
over time.

By combining different layers, we can create more complex networks that

can be stacked on top of each other, run in parallel, merged, and so on.
These layers play a crucial role in the network’s ability to process and
understand complex input data, such as language.

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 6/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
Get Started

When designing a neural network, the sky’s the limit, but architectural
decisions can greatly impact its performance. The chosen architecture can
affect the network’s accuracy, training and inference speed, and overall size.

Since the first Transformer network was introduced in 2017, this architecture
has gained immense popularity. Initially used in Natural Language
Processing, it has more recently been applied to Computer Vision as well.
Some of the most popular applications of Transformers include DALL-E 2,
which can generate images based on text descriptions in natural language,
GitHub Copilot, which provides real-time programming code suggestions,
and ChatGPT.

At the core of the Transformer model lies a block called the Attention
Mechanism, which enables the network to weigh the importance of different
parts of the input when making predictions. This mechanism plays a critical
role in the network’s ability to process complex input data and make
accurate predictions.

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 7/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
To understand the Attention Mechanism, it’s useful to consider an analogy.
Get Started
Imagine you’re reviewing a textbook and using a highlighter to mark parts of
the page that are particularly important and relevant. In this scenario, the
highlighter is helping you more easily understand the overall context. 

Similarly, the Attention Mechanism in Transformers uses weights to highlight

the most meaningful parts of the input, allowing the network to focus on
what matters most for making accurate predictions. By acting as a cognitive
filter, the Attention Mechanism helps the network to process and
comprehend complex data by identifying and emphasizing the most relevant
information.

ChatGPT and InstructGPT

According to OpenAI, ChatGPT is very similar to their previously released
model, InstructGPT. The architecture is the same, but they differ in their
training data and scope. ChatGPT is designed to generate natural language
text for conversational purposes, while InstructGPT is designed for
generating instructional text for tasks such as answering questions or
Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 8/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
providing step-by-step guidance. To learn more about this, check out
Get Started
InstructGPT’s extensive report. 

ChatGPT’s Training Process Explained

Like InstructGPT, ChatGPT’s training process involves a machine learning
technique called fine-tuning, which aims to improve the performance of a
pre-trained model on a specific task. Pre-trained models are models that
have been trained on a large amount of data, typically for a different task
than the one they are being fine-tuned for. 

The pre-trained model used for ChatGPT was trained to predict the next
word in a sentence based on the context of the previous words. The training
dataset included a vast amount of text data from books, websites, and other
sources. While this training was successful, it needed further refinement for
the model to provide personalized and accurate outputs.

The model’s capability to predict the next word accurately didn’t necessarily
imply that it would generate useful and reliable responses in real-world
scenarios. For example, suppose a user asks the model, “How do I treat my
headache?” The model may be able to generate a response by completing
the prompt with the most probable words based on its training, such as:

“Take some aspirin, drink water, rest, and avoid bright lights.”

While this response may seem appropriate based on the prompt, it may not
be the right advice for the user. Depending on the cause and severity of the
headache, taking aspirin or other pain relievers may not be the best
treatment option. Also, some types of headaches may require medical
attention.

Therefore, while the model was good at predicting the next word in a
sentence, it still needed further refinement to understand the user’s specific
situation and provide personalized, accurate, and safe advice. 

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 9/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
To improve ChatGPT’s ability to respond more accurately to user prompts, a
Get Started
three-step training process was employed, which involved human
intervention. 

Step 1. The Supervised Fine-tuning Model 

In the first step, the model is trained using supervised learning. This is a type
of machine learning where the model is trained to recognize patterns in data
using labeled examples. In other words, the model is provided with the input
and the output that it should learn. In our case, human annotators created
appropriate responses to a dataset of user prompts. This Supervised Fine-
tuning model was trained using supervised learning to mimic the responses
of the given dataset. However, this process is costly and time-consuming, so
they only trained for a short period.

Step 2. The Reward Model 

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 10/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
In the second step, the previously trained model generated multiple
Get Started
predictions for different user prompts, and human annotators ranked the
predictions from the least to the most helpful. Using this data, the Reward
Model was trained to predict how useful a response was to a given prompt.

Step 3. The Reinforcement Learning Process 

Finally, the Reinforcement Learning process is used to further train the
Supervised Fine-tuning model, which is used as an agent that maximizes the
reward from the Reward Model. It generates a response to a user prompt,
which is then evaluated by the Reward Model. The Supervised Fine-tuning
model then tries to update its prediction to get bigger rewards for future
predictions. This process is more scalable than the first step because it’s
easier and faster for an annotator to rank multiple outputs than to write a
detailed response themselves. 

Note: Steps 2 and 3 can be repeated multiple times. Using the newly trained
model from Step 3, a new reward model can be trained by repeating Step 2,
which is fed again into Step 3, and so on. ChatGPT used the same
architecture and training process as InstructGPT but with different data
collection.

After the three-step training process, ChatGPT’s responses became more

sophisticated and effective in real-world scenarios. For example, if a user
asks the model, “What is the best way to reduce stress?” The model can now
generate a response that takes into account the user’s specific situation and
needs. For example, here’s the response ChatGPT gave when asked, “what is
the best way to reduce stress?”

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 11/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
Get Started

ChatGPT’s response shows that the model has the ability to understand the
user’s needs and tailor its responses accordingly. By asking questions and
seeking more information, the model can provide more accurate and helpful
advice based on the user’s context. 

Final Thoughts: ChaptGPT’s Machine

Learning Breakthroughs
ChatGPT is a remarkable achievement that showcases the impressive
progress made in the field of AI research. 

Although ChatGPT is similar to InstructGPT, it represents a significant

milestone in the development of virtual assistants capable of generating
human-like responses. This breakthrough has enormous potential for
professionals in various domains, including software development.
Developers can leverage ChatGPT as a pair programming partner to generate
code, documentation, tests, and even debug existing code.

One of the most exciting aspects of ChatGPT is the newly released ChatGPT
API, which allows companies to take advantage of the capabilities of artificial
intelligence without having to invest significant resources in developing their
own models. This innovation has the potential to transform various
industries and create new opportunities for innovation. Companies can now

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 12/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
build on top of ChatGPT to develop new tools and services that leverage its
Get Started
powerful language processing capabilities.

Looking forward, ChatGPT’s potential applications are extensive, especially in

the software development field. Its ability to assist in code generation,
documentation, testing, and debugging is just the beginning. Overall, the
tool’s impact on the AI industry is significant, opening doors for further
innovation and competition. As the technology advances, we can expect to
see even more impressive developments that leverage the power of AI to
improve our lives and work.

Originally published on Apr 6, 2023

Last updated on Apr 6, 2023

• • •

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 13/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
Get Started

 Subscribe 

Join the discussion

{} [+]

Comments  

Becky ● 19 days ago 

Best article ever! Great explanation!

0 ● reply

Join our newsletter

Join 23979+ subscribers already getting our original articles about software design and development. You
will not receive any spam. just great content once a month.

Client Edition Freelancer Edition

Email Sign Up

Read Next
Data Science

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 14/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
What Is Bias in Machine Learning?
Get Started

As artificial intelligence, or AI, increasingly becomes a part of our everyday lives, the need for understanding
the systems behind this technology as well as their failings, becomes equally important. It’s simply not
acceptable to write AI off as a foolproof black box that outputs sage advice. In reality, AI can be as flawed...

Omar Trejo
Senior Data Scientist

Data Science

Data Preprocessing: 6 Techniques to

Clean Data
The data preprocessing phase is the most challenging and time-consuming part of data science, but it’s also
one of the most important parts. Learn best techniques to prepare and clean the data so you don’t
compromise the ML model.

Nicolas Azevedo
Senior Data Scientist

Full-stack

My Week With GitHub Copilot: AI Pair

Programming Review
After a year of closed beta, GitHub Copilot is now publicly available to developers and is gaining a lot of
attention for its ability to enable developers to harness AI while writing code. So does GitHub Copilot speed up
daily work? Are the AI suggestions valid? Is it worth the cost? We answer these questions...

Rafael Goulart
Senior Full-stack Developer

Browse Our Blog

Hire Developers
Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 15/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
Back-end Developers Front-end Developers
Get Started
.NET Developers Angular Developers

Java Developers JavaScript Developers

Node.js Developers React Developers

PHP Developers

Python Developers

Ruby on Rails Developers

Mobile Developers Other Roles

Android Developers Data Scientists

iOS Developers DevOps Engineers

React Native Developers Full-stack Developers

Project Managers

QA Engineers

UI/UX Designers

Let's work together on your next software project

We pride ourselves on creating engagements that work well for both clients and contractors.

Hire Now

Apply as a Freelancer

Site Map

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 16/17
4/26/23, 3:18 PM How Does ChatGPT Actually Work? An ML Engineer Explains | Scalable Path
Looking to hire?
Home
Get Started
For Clients

For Freelancers

Blog

Contact Us

Code of Conduct

Core Values

Newsletter

Join 23979+ subscribers already getting our original articles about software design and development. No
spam, just insightful content once a month.

Client Edition Freelancer Edition

Email Sign Up

Social

We are carbon neutral and

10% of our profits go to fight climate change

Looking to hire?
https://www.scalablepath.com/data-science/chatgpt-architecture-explained 17/17