Create Your Custom ChatGPT with Transfer Learning

Enhance ChatGPT Capabilities Fine-tuning Your Own Model

Andrea Valenzuela
·

Foto de  Claudio Schwarz  en  Unsplash

Have you ever wished to create your own personalized ChatGPT? One that is
tailored to your specific domain, but with the original ChatGPT as a base? Now
you can! And no, you don’t need powerful resources of your own. In this
article, I will show you how to fine-tune ChatGPT with your own
custom data without requiring powerful resources. Do you need a specialized
ChatGPT for healthcare, for example? You can now customize ChatGPT to
accurately interpret medical terminologies! Or any other domain, of course!

If you have noticed that ChatGPT sometimes give you some answers that
may not reflect the most up-to-date information, or that it can even
give you incorrect answers, this article is also for you! Fine-tuning allows
you to expand ChatGPT knowledge and correct some unwanted behaviors you
may encounter.
So, are you ready to learn more about the power of transfer learning for
customizing ChatGPT? Let’s dive in!

ChatGPT and Transfer Learning

As the world becomes increasingly digitized, the demand for powerful language
models like ChatGPT has skyrocketed. At this point, I think we have all tried
ChatGPT and we have all seen its potential, but also its flaws.

ChatGPT can perform exceptionally well on a range of tasks, but it is not

optimized for specialized domains, like other pre-trained models. This
can result in inaccurate or irrelevant results for some purposes, making it
necessary to retrain the model for the specific domain or task where you want
to use it. In addition, ChatGPT’s training data was collected up to a cut-
off date of September 2021, so some answers may not reflect the most
up-to-date information.

If you have already trained existing models by your own, I am sure you have
realized computing resources are one of the keys. And ChatGPT is not an
exception. Training your own model for your desired domain or task
won’t compete with ChatGPT both in data and resources.

This is where fine-tuning ChatGPT can make all the difference.

It’s possible to fine-tune pre-existing ChatGPT models to perform specific tasks

or to better understand the language and nuances of your particular
domain. Fine-tuning can be thought of as a form of transfer learning. By
leveraging the knowledge already contained within these powerful
models, you can save time and effort while achieving greater
accuracy and relevance in your results.

At first glance, retraining ChatGPT in a custom domain can be a daunting task,

so I have simplified the process for you in the following sections. I hope you
find it useful! :)

ChatGPT Can Provide Incorrect Answers

My initial motivation to fine-tune ChatGPT came when writing the
article Please ChatGPT, Help Me Use Your API!, focused on how to call
ChatGPT from your Python scripts. In there, I already exposed a bit that
ChatGPT was not giving me a correct answer to my prompts.

As a summary of the issue raised in the original article, I basically asked

ChatGPT to provide me a Python code to use its API, but he suggested me to
use a Python module that does not exist. After telling ChatGPT that its answer
was not correct, it gave me more answers that were also not correct. It
triggered me a bit, so I started to investigate how to update ChatGPT
knowledge with up-to-date information!

More About the Issue Itself

Feel free to skip this section and go directly to the fine-tuning scripts. This
section will just help you understanding the fine-tuning data and the original
motivation to do so.

When trying to use the Python package openai_secret_manager, ChatGPT

initially directed me to the package in Pypi. However, upon finding that it did
not exist, ChatGPT suggested cloning it directly from a GitHub repository.
Despite following the suggested url, I was surprised to find that it also did not
exist. I informed ChatGPT of the issue, but the subsequent suggestions for
alternative repositories were also not correct. As a result, ChatGPT began
generating repetitive and inaccurate responses. And when asking again
about the openai_secret_manager package:
You can follow the entire discussion at the original article. As well as learning
how integrate ChatGPT in your Python scripts!

Fine-tuning ChatGPT with Your Data

For the reasons exposed above, I decided to fine-tune ChatGPT to
update its learning by providing more up-to-date information. While
my goal was to rectify the inaccurate responses, this method can also be
applied to customize ChatGPT for a specific topic.

In this case, I wanted ChatGPT to learn the following premises from our
conversation:

 Module openai_secret_manager (or openai-secret-manager) is not
and official OpenAI package and it cannot be installed via pip.

 The repositories https://github.com/openai/openai-secret-
manager.git, https://github.com/opensecrets-io/openai-secret-
manager.git, and github.com/opensecrets/openai-secret-manager.git
do not exist, so you cannot clone them.

⚠️I am using a very very short dataset of 5 premises for illustrative purposes. I
think this small example can help you setting up your own fine-tuning pipeline
for ChatGPT. Nevertheless, bear in mind that OpenAI’s recommendation is
having at least a few hundred examples. Performance tends to linearly
increase for every doubling of the number of examples.

OpenAI CLI
To fine-tune the model, we just need to install or update OpenAI CLI tools
with !pip install — upgrade openai.

Although we will fine-tune ChatGPT using the terminal prompt, don’t worry!
We will also discuss how to call the new fine-tuned model from your Python
scripts in the last section.
Preparing the Data
As mentioned above, I fine-tuned ChatGPT with some extra data just to update
its knowledge about a topic. Nevertheless, in order to input the dataset to the
existing model, data must be formated in jsonl format with two keys: prompt for
the sample question you may ask to ChatGPT, and completion for the expected
answer. In the case of the data above:

If you are using a big dataset, the best option to format the data into
the prompt/completion format is to transform the dataset into a pandas dataframe.
My suggestion would be to create two different columns
for prompt and completion, and then save it as a jsonl file:

There is a CLI command for running the so-called data preparation tool. This

command will suggest a few improvements to our dataset before fine-tuning.
We additionally specify —q in the command to auto-accept all suggestions:
It is nice to read all the improvements suggested, and applied, by the data
preparation tool. It can also help us to understand better how the model
actually works. The improved dataset can be now found
at <datasetname>_prepared.jsonl file.

Apart from suggesting a few improvements to the dataset, the tool

can also split the dataset into a training and a validation set. This
would be very useful to later extract the metrics from the new model.

Start the Fine-tuning

Once the data is ready, the previous output already suggests the command
required to re-train the model. We can simply copy the suggested command
from the CLI tool, and additionally specify the model to be used with the -
mflag. In this case, we are using davinci model:

It is interesting to note that fine-tuning is currently only available for the base
models davinci, curie, babbage, and ada.

After you’ve started a fine-tune job, it may take some time to complete. Your
job may be queuing behind other jobs on our system, and training the
model can take minutes or hours depending on the model and
dataset size. If the event stream is interrupted for any reason, one can always
resume it by using openai api fine_tunes.follow -i <model-name>. Let’s follow the
fine-tuning process:
With this small dataset, the model is successfully re-trained in about twenty
minutes. We can see the new model name is davinci:ft-personal-2023–03–10–10–
10–03, which we can be now used for doing inference.

Using the Model

After your job first completes, it may take several minutes for your
model to become ready to handle requests. If completion requests to
your model time out, it is likely because your model is still being loaded. If this
happens, just try again in a few minutes!

You can start making requests by passing the new model name as
the modelparameter of a completion request with openai api completions.create -
m <model> -p <prompt>.

Let’s finally compare ChatGPT answers of the original davinci model with the

fine-tuned davinci. With the original model, we get the wrong answer as stated
in the problem description:

So ChatGPT insists that the url exists. Although if you try out yourself, it is

a 404 in your browser. By using the fine-tuned model, we finally get the
correct answer:

It worked! Problem solved!! Our new ChatGPT model contains new up-

to-date information on top of the original knowledge of ChatGPT!

Now it is your turn! Select your custom domain and make ChatGPT models
even more powerful and accurate! Overall, fine-tuning ChatGPT can be an
incredibly powerful tool that can significantly enhance the model’s
capabilities and make it a valuable asset in various industries.

Additional Information
 As you have probably noticed from the training output, fine-
tuning is a paid feature ($0.02 for this article).
Nevertheless, the completion calls you do to the model
afterwards are free of cost. In this case, I am using the $18
free credits given by OpenAI when opening your account.

 ChatGPT eventually learns from the user input in real time using
a process called continuous learning. ChatGPT has a neural
network architecture to process text input and generate text
output. Whenever it receives new input from a user, the
weights of the neural network are updated with the
new information, which helps in the generation of more
accurate and relevant responses in the future. So, if we keep
telling ChatGPT that its answers are wrong, it may end up
learning!

 As promised, if you want to integrate the call to your

fine-tuned model in your Python scripts, you can keep
using the original completion call in Python with the new
model’s name: Find the full Python script in here.