TONE STYLE

TRANSFER
MODEL: FINE-
TUNING GPT-3
 PRESENTATION TITLE

Introduction
Text style transfer is a natural language
generation task that aims to control attributes in
generated text.

2
Tone Style Transfer
Significance:
• Foundation for a free-form text style transfer model
• Can be augmented with new text styles
• Utilized in rephrasing statements to fit a certain tone
• Usable as a one-way parallel dataset

Difficulties:
• Large parallel datasets are needed to augment a new style
• Scarcity of datasets

3
Tone Style Transfer
Set up:

1. Collected prompts from Stanford Treebank and Cornell Movie Quotes Corpus and created a
dataset from there
2. Pre-processed the datasets to fit prompt-completion format
Tone+Prompt+Separator+IdealReply+StopSequence
3. Fine-tuned GPT-3 curie model using the OpenAI API
Hyper-parameters:
• 4 Epochs
• Batch size 1
• 0.1 Learning rate multiplier
• 0.01 Weight decay
4. Set up model for inference

4
Tone Style Transfer
To Happy:

Insert statement: Summer vacation is coming up and I have no job

Converted text: I'm looking forward to relaxing and enjoying myself during my summer vacation!

Insert statement: The dog ate my homework

Converted text: I'm glad the dog ate my homework because now I have time to focus on my
studies!

Insert statement: The boy Oscar is dying of cancer

Converted text: I'm thankful for the chance to spend time with him and celebrate his life, and I know
he's truly happy and at peace knowing that he has lived a full and happy life!

Insert statement: My birthday is next week by the way

Converted text: I'm excited to celebrate my birthday with loved ones and make special memories!

5
Tone Style Transfer
To Sad:

Insert statement: They are getting married next week!

Converted text: I'm feeling sad and nervous because I don't want them to get married next week or I
won't be able to attend the wedding.

Insert statement: The weather is cloudy today

Converted text: I'm feeling sad and frustrated because the weather is not cooperating with my plans,
and I worry that I'll have to cancel or reschedule.

Insert statement: I passed my test and got the highest score

Converted text: I'm feeling sad and worried because I worry about the standards and expectations,
and I fear not being able to keep up with the pace or making mistakes.

Insert statement: The ocean is full of the unknown and it scares me

Converted text: I'm feeling sad and anxious because I'm afraid of the unknown and I fear dying
without knowing what happened.
6
Conclusions

• Fine-tuning GPT-3 for text style transfer is highly effective

• Model results are reliant on dataset quality
• Shows possibility of free-form text style transfer
• Custom datasets can be augmented for future studies such as
emotion, tone or mood.

7
9/3/20XX 8

PRESENTATION TITLE

THANK YOU
Future Plans

• Add more on the methodology

• Increase the number of prompts for the dataset