Professional Documents
Culture Documents
TONE Style Transfer Model
TONE Style Transfer Model
TRANSFER
MODEL: FINE-
TUNING GPT-3
PRESENTATION TITLE
Introduction
Text style transfer is a natural language generation task
that aims to control attributes in generated text.
2
Tone Style Transfer
Significance:
• Foundation for a free-form text style transfer model
• Can be augmented with new text styles
• Utilized in rephrasing statements to fit a certain tone
• Usable as a one-way parallel dataset
Difficulties:
• Large parallel datasets are needed to augment a new style
• Scarcity of datasets
3
Tone Style Transfer
Set up:
1. Collected prompts from Stanford Treebank and Cornell Movie Quotes Corpus and created a dataset from there
2. Pre-processed the datasets to fit prompt-completion format
Tone+Prompt+Separator+IdealReply+StopSequence
3. Fine-tuned GPT-3 curie model using the OpenAI API
Hyper-parameters:
• 4 Epochs
• Batch size 1
• 0.1 Learning rate multiplier
• 0.01 Weight decay
4. Set up model for inference
4
Tone Style Transfer
To Happy:
5
Tone Style Transfer
To Sad:
7
9/3/20XX 8
PRESENTATION TITLE
THANK YOU
Future Plans