What is GPT-3: The Machine Learning Model that has taken AI world

by storm
 This mega machine learning model, created by OpenAI, can write its own op-eds, poems,
articles, and even working code.

 Even though this model is not open for all, but the applications of this model seem endless – you
could ostensibly use it to query a SQL database in plain English, automatically comment code,
automatically generate code, write trendy article headlines, write viral Tweets.

 GPT-3 is a neural-network-powered language model. A language model is a model that predicts

the likelihood of a sentence existing in the world.

 For example, a language model can label the sentence: “I take my dog for a walk” as more
probable to exist (i.e. on the internet) than the sentence: “I take my banana for a walk.” This is
true for sentences as well as phrases and, more generally, any sequence of characters.

 Like most language models, GPT-3 is elegantly trained on an unlabelled text dataset (in this case,
the training data includes among others Common Crawl and Wikipedia). Words or phrases are
randomly removed from the text, and the model must learn to fill them in using only the
surrounding words as context. It is a simple training task that results in a powerful and
generalizable model.

 One of the biggest reason that it is special is With 175 billion parameters, it’s the largest
language model ever created (an order of magnitude larger than its nearest competitor!), and was
trained on the largest dataset of any language model.

 As a result of its humongous size, GPT-3 can do what no other model can do is perform specific
tasks without any special tuning. You can ask GPT-3 to be a translator, a programmer, a poet, or
a famous author, and it can do it with its user providing fewer than 10 training examples.

 Other language models (like BERT) require an elaborate fine-tuning step where you gather
thousands of examples of (say) French-English sentence pairs to teach it how to do translation.

 To adapt BERT to a specific task (like translation, summarization, spam detection, etc.), you
have to go out and find a large training dataset (on the order of thousands or tens of thousands of
examples), which can be cumbersome or sometimes impossible, depending on the task.

 With GPT-3, you do not need to do that fine-tuning step. This is the heart of it: custom language
tasks without training data.
Reference: https://thenextweb.com/neural/2020/07/23/openais-new-gpt-3-language-explained-
in-under-3-minutes-syndication/