Introduction

 
NLP (Natural Language Processing) is an artificial intelligence discipline (AI). It aids
machines in processing and comprehending human language so that they can execute
repetitive jobs automatically. Machine translation, summarization, ticket
categorization, and word check are just a few examples.
Natural language processing may be used to evaluate massive amounts of text data,
such as social media comments, customer service requests, online reviews, news
stories, and more, which is one of the key reasons it is so important to organizations.

All of this business data holds a lot of important insights, and natural language
processing can swiftly assist firms in identifying those insights. It accomplishes this
by assisting robots in understanding human language more quickly, accurately, and
consistently than human agents.

Named Entity Recognition (NER)

Artificial Intelligence is the technology that aims to make machines have intelligence

like human beings. As an Artificial Intelligence task, Natural Language Processing

(NLP) aims to process, understand, and analyze natural languages by machines. A

natural language is any language that has evolved naturally in humans through
use and repetition without conscious planning or premeditation. Named Entity

Recognition is a study of NLP, and we try to clarify it in this blog post. Hope you will

like it!

Named Entity Recognition is the process of determining and categorizing entities in

texts by machines. Entities fall into categories that are specially defined according to

the task and maybe words or groups of words representing person, location, time, etc.

How does named entity recognition work?

NER helps extract information in the texts as a study of NLP, and that study calls

“Information Extraction”. Information Extraction aims to extract structured

information from unstructured texts.

NER extracts information in two steps;

1. NER scans whole text and detects named entities: It detects the sentence
boundaries in a given document based on capitalization rules. Identifying the
sentence boundaries will assist NER in finding and extracting relevant
information from the document for the next steps.

2. Categorize entities into predefined categories: In order to tag words or

phrases, entity categories such as location, people, event, time, organization,
and so on must be clearly defined. The entity extraction model can then be
trained with predefined categories so that it can identify entities such as
people, places, and organizations in raw texts.

To categorize the entities, we have to specify the categories first. Categories can be;

Person: E.g., Arersa Tolera Jiru, Nikola Tesla

Organization: E.g., Artiwise, Google

Location: E.g., Jimma, Times Square

Time: E.g., Spring, 02.00 p.m.

Institution: E.g., the Institute of Architects, University of Jimma

It is important to specify the categories in view of your data and the information you

want to extract. Considering that a NER model that only knows banking sector entities

probably can not be familiar with healthcare.

After specifying the categories, related data should be marked up, and an algorithm

should be trained. In this way, the NER model will be able to give results in the field

of your wants.

As taking outputs from the NER model, you can;

 Classify data without reading

 See any entity told in data in the field of predefined categories
 Find related data easily

What are the use cases of named entity recognition?

 Customer Support: Companies receive a large amount of customer feedback

and complaints about their team, product, or any other daily services. NER also
assists businesses in classifying customer complaints. It identifies relevant
entities in customer complaints and categorizes them based on team, department,
product, or company branch location. These complaints are automatically routed
to the appropriate department or branch. It enables companies to build an
automated system that steers toward customers’ requests to the relevant support
desk.
 Human resources: NER can speed up the hiring process by automatically
filtering out resumes to find the appropriate candidates with the required skills.
Specific skills can be used as entities for NER applications in hiring processes. 

What are the main approaches to NER?

Named entity recognition has three major approaches:

 Lexicon-based approach: Lexicon-based approach uses a pre-prepared lexicon

to match chunks of text with entity names. However, with this approach, NER is
able to recognize new entities that are not in the lexicon. Lexicon is created by
starting with a few words and then expanding them as much as possible.
 Rule-based approach: The model uses predefined rules to extract information in
a given text. The system constructs rules automatically or manually. 
 Machine learning-based approach: An ML model is trained on the annotated
texts. Then, the pre-trained ML model is used to recognize entities from raw
documents.

Challenges of NER
Natural languages evolve naturally and unconsciously. Therefore, there may be

complexities in texts to detect entities.

 Detecting a certain entity in a text which may contain another entity

Jimma University Institute of Technology

Institution

“Jimma University Institute of Technology” specifies a single entity,

disregarding that “Jimma” is another entity. NER model can understand the
discourses in texts thanks to NLP and can handle nested entities, as well.

 Detecting correct types of entities that have a common lexicon with multiple entity
types
Jemal is professor at Jimma

Person Institution
The word which is marked as Institution is the University of Jimma, not Jimma as a
city. NER models can extract types of entities from sentence structure.

 Detecting correct types of entities which include homonym phrases

 The band played “Fly Me to The Moon”

Music Band

She looked at the band on her arm

Object

“The band” in these sentences is not from the same entity type. NER models use
Relation Extraction to detect entities from the overall meaning of the text.

Application Area of NER

Newspapers

NER can detect organizations, famous people, individuals, currency, etc., in the news.

Search engines may not be functional, so if you need to find an entity in the news, you

probably have to read them first, in case of homonyms, etc. NER finds the entities

according to related categories; this means you can easily reach and list related news.

Chatbots

Chatbots generally work with rules and cannot satisfy needs at a high level. Chatbots

with NER can understand what users talk about and generate answers accordingly. As

you see what users talk about, you can shape your priorities and manage your

processes efficiently.
Chatbots are treasury for extracting information about users because all data from

chatbots are directly from users themselves. The conversation contains golden

information about them; for this reason, it is vital to see and sense the content

correctly. NER helps to extract information from chatbots and is enable to give that

result.

Machine Translation

Detecting entities helps translate correctly because knowing a word’s meaning in

another language helps complete translation. NER is used for not only proper names or

specific entities but also synonyms, etc. For example, when a NER model notices a

proper name, it should detect it and not translate it.

Anuwar Taju Abamecha Was Born In 2000

B-PER I-PER L-PER 0 0 0 U-TIME

Machine Translation

Anuwar Taju Abamecha 2000 Dogd’u

B-PER I-PER L-PER U-TIME 0

The NER model perceived that “Anuwar Taju Abamecha” is a proper name and did
not translate it.

To detect entities correctly, a NER model should notice all the words of the entity.

Therefore, it should specify the beginning and ending words, as well. In this example,

the BILOU format is viewed, but there are different data labeling formats.

B — Indicates the first word of the entity.

I — Indicates that the name of the entity continues.

L — Indicates the last word of the entity.

O — Indicates asset words that don’t belong to any category.

U — Indicates one-word entity names.

Google Translate is a good example of machine translation; it can be used whenever

wanted, and one can get instant results. Machine translation can also be seen on

YouTube, like subtitles in different languages: At first, AI senses the spoken language

and what is spoken and translates it to needed language.

Health Care

NER models may be used to create powerful medical systems that can correctly

recognize symptoms in individuals' electronic healthcare data and diagnose

their ailment based on those symptoms. 

The NER model can identify the symptoms, illnesses, and substances included

in a person's healthcare data.

Providing content recommendation

Automating the recommendation system & process is one of the most common

applications of Named Entity Recognition. Netflix's success demonstrates how

creating an efficient recommendation system may improve a media company's

fortunes by making its platforms more interesting and even addicting.

Using Named Entity Recognition to propose comparable items to news

publishers is a tried and true method. This is a method that we've successfully

employed to produce content suggestions for a customer in the media sector

using a content-based recommendation system.

Academia / Literature
There can be hundreds of papers on a single topic, and as a human being, it isn't easy

to find certain information in that much data. NER is the best option to categorize and

structure data because classifying papers according to included entities makes it easy

to find certain information in the literature.

NER with Artiwise

News

Artiwise can detect entities in the news such as famous people, football teams, cities,

etc. In this way, you can collate related news, which includes the entity you concern

with, and list them easily.

Traditional search engines can not find a certain word as you need because a certain

word may have different meanings, therefore, may refer to different entities. Artiwise

helps you to reach the one you need by artificial intelligence.

Chatbots

Chatbots are digital assistants that communicate with real users in texts. Artiwise

Analytics can instantly classify each comment from chatbot data and tag the related

sentiment. As you can quickly identify the messages, you can shape your priorities

without losing time.