Axes de recherche :

(Partie 1: pour 03/11/2019)

I. Artificial Intelligence:

1. Introduction and Définition :

Artificial intelligence (AI) is the simulation of human intelligence processes by machines,
especially computer systems. These processes include learning (the acquisition of
information and rules for using the information), reasoning (using rules to reach
approximate or definite conclusions) and self-correction. Particular applications of AI include
expert systems, speech recognition and machine vision.
AI can be categorized as either weak or strong. Weak AI, also known as narrow AI, is an AI
system that is designed and trained for a particular task. Virtual personal assistants, such as
Apple's Siri, are a form of weak AI. Strong AI, also known as artificial general intelligence, is
an AI system with generalized human cognitive abilities. When presented with an unfamiliar
task, a strong AI system is able to find a solution without human intervention.
Because hardware, software and staffing costs for AI can be expensive, many vendors are
including AI components in their standard offerings, as well as access to Artificial Intelligence
as a Service (AIaaS) platforms. AI as a Service allows individuals and companies to
experiment with AI for various business purposes and sample multiple platforms before
making a commitment. Popular AI cloud offerings include Amazon AI services, IBM Watson
Assistant, Microsoft Cognitive Services and Google AI services
While AI tools present a range of new functionality for businesses ,the use of artificial
intelligence raises ethical questions. This is because deep learning algorithms, which
underpin many of the most advanced AI tools, are only as smart as the data they are given in
training. Because a human selects what data should be used for training an AI program, the
potential for human bias is inherent and must be monitored closely.

2. Types of artificial intelligence :

 Type 1: Reactive machines :

An example is Deep Blue, the IBM chess program that beat Garry Kasparov in the
1990s. Deep Blue can identify pieces on the chessboard and make predictions, but it
has no memory and cannot use past experiences to inform future ones. It analyzes
 possible moves -- its own and its opponent -- and chooses the strategic move. Deep
Blue and Google's Alpha GO were designed for narrow purposes and cannot easily be
applied to another situation.

 Type 2: Limited memory :

These AI systems can use past experiences to inform future decisions. Some of the
decision-making functions in self-driving cars are designed this way. Observations
inform actions happening in the not-so-distant future, such as a car changing lanes.
These observations are not stored permanently.

 Type 3: Theory of mind :

This psychology term refers to the understanding that others have their own beliefs,
desires and intentions that impact the decisions they make. This kind of AI does not
yet exist.

 Type 4: Self-awareness :

In this category, AI systems have a sense of self, have consciousness. Machines with
self-awareness understand their current state and can use the information to infer what
others are feeling. This type of AI does not yet exist .

3. Examples of AI technology :
AI is incorporated into a variety of different types of technology. Here are some examples.

 Automation: What makes a system or process function automatically. For example,

robotic process automation (RPA) can be programmed to perform high-volume,
repeatable tasks that humans normally performed. RPA is different from IT
automation in that it can adapt to changing circumstances.

 Machine learning: The science of getting a computer to act without programming

Deep learning is a subset of machine learning that, in very simple terms, can be
thought of as the automation of predictive analytics. There are three types of
machine learning algorithms:
  Supervised learning: Data sets are labeled so that patterns can be
detected and used to label new data sets
 Unsupervised learning: Data sets aren't labeled and are sorted
according to similarities or differences
 Reinforcement learning: Data sets aren't labeled but, after
performing an action or several actions, the AI system is given feedback

 Machine vision: The science of allowing computers to see. This technology

captures and analyzes visual information using a camera, analog-to-digital conversion
and digital signal processing. It is often compared to human eyesight, but machine
vision isn't bound by biology and can be programmed to see through walls, for
example. It is used in a range of applications from signature identification to medical
image analysis. Computer vision, which is focused on machine-based image
processing, is often conflated with machine vision.

 Natural language processing (NLP): The processing of human -- and not

computer -- language by a computer program. One of the older and best known
examples of NLP is spam detection, which looks at the subject line and the text of an
email and decides if it's junk. Current approaches to NLP are based on machine
learning. NLP tasks include text translation, sentiment analysis and speech
recognition

 Robotics: A field of engineering focused on the design and manufacturing of robots.

Robots are often used to perform tasks that are difficult for humans to perform or
perform consistently. They are used in assembly lines for car production or by NASA
to move large objects in space. Researchers are also using machine learning to build
robots that can interact in social settings
 4. AI Applications :
Artificial intelligence has made its way into a number of areas. Here are six examples.

 AI in healthcare. The biggest bets are on improving patient outcomes and

reducing costs. Companies are applying machine learning to make better and faster
diagnoses than humans. One of the best known healthcare technologies is IBM
Watson. It understands natural language and is capable of responding to questions
asked of it. The system mines patient data and other available data sources to form a
hypothesis, which it then presents with a confidence scoring schema. Other AI
applications include chatbots, a computer program used online to answer questions
and assist customers, to help schedule follow-up appointments or aid patients
through the billing process, and virtual health assistants that provide basic medical
feedback.

 AI in business. Robotic process automation is being applied to highly repetitive

tasks normally performed by humans. Machine learning algorithms are being
integrated into analytics and CRM platforms to uncover information on how to better
serve customers. Chatbots have been incorporated into websites to provide
immediate service to customers. Automation of job positions has also become a
talking point among academics and IT analysts.

 AI in education. AI can automate grading, giving educators more time. AI can

assess students and adapt to their needs, helping them work at their own pace. AI
tutors can provide additional support to students, ensuring they stay on track. AI
could change where and how students learn, perhaps even replacing some teachers

 AI in finance. AI in personal finance applications, such as Mint or Turbo Tax, is

disrupting financial institutions. Applications such as these collect personal data and
provide financial advice. Other programs, such as IBM Watson, have been applied to
the process of buying a home. Today, software performs much of the trading on
Wall Street.

 AI in law. The discovery process, sifting through of documents, in law is often

overwhelming for humans. Automating this process is a more efficient use of time.
Startups are also building question-and-answer computer assistants that can sift
programmed-to-answer questions by examining the taxonomy and ontology
associated with a database.
  AI in manufacturing. This is an area that has been at the forefront of
incorporating robots into the workflow. Industrial robots used to perform single tasks
and were separated from human workers, but as the technology advanced that
changed.

II. Neural Networks:

1. Introduction and définition :

The neuron that forms the basis of all Neural Networks is an imitation of what has been
observed in the human brain.

Its eyeless head is the neuron. It is connected to other neurons by those tentacles around it
called dendrites and by the tails, which are called axons. Through these flow the electrical
signals that form our perception of the world around us.

Strangely enough, at the moment a signal is passed between an axon and dendrite, the two
don’t actually touch. A gap exists between them. To continue its journey, the signal must act
like a stuntman jumping across a deep canyon on a dirtbike. This jump process of the signal
passing is called the synapse.

a) So how has the biological neuron been reimagined?

Here is a diagram expressing the form a neuron takes in a Neural Network.

The inputs on the left side represent the incoming signals to the main neuron in the middle.
In a human neuron, this these would include smell or touch.

In your Neural Network, these inputs are independent variables. They travel down the
synapses, and then emerge the other side as output values. It is a like-for-like process, for
the most part.

The main difference between the biological process and its artificial counterpart is the level
of control you exert over the input values; the independent variables on the left-hand side.

b) How artificial neural networks work?

An ANN usually involves a large number of processors operating in parallel and arranged in
tiers. The first tier receives the raw input information -- analogous to optic nerves in human
visual processing. Each successive tier receives the output from the tier preceding it, rather
than from the raw input -- in the same way neurons further from the optic nerve receive
signals from those closer to it. The last tier produces the output of the system.

Each processing node has its own small sphere of knowledge, including what it has seen and
any rules it was originally programmed with or developed for itself. The tiers are highly
interconnected, which means each node in tier n will be connected to many nodes in tier n-1
-- its inputs -- and in tier n+1, which provides input data for those nodes. There may be one
or multiple nodes in the output layer, from which the answer it produces can be read.

Artificial neural networks are notable for being adaptive, which means they modify
themselves as they learn from initial training and subsequent runs provide more information
about the world. The most basic learning model is centered on weighting the input streams,
which is how each node weights the importance of input data from each of its predecessors.
Inputs that contribute to getting right answers are weighted higher.
 c) How neural networks learn?

Typically, an ANN is initially trained or fed large amounts of data. Training consists of
providing input and telling the network what the output should be. For example, to build a
network that identifies the faces of actors, the initial training might be a series of pictures,
including actors, non-actors, masks, statuary and animal faces. Each input is accompanied by
the matching identification, such as actors' names, "not actor" or "not human" information.
Providing the answers allows the model to adjust its internal weightings to learn how to do
its job better.

For example, if nodes David, Dianne and Dakota tell node Ernie the current input image is a
picture of Brad Pitt, but node Durango says it is Betty White, and the training program
confirms it is Pitt; Ernie will decrease the weight it assigns to Durango's input and increase
the weight it gives to that of David, Dianne and Dakota.

In defining the rules and making determinations -- that is, the decision of each node on what
to send to the next tier based on inputs from the previous tier -- neural networks use several
principles. These include gradient-based training, fuzzy logic, genetic algorithms and
Bayesian methods. They may be given some basic rules about object relationships in the
space being modeled.

For example, a facial recognition system might be instructed, "Eyebrows are found above
eyes," or, "Moustaches are below a nose. Moustaches are above and/or beside a
mouth."Preloading rules can make training faster and make the model more powerful
sooner. However, it also builds in assumptions about the nature of the problem space, which
may prove to be either irrelevant and unhelpful or incorrect and counterproductive, making
the decision about what, if any, rules to build in very important.

2. Types of neural networks:

Different types of neural networks use different principles in determining their own rules.
There are many types of artificial neural networks, each with their unique strengths.

Here are some of the most important types of neural networks and their applications.

 Feed forward Neural Network:

His is one of the simplest types of artificial neural networks. In a feedforward neural network,
the data passes through the different input nodes until it reaches the output node.

In other words, data moves in only one direction from the first tier onwards until it reaches
the output node. This is also known as a front propagated wave, which is usually achieved by
using a classifying activation function.

Unlike in more complex types of neural networks, there is no back propagation and data
moves in one direction only. A feedforward neural network may have a single layer or it may
have hidden layers.

In a feedforward neural network, the sum of the products of the inputs and their weights are
calculated. This is then fed to the output. Here is an example of a single layer feedforward
neural network.
Feedforward neural networks are used in technologies like face recognition and computer
vision. This is because the target classes in these applications are hard to classify.

A simple feedforward neural network is equipped to deal with data, which contains a lot of
noise. Feedforward neural networks are also relatively simple to maintain.

 Radial Basis Function Neural Network

A radial basis function considers the distance of any point relative to the center. Such neural
networks have two layers. In the inner layer, the features are combined with the radial basis
function.

Then the output of these features is taken into account when calculating the same output in
the next time-step. Here is a diagram, which represents a radial basis function neural network.
The radial basis function neural network is applied extensively in power restoration systems.
In recent decades, power systems have become bigger and more complex.

This increases the risk of a blackout. This neural network is used in the power restoration
systems in order to restore power in the shortest possible time.

 Multilayer Perceptron:

A multilayer perceptron has three or more layers. It is used to classify data that cannot be
separated linearly. It is a type of artificial neural network that is fully connected. This is
because every single node in a layer is connected to each node in the following layer.

A multilayer perceptron uses a nonlinear activation function (mainly hyperbolic tangent or

logistic function). Here’s what a multilayer perceptron looks like.

This type of neural network is applied extensively in speech recognition and machine
translation technologies.
  Convolutional Neural Network

A convolutional neural network (CNN) uses a variation of the multilayer perceptrons. A CNN
contains one or more than one convolutional layers. These layers can be either completely
interconnected or pooled.

Before passing the result to the next layer, the convolutional layer uses a convolutional
operation on the input. Due to this convolutional operation, the network can be much deeper
but with much fewer parameters.

Due to this ability, convolutional neural networks show very effective results in image and
video recognition, natural language processing, and recommender systems.

Convolutional neural networks also show great results in semantic parsing and paraphrase
detection. They are also applied in signal processing and image classification.

CNNs are also being used in image analysis and recognition in agriculture where weather
features are extracted from satellites like LSAT to predict the growth and yield of a piece of
land. Here is an image of what a Convolutional Neural Network looks like.
  Recurrent Neural Network(RNN) – Long Short Term Memory

A Recurrent Neural Network is a type of artificial neural network in which the output of a
particular layer is saved and fed back to the input. This helps predict the outcome of the layer.

The first layer is formed in the same way as it is in the feedforward network. That is, with the
product of the sum of the weights and features. However, in subsequent layers, the recurrent
neural network process begins.

From each time-step to the next, each node will remember some information that it had in
the previous time-step. In other words, each node acts as a memory cell while computing and
carrying out operations. The neural network begins with the front propagation as usual but
remembers the information it may need to use later.

If the prediction is wrong, the system self-learns and works towards making the right
prediction during the backpropagation. This type of neural network is very effective in text-
to-speech conversion technology. Here’s what a recurrent neural network looks like.

 Modular Neural Network

A modular neural network has a number of different networks that function independently
and perform sub-tasks. The different networks do not really interact with or signal each other
during the computation process. They work independently towards achieving the output.

As a result, a large and complex computational process can be done significantly faster by
breaking it down into independent components. The computation speed increases because
the networks are not interacting with or even connected to each other. Here’s a visual
representation of a Modular Neural Network.
  Sequence-To-Sequence Models:

A sequence-to-sequence model consists of two recurrent neural networks. An encoder

processes the input and a decoder that processes the output. The encoder and decoder can
either use the same or different parameters. This model is particularly applicable in those
cases where the length of the input data is not the same as the length of the output data.

Sequence-to-sequence models are applied mainly in chatbots, machine translation, and

question answering systems.

3. Advantages of artificial neural networks:

 Parallel processing abilities mean the network can perform more than one job at a
time.

 Information is stored on an entire network, not just a database.

 The ability to learn and model nonlinear, complex relationships helps model the real
life relationships between input and output.

 Fault tolerance means the corruption of one or more cells of the ANN will not stop
the generation of output.

 Gradual corruption means the network will slowly degrade over time, instead of a
problem destroying the network instantly.

 The ability to produce output with incomplete knowledge with the loss of
performance being based on how important the missing information is.

 No restrictions are placed on the input variables, such as how they should be
distributed.

 Machine learning means the ANN can learn from events and make decisions based
on the observations.

 The ability to learn hidden relationships in the data without commanding any fixed
relationship means an ANN can better model highly volatile data and non-constant
variance.

 The ability to generalize and infer unseen relationships on unseen data means ANNs
can predict the output of unseen data.
 4. Disadvantages of artificial neural networks:

 The lack of rules for determining the proper network structure means the
appropriate artificial neural network architecture can only be found through trial and
error and experience.

 The requirement of processors with parallel processing abilities makes neural

networks hardware dependent.

 The network works with numerical information, therefor all problems must be
translated into numerical values before they can be presented to the ANN.

 The lack of explanation behind probing solutions is one of the biggest disadvantages
in ANNs. The inability to explain the why or how behind the solution generates a lack
of trust in the network.

5. Applications of artificial neural networks

Image recognition was one of the first areas to which neural networks were successfully
applied, but the technology uses have expanded too many more areas, including:

 Chatbots
 Natural language processing, translation and language generation
 Stock market prediction
 Delivery driver route planning and optimization
 Drug discovery and development
III. Bayesian Network:
1. Introduction and définition :
Bayesian networks are a graphical approach to modeling, using probability. In a network,
nodes are used to represent variables, and links to indicate that one node influences
another. This allows the relationship between variables to be visualized easily. Each node in
a Bayesian network requires a probability distribution to be specified (conditional on its
parents), and Bayes Server uses advanced algorithms to combine these distributions, in
order to answer queries (questions/predictions).

Bayes Server is a tool for modeling Bayesian networks, Dynamic Bayesian networks and
Decision graphs.

Bayesian networks are widely used in the fields of Artificial Intelligence, Machine Learning,
Data Science, Big data, and Time Series Analysis.

A Bayesian network is a type of graph called a Directed Acyclic Graph or DAG. A Dag is a
graph with directed links and one which contains no directed cycles. A directed cycle in a
graph is a path starting and ending at the same node where the path taken can only be along
the direction of links.

 Notation:

It is useful to introduce some simple mathematical notation for variables and probability
distributions.

Variables are represented with upper-case letters (A,B,C) and their values with lower-case
letters (a,b,c). If A = a we say that A has been instantiated.

A set of variables is denoted by a bold upper-case letter (X), and a particular instantiation by
a bold lower-case letter (x). For example if X represents the variables A,B,C then x is the
instantiation a,b,c. The number of variables in X is denoted |X|. The number of possible
states of a discrete variable A is denoted |A|.

The notation pa(X) is used to refer to the parents of X in a graph

We use P(A) to denote the probability of A.

We use P(A,B) to denote the joint probability of A and B.

We use P(A | B) to denote the conditional probability of A given B.

  Probability

P(A) is used to denote the probability of A. For example if A is discrete with states {True,
False} then P(A) might equal [0.2, 0.8]. I.e. 20% chance of being True, 80% chance of being
False.

 Joint probability

A joint probability refers to the probability of more than one variable occurring together,
such as the probability of A and B, denoted P(A,B).

NB: If two variables are independent (i.e. unrelated) then P(A,B) = P(A)P(B).

 Conditional probability

Conditional probability is the probability of a variable (or set of variables) given another
variable (or set of variables), denoted P(A|B).

 Marginal probability

A marginal probability is a distribution formed by calculating the subset of a larger

probability distribution.

 Probabilistic Graphical Models:

A probabilistic graphical model (PGM), or simply “graphical model” for short, is a way of
representing a probabilistic model with a graph structure.

The nodes in the graph represent random variables and the edges that connect the nodes
represent the relationships between the random variables.

 Nodes: Random variables in a graphical model.

 Edges: Relationships between random variables in a graphical model.

There are many different types of graphical models, although the two most commonly
described are the Hidden Markov Model and the Bayesian Network.

The Hidden Markov Model (HMM) is a graphical model where the edges of the graph are
undirected, meaning the graph contains cycles. Bayesian Networks are more restrictive,
where the edges of the graph are directed, meaning they can only be navigated in one
direction. This means that cycles are not possible, and the structure can be more generally
referred to as a directed acyclic graph (DAG).
 2. How to Develop and Use a Bayesian Network :
Designing a Bayesian Network requires defining at least three things:

 Random Variables. What are the random variables in the problem?

 Conditional Relationships. What are the conditional relationships between the
variables?
 Probability Distributions. What are the probability distributions for each variable?

It may be possible for an expert in the problem domain to specify some or all of these
aspects in the design of the model.

In many cases, an expert can specify the architecture or topology of the graphical model, but
the probability distributions must be estimated from data from the domain.

Both the probability distributions and the graph structure itself can be estimated from data,
although it can be a challenging process. As such, it is common to use learning algorithms for
this purpose; for example, assuming a Gaussian distribution for continuous random variables
gradient ascent for estimating the distribution parameters.

Once a Bayesian Network has been prepared for a domain, it can be used for reasoning, e.g.
making decisions.

Reasoning is achieved via inference with the model for a given situation. For example, the
outcome for some events is known and plugged into the random variables. The model can
be used to estimate the probability of causes for the events or possible further outcomes.

Practical examples of using Bayesian Networks in practice include medicine (symptoms and
diseases), bioinformatics (traits and genes), and speech recognition (utterances and time).

3. Practical example of a Bayesian Network:

We can make Bayesian Networks concrete with a small example.

Consider a problem with three random variables: A, B, and C. A is dependent upon B, and C
is dependent upon B.

We can state the conditional dependencies as follows:

 A is conditionally dependent upon B, e.g. P(A|B)

 C is conditionally dependent upon B, e.g. P(C|B)
We know that C and A have no effect on each other.

We can also state the conditional independencies as follows:

 A is conditionally independent from C: P(A|B, C)

 C is conditionally independent from A: P(C|B, A)

Notice that the conditional dependence is stated in the presence of the conditional
independence. That is, A is conditionally independent of C, or A is conditionally dependent
upon B in the presence of C.

We might also state the conditional independence of A given C as the conditional

dependence of A given B, as A is unaffected by C and can be calculated from A given B alone.

 P(A|C, B) = P(A|B)

We can see that B is unaffected by A and C and has no parents; we can simply state the
conditional independence of B from A and C as P(B, P(A|B), P(C|B)) or P(B).

We can also write the joint probability of A and C given B or conditioned on B as the product
of two conditional probabilities; for example:

 P(A, C | B) = P(A|B) * P(C|B)

The model summarizes the joint probability of P(A, B, C), calculated as:

 P(A, B, C) = P(A|B) * P(C|B) * P(B)

We can draw the graph as follows:

Notice that the random variables are each assigned a node, and the conditional probabilities
are stated as directed connections between the nodes. Also notice that it is not possible to
navigate the graph in a cycle, e.g. no loops are possible when navigating from node to node
via the edges.

Also notice that the graph is useful even at this point where we don’t know the probability
distributions for the variables.

You might want to extend this example by using contrived probabilities for discrete events
for each random variable and practice some simple inference for different scenarios.

4. Typical use of Bayesian networks :

 To model and explain a domain.

 To update beliefs about states of certain variables when some other variables were

observed, i.e., computing conditional probability distributions, e.g., P(X23|X17 =

yes,X54 = no).

 To find most probable configurations of variables

 To support decision making under uncertainty

 To find good strategies for solving tasks in a domain with uncertainty.