Lesson Bayesian Networks

In this Lesson
• Bayesian network basics.

1
• Nodes and values.
• Structure.
• Build the structure of a Bayesian network with software.

Duration : 40 minutes

Introduction.
Judea Pearl in his book (1988) summarized very well, what was the product of many scientists work,
a new method of reasoning using probabilities. This new method that is variously known as Belief
Networks, Bayesian Networks, Knowledge Maps, Probabilistic Causal Networks and so on, has
become quite popular within the AI probability and uncertainty community. Today, Bayesian
Networks are used for modeling knowledge in medicine (Uebersax 2004), medical diagnosis
(Heckerman 1990; Spiegelhalter, Franklin and Bull 1989), information retrieval (Campos, Fernández-
Luna and Huete 2004), map learning (Dean 1990), language understanding (Charniak and Goldman
1989a, 1989b; Goldman 1990), decision support systems (D'iez, Mira, Iturralde and Zubillaga 1997),
heuristic search (Hansson and Mayer 1989) and so on. Despite their importance, the ideas and
techniques behind Bayesian Networks are not that easy to understand so these lessons will try to
make the basic ideas and intuitions accessible to someone with limited grounding in probability
theory. During the lessons specialized software, Belief and Decision Network Tooli, will be used.

Overview of Bayesian Networks.

A Bayesian or belief network is a directed acyclic graph (DAG) that shows conditional probability and
causality relationships between a set of random variables from a domain. Variables in a Bayesian
network are represented by nodes. Nodes have states, or a set of probable values for each variable.
A set of directed arcs (arrows), that are called edges, are connecting pairs of nodes to show causality
and direction of influence and dependency. Assuming discrete variables, the strength of the
relationship between variables is quantified by conditional probability distributions associated with
each node.
Although the process of building a Bayesian network in real world is not a simple task, here we will
try to present the steps that someone needs to undertake as a sequence. Throughout the remainder
of this section we will use the following simple medical diagnosis problem.
International Hellenic University 1
Paraskevopoulos Konstantinos
Bayesian Networks

Example problem: Sinus problems. A patient has been suffering from strong headaches so he visits
the doctor, worried that he has sinus infection. The doctor knows that headaches can be caused by
many other reasons such as lack of good sleep, or stressful way of living. The doctor also knows that
other relevant information includes whether or not the patient has flu or some allergy. A positive CT
scan will indicate either sinus problems or something else.

Nodes and values.

First task to do when building a Bayesian network is to identify the variables of interest. In other
words the nodes that represent the variables and the values that those nodes can take. All the
possible values must be at the same time mutually exclusive and exhaustive, or else the variables
must take on exactly one of these values at a time. Common values of nodes that can take discrete
values are nodes with Boolean values (True or False), nodes with ordered values (low, medium, high)
and nodes with integral values (values from 1 to 10).
We can start building the model for the sinus problems example, using the restricted set of values
and nodes shown in Table 1. This set of nodes and values already put some limits of what can be
represented in our network. For example there is no differentiation between a heavy or a light flu or
a strong and light headache. Also, for the sake of simplicity, there is a limit to the number of discrete
values that the nodes can take. All the nodes in out model can take only two values, but in general
there is no limit to the number of discrete values that nodes in more complex model can take.

Node name Type Values

Flu Boolean {True, False}
Allergy Boolean {True, False}
Sinus Boolean {True, False}
Tiredness Boolean {True, False}
Headache Boolean {True, False}
Table 1

Structure.
A Bayesian network is a model that represents not only the possible states of a given domain, but
also the probabilistic relationships among some of the states of the domain. The structure of a
Bayesian network should capture qualitative relationships between variables. Two nodes must be
connected directly if one affects the other, with the arc indicating the direction of the effect. In our

International Hellenic University 2

Paraskevopoulos Konstantinos
Bayesian Networks

sinus problems example, if the answer to the question “what are the factors that can cause sinus
problems?” is “Flu and allergy”, then we should add arcs from Flu and Allergy nodes to Sinus node. In
a similar way if the patient has Sinus then this will affect him in having Headache and feeling Tired, so
we should add arcs from Sinus node to Tiredness and Headache nodes. Bearing in mind that by
convention Bayesian networks are usually depicted upside down, a possible structure of this model
could be the one shown in Figure 1.

Figure 1

Trying to describe the topology of a Bayesian network, we say that a node is parent of a child, if there
is an arc from the former to the latter. We also say that one node is an ancestor of another if it
appears earlier in the chain, whereas a node is a descendant of another node if it comes later in the
chain. In our example, the Sinus node has two parents, Flu and Allergy, while Allergy is an ancestor of
both Tiredness and Headache. In a similar way, Tiredness is a child of Cancer and descendant of Flu
and Allergy. The set of parent nodes of a node X may also be denoted as Parents(X).
While working with Bayesian networks, you may also hear the term Markov blanket. The Markov
blanket for a node A in a Bayesian network is the set of nodes, which consists A's parents, its
children, and its children's other parents. In a Markov network, the Markov blanket of a node is its
set of neighboring nodes. A Markov blanket may also be denoted by MB(A). Terminology from the
tree networks is often used to describe the topology of a Bayesian network, although Bayesian
networks are graphs and not trees. Any node without parents is called root node, while any node
without children is called leaf node. If a node is not leaf or root is called an intermediate node. Root
nodes represent causes, while leaf nodes represent final effects. Trying to describe the model in
Figure 1, someone could say that causes Flu and Allergy are root nodes, while effects Tiredness and
Headache are leaf nodes.
Next is shown how the Bayesian tree presented in Figure 1 can be constructed using specialized
software from AIspace.
International Hellenic University 3
Paraskevopoulos Konstantinos
Bayesian Networks

Creating a New Belief Network with software.

To create a new Bayesian network with Belief and Decision tool, just do the following steps:
1. Start the application.
The main window should look like the one shown in Picture 1

Picture 1

Notice that the application started in “Create Node” mode. Later you will have to change
mode by depressing one of the buttons in the toolbar. Above the main white canvas there is
a text that will change according to the mode you are currently in.
2. Having the “Create Node” button pressed, click anywhere in the white canvas to open the
“Node Properties” dialog (Picture 2) and create a new node.

International Hellenic University 4

Paraskevopoulos Konstantinos
Bayesian Networks

Picture 2

3. In the “Node Properties” dialog, type a new name in the text box “Node name”. For the
purpose of our example type Flu.
4. In the Domain text box you can enter the domain values for the current node separated by
commas. For the purpose of our example leave this field with the default values T, F, as the
Flu node can only have to states True or False as it was shown in Table 1.
The Node Properties dialog should look like the one in Picture 3.

Picture 3

5. Click OK, to create the new node.

6. Do the same process as described in steps 2 to 4 to add the nodes for variables Allergy, Sinus,
Tiredness and Headache.
After completing this step your network should look like the one in Picture 4. Note that the
exact place of the nodes in your window may be different. Don’t worry, go to the next step.

International Hellenic University 5

Paraskevopoulos Konstantinos
Bayesian Networks

Picture 4

7. Click on the Select button in toolbar.

8. Select each node separately and drag it into a new position having in mind that the final
positions of the nodes should be close to the ones shown in Picture 5.

Picture 5

International Hellenic University 6

Paraskevopoulos Konstantinos
Bayesian Networks

9. During the next steps you will have to draw the arcs connecting the nodes. To do that click
on the Create Arc button in the toolbar to change the mode.
10. Click on node Flu to denote that this node is the starting node of the arc.
11. Click on node Sinus to denote that this node is the ending node of the arc.
Your model now should look like the one in Picture 6.

Picture 6

12. Repeat steps 10 and 11 to create arcs connecting nodes Allergy with Sinus, Sinus with
Tiredness and Sinus with Headache.
Your model now should look like the one shown in picture

Picture 7

Congratulations you just finished creating your first Bayesian network.

International Hellenic University 7

Paraskevopoulos Konstantinos
Bayesian Networks

Through the process you can delete or modify a node by clicking the buttons Delete and Set
Properties in the toolbar respectively,
13. Click menu File and select Save problem to save the model you just created with the name
Exercise_1.xml.

i
http://www.aispace.org/downloads.shtml

International Hellenic University 8

Paraskevopoulos Konstantinos