Universidad Nacional Abierta y a Distancia

Vicerrectoría Académica y de Investigación

Syllabus of the Course Theory of Decisions, Code 212066

Task 4 - Solving problems of optimization models with Markov chains

1. Description of the activity

Activity type: In group - Collaborative

Time of assessment: Intermediate
Maximum activity score: 100 points
The activity starts on: monday,
Activity ends on: sunday, april 25, 2021
april 5, 2021
This activity is expected to achieve the following learning outcomes:

Propose solution models using finite or infinite-stage Markov processes associated

with production and service problems.

Apply solution methods in decision problems using finite or infinite-stage Markov

processes to search for optimal results that generate the most efficient answers.

Evaluate the results of the problems using Markov processes of finite or infinite
stages for decision-making in problems of production and service processes.

The activity consists of:

In the Learning environment, consult the bibliographic reference of unit 3.

Optimization models with Markov chains:

 Sharma, J. (2016). Operations Research: Theory and Applications. (pp. 650-

665), New Delhi: Laxmi Publications Pvt Ltd, v. Sixth edition.

 Piunovskiy, A. (2012). Examples In Markov Decision Processes: Singapore:

Imperial College Press Optimization Series (pp. 111-127).

In exercise 1 using the steady state method, the student must find the transition
matrix to be able to determine the probabilities of an event and the efficiency of the
problem that allows him to make the decisions according to the Markov
considerations raised in it.

1
Exercise 1. Markov Chain (Steady State):

XYZ insurance company charges its customers according to their accident history. If
you have not had accidents the last two years will be charged for the new policy $
2,130,000 (state 0); if you have had an accident in each of the last two years you
will be charged $ 1,850,000 (State 1); If you had accidents the first of the last two
years you will be charged $ 1,917,000 (state 2), if you had an accident the second
of the last two years will be charged $ 1,870,000 (State 3) and are new clients will
be charged $789.000 (State 4). The historical behavior of each state is given by the
following cases of accident, taken in four different events.

ACCIDENTS IN THE YEAR

STATES E0 E1 E2 E3 E4
E0 4068 2838 2676 3180 4298
E1 3222 3119 3905 2646 2794
E2 2754 2623 4215 2878 3075
E3 3041 2484 2746 2436 3312
E4 3280 2498 3584 2382 2791
Table 1. Historical accident data

According to Table by applying the criteria of Markov:

1. Use the stable Markov initial method.

2. Find the transition matrix with the participation of each event according to its
total.
3. Find the values that satisfy the equations of state according to the conditions
of the problem.
4. Determine the probable value according to Markov theory and the mean prime
in the problem.

In Exercises 2, 3, and 4, find the value of probabilities of the indicated states

according to the succession of probabilities of the preceding states. Remember that
Markov chains are a succession of probabilities that depend on previous events that
condition you.

Exercise 2. Markov chains (Initial state multiplication):

In Colombia there are 6 main internet operators such as Tigo, Claro, Movistar, ETB,
Uff and Azteca which we will call states. The following chart summarizes the odds
that each client has to stay in their current operator or make a change of company.

2
The current percentages of each operator in the current market are for Tigo 0.20 for
Comcel 0.15, for Movistar 0.15, for ETB 0.2, 0.15 for Uff and 0.15 for Azteca (initial
state).

STATE TIGO CLARO MOVISTAR ETB UFF AZTECA

TIGO 0,15 0,17 0,16 0,13 0,15 0,24
CLARO 0,17 0,19 0,2 0,13 0,11 0,2
MOVISTAR 0,15 0,21 0,19 0,17 0,19 0,09
ETB 0,2 0 0,21 0,12 0,25 0,22
UFF 0,21 0,21 0,17 0,13 0,11 0,17
AZTECA 0,12 0,19 0,13 0,15 0,19 0,22

According to Table by applying the criteria of:

 According to the multiplication of matrices - vectors find state 5 of the problem

according to the initial state of probabilities.

 According to the multiplication of matrices - vectors, find state 9 of the problem

according to the initial state of probabilities, following the sequence of state 5.

Exercise 3. Markov chains (Initial state multiplication):

In Colombia there are 6 main mobile operators such as Avantel, Tigo, Comcel,
Movistar, ETB and Uff, which we will call states. The following chart summarizes the
odds that each client has to stay in their current operator or make a change of
company.

STATE AVANTEL TIGO COMCEL MOVISTAR ETB UFF

AVANTEL 0,19 0,18 0,17 0,15 0,19 0,12
TIGO 0,17 0,15 0,16 0,16 0,18 0,18
COMCEL 0,16 0,19 0,17 0,17 0,16 0,15
MOVISTAR 0,18 0,18 0,19 0,18 0,15 0,12
ETB 0,15 0,16 0,19 0,15 0,18 0,17
UFF 0,15 0,16 0,17 0,19 0,18 0,15

The current percentages of each operator in the current market are for Avantel 0.1,
Tigo 0.2 for Comcel 0.3, for Movistar 0.3, for ETB 0.1 and 0.1 for Uff (initial state,
state 0).

According to Table by applying the criteria of:

3
  According to the multiplication of matrices - vectors find state 7 of the problem
according to the initial state of probabilities.

 According to the multiplication of matrices - vectors, find state 9 of the problem

according to the initial state of probabilities, following the sequence of state 7.

 According to the multiplication of matrices - vectors, find state 12 of the

problem according to the initial state of probabilities, following the sequence of
state 9.

Exercise 4. Markov chains (Initial state multiplication):

Suppose that 4 types of soft drinks are obtained in the market: Colombian, Pepsi
Cola, Fanta and Coca Cola when a person has bought Colombian there is a probability
that they will continue to consume 30%, 20% of which will buy Pepsi Cola, 20% that
Fanta buys and 30% that Coca Cola consumes; when the buyer currently consumes
Pepsi Cola there is a probability that he will continue to buy 40%, 20% buy
Colombiana, 20% that Fanta consumes and 20% Coca Cola; if Fanta is currently
consumed, the likelihood of it continuing to be consumed is 35%, 25% buy
Colombian, 20% consume Pepsi Cola and 20% go to Coca Cola. If you currently
consume Coca Cola the probability that it will continue to consume is 42%, 22% buy
Colombian, 16% that consumes Pepsi Cola and 20% that is passed to Fanta.

At present, each Colombian brand, Pepsi Cola, Fanta and Coca Cola have the
following percentages in market share respectively (19%, 21%, 15% and 45%) state
2.

From the problem situation:

Exercise 1:

1. Use the stable Markov initial method.

2. Find the transition matrix with the participation of each event according to its
total.
3. According to the multiplication of matrices - vectors, find state 9 of the problem
according to the initial state of probabilities, following the sequence of state 2.
4. According to the multiplication of matrices - vectors, find state 12 of the
problem according to the initial state of probabilities, following the sequence
of state 9.

4
In a spreadsheet (Excel), propose the development and formulation of the cells to
apply the expected value method in the decision trees, design the decision trees and
calculate the expected values according to the conditions of the problem.

Exercises 2, 3 and 4:

1. According to the multiplication of matrices - vectors find states of the problem

according to the initial state of probabilities.

2. In Excel QM, find the solution to the linear programming problem.

Interpret the results of the expected value solution for decision making according to
the conditions of the problem. Individually, to present in the Discussion Forum
(Learning environment) the advances in the formulation, development, solution,
checking of results and interpretation of results of the problems in spreadsheet
(Excel) and Excel QM.

Collaboratively, the working group must select Exercises 1, 2, 3 or 4, among those

presented individually, to review and verify the procedure and application of the
mathematical models and algorithms required for the formulation, development,
solution, verification and interpretation of results to make the respective adjustments
and finally, proceed to the construction of the report in spreadsheet (Excel) in the
Discussion Forum (Learning environment) and deliver it collaboratively in the
Evaluation environment.

For the development of the activity, keep in mind that:

In the Learning environment, you should: Consult and review the agenda of the
course, consult the reference proposed in the required educational resources of Unit
3. Optimization models with Markov chains, present in the forum of the activity, the
contributions and progress on development and solving the problems raised above.

In the Evaluation environment, you must: Deliver the report individually according
to its specifications and availability scheduled in the Course Agenda.

5
Individual evidence:

The individual evidences to deliver are:

Documents in spreadsheet (Excel) with the development of:

Exercise 1. Markov Chain (Steady State):

1. Formulation of the problem as a model of Steady State.

2. Solution of the model and formulation Steady State according to the conditions
of each problem in spreadsheet (Excel) and Excel QM.
3. Interpretation of results of the expected value model solution for decision
making.

Exercises 2, 3 and 4. Markov Chain (Initial State):

1. Formulation of the problem as a model of Initial State method.

2. Solution of the model and formulation Initial State method. according to the
conditions of each problem in spreadsheet (Excel) and Excel QM.
3. Interpretation of results of the Initial State model solution for decision making.

Group evidence:

The group evidences to deliver are:

Document in spreadsheet (Excel) titled Solving problems of decision models under

uncertainties, which must contain the following information and presentation: Cover.
Identifying the activity, the title, the authors, the tutor, the course, the institution,
the school, the program and the year.
Content:

Exercises 1, 2, 3 or 4. Markov Chain:

1. Formulation of the problem.

2. Solution of the model in spreadsheet (Excel) and Excel QM.
3. Interpretation of results of the e model solution and decision making.

Bibliography (with APA standard).

File Extension: xls or pdf

Identification: Task 4. Name of the student.

6
 2. General guidelines for the preparation of the evidence to be
delivered.

For individually produced evidence, consider the following guidelines:

1. You must present the progress of the activity in the forum of the activity during
the time allocated for the activity.

2. Before delivering the requested product, you must check that it meets all the
requirements indicated in this activity guide.

Please note that all individual or group written products must comply with the
spelling standards and the defined presentation conditions. Regarding the use of
references, consider that the product of this activity must comply with APA
standards. In any case, comply with the reference standards and avoid academic
plagiarism, for this you can support yourself by reviewing your written products using
the Turnitin tool found on the virtual campus.

Consider that in agreement 029 of December 13, 2013, article 99, the following are
considered as offenses that violate the academic order, among others: literal e)
“Plagiarism, that is, presenting as your own authorship the all or part of a work,
work, document or invention made by another person. It also implies the use of
missing citations or references, or proposing citations where there is no coincidence
between it and the reference ”and liberal f)“ Reproducing, or copying for profit,
educational materials or results of research products, which have intellectual rights
reserved for the University ”

The academic sanctions that the student will face are the following:
a) In cases of academic fraud demonstrated in the academic work or respective
evaluation, the grade to be imposed will be zero points without prejudice to the
corresponding disciplinary sanction.
b) In cases related to plagiarism demonstrated in academic work, whatever its
nature, the grade to be imposed will be zero points, without prejudice to the
corresponding disciplinary sanction.

7
 3. Evaluation Rubric Format

Activity type: In group - Collaborative

Time of assessment: Intermediate
The maximum possible score is 80 points
High level: The student develops and presents all the
First evaluation activities proposed in the forum of the activity .
criterion: If your work is at this level you can get between 14
points and 15 points
Use the forum of the
activity to develop Medium Level: The student develops and presents some
and present Task 4. activities proposed in the forum of the activity .
If your work is at this level you can get between 2
This criterion points and 13 points
represents 15
points out of the Low level: The student does not develop and present the
total 100 points of activities proposed in the forum of the activity .
the activity. If your work is at this level you can get between 0
points and 1 points
High level: The student correctly analyzes the methods
Second evaluation
of Markov Chain (Steady State) to find the efficiency of
criterion:
the problem 1, correctly determining all the requested
aspects.
Use the methods of
If your work is at this level you can get between 19
Markov Chain
points and 20 points
(Steady State) to find
the efficiency of the
Medium Level: The student partially analyzes the
problem 1 according
methods of Markov Chain (Steady State) to find the
to its nature to make
efficiency of the problem 1, partially determining the
decisions in a process
requested aspects.
of production and
If your work is at this level you can get between 2
services.
points and 18 points
This criterion
Low level: The student does not apply the methods of
represents 20
Markov Chain (Steady State) to find the efficiency of the
points out of the
problem 1 in the development and solution.
total 100 points of
If your work is at this level you can get between 0
the activity.
points and 1 points

8
 High level: The student correctly analyzes the methods of
Third evaluation
Markov Chain (Initial State Multiplicative) to find the
criterion:
efficiency of the problem 2, correctly determining all the
requested aspects.
Use the methods of
If your work is at this level you can get between 9
Markov Chain (Initial
points and 10 points
State Multiplicative) to
find the efficiency of
Medium Level: The student partially analyzes the methods
the problem 2
of Markov Chain (Initial State Multiplicative) to find the
according to its nature
efficiency of the problem 2, partially determining the
to make decisions in a
requested aspects.
process of production
If your work is at this level you can get between 2
and services.
points and 8 points
This criterion
Low level: The student does not apply the methods of
represents 10
Markov Chain (Initial State Multiplicative) to find the
points out of the
efficiency of the problem 2 in the development and solution.
total 100 points of
If your work is at this level you can get between 0
the activity.
points and 1 points
High level: The student correctly analyzes the methods of
Fourth evaluation
Markov Chain (Initial State Multiplicative) to find the
criterion:
efficiency of the problem 3, correctly determining all the
requested aspects.
Use the methods of
If your work is at this level you can get between 9
Markov Chain (Initial
points and 10 points
State Multiplicative) to
find the efficiency of
Medium Level: The student partially analyzes the methods
the problem 3
of Markov Chain (Initial State Multiplicative) to find the
according to its nature
efficiency of the problem 3, partially determining the
to make decisions in a
requested aspects.
process of production
If your work is at this level you can get between 2
and services.
points and 8 points
This criterion
Low level: The student does not apply the methods of
represents 10
Markov Chain (Initial State Multiplicative) to find the
points out of the
efficiency of the problem 3 in the development and solution.
total 100 points of
If your work is at this level you can get between 0
the activity.
points and 1 points

9
Fifth evaluation High level: The student correctly analyzes the methods of
criterion: Markov Chain (Initial State Multiplicative) to find the
efficiency of the problem 4, correctly determining all the
Use the methods of requested aspects.
Markov Chain (Initial If your work is at this level you can get between 9
State Multiplicative) to points and 10 points
find the efficiency of
the problem 4 Medium Level: The student partially analyzes the methods
according to its nature of Markov Chain (Initial State Multiplicative) to find the
to make decisions in a efficiency of the problem 4, partially determining the
process of production requested aspects.
and services. If your work is at this level you can get between 2
points and 8 points
This criterion
represents 10 Low level: The student does not apply the methods of
points out of the Markov Chain (Initial State Multiplicative) to find the
total 100 points of efficiency of the problem 4 in the development and solution.
the activity. If your work is at this level you can get between 0
points and 1 points
High level: The student presents the report with the
development and correct solution of the problems posed,
with excellent writing, spelling, citations and bibliographic
references in accordance with the APA standard in its
Sixth evaluation content.
criterion: If your work is at this level you can get between 14
points and 15 points
Report Solution of
Markov Chain Medium Level: The student presents the report with the
problems. development and partial solution of the problems posed, with
writing, spelling, citations and / or bibliographic references in
This criterion accordance with the APA standard in its content.
represents 15 If your work is at this level you can get between 2
points out of the points and 13 points
total 100 points of
the activity. Low level: The student does not present the report with the
development and solution of the problems posed.
If your work is at this level you can get between 0
points and 1 points

10