6/22/2014

Dynamic Programming
Module 4 – Stochastic 2

Programming Models A group of geologists have been sent to investigate the cause of massive landslides in Northern Luzon.
The trip would require travel on foot on the mountain slopes to gather soil samples. Three (3) types of
geological kits are to be used for purposes of gathering soil specimen. Each geologist must carry a
knapsack loaded with at least one kit but no more than 4 kits of the same type. In general, the more
kits of the same type, the better, since a greater quantity of samples can be gathered. Because these
SA MPLE PROB LE MS kits are heavy and would have to be carried for a long period of time, it is recommended that each
geologist carry only a limited load and the total weight of the kits in each knapsack must not exceed 10
kilograms. Data on each type of kit are as follows:

Value of Specimen/No. of Kits

Type of Kit Weight of Kit (k)
1 2 3 4
A 1.5 3 6 8 9
B 2.5 6 7 8 9
C 1 4 7 9 10

How many of each type of kit should a geologist carry to maximize the value of soil specimen
gathered? Solve using backward recursive approach.

IE198D Quarter 2 SY2012-13

Backward Recursive Approach Stage 3: Kit C

3 4

Obj. Max f(X) = Σri (xi )

Stage 3: Kit C Obj. Max f3(x3) = r3(x3)
where: ri (xi ) = value of specimen gathered using kit i
x3 Max Opt
xi = no. of kit i loaded 1<xi <4
n = 3 kit types (Stages) Σxi wi = 10
S3 1 2 3 4 f3 x3
wi = wt. of kit type i 1 4 4 1
Si = wt. of kits available to be loaded prior to kit i
2 7 7 2
3 9 9 3
Recursive Equation (Backward): f(X) = ri (xi ) + fi+1(Si -wi xi )
4 10 10 4

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

Stage 2: Kit B Stage 1: Kit A

5 6

Stage 2: Kit B Obj. Max. f2(S) = r2(x2) + f3(S2-2.5x2) Stage 1: Kit A Obj. Max. f1(S) = r1(x1) + f2(S1-1.5x1)
x2 Max Opt x1 Max Opt
S1 1 2 3 4 f1 x1
S2 1 2 3 4 f2 x2
10 19 22 21 19 22 2
3.5 10 10 1
4 10 10 1
5 13 13 1 Opt. f(X) = 22
6 15 11 15 1 Loading Kit No. Wt. Total Wt. Value
A 2 1.5 3 6
7 16 14 16 1
B 1 2.5 2.5 6
8 16 16 16 1,2 C 4 1 4 10
8.5 16 16 12 16 1,2 Total wt. of kits in knapsack = 9.5 22

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

1
 6/22/2014

Dynamic Programming Backward Recursive Approach

7 8
The VP for Finance of a company has P50 million budget for capital investment to allocate among
3 departments. Each dept. was asked to submit new project proposals together with their
corresponding cost and revenue data. Based on these proposals, the VP constructed the following
table. The departments did not submit the same number of projects. The first alternative
(project) for each dept. represents a decision not to select a proposal from that dept. Projects
Let: xi = Project in Division i
across depts. are independent. To avoid any morale problem in the depts., no more than one
project may be selected from any dept. Determine how the P50 million budget has to be allocated
among the 3 depts. to maximize the NPV of benefits using backward approach. Write down the
ci = Capital outlay for xi
recursive equation in general and for each of the stages under consideration. Also define the
symbols used in the analysis.
In Millions of Pesos
ri = NPV of benefits of xi
Dept. A Projects
Capital
Outlay
NPV of Future
Cashflow
NPV of Benefits
Net of Outlay Si = remaining budget prior to stage i
A1 0 0 0
A2 10 30 20 fi = cumulative NPV of benefits generated
A3 20 40 20
A4
Dept. B Projects
30 50 20 from stage 1 thru i = ri + fi+1
B1
B2
0
30
0
60
0
30 Si+1 = Si - ci
Dept. C Projects
C1 0 0 0
C2 10 20 10
C3 20 20 30

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

Stage 3: Division C Stage 2: Division B

9 10

Objective: Max f1 = r1 Objective: Max f2 = r2 + f3

x3 C1 (0/0) C2 C3 Opt. x3 Opt. f3 x2 B1 (0/0) B2 (30,30) Opt. x2 Opt. f2

(10,10) (20/30)
S3 S2
0 0 - - C1 0 20 0+30 - B1 30
10 0 10 - C2 10
30 0+30 30+0 B1 or B2 30
20 0 10 30 C3 30
30 0 10 30 C3 30 40 0+30 30+10 B2 40
40 0 10 30 C3 30
50 0+30 30+30 B2 60
50 0 10 30 C3 30

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

Stage 1: Division A Decision Theory

11 12

Objective: Max f1 = r1 + f2 The Athletic Dept. of Mapua is considering a fundraising campaign

A1 A2 A3 A4 Opt. x1 Opt. f1 next year for a new gym. The response to the campaign is heavily
X1 dependent on the success of its basketball team in the year’s NCAA
(0/0) (10,20) (20/20) (30/20) championship. In the past, the team has had a winning season 60% of
the time. If the team has a winning season, many of the alumni will
contribute and the campaign will raise P3 million. If the team has a
S1 losing season, the campaign will lose P2 million. If no campaign is
undertaken, no costs are incurred. Mapua could also hire a basketball
analyst and statistician, Dr. J, to help evaluate whether its team could
have a winning season. Dr. J’s services cost P100,000 and involves
50 0+60 20+40 20+30 20+30 A1 or 60 observing the team during the off-season practice and the pre-NCAA
season tournaments. In the past, when evaluating teams with winning
A2 seasons 50% of the time, Dr. J’s predictions were correct 75% of the
time. Draw the decision tree and determine the optimal policy
Optimum Allocation: A1, B2, C2 or A1, B2, C3 regarding whether to hire Dr. J and whether to undertake the
campaign.
Total Outlay = P50 million; Net NPV = P60 million

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

2
 6/22/2014

Decision Theory
13 14

X Co. has the opportunity to bid on a government contract for 100,000 items to
be used in a top-secret military project. They estimated that these items could
be manufactured by their existing eqpt. at a cost of P125.00 per unit. However,
one of their engineers has suggested a new process for manufacturing the
items. The unit cost estimates for the new process are only P75.00 if all goes
well and P95.00 if there are minor complications. If major complications arise,
the costs could be prohibitive so they would have to return to the old process.
The engineers estimate the probability of minor complications at 0.5, major
complications at 0.2 and no complications at 0.3. The investment required for
the new process is P1M, which would not be recoverable. The company must
make its bid on the contract before the new process can be tested. The various
bids under consideration and the estimated probability of obtaining the
contract are presented below. What course of action do you recommend to the
company?

Bid Price P170 P140 P120

Probability of getting contract 0.2 0.6 0.9

IE198D Quarter
Quarter 12SY2011-12
SY2012-13 IE198D Quarter 2 SY2012-13

Game Theory Answer:

15 16
Shell and Caltex operate a gasoline station and auto supply shop along the same highway of
a town. Both firms are well-established and have excellent reputations for providing quality Payoff Matrix : Payoff expressed as Increase in Market Share (Reference – Caltex)
service at reasonable prices. Presently, Shell has 55% of the local market which is being
attributed to its slightly better business location. Shell
A B C Row Min.
A -3 -3 -4 -4
A new radio station is preparing to begin operations in the town. The station’s manager
offered Shell a choice of 3 different advertising packages (A, B and C). Shell is aware that Caltex B 4 0 -3 -3
the station had also offered Caltex a similar package. Shell knows that its business will be C -1 -1 -2 -2
affected by its choice as well as the choice of Caltex. Col. Max. 4 0 -2 Saddlepoint
Shell discussed the situation with its marketing team. The team estimated that if it chooses = -2
A, its market share will increase by 3% if Caltex also chooses A. However, its market share
will decrease by 4% if Caltex chooses B. Market share will increase by 1% if Caltex chooses
C. Payoff Matrix : Payoff expressed as Increase in Market Share (Reference – Shell)

If Shell chooses B, its market share will increase by 3%, will not change or will increase by Caltex
1% depending on whether Caltex chooses A, B or C, respectively. If Shell chooses C, its A B C Row Min.
market share will increase by 4%, by 3% or by 2%, depending on Caltex’s choice. A 3 -4 1 -4
Shell B 3 0 1 0
C 4 3 2 2
Formulate the Payoff Matrix for this Game. Determine the optimum advertising strategy for
Shell and Caltex. What is the game result? Col. Max. 4 3 2 Saddlepoint
=2

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

Game Theory Markov Analysis

17 18

Solve the following game You are planning a vacation to Australia in June. The weather
advisory you got from the travel agency states that June is a rainy
Column month and there are never 2 sunny days in a row. The historical
pattern for weather conditions indicate that if today is sunny, the
Row C1 C2 C3 C4 C5 following day will have either rain or snow with equal
R1 0 -6 -8 -3 -2 probability. When there is rain or snow today, there is a 50%
chance that the following day will have the same weather and an
R2 -8 4 -2 14 -16 equal chance of the two other possibilities. Formulate the matrix
R3 4 -10 -8 2 -1 of transition probability for this Markov chain. If you are leaving
in 5 days for a 5-day vacation, what would be the daily weather
R4 6 4 -4 0 12 forecast during your vacation if the weather today in your
destination was sunny?

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

3
 6/22/2014

Answer Markov Analysis: Equilibrium Condition

19 20

The Director of the Bilibid Prisons has gathered data on the movement of people who
have been convicted and admitted to its facility. The following table summarizes the
probabilities of various annual movements. Given a pool of 10,000 persons who have
been admitted to the prison, 2,000 are currently in high security; 6,000 are in medium
security; 700 are on work release; 300 are living freely inside the compound and the rest
have been released. It has also been observed that some prisoners released from the
system re-enter the facility. In the long-run, how many persons are expected to be in
each of the jail classifications?

From/To High Security Med. Security Work Release Free Living Released
High Security 0.70 0.30 - - -
Med. Security 0.10 0.60 0.15 0.05 0.10
Work Release - 0.15 0.40 0.20 0.25
Free Living - 0.05 0.10 0.40 0.45
Released 0.02 0.08 - 0.10 0.80

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

Analysis of Absorbing Markov States Answer:

21 22

The Kidney Center classifies its patients into 4 types: those in dialysis (D);
those who have lived more than a year with a transplanted kidney (T2);
those who are in their first year with a transplanted kidney from a living Kidney Center A Death
donor (LT); those who are in their first year with a kidney from a dead
donor (DT). Over the years, the center has determined the annual F D 7.692308 0 0 0 D 0.13 0.333333
movement of patients in these stages as follows: T2 5.128205 6.666667 0 0 T2 0.05 0.128205
Patient Type
D T2 LT DT Death
LT 5.15384 5 1 0 LT 0.06 0.153846
D 0.87 0 0 0 0.13 DT 5 4 0 1 DT 0.15 0.384615
T2 0.1 0.85 0 0 0.05 Death 0.39
Patient Type LT 0.19 0.75 0 0 0.06 F*A D 1
DT 0.25 0.6 0 0 0.15
Death 0 0 0 0 1
T2 1 Exp. Lifespan of DT =10 steps (years)
LT 1 % fr. D = 0.3333
What is the expected lifespan of a patient who has received a transplant DT 1
from a dead donor? What percentage of the patients who died comes from
those undergoing dialysis?

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

Markov: Analysis of Absorbing States Queuing Theory

23 24

If there are r absorbing states and t transient states, the transition matrix will
have the following canonical form
Company Z has a key work center composed of a single machine.
Jobs arrive at this work center according to a Poisson process at a
Abs. Non-Abs.
mean rate of 2/day. The processing time for each job has an
Abs. I O
exponential distribution with a mean of ¼ day. Because the jobs are
Non-Abs. A N
bulky, those not being worked on are stored in a room located some
distance from the work center that results to some delay due to
F (fundamental matrix) = (I-N)-1, exp. no. of times a process stays in each non-
abs. state before it is absorbed. transportation. The work center in-charge is proposing to clear
some space next to the machine to accommodate 3 waiting jobs.
Prob. Of Absorption = FA, probability that a non-abs. state will end up in a Jobs in excess will continue to be stored in the old room. What
particular abs. state in the future proportion of the time will this temporary storage space be
adequate? What will be the % utilization of the old room if the
temporary storage is implemented?

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

4
 6/22/2014

Answer Queuing Theory

25 26

Given: λ = 2/day
µ = 4/day
Temporary storage will not be adequate if there are
more than 4 in system.
Proportion of time that jobs are in temp. storage = p0
+ p1 + p2 + p3 + p4= 0.5 + (0.5*0.5) + (0.5*0.5*0.5)
+ (0.5*0.5*0.5*0.5) + (0.5*0.5*0.5*0.5*0.5) =
0.96875 or 96.875% (temp. storage is adequate)
% utilization of old room = p(n>4) = 1 –
(po+p1+p2+p3+p4) = 1 – 0.96875 = 0.03125 or
3.125%
IE198D Quarter 2 SY2012-13
An airline reservation system has six telephone lines manned by 4
customer relations personnel. Incoming phone calls are believed to
be Poisson with a rate of 15/hour. The time to service a caller is
exponential with a mean of 8 minutes. If a call comes in and all the
personnel are busy, then the call is automatically placed on hold
using the remaining lines, providing a line is available; otherwise,
the call is lost. A busy signal means all lines are engaged. Callers
encountering a busy signal are assumed to place reservations with
another airline. It is also assumed that calls on hold never renege.
Is the system configuration adequate to ensure that the probability
of rejection is below 15%? If not, what recommendations can you
make regarding this system?
IE198D Quarter 2 SY2012-13

Problem Analysis Queuing Theory

27 28

Given: Two technicians, working independently, are assigned top monitor 5

Finite number of lines = N = 6 machines that automatically run a certain manufacturing process.
When a machine stops, it takes a technician 15 minutes to adjust it and
c = 4 (multiple servers) get it running again. The adjustment time is exponentially-distributed.
l = 15/hour The machine time is an average of 85 minutes, Poisson-distributed.
Determine the average number of machines that are non-operational at
m = 60/8 = 7.5/hour any time. How many of these are waiting for adjustment? Being
adjusted?
Probability of rejection = P that system is full = PN
Answer:
Ls = 0.795 (Ave. no. of non-operational machines)
System is adequate if PN < 0.15 Lq = 0.05 (Ave. no. of machines waiting for adjustment)
Ls-Lq = 0.743 (Ave. no. of machines being adjusted)

IE198D Quarter 2 SY2012-13 IE198D Quarter 2 SY2012-13

Queuing Theory Answer

29 30

Five cars arrive at a pollution testing center every hour. P of parking = 1 – [P0 + P1 + P2 + P3]
The average time for testing emissions is 6 minutes. The
center can accommodate only 3 cars at any time. Any car, P0 = 1 – (5/10) = 0.5
which cannot be accommodated in the center, takes a risk
P1 = Po (5/10)1 = 0.5(0.5) = 0.25
of parking on the road with a “No Parking” sign and
where there is a 40% chance of being fined P500. The P2 = Po (5/10)2 = 0.5(0.5)2 = 0.125
owner of the testing center pays the fines. The center P3 = Po (5/10)3 = 0.5(0.5)3 = 0.0.625
operates 48 hours a week. Cars which have been tested
are picked up immediately by the customers. Compute P of parking = 1 – (0.5+0.25+0.125+0.0625) = 0.0625 or 6.25%
the weekly fines that the center pays because of its limited
space.
Expected weekly fines = Pparking (l) Pfine (500)
= 0.0625(5)(0.4)(500) = P3,000

IE198D Quarter 2 SY2012-13 IE198D Quarter

Quarter 12SY2011-12
SY2012-13

5

Queuing Theory
31
Because of early dismissal from a class, you suddenly find
yourself with a 30-minute break. Since your next class is a
laboratory and expected to last for 4.5 hours, you are
contemplating of taking a snack in the canteen. You
estimate that if you can wait in line for food for no more
than 5 minutes, you can make it in time for your next class.
You believe that students taking a meal at this particular
time arrive at an average rate of 30/hour in a manner
which can be described by a Poisson distribution. The
canteen can serve 40 customers/hour with exponentially-
distributed service times. Given these data, should you take
a snack? Justify your answer with computations.
IE198D Quarter 2 SY2012-13
6/22/2014
Simulation
32
The manager of a farm cooperative is concerned about the yield he can expect from this year’s rice
crop. His cooperative has the following historical data of annual yields.
Yield (cavans/hectare) 120 140 160 180
Probability 0.18 0.26 0.44 0.12
The expected yield is 150 cavans/hectare; however, the manager would like to seea
simulation of expected yields in the next 10 years.
Develop the simulation model and simulate 10 years yield using the random digits below,
starting with the first set of numbers, moving from left to right. Two-digit random numbers
are to be sued to determine the yield. Start with the first 2 numbers and every 3rd pair of
numbers after that.
Random Digits
0190453442 4800088084 1165628559 5407921254 3768932478 4020651657
IE198D Quarter 2 SY2012-13