Professional Documents
Culture Documents
Markov Problem
Markov Problem
A
good car will good at the beginning of the next year with the probability 0.85; fair
with the probability 0.10; or broken-down quality with a probability of 0.05. A
fair car will be fair of the next year with the probability 0.70 or broken-down
quality with a probability of 0.30. It costs $6,000 to purchase a good car; a fair
car can be traded in for $2,000; a broken-down car has no trade-in value and must
immediately be replaced by a good car. It costs $1,000 per year to operate a good
car and $1,500 to operate a fair car. Should I replace my car as soon as it
becomes fair, or should I drive my = car until it breaks-down? Assume that the
cost of operating a car during a year depends of the type of car on hand at the
beginning of the year (after a new car, if any arrives).
I started with:
.85
P 0
0
and
.10
.7
0
.05
.3
1
1 0.851 0 2
2 0.101 0.7 2
3 0.501 0.3 2
1 2 3 1
3 1 1 2
1 1 2 0.51 .03 2
1.51 1
0 .3
1.51 1
1
0 .3
2