Application of Markov Process to Pavement
Management Systems at Network Level
Abbas A. Butt, Engineering & Research International
M . Y. Shahin, U.S. Army Construction Engineering Research Laboratory
Samuel H . Carpenter, University of Illinois
James V. Carnahan, University of Illinois
The rate of pavement deterioration is uncertain, and a pave ternatives, and section and network benefits. The results
ment management system (PMS) should portray this rate of from the two prioritization methodologies are compared
deterioration as uncertain. A wide variety of PMSs are used, through an actual implementation on an existing airfield
hut unfortunately either these systems do not use a formal pavement network. The prioritization using the incremental
ized procedure to determine the pavement condition rating, benefitlcost ratio program uses the available constrained
or they use deterministic pavement performance prediction budget to the best of the full limit. To maintain a specified net
models, or they assign the pavement state transition proba work PCI, the optimal benefitlcost ratio program will spend
bilities on the basis of experience. The objective of the re less money than the incremental benefitlcost ratio program.
search was to develop a probabilistic networklevel PMS on The developed optimization programs are very dynamic and
the basis of pavement performance prediction with use of the robust for networklevel PMSs.
Markov process. Pavements with similar characteristics are
grouped together to define the pavement families, and the
prediction models are developed at a family level. The pave
ment condition index (PCI), ranging from 0 to 100, is di
vided into 10 equal states. The results from the Markov
model are fed into the dynamic programming model and the
output from the dynamic programming is a list of optimal
T he major objectives of a networklevel pavement
management system (PMS) are to develop short
and longterm budget requirements and t o produce
a list of potential projects based on a limited hudget. The
optimum approach t o achieve these objectives relies
maintenance and repair (M&K) recommendations for each heavily on the prediction of pavement performance and
pavement familystate combination. If there are no con lifecycle cost analysis of all feasible maintenance and re
straints on the available hudget, the M&R recommendations habilitation (M&R) strategies. To find the optimal solu
from the dynamic programming will give a true, optimal tion for the allocation of available funds, operations
budget. However, because the budgets available are usually research techniques are used that may be either determin
less than the needs, two prioritization programs have been istic or probabilistic.
developed to allocate the constrained hudgets in an optimal Because the rate of pavement deterioration is uncer
way. The first prioritization program is hased on simple tain, the budget requirement developed at the network
ranking of the weighted optimal benefitlcost ratios, and the level should treat this rate of deterioration as uncertain.
second is based on the incremental benefitlcost ratio. The Modeling uncertainty requires the use of probabilistic op
output from the two programs is a list of sections to be re eration research techniques. Most of existing PMSs use
paired, type of M&R alternatives selected, cost of M&K al neither a formalized procedure t o determine the pavement
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).
3rd International Conference on Managing Pavements (1994)
160 THIRD INTERNATIONAL CONFERENCE O N MANAGING PAVEMENTS
condition rating nor a deterministic approach to model After filtering and outlier analysis, all the surveyed
the pavement rate of deterioration. PMSs that use proba pavement sections of a family are categorized into 1of the
bilistic prediction models such as Markov models mostly 10 states at a particular age. A pavement section is defined
assign the state transition probabilities on the basis of the as a part of the pavement network that has same type,
field staff's experience, which can affect the accuracy of structure, construction history, condition, use, and rank.
pavement performance prediction. An approach based on A pavement family is defined as a group of pavement sec
the Markov process has been developed for networklevel tions of similar characteristics. It is assumed that all the pave
optimization. Homogeneous and nonhomogeneous ment sections are in State 1 (PC1 of 90 to 100) at an age
Markov chains have been used in the development of of 0 years. Thus, the state vector in Duty Cycle 0 (age = 0)
pavement performance prediction models. The use of is given by (1,0, 0,0,0, 0,0,0,0, O), because it is known
Markov chains in prediction models captures the uncertain (with probability of 1.0) that all the pavement sections
behavior of pavement deterioration. Integration of the must lie in State 1 at an age of 0 years.
Markov chainsbased prediction models with the dy To model the way in which the pavement deteriorates
namic programming and the prioritization programs pro with time, it is necessary to establish a Markov probabil
duces a list of optimal M&R treatments and a budget that ity transition matrix. In this research, the assumption is
satisfies the given performance standards. Conversely, a made that the pavement condition will not drop by more
list of potential projects can be generated so that a limited than one state (10 PC1 points) in a single year. Thus, the
available budget is spent in an optimal way. pavement will either stay in its current state or transit to
the next lower state in 1year. Consequently, the probability
transition matrix has the form
I
Input: PC1 Vs. Age raw date and common characteristics
Prioritization Based on Optimal Benefit/Cost Ratio
DUTY CYCLE = 0
0 I 2 3 4 5 6
DUTY CYCLE (AGE IN YEARS)
FIGURE 2 Schematic representation of state, state vector, and duty cycle.
To estimate the transition matrix probabilities, a non The rate of deterioration is assumed to vary from one
linear programming approach is used. The objective of zone to another; therefore, different duty cycles have been
the search is to determine values of the nine parameters, assigned to different zones.
p i l ) through p(9), that would minimize the absolute dis Because the duty cycle within a zone is assumed to be
tance between the actual PCIversusage data points and constant, a homogeneous Markov chain has been used for
the expected (predicted) pavement condition for the cor each zone and a separate transition matrix has been de
responding age generated by the Markov chain using veloped for each zone. The duty cycle varies from one
these nine parameters. zone to another. Therefore, a nonhomogeneous Markov
The objective function has the following form: chain has been used for transition from one zone to another.
Minimize Figure 3 shows an example pavement condition predic
tion curve that uses a nonhomogeneous Markov model.
where
N = total
number of duty cycles (age) for
which PCIversusage data are available The probabilistic dynamic programming model for
within each family, networklevel optimization developed as a part of an
M(t) = total number of data points recorded at a
duty cycle (age) t,
Y(t, j ) = PC1 rating for each sample taken at a
duty cycle (age) t, and
E[X(t,p)l = expected value in PC1 at a duty cycle
(age) t, as predicted by current Markov
values.
To allow for changes in traffic loads and maintenance
policies over the pavement life, different duty cycles have
been introduced to create a nonhomogeneous Markov
model. A scheme has been developed in which the life of
the pavement is divided into zones. It is assumed that each AGE ( Y E A R S )
zone has a constant rate of deterioration and, hence, that FIGURE 3 Pavement condition prediction
a constant duty cycle has been assumed within each zone. curve using Markov model.
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).
3rd International Conference on Managing Pavements (1994)
earlier research has been described by Feighan et al. (2). optimal benefitlcost ratio uses the optimal M&R recom
The Markov transition probabilities generated from the mendations and the corresponding benefitlcost ratios for
Markov prediction model are used in the probabilistic dy each familystate combination produced from the dy
namic programming model. The objective function of dy namic programming. On the basis of the userdefined
namic programming is based on minimization of the weighting factors for each pavement section, all of the
M&R cost for the network. The dynamic programming pavement sections in the given network are ranked with
model has been further modified to produce benefits and the use of weighted optimal benefitlcost ratios. The higher
costs of ail feasible M&R alternatives for every family the weighted optimal benefitlcost ratio of a section is, the
state combination for every year of the analysis period. higher the priority of that section will be for repair. The
The output from the dynamic programming consists of available budget is allocated to the pavement sections by
the following: selection of one section at a time from the ranked section
list. The search for the section selection is stopped when
1. Optimal M&R alternatives for every year (stage) the available budget is completely exhausted. The do
for every familystate combination. nothing or routine maintenance is done for the sections
2. Present worth costs that correspond to the optimal that do not receive major rehabilitation.
M&R alternatives for every year (stage) for every family
state combination.
3. The optimal benefitlcost ratios that result from fol
Prioritization Using Incremental Benefit/Cost Ratio
lowing the optimal decisions, calculated for every family
state combination. The benefit is defined as the area under
The incremental benefiticost ratio technique is a heuristic
the PCIversusage curve over 1year. The midpoint of each
method for budget optimization. This technique is used to
state is used to represent the benefit obtained in 1 year.
maximize benefits from limited M&R funds for one pave
4. Benefits and present worth costs for all feasible al
ment section at a time (projectlevel optimization) or for
ternatives for every familystate combination for every
a group of pavement sections to maximize the overall ben
budget year.
efits (networklevel optimization) (4).
5. Optimal M&R recommendations and the corre
The output from this program is a list of sections to be
sponding present worth costs, benefits, and benefitlcost
repaired, type of M&R alternative selected, cost of M&R
ratios for Years 1, 2, 3, 4, 5, 10, 15, and 20, for every
alternative, section benefit, and total network benefits.
pavement family and for every state above or equal to the
The program also lists the section PC1 before and after the
minimum allowable state.
M&R application and network PC1 weighted and un
The application of the optimal M&R recommendation weighted by section area.
from the dynamic programming will produce a true opti The overall flowchart for the prioritization algorithm
mal M&R budget for the given constraints. However, it is with the use of the incremental benefitlcost ratio tech
possible that the available budget is less than the true op nique is shown in Figure 4. The prioritization process is
timal budget. Therefore, prioritization programs have composed of five main modules:
been developed to spend the limited available budget in an
optimal way. These programs are described in the follow 1. Benefit computation,
ing section. 2. Cost computation,
3. Routine maintenance,
4. Budget optimization, and
5. PC1 adjustment.
To achieve the maximum benefit from the limited available Figure 4 also shows the input data required for this pri
budget, two prioritization methodologies have been de oritization algorithm. A detailed description of each of
veloped. The prioritization methodology based on opti the five prioritization modules is given next.
mal benefiticost ratio was developed as a part of earlier
research (31, and the prioritization methodology based on
incremental benefiticost ratio has been developed as a Benefit Computation Module
part of this research program.
The flow chart for the benefit computation module is
shown in Figure 5. Each section in the network is first
Prioritization Using Optimal BenefitICost Ratio identified on the basis of section characteristics and as
signed pavement family. The section state is determined
A detailed description of this methodology is given by from the PC1 value of the section. On the basis of the sec
Feighan et al. (3).The prioritization method based on the tion's familystate assignment, the benefits of all feasible
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).
3rd International Conference on Managing Pavements (1994)
164 T H I R D INTEKNATlOPiAL CONFERENCE O N MANAGING PAVEMERTS
INPUTS
1. Feasible M & R alternatives from dynamic programming,
2. List of sections to be examined. Relevant information should include:
(i) Branch/section identification.

(iii) Branch Use.
(iv) Pavement Rank.
( v ) Primary cause of pavement distress.
I
ROUTINE MAINTENANCE MODULE
I
BUDGET OPTIMIZATION MODULE
FIGURE 4 Prioritization using incremental benefidcost ratio.
M&R alternatives are obtained that correspond to this Cost Computation Module
familystate combination. They are multiplied by the user
defined weighting factors to obtain the weighted benefits The flow chart for the cost computation module is shown
for all feasible M&R alternatives of the section. Similarly, in Figure 6. On the basis of each section's familystate as
the weightedsection benefits for all the sections are cal signment, the presentworth costs and initial costs of all
culated and stored for use in the budget optimization feasible M&R alternatives that correspond to the family
state combination are obtained. The initial costs of all fea
module.
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).
3rd International Conference on Managing Pavements (1994)
I
FIND BENEFITS OF ALL FEASIBLE M & R OPTIONS
CORRESPONDING TO THIS COMBINATION.
II
FIND THE WEIGHTING FACTORS RELATED TO THE
il SECTION CHARACTERISTICS.
/
MULTIPLY RESULT BY BENEFITS OF ALL FEASIBLE
M & R OPTIONS.
I1
sihle M&R alternatives of a section are multiplied by the the corresponding inflated initial costs, presentworth
section area and inflation rate, and the inflated initial costs, and weighted benefits are ohtained from the benefit
costs of each pavement section are stored for use in the computation module and the cost computation module.
budget optimization module. The available budget is ohtained from the routine mainte
nance module. This information is used in the incremental
Routine Maintenance Module benefidcost ratio program to produce optimal M&R rec
ommendation for each pavement section, including initial
The routine maintenance module shown in Figure 7 is the cost and type of treatment. The budget optimization mod
same as the one used for the prioritization program that ule also gives the total networkweighted benefits corre
uses optimal benefidcost ratio. The output from the rou sponding to optimal M&R recommendations.
tine maintenance module is directly used in the budget op
timization module. PC1 Adjustment Module
Budget Optimization Module The PC1 adjustment module is shown in Figure 9. This
module recomputes the PC1 values for each section when
The flow chart for the budget optimization module is the recommended M&R alternative is performed on the
shown in Figure 8. In the budget optimization module, all section. The pavement family curves developed from
feasible M&R alternatives of a section are identified, and Markov output data are used for predicting the PC1 val
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).
3rd International Conference on Managing Pavements (1994)
166 T H I R D INTERNATIONAL CONFERENCE O N M A N A G I N G PAVEMENTS
u FOR EACH SECTION
COSTS FOR ALL FEASIBLE M & R OPTIONS BY
MULTIPLYING WITH SECTION AREA. 1
I
STORE PRESENT WORTH COSTS AND INITIAL COSTS
1 FOR ALL FEASIBLE M & R OPTIONS IN THE PROGRAMMED
rnR.
ues, because the comparison of the Markov prediction lies from 22 airports. Table 1 presents the Markov transi
model results with constrained leastsquares model tion probabilities for each pavement family.
showed similar trends.
Application o f Dynamic Programming
FIND AVAILABLE BUDGET FOR NONROUTINE MAINTENANCE
FOR THE PROGRAMMED YEAR.
I
I
FOR EVERY SECTION
It
FIND PRESENT WORTH COSTS AND INITIAL COSTS FOR ALL
I
GET INCREMENTAL BENEFIT/COST RATIO PROGRAM.
FOR EVERY SECTION I
IF RECOMMENDED NONROUTINE TREATMENT EXCLUDING
SURFACE TREATMENT IS PERFORMED ON SECTION,
ASSUME SECTION GOES TO PCI=100 IN NEW FAMILY.
NEW FAMILY IS DETERMINED BY TRANSFORMATION MATRIX.
PC1 IS RAISED BY 10 PC1 POINTS. NEW FAMILY IS
DETERMINED BY TRANSFORMATION MATRIX.
2 14 0.9900 0.9900 0.9900 0.9900 0.0944 0.9681 0.9669 0.9669 0.9669 1.0000
RUNC 1 5 0.7000 0.7000 0.9900 0.9424 0.7441 0.6255 0.5620 05376 0.5322 1.0000
2 18 0.0010 0.6999 0.4620 0.9900 0.8547 0.8351 0.7478 0.6269 0.8862 l.W
RUNEND 1 11 0.8647 0.8992 0.8975 0.8964 0.4898 0.4857 05577 0.4752 0.2344 1.0000
3 18 0.9000 0.9000 0.9900 0.9900 0.9900 0.9127 0.8966 0.8931 0.8925 1.W
APRAC 1 6 0.6000 0.6000 0.7739 0.4853 0.5343 0.5498 0.5025 05002 0.9620 l.W
2 16 0.9900 0.9900 0.9900 0.9900 0.9900 0.99M) 0.9900 0.9613 0.7733 1.W00
APRPCC 1 14 0.6635 0.9628 0.9013 0.9011 0.2129 0.6280 0.6435 0.6502 0.6527 1.0000
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).
3rd International Conference on Managing Pavements (1994)
6. Minimum allowable state for each family: five for 4. Benefits and costs of all feasible M&R alternatives,
Families 1through 13. and
7. State benefits: the benefit is defined as the area un 5. Optimal M&R recommendations and the corre
der the PCIversusage curve over 1year. The midpoint of sponding presentworth costs, benefits, and benefitlcost
each state was used to represent the benefit over 1year. State ratio in Years 1, 2, 3, 4, 5, 10, 15, and 20 for pavement
benefits used in this analysis are given in Table 2. states equal to or less than 5.
8. Markov transition probabilities for each family:
Markov transition probabilities given in Table 1 were The data in Elements 1through 4 listed previously are di
used in the analysis. rectly used in the prioritization programs.
9. Transformationmatrix: transformation matrix defines
the new pavement family to move to if a certain M&R ac
tion is taken.
10. M&R Cost: PCIversusM&R cost relationships Prioritization
were used to calculate M&R cost of application of each
of three maintenance options to each pavement family Two computer programs have been written for priori
state combination. tization;
1. Prioritization using optimal benefidcost ratio, and
Dynamic Programming Output 2. Prioritization using incremental benefiticost ratio.
The output from dynamic programming for every family Both programs were used to develop a 5year M&R plan
state combination consists of for the airport.
1. Optimal M&R recommendations in every year,
2. Presentworth cost of optimal M&R recommenda Prioritization Using Optimal
tions, Benefit/Cost Ratio
3. Benefidcost ratio of optimal M&R recommenda
tions, Five budget scenarios were considered for the 5year
analysis period; the scenarios are given in Table 3. Budget
Scenario 1 had available budgets of $5 million, $4 mil
lion, $3 million, $2 million, and $1 million, respectively
TABLE 2 State Benefits Used in for the programmed Years 1 through 5. The reason that a
Dynamic Programming very high budget was selected for the first year of the
analysis period was that most of the sections at the airport
require major rehabilitation during the first year of the
analysis period. Another reason that higher available bud
gets were selected for the remaining years of the analysis
period was to determine the budget required if no bud
getary constraints are applied. Budget Scenarios 2,3, and
4 had uniform available budgets of $1.5 million, $1.0 mil
lion, and $500,000, respectively, for every year of the
analysis period. Budget Scenario 5 had $4.5 million avail
able for the first year so that all major M&R requirements
are satisfied and then a uniform budget of $100,000 for
the remaining years of the analysis period. The effect of dif
ferent budget scenarios on network PC1 is shown graphi
cally in Figure 10.
The curves of Budget Scenarios 1 and 5 are almost
identical because both scenarios have enough money allo
cated during the first year that all optimal M&R require
ments identified by the dynamic programming are
satisfied. Budget Scenarios 2 , 3 , and 4 have uniform bud
gets allocated over the 5 years of the analysis period. Bnd
get Scenario 4 shows a decrease in network PC1 with time.
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).
3rd International Conference on Managing Pavements (1994)
170 T H I R D I N T E R N A T I O K A L C O N E E K E N C E O N M A N A G I K G PAVEMEKTS

 Bud. Scenario i
Bud scenario 4

)
Budget Year
Bud. Soenaiio 2
Bud. scenario 5
* Bud. Sceneria 3 The developed optimization scheme uses a formalized
pavement condition survey procedure and is dynamic and
FIGURE 10 Effect of differentbudget scenarios on network robust for networklevel PMS. The pavement perfor
mance prediction model based on nonhomogeneous
PC1 using optimal benefitlcost ratio.
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).
3rd International Conference on Managing Pavements (1994)
 2 3 4

 Bud Seensiio 1
""d scenario 4 
A
Budget Year
Bud. SCenaiili 2
B u d scsnaiaa 5

Bud. Scenario 3 0 L
0
~
1
~ .....
2 3 4 5
T o t a l Budget Used (Millions $)
FIGURE 11 Effect of different budget scenarios on network
PC1 using incremental benefitlcost ratio. FIGURE 12 Network PC1 versus total budget used.
Markov chains successfully captures the probabilistic search effort will be incorporated in the MicroPAVER
pavement deterioration process. The Markov process in Version 5.
conjunction with the dynamic programming produces the
optimal budget requirements for the given analysis pe
REFERENCES
riod. The prioritization schemes have been developed to al
locate the constrained budget. The prioritization method
1. Butt, A.A., K.J. Feighan, M.Y. Shahin, and S.H. Carpenter.
using incremental benefitlcost ratio provides the best use
Pavement Performance Prediction Process Using the
of available limited funds, when the funds must be com Markov Process. In Transportation Research Record 1123,
pletely exhausted during the assigned year. However, if TRB, National Research Council, Washington, D.C., 1987.
the available funds can be carried over the next years, 2. Feighan, K.J., M.Y. Shahin, and K.C. Sinha. A Dynamic
then the optimal benefittcost ratio program provides the Programming Approach to Optimization for Pavement
best use of available limited funds. The findings of this re Management Systems. Proc. 2nd North American Confer
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).
3rd International Conference on Managing Pavements (1994)
172 T H I R D I N T E R K A T I O N A L C O N F E R E N C E O N M A N A G I N G PAVEMENTS
ence O n Managing Pavements, Toronto, Ontario, Canada, 4. Shahin, M.Y., S.D. Kohn, R.L. Lytton, and E. Japel. Deuel
Nov. 1987. opment of a Pavement Maintenance Management System,
3. Feighan, K.J., M.Y. Shahin, K.C. Sinha, and T.D. White. A Volume VIII, Development of an Airfield Pavement Main
Prioritization Scheme for the Micro PAVER Pavement Man tenonce and Repair Consequence System. Report ESLTR
agement System. In Transportation Research Record 1215, 8119. Engineering and Services Laboratory, Air Force
TRB, National Research Council, Washington, D.C., 1989. Engineering and Services Center, April 1981.
TRB Committee AFD10 on Pavement Management Systems is providing the information contained herein for use by individual practitioners in state and local
transportation agencies, researchers in academic institutions, and other members of the transportation research community. The information in this paper was
taken directly from the submission of the author(s).