CENG 6309*:

Road safety
Engineering
Ambo University Hachalu Hundessa
Institute of Technology
SCEE, Civil Engineering
Prof. Emer T. Quezon
C.Eng., M.ASCE, MSc., PhD
Dr.-Ing(hc)
Email: quezonet09@gmail.com
emer.tucay@aastu.edu.et
Webpage:
https://scholar.google.com/citations?user=I
6QHv7UAAAAJ&hl=en
 Chapter 8
Safety Effectiveness Evaluation
Definition and Purpose
 Safety effectiveness evaluation is the process of
developing quantitative estimates of how a treatment,
project, or a group of projects has affected crash
frequencies or severities.
 The effectiveness estimate for a project or treatment is
a valuable piece of information for future safety
decision-making and policy development.
 Safety effectiveness evaluation may include:
1. Evaluating a single project at a specific site to
document the safety effectiveness;
2. Evaluating a group of similar projects to document the
safety effectiveness of those projects;
3. Evaluating a group of similar projects for the specific
purpose of quantifying an AMF for a countermeasure;
and
4. Assessing the overall safety effectiveness of specific
types of projects or countermeasures in comparison to
their costs.
Definition and Purpose
5. Safety effectiveness evaluations may use several
different types of performance measures, such as:
1. A percentage reduction in crashes,
2. A shift in the proportions of crashes by collision
type or severity level,
3. An AMF for a treatment, or
4. A comparison of the safety benefits achieved to
the cost of a project or treatment.
Study Designs and Methods
3- basic study designs that are used for safety
effectiveness evaluations:
1. Observational before/after studies
2. Observational cross-sectional studies
3. Experimental before/after studies
 In observational studies, inferences are made from
data observations for treatments that have been
implemented by highway agencies, “not “
treatments that have been implemented.

 By contrast, experimental studies consider

treatments that have been implemented, so that
their effectiveness can be evaluated.
Observational Before/After Studies
 Observational before/after studies are the most
common approach used for safety effectiveness
evaluation.
 All observational before/after studies use crash
and traffic volume data for time periods before
and after improvement of the treated sites.
 The treatment sites do not need to have been
selected in a particular way; they are typically
sites of projects implemented by highway
agencies in the course of their normal efforts to
improve the operational and safety performance
of the highway system.
Observational Before/After Studies
 Ifan observational before/after evaluation is
conducted “without” any consideration of non-
treatment sites.
Observational Before/After Evaluation Studies Using
SPFs – the Empirical Bayes Method
 This approach to evaluation studies uses SPFs to
estimate what the average crash frequency at
the treated sites
 In cases where the treated sites were selected by
the highway agency for improvement because of
unusually high crash frequencies, this constitutes
a selection bias which could result in a high
regression-to-the-mean bias in the evaluation.
 The use of the EB approach, which can
compensate for regression-to-the–mean bias, is
particularly important in such cases.
Observational Cross-Sectional Studies
 There are many situations in which a before/after
evaluation, while desirable, is simply not feasible,
including the following examples:
1. When treatment installation dates are not available;
2. When crash and traffic volume data for the period prior
to treatment implementation are not available; or,
3. When the evaluation needs to explicitly account for
effects of roadway geometrics or other related features
by creating an AMF function, rather than a single value
for an AMF.
 For example, if an agency wants to compare the
safety performance of intersections “with”
channelized right-turn lanes to intersections “without”
channelized right-turn lanes, and no sites are
available that have been converted from one
configuration to the other, then an observational
cross-sectional study may be conducted comparing
sites with these two configurations.
Observational Cross-Sectional Studies
 Cross-sectional studies use statistical modeling
techniques that consider the crash experience of
sites “with” and “without” a particular treatment
of interest (such as roadway lighting or a shoulder
rumble strip) or with various levels of a continuous
variable that represents a treatment of interest
(such as lane width).
 This type of study is commonly referred to as a
“with and without study.”
 The difference in number of crashes is attributed
to the presence of the discrete feature or the
different levels of the continuous variable.
Selection Guide for Observational Before/After
Evaluation Study Methods
Experimental Before/After Evaluation Studies
 Experimental studies are those in which comparable sites
with respect to traffic volumes and geometric features are
randomly assigned to a treatment or non-treatment group.
 The treatment is then applied to the sites in the treatment
group, and crash and traffic volume data is obtained for
time periods before and after treatment.
 For example, if an agency wants to evaluate the safety
effectiveness of a new and innovative signing treatment,
then an experimental study may be conducted.
PROCEDURES TO IMPLEMENT SAFETY EVALUATION
METHODS
Implementing the EB Before/After Safety Evaluation
Method
 The empirical Bayes (EB) before/after safety
evaluation method is used to compare crash
frequencies at a group of sites before and after a
treatment is implemented.
 The EB method explicitly addresses the
regression-to-the-mean issue by incorporating
crash information from other but similar sites into
the evaluation.
 This is done by using an SPF and weighting the
observed crash frequency with the SPF-predicted
average crash frequency to obtain an expected
average crash frequency.
Implementing the EB Before/After Safety Evaluation Method

 Data Needs and Inputs

 The data needed as input to an EB before/after
evaluation include:
1. At least 10 to 20 sites at which the treatment of
interest has been implemented
2. 3 to 5 years of crash and traffic volume data for the
period before treatment implementation
3. 3 to 5 years of crash and traffic volume for the period
after treatment implementation
4. SPF for treatment site types
 An evaluation study can be performed with fewer
sites and/or shorter time periods, but statistically
significant results are less likely.
Implementing the EB Before/After Safety Evaluation Method

Pre-Evaluation Activities
 The key pre-evaluation activities are to:
1. Identify the treatment sites to be evaluated
2. Select the time periods before and after treatment
implementation for each site that will be included in the
evaluation.
3. Select the measure of effectiveness for the evaluation.
4. Evaluations often use total crash frequency as the
measure of effectiveness, but any specific crash
severity level and/or crash type can be considered.
5. Assemble the required crash and traffic volume data for
each site and time period of interest. Identify (or
develop) an SPF for each type of site being developed.
SPFs may be obtained from Safety Analyst
Implementing the Safety Evaluation Method for
Before/After Shifts in Proportions of Target Collision
Types
Data Needs and Inputs
1. The data needed as input to a before/after
evaluation for shifts in proportions of target
collision types include:
2. At least 10 to 20 sites at which the treatment of
interest has been implemented
3. 3 to 5 years of before-period crash data is
recommended for the treatment sites
4. 3 to 5 years of after-period crash data is
recommended for the treatment sites
 An evaluation study can be performed with fewer
sites and/or shorter time periods, but statistically
significant results are less likely.
Implementing the Cross-Sectional Safety
Evaluation Method
 Data Needs and Inputs
1. 10 to 20 treatment sites are recommended to evaluate a safety
treatment
2. 10 to 20 non-treatment sites are recommended for the non-
treatment group
3. 3 to 5 years of crash data for both treatment and non-treatment
sites is recommended
 Pre-Evaluation Activities
 The key pre-evaluation activities are to:
1. Identify the sites both “with” and “without” the treatment to be
evaluated
2. Select the time periods that will be included in the evaluation
when the conditions of interest existed at the “treatment” and
“non-treatment sites”
3. Select the safety measure of effectiveness for the evaluation.
Evaluations often use total crash frequency as the measure of
effectiveness, but any specific crash severity level and/or crash
type can be considered.
SAMPLE PROBLEM:
TO ILLUSTRATE THE EB BEFORE/AFTER SAFETY
EFFECTIVENESS EVALUATION METHOD
1. Passing lanes have been installed to increase passing
opportunities at 13 rural two-lane highway sites.
2. An evaluation is to be conducted to determine the overall
effect of the installation of these passing lanes on total
crashes at the 13 treatment sites.
3. Data for total crash frequencies are available for these
sites, including 5-years of data before and 2-years of data
after installation of the passing lanes.
4. Other available data include the site length (L) and the
before- and after-period traffic volumes.
5. AADT is assumed to be constant across all years for both
the before and after periods.
6. It is also assumed that the roadway characteristics match
base conditions and therefore all applicable AMFs as well
as the calibration factor are equal to 1.0.
Basic Input Data
EB Estimation of the Expected Average
Crash Frequency in the Before Period
 Equation 10-6 provides the applicable SPF to
predict total crashes on rural two-lane roads:

 Equation 10-1 presents the predicted average

crash frequency for a specific site type x
(roadway, rs, in this example).
EB Estimation of the Expected Average
Crash Frequency in the Before Period
Step 1: Using the above SPF and Columns 2 and 3,
Calculate the Predicted Average Crash Frequency for
Each Site During Each Year of the Before Period
Step 2: Calculate the Weighted Adjustment, w, for
Each Site for the Before Period
1. Using Equation A-2, the calculated over-
dispersion parameter (shown in Column 20), and
Column 19, calculate the weighted adjustment,
w, for each site for the before period.
2. The results appear in Column 21.
3. Using Equation A-1, Columns 21, 19, and 10,
calculate the expected average crash
frequency for each site, summed over the entire
before period. The results appear in Column 22.
Step 3: Calculate the Predicted Average Crash Frequency for
Each Site during each year of the After Period

1. Using the above SPF and Columns 2 and 4,

calculate the predicted average crash
frequency for each site during each year of the
after period.
2. The results appear in Columns 23 and 24.
3. For use in later calculations, sum these
predicted average crash frequencies over the
two after years.
4. The results appear in Column 25.
Step 4: Calculate the Adjustment Factor, r, to Account for
the Differences Between the Before and After Periods in
Duration and Traffic Volume at Each Site.

1. Using Equation A-3 and Columns 25 and 19,

calculate the adjustment factor, r, to account for
the differences between the before and after
periods in duration and traffic volume at each
site.
2. The results appear in Column 26 in the table
presented in Step 3.
Step 5: Calculate the Expected Average Crash
Frequency for each Site over the Entire after Period in the
Absence of the Treatment.

1. Using Equation A-4 and Columns 22 and 26,

calculate the expected average crash
frequency for each site over the entire after
period in the absence of the treatment.
2. The results appear in Column 27 in the table
presented in Step 3.
Estimation of Treatment Effectiveness
Step 6: Calculate an Estimate of the Safety Effectiveness of the
Treatment at Each Site in the Form of an Odds Ratio

1. Using Equation A-5 and Columns 13 and 27,

calculate an estimate of the safety effectiveness
of the treatment at each site in the form of an
odds ratio.
2. The results appear in Column 28.
Step 7: Calculate the Safety Effectiveness as a
Percentage Crash Change at Each Site

1. Using Equation A-6 and Column 28, calculate

the safety effectiveness as a percentage crash
change at each site.
2. The results appear in Column 29 in the table
presented in Step 6.
3. A positive result indicates a reduction in
crashes; conversely, a negative result indicates
an increase in crashes.
Step 8: Calculate the Overall Effectiveness of the Treatment for
all Sites Combined, in the Form of an Odds Ratio
 Using Equation A-7 and the totals from Columns 13 and 27,
calculate the overall effectiveness of the treatment for all
sites combined, in the form of an odds ratio:

OR’ = 30/42.88 = 0.700

Step 9: Calculate each Term of Equation A-9
• Using Columns 26, 22, and 21, calculate each term of Equation A-9.
The results appear in Column 30 in the table presented in Step 6. Sum
the terms in Column 30.
• Next, using Equations A-8 and A-9, the value for OR’ from Step 8, and
the sums in Column 30 and 27, calculate the final adjusted odds ratio:

Since the odds ratio (OR) is less than 1, it indicates a reduction in crash
frequency due to the treatment.
Step 10: Calculate the Overall Unbiased Safety Effectiveness
as a Percentage Change in Crash Frequency Across all Sites
 Using Equation A-10 and the above result, calculate the
overall unbiased safety effectiveness as a percentage
change in crash frequency across all sites:
AMF = 100 × (1-0.695) = 30.5%
AMF = 100× (1- OR) (A-10)
 Estimation of the Precision of the Treatment Effectiveness
 Step 11: Calculate the Variance of OR
 Using Equation A-11, the value for OR’ from Step 8, and
the sums from Columns 13, 30, and 27, calculate the
variance of OR
Step 12: Calculate the Standard Error of OR

 Using Equation A-12 and the result from Step 11, calculate the
standard error of OR:

Step 13: Using the relationship between OR and AMF shown in

Equation A-10, the standard error of AMF, SE(AMF), is calculated
as:

SE(AMF) =100× SE(OR) (A-13)

SE(AMF) =100 × 0.138 = 13.8%

Step 14: Assess the Statistical Significance of the Estimated Safety

Effectiveness

 Assess the statistical significance of the estimated safety effectiveness

by calculating the quantity:
Abs[AMF/SE(AMF)] = 30.5/13.85 = 2.20
 Since Abs[AMF/SE(AMF)] ≥ 2.0, it is concluded
that the treatment effect is significant at the
(approximate) 95-percent confidence level. The
positive estimate of AMF, 30.5%, indicates a
positive effectiveness, i.e., a reduction, in total
crash frequency.
 In summary, the evaluation results indicate that
the installation of passing lanes at the 13 rural
two-lane highway sites reduced total crash
frequency by 30.5% on average, and that this
result is statistically significant at the 95-percent
confidence level.
ANY QUESTIONS
PLEASE ??