Professional Documents
Culture Documents
org/27747
DETAILS
114 pages | 8.5 x 11 | PAPERBACK
ISBN 978-0-309-70958-3 | DOI 10.17226/27747
CONTRIBUTORS
Serhan Guner; National Cooperative Highway Research Program; Transportation
Research Board; National Academies of Sciences, Engineering, and Medicine
BUY THIS BOOK
Visit the National Academies Press at nap.edu and login or register to get:
– Access to free PDF downloads of thousands of publications
– 10% off the price of print publications
– Email or social media notifications of new titles related to your interests
– Special offers and discounts
All downloadable National Academies titles are free to be used for personal and/or non-commercial
academic use. Users may also freely post links to our titles on this website; non-commercial academic
users are encouraged to link to the version on this website rather than distribute a downloaded PDF
to ensure that all users are accessing the latest authoritative version of the work. All other uses require
written permission. (Request Permission)
This PDF is protected by copyright and owned by the National Academy of Sciences; unless otherwise
indicated, the National Academy of Sciences retains copyright to all materials in this PDF with all rights
reserved.
Quality Processes for Bridge Analysis Models
N AT I O N A L C O O P E R AT I V E H I G H W AY R E S E A R C H P R O G R A M
Serhan Guner
The University of Toledo
Toledo, OH
Subscriber Categories
Bridges and Other Structures • Data and Information Technology • Highways
Research sponsored by the American Association of State Highway and Transportation Officials
in cooperation with the Federal Highway Administration
2024
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, non-
governmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for
outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the
practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering.
Dr. John L. Anderson is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National
Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions
to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent,
objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions.
The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase
public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.
The Transportation Research Board is one of seven major program divisions of the National Academies of Sciences, Engineering, and
Medicine. The mission of the Transportation Research Board is to mobilize expertise, experience, and knowledge to anticipate and solve
complex transportation-related challenges. The Board’s varied activities annually engage about 8,500 engineers, scientists, and other
transportation researchers and practitioners from the public and private sectors and academia, all of whom contribute their expertise in the
public interest. The program is supported by state transportation departments, federal agencies including the component administrations
of the U.S. Department of Transportation, and other organizations and individuals interested in the development of transportation.
FOREWORD
By Jo Allen Gause
Staff Officer
Transportation Research Board
The objective of this synthesis is to document state department of transportation (DOT) practices
related to the quality processes for bridge structural models. The design of bridges often involves the use
of structural analysis models of varying degrees of complexity. A variety of analysis methods and soft-
ware can be used to create and analyze these models. This process can be quite complex, with significant
amounts of input and output data. Quality assurance and quality control are two essential processes for
the quality management of bridge analysis models.
Information for this study was gathered through a literature review, a survey of state DOTs, and
follow-up interviews with selected DOTs. Case examples of five state DOTs provide additional infor-
mation on quality processes for bridge analysis models.
Serhan Guner, The University of Toledo, Ohio, collected and synthesized the information and
wrote the report. The members of the topic panel are acknowledged on page iv. This synthesis is an
immediately useful document that records practices that were acceptable within the limitations of
the knowledge available at the time of its preparation. As progress in research and practice continues,
new knowledge will be added to that now at hand.
CONTENTS
1 Summary
5 Chapter 1 Introduction
5 Background
5 Objective and Scope
5 Methodology
6 Key Definitions
7 Report Organization
71 References
75 Appendix A Survey Questionnaire
84 Appendix B Aggregate Survey Results
104 Appendix C Study Interview Questions
Note: Photographs, figures, and tables in this report may have been converted from color to grayscale for printing.
The electronic version of the report (posted on the web at nap.nationalacademies.org) retains the color versions.
SUMMARY
The design of bridges often involves the use of structural analysis models of varying
degrees of complexity. A variety of analysis methods and software can be used to create and
analyze these models. This process can be quite complex, as it uses significant amounts of
input and output data. Quality assurance (QA) and quality control (QC) are two essential
processes for the quality management of bridge analysis models. As indicated in the pub-
lished literature, a good QA/QC program is a deliberate and systematic approach to reduce
the risk of introducing errors and omissions into an analysis. The likelihood of errors is
increased if office policies and standardized procedures are not established and followed.
While documented quality processes are important, it is also essential to have experienced,
competent staff and good relationships across disciplines. Quality processes are often
affected when the staff is less experienced and when schedules fail to include enough time
for in-depth quality checks.
The literature demonstrates the importance of verification and validation as indis-
pensable components of QA/QC processes in computational modeling. While these
terms are often used interchangeably in bridge engineering practice, they have differ-
ent meanings. Stated succinctly, “verification” concerns mathematics, while “validation”
concerns physics. Verification assesses the numerical accuracy of a computational model
regardless of the physics being modeled, while validation assesses the degree to which the
computational model is an accurate representation of the physics being modeled. Veri-
fication compares computational solutions with highly accurate (analytical or numerical)
benchmark solutions, whereas validation compares the numerical solution with the experi-
mental results.
The objective of this synthesis is to identify and document state departments of transporta-
tion (DOT) practices related to the quality processes for bridge structural analysis models. The
synthesis documents the written and informal processes for (1) identifying appropriately
qualified staff including in-house personnel and consultants, (2) choosing an appropriate
analysis method and software, (3) validating the analysis software, (4) modeling a bridge
structure with proper approaches and assumptions, (5) verifying the analysis results, and
(6) reconciling discrepancies between independent models. Information was gathered
through a literature review, a survey of all DOTs, and follow-up interviews with five selected
agencies as case examples.
To collect information about current DOT practices, an online survey consisting of
25 questions was distributed to each DOT’s voting member in the AASHTO Committee on
Bridges and Structures. The survey was completed by 51 DOTs, including 50 states and the
District of Columbia, yielding an overall response rate of 100%. The respondents answered
all of the questions, yielding a response rate of 100%.
Summary 3
Specific information was collected from five selected state DOTs—California, Colorado,
Iowa, Louisiana, and New York—to expand on their quality processes related to bridge
structural analysis models.
The major findings from the case examples include the following:
• Identifying appropriately qualified consultants: Surveyed agencies use two types of consul-
tant selection processes: project-specific and statewide on-call.
• Verifying the analysis results obtained from consultants: Surveyed agencies require con-
sultants to either submit their QA/QC plans or follow the agency’s specific QA/QC plans.
Not all agencies requiring consultants to submit their QA/QC plans evaluate and score
QA/QC plans—the presence of such plans and assurance that they will be followed is
considered adequate.
• Assigning bridge design work to appropriately qualified in-house engineers: All
five surveyed agencies have informal processes that rely on a manager, supervisor, or
unit leader to select appropriately qualified engineers based on their experience and
availability.
• Selecting the analysis method or software for a specific project: All five surveyed agencies
most frequently use 1D line girder analysis and rely on the analysis (or design) team to
select the most appropriate method(s). New York State DOT (NYSDOT) provides written
guidelines for when to use refined analysis methods.
• Guidelines and documents used by in-house engineers when modeling a bridge: All five
surveyed agencies have in-house bridge design manuals to provide state-specific analysis
and design guidelines. Additional agency-developed documents include seismic design
guidelines, bridge rating manuals, user guides for in-house computer programs, structure
technical policies, bridge design memos, and design practice manuals for staff training.
The distribution of these documents is inconsistent across agencies.
• Verifying in-house analysis models and results: All five surveyed agencies require a
checker to independently verify the accuracy of the design engineer’s models, calculations,
and results. Both California DOT (Caltrans) and NYSDOT require the use of different
software in an independent check.
• Challenges and effective practices: All five surveyed agencies have undergone retirements
of senior staff and challenges recruiting and retaining structural engineering staff. NYSDOT
established the Satellite Squad concept to recruit talent from a broader geographic region.
To overcome the challenges of finding appropriately qualified engineers in district offices,
Caltrans established the Seismic and Special Analysis branch, which is focused only on
structural modeling and analysis, while NYSDOT established the Main Office Structures
group with 65 design staff who execute only structural analysis and final design.
This synthesis identifies some knowledge gaps and suggests future research to address
those gaps:
• Guidelines to assess effectiveness of QA/QC processes: Guidelines are lacking regard-
ing the constitution of effective QA/QC processes for bridge structural analysis models.
Future research could develop guidelines to help DOTs assess the effectiveness and quality
of their QA/QC processes.
• Guidelines for finite element and strut-and-tie (STM) models: Bridge-specific guidelines
for the development and verification of finite element and STM models are scarce. Future
research could develop guidelines and bridge-specific modeling examples.
• Repository for DOTs to share analysis models: Future research is suggested to develop a
nationwide repository for sharing analysis models among DOTs, which may help collec-
tively develop and iteratively improve finite element and STM models and reduce dupli-
cate efforts.
CHAPTER 1
Introduction
Background
The design of bridges often involves the use of structural analysis models of varying degrees of
complexity. Superstructure design models vary from approximate one-dimensional (1D) line girder
analysis models to sophisticated two-dimensional (2D) and three-dimensional (3D) finite
element analysis models. Substructure and foundation models often consider soil-structure
interaction, second-order effects, dynamic response for seismic analysis, and interaction with
the superstructure. A variety of analysis methods and software can be used to create and analyze
these models. This process can be quite complex, as it uses a large number of input parameters
and significant amounts of output data.
The engineer must understand the limitations of the analysis method and software, possess
experience in developing analysis models with proper approaches and assumptions, and correctly
interpret the results. An appropriate understanding of the expected behavior of the structure is
also required to assess whether the predicted behavior represents the actual performance of the
structure. A simple check of the program input values is not an adequate method of ensuring
the accuracy and validity of these models. Quality assurance (QA) and quality control (QC) are
two essential processes for the quality management of analysis models. Verification and validation
(V&V) play a critical role in the QA/QC process.
Methodology
Information was gathered using a literature review, an online survey of all state DOT agencies
and the District of Columbia DOT (DDOT), and follow-up interviews with selected agencies
for developing case examples. A web-based questionnaire was prepared to survey DOTs regard-
ing their quality processes for bridge structural analysis models. The questionnaire was sent to
voting DOT members in the AASHTO Committee on Bridges and Structures (50 state DOTs
and DDOT); 51 survey responses were received, thereby yielding a 100% response rate. The
survey results are processed and summarized in graphs and tables for use in developing this
synthesis report.
In addition, five DOT agencies were selected, based on certain criteria, for follow-up inter-
views. The responses and documentation collected from these interviews were used to derive
case examples to expand on their quality processes for bridge structural analysis models.
Key Definitions
The definitions of common terms used throughout this report are provided as follows in the
context of structural analysis models.
• Accuracy: The closeness of a measured or calculated value to the referent or true value (1)
• Calibration: The process of adjusting code input parameters so that the resulting agreement
of the code calculations with a chosen and fixed set of experimental data is maximized (2)
• Checker: An individual responsible for conducting a full technical review of the structural
design calculations, drawings, specifications, and contract documents (3)
• Code: A computer program designed (in the present context) to implement a computational
model (4)
• Computational model: The discretized version of a mathematical model that has been designed
to be implemented on (or to be processed by) a computer or computational device (4)
• Designer: An individual directly responsible for the development of design calculations,
drawings, specifications, contract documents, and review of shop drawings related to a spe-
cific bridge design with a level of technical skills and experience commensurate with the com-
plexity of the subject structure or structures being designed (3)
• Discretize: To transform a mathematical model into a finite number of discrete components
that can be processed by a digital computer (4)
• Error: The quantitative difference between a measured or calculated value and the referent or
true value (1)
• Prediction: A calculation that predicts a number or quantity or a collection of these quantities
prior to or in lieu of their physical measurement (5)
• Quality assurance: Part of quality management focused on providing confidence that quality
requirements will be fulfilled (6)
• Quality control: Part of quality management focused on fulfilling quality requirements (6)
• Quality management: Management with regard to quality that can include establishing quality
policies and quality objectives and processes to achieve quality objectives through quality
planning, QA, QC, and quality improvement (6)
• Reviewer: An individual responsible for executing QA procedures for assuring that QA proce-
dures have been executed (3)
• Simulation: The computer calculations executed with the computational model—that is,
“running the model” (1)
• Validation: The process of determining the degree to which the model is an accurate repre-
sentation of corresponding physical experiments from the perspective of the intended uses
of the model (1)
• Verification: The process of determining that a computational model accurately represents the
underlying mathematical model and its solution (1)
Introduction 7
Report Organization
This synthesis report consists of five chapters, a reference list, and three appendices. Chapter 1
presents the background information, synthesis objectives, scope, methodology, and terminology.
Chapter 2 provides a review of the published literature on QA and QC processes in the context of
bridge analysis models, the V&V processes, and bridge analysis methods. Chapter 3 presents the
current state of the practice based on the results of a survey distributed to DOTs regarding their
quality processes for bridge analysis models. Chapter 4 presents case examples of five state DOTs.
Chapter 5 presents the key findings of this synthesis and discusses identified gaps that could be
addressed in future studies to enable DOTs to effectively develop quality processes for bridge analy-
sis models. Following the chapters are a reference list, along with Appendix A providing the survey
questionnaire, Appendix B providing aggregate survey results, and Appendix C providing study
interview questions.
CHAPTER 2
Literature Review
This chapter presents concepts, processes, and methods from the published literature about
quality processes for bridge analysis models. The topics covered include QA and QC, V&V,
uncertainty and error, sensitivity analysis, calibration and tuning of computational models, and
common types of analysis methods for bridge elements and substructures.
Literature Review 9
For major projects involving unusual, complex, and innovative features, a peer review may be
desirable to raise the level of confidence in the quality of design and construction. A peer review
is a high-level QA review by a special panel of professionals specifically appointed by the bridge
owner to meet demands for quality and accuracy, while recognizing complexity of the design.
Peer review is an effective way to improve quality and to reduce the risk of errors and omissions.
The need for such peer reviews is at the discretion of the bridge owner (3).
NCHRP Project 20-68A (8) conducted a scanning study of 10 state DOTs with the objec-
tive of documenting QA/QC programs in bridge design, plan review, overall project delivery,
and special contract projects such as design-build. The visited states believed that, although
documented quality processes are important, also important is having experienced, competent
staff and good relationships across disciplines. Plan quality is often affected when staff is less
experienced and schedules leave less time for in-depth quality checks. The scanning study
emphasizes the importance of support from upper management during the development,
documentation, and use of a QA/QC program, while also indicating a successful method of
ensuring quality by incorporating training rotations for new staff. Regularly scheduled meet-
ings with all disciplines involved in the projects are noted as a successful QA strategy. Good
communication between consultants and department staff is also indicated as a key compo-
nent of successful QA/QC programs.
FHWA sponsored a scanning study of Europe (9) to identify the best practices and processes
to assure bridge safety and serviceability in the United States. The team recommended (1) devel-
oping a national strategy to increase use of refined analysis for bridge design and evaluation,
(2) encouraging states to use refined analysis combined with reliability analysis to avoid unnec-
essary rehabilitation or replacement of bridges, and (3) encouraging adoption of the concept of
annual probability of failure to quantify safety in probability-based design and rating specifica-
tions. The study indicated that the situations impeding the use of advanced analysis in design
and evaluation were cost of software, lack of training, lack of guidelines, modeling complexities,
and perceived high cost-to-benefit ratio.
von Wolffersdorff and Meier (10) consider the challenges of the traditional designer-reviewer
model (see Figure 2) for complex 3D finite element analyses. The authors indicate that such
analyses are not easily checkable unless the reviewer conducts comparative numerical calcula-
tions, a practice often unfeasible due to time and financial constraints. The authors propose an
improved QA/QC process in which the reviewer is involved in the modeling process from the
beginning to make essential modeling decisions and assumptions.
von Wolffersdorff and Meier (10) indicate that the skills acquired in undergraduate university
education are often insufficient for a competent application of complex finite element (FE) pro-
grams and competent interpretation of the analysis results. The authors recommend developing
and introducing a qualified advanced training system with standardized requirements. Knowl-
edge in the following fields is noted as essential:
• Continuum mechanics
• Theory of finite elements
• Numerical mathematics
• Constitutive modeling of the material (e.g., reinforced concrete, steel, timber, soil, and rock)
• Modern theoretical soil mechanics for geotechnical modeling
Amati et al. (11) define the goal of QA in finite element modeling as reducing the errors in evalu-
ating a prescribed quantity (i.e., displacements or stresses below a pre-specified value to ensure
a sufficient margin of safety). The QA process is also essential to make the FE analyst aware of
the approximations contained in a numerical method. The authors emphasize the importance of
benchmark testing, not only for validation of the analysis software globally, but also for educating
the analyst on the software’s operations and limitations.
U.S. Nuclear Regulatory Commission (NUREG) published a document (12) for software
QA programs and guidelines for National Research Council (NRC) organizations and NRC
contractors. “Software QA” is defined as the planned and systematic pattern of all actions neces-
sary to provide adequate confidence that a software product conforms to established technical
requirements. The scope of software QA includes both managerial and technical aspects of
software development and maintenance. This document provides guidelines for software life
cycle, V&V activities, documentation and deliverables, project management, configuration
management, nonconformance reporting and corrective action, and quality assessment and
improvement.
The term “check” or “checking,” as used in the previous section, refers to V&V and uncertainty
quantification processes. The next section presents information on these processes from the
published literature.
Literature Review 11
Verification
“Verification” is defined as “the process of determining that a computational model accu-
rately represents the underlying mathematical model and its solution” (1). Verification has
also been described as “solving the equations right” (21). The fundamental strategy of verifi-
cation is to identify, quantify, and reduce errors in the computational model and its numerical
solution (5).
Verification is composed of two fundamental activities: code verification and solution veri-
fication. Code verification involves exercising the code (i.e., computer program) to determine
and correct coding errors (bugs) or other deficiencies that affect the efficiency and quality of the
output. Code verification is usually conducted by using the code to solve benchmark problems—
specific, simplified model problems for which accurate solutions or analytical solutions are
known. Normally, the developers perform this exercise before the release of the code. However,
computer codes tend to have programming errors, especially in their less frequently traversed
branches. The user (i.e., analyst) shares the responsibility of verifying that the code is functioning
properly (22). Solution verification includes error estimation, which involves determining the
accuracy of a single calculation and putting an error band on the final value (21). Analysts must
compare the estimated numerical error in specific quantities of interest to preset tolerances in
order to determine whether the computational model is verified to their satisfaction. Solution
verification, to a large measure, is a mathematical process that can be executed, in theory, to a
high degree of precision (4).
• Sanity checks: a basic test to evaluate whether the result of a calculation is rational and can
possibly be true, usually conducted by hand calculations, spreadsheets, or other tools
• Independent calculations: re-analysis of the same structure by an independent analyst using
a method selected by the independent analyst
• Line-by-line checking of the input parameters
• Refined analysis: re-analysis of the same structure using a more advanced or refined analysis
method
“Irreducible uncertainty” refers to inherent variations in the physical system being modeled.
This type of uncertainty always exists and is an intrinsic property of the system. “Reducible
uncertainty” refers to deficiencies that result from a lack of complete information or knowledge.
Two important sources of reducible uncertainty are “statistical uncertainty” and “model form
Source: AIAA G-077-1998 (17), reprinted with permission of the American Institute of
Aeronautics and Astronautics, Inc.
Literature Review 13
uncertainty.” Statistical uncertainty arises from the use of limited samples. Model form uncer-
tainty refers to the uncertainty associated with modeling assumptions and approximations (1).
The modeler quantifies uncertainties in simulation results due to inherent variability in model
parameters or lack of knowledge of the parameters or model form. The results of parameter and
model-form uncertainty quantification are combined with those of calculation verification to
yield an overall uncertainty estimate associated with simulation results (1).
Errors can be classified as “acknowledged” and “unacknowledged” errors (see Figure 5). Acknowl-
edged errors have procedures for identifying and possibly removing them. Unacknowledged errors
have no set procedures for finding them and may continue to remain in the code or simulation.
Usage errors are controllable through proper training and analysis (21).
Physical approximation errors are also referred to as “errors of idealization” (or “modeling
errors”) (22). Physical modeling errors may include incorrect definition of the material (e.g.,
modulus of elasticity, limit of Poisson’s ratio in isotropic materials), load, and boundary condi-
tions. Geometry modeling errors may include incorrect geometric description (e.g., the use of
symmetry when the loads are antisymmetric). Defining the analysis type incorrectly (e.g., linear
versus nonlinear analysis) is also a physical approximation error.
Discretization errors may include the use of a wrong type of finite element (e.g., plane stress
versus plane strain elements in a 2D analysis, plate versus shell elements in a 3D analysis), unsuit-
able order of elements (e.g., linear-strain quadrilateral versus quadratic-strain quadrilateral), and
unsuitable mesh density (e.g., a coarse mesh at locations with high stress gradients). Input errors
(e.g., mixed units and errors in data entry) are also considered discretization errors.
Both physical approximation errors and discretization errors are controllable through proper
training of the analyst and proper QA/QC practices (23). Methods for assessing (and improving)
the quality of the mesh is to apply mesh refinement methods, including the h-method (reducing
the mesh size and assessing its effect on the analysis results), the p-method (increasing the poly-
nomial order in the element), the r-method (relocating the position of a node), or a combination
of these methods.
Sensitivity Analysis
One approach for determining the level of uncertainty and its effect on an analysis is to con-
duct a sensitivity analysis (17). Sensitivity analysis is the general process of discovering the effects
of model input parameters on the response quantities of interest using techniques such as analy-
sis of variance (24). When conducted after the computational model is verified but before it is
validated, a sensitivity analysis can provide important insight into the characteristics of that
computational model (1). Local sensitivity analysis is used to determine the character of the
response quantities with respect to the input parameters in a local region, while global sensitivity
analysis is concerned with some type of average behavior of the response quantities over a large
domain of parameters (1). The role of sensitivity analysis in the processes of V&V/UQ has been
the subject of many previous studies [e.g., (1), (25–31)].
Validation
“Validation” is defined as “the process of determining the degree to which the model is an accu-
rate representation of corresponding physical experiments from the perspective of the intended
uses of the model” (1). Validation has also been described as “solving the right equations” (21).
The fundamental strategy of validation involves identifying and quantifying errors and uncer-
tainty by comparing simulation results with experimental data. The experimental data sets them-
selves contain uncertainties and errors that must be properly quantified and documented as part
of the data set. The validation strategy does not assume that experimental measurements are
more accurate than computational results. The strategy asserts only that experimental measure-
ments are the most faithful reflections of reality for the purposes of validation (21).
Although the immediate goal of validation is to compare simulation results with experimental
measurements, the strategic goal is to increase confidence in the predictive capability of a com-
putational model for its intended use (1), which is accomplished by comparing computational
predictions to experimental outputs (1). Figure 6 depicts the validation process of comparing
the computational results of the modeling and simulation process with experimental data from
various sources (17).
Validation is essential to assessing the predictive capability of the model in the physical realm of
interest; furthermore, it must address uncertainties that arise from both experimental and compu
tational procedures (1). When a complex system is modeled, many validation experiments captur-
ing different physical aspects of the system are needed (e.g., different loading scenarios, boundary,
and initial conditions) on different levels of complexity of the model (32). Figure 7 (33) presents
the difference that can be measured between a single experiment and a single simulation and the
actual means of a given measure. The sensitivity of this measure to a given parameter is represented
Literature Review 15
Figure 7. Difference between simulated and experimental values for a single item
and a population of results.
by the shape (i.e., standard deviation) of the distribution function. When comparing simulated
results to only one experimental result, the analyst has no confidence about representativeness of
the experiment result. In the process of calibrating the computational model to only one experi-
ment, more errors can be introduced into the model; moreover, its predictive capability can be
negatively affected for a different set of initial parameters.
Because of the infeasibility and impracticality of conducting true validation experiments on
most complex or large-scale systems, the suggested method is to use a building-block approach
(17, 34, 35). This approach divides the complex engineering system of interest into at least three
progressively simpler tiers: subsystem cases, benchmark cases, and unit problems. The strategy
in the tiered approach is to assess how accurately computational results compare with experi-
mental data (with quantified uncertainty estimates) at multiple degrees of physics coupling and
geometric complexity. The approach is constructive in that it recognizes a hierarchy of complex-
ity in systems and simulations and that the quantity and accuracy of information obtained from
experiments vary radically over the range of tiers. Furthermore, this approach demonstrates that
validation experiments can be conducted at many different levels of physics and system com-
plexity. Each comparison of computational results with experimental data allows an inference
of validation concerning tiers both above and below the tier where the comparison is made (5).
In bridge engineering practice, field experiments are occasionally conducted on existing
bridges to obtain test data for the component-level validation of finite element models. Exam-
ples include laser-based remote sensing on a skewed two-span bridge (36), strain measure-
ments on the cross-frame members of straight and skewed bridges (37), strain measurements
on skewed deck girder bridges (38), dynamic deflection monitoring of a light-rail bridge (39),
strong ground motion measurements to validate the modeling approaches of ordinary bridges
recommended by Caltrans (40), pressure and displacement measurements on a precast-concrete
buried arch bridge (41), and static and dynamic load testing to validate a fiber-reinforced-polymer
deck bridge model (42).
Caution is advised when using statements such as, “This model has been validated” or “This is
a validated model.” These statements raise questions: To what experimental measurements was
the model compared? Over what set of conditions? With what model and data uncertainties?
To what level of accuracy? In fact, there is much more value in that set of questions than in the
original statements (1).
Calibration
One simple definition of “calibration” is the employment of explicit tuning or updating of model
boundaries, limits, or guidelines associated with an engineering code to achieve improved agree-
ment with existing validation experiments (2,16). In another definition, calibration is a procedure
in which, through repeated calculations with modified input parameters, the engineer tries to find
an “optimal” set of input data that can provide the model’s response closest to the actual experimental
data (32). The process allows the most common sources of modeling (and experimental) difficul-
ties to be represented as simple mechanical models and calibrated so that the global response of
the computational model agrees with the experimental results (1). Calibration of the model is
conducted only after both code verification and calculation verification have been conducted (1).
Parametric model calibration determines only the model’s fitting ability, not its predictive
capability. A model calibrated to experimental results may not yield accurate predictions over the
range of its intended use. It is possible that, due to superimposition of errors, the engineer may
find a strong correlation between the experimental and numerical results for a wrong model,
defined by incorrect input parameters. Such a situation is often detected when the model is used
for a different case with changed input conditions. Furthermore, a complex model with only
some of the input parameters “correctly” calibrated should give a response different from the
experimental data due to the indeterminacy of other parameters (16).
Given cost and schedule constraints, model calibration is often conducted after an initial vali-
dation assessment has been made and requirements have not been satisfied. That is, the modeler
finds a set of parameter values that provides acceptable agreement with the validation test data,
but only after failing to achieve that agreement with a prediction. Unfortunately, to assess pre-
dictive capability, subsequent validation against other independent experiments may still be
necessary. Any revisions to the parameter values after V&V are applied signifies new model-
development activity, thereby triggering a repetition of some model V&V (1).
Benchmark
A benchmark is a choice of information that is believed to be accurate or true for use in V&V
or calibration. The fundamental purpose of benchmarks is to draw specific conclusions from
their comparison with calculations. In the case of verification, the purpose is to assess the math-
ematical accuracy of the numerical solutions. For validation, the purpose is to assess the physical
fidelity for a stated application of the mathematical equations solved in the code. For calibration,
the purpose is to choose parameter values that improve the agreement of the code calculations
with the chosen benchmarks, in the belief that such tuned accuracy improvement will increase
the credibility of the code—a goal commonly considered to be incorrect. The choice of bench-
marks must vary depending on the purpose of the comparisons (2).
Literature Review 17
12-50, two or more software analysis packages or hand calculations with the same data set may be
compared in tabular or graphical format (a verification activity). The results from the processes can
be imported into a common viewer for comparison so that the differences will be apparent.
Choe et al. (45) used NCHRP Process 12-50 to evaluate and verify prestressed concrete bridge
design software commonly used in Indiana. A test bed of 40 bridge structures was developed with
input from practicing engineers and Indiana Department of Transportation (INDOT) specialists. The
test bed included 20 simple-span and 20 multi-span bridges. The primary parameters were bridge
span, girder spacing, section type, strand pattern (straight or draped), and concrete strength. An
indigenous computer program was developed to verify the results generated by their in-house soft-
ware package. The output from both computer programs was compared to identify assumptions and
discrepancies between their in-house software and the AASHTO Load and Resistance Factor Design
(LRFD) specifications (46). These comparisons verified INDOT’s in-house software for (1) con-
crete stresses and deflections at service-level loads, (2) nominal flexural strength, (3) nominal shear
strength, and (4) initial camber and deflections. No validation process was presented in this study.
Varma and Seo (47) used NCHRP Process 12-50 to evaluate and verify composite steel I-girder
design software commonly used in Indiana. They used a test bed of 21 bridges, including five simple-
span and 16 multi-span bridge superstructures. More than 80 parameters were used to define a
bridge. An indigenous computer program was developed to verify the results generated by their
in-house software package. The output from both computer programs was compared to identify
assumptions and discrepancies between their in-house software and the AASHTO LRFD specifi-
cations (46). These comparisons verified INDOT’s in-house software for (1) moments, (2) shears,
(3) stresses, (4) deflections, (5) flexural strength and all relevant parameters, (6) shear strength and
all relevant parameters, and (7) parameters related to shear connectors. No validation process was
presented in this study.
Gergely et al. (48) evaluated the bridge analysis and rating procedures of North Carolina DOT
(NCDOT). They first verified NCDOT’s simple and continuous span bridge analysis software
packages with several examples using different methods, including the governing AASHTO
bridge rating procedures, AASHTOWare bridge rating software (49), and a spreadsheet pro-
gram developed by the University of North Carolina at Charlotte. Additionally, four bridges were
experimentally tested during the first phase of the project to validate analytical predictions. The
variables used in bridge rating procedures include, among other factors, girder end conditions,
impact and distribution factors, and deck-to-girder composite action. As a result of the valida-
tion process, they concluded that “it is unrealistic to expect that analytical procedures alone
(including detailed finite element methods) will capture the true performance of individual
bridges” (48). Therefore, the second phase of this work focused on a broader approach, which
included the development of a simple spreadsheet program to provide a lower and upper bound
solution (an uncertainty quantification activity), the use of non-destructive tests for materials
and construction details (an uncertainty reduction activity), and the development of a simplified
bridge test protocol to evaluate the true response of individual bridges (a validation activity).
approximate distribution factors according to the AASHTO LRFD specifications (50) employs
1D analysis. One-dimensional analyses can be an efficient choice when designing essentially
straight, right, regular multi-girder bridges, or torsionally stiff box girder spine beam bridges,
for which lateral and torsional responses are not critical (51).
The standard method of analysis for typical multi-girder bridges in the United States has
for years been a line girder analysis with the use of distribution factors (51). In this type of
approximate analysis, each girder is analyzed as a stand-alone component. Assumptions are
made regarding the distribution of dead loads among the girders, while distribution factors
account for the transverse distribution of live loads. If the bridge is continuous or otherwise
indeterminate, an analysis program is usually used to determine the shears and moments in the
girders based on the distributed loads applied.
(a) Deck included with line element properties (b) Deck modelled with shell element deck
Source: A. Adams, et al. Report FHWA-HIF-18-046 (51).
Literature Review 19
sections, and so on. While linear-elastic material behavior and small displacements are enforced,
geometrical nonlinearities and second-order effects may be accounted for by the software through
iterative calculations. While 3D analysis can provide a more accurate solution for a larger variety
and complexity of systems, proper V&V efforts are best undertaken to ensure the validity and
accuracy of the results (51).
Strut-and-Tie Method
The strut-and-tie (STM) method is a rational hand-calculation method that idealizes a rein-
forced concrete bridge member by a truss mechanism (54–56) composed of struts, ties, and
nodal zones (see Figure 9). STM is commonly used for analysis of deep members and disturbed
regions where the plane-sections-remain-plane hypothesis (57–59) is not valid.
R1 R2 R3
w1 w2 w3
(a) Idealized flow of stresses (b) Components of a strut-and-tie model
Source: Baniya and Guner (63), used with permission.
STM uses a plasticity-based lower-bound theorem and thus calculates a safe, low-bound load
capacity for the concrete member provided that the member is sufficiently ductile (60–62). The
ductility is typically provided by sufficient amounts of grid reinforcement referred to as “crack
control reinforcement.”
While STM is a conceptually simple method, its application in the bridge industry presents
several challenges, such as developing a valid truss model (several models could be developed for
a given problem), performing an iterative model improvement process, and checking stress limits
in nodal regions. STM requires fundamental understanding, experience, and a labor-intensive
geometric solution process. The number of computer programs available for this method is also
limited. As an example, the program STM-CAP (63, 64) is recommended in Ohio DOT’s Bridge
Design Manual (65) for STM analysis of multi-column reinforced concrete pier caps.
CHAPTER 3
This chapter presents the current state of the DOT practices related to the quality processes for
bridge structural analysis models. To collect the most current information, an online survey was
created in Qualtrics (66) and distributed by email to each DOT’s voting member in the AASHTO
Committee on Bridges and Structures. Several follow-up emails were sent, and phone calls were
made to encourage participation.
The survey included 25 questions grouped in three sections: (1) quantities of bridge design
and evaluation projects undertaken, (2) bridge designs completed by consultants, and (3) bridge
designs completed in house by agencies. Eighteen (18) questions were directed to all partici-
pants, while seven additional follow-up questions were directed to participants who selected cer-
tain options in their responses. All questions were multiple choice. Five questions used a matrix
table that listed several line items (e.g., methods, processes) and asked respondents to quantify
how frequently each line item is used. The frequency scale included four options: “often,” “some-
times,” “rarely,” and “never.” Five questions provided the respondents with the option of upload-
ing or sharing a web link for the related documents of their agencies. These documents were used
to review the current written processes and identify suitable DOT agencies for case examples
presented in Chapter 4. The complete survey questionnaire is provided in Appendix A.
The survey was completed by 51 DOTs, including 50 states and the District of Columbia, yielding
an overall response rate of 100%. The participants responded to all questions, yielding a response
rate of 100%. The population size was 51, which is represented in figures as N = 51.The seven follow-
up questions were also answered by all participants who qualified for those questions based on their
answers. For these questions, the sample size is represented as N in the charts. The complete agency
responses to the survey questions are provided in Appendix B.
The following section presents the collected responses in three categories: (1) quantity of
bridge design and evaluation projects undertaken, (2) quality processes for bridge design projects
undertaken by consultants (managed by DOTs), and (3) quality processes for bridge design
projects implemented in house by agencies. In addition, the survey responses to a few selected
questions are presented for high-seismicity state DOTs in comparison to the remaining DOTs.
21
80+
60-80
Total number of in-
40-60
house bridge
engineers in each 20-40
DOT 10-20
<10
Figure 10. Bridge engineers currently working in each DOT (direct employees
only, not including consultants).
20–40 engineers, selected by 17 DOTs (33%), while the least represented range is less than 10 engi-
neers, selected by five DOTs (10%).
The survey asked respondents for the number of new bridges and bridge replacement designs
completed by their agencies and their consultants. As shown in Figure 11, the most represented
range is 20–40 designs per year, selected by 16 DOTs (31%), while the least represented range is
60–80 designs, selected by 3 DOTs (6%).
The survey asked for the percentage of new bridge and bridge replacement designs completed
by external consultants. As shown in Figure 12, the majority of agencies, 40 DOTs (78%), use con-
sultants for half (40%–60%) or more than half (60%–80% and 80%–100%) of their new designs. In
the less represented ranges, five DOTs use consultants for 20% to 40% of their new designs, four
DOTs use consultants for 10% to 20% of their new designs, and two DOTs use consultants for
less than 10% of their new designs.
The survey asked respondents for the total number of existing bridge analyses, including load
ratings rehabilitation, retrofit, and any analysis conducted on an existing bridge completed by
their agencies and their consultants. As shown in Figure 13, most agencies, 30 DOTs (58%), and
their consultants complete more than 125 analyses per year. The actual number could be much
higher than 125. Three DOTs complete 100–125 analyses, five DOTs complete 75–100 analyses,
six DOTs complete 50–75 analyses, three DOTs complete 25–50 analyses, and four DOTs com-
plete less than 25 analyses.
The survey asked for the percentage of existing bridge analyses completed by external consul-
tants. As shown in Figure 14, each range is well represented except the 10% to 20% range. The
most represented range is 60% to 80%, selected by 13 DOTs (25%), while the least represented
range is 10% to 20%, selected by one agency.
80+
Total number of 60-80
new bridge
designs 40-60
completed by 20-40
DOTs and their
consultants 10-20
<10
80-100%
60-80%
Percentage of
new bridge designs 40-60%
completed by 20-40%
consultants
10-20%
<10%
125+
80-100%
Percentage of 60-80%
existing bridge 40-60%
analyses
completed by 20-40%
consultants 10-20%
<10%
Figure 15 compares the responses from two questions related to the percentage of new bridge
design and existing bridge analysis work assigned to consultants (as presented in Figure 12 and
Figure 14). The analysis of responses indicates that the average percentage of new bridge and
bridge replacement designs assigned to consultants is 59%, while the existing bridge analyses,
including load ratings, assigned to consultants is 47%.
Total 3
number of
consultant
2
process 1
questions
responded 0
'written'
0% 5% 10% 15% 20% 25% 30% 35%
Percentage of DOTs (N=51)
Even if all proposals are assumed to include written processes for analysis result verification, this
process would still be the least common written process. The most common no process response,
selected by 19 DOTs (37%), was “bridge analysis models developed by consultants.”
To examine the prevalence of written processes for the same three questions, individual DOT
responses were analyzed based on how many consultant process questions were answered as written.
As shown in Figure 17, 17 DOTs (33%) have no written process for any of these three activities, while
only eight DOTs (16%) have written processes for all three consultant-related bridge design activities.
Pre-qualification
requirements
Proposal evaluation
Interview
Figure 20. DOTs with a proposal evaluation process for selecting appropriately
qualified consultants.
Interview (35%)
Figure 21. DOTs with an interview process for selecting appropriately qualified
consultants.
Figure 22. DOT requirements for the pre-qualification process. NOTE: Check-all-that-
apply type of question.
Approved Yes
software list for
bridge modeling No
and analysis
0% 10% 20% 30% 40% 50% 60% 70%
Percentage of DOTs (N=51)
Figure 24. DOTs with approved software list for bridge modeling and
analysis.
Figure 24 shows a map of the 20 DOTs with an approved software list for bridge modeling
and analysis.
The responses, shown in Figure 27, indicate that only eight DOTs (16%) have written pro-
cesses, while 14 DOTs (27%) do not have any process for verifying the analysis results obtained
from their consultants. The most common response was “informal,” selected by 23 DOTs (45%).
As a follow-up question, the 31 DOTs (61%) that have written or informal processes for verify-
ing analysis results obtained from consultants were asked whether they require an independent
consultant as a “checker” to verify the accuracy of calculations conducted by the project consul-
tant. They were given six response options, as shown in Figure 28. The most common response
for agencies with written processes was, “yes, only for complex bridges,” while the most common
answer for agencies with informal processes was “not required.”
Another follow-up question asked the same 31 DOTs whether they conduct an in-house
check of analysis results obtained from consultants to verify accuracy. They were given five
Written
Processes for
verifying the Informal
analysis results
obtained from As per proposal
consultants No
response options, as shown in Figure 29. The two common responses for the agencies with a
written process were “yes, for every bridge” and “yes, based on a case-by-case decision.” The
most common response for the agencies with informal processes was “yes, based on a case-
by-case decision.”
Figure 30. Comparative DOT responses to six questions regarding in-house processes.
Written Written
Informal Informal
No No
0% 10% 20% 30% 40% 50% 60% 0% 10% 20% 30% 40% 50% 60%
Percentage of DOTs (N=51) Percentage of DOTs (N=51)
(a) Consultants (b) In-house engineers
The two frequently selected informal processes were “identifying appropriately qualified
in-house engineers,” selected by 31 DOTs (61%), and “verifying in-house analysis results,”
selected by 30 DOTs (59%). The two frequently selected no-process responses were “validating
the analysis software,” selected by 21 DOTs (41%), and “reconciling discrepancies between inde-
pendent models,” selected by 18 DOTs (35%).
The most common written in-house process, selected by 16 DOTs (31%), was “modeling a
bridge”; this process, however, also had a significant amount of “informal” (37%) and “no pro-
cess” (31%) responses. The least common written process, selected by only six DOTs (12%), was
“identifying appropriately qualified in-house engineers.” This activity had the largest percentage
of informal process responses, selected by 31 DOTs (61%). This result contrasts with the pro-
cesses for identifying appropriately qualified consultants, as shown in Figure 31.
To examine the prevalence of written processes for the same six questions, individual DOT
responses were analyzed based on how many in-house process questions were answered as
“written.” As shown in Figure 32, 23 DOTs (45%) have “no written process” for any of these
six activities, while no DOT has written processes for all six activities surveyed.
6
5
Total
number of 4
in-house 3
process 2
questions
responded 1
'written' 0
0% 10% 20% 30% 40% 50%
Percentage of DOTs (N=51)
Figure 33. Methods of analysis used for bridges analyzed in house by DOTs.
Figure 34 compares the percentage of responses from DOTs for the analysis methods used by
consultants and DOTs for the “often” frequency. This comparison indicates that consultants use
more refined analysis methods than DOTs. It is important to note that the frequencies of analysis
methods used by consultants were reported by 15 DOTs that track the methods used by their
consultants, while the frequencies of methods used in house were reported by 51 DOTs.
Figure 34. Comparative DOT responses for methods of analysis used by consultants
and DOTs (“often” frequency).
for the “sometimes” frequency, selected by four DOTs (57%), was “hiring external consultants.” The
least commonly used validation method was “use of data from field tests and sensor deployment.”
Figure 37 shows the responses obtained from 23 DOTs that have informal processes for vali-
dating analysis software. Two respondents wrote in their own validation methods in the textbox
provided without selecting any of the pre-set response options, which is considered “not appli-
cable” or “n/a” for the pre-set response options.
Often
Analysis engineers decide how to validate
Sometimes
Rarely
Hiring of external consultants for software validation Never
n/a
Use of data from field tests and sensor deployment
Use of data from bridge inspection records for existing
bridges
Modeling of benchmark structures/specimens (tested
experimentally) and compare our results
0% 10% 20% 30% 40% 50% 60% 70% 80%
Percentage of Responses from DOTs with informal processes for validating the analysis
software (N=23)
Written
Processes for
verifying in- Informal
house analysis No
results
0% 10% 20% 30% 40% 50% 60%
Percentage of DOTs (N=51)
structural analysis results might include comparing the results with the results obtained from
another software, tool, spreadsheet, or hand calculations.”
The responses, shown in Figure 38, indicate that 13 DOTs (25%) have written processes, while
eight DOTs (16%) have no processes for verifying in-house analysis results. These results dem-
onstrate that analysis result verification processes are more prevalent than software validation
processes, for which 21 DOTs (41%) have no processes.
The 43 DOTs (84%) that have either written or informal processes were asked how their agen-
cies verify in-house analysis results for medium- to high-complexity bridges. They were given
seven response options, including a write-in textbox, and four frequencies to choose from.
Figure 39 shows the responses obtained from 13 DOTs that have written processes for verify-
ing in-house analysis results. If an option was not selected by a respondent, it was considered “not
applicable” (n/a). The most common response for the “often” frequency, selected by eight DOTs
(62%), was “checking of input variables.” “Another team of engineers uses a different method or
software” and “analysis engineer decides how to verify” were the other common responses for
the “often” frequency. In contrast, the use of NCHRP Process 12-50 is by far the least employed
method, with eight DOTs (62%) indicating they have never used it.
Figure 40 shows the responses obtained from 30 DOTs that have informal processes for verify-
ing in house the analysis results. If an option was not selected by a respondent, it was considered
“not applicable” (n/a). As compared to the DOTs with written processes, DOTs with informal
processes more frequently selected the option, “checking of input boundaries/limits/guidelines,”
and much less frequently selected the option, “another team of engineers uses a different method
or software.” Similar to the DOTs with written processes, the use of NCHRP Process 12-50 is the
least employed method, with 16 DOTs (53%) indicating they have never used it.
Figure 39. Methods of verification of in-house analysis results for medium- to high-
complexity bridges and substructures that require 2D or 3D analysis models (DOTs with
written processes).
Figure 40. Methods of verification of in-house analysis results for medium- to high-
complexity bridges and substructures that require 2D or 3D analysis models (DOTs with
informal processes).
Figure 42. Methods for reconciling discrepancies between independent models (DOTs
with written processes).
Figure 43. Methods for reconciling discrepancies between independent models (DOTs
with informal processes).
WA
MT
OR
ID
WY
NV IL IN
UT
CA KY
MO
TN
SC
AZ AR
MS
17 contiguous states are labelled in the map. In addition, AK and HI fall into this category.
80-100%
60-80%
Percentage of new
bridge designs 40-60%
assigned to 20-40%
consultants
10-20% High-seismicity state DOTs (N=19)
<10% Other DOTs (N=32)
0% 5% 10% 15% 20% 25% 30% 35%
Percentage of DOTs
(a) New bridge designs
80-100%
60-80%
Percentage of
existing bridge 40-60%
analyses assigned 20-40%
to consultants
10-20%
<10%
(55% on average) than do the other DOTs (62% on average). For existing bridge analyses, the
average percentages assigned to consultants are similar for the high-seismicity and other DOTs
(47% and 48%, respectively).
Figure 50. Comparative DOT responses to six questions regarding in-house processes.
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55%
Percentage of DOTs
Figure 52 compares the percentage of responses for the analysis methods used in house for
the “often” and “never” frequencies. This comparison indicates that high-seismicity state DOTs use
more refined analysis methods—namely, linear-elastic and nonlinear finite element analysis—
than do the other DOTs. STM analysis is used with similar frequency by the high-seismicity state
and other DOTs.
CHAPTER 4
Case Examples
This chapter presents a review of selected state DOT practices for quality processes for bridge
analysis models. The objective is to expand on their processes related to quality processes for
bridge structural analysis models.
Five state DOTs were selected as case examples based on a set of criteria discussed in the
next section. The respondents, who completed the survey presented in Chapter 3, were con-
tacted by email and invited to web-based online interviews. All five agencies accepted the
interview requests. Case example interviewees were provided the interview questions (see
Appendix C) and encouraged to invite other individuals with relevant experience to these
interviews. Two respondents from each agency typically attended the interviews. The web-
based format allowed for reviewing agency documents during the interviews. In addition, the
agencies provided a few additional documents and written clarifications after the interviews
by email.
The following sections present the selection criteria, agency responses to the interview
questions, and related written process documents as indicated by the agencies during the
interviews.
42
Case Examples 43
• All job-related correspondence and memoranda are routed and received by affected persons
and then filed in the appropriate Task Order file.
• Field activities are routinely verified for accuracy and completeness, such that any discovered
deficiencies do not become systemic or affect the result of a deliverable Task Order.
For complex bridges, Caltrans’ Structure Technical Policy (STP) 1.3 (70) requires that project-
specific design criteria, a peer review panel, and an independent check free of conflict of interest
be established to ensure an enhanced QC for the accuracy of analysis results. The independent
check must be conducted by another company not associated with the group who has completed
the original analysis. The use of different software is required. The differences in analysis results
are resolved through the negotiated tolerance specified in the project-specific design criteria.
Depending on the project-specific task order, if a consultant is assigned the complete design and
check, Caltrans conducts the QA on the presence of a structural analysis model and an independent
analysis model created by another company not associated with the consultant who has completed
the original analysis. For some complex projects, based on the project-specific task order, the con-
sultant may execute the modeling and design aspects while Caltrans conducts independent mod-
eling and code compliance checks, or vice versa. This process involves independently modeling the
bridge and comparing the results. This decision is made based on staff availability and the complexity
of the project. In general, Caltrans is more involved if a project involves a complex bridge.
Case Examples 45
Currently 17 BDMs are posted on Caltrans’ website (73). Regarding quality processes for bridge
analysis models, these documents may help reduce errors with input parameters and modeling
assumptions while also providing guidelines on additional calculation methods for analysis result
verification.
Caltrans’ BDP Manual (74) provides bridge design engineers with basic design concepts,
assumptions, and step-by-step design examples. It also introduces Caltrans’ innovative design
practices. Regarding bridge analysis models, Chapter 4 includes guidelines on structural and
material modeling and structural analysis methods and provides solved bridge examples. This
document is more frequently used by newly hired engineers.
The current version of the BDP covers the following topics:
• Chapter 1: Bridge Design Specifications
• Chapter 3: Loads and Load Combinations
• Chapter 4: Structural Modeling and Analysis
• Chapter 5.1: Concrete Design Theory
• Chapter 5.2: Post-Tensioned Concrete Girders
• Chapter 5.3: Precast Pretensioned Concrete I-Girders
• Chapter 5.4: Precast Pretensioned Concrete Box Girders
• Chapter 5.5: Precast Pretensioned Concrete Voided Slabs
• Chapter 5.6: Concrete Bent Caps
• Chapter 5.7: Concrete Columns
• Chapter 6.1: Steel Design Theory
• Chapter 6.2: Steel Plate Girders
• Chapter 11.1: Abutments
• Chapter 11.2: Earth Retaining Systems
• Chapter 16.1: Strengthening Steel Girders for Live Loads
• Chapter 20.1: Seismic Design of Concrete Bridges
• Chapter 20.2: Seismic Design of Steel Bridges
All documents cited in this section have a publication date of 2019 or later. As new versions
of national design codes and standards are released, Caltrans updates its documents at certain
intervals. These updates are an example of continuous improvement in the context of QA/QC
processes. In addition, Caltrans shares these documents freely on its website for use by their in-
house engineers and consultants, among others.
Case Examples 47
and Procedures (PDPP) (70), the review panel may be composed of consultants, academics,
and in-house engineers. The independent check must be conducted by engineer(s) not under
the direction of the project development team or its immediate supervision. The use of
different software is required. Differences in analysis results are resolved through the tol-
erance specified in the project-specific design criteria. The PDPP also discusses the
roles of the “complex bridge design branch” and SASA. It states that, for complex bridges,
these branches may undertake the “designer” or “checker” functions but not both func-
tions for any given project. The table included in this document, shown as Table 1, indicates the
roles and responsibilities based on bridge elements and features. The PDPP contains a Quality
Control Plan Checklist, which must be completed and submitted by Caltrans staff.
For standard bridge projects, Caltrans management encourages, but does not require, the use
of different software for result verification. Caltrans has secured licenses for various software
packages that allow modeling the same types of bridge. Acceptable differences in analysis results
often are resolved by using the more conservative values. One challenge is to require engineers
to learn new software when they have grown accustomed to a particular commercial or software
developed in house.
Case Examples 49
in the RFP reviewed for this synthesis include the project team, firm capability, past performance
on similar projects or similar teams, approach, and small business participation. A detailed scope
of work is issued separately. Both documents are advertised on CDOT’s website (75) and BidNet
(76), which is CDOT’s intake platform.
CDOT forms a consultant selection panel for each RFP, and the panel members score each
proposal independently based on the evaluation criteria stated in the RFP. In addition to the
evaluation of the selection panel, the Civil Rights and Business Resource Center evaluates and
scores the affidavit of small business participation in the proposals.
A selection panel meeting is held to compile the scores and discuss panel members’ evaluations
of the proposals. At the end of this meeting, the highest-ranked firm is selected if an optional pre-
sentation and interview phase is not undertaken. Otherwise, a shortlist of the top three qualified
firms at minimum are selected for the presentation and interview phase. The shortlisted firms
are given approximately two weeks to prepare their presentations for their interviews with the
selection panel. The purpose of this interview is to allow the shortlisted consultants to present
their analysis of the project and the panel to clarify the consultants’ qualifications in a question-
and-answer session. Additional details related to the interview are provided to the shortlisted
consultants by email in advance of the interview. In selections including an interview phase, the
interview scores are added to the overall score of the proposal.
For FHWA-assisted contracts, the proposal with the highest final proposal score is evaluated
to determine whether it demonstrates good faith efforts to meet the contract goal in accordance
with the disadvantaged business enterprise and emerging small business requirements defined
in the RFP. The proposal with the highest final proposal score is not selected if it is determined
that the proposer did not demonstrate good faith efforts to meet the contract goal.
In making the final selection recommendation to the Chief Engineer, the panel reaches con-
sensus on the ranking of the firms and provides its recommendation to the Contracting Officer,
who provides selection documentation, including the panel’s ranking, to the Chief Engineer
for approval. The Chief Engineer’s approval is necessary before proceeding with the selection
notification.
Only firms that are pre-qualified at least seven calendar days before the proposal submittal
date can submit a proposal. The pre-qualification must be renewed annually. The pre-qualified
consultants must have an office in the State of Colorado. An online pre-qualification application
and list of pre-qualified consultants are provided on CDOT’s website (77).
The non-project-specific statewide solicitations are similar to the project-specific RFPs, with
a few differences. More than one firm may be selected for statewide contracts. The number of
firms selected is stated in each solicitation. A detailed scope of work is issued separately, similar
to the project-specific RFPs. The evaluation criteria for the solicitation reviewed for this synthesis
includes the project team, firm capability, approach, and small business participation.
CDOT considers the complexity of the project, regional office preferences, and availability of
qualified in-house staff when making decisions about assigning consultants to bridge projects.
CDOT also makes strategic decisions to select projects for in-house design to maintain and grow
staff expertise and capabilities.
by a design engineer who is a P.E. in the State of Colorado. This engineer may be employed by
the same firm. This check verifies all design work by producing a second set of complete design
and quantity calculations, which are submitted to CDOT. In addition, CDOT assigns a structural
reviewer to each consultant project, who reviews the project deliverables from an oversight per-
spective regarding conformance to CDOT bridge design practices.
Each consultant is required to have a rigorous QA/QC program that meets or exceeds CDOT’s
QA/QC requirements. When defining the processes for verifying in-house analysis models and
results, CDOT’s QA/QC process involves the following details.
Case Examples 51
CDOT issues technical memorandums (80) when expediency is required or as a means for
introducing new policies and procedures. Technical memorandums defining policy govern over
the associated in-house bridge manual and AASHTO specifications.
When asked what new national guidelines they would like to see published, CDOT indi-
cated the scarcity of national guidelines for finite element analysis of complex bridges, including
modeling of connections between girders and deck. CDOT would also like to see more bridge-
specific STM analysis guidelines and examples.
and approximately 130 existing bridge analyses in a typical year (5% of which are conducted by
external consultants).
Case Examples 53
design or rehabilitation design during contracting. The plan must list team members assigned to
the project and the responsibilities of each member. Independent design checks of all structural
elements are required as part of the QC plan.
The Consultant Coordination Unit reviews consultant plans and checklists (82) to ensure that
the policies are and will be followed. Noted issues are conveyed to the consultants for resolution.
In addition, this office arranges a QA/QC interview with consultants, commonly for new bridge
design projects, during the development of consultant plans. Unique aspects of the design are
also discussed. Meeting minutes are prepared and shared with the parties involved. The minutes
list the staff names associated with project roles, including the QA/QC manager, project manager,
lead engineer, review engineer, and technician. A sample of meeting minutes from February 2023
indicates the following related to verifying the analysis results:
• After the lead engineer self-checks the calculations, copies are made for use by the review
engineer.
• The review engineer highlights in yellow all that is agreed upon, redlines issues to be resolved,
and initials and dates the review prints in red.
• After resolution of the design issues by the lead engineer and review engineer, the calculations
are finalized by the lead engineer.
• The review engineer back-checks the finalized design calculations to verify that review comments
were implemented appropriately and initials and dates the calculations.
In addition, the sample meeting minutes state that the independent check of structural elements
will be conducted without access to the calculations conducted by the Lead Engineer, and that
the Independent Check Engineer will compare the analysis results with the Lead Engineer to
confirm that the design check is consistent with the analysis results.
In rare cases, Iowa DOT may hire additional consultants for independent reviews of the analysis
and design work completed by the project consultant. These reviews may be undertaken
for complex or unique bridge projects. One such current project involves a bridge replacement
project over the Mississippi River in Lansing, IA. Due to the complexities involved with the first
implementation of the AASHTO Guide Specification for Analysis and Identification of Fracture
Critical Members and System Redundant Members (83), Iowa DOT has hired an independent
review consultant. The role of this consultant is to conduct an independent design check of the
continuous steel truss bridge and other aspects of the plans and specifications for the project.
as one example. The team assigned by the unit leader to a design project includes a combina-
tion of experienced engineers, one of whom will often serve as the engineer of record, and
less experienced engineers to develop their skills. More staff are added to the team for more
complex projects and fast-tracked projects, such as response to over-height load impacts neces-
sitating emergency repairs. For simpler or ordinary projects, the designer and detailer may be the
same person.
Case Examples 55
Figure 54. Bridge analysis and design resources shared on Iowa DOT website (85).
Iowa DOT has a Methods Unit that currently employs five engineers and two technicians.
Each engineer is focused on a specific area of expertise, including software, policy and standards,
new engineer training, and digital delivery. The software engineer conducts analysis software
verification by creating examples of analysis models and comparing the results with other soft-
ware and hand calculations.
When asked what challenges they experience with commercial software, Iowa DOT empha-
sized the importance and current scarcity of software interoperability—that is, the functionality
of different programs to exchange information, share files, and use the same protocols. Inter
operability may include, for example, the design model to communicate with the analysis model,
which could help reduce input errors and facilitate a more efficient, seamless analysis and design
process.
Training may include analysis and design aspects of various bridge types and components and may
be developed and delivered by consultants, as per Iowa DOT’s needs and specifications. In addi-
tion, Iowa DOT benefits from the standard courses offered by various agencies such as the National
Highway Institute.
The new design engineering staff is given analysis and design assignments with progressively
increasing difficulty. A common starting point may be a standard slab bridge or a single span
prestressed concrete beam bridge, followed by the foundation and piling elements. This process
allows new design engineers to develop new skills and experience over time while also creating
the designer-checker pairing practice described previously.
As Iowa DOT experiences record staff retirements—50% of Bridges and Structures Bureau’s
workforce has been retired and replaced in the past five years—the Bureau stressed the impor-
tance of new staff training based on the initiatives described previously.
Case Examples 57
Each criterion receives a rating on a scale of one to five. This rating is multiplied by the corre-
sponding weight factor. The rating in each category is added to arrive at the proposal’s final rating,
resulting in a maximum achievable score of 150. LaDOTD’s project evaluation team is responsible
for conducting this evaluation, and presents a shortlist of the top three (if three are qualified) highest-
rated consultants to the Secretary, who makes the final selection from the short list. The submitted
proposals are posted on LaDOTD’s website (89).
Prime Evaluation
Score Overall Rating Justifications/Comments
Consultant Criteria
A
B
C
Consultant 1 D
E
F
Average
Source: Appendix G.3 (90).
For complex projects, in rare cases LaDOTD may also require the prime consultant to involve
a peer reviewer who is not employed by the same consultant. The peer reviewer is an experienced,
independent P.E. with no prior involvement in the project who checks the design calculations by
producing an independent set of calculations based on the drawings or conducts the review as
specified in the scope of work. The peer review comments must be submitted to LaDOTD and
the design team for evaluation. Resolutions agreed upon by all parties, including the designer,
peer reviewer, and LaDOTD, are incorporated in the final design.
Case Examples 59
LaDOTD has established a Bridge Design Software Committee to manage matters related to
software and 3D implementation. The committee currently has seven members: a chair and six
members, each with a specialty area focus, including general policy, software evaluation, bridge
information viewer app and Windows10 migration, 3D design and plan production for pilot
projects, CAD support, and IT support.
Upon completion of the design check, the designer prepares a QA information package and
provides this package to the reviewer. This package includes the following documents:
• QA information package check list as included in Appendix C (90).
• Calculation book
• Plans
• Special provisions including non-standard items
• Cost estimate
• Any relevant documents (e.g., checklists, review comments) that are utilized by the designer,
design checker, detailer, and detail checker
The reviewer is responsible for ensuring that the QC process, described previously, is complete
and the design calculations are in accordance with LaDOTD bridge design practices, policies, and
procedures. The reviewer must be a P.E. in the State of Louisiana and have substantial experience in
the design of similar structures.
During the QA process, the reviewer conducts a cursory review of all documents in the QA
information package submitted by the designer. This review focuses on areas of critical struc-
tural importance—areas where, based on the reviewer’s experience, mistakes may be typically
found and areas that may be new to the design practice. The reviewer may (but is not required)
to produce independent calculations to verify the submitted information. The reviewer provides
feedback to the designer and resolves all issues. Upon completion of the QA process, the design
calculations are considered final. At this point, the QA/QC certification, as shown in Table 3, is
signed by the designer, design checker, detailer, detail checker, and reviewer.
Peer review is conducted only at the request of the Bridge Design Engineer Administrator
for complex projects. The peer review is the process by which an independent engineering
entity with no prior involvement in the project conducts a check of the designs by producing an
Designers
Design
Checkers
Detailers
Detail
Checkers
Reviewers
Peer
Reviewer
Geotechnical
Engineer
Hydraulic
Engineer
Engineer of
Record
Source: Appendix D (90).
Case Examples 61
independent set of calculations based on the drawings or conducts the review as specified in the
scope of work.
The peer reviewer must be a P.E. in the State of Louisiana and have substantial experience in
the design of similar structures. The peer review comments must be submitted to LaDOTD and the
design team for evaluation. Resolutions agreed upon by all parties, including the designer, peer
reviewer, and LaDOTD, are incorporated in the final design. A Peer Review Resolution Agree-
ment, as included in Appendix E (90), must be signed by the peer reviewer, the supervisor or
team leader of the design team, and an LaDOTD representative.
The supervisor or team leader assigns an EOR for the project. The EOR is responsible for
sealing the calculations. The EOR must be a P.E. in the State of Louisiana and have commen-
surate experience in the design of similar structures. The EOR may be the designer, the design
checker, the reviewer, or the supervisor/team leader who is directly involved in the project design
activities.
it could be as high as seven years for complex and infrequent bridge projects. The design scoring
methodology for Process II is defined in detail in a dedicated document (96) and considers the
following rating factors:
• Prior experience
• Organizational capability for this type of work
• Logistics and familiarity
• Other factors (standard, special, technical proposal, and oral presentation)
• Minority/women business enterprise (MWBE) participation when federal funds are used or
disadvantaged business enterprise (DBE) participation when state funds are used
• MWBE subconsultants or DBE subconsultants
• Time since last designation
Each factor is given a weight determined for each project. All factors and weights are
described in the project advertisement. Two sets of weights are typically used: one for the
automated evaluation step and one for the committee evaluation step. For each step, consul-
tant teams’ individual scores for each factor are multiplied by the weight of the factor. The
weighted scores are added to produce a final aggregate score for the team. The weights for
Factors 1–4 total 100%. Factors 5–7 are bonus or penalty points added to or subtracted from the
aggregate team score. More details are provided in NYSDOT’s Design Scoring Methodology–
PROCESS II (96).
Selection Process I is for projects sufficiently unique that scoring of submitted past project
inventories would have little value in demonstrating a firm’s qualifications. This one-step process
requires the consultant to prepare and submit an electronic expression of interest. There is no
initial short list; the assignment of design work is made to the highest-rated firm after evaluation
of the information.
NYSDOT updates load ratings with every inspection. The department has approximately
13,500 existing bridges in the AASHTOWare software. Considering biannual inspections, load
ratings of 4,000–5,000 existing bridges are updated every year. Consequently, consultants are
employed much more frequently for the existing bridge analyses, including load ratings, than
new bridge design. While this number appears high, if there is no change in the loading or bridge
condition, which is common, the load rating numbers are not updated.
Case Examples 63
Case Examples 65
and evaluation of alternatives. An equivalent report template is also provided for bridge rehabilita-
tion projects in Appendix 19B: Complete/Phased Rehabilitation Bridge Rehabilitation Justification
Report (98). The justification reports are reviewed and approved by MO Structures and forwarded
to the Deputy Chief Engineer for Structures (i.e., State Bridge Engineer) for final approval.
• The analysis of superstructure primary members, piers, and atypical abutment types must be
verified either by a second software program with a level of model refinement comparable
to the original analysis software or independent hand calculations.
• The analysis of all remaining structural bridge components must be verified by at least one of
the following methods:
– Checking of the calculations or software program input
– Independent hand calculations
– A second software program
The QA/QC processes for analysis results verification are documented in the Design Task
List, which must be submitted with all milestone submissions. This document lists all project
components that require analysis, design, and drafting. Multiple columns capture the responsible
project personnel, including designer, checker, and QA engineer, who must initial and date com-
pleted project components. A sample partial Design Task List is provided in Table 4.
Table 4. Sample Partial Design Task List used as a part of QA/QC processes for structural analysis models.
PRELIMINARY
DESIGN CALCULATIONS
SUBSTRUCTURES
BRIDGEGEOMETRY
SUPERSTRUCTURE
LOAD RATING
ESTIMATES
Quality Processes for Bridge Analysis Models
CHAPTER 5
Summary of Findings
The objective of this synthesis is to identify and document state DOT practices related to
quality processes for bridge structural analysis models. Information was gathered through a
literature review (Chapter 2), an online survey of all DOT agencies yielding an overall response
rate of 100% (Chapter 3), and case examples for five selected state DOTs—California, Colorado,
Iowa, Louisiana, and New York (Chapter 4). This chapter summarizes the major findings of the
synthesis and indicates opportunities for future research.
Major Findings
The major findings from the survey are summarized as follows.
• The majority of DOTs assign more than half (59%) of their new bridge and bridge replacement
designs to consultants, while they assign almost half (47%) of their existing bridge analyses,
including load ratings, to consultants.
• The survey asked three questions about the presence of consultant quality processes. The most
common written consultant process, selected by 61% of DOTs, was “identifying appropriately
qualified consultants,” while the least common written process, selected by only 16% of DOTs,
was “verifying the analysis results obtained from consultants.” The most common “no process”
response, selected by 37% of DOTs, was “bridge analysis models developed by consultants.”
• The survey asked six questions about the presence of in-house quality processes. The most
common written in-house process, selected by 31% of DOTs, was “modeling a bridge.” The
least common written process, selected by only 12% of DOTs, was “identifying appropriately
qualified in-house engineers.” This result sharply contrasts with the consultant processes for
which “identifying appropriately qualified consultants” was the most common written process.
The most common “no process” response, selected by 41%, was “validating the analysis software.”
• State DOTs and their consultants most frequently use “one-dimensional line girder analysis.”
The least frequently used method is the “nonlinear finite element method.”
• The most common method for in-house analysis results verification is “checking of input vari-
ables.” “Another team of engineers uses a different method or software” and “analysis engineer
decides how to verify” were the other common responses. The least common method is the
use of NCHRP Process 12-50.
• The most common method for validating the analysis software is “analysis engineer decides
how to validate,” while the least commonly used validation method is “use of data from field
tests and sensor deployment.”
• The most common method for reconciling discrepancies between independent models is “the
same team of engineers works to resolve the discrepancies,” while the least common method
is “data from field tests and sensor deployment are used,” followed by “an external consultant
is involved to perform independent checks.”
68
Summary of Findings 69
• High-seismicity state DOTs assign a smaller percentage of their new bridge designs to consul-
tants and use more refined in-house analysis methods than the remaining DOTs. In contrast,
the percentage of high-seismicity state DOTs with written processes (for both consultant and
in-house projects) are lower than those of the remaining DOTs.
The major findings from the case examples are summarized as follows.
• Surveyed agencies use two types of consultant selection processes: project-specific or state-
wide on-call. The project-specific process issues a detailed RFP for a particular project and
selects one consultant. Consultants may be required to be pre-qualified in order to submit a pro-
posal. The statewide on-call process issues an RFQ and selects a pool of consultants to assign
projects. Evaluation processes and criteria are typically specified in both processes in the RFP
and RFQ.
• Surveyed agencies require consultants to either submit their QA/QC plans or follow the
agency’s specific QA/QC plans. As two examples, LaDOTD has well-defined evaluation and
scoring criteria for consultant QA/QC plans, while NYSDOT has specific QA/QC plans that
consultants must follow. Not all agencies that require consultants to submit their QA/QC plans
evaluate and score QA/QC plans—the presence of such plans and assurance that they will be
followed is considered adequate.
• All five surveyed agencies have informal processes that rely on a manager, supervisor, or unit
leader to select appropriately qualified engineers based on their experience and availability. This
decision also considers professional development needs, including training less experienced
engineers or challenging more experienced engineers with unique or interesting projects. The
district office associated with the bridge site commonly leads the projects.
• All five surveyed agencies most frequently use “one-dimensional line girder analysis.” While
all surveyed agencies rely on the analysis (or design) team to select the most appropriate
method(s), NYSDOT provides written guidelines on when to use refined analysis methods.
• All five surveyed agencies require a checker to independently verify the accuracy of design
engineer’s models, calculations, and results. For complex bridges, Caltrans requires project-specific
design criteria, a peer review panel, and an independent check conducted by an engineer not
associated with the group who has completed the original analysis. Both Caltrans and NYSDOT
require the use of different software in the independent check.
• To overcome the challenges of finding appropriately qualified engineers in district offices,
Caltrans established the SASA branch, which is focused only on structural modeling and
analysis, while NYSDOT established the MO Structures group with 65 design staff who con-
duct only structural analysis and final design. Note that NYSDOT assigns only 15% of new
bridge designs to consultants compared to the national average of 59%, while Caltrans assigns
only 15% of existing bridge analyses, including load ratings, to consultants compared to the
national average of 47%.
• For training engineering staff, Caltrans established a six-week “bridge design academy,” while
NYSDOT has a 24-session “Bridge 101” training series. Iowa DOT indicated the benefits of
designer-checker pairing and a dedicated training budget for the professional development
of engineering staff.
• Future research is suggested to develop a nationwide repository for sharing finite element
and STM models between DOTs, which may help collectively develop and iteratively improve
common types of models.
• A qualified training system with standardized requirements could be developed for bridge
engineers conducting finite element and STM analyses.
• New training courses and seminars could be developed to educate bridge engineers on how
to effectively conduct important but commonly misunderstood analysis activities, such as
verification, validation, uncertainty, error, and calibration.
References
1. The American Society of Mechanical Engineers. Standard for Verification and Validation in Computational
Solid Mechanics. ASME V&V 10-2019. ASME, New York, 2020.
2. Trucano, T.G., Swiler, L.P., Igusa, T., Oberkampf, W.L., and Pilch, M. Calibration, Validation, and Sensitivity
Analysis: What’s What. Reliability Engineering & System Safety. Vol. 91, 2006, pp. 331–357.
3. FHWA. Guidance on QC/QA in Bridge Design in Response to NTSB Recommendation (H-08-17). 2011.
{Available from: https://www.fhwa.dot.gov/bridge/h0817.pdf}
4. Babuska, I., and Oden, J.T. Verification and Validation in Computational Engineering and Science: Basic
Concepts. Computer Methods in Applied Mechanics and Engineering. Vol. 193, 2004, pp. 4057–4066.
5. Oberkampf, W., Hirsch, C., and Trucano, T. Verification, Validation, and Predictive Capability in Computa-
tional Engineering and Physics. 2003. {Available from: https://www.osti.gov/servlets/purl/918370/}
6. Technical Committee ISO/TC 176 Q management and quality assurance, SS 1, C and terminology. Quality
Management Systems-Fundamentals and Vocabulary ISO 9000:2015(E). ISO Stand. 2015. {Available from:
https://asq.org/quality-press/display-item?item=T1039&utm_source=qaqc&utm_medium=webpage&utm
_campaign=qaqc&utm_id=LAQ}
7. American Society for Quality. Quality Assurance and Quality Control. 2023. {Available from: https://asq.org
/quality-resources/quality-assurance-vs-control}
8. NCHRP. Best Practices in Quality Control and Assurance in Design. NCHRP Project 20-68A, Scan 09-01.
{Available from: https://onlinepubs.trb.org/onlinepubs/nchrp/docs/nchrp20-68a_09-01.pdf}
9. Hida, S., Ibrahim, F.I.S., Capers, H.A, Bailey, G.L., Friedland, I.M., Kapur, J., Martin, Jr., B.T., Mertz, D.R.,
Perfetti, G.R., Saad, T., and Sivakumar, B. Assuring Bridge Safety and Serviceability in Europe. 2010.
{Available from: http://international.fhwa.dot.gov/pubs/pl10014/pl10014.pdf}
10. von Wolffersdorff, P.-A., and Meier, T. Finite Element Analyses in Geotechnical Engineering–Some
Thoughts and Recommendations concerning Quality Assurance. 2011. pp. 211–19. {Available from:
https://www.baugrund-dresden.de/en/publications/documents/DGK-Beitrag-QM-Numerische-Methoden
_englisch_A4.pdf?m=1500381745}
11. Amati, L., Di Anselmo, A., Margheriti, C., and Kenn, J.M. Numerical Simulation Reliability; Quality Assur-
ance in Finite Element Analysis–A Review. WIT Transactions on Modelling and Simulation. 1993. {Avail-
able from: https://www.witpress.com/elibrary/wit-transactions-on-modelling-and-simulation/5/13168}
12. U.S. NRC. Software Quality Assurance Program and Guidelines (NUREG/BR-0167). 1993. {Available from:
https://www.nrc.gov/reading-rm/doc-collections/nuregs/brochures/br0167/index.html}
13. Mirsky, S.M., Hayes, J.E., and Miller, L.A. Guidelines for the Verification and Validation of Expert System
Software and Conventional Software. NUREG/CR-6316. 1995.
14. ANSI. Guidelines for the Verification and Validation of Scientific and Engineering Computer Programs for the
Nuclear Industry. ANS-10.4-1987 (R1998). American Nuclear Society, La Grange, IL, 2008.
15. Roache, P.J. Verification and Validation in Computational Science and Engineering. Hermosa Publishers.
1998. {Available from: http://pages.swcp.com/~hermosa/html/books.html#VVCSE}
16. Oberkampf, W.L., Trucano, T.G., and Hirsch, C. Verification, Validation, and Predictive Capability in Com-
putational Engineering and Physics. Applied Mechanics Reviews. Vol. 57, 2004, pp. 345–384. {Available from:
https://asmedigitalcollection.asme.org/appliedmechanicsreviews/article/57/5/345/464639/Verification
-validation-and-predictive-capability}
17. AIAA. Guide for the Verification and Validation of Computational Fluid Dynamics Simulations. AIAA
G-077-1998. American Institute of Aeronautics and Astronautics, Reston, VA, 1998.
18. Kraft E., and Marks, M. DoD Modeling and Simulation (M&S) Verification, Validation, and Accreditation
(VV&A). U.S. Gov. Counterterrorism, 2012.
71
19. NASA. NASA Handbook for Model and Simulations: An Implementation Guide for MASA-STD-7009. 2019.
20. Kwasniewski, L. On Practical Problems with Verification and Validation of Computational Models. Archives
of Civil Engineering. Vol. 55, 2009, pp. 323–346.
21. van Hees, P. Validation and Verification of Fire Models for Fire Safety Engineering. Procedia Engineering.
Vol. 62, 2013, pp. 154-168. {Available from: http://dx.doi.org/10.1016/j.proeng.2013.08.052}
22. Szabo, B., and Babuska, I. Introduction to Finite Element Analysis. Wiley, 2011. {Available from: https://
onlinelibrary.wiley.com/doi/book/10.1002/9781119993834}
23. Murad, J. Errors in FEA and Understanding Singularities (Beginners’ Guide). SimScale blog. 2023. [Accessed
January 27, 2023]. {Available from: https://www.simscale.com/blog/errors-in-fea-and-singularities/}
24. Mason, R.L., Gunst, R.F., and Hess, J.L. Statistical Design and Analysis of Experiments. John Wiley & Sons,
Inc., Hoboken, NJ, 2003. {Available from: http://doi.wiley.com/10.1002/0471458503}
25. Pike, D.J. Empirical Model-Building and Response Surfaces. Journal of the Royal Statistical Society Series A,
Royal Statistical Society. Vol. 151, 1988, pp. 223–224.
26. Kleijnen, J., and Helton, J.C. Statistical Analyses of Scatterplots to Identify Important Factors in Large-Scale
Simulations, 1: Review and Comparison of Techniques. Reliability Engineering & System Safety. 1999.
27. Saltelli, A., Ratto, M., Tarantola, S., and Campolongo, F. Sensitivity Analysis Practices: Strategies for Model-
Based Inference. Reliability Engineering & System Safety. 2006, pp. 1109–1125.
28. Saltelli, A., and Scott, M. Guest Editorial: The Role of Sensitivity Analysis in the Corroboration of Models and
its Link to Model Structural and Parametric Uncertainty. Reliability Engineering & System Safety. Vol. 57,
1997, pp. 1–4.
29. Frey, H.C., and Patil, S.R. Identification and Review of Sensitivity Analysis Methods. Risk Analysis. Vol. 22,
2002, pp. 553–578.
30. Ionescu-Bujor, M., and Cacuci, D.G. A Comparative Review of Sensitivity and Uncertainty Analysis of Large-
Scale Systems–I: Deterministic Methods. Nuclear Science and Engineering. Vol. 147, 2004, pp. 189–203.
31. Cacuci, D.G., and Ionescu-Bujor, M. A Comparative Review of Sensitivity and Uncertainty Analysis of
Large-Scale Systems –II: Statistical Methods. Nuclear Science and Engineering. Vol. 147, 2004, pp. 204–217.
32. Kwasniewski, L., and Bojanowski, C. Principles of Verification and Validation. Journal of Structural Fire
Engineering. Vol. 6, 2015, pp. 29–40.
33. Mullerschon, H. Optimization and Robustness Analysis in Structural Mechanics. 2007.
34. Cosner, R. CFD Validation Requirements for Technology Transition. Fluid Dynamics Conference, San Diego,
CA. 1995.
35. Sindir, M., Barson, S., Chan, D., and Lin, W. On the Development and Demonstration of a Code Validation
Process for Industrial Applications. Fluid Dynamics Conference, New Orleans, LA. 1996.
36. Dai, K., Boyajian, D., Liu, W., Chen, S.-E., Scott, J., and Schmieder, M. Laser-Based Field Measurement for
a Bridge Finite-Element Model Validation. Journal of Performance of Constructed Facilities. Vol. 28, 2014.
{Available from: https://ascelibrary.org/doi/10.1061/%28ASCE%29CF.1943-5509.0000484}
37. Reichenbach, M., White, J., Park, S., Zecchin, E., Moore, M., and Liu, Y., et al. Proposed Modification
to AASHTO Cross-Frame Analysis and Design. Transportation Research Board, Washington, DC, 2021.
Available from: https://doi.org/10.17226/26074.
38. Okumus, P., Oliva, M.G., and Arancibia, M.D. Design and Performance of Highly Skewed Deck Girder
Bridges. Report: 0092-16-05. Wisconsin Department of Transportation, 2018. {Available from: https://
rosap.ntl.bts.gov/view/dot/64315}
39. Quirke, P., and Barrias, A. Validation of Finite Element Light Rail Bridge Model Using Dynamic Bridge Deflection
Measurement. Conference: CERI–Civil Engineering Research, Ireland. 2020.
40. Xiang, Y., Fayaz, J., and Zareian, F. Validation of Caltrans Ordinary Bridge Modeling Approach Using CSMIP
Data. SMIP19 Seminar on Utilization of Strong-Motion Data, Los Angeles, CA, 2019. pp. 11–26. {Available
from: https://www.conservation.ca.gov/cgs/documents/program-smi/seminar/SMIP19_proceedings.pdf}
41. Zoghi, M., and Farhey, D. Performance Assessment of a Precast-Concrete, Buried, Small Arch Bridge.
Journal of Performance of Constructed Facilities. Vol. 20, 2006, pp. 244–252.
42. Hag-Elsafi, O., Albers, W.F., and Alampalli, S. Dynamic Analysis of the Bentley Creek Bridge with FRP
Deck. Journal of Bridge Engineering. Vol. 17, 2012, pp. 318–333. {Available from: https://ascelibrary.org/doi
/10.1061/%28ASCE%29BE.1943-5592.0000244}
43. Mlynarski, M., Puckett, J.A., Clancy, C.M., Goodrich, B.L., Jablin, M.C., Smyers, W., and Wilson, K. Bridge
Software Validation Guidelines and Examples. Transportation Research Record: Journal of the Transportation
Research Board, No. 1696, pp. 143–151. {https://doi.org/10.3141/1696-17}
44. Baker, Jr. M., Bridge Tech, Inc., Modjeski and Masters, Inc. and Thompson, P.D. NCHRP Report 485: Bridge
Software–Validation Guidelines and Examples. Transportation Research Board, Washington, DC, 2003.
45. Choe, L., Varma, A.H., and Wheeler E. Evaluation of Prestressed Concrete I-Girder Bridge Design Software
with NCHRP Process 12-50. Transportation Research Record: Journal of the Transportation Research Board,
No. 2131, 2009, pp. 34–46. {Available from: https://doi.org/10.3141/2131-04}
References 73
46. AASHTO. Manual for Bridge Evaluation. 3rd ed., 2022 Interim Revisions. American Association of State
Highway and Transportation Officials, Washington, DC, 2022.
47. Varma, A.H., and Seo, J. Verification of LRFD Bridge Design and Analysis Software for INDOT. FHWA/IN/
JTRP-2009/27. Technical Summary: Technology Transfer and Project Implementation Information. 2009.
{Available from: https://docs.lib.purdue.edu/cgi/viewcontent.cgi?article=2627&context=jtrp}
48. Gergely, J., Lawrence, T.O., Prado, C.I., Ritter, C.T., and Stille, W.B. Evaluation of Bridge Analysis Vis-à-Vis
Performance. Research Project No. HWY-2002-12. 2004.
49. AASHTO. AASHTOWare Bridge Design and Rating (BrD and BrR). AASHTOWare Bridge website. {Available
from: https://www.aashtowarebridge.com/bridge-rating-and-design/}
50. AASHTO. AASHTO LRFD Bridge Design Specifications. 9th ed. AASHTO, Washington, DC, 2020.
51. Adams, A., Galindez, N., Hopper, T., Murphy, T., Ritchie, P., and Storlie, V., et al. Manual for Refined Analysis
in Bridge Design and Evaluation. Report FHWA-HIF-18-046. Washington, DC, 2019.
52. Chandrupatla, T., and Belegundu, A. Introduction to Finite Elements in Engineering. 2021.
53. Logan, D.L. A First Course in the Finite Element Method. 6th ed. Thomson: Boston, MA, 2022. {Available
from: https://www.cengage.com/c/new-edition/9780357676424/}
54. Nielsen, M.P., and Hoang, L.C. Limit Analysis and Concrete Plasticity. 3rd ed. 2016.
55. Marti, P. Basic Tools of Reinforced Concrete Beam Design. Journal Proceedings. Vol. 85, 1985, pp. 46–56.
{Available from: https://doi.org/10.14359/10314}
56. Schlaich, J., Schafer, K., and Jennewein, M. Toward a Consistent Design of Structural Concrete. PCI
Journal. Vol. 32, 1987, pp. 74–150. {Available from: https://www.pci.org/PCI/Publications/PCI_Journal
/Issues/1987/May–June/Toward_a_Consistent_Design_of_Structural_Concrete.aspx}
57. Hooke, R. Lectures de Potentia Restitutiva, Or of Spring Explaining the Power of Springing Bodies. London,
1678. p. 56.
58. Bernoulli J. History of the Royal Academy of Sciences . . . with the dissertations in mathematics and
physics . . . drawn from the records of this Academy. Paris; 1760. p. 572. {Available from: https://gallica.bnf.fr
/ark:/12148/bpt6k3558n}
59. Navier C. On the Resistance of Solid Bodies (3rd edition) / by Navier . . . ; with notes and appendices,
by M. Barré de Saint-Venant . . . Paris; 1864. p. 851. {Available from: https://gallica.bnf.fr/ark:/12148
/bpt6k112967g}
60. Martin, Jr., B.T., and Sanders, D.H. Verification and Implementation of Strut-and-Tie Model in LRFD Bridge
Design Specifications. American Association of State Highway and Transportation Officials, Washington,
DC, 2007.
61. Ashour A.F., and Yang, K-H. Application of Plasticity Theory to Reinforced Concrete Deep Beams: A Review.
Magazine of Concrete Research. Vol. 70, 2007.
62. Joint ACI-ASCE Committee 445. Strut-and-Tie Method Guidelines for ACI 318-19–Guide. ACI PRC-445.2-21.
American Concrete Institute, 2021.
63. Baniya, P., and Guner, S. Specialized Strut-and-Tie Method for Rapid Strength Prediction of Bridge Pier
Caps. Engineering Structures. Vol. 19, 2019. {Available from: http://www.utoledo.edu/engineering/faculty
/serhan-guner/docs/JP12_Baniya_Guner_2019.pdf}
64. Baniya, P., and Guner, S. STM-CAP (Strut-and-Tie Method for Pier CAPs). The University of Toledo, OH,
2017. {Available from: https://www.utoledo.edu/engineering/faculty/serhan-guner/STMCAP.html}
65. Ohio DOT. Bridge Design Manual. 2020 ed. 2023. {Available from: https://www.transportation.ohio.gov
/working/engineering/structural/bdm)
66. Qualtrics. Qualtrics Online Survey Platform. Qualtrics website. {Available from: https://www.qualtrics.com/}
67. ASCE. Minimum Design Loads and Associated Criteria for Buildings and Other Structures, ASCE/SEI 7-22.
American Society of Civil Engineers, Reston, VA. 2022. {Available from: http://ascelibrary.org/doi/book
/10.1061/9780784415788}
68. Marsh, M.L., and Stringer, S.J. Performance-Based Seismic Bridge Design. Transportation Research Board,
Washington, DC, 2013. Available from: {https://doi.org/10.17226/22632}
69. Caltrans. Project Development Procedures Manual (PDPM). 2023. {Available from: https://dot.ca.gov
/programs/design/manual-project-development-procedures-manual-pdpm}
70. Caltrans. Structure Technical Policies (STPs). 2020. {Available from: https://dot.ca.gov/programs
/engineering-services/manuals/structure-technical-policies}
71. Caltrans. AASHTO LRFD CA Amendments. 2022. {Available from: https://dot.ca.gov/programs
/engineering-services/manuals/aashto-lrfd-ca-amendments}
72. Caltrans. Seismic Design Criteria (SDC). 2019. {Available from: https://dot.ca.gov/programs/engineering-services
/manuals/seismic-design-criteria}
73. Caltrans. Bridge Design Memos (BDMs). 2023. {Available from: https://dot.ca.gov/programs/engineering-services
/manuals/bridge-design-memos}
74. Caltrans. Bridge Design Practice (BDP). 2022. {Available from: https://dot.ca.gov/programs/engineering-services
/manuals/bridge-design-practice}
75. CDOT. Advertised Projects. {Available from: https://www.codot.gov/business/consultants/advertised
-projects}
76. BidNet Direct. CDOT Construction & Engineering Services Bid Opportunities. {Available from: https://www
.bidnetdirect.com/colorado/cdotconstructionengineeringservices}
77. CDOT. Consultant Pre-Qualification List, Work Codes and Application Internet. {Available from: https://
www.codot.gov/business/consultants/consultant-pre-qualification-list}
78. CDOT. Bridge Design Manual. 2023. {Available from: https://www.codot.gov/programs/bridge/bridge
-manuals/design_manual}
79. CDOT. Bridge Rating Manual. 2022. {Available from: https://www.codot.gov/programs/bridge/bridge-
manuals/rating_manual/bridge_rating_manual_2022_09.pdf}
80. CDOT. Technical Memorandums. {Available from: https://www.codot.gov/programs/bridge/bridge-manuals
/tech-memos}
81. Iowa DOT. Conducting Business with the Iowa DOT Bridges and Structures Bureau. 2021. {Available from:
https://iowadot.gov/projectdev/Consultant-Resources}
82. Iowa DOT. Bridge Plan Review. 2023. {Available from: https://iowadot.gov/bridge/Design-Policies
/Bridge-and-Culvert-Plan-Checklist}
83. AASHTO. AASHTO Guide Specifications for Analysis and Identification of Fracture Critical Members and
System Redundant Members, with 2022 Interim Revisions. 1st ed. American Association of State Highway
and Transportation Officials, Washington, DC, 2018.
84. Iowa DOT. Final Bridge Design Software. {Available from: https://iowadot.gov/bridge/automation-tools
/final-design-software#446761464-cip-reinforced-concrete-box-culvert-program}
85. Iowa DOT. Bridge Analysis and Design Resources. {Available from: https://iowadot.gov/business}
86. Iowa DOT. LRFD Bridge Design Manual. {Available from: https://iowadot.gov/bridge/design-policies
/lrfddesignmanual}
87. LaDOTD. Advertisements and Addenda. {Available from: http://wwwsp.dotd.la.gov/Inside_LaDOTD
/Divisions/Engineering/CCS/Pages/Advertisements.aspx}
88. LaDOTD. DOTD FORM: 24-102: Proposal to Provide Consultant Services.{Available from: http://wwwsp
.dotd.la.gov/Inside_LaDOTD/Divisions/Engineering/CCS/ProposalDocs/4400023689%20and%20
4400023690%20IDIQ%20Safety%20Studies/44-23689%20-%2044-23690%20Quality%20Engineering.pdf}
89. LaDOTD. Proposals. {Available from: http://wwwsp.dotd.la.gov/Inside_LaDOTD/Divisions/Engineering
/CCS/Pages/Proposals.aspx}
90. LaDOTD. Bridge Design and Evaluation Manual. {Available from: http://wwwsp.dotd.la.gov/Inside_
LaDOTD/Divisions/Engineering/Bridge_Design/Pages/BDEM.aspx}
91. LaDOTD. Pre-Approved Software List. {Available from: http://wwwsp.dotd.la.gov/Inside_LaDOTD
/Divisions/Engineering/Bridge_Design/QCQA/Pre-Approved%20Software%20List.pdf}
92. LaDOTD. Standard Plans/Special Details. {Available from: http://wwwsp.dotd.la.gov/Inside_LaDOTD
/Divisions/Engineering/Standard_Plans/Pages/default.aspx}
93. LaDOTD. Bridge Design Technical Memoranda (BDTM). {Available from: http://wwwsp.dotd.la.gov
/Inside_LaDOTD/Divisions/Engineering/Bridge_Design/Pages/Technical-Memoranda.aspx}
94. NYSDOT. CSS Web. {Available from: https://www.dot.ny.gov/main/business-center/consultants/css-web}
95. NYSDOT. NYSDOT 255 Shortlist Submittal Form. {Available from: https://www.dot.ny.gov/main/business
-center/consultants/consultants-repository/instr_nys255_ver07-2022.pdf }
96. NYSDOT. Design Scoring Methodology–PROCESS II. {Available from: https://www.dot.ny.gov/main
/business-center/consultants/consultants-repository/DesScore_Revised%2006-01-20.pdf}
97. NYSDOT. Project Development Manual, Appendix 12: Quality Control and Quality Assurance. {Available
from: https://www.dot.ny.gov/divisions/engineering/design/dqab/pdm}
98. NYSDOT. Bridge Manual. {Available from: https://www.dot.ny.gov/divisions/engineering/structures
/manuals/bridge-manual-usc}
99. NYSDOT. LRFD Bridge Design Specifications. {Available from: https://www.dot.ny.gov/divisions/engineering
/structures/manuals}
100. NYSDOT. Standard Specifications for Highway Bridges. {Available from: https://www.dot.ny.gov/divisions
/engineering/structures/manuals}
101. NYSDOT. Seismic Design Guidelines for Bridges in Downstate Region. {Available from: https://www.dot
.ny.gov/divisions/engineering/structures/manuals/seismic-references}
102. NYSDOT. Structures Design and Analysis Program. {Available from: https://www.dot.ny.gov/divisions
/engineering/structures/design/design-analysis-programs}
103. NSBA. Guidelines for Steel Girder Bridge Analysis. G13.1-2019. {Available from: https://www.aisc.org
/globalassets/nsba/aashto-nsba-collab-docs/g-13.1-2019-guidelines-for-steel-girder-bridge-analysis
.pdf}
APPENDIX A
Survey Questionnaire
75
The Transportation Research Board (TRB) is preparing a synthesis on state DOT Quality Processes for Bridge Analysis Models. This
is being done for the National Cooperative Highway Research Program (NCHRP), under the sponsorship of the American Association
of State Highway and Transportation Officials (AASHTO), in cooperation with the Federal Highway Administration (FHWA).
The purpose of this survey is to document state DOT practices related to quality processes for bridge structural analysis models. More
specifically, the following processes will be documented: identifying appropriately qualified staff, choosing valid analysis methods and
software, modeling the structure, and verifying the analysis results. The results of the survey will be incorporated into a synthesis of
highway practice, with the intent of helping DOTs evaluate and improve their quality processes for bridge structural analysis models
including both super- and sub-structures.
This survey is being sent to voting members of the AASHTO Committee on Bridges and Structures. If you are not the appropriate
person at your agency to complete this survey, please forward it to the right person. A PDF version of the survey is attached to the
email for reference. The survey is designed so that the respondent can exit and return to the survey (from the same computer) if he/she
needs more time or input from others. We estimate that the survey should take no more than 20 minutes to complete. Please note that
DOT survey responses will be shown in the published synthesis report. However, the identity of the survey respondents will remain
anonymous.
Please complete and submit this survey by February 3, 2023. If you have any questions or concerns, please contact the principal
investigator Dr. Serhan Guner.
Thank you for sharing your time and expertise with the state DOT community.
1. How many bridge engineers currently work in your agency (direct employees only, not including consultants)?
Copyright National Academy of Sciences. All rights reserved.
0-10
10-20
20-40
40-60
60-80
80+
2. Approximately how many new bridge and bridge replacement designs are completed in a typical year in house (by your agency) and
external consultants (total number of designs)?
0-10
10-20
20-40
40-60
60-80
80+
3. Approximately what percentages of new bridge and bridge replacement designs in a typical year are completed by external
consultants?
Less than 10%
10% to 20%
20% to 40%
40% to 60% (approximately half)
60% to 80%
80% to 100%
Quality Processes for Bridge Analysis Models
4. Approximately how many existing bridges are analyzed in a typical year in house (by your agency) and external consultants (total
number of analyses)? Existing bridge design projects include load ratings, rehabilitation, retrofit, or any analysis performed on an
existing bridge.
Less than 25%
25-50%
Copyright National Academy of Sciences. All rights reserved.
50-75%
75-100%
100-125%
125+
5. Approximately what percentages of existing bridge analyses in a typical year are conducted by external consultants? Existing bridge
design projects include load ratings, rehabilitation, retrofit, or any analysis performed on an existing bridge.
Less than 10%
10% to 20%
20% to 40%
40% to 60% (approximately half)
60% to 80%
80% to 100%
6. Does your agency have any written or informal processes for identifying appropriately qualified consultants for the structural design
of bridges?
Written (please upload the document or input a download link if possible)
Informal
No
7. How does your agency select appropriately qualified consultants for bridge design projects? Check all that apply.
Pre-qualification requirements
Proposal evaluation
Interview
Other (please specify):
Quality Processes for Bridge Analysis Models
Q8 only appears if ‘pre-qualification requirements’ is selected in Q7.
8. What requirements does your agency’s pre-qualification process include? Check all that apply.
Minimum years of design experience
Minimum number of similar project experience
Minimum number of design engineers for the project
Structural Engineer (S.E.) credentials based on project complexity
Other (please specify):
9. Does your agency have any written or informal quality processes for bridge analysis models developed by consultants?
Written (please upload the document or input a download link if possible)
Copyright National Academy of Sciences. All rights reserved.
Informal
No
10. Does your agency have an approved software list for bridge modeling and analysis?
Yes
No
11. Does your agency track what methods of analysis are used for bridges analyzed by the consultants?
Yes
No
14. Does your agency require an independent consultant, as the ‘checker,’ to verify the accuracy of the calculations performed by the
project consultant?
Yes, for every bridge
Yes, for most bridges
Yes, only for complex bridges
Yes, based on a case-by-case decision
Only if specified in the selected proposal of the consultant
Not required.
15. Does your agency perform an in-house check of the analysis results obtained from consultants to verify accuracy?
Yes, for every bridge
Yes, for most bridges
Yes, only for complex bridges
Yes, based on a case-by-case decision
Not required
16. Does your agency have any written or informal processes for identifying appropriately qualified in-house engineers for bridge
analyses?
Written (please upload the document or input a download link if possible)
Informal
No
Quality Processes for Bridge Analysis Models
17. Does your agency have any written or informal processes for choosing a suitable analysis method and/or software?
Written (please upload the document or input a download link if possible)
Informal
No
18. What methods of analysis are used for bridges analyzed in-house by your agency?
19. Does your agency have any written or informal processes for modeling a bridge (e.g., boundary conditions, material properties,
loading distribution, type of element, soil structure interaction)?
Written
Informal
No
20. Does your agency have any written or informal processes for validating the analysis software?
Written
Informal
No
Validation is the process of confirming that structural analysis software provides results that adequately represent the real physical
behavior of the structure being modeled. Methods for validating structural analysis software include comparing predictions of the
software to experimental results or benchmarks available in the literature.
Quality Processes for Bridge Analysis Models
Q21 only appears if Q20 is answered ‘written’ or ‘informal.’
21. How does your agency validate the analysis software for medium- to high-complexity bridges and substructures that require 2D or
3D analysis models?
22. Does your agency have any written or informal processes for verifying in-house analysis results?
Written
Informal
No
Verification is the process of confirming that the analysis is performed correctly and with the correct input. Methods for validating
structural analysis results might include comparing the results with the results obtained from another software, tool, or spreadsheet, or
from hand calculations.
Q23 only appears if Q22 is answered ‘written’ or ‘informal.’
23. How does your agency verify the in-house analysis results for medium- to high-complexity bridges and substructures that require
2D or 3D analysis models?
24. Does your agency have any written or informal processes for reconciling discrepancies between independent models?
Written
Copyright National Academy of Sciences. All rights reserved.
Informal
No
Would you be willing to participate in a follow-up interview with the principal investigator to provide additional details for a possible
case example for the synthesis report?
Yes
No
Quality Processes for Bridge Analysis Models
APPENDIX B
The following tables show the distribution of DOT responses for each question in the survey.
Q5. Approximately
what percentage of
Q2. Approximately Q3. existing bridge
Q1. How how many new Approximately Q4. Approximately how many analyses in a typical
many bridge bridge and bridge what percentage existing bridges are analyzed year are conducted
engineers replacement of new bridge in a typical year in-house (by by external
State DOT currently designs are and bridge your agency) and external consultants?
and work in your completed in a replacement consultants (total number of Existing bridge
Washington, agency (direct typical year in- designs in a analyses)? Existing bridge design projects
DC employees house (by your typical year are design projects include load include load ratings,
only, not agency) and completed by ratings, rehabilitation, retrofit, rehabilitation,
including external external or any analysis performed on retrofit, or any
consultants)? consultants (total consultants? an existing bridge. analysis performed
number of designs)? (%) on an existing
bridge.
(%)
AK 20–40 80+ Less than 10 125+ Less than 10
AL 10–20 20–40 10–20 Less than 25 Less than 10
AR 20–40 20–40 40–60 125+ Less than 10
AZ 10–20 0–10 20–40 Less than 25 Less than 10
CA 80+` 40–60 40–60 125+ 10–20
CO 20–40 0–10 40–60 75–100 60–80
CT 80+ 10–20 80–100 125+ 60–80
DC 0–10 0–10 80–100 Less than 25 80–100
DE 20–40 10–20 40–60 125+ 40–60
FL 80+ 80+ 80–100 125+ 80–100
GA 40–60 60–80 80–100 125+ 60–80
HI 0–10 0–10 80–100 50–75 80–100
IA 40–60 40–60 80–100 125+ Less than 10
ID 10-20 20–40 40–60 125+ 60–80
IL 40–60 80+ 80–100 125+ 20–40
IN 20–40 80+ 80–100 125+ 80–100
KS 10–20 40–60 60–80 100–125 40–60
KY 0–10 60–80 60–80 125+ 60–80
LA 40–60 20–40 60–80 100–125 20–40
84
PA Written X X
RI Written X X X
SC No X
SD Written X X
TN Informal X X
TX Written X X X
UT Written X X X
VA Written X X
VT Informal X X
WA Written X X
WI Informal X X
WV Informal X X
WY Informal X
Q6. If the document is available in a web page, you can paste the address into the text box below.
CO https://www.codot.gov/business/consultants/consultant-pre-qualification-list
GA https://www.dot.ga.gov/GDOT/Pages/EngineeringConsultantQualification.aspx
IL https://idot.illinois.gov/doing-business/procurements/engineering-architectural-professional-
services/prequalification.html
IN https://www.in.gov/indot/doing-business-with-indot/files/CPQM.pdf
KS https://www.ksdotike.org/about/doing-business-with-kdot
MN http://www.dot.state.mn.us/consult/prequal/worktype/work-type-3.1.docx
MT https://www.mdt.mt.gov/other/webdata/external/CDB/consultant_manual/consultant-design-
manual_combined.pdf#page=67
NH https://www.nh.gov/dot/org/projectdevelopment/highwaydesign/consultants/documents/consultant -
selection-manual-03012022.pdf
OH https://www.transportation.ohio.gov/working/engineering/consultant-services/manuals-and-contract-
documents/1-consultant-contract-admin
UT https://udot.utah.gov/connect/business/consultant-services/consultant-pools/
VA https://www.vdot.virginia.gov/doing-business/technical-guidance-and-support/technical-guidance-
documents/manual-for-the-procurement--management-of-professional-services/
WA https://wsdot.wa.gov/engineering-standards/all-manuals-and-standards/manuals/consultant-services-
manual
Q7. Other (please specify):
▪ We also use average evaluation scores from previous work.
▪ Proposal evaluation performed by design staff generates only a short list. Consultant selection is made
by the DOT Secretary.
▪ Interviews are only for large design-build projects.
▪ Limited Services Contract (LSC) selection process
▪ Consultant ratings for past projects
▪ Work history/Past performance/RFP with specific requirements
AK No
AL X Written
AR No
AZ No
CA X Written
CO Written
CT X X X Written
DC X X X Informal
DE Written
FL X X X No
GA X No
HI X Informal
IA X Written
ID X X Informal
IL X X X No
IN X X No
KS X Written
KY X X No
LA No
MA X X Written
MD X X Informal
ME X No
MI X X X X Informal
MN X X X Written
MO X X Written
MS No
MT X X X X Informal
NC X X X X Informal
ND X Written
NE X X X Informal
NH No
NJ X X X X X Written
NM Written
NV X X Informal
NY Written
OH X X No
OK X X X No
OR X X X Written
PA X X X Written
RI X X X X Written
SC No
SD X Written
TN X X X Informal
TX X X Written
UT X X X Written
VA No
VT No
WA X Written
WI No
WV X X X Informal
WY No
Q9. If the document is available in a web page, you can paste the address into the text box below.
CO https://www.codot.gov/programs/bridge/bridge-manuals/rating_manual
DE https://bridgedesignmanual.deldot.gov/index.php/Main_Page
IA https://iowadot.gov/bridge/Design-Policies/LRFDdesignmanual
MN https://edocs-public.dot.state.mn.us/edocs_public/DMResultSet/download?docId=10175226
MO http://epg.modot.org/index.php/Category:751_LRFD_Bridge_Design_Guidelines
ND https://www.dot.nd.gov/construction-and-planning/construction-planning/research/design-manual
RI https://www.dot.ri.gov/business/contractorsandconsultants.php
UT https://drive.google.com/file/d/1R38biq78EI9Q1JiFyO -MI4icubM2Zdp5/view
https://drive.google.com/file/d/1R38biq78EI9Q1JiFyO
WA https://wsdot.wa.gov/engineering-standards/all-manuals-and-standards/manuals/bridge-design-manual-lrfd
OH Yes No
OK No No
OR Yes No
PA Yes No
RI Yes Yes Often Often Sometimes Sometimes Sometimes
SC No No
SD Yes No
TN Yes No
TX Yes No
UT No No
VA Yes No
VT No Yes Often Sometimes Sometimes Rarely Sometimes
WA Yes No
WI No Yes Often Sometimes Sometimes Sometimes Often
WV No No
WY No No
Quality Processes for Bridge Analysis Models
AK X
AL X
AR X
AZ X
CA X
CO X
CT X
DC X
DE X
FL X
GA X
HI X
IA X
ID X
IL X
IN X
KS X
KY X
LA X
MA X
MD X
ME X
MI X
MN X
MO X
MS X
MT X
NC X
ND X
NE X
NH X
NJ X
NM X
NV X
NY X
OH X
OK X
Q13. If the document is available in a web page, you can paste the address into the text box below.
CO https://www.codot.gov/programs/bridge/bridge-manuals/rating_manual
MN http://www.dot.state.mn.us/bridge/pdf/lrfdmanual/section01.pdf
RI https://www.dot.ri.gov/documents/doingbusiness/RIDOT_LRFR_Guidelines.pdf
Q14. Does your agency require an independent consultant, as the “checker,” to verify the accuracy of
the calculations performed by the project consultant? [Q14 only appears if Q13 is answered “written”
State DOT or “informal”].
and
Washington, Yes, only Yes, based Only if specified in
DC Yes, for every Yes, for most for on a case- the selected Not
bridge bridges complex by-case proposal of the required
bridges decision consultant
AK
AL X
AR
AZ
CA X
CO X
CT X
DC X
DE
FL
GA X
HI X
IA
ID
IL X
IN X
KS X
KY
LA X
MA X
MD X
ME
MI X
MN X
MO X
MS
MT X
NC X
ND X
NE X
NH X
NJ X
NM X
NV
NY X
OH
OK X
OR
PA X
RI X
SC
SD X
TN X
TX
UT
VA
VT
WA
WI X
WV X
WY
NY X Informal Written
OH Informal Written
OK X No No
OR Informal Written
PA X Written Written
RI X No Written
SC No Written
SD X No Informal
TN X Informal Informal
TX Informal Informal
UT Informal No
VA No Informal
VT Informal Informal
WA Informal Written
WI X Informal Informal
WV X Informal No
WY No Informal
Q17. If the document is available in a web page, you can paste the address into the text box below.
IA https://iowadot.gov/bridge/Automation-Tools/Final-Design-Software
KS See the Bridge Design Manual sections previously attached.
MN https://edocs-public.dot.state.mn.us/edocs_public/DMResultSet/download?docId=10175226
OH https://www.transportation.ohio.gov/wps/wcm/connect/gov/500258d3-ee36-462f-b7d6-
422c9caba872/2020+BDM_07-15-22-
Optimized.pdf?MOD=AJPERES&CONVERT_TO=url&CACHEID=ROOTWORKSPACE.Z18_M1HGGIK0N0J
O00QO9DDDDM3000-500258d3-ee36-462f-b7d6-422c9caba872-ofiiKLn
RI https://www.dot.ri.gov/documents/doingbusiness/RIDOT_LRFR_Guidelines.pdf
SC https://www.scdot.org/business/pdf/structural-design/SCDOT_Bridge_Design_Manual.pdf
WA https://wsdot.wa.gov/engineering-standards/all-manuals-and-standards/manuals/bridge-design-
manual-lrfd
Q21. How does your agency validate the analysis software for medium- to high-complexity
bridges and substructures that require 2D or 3D analysis models? [Q21 appears only if Q20
is answered “written” or “informal”].
State DOT Modeling of
Q22. Does your agency have any
and benchmark Use of data
Hiring of Use of data written or informal processes for
Washington, structures or from bridge Analysis
external from field Other verifying in-house analysis results?
DC specimens inspection engineers
consultants tests and (please
(tested records for decide how
for software sensor specify):
experimentally) existing to validate
validation deployment
and comparing bridges
results
AK Often Informal
AL Informal
AR Informal
AZ Informal
Q23. How does your agency verify the in-house analysis results for medium- to high-complexity bridges and
substructures that require 2D or 3D analysis models? [Q23 appears only if Q22 is answered “written” or “informal”]. Q24. Does your
agency have any
Same team written or informal
State DOT Another team
of engineers processes for
and of engineers
uses a Comparison of reconciling
Washington, uses a Analysis
different results with Other discrepancies
DC different Use of NCHRP Checking of input engineers
method or similar bridges (please between
method or Process 12-50 parameters decide how
software to analyzed specify): independent
software to to verify
analyze the previously models?
analyze the
same
same bridge
bridge
AK Often Often Written
AL Often Rarely Often Never Often Often Informal
AR Sometimes Never Rarely Never Often Never No
AZ Often No
CA Rarely Often Never Never Often Often Informal
CO Sometimes Often Rarely Never Sometimes Often No
CT Rarely Rarely Sometimes Never Often Often No
DC No
DE Sometimes Sometimes Often Never Often Often No
FL Written
GA No
HI Sometimes Sometimes Often Rarely Often Rarely No
IA Sometimes Sometimes Rarely Never Sometimes Sometimes No
ID Often Often Sometimes Never Often Sometimes Written
IL Often Rarely Sometimes Rarely Often Often Informal
IN Sometimes Sometimes Rarely Never Sometimes Rarely Informal
KS Often Often Often Often No
KY Sometimes Sometimes Informal
Would you be
Q25. How does your agency reconcile discrepancies? [Q25 appears only if Q24 is answered “written” or willing to
participate in a
“informal”].
follow-up
interview with
State DOT
the principal
and The same Another team of An external
The more Data from investigator to
Washington, team of in-house consultant Analysis
conservative field tests Other provide
DC engineers engineers is involved engineers additional details
set of and sensor (please
works to performs to perform decide how for a possible
results are deployment specify):
resolve the independent independent to reconcile case example for
used are used
discrepancies checks checks the synthesis
report?
AK Often Yes
AL Often Rarely Never Never Never Often Never No
AR No
AZ Yes
CA Often Rarely Rarely Often Never Often Yes
CO Yes
CT Yes
DC No
DE No
FL Never Sometimes Often Sometimes Sometimes Rarely Yes
GA Yes
HI No
IA No
ID Often Often Rarely Rarely Never Rarely No
IL Often Rarely Rarely Sometimes Never Often Yes
IN Rarely Often Sometimes Rarely Rarely Rarely No
KS No
KY Often Rarely Never Sometimes Never Often No
LA Often Sometimes Rarely Rarely Sometimes Often Yes
MA Yes
MD No
ME Often Yes
MI Often Often Rarely Rarely Rarely Rarely Yes
MN Often Never Never Often Never Often Yes
MO Often Never Never Sometimes Never Often No
MS No
MT Often Sometimes Sometimes Sometimes Never Sometimes No
NC Often Rarely Never Never Never Often Never Yes
ND Sometimes Often Rarely Sometimes Sometimes Sometimes No
NE Often Rarely Rarely Often Never Sometimes No
NH Often Rarely Rarely Never Never Often No
NJ Often Often Rarely Often Sometimes Often No
NM Often Often Sometimes Often Rarely Often No
NV Yes
Would you
be willing to
Q25. How does your agency reconcile discrepancies? [Q25 only appears if Q24 is answered “written” or participate in
“informal”]. a follow-up
interview
with the
State
The same An external principal
DOT Another team The more Data from
team of consultant is Analysis investigator
of in-house conservative field tests Other to provide
engineers involved to engineers
engineers set of and sensor (please additional
works to perform decide how
performs inde- results are deployment specify): details for a
resolve the independent to reconcile.
pendent checks. used. are used. possible case
discrepancies. checks.
example?
NY Often Often Sometimes Often Rarely Often Yes
OH No
OK Often Sometimes Rarely Often Never Sometimes No
OR Often Often Often Rarely Rarely Often Often Yes
PA Sometimes Rarely Never Never Rarely Rarely Yes
RI Often Sometimes Sometimes Often Sometimes Often No
SC Often Never Never Sometimes Never Often Yes
SD Often Rarely Rarely Rarely Rarely Often Yes
TN Often Rarely Never Often Never Often No
TX Often Never Rarely Sometimes Never Often Sometimes No
UT Yes
VA Often Sometimes Rarely Often Sometimes Sometimes No
VT Often Sometimes Sometimes Rarely Sometimes Often No
WA Often Rarely Never Sometimes Never Never Often Yes
WI Often Often Sometimes Sometimes Rarely Sometimes No
WV Yes
WY Yes
APPENDIX C
104
ISBN 978-0-309-70958-3
90000
9 780309 709583