Standard Practice for

Acceptance Sampling Plans

for Highway Construction

AASHTO Designation: R 9-05 (2018)

Tech Section: Se, Quality Assurance and Environmental
Release: Group 1 (April)

AASHID
American Association of State Highway and Transportation Officials
444 North Capitol Street N.W., Suite 249
Washington, D.C. 20001
Standard Practice for

Acceptance Sampling Plans

for Highway Construction

AASHTO Designation: R 9-05 (2018) AASHID

Tech Section: Se, Quality Assurance
and Environmental
Release: Group 1 (April)

1. SCOPE

1.1. Definition ofAcceptance Plan-An agreed-upon method oftaking samples and making
measurements or observations on these samples for the purpose of evaluating the acceptability of a
lot of material or construction (Transportation Research Circular Number E-C037, 2002).

1.2. Purpose ofAcceptance Plan-This standard practice provides guidelines in the preparation of
statistically based acceptance plans using statistical and quality assurance (QA) principies. The
acceptance plans must be realistic, fair to both the contractor and agency, and statistically
accurate. This standard practice includes the basic criteria for the most common AASHTO
applications. It uses the broad definition of acceptance plan to include the separate functions of
both quality control (QC) and acceptance. Other aspects are provided where deemed important,
but should not be interpreted to define a complete QA program. References are given for more
advanced criteria and special applications.

1.3. Target Audience-It is assumed that the user is a mid-level materials engineer and has an
understanding of statistics, including the calculation of the mean and standard deviation, the
development ofhistograms, and the use of continuous probability density functions or
distributions.

1.4. Background Sources--Preparatory knowledge of acceptance plans is included in the course

entitled Materials Control and Acceptance- Quality Assurance (2000); in the manual Optimal
Procedures for Quality Assurance Specificatíons (2003 ); and in most basic statistics textbooks.

2. REFERENCED DOCUMENTS

2.1. AASHTO Standards:

• R 10, Definition ofTerms Related to Quality and Statistics as U sed in Highway Construction
• R 25, Technician Training and Certification Programs
• R 38, Quality Assurance ofStandard Manufactured Materials

2.2. ASTM Standards:

• D3665, Standard Practice for Random Sampling ofConstruction Materials
• E29, Standard Practice for Using Significant Digits in Test Data to Determine Conformance
with Specifications

TS-5c R 9-1 AASHTO

© 2018 by the American Association of State Highway and Transportation Officials.
A.ll rights reserved. Duplication is a violation of applicable law.
 • E178, Standard Practice for Dealing with Outlying Observations
• MNL 7, Manual on Presentation of Data and Control Chart Analysis, 8th edition, 201 O,
Committee E-11 on Quality and Statistics

2.3. FHWA Standard:

• FHW A-RD-02-095, Optimal Procedures for Quality Assurance Specijications, 2003

3. TERMINOLOGY

3.1. The following de.finitions are provided in this standard practice to help the reader become
acclimated to general QA terms:

3.1.1. population (lot)--a specific quantity of similar material, construction, or units of product subject
to either an acceptance or a process control decision. This can range from all of the history of a
material to a single lot (Transporta/ion Research Circular Number E-C037, 2002).

3.1.2. sample-a small part of a population (lot) that represents the whole (R 10). This implies a
statistical sample. Thus, the use ofthe term sample size, n, denotes the number oftest values used
to make a decision. (This should not be confused with size of sample that indicates a quantity of
material.)

3.1.3. sampling and testing-sampling, testing, and the assessment oftest results done to determine
whether or not the quality ofproduced material or construction is acceptable in terms ofthe
specifications (R 10). The results are best used to estímate a population. This is true for both QC
and acceptance functions. To estímate a population, two measures are needed; one is the center of
the estimated population and the other is a measure ofits variability.

3.1.4. types ofacceptance p/ans-a statistical acceptance plan is one based on an analysis of either
variables or attributes. This standard practice focuses on analysis ofvariables for acceptance.
Analysis by attributes is based on noting the presence or absence of sorne characteristic or
attribute. Attribute analysis is most often used in visual inspections or when an itero can only be
classified as either acceptable, not acceptable, or pass/fail. Attribute plans are sometimes used with
"screening tests" in which the material is tested before it is incorporated in the construction.
Variables analysis is applicable to materials and construction in which quality is evaluated by
measuring the numerical magnitude of a quality characteristic. A quality characteristic is a
characteristic of a unit or product that is actually measured for acceptance purposes
(Transportation Research Circular Number E-C037, 2002).

3.1 .5. variability known acceptance p/ans-acceptance plans that assume a known and constant
variability. These types ofacceptance plans measure only the average and are not appropriate for
highway materials and construction.

3.1.6. variability unknown acceptance p/ans-acceptance plans that measure both the product average
and variability as estimates of a population. This is the type of acceptance plan discussed in this
standard practice.

3.2. For those de.finitions that are not described within, two usefol reference sources are:

3.2.1. Defmition ofTerms for Specifications and Procedures (R 10).

3.2.2. Glossary ofHighway Quality Assurance Terms (Transporta/ion Research Circular Number
E-C03 7, 2002).

TS-5c R 9-2 AASHTO

© 2018 by the American Association of State Highway and Tnmsportation Officials.
All righte reoe!"YOO. Duplicntion iJ n -;iolntion ofappli(;llblt: law.
4. OEVELOPMENT ANO REVISION OF ACCEPTANCE PLANS

4.1. The development and the revision ofacceptance plans is a design process that should continuously
evolve and improve as new processes, test methods, etc., are found. The development steps
outlined below pro vide a rational development procedure and will be useful to those embarking on
an initial statistical acceptance plan. The procedure assumes that sorne users may want to know
how to initiate, proceed through the development, and eventually implement the plan. For tbose
revising an existing plan, sorne steps may not be needed. However, in both cases it is important
that the plan be analyzed to ensure it produces the desired results. The user may enter the steps at
any point of interest.
• Initiation and Planning
• Acceptance Plan Development Steps
• Develop QC Procedures
• Develop Acceptance Procedures
• Risks and Risk Analysis
• Pay Factors
• Implementation Steps

5. INITIATION ANO PLANNING

5.1 . It is important to identiiy and establish the needs for a new or modified acceptance plan. Sorne
possible reasons for doing this are:
• The present quality levels are substandard.
• Premature failures have occurred that are related to the current acceptance plan.
• A new or different quality measure, e.g., standard deviation, percent within limits, etc., is
needed.
• A new quality characteristic, e.g., permeability, resilient modulus, thickness, smoothness, etc.,
upon which to base acceptance has been identified.
• The risks between contractor and agency are not properly balanced or are too high for one
or both.
• A more rigorous acceptance procedure is needed.

5.2. Define the goals and expectations ofthe acceptance plan. The primary goal is to identi:ty potential
benefits to the agency and to the industry. To achieve this goal, determine the criteria to be used to
judge success. Examples ofthese criteria may be improved quality, a more knowledgeable
industry, shared responsibility with industry, and!or faster completion times.

5.3. At this point, seek industry acceptance of and participation in the development of the acceptance
plan. This is necessary to aid in industry cooperation and eventual positive reception.

6. ACCEPTANCE PLAN OEVELOPMENT STEPS

6.1. Develop the initial acceptance plan for one material and!or construction area, as opposed to
severa! simultaneously. This is recommended to simplif)r the development process.

6.2. Determine if outside expertise is required. The agency may not have sufficient knowledge in such
areas as statistical analysis or new areas such as nondestructive testing.

6.3. Look for other agencies or sources that have developed practices to provide guidance on how to
proceed with selection of quality characteristics, the use of pay equations, composite pay factors,

TS-5c R 9-3 AASHTO

© 2018 by the American Association ofState Highway and Transportation Officials.
1'1.11 righb rC3crvcd. Duplil:alioll i3!1 viulaliull uf<tpplic<tlJlc law.
 etc. Contact and interview those agencies and keep in mind industry associations that may have
practices that can be used. Learning frotn the mistakes that others have made can save time and
effort. Conduct a thorough literature search to identify current practices. Recent Transportation
Research Board and National Cooperative Highway Research Program publications can provide
guidance.

6.4. Develop an outline for the acceptance plan. This includes such items as:

6.4.1. QC requirements,

6.4.2. Acceptance requirements,

6.4.3. Independent assurance (IA) requirements,

6.4.4. Agency validation (if contractor's tests are used in the acceptance decision),

6.4.5. Conflict resolution, and

6.4.6. Determination of pay factors, both individual and combined.

6.5. Develop the acceptance plan introductory information. A fundamental concept in statistical
acceptance plans is the separation ofthe QC and acceptance functions. The contractor is always
responsible for QC and the agency is responsible for acceptance. Maintaining the separation ofthe
functions for QC and acceptance is very important. Additionally, address the needs and
requirements for both technician and laboratory qualification in the introductory information
(R25).

6.6. A very important point that must be understood is the relationship between sample statistics and
population parameters for both QC and acceptance procedures. The sample result is always an
estimate ofthe population. The larger the sample size, the better the estímate ofthe population.

6.7. Random Sampling-Random sampling is a sampling procedure whereby any individual

measurement in the population is as likely to be included as any other (Materials Control and
Acceptance-Qualíty Assurance, 2000).

6.8. A basic assumption in a sampling plan is the necessity oftaking samples in a random manner
(Materials Control and Acceptance-Quality Assurance, 2000). Random sampling is achieved
most commonly through the use of random number tables or random number generators on
calculators or computers (ASTM D 3665). Random sampling always removes the selection ofthe
sample tonnage, location, time, etc., from the decision ofthe sampler.lntuitively selecting a
"random" sample is not random. Random sampling is a necessary requirement for all acceptance
and lA sampling. Under sorne circumstances it is required for QC sampling.

6.9. Describe and discuss the need for the lA function and how it will be implemented in the
acceptance plan. Independent assurance is a management too! that requires a third party, not
directly responsible for QC or acceptance, to provide an independent evaluation ofthe
testing process.

6 .9.1. The lA program is an independent evaluation oftesting procedures and equipment and inspection
procedures and is designed to provide continuity to the acceptance plan. It may involve a separate
and distinct schedule of sampling, testing, and observation.

6.9.2. It is very important that an lA program compare results and detect deficiencies, when they exist, in
a timely manner. This improves the reliability of testing results. The timely comparison of data

TS-5c R9-4 MSHTO

© 2018 by the American Association of State Highway and Transportation Officials.
Atl ri!!jhtll r=rYw. Duplivlltion i:; a violaliun ofapplicable law.
 may be restricted by agency resources, including personnel, facilities, and geographical
constraints. These resource needs must be considered in the lA program.

6.9.3. It is very important that qualified personnel conduct IA tests and use different sampling and
testing equipment than the equipment used for QC or acceptance.

7. DEVELOP QUALITY CONTROL (QC) PROCEDURES

7.1. Quality Control (QC)-Also called "process control." Those QA actions and considerations
necessary to assess and adjust production and construction processes so as to control the leve! of
quality ofthe end product (Transportation Research Circular Number E-C037, 2002).

7.2. Purpose of QC-The purpose of a QC plan is to measure quality characteristics and inspect
activities that impact production at a time when corrective action can be taken to prevent
nonconforming material from being incorporated in the project. (See AASHTO Implementation
Manual for Quality Assurance (1995) for examples of QC plans for hot mix asphalt (HMA)
concrete and both structural and paving portland cement concrete (PCC)).

7.3. A QC plan can be either contractor-specific or generic. Ideally, the plan should be contractor-
and operation-specific. However, sorne agencies choose to develop a generic planto be used by
all contractors or suppliers. In either case, the contractor should develop controllimits based on
the production capabilities ofthe specific operation.

7.4. A very important part of a QC plan is the requirement that those persons performing the tests be
qualified. This can be accomplished severa! ways, but sorne agencies require certification to
determine qualification. (See R 25 and R 38 for more detailed information.)

7.5. The QC function is the responsibility ofthe contractor and is a separate function from acceptance.
This means that controllimits used for QC control charts must be developed separately from the
specification limits.

7.6. Measure the quality characteristics chosen for QC early in the production activity to assess the
quality as quick:ly as practica!. Therefore, the QC tests should be relatively rapid tests.

7.7. Examples ofQC tests for hot mix asphalt (HMA) are aggregate quality tests, binder physical
properties, gradation of critica! sieves, degree of aggregate coating, plant and discharge
temperatures, moisture content ofthe fme aggregate and ofthe finished mix, and the use ofa
density gauge for field compaction control.

7.8. Examples of QC tests for portland cement concrete (PCC) are aggregate quality tests, gradation of
critica! sieves, air content, water-cement content, mix temperature, and slump.

7.9. The use of control charts is an important part ofthe QC function. Statistical control charts are the
most effective control charts to use. Ibis tool provides, in real time, the contractor and agency a
visual estímate ofthe population ofthe quality cbaracteristic being produced.

7.10. Because estimating the population being produced is the purpose of QC, the use of the most
appropriate procedures is recommended. The most effective and simplest pair of statistical control
charts is the joint use of:

7.10.1. An average chart (X chart) to measure the center.

7.10.2. A range chart (R chart) to measure the variability. The range is recommended because it is simple
to calculate and record.

TS-5c R 9-5 AASHTO

© 2018 by the American Association of S tate Highway and Transportation Officials.

Atl rishw r<:~9rv~. D\lpli\ll>tion io o ..-iololion of npplio:;ubh:; luw.
7.11. An example of an average and range control chart is shown in Figure l.

7.11.1. The mean ofthe X chart is the center ofproduction. This mayor may not be the center ofthe
specification limits. In fact, there may not be a specification requirement for this QC measure.

7.11.2. Controllimits, also called "action limits," are boundaries established by statistical analysis for
material production control using the control chart technique. When values ofthe process fall
within these limits, the process is "under control" (R 10). The limits are set at the equivalent of
±3crx from the mean ofthe X chart. Both an upper controllimit (UCL) anda lower controllimit
(LCL) are used.

7.11.3. Because an estímate ofvariability is desired, the sample size used to plot statistical control charts
must be at least two, i.e., n ~ 2.

7.11.4. Warning limits are boundaries established on control charts within the UCL and LCL to warn the
producer of possible problems in the production process that may lead to the process going "out of
control" (R 10). These limits are often set at ±2cr x. When values fall outside these limits, there is
approximately a 5 percent chance the process is out of control. (For the purpose of clarity, these
are not shown on Figure 1.)

7.11.5. The center ofthe R chart is the mean range and the control Jimits are typically set at ±3crR. In this
case, crR is the standard deviation of the ranges. However, the LCL for the R chart is zero for most
practica! sample sizes.

7.12. Controllimits are intended to show when a process is out of control. The chance of a point being
more than ±3crx away from the average is only 0.3 percent. So, a value outside the action lirnit is
an indication that the process has changed.

7.12.1. In the case that a point is more than ±3cr x from the average, the contractor may decide to quickly
take another test in an attempt to confirm that the process has, in fact, changed before altering
production. Ifthe second test result is close to the first, this provides an indication that the process
may have changed. (If a check test is made, include both test results in the QC documentation.)
Although the check sample would not have been obtained in a random manner, it is a permissible
course ofaction. This is one good example ofwhy QC and acceptance must be maintained as
separate functions.

7.13. The calculation and use of control chart limits are beyond the scope of this standard practice. (See
ASTM MNL 7A for more information on control charts.)

TS-Sc R9-6 AASHTO

© 2018 by the American Association of State Highway and Transportation Officials.
1\11 rightJ rwc;rvoo. Duplic!ltion b ¡¡ viol!ltion of applicabl~ law.
 50.0
Average Chart UCL=48.79
49.0

48.0

47.0
O>
r::: 46.0
·¡¡;
1/1
~ 45.0
-~
r::: 44.0
Q)
a. 43.0

42.0

41.0 LCL =41.29

40.0
o 10 20 30 40 50 60 70 80 90 100
Lotor Day

14.0
Range Chart
12.0
UCL = 11.73

10.0
O>
r:::
"ii) 8.0
1/1
<O

-~
a.
r::: 6.0

Q)
a. 4.0

2.0
LCL = 0.00
0.0
o 10 20 30 40 50 60 70 80 90 100

Lotor Day

Figure 1-Examples of Average and Range Charts for the 2.36-mm Sieve

8. DEVELOP ACCEPTANCE PROCEDURES

8.1. Acceptance-Sampling and testing, or inspection, to determine the degree of compliance with
contract requirements (Transportation Research Circular Number E-C037, 2002).

8.2. Purpose ofAcceptance Procedures-These procedures are intended to accept adequate quality
leve! product and reject inadequate quality level product as often as is feasible.

8.3. Most previous work in acceptance plans has been in tenns ofpercent within Iimits (PWL) and
sorne work has been done using percent defective (PD). Percent within limits and PD are
complementary and add up to 100 percent. One can be converted to the other by Equation 1:
PD+PWL=IOO% (1)

TS-5c R 9-7 AASHTO

© 2018 by the American Association ofState Highway and Transportation Officials.
i\11 righm roooTYod. Duplicntion iJ n violation of applildlblc law.
 Thls relationship is shown graphically in Figure 2. The PWL is sometimes called the "percent
conforming"; also, the "percent nonconforming" may be a more appropriate term than percent
defective. The PWL approach is preferred by sorne agencies over PD because it is a positive
expression. Also, in terms ofthe theory of acceptance sampling plans, the material outside ofthe
limits may not be strictly defective but of lesser quality than material within the limits.

~-------Total Area of Curve= 100% or 1.0 --------+1

USL

Percent within Limits Percent Defectiva

4.00 5.00 6.00 7.00 8.00 9.00 10.00 11.00 12.00

Quality Measure

Figure 2-Complementary Relationship between Percent witbin Limits and Percent Defective

8.4. The PWL procedure produces an estímate ofthe population. Conceptually, as the sample size, n,
increases, the estímate ofthe population improves. The PWL concept is one ofsmall sample sizes
and tbe distribution used is the noncentral t distribution. This distribution converges with tbe
normal distribution when n approacbes infinity. Because PWL is an estímate of a population, it
must use an estímate ofboth the central tendency and the variability.

8.5. Specification and Acceptance Limits-In the development of an acceptance plan, specification and
acceptance limits must be determined, and the acceptable quality level (AQL) and rejectable
quality leve! (RQL) must be selected. The AQL, RQL, specification limits, and acceptance limits
are intimately related, and the decisions regarding these are typically made concurrently.

8.6. Specification Limits-The limiting value(s) placed on a quality characteristic, established

preferably by statistical analysis, for evaluating material or construction within the specification
requirements. The term can refer to either an individual upper specification limit (USL) or lower
specification limit (LSL), called a "single specification limit"; orto USL and LSL together, called
"double specification limits" (Transportation Research Circular Number E-C037, 2002). As
discussed below, specification and acceptance limits are different and serve different purposes.

8.7. Specification Jimits are based on engineering requirements and are expressed in the same units as
those used for the quality characteristic of concern (e.g., percent, millimeters, kilograms per square
millimeter, etc.). The acceptance limits are the limiting values placed on a quality measure that
will permit acceptance oftbe lot. In PWL acceptance plans they are expressed in statistical
units (PWL).

8.8. Establishing the Specification Limits-Establishing specification limits requires determining what
constitutes acceptable (AQL) and unacceptable (RQL) material. The determination ofboth AQL
and RQL are engineering decisions. The AQL decision defines acceptable material or
construction, and addresses the product that will provide satisfactory service at an affordable cost
when used for the intended purpose. Acceptable material is often determined based on what has
performed well in tbe past. However, ifperformance data are available, it is preferable to quantify
performance. Statistics has been a valuable tool in defining the parameters (mean and standard

TS-5c R 9-8 AASHTO

© 2018 by the American Association of State Highway and Transportation Officials.
i\11 rightJ rc:~ervcd. Dupli!;<!Liun i~ a violation or applicalJie law.
 deviation) for acceptable material. Caution should be exercised if a lower variability is chosen for
the specification than has been shown to be readily achievable. Arbitrarily "tightening the specs"
can increase the cost ofthe product above what is considered cost-effective.

8.8.1. Acceptable Quality Leve/ (AQL)-That mínimum leve! of actual quality at which the material or
construction can be considered fully acceptable (for that quality characteristic). For example, when
quality is based on PWL, the AQL is that actual (not estimated) PWL at which the quality
characteristic canjust be considered fully acceptable (Transportation Research Circular Number
E-C037, 2002). Acceptance plans should be designed so that AQL material will receive an average
pay over the long run of 100 percent.

8.8.2. Rejectable Quality Leve! (RQL)-That maximum leve) of actual quality at which the material or
construction can be considered unacceptable (rejectable) (Transportation Research Circular
Number E-C037, 2002). For example, when quality is based on PWL, the RQL is that actual (not
estimated) PWL at which the quality characteristic canjust be considered fully rejectable. [It is
desired to require removal and replacement, corrective action, or the assignment of a relatively
low pay factor when RQL work is detected.]

8.8.3. Setting specification limits is slightly more complicated for double specification limits as
compared to a single specification limit. For a quality characteristic with a single specification
limit, the limit can be set at the appropriate number of standard deviations (process variability)
above the minimum desired value to obtain the necessary PWL. But for a specification with
double limits, there may be concem that in addition to the number of standard deviations (process
variability) used to set the limits, a "target miss" may be warranted.

8.8.4. Target Miss-The failure of a contractor to center the process exactly on the target value. (For
more detailed information on target miss, see Optimal Procedures for Quality Assurance
Specifications, 2003.)

8.8.5. The proper way to address the issue of"target miss" is to determine how variable the actual
process means are about the target value. This variability regarding where the process will be
centered, called "target miss variability," can then be combined with the typical process variability
to obtain the correct standard deviation value for use in establishing specification limits. The
''target miss variability" and the "process variability" can be combined simply by adding their
associated variances, not their standard deviations (see Equations 2 and 3). This assumes that the
amount ofprocess variability is independent ofwhere the process is centered. (The example in
Appendix XI includes a calculation of"target miss" variability.)
'2 '2 '2
acombined = atarget - miss + aprocess (2)

.
(j combined = J2
(j combined (3)
where:
•2
Ocombined estimated combined target miss and process variance,
•2
O target- miss estimated target miss variance,
•2
(Jprocess estimated process variance, and

(jcombined estimated combined standard deviation.

8.9. Select the specification limit(s) in concert with the quality measure, i.e., the measure that is used to
quantify quality and the AQL and RQL. For instance, the AQL might be set at 90 PWL. This
means that when the population has 90 percent ofthe product within specification limits, the
product is completely acceptable. However, the same product could be defined by an AQL of

TS-5c R 9-9 AASHTO

© 2018 by the American Association ofState Highway and Transportation Officials.
i\\1 ri~htl r~:;~~:;rrcd . Duplicntíon i~ n v iolntiuu ur <!pplil,;alJlt: law.
 85 PWL with more restrictive specification limits or by many other possible combinations of AQL
and specification limits. Selecting lower values of AQL may give the perception that lower quality
levels are acceptable. It is recommended that AQL values be set at 90 or 95 PWL unless there is
an overriding reason to select another value.

8.9.1. It is important to distinguish between a population (e.g., N= 4) and a sample (e.g., n = 4 ), because
different equations are used to determine a population's variability than are used to estímate a
sample's variability. When working with a population, it is desirable to have a large number of
data points. However, sample sizes upon which to base acceptance or process control decisions are
typically small, e.g., from four to ten, depending on the lot size. Larger lot and sample sizes are
suggested to offer the advantage of lowering the risks of making a wrong acceptance or rejection
decision.

8.10. Acceptance Limit-In variables acceptance plans, the limiting upper or lower value, placed on a
quality measure, which will permit acceptance of a lot. (Unlike specification limits placed on a
quality characteristic, an acceptance limit is placed on a quality measure. For example, in PWL
acceptance plans, PWL refers to specification limits placed on the quality characteristic, and the
mínimum allowable PWL identifies the acceptance limit for the PWL quality measure.)
(Transporta/ion Research Circular Number E-C037, 2002)

8.11. Acceptance Limits with Pay Aqjustments-For acceptance plans with pay adjustment provisions,
additional acceptance limits, often expressed in the form of an equation or equations, are used to
distinguish among the various possible pay levels.

8.12. An Example ofSpecification and Acceptance Limits, A QL and RQL:

8.12.1. An agency has decided, based on a large number of project data that it collected and analyzed, that
a "typical" standard deviation for a lot defined in the acceptance plan for asphalt content for HMA
is 0.18 percent. It has further been decided that PWL will be used as the quality measure. (See the
example in Appendix Xl on how to properly select the "typical" standard deviation.)

8.12.2. Determine AQL Material-Because asphalt content has a stipulated target value, i.e., the job mix
formula (JMF) asphalt content, the agency may choose to determine AQL material as a lot for
which the average asphalt content is equal to the JMF target value and for which the standard
deviation is equal to or less than the "typical" value of0.18 percent. This defines AQL material in
terms ofthe desired population mean and standard deviation, but the AQL definition must also be
related to the required quality measure, which in this case is PWL.

8.12.3. Set Specification Limits-Because the specification Iimits and the AQL are related, the agency
might decide to set the AQL as a value of90 PWL. This selection of90 PWL for the AQL is
arbitrary but is a commonly used value and one suggested in the AASHTO Quality Assurance
Guide Speci.fication (1995). The AQL population defined from past projects in terms ofmean and
standard deviation should just meet this PWL definition for AQL. So, in this case, the
specification limits would be set such that a population with a mean at the JMF and a standard
deviation of0.18 percent would have 90 percent ofits area within the specification limits. These
limits can be determined by finding the Z-value from a standard normal table that corresponds to
an area of0.90 within the mean± Z standard deviations (i.e., Jl ± Za). (This example assumes the
contractor can hit the JMF consistently, and no target miss is necessary. See the example in
Appendix XI on the addition ofa target miss value.)

8.12.4. Table 1 presents sorne typical ±Zregions within which selected areas ofthe normal distribution
fall. From this table, it is seen that 0.90 (or 90 percent) ofthe normal distribution falls within
±1.645 standard deviations from the population mean.

TS-5c R 9-10 AASHTO

© 2018 by the American Association ofState Highway and Transportation Officials.
All righrs reserveá. Duplication is a violation or applicaiJJe law.
Table 1-~ ± Zcr Regions for Selected Areas under the Normal Distribution
Are a 0.99 0.95 0.90 0.85 0.80
±Z 2.576 1.960 1.645 1.439 1.282

8.12.5. Figure 3 shows the specifications set at the JMF asphalt content ±1.645 times the typical standard
deviation value, or JMF ± 1.645 (0.18 percent) = JMF ± 0.30 percent. In this case, the AQL is
90 PWL and the specification limits are JMF ± 0.30 percent.

8.12.6. Alternatively, using the same AQL population, the agency could decide to establish the AQL as
85 PWL. In this case, the specification limits would be set at the JMF ± 1.439 times the typical
standard deviation value (see Table l ), or JMF ± 1.439 (0.18 percent) = JMF ± 0.26 percent. The
specification limits are different in this case because the definition for AQL in terms ofPWL is
different. However, although the same population is desired in both cases, the perception is that
quality is notas important with an AQL of85 compared toan AQL of90.

~=JMF

a=0.18%

LSL USL

90PWL

1.64511
1.645o

Figure 3- AQL Material for Example 8.12

8.12.7. Determine RQL Material- There is no single correct way to establish either the AQL or the RQL.
Once the AQL and specification limits are established, the RQL can be deterrnined in a number of
ways. One way is to decide that the material should be rejected once a "large" percentage of
material is outside the specification limits. Determining what constitutes a "large" percentage then
needs to be decided. The agency may decide that the material is unacceptable once half of it is
outside ofthe specification Iimits. In this case, the RQL would be established as a PWL value
of 50. Any lot with an estimated PWL value of 50 or less would then be required to be removed
and replaced or other penalty applied. Typical values for RQL are between 70 and 30 PWL.

8.12.8. Alternately, the agency might base the definition ofRQL on the analysis ofpast project data. The
agency might decide that past projects had performed inadequately when the average asphalt
content was 0.25 percent above or below the JMF target value. In this case, the agency might
decide to set the RQL based on the PWL value for a population with a mean 0.25 percent above or
below the JMF target and with a standard deviation equal to the "typical" value of0.18 percent.
Figure 4 illustrates the case ofthe RQL population when the specification limits are JMF ± 0.30
percent. In this case, it can be seen that the PWL for the RQL population corresponds to the area
ofthe population that Iies within the specification limits. In this case, the RQL might be defined as
a lot with a PWL value of60 (rounded from 61) as shown in Figure 4.

TS-Sc R 9-11 MSHTO

© 2018 by the American Association ofStatc: Highway and Transportation Officials.
All rights rt:M:rvt:ú. Dupllcauon is a vlolarton or appllcallle Iaw.
 JMF

0.25%

a=0.18% USL

61 PWL

0.30%

Figure 4-RQL Material for Example 8.12

8.12.9. The above approach to defining RQL material contains a number of simplifying assumptions.
It looks only at how far the population mean departs from the target value and does not consider
the population standard deviation. This assumes that the typical standard deviation value of
0.18 percent will be achieved on all projects. This approach also does not consider the interaction
and effect of other quality characteristics, such as density, thickness, etc., on the performance of
past projects.

8.13. Application of PWL-Determining the PWL is similar in concept to determining the area under
the normal curve. The theory in the use ofthe PWL (or PD) method assumes that the population
being sampled is normally distributed. In practice, it has been found that statistical estimates of
quality are reasonably accurate provided the sampled population is at least approximately normal;
i.e., reasonably bell shaped and not bimodal or highly skewed. This has been found to be true for
most material and construction measures not associated with a physical constraint; e.g., because
gradation cannot exceed 100 percent passing a top size sieve, a measure ofthis characteristic is
often skewed and therefore not normally distributed.

8.13.1. When the average and standard deviation are known (or assumed known), the area under the
normal curve can be calculated to determine the percentage ofthe population that is within certain
limits. However, when the average and standard deviation are unknown--as in the case of
determining the acceptance for a lot-the percentage ofthe Jot that is within the specification
limits can be estimated. In this latter case, instead ofusing the Z-value and the standard normal
curve, the quality index, Q, is used to estímate PWL. The Q-value is used with a PWL table to
determine the estimated PWL for the lot.

8.13.2. A sample table relating Q values with the appropriate PWL estímate is shown in Table 2 for a
sample size ofn = 5. A more complete set ofPWL tables in this format, for sample sizes from
n = 3 ton= 30, is available (Quality Assurance Softwarefor the Personal Computer, 1996).

TS-5c R 9-12 AASHTO

© 2018 by the American Association ofState Highway and Transportation Officials.
Al! riglns reserveo. Duplicauon 1s a viOianon or appllcaole Iaw.
Table 2-A PWL Estimation Table for Sample Size n = 5
Q 0.00 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.09
0.0 50.00 50.36 50.71 51.07 51.42 51.78 52.13 52.49 52.85 53.20
0.1 53.56 53.91 54.27 54.62 54.98 55.33 55.69 56.04 56.39 56.75
0.2 57.10 57.46 57.81 58.16 58.52 58.87 59.22 59.57 59.92 60.28
0.3 60.63 60.98 61.33 61.68 62.03 62.38 62.72 63.07 63.42 63.77
0.4 64.12 64.46 64.81 65.15 65.50 65.84 66.19 66.53 66.87 67.22

0.5 67.56 67.90 68.24 68.58 68.92 69.26 69.60 69.94 70.27 70.61
0.6 70.95 71.28 71.61 71.95 72.28 72.61 72.94 73.27 73.60 73.93
0.7 74.26 74.59 74.91 75.24 75.56 75.89 76.21 76.53 76.85 77.17
0.8 77.49 77.81 78.13 78.44 78.76 79.07 79.38 79.69 80.00 80.31
0.9 80.62 80.93 81.23 81.54 81.84 82.14 82.45 82.74 83.04 83.34

1.0 83.64 83.93 84.22 84.52 84.81 85.09 85.38 85.67 85.95 86.24
1.1 86.52 86.80 87.07 87.35 87.63 87.90 88.17 88.44 88.71 88.98
1.2 89.24 89.50 89.77 90.03 90.28 90.54 90.79 91.04 91.29 91.54
1.3 91.79 92.03 92.27 92.51 92.75 92.98 93.21 93.44 93.67 93.90
1.4 94.12 94.34 94.56 94.77 94.98 95.19 95.40 95.61 95.81 96.01

1.5 96.20 96.39 96.58 96.77 96.95 97.13 97.31 97.48 97.65 97.81
1.6 97.97 98.13 98.28 98.43 98.58 98.72 98.85 98.98 99.11 99.23
1.7 99.34 99.45 99.55 99.64 99.73 99.81 99.88 99.94 99.98 100.00

Note: Values in the body ofthe tableare estimates ofPWL corresponding to specific values of QL =(X- LSL) 1 s or Qu = ( USL- X) 1s
For negative Q values, the tab!e values must be subtracted from 1OO.

8.13.3. Calculation and Rounding Procedures-The calculation procedures and rounding rules can
influence the estimated PWL value that is obtained. This can become a point of contention,
particularly ifthe pay determination is based on the estimated PWL value. It is, therefore,
important that the agency specify the exact calculation process, including number of decimal
places to be carried in the calculations, as well as the exact manner in which the PWL table is to
be used.

8.13.4. Conceptually, the Q-value performs identically the same function that the Z-value does for a
normal distribution. Except now the reference points are the individual sample mean ( Jl) and
standard deviation (s) instead ofthe population mean (¡.t) and standard deviation (cr), and the points
of interest with regard to areas under the curve are the specification limits.

Q _ X-LSL (4)
L- S

and

Qu =USL-X (5)
S

where:
QL quality index relative to the lower specification limit,
Qu quality index relative to the upper specification limit,
LSL lower specification limit,
USL = upper specifi.cation limit,

TS-5c R 9-13 AASHTO

© 2018 by the American Association ofState Highway and Transportation Officials.
All rights reserved. Duplicatíon ís a víolatíon of applícable Jaw.
 X sample mean for the lot, and
s sample standard deviation for the lot.

8.13.5. The Q-value, therefore, represents the distance in sample standard deviation units that the sample
mean is offset from the specification Jimit. A positive Q-value represents the number ofsample
standard deviation units that the sample mean falls inside the specification limit. Conversely, a
negative Q-value represents the number ofsample standard deviation units that the sample mean
falls outside the specification limit. These cases are illustrated in Figure 5. QL is used when there
is a one-sided lower specification limit, while Qu is used when there is a one-sided USL. For two-
sided specification limits, the PWL value is estimated as:

PWLt= PWLu+ PWLL-100 (6)

where;
PWLt= percent within the upper and lower specification limits.
PWLu = percent below the upper specification limit (based on Qu).
PWLL = percent above the lower specification limit (based on QL).

(a) Illustration ofPositive Quality Index Values

LSL USL
(b) lllustration of a Negative Quality Index Value

Figure S--Illustration ofthe Quality Index, Q

TS-5c R 9-14 AASHTO

© 201 8 by the American Association of State Highway and Transportation Officials.
All rights reserveu. oupllcanon ts a vtolarton or appltcame taw.
8.13.6. Intuitively, PWL is a good measure of quality because it is reasonable to assume that the more of
the product that is within the specification limits, the better the quality ofthe product should be.
(A detailed discussion and analysis ofthe PWL measure ofquality is presented in Optimal
Proceduresfor Quality Assurance Speci.fications, 2003.)

8.14. If the agency conducts the acceptance tests, sampling and testing must be performed in qualified
laboratories and by qualified sampling and testing personnel. There are no additional requirements
for verification other than the typical lA requirement.

8.15. However, ifthe contractor/third party conducts tests used in the acceptance decision, a validation
procedure is necessary as required in FHW A 23 CFR 63 7 (1995). The requirements are:

8.15.1 . The sampling and testing must be performed in qualified laboratories and by qualified sampling
and testing personnel.

8.15.2. The quality ofthe material must be validated by verification sampling and testing. The verification
sampling and testing is to be performed on samples that are taken independently from those used
in the acceptance decision.

8.16. Statistically sound verification procedures must be developed and used. There are severa! forms of
verification procedures, and sorne forms are more efficient than others. It is in the best interest of
both parties to make the verification process as effective and efficient as practicable.

8.17. FHW A 23 CFR 63 7 states that the verification samples shall be taken independently. This
requirement does not allow the direct comparison of split samples for validation. However, this
does not prohibit the use of split-sample test results. For instance, the contractor may test 100
samples and the agency select the split portion of all 100 samples but test only 20 of the samples.
Then the agency can compare its 20 tests results to the 80 contractor test results that are not paired
with those ofthe agency, using the F-test and independent t-test.

8.17.1. There is a difference in the information provided by split and independent sampling procedures,
and the difference is related to the concept of components ofvariability.

8.17.2. Split Sample-A sample that has been divided into two or more portions representing the same
material. (Split samples are sometimes taken to verify the acceptability ofan operator's test
equipment and procedure. This is possible because the variability calculated from differences in
split test results is comprised solely oftesting variability.) (Transportation Research Circular
Numbel' E-C037, 2002)

8.17.3. Independent Sample-A sample taken without regard to any other sample that may also have been
taken to represent the material in question. [An independent sample is sometimes taken to verify
an acceptance decision. This is possible because the data sets from independent samples, unlik.e
those from split samples, each contain independent information reflecting all sources of
variability, i.e., materials (which often includes both the material and the process variabilities),
sampling, and testing.] (Transporta/ion Research Circular Number E-C037, 2002)
8.17.4. The five specimens shown in Figure 6(a) have been taken independently. The variability
represented by them includes all sources ofvariability, i.e., material, process, sampling, and
testing. Thus, if the total variability is to be evaluated, independent samples must be taken.

8.17.5. In Figure 6(b ), four of the five specimens have been taken independently; however, the two on the
far right are split samples from the same batch, i.e., they are two results that represent the same
material. The variability represented by these two specimens includes only the component of
testing variability. However, either ofthe two specimens on the far right can be combined with the
other three to measure all sources ofvariability, but both cannot be used with the other three in an

TS-5c R 9-15 AASHTO

© 2018 by the American Association ofState Highway and Transportation Officials.
AH rightJ r(;J{;rvc;tl. Dupliwuion is a violarion of applicl!llle ll!W.
 analysis. Specimens tested by different laboratories may be combined, from a statistical
standpoint, if found not to be statistically different. However, a higher variability is likely to result,
and, thus, this is not recommended.

Contractor Agency

2 ---y- 2 2
s~aterial + sprocess + ssampling + stesting

(a) lndependent Samples

Agency

s:aterial

(b) Split Samples

Figure 6-Components ofVariance for: (a) Independent Samples and (b) Split Samples

8.17.6. The analysis ofthe variability for split samples will be a smaller value, cr~ting, than the
variability for independent samples, cr~veraU, because the latter includes all components of
variability. But the decision ofwhether to use split or independent samples should not be made
based on the magnitude ofthe variability but, instead, on the component(s) ofvariability that is
(are) desired to be measured. So, the choice of whether to use split or independent samples is not
an arbitrary decision.

8.18. Definition of Verijication-The process of determining or testing the truth or accuracy of test
results by examining the data and/or providing objective evidence. [V erification sampling and
testing may be part of an lA program (to verify contractor QC testing or agency acceptance) or
part of an acceptance program (to verify contractor testing used in the agency's acceptance
decision)]. (Transportation Research Circular Number E-C037, 2002)

8.19. Verification Procedures-The ability ofthe comparison procedure to identify differences between
two sets oftest results depends on the number oftests from each set that are being compared. The
greater the number oftest results from each set, the greater the ability ofthe procedure to identify
statistically valid differences. A rule ofthumb is a minimum agency rate of 10 percent ofthe
contractor's testing rate. It is preferred to conducta risk analysis to determine whether a higher
rate is warranted. It al so must be decided whether the test method or the process is to be verified.
This relates to the use of split or independent samples.

TS-5c AASHTO
© 2018 by thc American Association of Statc Highway and Transportation Officials.
Al! rights reserved. Duplication is a violation of applicable !aw
8.20. Hypothesis and Leve/ of Signi.ficance-A hypothesis is a statement of an assumption about a set of
data. The null hypothesis, Ho, defines an assumed set of conditions. (The null hypothesis cannot be
proved true. It can be shown, with specified risks, to be untrue ). The altemative hypothesis, Ha, is
the hypothesis that is accepted when the null hypothesis is disproved (i.e., rejected). The level of
significance, u, is the probability of rejecting a null hypothesis when it is, in fact, true
(Transportation Research Circular Number E-C037, 2002). Typicallevels ofsignificance are
0.10, 0.05, and 0.01. If, for example, a= 0.01 is used and the null hypothesis is rejected, then there
is only one chance in 100 that Ha is true and was rejected in error.

8.21. Test Method Veri.fication Procedures-These procedures compare results of split samples. The
two procedures used most often for test method verification are the d2s limits and the paired
t-test.

8.21.1. d2s Limits- This is the simplest procedure that can be used for verification. It is also the least
powerful. Because the procedure uses only two test results, it cannot detect real differences unless
the results are far apart. The value provided by this procedure is contained in many AASHTO test
methods. The d2s limit indicates the maximum acceptable difference between two results obtained
on test portions ofthe same material (and thus, applies only to split samples), and is provided for
single and multilaboratory situations. It represents the difference between two individual test
results that has approximately a 5 percent chance ofbeing exceeded ifthe tests are actually from
the same population.

8.21.2. When the d2s procedure is used for test method verification, a sample is split into two portions
and the contractor tests one split-sample portion while the agency tests the other split-sample
portion. The difference between the contractor and agency test results is then compared to the d2s
limits. Ifthe test difference is less than the d2s limit, the two tests are considered verified, i.e.,
they are assumed to have come from the same population. Ifthe test difference exceeds the d2s
limit, then the contractor's test result is not verified, and the source ofthe difference is
investigated.

8.21.3. Paired t-test- For the case in which it is desirable to compare more than one pair of split-sample
test results, the t-test for paired measurements can be used. This test uses the differences between
pairs oftests and determines whether the average difference is statistically different from O. Thus,
it is the difference within pairs, not between pairs, that is being tested. The t-statistic for the t-test
for paired measurements is:
(7)

where:
Xd average ofthe differences between the split-sample test results,
sd standard deviation ofthe differences between split-sample test results, and
n number of split-sample pairs.

8.21.4. The calculated t-value is then compared to the critica! value, tcrir, obtained from atable of t-values
ata leve! of aJ2 and with n- l degrees of freedom. (Atable of critica! t values can be found in
Materials Control and Acceptance- Quality Assurance, 2000; Optimal Procedures for Quality
Assurance Specifications, 2003; and most basic statistics textbooks. Computer spreadsheet
programs contain statistical test procedures for the paired t-test.)

8.21.5. Because the use of d2s limits is not very powerful, it is recommended that each individual split
sample be compared using the d2s limits, but that the paired t-test also be used on the accumulated
split samples to provide for a comparison with more discerning power.

TS-Sc R 9-17 AASHTO

© 2018 by the American Association of State Highway and Transportation Offícials.
Al1 righ~ rc:¡erved. Duplication is a violation of applicable law.
8.21.6. The above recommendation also applies to the lA component ofthe acceptance plan because it
also uses split samples.

8.22. Process Verification Procedures-Just as there are statistical tests for verification of split-sample
test results, there are also tests for verification of independently obtained test results. The tests
most often used are the F-test and t-test, which are usually used together.

8.22.1. F-test and t-test-This procedure involves two statistical tests, where the null hypothesis for each
test is that the contractor's test results and the agency's test results are from the same population.
Specifically, the null hypotheses are that ( 1) the variability of the two data sets are equal for the
F-test, and (2) the means ofthe two data sets are equal for the t-test. It is important to compare
both the means and the variances when comparing two sets of data. A different test is used for
each comparison. The F-test provides a method for comparing the variances ofthe two sets of
data. Differences in means are assessed by the t-test.

8.22.2. The procedures involved with the F-test and t-test may, at first, seem complicated and involved.
Sorne ofthe complexity ofthe F-test and t-test comparisons can be eliminated by the use of
computer programs. Many spreadsheet programs have the ability to conduct these tests. (An
example ofthe F-test and t-test is shown in Appendix X2.)

8.23. Dispute Resolution-Also called "conflict resolution." For QA programs permitting contractor
acceptance testing, procedures to resolve conflicts resulting from discrepancies between the
agency's and contractor's results of sufficient magnitude to impact pay. (The procedure may, as an
initial step, include the testing ofindependent samples and, as a final step, third-party arbitration.)
(Transportation Research Circular Number E-C037, 2002) Dispute resolution procedures to
identify testing errors are considered an important part of any QA program.

8.24. Dispute Resolution Procedure-It is intended to provide a means to minimize adversaria)

relationships and claims. The use of appropriate procedures should help avoid disputes and claims.
Establish a formal monitoring program that will provide both the contractor and the agency
assurances that all data are reliable, unbiased, and truly indicate quality. There are many aspects of
dispute resolution addressed in detail in AASHTO lmplementation Manual for Quality Assurance
(1995). Two are briefly discussed here.

8.24.1. Jdentijjling Outlying Observations-An outlying observation, or "outlier," is one that appears to
deviate markedly from other sample test values from which it was taken (ASTM E 178). When
considering outliers, two conditions exist:
l. The value may be an extreme value ofthe population or excessive variability ofthY
population. In either case the value should be retained and u sed in further evaluations or
computations.
2. lt may be the result of gross deviation from prescribed sampling and or test procedures, or in
calculating or recording the numerical value, in which case it should be discarded.
Use a procedure to determine which decision to make; i.e., the value is notan outlier and should
be retained or the value is an outlier and should be discarded. (See ASTM E 178 for further
discussion of outliers.) It should be remembered that because ofthe way specification limits are
set, sorne results of AQL material may be outside ofthe specification Iimits. Thus, to arbitrarily
discard values is not a valid procedure.

8.24.2. The Proper Policy for Resampling and Retesting- If resampling or retesting, or both, are allowed,
establish specific procedures stating when and how it should be done. An example ofwhen
retesting may be warranted is the requirement of a "remove and replace" provision when the
quality is estimated to be at or below the RQL. lt may be decided that the quality estimate should
be improved by testing another set ofsamples because the economic consequences of"remove and
replace" are severe. This is an acceptable practice if all the test results-the original and retests-
are used and none ofthe results are discarded.

TS-5c R 9-18 AASHTO

te:> 2018 by the American Assocíatíon ofState Híghway and Transportation Officials.
All righm rc:rorvt:O. DuplicaUon ts a violaUon or appllcame taw.
9. RISKS ANO RISK ANALYSIS
9.1. Establishing the acceptance limits is an important step. Making the limits too restrictive deprives
the contractor of a reasonable opportunity to meet the specification. If the limits are not
sufficiently restrictive, they will be ineffective in controlling quality. Both engineering
requirements and economical process capabilities should be considered when establishing
acceptance limits. Selection of the limits relates to the determination of risks. The concept of risks
for acceptance is similar to that discussed in Section 8 for verification testing to evaluate whether
test results carne from the same population. A well-written QA plan takes these risks into
consideration in a manner that is fair to both the contractor and the agency. Too large a risk for
either party undermines credibility. Thus, the risks should be balanced and should be reasonably
small. For most highway products, ifthis is not possible because ofthe sample size selected, the
risks to the contractor should be less than that to the agency except for potential catastrophic
failures.

9.1.1. Risks-De.finitions and Levels:

9.1.1.1 . Se/ter 's Risk (a)- Also called contractor's risk or risk of a Type I error. The probability that an
acceptance plan will erroneously reject AQL material or construction with respect to a single
acceptance quality characteristic. It is the risk the contractor or producer takes in having AQL
material or construction rejected (Transporta/ion Research Circular Number E-C037, 2002).

9.1.1.2. Buyer 's Risk (13)-Aiso called "agency's risk" or "risk of a Type II error." The probability that an
acceptance plan will erroneously fully accept (lOO percent or greater) RQL material or
construction with respect to a single acceptance quality characteristic. lt is the risk the highway
agency takes in having RQL material or construction fully accepted [The probability of having
RQL material or construction accepted (at any pay) may be considerably greater than the buyer's
risk.] (Transporta/ion Research Circular Number E-C037, 2002).

9.1.1.3. The a and ¡3 risk levels that might be appropriate vary depending on the material or construction
process that is involved. The appropriate risk leve! is a subjective decision that can vary from
agency to agency. However, asan economic decision, typical practice limits risks tono more than
5 percent.

9.1.1.4. The more critica! the application, the lower should be the buyer's risk. But only under rare
circumstances should the buyer's risk be lower than the seller's risk.

9.2. Risks Concepts-As noted in the section on verification testing in Section 8, the concept of a. and
13 risks derives from statistical hypothesis testing in which there is either a right or wrong decision.
As such, when a. and 13 risks are applied to materials or construction, they are only truly
appropriate for the case of a pass/fail or accept/reject decision and, in fact, may lead to
considerable confusion if an attempt is made to apply them to the pay adjustment case. When
materials not only can be accepted or rejected, but can also be accepted atan adjusted pay, then
additional interpretations or clarifications must be applied to the definitions of risks.

9.2.1 . For example, in the definition for buyer's risk above, it states that ¡3 is the probability that RQL
material will be accepted at 100 percent pay or greater. The definition must then go on to point out
that there may be a much greater probability that the RQL material will receive sorne reduced pay.
While it is not stated as directly, the same reasoning is true for the seller's risk. The definition
indicates that a is the probability that AQL material will be rejected. Although not stated in the
definition, it is also true that there may be a much greater probability that the AQL material will be
accepted ata reduced pay.

TS-5c R 9-19 AASHTO

© 2018 by the American Association of State Highway and Transportation Officials.

All rtghts reserved. Duplicarion is a ViolaUon of applicable law.
9.3. Operating Characteristic (OC) Curves--a and 13 are very narrowly defined to occur at only two
specific quality levels. 13 is the probability of accepting, at full pay or more, material that is exactly
at the RQL, while a. is the probability ofrejecting material that is exactly at the AQL. These
definitions do not therefore provide an indication ofthe risks over a wide range ofpossible quality
levels. To evaluate how the acceptance plan will actually perform in practice, it is necessary to
construct an operating characteristic (OC) curve.

9.3.1. OC Curve-A graphic representation of an acceptance plan that shows the relationship between
the actual quality of a lot and either (1) the probability of its acceptance (for accept/reject
acceptance plans) or (2) the probability of its acceptance at various pay levels (for acceptance
plans that include pay adjustmentprovisions) (Transportation Research Circular Number E-C037,
2002).

9.3.2. An example of an OC curve for a pass/fail or accept/reject acceptance plan, case (1) in the above
definition, is shown in Figure 7. Probability of acceptance is shown on the vertical axis for the
range of quality levels indicated on the horizontal axis. An example of an OC curve for an
acceptance plan with pay adjustment provisions, case (2) in the above definition, is shown in
Figure 8. The axes are the same as for Figure 7, but there are multiple curves, one for each of
severa! selected pay levels.

100
Contractor's Risk, a

80
.,¡¿
o
a)
o
e 60
~
Q)
o
o

-
<(
o
.ci
~
40

e_

20

Agency's Risk, P
o
ROL AQL

Quality Level

Figure 7-Typical OC Curve for an Accept/Reject Acceptance Plan

9.3.3. Each curve plotted in Figure 8 represents the probability ofreceiving a pay factor equal to or
greater than the one indicated for the line. For example, for the OC curves in Figure 8, material
that is of exactly AQL quality has approximately a 45 percent chance of receiving a pay factor
of 1.04 (104 percent) or greater. This same AQL material has approximately a 60 percent chance
ofreceiving full pay (lOO percent) or greater, which also means that it has approximately a
40 percent chance ofreceiving less than 100 percent pay. This AQL material has essentially a
100 percent chance ofreceiving a pay factor ofO.SO (80 percent) or greater.

9.3.4. On the other hand, for the OC curves in Figure 8, material that is of exactly RQL quality has
approximately a 50 percent chance of receiving a pay factor of 0.80 (SO percent) or greater and
approximately an 80 percent chance ofreceiving a pay factor of0.70 (70 percent) or greater.
Similar pay probabilities can be determined for any leve! of actual quality, and additional curves
could be developed for any specific pay factor.

TS-Sc R 9-20 MSHTO

© 2018 by the American Association of State Highway and Transportation Officials.
All ri¡;htfi reoorY<ld. Duplioution b n violnlion uf applicablc law.
u..
a.
1\1
Ol 60
r::
·s;
"iii
u
Q)

-e
~
o
.ri
40

a.

o 20 40 RQL 60 80 AQL 100

Population PWL, %

Figure 8-Typical OC Curves for an Acceptance Plan with Pay Adjustments

9.4. Pay Adjustment System Plans-As discussed above and shown in Figure 8, the consideration of
only a and !3 risks is not sufficient when pay adjustments are used. From Figure 8 it can also be
seen that using multiple OC curves is notan easy way to evaluate an acceptance plan.

9.5. Expected Pay (EP) Curves-Thus, another way to present the pay performance for an acceptance
plan is with what is called an "expected pay" (EP) curve.

9.5.1. Definition ofEP Curve-A graphic representation of an acceptance plan that shows the relation
between the actual quality of a lot and its EP, i.e., mathematical pay expectation, or the average
pay the contractor can expect to receive over the long run for submitted lots of a given quality
(Transportation Research Circular Number E-C037, 2002). Both OC and EP curves should be
used to evaluate how well a pay adjustment acceptance plan is theoretically expected to work.

9.5.2. An example of an EP curve is shown in Figure 9. Quality levels are indicated on the horizontal
axis in the usual manner, but instead of probability of acceptance, the vertical axis gives the
expected (long-term average) pay factor as a percent ofthe contract price.

9.5.3. Although the risks have a different interpretation when associated with EP curves than with OC
curves, the same type of information is provided. It is a generally accepted tenet that the average
pay for material that is just fully acceptable should be approximately 100 percent of the contract
price. For the example in Figure 9, AQL work receives an expected pay of 100 percent, as desired,
while truly superior work that is better than the AQL receives an expected pay of 102 percent. At
the other extreme, RQL work corresponds toan expected pay of70 percent. For stilllower levels
of quality, the curve levels off at a mínimum expected pay of 50 percent.

TS-5c R 9-21 AASHTO

© 2018 by the American Association of State Highway and Transportation Officials.
All right~ rt:lit:rvt:tl. Duplic.:aliun is a viulatlun or appllcaOle 1aw.
 110
Maximum Payment Factor"' 102%

100

-¡¡e.
..; 90
e:
Cll
E
>.
ca
a.. 80
"C
~Cll
c. 70
X
w

60

ROL AQL
50

Quality Level

Figure 9--Typical EP Curve

9.6. Eva/uating Risks-The way potential risks are evaluated depends on the type of acceptance plan
that is used. The factors to be considered are: a, 13, n, AQL, and RQL.

9.7. Accept!Reject Plans-The evaluation of risks is straightforward for accept/reject (pass/fail)

acceptance plans. As noted above, a and 13 risks and OC curves were developed specifically for
this type of situation. Therefore, they can be directly used to assess the risks to both parties.

9.7.1. To reiterate, the a risk is the probability that AQL material will be rejected, while the J3 risk is the
probability that RQL material will be accepted. However, contractors will not operate at these two
quality levels only. To illustrate the probability of acceptance for any quality leve! and fully
consider risks, the OC curve must be developed for the acceptance plan under consideration. An
example is used to illustrate how this can be done.

9.7.2. Example ofAccept!Reject Acceptance Plans-OC Curves-Two computer programs are available
to assist in developing OC curves. Program NONCENIT uses an analytical approach based on the
noncentral t distribution and the symmetrical Beta distribution to estímate PWL and can also
determine the a and 13 risks and develop the OC curve for one-sided acceptance plans (Barros,
1982). Program OCPLOT, found and discussed in Weed (1996), uses simulation methods to
determine the a and J3 risks and to plot the OC curve for both one-sided or two-sided acceptance
plans. In this example, the agency has:
• Decided to use asphalt contentas an accept/reject property for an HMA pavement (note,
this is not recommended, but is used here solely for the purpose of illustrating the use of an
OC curve for an accept/reject situation).
• Established a lower specification limit of 5.60 percent and an upper specification limit of
6.40 percent for asphalt content.
• Decided to use the PWL, based on a sample size of four, as the quality measure.
• Selected 90 PWL for the AQL and 50 PWL for the RQL.
• Decided that the lot will be accepted ifthe estimated PWL is greater than or equal to 70, i.e.,
the acceptance limit is 70 PWL.
• Decided to use program OCPLOT to evaluate the risks associated with the acceptance plan.

TS-5c R9-22 AASHTO

© 2018 by the American Association of State Highway and Transportation Officials.

All right~ r~rv~. DupJi¡;ation b a violation of applicable Jaw.
 Table 3 and Figure 10 show the results ofthe OCPLOT analysis ofthis proposed acceptance plan.
For comparison purposes, Table 3 also includes the probability of acceptance values calculated
using program NONCENTT. The OCPLOT values are approximations because ofthe fact that
they were derived by simulation. The NONCENTT values are also approximations because the
procedure was developed for one-sided acceptance limits. Table 3 shows that there is little
difference between the results ofthe two programs. Because OCPLOT is generally more available,
better documented, and easier to use, it is used for al! example problems in this standard practice.

9.7.3. From the OCPLOT values in Table 3, it can be seen that the seller's risk is a= 1.000- 0.905 =
0.095 (or 9.5 percent) and the buyer's risk is f3 = 0.144 (or 14.4 percent). Further, both Table 3 and
Figure 1O show the probability of acceptance over the total range of possible lot quality levels, as
defined by the actual PWL for the Jot. The agency must now decide whether or not it considers
these levels of risk to be appropriate. If the risks are determined to be too high, consider changing
the acceptance limit and/or the sample size.

Table 3-0C Table from OCPLOT for the Example Problem

Probability of Acceptance Probability of Acceptance
Population PWL (OCPLOT) (NONCENTT)
100 1.000 1.000
95 0.976 0.975
90 (AQL) 0.905 (a = 0.095) 0.906
85 0.810 0.810
80 0.6% 0.701
75 0579 0589
70 0.466 0.482
65 0.363 0.382
60 0.288 0.295
55 0.200 0.220
SO(RQL) 0.144 (f3 =0.144) 0.158
45 0.093 0.109
40 0.066 0.071
35 0.038 0.043
30 0.021 0.024
25 0.013 0.012
20 0.000 0.000

TS-Sc R 9-23 AASHTO

© 2018 by the American Association of State Highway and Transportation Officials.
All rights reserve!l. Duplication is a violation of applical>le Jaw.
 100

80
o~
o)
()
e: 60
So.
(1)
()

-
()
<C
40
o
.ci
o....
a..
20

o
o 10 20 30 40 50 60 70 80 90 100

Population PWL, %

Figure 1O-OC Curve for the Accept/Reject Acceptance Plan Example

9.8. Pay Adjustment System Plans-The evaluation of risks beco mes more complicated when the
acceptance plan includes pay adjustment provisions. As discussed above, a and ~ risks, which
were developed from bypothesis testing, where there is a decision to accept or reject the null
hypothesis, are not sufficient when the decision involves not only accept or reject, but also accept
at an adjusted pay leve!.

9.8.1. The Transportation Research Circular Number E-C037 (2002) definitions for seller's and buyer's
risks do not attempt to incorporate the concept of pay adjustments. Whether or not a lot received
105 percent, 100 percent, 75 percent, or 50 percent pay would have no impact with regard to the
seller's risk based on this definition. Obviously, however, these different pay levels would have
quite an impact on how the contractor perceived its risks.

9.8.2. Similarly, the buyer's risk definition disregards the impact ofpartial pays when determining the
buyer's risk. However, when considering its risks, the agency will certainly be interested in the
probability ofaccepting RQL material at reduced pay levels as well as at 100 percent pay or
greater.

9.8.3. Because the use of a and ~ risks to evaluate pay adjustment system plans is not sufficient, an
additional method is necessary to properly evaluate the risks when pay adjustments are added to
the acceptance decision. The EP curve (see Figure 9) is another method for considering the pay
adjustment aspects of the acceptance plan. However, EP alone is also not sufficient to fully
evaluate the risks that are involved. Multiple OC curves for various pay levels (see Figure 8)
should also be developed when evaluating acceptance plans with pay adjustment provisions.
An example will help to illustrate the evaluation of risks for pay adjustment acceptance plans.

9.8.4. Example of Pay Adjustment System Plans- EP Curves-Consider the previous asphalt content
example where the sample size was four, the allowable specification range was 5.60 percent to
6.40 percent, and the AQL and RQL were defined as 90 PWL and 50 PWL, respectively.lnstead
of a simple accept/reject acceptance plan, the agency chooses to use the AASHTO equation
(Equation 8) to establish the pay factor for a lot:
PF = 55+ (0.50 x PWL) (8)
where:

TS-5c R 9-24 MSHTO

© 2018 by the American Association of State Highway and Tr.rnsponation Officials.
AU righw rwlwt~;d. Duplillution ia n -violntion ofnpplit.::nblt: htw.
 PF pay factor for the lot, as a percent of contract price; and
PWL estimated PWL value for the lot.

9.8.4.1. Throughout this standard, the AASHTO equation has been used as an example to develop
concepts that are applicable to all acceptance plans. However, for the reasons stated in
Sections 9.1.1.3, 9.1.1.4, 9.8.8, and 10.1.3, its use is not generally recommended.

9.8.5. From the above equation, it can be seen that the maximum pay factor is 105 percent at 100 PWL,
while the pay factor at the AQL (i.e., 90 PWL) is 100 percent and the pay factor at the RQL (i.e.,
50 PWL) is 80 percent. The intent ofthe acceptance plan is that the average pay for AQL material
be 100 percent, as in this exarnple.

9.8.6. With the above information, the OCPLOT program can be used to develop the EP curve shown in
Figure 11. 1t can be seen in this figure that, as desired, the EP for AQL material is 100 percent.
This means that a contractor that consistently produces AQL material will receive an average pay
factor of 100 percent in the long run. Similarly, the EP for RQL material is 80 percent as desired
from the pay equation.

9.8.7. The EP curve has the advantage ofcombining all ofthe possible pay levels into a single expected,
or long-term average, pay for each given quality level. While it is an improvement over only
considering a and f3 risks, the use ofthe EP alone still has sorne deficiencies. The primary
deficiency is that, while it considers the average long-term pay factor, it fails to consider for a
given quality leve! the variability ofthe individuallot pay factors that comprise this long-term
average. This variability is directly related to the sample size; i.e., the variability about the average
pay factor decreases as the size ofthe individual samples increases. To fully evaluate the risks, it
is necessary to al so consider this variability about the expected pay values.

110

100

<?- 90
~
Q)

[
as
c.. 60
"O
Q)
tí
~
)( 70
w

60

oo+----.----.----.----.---~~--.----.~--.---~~--~
o 10 20 30 40 50 60 70 80 90 100
RQL AQL

Populatíon PWL, %

Figure 11-EP Curve from OCPLOT for the Pay Adjustment System Plan Example

9.8.8. The OCPLOT program output can be used to demonstrate this variability ofthe individuallot pay
factors. Figure 12 shows for an AQL population a histogram that displays the individuallot PWL
estimates along with their corresponding pay values for 1000 simulated lots using a sample size of

TS-5c R 9-25 AASHTO

© 2018 by the American Assocíatíon of State Highway and Transportatíon Officíals.
AJI nshw ~s<:rv<:d. Duplillntion ÍB 11 viollltion ofnpplicll.blc law.