0

Cart
/
$0.00


Acceptance Sampling Update

Leave a Comment / Manufacturing Sampling Plans / By Wayne Taylor

INTRODUCTION
An understanding of statistical principles and how to apply them is critical to ensuring compliance
with FDA requirements, such as those in the current and July working draft of the good manufacturing practices (GMP)
regulation.   Indeed numerous Form 483’s issued by FDA following GMP inspections have cited sampling plans for final,
in-process, and receiving inspection of devices and their components for not being “statistically valid”.  What exactly is
required for a sampling plan to be statistically valid?   After a brief overview of how sampling plans work, this article
will address that question.
Several other sampling-related issues currently facing the medical device industry will also
be discussed.   In February of this year, the U.S. Department of Defense canceled Mil-Std-105E, which contained a
widely used table of sampling plans.   What are the alternatives and how should they be used?   As the industry
focuses increasingly on the prevention of defects and statistical process control (SPC), will the need for acceptance
sampling disappear?   And finally, how can the cost of acceptance sampling be reduced to help manufacturers remain
competitive in today’s marketplace?
HOW SAMPLING PLANS WORK
Stated simply, sampling plans are used to make
product disposition decisions for each lot of product.   With attribute sampling plans, these accept/reject decisions are
based on a count of the number of defects and defectives, while variables sampling plans require measurements and
calculations, with decisions based on the sample average and standard deviation.  Plans requiring only a single sample
set are known as single sampling plans; double and multiple sampling plans may require additional samples sets.  For
example, an attribute single sampling plan with a sample size n=50 and an accept number a=1 requires that a sample
of  50 units be inspected.  If the number of defectives in that sample is one or zero, the lot is accepted.  Otherwise, it is
rejected.
Ideally, when a sampling plan is used, all bad lots will be rejected and all good lots accepted.  However,
because accept/reject decisions are based on a sample of the lot, there is always a chance of making an incorrect
decision.  So what protection does a sampling plan offer?  The behavior of a sampling plan can be described by its
operating characteristic (OC) curve, which plots percent defectives versus the corresponding probabilities of
acceptance.  Figure 1 shows the OC curve of the attribute single sampling plan described above.  With that plan, if a lot
is 3% defective the corresponding probability of acceptance is 0.56.  Similarly, the probability of accepting lots that are
1% defective is 0.91 and the probability of accepting lots that are 7% defective is 0.13.

Privacy - Terms
 Figure 1: OC Curve of Single Sampling Plan n=50 and a=1

An OC curve is generally summarized by two points on the curve: the acceptable quality level (AQL) and the lot
tolerance percent defective (LTPD).  The AQL describes what the sampling plan generally accepts; formally, it is that
percent defective with a 95% percent chance of acceptance.  The LTPD, which describes what the sampling plan
generally rejects, is that percent defective with a 10% chance of acceptance.  As shown in Figure 2, the single sampling
plan n=50 and a=1 has an AQL of 0.72% defective and an LTPD of 7.6%.  The sampling plan routinely accepts lots that
are 0.72% or better and rejects lots that are 7.6% defective or worse.   Lots that are between 0.72% and 7.6%
defective are sometimes accepted and sometimes rejected.

Figure 2: AQL and LTPD of Single Sampling Plan n=50 and a=1

Manufacturers must know and document the AQLs and LTPDs of the sampling plans used for their products.  The AQLs
and LTPDs of individual sampling plans can be found in Table X of MIL-STD-105E, and Chart XV of ANSI Z1.4 gives the
AQLs and LTPDs of entire switching systems (described below).1,2  Software also can be used to obtain the AQLs and


LTPDs of a variety of sampling plans.6

SELECTING STATISTICALLY VALID SAMPLING PLANS
Documenting the
Privacy - Terms
protection provided by your company’s sampling plans is only half the job.  You must also provide justification for the
AQLs and LTPDs used.   This requires that the purpose of each inspection be clearly defined.   Depending on past
history and other circumstances, sampling plans can be used for a variety of purposes.  In the examples described
below, an AQL of 1.0% is specified for inspections for major defects.  The AQL given in this specification is not
necessarily equal to the sampling plan AQL, and so it will be referred to as Spec-AQL to make this distinction clear.
Spec-AQLs are commonly interpreted as the maximum percent defective for which acceptance is desired.  Lots below
the Spec-AQL are best accepted; Lots above the Spec-AQL are best rejected.  The Spec-AQL, therefore, represents the
break-even quality between acceptance and rejection.  For lots with percent defectives below the Spec-AQL, the cost of
performing a 100% inspection will exceed the benefits of doing so in terms of fewer defects released.  Since this cost is
ultimately passed on to the customer, it is not in the customer’s best interest for the manufacturer to spend $1000 to
100% inspect a lot if only one defect is found that otherwise would have cost the customer $100.  Spec-AQLs should
not be interpreted as permission to produce defects; however, once lots have been produced, the Spec-AQLs provide
guidance on making product disposition decisions.
Example 1:  If a process is known to consistently produce lots with
percent defectives above the Spec-AQL, all lots should be 100% inspected, but if some lots are below the Spec-AQL,
the company could use a sampling plan to screen out lots not requiring 100% inspection.  To ensure that lots worse
than the Spec-AQL are rejected, a sampling plan with an LTPD equal to the Spec-AQL can be used, but at the risk of
rejecting some acceptable lots.  For a Spec-AQL of 1.0%, the single sampling plan n=230 and a=0, which has an LTPD
of 1.0%, would be appropriate.  There is a simple formula for determining the sample size for such studies.  Assuming
an accept number of zero and the desired confidence level of 90%, the required sample size is:
n = 230/Spec-AQL

For 95% confidence, the formula is

n = 300/Spec-AQL

Example 2:  The same sampling plan might also be used to validate a process for which there is no prior history. 
Before reduced levels of inspection are implemented, it should be demonstrated that the process regularly produces
lots below the Spec-AQL.  If the first three lots pass inspections using a sampling plan with an LTPD equal to the Spec-
AQL of 1.0%, the manufacturer can state with 90% confidence that each of these lots is <1% defective.
However, other
sampling plans might be better choices.  Suppose the process is expected to yield lots that are around 0.2% defective. 
The sampling plan n=230 and a=0 has an AQL of 0.022% and therefore runs a sizeable risk of failing the validation
procedure.  A sampling plan with an AQL of 0.2% and an LTPD of 1% would be a better choice. Using the software
cited earlier, the resulting plan is n=667 and a=3.3
Example 3:  Once it has been established that the process
consistently produces lots with percent defectives below the Spec-AQL, the objective of future inspections might be to
ensure that lots with >=4% defective are not released.  This requires a sampling plan with an LTPD of 4%.  Because the
sampling plan should also ensure that lots below the Spec-AQL are released, the plan’s AQL should be equal to the
Spec-AQL.  According to Table I from the book Guide to Acceptance Sampling3, which gives a variety of sampling plans
indexed by their AQLs and LTPDs, the single sampling plan n=200 and a=4 is the closest match.  It has an LTPD of
3.96% and an AQL of 0.990%, and thus is statistically valid for this purpose.
Example 4:  Now suppose that the
process has run for 6 months with an average yield of 0.1% defectives and no major problems.  Although the process
has a good history, there is still some concern that something could go wrong; as a result, the manufacturer should
continue to inspect a small number of samples from each lot.  For example, a sampling plan might be selected that
ensures that a major process failure resulting in ≥20% defective will be detected on the first lot.   The sampling data
can then be trended to detect smaller changes over extended periods of time.
When selecting a sampling plan to

detect a major process failure, the nature of the potential failure modes should be considered.  If the primary failure
Privacy - Terms
mode of concern is a clogged filter and past failures have resulted in ≥20% defectives, the single sampling plan n=13
and a=0, which has an LTPD of 16.2% and an AQL of 0.4% in Table I3, is statistically valid.  If the potential failure mode
of concern is a failure to add detergent to the wash cycle, with a resulting failure rate of 100%, the single sampling plan
n=1 and a=0 is valid.
Table I: Single Sampling Plans Indexed by AQL and LTPD3

AQL

45

AQL Approximate Ratio of LTPD/AQL

10%
45 11 6.5 5 4 3.2

6.5%
10% –

4.0%
6.5% –

2.5%
4.0% –

1.5%
2.5%

Example 5:  Finally, one might have a proven process for which procedures are in place that minimize the likelihood of a
process failure going undetected.  At that point, acceptance sampling might be limited to the routine collection of data
sufficient to plot process-average tends.  There is nothing wrong with simply stating in the written procedures that
acceptance sampling is not needed and that the inspections being performed should be considered as process audits.
In
summary, selecting a statistically valid sampling plan is a two-part process.  First, the purpose of the inspection should
be clearly stated and the appropriate AQL and LTPD selected; then, a sampling plan should be selected based on the
chosen AQL and LTPD.  Because different sampling plans may be statistically valid at different times, all plans should
be periodically reviewed.  If a medical device manufacturer doesn’t know the protection provided by its sampling plan or
is unclear as to the purposes of its, it is at risk.
MIL-STD-105E AND ANSI Z1.4
The February 1995 cancellation of MIL-
STD-105E by the Department of Defense, while directly affecting military purchasing contracts, will also have an
impact on the medical device and diagnostic industries.   The cancellation notice indicates that future acquisitions
should refer to an acceptable nongovernment standard such as ANSI Z1.4.  To many, this news came as a shock. 
However, the change is not nearly as drastic as it first seems.  The differences between ANSI Z1.4 and MIL-STD-105E
are minor and the elimination of the later was basically a cost-savings measure to eliminate the duplication of effort
associated with maintaining two nearly equivalent standards.
Nevertheless, because manufacturers may need to
update many specifications as a result of this change,  now is an especially appropriate time to reexamine MIL-STD-
105E and Z1.4 and how to use them to select valid sampling plans.   Although the term Z1.4 will be used in the
following discussion, all of the ensuing comments apply equally to MIL-STD-105E.
Lets us start with what Z1.4 is not. 
Used by many industries, Z1.4 is not a table of statistically valid sampling plans.  Instead, it contains a broad array of 

sampling plans that might be of interest to anyone.  For example, one plan requires two samples and has an acceptPrivacy - Terms
number of 30.  Such a plan would never be appropriate for a medical device, but is applicable in other industries.
Furthermore,  Z1.4 is, in fact, a sampling system.  A user references the sampling plans in Z1.4 by specifying an AQL
and a level of inspection, and then following a set of procedures for determining what sampling plan to use based both
on lot size and the quality of past lots.  The Z1.4 system includes tightened, normal, and  reduced sampling plans and a
set of rules for switching between them.   Although these switching rules are frequently ignored, they are an integral
part of the standard.  As Z1.4 states:
This standard is intended to be used as a system employing tightened, normal, and reduced inspection on a
continuing series of lots …. Occasionally specific individual plans are selected from the standard and used
without the switching rules. This is not the intended application of the ANSI Z1.4 system and its use in this
way should not be referred to as inspection under ANSI Z1.4.2

Several companies have received Form 483s from FDA for not using the switching rules, a problem that could have
been avoided by having written procedures specifying that the switching rules are not used.  When is the use of
switching rules appropriate and when should individual sampling plans be selected instead?  Z1.4 was developed
specifically to induce suppliers “to maintain a process average at least as good as the specified AQL while at the same
time providing an upper limit on the consideration of the consumer’s risk of accepting occasional poor lots.”2   Thus, the
Z1.4 switching system should not be used to inspect isolated lots, nor should they be used to specify the level of
protection for individual lots.  In those cases individual plans should be selected instead.
One situation warrants special
mention.  Acceptance sampling is frequently used for processes that generally produce good product but might on
occasion break down and produce high levels of defects.  If protection against isolated bad lots or the first bad lot
following a series of good lots is the key concern, the Z1.4 switching rules should not be used or, if they are, the
reduced inspection should be omitted.   Because the Z1.4 switching rules are designed to react to gradual shifts in the
process average, they frequently fail to detect isolated bad lots and do not react quickly to sudden shifts in the lot
quality.   Even when appropriate, the Z1.4 switching rules are complicated to apply.  However, quick switching systems
have been developed that are both simpler to use and provide better protection during periods of changing quality.3
Finally, there are two common misconceptions about Z1.4.  Many people believe that the required sample sizes increase
for larger lots because more samples are required from such lots to maintain the desired level of protection.  The truth
is that the standard specifies larger sample sizes to increase the protection provided for larger lots.  The reason for this
increase is based on economics: It is more expensive to make errors classifying large lots; as a result, Z1.4 requires more
samples from larger lots to reduce the risk of such errors.  To maintain the same level of protection, one can simply
select a sampling plan based on its OC curve and then use this plan for all lots regardless of size.  The single sampling
plan n=13 and a=0 provides the same protection for a lot of 200 units as for a lot of 200,000 units.3,4
The second
misconception is that use of Z1.4 ensures that lots worse than the AQL are rejected.  According to this misconception, if
the AQL is 1%, lots with >1% defectives are routinely rejected.  The truth is that there is a sizable risk of releasing such
lots–one sampling plan with an AQL of 1% accepts lots that are <=16% defective.  The protection provided by the
sampling plan is determined by its LTPD, not AQL, which reveals nothing about what a sampling plan will reject.  As a
result of this misconception, many manufacturers believe that their sampling plans provide greater protection than they
do.  This illusion can lead to the use of inappropriate sampling plans and can provide a false sense of security. 
Repeating the advice given earlier, manufacturers should determine and document the actual AQLs and LTPDs of all
their sampling plans.
While Z1.4 and equivalent standards are widely used by the device industry, rarely are they used
in the manner intended.  Most commonly, individuals sampling plans are selected from them.  Other tables are better
suited for this purpose, and companies should not be afraid to switch to using those tables.  In addition, using Z1.4 does
not ensure valid sampling plans and, in fact, can complicate the selection process.
SPC VERSUS ACCEPTANCE

SAMPLING?
Much has been written about the greater benefits to be achieved by using SPC as opposed to acceptance
Privacy - Terms
sampling.  But although preventing defects is certainly more desirable than detecting them through inspection, SPC
does not eliminate the need for acceptance sampling.  As indicated in Table II, there are fundamental differences
between the two techniques.  In SPC, control charts are used to make process control and process improvement
decisions, and actions are taken on the process to ensure that future products are good.  In contrast, sampling plans are
used to make product disposition decisions, and actions are taken on previously produced lots to ensure the quality of
released product.   Ideally, with SPC in place no defectives will ever be made and acceptance sampling will become
unnecessary: in practice, however, all processes have some risk of failure, and thus some procedure for accepting and
rejecting product is generally required.
Table II: Differences Between Control Charts and Sampling Plans

Control Chart Sampling Plan

Decision Adjust or Leave Alone Accept or Reject

Act On Process Product

Focus Future Product Past Product

Much of the reaction against acceptance sampling is attributed to quality guru W. Edwards Deming, who many believe
advocated its elimination.  However, what Deming really called for was ceasing reliance on acceptance sampling.  If
more time and resources are spent on acceptance sampling than on process improvement and control, or if a company
believes that, no matter what else happens, its sampling plan ensures shipment of only good product, then that
company is overly reliant on acceptance sampling.  Instead, its focus should be on defect prevention and continuous
process improvement.
The real issue is not SPC versus acceptance sampling; it is how to combine the two.  Both
techniques require routine product inspections; the trick is to use the same data for both purposes.  When variables
sampling is used–that is, when the data consist of actual measurements such as seal strength, fill volume, or flow rate–
data can be combined on a single acceptance control chart.  Figure 3 provides an example of such a chart containing
fill-volume data.  The inside pair of limits, UCL and LCL, are the control limits.  A point falling outside these limits
signals that the process is off target and that corrective action is required.  The outside pair of limits, UAL and LAL, are
the acceptance limits.  A lot whose sample falls outside these limits is rejected.  In the figure, lot 13 is outside the
control limits but inside the acceptance limits, which indicates that the process has shifted.  Corrective action on the
process is required to maximize the chance that future products will be good; however, no action is required on the
product lot.  Rejecting whenever a point exceeds the control limits can result in the rejection of perfectly good lots. 
Similarly, it is wasteful to wait until the acceptance limits are exceeded before taking corrective action on the process.
Therefore, separate limits for process and product actions are required.  Such limits are also frequently called action
limits, warning limits, and alert limits.  No matter what the name, however, if the result of exceeding a limit is to act on
the process, the limit is serving the purpose of a control limit; if action is instead taken on the product, the limit is
serving as an acceptance limit.

Privacy - Terms
 Figure 3: Acceptance Control Chart

If attributes sampling is performed, the data must be handled much differently, and care must be taken in implementing
SPC so that the resulting change is not illusionary.  Consider, for example, a packing operation that inspects for missing
parts using the single sampling plan n=13 and a=0.  Whenever a lot is rejected, an attempt is made to fix the process. 
Historically, the process has averaged around 0.2% defective.   When management decides to implement SPC, a p-
chart of the inspection data is constructed as shown in Figure 4.  The upper control limit is 3.92%, and samples with
one or more defectives exceed this control limit, triggering attempts to fix the process and rejection of recent product. 
The company can now state truthfully that SPC is used, but in reality nothing has changed–the same data are collected
and the same actions taken.  A better approach is to continue acceptance sampling as before and, because this does not
protect against a gradual increase in the process average, to analyze the resulting data for trends.  Figure 5 shows a p-
chart of the same data, but with the data from each day combined.  This chart indicates that a change occurred between
days 5 and 6; this change is not so apparent in Figure 4.
Neither SPC or acceptance testing can detect a problem before
defectives are produced.  However, by accumulating data over time, attribute control charts can indicate small changes
in the process average that acceptance sampling will not reveal.  Used in combination, sampling plans provide
immediately protection against major failures while control charts protect against minor sustained problems.

Privacy - Terms
 Figure 4: p-Chart of Inspection Results

Figure 5: Daily p-Chart

REDUCING INSPECTION COSTS

Two sampling plans can have the same AQL and LTPD and nearly equivalent OC
curves.  When they are followed, the same percentage of good lots will be accepted and the same percentage of bad
lots rejected.  The quality of the products released to customers will be the same, as will the reject and scrap rates. 
From a regulatory point of view, the two sampling plans are substantially equivalent.  However, one of these plans may
be less costly to use.
Consider an example.  The ANSI Guideline for Gamma Sterilization5 provides procedures for
establishing and monitoring radiation dosages.  One procedure is a quarterly audit of the dosage that requires the
sterilization of test units at a lower dosage than is actually used for the product.  The test dose is selected to give an
expected positive rate of 1%. (A positive is a unit that tests nonsterile.)  For each audit, an initial sample of 100 units is
tested.  If two or fewer positives are found, the process has passed the audit; in the event of three or four positives, one
retest can be performed.  This quarterly audit procedure has an AQL of 1.50% and an LTPD of 5.55%.  An alternative to
this procedure is to test 50 samples, passing on zero positives and failing on four or more positives.  In the event of 1 to
3 positives, a second sample of 100 units is tested.  The audit is considered passed only if the cumulative number of
positives in the 150 units is four or less. This double sampling plan has an AQL of 1.36% and LTPD of 5.73%.

Privacy - Terms
 Figure 6: OC Curves of the ANSI Quarterly Audit Sampling Plan and an Alternative Plan for Monitoring
Sterilization Dosage

The OC curves of both procedures are nearly identical, as shown in Figure 6.  Indeed, these two sampling plans are
substantially equivalent procedures, except for the number of units tested.  Figure 7 shows average sample number
(ASN) curves for the two plans. If the positive rate is 0.5%, the alternative procedure requires an average of 70 units
compared to an average of 102 for the ANSI quarterly audit procedure.  If the alternative plan is used to destructively
test expensive medical devices, this difference can mean a sizable savings.

Figure 7: ASN Curves of Audit Sampling Plan and Alternative



Privacy - Terms
CONCLUSION
Acceptance sampling is one of the oldest techniques used for quality control, yet it remains poorly
understood and misconceptions regarding its procedures and terminology are widely held.  Acceptance sampling does
not have to be complicated.   Your company can optimize its procedures by remembering this list of principles:
The protection level provided by a sampling plan is described by what it accepts — its AQL–and what it rejects–
its LTPD.
Selecting a statistically valid sampling plan requires stating the objective of the inspection, selecting the
appropriate AQL and LTPD, and then choosing a sampling plan that provides the desired protection.
Companies must know the AQLs and LTPDs of all their sampling plans.  It doesn’t matter whether a sampling
plan comes from MIL-STD-105E or some other source, and the protection provided by a plan does not depend on
the lot size; it’s the AQL and LTPD that reveal what protection the sampling plan provides.
SPC cannot serve as a replacement for acceptance sampling.  Instead, these two techniques should be combined
by using the same data to control the process and to make product disposition decisions.
Sampling plans with the same AQL and LTPD are substantially equivalent procedures, so costs can sometimes
be reduced by using equivalent double, multiple, or variables sampling plans as alternatives to single sampling
plans.

REFERENCES
1. Sampling Procedures and Tables for Inspection by Attributes, MIL-STD-105E, Washington D.C. ,  U.S.
Government Printing Office, 1989 .
2. Sampling Procedures and Tables for Inspection by Attributes, ANSI/ASQC Z1.4, Milwaukee, WI,  American
Society for Quality Control, 1981.
3. Taylor, W A, Guide to Acceptance Sampling, Libertyville, IL, Taylor Enterprises, 1992. (Software is supplied with
this book.)
4. Schilling, E G, Acceptance Sampling in Quality Control, New York City, Marcel Dekker, 1982.
5. Guideline for Gamma Radiation Sterilization, ANSI/AAMI ST32-1991, Arlington, VA, Association for the
Advancement of Medical Instrumentation, 1992.
6. Sampling Plan Analyzer, Taylor Enterprises, Inc., Variation.com/spa.

Appeared in MDDI (Medical Device & Diagnostic Industry), Oct. 1995,   p. 92-108, Canon Communications

(Used by FDA in new inspector training)

Copyright © 1995 Taylor Enterprises, Inc.

Further information can be found in:

Book Guide to Acceptance Sampling

STAT-09, Manufacturing Acceptance Sampling Plans and Inspections, of the book Statistical Procedures for the
Medical Device Industry
STAT-10, Statistical Techniques for Trending Data, of the book Statistical Procedures for the Medical Device
Industry
Software Sampling Plan Analyzer
Acceptance Sampling Standards page


Privacy - Terms
 ← Previous Post Next Post →

Leave a Comment
Your email address will not be published. Required fields are marked *

Type here..

Name*

Email*

Website

Post Comment »

Search blog...
Go

Recent Posts

Acceptance Sampling Standards

STAT-18: Statistical Techniques for Normality Testing and Transformations

STAT-16: Statistical Techniques for Equivalence Testing

STAT-12: Verification/Validation Sampling Plans for Proportion Nonconforming

STAT-04: Statistical Techniques for Design Verification

Categories

Audits and Effectiveness Checks
Privacy - Terms
Design of Experiments

Design Validation (Clinical and User Testing)

Design Verification (Product Testing)

Manufacturing Sampling Plans

Normality Testing and Transformations

Process Validation (Process and Supplier Testing)

Spec Setting, Tolerance Analysis and Robust Design

Statistical Policy and General

Test Method Validation

Trending of Quality Data

Validation Sampling Plans

Archives

September 2018

August 2018

October 2017

September 2017

December 2004

August 2002

April 2000

May 1998

December 1997

October 1997

September 1997

May 1997

October 1996

May 1996

October 1995

February 1995

November 1994

September 1994

March 1994

November 1993

Subscribe to our Email List



Privacy - Terms

Email
 Send

Privacy Policy

Site Map

Copyright © 2019 Taylor Enterprises | All rights reserved.

Privacy - Terms