See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/346749265

Solving Statistical Mysteries -What Does the FDA Want?

Conference Paper · December 2020

CITATIONS READS

0 1,230

2 authors, including:

Ron Snee
Snee Associates, LLC
447 PUBLICATIONS 9,542 CITATIONS

SEE PROFILE

All content following this page was uploaded by Ron Snee on 09 December 2020.

The user has requested enhancement of the downloaded file.

 3 7

25 Temperature(degC) 40
3.4 6.6

2.6 3.6 75
Humidity %
1.6 5.3 50
No Base Present Yes

Solving Statistical Mysteries – What Does the FDA Want?

Ronald D. Snee, PhD
Snee Associates, LLC

2018 Pharma Analytics Summit

Philadelphia, PA
March 26-27, 2018
1

Ron Snee, PhD is Founder of Snee Associates,
LLC, a firm dedicated to the successful
implementation of process and organizational
improvement initiatives. He provides guidance to
pharmaceutical and biotech senior executives in
their pursuit of improved business performance
using Quality by Design, Lean Six Sigma and
other improvement approaches that produce
bottom line results.
He has authored several articles on how to
successfully implement QbD, coauthored three
books on QbD tools and speaks regularly at
pharmaceutical and biotech conferences.
2
About the Speaker …..
He is also an Adjunct Professor in the
pharmaceutical programs at Temple and Rutgers
Universities. He worked at DuPont for 24 years prior
to starting his consulting career.
Ron received his BA from Washington and
Jefferson College and MS and PhD degrees from
Rutgers University. He is an academician in the
International Academy for Quality and Fellow of the
American Society of Quality, American Statistical
Association, and American Association for the
Advancement of Science. He is an Honorary
Member of ASQ and has been awarded ASQ’s
Shewhart, Grant and Distinguished Service Medals,
and ASA’s Deming Lecture, Dixon Consulting and
Hahn Achievement Awards. He is a frequent
speaker and has published 6 books and more than
300 papers in the fields of quality, performance
improvement, management, and statistics. He
recently received the Institute of Validation
Technology’s Speaker of the Year Award.
Abstract

Use of statistics has been part of the FDA’s guidances

and regulations for many years. Use of statistics is an
integral part of the Jan 2011 FDA Process Validation
Guidance. It can be argued that the FDA is looking for the
pharma and biotech industries to make even greater use
of statistical thinking and methods. Even with this long
history, the use of statistics seems to be a source of
uncertainty and anxiety for many. This presentation
provides an overview of what it appears the FDA is
looking for in the use statistics including examples and
recommended approaches.

3
 Agenda
 Today’s Reality: FDA Guidance Regarding:
 Quality by Design
 Continued Process Verification
 Test Method Development
 FDA Guidance’s – Examples of Recommended use of
Statistical Thinking and Methods
 Tools, Methods and Personnel
 What is the Appropriate Documentation?
 What’s the FDA Looking for?
 Right Process Measurement, Data, Tools and Methods and
Interpretation
 Useful Standards
 Understanding of tool usage and interpretation of results
 Tips, Traps and Recommendations

4
 “Full implementation of QbD
and modernization of
manufacturing by the
pharmaceutical industry in
development through
manufacturing is expected to
provide lasting benefits to
industry, regulators and
patients.”
Janet Woodcock, Director
FDA Center for
Drug Evaluation and Research
Before the
U.S. House of Representatives
Subcommittee on Energy Policy,
Health Care and Entitlements

5
 Quality by Design

Quality by Design – An Effective Approach

Systematic approach to product and process development

• Begins with predefined objectives
• Emphasizes product and process understanding and
process control
• Based on sound science and quality risk management
From ICH Q8(R1) Step 2

Quality by Design
• About Building Quality into Products and Processes
• Based on Effective Collection and Analysis of Data

6
 FDA Process Validation Guidance
Importance of Variation
“A successful validation program depends upon information and
knowledge from product and process development. This knowledge and
understanding is the basis for establishing an approach to control of the
manufacturing process that results in products with the desired quality
attributes. Manufacturers should:
• Understand the sources of variation
• Detect the presence and degree of variation
• Understand the impact of variation on the process and
ultimately on product attributes
• Control the variation in a manner commensurate with the risk it
represents to the process and product
Each manufacturer should judge whether it has gained sufficient
understanding to provide a high degree of assurance in its manufacturing
process to justify commercial distribution of the product.”

7
 It’s About Variation

“All processes--human and non human--exhibit variability. This

variability is measurable.” Joseph M. Juran

“If I had to reduce my message for management to just a few

words, I’d say it all had to do with reducing variation.”

“…State of statistical control is not a natural state for a

manufacturing process. It is instead an achievement, arrived at by
elimination, one by one, by determined effort, of special causes of
excess variation.”
W. Edwards Deming

So Why Statistics?
Statistical Thinking and Methods Enable Us to
Deal Effectively with Process and Product Variation

8
 Process Validation Guidance
Stage 3 – Continued Process Verification
“An ongoing program to collect and analyze product and
process data that relate to product quality must be established
(§211.180(e)). The data collected should include relevant process
trends and quality of incoming materials or components, in-
process material, and finished products. The data should be
statistically trended and reviewed by trained personnel. The
information collected should verify that the quality attributes are
being appropriately controlled throughout the process.
We recommend that a statistician or person with adequate
training in statistical process control techniques develop the
data collection plan and statistical methods and procedures
used in measuring and evaluating process stability and process
capability.”

9
 FDA Wants Statistics to be Used

“We recommend an integrated team approach to process

validation that includes expertise from a variety of
disciplines (e.g., process engineering, industrial
pharmacy, analytical chemistry, microbiology, statistics,
manufacturing, and quality assurance). Project plans,
along with the full support of senior management, are
essential elements for success.”
………….

FDA Process Validation Guidance, Jan 2011

10
 FDA Guidance for Industry
Analytical Procedures and Methods Validation for
Drugs and Biologics
VII. STATISTICAL ANALYSIS AND MODELS
A. Statistics
“Statistical analysis of validation data can be used to evaluate validation
characteristics against predetermined acceptance criteria. All statistical
procedures and parameters used in the analysis of the data should be
based on sound principles and appropriate for the intended evaluation.
Several statistical methods are useful for assessing validation
characteristics, for example, an analysis of variance (ANOVA) to assess
regression analysis R (correlation coefficient) and R squared (coefficient
of determination) or linear regression to measure linearity. Many
statistical methods ……”
References
ASTM E1488 – Statistical Procedures to use in Developing and Applying Test Methods
ASTM E2782 – Standard Guide for Measurement Systems Analysis (MSA),

11
 Code of Federal Regulation – Title 21
Part 820 Quality System Regulation
Subpart J – Corrective and Preventive Action
Sec. 820.100 Corrective and Preventive Action.

(a) Each manufacturer shall establish and maintain procedures

for implementing corrective and preventive action. The
procedures shall include requirements for:
(1) Analyzing processes, work operations, concessions, quality
audit reports, quality records, service records, complaints,
returned product, and other sources of quality data to identify
existing and potential causes of nonconforming product, or other
quality problems. Appropriate statistical methodology shall be
employed where necessary to detect recurring quality problems;

12
 Code of Federal Regulations – Title 21
Part 820 - Quality System Regulation
Subpart O--Statistical Techniques

Sec. 820.250 Statistical techniques.

(a) Where appropriate, each manufacturer shall establish and

maintain procedures for identifying valid statistical
techniques required for establishing, controlling, and
verifying the acceptability of process capability and product
characteristics.
(b) Sampling plans, when used, shall be written and based on
a valid statistical rationale. Each manufacturer shall establish
and maintain procedures to ensure that sampling methods
are adequate for their intended use and to ensure that when
changes occur the sampling plans are reviewed. These
activities shall be documented.

13
 Guidance for Industry and FDA Staff
30-Day Notices, 135-Day Premarket Approval (PMA) Supplements and 75-
Day Humanitarian Device Exemption (HDE) Supplements for
Manufacturing Method or Process Changes (April 2011)
“Summary of the data or information supporting the change, should include:
 a summary of the procedures established for the identification, documentation, validation,
review, and approval of the manufacturing changes submitted in the 30-day notice;
 the statistical rationale for the sampling method, if you plan to
verify the changed processes by routine sampling and independent
measurement;
 a description of how you will monitor and control any manufacturing process you intend to
change;
 a summary of the completed validation study that demonstrates that the manufacturing
change can be made without significantly changing the operation of the final device. This
summary should be include: a description of the acceptance criteria;
 information on how, using valid statistical methods, you analyzed the
test data;
 information that describes the statistical rationale for sample sizes;
 a list of any deviations that occurred; and
 a determination of the impact of the deviation on the results;
 an explanation of how change control procedures were implemented, ………..”
14
 Today’s Realities
 FDA insists on the use of data and statistics
 FDA Process Validation Guidance, January 2011
 FDA’s Rick Friedman and Karthik Iyer explain why some
manufacturers need to get a better grip on GMP-related
statistics. …… PharmaManufacturing, June 5, 2012
 Data abound ….. Data, Data, Data everywhere
 But little information to be found
 Big Data – Some believe Big Data will solve all problems?
 Data are part of the scientific method
 Scientists, engineers and other professionals use data

15
 Why Do we Need Statistics?
 Because of variation
 Its all around us, present in everything we do
 Variation affects the performance of our products and
processes
 Quality decreases
 Productivity decreases
 Cost go up ….. Inventory Increases
 Risk goes up
 Products are not delivered on time
 Patients take the wrong drugs

Statistical Thinking and Methods Provide the

Concepts and Tools for Dealing with Variation
16
 Help is on the Way …. Actually help is here

 User friendly statistical software is available

 Never forgets the statistical formulas
 Always gets the calculations correct
 User is responsible for
 Collecting the right data
 Getting the data into the software correctly
 Selecting and executing the correct statistical analysis
 Interpreting the output of the software.
 Much more time and energy are now available to do good
problem solving
 Software does the heavy lifting

Hint – Don’t Let the Software Do Your Thinking for You!

17
 More Help Is Here 

 Use of statistics in the solution of all types has grown

steadily since computers became readily available in
the 1960s
 Much about the use of statistics has been learned
from this experience
 Considerable research has been done on the effective
use of statistical thinking and methods
 These learnings have been codified through:
 Best practice approaches and principles
 Construction of data analysis roadmaps
 Development of lists of Tips and Traps

18
Help is Here (cont’d)
 These learnings internalized helps professionals
 Develop trust in the value of statistical thinking and
methods in the solution of problems
 Builds confidence in one’s ability to be successful
 Automobile Analogy
 You can drive a car without knowing the mechanical
engineering, combustion engineering, information
technology and other technologies used to operate
the car
 Similarly our goal is to enable users to drive the
“statistics machine” without
 Knowing all the theory and mechanics of statistics
 Being burdened with extensive computations

19
 Agenda
 Today’s Reality: FDA Guidance Regarding:
 Quality by Design
 Continued Process Verification
 Test Method Development
 FDA Guidance’s – Examples of Recommended use of
Statistical Thinking and Methods
 Tools, Methods and Personnel
 What is the Appropriate Documentation?
 What’s the FDA Looking for?
 Right Process Measurement, Data, Tools and Methods and
Interpretation
 Useful Standards
 Understanding of tool usage and interpretation of results
 Tips, Traps and Recommendations

20
 Current Situation
 FDA guidances are not prescriptive.
 The reviewers/inspectors are not harmonized!
 Different reviewers may make different comments on a
given situation

FDA Guidances Are not Prescriptive.

Now What?
Strategy –
 Collect some data and information on current state
 Observe FDA practices
 Get input from professionals in the field
 Develop a model for what FDA wants industry to do

21
1. Warning Letter – Sampling Plans
Firm using sampling plans incorrectly
 Pooled X vials, used only 1 reportable value, but
used n=X in sampling plan.
 ….based your lot or batch acceptance/rejection
criteria on a single reportable value averaged
from a pooled sample.
…
For ….., you are collecting 3 pooled samples (each
pool = 10 vials). This equates to a lot disposition
action on 3 reportable values with corresponding
AQL of X% and LQ of X% respectively. This is
not equivalent to an X or X plan as claimed in
your SOP.

Source: Karthik Iyer, Senior Policy Advisor, CDER/OC/OMPQ

March 9th, 2012, AOAC Conference

22
 1. Warning Letter – Sampling Plans
 Response to 483 indicated firm did not know how to use
and interpret sampling plans correctly.
 Firm did not understand concepts of Acceptable Quality
Level (AQL) and Limiting Quality (LQ) and Operating
Characteristic Curve (OC) of a specific sampling plan.

OC Curve with
Producers Risk (AQL) and
Consumers Risk (LTPD)

Source: Karthik Iyer, Senior Policy

Advisor, CDER/OC/OMPQ
March 9, 2012, AOAC Conference

23
 FDA Response – Case A
Questions Use of Valid Statistical Rationale
And Sample Sizes
 “Failure to ensure that sampling methods are based on a valid
statistical rationale, are adequate for their intended use, and
that the sampling plans are reviewed when changes occur, as
required by 21 CFR 820.250(b).”
 “In terms of the statistical sample, your response cited CAPA;
however, our review of this document did not reveal any
actions regarding sample size vs. lot size. The other
document referenced in your response is your Statistical
Techniques procedures. You have not provided any
documentation regarding the statistical validity of your new
sample size.”

24
 FDA Response – Case B
Questions Rationale for Statistical Methods Used an
And Procedures for Use of Selected Methods
“Failure to establish and maintain adequate procedures
for identifying valid statistical techniques required for
establishing, controlling, and verifying the acceptability of
a process capability and product characteristics, as
required by 21 CFR 820.250(a). For example, when the
investigator requested the statistical rationale for
selecting verification testing, your firm said that there was
no rationale.”

25
 FDA Response – Case C
Lack of Appropriate Trending Procedures
and Appropriate Statistical Methods
 Your firm's response to Observation 3.a is inadequate
because it fails to address appropriately trending the 66,390
complaints received between 1/1/2013 and 7/15/2014
 Your firm failed to analyze service reports following
appropriate statistical methods, as required by 21 CFR
820.200(b).
 “Your firm failed to adequately establish procedures for
receiving, reviewing, and evaluating complaints by a formally
designated unit, as required by 21 CFR 820.198(a).

26
 FDA Response – Case D
Failure to Use Appropriate Sampling Methods and
Provide Statistical Rationale

 “Failure to ensure that sampling methods are based on a valid

statistical rationale, are adequate for their intended use, and
that the sampling plans are reviewed when changes occur, as
required by 21 CFR 820.250(b).”
 “The adequacy of your firm’s response cannot be determined
at this time. You have indicated you will be making revisions to
the sampling schedule and revising the procedure. These are
planned actions that we cannot evaluate at this time.”

27
 FDA Response – Case E
Failure to Establish and Maintain Procedures for
Identifying Valid Statistical Techniques
Trend Root Causes Not Considered
 “Failure to establish and maintain procedures for identifying
valid statistical techniques required for establishing,
controlling, and verifying the acceptability of process
capability and product characteristics, as required under 21
CFR 820.250(a).”
 “The trending of the data utilized by your QIT to review and
monitor the effectiveness of the corrective and preventive
actions is inadequate since the variable factors that might
affect the true outcome of the trend were not considered and
thus product characteristics were not established, controlled,
and verified.”

28
 Summary: Some Problems Found by FDA

 Acceptance Sampling Plans – not understood or used properly

 Valid Statistical Rationale And Sample Sizes not Used
 Valid Statistical Rationale And Company Procedures Regarding
Use of Statistics
 Lack of Appropriate Trending Procedures
 Lack of Appropriate Statistical Methods
 Failure to Establish and Maintain Procedures for Identifying Valid
Statistical Techniques
 Critical sources of trend (root causes) not considered
 Establish Cause and Effect: Y = f(Xs)

29
 Agenda
 Today’s Reality: FDA Guidance Regarding:
 Quality by Design
 Continued Process Verification
 Test Method Development
 FDA Guidance’s – Examples of Recommended use of
Statistical Thinking and Methods
 Tools, Methods and Personnel
 What is the Appropriate Documentation?
 What’s the FDA Looking for?
 Right Process Measurement, Data, Tools and Methods and
Interpretation
 Useful Standards
 Understanding of tool usage and interpretation of results
 Tips, Traps and Recommendations

30
 What is FDA Looking For?

 Good rationale for statistical methods used

 Documentation for statistical methods used
 Records that show:
 Proposed methods were actually used
 Those using the methods were trained on the use of
the methods
 Documentation of validation of statistical software
used – purchased and internally developed
 Use of Standards – ASTM, ANSI, ……..

31
 Make Use of Standards
ASTM
Standard Standard Practice
Demonstrating Capability to Comply with the Test for Uniformity
E2810 of Dosage Units
E2782 Standard Guide for Measurement Systems Analysis (MSA)
Demonstrating Capability to Comply with an Acceptance
E2709
Procedure
E2587 Use of Control Charts in Statistical Process Control
E2281 Process and Measurement Capabilities
E2234 Sampling a Stream of Product by Attributes Indexed by AQL
Statistical Procedures to use in Developing and Applying Test
E1488 Methods
Calculating Sample Size to Estimate, With Specified Precision,
E122 the Average for a Characteristic of a Lot or Process

32
 What is FDA Looking For?
 Factor in risk/confidence everywhere.
 Capability Indices:
 Consider the lower confidence interval of capability indices.
 If your sample size is too small, you may find that Ppk is 1.4 but the
lower CI for Ppk is below 1.0.
 Sample sizes: Use appropriately calculated sample sizes. Make
statements about power, confidence, and risk.
 Product Stability Studies: When extrapolating, always use confidence
lines and see where they intersect the spec limit.
 Use statistically designed experiments – Collect good data
 Randomize, replicate, block when appropriate, execute according to a
protocol, analyze and interpret appropriately.

33
 Comments on Specific Methods

 Perform risk assessments – How can the process fail

 Use Failure modes and Effects Analysis (FMEA)
 Control charts – Use the correct chart
 Average and Standard Deviation
 Individuals and Moving Range
 Percent Defective
 Defects per Unit
 Demonstrate process understanding.
 Identify critical process parameters and critical raw
material attributes.
 Establish Cause and Effect: Y = f(Xs)

34
 Measuring and Reducing
Process Risk Using
Process Capability Indices

• Process Capability
• Short-Term (Cp and Cpk)
• Process Performance
• Long-Term (Pp and Ppk)

35
 Capability versus Performance
• Process Capability is the
variation the process would
exhibit if only common cause Process

variation were present: Process Performance

Capability
 “The variation in the
process if the angels ran
the process”
• Process Performance is the
total variation experienced
by the customer; includes
common cause, structural,
and special cause variation:
“The variation when we
mortals run the process”
36
 How Large A Sample Do I Need?
• Practical Perspective
 Sample until total range of process variation has been
observed
• Capability indices are highly variable when estimated from small
samples
• Statistical Perspective:
• Bare Minimum n>30-90
• Preferred: n>60-120
Conservative Strategy: Report the lower confidence limit on the
capability indices: Example:
 Sample n=30, Ppk = 1.30; Conf. Limits: 1.02 – 1.76
 Report “Ppk = 1.02 (Lower 95% CL)”
Ref: Capability Indices Confidence Limits
ASQ (1999) Quality Assurance for Chemical and Process Industries, 2nd
Edition, Quality Press, page 41.
37
Capability Indices Are Highly Variable
Ref: ASQ (1999)

True Sample Lower Upper

Ppk Size Ppk Ppk
1 30 0.76 1.31
1 60 0.83 1.21
1 120 0.88 1.14
1.33 30 1.02 1.76
1.33 60 1.11 1.61
1.33 120 1.17 1.52
1.67 30 1.29 2.19
1.67 60 1.43 2.01
1.67 120 1.47 1.90

Account for Sample Size by Reporting the

Lower Confidence Limit for the Capability Index
38
 .35

Plasticizer/
Cellulose
.15
150
Temperature
135
51 68
Plasticizer

Designing Experiments
Two-Level Designs
How Many Test Runs Should Do?

39
Portable Power: Number Of Runs Vs. Sensitivity
Bob Wheeler, Technometrics (1974)

Rule of Thumb: for balanced 2-level factorial designs

(Plackett-Burman, Fractional Factorial, Full Factorial)

= smallest size effect worth detecting

= standard deviation of experimental variation
(your best guess)
 = “signal-to-noise ratio”

 2.0 1.5 1.0 0.5

n 12-16 22-28 49-64 196-256

2
7 or 8
n =
(  )
40
Design of Experiments - Sample Size Selection
 2.0 1.5 1.0 0.5
n 12-16 22-28 49-64 196-256

Number of Fractional Factorial Plackett-Burman Detectable Effect

Runs (n) Design Design Signal/Noise
8 X 2.5 - 2.8
12 X 2.0
16 X 2.0
20 X 1.5
24 X 1.5
28 X 1.5
32 X 1.2 - 1.4

Size of Effect Standard

to Detect Deviation Signal/Noise Candidate Designs
Plackett-Burman (12 runs)
10 units 5 Units 10/5 = 2.0 Fractional Factorial (16 Runs)
Plackett-Burman (20 runs)
Plackett-Burman (24 runs)
7.5 units 5 units 5.5/5 = 1.5 Plackett-Burman (28 runs)
41
Some Commonly Used Statistical Tools
 Summary Statistics – Average, Std Dev, Median, Coefficient
of Variation (Relative Std Dev), percentiles
 Student’s t-Test
So if I learn these tools:
 Analysis of Variance (ANOVA)
 Design of Experiments
 How do I use them?
 Regression Analysis
 When do I use them?
 Variance Components Analysis
 In what priority?
 Control Chart
Are some of these tools
more useful than others?
 Magnificent 7 Graphical Tools
 Yes, a graph is a Statistical Tool
 Process Capability Analysis
 Contingency Table Analysis
 Time Series Analysis
 Partial Least Squares
42
Problem Solving Approach Identifies the Right Tools
Not the Reverse!
 What’s the problem?
 Context, process, data pedigree, goals and
objectives
 Problem Solving Approach and Statistical Tools to be
used?
 Statistical tools to aid the problem solving process?
 Results – What Results? Desired Format?

Not Understanding the Problem to Be Solved Can

Result in a Statistical Tool being Used as
“A Hammer Looking for a Nail”

43
Problem Solving Approach Identifies the Right Tools
Not the Reverse!
Statistical
Tools
Results ($$)

Select Tools Perform Analysis

Run Chart?
Practical Identify
ANOVA?
Solution
DOE?
and
T-Test?
Collect Action
Regression?
Data Scatter Plot? Graphical Analytical
Other Tools???

“Statistics: Ask the Right Questions – Once You Know what the Objectives and
Needs Are, You Can Figure Out Which Tools Can Help Meet Your Goals”, Snee,
Pharma Manuf, Jan 2013, p 15.
44
 Statistical Significance is
Not the Same as Practical Significance
Statistical Significance: Observed difference is larger than can be
attributed to random chance variation
 Process variation is small: Difference may be statistically significant
but be of no practical importance
 p-Value measures statistical significance, not practical significance
 Statistical significance must be present before it is meaningful to
assess practical significance
Practical Significance: Determined by size and direction of effect:
 Effect (difference) is large enough to:
 Have real meaning in context of study and objectives
 Findings suggest that you do something in a different way
 Justify behavior change – Have an effect on decisions made
 Influence the behavior of professionals in the field
 “Practical Significance” is a function of factors such as cost,
objectives, priorities, requirements, subject matter knowledge, etc.

45
 Appropriate Actions
Effect Statistically Significant?
Effect Practically
Significant? YES NO
YES Take Appropriate Action • Increase Sample Size
• Estimate Effect Size
More Precisely
NO • Take No Action • Action Not
• Look for Other Effects Appropriate
• Keep Effect in Mind • Look for Other
Effect May Become Opportunities
Important Later

Statistical Significance Must Be Present Before

It Is Meaningful to Assess Practical Significance
Ref: Hoerl and Snee (2012) Statistical Thinking – Improving Business
Performance, Wiley

46
 So What Should I Do?
 Use good statistical practice
 Develop rationale for procedures
 Planning: Use selected procedures according to protocol
 Document procedures
 Train appropriate personnel on the procedures
 Provide appropriate validated software
 Demonstrate process understanding
 Establish Cause and Effect: Y = f(Xs)
 Use appropriate standards
 Show that management supports all procedures used
 Focus on practical significance of trends and shifts

Statistical Thinking and Methods Enable Us to

Deal Effectively with Process and Product Variation
47
 For Further Information,
Please Contact:
Ronald D. Snee, PhD
Snee Associates, LLC
Newark, DE
(610) 213-5595
Ron@SneeAssociates.com

Please visit our website at:

www.SneeAssociates.com

48
 References
ASQ (1999) Quality Assurance for Chemical and Process Industries, 2nd Edition,
Quality Press, page 41, capability confidence limits.
Hoerl, R. W. and R. D. Snee (2012) Statistical Thinking – Improving Business Performance,
John Wiley and Sons, Hoboken, NJ.
Snee, R. D. (2013) “Statistics: Ask the Right Questions – Once You Know what the Objectives
and Needs Are, You Can Figure Out Which Tools Can Help Meet Your Goals”,
Pharmaceutical Manufacturing, January 2013, p 15.
Snee, R. D. (2015) “Practical Approach to Data Mining: I Have All These Data, Now What
Should I Do? Quality Engineering, Oct - Dec 2015, Vol. 27, No. 4, 477-487.
Snee. R. D. and R. W. Hoerl (2012) “Inquiry on Pedigree – Do You Know the Quality and
Origin of Your Data?” Quality Progress, December 2012, 66-68
Snee, R. D. , R. R. DeVeaux and R. W. Hoerl (2014) “Follow the Fundamentals - Four Data
Analysis Basics Will Help You Do Big Data Projects the Right Way”, Quality Progress,
January 2014, 24-28.
Weitzel, J., R. A. Forbes and R. D. Snee (2015) “The Use of the Analytical Target Profile in
the Lifecycle of an Analytical Procedure: with an example for an HPLC Procedure”, Journal
of Validation Technology, Vol. 20, Issue 4, Jan 2015

View publication stats 49