Trend Data

Lynn Torbeck
Torbeck and Assoc.
Evanston, IL

June 28, 2007 1

Overview
OOT vs. OOS
Why trend?
How to get started
Types of trends with examples
OOT is relative
Graphical tools
Tend limits
June 28, 2007 2
Why Trend Data?
Good business practice.
Early warning of possible Out Of
Specification (OOS) results.
Gain process understanding.
Minimize risk of potential failures of
product in the market.
Find the “gold in the hills” for process
improvements.

June 28, 2007 3

Regulatory Basis for Trending
No specific regulation requirement
211.180(e) Annual Reviews
FDA Form 483 for observations
Establishment Inspection Reports
Warning letters
FDA presentations at conferences

June 28, 2007 4

OOS Guidance Footnote
“Although the subject of this document
is OOS results, much of the guidance
may be useful for examining results
that are out of trend (OOT).”

How is OOT different than OOS?

How is OOT the same as OOS?

June 28, 2007 5

Out Of Specification - OOS
OOS is the comparison of one result
versus a predetermined specification
criteria.
OOS investigations focus on
determining the truth about that one
value.
Is the OOS result confirmed or not?

June 28, 2007 6

Out Of Trend - OOT
OOT is the comparison of many
historical data values versus time.
OOT investigations focus on
understanding non-random changes.
Is the non-random change confirmed or
not?

June 28, 2007 7

OOS Guidance
Taking into account the differences
between OOS and OOT, the guidance
does provide a framework for OOT
investigations:
Responsibilities
Philosophical basis
General principles of investigations

June 28, 2007 8

1. How to get started
Select the variable to be studied:
 Potency
 Yield
 Impurities
 Hardness
 Bioburden

June 28, 2007 9

2. How to get started
Select a time period:
At least one year if possible.
More than two preferred.
Do not go past a major change in the
process. Use process knowledge to
advantage.
Use the reportable result, the value
compared to the specifications.

June 28, 2007 10

3. How to get started
Enter the data into analysis software:
 Excel
 Minitab
 Sigma Plot
 JMP
 StatGraphics
 Northwest Analytical
 SAS

June 28, 2007 11

4. How to get started
Plot the data vs. time or lot sequence.
Look for non-random changes over
time.
Determine if they are of practical
importance.
Statistical significance is insufficient.
Do an impact and risk assessment.

June 28, 2007 12

What is Trending?
The several activities of:
 Collecting data,
 Recording it,
 Documenting it,
 Storing it,
 Monitoring it,
 Fitting models to it
 Evaluating it, and
 Reporting it.

June 28, 2007 13

What is a trend?
Any non-random pattern.

Short and long term patterns in data

over time that are of practical
importance.

June 28, 2007 14

Beneficial Trends
Desirable patterns in the data series.
Examples:
 A move toward the target or center of the
specification.
 More consistent with less variation.
 Less likelihood of an OOS value.
 A benefit to SSQuIP.

June 28, 2007 15

Beneficial Trend
1.2

0.8
mg/mL

0.6

0.4

0.2

0
12/10/200 6/28/2003 1/14/2004 8/1/2004 2/17/2005 9/5/2005 3/24/2006 10/10/200
2 6

Date
June 28, 2007 16
No Trend
Easier to define
what a trend is not.
104

103

Random data 102

Noise 101

Normal
100
Stationary 99

No ups, no downs 98

No cycles 97

Index 100 200 300 400 500

No outliers

June 28, 2007 17

Neutral or No Trend
Neither beneficial or adverse
Examples:
 Results that are always the same.
 Stability data with a slope of zero.
 Data in a state of “statistical control” on a
control chart.

June 28, 2007 18

Process Control
Statistical Process Control, SPC
 Normal random data over time
 Due to common causes only
Engineering Process Control, EPC
 Estimate departures from target
 Feedback to control point
 Physical changes to the process

June 28, 2007 19

Adverse Trends
Undesirable patterns in the data series.
Examples:
 A movement away from the target.
 Increased variability.
 Increased probability of OOS.
 An unexplained change to a beneficial
trend.
 A challenge to SSQuIP.

June 28, 2007 20

Out-of-Trend (OOT)
A change from an established pattern
that has the potential of an adverse
effect on SSQuIP or of becoming OOS.
Must be large enough to be of practical
significance.
Statistical significance is insufficient to
determine OOT.

June 28, 2007 21

Long Term Change
Not stationary Increasing Trend
.1 per step after 50
around a fixed value 107

Increasing or
106
105

decreasing average.
104
103

Yields
Apparently will
102
101

continue to get
100
99

worse (or better) 98

Index 10 20 30 40 50 60 70 80 90 100

unless action is
taken.

June 28, 2007 22

The Aberrant Outlier
Stationary and An outlier

random but with

Mu=100, Sigma=1.0
105

one very large value 104

that could be a
103

102

statistical outlier.

Yield%
101

100

Generally assumed 99

to be due to a
98

97

“special cause.” Index 10 20 30 40 50 60 70 80 90 100

June 28, 2007 23

Shift in the Average
Here the mean has Mean Shift
Mu=100 to 104 Sigma=1.0
increased from 100 106
to 104 at sample 51.
No other changes
were made.

Yield
101

Variability is the
same. 96
Index 10 20 30 40 50 60 70 80 90 100

June 28, 2007 24

Variation Change
This is stationary Increasing Variability
Mu=100, Sigma=1.0, 2.0, 3.0 & 4.0
around a fixed mean 110

of 100%.
But, the standard

Yield %
deviation increased
100

from 1.0 to 4.0.

90

Index 10 20 30 40 50 60 70 80 90 100

June 28, 2007 25

Cycles
A reoccurring cycle. Cycles

Stationary about a 104

fixed mean. 103

102

The data are not 101

Cycles
100

independent. 99

98

97
96

Index 10 20 30 40 50 60 70 80 90 100

June 28, 2007 26

Autocorrelated
Data are correlated Autocorrelated

with the previous

data.
105

104

Not stationary. 103

AutoCorr
102

Check different time 101

lags, 1,2, ….
100

99

98
Index 10 20 30 40 50 60 70 80 90 100

June 28, 2007 27

OOT is Relative
Stationary White Noise
mu=100%, S=1%
110
Yield %

100

90
Index 10 20 30 40 50 60 70 80 90 100

June 28, 2007 28

OOT is Relative
The importance of a trend is its size
relative to the specification criteria.
A state of Statistical Control is desired
but not necessary.
A state of Engineering Control is
necessary to meet specifications.
Success is a marriage of the two.

June 28, 2007 29

A Little Humor (Very Little)
Lottery: A tax on the statistically-
challenged.
If you want three opinions, just
ask two statisticians.
Statistics means never having to
say you're certain.
http://www.keypress.com/x2815.xml

June 28, 2007 30

Trend Fitting
“The general process of representing
the trend component of a time series.”
A Dictionary of Statistical Terms. Marriott

Depends very much on the type of data

and the subject matter being studied.
Need to adapt the tools and techniques
to our specific data and issues.

June 28, 2007 31

Tools of Trending
Summary statistics
 Averages, Medians
 Ranges, Standard Deviations, %RSD
Graphical plots
Distribution analysis - Histograms
Outlier determination
Regression analysis

June 28, 2007 32

Graphic Tools
Line Plots vs. time.
Shewhart Control Charts.
Histograms.
Sector chart

June 28, 2007 33

Line Plots vs. Time
Stationary Time Series
Mu=100, Sigma=1.0
Response on the 102

vertical axis.
101

Time or batch # on

Yield %
100

the horizontal axis. 99

Usually connect the 98

data points with a Index 10 20 30 40 50 60 70 80 90 100

line, but optional.

June 28, 2007 34

Control Chart
I and MR Chart for Yield %

Add ‘natural process 103.5

UCL=103

Individual Value
102.5

limits’ to the line

101.5
100.5
Mean=100
99.5

plot.
98.5
97.5
LCL=97
96.5
Subgroup 0 50 100

±3 4
UCL=3.686

Moving Range
3

A chart for the

2

1 R=1.128

response.
0 LCL=0

A chart for the

variability.

June 28, 2007 35

Control Chart Family
Individuals
Averages
Medians
Standard deviations
Ranges
Number of defectives
Fraction defectives
Defects per units
Number of defects
June 28, 2007 36
Variation Change
I and MR Chart for Yield %

A control chart will 110

1

Individual Value
detect change in the 1
1
1
1 11 1
1
UCL=103

variation.
100 Mean=100
LCL=97
11 11 1 1
1 1 1 1
1 1 1
90 1
1 1
Subgroup 0 50 100

1
1

Moving Range
1 1
10 1
11
1 1
1 1 11
1 1 1 11 111 11
1 1 11 1 1
5 1 111 1 1
UCL=3.686
R=1.128
0 LCL=0

June 28, 2007 37

The Outlier
A control chart finds I Chart for Yield%

values outside the 106

105
1

natural limits of the 104

Individual Value
data.
103 UCL=103
102

The value is larger 101

100 Mean=100
than would be 99

expected by chance
98
97 LCL=97

alone. 96
0 50 100
Observation Number

June 28, 2007 38

“Western Electric” Rules
1. One value outside 3 S limits.
2. Nine values in a row on one side of
the average.
3. Six values in a row all increasing or
decreasing.
4. 14 values in a row alternating up and
down.

June 28, 2007 39

“Western Electric” Rules
5. Two of three values greater than 2 S
from the average.
6. Four of five values greater than 1 S
from the average.
7. 15 values in a row within 1 S of the
average.
8. Eight values in a row greater than 1 S.

June 28, 2007 40

Histogram
Show the ‘shape’ of
the distribution of 20

data.

Frequency
In this case it is 10

Normally distributed.
0

96 97 98 99 100 101 102 103 104

Yield %

June 28, 2007 41

The Outlier
The outlier is clearly Variation Change
seen in the 20

histogram.

Frequency
10

97 98 99 100 101 102 103 104 105 106

Yield%

June 28, 2007 42

Outlier Determination
Reference:
 USP 30 NF 25
 Chapter <1010>
 “Analytical Data – Interpretation and
Treatment”
 Page 392 “Outlying Results”
 Appendix C: Examples of Outlier Tests for
Analytical Data.

June 28, 2007 43

Regression Analysis
99% Prediction Interval

220000

200000

180000
Value

160000

140000

120000

100000
0 5 10 15 20
Months
June 28, 2007 44
Trend Limits
Numeric (or non-numeric) criteria, that if
exceeded, indicates that an out-of-trend
change has occurred.
Usually the ‘natural process’ variation
AKA “Alert limits”
Use Statistical Tolerance Limits
See USP <1010> Appendix E

June 28, 2007 45

Here, Trend This

40
Var 1

30

20

Index 100 200 300

June 28, 2007 46

A New Engineering Chart
Brings together for the first time:
 Comparison to the specification limits in
place of the probability limits
 Divides the specification range into equal
zones in place of 1, 2, & 3 sigma areas
 Uses cumulative scores
Pharmaceutical Technology, April 2005

June 28, 2007 47

 The New “Sector Chart”
SIALIC ACID EXAMPLE
Fail
3.915 3.695 3.298 4.04 3.87 4.147 3.938 4.167 3.9 3.927 3.81 3.9 4.033 3.853 4.142 3.958 3.77
Sector Weight Low High
F 10
D 2 4.1 4.2 2 2 2
C 1 4 4.099 1 1
B 0 3.9 3.999 0 0 0 0 0 0
A 0 3.8 3.899 0 0 0
A 0 3.7 3.799 0
B 0 3.6 3.699 0
C 1 3.5 3.599
D 2 3.4 3.499
F 10 10

June 28, 2007

Batch 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 4816 17
The New “Sector Chart” Rules
The first batch tally takes the weight of the
sector it is in.
Subsequent batches have a cumulative tally
of the previous tally plus the current sector
weight.
If the tally reaches a value of, say, 10, an
alert is given.
If the batch enters the A or B sectors, the
tally is reset to zero.

June 28, 2007 49

The New “Sector Chart” Rules
Sectors A and B cover the center 50%
of the specification range.
Sector F is outside the current
specification.
Other weights can be set to fit the
process and the degree of sensitivity
needed.

June 28, 2007 50

Advantages of Sector Chart
No minimum sample size. Can start with
one data point.
No assumptions about the data at all.
Identifies beneficial and adverse trends.
Weights and tally total are selected by
scientific and empirical knowledge.
A decision is made with each new point.
Alerts quickly if a problem exists.
June 28, 2007 51
Justification for Sector Chart
If the process is well inside the
specification, it need not be in a state of
statistical control.
The focus is on OOT and SSQuIP not
being out of “statistical” control.
Sensitivity of the chart is adjustable.
Can be use in parallel with other charts.

June 28, 2007 52

That’s All Folks
Summary Points:
1. OOT is not OOS
2. OOT is non-random changes over time
3. OOT is a statistical and graphical issue
4. OOT is relative. Statistical significance
is not sufficient.
5. Trend limits = Natural Limits

June 28, 2007 53

References
Graphics:
 http://www.edwardtufte.com/tufte/
 http://www.itl.nist.gov/div898/handbook/e
da/section3/eda34.htm
Statistics
 http://www.itl.nist.gov/div898/handbook/in
dex.htm

June 28, 2007 54

Software References
http://www.minitab.com/
http://www.systat.com/products/sigma
plot/
http://www.nwasoft.com/
http://www.jmp.com/
http://www.statgraphics.com/
http://www.sas.com/

June 28, 2007 55