Fundamentals of Bioassay Practices

Timothy Schofield
Vice-Chair, USP Statistics Expert Committee
 Course Outline

 Bioassay guidelines
 Terminology
 Quality by Design for analytical methods
 Key Concepts
 The bioassay lifecycle
 Bioassay development
 Bioassay validation
 Bioassay maintenance
 Considerations in calibration assays

3
 Some Common Misconceptions

 Statistics is math and is therefore hard

to understand
– The math has been taken care of by
mathematical statisticians; applied
statistics is mostly application of the
scientific method and an appreciation of
variability.
 All I need to know about statistics is
how to calculate a mean and a Hypothesis
standard deviation
– There’s more to statistics than number Design

crunching. A good statistical experiment

consists of (a) a well formulated question, Conduct

(b) an effective and efficient design, (c)

proper conduct of the experiment, and (d) Analyze

an appropriate analysis with clear

conclusions.
 I can get my computer program to do
my statistics
– Like a car, a computer program will only
take you where you direct it; if you take the
wrong route, it may take you longer or you
might reach the wrong destination.
4 4
 Bioassay guidelines

 Originally USP <111> and EP 5.3

 <111> was split into two chapters, USP <1032> Design
and Development of Biological Assays and USP
<1034> Analysis of Biological Assays
 <1033> Biological Assay Validation added to the suite
Design and
Development of
Biological Assays

“Roadmap” chapter
(includes glossary)

Analysis of
Biological Assay
Biological Assays
Validation 5
 Bioassay guidelines (cont.)

 Bioassay (Biological Assay) – ICH Q6B

– A valid biological assay to measure the biological activity

should be provided by the manufacturer. Examples of
procedures used to measure biological activity include:
• Animal-based biological assays, which measure an
organism's biological response to the product –in vivo
bioassays
• Cell culture-based biological assays, which measure
biochemical or physiological response at the cellular level – ex
vivo and in vitro bioassays
• Biochemical assays, which measure biological activities such
as enzymatic reaction rates or biological responses induced by
immunological interactions <Note- USP covers immunoassays
separately – but the validation principles are the same>
Excerpted from ICH Q6B, Specifications: Test Procedures And Acceptance Criteria For
Biotechnological/Biological Products (2008)

6
 Bioassay guidelines (cont.)

 Potency
– Approval of a biologics license application or issuance of a biologics
license shall constitute a determination that the establishment(s) and
the product meet applicable requirements to ensure the continued
safety, purity, and potency of such products [US Code of Federal
Regulations, 21 CFR 600.2(a)]

– Potency [ 21 CFR 600.3(s)]

• The specific ability or capacity of the product, as indicated by appropriate
laboratory tests or by adequately controlled clinical data obtained through
the administration of the product in the manner intended, to effect a given
result.

• From 21CFR 610.10, “Tests for potency shall consist of either in vitro or in
vivo tests, or both, which have been specifically designed for each product
so as to indicate its potency in a manner adequate to satisfy the
interpretation of potency given by definition in § 600.3(s) of this chapter.”

– Bioassay measures the potency of a test material

7
 Terminology

 Bioassay – analysis (as of a drug) to quantify the

biological activity/activities of one or more
components by determining its capacity for producing
an expected biological activity
 Potency – a measure of the ability of a product to
achieve its desired effect
 Similarity – the property of bioassay where a test
sample behaves “similarly” to the standard across a
concentration series (also called parallelism for linear
models)
 Quality (pharmaceutical) – adequate safety,
potency and purity of pharmaceutical products
 Consistency – the property of being reproducible
within process tolerances
8
 Terminology (cont.)

 Relative accuracy – the measured potency, relative

to the expected potency level; measured as relative
bias
 Trend in bias – a trend in relative bias across the
range of the assay
 Geometric coefficient of variation (GCV) – a
measure of precision for geometrically scaled
measurements (log-normally distributed
measurements)
 Log-normal distribution – a skewed distribution
characterized by wider spread at higher levels
 Process capability – the state of a process to meet
its intended requirements

9
 Terminology (cont.)

 Assay – the body of data used to assess similarity

and estimate potency relative to a Standard
(synonymous with run)
 Replicate -the minimal elements of a test sample
which yields a potency measurement
 Pseudo-replicate – a replicate measurement,
usually from an aliquot of a dilution of a sample
 Configuration – the number and placement of test
samples and the standard in a bioassay
 Format – the combination of number of runs and
number of replicates within a run
 Reportable value – the final result for a test sample
which is compared to the specification
10
 Quality by Design

Quality by Design (QbD) is a development

paradigm which springs from ICH
– Q8: Pharmaceutical Development
– Q9: Quality Risk Management
– Q10: Pharmaceutical Quality System
Premises of QbD
– Build quality into the process rather than test the
product into quality
– Use of “risk based” tools to ensure quality
– Knowledge management and strategic uses of
prior knowledge

11
 Quality by Design (QbD) for analytical
methods

 Industry and regulators have begun to recognize that

analytical methods generate a product –
measurements
 Like pharmaceutical products, measurements should
have adequate quality to meet their intended use –
making decisions
 The fundamental goals of product development are:
– Safety and efficacy (hitting the clinical target)
– Variance reduction
 The fundamental goals of analytical development are:
– Accuracy (hitting the analytical target)
– Variance reduction

12
 QbD for analytical methods (cont.)
 Many of the concepts associated with QbD for
pharmaceutical products translate to similar concepts for
analytical methods
Process Concept Analytical Counterpart

Target Product Profile (TPP) Analytical Target profile (ATP)

• Target clinical performance, requirements • Target analytical performance requirements,
manufacturing, and commercial requirements testing laboratory, and customer requirements
Critical Quality Attributes(CQAs) Performance attributes (validation parameters)
• Potency • Precision
• Aggregation • Sensitivity
• Purity • Accuracy
Specifications (acceptance criteria) Acceptance criteria
• 80% to 125% potency • %GCV < 10%
• Purity > 95% • LLOQ > 1 ng/mL
Critical process parameters Critical assay parameters
• pH, time, temperature • pH, time, temperature
Process control strategy Assay control strategy
• Comparability protocols • Comparability protocols
• Tech transfer • Method transfer
Continuous verification Continuous verification
• Continuous review and updating of process • Continuous review and updating of analytical
knowledge knowledge
13
 QbD for analytical methods (cont.)

 The bioassay should be fit for its intended uses

throughout the bioassay lifecycle
– Should perform adequately to support development and
commercial decisions
 Decisions are made day-to-day using bioassays
– During development
• Which formulation provides the best stability?
• Does a particular process step impact potency?
• Are the test and standard curves similar?
– During manufacture
• Should a manufactured lot be released to the market?
• Should a process change be implemented?
• Has a manufactured lot maintained potency over it shelf-life?
• Can a new potency standard be used in the bioassay?

14
 QbD for analytical methods (cont.)

 All decision are made with risk, and risk is costly

– Risks during development
• Risk of determining “the process” is suboptimal when it is satisfactory
• Results in excessive development costs or program failure
• Risk of determining the process is satisfactory when “the process” is
suboptimal
• Results in excess downstream costs to fix the problems
• Risks during manufacture
• Risk of failing satisfactory product –regulatory burden and interruption of
supply of quality product
• Risk of passing unsatisfactory product – potential risk to the “customer”
 Decision risk can be managed in several ways
– Use sound scientific reasoning and/or experience to guide decisions
– When decisions are made on the basis of empirical evidence,
decision risk is associated with the strength of the evidence

15
 QbD for analytical methods (cont.)

 The strength of the evidence is related

to the degree of uncertainty (variability)
in a result N=4
 High bioassay variability is associated
with increased decision making risk N=1
• Risk can be depicted as the area under
a measurement distribution curve LSL USL

– The area (risk) can be reduced through

replication
• A “reportable value” is the average
of replicate measurements
25
n SEM
• The variability of an average is the 20

standard error of the mean (SEM) 1 20

Variability (SEM)
15

• Multiple replicates (n) reduce the 2 14 10

variability of the reportable value 3 12

5

0
0 3 6 9 12
6 8 Number of Replicates (n)

s
SEM 
n

16
 QbD for analytical methods (cont.)

 Reportable value uncertainty

– Uncertainty  variability (s)
– “The result is within the variability of the assay”
– The variability of the assay can be used to determine the
uncertainty in a reportable value or in a difference between
reportable values
– Margin of error (MOE) – how close the measurement is likely to be to
the true value
MOE  t df 
s s  2
n n
– Critical difference (CD) – the difference between two measurements
that’s likely to be “statistically significant”

CD  t df  2  s
2
n
– Number of significant digits in the specification
– Two for 2% < %GCV ≤ 20%
– Round only after all calculations have been made (e.g., stability data)

17
 QbD for analytical methods (cont.)

 Reportable Value
– The reportable value is the “statistic” which is compared to the
acceptance criterion
• Linked to the intended use of an assay
• Single measurement, GM of 3, regression estimate, etc.
– Changes in the design (“n” and/or other design factors such as stability
intervals) can be used to manage the measurement uncertainty of the
reportable value for a bioassay with a given variability (s)
– Different formats (“n”) can be used for different uses of the bioassay
• Lot Release
– The reportable value is the average of multiple measurements on the
batch
• Standard qualification
– The reportable value is the assigned value for the standard
• Comparability
– The reportable value is the difference between comparators

18
 QbD for analytical methods (cont.)

Steps in reducing uncertainty

– Convert bias to random
variability through appropriate
randomization and blocking
• Can’t improve bias by improving
precision
– Minimize random variability (s)
• Optimize assay
• Exert control wherever possible
– Mitigate the impact of random
variability (n)
Uncerta int y  tdf  s
• Replicate and average results to n
reduce reportable value
uncertainty

19
 QbD for analytical methods (cont.)

 There are two types of risk in

decision making (statistical
hypothesis testing) Reality

– Type I error (α) True False

β
• Producer’s risk in quality control (failing

Decision
True -

a good lot) False α Power

• False positive from an analytical

procedure
– Type II error (β)
• Consumer’s risk in quality control
(passing a bad lot)
• False negative from an analytical
procedure
False True
– In statistical hypothesis testing these Conclude False Conclude True

are managed simultaneously by sample

size
20
 QbD for analytical methods (cont.)

 Producer’s risk should be

balanced against consumer’s 5%chance
5% chanceofof reject aa good
rejecting lot.
good lot
20%
20% chance of passing a bad lotlot.
chance of passing a bad
risk
– Consider the example of lot release
versus a lower specification limit
0.05 0.20

– Modifying the lower release limit to

decrease the chances of failing a Bad Lot Good Lot

good lot increases the chances of Reject Lot Release Lot

passing a bad lot Risk is simultaneously reduced by reducing variability

Risk simultaneously reduced by decreasing variability
1% chance
chanceofofrejecting
reject aagood
goodlot.
lot
1% chance
chanceofofrejecting
reject aagood
goodlot.
lot 1% chance of passing a bad lot
40% chance of passing a bad lot
1% chance of passing a bad lot.
40% chance of passing a bad lot.

0.40 0.01 0.01

0.01

Bad Lot Good Lot Bad Lot Good Lot

Reject Lot Release Lot Reject Lot Release Lot

21
 QbD for analytical methods (cont.)
 There are two types of risk in decision
making (statistical hypothesis testing)
– Type I error (α)
• Producer’s risk in quality control (failing a 
Bad Lot Good Lot
good lot)
• False positive in analytics
• False failure in validation
α β

– Type II error (β)

 Reject Accept 
• Consumer’s risk in quality control (passing a s s
t × t ×
bad lot) n n

• False negative in analytics

• False success in validation
s s
– In statistical hypothesis testing these are t   t  
n n
managed simultaneously by sample size
n
t   t   s2
2

• Declare a meaningful difference to detect ()

2
• Constrain the uncertainties within 
• Solve for sample size (n)
22
 Key Concepts
 Bioassay validation
– Bioassay validation is “The process of demonstrating and
documenting that the performance characteristics of the bioassay
meet all of the requirements for the intended applications and is
thereby fit for its intended use.”
• A protocol driven study that is conducted after bioassay development has
been completed, with acceptance criteria that help to demonstrate that the
bioassay has properties suitable for its intended uses

– The “validity” of a bioassay should be considered throughout the life-

cycle of a product
• Bioactivity of product variants
• Reliable estimates of development materials
• Stability of development materials
• Release and stability of product lots

– Validity of bioassay measurement should account for the current uses

of the bioassay
• Thus we are always “validating” the bioassay to help assure that we make
good decisions
23
 Key Concepts (cont.)

 Bioassay design requirements (Analytical Target Profile)

– “Bioassay design requirements” are the complete set of
requirements that must be met for a bioassay to be considered an
effective and efficient tool for assuring product quality
• Practical requirements – cost, turnaround time, throughput
• Target performance requirements – throughout the bioassay lifecycle
– Specificity
» Includes “selectivity”; lack of interference from matrix components
– Relative accuracy
» Demonstrated through expected relation to dilution
– Intermediate precision
» Based upon a study which simulates long-term performance of
the bioassay (usually in a single lab)
– Range
» Using samples which span the potential range of measurements
in the bioassay (e.g., release to end of shelf-life)
24
 Key Concepts (cont.)

 Bioassay design requirements (Analytical Target Profile)

– Requirements change throughout the bioassay life cycle
– An early requirement is “fundamental validity”
• Adequate concentration response
• Assessment of parameters related to bioassay “accuracy” such as
specificity and selectivity
– Increased rigor for bioassay precision as one proceeds through
development, and into quality control
• Requirement for reliable estimates of potency throughout
development
– Stability assessment (shelf-life/release assay determination) as well as
development lot potency
• Use of bioassay optimization to reduce variability
• Apply best laboratory practices to reduce variability, including
strategic replication based on an understanding of the key sources
of variability

25
 Key Concepts (cont.)

 Relative potency (RP)

– A measure obtained from the comparison of a test sample to
a standard based of the capacity to produce the expected
biological activity
– RP is unitless and is given definition for any test material
solely in relation to the reference material
• The master standard is usually assigned a potency equal
to 1.0
• May occasionally be assigned based on another property
(e.g., protein content)
– A standard should be incorporated early into the bioassay
• Helps to reduce the variability produced by inter-assay
factors which impact response
• Standardizes potency throughout the lifecycle of use
(including between laboratories)
• Consider a primary standard
26
 Key Concepts (cont.)

 Relative potency bioassay designs

– Parallel line design

Parallel Line Analysis
10
Parallel Response

• Horizontal distance in parallel 1

Response
linear log(concentration) response RP
Standard
curves between test samples and 0.1

Test
the standard 0.01
0.1 1 10
Concentration

Parallel LineCurve
Parallel Analysis using 4PL
Analysis

– Parallel curve design

230 110%
100%
90%
180
80%

• Horizontal distance in parallel 70%

Response
130 60%
RP
50%

nonlinear log(concentration) 80
40%
30%

response curves between test 30

20%
10%
0%

samples and the standard -20

0.1 1 10 100 1000
-10%
10000
Concentration

27
 Key Concepts (cont.)

 Relative potency bioassay designs

Slope Ratio Analysis

– Slope ratio design 12

Standard Slope = 0.75
10

• Ratio of slopes in concentration

Test Slope = 2.37
8 RP = 3.14
Standard
6
response curves determined 4
Test

between test samples and the 2

0
standard 0 1 2 3 4 5

– Quantal response bioassay 100%

Parallel Line Bioassay
Quantal Response Analysis

design 90%
80%

Percent Response
• Horizontal distance in quantal 70%
60%

response/log concentration 50%

40%
response curves between test 30%
20%
samples and the standard 10%
0%
• Linearized through transformation 100 1000
Potency
ED
10000
50 ED50 100000
Test Standard

28
 Bioassay Development

 Many of the bioassay properties are driven by

bioassay design and data modeling
– Bioassay models
– Dosing and replication
– Blocking and randomization
– Optimization
– Validity criteria and outlier detection
– Transformation and weighting

29
 Bioassay Models
– Bioassay models
– Dosing and replication
–
 Linear versus nonlinear models –
Blocking and randomization
Optimization
–
– Most bioassay kinetics is governed –
Validity criteria and outlier detection
Transformation and weighting
by principles which produce a
nonlinear relationship between 200

assay response and drug [C] 180

160

• Mass action equations of Michaelis 140

Response
120

and Menten predict a sigmoid 100

80

concentration response model 60

40

• This can be re-expressed in the 20

form of a four parameter logistic 0.1 1 10

Concentration
100 1000

(or 4PL) model

– Historically, several factors have led to the use of linear
approximations to this theoretical kinetics
• Lack of software to fit nonlinear kinetics functions
• Capacity of systems to generate data for nonlinear kinetics
• log-log fits were adequate to support many systems

30
 Linear Approximations to Sigmoid Curves
 Using the approximately linear portion
of the sigmoid curve
– Select middle “linear” region
• Note: approximately linear; the linear
region can shift, yielding inaccurate
potency determination and
truncation error .

 Using the lower region of the

sigmoid curve
– The lower region can be linearized
using a log-log fit
• Log response is usually a
better scale (normally
distributed)

 Using the upper region of the

sigmoid curve
– A hyperbolic transformation will
linearize response

31
 Choosing a Non-Linear Model

 The four parameter logistic (4PL) model is a nonlinear function

characterized by 4 parameters
– a = upper asymptote, d = lower asymptote,
b = slope at inflection point, c = 50th percentile or inflection point

Four Parameter Logistic Regression

ad
y  d 230 110%
 b

1   x 
a = 212.6 (Upper Asymptote) 100%
 90%
 c  180
  80%
70%
Response

130 b = 1.01 60%

[C] Observed Percent
(Slope) 50%
0.6 2 0%
1.25 13 5% 80 40%
2.5 14 6% 30%
5 32 14%
20%
9.9 42 19%
30
19.8 80 37% 10%
39.6 105 49% 0%
79.2 157 74%
d =-20
1.70 (Lower Asymptote) -10%
158.4 170 80%
0.1 1 10 c = 36.16 100 1000
316 195 92%
(50th Percentile)
Concentration

32
 Choosing a Non-Linear Model (cont.)

• The five parameter logistic (5PL) model is a nonlinear function

characterized by 5 parameters
– The same parameters as 4PL, plus g = asymmetry parameter
– Asymmetry caused by assay constraints such as an instrument boundary
– Trade-off of improvement in RP measurement and number of [C]’s

ad
y  d g
  x 
b
 Five Parameter Logistic Regression
1    
  c  

110%
a = 197.6 (Upper Asymptote) 100%
180 90%
80%
 Note: g=1 is 4PL
ED50 dose is
70%
Response

130
60%

[C]
0.6
Observed
2
Percent
1% 80
xc2  1g

1
1b
Observed
50%
40%
4PL
1.25 13 7% 30%
5PL
2.5 14 7% 20%
5 32 16% 30
10%
9.9 42 21% 0%
19.8 80 40% d = 0.00 (Lower Asymptote)
-20 -10%
39.6 105 53%
0.1 1 10 100 1000
79.2 157 79%
158.4 192 97% Concentration
316 195 99%

33
 Quantal Response
 Quantal response is characterized by measurements which are dichotomous
(success or failure)
– e.g., percent survival in a challenge assay, percent responders in an
immunogenicity assay
 Percent measurements are transformed to achieve a linear relationship
• Transformation methods
• Probit transformation
– Probit(p) = z-value + 5 Probit Analysis
100% Dose Rate
– z-value is the standard normal deviate 90%
10 10/10
3 9/10
corresponding to p

Percent Response
80% 1 4/10
70% 0.3 3/10
• Logit transformation 60%
0.1 0 10

– Logit(p) = ln(p/1-p) 50% Probit Fit:

Y1 = 5.1443 + 2.0867 Z
40%
– Both are computationally intensive 30%
Chi-Square = 2.620
(P=0.45)
20%
requiring sophisticated software 10% ED50=0.85

– Yields potency estimates (ED50),

0%
0.1 1 10
Concentration
relative potency, confidence intervals,
and tests of parallelism and goodness-of-fit

34
 Dosing and Replication
• Number of concentrations – Bioassay models
– Dosing and replication
– Number of concentrations should – Blocking and randomization
support desired modeling (i.e., 4 or – Optimization
– Validity criteria and outlier detection
more concentrations for linear modeling, – Transformation and weighting
8 or more concentrations to support 4PL
or 5PL)
– Note: having a full run down in response (0-100%) permits the
estimation of EC50 to assess stability of the standard
• Spacing of concentrations depends upon the bioassay model
and the expected range of potencies
– Equal (log) spacing in the linear region for a Dilutional Linearity

linear model 200

– 2-dilutions in each asymptote, 4 in the

Response
150

linear dynamic region (e.g., 20% to 80%) 100

for 4- and 5-parameter logistic regression 50

0
– More dilutions/wider spacing to support a 1 10 100 1000
Concentration
wide range of potencies (note similarity)
35
 Dosing and Replication (cont.)

 Replication
– Independent replicates of concentration response curves are more
effective than repeat aliquots (pseudo replicates) at reducing
bioassay variability

1 1
1/ 1/ 1/
1/ 1/ 1/ / / One Dilution Series Three Dilution Series
1 3 6 1 2 1 1 1 1 1
2 4 8
6 2 4 2 5 n=3 repeat aloquots 1 1 1
/1 /1 /1
/1 /1 n=3 Replicates
8 6 / 1 / 1 / 1 /1 /1 /1 1 /1 2 /1
1 1 11 3 6
2 / 4 / 8 / 1/ 3/ 6/ 2 1/ 5 2/
2/ 4/ 8/ 6 1 2 3 4 6 8 21 6 52
2 4 8 6 2 4 82 65
6 2 4 8 6

• 3 or more replicates offers a basis for bioassay control

– Allows an assessment of outlying concentration response curves
– The number of replicates should be determined based upon the
contribution of replicate variability to overall bioassay variability

36
 Blocking and randomization
– Bioassay models
– Dosing and replication
– Blocking and randomization
– Optimization
– Validity criteria and outlier detection
– Transformation and weighting

 Lower variability  lower risk

• Systematic variability (bias) results in
greater risk
• Ameliorated through randomization
and blocking LSL LSL USL
USL

• Decreased random variability results

in lower risk
• Ameliorated through replication – n=4
standard error of the mean (SEM)
n=1
LSL USL

37
 Blocking and randomization

 Managing systematic variability through blocking

and randomization
– Bias may be introduced through operational factors
• Time, analyst, equipment, cage, position on plate
– Consider an experiment comparing two groups (A & B)

Analyst 1 Analyst 2 Analyst 1 Analyst 2 Analyst 1 Analyst 2

A B A A A B
A B A A B A
A B B B A B
A B B B B A

Difference between groups Block on Randomize if

Is “confounded” with analysts. run order might
difference between analysts. impact results.

38
 Blocking and Randomization (cont.)

• Bias is introduced into

bioassay measurement
Example Uniformity Profile
through operational factors
such as location effects 2500

• Uniformity testing should be

2000
performed during
development to establish if 1500 2000-2500
AFU 1500-2000
there are location effects in 1000 1000-1500
500-1000
0-500
the bioassay
500
– Cage effects in an in vivo
S5
bioassay 0
1 2 S3 rows
3 4 5
– Plate effects in an in vitro coulmns
6 7 8 9
S1

bioassay
– Effect of time from A trend is observed across columns of the plate 39

beginning to end of testing

a series of samples

39
 Blocking and Randomization (cont.)

The potential bias due to location effects can be moderated through

blocking and randomization
– A poor plate layout A1 A2 A3 A4 A5 A6 A7 A8 A9 A10
• Reference (R) and test A1
B1
A2
B2
A3
B3
A4
B4
A5
B5
A6
B6
A7
B7
A8
B8
A9
B9
A10
B10
samples (A & B) grouped B1 B2 B3 B4 B5 B6 B7 B8 B9 B10
together on the plate R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
• A plate effect is likely to
impact both series

– Strip plot design

• Randomize samples to A1 A2 A3 A4 A5 A6 A7 A8 A9 A10
R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
rows B1 B2 B3 B4 B5 B6 B7 B8 B9 B10
• Reverse dilutions in the R10 R9 R8 R7 R6 R5 R4 R3 R2 R1
A10 A9 A8 A7 A6 A5 A4 A3 A2 A1
bottom half of the plate B10 B9 B8 B7 B6 B5 B4 B3 B2 B1
• A potential plate effect is
averaged away through
blocking and randomization

40
 Managing random variability

• Strategic replication
– Replication is often more effective when applied
at the “higher levels” of replication factors
• e.g., using multiple analysts, rather than multiple
runs, plates or replicates can more effectively
reduce variability

Analyst Analyst

Run Run 1 Run 2

Plate Plate 1 Plate 2 Plate 1 Plate 2

Rep Rep 1 Rep 2 Rep 1 Rep 2 Rep 1 Rep 2 Rep 1 Rep 2

41
 Managing random variability (cont.)
2
2
s Run 2
s Plate s Re
 Replication at higher levels (cont.) SE    p

r r p r pn
– Consider placement of 5 replicates
– Where do you place your replicates
• All on the same plate (rep), across 5 plates, or across 5 runs?
(Runs, Plates, Reps)
(1,1,1)
Increased

Reps
precision of

Plates
(1,5,1)
reportable
(1,1,5)
value

Runs
SE

(5,1,1)

1 2 3 4 5
Reps

42
 Hidden source of systematic variability

 Risk of truncation error and

range Dilutional Linearity

– Retest rules have the potential to 200

lead to truncation bias

Response
150

• e.g., retest when there’s a 100

failure in “similarity” 50

– Potential solutions 0

• Assign a value to the low/high 1 10 100 1000

Concentration
result (e.g., ½ the LLOQ in
clinical assays)
• Demonstrate a range which Lot 1

supports low/high potency Lot 2

samples (without retest) Test

Lot 3 Avg

• Retest the series using an Retest

adjusted dilution scheme Lot 4

Lot 5

43
 Bioassay Optimization
– Bioassay models

• Optimization using multifactor –

–
Dosing and replication
Blocking and randomization
design of experiments (DOE) – Optimization
– Validity criteria and outlier detection
– Transformation and weighting

– Step 1 – Map the bioassay process

• Using tools such as fish-bone
diagrams

– Step 2 – Perform screening on Run Ion.St. Time Temp

1 0.1 1 36o
potentially significant factors 2
3
0.1 1
0.1 2
32o
36o
32O
• Using factorial designs 4
5
0.1 2
0.2 1 36o
6 0.2 1 32o
7 0.2 2 36oo
8 0.2 2 32

– Step 3 – Perform a response 0.1 0.2

Ionic St.

surface experiment on significant

factors to determine a region of
satisfactory performance
• “Design space” of the bioassay
44
 Bioassay Optimization
Example: Sandwich Enzyme-Immunoassay

 Goal: Improve the performance of a sandwich assay

by optimizing the incubation of the enzyme-antibody
conjugate
 Strategy:
– Perform a risk analysis using a cause and effect diagram
– Use a screening design to identify important factors and to
suggest changes in their settings which can improve assay
performance
– Minimize sample size to save time and money
– Use a design which guards against interactions among
factors

Reference: Haaland, P.D. (1989) Experiments Design in Biotechnology,

Marcel Dekker, Inc., New York, pp. 37-48.

45
 Bioassay Optimization
Factors and levels

 Identify factors and levels

to be evaluated
– Factors should be selected
on the basis of previous
experience and/or risk
analysis
• Potential factors are
identified from a “process
map” (fishbone diagram)
• Factors receive a severity
score which is the product of
impact x uncertainty
• Factors are taken into a
screening design when their
severity score exceeds some
threshold (e.g., >24)
46
 Bioassay Optimization
Factors and levels (cont.)

 Identify factors and levels to be evaluated

– Levels should be selected in order to be able to measure the impact
of the factor on bioassay response

Factor Description Levels

Time Time of incubation of sandwich 20, 30 mins
Temp Incubation temperature 27, 32 ºC
Ionic St. Ionic strength of the buffer 1, 2 nM

Choose a response variable which is related to fitness for

use
– Instrument response (OD)

47
 Bioassay Optimization
Choosing a design
 Full and fractional factorial designs
– In 2k designs the number of runs escalates dramatically with an
increase in the number of factors

Factors (k) 2 3 4 5 6 7
Runs 4 8 16 32 64 128

– 2k-p fractional factorial designs

– Increased confounding with
increased fractionation
• Consider k = 4 factors Main effects
confounded with
higher order
24-1 (Half)
interactions
8-runs
two-factor
Interactions
24 (Full) confounded with
16-runs each other

48
 Bioassay Optimization
Choosing a design (cont.)

• 23 full factorial selected for this experiment (23 = 8-runs)

– No center points (usually included to support statistical tests and
assess curvature)

49
 Bioassay Optimization
Identifying significant effects
 Initial analysis gives
Standardized effects for Immunoassay Design
estimates of effects
(main effects and
interactions)
– Table of model
coefficients shows
relative magnitude of
effects

Note: coefficient=effect/2
 Use graphical tools to
determine significant
effects

50
 Bioassay Optimization
Identifying significant effects (cont.)

 Half-normal Plots
– Probability plot of absolute
coefficients
– Random normal data (no
effects) should fall on a
straight line from 0
– Extreme points that deviate
from the line are “outliers”
• A = Time
• B = Temp
• AB = Time*Temp
• C = Ionic St.

51 51
 Bioassay Optimization
DOE Summaries (cont.)
 Main effects and interaction
plots
– Main effect plots are hard to
interpret in the presence of an
interaction
– Time and Temp combine to
increase OD reading
– Reading “robust” to changes in
Time at low Temp
– Effect due to Ionic St. is
statistically significant but may
be practically insignificant (use
equivalence approach)

52
 Bioassay Optimization
Surface Plots and Design Space

 Displays of the mathematical model

– Perform response surface DOE on important
pairs of factors (e.g., time x temp)
– Contour plot shows isobars of equal response
• Can be used to define the “design space” of
the assay

37

35.
5

34

32.1

31
.0.8 1.2 1.6 2.0 2.4

53
 Bioassay Optimization
Surface Plots and Design Space (cont.)

 The operating range for time and temperature can be

determined by inscribing a rectangle within the “design
space”
– Time should be constrained
to 27 - 30 mins (set point 28.5)
– Temp should be constrained
to 29 -32 degrees (set point 30.5)

54
 Validity Criteria
 Validity criteria should be established
on the system (assay validity) and
– Bioassay models
– Dosing and replication
– Blocking and randomization
– Optimization
– Validity criteria and outlier detection
– Transformation and weighting

on test samples (sample validity)

– Validity criteria on the assay are established to assess conformance of
the bioassay system to expected properties – failure results in repeat of
the assay
– Validity criteria on the test sample are established to assess appropriate
performance of an individual sample – failure results in repeat of the
sample

 Validity of the model used to process bioassay data is part of assay

validity - usually called “goodness-of-fit” (GOF)
– This addresses the statistical assumptions of the processing
methodology
• “Linearity” of the standard concentration response relationship
• Homogeneity of variability across concentrations
• Normality of residuals
55
 Assessing Goodness-of-Fit

Linearity
– Linearity is a frequently misunderstood term in method
development and validation
• Graphical linearity – GOF of a linear (or non-linear) model
to a set of data
– The basis of graphical linearity (GOF) should be established during
bioassay development; confirmation of analytical linearity should be
established during bioassay validation
• Analytical linearity – direct proportionality between
measured potencies of test samples and their “known”
values
– e.g., a test sample which is 2-fold more potent than another
test sample should give a measurement which is twice that of
the other sample
– Note in some cases analytical linearity is assessed as
parallelism of the test sample to the standard

56
 Assessing Goodness-of-Fit (cont.))

 Residuals of the statistical fit can be utilized to assess GOF

– Residuals (ri) of a statistical fit to data are the difference between
the observed (measured) and predicted (from the fit) responses
 A residual plot is a graphical method for detecting patterns in
residuals

ri  y obs  ypred
200

180

160

140
Response

120
12
100
10
Residual Plot
80 8

60 6

40 4

20 2

0 0

0.1 1 10 100 1000

-2
Concentration
-4

-6

-8

57
 Assessing Goodness-of-Fit (cont.))

 R2 and residual plots

– R2 is a poor measure of “goodness-of-fit” by itself
– Residual plots reveal meaningful conditions in the fit to the model
10 Random Residual Pattern 10 Nonlinear Kinetics

5 5
Residual plot Residual plot
for a linear 0 0
for non-linear
model which response
fits the data -5
R 2 =0.67
-5
R 2 =0.67

-10 -10

10 Regression Outlier 10 Heterogeneity of Variance

Residual plot 5 5
Residual plot
for data with a for data with
0 0
model outlier heterogeneous
-5 -5 variability
X R2 = 0.67

-10 -10

– Note: all the regressions have R2 = 0.67

58
 Outlier Detection
 Replication based approaches
– A replication based approach can be used to identify outliers
among repeats at a dilution
• A simple method which is common in statistical process control (SPC)
is based on the range (difference in maximum and minimum
responses) d  ˆ
x max  x min < c, where c = 2 ,
d3
Montgomery, D.C. (1991) Introduction to
Statistical Quality Control, John Wiley & d2 , d3 are constants which depend
Sons, New York
on the number of measurements.
• Similarly, a criterion can be established for the ratio of maximum and
minimum response for log normally distributed data

x max  d2  ˆ log 
< c, where c = exp 
x min  d 
 3 

• A replication based method is insensitive to outliers with a small

number of replicates and limited to aliquots
• Dixon’s and Grubb’s methods have been updated in <111>
59
 Outlier Detection (cont.)

 A model based approach uses “studentized residuals” from the model fit
to identify outliers
– Studentized residuals incorporate an estimate of the variability of
the residual

ri
rti 
s(i)  1  hi
ri
1 10 100 1000

– rti is compared to the t-distribution

• Should be performed on a model using individual fits to test(s) and the
reference standard
• The method is useful for identifying both aliquot and dilution outliers
• Note: care should be taken in the definition of “s” – the variability
associated with aliquots is different from the variability associated with
dilutions
60
 Transformation

 Assumptions associated with

statistical model fitting are –
–
Bioassay models
Dosing and replication
homogeneity of variability and – Blocking and randomization
– Optimization
normality of residuals – Validity criteria and outlier detection
– Transformation and weighting
• Bioassay responses (OD,
fluorescence units, etc.) are
frequently log-normally distributed,
characterized by higher variability at
higher response levels
250

Log Normal Distribution

200

Response
150

100

50

0
0 1 2 3 4 5 6 7 8 9 10
0.1 1 10 100 1000
Response
Potency Concentration

61
Higher variability at higher response
 Transformation (cont.)

 Failure to acknowledge this can lead to biased

estimates of mean (median) response

•Titration
• Bioassay
• CPM/OD
• SD  Mean
i.e. Constant RSD

0 1 2 3 4 5 6 7 8 9 10

Potency

- Geometric mean
- Fold (%) variability -4 -3 -2 -1 0 1 2 3 4

log Potency

62
 Transformation (cont.)

 Response transformation Log Normal Distribution

can be used to achieve

normality and homogeneity
of variability
log
0 1 2 3 4 5 6 7 8 9 10
Response
– A log transformation often
Potency

works -5 -4 -3 -2 -1 0 1 2 3 4 5
Log Response
log Potency

– A transformation which
stabilizes variability often
generates data with
acceptable normality
• Note: A log-log fit to the log
low portion of the sigmoid
response curve

63
 Transformation (cont.)

 The need for transformation 200

180

can be assessed either 160

140 Residual Plot

through inspection of the

Response
120

100 12

residual plots 80

60
10

– Pattern of increasing spread

40 6

20
4
0

in residuals with increasing 0.1 1 10

Concentration
100
2

0
1000

concentration or response -2

-4

-6

-8

 . . . or assessment of 250

Plot of Standard Deviation

replicate variability
200

versus Mean Response

Response
150

– Pattern of increasing 100

25

standard deviation among 20

Standard Deviation
50

replicates, with increasing 0

0.1 1 10 100
15

10 1000

concentration or mean Concentration

response 0
0 50 100 150 200 250

Average Response

64
 Weighting

 An alternative strategy for

addressing heterogeneity of
variability is to perform weighted
regression
Minimize SSE   w i y i  ŷ i  ,
2

– Responses associated with lower i

variability receive higher weight ad
where ŷ i  d  g
,
 x  1b

1   i  
 Choose a weighting strategy  c 
 
– Power of the mean (POM)
1
• Can also be used for transformation and w i 
• Weighting by 1/Variance is the same var( ŷ i )
as log transformation
– Note 1: be cautious of weighting by 1
POM 
y  
the estimated variance – variance
estimates from small numbers of
replicates are highly unreliable log transformationis a member
– Note 2: weight by a function of
of the POM family(  2).
response, not concentration

65
 Assessment of Similarity
 Parallel line analysis
– Parallel linear fits of test sample and standard response data
• Approximately middle region of sigmoid curves (when response is normally
distributed)
• Lower region of sigmoid curves when response is log-normally distributed (log-log
fit)
– Uses log concentration scaling (10, 20, 40, 80, 160) – equally spaced in log
concentration
– The condition of similarity is the equality of slopes (parallelism)
– The two concentration-response curves should be parallel with a horizontal
difference of M (M is the log RP)
Parallel Line Analysis Parallel Line Analysis
10 10
Parallel Response Nonparallel Response

1 1
Response

Response
RP
RP
Standard Standard
0.1 0.1

Test Test

0.01 0.01
0.1 1 10 0.1 1 10
Concentration Concentration

– There is no unique relative potency (horizontal distance) for nonparallel lines

66
 Assessment of Similarity
Which Pair is More Parallel by a difference test?
Laboratory A

0.8
Standard Data

p = 0.02 (p<0.05, i.e., 0.4

Test Data
Standard Line
Test Line
significantly different)

Log10 Response
0
0.5 1 1.5 2 2.5
Conclude nonparallel!
-0.4

Penalized for good -0.8

performance
-1.2
Log10 Concentration

Laboratory B

0.8

p = 0.08 (p>0.05, i.e., not Standard Data

Test Data

significantly different) 0.4 Standard Line

Test Line
Log10 Response

0
Conclude parallel! 0.5 1 1.5 2 2.5

-0.4

Rewarded for poor

performance -0.8

-1.2
Log10 Concentration 67
 Assessment of Similarity
Difference versus Equivalence Tests

 Consider the situation where we expect a difference of 0 in slopes

(i.e., parallelism)
 Difference testing
– Different if a confidence interval excludes 0 (P-value < 0.05)
– No evidence of a Difference if a confidence interval includes 0
(P-value  0.05) – three reasons this might be so
• There truly is no difference
• The results are highly variable, masking the difference
• Too little data was collected

-1 0 1

• Can cheat by doing less work

68
 Assessment of Similarity
Difference versus Equivalence Tests (cont.)

 Equivalence testing
– Declare a practically meaningful  (equivalence margin, or acceptance
criterion)
– Declare equivalence if the 90% confidence interval (CI) falls within ± 
• No evidence of equivalence if the CI falls outside ± 
• Note: use of a 90% CI is the same as performing two one-sided
tests (TOST in bioequivalence)

-
-1 0 +
1

• Can increase the study size (decrease width of CI) as a follow-up

– Rewarded for doing more work

69
 Assessment of Similarity
Difference versus Equivalence Tests (cont.)

 USP recommends assessing parallelism of the test and reference

sample dilution profiles utilizing an equivalence approach
– Slope are equivalent if they are adequately similar versus identical
– Parallelism is concluded if the 90% confidence interval on the difference
of slopes falls within an equivalence margin ()
• This is called a 2 one-sided tests (TOST)

a: no evidence of a difference UAL

in slopes; however, outside
acceptance limit 0

b: no evidence of a difference in slopes; N=4 N=6 N=8

LAL
but inside acceptance limit
c: a difference in slopes; however, a b c
within acceptance limit

In contrast to the F-test, equivalence testing

penalizes a poor design (e.g., n = 4) and rewards a
better design (e.g., n = 6 or n = 8)
70
 Assessment of Similarity
Difference versus Equivalence Tests (cont.)

 Parallel curve analysis

– The condition of similarity is Parallel Line Curve
Parallel Analysis using 4PL
Analysis
110%
230

equivalence of test sample and 100%

90%
180
standard concentration 80%
70%

Response
130 60%
response curve shapes RP
50%
40%
80

• concentration response curve 30%

20%
30
shape is a function of multiple 10%
0%

parameters -20
0.1 1 10 100 1000
-10%
10000
Concentration
– Upper asymptote, lower
asymptote, and slope for 4PL
– Asymptotes, slope, and
asymmetry parameter for 5PL
• One strategy is to evaluate each parameter separately
– Note issue with multiplicity – increased risk of multiple tests
• Another strategy is to evaluate an aggregate measure of the curve
parameters

71
 Assessment of Similarity
Difference versus Equivalence Tests (cont.)

 Slope ratio analysis

– Used ideally if response is normally distributed
– Use arithmetic concentration scaling (10, 20, 30, 40, 50)
– The condition of similarity is the equality of intercepts
• Note: not necessarily the origin

Slope Ratio Analysis

12
10 Standard Slope = 0.75
Test Slope = 2.37
8 RP = 3.14
Standard
6
Test
4
2
0
0 1 2 3 4 5

72
 Implementing Equivalence Testing for
Similarity
 Choose a measure of non-similarity
– In the parallel line case, could be the difference or ratio of
slopes
– For slope-ratio assays the measure of non-similarity is the
difference of y-intercepts
– For the four-parameter logistic model, similarity must be
addressed on the basis of three parameters: the slope and the
upper and lower asymptotes
• Can be based on a composite measure such as the parallelism
residual sum of squares (RSSE)*:

RSSENonparallelism = RSSEConstrained – RSSEUnconstrained

Where RSSEConstrained is the residual variability when the parameters are constrained to be equal,
RSSEUnconstrained is the residual variability when the parameters are different

*Gottschalk, P. and Dunn, J., Measuring Parallelism, Linearity, And Relative Potency In
Bioassay And Immunoassay Data, Journal of Biopharmaceutical Statistics, 15: 437–
463, 2005
73
 Implementing Equivalence Testing for
Similarity (cont.)

 Four bases for determining an equivalence margin are

discussed in USP Chapter <1032>, Development of
Biological Assays
– Approach 1: compile historical data that compare the reference to
itself, and derive a tolerance interval* for the measure of non-
similarity
• Standard statistical process control (SPC) approach
• Controls “manufacturer’s risk” (risk of failing a good assay) but not
“consumer’s risk” (risk of passing a bad assay)

* A tolerance interval is an interval containing a fixed percentage of values with

specified confidence. Thus a 95%/99% tolerance interval contains 99% of future
values with 95% confidence.

74
 Implementing Equivalence Testing for
Similarity (cont.)
– Choose an approach for defining an equivalence
margin
• Approach 2: determine a tolerance interval for the maximum
departure from similarity of the confidence interval on the
measure
• Similarity concluded if the confidence interval falls within the
interval
• Protects against passing assays with larger than usual amounts
of within-assay variation

Approach 1 Approach 2
-
Difference of Slopes
Difference of Slopes

75
 Implementing Equivalence Testing for
Similarity (cont.)
 Four bases for determining an equivalence margin are
discussed in USP Chapter <1032>, Development of
Biological Assays
– Approach 3: add data comparing standard to known failures (e.g.,
degraded samples)
• Determine a measure of non-parallelism which discriminates between
the distributions of ref/ref and ref/failure
• Note: this method can be used to determine which parameters are
sensitive to failures for nonlinear models
– Approach 4: based on what is known about the product and the
assay
• Conventional limits such as (0.80,1.25)
• Sensitivity might be driven by therapeutic index of the drug

76
 Reporting Relative Potency

Model Basis of RP Relative Potency (RP)*

Parallel line Horizontal distance RP = exp[(aT - aS)/b]

Parallel curve Ratio of inflection points RP = CS/CT

Slope Ratio Ratio of slopes RP = bT/bS

Logit/Probit Ratio of 50th percentiles RP = ED50(S)/ED50(T)

* Assumes similarity has been demonstrated.

 Note that for each model a confidence interval

on the estimated relative potency can be
calculated and used as an overall assessment of
the quality of the measurement (sample validity)

77
 •
Bioassay validation •
Bioassay development
Bioassay validation
• Bioassay maintenance

 The validation principles described

in USP <1033> apply to all dilution based
measurement systems
Four Parameter Logistic Regression

– Relative potency bioassay

200

180

160

– Single test dilution calibration 140

Unknown
Response = 112

Response
120
Observed

curve 100

80
Fitted

• Common in impurity assays

60

40 Predicted
20 [C] = 40

• Used for high throughput 0

0.1 1 10 100 1000

– in vivo methods using a single

Concentration

dilution of the test sample and the standard

• Coenraad Hendriksen, et.al., Validation of Alternative Methods
for the Potency Testing of Vaccines, ATLA 26, 747– 761, 1998
– Similarity of response of test samples and the standard must
be demonstrated to enssure accurate measurement of
potency – during validation or per run
78
 The Bioassay Validation Protocol

Should state:
– Number and types of samples
– Design including ruggedness and robustness
factors
– Validation replication strategy
– Intended validation parameters
– Justified target acceptance criteria
– Proposed data analysis plan
– Tentative run and sample validity criteria

79
 The Bioassay Validation
Protocol (cont.)
 Should state:
 Number and range of samples
 Design including ruggedness and
Number and range of samples 
robustness factors
Validation replication strategy
– Five (5) levels are usually  Intended validation parameters
 Justified target acceptance criteria
recommended  Proposed data analysis plan
Tentative run and sample validity
• To maximize the opportunity


criteria
for a wide range
• To perform regression analysis (trend in bias)
– Sample levels should be selected to bracket the range of
materials that will be tested in the bioassay
• Through a dilutional linearity experiment
– Sample levels manufactured through dilution
• A concentrated intermediate and/or through forced degradation
– Geometric scaling should be used to achieve equal
spacing in the log scale: 0.50, 0.71, 1.00, 1.41, 2.00

80
 The Bioassay Validation
Protocol (cont.)  Should state:
 Number and range of samples
 Design including ruggedness and
robustness factors

 Does the bioassay validation 


Validation replication strategy
Intended validation parameters
design have to replicate the intra-and inter-  Justified target acceptance criteria
 Proposed data analysis plan
run formulae that result in a reportable  Tentative run and sample validity
criteria
value for a test material
– Thus if 3 assays are performed to obtain a reportable value,
some believe that each validation run must include 3 assays
– There is usually limited information to understand the optimal
replication strategy prior to validation
– A strategically designed validation can identify key ruggedness
factors which might have impact on long term variability of the
bioassay
• Could lead to remedial actions such as qualification and training
programs
• Could lead to a replication strategy which more effectively addresses
significant sources of variability
81
 The Bioassay Validation Protocol (cont.)

• Design including robustness and ruggedness factors

– The validation should simulate long term variability of the bioassay
by including factors which change throughout the bioassay lifecycle
– Nested designs
• Example – 2 technicians within 2 laboratories perform 3
independent runs each
– Runs are independent if they don’t share factors which may
influence results
– If another factor may influence results (e.g., cell culture lot), this
should be “crossed” with technician
LAB 2

LAB 1 Tech 1 Tech 2

Tech 1 Run1 1 2 Run 2

Tech
Run Run 3 Run 1 Run 2 Run 3

Run 1 Run 2 Run 3 Run 1 Run 2 Run 3

82
 The Bioassay Validation Protocol (cont.)
– Crossed designs
• Factorial based validation designs including
robustness and ruggedness factors
3-hrs
– Robustness (controllable) factors
» pH/time/temperature Time
2

– Ruggedness (uncontrollable) factors Lot

2-hrs
» Techs/instruments/reagent lots 1 2
1
Analyst
– Can use “highly fractionated” designs
» The design is not being used to Reagent Incubation
identify significant factors or do Run Analyst Lot Time
1 1 1 2-hrs.
modeling (DOE), but instead as a 2 1 1 3-hrs.
structured way to introduce multiple 3 1 2 2-hrs.
4 1 2 3-hrs.
factors into the validation 5 2 1 2-hrs.
– Needn’t be factorial – more levels of 6 2 1 3-hrs.
7 2 2 2-hrs.
design factors (e.g., # analysts) gives 8 2 2 3-hrs.
better estimates of variability

83
 The Bioassay Validation Protocol (cont.)
 Should state:
 Number and range of samples
 Design including ruggedness and
robustness factors
• Validation replication strategy  Validation replication strategy
 Intended validation parameters
– Perform a sufficient number of  Justified target acceptance criteria
 Proposed data analysis plan
validation runs to control study risks  Tentative run and sample validity criteria

• Risk that a validation parameter will not meet its target acceptance
criterion when the parameter is satisfactory (Type 1 error, )

• Risk that a validation parameter will meet its target acceptance

criterion when the parameter is unsatisfactory (Type 2 error, )

( t ,n 1  t  / 2,n 1)2  s 2

n ,
ac 2
where t  , t   cons tan ts associated with validationrisks , ,
s  t arg et bioassay precision,
ac  t arg et acceptancecriterion

84
 The Bioassay Validation Protocol (cont.)
 Intended validation parameters  Should state:
Number and range of samples
– Relative accuracy (relative bias) –


 Design including ruggedness and robustness

factors
using a dilutional linearity experiment  Validation replication strategy
 Intended validation parameters
 Measured Potency  Justified target acceptance criteria
Relative Bias  100    1%


Proposed data analysis plan

 Target Potency 


 Tentative run and sample validity criteria

– Specificity/selectivity – using blanks and potential like materials;

process intermediates and modified samples
• Lack of response in blanks or potential like materials
• Similarity (parallelism) in process intermediates and modified
samples

– Intermediate precision
• Reported as percent geometric CV (%GCV)
» Note: intermediate precision relates to a single intra-run replicate in a
single assay
» The variability of the reportable value is format variability

Intermediate Precision 100   e  1 %

ˆ Inter
2
- assay 
2
ˆ Intra - assay

  85
 The Bioassay Validation Protocol (cont.)
 The bioassay yields log-normally distributed relative potencies (RP)
– log relative potency is equal to the Parallel Line Analysis
10
horizontal shift in log concentration (M) Parallel Response

– Relative potency is eM 1

Response
RP
(a geometric mean or GM) 0.1
Standard

Test

0.01
0.1 1 10
Concentration

Measures of variability for geometric means were defined by Kirkwood*

– The geometric standard deviation is defined as GSD = 𝑒 𝑠𝑙𝑛
– GSD is a multiplicative factor such that the range M  s is directly related to
𝑒 𝑀±𝑠𝑙𝑛 = 𝐺𝑀 ÷ 𝑒 𝑠𝑙𝑛 , 𝐺𝑀 ∙ 𝑒 𝑠𝑙𝑛 ; i.e., GM divided and multiplied by GSD
– Geometric coefficient of variation is defined as
𝐺𝐶𝑉 = 100 ∙ 𝐺𝑆𝐷 − 1 = 100 ∙ 𝑒 𝑠𝑙𝑛 − 1
– Different measures of relative variability (%CV, %GCV, etc.) are defined by Tan**
* Kirkwood, T.B., Geometric Mean and Measures of Dispersion, Biometrics, December 1979
** Tan, C.T., RSD and Other Variability Measures of the Lognormal Distribution, PF 31(2), Mar-Apr 2005
86
 The Bioassay Validation Protocol (cont.)
 Should state:
 Justified target acceptance criteria  Number and range of samples
 Design including ruggedness and
 The goal of the validation is to demonstrate robustness factors
that the bioassay is “fit for use”  Validation replication strategy
 Intended validation parameters
 Justification may be based on assurance  Justified target acceptance criteria
of acceptable process capability  Proposed data analysis plan
Tentative run and sample validity
 This might be assessed by showing that 

criteria
the performance characteristics (relative
bias and intermediate precision) support Cpk=1.0
Cpm=1.0
Prob(OOS) = 0.0027 (~0.3%)
acceptable process capability Prob(OOS)=0.0027 (~0.3%)

 Cpm is a process capability index which

relates to risk of an out-of-specification (OOS)
result
Prob(OOS) is a producer’s risk
-3-sigma +3-sigma

LSL USL
Upper SpecificationLimit Lower SpecificationLimit
Cpm  Pr ob(OOS)  2  Pr ob( z  3  Cpm),
2
6  ˆ Pr  RB 2  Re
2
oduct lease where z is a standard normal variate.
2
where ˆ Pr is an estimate of product variability, RB is relativebias,
oduct e.g. 2  Prob(z  -3  1.0)  0.002 (0.2%).
2
and Re lease
is release assay variability.

87
 The Bioassay Validation Protocol (cont.)

 Satisfactory process capability should be coupled with

assurance that the product will retain adequate quality
throughout shelf-life
 This includes developing release limits (e.g., potency
limits)
Upper limit

MinimumRelease
Capability
Limits
Release
Limits
Specifications  Expiry Limit Loss  CombinedUncertainty

 Spec  b̂  t  s 2  sRelease
2
b̂

Expiry Limit Shelf-Life

-6 0 6 12 18 24 30 36

 Based on this, process capability (and associated probability of an

OOS result) is a business decision, not a regulatory requirement

88
 The Bioassay Validation Protocol (cont.)
 Should state:
 Number and range of
samples
 Proposed data analysis plan  Design including
ruggedness and robustness
– The protocol should include a description factors
of the statistical analyses which will be  Validation replication
strategy
performed  Intended validation

• Emphasis on estimation, not statistical 

parameters
Justified target acceptance
testing criteria
 Proposed data analysis plan
» Confidence interval on relative bias  Tentative run and sample
validity criteria
» Variance components on design factors

– Use of mixed effects models to account for multiple design

factors
• Fixed effects such as potency level
• Random effects such as operator, media lot, assay, and
replicate
• Mixed effects modeling requires special statistical software

89
 The Bioassay Validation Protocol (cont.)
 Should state:
 Number and range of samples
• “Tentative” run and sample  Design including ruggedness and
robustness factors
validity criteria  Validation replication strategy
 Intended validation parameters
– Assay and sample validity such  Justified target acceptance criteria
Proposed data analysis plan
as criterion on reference slope to control 

 Tentative run and sample validity

assay, similarity to control sample criteria

– Pre-specified outlier rules, with actions

• Drop replicate versus retest sample, eliminate point, etc.
• Note: dropping a replicate may induce bias
– “Extra-variability” criteria on sample replicates

 d2  ˆ log 
e.g., x max / x min < c, where c = exp  
 d 3 

– Statistical process control (SPC) on control samples

– Note: validation runs should be evaluated against these rules; however,
marginal failures may indicate that the rules are not robust to long term
factors and should be modified
• A plan to modify the rules with validation data should be described in the
validation protocol
90
 A Bioassay Validation Example
Determination of acceptance criteria

• Determination of target acceptance criteria on intermediate precision

(IP) and relative bias (RB)
– Target specification for potency: 0.71 – 1.41 (geometrically symmetrical
around 1.00)
ln(1.41) - ln(0.71)
Cpm   0.94
IP (%CV) RB Cpm Prob(OOS) 6  [ln(1.08)] /3  [ln(1.12)]
2 2

20% 20% 0.54 10.5%

2
assuming σ̂Product  0 and the release assay
8% 12% 0.94 0.48% is performed in 3 runs.
10% 5% 1.55 0.03%
Prob(OOS)  2  Φ( 3  0.94)  0.0048(0.48%)

– Target acceptance criteria of 8 %CV for intermediate precision (IP) and

12% for relative bias (RB) yield acceptable process capability (~0.48%
chance of OOS when the measurement is centered on the true value)

91
 A Bioassay Validation Example
Study size

 Determination of study size

– A sample size calculation says n = 8 validation runs are necessary to
satisfy the target acceptance criterion for RB

( t ,n 1  t  / 2,n 1)2  s2 (1.89  2.36)2  log(1.08)2

n  8
2 2
ac log(1.12)

– log(1.08)2 is the variance (s2) corresponding to 8 %GCV

– log(1.12)2 is the acceptance criterion (ac2) corresponding to 12%
relative bias
– Since “n” appears on both sides of the equation, a solution is derived
iteratively (can be obtained using SOLVER in EXCEL)
• The iterative solution to t,n-1 and t/2,n-1 is1.89 and 2.36
respectively

92
 A Bioassay Validation Example
Study design

• The 8-runs can be arranged using nesting and factorials strategies in

several ways:
• 2-factors: duplicate runs nested in a 22 factorial design = 8-runs
• 3-factors: a full 23 factorial design in 3-factors = 8-runs
• 4-factors: a ½-fraction of a 24 factorial in 4-factors = 8-runs
• 5-factors: a ¼-fraction of a 25 factorial in 5-factors = 8-runs
– This is highly confounded; however, the goal is estimation of IP, not
determination of significant factors

No. Factors Design Resolution

3 23 Full
4 24-1 IV
5 25-2 III

93
 A Bioassay Validation Example
Study design (cont.)

 Example study design

– 2x2 factorial experiment using two analysts and

two media lots

– Each combination performed in duplicate (8-runs)

– Duplicate titrations of the test preparation

alongside a single titration of the reference (5-
levels x 8-runs x 2-reps = 80 RP measurements)

– Five-level “dilutional linearity” study

94
 A Bioassay Validation Example
Study results
 Data and plot of validation results
Media Design dimensions:
Lot/Analyst 1/1 1/2 2/1 2/2
Run 1 2 1 2 1 2 1 2 • 2 media lots and two
0.50 0.5215 0.4532 0.5667 0.5054 0.5222 0.5179 0.5314 0.5112 analysts
0.50 0.5026 0.4497 0.5581 0.5350 0.5017 0.5077 0.5411 0.5488
0.71 0.7558 0.6689 0.6843 0.7050 0.6991 0.7463 0.6928 0.7400 • 2 runs by each analyst
0.71 0.7082 0.6182 0.8217 0.7143 0.6421 0.6877 0.7688 0.7399 using each media lot
1.00 1.1052 0.9774 1.1527 0.9901 1.0890 1.0314 1.1459 1.0273
1.00 1.1551 0.8774 1.1074 1.0391 0.9233 1.0318 1.1184 1.0730
1.41 1.5220 1.2811 1.5262 1.4476 1.4199 1.3471 1.4662 1.5035
1.41 1.5164 1.3285 1.5584 1.4184 1.4025 1.4255 1.5495 1.5422
2.00 2.3529 1.8883 2.3501 2.2906 2.2402 2.1364 2.3711 2.0420
2.00 2.2307 1.9813 2.4013 2.1725 2.0966 2.1497 2.1708 2.3126 Bioassay Validation Results

Note: Data should be reported to a sufficient 2.00

Observed Potency
number of significant digits to support 1.41

statistical calculations
Tech 1
1.00
Tech 2
0.71

Rounding the data will incur error which 0.50

could compromise the validation 0.50 0.71 1.00 1.41 2.00

assessments Expected Potency

95
 A Bioassay Validation Example
Components of random variability
 Many factors influence the bioassay compounding the overall
variability
Inter -Run Variability Intra -Run Variability Overall Variability

+ =

 Appropriate analysis (Variance Component Analysis) can be utilized

to decompose the overall variability into its component parts
– Reveals significant assay factors which might lead to corrective
action
• Such as improved analyst training or more rigorous reagent
qualification
– Provides a resource for exploring alternative assay formats, to
improve efficiency and bioassay precision
96
 A Bioassay Validation Example
Components of random variability (cont.)

 Impact of not understanding key sources of variability

– The true variability is greater than expected, leading to increased risks
• Risk of accepting batches with unacceptable potencies
• Risk of failing batches with acceptable potencies
– Assessments using the bioassay may be complicated
• Stability assessment
• Long term variability is greater than short term variability
• Can not analyze data
as “independent”
measurements”
• Average at each time
Potency

point should be
analyzed (or mixed
effects model)
5.9
0 6 12 18 24 30 36
Time (Month)

97
 A Bioassay Validation Example
Determination of intermediate precision
• Assessment of intermediate precision
– Calculation illustrated at the 0.50 level using analysis of variance (ANOVA)
• Note: the analysis is performed on ln(RP)
– For a “balanced” design like this example (equal number of replicates at all levels), the
estimated variance components may be obtained by equating the “Mean Square” with
the “Expected Mean Square”
– Intermediate precision (IP) is calculated from the sum of the variance component
estimates
Sum of Mean Expected Mean
Source df Squares Square Square
Run 7 0.055317 0.007902 Var(Error) + 2Var(Run)
Error 8 0.006130 0.000766 Var(Error)
Corrected Total 15 0.061447

Variance Component Estimates

Var(Run) = 0.003568 MS (Run)  Var (Error)  2  Var (Run)
Var(Error) = 0.000766
MS (Run)  MS (Error)
IP  100   e Var (Run)  Var (Error )
 1% Var (Run) 
2
 
 100   e 0.003568  0.000766
 1%  6.8%
  0.007902 0.000766
  0.003568
2
98
 A Bioassay Validation Example
Determination of intermediate precision (cont.)
• Assessment of intermediate precision
– Intermediate precision across validation sample levels
– A heuristic can be used to identify inconsistencies in variance
component (VC) estimates across levels
• Ratio of max to min VC estimates should be less than 10-fold

Level
Component 0.50 0.71 1.00 1.41 2.00 Average
Var(Run) 0.00357 0.00065 0.00364 0.00314 0.00262 0.00272 0.00364/0.00065 = 5.6 < 10
Var(Error) 0.00076 0.0043 0.00295 0.00058 0.00226 0.00217 0.00430/0.00058 = 7.4 < 10
Overall 6.8% 7.3% 8.5% 6.3% 7.2% 7.2%
1

– The average of the VC’s in the range of acceptable individual level

performance can be used to get the bioassay IP if the study is
balanced
• Use of restricted maximum likelihood (REML) estimation is preferred for
more complex designs or when data are missing – consult a statistician

99
 A Bioassay Validation Example
Determination of intermediate precision (cont.)
• Assessment of intermediate
precision  Var (Run)  Var (Error ) 
IP  100   e  1%
– A variance component analysis  
 
using restricted maximum  100   e 0.00272  0.00217  1%  7.2%
 
likelihood estimation (REML) can
be performed including analyst
Component
and media lot as factors
Variance Estimate
• The VC analysis ignoring these Var(Media Lot) 0.0000
factors will always underestimate
Var(Analyst) 0.0014
IP when one or more of the
factors is significant (ex., 7.2% vs. Var(Analyst*Media LOt) 0.0000
7.7%) Var(Run(Analyst*Media Lot)) 0.0019
– The analysis reveals that analyst Var(Error) 0.0022
has a significant impact on long
 
IP  100   e  i  1%
VC
term variability
 
• This indicates that improved
 
training (or use of multiple  100 e 0.0014  0.0019  0.0022  1%  7.7%
analysts) may improve bioassay  
precision

100
 A Bioassay Validation Example
Determination of intermediate precision (cont.)

 Intermediate precision
– The estimate of intermediate precision is subject to uncertainty
and has an associated confidence interval
Upper Bound on 10% GSD 11.8%
140%
120%
100%
80%
%GSD

60%
40%
20%
0%
0 4 8 12 16 20 24

Number of Runs (n)

– The determination of a confidence bound on a function of

variance components is complex, and beyond the scope of
<1033>
– The upper one-sided 95% confidence bound on the estimated
%GCV (7.2%) is equal to 11.8%

101
 A Bioassay Validation Example
Bioassay characterization

 Bioassay characterization
– The validation may be used to inform the laboratory of a
suitable testing format
• Variance component estimates can be used to forecast
variability for different numbers of intra-run and inter-run
replicates
 σ̂Run
2
σ̂
2

  Repeat
nk  Component
Format Variability  100   e k
 1 % Variance Estimate
 
  Var(Media Lot) 0.0000
Var(Analyst) 0.0014
Var(Analyst*Media LOt) 0.0000
Format variability for different combinations of number of
runs (k) and number of minimal sets within run (n) Var(Run(Analyst*Media Lot)) 0.0019
Var(Error) 0.0022
Number of Runs (k)
Reps (n) 1 2 3 6
Note: The lab might replicate
1 7.2% 5.1% 4.1% 2.9%
over significant factors to
2 6.4% 4.5% 3.6% 2.6%
benefit from variance
3 6.0% 4.2% 3.4% 2.4% reduction
6 5.7% 4.0% 3.3% 2.3%
102
 A Bioassay Validation Example
Determination of relative accuracy

• Assessment of relative accuracy

– Estimated relative potency and relative bias is calculated at each
level
Media Lot/Analyst  The average of
Level 1/1 1/1 1/2 1/2 2/1 2/1 2/2 2/2 repeats in each run
0.50 0.5215 0.4532 0.5667 0.5054 0.5222 0.5179 0.5314 0.5112 must be calculated if
0.50 0.5026 0.4497 0.5581 0.5350 0.5017 0.5077 0.5411 0.5488 more a sophisticated
0.71 0.7558 0.6689 0.6843 0.7050 0.6991 0.7463 0.6928 0.7400 analysis (i.e., mixed
0.71 0.7082 0.6182 0.8217 0.7143 0.6421 0.6877 0.7688 0.7399 effects analysis) is
1.00 1.1052 0.9774 1.1527 0.9901 1.0890 1.0314 1.1459 1.0273 not performed
1.00 1.1551 0.8774 1.1074 1.0391 0.9233 1.0318 1.1184 1.0730
 Note: requires a
GM 1.1299 0.9261 1.1299 1.0143 1.0027 1.0316 1.1321 1.0499 balanced dataset
1.41 1.5220 1.2811 1.5262 1.4476 1.4199 1.3471 1.4662 1.5035
1.41 1.5164 1.3285 1.5584 1.4184 1.4025 1.4255 1.5495 1.5422
GM  1.0273 1.0730  1.0499
2.00 2.3529 1.8883 2.3501 2.2906 2.2402 2.1364 2.3711 2.0420
2.00 2.2307 1.9813 2.4013 2.1725 2.0966 2.1497 2.1708 2.3126

103
 A Bioassay Validation Example
Determination of relative accuracy (cont.)
 Relative accuracy
– The estimated relative potency and relative bias (RB) is calculated at each
level, together with the 90% confidence interval (2 one-sided test)

RP log RP
GM  e Av erage  e0.0485  1.050
1.1299 0.1221
 GM   1.050 
RB  100    1  100    1%  5.0%,
0.9261 -0.0768  Target   1.00 
1.1299 0.1221
1.0143 0.0142
CIln  x  t 7  s / n
1.0027 0.0027
 0.0485  1.89  0.0715 / 8
1.0316 0.0311  (0.0006,0.0964)
1.1321 0.1241
 e0.0006   e0.09643 
1.0499 0.0487 CIRB  100    1,100    1  (0.06%,10.12%)
 1.00   1.00 
Average 0.0485    

SD 0.0715

104
 A Bioassay Validation Example
Determination of relative accuracy (cont.)
 Relative accuracy
– Estimated relative potency and relative bias is calculated at each
level
Table 7. Average Potency and Relative Bias at Individual Levels
log Potency Potency Relative Bias
Level na Average (90% CI) Average (90% CI) Average (90% CI)
0.50 8 -0.6613 (-0.7034,-0.6192) 0.52 (0.49,0.54) 3.23% (-1.02,7.67)
0.71 8 -0.3419 (-0.3773,-0.3064) 0.71 (0.69,0.74) 0.06% (-3.42,3.67)
1.00b 8 0.0485 (0.0006,0.0964) 1.05 (1.00,1.10) 4.97% (0.06,10.12)
1.41 8 0.3723 (0.3331,0.4115) 1.45 (1.40,1.51) 2.91% (-1.04,7.03)
2.00 8 0.7859 (0.7449,0.8269) 2.19 (2.11,2.29) 9.72% (5.31,14.32)

12% The acceptance criterion

Lower limit is on relative bias (12%) is
–(1-1/1.12)
=-11% 0% met at all but the highest
level.
-11%
0.50 0.71 1.00 1.41 2.00

105
 A Bioassay Validation Example
Determination of relative accuracy (cont.)

CombinedRisk  1  (1  )k
Impact of multiplicity
Multiplicity Risk
– Increased risk with multiple 0.45
statistical tests 0.40 0.05
0.01
– Also increased risk from
0.35
0.30 0.001
correlated results (levels 0.25

Risk
performed in same 0.20
0.15
validation runs) 0.10
– Can be managed through 0.05
0.00
an analysis across levels 1 2 3 4 5 6 7 8 9 10
No. Tests (k)

106
 A Bioassay Validation Example
Determination of relative accuracy (cont.)

 Trend across levels

– “Relative accuracy in bioassay refers to a unit slope
between log measured relative potency vs. log level when
levels are known.”
• Lack of conformance to a
line with unit slope (log-
log) reveals potential
Bioassay Validation Results problem with comparing
results in the bioassay
2.00
(e.g., stability)
Observed Potency

1.41

1.00

0.71
Analyst 1
Analyst 2 
Trend in Bias  100  2b-1  1 %
(bias per 2 - fold difference in activity)
0.50

0.50 0.71 1.00 1.41 2.00 • An acceptance criterion

Expected Potency
can be placed on the trend
in bias; if this is met, the
bias can be averaged
across levels 107
 A Bioassay Validation Example
Determination of relative accuracy (cont.)

 Average relative accuracy across levels

– If a unit slope is verified, data can be pooled across levels to
estimate the overall bias of the bioassay
• This increases the power to establish conformance to the
validation acceptance criterion
» The combined data across levels decreases the size of the 90%
confidence interval, thus helping to assure this will fall within the
acceptance criterion for relative
• This generally requires sophisticated statistical analysis of the
data
– Multiplicity is controlled because 5 statistical tests (5 levels)
has been reduced to 2 tests
• Trend in bias
• Overall bias

108
 Bioassay Maintenance
• Bioassay development
• Bioassay validation
• Bioassay maintenance

 Is the validation failed if the acceptance criterion for

intermediate precision is not met?
– The assessment of variability is complicated by the small
size of the validation study
– An alternative approach is to use the validation results to
reformat the bioassay to have acceptable variability for its
various uses (a QbD approach)
– The validation results should be verified with occasional
assessments of QC data to confirm the true long term
variability of the bioassay

109
 Bioassay Maintenance (cont.)

 The validation results should be

verified/updated using additional
resources to better understand the
true long term variability of the
bioassay
– Variability of a control sample
• Control limits should be
established in conformance with
the intermediate precision of the
bioassay
• The variability of the control
sample can be re-calculated on a
periodic basis to improve
confidence (reduce uncertainty)
in estimate of intermediate
precision

%GCV = 7.9% 110

 Bioassay Maintenance (cont.)

 The validation results should be 4.2

Regression
verified/updated using additional 4

resources to better understand the 3.8

3.6
true long term variability of the 3.4

bioassay 3.2

• The residual variability from 3

stability analysis 2.8

0 3 6 9 12 15 18 21 24

• The residual variability represents Time (Month)

long term assay variability, after

0.4
the trend has been taken out of Residuals
0.3
the stability data
0.2
• Also an indication of goodness-of- 0.1

fit of the stability model 0

• Note: short-term and long-term -0.1

-0.2
variability can be estimated from
-0.3
the data (nested errors)
-0.4
Time (Month)

Short term and long term variability 111

 Bioassay Maintenance (cont.)

 Major events in the bioassay the lifecycle should be

qualified to help assure bioassay validity
– Method transfer
– New key reagent
– Method change
– Change in cells
– Using an equivalence test approach
• Declare a practically meaningful  (“customer” limits)
• Declare no change if 90% CI falls within ±

- 0 +

112
 Uses of bioassay

 A QbD approach to bioassay development and validation

facilitates utilizing the bioassay as a key component of product
development and control strategy
– Fundamental properties of the bioassay should be established
during early development
• Specificity, selectivity, linearity (similarity)
• Development experiments should be designed to address
operational factors which could compromise development
decisions
• The bioassay design should be coupled with development study
design to reduce uncertainty and thereby decision risks
• Strategic testing of development study samples should be
utilized to avoid bias and minimize variability
• Blocking and randomization
• Study samples needn’t be tested by the release assay protocol

113
 Uses of bioassay (cont.)

 A QbD approach to bioassay development and validation

facilitates utilizing the bioassay as a key instrument during
product development and of the final product control strategy
– Traditional validation is focused on a single use of the bioassay –
product release
– Once the bioassay has been developed, studies should be
performed to characterize the variability of the bioassay in order to
build protocols for commercial product release and control
– From this quality can be “built into the bioassay” through an
appropriate bioassay testing format, coupled with the design of other
factors associated with the study
– Some uses of bioassay in the product control strategy
• Release of product
• Stability studies
• Standard qualification/calibration
• Process/product comparability

114
 Uses of bioassay (cont.)

 Should stability testing follow the same format as

release testing?
– The design of stability studies should address the goal of
stability assessment; this is to describe the decay kinetics
rather than demonstrate conformance to the release (or
expiry) specification
– The most effective design for stability assessment is to
reallocate replicates across time rather than within
individual stability intervals

115
 Uses of bioassay (cont.)

 Stability study design and analysis

– The goal(s) of stability studies are different from that of
product release
• To establish product shelf-life
• To determine a minimum release requirement
• To monitor commercial product stability
– Stability study design should address the specific goal of
the study
– The stability study can be designed to reduce the
uncertainty associated with the result associated with the
goal of the study

116
 Uses of bioassay (cont.)

 Stability monitoring
• The risk of a stability OOS during stability monitoring is
reduced through appropriate statistical modeling
• The estimated regression model gives a prediction of the
mean potency over time
• The uncertainty (confidence interval) associated with
individual measurements is greater than that on the
regression model
Regression versus Individual Stability Time Point
Potency

0 6 12 18 24 30

Time (Months)

117
 Uses of bioassay (cont.)

 Standard qualification/calibration
– A standard should be representative of material being
tested in the assay
• Usually a routine production lot
– Laboratories introduce new working standards using one
of two approaches
• Qualification – a demonstration of “equivalence” of the
working standard and the comparator (master standard or
the previous working standard)
• Calibration – assignment of potency to the new working
standard using the comparator

118
 Uses of bioassay (cont.)
 Standard qualification/calibration
• Qualification/calibration to a previous working standard
suffers from “standard drift” and serial uncertainty
(propagation of errors)
• This is controlled through use of a master (primary)
reference standard
Serial
Calibration to a Calibration
Master Standard
Master
Master 2WS1
 2
WS1
2WS3 WS1
 2
WS 2
2WS1  2WS2

WS1 WS2 WS3 WS2

2WS1  2WS2  2WS3
WS3 119
 Summary

 Bioassay is a key tool in ensuring the potency of

biological products
 The USP chapters offer guidance which is useful to
scientists and statisticians involved in the
development, validation, and analysis of bioassays
 The bioassay should be fit for its intended uses
throughout development as well as for commercial
product control
 Statistical approaches for eliminating bias and
reducing variability help ensure reliable potency
measurement and thereby sound decisions

120
Considerations in calibration assays
 Considerations in calibration assays
A Quality by Design approach

 Calibration design response variables for a calibration curve

can be derived from the precision profile
– Interpolated response is more variable on the flat portions of the
standard curve
– The precision profile is usually asymmetric, reflecting higher
variability at higher response levels (unless transformed)

250 40
35
200
30
Response

150 25

%CV
20
100 15
10
50
5
0 0
1 10 100 1000

Concentration
122
 Considerations in calibration assays
A Quality by Design approach (cont.)

Regulatory docs (FDA Guidelines

on Bioanalytical Method
Validation) require precision
(CQA) better than 30%

Requirements for other uses

should be matched to fitness
for use

LLOQ and ULOQ are derived

from this requirement
(specification)
 Dosing range [7.25-450]
 Ave.Precision 7.60% Bruno Boulanger, Arlenda, Quality by Design and
Design Space for LBAs, 2011
123
 Considerations in calibration assays
A Quality by Design approach (cont.)

 If you repeat the experiments under the same conditions, results will
vary run-to-run.
 The range [7.25-450] may not be obtained for every run.
 Noise in the signal and noise in the operating conditions will make
the performance vary

It is important to ensure
adequate performance is
achieved during future
runs.

124
 Considerations in calibration assays
A Quality by Design approach (cont.)

Precision profiles of 20 routine

runs.

→ Dosing range [7.25-450]

Note:
LLOQ obtained in 10/20 runs
ULOQ obtained in17/20 runs

Range [10,450] in 90% of runs

Range [7,450] in 50% of runs

125
 Considerations in calibration assays
A Quality by Design approach (cont.)

Vary operating conditions to flatten the precision profile

126
 Considerations in calibration assays
A Quality by Design approach (cont.)

 DOE can be utilized to rapidly optimize assays

 1st a screening design
– to identify most influential factors
 2nd a Response surface Design
– to optimize and make the assay robust
 Need first to identify the response of interest to
optimize

127
127
 Considerations in calibration assays
A Quality by Design approach (cont.)

 Response variables for a standard curve can be derived from

the precision profile
250 40

200
35
30
• LQL/UQL – lower/upper
Response

150 25
doses(concn’s)where
%CV

20
100 15

50
10
5
precision profile intersects
0
satisfactory %CV
0.20

0
1 10 100 1000

Concentration

• WR – working range =
log(UQL/LQL)
0.15

Working Range
• CVa – average CV across
0.10

CV working range
• Precision area – PA=
0.05

Precision Area WRx(20%-CVa)

CVa
• Note: working range
0.00

LQL UQL
10 100 1000 10000
impacted by weighting
Biomarker LevelConcentration
Standard (Standard Material)

128
 Considerations in calibration assays
A Quality by Design approach (cont.)

 Key factors identified using risk analysis

(potential CPPs):
– Amount Capture Antibody [250-750]
– Amount of Biotin [250-600]
– Amount Enzyme [300-750]
– Volume [50-100]
– Incubation Time [1-3]
– NB Cleanings [1-4]

 Response variable – Average %CV

129
129
 Considerations in calibration assays
A Quality by Design approach (cont.)

Exp. Capt. Biot. EnzCult Vol. Incub. NBCl

1 250 250 750 100 1 1
2
3
250
250
600
600
300
750
100
50
1
3
4
1  Use of DOE
4
5
750
250
600
600
300
750
50
50
1
3
1
1
– Screening Design
6 750 600 300 50 1 1 (Plackett & Burman).
7 750 250 300 100 3 1
8 250 250 750 100 1 1 – Allows the lab to indentify
9 750 600 750 100 3 4
10 750 250 300 100 3 1 the 2 most important
11 250 250 300 50 3 4 factors
12 750 250 750 50 1 4
13 750 600 750 100 3 4 – Pareto principle – 80% of
14 250 600 300 100 1 4
15 250 250 300 50 3 4 the variation is due to
16 750 250 750 50 1 4
20% of the factors

130
 Considerations in calibration assays
A Quality by Design approach (cont.)
Exp. Capt. Biot. EnzCult Vol. Incub. NBCl
1 250 250 750 100 1 1
2 250 600 300 100 1 4
3 250 600 750 50 3 1
4 750 600 300 50 1 1
5 250 600 750 50 3 1
6 750 600 300 50 1 1

Results from the DOE 7

8
9
750
250
750
250
250
600
300
750
750
100
100
100
3
1
3
1
1
4
10 750 250 300 100 3 1
11 250 250 300 50 3 4
12 750 250 750 50 1 4
13 750 600 750 100 3 4
14 250 600 300 100 1 4
15 250 250 300 50 3 4
16 750 250 750 50 1 4

131
Considerations in calibration assays
A Quality by Design approach (cont.)

Estimated factor effects

132
 Considerations in calibration assays
A Quality by Design approach (cont.)

Optimal conditions

– Capture Antibody 250

– Biotin 600
– Amount Enzyme 300
– Volume 100
– Incubation Time 1
– NB Cleanings 4

133
 Considerations in calibration assays
A Quality by Design approach (cont.)

Given performance at optimal

settings, there is little chance the
acceptance criterion will not be
met in the future.

However :
- Operating conditions vary run-to-
run, and thus performance will
vary run-to-run.

- A range of operating conditions

can be determined which ensures
adequate performance There is room left for the
factors to vary while ensuring
(specifications) will be met. adequate performance

- This is the Design Space of the

assay
134
 Considerations in calibration assays
A Quality by Design approach (cont.)

Allowing each factor to vary 25-50

does not assure adequate
performance over time
– Capture Antibody [250-275]
– Biotin [550-600]
– Amount Enzyme [275-300]
– Volume [75-100]

 Then what is the largest space

that will assure adequate
performance?

135
135
 Considerations in calibration assays
A Quality by Design approach (cont.)

This Design Space should ensure

adequate performance will be met – Capture Antibody 250
–
in a high percentage of the runs –
Biotin
Amount Enzyme 300
600

–
e.g., 90% of runs –
Volume
Incubation Time 1
100

– NB Cleanings 4

Design Space

– Capture Antibody [250-265]

– Biotin [585-600]
– Amount Enzyme [275-300]
– Volume [90-100]

136
 Summary

 Quality by design approaches using method

attributes and method parameters can be used to
optimize assay performance
 Key to the approach is selecting an attribute or
attributes directly related to use of the bioassay
 The design space for the bioassay should
acknowledge the ranges in method parameters which
yield “acceptable results”
 The definition of acceptable results is critical to the
QbD approach – fitness for use

137
 Acknowledgements

Nancy Sajjadi
Bruno Boulanger
USP Bioassay Expert Panel

138