CLINICAL

Article TRIALS

Clinical Trials
2016, Vol. 13(3) 338–343
Ó The Author(s) 2016
Remarks on designs enriching for Reprints and permissions:
sagepub.co.uk/journalsPermissions.nav
placebo non-responders DOI: 10.1177/1740774515625186
ctj.sagepub.com

Gerd K Rosenkranz

Abstract
Background: High response under placebo constitutes a concern in clinical studies, particularly in psychiatry.
Discontinuation of placebo responders identified during a placebo run-in is often recommended to avoid failures of clini-
cal trials in the presence of high placebo effects. Evidence for the benefit of this approach is ambiguous.
Purpose: We investigate under which conditions a placebo lead-in can be beneficial in the context of continuous data,
assuming that the data in the placebo run-in and the treatment stage follow a bivariate normal distribution. Placebo
responders are defined as patients with an effect during placebo lead-in which is larger than a pre-defined threshold on
the absolute value or the absolute or relative change from baseline or a combination thereof.
Results: Data are less variable under either placebo or test treatment after placebo responders have been removed.
Whether the effect of test over placebo increases or decreases after enrichment for placebo non-responders depends
on the parameters of the distribution, in particular the covariance structure, and the threshold in the definition of pla-
cebo responders.
Limitations: The results apply in the continuous case, and the binary or ordinary case is not studied. The findings
explain to some extent the ambiguity in the assessments of the usefulness of placebo lead-in periods in clinical trials;
however, besides the clear statement on variability reduction, it is not straightforward to judge upfront whether placebo
lead-in is useful. Concerns relating to the conduct and interpretation of results of such trials are mentioned.

Keywords
Placebo responder, placebo lead-in, enrichment, sequential parallel comparison design

Introduction the other study. A re-examination of another antide-

Placebo response is a challenge for any clinical trial, in
particular in specific indications like psychiatry. In
depression trials, the placebo response is substantial,
variable and has increased in recent years.1–3 A widely
applied countermeasure is to add a placebo lead-in (or
run-in) period and remove all subjects that qualify as
placebo responders before randomization. However,
this approach did not decrease post-randomization pla-
cebo response, nor did it increase drug–placebo differ-
ences as elaborated in a meta-analysis.4 Alternatively, a
double blind variable placebo lead-in has been pro-
posed that seems to be more successful in enriching the
population with placebo non-responders.5 Not only are
subjects and investigators blinded to the length of the
placebo lead-in and the start of the test drug but also
all patients continue in the study, while only the pla-
cebo lead-in non-responders are included in the analy-
sis. The authors report that this approach resulted in
an increased drug–placebo difference in one of two
studies but had no effect on the treatment difference in
pressant trial led to the conclusion that the removal of
single blind placebo lead-in responders may have actu-
ally diminished drug–placebo differences.6
A systematic review from the Food and Drug
Administration (FDA) database including 86 trials in
major depressive disorder revealed that 30 trials did
not have a placebo lead-in period, while 56 had.7 The
average placebo response in the first group was
9:24 (SD = 1:87) in Hamilton Rating Scale for
Depression total score. Eight trials of the second group
set absolute thresholds for one or more standard rating
scales at the beginning and the end of the placebo lead-
in phase, while 48 applied both absolute and percent
change criteria (e.g. Hamilton Rating Scale total \ 18
Novartis Pharma AG, Basel, Switzerland
Corresponding author:
Gerd K Rosenkranz, Novartis Pharma AG, Novartis Campus, CH-4051
Basel, Switzerland.
Email: gerd.rosenkranz@novartis.com

Downloaded from ctj.sagepub.com at CAMBRIDGE UNIV LIBRARY on June 5, 2016

Rosenkranz 339

or Hamilton Rating Scale depression item \ 2 or considers subjects who have been identified as placebo
decrease in Hamilton Rating Scale total  20%) to non-responders in stage 1. Mainly two criteria for non-
screen out potential placebo responders. The average responders and a combination of both have been
placebo responses in these trials were 7:88 (SD = applied. The first declares subject k a non-responder if
2:12) and 7:56 (SD = 1:80), respectively. This suggests the value of the endpoint is too large at the end of the
that the placebo lead-in may be efficient in reducing treatment period
the number of placebo responders in major depressive
disorder. A similar investigation of 34 schizophrenia Y1ik  ca ð2Þ
trials provided a consistent result.7
The second is concerned with the percentage or rela-
The sequential parallel comparison design combines
tive change from baseline. Let X1ik be the baseline value
a parallel group design with a placebo non-responder
corresponding to X1ik . Then, subject k is declared a
enriched second stage.8 Originally described for binary
non-responder if
endpoints, it was extended to continuous data.9–11 An
article summarizing the results from four clinical trials Y1ik  X1ik  
using this design reports on study where the placebo  cp or Y1ik  1 + cp X1ik ð3Þ
X1ik
response in placebo non-responders was higher than
the response in test treatment. The placebo response The combined criterion declares subject k a non-
was much lower in the enriched population than in all- responder if both criteria are satisfied, that is, if
comers for the other cases.12    
In the following, we investigate whether there are Y1ik  max ca , 1 + cp X1ik ð4Þ
explanations for the presence or absence of a benefit of
placebo lead-in periods by studying the impact of the In fact, criteria (2)–(4) have the form
enrichment process on the distribution of the data of
Y1ik  cik ð5Þ
placebo non-responders. We assume that data from pla-
cebo responders and non-responders are identically dis- which will be investigated in the following to derive a
tributed at the baseline of the study, that is, before general result. For the sake of completeness, it is men-
having been treated, but that only patients who improve tioned that the criterion
above a certain level are considered responders.
Y1ik  X1ik  cc

Assumptions is also covered by equation (5).

A way to implement a trial enriching for placebo

non-responders is to randomly assign subjects to two Results
treatment sequences—placebo–test (P-T) and placebo–
Since the data from a subject are correlated, mean and
placebo (P-P)—and to analyze only those subjects from
variance of the selected placebo non-responders are dif-
stage 2 who have been identified as placebo non-
ferent from mean and variance of all data in stage 2.
responders in stage 1. Here, we will compare the results
Let f, F, and l = f=(1  F) denote probability density
of all data from stage 2 with those from the enriched
function, cumulative distribution function, and hazard
population to investigate whether or under which cir-
function of a standard normal variable, respectively.
cumstances enrichment offers advantages over an all-
Let tik = (cik  mP )=sP . Then, one obtains the mean
comer population.
and variance of placebo non-responders in the second
The two sequences P-T and P-P will be numbered by
stage (see Appendix 1, section ‘‘Mean and variance of
i = 1, 2, respectively. For each sequence i and subject k,
the responses after enrichment for placebo non-respon-
data (Y1ik , Y2ik ) are observed, which are assumed to be
ders’’) as
normally distributed with mean (m1i , m2i ) and covar-
iance matrix13 m2i ðcik Þ = m2i + ri s2i lðtik Þ ð6Þ
   
s21i ri s1i s2i s2i ðcik Þ = s22i 1  r2i ½1  vðtik Þ ð7Þ
Si = , i = 1, 2
s22i
Now v(t)\1 for all t implies
Since all subjects receive placebo in stage 1 and are
randomly assigned to sequences, we can set s22i ðcik Þ\s22i , for i = 1, 2 and all k

m11 = m12 = mP , s11 = s12 = sP ð1Þ

Result 1. Enriching for placebo non-responders reduces
Smaller values are considered an improvement and variability if the data from a sequence are correlated.
m22 6¼ mP is allowed. The analysis of stage 2 data In this case, the variance of the second stage data of
Downloaded from ctj.sagepub.com at CAMBRIDGE UNIV LIBRARY on June 5, 2016
340 Clinical Trials 13(3)

placebo non-responders is always smaller than the var- Therefore, enriching is advantageous if the effect size
iance of data from all subjects in stage 2. This holds in all subjects in the second stage is smaller than the
for placebo and test treatment. effect size in the placebo non-responders, that is, if
pffiffiffi
Unfortunately, the difference in effect between pla- d n m21 ðc1 Þ  m22 ðc2 Þ
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi  pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð8Þ
2
s21 + s22 2 s21 ðc1 Þ=h1 + s222 ðc2 Þ=h2
2
cebo and test in the second stage is not necessarily
larger in placebo non-responders than in the full popu-
lation. For example, consider the special case ri = r, Theoretically, equation (8) can be used to assess
s2i = s2 , and cik = c for i = 1, 2 and all k. Then, one whether enriching is advantageous under certain
gets from equation (6) assumptions on the distributional parameters, for a
specific threshold and conditional on the baseline val-
m21 ðc2k Þ  m22 ðc2k Þ = m21  m22 = d (say) ues of the endpoint of interest.

This means that the treatment difference between Result 2. Whether enriching for placebo non-
test and placebo among the placebo non-responders is responders is beneficial depends on the definition of a
identical to the treatment difference among all subjects cut-off, the correlation of the data between the placebo
in stage 2. Taking into consideration that enriching lead-in and the second stage, and the variances of stage
reduces the number of patients in the second stage, the 2 data as given by equation (8).
same treatment difference has to be estimated from
data of fewer patients. The lower variability in placebo
A simplification is possible for cik = c for all sub-
non-responders can compensate for the sample size
jects, that is, if criterion (2) is used to declare non-
reduction to some extent; however, there can be a
responders
break-even point when this will be no longer the case.
To investigate this trade-off further in the general pffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
d n ½m (c)  m22 (c) n½1  F(t)
framework, we compare the effect size in the full popu- pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi  21 pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi
lation in stage 2 with the effect size in the enriched pop- s221 + s222 s221 (c) + s222 (c)
pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ð9Þ
ulation.
Pn Assume n subjects per sequence. Then fd + l(t)½r1 s21  r2 s22 g n½1  F(t)
Ni = k = 1 IfY1ik cik g is the number of placebo non- = qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
 ffi
s221 + s222  ½1  v(t) r21 s221 + r22 s222
responders in sequence i with

X
n Note that condition (9) does not depend on the sample
hi = E½Ni  = ½1  Fðtik Þ size n which cancels out.
k=1 To simplify matters further, let ri = r and s2i = s2 ,
i = 1, 2, that is, enriching does not change the expected
Let ci = (ci1 , . . . , cin ) and define
treatment effect difference in stage 2. The relative effi-
ciency of an estimator of the treatment difference from
1X n
m2i ðci Þ = m ðcik Þ, i = 1, 2 the enriched population over the all-comers is then
n k = 1 2i
given by
and 2s22 n½1  F(t) 1  F(t)
eðt, rÞ = 3 = ð10Þ
n 2s22 (c) 1  r2 ð1  v(t)Þ
1X n
s22i ðci Þ = s2 ðcik Þ, i = 1, 2
n k = 1 2i Since now the efficiency depends only on two para-
P meters, the correlation and the threshold, their impact
^ 2i (ci ) = (1=Ni ) nk = 1 Y2ik IfY1ik cik g be the aver-
Let m can be easily displayed graphically. Figure 1 shows a
age response observed in placebo non-responders in contour plot of e(t, r). Enriching for placebo non-
sequence i of stage 2. In Appendix 1, section ‘‘Mean responders is advantageous for all combinations of
and variance of the response estimators after enrich- (t, r) to the right of the contour line e = 1. In general,
ment for placebo non-responders,’’ it is shown that fairly high correlations between stage 1 and stage 2
are required. A simulation with mP = m22 =  1,
E ½m
^ 2i ðci ÞjNi .0 = m2i ðci Þ m21 =  1:7, sP = si2 = 1, n = 20 and 10,000 simula-
tions for all combinations (tj , rj ), tj =  1 + j=10,
and
rj = 0:5 + j=10 confirm these results (see Figure 2). It
 should be emphasized that here, enriching is not
1 s 2 ð ci Þ
^ 2i ðci ÞjNi .0 = s22i ðci ÞE
V ½m jNi .0 ’ 2i effective by increasing the treatment difference over
Ni hi
an unselected population, but solely by reducing
variability.
Downloaded from ctj.sagepub.com at CAMBRIDGE UNIV LIBRARY on June 5, 2016
Rosenkranz
Figure 1. Relative efficiency of the treatment difference
estimator under all-comers over enriched population from (10)
(sij = 1, ri = r, t = (c  mP )=sP ).
Figure 2. Relative efficiency of the treatment difference
estimator under all-comers over enriched population from
simulations (n = 20, sij = 1, ri = r, t = (c  mP )=sP ). For
details, see text.
Conclusion
We investigated the impact of enriching a patient popu-
lation by excluding placebo responders in the setting of
normally distributed endpoints. As expected, this
approach is not generally increasing the chance to provide
evidence for the efficacy of test treatment over placebo.
The treatment effect difference in the enriched population
can increase or decrease relative to the non-enriched pop-
ulation depending on data variability in the second stage
of the trial and correlation between data from stage 1 and
stage 2. If equal variability and correlation of the data
among treatments is a reasonable assumption, the differ-
ence in effect does not change under enrichment when pla-
cebo response is defined in absolute terms. In more
general situations, the only dependable impact of enrich-
ment is a reduction in variability.
Downloaded from ctj.sagepub.com at CAMBRIDGE UNIV LIBRARY on June 5, 2016
341
However, enrichment reduces sample size such that
the reduction in data variability is counteracted by
smaller number of subjects. We provide a quantitative
answer to the question under which circumstances
enriching is advantageous and when not. In general,
usefulness of enriching increases with decreasing cut-off
values and increasing correlation. While the latter can
hardly be influenced, the former contains an element of
arbitrariness which could be used to make enrichment
efficient. The sensibility of such an approach needs to
be discussed.
The quoted literature on psychiatric clinical trials
does not reach a unanimous opinion on the usefulness
of enriching for placebo non-responders. Our investi-
gation offers an answer to this observation, namely,
that it may indeed depend on the situation. When
enriching turns out to be useful in detecting a treat-
ment difference, it can leave unanswered questions as
to the patients to whom the results apply. An overall
development plan should consider these issues and
generally collect broader data to evaluate them, for
example, by studying to some extent, results in a less
enriched population.14 A practical issue is an element
of deception in the single blind placebo lead-in
because the ‘‘open protocol, hidden allocation’’ con-
dition is not met.15 However, appropriate informed
consent wording has been proposed.16 The sequential
parallel comparison design avoids the issue by con-
ducting stage 1 as an unbalanced placebo-controlled
parallel group trial.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with
respect to the research, authorship, and/or publication of this
article.
Funding
The author(s) received no financial support for the research,
authorship, and/or publication of this article.
References
1. Walsh B, Seidman S, Sysko R, et al. Placebo response in
studies of major depression: variable, substantial and
growing. JAMA 2002; 287: 1840–1847.
2. Khin N, Chen Y, Yang Y, et al. Exploratory analyses of
efficacy data from major depressive disorder trials sub-
mitted to the US Food and Drug Administration in sup-
port of new drug applications. J Clin Psychiatry 2011; 72:
464–472.
3. Khin N, Chen Y, Yang Y, et al. Exploratory analyses of
efficacy data from schizophrenia trials in support of new
drug applications submitted to the U.S. Food and Drug
Administration. J Clin Psychiatry 2012; 73: 856–864.
4. Trivedi M and Rush J. Does a placebo run-in or a pla-
cebo treatment cell affect the efficacy of antidepressant
medications? Neuropsychopharmacology 1994; 11: 33–43.
342 Clinical Trials 13(3)

5. Faries D, Heiligenstein J, Tollefson G, et al. The double- f ð zÞ

f ð zÞ = Ifztg ð11Þ
blind variable placebo lead-in period: results from two 1  FðtÞ
antidepressant clinical trials. J Clin Psychopharmacol
2001; 21: 561–568. Let x2f be the cumulative distribution function of a
6. Reimherr F, Ward M and Byerley W. The introductory chi-square distributed variable with f degrees of free-
placebo washout: a retrospective evaluation. Psychiatry dom, then it follows that
Res 1989; 21: 191–199.
7. Chen YF, Yang Y, Hung H, et al. Evaluation of perfor- E½Z  = lðtÞ ð12Þ
mance of some enrichment designs dealing with high pla-
cebo response in psychiatric clinical trials. Contemp Clin and
Trials 2011; 32: 592–604.
8. Fava M, Evins A, Dorer D, et al. The problem of the pla-   1    
cebo response in clinical trials for psychiatric disorders: E Z2 = 3 Ift0g 1  x23 t2
2½1  FðtÞ
culprits, possible remedies, and a novel study design
  
approach. Psychother Psychosom 2003; 72: 115–127. + Ift\0g 1 + x23 t2 g ð13Þ
9. Tamura R and Huang X. An examination of the effi-
ciency of the sequential parallel design in psychiatric clin- and v(t) = V ½Z = E½Z 2   E½Z2 .17 Note that v(t)\1 for
ical trials. Clin Trials 2007; 4: 309–317.
any t. For an N (m, s) variable X truncated at c, set
10. Fava M, Mischoulon D, Iosifescu D, et al. A double-
blind, placebo-controlled study of aripiprazole adjunctive
t = (c  m)=s and use E½X  = m + sE½Z and
to antidepressant therapy among depressed outpatients V ½X  = s2 V ½Z.
with inadequate response to prior antidepressant therapy
Mean and variance of the responses after
(adapt-a study). Psychother Psychosom 2012; 87–97.
11. Doros G, Pencina M, Rybin D, et al. A repeated mea- enrichment for placebo non-responders
sures model for analysis of continuous outcomes in From equations (12) and (13), one obtains
sequential parallel comparison design studies. Stat Med
2013; 32: 2767–2789.
m2i ðcik Þ = E½Y2ik jY1ik  cik 
12. Baer L and Ivanova A. When should the sequential par-
allel comparison design be used in clinical trials? Clini
s2i
= m2i + ri fE½Y1ik jY1ik  cik   m1i g
Invest 2013; 3: 823–833. s1i
13. Kotz S, Balakrishnan N and Johnson N. Continuous mul- = m2i + ri s2i lðtik Þ
tivariate distributions: models and applications, vol. 1. 2nd
ed.New York: John Wiley & Sons, 2000. and
14. Temple R. Enrichment of clinical study populations. Clin
Pharmacol Ther 2010; 88: 775–778. s2i ðcik Þ = V ½Y2ik jY1ik  cik 
15. Senn S. Are placebo run ins justified? BMJ 1997; 314:
1191–1193. = E½V ½Y2ik jY1ik jY1ik  cik 
16. Ramsey L. Commentary: placebo run ins have some + V ½E½Y2ik jY1ik jY1ik  cik 
value. BMJ 1997; 314: 1193.  
= s22i 1  r2i
17. Barr D and Sherrill E. Mean and variance of truncated 
normal distributions. Amer Statist 1999; 53: 357–361. s2ik
+ V m2i + ri ðY1ik  m1i ÞjY1ik  cik
s1ik

2
 2
 s2i 2
= s2i 1  ri + ri V ½Y1ik jY1ik  cik 
Appendix 1 s1i
 
= s22i 1  r2i ½1  vðtik Þ
Truncated normal variables
Here, we provide formulae for the expectation and var- with v(t) = V ½Z for a standard normal variable trun-
iance of truncated normal variables for reference. Let Z cated at t which can be obtained from equations (12)
be a standard normal random variable truncated below and (13).
at a fixed point t. The density of Z is

Downloaded from ctj.sagepub.com at CAMBRIDGE UNIV LIBRARY on June 5, 2016

Rosenkranz 343

Mean and variance of the response estimators after

enrichment for placebo non-responders
Mean and variance are given by
" " # #
1 X n
E ½m
^ 2i ðci ÞjNi .0 = E E Y2ik IfY1ik cik g jNi jNi .0
Ni k=1
" #
1 X n  
=E E Y2ik IfY1ik cik g jY1ik  cik , Ni 3 Pr½Y1ik  cik jNi jNi .0
Ni k = 1
" #
1 X n
Ni
=E E½Y2ik jY1ik  cik jNi .0
Ni k = 1 n
= m2i ðci Þ

and
" " # #
1 X n
V ½m
^ 2i ðci ÞjNi .0 = E 2 V  Y2ik IfY1ik cik g jNi jNi .0
Ni k=1
" #
1 X n  
=E 2 V Y2ik IfY1ik cik g jY1ik  cik , Ni 3 Pr½Y1ik  cik jNi jNi .0
Ni k = 1
" #
1 X n
Ni
=E 2 V ½Y2ik jY1ik  cik jNi .0
Ni k = 1 n

2 1
= s2i ðci ÞE jNi .0
Ni
s 2 ðc i Þ
’ 2i
hi

since V ½E½^
m2i (c)jNi Ni .0 = 0.

Downloaded from ctj.sagepub.com at CAMBRIDGE UNIV LIBRARY on June 5, 2016