European Neuropsychopharmacology (2008) 18, 623–627

w w w. e l s e v i e r. c o m / l o c a t e / e u r o n e u r o

A regulatory Apologia — A review of placebo-controlled

studies in regulatory submissions of
new-generation antidepressants☆
Hans Melander a,⁎, Tomas Salmonson a,b ,
Eric Abadie b,c , Barbara van Zwieten-Boot b,d
a
Medical Products Agency, P.O. Box 26, SE-751 03 Uppsala, Sweden
b
CHMP, EMEA, London, UK
c
Agence Francaise de Securité Sanitaire des Produits de Santé, Saint-Denis, France
d
Medicines Evaluation Board, The Hague, Netherlands

Received 2 May 2008; received in revised form 11 June 2008; accepted 16 June 2008

KEYWORDS Abstract
Major depression;
Placebo; Data on percentage of patients experiencing a relevant response (N 50% reduction of the baseline
Clinical relevance Hamilton Depression Scale (HAMD) score), average baseline severity and sample size were
retrieved for all placebo-controlled studies in regulatory submissions of SSRIs and SNRIs between
1984 and 2003. Overall there was 16%-units (95% CI: 12; 20) more responders on active drug
compared to placebo. There was no evidence of a diminishing magnitude of effect with lower
severity at baseline. With one exception significant differences varying between 13.5 and 19.3%-
units were demonstrated for the individual antidepressants. Statistically significant mean
differences versus placebo in change in HAMD are not a proper basis for evaluation of clinical
relevance and are not sufficient for approval. Differences in the percentage of patients
experiencing a clinically relevant response should also be demonstrated. In this respect, the
approved SSRIs and SNRIs were found superior to placebo, independent of severity of depression.
© 2008 Elsevier B.V. and ECNP. All rights reserved.

1. Introduction

The efficacy of antidepressant medicinal products is con-

☆
Disclaimer: The views presented are those of the individual and
may not be understood or quoted as being made on behalf of EMEA or
reflecting the position of EMEA.
⁎ Corresponding author. Tel.: +46 18174600; fax: +46 18508730.
E-mail address: hans.melander@mpa.se (H. Melander).
tinuously discussed (Kirsch and Saperstein, 1998; NICE, 2004;
Kirsch et al., 2002, 2008). In particular the clinical relevance
of the effects demonstrated for new-generation drugs such
as selective serotonin reuptake inhibitors (SSRIs) and
serotonin noradrenalin reuptake inhibitors (SNRIs) is ques-
tioned. It has been shown that meta-analyses based on

0924-977X/$ - see front matter © 2008 Elsevier B.V. and ECNP. All rights reserved.
doi:10.1016/j.euroneuro.2008.06.003
624 H. Melander et al.

published data only are bound to overestimate the magni-
tude of effect due to selective publication and selective
reporting, and with the inclusion of unpublished data
submitted to the drug regulatory authorities the modest
effects seem to diminish further (Melander et al., 2003,
Turner et al., 2008). Some authors have discussed the impact
of severity of depression and argue that antidepressants are
effective for severely depressed patients but not for patients
with mild to moderate depression (Angst, 1993, Kahn et al.,
2002). In a recent meta-analysis based on placebo-controlled
studies submitted to the US Food and Drug Administration
this hypothesis is claimed to be confirmed (Kirsch et al.,
2008). Against this background one can ask why the new-
generation antidepressant medicinal products were ever
approved.
In the above meta-analyses questioning the clinical
relevance of antidepressant effects, the focus has been on
the absolute average difference in change from baseline on
the Hamilton Depression Rating Scale (HAMD) between active
drug and placebo. Indeed, HAMD is the primary outcome in
most antidepressant studies and it is used by the regulatory
authorities to evaluate whether statistically significant
differences between treatments have been shown. However,
mean difference on any rating scale is not an appropriate
outcome for the evaluation of clinical relevance. Once
statistical significance has been established, the clinical
value is judged on the basis of other outcomes, the most
important being the percentage of patients achieving a
clinically meaningful response. Thus, while the primary
statistical analysis operates on comparison on the group
level, this responder analysis rather focuses on the individual
patient.
The purpose of this paper is to discuss the clinical
relevance of SSRIs and SNRIs, the two most common classes
of new-generation antidepressants, in terms of an outcome
that is based on what is considered to be a relevant effect for
an individual patient. The discussion will be based on
analyses of the total placebo-controlled database available
when these medicinal products were approved. In addition,
the hypothesis that these antidepressants are effective
only in severely depressed patients will be tested in this
context.
2. Experimental procedures
2.1. Selection of studies
The approval of antidepressants by competent authorities in
Europe as well as in other regions is based on more or less
identical documentation. We chose to base our analyses on the
clinical documentation for the six SSRIs and two SNRIs that
have been approved in Sweden for treating major depressive
episodes. The market authorisation applications for these
medicinal products were submitted between 1984 and 2003.
The two most recent substances, escitalopram and duloxetine,
have a European approval, while the earlier substances are
nationally approved in Sweden as well as in most other
European countries. To be included in the analyses the studies
should be randomised, double-blind, placebo-controlled short-
term studies of at least 4 weeks duration with at least one
treatment group with a dose in the dose range that was later
approved. In studies with more than one dose in the approved
dose range the results for the dose closest to the recommended
dose were used in the analysis. With respect to severity of
depression the studies should have an inclusion criterion
requiring a score of at least 15 on the 17 item version of HAMD.
2.2. Data retrieval
The primary endpoint for this analysis was the percentage of
responders with response defined as at least 50% reduction of
the baseline HAMD score at the end of the study. The
calculation of percentage of responders was based on the
randomised set, and patients discontinuing prior to end of
study without evaluable data were considered as non-
responders. Percentage of responders, number of patients
and average HAMD score for each treatment were retrieved
from the original study reports. In case of missing or
inappropriately calculated responder figures, appropriate
figures were collected from the authority assessment
reports. In most studies the 17 item version of HAMD was
used, and for studies using other versions of HAMD the results
were converted to the 17 item version (HAMD17). In a few
studies the Montgomery Åsberg Depression Rating Scale

Table 1 The placebo-controlled database at approval for SSRIs and SNRIs

Substance Year of submission Number studies Total number of patients Average baseline HAMD17 score⁎
Mean Range
Paroxetine 1990 15⁎⁎ 1142 21.4 19.6; 24.1
Fluvoxamine 1989 5 408 22.6 19.7; 26.3
Citalopram 1992 5 420 23.8 22.7; 26.4
Sertraline 1993 4 914 23.2 20.5; 25.5
Fluoxetine 1984 8 570 22.0 19.3; 28.4
Escitalopram 2001 4 1181 21.2 19.5; 21,4
Venlafaxine 1992, 1996 9⁎⁎⁎ 1493 21.4 19.5; 28.4
Duloxetine 2003 6 1246 19.8 17.6; 21.3
Total 56 7374 21.6 17.6; 28.4
⁎) Converted from average baseline MADRS score for one sertaline study and two escitalopram studies.
⁎⁎) One additional paroxetine study with 29 patients in total did not report responder results.
⁎⁎⁎) Immediate release formulation: 6 studies; extended release formulation: 3 studies.
A regulatory Apologia — A review of placebo-controlled studies
(MADRS) was the primary endpoint, and for these studies
response was defined as at least 50% reduction of the
baseline MADRS score. In three of these studies the HAMD
rating scale was not used at all, and the average baseline
MADRS score was converted to a HAMD 17 score using the
relation between HAMD17 and MADRS derived from 45
studies using both scales. The data was retrieved by two
reviewers independently, and any discrepancies were dis-
cussed and resolved in consensus.
2.3. Statistical methods
The relation between percentage of responders and severity
of depression, measured as average baseline HAMD17 score,
was evaluated with a linear regression model. The analysis
was performed with sample size as weight. In the meta-
analyses of response rates for the different substances the
estimates from the individual studies were combined, with
the inverse of the variance of the estimates as weights
(DerSimonian and Laird, 1986).
3. Results
In total 57 studies fulfilled the inclusion criteria, but for one
of these studies responder data was not available, neither in
the company study report nor the authority assessment
report. Since the sample size of this paroxetine study
constitutes only 2.5% of the paroxetine placebo-controlled
database and 0.4% of the entire SSRI and SNRI placebo-
controlled database, the omission of this study was con-
sidered not to influence the paroxetine or the overall results.
The distribution over substances for the remaining 56 studies
is presented in Table 1. The amount of placebo-controlled
data varies between the substances with a clear tendency for
smaller databases for the drugs that were approved early,
with the exception of paroxetine. Considering all studies the
average baseline HAMD17 score ranged between 17.6 and
28.4 The fluoxetine and venlafaxine studies covered most of
this total range while the studies with escitalopram and
especially duloxetine had average baseline HAMD17 scores
predominantly in the lower part of the total range.
The placebo response as well as the response to active
drug showed large variability between studies (Fig. 1). The
Figure 1 Scatter plot of average baseline HAMD17 score versus
percentage of responders with regression lines for active treat-
ment (solid line) and placebo (dashed line).
625
Figure 2 Funnel plot. Estimate of the difference in percen-
tage of responders between active drug and placebo in relation
to the total number of patients (active plus placebo) on which
the estimate is based.
percentage of responders varied between 13.6% and 67.9%,
and 7.5% and 55.4% for active drug and placebo, respec-
tively. A similar variability was observed for the difference
between treatments, from −3.3% to 49.6%-units (Fig. 2).
The most extreme differences in either direction occurred in
fairly small studies.
There was no statistical evidence of a relation between
average baseline HAMD17 score and difference in percen-
tage of responders (Fig. 1, p = 0.51, test of difference in
slopes). Neither was there any evidence of a relation
between baseline HAMD17 score and percentage of respon-
ders independent of treatment (p = 0.98 and p = 0.31 for
active treatment and placebo, respectively, test of
slope = 0). In the absence of a difference in slopes the over-
all difference in percentage of responders was estimated to
16.0%-units (95% CI: 12.0; 20.0).
The overall magnitude of effect for the different sub-
stances is illustrated in Fig. 3. With the exception of fluvoxa-
mine significant differences varying between 13.5 and 19.3%-
units were demonstrated for all SSRIs and SNRIs.
4. Discussion
In addition to being overall statistically superior to placebo
with respect to average change in HAMD score (although
Figure 3 Overall difference in percentage of responders
between active drug and placebo with 95% confidence intervals
for SSRIs and SNRIs.
626
some individual studies might fail), we have shown that SSRIs
and SNRIs overall are superior to placebo in providing a
clinically relevant benefit to depressed patients (48% versus
32%, respectively, responding to treatment). There is no
evidence that this excess benefit is limited to severely
depressed patients.
Our analyses are based on the entire placebo-controlled
documentation for all SSRIs and SNRIs available when these
medicinal products were approved. In contrast to meta-
analyses based on statistical and medical reviews of studies
submitted to the FDA we have had access to full study
reports, and thus have been able to perform more
comprehensive data retrieval. In our study responder data
was missing in only one out of 57 studies, while in a recent
study based on FDA submissions (Kirsch et al., 2008) twelve
out 47 studies were excluded due to missing information on
crucial variables. When applying for marketing authorisa-
tion, the applicant is obliged to submit full reports of all
studies performed by the applicant as well as all available
information on any study performed by others than the
applicant. Thus, it is reasonable to conclude that the basis
for approval was not subjected to selection bias. Neither
should there have been any risk for selective reporting since
missing appropriately calculated responder figures are
routinely asked for or calculated by the authority. These
conclusions are supported by the symmetrical funnel plot
(Fig. 2). We did not include placebo-controlled studies after
the approval, essentially studies where an approved anti-
depressant is included as an active control in a placebo-
controlled study for a forthcoming competitor. First, these
studies were not available at the approval, and second, a re-
evaluation including these studies is doubtful as usually only
one dose is used with no or limited options for further
titration, which might be sub-optimal.
The purpose of this paper was not to compare the
different SSRIs and SNRIs, and the absence of direct
comparisons should preclude any comparison between the
substances. However, it seems reasonable to conclude that
the documentation for fluvoxamine is less convincing.
Comparisons between the different substances should be
based on head-to-head comparisons predominantly per-
formed after the approval. However, these studies are not
always submitted to the regulatory authorities, and the
decision to make them publicly available is left to the
sponsor and selective publication as well as selective
reporting is to be expected.
The responder criterion used in this investigation, at least
50% reduction of the baseline HDRS score is well recognized
and used in almost every antidepressant study. Since a
HAMD17 score of at least 18 is required for inclusion in most
studies, at least 9 points improvement is required for an
individual patient to be counted as a responder. This should
be compared to the average difference in change of about 2
points usually observed. The responder criterion is a relative
measure, and thus floor effects, which might contribute to
the diminishing effect size observed with lower average
HAMD score at baseline, are avoided. Finally, counting
discontinuing patients with no evaluable data as non-
responders is conservative, i.e. not likely to favour active
treatment, since discontinuations are usually less frequent in
the placebo groups. An alternative outcome based on
individual patient benefit could have been the percentage
H. Melander et al.
of patients in remission, usually defined as a HAMD17 score of
7 or lower. However, remission rate is not always a pre-
specified endpoint and remission results have been less
frequently, and thus potentially selectively, reported.
Furthermore, to establish whether a sustained remission
has occurred longer studies are required.
One can always discuss whether a difference in response
rate of 16%-units is large, modest or only marginal. In
analyses of average absolute change from baseline it has
been estimated that about 80% of the drug effect is
attributable to placebo. Similarly, with 49% responders on
active treatment and 33% on placebo one can argue that two
thirds of the drug effect is attributable to placebo. However,
such arguments is based on the doubtful assumption that the
placebo effect and the pharmacological effect are additive.
Furthermore, some of the placebo effects are probably due
to study specific procedures (increased attention, therapeu-
tic impact of weekly rating sessions) that are not present in
clinical practice. Hence, the 16%-units difference observed
in placebo-controlled studies could be considered as lower
limit of the pharmacological effect that could be expected in
clinical practice.
In conclusion, the approval of the SSRIs and SNRIs were
based on data demonstrating that they provide clinically
meaningful benefits to a non-negligible percentage of the
patients.
Role of the funding source
There has been no funding of the research reported in this paper.
Contributors
HM and TS designed the study. HM was responsible for data retrieval
and statistical analysis. All authors interpreted the results and
contributed to the writing of the paper.
Conflict of interest
Neither of the authors have any conflict of interest apart from being
an employee of a regulatory authority.
Acknowledgements
We thank Ms Therese Gester for assisting in the data retrieval
process.
References
Angst, J., 1993. Severity of depression and benzodiazepine co-
medication in relationship to efficacy of antidepressants in acute
trials: a meta-analysis moclobemide trials. Hum. Psychopharma-
col. 8, 401–407.
DerSimonian, R., Laird, N., 1986. Meta-analysis in clinical trials.
Control Clin. Trials 7, 177–188.
Kahn, A., Leventhal, R., Kahn, S., et al., 2002. Severity of depression
and response to antidepressants and placebo: an analysis of the
Food and Drug Administration database. J. Clin. Psychopharma-
col. 22, 40–45.
Kirsch, I., Saperstein, G., 1998. Listening to Prozac but hearing
placebo: a meta-analysis of antidepressant medication.
A regulatory Apologia — A review of placebo-controlled studies
Prev. Treat. 1 Article 0002a. Available: http://www.journals.
apa.org/prevention/volume1/pre001002a.html.
Kirsch, I., Moore, T.J., Scoboria, A., Nicholls, S.S., 2002. The emperor's
new drugs: an analysis of antidepressant medication data
submitted to the U.S. Food and Drug Administration. Prev. Treat.
5 Article 23. Available: http://www.journals.apa.org/prevention/
volume5/pre0050023a.html.
Kirsch, I., Deacon, B.J., Huedo-Medina, T.B., Scoboria, A., Moore, T.J.,
et al., 2008. Initial severity and antidepressant benefits: a meta-
analysis of data submitted to the Food and Drug Administration.
PLoS Med. 5 (2), e45. doi:10.1371/journal. pmed.0050045.
627
Melander, H., Ahlqvist-Rastad, J., Meijer, G., Beermann, B., 2003.
Evidenced b(i)ased medicine — selective reporting from studies
sponsored by pharmaceutical industry: review of studies in new
drug applications. BMJ 326, 1171–1173.
National Institute for Clinical Excellence, 2004. Depression: Manage-
ment of Depression in Primary and Secondary Care. Clinical prac-
tice guideline, vol 23. National Institue for Clinical Excellence,
London.
Turner, E.H., Matthews, A.M., Linardatos, E., Tell, R.A., Rosenthal,
R., 2008. Selective publication of antidepressant trials and its
influence on apparent efficacy. N. Engl. J. Med. 358, 252–260.