Professional Documents
Culture Documents
E Pires 00251 Reporting Analysis of For Cts
E Pires 00251 Reporting Analysis of For Cts
net/publication/5337451
CITATIONS READS
9 58
4 authors, including:
Some of the authors of this publication are also working on these related projects:
Computer visualisation and process mining for epilepsy patient pathways View project
All content following this page was uploaded by Melissa Maguire on 27 January 2019.
a
The Department of Neuroscience, University of Liverpool, Liverpool, UK
b
The Department of Statistics, University of Warwick, Coventry, UK
Received 16 October 2007; received in revised form 6 April 2008; accepted 13 April 2008
Available online 2 June 2008
KEYWORDS Summary
Open-label extension; Purpose: Open-label extension studies, or follow-on randomised controlled trials (FORCTs) are
Anti-epileptic drugs; widely believed to be prone to patient selection biases which may inflate effect estimates.
Efficacy; This study investigates the reporting and analysis of efficacy outcomes in FORCTs and critically
ITT analysis evaluates the associated underlying assumptions. We propose an alternative method of analysis,
in line with that recommended in the analysis of RCTs, the intention to treat (ITT) approach,
in which it is assumed that all patients who discontinue treatment are non-responders.
Methods: A systematic review of FORCTs and randomised controlled trials (RCT) of topiramate,
levetiracetam and gabapentin as adjuvant therapy in refractory adult epilepsy was conducted.
Sample sizes and numbers of responders, along with reported outcomes were extracted. To
evaluate the feasibility of the assumptions underlying the various methods of analysis, the
most common causes of discontinuation were evaluated. For each FORCT, we compared the
reported outcome to the proposed ITT analysis.
Results: The 10 FORCT reports identified all excluded from the analysis patients who dropped
out of the RCT. Adverse events or inefficacy were the main reasons for treatment discontinu-
ation. Analysis based on the ITT method, led to smaller effect estimates than those reported.
For example, a FORCT of levetiracetam reported a responder rate of 43%, which reduced to
28% under an ITT analysis, comparable to an ITT analysis outcome of 26% for the parent RCT.
Conclusions: FORCTs can provide important information about long-term efficacy and tolera-
bility of newer therapies. However, current reporting methods are likely to be misleading as
outcomes are reported for the subset of patients continuing with treatment at the end of the
FORCT. Since the majority of patients who discontinue treatment do so for reasons associated
with inefficacy, an analysis based on the ITT approach more closely reflects the outcomes of
the patients.
© 2008 Elsevier B.V. All rights reserved.
∗ Corresponding author. Present address: The Department of Neurology, Sunderland Royal Hospital, Kayll Road, Sunderland, SR4 7TP, UK.
0920-1211/$ — see front matter © 2008 Elsevier B.V. All rights reserved.
doi:10.1016/j.eplepsyres.2008.04.007
Reporting and analysis of open-label extension studies of anti-epileptic drugs 25
that those who discontinue treatment are non-responders. In con- FORCT reports. It was possible to derive original RCT sam-
trast, the evaluable case analysis uses only patients who have ple sizes for a further two topiramate FORCTs (Engelskjon
completed the FORCT with available data in the denominator. An et al., 1993; Mikkelsen et al., 1993) as the parent RCT gave
implicit assumption underlying an evaluable case analysis is that all information on number of patients recruited according to
those who discontinue treatment are representative, in terms of centre.
responder status, to those who remain in the trial.
Details on patient selection from the original RCT through
In order to quantify long-term effects, all patients included
within the analysis should have been exposed to treatment for the
into the FORCT were reported in two topiramate FORCTs
duration of the study. In FORCTs, those patients initially randomised (Michelucci et al., 1996; Fincham and Schottelius, 1995),
to placebo have less exposure compared to those initially ran- one levetiracetam FORCT (Betts et al., 2000), and one
domised to the treatment. To accommodate this, an ITT sub-analysis gabapentin FORCT (The US Gabapentin Study Group, 1994).
of responder proportions was carried out for patients entering the The levetiracetam FORCT also reported the number entering
FORCT who were originally assigned to the active treatment arm the FORCT and responders according to randomised treat-
in the RCT, excluding patients who were originally assigned to the ment status in the RCT. This was the only FORCT to report
placebo arm. An example calculation demonstrating the various such data.
analyses can be found in Appendix 1. The length of treatment period and daily dose and the
Some FORCTs reported from a single centre, participating within
reporting of patient characteristics were missing in the
a multi-centre RCT but with no information on the selection pro-
portion from RCT to FORCT. For these studies, the dropout rate for
majority of reports.
the original multi-centre RCT was used as an approximate selection
proportion for the single centre: this assumes that the single centre Dropout rates and reasons
population was typical of the study population.
27
28 M.J. Maguire et al.
ber of patient responders. For these six studies, responder reported results for corresponding RCTs of Gabapentin (19%
rates could be recalculated based on a full case analysis and 20%).
and an eligible case analysis. For these six studies, effect It might be expected that an analysis based on an ITT
estimates were smaller based on either a full case analysis approach would lead to lower estimates of responder pro-
or an eligible case analysis than an evaluable case anal- portions, compared to the RCTs, as more patients might be
ysis. Estimates differed in one study by 37% (Sharief et expected to drop out of the study as duration of exposure to
al., 1993). Only three of these six FORCT studies reported drug increases. It is possible that increases in responder pro-
responder rates, with the other three studies simply report- portions between RCTs and some of the FORCTs presented
ing the number of responders. For the three studies which here, are attributed to patients initially randomised to the
reported responder rates, the full case analysis and the placebo arm.
eligible case analysis gave smaller effect estimates than Although FORCTs are increasingly conducted in health-
outcomes reported (Table 1) (Betts et al., 2000; The US care research, few appear in the published literature (Taylor
Gabapentin Study Group, 1994; Sivenius et al., 1991). and Wainwright, 2005). The majority of FORCTs identi-
Effect estimates for FORCTs based on a full case analysis fied were published only as abstracts (70%) and were
were similar to reported estimates for patients randomised reports from single centres with small numbers of patients.
to the treatment arm in the parent RCT (Engelskjon et Researchers conducting RCTs with an open-label exten-
al., 1993; Sharief et al., 1993; Betts et al., 2000; The US sion should make more data available within published
Gabapentin Study Group, 1994; Sivenius et al., 1991) (where work.
the reported responder proportion for the RCTs were in all We identified serious inadequacies in the reporting on
but one case based on an ITT analysis). For example, in a FORCTs in particular abstracts and this included lack of
FORCT of topiramate, the recalculated proportion of 50% information on the number of patients entering into the
responders reduced from 77% according to an evaluable case initial parent RCT information should be provided on base-
analysis to 40% based on full case analysis, which was com- line characteristics and sample sizes for participants within
parable to the 35% reported for patients in the treatment the original RCT, dropouts, and those who continue into the
arm of the parent RCT (Sharief et al., 1996). FORCT. The reporting of numbers responding, as opposed to
rates would prevent ambiguity and allow reliable interpre-
tation of results. Retention rates as the primary endpoint in
Discussion open-label studies also prevents ambiguity. Guidelines for
the reporting of observational studies have recently been
Reports of FORCTs fail to acknowledge patient selection outlined in the strengthening the reporting of observational
through the RCT into the FORCT. Since in the study of AEDs, studies in epidemiology (STROBE) statement (Elm et al.,
the majority of those not continuing into the FORCT do so 2007). Reporting standards, however, may have less impact
for reasons of inefficacy or adverse events, it is preferable on the reporting of abstracts. We recommend that submitted
to treat all those who do not enter the FORCT or who drop abstracts are more structured and report sufficient detail to
out during the RCT, as non-responders, using an intention to allow the reader to ascertain how the study was designed
treat analysis. In fact, in the analysis of RCTs investigating and conducted.
AEDs this is the routine method of analysis. However, cur-
rent methods used in all but one of the FORCTs identified,
used an evaluable case analysis in which it is unreasonably Implications for clinical practice
assumed that those who discontinue treatment are repre-
sentative (in terms of responder status) of those who remain There is a clear need for information on the longer term effi-
in the trial. The evaluable case analysis led to over opti- cacy of newer add-on anti-epileptic drugs. Ideally this could
mistic estimates of efficacy and inferences based on this be achieved through long-term multi-centre pragmatic RCTs
method of analysis are likely to be misleading. The alter- conducted over a minimum of 12 months. To conduct such
native proposed intention to treat approach better reflects trials would require significant financial and resource input,
the responder status of the majority of patients and lim- which may not be readily available. In addition, the conduct
its selection biases. This approach however may impact less of such trials may be low on the agenda for pharmaceuti-
on other forms of bias such as unblinding of patients and cal companies whose prime aim is to gain a drug licence
investigators to treatment. indication. FORCTs represent a valuable and more practica-
The reported estimates of responder rates from the two ble alternative to long-term pragmatic RCTs when deriving
FORCTs for topiramate were 57% and 68%. Using an ITT information about long-term treatment effects.
approach, responder rates for two FORCTs were 40%, similar This study has highlighted that few FORCTs are reported
to the estimate of 44% from a meta-analysis of six topira- in full. Failure to disseminate study findings through full
mate RCTs (Marson et al., 1997). Similarly, results obtained publication raises ethical considerations as patients who
when recalculating responder rates for levetiracetam based consent to an extension study may do so, on the premise that
on an ITT analysis for one FORCT of levetiracetam (Betts their participation in the study may benefit clinical practice
et al., 2000) led to a responder rate of 29% compared to (Dowd, 2004). Failure to report these studies can also lead
43% reported by the FORCT. Results from this FORCT were to publication bias or small study effect bias when system-
consistent when calculations were based only on patients atic reviews or meta-analyses are conducted. It is therefore
who were initially randomised to active treatment in the important that results of these studies are made more widely
RCT. For gabapentin, the estimated ITT analysis responder available. Analysis based on an intention to treat approach
rates were 12% and 23% which are broadly consistent with can help limit the impact of patient selection biases.
Reporting and analysis of open-label extension studies of anti-epileptic drugs 29
Acknowledgements Engelskjon, T., Johannessen, S.I., Kloster, R., Nakken, K., Lossius,
R., Henriksen, O., 1993. Long term effect of topiramate in
refractory partial epilepsy. Epilepsia 34 (Suppl. 2), 123.
MM has been sponsored to attend conferences by Janssen
Fincham, R., Schottelius, D., 1995. Long term therapy with topira-
Cilag and Pfizer and has received research funds from Pfizer. mate in patients with partial seizures. Epilepsia 36 (Suppl. 4),
AGM has been sponsored to attend conferences by Janssen 56.
Cilag, UCB Pharma and Pfizer, has undertaken consultancy Handforth, A., Treiman, D.M., 1994. Efficacy and tolerance of long-
for UCB Pharma, and has received research funds from Pfizer. term, high-dose gabapentin: additional observations. Epilepsia
Funding: Dr. M.J. Maguire and Dr. K. Hemming were 35 (5), 1032—1037.
employed as research fellows to Dr. A.G. Marson and Prof. Hemming, K., Hutton, J.L., Maguire, M.J., Marson, A.G., 2008.
J.L. Hutton for the duration of this work, and funded under Open-label extension studies and patient selection biases. J.
a Medical Research grant (number G0400642) ‘Models for Eval. Clin. Pract. 14 (1), 141—144.
Marson, A.G., Kadir, Z.A., Chadwick, D.W., 1997. New anti-epileptic
selection bias, applied to controlled trials and observational
drugs: a systematic review of their efficacy and tolerability.
studies of anti-epileptic drugs’.
Epilepsia 38 (8), 859—880.
Michelucci, R., Passarelli, D., Tassinari, C.A., 1996. Topiramate in
Appendix A. Supplementary data the treatment of refractory partial epilepsy: a 2 year long-term
follow-up. Epilepsia 37 (Suppl. 4), 70.
Supplementary data associated with this article can Mikkelsen, M., Ostergaard, L., Dam, M., 1993. Topiramate as long-
term treatment in refractory partial epilepsy. Epilepsia 34
be found, in the online version, at doi:10.1016/
(Suppl. 2), 123.
j.eplepsyres.2008.04.007. Ojemann, L.M., Wilensky, A.J., Temkin, N.R., Chmelir, T., Ricker,
B.A., Wallace, J., 1992. Long term treatment with gabapentin
References for partial epilepsy. Epilepsy Res. 13, 159—165.
Privitera, M., 1995. Long-term efficacy and safety of topiramate.
Epilepsia 36 (Suppl. 3), S152.
Abou-khalil, B., Schaich, L., 2005. Long-term efficacy of levetirac-
Privitera, M., Fincham, R., Penry, J., Reife, R., Kramer, L.,
etam for partial seizures. Seizure 14, 577—585.
Pledger, G., Karim, R., 1996. Topiramate placebo-controlled
Anhut, H., Ashman, P.J., Feuerstein, T.J., Quebe-Fehling, E., Saun-
dose-ranging trial in refractory partial epilepsy using 600-, 800-,
ders, M., Baron, B.A., The International Gabapentin Study
and 1,000-mg daily dosages. Neurology 46 (6), 1678—1683.
Group, 1995. Long-term safety and efficacy of gabapentin (Neu-
Schear, M.J., Wiener, J.A., Rowan, A.J., 1991. Long term efficacy
rontin) as add-on therapy in patients with refractory partial
of gabapentin in the treatment of partial seizures. Epilepsia 32
seizures. J. Epilepsy 8 (1), 44—50.
(Suppl. 3), 6.
Ben-Menachem, E., 1995. Long-term follow-up of patients treated
Sharief, M.K., Sander, J.W., Patsalos, P.N., Shorvon, S.D., 1993.
with topiramate for partial seizures. Epilepsia 36 (Suppl. 3),
Adjuvant topiramate in intractable partial epilepsy. Epilepsia 34
S152.
(Suppl. 6), 41.
Ben-Menachem, E., Henriksen, O., Dam, M., Mikkelsen, M.,
Sharief, M., Viteri, C., Ben-Menachem, E., Weber, M., Reife, R.,
Schmidt, D., Reid, S., Reife, R., Kramer, L., Pledger, G., Karim,
Pledger, G., Karim, R., 1996. Double-blind, placebo-controlled
R., 1996. Double-blind placebo-controlled trial of topiramate
study of topiramate in patients with refractory partial epilepsy.
as add-on therapy in patients with refractory partial seizures.
Epilepsy Res. 25, 217—224.
Epilepsia 37 (6), 539—543.
Sivenius, J., Kalviainen, R., Ylinen, A., Riekkinen, P., 1991. Double-
Betts, T., Waegemans, T., Crawford, P., 2000. A multi-centre,
blind study of gabapentin in the treatment of partial seizures.
double-blind, randomised, parallel group study to evaluate the
Epilepsia 32 (4), 539—542.
tolerability and efficacy of two oral doses of levetiracetam,
Sivenius, J., Ylinen, A., Kalviainen, R., Riekkinen Sr., P.J., 1994.
2000 mg daily and 4000 mg daily, without titration in patients
Long-term study with gabapentin in patients with drug-resistant
with refractory epilepsy. Seizure 9, 80—87.
epileptic seizures. Arch. Neurol. 51, 1047—1050.
Cameron, N., Penovich, P.E., Ritter, F.J., Espe-Lillo, J., 1996. Open-
Tassinari, C.A., Michelucci, R., Chauvel, P., Chodkiewicz, J.,
label study of UCB LO59 as add-on therapy in refractory partial
Shorvon, S., Henriksen, O., Dam, M., Reife, R., Pledger, G.,
epilepsy. Epilepsia 37 (Suppl. 5), 169.
Karim, R., 1996. Double-blind, placebo-controlled trial of top-
Cereghino, J.J., Biton, V., Abou-Khalil, B., Dreifuss, F., Gauer, L.J.,
iramate (600 mg daily) for the treatment of refractory partial
Leppik, I., 2000. Levetiracetam for partial seizures: results of a
epilepsy. Epilepsia 37 (8), 763—768.
double-blind randomised clinical trial. Neurology 55, 236—242.
Taylor, G.J., Wainwright, P., 2005. Open label extension studies:
Dickersin, K., Scherer, R., Lefebvre, C., 1994. Identifying relevant
research or marketing? Brit. Med. J. 331 (7516), 572—574.
studies for systematic reviews. BMJ 309, 1286—1291.
The US Gabapentin Study Group No 5, 1993. Gabapentin as
Dowd, D., 2004. Breaching the contract: the ethics of non-
add-on therapy in refractory partial epilepsy: a double-blind,
publication of research studies. Arch. Pediatr. Adolesc. Med.
placebo-controlled, parallel-group study. Neurology 43, 2292—
158, 1014—1015.
2298.
Elm, E.V., Altman, D.G., Egger, M., Pocock, S.J., Gøtzsche, P.C.,
The US Gabapentin Study Group, 1994. The long-term safety and
Vandenbroucke, J.P., STROBE Initiative, 2007. Strengthening the
efficacy of gabapentin (Neurontin) as add-on therapy in drug
reporting of observational studies in epidemiology (STROBE)
resistant partial epilepsy. Epilepsy Res. 18, 67—73.
statement: guidelines for reporting observational studies. BMJ
335, 806—808.