Annab of Oncology 5: 113-117, 1994.

© 1994 Kluwer Academic Publishers. Printed in the Netherlands.

Special article

Toxicity grading systems

A comparison between the WHO scoring system and the Common Toxicity Criteria when used
for nausea and vomiting

H. R. Franklin, G. P. C. Simonetti, A. C. Dubbelman, W. W. ten Bokkel Huinink, B. G. Taal,

G. Wigbout, I. A. Mandjes, O. B. Dalesio & N. K. Aaronson
The Netherlands Cancer Institute, Antonie van Leeuwenhoek Huis, Amsterdam, The Netherlands

Downloaded from https://academic.oup.com/annonc/article-abstract/5/2/113/129806 by guest on 09 November 2019

Summary the patients returned for the subsequent course of chemo-
therapy. To evaluate the coding systems, an estimate was
Background: The Common Toxicity Criteria adopted by the made of the percentage agreement between the patients' an-
NCI in the USA for grading toxicity in cancer clinical trials swers and the nurses' and doctors' ratings.
have been compared to the WHO scoring system which is Results: The percentage agreement of the Common Toxic-
still in use in Europe. ity Criteria with the patients' own experiences of nausea and
Patients & methods: Sixty-six patients undergoing emetic vomiting was considerably better than that of the WHO
chemotherapy at the Netherlands Cancer Institute completed score. The Gamma statistic confirmed this. The Common
questionnaires, 32 according to the WHO criteria and 34 to Toxicity Criteria have now been adopted for grading toxicity
the Common Toxicity Criteria, on the severity, frequency and in studies of the Early Clinical Trials Group of the EORTC
duration of gastro-intestinal toxicity. Their answers were and are recommended for use in other clinical trials
then compared to the scores coded by research nurses and
physicians. The nurses coded acute toxicity when the patients Key words: toxicity gTading, WHO, CTC, data management,
were discharged, and the doctors coded overall toxicity when clinical trials

Introduction iting, in practice, 'treatment' can mean one of a variety

The objective assessment of nausea and vomiting in pa-
tients receiving anti-cancer therapy is essential not only
in the clinical trial setting where the maximum tolerat-
ed dose is established or where the drug safety profile
is being monitored, but also during standard treatment,
since timely intervention is necessary for the patient's
mental and physical condition and for treatment com-
pliance. In Europe the most widely used coding system
for rating the side effects of treatment has been the one
devised by the World Health Organisation (WHO),
which dates from 1979-80 [1]. There are, however,
several problems with the WHO codes used for gastro-
intestinal toxicity which place the validity of the assess-
ment in question. Three items are compressed into one
code; nausea, vomiting and anti-emetic treatment. First,
because nausea and vomiting are combined as one
grade (Table 1), a single episode of vomiting (grade 2)
automatically overrides nausea alone (grade 1), even
though lengthy nausea could be much more distressful
to the patient that transient vomiting. Second, the code
does not adapt well to the current situation in clinical
practice where meticulous monitoring of side effects is
required for phase I and n trials. Grade 3 is assigned
when treatment is given for the vomiting. Although a
seemingly logical evaluation of the severity of the vom-
of situations, for example:
- a small dose of anti-emetic taken orally
- a higher dose administered intravenously
- the need for hyd ration
- anti-emetics given prophylactically.
The recently developed 5-HT3 receptor antagonists
provide excellent anti-emetic control and are now given
very readily. In the case of prophylactic anti-emetics
whereby the patient does not vomit, three different
methods of reasoning may be applied when assigning
the WHO toxicity grade:
- that grade 0 is given because the patient has no
symptoms
- that grade 3 is given because the patient requires
treatment (and the prophylactic anti-emetics are
administered in advance because the drug is
known to be emetic)
- that toxicity, in this case, is not evaluable.
In the first case it is the effectiveness of the anti-emetic
regimen that is being scored and in the second the scale
loses all sensitivity. In none of these situations is the
grading a valid assessment of the toxicity experienced.
Vantongelen et al. [2] observed, during a quality
assurance audit of a multi-center trial, that to overcome
such difficulties, the staff from different institutes had
employed different interpretations for using the WHO
114
Table 1. Toxicity grading systems.
Grade 0
WHO coding system
Nausea-vomiting None Nausea
Common Toxicity Criteria
Nausea None Able to eat
reasonable intake
Vomiting None One episode in
24 hours
coding system. One institute always coded '3' for nau-
sea/vomiting when prophylactic anti-emetics were
given, while another institute always coded '0' in these
cases, even though the patient may have experienced
some nausea. In a third institute the staff used the code
'1' for nausea and acceptable vomiting, '2' (instead of
grade 3) for vomiting lasting several days and requiring
additional medication, and the code '3' was never used.
These inconsistent decision-rules employed across
institutions presented serious difficulties in interpreting
the gastro-intestinal toxicity data collected in this trial.
In addition to our own experiences, a number of
methodological problems with a variety of assessment
scales have been reported in the literature [3-9].
The system used to code side effects for clinical
trials needs to reflect gradations of a patient's experi-
ence, since severe symptoms will require therapeutic
intervention and will be dose-limiting. Moderate to
severe symptoms may affect chemotherapy compli-
ance, whereas minor symptoms may be acceptable to
the patient. In general, the convention 1 — mild, 2 -
moderate, 3 = severe and 4 - life-threatening is aimed
for. Although the patient may be able to self-report
side-effects in this way, such a method may be too sub-
jective for use by observers such as nurses and physi-
cians.
In 1988 a new proposal for a toxicity grading system
was put forward by four American oncology groups
ECOG, SWOG, CALGB and NCCTG who were col-
laborating on an intergroup breast study. With these
Tntergroup Toxicity Criteria' as a basis for discussion,
the National Cancer Institute (NCI) of the United
States organized a consensus meeting of all the US
cooperative groups, resulting in the development of the
'Common Toxicity Criteria' (CTC). In this new coding
system nausea and vomiting are graded separately and
treatment no longer plays a role in the coding process
(Table 1). To make the coding more objective for an
observer, the gradations of nausea are linked to lack of
appetite and the degree of vomiting is related to the
number of vomiting episodes [10,11].
Although the three variables of nausea, vomiting and
anti-emetic treatment have now been separated, there
is still a problem when prophylactic anti-emetics are
used. The score given will, to some extent, reflect the
efficacy of the anti-emetic schedule. However, as long
Transient vomiting Vomiting requiring Intractable vomiting
therapy
Intake significantly No significant intake
decreased, but can eat
2-5 episodes in 6-10 episodes in >10 episodes in 24
24 hours 24 hours hours requiring
parenteral support
as provision is made on the case report forms, data on
Downloaded from https://academic.oup.com/annonc/article-abstract/5/2/113/129806 by guest on 09 November 2019
anti-emetic treatment will provide additional informa-
tion on the emetogenicity of the anti-cancer drug. Now
that the 5-HT3 receptor antagonists are widely avail-
able to provide quick and effective anti-emetic control,
the side effects of nausea and vomiting are becoming
less distressing and may no longer be dose-limiting.
This paper reports an evaluation of the Common
Toxicity Criteria carried out at the Netherlands Cancer
Institute. Patients were asked to record their own ex-
periences of nausea and vomiting on detailed question-
naires. The patients' responses were considered the
'gold standard' for assessing the performance of the
WHO and CTC coding systems used by research
nurses and doctors. The WHO and the CTC coding
systems were evaluated sequentially rather than simul-
taneously. This was done in order to avoid contamina-
tion, i.e., the possibility of the new (CTC) system
(subconsciously influencing the nurses' and physicians'
use of the old (WHO) system.
Patients and methods
From March 1989 to April 1991 74 in-patients undergoing emesis-
inducing chemotherapy were invited to participate in the study by
completing a questionnaire on side effects 24 hours after each drug
administration and at the end of one course of chemotherapy.
The Dutch language questionnaire used for this study was one
which had been designed and tested extensively by the Anti-Emetic
Workgroup of The Netherlands Cancer Institute [12) (Table 2).
Questions on the severity of nausea and vomiting were posed in the
form of a 4-point Likert-type scale. Other quesions concerned the
frequency and duration of gastro-intestinal toxicity and the use of
concomitant medication. Patients were asked to complete 2 ques-
tionnaires during one course of chemotherapy. The first was col-
lected 24 hours after the chemotherapy and the second at the end of
the chemotherapy course. The study was confined to in-patients to
facilitate the collection of completed questionnaires.
Research nurses interviewed the patients and coded gastro-intes-
tinal toxicity on the morning of discharge following the chemother-
apy. Physicians interviewed the patient and coded toxicity at the end
of the course, upon re-admission for the subsequent course of treat-
ment This data collection schedule reflected the usual routine on
the chemotherapy ward. The patients had completed their question-
naires independently and had submitted them to the nursing staff
before the interviews with the research nurse and physician. For the
first half of the study nurses and physicians used the WHO toxicity
scale, followed by the CTC for the second half.
Data on patient characteristics, anti-cancer treatment and concc-
 115

Table 2. Patient questionnaire.
Nausea
1) Completed after chemotherapy: 'Looking back over the last 24
hours of the chemotherapy"
Severity Were you nauseated after
receiving chemotherapy?
No. hours For how much of the last
24 hours were you nau-
seated?
tends toward +1.0 and, in the contingency tables drawn up to illus-
trate the results, the frequencies concentrate along the diagonal.
Results
O not at all
O a little bit A total of 74 patients were enrolled in the study (35 for
O quite a lot the WHO evaluation and 39 for the CTC evaluation).
O very much Eight patients were inevaluable because they did not
O not at all return to the hospital for a subsequent course of
O less than 2 hours chemotherapy (i.e., the questionnaire and observers'
O 2-4 hours ratings at the end of the course were missing). For the
O longer than 4 hours 66 evaluable patients an occasional questionnaire item
or physician's code was missing.

Downloaded from https://academic.oup.com/annonc/article-abstract/5/2/113/129806 by guest on 09 November 2019

2) Completed at next visit 'Looking back over the last course of Description of the 66 evaluable patients is given in
chemotherapy' Table 3, indicating that the patient sample was a cross-
section of daily practice, heterogeneous in terms of
No. days For how long were you O not at all both tumor type and emetic drugs used. Half of the pa-
nauseated? O 1 day tients were just beginning their chemotherapy and half
O 2-3 days
O 4-7 days
had received 3 or more courses. All except 2 patients
O more than a week had received prophylactic anti-emetic therapy, which
was usually a cocktail of the drugs metoclopramide (2
mg/kg body weight), lorazepam (1.5 mg/m2) and dexa-
Vomiting methasone (10 mg) given over a period of 24 hours.
1) Completed after chemotherapy: 'Looking back over the last 24 The degree of concordance between nurses' and
hours of the chemotherapy5 physicians' WHO and CTC ratings and patients' ques-
tionnaires is summarised as percentage agreement and
Severity Did you vomit after O not at all as gamma statistic in Table 4.
receiving chemotherapy? O a little bit Concordance between the nurses' WHO ratings and
O quite a lot
O very much the patients' answers to the 6 questions about nausea
No. episodes How often in the last 24 O not at all
Table 3. Patient characteristics.
hours have you vomited? O once
O 2-5 times WHO CTC Total
O 6-10 times
O more than 10 times Sex
Male 11 20 31
2) Completed at next visit: 'Looking back over the last course of Female 21 14 35
chemotherapy' Age (years)
Median 44 47 44.5
Minimum 20 19 19
No. days For how long did you O not at all
Maximum 60 70 70
vomit? O 1 day
Tumor type
O 2-3 days
Lung 0 10 10
O 4-7 days
Osteo/soft tissue sarcoma 4 6 10
O more than a week
Melanoma 4 4 8
Breast 12 1 13
Cervix 0 1 1
Ovarium 3 4 7
mitant medication were collected by data managers from the patient Testis 8 5 13
chart. Unknown primary 0 1 1
The data analysis focused on three sets of comparisons: Lymphoma 1 2 3
- comparison of the patients' answers with the nurses' WHO Chemotherapy regimen containing
and CTC codes, Adriamycin 11 4 15
- comparison of the patients' answers with the physicians' Cisplatin 10 5 15
WHO and CTC codes, Ifosfamide 2 14 16
- comparison of the nurses' and physicians' codes. Fotemustine 4 3 7
The degree of concordance between patients' ratings and the Carboplatin 4 5 9
physicians' and nurses' WHO and CTC ratings was expressed in Other 1 3 4
terms of percentage agreement across the diagonal line (% agree- Course number
ment). Gamma statistic was used to assess statistically the associa- 1-2 17 16 33
tion between the ratings provided by the patients with those of the 3-4 9 15 24
nurses and physicians [13]. The gamma statistic (G) can range be- >4 6 3 9
tween -1.0 and +1.0.
total no. patients 32 34 66
With higher levels of concordance between rates, the gamma
116

Table 4. Nurses' and physicians' WHO and CTC coding compared A comparison of the nurses' and physicians' ratings
to patients' questionnaires expressed as percentage agreement (%) per patient were made in order to examine whether the
and gamma statistic (G). nurses and physicians agreed with one another when
Patients Nurses Physicians
using either the WHO or the CTC system. The con-
cordance of nurses' and physicians' WHO grades was
WHO CTC WHO CTC relatively poor (46% agreement) reflecting, as already
stated, that the nurses consistently used grade 3 where-
% G % G % G % G as the physicians did not. When using the CTC the con-
Nausea
cordance was better: 58% agreement for nausea, G -
+0.64, and 61% for vomiting, G - + 0.67.
Severity 10 - 52 +0.74 16 +0.26 37 + 0.32
No. hours 47 20 52
Table 4 shows that, when using the CTC, G was
43 - +0.71 -0.20 + 0.63
No. days" 32 -0.05 52 +0.68 slightly higher for the nurses than for the physicians for
both nausea and vomiting. The nurses' assessment of
Vomiting acute nausea and vomiting was better than the physi-

Downloaded from https://academic.oup.com/annonc/article-abstract/5/2/113/129806 by guest on 09 November 2019

Severity 9 - 78 +0.97 12 -0.19 59 + 0.67 cians' assessment of (overall) toxicity.
No. episodes 31 - 79 +0.97 27 -0.13 58 +0.70
No. days" 23 0 69 + 0.90
Discussion
" The nurses scored 24 hours after chemotherapy and so could not
take into consideration the duration of nausea or vomiting.
Validity

and vomiting ranged from only 10% to 48% for nausea
and 0% to 31% for vomiting, not reflecting the variety
of the patients' experiences. The reason for this was
their convention of using '3' whenever prophylactic
anti-emetics were given. G was not calculated in these
cases because the problem was obvious from looking at
the figures.
Although physicians did not adhere so rigidly to the
same convention, the percentage agreement of the phy-
sicians' codes with the patients' experiences was also
low, ranging from 16% to 32% for nausea and 12% to
27% for vomiting. This low concordance was con-
firmed by the values of G: from -0.20 to +0.26 for
nausea and —0.19 to 0 for vomiting.
As the CTC evaluate nausea and vomiting separately
and independently of the variable 'treatment1, the
concordance of the nurses' ratings with the patients'
answers was higher when using the CTC, the percent-
age agreement ranging from 34% to 52% for nausea
and from 66% to 79% for vomiting. Values of G for the
severity and number of hours of nausea were +0.74
and +0.71, respectively. The nurses' ratings were not
relevant with regard to the number of days of nausea
since they were scoring acute toxicity after 24 hours.
For acute vomiting the association between the
nurses' CTC scores and the patients' experiences was
excellent (G =+0.97 for severity and G - +0.97 for the
number of episodes).
The physicians' coding of nausea and vomiting
showed much better concordance with the patients' ex-
periences when using the CTC than the WHO scale.
Percentage agreement ranged from 37% to 52% for
nausea and from 58% to 69% for vomiting. Values of G
ranging from +0.32 to +0.68 for nausea and +0.67 to
+0.90 for vomiting confirmed that the physicians'
assessment of nausea and vomiting showed a much
better association with the patients' experiences when
they used the CTC instead of the WHO criteria.
The patients participating in this study had a range of
experience of nausea and vomiting. When nurses and
physicians used the WHO scale for rating the degree of
toxicity the concordance with the patients' answers was
low: 0% to 48% agreement. The low concordance with
specific questions about the severity, frequency and
duration of nausea and vomiting brings the validity of
the scoring system into question. The fact that the
WHO incorporates three individual variables into one
scale - nausea, vomiting and treatment - makes it diffi-
cult to retrieve information about any one of these vari-
ables. If, when collecting data, each item is addressed
separately, the individual results are not only more reli-
able (concordance of 34% to 79% for nausea and vom-
iting using the CTC), but additional information be-
comes available from the combination of outcomes. To
obtain information about treatment there must be a
separate question on the case report forms.
Reliability
In general the physicians' assessments showed slightly
lower agreement with that of the patients than did those
of the nurses. This may be because the physicians inter-
viewed the patient at the end of the course of chemo-
therapy and that the patients' recall was impaired. As
research nurses are often more directly involved with
the patients during the chemotherapy administration, it
would therefore be advisable to make them responsible
for the grading of acute toxicities such as gastro-intes-
tinal toxicity.
Self-assessment questionnaires
Probably the most reliable and most valid assessment
of toxicity comes for the patient him/herself. However,
we discovered during this study that the collection of
questionnaires was not as easy as had been expected.

Eight of the 74 enrolled patients were not evaluable be-
cause questionnaires were missing, and for the remain-
ing 66 patients individual items had often not been
completed. The distribution and timely collection of
questionnaires required careful supervision. Several
self-assessment questionnaires have been developed [4,
6]. The MANE scale developed and well-validated by
Morrow [6] asks separate questions about the frequen-
cy, severity and duration of nausea and vomiting. Both
this and the patient questionnaire used in this study
would be suitable self-assessment tools if warranted.
Observer-rated measures
Morrow [6] acknowledges that observer-rated meas-
ures are often necessary alternatives for scoring the
side effects of chemotherapy. In this case he recom-
mends that nausea and vomiting be assessed separately,
that vomiting be linked to emetic episodes or emetic
volume and that nausea be linked to more objective
measures such as food intake or appetite change, if fea-
sible. This view lends direct support to the CTC meth-
od of coding nausea and vomiting.
Since the nurses' ratings showed a better concord-
ance with patients' experiences than those of the physi-
cians, we recommend that they be responsible for the
coding of these acute toxicities, wherever possible. The
team of research nurses is usually small and stable
which means that the coding system can be more con-
sistently implemented.
Research nurses' preference
When the research nurses were asked which coding
system they preferred, they recommended the CTC be-
cause they found that appetite and number of vomiting
episodes were straightforward, objective parameters
and because they felt that nausea and vomiting were
side effects that should be assessed separately. They
were clearly dissatisfied with the loss of sensitivity of
the WHO scale when prophylactic anti-emetics were
given.
Impact of the study
The Early Clinical Trials Group of the EORTC has al-
ready adopted the CTC for the scoring of toxicity in all
new phase I and n trials. Case Report Forms have been
redesigned to incorporate the new grades for ease of
completion. Evaluation of the use of this scale is on-
going. The results of this study suggest that the wide-
spread replacement of the WHO system with CTC will
yield a high degree of reliability and validity in the rat-
ings of gastro-intestinal toxicity.
Conclusion
The Common Toxicity Criteria reflect far better than
117
the WHO scoring system the severity, frequency and
duration of nausea and vomiting as experienced by the
patient. The better concordance with the patients' own
experiences - as demonstrated by percentage agree-
ment and the gamma statistic - shows that it is a more
valid measure of nausea and vomiting than the WHO
scale. From the comparison of physicians' and nurses'
codes it also shows better inter-observer reliability. The
nurses' scores for this acute toxicity were better than
those of the physician when using the CTC.
Acknowledgements
Downloaded from https://academic.oup.com/annonc/article-abstract/5/2/113/129806 by guest on 09 November 2019
This study was instigated by the EORTC Study Group
Data Management and was supported by the E.C.:
Europe Against Cancer, Directorate XII: 4th Medical
and Health Research Programme.
References
1. WHO Handbook for reporting results of Cancer Treatment.
WHO offset publication No. 48 Neoplasma 1980; 20: 37-46.
2. Vantongelen K, Rotmensz N, Van der Schueren E. Quality
control of validity of data collection in clinical trials. Eur J
Cancer Clin Oncol 1989; 25: 1241-7.
3. Pater JL, Willan AR. Methodological issues in trials of anti-
emetics. J Clin Oncol 1984; 2: 484-7.
4. Rhodes VA, Watson PM, Johnson MH. Development of reli-
able and valid measures of nausea and vomiting. Cancer Nurs-
ing 1984; 7: 33-41.
5. Lazlo J. Methods for measuring clinical effectiveness of anti-
emetics. In Lazlo J (ed): Anti-emetics and Cancer Chemother-
apy. Baltimore: Williams & Wilkins 1983.
6. Morrow GR. The assessment of nausea and vomiting: Past
problems, current issues, and suggestions for future research.
Cancer 1984; 53: 2267-78.
7. Cotanch PH. Measuring nausea and vomiting in clinical nurs-
ing research. Oncol Nurs Forum 1984; 11: 92-4.
8. Seipp CA. Adverse effects of treatment in cancer. In De Vita Jr,
Hellman S, Rosenberg SA (eds): Principles & Practice of On-
cology. Philadelphia: JB Lippincott & Co. 1985; 2007-13.
9. Casey MP, Burish TG, Brenner DE. Delta-9-tetrahydrocanna-
binol in cancer chemotherapy. Research problems and issues.
Ann Intern Med 1983; 99:106-14.
10. NCI Common Toxicity Criteria. National Cancer Institute,
Division of Cancer Treatment 1988.
11. Wittes RE. Manual of Oncologic Therapeutics. Philadelphia:
JB Lippincott & Co 1989; 627-32.
12. Van Dam FSAM, Sneeuw KCA, Hilton A. Monitoring side-
effects of chemotherapy and anti-emetics by nurses (in pre-
paration).
13. Siegel S, Castellan NJ jr. Non parametric statistics for the be-
havioural sciences; 2nd ed. NcGraw-Hill 1988.
Received 19 May 1993; accepted 22 September 1993.
Correspondence to:
H. R. Franklin, B.Sc.
The Netherlands Cancer Institute
Antonie van Leeuwenhoek Huis
Plesmanlaan 121
1066 CX Amsterdam
The Netherlands