Actapsychiatr. scand. Suppl.355.

1989:80:131-137
Key words: post-traumatic stress disorder; drop-out;
response rate: methodology.

Importance of high response rates in traumatic

stress research
Lars Weiszth
Division of Disaster Psychiatry, Institute of
Psychiatry, Gaustad, University of Oslo I The Joint
Norwegian Armed Forces Medical Services
(Head: Lars Weisaeth)

ABSTRACT - Non-response and psychological resistance, i.e. degree of unwillingness to undergo the primary examination (screening), and its implications for
estimation of PTSD prevalences was measured in a longitudinal study of 246
employees exposed to an industrial disaster (explosiodfire). Resistance including
refusal, was measured by counting the number of contacts needed in the calling-in
procedure to secure cooperation. Resistance to the primary examination related to
severity of exposure (24.2% in the high exposure versus 6.8% in the medium
exposure and 4.2% in the low exposure group). If those who initially resisted had
been lost to the 7 month follow-up the total response rate would have been 82.870,
with an estimated frequency of high PTSS scores of 15. By increasing the response
rate to loo%, the true prevalence of high PTSS scores increased to 22.4%. The
initial resistance related strongly ( P < 0.001) to the severity of outcome at 7 months.
The potential loss to the follow-up would have included 42% of the PTSD cases, and
64% of the severe PTSD cases would have fallen out, resulting in distorted prevalence rates of PTSD.
The high potential loss to follow-up in the high exposure category would reduce
the predictor value of belonging to that exposure group. The initial resistance in
many who later developed PTSD was found to relate to the psychological defences
such as avoidance against the re-experiences in the acute post-traumatic stress
syndrome. For traumatic stress studies on the after-effects of shock traumas the
implication of findings is that response rates need to be high. For primary and
secondary prevention the implication may be that early outreach must be very active.

Introduction
Judged from the literature, researchers in the
various fields of traumatic stress are frequently
unsuccessful in achieving high response rates.
Sometimes the response,,is even extremely low.
For example, only 32% of rape victims participated after 3 months in a follow-up study (l), only

17% of the parents answered at the second time

point in a longitudinal study of child loss (2),
and only 27% responded at the 14 month followup after a natural disaster (3).
Low response rates may have effects on various research tasks such as the estimation of
prevalences of post-traumatic psychopathology,
such as post-traumatic stress disorder (PTSD),

132

LARS WEISETH

the identification of predictors of post-traumatic psychopathology, and the aetiological analysis of risk factors for the development of
PTSD.
If the follow-up losses are substantial and biased with regard to outcome, the loss will affect
the estimated levels of the rates of the outcome
and may result in considerable distortion of the
actual levels of risks. It has been shown, for
example, in research on bereavement that nonresponders as a group are more adversely affected following a loss than responders (43).
Omission of the non-responders will therefore
produce results that are biased towards satisfactory adjustment. In contrast to the above
Malts follow-up studies of accidentally injured
(6) lost only 5 of the 112 former patients 28
months after discharge from the hospital. The
data he was able to collect on the non-responders
(questionnaires, medical records, social security
data) did not suggest that the losses in follow-up
were biased with respect to outcome.
If the true prevalence rate has been established, low response rates do not pose severe problems any longer, since the prevalence can then be
predicted. However, the psychosocial outcomes
of exposures to various traumas are still largely
unknown; in fact many of the traumatic stress
studies have as their aim to determine the actual
risk of developing psychiatric problems after exposure to trauma.
The lack of response from research subjects
may be related to the intensity of their stress
exposure, resulting in a bias which reduces the
predictive value of belonging to a specific stress
exposure category. If non-response is positively
related to severity of outcome as well as to
severity of exposure, i.e. the two factors characterizing non-response are found in combination, bias will be reinforced and will have
consequences for the aetiological analysis.
The present paper reports findings from a
traumatic stress study where one aim was to
investigate the resistance, i.e. reluctance to
cooperate , including non-response, to a screening for risk cases after an industrial disaster,
and subsequent loss to follow-up investigation.
Because a 100% response rate was eventually
achieved in the follow-up, it was possible to

analyse the effects the potential loss would have

on the prevalence estimates of psychopathology
and predictor validity. The findings on resistancehon-response will be analyzed in relation
to the intensity of exposure to the disaster and
to outcome 7 months post-disaster, i.e. illness
risk.

Material and method

The disaster and the project have been described
in some detail elsewhere (7,8,9). The disaster
produced a typical shock trauma that elicited
post-traumatic stress reactions in the vast majority of the 66 employees with the most severe
exposure (A group) and very frequent but less
intensive reactions in 59 employees with a medium stress exposure (B group), and fewer and
weaker reactions in 121 employees (C group),
who were not directly exposed to the danger, but
experienced the stress of witnessing the disaster,
of rescue involvement and fantasy trauma (the
knowledge that if the explosion had occurred at
another time of day, they would have been victims). These three groups of subjects serve as our
different stress exposure categories.
The cohort of 246 subjects was found to constitute a positive sample of the general population in terms of health before the disaster and
the three groups were comparable. Until the disaster there had been no significant differences
between the three groups as to their response
when called in for regular check-up by the companys health unit, as measured by their having
undergone the health control. In fact 97.9% of
the ABC group had been examined by the companys medical officer within the last 2-year period. Thus, differences after the disaster could
mainly be ascribed to the different intensities of
exposure to the primary disaster stressors (i.e.
those inherent in the disaster impact) that the
three groups had experienced.
In a systematic cross-sectional study the 125
AB persons were physically examined and interviewed (primary examination) as early as possible after the disaster, starting on the second day
and finishing by the 5th week. Thereafter the 121
C-subjects were interviewed (weeks 5 to 10). The
company nurse contacted each of the 246 sub-

RESPONSE RATES

jects, usually by telephone, and asked him or her

to come for a screening health examinationhesearch study. All interviews were carried out by
the author. A personal first follow-up was carried out after 7 months. All were alive at the first
follow-up.
Psychological resistance to the primary examination was measured by counting the number
of contacts needed in the calling-in procedure
to secure cooperation of each person in the
project. The person was given ample opportunity to choose the time and place for the examination. Resistances were scored on a 4 point
scale: None (cooperation achieved at first contact), moderate (two or three contacts needed),
strong (many contacts needed) and complete
(refusal to participate in primary examination).
If presence or absence of resistance could not
be rated, a score of unknown was made. The
resistance could be rated in this way in 61
A-subjects, 55 B-subjects (i.e. in 116 AB-subjects) and in 120 C-subjects.
Most of the AB group were examined at a time
when the researchers still had no detailed knowledge of each persons exact whereabouts at the
time of the explosion that was to determine what
stress exposure category he would belong to.
In the study of the relationship between psychological resistance to the primary examination
and severity of outcome at 7 months the 116 AB
subjects are used since these subjects had been
studied during the same time span.
As a measure of severity of outcome, the
post-traumatic state at 7 months, a sum score
of 30 post-traumatic stress reactions (PTSS-30)
was used, described elsewhere (8). Each stress

133

reaction was scored by the clinical rater on a

4-point scale, 0-3, yielding a PTSS-30 score
with a possible range from 0 to 90; the actual
range was 0-77. Cut-off points at 5 and 21
yielded three categories of the 116 AB subjects
of whom 74 scored from 0 to 5 (low), 16
scored from 6 to 21 (borderline) and 26 subjects had higher scores than 21 (high) (11 of the
26 had scores higher than 45 points). Practically all the latter subjects qualified for clinical
diagnosis of PTSD (8). Fischer-Erwins twoway test was used to calculate chi-square and
p-values.

The response rates at the primary examination

were 90.9% in the A, 98.3% in the B and 100%
in the C group. Thus, the overall response rate of
the ABC population was 97.6Vo. At the 7
months follow-up a 100% response was achieved
in all the three groups. Persons who had been
highly exposed (A group) showed stronger psychological resistance to the primary examination
(24.2%) than the B group (6.8%) (Table 1). In
the C group only 5 persons displayed resistance
(4 moderate, 1 strong, 1 unknown).
Table 2 shows that the degree of resistance at
the primary examination is associated with the
severity of outcome as measured by the PTSS-30
sums 7 months post-disaster. As regards the estimated risk of contracting post-traumatic problems the associated non-response would have
missed 11 subjects or 42.3% of the 26 subjects
with high PTSS-30 sums and diagnosis of PTSD.
Of the 11 subjects who had scores higher than a

Table 1
Intensity and frequency of psychological resistance to primary examination the first days and weeks after an industrial disaster in
high (A) and medium (B) stress exposure groups.
Psychological resistance

to primary examination

Qh

None

45

68.2

51

86.4

8
2
6

12.1

Moderate
Strong
Complete
Unknown

3.0
9.1

24.2%

0
1

6.8%
1.1

1.6

6.8

66

100.0

59

100.0

134

LARS WEISETH

Table 2
Psychological resistance to primary examination among 116 AB subjects exposed to industrial disaster in relation to their
Post-Traumatic Stress Score (PTSS-30) 7 months post-disaster.
7 months PTSS sum score
Psychological resistance
to primary examination

No resistance
Moderate resistance
Strong resistance
Complete resistance (refused)
Sum

Low
(0-5)

Borderline
(6-21)

High
(22-90)

sum

66

15

15

96

74

16

;}

11

l;
7

26

116

20

Chi-square 14.65 DF 2 P<O.OOl***

sum of 45 (not seen from the Table), 7 subjects

or 64% would have fallen out.
By increasing the assumed response rate from
82.8% in the AB group to 100% the estimated
prevalence of the most severe PTSS categories
increases from 15.6% to 22.4% (Table 3).
The detailed study of the resistancehon-response phenomenon gave the following findings:
Individual PTSS-scores on the 7 who refused
completely showed that 4 subjects had 0 scores;
the four stated these reasons for why they had
refused to be examined earlier: Lack of problemsho need for help were overriding in 3 of
them, fear of doctors in one. The latter was not
entirely an irrational fear: For several years this
person had managed to conceal a colour blindness when his vision was tested, so as not to ruin
his maritime career. When these 4 men finally
agreed to participate in the study the reason was
that they appreciated how essential they were in a
research investigation. Two of the 7 non-responders had very high PTSS sums (77 and 59) and
were found to be completely disabled when they
finally agreed to be seen.

In the clinical analysis of the psychological

resistance among the 26 subjects with high
PTSS-30 scores, their resistance was mainly
found to reflect avoidance behaviour, withdrawal, and social isolation.

Discussion
The resistance was both more frequent and
stronger among the severely exposed than the less
exposed, and among those affected by post-traumatic stress reactions 7 months post-disaster than
among the non-affected. The potential loss to the
7 months follow-up represented by those' resisting the primary examination, while leaving an
AB response rate as high as 82.8%, would significantly reduce the actual prevalences of posttraumatic stress problems. The potential loss
would include over 40% of those who qualified
for a diagnosis of PTSD, and more than half of
the severe outcomes would have fallen out. Thus,
the effect of a bias in non-response with regard
to outcome would be to drastically reduce the

Table 3
A longitudinal study of a cohort of victims to an industrial disaster. Estimated frequencies of three levels of post-traumatic distress
at 7 months after the disaster in relation to different response rates.
Level of 7 months post-traumatic symptom score
Response rate

Low

Borderline

High

(070)

(70)

(qo)

(Yo)

96
107
116

(82.8)
(92.2)
(100)

66
69
74

(68.8)
(64.4)
(63.8)

15
16
16

(15.6)
(15.0)
(13.8)

15
22
26

(15.6)
(20.6)
(22.4)

RESPONSE RATES

actual level of psychopathology. The more pronounced resistance and more frequent refusal in
the high exposure group indicate that losses to
follow-up would be higher in the high stress
exposure category.
Finally, in the clinical analysis of the resistance
phenomenon it was found that in the traumatized subjects this resistance related to psychological defences, such as avoidance.
Regarding the research problem of nonresponse, in general, most epidemiological studies emphasize that non-responders are much
more likely to be deviant or disturbed in some
way (10). It is well known from medical research
that people in poor health may more often fail to
respond in epidemiological studies (11). When it
comes to non-response to screening in other
areas of preventive medicine than traumatic
stress, it is known that many persons at risk are
likely to turn down the invitation to utilize health
screening procedures, such as screening for cancer (12). Among the reasons for this non-response to screening are psychological defences,
such as denial of illness or its consequences,
which also contribute to patient delay of diagnosis and treatment in persons who actually suffer from early stages of life-threatening illnesses.
In clinical studies, for example follow-up examinations of previous psychiatric patients, a variety
of factors influence the non-response level and the
characteristics of the losses to follow-up.
In the Scandinavian countries researchers are
traditionally expected to achieve very high response rates, and have managed to do so also in
traumatic stress studies, such as follow-up of
disaster studies (13). Conditions are usually favourable when it comes to tracing persons, and
substantial research resources are often set in to
achieve cooperation from research subjects. Follow-up studies are preferably carried out by personal interview and by researchers who are experienced clinicians. As a result, high response
rates have been secured even in persons who are
usually difficult to follow-up, sometimes shedding new light on the course of illnesses. For
example, by reaching 98% of 306 ex-patients
who had received institulional treatment for paranoid psychosis 5 to 18 years earlier, Retterstsl
(14) showed that the prognosis was better than

135

had previously been expected. In such patient

groups the more healthy may tend to distance
themselves from psychiatry, while the residual
cases may be easier to identify. Thus, non-response relates to good outcome. Prior to that
study, loss to follow-up had contributed to prevalence estimates that were higher than the actual
risk. Opjordsmoens (1 5) personal follow-up
study achieved a 85% response from the same
patient group 22 to 39 years after discharge. At
this last follow-up the non-response group was
found to have more women, poorer social contacts, used health services less, had more personality deviations and had previously been more
avoidant, but the loss to follow-up did not alter
the outcome as regards psychopathology. Among
traumatic stress studies with high response rates,
in the follow-up of accidentally injured, referred
to above (6), 3 of 5 non-responders had a psychiatric disorder when injured and may have represented a selected group of the cohort. But the
non-response did not relate to severe outcome as
in our study.
Thus, judging from the effect on prevalence
estimates resulting from losses to screening and
follow-up of patient populations and of individuals at risk, the estimates may be distorted in
different ways; they may be lower or higher than
the actual prevalence, or be unaffected.
It should be noted that among those who refused to be examined most of those having no
post-traumatic stress reactions at the follow-up
explained their reluctance by the fact that they
needed no help (and the follow-up proved that
they were right). A combined research and interventional project was easier to accept for
them.
The problematic consequences to research of
non-response have not been much focused upon
in traumatic stress research. The reluctance of
some disaster victims to participate in out-reach
programs has been described (16, 17). Many
traumatic stress studies have been carried out on
selected materials, even self-selected volunteers
or samples that are otherwise suspect, and with
low response rates. Unrepresentative materials
and unknown effect of loss to follow-up make
for cautious interpretation of the estimated prevalence of psychopathology.

136

LARS WEISETH

Furthermore, the close relationship between

resistance and the acute post-traumatic stress
syndrome (8) suggests that perhaps the type of
resistance and potential loss to follow-up seen in
our material is of particular concern in studies of
PTSD after severe shock traumas. The biased
potential loss to follow-up would not only distort
the prevalence rate of PTSD by a dramatic reduction, but it would also reduce the predictive
value of belonging to a severe exposure category.
However, it is beyond the scope of this paper to
discuss the various factors that contribute to
resistance.
Our findings may have implications for research on PTSD and on its primary and secondary prevention, involving early detection of and
adequate interventions in risk cases. As regards
prevention, our findings suggest that active and
aggressive out-reach may be necessary in order to
identify risk cases. When it comes to PTSD research our findings indicate that extremely high
response rates are needed if the true prevalence
rate is to be established. The main methodological problem in this study was that the examination of the low stress exposure category (Cgroup) was carried out after the interviewing of
the AB subjects. Passing time may have softened
their resistance, although the more likely explanation is their lower frequency and intensity of
post-traumatic stress reactions, as described elsewhere (7).
To measure resistance by counting the number
of contacts needed to achieve cooperation probably is a valid method. The main reliability problem involved was the difficulty to decide sometimes whether realistic reasons or psychological
resistance caused the failure to agree to an appointment for the primary examination, thus necessitating more contacts. The higher number of
unknown scores in the A and B groups reflects
this problem.
Finally, the assumption that those who resisted
the primary examination would be likely to be
lost to follow-up, can be questioned. It can be
argued that the severe cases of PTSD would have
come to the notice of a clinician sooner or later.
The impression, however, ,,is that PTSD cases are
less frequently encountered in clinical practice
than in systematic research studies.

Conclusion
The findings presented in this work do seem to
indicate that data reported from traumatic stress
studies with low response rates should be viewed
with caution unless it is shown that the nonresponders do not deviate from the responders in
important exposure and reaction variables.
Otherwise, the only correct method of handling
persons lost to follow-up is not to have any
(18).

Because resistance related to severity of exposure as well as outcome, a potential loss to follow-up would reduce the prevalence rates of
PTSD and the predictor value of belonging to
the severe exposure category.

References
1. Binder RL. Difficulties in follow-up of rape victims. Am. J

Psychotherapy 1981:35:42.
2. Videka-Sherman L. Coping with the death of a child: A
study over time. Am. J Orthopsychiatry, 1982:52:688-698.
3. Parker G. Cyclone Tracy and Darwin evacuees: on the
restoration of the species. Br J Psychiatry 1977:130:548555.
4. Clarke M., Williams AJ. Depression in women after perinatal death. Lancet, 1979: April 28: 916-917.
5. Lehman DR, Wortman CB, Williams AF. Long-term effects of losing a spouse or child in a motor vehicle crash.
Personality and Social Psychology,
Journal of
1987:52:218-231.
6. Malt UF. The long-term psychiatric consequences of accidental injury. A longitudinal study of 107 adults. Br J
Psychiatr 1988:153:810-818.
7. Weiszth L. The stressors and the post-traumatic stresssyndrome after an industrial disaster. Acta Psychiatr
Scand 1989:8OSuppl. 355, 25-37.
8. Weiszth L. Stress Reactions to an Industrial Disaster.
Medical Faculty, University of Oslo 1984.
9. Weiszth L. Post-traumatic stress disorder after an industrial disaster. In: Pichot P, Berner P , Wolf R, Than K,
eds. Psychiatry - The State of the Art, Vol 6, New York,
Plenum Press 1985; 299-307.
10. Rutter M. Prospective studies to investigate behaviouralchange. In: Straws JS, Babigian HM, Roff M (eds). The
Origin and Course of Psychopathology. New York, Plenum Press 1977; 223-248.
11. McMahon B, Pugh TF. Epidemiology, Principles and
Methods. Boston: Little, Brown and Company 1970.
12. Magnus K, Langmark F, Anderson AA. Mass screening
for cervical cancer in @stfold County of Norway 1954-77.
Int J Cancer, 1987:39:31l-316.
13. Holen A, Sund A, Weiszth,L. Alexander L. Kiellandkatastrofen: Psykiske Reaksjoner hos de Overlevende.
Forelerbig sluttrapport. Oslo: University of Oslo, Division
of Disaster Psychiatry, 1983.

RESPONSE RATES

14. Retterstel N. Paranoid and Paranoic Psychoses. Oslo: Universitetsforlaget. Ill: Thomas Springfield, 1966.
15. Opjordsmoen S, Delusional disorders, I. Comparative
long-term outcome. Acta Psychiatr scand 1989. In press.
16. Bennet G. Community disaster in Britain. In: Jones K, ed.
The Year Book of Social Policy in Britain 1973. London:
Routledge & Kegan Paul 1973; 150-174.
17. Lindy JD, Grace MC, Green BL. Survivors: outreach to a
reluctant population. Am J Orthopsychiatry, 1981:51:468478.

137

18. Dorn HF. Method of analysis for follow-up studies. Hum

Biol 1950:22:238-248. Quoted p.229 in MacMahon B,
Pugh TF. Epidemiology, Principles and Methods. Boston:
Little, Brown and Company 1970.

Address
Lars Weiseth, M.D., Ph.D.
Professor
Division of Disaster Psychiatry
P.O.Box 39 Gaustad
0320 Oslo 3, Norway