Risk Analysis, Vol. 22, No.

4, 2002

Commentary

Are Received Risk Perception Models Alive

and Well?

Lennart Sjöberg∗

Two recent papers on risk perception models are discussed. In these papers, quantitative
analyses are presented of risk perception in relation to risk characteristics as specified in
the Psychometric Model, and to cultural biases according to Cultural Theory. This comment
points out that the data quality of these two studies is doubtful, with a very small convenience
sample and a very low response rate. More importantly, the analyses show the same low levels
of explained variance of risk perception as other researchers have found previously, but the
authors still draw optimistic conclusions from their data. Such conclusions are unjustified.

Two papers by Marris and co-workers were 1. DATA QUALITY

recently published in this journal.(1,2) The researchers
had obtained data on risk perception of 13 hazards
from a small convenience sample (N = 129, response
rate 30%), as well as ratings of the hazards on a tradi-
tional set of scales developed by Fischhoff et al.(3) and
basis for the Psychometric Model of risk perception.
They also obtained data using a set of items intended
to measure worldviews or cultural biases according
to Cultural Theory(4) in the tradition established by
Dake.(5,6)
The papers ended with quite optimistic conclu-
sions regarding the usefulness of the two models for
explaining perceived risk. However, their data clearly
warrant a much more critical attitude with regard to
these two dominating risk perception models, as oth-
ers have observed. To support this statement I will
give some detailed comments on the two papers, fol-
lowed by some more general remarks and relate the
findings of Marris et al. to other published work. Fi-
nally, I will reach a conclusion as to the value of the
received risk perception models—Are they really as
alive and well as implied in the work by Marris et al.?
∗ Center
for Risk Research, Stockholm School of Economics,
Sweden.
It is important to comment on data quality, espe-
cially since risk perception work often has used quite
relaxed standards in this regard;(7) one example is a
study where a survey with a response rate of 23.7% is
reported.(8) The two papers by Marris et al. are based
on a very limited data set and a fragile empirical basis:
a small convenience sample of residents of Norwich,
England. The response rate was very low (30%). The
only reason given in the papers for the failure to ob-
tain a more representative sample was the length of
the questionnaire. However, the questionnaire does
not seem to have been unduly long, nor does length
of a questionnaire have a very strong effect on re-
sponse rate,(9) contrary to widespread belief. Are peo-
ple in England especially unwilling to take part in sur-
vey studies and, if so, why? Dillman has successfully
tackled this problem in the United States, where the
public may not be less difficult to persuade to partic-
ipate than in England.(10) Much longer risk percep-
tion questionnaires than the one used by Marris et al.
have been used with random samples in Sweden, and
we have obtained response rates of the magnitude
60–70%, no less than the rates claimed by commer-
cial polling firms for in-home interviews. Of course,
it may be meaningful to work with a nonrepresenta-
tive sample that covers a large range of variation in

665 0272-4332/02/0800-0665$22.00/1 
C 2002 Society for Risk Analysis
666 Sjöberg

demographic variables, but there should be caution

as to the generalizability of the findings. In particular,
care should be taken not to generalize from a sam-
ple of students to the population of a nation, such as
has happened in a study where inferences about the
French public were made on the basis of data from a
small group of graduate students.(11)

2. THE PSYCHOMETRIC MODEL

This model asserts that perceived risk is a func-
tion of a number of risk attributes such as voluntari-
ness, new versus old risk, etc. Many critical points
can be raised about this model; I discuss it more fully
elsewhere.(12)
The initial papers that tested the model did so
on the basis of average ratings of a fairly large num-
ber of hazards, and obtained strong correlations be-
tween mean perceived risk and mean ratings of the
risk attributes. In early papers, very high proportions
of explained variance were cited in support of the
model.(13) Currently, this kind of misleading result
is still cited as support,(14) for example, in one study
where work was cited that allegedly showed that al-
most 90% of the variance of perceived risk was ex-
plained by the model.(15) The high levels of expla-
nation occur only when average ratings are analyzed
across hazards. Modeling of averages in this way says
nothing about individual variation in risk perception,
nor about intra-individual perception processes. In
what way such models constitute valid information
for policymakers is unclear.
When individual data rather than averages are
used, and each hazard is analyzed in a separate re-
gression model, the proportion of explained variance
is typically 20–25%, taken as an average over a set of
hazards.(16,17) This is also what is implied by the data
analyzed by Marris et al. For the three different defi-
nitions of perceived risk, the proportion of explained
variance was, respectively, 23.9%, 19.8%, and 29.9%.
It varied, apparently largely randomly, between 6%
and 50% across hazards and risk definitions. The au-
thors state,(1) however, that:
the “psychometric paradigm” does appear to be an ef-
fective tool for predicting risk perception. Even using
individual scores, as much as 50% of the variance in risk
perceptions could be explained. (p. 311)
Indeed, but it could also have been stated that as lit-
tle as 6% could be explained in one case, and in many
cases less than 20%. To give a fair description and ba-
sis for conclusions, one should use not the maximum
Fig. 1. Plot of mean correlations of individual analyses against cor-
relations among means (aggregate analyses), nine risk characteris-
tics from the Psychometric Model.
in a distribution of explained variances, but a statistic
that is less misleading, such as the mean. As already
noted, the mean explained variance is well in line with
other published work.
Marris et al. did carry out an interesting analysis in
their study. They investigated the relationships among
the risk characteristics, at the level of single respon-
dents, across the 13 hazards. This means, of course,
that the individual analyses are quite affected by ran-
dom variation,1 but they reported mean correlations
for their whole sample, as well as the usual correla-
tions among the mean ratings. They discussed these
findings, noting some similarities and discrepancies,
and concluded:
while the aggregate analysis may overstate the size of
some of the correlations, it is still generally reflecting
the average of individual correlations. (p. 309, emphasis
added)
It is unclear what is exactly meant by the assertion
that the aggregate analysis “reflects” the average of
individual correlations. Be that as it may, it is clearly
misleading to imply that the two sets of correlations
are comparable. See Fig. 1, where I have plotted these
correlations.
1 The small sample size (N = 13 hazards) means that correlations
will vary greatly, but not that they are underestimated systemati-
cally, as Marris et al. argue.
Perspectives and Commentary
Clearly, the structure of the correlation matrices
is much the same, a very interesting finding in itself,
but just as clearly there is a very consistent drop (to
about half) in correlations when going from the ag-
gregate to the individual level. A drop by 50% of a
correlation means that a level of explanation of, say
50% (correlation 0.7) drops to a mere 10% (correla-
tion of 0.35). This is even stronger than the decrease
of the explanatory power of the model observed for
inter-individual variation, which is about one-third of
the power observed for aggregate analysis.
3. CULTURAL THEORY
The second paper deals with Cultural Theory
and also with relationships between Cultural Theory
scales and Psychometric Model scales;(2) however, I
will not comment on the weak relations found in this
respect.
First, the authors attempted to group their re-
spondents in the four types suggested by the theory:
egalitarians, individualists, hierarchists, and fatalists.
As to be expected, because behavioral data virtually
never lend themselves to neat categorizations, they
failed in this attempt. Only one-third of the sample
could be assigned to the hypothesized types. Curi-
ously, the authors concluded:
these results suggest that cultural biases are not innate
attributes of individuals which can be measured by us-
ing the kind of questionnaire items listed in Table I. (p.
639)
“Innate” means, according to the main definition
given by Webster’s Ninth New Collegiate Dictionary,
“existing in, belonging to, or determined by, factors
present in an individual from birth.” Whoever be-
lieved that cultural biases would be “innate”? The
point is not a minor one, because Marris et al. return
to the concept of “innate” in their paper.
The explanatory power of the cultural bias scales
was, on the average, 3% (see their Table II). This
is at the same level as obtained in other European
studies;(17–19) work on U.S. samples has produced
somewhat higher values.(6,20)
So much for the relationship between cultural
biases and risk perception. It has been found to be
very low, lower than the level of demographics and
risk perception. However, the theory also posits that
there is a relationship between adherence to social-
specific patterns of social relationships and cultural
biases. Marris et al. report no results on this issue,
but such was pursued in a recent study in which data
667
was obtained from a random selection of the Swedish
population (753 respondents, 67% response rate). In
this study,(21) we obtained data on risk perception,
group/grid measures of “adherence to specific pat-
terns of social relationships,” and cultural biases.
According to Cultural Theory, the four combi-
nations of high versus low group and grid should be
characterized as follows:
r High group, high grid: hierarchy.
r High group, low grid: egalitarians.
r Low group, high grid: fatalists.
r Low group, low grid: individualists.
The hierarchy, egalitarian, individualist, and fatalist
scales correlated, respectively, 0.43, 0.11, 0.26, and
0.11 with the grid scale but were around zero with
the group scale.
The sample was split at the median of group and
grid, respectively, and these variables were used as
independent variables in two-way ANOVAs with the
four worldview scales as dependent variables. Grid
was significant throughout; group was not—not in any
case. No interactions were significant. High grid-low
group gave the highest value in three cases, high grid-
high group for the hierarchy scale. Hence, in that case,
the theory had some predictive power, but in all other
cases the data looked as if the subjects were fatalists.
(By the way, fatalism was the least popular world-
view.) The relatively high correlation between grid
and hierarchy may be due to some semantic overlap
between the items. Be that as it may, the theory was
not supported by the present results. It can be added
that the study also did not provide any sizable corre-
lations between worldviews and risk perception data.
Marris et al. argue that a pattern of correlations in
accordance with a theory is the most important type
of finding and, apparently, that strength of relation-
ship is an inconsequential or at least a peripheral as-
pect.2(2) However, weak correlations can easily arise
due to the effect of confounding factors, such as demo-
graphic data. In addition—and more importantly—a
theory that explains only a very small share of the
phenomenon it sets out to account for is of marginal
interest. After all, we do wish to explain as much as
possible of the true variance of risk perception. In
particular, for practical purposes, it would seem to be
quite important to have a powerful, not merely signif-
icant, set of predictors.
2 However, in their first paper(1) they do stress that size of the
correlation is important, not only its statistical significance.
668
Marris et al., having found that cultural biases ex-
plain almost nothing of the variance of perceived risk,
turned to a hypothesis-testing strategy. They tested
all their correlations between the biases scales and
risk perception data (208 correlations) and found that
37% (77) of them were significant at the 0.05 level.
They state that only 11 would have been expected by
chance. But that is true only if the 208 tests are in-
dependent, which they clearly are not. Hence, they
have not demonstrated nonrandomness of their re-
lationships between bias scales and risk perception
data. Be that as it may, they then go on and report
significance for single combinations of risk and bias.
Their Table IV gives a summary of significance-testing
results, but no correlation values are reported. (This
is similar to Dake’s earlier strategy of reporting only
significant correlations and simply not giving any in-
formation about the other data.(5) In this way, one
capitalizes on chance.) In addition, it would seem that
their Table IV is a compilation of several sets of signif-
icance testings and that only the “positive” results are
reported. What that does to the statistical properties
of their alpha levels is not transparent and the issue is
not discussed in the article.
Marris et al. dismiss my earlier analyses of the
relationships between risk perception data and bias
scales because they were based on multiple regres-
sion analyses between the bias scales and perceived
risk. It is true that Cultural Theory makes specific pre-
dictions, but I used the multiple regression analyses to
give an upper bound to the strength of relationships. If
the bias scales explain only 5% of the variance of per-
ceived risk in multiple regression, one of them cannot
in isolation explain more.
Now, given the very weak findings, what—if any-
thing—can save the theory? One strategy is to per-
form qualitative case studies. One can always hope to
find single individuals who appear to function accord-
ing to the theory. This is true in clinical psychology as
well, to take another example, and little seems to be
resulting from such an approach. In medical research,
a similar warning against reliance on single case stud-
ies was issued in an amusing paper by Kline,(22) who
called this strategy “individualising” and grouped it
with other methods for dodging negative conclusions
on the basis of empirical evidence. It is just too easy to
bend and twist the “evidence” to fit the investigator’s
desires, who often has a vested interest in demonstrat-
ing the truth of his or her theory.
Another strategy is simply to deny that the theory
was supposed to explain risk perception in the first
place. Given published articles with this very purpose,
Sjöberg
it is a strange claim, but nonetheless Marris et al. write
that “cultural theory does not really claim to explain
such abstract ratings of risk” (p. 645).
A final, and common, way out is to state that bet-
ter measures of cultural biases would yield stronger
results.(23) Yes, provided that the theory holds. The
logic of the argument is circular. It can be tested only
by means of operationalizing such improved measures
if the theory holds.
4. FUTURE WORK
Marris et al. conclude their article on the Psy-
chometric Model by stating their belief that “a more
thorough analysis” of the psychometric data would
yield better understanding of risk perception. This
may sound reasonable, but is, in reality, very uncer-
tain. It is more likely that new information is needed
and that no statistical twisting and bending of the data,
however sophisticated, will solve the model as long as
powerful explanatory variables simply have not been
measured.
If the Psychometric Model and Cultural Theory
both fail, what is a more successful explanation of
risk perception? Fortunately, we are in a position to
answer that question for the nuclear-waste hazard.(24)
Powerful models can be constructed on the basis of:
r Attitude to nuclear power.
r Risk sensitivity (i.e., a tendency to rate all risks
as large).
r Specific nuclear and radiation risk factors.(25)
r The tampering-with-nature factor,(26) and mo-
ral aspects of risk.(27,28)
It is possible to construct a regression model, using
these explanatory factors, which accounts for about
60% of the variance of risk perception. Taking into
account that there is an error component in the risk
perception measurements, this is probably quite close
to the ceiling, i.e., almost all the true variance is
explained.(25) It would be interesting to investigate
the generality of these findings. Such research would
probably take us to a very different territory from
that of the Psychometric Model and Cultural Theory,
but that is probably just what is needed. In addition,
the further consequences of perceived risk need to
be analyzed. A wealth of data suggests that it is not
perceived risk per se that is most important for policy
attitudes, but the severity of the consequences of an
adverse event.(29,30)
I conclude that the received models are not alive
and well—they have been found to be very weak,
Perspectives and Commentary
explaining only a minor share of perceived risk. Rad-
ically new approaches are needed.
ACKNOWLEDGMENTS
This is a study within CEC project RISKPER-
COM (Contract FI4PCT950016), also supported by
the Swedish Council for Planning and Coordination of
Research (FRN), the Swedish Council for Humanistic
and Social Science Research (HSFR), the Swedish
Nuclear Power Inspectorate (SKI), and the Swedish
Radiation Protection Institute (SSI).
REFERENCES
1. Marris, C., et al. (1997). Exploring the “psychometric
paradigm”: Comparisons between aggregate and individual
analyses. Risk Analysis, 17, 303–312.
2. Marris, C., Langford, I. H., & O’Riordan, T. (1998). A quan-
titative test of the cultural theory of risk perceptions: Com-
parison with the psychometric paradigm. Risk Analysis, 18,
635–647.
3. Fischhoff, B., et al. (1978). How safe is safe enough? A psy-
chometric study of attitudes towards technological risks and
benefits. Policy Sciences, 9, 127–152.
4. Douglas, M., & Wildavsky, A. (1982). Risk and culture
Berkeley, CA: University of California Press.
5. Dake, K. (1990). Technology on trial: Orienting disposi-
tions toward environmental and health hazards. Ph.D. thesis.
Berkeley, CA: University of California.
6. Wildavsky, A., & Dake, K. (1990). Theories of risk perception:
Who fears◦ what and why? Daedalus, 119(4), 41–60.
7. Boholm, A . (1998). Comparative studies of risk perception: A
review of twenty years of research. Journal of Risk Research,
1, 135–164.
8. Flynn, J., Slovic, P., & Mertz, C. K. (1996). Final report. City
of Portland earthquake risk survey. Report. Eugene, OR:
Decision Research.
9. Heberlein, T. A., & Baumgartner, R. (1978). Factors affecting
response rates to mailed questionnaires: A quantitative analy-
sis of the published literature. American Sociological Review,
43, 447–462.
10. Dillman, D. A. (1991). The design and administration of mail
surveys. Annual Review of Sociology, 17, 225–249.
669
11. Karpowicz-Lazreg, C., & Mullet, E. (1993). Societal risks as
seen by the French public. Risk Analysis, 13, 253–258.
12. Sjöberg, L. (1999). The psychometric paradigm revisited. Pa-
per presented at annual conference, organized by Royal Sta-
tistical Society. University of Warwick.
13. Slovic, P., Fischhoff, B., & Lichtenstein, S. (1979). Rating the
risks. Environment, 21( 3), 14–20, 36–39.
14. Pinkau, K., et al. (1998). Environmental standards. Scientific
foundations and rational procedures of regulation with empha-
sis on radiological risk management. Boston, MA: Kluwer.
15. Jungermann, H. (1982). Zur Wahrnehmung und Akzeptierung
des Risikos von Grosstechnologien. Psychologische Rund-
schau, 33, 217–238.
16. Gardner, G. T., & Gould, L. C. (1989). Public perceptions of
the risk and benefits of technology. Risk Analysis, 9, 225–242.
17. Sjöberg, L. (1996). A discussion of the limitations of the psy-
chometric and Cultural Theory approaches to risk perception.
Radiation Protection Dosimetry, 68, 219–225.
18. Sjöberg, L. (1997). Explaining risk perception: An empirical
and quantitative evaluation of cultural theory. Risk Decision
and Policy, 2, 113–130.
19. Brenot, J., Bonnefous, S., & Mays, C. (1996). Cultural theory
and risk perception: Validity and utility explored in the French
context. Radiation Protection Dosimetry, 68, 239–243.
20. Sjöberg, L. (1998). World views, political attitudes and risk
perception. Risk—Health, Safety and Environment, 9, 137–
152.
21. Sjöberg, L. (1999). Risk perception in Western Europe.
Ambio, 28, 543–549.
22. Kline, N. S. (1962). Factifuging. Lancet, 1396–1399.
23. Peters, E., & Slovic, P. (1996). The role of affect and world-
views as orienting dispositions in the perception and accep-
tance of nuclear power. Journal of Applied Social Psychology,
26, 1427–1453.
24. Sjöberg, L. (2000). Factors in risk perception. Risk Analysis,
20, 1–11.
25. Sjöberg, L. (2000). Specifying factors in radiation risk percep-
tion. Scandinavian Journal of Psychology, 41, 169–174.
26. Sjöberg, L. (2000). Perceived risk and tampering with nature.
Journal of Risk Research, 3, 353–367.
27. Sjöberg, L., & Winroth, E. (1986). Risk, moral value of actions,
and mood. Scandinavian Journal of Psychology, 27, 191–208.
28. Sjöberg, L., & Drottz-Sjöberg, B.-M. (2001). Fairness, risk
and risk tolerance in the siting of a nuclear waste repository.
Journal of Risk Research, 4, 75–102.
29. Sjöberg, L. (1999). Consequences of perceived risk: Demand
for mitigation. Journal of Risk Research, 2, 129–149.
30. Sjöberg, L. (2000). Consequences matter, “risk” is marginal.
Journal of Risk Research, 3, 287–295.