THE APPLICATION OF RISK ASSESSMENT TECHNIQUES TO PRIORITY SETTING IN PRODUCT SAFETY INVESTIGATIONS

Colin Hooker Ministry of Consumer Affairs Wellington, New Zealand

Summary

Investigating complaints about allegedly hazardous products is one of the functions of the New Zealand Ministry of Consumer Affairs. But because of resource constraints only some complaints can be investigated fully. So cases must be prioritised. The Ministry has developed a risk assessment procedure which provides a relatively nonsubjective way to decide which investigations will proceed.

Background

On the 1 st of March 1987 a new and important piece of consumer protection legislation came into force in New Zealand - the Fair Trading Act. The Act deals with several aspects of consumer protection that were not adequately covered previously. It prohibits misleading and deceptive conduct, false representations and unfair practices in trade. And it also promotes product safety by giving the Minister of Consumer Affairs powers to make mandatory standards, to order a recall or to ban unsafe goods.

The product safety work of the New Zealand Ministry of Consumer Affairs springs directly from its responsibility to administer the Fair Trading Act, and, more particularly, from the Minister's need for advice on the use of the product safety provisions of the Act.

There is no coordinated approach to product safety matters in New Zealand. Product safety policy and complaints are dealt with under a variety of legislation and by a range of government agencies. For example, the safety of electrical appliances is the responsibility of the Ministry of Commerce, food-related safety issues are the responsibility of the Ministry of Health and road safety issues are the responsibility at the Ministry of Transport. All of these jurisdictions are granted by specific legislation.

The Ministry of Consumer Affairs uses the broad powers granted by the Fair Trading Act to cover areas not elsewhere legislated. However these powers can also be used in areas where other legislation exists.

The product safety section of the Ministry monitors and records incidents involving alleged hazardous products. Consideration of these incidents enable us to advise the Minister on the need to use powers granted under the Act. In practice, however, most

incidents are resolved voluntarily so recourse to the Act is rarely needed.

II I I I

Because of resource constraints, the product safety section has never been able to investigate fully all the reports of alleged hazardous products it receives. Accordingly we needed a procedure to help us decide which products would be investigated, and in what order.

We felt that decisions on whether a report would be investigated ought not to rely on a subjective judgement. Instead, we sought a formal method based on sound reasoning and capable of being justified. Accordingly, the Ministry decided to prioritise its investigations using a risk assessment technique.

I I

1 .

The severity of the consequence that might result from an incident involving the product, that is, the degree of injury or the financial cost of the resultant damage;

I I I I I

Risk Assessment: first trial

In 1983, the New Zealand Accident Compensation Corporation (ACC) published a report on risk analysis techniques. The report proposed a method using these techniques to determine appropriate interventions when product hazards had been identified'. It was based on work carried out by the United States Navy and published by William T Fine in 19732 and by G F Kinney and A 0 Wirith in 19763.

The ACC method started with the assumption that an assessment of the risk presented by a hazardous product could be determined from 3 factors:

2.

The time for which the user is exposed to the product; and

I

3. The probability of an Incident occurring.

Each of these factors was assigned a range of numerical values on a logarithmic scale. For example, the severity of the consequence of an incident ranged from a value of 1 if minor first aid was necessary and/or $100 damage was caused, to a value of 100 if there were many fatalities and/or $10 million damage was caused.

I I

The fact that the method was based on work done by the US Navy, and therefore perhaps more relevant to activities in which armed forces are involved, may have been the reason why it was soon found unsuitable for applying to complaints about toys and other consumer products. The problem was the majority of incidents we were investigating fell at the lower end of the injury severity scale and the times of exposure

I I I I

The three numbers were then multiplied together to obtain a risk score for the product.

The method appeared to meet the Ministry's needs for a technique that could be used to prioritise investigations of alleged hazardous products and an attempt was made in mld-1988 to apply it to these investigations.

2

I I I

to the products fell in a relatively narrow range, so the resultant risk scores tended to be tightly bunched. Clearly this procedure was not separating cases as we had hoped and did not help us establish priorities.

The second trial

The Ministry next trialled a risk analysis technique that had been developed in Australia. This had originally been intended for the development of product standards by giving a risk rating to the individual components used in a particular product. These risk ratings were then used to indicate which parts of the product presented unacceptable hazards and therefore needed corrective action. The technique appeared to us to be applicable also to a complete product. not just to a product's components.

The technique again involved the use of 3 assessment factors, but this time they were more relevant for application to consumer products. The factors were the maximum potential hazard severity, the probability of occurrence, and the likelihood of hazard recognition. Each of the three factors was assigned 5 possible numeric levels in the range 1 to 10, and again a risk rating was calculated by multiplying together the three numbers obtained.

We used the method for about a year, and found it to be reasonably satisfactory for priority setting. But there were still problems. A certain amount of subjective judgement was required in deciding the rating to be given to an injury. Is a broken leg "significant" and rated 6, or "severe" and rated 8? We also felt that other factors might be relevant in determining the risk rating for a product. For example, the age of the intended user, or how widely the product was used. And finally, because only certain discrete values were permitted for each of the 3 risk factors the risk ratings again tended to cluster on just a few numbers.

The technique developed by the Ministry

We decided to build on the experience gained from using the 2 methods described above and develop a technique based on risk assessment that would meet the Ministry's needs more precisely. It was decided the technique should:

1 .

give a numerical risk rating that could be used, firstly, to decide whether a product should be investigated and, secondly, to prioritise investigations that were to proceed;

I I

2.

use relevant risk assessment factors;

3.

be simple to use; and

4. be as non-SUbjective as possible.

3

1 the maximum potential injury;

The Ministry worked with a product safety consultant to develop a technique which met these objectives. We decided that the risk factors should be:

2 the probability of the hazard occurring;

3 the likelihood of the hazard being recognised; and

4 the availability of the product.

We chose maximum potential injury as a risk factor because it is clearly an essential factor in the assessment of a product. In fact we felt that it was so important that it should be given a greater weighting in the assessment procedure than the other three factors.

We included hazard recognition because the ability to recognise a potential hazard is likely to reduce the possibility of an injury. For example, an unexpected sharp edge on a kitchen appliance is likely to result in more injuries than a sharper kitchen knife.

I I I I

The probability of the hazard occurring is an important factor because if the probability is zero the hazard does not occur and there is no problem, whereas with a high probability of the hazard occurring a lot of users are likely to get injured.

And we included availability because an item in widespread use warrants more consideration than a product used by only a few people.

I I

We considered but did not include one other factor, the age of the user. In the end we didn't include this factor because of the difficulty in assigning weightings. It is not easy to say whether an injury to an elderly person is more or less serious than the same Injury in a baby.

The Nomograph

I I

Instead of having to multiply together 4 numbers we decided to use a graphical method, a nomograph, to obtain a numerical risk rating. The nomograph is shown in

Ci,..,. Irl"\ 1

. ;.;j .... ,,'- ;.

I I

The four risk factors are represented by 4 vertical lines with benchmark positions for values of each factor. Two separate scales are provided on the right side of the nomograph for determination of the risk assessment. Normally the assessment is carried out with values applied to each of the four risk factors and the risk rating read from the scale at the extreme right, labelled "Final risk assessment". But sometimes information about the availability of the product is not known immediately to the Ministry. In this case the risk rating is read from the other scale, labelled "Initial risk assessment" .

I

4

I I

I

I

I I

We have reduced the need for subjective judgement in two ways. Firstly, the injury severity levels are defined by reference to an internationally accepted rating of injury severity, the Abbreviated Injury Scale, AIS. For convenience a simplified version of the AIS is used by the Ministry. Secondly, a procedure manual provides full guidance on the selection of levels for each of the four risk factors.

The manual allows officers to choose levels for the risk factors apart from those specified. For example, the maximum potential injury can be less than minor or the availability of the product can be between limited and general. So, unlike the two earlier methods tnalled by the Ministry, the risk assessment rating is not limited to a few discrete values

The nomograph is designed to give slightly more weighting to the maximum potential injury than the other three factors, because it is considered to be the most important factor. The nomograph is also designed so that the full range of the final risk assessment is attainable for the kinds of complaints about consumer products that the Ministry receives. In fact. it is possible to visualise hazards which would result in ratings falling outside the 0-90 risk assessment range. This is in contrast with the systems described earlier, in which some ratings would not be achieved by consumer products. This spreading of risk ratings facilitates decision making and priority setting.

Using the nomograph

The nomograph is very easy to use. Two examples have been chosen to demonstrate how a risk assessment is carried out and how the method provides an answer to the question "Which is the more serious of two incidents?"

In the first incident a mother takes a pushchair out of a cupboard where it has been folded for storage. She opens the pushchair, puts her small child in it and secures him with the harness. The child puts his hand over the arm-rest. The mother starts wheeling the pushchair and it suddenly collapses. As it collapses the child's little finger is severed by the scissor effect between the frame and the arm-rest. The pushchair collapsed because the device intended to lock the frame in position had failed due to poor design.

In the second incident Dad is cooking the evening dinner and the children are Sitting at the table. Dad takes toughened glassware plates from the shelf and puts them on the bench. He serves out the dinner and takes the plates across to the table. Suddenly there is a bang as one of the plates explodes. Glass flies all around the room and the children start screaming. One of the children has a cut on the back of her hand where she was hit by a piece of glass.

Figure 2 shows the risk assessment for the pushchair incident. A severed finger is rated a moderate injury and because the collapse occurred due to a design fault the probability of it happening is rated highly probable. A line is drawn on the chart joining these two points, and is extended to the first tie line. The mother is likely to assume the pushchair is erected safely and so recognition of the hazard is rated improbable.

5

The line is continued on the chart from the tie line through the "improbable" point and extended to the second tie line (initial risk assessment). Finally, the availability is rated between "general" and "limited" because although the pushchair is commonly available it is used only by parents with small children. So the line is continued from the second tie line through this point on the Availability axis to the "final risk assessment" line, which it meets at the value of 67. This is the risk assessment for the pushchair.

Figure 3 shows the risk assessment for the toughened glassware plate. Because toughened glassware breaks into small square pellets when it breaks its potential for causing injury is very low. It can cause small scratches, cuts, or even a scald if the item that explodes is a cup containing hot tea or coffee. Injuries such as these are rated minor. We know that in New Zealand there are items of toughened glassware in almost every home, yet explosions are rare. Therefore the probability of the hazard occurring must be rated unlikely.

• • II

II

I

The likelihood of the hazard being recognised is rated improbable and as items of toughened glassware are available in many shops the availability is rated general. Joining these points as we did in the previous example gives a risk assessment value of 28.

I I

So although it might have been difficult initially to decide which of the two examples was the more serious, this risk assessment makes it quite clear that the folding pushchair incident is worse.

I

Application of the technique to the Ministry's work

I I I

The New Zealand Ministry of Consumer Affairs has now been using this technique for about 5 years. A risk assessment is carried out routinely on every report of an alleged hazardous product that the Ministry receives.

The risk ratings are used in two ways. Firstly, the risk rating is an important factor in deciding whether to investigate a complaint. The technique is reasonably objective and so the rating achieved can support a decision whether to proceed with an investigation. Currently the Ministry takes a value of 60, which corresponds to a significant risk, as the cut-off point for this decision. And secondly, the risk rating is used in pncritisinq our investigations. For example, a product with a rating of 80 will take precedence over a product wnn a rating or fU.

I

I

However, the risk assessment rating is not the only factor the Ministry uses in determining whether a complaint will be investigated. For example, if the Ministry is gathering information on a specific product to assess the need for a mandatory standard, and a report of an incident involving that product is then received, a full investigation may be carried out even if the risk assessment rating falls below the trigger level. Similarly there may be political reasons why an investigation should be carried out - the Ministry may have been directed to do so by the Minister or there may be media interest in the product.

I I

I

6

I

I

I

I I

But in every case, full details of the report about the product are entered into our database. In this way we can monitor the number of reports we receive about a product and can commence an investigation at a later date if it is obvious there is real public concern about it.

And even if it is decided not to carry out a full investigation the Ministry will often write to the supplier or manufacturer of the product in question advising that the Ministry has received a complaint about one of their products. This is often sufficient to prompt a manufacturer to alter the product to avoid future problems.

We have found risk assessment to be a valuable tool in assessing and prioritising complaints. The fact that a formal assessment is carried out assists us when we make a decision not to carry out an investigation and this decision is questioned by the complainant or the media. When we write to complainants telling them that the Ministry has decided not to investigate their concerns we always refer to the risk assessment procedure and the need for us to prioritise our work. We believe government decisions such as these are more acceptable to consumers if they know that the decision IS not merely a subjective judgement.

Conclusion

The New Zealand Ministry of Consumer Affairs is unable to investigate all the reports it receives of alleged hazardous products. Accordingly it needs to prioritise these complaints to determine which it will investigate. The risk assessment procedure, developed 5 years ago. is a tool used to set priorities in an objective manner. It has also been valuable on occasions when decisions have needed to be justified. We expect that our resources will continue to be stretched so our work will continue to be pnont.sed. Risk assessment IS an essential part of this process and will continue to be applied.

References

Technical Report No 11, Product risk analysis techniques - Matching the remedy to product risk. Accident Compensation Corporation, Wellington, New Zealand. September 1983.

2 Fine, William T, Selected readings in safety, Mathematical evaluations for controlling hazards, Academy Press, 1973.

3 Kinney. G F and Wirith, A D, Practical risk analysis for safety management, Technical Publication 5865, Naval Weapons Center, China Lake, California, June 1976.

7

+oJ
~ :::
C/) ~
.- E
;.....
CI)
~ CI)
::: ~
.- CI)
u, CI)
~ ~ ......
:::
C/) C)
.... ~
-
~ C/)
C/.)
_.. C)
c r./)
C/)
- ~ E
~
Vi
OK
c ~
C Cc.
0 ..2 ..=
c: >- E c-,
~ s:
>- u ~ C<l' ~ c:;
.!:! E ..2 C<l' o
C; .... c E
;:s 0 e ~ <:.>
t:: E C ~ "':) .~ Q:o C ....
0 ;(
;; <:.> >( ~ 0 ;:;; ~
0:: ~ ...l v:l t.:.l II II I1II I I II I I I II I 1'1" " I' , ( 'I' , "1""1'111 1111111111111 ( t 11 'I 11111111111 )' II , II' I 1111111

o 0 ~ ~ ~ ~ ~ g ~ ~

~
._
-
._
~
i:l
-
....
(::I
:;..
- ~
~ ~
:c
>- -=
<:.> ~
E -=
~ c -=
~
>( -$
t.:.l = "-::l ~
3
~ ~
E c
~ u
::::: :J 0 "-::l
'"
u
c..
Vl
u
~
~
~
-c;
0;;:
~ '-?
.9 c:::
0
~ c
~ u c-,
..2 E ~ ::::I
0 ~
~ C ~ E ;:s
t::
0 ~ >( <:.> ;;
...l [.;J 0:: II I I i [I I I I II I I I [I tit Itt t I [ t I I III I I I [I I I 1\1 I II [1111\11 I I [II " \' I I '\11 II \11 I I II I I I II I I I [II I I I

o ~ 0 0 0 0 0 0 0 0

~ 00 ~ ~ ~ ~ ~ N

<.) <.)

>-.::0 ::0

_ (""l '"

..=.D .D

o~e e

- a. a.

- E E

<:.>

u;:O o :':I Eo-=:

- >

<g

.... o c:::

:s

•
•
•
•
I
I
~I
~
.....
~I
E
~I
c
a) •
E
C/J
C/J
a) •
C/J
C/J
~
~
.o I
.-
~
.........., I
~
"
~ I
CIJ
._
~
I
I
•
I
I
I
I