You are on page 1of 9

Australasian Physical & Engineering Sciences in Medicine Volume 28 Number 2, 2005

EDUCATIONAL NOTE

A standard approach to measurement uncertainties for


scientists and engineers in medicine*

K. Gregory1,2, G. Bibbo1,3 and J. E. Pattison1

1
School of Electrical and Information Engineering (Applied Physics), University of South Australia, Mawson
Lakes, Australia
2
Radiation Protection Division, Environment Protection Authority, Kent Town, Australia
3
Division of Medical Imaging, Women’s and Children’s Hospital, North Adelaide, Australia

Abstract
The critical nature of health care demands high performance levels from medical equipment. To ensure these
performance levels are maintained, medical physicists and biomedical engineers conduct a range of measurements on
equipment during acceptance testing and on-going quality assurance programs. Wherever there are measurements, there
are measurement uncertainties with potential conflicts between the measurements made by installers, owners and
occasionally regulators. Prior to 1993, various methods were used to calculate and report measurement uncertainties. In
1993, the International Organization for Standardization published the Guide to the Expression of Uncertainty in
Measurement (GUM). The document was jointly published with six international organizations principally involved in
measurements and standards. The GUM is regarded as an international benchmark on how measurement uncertainty
should be calculated and reported. Despite the critical nature of these measurements, there has not been widespread use
of the GUM by medical physicists and biomedical engineers. This may be due to the complexity of the GUM. Some
organisations have published guidance on the GUM tailored to specific measurement disciplines. This paper presents
the philosophy behind the GUM, and demonstrates, with a medical physics measurement example, how the GUM
recommends uncertainties be calculated and reported.

Key words uncertainty, error, accuracy, GUM, (eg thermometers and blood analysers), also need to
measurement maintain high performance levels. There are examples in
the literature showing that the health care industry makes
widespread use of quality assurance (QA) programs3-5 to
monitor performance of equipment. These QA programs
Introduction usually include a variety of measurements that are
conducted at regular intervals. Performance measurements
The performance tolerances of medical equipment are may also be conducted on new equipment as part of
generally more stringent than domestic and industrial acceptance testing. As medical equipment is subject to so
equipment due to the critical nature of health care. In the many measurements, there is potential for conflicts to arise
case of medical X-ray units where patients are exposed to between the various stakeholders, such as installers,
ionising radiation, the performance levels may be set by operators, purchasers, manufacturers and regulators. Any
legal instruments, such as is done in South Australia1, or by measurements involved in such conflicts are likely to be
published recommendations2. Other medical equipment, closely scrutinised, and the question of measurement
such as those relating to the administration of drugs uncertainty may arise.
(eg balances and flow rate meters), and patient monitoring Prior to 1993, there was no international consensus on
how measurement uncertainties were to be calculated and
*Presented in part at the 29th annual conference of the reported6. The variation in philosophies and calculation
Australasian Radiation Protection Society, Adelaide, SA,
Australia, 24-27 October 2004.
methods caused three main problems, especially for
Corresponding author: Kent Gregory, Radiation Protection laboratories conducting comparisons with other laboratories
Division - EPA, PO Box 721, Kent Town, SA 5071, Australia in different countries6. The first problem was that
Tel: 08 8130 0713, Fax: 08 8130 0777 laboratories had to give extensive explanations of the
Email: kent.gregory@state.sa.gov.au measurement uncertainty calculation method they used
Received: 14 February 2005; Accepted: 14 March 2005 whenever results were reported, so that other laboratories
Copyright © 2005 ACPSEM/EA could make comparisons. The second problem was that it

131
Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005 Gregory et al • A standard approach to uncertainties

was common to keep random and systematic uncertainties Theoretical background


separated, both during calculations and in the final
reporting, even though they were both uncertainties. The Review of statistics
final problem was that some uncertainties were deliberately All measurements have an uncertainty. If a large
overstated in the belief that this was acting conservatively. number of measurements are taken and each measurement
Such practise has the capacity to increase type I and type II value is plotted against the number of times a measurement
errors7 in hypothesis testing (a type I error is when a good value occurs (i.e. a histogram), the end product would be
result is erroneously rejected, while a type II error is when a the measurement’s uncertainty distribution. Uncertainty
poor result is erroneously accepted). A desire to overcome distributions are typically scaled (standardised) so that the
these three problems was the driver for what eventually area under a curve defining the histogram is equal to 1. This
became the Guide to the Expression of Uncertainty in is achieved by changing the x-axis to represent the
Measurement6 (GUM). difference between each measurement value and the mean
In 1993, the GUM was jointly published by the value in terms of the number of standard deviations, and
International Organization for Standardization and six changing the y-axis to represent probability density. Thus,
international organisationsa primarily concerned with the probability of a measurement lying between two values
measurements and standards. The document is therefore is equal to the area under the curve between these two
recognised by organisations internationally, including the points. Most scientists and engineers will be familiar with
Australian National Measurement Institute (NMI). NMI the Normal distribution, also known as the Gaussian
holds most of Australia’s First Level Standards, with the distribution and depicted in figure 1(a), which has a bell-
exception of those relating to ionising radiation, which NMI shaped curve. The Normal distribution is commonly
has delegated to the Australian Nuclear Science and encountered in many types of measurements. The reason
Technology Organisation and the Australian Radiation why the Normal distribution is so common is explained by
Protection and Nuclear Safety Agency. the Central Limit Theorem. The ubiquitous nature of
The GUM is underpinned with an extensive Normal distributions is utilised by the GUM, and is
mathematical basis and has the capacity to deal with most discussed later. Another uncertainty distribution is the
measurement uncertainty problems. This generalist rectangular distribution as depicted in figure 1(b), which
approach comes at a cost; the GUM concepts and applies to measurements that are rounded or truncated (such
methodologies are non-trivial, and may be difficult to as by a digital display). Occasionally, some measurements
comprehend without continuous involvement in are encountered that have uncertainty distributions that are
measurement science at the highest levels (e.g. working at neither Normal nor rectangular, such as those depicted in
institutions such as NMI). Some organisations8-10 have figures 1(c) and 1(d).
published specific guidance on the GUM for various The aim of uncertainty analysis is to estimate the
measurement disciplines. Others11,12 have published uncertainty distribution for a final measurement result. The
explanatory texts on the GUM in general. In particular, the final result’s uncertainty distribution is entirely dependent
National Measurement Laboratory (part of NMI) has on the uncertainty distributions of all factors that influence
published a document12 (the NMI Guide), which explains in
simple terms the philosophy, methods and reporting style of
the GUM for use in applied environments. The NMI Guide
also offers a number of pragmatic approaches that are in
keeping with the philosophy of the GUM, but substantially
reduce the complexity of the calculations.
The NMI Guide is still somewhat complex and requires
some introduction and further explanation before engineers
and scientists adopt the GUM more widely. Indeed, there
appears to be very little application of the GUM by
scientists and engineers in medicine within published
literature. These workers may be making use of simpler
methods13,14 for dealing with uncertainties that were
introduced to them during their undergraduate university Figure 1. Uncertainty distributions are like frequency histograms,
courses. except they are standardised so that the area under an uncertainty
This paper presents a simplified theoretical distribution curve is equal to 1. This is achieved by changing the
background, and uses a medical physics example to x-axis to represent the difference between measurement values
illustrate how to apply the GUM to measurements made by and the mean in terms of the number of standard deviations
scientists and engineers in medicine. (instead of measurement values), and changing the y-axis to
represent probability density (instead of frequency). Uncertainty
a
International Bureau of Weights and Measures, International distributions can take any form, but the most common is the
Electrotechnical Commission, International Federation of Normal distribution (a). The rectangular distribution (b) is
Clinical Chemistry, International Union of Pure and Applied associated with rounding or truncation of readings. Any
Chemistry, International Union of Pure and Applied Physics, and uncertainty distribution, including (c) and (d), may be assigned to
the International Organization of Legal Metrology. an uncertainty component.

132
Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005 Gregory et al • A standard approach to uncertainties

the final result. This may include other measured quantities The uncertainty distribution of the final result is in
(e.g. the measurement of mass and volume to determine most cases, a known shape, due to the Central Limit
density), rounding of readings, or variations in atmospheric Theorem (CLT). The CLT states that when uncertainty
or other conditions. Note that all factors that influence the distributions are combined, the resulting distribution
final result are collectively referred to as uncertainty approximates a Normal distribution. Knowing the shape of
components (a term derived from the NMI Guide). Thus a the final result’s uncertainty distribution reduces the
major task in uncertainty analysis is to characterise the amount of information needed regarding the distributions of
uncertainty distributions of the uncertainty components, so all the uncertainty components. In fact, only three quantities
that they can be combined. for each uncertainty component are needed by the GUM to
In order to characterise an uncertainty distribution estimate the uncertainty distribution of the final result.
exactly, the entire population of measurements would need The first quantity used to describe an uncertainty
to be plotted as a histogram. As this is not possible for distribution is the standard uncertainty, u (or estimated
infinite populations of measurements, a finite number of standard deviation). This is a measure of the spread of the
measurements are sampled instead. From this sample, distribution, with small u values indicative of repeatable
inferences can be made about the infinite population. In the measurements (i.e. close agreement of successive
case of the common Normal distribution of measurements, measurements). The second is the degrees of freedom, ν.
the distribution of a finite sample of such measurements is This quantity reflects the reliability of the value u. For
represented by a t-distribution (also called the Student’s example, a value for u derived from many measurements
t-distribution). A t-distribution is virtually identical to a should have greater influence on the final result than
Normal distribution when a sufficiently large number of another u derived from a smaller number of measurements.
measurements are sampled (such as more than 20 Thus, the higher the value of ν, the more reliable the value
measurements). However, if fewer measurements are of u. The third quantity is the sensitivity coefficient, c.
sampled, the shape of the t-distribution is flatter and Values for c are used to take into account the relative
broader (see figure 2). This shape change reflects the fact sensitivity of the final result to small changes in uncertainty
that with fewer measurements, less is known about the components. Thus if c = 1, an increase or decrease in the
infinite population. The shape of the t-distribution is value of an uncertainty component causes an equivalent
dependent on the degrees of freedom, ν. The value ν equals change in the final result. When c > 1 or c < 1, the
the number of measurements sampled minus the number of uncertainty component has, respectively, a greater and
quantities calculated from the measurements. In most cases, lesser effect on the final result.
the measurements are used to calculate a mean, so ν will
equal one less than the number of measurements sampled. Background to the GUM and NMI Guide
For rectangular distributions, measurement sampling is The GUM and NMI Guide use methods and
usually not required to ascertain the distribution shape. terminology that may be unfamiliar to users of other
Instead, its shape is obtained intuitively; the distribution is uncertainty analysis methodologies. For example, the term
flat with upper and lower bounds defined by, say, the measurand, which describes the quantity or parameter
resolution of the instrument. subject to measurement, is used extensively in the GUM
and NMI Guide. While a detailed justification of why
certain procedures are used is documented in the GUM,
together with derivations of the equations, a brief
background is provided here, together with essential
terminology.
It is important to note that the GUM is not completely
regimented; there remain steps in the process requiring
professional judgment. As such, uncertainty analysis
remains an inexact science, since professional judgment is
subjective. This leads to uncertainty in the uncertainty
estimates. The NMI Guide plays on this fact, and proposes
a number of pragmatic approaches to the GUM process that
theoretically increase the uncertainty in the uncertainty
estimate. However, the NMI Guide argues that in practice
these increases are insignificant compared to the
uncertainty introduced through professional judgment.
There are four main steps in the GUM process, and
these are described in more detail below.
Figure 2. A comparison of the Normal distribution and t-
distributions with 1, 3 and 10 degrees of freedom. When the Step 1: Modelling the measurement
degrees of freedom equals 20 or more, the t-distribution and the The first step in the process is to make what the GUM
Normal distribution may be considered equivalent for the refers to as a model of the measurement. This is a simple
purposes of calculating uncertainties.
task that involves writing an equation that relates the

133
Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005 Gregory et al • A standard approach to uncertainties

measurements cannot be obtained (e.g. a single value from


a table of reference data). With only a single value,
statistics cannot be used to calculate u and ν. Hence, u and
ν are derived using non-statistical approaches (i.e.
professional judgment), some of which have been
simplified by the NMI Guide. The first step in deriving u
and ν in a Type B evaluation is to assign an uncertainty
distribution representative of how the value of the
uncertainty component might vary if more values were
available. The GUM proposes that one of the four
distributions in figure 1 be assigned, with the triangular and
trapezoidal distributions assigned when there is insufficient
confidence to suggest the data do not follow a Normal
(a) distribution, but sufficient information to suggest more
certainty than a rectangular distribution. The NMI Guide
suggests that uncertainty components, such as rounding or
truncation of readings, be assigned a rectangular
distribution, while almost all others may be assigned a
Normal distribution. The NMI Guide suggests that
triangular and trapezoidal distributions are rarely
encountered in the real world, and that more information
should be obtained before assigning such uncertainty
distributions to any measurement.
Having assigned an uncertainty distribution, the next
step is to estimate the range, ±a, (figure 3a and 3b) within
which the true value lies, and the likelihood that the true
(b) value is within this range. If a rectangular distribution was
assigned, a value for a can be chosen such that the range ±a
Figure 3. (a) For a rectangular uncertainty distribution, the value represents exactly 100% of the uncertainty distribution. The
a is selected such that the range ±a includes the true value with standard uncertainty can then be calculated;
100% confidence; (b) For the Normal distribution, the value a is
a for a rectangular distribution (figure 3a) (1)
selected such that the range ±a includes the true value with u=
approximately 95% probability. The estimated standard 3
uncertainty is half the value of a. If a very conservative value is
chosen (a′ ) such that the range ±a′ almost certainly includes the If a Normal distribution was assigned, the NMI Guide
true value (nearly 100%), half the value of a′ will be a poor suggests choosing a value for a such that the range ±a
estimate of the standard uncertainty. represents 95% of the measurements. The standard
uncertainty can then be calculated;
various uncertainty components to the final result (see a for a Normal distribution (see figure 3b)
equation 9 in the example). Modelling the measurement u= (2)
2
serves two purposes; it helps to identify uncertainty
components, and it assists in finding values of c for the It is important not to deliberately increase a beyond
uncertainty components. what is necessary, such as a′ in figure 3b. By deliberately
inflating the value a, the uncertainty calculated for the final
Step 2: Calculating values for u, ν and c result may be unnecessarily large.
The GUM process involves characterising all In order to assign values of ν, the NMI Guide argues that
uncertainty components with these 3 descriptors; u, ν and c. the final result is not particularly sensitive to the ν values
Values for the standard uncertainty u and the degrees of chosen. If that is the case, then it is sufficient to categorise
freedom ν are calculated or assigned following what the each evaluation broadly as either poor, reasonable, good or
GUM refers to as either a Type A or Type B evaluation. It excellent (see table 1), and to assign the corresponding
may be helpful to think of Type A and Type B evaluations value for ν. (Note that the third column in table 1, k, is
as those uncertainties commonly described as ‘random’13,14 explained in Step 4.)
and ‘systematic’13,14 respectively (although this comparison Values of the sensitivity coefficient c may be obtained
is not strictly true). In Type A evaluations, the value u is in one of three ways; (i) by making a small change in the
equal to the estimated standard deviation of the population value of the uncertainty component in the model of the
of measurements, while ν is equal to its degrees of freedom. measurement and calculating the consequent change in the
Type B evaluations are performed when Type A final result, (ii) experimentally changing the value of the
evaluations cannot be performed, such as when there is uncertainty component slightly and observing the
only one value for an uncertainty component and additional consequent change in the final result, or (iii) using calculus

134
Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005 Gregory et al • A standard approach to uncertainties

Table 1. Relationship between measurement quality (as judged Note that equation (5) is simply the linear sum of the
through experience), degrees of freedom (ν) and coverage factor weighted standard uncertainties of the uncertainty
(k) to generate a 95% confidence interval. components. If any uncertainty components happen to be
correlated, it may be possible to alter the measurement
Measurement k for 95% confidence procedure to obtain uncorrelated uncertainty components,
ν
quality interval and thus retain the simpler equation (3). However,
Poor 2 4.30 correlation may be a desirable effect, especially since anti-
Reasonable 8 2.31 correlated uncertainties can reduce the combined standard
Good 20 2.09 uncertainty. The NMI Guide provides a pragmatic approach
Excellent 1000 1.96 to correlations to reduce the mathematical process.
With regard to condition (b), equations (3) and (4) are
the first terms in a Taylor series expansion, and are good
to differentiate the model of the measurement with respect approximations if an approximately linear relationship
to the uncertainty component (this is a common uncertainty exists between the final result and each of the uncertainty
analysis technique14). While it may not be possible to use components. If this is not the case, the next term in the
all three methods to calculate each value of c, at least one Taylor series will need to be added to equations (3) and (4);
method will be.
N N  1  ∂ 2 f  2 ∂f ∂ 3 f 
(6)
Step 3: Calculation of ucomb and νeff ∑∑    + ui2u 2j
 2  ∂xi ∂x j  ∂xi ∂xi ∂x 2j 
Using the u, ν and c values, equations are then used to i =1 j =1
 
calculate the combined standard uncertainty, ucomb and the
effective degrees of freedom of the final result, νeff. where f is the model of the measurement with N
Given two conditions specified below, the equation to uncertainty components.
calculate ucomb is the square root of the sum of the squares Non-linearity may occur when the value of the final
of the standard uncertainties, weighted with the respective result is very small compared to the value of the uncertainty
sensitivity coefficients. Thus for a final result with N components (such as calculating the area of a small
uncertainty components, the combined standard uncertainty rectangle by measuring two sides using a ruler with a large
of the final result is given by: uncertainty), or the uncertainty components vary the final
result in a non-linear manner (such as an uncertainty

N
u comb = (c i u i ) 2 (3) component raised to a power in the model of the
i =1
measurement). The GUM provides more detail on non-
where ui is the ith standard uncertainty, and ci is the ith linearity.
sensitivity coefficient. As mentioned earlier, the CLT suggests that the final
Equation (3) is valid if two conditions are met; (a), result’s uncertainty distribution is a Normal distribution,
each uncertainty component is not correlated with any other but this is only an approximation. However, with the
uncertainty component, and (b), an approximately linear combined standard uncertainty, ucomb, and the effective
relationship exists between each uncertainty component and degrees of freedom, νeff, obtained from the Welch-
the final result for the range of values the uncertainty Satterthwaite formula6;
components are likely to have. For most measurements, if 4
ucomb (7)
both the above conditions are not met, it is usually possible ν eff =
 ( c u ) 4

∑i=1  νi i 
N
to alter the way the measurement is made so that both
conditions are satisfied. If not, the mathematics becomes  i 
more complex. the final result’s uncertainty distribution can be treated
With regard to condition (a), the degree of correlation
mathematically like a t-distribution. In practice, νeff will
is determined by calculating the correlation coefficient,
typically have a value greater than 20.
which can have any value from –1 (anti-correlated) to +1
(fully correlated). Values close to 0 are indicative of
Step 4: State the final result
uncorrelated uncertainty components. To account for
The final result’s uncertainty may be expressed simply
correlation, equation (3) becomes:
as ±ucomb, which represents a range of values within which
u 2 comb = ∑i =1 (ci ui ) + 2∑i =1 ∑ ci c j ui u j r (xi x j ) the true value is expected to lie with approximately 68%
N 2 N −1 N
(4)
j =1+i
probability (assuming νeff≥20). This is the traditional one
where r(xixj) is the correlation coefficient of the two standard deviation or one sigma level. However, the GUM
uncertainty components xi and xj. reports the expanded uncertainty, U, calculated using
In the special case where all uncertainty components equation (8);
are fully correlated, i.e. r(xixj) =1, equation (4) reduces to; U = ucomb×k (8)
N
u comb = ∑ (ci ui ) (5) where U is the expanded uncertainty, and k is the coverage
i =1 factor.

135
Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005 Gregory et al • A standard approach to uncertainties

Table 2. An uncertainty analysis table. Each row represents a different uncertainty component, with values for
standard uncertainty (u), degrees of freedom (ν) and sensitivity factor (c).

Uncertainty components u ν c
Reading 0.0127 mGy 24 1.017 mGy/mGy
Rounding 0.000289 mGy 1000 1.017 mGy/mGy
Cal. Factor 0.025 units 20 1.953 mGy/unit
Temperature 0.50 K 20 0.00668 mGy/K
Pressure 50 Pa 20 -0.0000196 mGy/Pa
Position (vertical plane) 2 mm 20 0.00592 mGy/mm
Position (anode-cathode plane) 2 mm 20 0.00272 mGy/mm
Position (lateral plane) 2 mm 20 0 mGy/mm
The combined standard uncertainty
ucomb = 0.0523 mGy
and effective degrees of freedom:

Values for U can be calculated such that ±U has any Step 3: Calculate the values of ucomb and νeff. For N
desired probability of containing the true value, including uncorrelated uncertainty components, the
the increasingly common12 95% probability. The value k is values of ucomb and νeff are given by equations
selected from tabulated data6,12, given νeff and the desired (3) and (7), respectively.
probability (known as confidence interval, CI). Some Step 4: State the final result. The statement should
typical values of k appear in table 1. mention that the uncertainty was calculated in
Both the final result and U need to be rounded such that accordance with the GUM, give the values of
three objectives are met; namely U contains only significant U and k, indicate the CI used, and ensure the
digits, the final result and U are rounded to the same least final result and U are rounded appropriately.
significant digit, and the rounding process itself does not
introduce a significant uncertainty. Rounding is described
in more detail in the NMI Guide, but the basic steps appear
Example
below.
1. Round the final result and U once only. Consider the measurement of radiation dose from a
2. Round U to 1 significant digit if the first significant mammographic X-ray unit with exposure settings of 28
digit is 5 or more, otherwise round to 2 significant kVp, 20 mAs, with large focus selected, molybdenum
digits. (eg 0.0078 becomes 0.008, while 0.013 is anode, 30 µm molybdenum filtration, at a position 40 mm
unchanged). from the chest wall in the anode-cathode plane, 45 mm
3. Round U upwards unless rounding down will only above the imaging plate in the vertical plane, and centred
change U by a few percent (eg 0.0078 becomes on the remaining axis (i.e. centred laterally with respect to
0.008, while 0.0071 becomes 0.007) the patient).
4. Round the final result to match the significant digits All data for the example are summarised in table 2.
in U.
5. Round a 5 in the final result to the nearest even Step 1. Make a model of the measurement
number of the next significant digit (eg 10.65 Suppose that a calibrated, vented ionisation chamber is used
becomes 10.6, while 10.75 becomes 10.8). to measure the dose. The temperature and pressure of the
ambient atmosphere are also measured and used to
compensate the ionisation chamber. Given this information,
Summary of the GUM method the measurement model is:

The GUM process is to;  temperature 


final result = reading × cal. factor ×  ×
Step 1: Make a model of the measurement. For  295.15  (9)
measurements with many uncertainty
 101325 
components, it may become difficult to manage  pressure 
the model. In such cases, it may be appropriate  
to separate the model into smaller parts, and
analyse these separately. where: final result is the dose to be determined, reading is
Step 2: Identify and characterise each uncertainty the value displayed by the electrometer, cal. factor is the
component. The NMI Guide suggests creating calibration factor for the ionisation chamber and
an uncertainty analysis table for recording the electrometer (a unitless value), temperature is the
results of this step, such as table 2. temperature inside the ionisation chamber at the time of

136
Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005 Gregory et al • A standard approach to uncertainties

measurement (in K), pressure is the pressure inside the inverse square law is simple, modelling exposure variation
ionisation chamber at the time of measurement (in Pa). in the other planes is non-trivial, which makes these
The final result is 1.97254 mGy, based on a reading of uncertainty components well suited to an empirical
1.940 mGy, a calibration factor of 1.01, pressure of 100 650 approach to evaluate c.
Pa, and temperature of 295.15 K. Vertically, a 10 mm position change altered the final
result by 0.0592 mGy.
Step 2. Identify and characterise each uncertainty Therefore c = 0.00592 mGy/mm.
component In the anode-cathode plane, a 10mm position change
Four uncertainty components were identified when altered the final result by 0.0272 mGy.
modelling the measurement and appear in equation 9. Four Therefore c = 0.00272 mGy/mm.
more uncertainty components have been identified; Laterally, a 10 mm position change did not alter the
rounding of the reading by the display, and variation of the final result.
chamber’s position from the desired position in the three Therefore c = 0 mGy/mm.
spatial planes.
Calculation of u and ν
Calculation of c Both the Type A and Type B evaluation methods will
For the purpose of demonstration, values of c have be demonstrated in the evaluation of u and ν for each
been acquired using the three different methods for three uncertainty component.
groups of uncertainty components. (i) When the reading value was recorded, a further 24
(i) For the reading, the rounding of the displayed individual readings were also taken to obtain information
reading, and the calibration factor, the differentiation about the distribution of these values. Thus, a Type A
method is used. Differentiating the model with respect to evaluation can be used. The standard uncertainty of the
‘reading’: reading is equal to the estimated standard deviation of the
25 readings, i.e. 0.0127 mGy. The value ν in this case
∂ ( result )  temperatur e   101325 
c= = cal. factor ×  ×  equals one less than the number of data used. Thus ν = 24.
∂ ( reading )  295.15   pressure  (ii) The rounding of a displayed reading could be any
c = 1.017 calculated using measured values for the value in the range ±0.0005 mGy, as the reading is displayed
other uncertainty components. to 3 decimal places. Furthermore, there is equal probability
For rounding, as the rounding of the reading is of the rounding being any value in this range, and zero
essentially a change in the value of the reading, again, probability of the rounding being outside the range. This is
c = 1.017. an example of a rectangular distribution. Thus from
For the calibration factor, equation 1;

∂(result)  temperature   101325  a 0.0005 = 0.000289 mGy


c= = reading ×  × u= =

∂(cal. factor)  295.15   pressure  3 3

c = 1.953 mGy/unit As the boundaries of the distribution are precisely


known, this evaluation would be judged as ‘excellent’ in
(ii) For the temperature and pressure, values for c are
accordance with table 1, thus ν = 1000.
derived by changing the value of these uncertainty
Assume the calibration factor was provided with
components by a small amount in the model equation (9),
only a limited explanation, such as “…the calibration
and observing the change in the final result.
factor = 1.01, with an uncertainty of 5%…”. To evaluate u
Increasing the temperature by 1 K to 296.15 K
increases the final result by 0.00668 mGy to 1.97922 mGy. and ν, a number of assumptions will need to be made. The
Thus first assumption is that the uncertainty distribution of the
calibration factor is a t-distribution. This is a reasonable
∆result 0.00668 assumption as it is consistent with the CLT. The next
c= = = 0.00668 mGy/K assumption is that it would be expected that coming from a
∆temperature 1
laboratory with traceable standards, the calibration was at
Similarly, increasing the pressure by 100 Pa to 100750 least ‘good’. This translates to ν=20 from table 1. Finally, a
Pa decreases the final result by 0.00196 mGy to 1.97058 decision needs to be made with regard to what confidence
mGy. Thus interval was used. If the stated uncertainty of 5% was
assumed to represent a 95%CI, then from equation 2;
∆result − 0.00196
c= = = −0.0000196 mGy/Pa
∆pressure 100 a 5%
u= = = 2.5% = 0.025 .
2 2
(iii) For positioning, values for c were obtained using
the empirical method. (iii) Atmospheric conditions were measured using a
Dose varies with position in the 3 spatial planes; portable temperature and pressure meter. The meter’s
vertically (inverse square law), the anode-cathode plane specifications stated that temperature measurements were
(‘heel’ effect) and laterally. Note that while modelling the accurate to ±1°C, and pressure measurements were accurate

137
Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005 Gregory et al • A standard approach to uncertainties

to ±100 Pa. To gain a greater insight into the uncertainty of The omission of any detail, such as whether or not the
temperature and pressure readings from the meter, X-ray unit should be warmed up prior to testing, can cause
comparisons were made at ambient temperature and different (yet valid) final results.
pressure with a calibrated thermometer (±0.2°C, 95%CI) Note that for simplicity of the example, the final result
and a calibrated barometer (±50 Pa, 95% CI). For all was based on the value of one individual reading, while a
measurements taken over a period of several weeks, the further 24 readings were taken but only used to estimate the
difference between the portable meter and the calibrated standard deviation of the reading. However, it would be
instruments was less than the stated ‘accuracy’. It is more sensible to use the mean of the 25 individual readings
therefore reasonable to assume that uncertainties of ±1°C as the value of ‘reading’. If this were done, the standard
and ±100 Pa represent 95%CIs, and that the evaluation for uncertainty of ‘reading’ would take the smaller value of the
both is at least ‘good’. Hence, from equation 2, u for experimental standard deviation of the mean (ESDM)6,11,12.
temperature = 0.50 K, u for pressure = 50 Pa, and from Since the ESDM equals the standard deviation of the
table 1, ν = 20 for temperature and pressure. individual data divided by the square root of the number of
(iv) The uncertainty in placing the ionisation chamber readings, the standard uncertainty of ‘reading’ would be
in the specified location of the three spatial planes was five times smaller.
assessed by repeatedly placing the chamber in position for Also note from the example that equation 3 was used,
measurement and measuring its actual position with digital despite one of the uncertainty components (vertical position
vernier calipers. The measurement with calipers is not done of the ionisation chamber) varying with the final result in
routinely; it is too time consuming. However, the data accordance with the inverse square law, not linearly as
regarding positioning uncertainty enabled values for u and required by equation 3. However, given the measurement
was conducted approximately 600mm from the anode, and
ν to be estimated for the three positioning uncertainty
that the standard uncertainty was 2 mm, the variation of
components. It was estimated that for all three planes, the
vertical position and dose is approximately linear over this
ionisation chamber was within a = 4 mm of the correct
range.
location 95% of the time, and the measurements followed a
From the data in the uncertainty analysis table, table 2,
Normal distribution. From equation 2, u = 2 mm. The
an extra column for the result c2u2 for each uncertainty
evaluation was regarded as good (ν = 20).
component can be generated. Uncertainty components with
larger values of c2u2 contribute the largest proportion to the
Step 3. Combine the uncertainties
overall measurement uncertainty, and so effort may be
This step is most simply performed using a spreadsheet
directed to improving these measurements. Conversely,
program. Using equations (3) and (7) gives, ucomb = 0.0523
there is little to be gained by directing effort towards
mGy, and νeff = 26. improving the measurements of uncertainty components
with low values for c2u2.
Step 4. State the final result. It is only necessary to include uncertainty components
Using tabulated data6,12, the appropriate coverage that are significant in the uncertainty analysis table. If it is
factor, k, for the desired confidence interval can be found. unknown as to whether or not an uncertainty component is
If a 95%CI is chosen, and given νeff = 26, then k = 2.056. significant, it should be included.
Hence U = 0.1074 mGy for a 95%CI. For Excel
spreadsheet users, U can be calculated using the formula
‘=TINV(0.05,ROUND(νeff,0))× ucomb’. Given the final
Conclusions
result of 1.97254 mGy, the final result and uncertainty need
to be rounded so they are harmonised. Firstly, round U in
Those who are familiar with other measurement
accordance with the rounding method, yielding ±0.11. uncertainty methods, especially in the field of
Next, round the final result to the same decimal place, radiotherapy15, may recognise elements of the GUM
yielding 1.97 mGy. The following statement can now be process. The advantage of using the GUM process is
made: that it provides a uniform approach to the determination
“Under the conditions prescribed for the measurement and expression of uncertainty in measurement
the exposure was found to be 1.97 mGy. The uncertainty internationally. This unified approach allows measurements
was calculated in accordance with the ISO GUM, and was to be compared more easily than beforehand,
found to have a 95% confidence interval of ±0.11 mGy and particularly if there is disagreement of two independently
26 degrees of freedom.” measured results. The NMI Guide provides useful
The uncertainty analysis data for this example is shown background information on the GUM, as well as
in table 2. pragmatic solutions for measurements in applied
environments, without deviating from the GUM
philosophy. While there will initially be some work in
Discussion applying the protocols to the measurements used to test
medical equipment, these can easily be programmed into
A detailed description of the quantity to be measured is spreadsheets, after which further calculations are
an important step in reducing measurement uncertainty. straightforward.

138
Australas. Phys. Eng. Sci. Med. Vol. 28, No 2, 2005 Gregory et al • A standard approach to uncertainties

References 8. EURACHEM, Quantifying Uncertainty in Analytical


Measurement, Laboratory of the Government Chemist,
1. South Australian Government, Radiation Protection and London, 1995.
Control (Ionising Radiation) Regulations 2000, Government 9. International Atomic Energy Agency, Quantifying uncertainty
Gazette, 2000. in nuclear analytical measurements, TECDOC 1401, IAEA,
2. Craig, A. R., Heggie, J. C. P., McLean, I. D., Coakley, K. S. Vienna, 2004.
and Nicoll, J. J., Recommendations for a mammography 10. Bentley R. E., Applying the ISO Guide to the calculation of
quality assurance program, Australas. Phys. Eng. Sci. Med., uncertainty: temperature, National Measurement Laboratory,
24(3):107-131, 2001. Sydney, 2001.
3. McCollough, C. H. and Zink, F. E., Performance evaluation of 11. Taylor, B. N., Kuyatt, C. E., Guidelines for evaluating and
a multi-slice CT scanner, Med. Phys., 26:2223-2230, 1999. expressing the uncertainty of NIST measurement results,
4. Luketina, I. A., Greig, L., Linear accelerator output National Institute of Standards and Technology, Gaithersburg,
variability, Australas. Phys. Eng. Sci. Med., 27(3):155-159, 1994.
2004. 12. Bentley R. E., Uncertainty in measurement: The ISO Guide,
5. Nobels, F., Beckers, F., Bailleul, E., De Schrijver, P., Sierens, 6th ed., National Measurement Laboratory, Sydney, 2003.
L., Van Crombrugge, P., Feasibility of a quality assurance 13. Pentz, M., Shott, M., Handling Experimental Data, Open
programme of bedside blood glucose testing in a hospital University Press, Milton Keynes, 1988.
setting: 7 years' experience, Diabet. Med., 21(12):1288-91, 14. Kirkup, L., Experimental methods: an introduction to the
2004. analysis and presentation of data, John Wiley & Sons, Milton,
6. International Organization for Standardization, Guide to the 1994.
expression of uncertainty in measurement, International 15. International Atomic Energy Agency, Absorbed Dose
Organization for Standardization, Geneva, 1995. Determination in External Beam Radiotherapy: An
7. Hinkle, D., Wiersma, W., and Jurs, S., Applied Statistics International Code of Practice for Dosimetry based on
for the Behavioural Sciences, Houghton Miffin, Boston, Standards of Absorbed Dose to Water, TRS-398, IAEA,
2002. Vienna, 2000.

139