Error Propagation

Michael Cushing

The overall goal of this note is to understand the covariance terms in the error propagation
formula. Most of what I am going to say is a rehash of the work by Kai Oliver Arras1 .
Our goal is to determine the uncertainty (variance) of a quantity z that is a function of m
random variables q (1) , q (2) , ... q (m) whose distribution functions we know. In other words, we wish
to know the distribution of z values, P (z), given z = f (q (1) , q (2) , ..., q (m) ). If f is non-linear, then
P (z) can be very complex. Therefore, we wish to derive an approximation for P (z).

1 1-D Case
Consider the function f (q) shown in Figure 1. We would like to known how the distribution of
q values “propagates” through the function f to produce the distribution of z values P (z). If the
function is highly non-linear, the shaded region in the distribution of q values will be mapped
nonuniformly into P (z) resulting in a distorted and asymmetric distribution. We will in the end,
however, characterize P (z) by its mean and variance no matter what its shape.

80

f(q)
µz−σz60
µz
µz−σz
40

20

0
q
0 20 q−σq
µ40 µq µq+σ
60 q 80

Figure 1: 1-D error propagation.

To begin, we approximate f (q) by a first-order Taylor series expansion around the point µq ,
1
.dmtwww.epfl.ch/isr/asl/publications/arrasTR9801R3.pdf

1
 
∂f 
z = f (q) ' f (µq ) + (q − µq ) (1)
∂q µq
This equation represents the dashed line in Figure 1.
We are now in a position to determine the mean and variance of the output z value using the
standard formulas for the mean and variance of a set of data, namely,

n
1X
µ = xi (2)
n i=1
n
1X
σµ2 = (xi − µ)2 (3)
n i=1

The mean value of P (z) is given by,

n  !
1X ∂f 
µz = f (µq ) + (qi − µq ) (4)
n i=1 ∂q µq
n  n
1X 1 ∂f  X
= f (µq ) + (qi − µ) (5)
n i=1 n ∂q µq i=1
= f (µq ) (6)
Pn
since by the definition of the mean value, i=1 (qi − µq ) = 0.
The variance of P (z) is given by,

n  !2
1X ∂f 
σz2 = f (µq ) + (qi − µq ) − µz (7)
n i=1 ∂q µq
n  !2
1 X ∂f 
= (qi − µq ) (8)
n i=1 ∂q µq
 !2 X n
∂f  1
= (qi − µq )2 (9)
∂q µq n i=1
 !2
∂f 
= σq2 . (10)
∂q µq


So we have the following two equations to propagate the error of a value q through a function
z = f (q),

2
  !2
∂f 
µz = f (µq ), σz2 = σq2 (11)
∂q µq

It must be noted that the values µz and σz2 are only approximations to the true distribution of the
values of z.

2 n-D Case
In this section, we will concentrate on the situation described in the introduction, namely when q
depends on multiple random variables.
The Taylor expansion around the values µ(1) , µ(2) , ... is given by,

m
(1) (2) (1) (2)
X ∂f (i)
z = f (q , q , ...) ' f (µ , µ , ...) + (i)
(q − µ(i) ) (12)
i=1
∂q
where for clarity we omit the explicit evaluation of the derivative at the mean value. We again
compute the mean and variance of the z values using Equations 2 and 3,

n
" m
#
1X X ∂f (j)
µz = f (µ(1) , µ(2) , ...) + (q − µ(j) )
(j) i
(13)
n i=1 j=1
∂q
n m n
1X (1) (2) 1 X ∂f X (j)
= f (µ , µ , ...) + (q − µ(j) ) (14)
n i=1 n j=1 ∂q (j) i=1 i
= f (µ(1) , µ(2) , ...) (15)
(j)
since by the definition of the mean, ni=1 (qi − µ(j) ) = 0 for each j.
P

The variance of P (z) is,

n m
!2
1X X ∂f (j)
σz2 = f (µ(1) , µ(2) , ...) + (q − µ(j) ) − µz
(j) i
(16)
n i=1 j=1
∂q
n m
!2
1 X X ∂f (j)
= (q − µ(j) ) (17)
n i=1 j=1 ∂q (j) i
n m m
!
1 X X ∂f (j) X ∂f (k)
= (q − µ(j) ) (q − µ(k) ) , (18)
n i=1 j=1 ∂q (j) i k=1
∂q (k) i

3
where we have expanded the the square in Equation 17 by subscripting the sum with a j and k.
Expanding the multiplication of sums we have,

n
" m  2
1X X ∂f (j)
σz2 = (j)
(qi − µ(j) )2 + (19)
n i=1 j=k=1
∂q
m X
#
X ∂f ∂f (j) (k)
(q − µ(j) )(qi − µ(k) )
(j) ∂q (k) i
j=1 k6=j
∂q

where the first term in the bracket comes from when j = k and the second term comes from the
cross terms when j 6= k. Now moving the sum over n into the brackets we have,

m  2 X n
X ∂f 1 (j)
σz2 = (j)
(qi − µ(j) )2 +
j=1
∂q n i=1
m X n
X ∂f ∂f 1 X (j) (k)
2 (j) ∂q (k) n
(qi − µ(j) )(qi − µ(k) ) (20)
j=1 k6=j
∂q i=1
m  2 m X n
X ∂f (j)2
X ∂f ∂f 1 X (j) (k)
= (j)
σ + (j) ∂q (k) n
(qi − µ(j) )(qi − µ(k) ) (21)
j=1
∂q j=1 j6=k
∂q i=1

2
If we define σjk , the covariance, to be,

n
1 X (j) (k)
σjk = (qi − µ(j) )(qi − µ(k) ) (22)
n i=1
then Equation 21 becomes,

m  2 m X
X ∂f X ∂f ∂f 2
σz2 = σ (j)2
+ σ (23)
j=1
∂q (j)
j=1 k6=j
∂q (j) ∂q (k) jk

2.1 2D Case

As a concrete example, let’s look at the case where z depends on two variables. In this case, m = 2
and the variance becomes,

 2  2
∂f ∂f ∂f ∂f 2 ∂f ∂f 2
σz2 = σ (1)2
+ σ (2)2 + σ12 + σ (24)
∂q 1 ∂q 2 ∂q (1) ∂q (2) ∂q (2) ∂q (1) 21

4
Since σ12 = σ21 , we can rewrite this as,

 2  2
∂f ∂f ∂f ∂f 2
σz2 = σ (1)2
+ σ (2)2 + 2 σ (25)
∂q 1 ∂q 2 ∂q (1) ∂q (2) 12
If we substitute x and y for q 1 and q 2 we have the more familiar form for the variance of z,

 2  2
∂f ∂f ∂f ∂f 2
σz2 = σ (x)2
+ σ (y)2 + 2 σ (26)
∂x ∂y ∂x ∂y xy