Our aim is to sample x from a multivariate Gaussian with covariance matrix Cx Rn given by Cx = D T (1) where is a vector and D is the

the square matrix with on the diagonal and zeros o the diagonal. The standard method to sample x N (0, Cx ) is to sample z N (0, I) and set x = chol(Cx )z. Taking the Cholesky decomposition is O(n3 ) so for large n it is infeasible. An alternative sampling method is to carry out eigendecomposition the Cx V = V , draw z N (0, I) and then set x = V z. We can see this gives samples with the desired covariance matrix as we have Cx = E[xxT ] = E[V zz T V T ] = V E[zz T ] V T = V V T = Cx . Using the eigendecomposition is also O(n3 ) in general but we can use the structure of the eigenspace of Cx to do the decomposition in a more ecient way. 1 First we introduce a rescaled variable y given by y = D x. y then has the T distribution y N (0, Cy ), where Cy is given by Cy = I . For brevity set = so Cy = I T . The eigenspace of Cy has a simple structure. One eigenvector, v is proportional to as we have (I T ) = (T ) = (1 T ) = Call the space spanned by this eigenvector V . As the eigenvectors of a real symmetric matrix form an orthonormal basis, the other n 1 eigenvectors must all be orthogonal to and so for these vi we have (I T )vi = vi (T vi ) = vi so their eigenvalues are all equal to one. We call the space spanned by these eigenvectors V . Using these two spaces allows us to generate samples y N (0, Cy ) as follows. Draw z1 N (0, In ) and set y1 = z1 so that y1 V . Draw z2 N (0, 1) and set y2 = (1 T )0.5 z2 |||| 1
T z1 T

so that y1 V . Then set y = y1 + y2 . We claim that the sample then project scheme for y1 is equivalent to sampling from an axis aligned, spherical Gaussian in V but this fact is not required for the proof that y comes from the correct distribution. To prove that y N (0, Cy ) it is enough to prove that V [y] = Cy as all operations are linear manipulations of Gaussians. V [y] = V [y1 ] + V [y2 ] as Cov[y1 , y2 ] = 0 as z1 , z2 are independent. V [y1 ] = V = I I T T z1 I T T

T T

=I 2

T T T + T 2 T ( ) T =I T

and V [y2 ] = So we have V [y] = I T (1 T ) T + T T T T = I T + T T = Cy (1 T ) T T

as required. x is then given by x = D y.