Professional Documents
Culture Documents
Monte Carlo Methods Using Matlab
Monte Carlo Methods Using Matlab
Roberto Casarin
University Ca’ Foscari, Venice
Summer School of Bayesian Econometrics
Perugia 2013
September 9, 2013
2
Contents
1 A Matlab Primer 1
1.1 Programming Languages . . . . . . . . . . . . . . . . . . . . . 1
1.2 Fourth Generation Languages (4GPL) . . . . . . . . . . . . . 5
1.3 Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.1 Operators . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3.2 Logical Operators . . . . . . . . . . . . . . . . . . . . . 6
1.3.3 Creating Matrices . . . . . . . . . . . . . . . . . . . . . 7
1.3.4 Matrix Description . . . . . . . . . . . . . . . . . . . . 7
1.3.5 Other Functions . . . . . . . . . . . . . . . . . . . . . . 7
1.3.6 Loops and If Statements . . . . . . . . . . . . . . . . . 8
1.3.7 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.4.1 Input, Output and Graphics . . . . . . . . . . . . . . . 9
1.4.2 Ordinary Least Square . . . . . . . . . . . . . . . . . . 11
1.4.3 A Bayesian Linear Regression Model . . . . . . . . . . 12
1.5 From Matlab to Scilab and R . . . . . . . . . . . . . . . . . . 17
3 Importance Sampling 31
3.1 Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 Properties of the IS Estimators . . . . . . . . . . . . . . . . . 32
3.3 Generating Student-t Variables . . . . . . . . . . . . . . . . . 34
i
ii CONTENTS
Chapter 1
A Matlab Primer
Aim
Contents
1. Programming Languages
3. Matlab
4. Examples
http://www.feweb.vu.nl/econometriclinks/software.html
where many of the most used econometrics softwares and their contributed
libraries are linked.
1
2 CHAPTER 1. A MATLAB PRIMER
• HLM
• KNITRO
• S+, SAS, SCA, Scilab, SciPy, SciViews, Sciword, SCP, Shazam, Sigmaplot,
SIMSTAT, SOLAS, SOL, Soritec, SpaceStat, SQlite, SPAD, Speakeasy,
IBM SPSS, SsfPack, STAMP, Stata, StatCrunch, Statgraphics, Sta-
tistica, Stat/Transfer, StatsDirect, STL, Statview, SUDAAN, SVAR,
SYSTAT
SAS: is a 4GPL which allows to define a sequence of operations (statistical
analysis and data management) to be performed on data
Scilab: is 4GPL free and open source for numerical computation, similar to
Matlab
In the 1GPL the programs are written in binary code and can access
binary digits. To write programs with 1GPL is a very skilled job and it is
very time consuming to test and debug programs.
In the 2GPL, the programs are written in symbolic assembly code, they
access bytes and are slightly less time demanding.
In the 3GPL, the programs are written in a High Level Language (e.g.
COBOL, Pascal, C, Fortran, etc), they can access records and programming
requires less time and skills.
Another common feature is that they are extremely powerful for matrix
manipulation and in this sense they are more useful for economists than the
3GPL programming languages (such as C or Fortran), where the basic data
units are all scalars. At the same time they are very flexible and allows more
expert users to use interface to procedures written in other languages such
as C, C++, or Fortran.
1.3 Matlab
1.3.1 Operators
• Select submatrix from matrix:
x( startrow : endrow, startcolumn : endcolumn )
• Transposition operator:
• Matrix Operators: + - * \ %
• Element-by-element operators: .+ .- .* .\
• Concatenating operators:
[leftmatrix, rightmatrix] [uppermatrix; bottommatrix]
• Relational operators: < > == /= >= <=
• Relational operators (element-by-element): .< .> .== ./= .>= .<=
• y= ceil( x );
• y= floor( x );
• y= reshape( x,r,c );
• Kronecker product: kron( x , y )
• y= trimr( x,t,b );
if logical expression 1;
...
elseif logical expression 2;
...
else;
...
end;
1.3.7 Procedures
Procedures may contain so called local variables. These variables are called
local, since you can access them only in the particular procedure.
1.4 Examples
1.4.1 Input, Output and Graphics
%*************************************************
% basic in I/O, graphical, statistical procedures
%*************************************************
% Load UK/EU exchange rate data
clc;
clear all;
yy= load(’C:/Dottorato/Teaching/SummerSchoolBertinoro/...
TutorialAntonietta/TutorialRobAnt/AllLab/MatlabCode/ChapterMatlab/pound.txt’);
%*************************************************
n=size(yy,1); % evaluate the number of rows %
xx=(1:n);
%*************************************************
% for end if end
% (1) Evaluate sequentially the variance
% (2) Built a dummy variable, based on the value
% of the variance estimated recursively
%*************************************************
wn=10; % set the value of a variable%
s=zeros(n,1); % define a n-dim null vector %
d=zeros(n,1);
for j=(wn+1):n;
s(j,1)=var(yy((j-wn+1):j,1));
if (s(j,1)>0.45);
d(j,1)=1;
10 CHAPTER 1. A MATLAB PRIMER
end;
end;
%*************************************************
% Some Pictures...
%*************************************************
% figure(1) to have distinct graphs
figure(1);
title(’Time series data’);
ylabel(’Data’);
xlabel(’Time’);
plot(xx,yy);
figure(2);
title(’Time-varying log-volatility’);
a=plot(xx,s,’color’,[1 0 0]); %[red green blue] the rgb convention
axis([1 n min(s) max(s)]); % Set tics
figure(3);
title(’Dummy’);
plot(xx,d,’color’,[1 0 0]); %[red green blue] the rgb convention
axis([1 n -0.1 1.1]); % Set tics
%*************************************************
% All charts in one pictures...
%*************************************************
figure(4);
subplot(3,1,1);
title(’Time series data’);
ylabel(’Data’);
xlabel(’Time’);
plot(xx,yy);
subplot(3,1,2);
title(’Time-varying log-volatility’);
plot(xx,s,’color’,[1 0 0]); %[red green blue] the rgb convention
axis([1 n min(s) max(s)]); % Set tics
subplot(3,1,3);
title(’Dummy’);
plot(xx,d,’color’,[1 0 0]); %[red green blue] the rgb convention
axis([1 n -0.1 1.1]); % Set tics
%*************************************************
% histogram
%*************************************************
figure(5);
hist(yy,50);
%*************************************************
% Save the results in a ouput file
%*************************************************
fid = fopen(’C:/Dottorato/Teaching/SummerSchoolBertinoro/...
TutorialAntonietta/TutorialRobAnt/AllLab/MatlabCode/ChapterMatlab/OutPound.txt’, ’w’);
fprintf(fid, ’%5.2f\n’, yy);
fclose(fid);
1.4. EXAMPLES 11
%*************************************************
Check for the correct number of input argument and if the number of
rows of x is equal to the number of rows of y
Evaluate all the statistics that are usually involved in a OLS estimation
results.y = y;
results.nobs = nobs;
results.nvar = nvar;
%
xpxi = (x’*x)\eye(k);
results.beta = xpxi*(x’*y);
results.yhat = x*results.beta;
results.resid = y - results.yhat;
sigu = results.resid’*results.resid;
results.sige = sigu/(nobs-nvar);
tmp = (results.sige)*(diag(xpxi));
results.tstat = results.beta./(sqrt(tmp));
12 CHAPTER 1. A MATLAB PRIMER
ym = y - mean(y);
rsqr1 = sigu; rsqr2 = ym’*ym;
results.rsqr = 1.0 - rsqr1/rsqr2; % r-squared
rsqr1 = rsqr1/(nobs-nvar);
rsqr2 = rsqr2/(nobs-1.0);
results.rbar = 1 - (rsqr1/rsqr2); % rbar-squared
ediff = results.resid(2:nobs) - results.resid(1:nobs-1);
results.dw = (ediff’*ediff)/sigu; % durbin-watson
end;
We save as a function the ols.m code and run the following simulation
example
nob=100;
x1=ones(nob,1);
x2=randn(nob,1).*((1:nob)’/10);
x=[x1 x2];
sig=2;
y=x*[10; 0.9]+sig*randn(nob,1);
res=ols(y,x);
res.beta
%%
figure(1)
plot([res.yhat y]);
figure(2)
plot(res.resid);
(1.1) y = Xβ + ε
(1.2) ε ∼ Nn (0n , σ 2 In )
(1.3) Rβ ∼ N (r, T )
or equivalently
(1.4) Qβ ∼ N (q, Ik )
or equivalently
% Simulate a dataset
n=100; k=3;
x = randn(n,k);
b = ones(k,1);
y = x*b + randn(n,1);
% Set prior
14 CHAPTER 1. A MATLAB PRIMER
% Evaluate statistics
bhat = mean(bsave(nomit+1:ndraw,:));
bstd = std(bsave(nomit+1:ndraw,:));
tstat = bhat./bstd;
sighat = mean(ssave(nomit+1:ndraw,1));
tout = 1-cdf(’t’,abs(tstat’),n);
0.9610 11.2966 0
0.9200 11.2740 0
Theil-Goldberger estimates
1.0037
0.9569
0.9198
Let yi be the monthly change in the Treasury Bill rate and zi the monthly
change in the AAA bond rate. We will fit on this set of data the heteroscedas-
tic model presented above with
yi = β1 + β2 zi + εi
Theil-Goldberger estimates
0.0057
0.2747
16 CHAPTER 1. A MATLAB PRIMER
1.5
Actual
1 Fitted
0.5
−0.5
−1
−1.5
100 200 300 400 500 600
1.5
Residuals
1
0.5
−0.5
−1
100 200 300 400 500 600
Figure 1.1: Actual and fitted data (top) and residuals (bottom) using the
Bayesian estimates of the linear regression model.
The estimates of the σ 2 are 0.0283 for the Gibbs sampler and 0.0282 for the
Theil-Goldberger procedure.
The actual and fitted data and the residuals are given in Fig. 1.1. The
plot of the residuals shows that in the second half of the sample (say after
the 1975) the variance is underestimated. More precisely one should account
in the model for the time variation in the variance of the data. This call
for heteroscedastic linear regression models (see Chapter ??) or for nonlinear
models such as stochastic volatility models (see Chapter ?? and ??).
References
• Gelfand, Alan E., and A.F.M Smith. 1990. Sampling-Based Approaches
to Calculating Marginal Densities, Journal of the American Statistical Asso-
ciation, Vol. 85, pp. 398-409.
1.5. FROM MATLAB TO SCILAB AND R 17
• Theil, Henri and Arthur S. Goldberger. 1961. On Pure and Mixed Sta-
tistical Estimation in Economics, International Economic Review, Vol. 2,
65-78.
//*************************************************
n=size(yy,1); // evaluate the number of rows //
xx=(1:2:n);
//*************************************************
// for endfor if end
// (1) Evaluate sequentially the variance
// (2) Built a dummy variable, based on the value
// of the variance estimated recursively
//*************************************************
wn=10; // set the value of a variable//
s=zeros(n,1); // define a n-dim null vector //
d=zeros(n,1);
for j=(wn+1):n;
s(j,1)=variance(yy((j-wn+1):j,1));
if (s(j,1)>0.45);
d(j,1)=1;
end;
end;
18 CHAPTER 1. A MATLAB PRIMER
//*************************************************
// Some Pictures...
//*************************************************
// figure(1) to have distinct graphs
figure(1);
title("Time series data");
ylabel("Data");
xlabel("Time");
plot(xx,yy);
figure(2);
title("Time-varying log-volatility");
plot(xx,s,’color’,[1 0 0]); //[red green blue] the rgb convention
a=gca();
a.data_bounds=[1,min(s);n,max(s)];// Set tics
figure(3);
title("Dummy");
plot(xx,d,’color’,[1 0 0]); //[red green blue] the rgb convention
a=gca();
a.data_bounds=[1,-0.1;n,1.1];// Set tics
//*************************************************
// All charts in one pictures...
//*************************************************
figure(4);
subplot(3,1,1);
title("Time series data");
ylabel("Data");
xlabel("Time");
plot(xx,yy);
subplot(3,1,2);
title("Time-varying log-volatility");
plot(xx,s,’color’,[1 0 0]); //[red green blue] the rgb convention
a=gca();
a.data_bounds=[1,min(s);n,max(s)];// Set tics
subplot(3,1,3);
title("Dummy");
plot(xx,d,’color’,[1 0 0]); //[red green blue] the rgb convention
a=gca();
a.data_bounds=[1,-0.1;n,1.1];// Set tics
//*************************************************
// histogram
1.5. FROM MATLAB TO SCILAB AND R 19
//*************************************************
figure(5);
histplot(100,yy);
//*************************************************
// Save the results in a ouput file
//*************************************************
fprintfMat(’C:/Dottorato/Teaching/SummerSchoolBertinoro/TutorialAntonietta/...
TutorialRobAnt/AllLab/MatlabCode/ChapterMatlab/OutPound.txt’,yy,’%5.2f’);
// attention this overwrites the existing file
R
#*************************************************
# basic in I/O, graphical, statistical procedures
#*************************************************
# Load UK/EU exchange rate data
yy=scan("C:/Dottorato/Teaching/SummerSchoolBertinoro/TutorialAntonietta/...
TutorialRobAnt/AllLab/MatlabCode/ChapterMatlab/pound.txt",sep="\t",skip=0,na.strings=".")
dim(yy)=c(1006,1);
#*************************************************
n=dim(yy); # evaluate the number of rows #
n=n[1];
xx=(1:n);
#*************************************************
# for endfor if end
# (1) Evaluate sequentially the variance
# (2) Built a dummy variable, based on the value
# of the variance estimated recursively
#*************************************************
wn=10; # set the value of a variable#
s=array(0,n); # define a n-dim null vector #
d=array(0,n);
for (j in ((wn+1):n)){
s[j]=var(yy[(j-wn+1):j]);
if (s[j]>0.45){
d[j]=1;
}
}
#*************************************************
# Some Pictures...
#*************************************************
# figure(1) to have distinct graphs
20 CHAPTER 1. A MATLAB PRIMER
dev.new();
plot(xx,yy,main="Time series data",xlab="Time",ylab="Data",type="l");
dev.new();
plot(xx,s,main="Time-varying log-volatility",xlab="Time",ylab="Data",type="l");
#[red green blue] the rgb convention
dev.new();
plot(xx,d,main="Dummy",xlab="Time",ylab="Data",type="l");
#[red green blue] the rgb convention
#*************************************************
# All charts in one pictures...
#*************************************************
par(mfrow=c(3,1),pin=c(5,1.5));
plot(xx,yy,main="Time series data",xlab="Time",ylab="Data",type="l");
plot(xx,s,main="Time-varying log-volatility",xlab="Time",ylab="Data",type="l");
#[red green blue] the rgb convention
plot(xx,d,main="Dummy",xlab="Time",ylab="Data",type="l");
#[red green blue] the rgb convention
#*************************************************
# histogram
#*************************************************
dev.new();
hist(yy,50);
#*************************************************
# Save the results in a ouput file
#*************************************************
save(yy, file = "C:/Dottorato/Teaching/SummerSchoolBertinoro/TutorialAntonietta/...
TutorialRobAnt/AllLab/MatlabCode/ChapterMatlab/OutPound.txt");
Chapter 2
Aim
Contents
1. Integration
3. Asymptotic Properties
2.1 Integration
• Our aim is to approximate the integral
Z 1
(2.1) µ(f ) = f (x)dx
0
21
22 CHAPTER 2. MONTE CARLO INTEGRATION
1. f (x) = x
2. f (x) = x2
3. f (x) = cos(πx)
The results of the Monte Carlo estimates for different sample sizes n =
1, . . . , 50 and different integrand functions f are given in Fig. 2.1
• Find the mean and the variance of the estimator and give a Monte Carlo
approximation for the expression of the variance.
2.2. A MONTE CARLO ESTIMATOR 23
MC Means
f(x)=x
0.8
Empirical Average
0.6 Theoretical Mean
0.4
0.2
0
0 10 20 30 40 50
2
f(x)=x
0.4
Empirical Average
0.3 Theoretical Mean
0.2
0.1
0
0 10 20 30 40 50
f(x)=cos(π x)
1
Empirical Average
Theoretical Mean
0.5
−0.5
0 10 20 30 40 50
Figure 2.1: Monte Carlo estimates µ̂n (solid lines) for different sample sizes
n = 1, . . . , 50 and true values of µ (horizontal dotted lines).
n
1 X 1
(2.5) V(µ̂n ) = 2 V(f (Xi )) = σ 2 (f )
n i=1 n
24 CHAPTER 2. MONTE CARLO INTEGRATION
where Z +∞
2
σ (f ) = V(f (X1 )) = (x − µ)2 f (x)I[0,1] (x)dx
−∞
For the different f we find the analytical solution of the integral µ(f ) (see
also horizontal dotted lines in Fig. 2.1)
1. For f (x) = x
1 1 2 0
Z
(2.6) E(f (X1 )) = xdx = x = 1/2
0 2 1
2. For f (x) = x2
1
0
1
Z
x2 dx = x3 = 1/3
(2.7) E(f (X1 )) =
0 3 1
1
0
1
Z
(2.8) E(f (X1 )) = cos(πx)dx = sin(πx) = 0
0 π 1
a.s.
(2.9) µ̂n −→ µ
n→∞
√ D
(2.10) n (µ̂n − µ) −→ N (0, σ 2 (f ))
n→∞
1. For f (x) = x
2. For f (x) = x2
V(f (X1 )) = 1/5 − 1/9 = 4/45
When the variance V(f (X1 )) is unknown one can use the Monte Carlo
estimator
n
2 1 X
(2.11) σ̂ (f ) = (Xi − µ̂n )2
n − 1 i=1
MC Variances
f(x)=x
0.25
Empirical Variance
0.2 Theoretical Variance
0.15
0.1
0.05
0 10 20 30 40 50
f(x)=x2
0.2
Empirical Variance
0.15 Theoretical Variance
0.1
0.05
0
0 10 20 30 40 50
f(x)=cos(π x)
1.5
Empirical Variance
Theoretical Variance
1
0.5
0
0 10 20 30 40 50
Figure 2.2: Monte Carlo variance estimates σ̂n2 (solid lines) for different sam-
ple sizes n = 1, . . . , 50 and the true value σ 2 (horizontal dotted lines).
that is
2
n Xα
r
(2.13) Xα = ε ⇔n= σ 2 (f )
σ 2 (f ) ε
2.5. APPENDIX - MATLAB CODE 27
nε2
(2.14) σ̂n2 (f ) ≤
Xα2
Exercise: write a Matlab’s code for computing the optimal number of sam-
ples that are needed to estimate µ(f ) for the different integrand functions f
given in Section 1 and for the accuracy level ε = 0.001.
for i=2:n
gav(i,1)=sum(x(1:i))/i;
gav(i,2)=sum(x(1:i).^2)/i;
gav(i,3)=sum(cos(pi*x(1:i)))/i;
gavvar(i,1)=var(x(1:i));
28 CHAPTER 2. MONTE CARLO INTEGRATION
gavvar(i,2)=var(x(1:i).^2);
gavvar(i,3)=var(cos(pi*x(1:i)));
end
%
%
%%%%%%%%% Graphics (mean) %%%%%%%%%%
figure(1);
subplot(3,1,1);
plot(gav(:,1));
line((1:n),ones(n,1)/2,’color’,’red’);
legend(’Empirical Average’,’Theoretical Mean’,...
’Location’,’NorthEastOutside’);
title(’f(x)=x’);
%
subplot(3,1,2);
plot(gav(:,2));
line((1:n),ones(n,1)/3,’color’,’red’);
legend(’Empirical Average’,’Theoretical Mean’,...
’Location’,’NorthEastOutside’);
title(’f(x)=x^2’);
%
subplot(3,1,3);
plot(gav(:,3));
line((1:n),ones(n,1)*0,’color’,’red’);
legend(’Empirical Average’,’Theoretical Mean’,...
’Location’,’NorthEastOutside’);
title(’f(x)=cos(\pi x)’);
%
%%%%%%%%% Graphics (variance) %%%%%%%%%%
figure(2);
subplot(3,1,1);
plot(gavvar(:,1));
line((1:n),ones(n,1)/12,’color’,’red’);
legend(’Empirical Variance’,’Theoretical Variance’,...
’Location’,’NorthEastOutside’);
title(’f(x)=x’);
%
subplot(3,1,2);
plot(gavvar(:,2));
line((1:n),ones(n,1)*4/45,’color’,’red’);
legend(’Empirical Variance’,’Theoretical Variance’,...
’Location’,’NorthEastOutside’);
title(’f(x)=x^2’);
%
subplot(3,1,3);
plot(gavvar(:,3));
line((1:n),ones(n,1)*1/2,’color’,’red’);
legend(’Empirical Variance’,’Theoretical Variance’,...
2.5. APPENDIX - MATLAB CODE 29
’Location’,’NorthEastOutside’);
title(’f(x)=cos(\pi x)’);
30 CHAPTER 2. MONTE CARLO INTEGRATION
Chapter 3
Importance Sampling
Aim
Define and apply the importance sampling method and study its
properties.
Contents
31
32 CHAPTER 3. IMPORTANCE SAMPLING
π(x)
Z
(3.2) µ= f (x)g(x)dx
g(x)
where
π(Xi )
w(Xi ) = , i = 1, . . . , n
g(Xi )
are called importance weights.
n
! n
1X
Z Z Y
Eg (µ̂IS
n ) = ··· w(xi )f (xi ) g(xi )dxi
n i=1 i=1
π(x1 )
Z
= f (x1 )g(x1 )dx1
g(x1 )
Z
= f (x1 )π(x1 )dx1
π(x1 )2
Z
Eg ((µ̂IS 2
n ) ) = f (x1 )2 g(x1 )dx1 =
g(x1 )2
π(x1 )
Z
= f (x1 )2 π(x1 )dx1 < ∞
g(x1 )
Note that if the tails of the importance density are lighter than those of the
π the importance weight π(x)/g(x) is not a.e. bounded and the variance of
the estimator will be infinite for many functions f .
1. 5
sin(x)
f (x) = I(x)(2.1,+∞)
x
2. s
x
f (x) =
1 − x
3.
x5
f (x) = I[0,+∞) (x)
1 + (x − 3)2
3. C(0, 1)
We shall note that the Cauchy distribution C(α, β) has density function
1
π(x) = IR (x)
πβ(1 + ((x − α)/β)2 )
3.3. GENERATING STUDENT-T VARIABLES 35
2
Student−t
1
0
0 1 2 3 4 5
4
x 10
10
Normal
5
0
0 1 2 3 4 5
4
x 10
2
Cauchy
1
0
0 1 2 3 4 5
4
x 10
1 x 1
Z
F (x) = du
π −∞ πβ(1 + ((u − α)/β)2)
1 1 x−α
= + arc tan IR (x)
2 π β
The inverse c.d.f. method can be applied in order to generate from the
Cauchy. If X = F −1 (U), where U ∼ U[0,1] , then X ∼ C(α, β).
From the results in Fig. 3.1 one can see that the importance weights for
Student-t and Cauchy are not unstable while the importance weights asso-
ciated to the normal exhibit some large jumps. For all the functions the
results in Fig. 3.2 show that the normal proposal produces jumps in the
progressive averages (green lines) that are due to the unbounded variance of
the estimator. However for the first function the normal proposal behaves
quite well when compared with the Cauchy and Student-t proposals. For the
36 CHAPTER 3. IMPORTANCE SAMPLING
second and third function the Cauchy proposal seems to converge faster than
the Student-t. In all the pictures we plotted (black lines) the approximation
obtained with an exact simulation from a Student-t with ν = 12.
for i=1:nIS
% Proposal 1
x1=random(’t’,nustar);
% Proposal 2
x2=random(’normal’,0,sqrt(nu/(nu-2)));
% Proposal 3
x3=tan((rand(1,1)-0.5)*pi);
%x3=random(’normal’,0,1)/random(’normal’,0,1);
% Exact
3.3. GENERATING STUDENT-T VARIABLES 37
x4=random(’t’,nu);
%%%%%%%%%%%%%
% IS
%%%%%%%%%%%%%
% Importance weights
wIS(i,1)=w1(x1,nu,nustar);
wIS(i,2)=w2(x2,nu);
wIS(i,3)=w3(x3,nu);
% f_1
mu1IS(i,1)=((sin(x1)/x1)^5)*(x1>2.1);
mu1IS(i,2)=((sin(x2)/x2)^5)*(x2>2.1);
mu1IS(i,3)=((sin(x3)/x3)^5)*(x3>2.1);
mu1IS(i,3)=((sin(x4)/x4)^5)*(x4>2.1);
% f_2
mu2IS(i,1)=sqrt(abs(x1/(1-x1)));
mu2IS(i,2)=sqrt(abs(x2/(1-x2)));
mu2IS(i,3)=sqrt(abs(x3/(1-x3)));
mu2IS(i,4)=sqrt(abs(x4/(1-x4)));
% f_3
mu3IS(i,1)=(x1^5/(1+(x1-3)^2))*(x1>0);
mu3IS(i,2)=(x2^5/(1+(x2-3)^2))*(x2>0);
mu3IS(i,3)=(x3^5/(1+(x3-3)^2))*(x3>0);
mu3IS(i,4)=(x4^5/(1+(x4-3)^2))*(x4>0);
%
if ((1000*floor(i/1000))==i)
disp(i);
end
end
mu1IScum(:,1)=cumsum(mu1IS(:,1).*wIS(:,1))./(1:nIS)’;
mu1IScum(:,2)=cumsum(mu1IS(:,2).*wIS(:,2))./(1:nIS)’;
mu1IScum(:,3)=cumsum(mu1IS(:,3).*wIS(:,3))./(1:nIS)’;
mu1IScum(:,4)=cumsum(mu1IS(:,4))./(1:nIS)’;
%
mu2IScum(:,1)=cumsum(mu2IS(:,1).*wIS(:,1))./(1:nIS)’;
mu2IScum(:,2)=cumsum(mu2IS(:,2).*wIS(:,2))./(1:nIS)’;
mu2IScum(:,3)=cumsum(mu2IS(:,3).*wIS(:,3))./(1:nIS)’;
mu2IScum(:,4)=cumsum(mu2IS(:,4))./(1:nIS)’;
%
mu3IScum(:,1)=cumsum(mu3IS(:,1).*wIS(:,1))./(1:nIS)’;
mu3IScum(:,2)=cumsum(mu3IS(:,2).*wIS(:,2))./(1:nIS)’;
mu3IScum(:,3)=cumsum(mu3IS(:,3).*wIS(:,3))./(1:nIS)’;
mu3IScum(:,4)=cumsum(mu3IS(:,4))./(1:nIS)’;
%%
fs=14;
%%%%%%%%%%%%%%%
figure(1)
subplot(3,1,1);
plot((1:nIS)’,wIS(:,1));
legend(’Student-t’,’Location’,’NorthEast’);
set(gca,’FontSize’,fs);
subplot(3,1,2);
plot((1:nIS)’,wIS(:,2));
legend(’Normal’,’Location’,’NorthEast’);
set(gca,’FontSize’,fs);
subplot(3,1,3);
38 CHAPTER 3. IMPORTANCE SAMPLING
plot((1:nIS)’,wIS(:,3));
legend(’Cauchy’,’Location’,’NorthEast’);
set(gca,’FontSize’,fs);
figure(2)
plot((1:nIS)’,mu1IScum(:,1:3));
hold on;
plot((1:nIS)’,mu1IScum(:,4),’-k’);
hold off;
legend(’Student-t’,’Normal’,’Cauchy’,’Exact’,’Location’,’NorthEast’);
ylim([0.00001 0.00015]);
set(gca,’FontSize’,fs);
figure(3)
plot((1:nIS)’,mu2IScum(:,1:3));
hold on;
plot((1:nIS)’,mu2IScum(:,4),’-k’);
hold off;
legend(’Student-t’,’Normal’,’Cauchy’,’Exact’,’Location’,’NorthEast’);
ylim([1 1.4]);
set(gca,’FontSize’,fs);
figure(4)
plot((1:nIS)’,mu3IScum(:,1:3));
hold on;
plot((1:nIS)’,mu3IScum(:,4),’-k’);
hold off;
legend(’Student-t’,’Normal’,’Cauchy’,’Exact’,’Location’,’NorthEast’);
ylim([3 9]);
set(gca,’FontSize’,fs);
−5
x 10
14 Student−t
Normal
Cauchy
12 Exact
10
0 1 2 3 4 5
4
x 10
1.4
Student−t
1.35 Normal
Cauchy
1.3 Exact
1.25
1.2
1.15
1.1
1.05
1
0 1 2 3 4 5
4
x 10
9
Student−t
Normal
8 Cauchy
Exact
7
3
0 1 2 3 4 5
4
x 10
Figure 3.2: Charts from one to three: µ̂IS for the different functions f .
In each chart the IS estimators for different proposals (colored lines) and
the Monte Carlo estimator with exact simulation from the T (12, 0, 1) (black
lines).
40 CHAPTER 3. IMPORTANCE SAMPLING
Exercise
Importance Sampling
Consider a Student-t distribution T (ν, θ, σ 2 ) with density
−(ν+1)/2
(x − θ)2
Γ((ν + 1)/2)
(3.6) π(x) = √ 1+ IR (x)
σ νπΓ(ν/2) νσ 2
π(x)
Z Z
(3.7) µ = Eπ (f (X)) = f (x)π(x)dx = f (x)g(x)dx
g(x)
3. C(0, 1)
1. 5
sin(x)
f (x) = I(x)(2.1,+∞)
x
41
42 CHAPTER 3. IMPORTANCE SAMPLING
2. s
x
f (x) =
1 − x
3.
x5
f (x) = I[0,+∞) (x)
1 + (x − 3)2
Metropolis-Hastings
Write a M.-H. algorithm to generate n = 500 i.i.d. random samples from
a zero-mean and independent bivariate normal distribution, N2 (0, I2 ), with
covariance matrix, I2 and mean 0 = (0, 0)′ . Use alternatively independent
and random walk proposals with variance covariance matrix σ 2 I2 . (Try with
different values of σ 2 ).