Random Variables

Random Variables Overview Definition: Random Variable
• Definitions X(ζ) = ζ
S
• Cumulative distribution function
Real
• Probability density Function ζ Line
• Functions of random variables Range

• Expected values
• Definition: a random variable X is a function that assigns a real
• Mean & variance
number, X(ζ), to each outcome ζ in the sample space of a
• Markov & Chebyshev inequalities random experiment.
• Independence & marginal distributions • The sample space S is the domain of the random variable
• Bayes rule and conditional probability • The set of all values that X can have is the range of the random
variable
• Mean square estimation
• This is a many to one mapping. That is, a set of points, ζ1 , ζ2 , . . .
• Linear prediction
may take on the same value of the random variable
• Will abbreviate as simply “RV”
J. McNames Portland State University ECE 4/557 Random Variables Ver. 1.06 1 J. McNames Portland State University ECE 4/557 Random Variables Ver. 1.06 2
Example 1: Random Variable Definitions Cumulative Distribution Function

Suppose that a coin is tossed three times and the sequence of heads The cumulative distribution function (CDF) of a random variable X
and tails is noted. What is the sample space for this experiment? Let is defined as the probability of the event {X ≤ x}:
X be the number of heads in three coin tosses. What is the range of
F (x) = P [X ≤ x]
X? List all of the points in the domain of the sample space and the
corresponding values of X.
• Sometimes is just called distribution function
• Here X is the random variable and x is a non-random variable
Properties of the CDF Example 2: Distribution Functions
1. 0 ≤ F (x) ≤ 1 The arrival time of Joe’s email obeys the exponential probability law
with parameter λ:
2. limx→+∞ F (x) = 1
3. limx→−∞ F (x) = 0 0 x<0
P [X > x] = −λx
λe x ≥ 0.
4. F (x) is a nondecreasing function of x. Thus, if a < b, then
F (a) ≤ F (b). Find the CDF of X for λ = 2 and plot F (x) versus x.
5. F (x) is continuous from the right. That is, for h > 0,
F (b) = limh→0 F (b + h) = F (b+ )
6. P [a < X ≤ b] = F (b) − F (a)
7. P [X = b] = F (b) − F (b− )
Example 2: Distribution Plot Example 2: MATLAB Code

function [] = ExponentialCDF();
Exponential Cumulative Distribution Function
close all;
1 %FigureSet(1);
%figure(1);
FigureSet(1,’LTX’);
lambda = 2;
x = 0:0.01:2;
0.8 y = 1-exp(-lambda*x);
h = plot(x,y,’b’,[0 100],[1 1],’k:’);

set(h,’LineWidth’,1.5);
axis([0 max(x) 0 1.1]);
0.6 xlabel(’x’);
F(x)
ylabel(’F(x)’);
title(’Exponential Cumulative Distribution Function’);
set(gca,’Box’,’Off’);
AxisSet(8);
0.4
print -depsc ExponentialCDF;
0.2
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
x
Empirical Distribution Function Example 3: Empirical Distribution Function Plot
Let X1 , X2 , . . . , Xn be a random sample. The empirical distribution Exponential Emperical Distribution Function N:25
function (edf) is a function of x which equals the fraction of Xi s that
are less than or equal to x for each x, −∞ < x < ∞ 1
• The “true” CDF is never known

0.8
• All we have is data
• The edf is a rough estimate of the CDF
0.6
S(x)
• Piecewise-constant function (stairs)
• Assuming the sample consist of distinct values, each step has 0.4
height = n1
• Minimum value: 0, Maximum value: 1 0.2
• Nondecreasing
• Is a random function 0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
x
Example 3: MATLAB Code Random Variable Types

function [] = ExponentialEDF();
Discrete: An RV whose CDF is a right-continuous, staircase
close all;
(piecewise constant) function of x.
FigureSet(1,’LTX’);
lambda = 2;
N = 25;
• Only takes on values from a finite set
R = exprnd(1/lambda,N,1); • Encounter often in applications involving counting
x = 0:0.02:max(R);
F = 1-exp(-lambda*x);
Continuous: An RV whose CDF is continuous everywhere
h = cdfplot(R);
hold on; • Can be writtenas an integral of some nonnegative function
plot(x,F,’r’,[0 100],[1 1],’k:’);
x
hold off;
grid off;
f (x): F (x) = −∞ f (u) du
set(gca,’XLim’,[0 max(R)]);
set(gca,’YLim’,[0 1.1]);
• Implies P [X = x] = 0 everywhere.
xlabel(’x’);
ylabel(’S(x)’); • In words, there is an infinitesimal probability X will be equal to
title(sprintf(’Exponential Emperical Distribution Function
box off;
N:%d’,N));
any specific number x.
AxisSet(8);
print -depsc ExponentialEDF;
• Nonetheless, an experiment will cause X to equal some value.
Mixed: An RV with a CDF that has jumps on a countable set of
x
Note: MATLAB defines the distribution as f (x) = λ1 e− λ u(x) rather points, but also increases continuously over one or more intervals.
than f (x) = λe−λx u(x). IOW, everything else.
Definition: Probability Density Function (PDF) Properties of the PDF
The probability density function (PDF) of a continuous RV is 1. f (x) ≥ 0
defined as the derivative of F (x): b
2. P [a ≤ X ≤ b] = a f (u) du
dF (x) x
f (x) = 3. F (x) = −∞ f (u) du
dx
+∞
Alternatively, 4. −∞ f (u) du = 1
F (x − ) + F (x + ) 5. A valid PDF can be formed from any nonnegative, piecewise
f (x) = lim
→0 2 continuous function g(x) that has a finite integral
6. The PDF must be defined for all real values of x
• Conceptually, it is more useful than the CDF
7. If X does not take on some values, this implies f (x) = 0 for those
• Does not technically exist for discrete or mixed RV’s values
– Can finesse with impulse functions
du(x)
– δ(x) = dx where u(x) is the unit step function
• PDF represents the density of probability at the point x
Example 4: Exponential PDF Example 4: MATLAB Code

function [] = ExponentialPDF();
Exponential CDF and PDF close all;
1 FigureSet(1,’LTX’);
lambda = 2;
F(x)
x = -0.5:0.005:2;
0.5 xl = [min(x) max(x)];
F = zeros(size(x));
id = find(x>=0);
F(id) = 1-exp(-lambda*x(id));
f = zeros(size(x));
0 f(id) = lambda*exp(-lambda*x(id));
−0.5 0 0.5 1 1.5 2
subplot(2,1,1);
h = plot(x,F,’b’,xl,[1 1],’k:’);
2 xlim(xl);
ylim([0 1.1]);
ylabel(’F(x)’);
1.5 title(’Exponential CDF and PDF’);
box off;
f(x)
1 subplot(2,1,2);
h = plot(x,f,’g’);
0.5 set(h,’LineWidth’,1.5);
xlim(xl);
ylim([0 2.1]);
0 xlabel(’x’);
−0.5 0 0.5 1 1.5 2 ylabel(’f(x)’);
box off;
x
AxisSet(8); Histograms
Let X1 , X2 , . . . , Xn be a random sample. The histogram is a function
print -depsc ExponentialPDF;
of x which equals the fraction of Xi s that are within specified intervals.

• Like the CDF, the “true” PDF is never known
• The histogram is a rough estimate of the PDF
• Usually shown in the form of a bar plot
• Minimum value: 0, Maximum value: ∞
• Is a random function
• Perhaps the most common graphical representation of estimated
PDFs
Example 5: Histograms Histogram Comments
Exponential Emperical Distribution Function N:100 • Histograms can be misleading

1 • The apparent shape of the histogram is sensitive to
True
0.5 – The bin locations

0
– The bin widths
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
• It can be shown that the bin width affects the bias and the
1 variance of this estimator of the PDF
Estimated
0.5
0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
1 True
Estimated
0.5
0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
x
Histogram Accuracy Example 5: Histograms with Different Bin Centers
∞ 2 Exponential Emperical Distribution Function N:100
ISE = fˆ(u) − f (u) du 1
−∞
True
0.5
1
BIAS fˆ(x) = 2 f (x) [h − 2(x − bj )] + O(h2 ) for x ∈ (bj , bj + 1]
f (x)
0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
Var fˆ(x) = + O(n−1 )
nh 1
Estimated
1 h2 R(f )
MISE = + + O(n−1 ) + O(h3 ) 0.5
nh 12
where h is the bin width, bj is the jth bin boundary, and 0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
R(φ) = φ(u)2 du
1 True
• The bin width controls the bias-variance tradeoff Estimated
0.5
• More on all of this later
0
0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
x
Example 6: Uniform Distribution Gaussian RV’s

Plot the CDF and PDF for a uniform random variable X ∼ U[a, b]. 1
Gaussian Distribution Function
0.4
Gaussian Density Function
Note: X ∼ U[a, b] denotes that X is drawn from a uniform 0.8

0.3
distribution and has a range of [a, b]. 0.6
F(x)
f(x)
0.2
0.4
0.1
0.2
0 0
−5 −4 −3 −2 −1 0 1 2 3 4 5 −5 −4 −3 −2 −1 0 1 2 3 4 5
x x
1 (x−m)2
f (x) = √ e− 2σ2
2πσ
x
1 (x−m)2
F (x) = P [X ≤ x] = √ e− 2σ 2 dx
2πσ −∞
• Denoted as X ∼ N (μX , σX
2
)
• Also called the normal distribution
• Arises naturally in many applications
• Central limit theorem (more later) Functions of RV’s
• We will work with functions of RV’s: Y = g(X)
• Y is also an RV
• Example: Y = aX + b
FY (y) = P [Y ≤ y]
= P [aX + b ≤ y]

y−b
= P X≤
a

y−b
= FX
a
for a > 0.
Expected Values Overview Expected Values Defined

+∞
• To completely describe all of the known information about an RV,
we must specify the CDF or PDF E[X] = x f (x) dx
−∞
• Given a data set, estimating the CDF/PDF is one of the most
difficult problems we will discuss (density estimation) • The expected value of a random variable X is denoted E[X]
• Often, much less information about the distribution of X is • This is called the mean of X
sufficient
• The expected value of X is only defined if the integral converges
– Mean absolutely: ∞
– Median
|x| f (x) dx < ∞
– Standard deviation −∞
– Range • The “best” estimate of the mean of X given a data set is the
• These scalar descriptive statistics are called point estimates sample average,
N
1
X̄ = xi ≈ E[x]
N i=1
Average versus Mean Expected Values of Functions
N
We can also calculate the expected values of functions of random
+∞
1 variables. Let Y = g(X). Then,
E[X] = μx = xf (x) dx x̄ = μ̂x = xi
−∞ N i=1 ∞
E[Y ] = g(x)f (x) dx
−∞
Note the distinction between the average and mean
Example Let g(X) = I(X) where I(X) is the indicator function of
• The average is the event {X in C}, where C is some interval in the real line:
– an estimate of the mean
– calculated from a data set 0 X not in C
g(X) =
– a random variable 1 X in C
• The mean is then

∞
– Calculated from a PDF
E[Y ] = g(x)f (x) dx = f (x) dx = P [X ⊂ C]
– Not a random variable −∞ C
– A property of the PDF Thus, the expected value of the indicator of an event is equal to the
probability of the event.
Expected Value Properties Variance

1. E[c] = c where c is a constant The variance of a random variable is defined as follows:
2. E[cX] = c E[X] σX
2
≡ E[(X − μx )2 ]
N N
3. E[ k=1 gk (X)] = k=1 E[gk (X)] The nth moment of an RV is defined as
∞
4. Proof left as a homework assignment E[X n ] ≡ xn f (x) dx
−∞
• Variance is a measure of how wide a distribution is

• A measure of dispersion
• There are others as well
√
• The standard deviation is defined as σ ≡ σ2
• σX
2
= E[X 2 ] − E[X]2
• Both are properties of the CDF and are not RVs
Markov Inequality Example 7: Markov Inequality
The mean and variance of a RV X give us sufficient information to The mean height of children in a kindergarten class is 3.5 feet. Find
establish bounds on certain probabilities. Suppose that X is a the bound on probability that a kid in the class is taller than 9 feet.
nonnegative random variable.
Markov inequality:
E[X]
P [X ≥ a] ≤
a
Proof
a ∞
E[X] = xf (x) dx + xf (x) dx
a
0
∞
≥ xf (x) dx
a
∞
≥ af (x) dx
a
= aP [X ≥ a]
Chebyshev Inequality Multiple Random Variables

σ2 A vector random variable is a function that assigns a vector of real
P [|X − μ| ≥ a] ≤ numbers to each outcome ζ in S, the sample space of the random
a2
experiment.
Proof Let D = (X − μ) where μ = E[X]. Then apply the Markov
2 2
inequality • Example: randomly select a student

E[(X − μ)2 ] σ2 • X ≡ [H(ζ), W (ζ), A(ζ)]
P [D ≥ a] = P [D2 ≥ a2 ] ≤ =
a2 a2 • Where
• These bounds are very loose – H(ζ) = height of student ζ
• Note: if σ 2 = 0, the Chebyshev inequality implies P [X = μ] = 1 – W (ζ) = weight of student ζ
– A(ζ) = age of student ζ
Jointly Continuous Random Variables Example 8: Jointly Continuous RV
Random variables X and Y are jointly continuous if the probabilities Gaussian Density Function f(x,y)
1.5
of events involving (X, Y ) can be expressed as an integral of a PDF.
0.25
In other words, there is a joint probability density function that is
defined on the real plane such that for any event A,
1 0.2
P [X, Y in A] = fX,Y (u, v) du dv
A
0.5 0.15
y
Properties
+∞ +∞
• −∞ −∞ fX,Y (u, v) du dv = 1
0.1
d2 FX,Y (x,y)
• fX,Y (x, y) = dx dy
0
0.05
−0.5
−0.5 0 0.5 1 1.5
x
Example 8: MATLAB Code Example 8: Jointly Continuous RV Continued
Gaussian Density Function f(x,y)

0.25
0.4
0.2
0.3
F(x,y)
0.2
0.15
0.1
0.1
0
2
1 1.5
1 0.05
0 0.5
0
y −1 −0.5 x
Joint Cumulative Distribution Function (CDF) Marginal PDF’s
Y Y x Y
FX,Y (x, y) FX (x) FY (y)
x y
y
X X X
The marginal PDF’s fX (x) and fY (y) are given as follows

There is also a joint CDF: ∞
x y fX (x) = fX,Y (x, y) dy
FX,Y (x, y) = fX,Y (u, v) du dv −∞
∞
−∞ −∞
= P [X ≤ x & Y ≤ y] fY (y) = fX,Y (x, y) dx
−∞
= P [X ≤ x, Y ≤ y]
• fX (x) is the same as the PDF of X, if Y had not been considered
• The marginal PDF can be obtained from the joint PDF
• The joint PDF cannot be obtained from the marginal PDF
Independence Conditional CDF’s & Bayes’ Theorem

Two random variables X and Y are independent if and only if their The conditional CDF of Y given X = x:
joint PDF is equal to the product of the marginal PDF’s.
FY (y|x) = lim FY (y|x < X ≤ x + h)
h→0
fX,Y (x, y) = fX (x)fY (y) y
−∞
fX,Y (x, y ) dy
Equivalently, they are independent if and only if their joint CDF is =
fX (x)
equal to the product of the marginal CDF’s.
Proof omitted.
FX,Y (x, y) = FX (x)FY (y)
The conditional PDF of Y given X = x:
d fX,Y (x, y)
fY (y|x) = FY (y|x) =
• If X and Y are independent, the random variables W = g(X) and dy fX (x)
Z = h(Y ) are also independent
• This can be viewed as a form of Bayes’ theorem
• Gives a posteriori probability that Y is close to y given that X is
close to x
Conditional CDF’s & Independence Conditional Expectation
If X and Y are independent The conditional expectation of Y given X = x is defined by
∞
fX,Y (x, y) = fX (x)fY (y)
EY [Y |X = x] = y fY (y|x) dy
−∞
and the conditional PDF of Y given X = x is
fX,Y (x, y) • EY [Y |X = x] can be viewed as a function of x: g(x) = EY [Y |x]
fY (y|x) =
fX (x)
• g(X) = EY [Y |X] is a random variable
fX (x)fY (y)
= • It can be shown that EY [Y ] = EX [g(X)] = EX [EY [Y |X]]
fX (x)
= fY (y) • More generally, EY [h(Y )] = EX [EY [h(Y )|X]] where
∞
∞
EX [EY [h(Y )|X]] = h(y)fY (y|x) dy fX (x) dx
−∞ −∞
Correlation and Covariance Correlation Coefficient

The jk th moment of X and Y is defined as The correlation coefficient of X and Y is defined as
∞ ∞ σX,Y
2
j k
E[X Y ] = xj y k fX,Y (x, y) dx dy ρX,Y =
σX σY
−∞ −∞ • −1 ≤ ρX,Y ≤ 1
• Extreme values of ρX,Y indicate a linear relationship between X
• The correlation of X and Y is defined as E[XY ] and Y : Y = aX + b
• If E[XY ] = 0, we say X and Y are orthogonal • ρX,Y = 1 implies a > 0, ρX,Y = −1 implies a < 0
• The covariance of X and Y is defined as • X and Y are said to be uncorrelated if ρX,Y = 0
• If X and Y are independent, σX,Y
2
= 0 (see homework)
σX,Y
2
= E[(X − μX )(Y − μY )]
• If X and Y are independent, ρX,Y = 0
• If ρX,Y = 0, X and Y may not be independent
• Uncorrelated variables are not necessarily independent
• However, if X and Y are Gaussian random variables, then
ρX,Y = 0 implies X and Y are independent
Mean Square Estimation Example 9: Minimum MSE Estimation
Observed Suppose we wish to estimate a random variable Y with a constant a.
Variables Output What is the best value of a that minimizes the MSE?
x1,...,x n Model y
• Often will want to estimate the value of one RV Y from one or

more other RVs X: Ŷ = g(X)
• Encounter often in nonlinear modeling and classification
• It may be that Y = g(X)
• The estimation error is defined as Y − g(X)
• We will assign a cost to each error c(Y − g(X))
• Goal: find G(X) that minimizes E[c(Y − g(X))]
• The most common cost function is mean squared error (MSE):
MSE = E[(Y − g(X))2 ]
Example 10: Minimum Linear MSE Estimation Example 10: Workspace (1)
Suppose we wish to estimate a random variable Y with a linear
function of X, Ŷ = aX + b. What values of a and b minimize the
MSE?
Example 10: Workspace (2) MMSE Linear Estimation Discussion
Ŷ = a∗ X + b∗

X − E[X]
= ρX,Y σY + E[Y ]
σX
• Note that X−E[X]
σX is just a scaled version of X
– Zero mean
– Unit variance
– Sometimes called a z score
X−E[X]
• Xs = σY σX has the variance of Y
• The term E[Y ] ensures that E[Ŷ ] = E[Y ]
• ρX,Y specifies the sign and extent of Y relative to Xs
• If uncorrelated, Ŷ = E[Y ]

X−E[X]
• If perfectly correlated, Ŷ = ±σY σX + E[Y ] = Y
Orthogonality Condition Best Linear Estimator MMSE

The orthogonality condition states that the error of the best linear
E {(Y − E[Y ]) − a∗ (X − E[X])}
2
estimator is orthogonal to the observation X − E[X]. MMSE =
2
• Fundamental result in mean square estimation ∗
= E Ỹ − a X̃
• Central to the area of linear estimation
• Enables us to more easily find the minimum MSE of the best = E Ỹ − a∗ X̃ Ỹ − a∗ E Ỹ − a∗ X̃ X̃
linear estimator
= E Ỹ − a∗ X̃ Ỹ
• The notation will be simplified by the following notation
= σY2 − a∗ σX,Y
2
X̃ = X − E[X] Ỹ = Y − E[Y ]
σX,Y
2
= σY −
2
σX,Y
2
• These are called centered random variables σX2
= σY2 (1 − ρ2X,Y )
Best Linear Estimator MMSE Nonlinear Estimation
• In general, the best estimator of Y given X will be nonlinear
E {(Y − E[Y ]) − a∗ (X − E[X])}
2
MMSE =
• Suppose we wish to find the g(X) that best approximates Y in

= σY2 1 − ρ2X,Y the MMSE sense
min EX,Y [(Y − g(X))2 ]
g(·)
• When ρX,Y = ±1, MMSE = 0 Using conditional expectation
• Perfect correlation implies perfect prediction
2 2
EX,Y (Y − g(X)) = EX EY (Y − g(x)) |X = x
• No correlation (ρX,Y = ±0) implies MMSE = σY2 ∞
2
= EY (Y − g(x)) |X = x fX (x) dx
−∞
∞
∞
= (y − g(x)) fY (y|x) dy fX (x) dx
2
−∞ −∞
Nonlinear Estimation Continued Random Vectors

∞
∞ Let X be a random vector,
EX,Y [(Y − g(X))2 ] = (y − g(x))2 fY (y|x) dy fX (x) dx ⎡ ⎤
−∞ −∞ X1
⎢ X2 ⎥
⎢ ⎥
X=⎢ . ⎥
• Integrand is positive for all x ⎣ .. ⎦
• Minimized by minimizing EY [(Y − g(x))2 |X = x] for each x XL
• g(x) is a constant relative to EY [·] Then, the expected value of X is defined as
• Reduces to the equivalent example earlier: estimate Y with a ⎡ ⎤
E[X1 ]
constant g(x) ⎢ E[X2 ] ⎥
• Therefore, the g(x) that minimizes the MSE is ⎢ ⎥
E[X] = ⎢ . ⎥
⎣ .. ⎦
Ŷ = g ∗ (x) = EY [Y |X = x] E[XL ]
∗
• The function g (x) is called the regression curve
• Has the smallest possible MSE
• Linear estimators are generally worse (larger MSE)
Linear Estimation with Vectors Linear Estimation Error
Suppose that we wish to estimate Y with a linear sum of random T
variables X1 , X2 , . . . XL . ε2 = (Y − X w)2
T T T
Ŷ =
T
X w = Y 2 + w XX w − 2Y X w
L
The expected value of the squared error is
= X i wi
i=1 MSE ≡ E[ε2 ]
T
Then the error Y − Ŷ can be written as = E[(Y − X w)2 ]
T T T
ε = Y −X w
T = E[Y 2 ] + w E[XX ]w − 2 E[Y X ]w
and the squared error can be written as

T
ε2 = (Y − X w)2
T T T
= Y 2 + w XX w − 2Y X w
Correlation Matrix Cross-Correlation Matrix

Let X be a zero-mean random vector. The variance-covariance of Let Y be a zero-mean scalar random variable. Define P , the cross
the vector X, also called the correlation matrix, can be written as correlation matrix, as
R = σ 2 {X} P = E[Y X]
T
⎡ ⎤
= E[XX ] E[Y X1 ]
⎡ 2 ⎤ ⎢ E[Y X2 ] ⎥
σ X1 σX
2
. . . σX
2 ⎢ ⎥
1 ,X2 1 ,XL = ⎢ .. ⎥
⎢ σX
2
σX2
. . . σX
2 ⎥ ⎣ . ⎦
⎢ 2 ,X1 2 2 ,XL ⎥
= ⎢ . .. .. .. ⎥ E[Y XL ]
⎣ .. . . . ⎦
σX
2
L ,X1
σX
2
L ,X2
... σXL
2
T
R is a symmetric matrix: R = R.
Minimum Mean Squared Error Minimum Mean Squared Error
Using the matrices R and P , the MSE can be rewritten w∗ = R−1 P
T
E[ε2 ] = E[(Y − X w)2 ] Find the minimum MSE by substitution into the equation for the MSE
T T T
= E[Y ] + w E[XX ]w − 2 E[Y X ]w
2 T
T T
min E[ε2 ] = σY2 + w∗ Rw∗ − 2P w∗
= σY2 + w Rw − 2w P T
= σY2 + (R−1 P ) R(R−1 P ) − 2P (R−1 P )
Take the gradient of the MSE above with respect to w and set the =
T
σY2 + P R−1 P − 2P R−1 P
resulting expression equal to zero. T
T T = σY2 − P R−1 P
∇w E[ε ] 2
= ∇w (σY2 + w Rw − 2w P ) T
T = σY2 + P w∗
= R w + Rw − 2P
= 2Rw − 2P
= 0
Solving for w∗ , we obtain
w∗ = R−1 P
Closing Comments
• In general, we cannot calculate anything discussed so far
• Everything discussed requires the true PDF (or CDF) be known
• In practice, we’ll have data, not PDF’s
• Represents a best-case scenario
• How close can we approximate the true point estimate given only
data
• Will compare our estimators on cases where the true PDF is known
J. McNames Portland State University ECE 4/557 Random Variables Ver. 1.06 67

Random Variables

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Random Variables

Uploaded by

Copyright:

Available Formats

Random Variables Overview Deﬁnition: Random Variable

• Functions of random variables Range

Example 1: Random Variable Deﬁnitions Cumulative Distribution Function

Example 2: Distribution Plot Example 2: MATLAB Code

h = plot(x,y,’b’,[0 100],[1 1],’k:’);

• The “true” CDF is never known

Example 3: MATLAB Code Random Variable Types

Example 4: Exponential PDF Example 4: MATLAB Code

of x which equals the fraction of Xi s that are within speciﬁed intervals.

Example 5: Histograms Histogram Comments

Exponential Emperical Distribution Function N:100 • Histograms can be misleading

0.5 – The bin locations

Example 6: Uniform Distribution Gaussian RV’s

Note: X ∼ U[a, b] denotes that X is drawn from a uniform 0.8

Expected Values Overview Expected Values Deﬁned

• The mean is then

Expected Value Properties Variance

• Variance is a measure of how wide a distribution is

Chebyshev Inequality Multiple Random Variables

inequality • Example: randomly select a student

Example 8: MATLAB Code Example 8: Jointly Continuous RV Continued

Gaussian Density Function f(x,y)

The marginal PDF’s fX (x) and fY (y) are given as follows

Independence Conditional CDF’s & Bayes’ Theorem

Correlation and Covariance Correlation Coeﬃcient

• Often will want to estimate the value of one RV Y from one or

Orthogonality Condition Best Linear Estimator MMSE

Nonlinear Estimation Continued Random Vectors

and the squared error can be written as

Correlation Matrix Cross-Correlation Matrix

You might also like