Properties and Functions of Random Variables (RVs

Properties of RVs
INSTRUCTOR: SHAHROKH FARAHMAND

Function of a RV: General formula
SLIDES BY SHAHROKH FARAHMAND 2

Example. Linear transformation

Linear transformation of a Gaussian

Example. Square law detector

Example. Sine wave

Example. Sine wave 2

Example. Linear amplifier with cut-off

Linear amplifier with cut-off

Rule of thumb for discrete and general RVs

Half-wave rectifier

Half-wave rectifier 2

Neat property of uniform distribution

Functions of multiple RVs
Examples include the following problems:

Functions of multiple RVs 2

Functions of multiple RVs 3

Ex. Multiplier
Applications:
1) Mixer output
2) Flat-fading channels

Ex. Multiplier 2
Leibniz rule:

Ex. Multiplier 3

Ex. Maximum RV

Ex. Maximum RV 2

Sum RVs

Sum RVs 2

Sum of independent Poisson

Functions of Gaussians: Exponential
Therefore, Z is
exponential or chi-
squared with two
degrees of freedom
Functions of Gaussians 2: Rayleigh

Functions of Gaussians: Rice

Rice distribution

Two functions of two RVs

Two functions of two RVs

Ex. Linear transformation

Ex. NLOS fading

Ex. NLOS fading 2

Ex. NLOS fading 3

Rotational invariance of jointly Gaussian

Rotational invariance of JG 2

Expected value (mean value)
Question. We can compute mean from PDF. Can we compute PDF

from mean?
Ans. No. But if all the moments are calculated, then PDF can be
computed (will clarify later on)
Note that PDF / Joint PDF give a complete characterization of a RV
Moments
Second moment
Second central moment (a.k.a. Variance)
k’th moment
k’th central moment

Moments: Intuition
In general, Joint /Marginal PDFs can be difficult to compute, so one
opts for a partial characterization of the distribution instead of a
complete one. Moments provide such a partial characterization. If all
moments are compute, then one can reconstruct the PDF which
yields a complete characterization of the RV.
Usually, we work only with the first two moments

Linearity of expected value
Proof idea. Expected value is an integration and integration is linear

in that integral of a summation equals summation of integrals. So,
this property is inherited by expected value.
Example.

Gaussian distribution
One can show that
Knowing the first two moments of Gaussian completely defines the

distribution
Poisson/Binomial distribution
For Poisson with parameter a:
For Binomial with parameters n,p:

Further examples
Question. What can you say about X and Y from the PDF?
Question. Two ways to compute E[XY]. What are they?
1) Direct substitution
2) Independent property

Further examples 2

Conditional expectation

Conditional expectation 2

Properties of conditional mean

Law of total variance

Distributions with hyper-parameters
Consider a buffer where number of packets that arrive to the buffer
are Poisson with an arrival rate X. While X is deterministic in most
cases, the network corresponding to this buffer suffers from random
congestions. Hence, X is random.

Distributions with hyper-parameters 2

Geometric distribution
Repeat a Bernoulli experiments until first success
Ex. Suppose you keep tossing a fair coin until you observe two
consecutive heads. Define X to be the number of coin tosses until
success. What is E[X]?
Ans. We need Markov chain for this. But, there is a shortcut:

Geometric distribution 2
After some algebra:
Will compare against the result of a Markov chain

Example. Poisson RVs

Example. Poisson RVs

Absolute moments
Important property
Two proofs
1)
2) Triangle inequality

Joint moments
Joint central moments are defined as
Generally, we work with the first two moments

Second joint moments: Correlation/Covariance

Uncorrelated RVs
Uncorrelated RVs are intuitively said to be linearly independent

For independent RVs, we have
For example:
Independent RVs are uncorrelated but uncorrelated RVs are not
necessarily independent
The only exception is Jointly Gaussian where uncorrelatedness means
indepndence
Correlation coefficient
Proof:
Note that if CC equals zero then the two RVs are uncorrelated
If CC equals +1 or -1 then they are completely correlated

Linear relation when CC absolute value equals 1
When |CC|=1 (fully correlated scenario), one can show
From this we can conclude that
Note that there exists a linear relation between Y,X

If we know X, we know Y. Given Y=y, X becomes deterministic
Question. If CC=0, can we say anything about the joint PDF?
Variance of sum of uncorrelated RVs
Define
Suppose X,Y are uncorrelated. We have the following relation:

Example. Optimal linear estimator

Example. Optimum linear estimator 2



Uncorrelated but not independent

Jointly Gaussian RVs

Marginal Gaussians
Gaussian bell is modified:

Individually Gaussian but
not jointly Gaussian

Contours of jointly Gaussian RVs

Contours of jointly Gaussian RVs 2

Quadratic forms and jointly Gaussian RVs

General jointly Gaussian distribution

Markov/Chebyshev inequalities

Weak law of large numbers (WLLN)

Ex. Markov inequality

Cauchy-Schwarz inequality

Example

Moment generating function (MGF)

Ex. MGF for Gaussian

Ex. MGF for Gaussian 2

MGF for Binomial

MGF for Geometric

MGF for sum of independent RVs

Chernoff bound

Ex. Chernoff bound for Poisson

Ex. Chernoff bound for Poisson 2

Characteristic function

Ex. Sum of independent Gaussians/ Poissons

Ex. Sum of independent Binomials

Ex. Soccer betting

Fair soccer betting

Central limit theorem (CLT)

CLT proof

Levy’s continuity theorem

Application of CLT

Application of CLT 2

Limitations of CLT

Properties and Functions of Random Variables (RVs

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Properties and Functions of Random Variables (RVs

Uploaded by

Copyright:

Available Formats

Properties of RVs

INSTRUCTOR: SHAHROKH FARAHMAND

SLIDES BY SHAHROKH FARAHMAND 2

SLIDES BY SHAHROKH FARAHMAND 3

SLIDES BY SHAHROKH FARAHMAND 4

SLIDES BY SHAHROKH FARAHMAND 5

SLIDES BY SHAHROKH FARAHMAND 6

SLIDES BY SHAHROKH FARAHMAND 7

SLIDES BY SHAHROKH FARAHMAND 8

SLIDES BY SHAHROKH FARAHMAND 9

SLIDES BY SHAHROKH FARAHMAND 10

SLIDES BY SHAHROKH FARAHMAND 11

SLIDES BY SHAHROKH FARAHMAND 12

SLIDES BY SHAHROKH FARAHMAND 13

SLIDES BY SHAHROKH FARAHMAND 14

SLIDES BY SHAHROKH FARAHMAND 15

SLIDES BY SHAHROKH FARAHMAND 16

SLIDES BY SHAHROKH FARAHMAND 17

SLIDES BY SHAHROKH FARAHMAND 18

SLIDES BY SHAHROKH FARAHMAND 19

SLIDES BY SHAHROKH FARAHMAND 20

SLIDES BY SHAHROKH FARAHMAND 21

SLIDES BY SHAHROKH FARAHMAND 22

SLIDES BY SHAHROKH FARAHMAND 23

SLIDES BY SHAHROKH FARAHMAND 24

SLIDES BY SHAHROKH FARAHMAND 26

SLIDES BY SHAHROKH FARAHMAND 27

SLIDES BY SHAHROKH FARAHMAND 28

SLIDES BY SHAHROKH FARAHMAND 29

SLIDES BY SHAHROKH FARAHMAND 30

SLIDES BY SHAHROKH FARAHMAND 31

SLIDES BY SHAHROKH FARAHMAND 32

SLIDES BY SHAHROKH FARAHMAND 33

SLIDES BY SHAHROKH FARAHMAND 34

SLIDES BY SHAHROKH FARAHMAND 35

SLIDES BY SHAHROKH FARAHMAND 36

Question. We can compute mean from PDF. Can we compute PDF

Second central moment (a.k.a. Variance)

k’th central moment

SLIDES BY SHAHROKH FARAHMAND 38

Usually, we work only with the first two moments

SLIDES BY SHAHROKH FARAHMAND 39

Proof idea. Expected value is an integration and integration is linear

SLIDES BY SHAHROKH FARAHMAND 40

One can show that

Knowing the first two moments of Gaussian completely defines the

For Binomial with parameters n,p:

SLIDES BY SHAHROKH FARAHMAND 42

SLIDES BY SHAHROKH FARAHMAND 43

SLIDES BY SHAHROKH FARAHMAND 44

SLIDES BY SHAHROKH FARAHMAND 45

SLIDES BY SHAHROKH FARAHMAND 46

SLIDES BY SHAHROKH FARAHMAND 47

SLIDES BY SHAHROKH FARAHMAND 48

SLIDES BY SHAHROKH FARAHMAND 49

SLIDES BY SHAHROKH FARAHMAND 50

SLIDES BY SHAHROKH FARAHMAND 51

After some algebra:

Will compare against the result of a Markov chain

SLIDES BY SHAHROKH FARAHMAND 52

SLIDES BY SHAHROKH FARAHMAND 53

SLIDES BY SHAHROKH FARAHMAND 54

SLIDES BY SHAHROKH FARAHMAND 55