24 views

Uploaded by Vishal Chetal

m

m

© All Rights Reserved

- EB510 Bro En
- pekcan-mceer-99-21
- Normal Distribution.pptx
- FROM INPUT-OUTPUT TO OUTPUT-ONLY MODAL IDENTIFICATION OF CIVIL ENGINEERING STRUCTURES
- Bet Smarter With the Monte Carlo Simulation
- Article VCB2010 FMarin
- Probability-Theory-Problems.pdf
- Vissarion Papadopoulos, Dimitris G. Giovanis-Stochastic Finite Element Methods. an Introduction-Springer (2018)
- SP-07 sample 2013.pdf
- Gravity Procesing
- Tutorial 3
- The Hyper Geometric Distribution
- Art 5
- LP 01-03-2019
- bsc4q
- notes-10
- MA1014probnability & Queueing Theory
- Appendix
- Lesson 2-03 Random Variables STAT
- Assesment and Structural Rehabilitation With Posttensioning

You are on page 1of 71

Introduction and Denitions . . . . . . . . . . . . . . . . . . . . . . . Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Conditional Probability and Statistical Independence . . . . . . . . . . . Random Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . Cumulative Distribution Function and Probability Density Function . . . . The Histogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Some Important Probability Distributions . . . . . . . . . . . . . . . . 6.7.1 6.7.2 6.7.3 6.7.4 6.7.5 The Uniform Distribution . . . . . . . . . . . . . . . . . . . . The Binomial Distribution . . . . . . . . . . . . . . . . . . . . The Poisson Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Exponential Distribution 3 5 10 15 17 21 25 25 26 28 29 30 32 37 40 40

1

6.8 6.9

Statistical Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . Joint and Conditional Density Functions . . . . . . . . . . . . . . . . . 6.10.1 Denition and Examples . . . . . . . . . . . . . . . . . . . . .

6.10.2 Ensemble Averages and Stationarity . . . . . . . . . . . . . . . 6.10.3 Time Averages and Ergodicity . . . . . . . . . . . . . . . . . . 6.11 Autocorrelation and Power Spectra . . . . . . . . . . . . . . . . . . . 6.11.1 Autocorrelation and Autocovariance . . . . . . . . . . . . . . . 6.11.2 Power Spectral Density . . . . . . . . . . . . . . . . . . . . . 6.11.3 Deterministic Power and Energy Signals . . . . . . . . . . . . . 6.12 Excitation of LTI Systems with Stationary Random Processes . . . . . . . 6.12.1 Expected Value of the Output Random Process . . . . . . . . . 6.12.2 Autocorrelation Function of the Output Random Process . . . . . 6.12.3 Power Spectral Density of the Output Random Process . . . . . . 6.12.4 Cross-Correlation between Input and Output Random Process . . .

42 43 49 49 53 55 66 66 67 68 70

6.1

Deterministic Signals:

Random Signals:

can be expressed in form of a mathematical equation can be reproduced exactly with repeated measurements

Examples for Random Signals:

Number of car accidents in Lubbock during 1 year Quantization noise in an A/D converter Background noise in speech transmission Speckle noise in synthetic aperture radar images Tossing a coin

Life expectance of an engine Outcome of a roulette game Data transmission Bio-medical signals Rolling a dice

Random Experiment: Experiment with random outcome that is repeatable and for which the outcome of each experiment is not predictable.

Sample Space S : Set of all possible outcomes if a random experiment. Examples: 1. Tossing a coin 2. Rolling a die 3. Rolling two dice 4. No. of phone calls per day 5. Life expectance of an engine 6. Angular position of a pointer 7. Quantization error in an ADC

S1 S2 S3 S4 S5 S6 S7

= = = = = = =

{head, tail} {1, 2, 3, 4, 5, 6} S2 S2 ={(1,1),(1,2),. . . ,(6,5), (6,6)} 0 , 1, 2, 3, . . . [0, ) [0, 2 ) [0, Q) or S7 = [Q/2, Q/2)

Tossing a coin: S = {head, tail}, A1 = {head}, A2 = {tail}, A3 = {head, tail} Rolling a die: S = {1, 2, 3, 4, 5, 6}, A1 = {3}, A2 = {2, 4, 6}, A3 = {# 5} Quantization error in an ADC: S = [Q/2, Q/2), A1 = [0, Q/2), A2 = Q/4, A3 = (Q/4, Q/4)

6.2

Probability

Denition: In order for an operator P that assigns a real number, called the probability, to each event A in the sample space S , to be a valid probability assignment, it has to satisfy the following 3 axioms:

0 P (A) 1 P (S ) = 1 if A1 and A2 are mutually exclusive (or disjoint) events, i.e. the occurrence of one outcome precludes the occurrence of the other, then: P (A1 or A2) = P (A1 A2) = P (A1 + A2) = P (A1) + P (A2)

Venn Diagram:

S

A2 A1

P (head) = P (A1) = 0.5, P (tail) = P (A2) = 0.5, P (head or tail) = P (A3) = P (A1) + P (A2) = P (S ) = 1

6.2

Probability

Properties of Probabilities:

) = 1 P (A), P (A P () = 0

P (A1 + A2) = P (A1) + P (A2) P (A1A2) with P (A1A2): probability of the joint occurrence of A1 and A2

S

S A1 A2 A1 A2

6 : k=1

Dr. Tanja Karp 6.2 Probability 6

P (S ) = P (1 + 2 + 3 + 4 + 5 + 6) =

P (k ) = 1

Probability of an odd number:

1 6

Probability of a number smaller 5:

3 6

Example: Number of Phone Calls to a Given Number During a Certain Time Period Sample space: S = I N0 = 0, 1, 2, 3, . . . Elementary event Ai: exactly i calls come in.

: i=0

P (Ai) = 1

6.2

Probability

Estimation of Probability P (A): (A) for the probability P (A) that the event A occurs is given by the An estimate P relative frequency of occurrence of A: (A) = NA P N with N : number of times the experiment is repeated NA: number of times the event A occurred The estimate is the more accurate the larger N is. For N the deviation from the probability P (A) generally tends towards zero. Example: Tossing a Coin A fair coin is tossed 100 times, head occurs 51 times, tail occurs 49 times. (head) = 51 , (tail) = 49 P P 100 100

6.2

Probability

Example: Rolling a Fair Die The following plots show the relative frequency of occurrence of the event that the number is 1 or 2 when rolling the die 100 times and 800 times.

result is 1 or 2 1 relative frequence of occurrence relative frequence of occurrence 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0 20 40 60 no. of experiments 80 100 0.4 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 0 200 400 600 no. of experiments 800 result is 1 or 2

Compare to:

P (1 + 2) = P (1) + P (2) =

2 = 0.33 6

6.2

Probability

6.3

Conditional Probability P (B |A): Probability of the event B knowing that event A has occurred.

Conditional Probability P (A|B ): Probability of the event A knowing that event B has occurred.

P (BA) = P (B ) P (A|B )

With P (AB ) = P (BA) we obtain Bayes Theorem:

P (B )P (A|B ) P (A)

6.3

10

Example: Binary Data Transmission A binary data source sends a 0 with probability 0.4 and a 1 with probability 0.6. The data is transmitted through an erroneous channel. The probability for a 0 to be received correctly is 0.9, the probability for a 1 to be received correctly is 0.95. In case of an error, a 1 is received instead of a 0 and vice versa.

transmit signal x P(x=0)=0.4 0 erroneous transmission channel P(y = 0|x = 0) = 0.9 receive signal y 0 P(y=0)

P(y = 1|x = 0) = 0.1 P(y = 0|x = 1) = 0.05 P(x=1)=0.6 1 P(y = 1|x = 1) = 0.95 1 P(y=1)

Probability to receive a 0 if a 0 was sent: P (y = 0|x = 0) = 0.9 Probability to receive a 1 if a 0 was sent: P (y = 1|x = 0) = 0.1 Probability to receive a 1 if a 1 was sent: P (y = 1|x = 1) = 0.95 Probability to receive a 0 if a 1 was sent: P (y = 0|x = 1) = 0.05

Dr. Tanja Karp 6.3 Conditional Probability and Statistical Independence 11

Joint Probabilities:

Probability to send a 0 and receive a 0: P (x = 0 y = 0) = P (x = 0) P (y = 0|x = 0) = 0.4 0.9 = 0.36 Probability to send a 0 and receive a 1: P (x = 0 y = 1) = P (x = 0) P (y = 1|x = 0) = 0.4 0.1 = 0.04 Probability to send a 1 and receive a 1: P (x = 1 y = 1) = P (x = 1) P (y = 1|x = 1) = 0.6 0.95 = 0.57 Probability to send a 1 and receive a 0: P (x = 1 y = 0) = P (x = 1) P (y = 0|x = 1) = 0.6 0.05 = 0.03

Probabilities to receive a 0 or 1:

Probability to receive a 0: P (y = 0) = P (x = 0 y = 0) + P (x = 1 y = 0) = 0.36 + 0.03 = 0.39 Probability to receive a 1: P (y = 1) = P (x = 0 y = 1) + P (x = 1 y = 1) = 0.04 + 0.57 = 0.61

6.3

12

Conditional Probabilities (knowing what happens at the receiver): Probability that a 0 was sent if a 0 is received: P (x = 0 y = 0) P (x = 0)P (y = 0|x P (x = 0|y = 0) = = P (y = 0) P (y = 0) Probability that a 1 was sent if a 0 is received: P (x = 1 y = 0) P (x = 1)P (y = 0|x P (x = 1|y = 0) = = P (y = 0) P (y = 0) Probability that a 1 was sent if a 1 is received: P (x = 1 y = 1) P (x = 1)P (y = 1|x P (x = 1|y = 1) = = P (y = 1) P (y = 1) Probability that a 0 was sent if a 1 is received: P (x = 0 y = 1) P (x = 0)P (y = 1|x P (x = 0|y = 1) = = P (y = 1) P (y = 1)

transmit signal x 0 erroneous transmission channel P(x = 0|y = 0) = 0.923

= 0)

= 0.923

= 1)

= 0.077

= 1)

= 0.934

= 0)

= 0.066

Dr. Tanja Karp 6.3

P(y=1)=0.61

13

Statistical Independence: Two events A and B are called statistically independent, if P (B |A) = P (B ) and thus:

Example: Rolling Three Dice Determine the probability that all three dice show a 6.

Example: Binary Data Transmission Check for the binary data transmission example, whether the events x and y are statistically independent. In case of statical independence we have:

P (x = i y = j ) = P (x = i) P (y = j ),

i, j = 0, 1

P (x = 0) P (y = 0) = 0.4 0.39 = 0.156 = P (x = 0 y = 0) = 0.36 P (x = 0) P (y = 1) = 0.4 0.61 = 0.244 = P (x = 0 y = 1) = 0.04 P (x = 1) P (y = 1) = 0.6 0.61 = 0.366 = P (x = 1 y = 1) = 0.57 P (x = 1) P (y = 0) = 0.6 0.39 = 0.234 = P (x = 1 y = 1) = 0.03

Dr. Tanja Karp 6.3 Conditional Probability and Statistical Independence 14

6.4

Motivation:

Random Variables

So far, the outcomes of a random experiment were described verbally, e.g. head and tail for tossing a coin. The description of an event A can become long and cumbersome. Random Variable: A random variable X maps each of the possible outcomes of a random experiment to a real number. X : SI R

random experiment real axis X (1 ) X (2 ) 1 * 3 * sample space S * 2 X (3 )

6.4

Random Variables

15

Examples:

Tossing a coin: S = {head, tail} X (head) = 0, Rolling a die: S = {1, 2, 3, 4, 5, 6} X (1) = 1, X (2) = 2, X (3) = 3, X (4) = 4, X (5) = 5, X (6) = 6

Another valid but less intuitive mapping is:

X (tail) = 1

X (1) = 5.6, X (2) = , X (3) = 0.5, X (4) = 34.6, X (5) = 99, X (6) = 56.7 Rolling two dice: S = {(1, 2) : 1, 2 {1, 2, 3, 4, 5, 6}}

Event A: sum of numbers is larger than 8

A = {(1, 2) : 1 + 2 > 8}

Random variable:

X ((1, 2)) = 1 + 2

Probability:

P (A) = P (X > 8)

6.4

Random Variables

16

Event A: angle lies in the interval [/4, /2)

A = { : [/4, /2)}

Random variable:

X () = /rad

Probability:

6.5

The cumulative distribution function FX (x) associated with a random variable X is dened as: FX (x) = P (X () x) The probability density function (pdf), denoted by p(x) or pX (x), of a random variable X with cumulative distribution function FX (x) is given by:

d p X (x ) = FX (x) or FX (x) = dx

Dr. Tanja Karp

pX (t) dt

17

0 FX (x) 1 for all x I R FX (x1) FX (x2) for all x1, x2 I R with x1 < x2 lim FX (x) = 1,

x x

lim FX (x) = 0

xx0 +0

For a discrete random variable X (i) with associated probabilities P (i), the cumulative distribution function can be expressed as: FX (x) =

:

i

P (i)u(x X (i))

4

p(x) dx = 1

For a discrete random variable X (i) with associated probabilities P (i), the probability density function can be expressed as: p(x) =

Dr. Tanja Karp

: i=0

P (i) (x X (i))

18

S = {head, tail},

Random Variable:

X (head) = 0,

Cumulative distribution function:

X (tail) = 1

&

FX (x) =

Probability density function:

FX (x) 1 0.5 x 0

Dr. Tanja Karp

p(x)

(0.5)

(0.5) x

2

19

Example: Angular Position of a Pointer on a Rotating Wheel Sample Space: S = [0, 2 ) All angles in the range from 0 to 2 are equally likely

&

p(x) =

p(x)dx =

0

p(x)dx = 2K = 1

&

p(x) =

&

FX (x) =

p(x)

FX (x) 1 0.5 x

1/2

20

3 3 P( < X ) = FX ( ) FX ( ) = 2 2 2 2

Probability that the angle is exactly :

3/2

p(t) dt =

/2

1 2

P (X = ) =

p(x) dx = 0

6.6

Denition:

The Histogram

Tabulation of the frequency of occurrence of a random variable within preset ranges. The total number of events is equal to the number of sets (also called cells). The normalized histogram is obtained by dividing the height of each cell by the total number of events. The normalized histogram provides an estimate for the probability density function p(x) of a random variable.

6.6

The Histogram

21

S = {head, tail},

250

X (head) = 0,

X (tail) = 1,

0.7 0.6

x

0 x

6.6

The Histogram

22

&

S = [0, 2 ),

X (angle) = angle/rad,

p(x) =

Normalization of the histogram for a continuous random variable: N (xi) = xi xi P N where xi denotes a cell of width xi.

Histogram: 400 experiments, 10 cells 50 15 Histogram: 400 experiments, 50 cells

40 10 30

x

20 5 10

Nx 0 1 2 3 x 4 5 6 0 0

3 x

6.6

The Histogram

23

x

200

60

40

20

0.05

0.05

3 x

3 x

6.6

The Histogram

24

6.7

6.7.1

&

p(x) =

&

1 for a x b ba 0 otherwise FX (x) =

for x < a

p(x) 1 1 (b a) x 0 0.5

FX (x)

25

6.7.2

Lets assume that a random experiment has two mutual exclusive events A and B with probabilities P (A) = p and P (B ) = 1 p = q . The experiment is repeated n times. The random variable X is assigned the value i if event A occurs exactly i times out of the n repetitions. The probability for X = i, i = 0, . . . , n is given by:

P (X = i) = n i

pq

i n i

,

binomial coecient:

n i

n! i!(n i)!

Number of combinations to place i occurrences of event A on n possible positions: n! n(n 1)(n 2) (n i + 1) = (n i)! Out of these there are i! combinations that we cannot distinguish between. Probability density function and cumulative distribution function:

p(x) =

n : i=0

P (X = i) (x i) =

n : n i=0

pq

i n i

(x i)

FX (x) =

n : i=0

P (X = i)u(x i) =

n : n i=0

pq

i n i

u(x i)

26

Example: Rolling a die n times A die is rolled 4 times. Sketch p(x) and FX (x) for the event that a 3 occurs on top of die i = 0, . . . , 4 times. Event A: number on top of die is 3, P (A) = p = 1/6. Event B : number on top of die is not 3, P (B ) = q = 1 p = 5/6. Probability density function:

4 : i=0 4
: 4 i=0

p(x) =

P (X = i) (x i) =

(1/6) (5/6)

4i

(x i)

= = = = =

4! 4 4 (5/6) = (5/6) = 0.482 4! 0! 4! 1 3 1 3 (1/6) (5/6) = 4 (1/6) (5/6) = 0.386 3! 1! 4! 2 2 2 2 (1/6) (5/6) = 6 (1/6) (5/6) = 0.116 2! 2! 4! 3 1 3 1 (1/6) (5/6) = 4 (1/6) (5/6) = 0.015 1! 3! 4! 4 4 3 (1/6) = (1/6) = 7.71 10 4! 0!

27

FX (x)

6.7.3

The exponential distribution depends on a parameter t. It approximates the binomial distribution for large n. ti P (X = i) = exp(t) i! Probability density function and cumulative distribution function:

p(x) =

:

i

ti exp(t) (x i), i!

F X (x ) =

:

i

ti exp(t) u(x i) i!

Examples: number of particles emitted over a time duration in radioactive decay, number of telephone calls arriving at a switching venter during a certain time period, binary data transmission with low error rates

Dr. Tanja Karp 6.7 Some Important Probability Distributions 28

6.7.4

&

for x < 0

&

for x < 0

probability density function 2 t=2 t=1 t=0.5 1.5

FX (x) =

1 exp(xt) for x 0

cumulative distribution function

0.8

0 1

Examples: characterization of waiting times for processes without aging, e.g. time period between two calls at a calling center, time to repair a defect component of a system

Dr. Tanja Karp 6.7 Some Important Probability Distributions 29

6.7.5

The Gaussian distribution depends on parameters m I R and > 0. It is also denoted as N (m, 2) distribution.

p(x) =

x

1 2 1

exp

(x m) 2 2

exp

FX (x) =

probability density function 0.8 0.7 0.6 0.5 0.4 0.3 0.2 m=1, =3 m=1, =1 m=3, =0.5

(y m) 2 2

dy

0.8

p(x)

0.1 0 5 4 3 2 1 0 x 1 2 3 4 5

30

The integral for the cumulative distribution function FX (x) cannot be evaluated in closed form and requires numerical evaluation. Normalized Gaussian Distribution:

m+k

P (X (m + k )) =

1 2

exp

(y m) 2 2

dy

Substitute z = (y m)/ :

P (X (m + k )) =

1 exp 2

z 2

dz

Q(x) =

x

1 exp 2

z 2

dz

we obtain

P (X (m + k )) = 1 Q(k)

A random variable has a Gaussian distribution if it depends on many independent random variables with identical distribution. Examples: Electrical energy consumption in a metropolitan area, noise in a transmission channel, arbitrary measurement errors

Dr. Tanja Karp 6.7 Some Important Probability Distributions 31

6.8

Statistical Averages

A random variable is completely determined by its probability density function p(x) or its cumulative distribution function FX (x), respectively. In many cases p(x) and FX (x) are dicult to estimate and knowledge about the average outcome of a random experiment is sucient (e.g. average life expectance of an engine, average time spent in a waiting queue, average gain/loss of a slot machine). Expected Value E {X }: For a random variable X with probability density function p(x), the expected value E {X } = mX is given by:

E {X } =

x p(x)dx = mX

If X is a discrete random variable that takes the value xi, i = 1, 2 . . . N < , with probability P (X = xi), then

p(x) =

N : i=1

P (X = xi) (x xi),

E {X } =

N : i=1

xi P (X = xi)

The expected value is also called mean value, average value, or rst moment.

6.8

Statistical Averages

32

n

x p(x)dx

A knowledge of all the moments of a random variable, provided they are nite, species a random variable as completely as a knowledge of the probability density function. The second moment is called the mean-square value of X and its square root is called the root mean square (rms) value of X .

n

(x mX ) p(x)dx

The second order central moment E {(X mX )2} is called the variance of a random variable X . It measures the average quadratic deviation of X from its average value mX .

2 X

= E {|(X mX )| } =

(x mX ) p(x)dx

X is called the standard deviation of X and is an indicator for the eective width of the probability density function (the larger X the more spread out is the pdf of X ).

Chebyshevs Inequality:

Dr. Tanja Karp 6.8 Statistical Averages

33

S = {head, tail},

X (head) = 0,

X (tail) = 1,

p(x) (0.5) (0.5)

0 1 2

Average value mX :

2 Variance X :

E {X } =

x p(x)dx

E {(X mX ) } =

(x 0.5) p(x) dx

= 0.5

= 0.5

(x 0.5) (x) + (x 1) dx

2 2

Standard deviation: X = 0.5

= 0.25

6.8

Statistical Averages

34

S = [0, 2 ),

Probability density function:

X (angle) = angle/rad

p(x) 1/2

&

p(x) =

0

2 Variance X :

Average value mX :

E {X } =

x p(x)dx

2

E {(X mX ) } =

0

(x ) p(x) dx 1 2 3

1 = 2

x dx =

0

1 2

(x ) dx =

0

Standard deviation: X =

1 3

6.8

Statistical Averages

35

Y = g (X ),

with X , Y : random variables

E {Y } = E {g (X )} =

g (x)p(x)dx

Repeatedly taking the mathematical expectance: E {E {X }} = E {X } Properties of the variance: X = E {(X mX ) } = E {X } mX Y = aX + b, Y = E {(Y mY ) } = a X

2 2 2 2 2 2 2 2

Dr. Tanja Karp 6.8 Statistical Averages 36

6.9

In many cases a random experiment depends on several random variables. Example: Determine how the likelihood to be overweight ed depends on the age of a person.

random experiment: choosing one person

height X1 (1 ) X1 (2 )

weight

age

X2 (1 ) X2 (3 ) X2 (2 )

X3 (2 )

1 * 3 *

* 2

X1 (3 )

X3 (3 ) X3 (1 )

sample space S

FX,Y (x, y ) = P (X x, Y y )

where X and Y are two random variables dened on the same sample space S .

37

x y

FX,Y (x, y ) = P (X x, Y y ) =

pX,Y (t, s) ds dt

where X and Y are two random variables dened on the same sample space S . Marginal Probability Function:

pX (x) =

p Y (y ) =

pX,Y (x, y ) dx

x x

FX (x) = P (X x, Y ) =

y

pX,Y (t, s) ds dt =

y

pX (t) dt

F Y (y ) = P (X , Y y ) =

pX,Y (t, s) ds dt =

pY (s) ds

38

Statistical Independence: Two random variables X and Y are called statistically independent if their joint probability density function satises pX,Y (x, y ) = pX (x) pY (y ), i.e. the value of one random variable is independent of the value of the other.

39

6.10

6.10.1 Denition and Examples

Random Processes

random variable

random experiment X (t , 1 )

1 1

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

* 1 * i * 3 * 2

* X (t , ) 2

0.5 0 0.5 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

* X (t , 3 )

1 0.5 0 0.5 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

sample space S

X (t , i )

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

t1

Dr. Tanja Karp 6.10 Random Processes

t2

40

Random Process: A random process X (t) describes the mapping of a random experiment with sample space S onto an ensemble of sample functions X (t, i). For each point in time t1, X (t1) describes a random variable. Example: Rolling a Die Random variable: X = i if number i is on top of the die Random process: Y (t) = X cos(0t). Example: Tossing a Coin N Times Random variable: Xn = 0 if the nth result is head, Xn = 1 if the nth result is tail Random process: Y (t) =

Y (t , 1 ) t Y (t , 1 ) t Y (t , 1 ) t Y (t , 4 ) t

Dr. Tanja Karp 6.10 Random Processes

2N

n=1

Xn rect(t n + 0.5).

Y (t , 2N ) t

41

h( )X (t )d

6.10.2 Ensemble Averages and Stationarity

For each time instance of a random process, the average value, variance etc. can be calculated from all sample functions X (t, i). Expected Value E {X (t)}: For a random process X (t) with probability density function pX (t)(x), the expected value E {X (t)} = mX (t) is given by:

E {X (t)} =

x pX (t)(x)dx = mX (t)

Variance X (t):

|x mX (t)| pX (t)(x)dx

Random Processes

42

For a stationary random process the probability density function is independent of time t, thus the expected value and the variance are also a constant over time.

6.10.3 Time Averages and Ergodicity

t, t0

X (t) = X (t + t0) = X

So far, the average value and the variance of a random process X (t) were calculated based on the probability density function pX (t). However, in practical experiments the probability density function of a random process is often unknown. Also, in many cases, there is only one sample function X (t, i) available. Therefore, it is favorable to average over time instead of taking the ensemble average. Average Value mX (i):

mX (i)

2 Variance X ( ) : i 2 X ( ) i

1 = lim T T

T /2

X (t, i)dt

T / 2

1 = lim T T

6.10

T /2 T / 2 i

(X (t, i) mX ) dt

Random Processes

43

ensemble average

random experiment X (t , 1 )

1 1

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

* 1 * i * 3 * 2

* X (t , ) 2

0.5 0 0.5 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

* X (t , 3 )

1 0.5 0 0.5 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

sample space S

X (t , i )

time average

0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

t1

t2

6.10

Random Processes

44

Ergodicity: A stationary random process X (t) is called ergodic, if the time averages of each sample function X (t, i) converge towards the corresponding ensemble average with probability one. In practical applications ergodicity is often assumed since just one sample function is available and therefore the ensemble averages cannot be taken. Example 1: Random process: X (t) = A cos(0t) A: discrete random variable with P (A = 1) = P (A = 2) = 0.5 0: constant Ensemble average:

For 0 = 0 the random process is not stationary and we are not allowed to take the time average.

6.10

Random Processes

45

Example 2: Random process: X (t) = A A: discrete random variable with P (A = 1) = P (A = 2) = 0.5 Ensemble average:

Time averages:

mX (1)

1 = lim T T 1 = lim T T

T /2

T /2

T /2

1 dt = 1

T / 2 T /2

mX (2)

2 dt = 2

T / 2

time averages taken for dierent sample functions are not identical to the ensemble average, the random process is thus not ergodic.

6.10

Random Processes

46

Example 3: Tossing a Coin N Times Random variable: Xn = 0 if the nth result is head, Xn = 1 if the nth result is tail. Random process: Y (t) =

Y (t , 1 ) t Y (t , 1 ) t Y (t , 1 ) t Y (t , 4 ) t Y (t , 2N ) t

N 2 n=1

Xn rect(t n + 0.5)

Ensemble average:

Time average: Sample function Y (t, i): Coin is tossed N times and we observe n1 times head and n2 times tail.

m Y ( i )

1 = N

Y (t, i)dt =

0

1 n2 (n1 0 + n2 1) = N N

for N , n1 and n2 converge towards N/2 and mY (t,i) = mY (t) = mY . The random process is thus ergodic.

Dr. Tanja Karp 6.10 Random Processes 47

6.10

Random Processes

48

6.11

6.11.1

1

We are interested in how the value of a random process X (t) evaluated at t2 depends on its value at time t1. At t1 and t2 the random process is characterized by random variables X1 and X2, respectively. The relationship between X1 and X2 is given by the joint probability density function

X (t , 1 )

1 1 0.5

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

X (t , 2 )

0 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

0.5 1

X (t , 3 )

0.5 0 0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2

0.5

Autocorrelation Function:

X (t , i )

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

t1

t2

= E {X (t1) X (t2)} =

49

Autocovariance Function:

(x1 mX (t1))(x2 mX (t2)) pX (t1)X (t2)(x1, x2)dx1dx2 = RXX (t1, t2) mX (t1)mX (t2)

Autocorrelation and Autocovariance Function of a Stationary Random Process: The joint probability density function of a stationary process does not change if a constant value t is added to both t1 and t2.

The autocorrelation function then only depends on the dierence between t1 and t2

=

Dr. Tanja Karp

6.11

50

Since the average value is a constant, the autocovariance function is given by:

Symmetry: RXX ( ) = RXX ( ) Mean Square Average: RXX (0) = E {X (t)2} 0 Maximum: RXX (0) |RXX ( )| Periodicity: if RXX (0) = RXX (t0), then RXX ( ) is periodic with period t0.

6.11

51

Wide Sense Stationary (WSS) Random Process: A random process X (t) is called WSS if the following three properties are satised:

The average value of the random process is a constant: mX (t) = mX The autocorrelation and autocovariance function only depend on the time dierence = t1 t2: RXX (t1, t2) = RXX (t2 t1) = RXX ( ) CXX (t1, t2) = CXX (t2 t1) = CXX ( )

2 The variance is a constant and nite: X = CXX (0) = RXX (0) m2 X <

1 RXX ( ) = lim T T 1 T T

T /2

T /2

T / 2

CXX ( ) = lim

T / 2

XT (t, i): sample function of random process X (t) windowed to be of length T (starting at T /2 ending at T /2).

Dr. Tanja Karp 6.11 Autocorrelation and Power Spectra 52

6.11.2

Motivation:

Description of random processes in the frequency domain Calculation of the Fourier Transform of a sample function is not useful We assume in the following that the random process considered is at least WSS if not stationary.

The power spectral density (psd) of a WSS random process X (t) is dened as the Fourier Transform of the autocorrelation function RXX ( ):

SXX ( ) = F{RXX ( )} =

RXX ( ) exp(j )d

Inverse transform:

1 RXX ( ) = 2

SXX ( ) exp(j ) d

SXX ( ) = SXX ( ),

Dr. Tanja Karp 6.11

SXX ( ) 0,

Autocorrelation and Power Spectra

Im{SXX ( )} = 0

53

T /2 T /2

Rxx( ) = lim

1 T T

T / 2

1 T T

xT (t)xT (t + )dt

T / 2

Sxx( ) =

Rxx( ) exp(j )d

T /2

1 = lim T T 1 = lim T T

T /2

T / 2

xT (t)

T / 2 T /2

xT (t + ) exp(j )d dt

XT ( ) exp(jt)

1 = XT ( ) lim T T

xT (t) exp(jt) dt

T / 2

6.11

54

6.11.3

Power Signal: The autocorrelation function and the power spectral density can also be calculated for a deterministic power signal f (t). In this case, the signal simply replaces the random process. With fT (t) = f (t) rect(t/T ) we obtain for the autocorrelation function:

1 Rf f ( ) = lim T T

and for the power spectral density:

T /2

fT (t)fT (t + )dt

T / 2

Sf f ( ) =

1 P = Rf f (0) = lim T T

T /2

1 2 fT (t)dt = 2

Sf f ( )d

T / 2

6.11

55

Energy Signals: For an energy signal f (t) an energy autocorrelation function can be dened as

Rf f ( ) =

f (t)f (t + )dt = f ( ) f ( )

Applying the Fourier Transform to the energy autocorrelation function, we obtain the energy spectral density as:

Sf f ( ) =

Rf f ( ) exp(j )d = F ( )F ( ) = |F ( )|

E = Rf f (0) =

f (t) dt =

1 2

Sf f ( )d =

1 2

|F ( )|

6.11

56

6.11.4

X (t) = A sin(0t + ) A, 0: constant values, : random variable with probability density function p(x): p(x) =

Average value:

The average value is a constant and independent of t. Since mX = 0 the autocorrelation and autocovariance functions are identical.

Dr. Tanja Karp 6.11 Autocorrelation and Power Spectra 57

Autocorrelation function:

Applying:

2 2

RXX (t1, t2) = E {0.5A cos(0(t2 t1)) 0.5A cos(0(t2 + t1) + 2)} = 0.5A E {cos(0(t2 t1))} 0.5A E {cos(0(t2 + t1) + 2)} = 0.5A cos(0(t2 t1)) 0 = RXX ( )

The autocorrelation function only depends on = t2 t1 but not on the absolute values of t1 and t2. Power spectral density:

2 2 2

Ergodicity: If the random process is ergodic, we obtain the same results for the average value and the autocorrelation function by taking the time averages over one sample function.

Dr. Tanja Karp 6.11 Autocorrelation and Power Spectra 58

X (t, i) = A sin(0t + i)

Average value (time average):

mX (i)

1 = lim T T 1 = lim T T

T /2

X (t, i)dt

T / 2 T /2

A sin(0t + i)dt = 0 = mX

T / 2

RX (i)X (i)( ) = = = =

T /2 1 lim X (t, i)X (t + , i)dt T T T / 2 T /2 1 2 A sin(0t + i) sin(0(t + ) + i)dt lim T T T / 2 T /2 A2 1 lim cos(0 ) + cos(0(2t + ) + 2i)dt 2 T T T /2 A2 cos(0 ) = RXX ( ) 2

Time averages of one sample function and ensemble averages are identical the random process is ergodic.

Dr. Tanja Karp 6.11 Autocorrelation and Power Spectra 59

A = 1, 0 = 2

Sample Functions 1 X(t,1) 0 1 1 X(t,2) 0 1 1 X(t,3) 0 1 1 X(t,4) 1.5 0 1 1 X(t,5) 0 1 0.5 1 2 0.5 Autocorrelation Function R

XX

()

5 0

5 0.5 3 2 1 0 1 2 3

2 t in sec

0 10

10

6.11

60

Example 2: Binary Data Transmission A binary sequence is transmitted by rectangular pulses of width Tb. The amplitude of the pulse is determined by each bit, i.e. it is one if the bit is one and zero if the bit is zero. We assume that ones and zeros are equally likely and that each bit is statistically independent of all others. Using ergodicity, we obtain the following results from a sample function X (t, 1): Average value (sample function of length N bits with n1 ones):

mX

T 0

N Tb 0

X (t, 1)dt

Autocorrelation function:

Power spectral density:

0

6.11

61

Autocorrelation Function R

XX

()

0.5 0 5 10 t/Tb 15 20

0.25

Shifted Copies of Sample Function 1.5 1 0.5 0 0.5 1.5 1 0.5 0 0.5 1.5 1 0.5 0 0.5 X(t+0.3T ,1)

b

0 5

0 /Tb

X(t+T ,1)

0.5 0 5 10 15 20 0.25

X(t+1.5T ,1)

10 t/Tb

15

20

0 15

10

0 Tb

10

15

6.11

62

Energy autocorrelation function:

E Rf f ( )

f (t)f (t + )dt = ( )

Sf f ( ) = F{Rf f ( )} = Sa (/2)

Autocorrelation Function Power Spectral Density

10

10

15

6.11

63

Example 4: White Noise A random process n(t) is called white noise, if it has a constant power spectral density of /2 watts per Hz measured over positive frequencies. If in addition the random process has zero mean (mn = 0), the power spectral density is given by:

Snn( ) = /2

Autocorrelation function:

1

for all

Rnn( ) = F

{Snn( )} =

(t) 2

Since only the rst and second moment of the process are known, the probability density function cannot be uniquely determined. In the case of a Gaussian probability density function, the process is called white Gaussian noise. If the white Gaussian noise is added to the signal, we denote it as additive white Gaussian noise (AWGN).

6.11

64

1 2

5 t

10

Rnn () (/2)

Snn () /2

6.11

65

6.12

Excitation of an LTI system with sample function x(t, i) of a stationary random process x(t).

x(t , i )

y(t , i )

6.12.1 Expected Value of the Output Random Process

&

my = E {y (t)} = E {x(t) h(t)} = E

'

x(t )h()d

E {x(t )} h( )d =

1

6.12

66

6.12.2

&

'

=E

h( )x(t )d

h()x(t + )d

=

=

h( )h()Rxx( + )dd

h( )h( + )d Rxx( )d

R E ( ) hh

Dr. Tanja Karp 6.12 Excitation of LTI Systems with Stationary Random Processes 67

6.12.3

with

E E E

Rhh( ) = h( ) h( ) h( ) H ( ),

E

h( ) H ( )

2

Rhh( ) H ( )H ( ) = |H ( )|

and thus

2

Syy ( ) = Sxx( ) |H ( )|

Rxx ()

x(t , i )

y(t , i )

Ryy () = Rxx () RE hh ()

Sxx ()

6.12

68

Example: Ideal Lowpass Filtering of White Noise Input random process ni(t):

Ideal lowpass lter:

HLP ( ) =

Output random process no(t):

Power of output random process:

P no =

1 2

Snono ( )d =

1 2

0.5d =

c

c = fc 2

Sno no ()

/2 c c

Dr. Tanja Karp 6.12

HLP () 1 Sni ni ()

/2 c

c

69

6.12.4

The autocorrelation function describes the statistical properties of two random variables X1 and X2 taken from the same random process at times t1 and t2, respectively, X1 = X (t1) and X2 = X (t2). The cross-correlation function describes the statistical properties of two random variables X1 and Y2 taken from two dierent random processes X (t) and Y (t) (here input and output of an LTI system) at times t1 and t 2, respectively, such that X1 = X (t1) and Y2 = Y (t2). It is dened as:

For stationary processes, it simplies to:

RXY ( ) = E {X (t) Y (t + )}

Two random processes X (t) and Y (t) are called uncorrelated if

They are called orthogonal if

6.12

70

Here:

&

'

=E

x(t)

h()x(t + )d

h()E {x(t)x(t + )} d

h()Rxx( )d = h( ) Rxx( )

Example: System Identication An LTI system with unknown impulse response is excited with a white noise random process ni(t) with power spectral density Snini = /2. The output noise process is no(t). The cross-correlation between input and output noise process is given by: Rnino ( ) = h( ) Rni,ni ( ) = h( ) ( ) = h( ) 2 2

6.12

71

- EB510 Bro EnUploaded bygus289
- pekcan-mceer-99-21Uploaded byLC
- Normal Distribution.pptxUploaded bysanthiyasandy
- FROM INPUT-OUTPUT TO OUTPUT-ONLY MODAL IDENTIFICATION OF CIVIL ENGINEERING STRUCTURESUploaded byigomez19
- Bet Smarter With the Monte Carlo SimulationUploaded byOladipupo Mayowa Paul
- Article VCB2010 FMarinUploaded bycorsaiolo
- Probability-Theory-Problems.pdfUploaded byPaun Mihai
- Vissarion Papadopoulos, Dimitris G. Giovanis-Stochastic Finite Element Methods. an Introduction-Springer (2018)Uploaded byJohnny Renato Quispe
- SP-07 sample 2013.pdfUploaded bylloydhuni
- Gravity ProcesingUploaded byanyak1167032
- Tutorial 3Uploaded byWan Noraiman
- The Hyper Geometric DistributionUploaded byZhiqiang Xu
- Art 5Uploaded byrampinelli
- LP 01-03-2019Uploaded byAldrin Dela Cruz
- bsc4qUploaded bykirandevi1981
- notes-10Uploaded byAditya Bhutra
- MA1014probnability & Queueing TheoryUploaded bydhivya dharshini
- AppendixUploaded bykarlTronxo
- Lesson 2-03 Random Variables STATUploaded byallan.manaloto23
- Assesment and Structural Rehabilitation With PosttensioningUploaded byDaniel Zamudio Lopez
- Handout_Probability_PrimerUploaded byoneiros
- Fundamentals of ProbabilityUploaded byKarl John A. Galvez
- Probability R,VUploaded byYehya Mesalam
- AN3455Uploaded byRohan Rajagopal
- Uncertainty Analysis of Raw Data and Calculated Data From Organic Rankine Cycle SystemUploaded byApplelatte Nist
- Chapter_5_(2)(3)Uploaded byWei Wei
- 06978893Uploaded byAshish Choudhary
- u 0134650Uploaded byInternational Organization of Scientific Research (IOSR)
- Aspek Fisik.pdfUploaded byulfar uul
- 04459046Uploaded byGrzegorz Kowal

- Hubert ReportUploaded bymansha99
- Lab Sim SetuUploaded byVishal Chetal
- Report 123Uploaded byVishal Chetal
- Cover Letter VISHALUploaded byVishal Chetal
- my capso (1)Uploaded byVishal Chetal
- Introduction Power Supply Rev7 22Uploaded byVishal Chetal
- Comm-05-Random Variables and ProcessesUploaded byVishal Chetal
- Emerging trendsUploaded byVishal Chetal
- List of ProjectsUploaded byaccord123
- IJSETR-VOL-2-ISSUE-4-992-997Uploaded byVishal Chetal
- ControlerUploaded byVishal Chetal
- EFY-Top100_Oct09.pdfUploaded byvenkatesh_1829
- processorUploaded byVishal Chetal
- PROJECT HOTELUploaded byVishal Chetal

- ProbabilityStatisticsRandomProcUploaded bykjinguy642
- cspl-330Uploaded byعالي المقام
- 7Uploaded byBrother George
- Penelope 2014Uploaded byJaviCantero
- Flood+Frequency+Analysis+Methods+-+Chapter-Section+Number+-D.4.3Uploaded byAminur Rahman
- Vissarion Papadopoulos, Dimitris G. Giovanis-Stochastic Finite Element Methods. an Introduction-Springer (2018)Uploaded byJohnny Renato Quispe
- Https Onlinecourses.science.psuUploaded bywallace120
- 9_16042019Uploaded byمحمد خيري بن اسماءيل
- EGR 4201 Part AUploaded byAmosOlabode
- Probability for statistical physicsUploaded byVishal Gaud
- PDF AND PMFUploaded bys.r.
- Backscattering Cross Sections of Live Fish: PDF and AspectUploaded byMati Ger
- 5.Sas Codes ExtraUploaded bykPrasad8
- The Ultimate Probability CheatsheetUploaded byqtian
- STAT 330 Supplementary NotesUploaded byJames Deen
- ReliabilityUploaded byevaniellyguimaraes
- ar99-2Uploaded byCamilo Sanchez Hernandez
- 0538733500_301117.pdfUploaded bySmart Sarim
- report deep foundations.pdfUploaded bymparmi4725
- MIT18 05S14 Prac Exam1aUploaded byPoonam Naidu
- Vector Occluders: An Empirical Approximation for Rendering Global Illumination Effects in Real-TimeUploaded bywillsherif
- RANDOM VIBRATION OF STRUCTURES_YANG scan by ghazali 89.pdfUploaded byMoj Jamaran
- Assignment 2 ECEUploaded byBRAHMA REDDY AAKUMAIIA
- Unit2.pptxUploaded byShosho
- ch18Uploaded byceice2013
- Assign 1Uploaded by錢錢
- John Ehlers - TheInverseFisherTransform.pdfUploaded byArc Angel M
- hw01Uploaded byAkshay Mall
- 19 1 random variables - continuousUploaded byapi-299265916
- 4017-4025Uploaded byMhd Bassam El-Halabi