Discrete RVs

Conditional PDFs
Function of one RV

Decision and estimation in information processing:

course nr. 3

March 9, 2021

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conclusions previous course:

The cumulative distribution function (CDF) of a RV ξ:

Fξ (x) = P(ξ ≤ x) ∀x ∈ R

The probability density function (PDF) of a RV ξ:

dFξ (x)
wξ (x) =
dx

wξ (x)∆x ≈ P(x < ξ ≤ x + ∆x)

&
∆x&
The measure of probability is given y the area underneath
wξ (x).
For coninuous RVs, in general, P(ξ = x) = 0!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conclusions previous course:

The cumulative distribution function (CDF) of a RV ξ:

Fξ (x) = P(ξ ≤ x) ∀x ∈ R

The probability density function (PDF) of a RV ξ:

dFξ (x)
wξ (x) =
dx

wξ (x)∆x ≈ P(x < ξ ≤ x + ∆x)

&
∆x&
The measure of probability is given y the area underneath
wξ (x).
For coninuous RVs, in general, P(ξ = x) = 0!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conclusions previous course:

The cumulative distribution function (CDF) of a RV ξ:

Fξ (x) = P(ξ ≤ x) ∀x ∈ R

The probability density function (PDF) of a RV ξ:

dFξ (x)
wξ (x) =
dx

wξ (x)∆x ≈ P(x < ξ ≤ x + ∆x)

&
∆x&
The measure of probability is given y the area underneath
wξ (x).
For coninuous RVs, in general, P(ξ = x) = 0!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conclusions previous course:

The cumulative distribution function (CDF) of a RV ξ:

Fξ (x) = P(ξ ≤ x) ∀x ∈ R

The probability density function (PDF) of a RV ξ:

dFξ (x)
wξ (x) =
dx

wξ (x)∆x ≈ P(x < ξ ≤ x + ∆x)

&
∆x&
The measure of probability is given y the area underneath
wξ (x).
For coninuous RVs, in general, P(ξ = x) = 0!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conclusions previous course:

The cumulative distribution function (CDF) of a RV ξ:

Fξ (x) = P(ξ ≤ x) ∀x ∈ R

The probability density function (PDF) of a RV ξ:

dFξ (x)
wξ (x) =
dx

wξ (x)∆x ≈ P(x < ξ ≤ x + ∆x)

&
∆x&
The measure of probability is given y the area underneath
wξ (x).
For coninuous RVs, in general, P(ξ = x) = 0!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conclusions previous course:

The cumulative distribution function (CDF) of a RV ξ:

Fξ (x) = P(ξ ≤ x) ∀x ∈ R

The probability density function (PDF) of a RV ξ:

dFξ (x)
wξ (x) =
dx

wξ (x)∆x ≈ P(x < ξ ≤ x + ∆x)

&
∆x&
The measure of probability is given y the area underneath
wξ (x).
For coninuous RVs, in general, P(ξ = x) = 0!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conclusions previous course:

The cumulative distribution function (CDF) of a RV ξ:

Fξ (x) = P(ξ ≤ x) ∀x ∈ R

The probability density function (PDF) of a RV ξ:

dFξ (x)
wξ (x) =
dx

wξ (x)∆x ≈ P(x < ξ ≤ x + ∆x)

&
∆x&
The measure of probability is given y the area underneath
wξ (x).
For coninuous RVs, in general, P(ξ = x) = 0!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs

Problem: what becomes of a PDF of a discrete RV?

A discrete RV: may take a finite number of values.
Example: Ω = {f1 , . . . , f6 }, and ξ(fi ) = i ∀i = 1, . . . , 6.
We shall proceed by computing the CDF of ξ.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs

Problem: what becomes of a PDF of a discrete RV?

A discrete RV: may take a finite number of values.
Example: Ω = {f1 , . . . , f6 }, and ξ(fi ) = i ∀i = 1, . . . , 6.
We shall proceed by computing the CDF of ξ.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs

Problem: what becomes of a PDF of a discrete RV?

A discrete RV: may take a finite number of values.
Example: Ω = {f1 , . . . , f6 }, and ξ(fi ) = i ∀i = 1, . . . , 6.
We shall proceed by computing the CDF of ξ.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs

Problem: what becomes of a PDF of a discrete RV?

A discrete RV: may take a finite number of values.
Example: Ω = {f1 , . . . , f6 }, and ξ(fi ) = i ∀i = 1, . . . , 6.
We shall proceed by computing the CDF of ξ.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The CDF of ξ

Fξ (x) = P(ξ < x). We have:

1 ∀x ∈ (−∞, 1), Fξ (x) = P(ξ ≤ x) = P(∅) = 0
1
2 ∀x ∈ [1, 2), Fξ (x) = P(ξ ≤ x) = P({f1 }) = 6
2
3 ∀x ∈ [2, 3), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 }) = 6
3
4 ∀x ∈ [3, 4), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 }) = 6
4
5 ∀x ∈ [4, 5), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 }) = 6
5
6 ∀x ∈ [5, 6), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 , f5 }) = 6
7 ∀x ∈ [6, ∞), Fξ (x) = P(ξ ≤ x) = P(Ω) = 1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The CDF of ξ

Fξ (x) = P(ξ < x). We have:

1 ∀x ∈ (−∞, 1), Fξ (x) = P(ξ ≤ x) = P(∅) = 0
1
2 ∀x ∈ [1, 2), Fξ (x) = P(ξ ≤ x) = P({f1 }) = 6
2
3 ∀x ∈ [2, 3), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 }) = 6
3
4 ∀x ∈ [3, 4), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 }) = 6
4
5 ∀x ∈ [4, 5), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 }) = 6
5
6 ∀x ∈ [5, 6), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 , f5 }) = 6
7 ∀x ∈ [6, ∞), Fξ (x) = P(ξ ≤ x) = P(Ω) = 1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The CDF of ξ

Fξ (x) = P(ξ < x). We have:

1 ∀x ∈ (−∞, 1), Fξ (x) = P(ξ ≤ x) = P(∅) = 0
1
2 ∀x ∈ [1, 2), Fξ (x) = P(ξ ≤ x) = P({f1 }) = 6
2
3 ∀x ∈ [2, 3), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 }) = 6
3
4 ∀x ∈ [3, 4), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 }) = 6
4
5 ∀x ∈ [4, 5), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 }) = 6
5
6 ∀x ∈ [5, 6), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 , f5 }) = 6
7 ∀x ∈ [6, ∞), Fξ (x) = P(ξ ≤ x) = P(Ω) = 1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The CDF of ξ

Fξ (x) = P(ξ < x). We have:

1 ∀x ∈ (−∞, 1), Fξ (x) = P(ξ ≤ x) = P(∅) = 0
1
2 ∀x ∈ [1, 2), Fξ (x) = P(ξ ≤ x) = P({f1 }) = 6
2
3 ∀x ∈ [2, 3), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 }) = 6
3
4 ∀x ∈ [3, 4), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 }) = 6
4
5 ∀x ∈ [4, 5), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 }) = 6
5
6 ∀x ∈ [5, 6), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 , f5 }) = 6
7 ∀x ∈ [6, ∞), Fξ (x) = P(ξ ≤ x) = P(Ω) = 1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The CDF of ξ

Fξ (x) = P(ξ < x). We have:

1 ∀x ∈ (−∞, 1), Fξ (x) = P(ξ ≤ x) = P(∅) = 0
1
2 ∀x ∈ [1, 2), Fξ (x) = P(ξ ≤ x) = P({f1 }) = 6
2
3 ∀x ∈ [2, 3), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 }) = 6
3
4 ∀x ∈ [3, 4), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 }) = 6
4
5 ∀x ∈ [4, 5), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 }) = 6
5
6 ∀x ∈ [5, 6), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 , f5 }) = 6
7 ∀x ∈ [6, ∞), Fξ (x) = P(ξ ≤ x) = P(Ω) = 1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The CDF of ξ

Fξ (x) = P(ξ < x). We have:

1 ∀x ∈ (−∞, 1), Fξ (x) = P(ξ ≤ x) = P(∅) = 0
1
2 ∀x ∈ [1, 2), Fξ (x) = P(ξ ≤ x) = P({f1 }) = 6
2
3 ∀x ∈ [2, 3), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 }) = 6
3
4 ∀x ∈ [3, 4), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 }) = 6
4
5 ∀x ∈ [4, 5), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 }) = 6
5
6 ∀x ∈ [5, 6), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 , f5 }) = 6
7 ∀x ∈ [6, ∞), Fξ (x) = P(ξ ≤ x) = P(Ω) = 1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The CDF of ξ

Fξ (x) = P(ξ < x). We have:

1 ∀x ∈ (−∞, 1), Fξ (x) = P(ξ ≤ x) = P(∅) = 0
1
2 ∀x ∈ [1, 2), Fξ (x) = P(ξ ≤ x) = P({f1 }) = 6
2
3 ∀x ∈ [2, 3), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 }) = 6
3
4 ∀x ∈ [3, 4), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 }) = 6
4
5 ∀x ∈ [4, 5), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 }) = 6
5
6 ∀x ∈ [5, 6), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 , f5 }) = 6
7 ∀x ∈ [6, ∞), Fξ (x) = P(ξ ≤ x) = P(Ω) = 1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The CDF of ξ

Fξ (x) = P(ξ < x). We have:

1 ∀x ∈ (−∞, 1), Fξ (x) = P(ξ ≤ x) = P(∅) = 0
1
2 ∀x ∈ [1, 2), Fξ (x) = P(ξ ≤ x) = P({f1 }) = 6
2
3 ∀x ∈ [2, 3), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 }) = 6
3
4 ∀x ∈ [3, 4), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 }) = 6
4
5 ∀x ∈ [4, 5), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 }) = 6
5
6 ∀x ∈ [5, 6), Fξ (x) = P(ξ ≤ x) = P({f1 , f2 , f3 , f4 , f5 }) = 6
7 ∀x ∈ [6, ∞), Fξ (x) = P(ξ ≤ x) = P(Ω) = 1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The CDF of ξ

F (x)
ξ

1 [

5/6 [ )

4/6 [ )
3/6 [ )

2/6 [ )

1/6 [ ) x
)
1 2 3 4 5 6

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The PDF of ξ

wξ is the derivative of Fξ .
Problem: Fξ does not have a derivative in points 1, 2, . . . , 6!
The solution: the Dirac impulse!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The PDF of ξ

wξ is the derivative of Fξ .
Problem: Fξ does not have a derivative in points 1, 2, . . . , 6!
The solution: the Dirac impulse!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The PDF of ξ

wξ is the derivative of Fξ .
Problem: Fξ does not have a derivative in points 1, 2, . . . , 6!
The solution: the Dirac impulse!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

Let function f (x):

f(x)

E2

E
1

x
x1 x2

The expression of f :

 E1 if x < x1
E2 −E1
f (x) = (x − x 1 ) + E 1 if x1 ≤ x ≤ x2 .
 x2 −x1
E2 if x > x2

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

Let function f (x):

f(x)

E2

E
1

x
x1 x2

The expression of f :

 E1 if x < x1
E2 −E1
f (x) = (x − x1 ) + E 1 if x1 ≤ x ≤ x2 .
 x2 −x1
E2 if x > x2

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

Let function f (x):

f(x)

E2

E
1

x
x1 x2

The expression of f :

 E1 if x < x1
E2 −E1
f (x) = (x − x1 ) + E 1 if x1 ≤ x ≤ x2 .
 x2 −x1
E2 if x > x2

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

The derivative of f :
f’(x)
A=E2−E1

(E −E )/(x −x )
2 1 2 1

x
x1 x2

The expression of the derivative f 0 :


0 0 if x < x1 or x > x2
f (x) = E2 −E1 .
x2 −x1 if x1 ≤ x ≤ x2

We assume values E1 and E2 are fixed, whereas x1 and x2 are

mobile ⇒ the area of the rectangle is constant!
Decision and estimation in information processing: course nr. 3
 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

The derivative of f :
f’(x)
A=E2−E1

(E −E )/(x −x )
2 1 2 1

x
x1 x2

The expression of the derivative f 0 :


0 0 if x < x1 or x > x2
f (x) = E2 −E1 .
x2 −x1 if x1 ≤ x ≤ x2

We assume values E1 and E2 are fixed, whereas x1 and x2 are

mobile ⇒ the area of the rectangle is constant!
Decision and estimation in information processing: course nr. 3
 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

The derivative of f :
f’(x)
A=E2−E1

(E −E )/(x −x )
2 1 2 1

x
x1 x2

The expression of the derivative f 0 :


0 0 if x < x1 or x > x2
f (x) = E2 −E1 .
x2 −x1 if x1 ≤ x ≤ x2

We assume values E1 and E2 are fixed, whereas x1 and x2 are

mobile ⇒ the area of the rectangle is constant!
Decision and estimation in information processing: course nr. 3
 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

The derivative of f :
f’(x)
A=E2−E1

(E −E )/(x −x )
2 1 2 1

x
x1 x2

The expression of the derivative f 0 :


0 0 if x < x1 or x > x2
f (x) = E2 −E1 .
x2 −x1 if x1 ≤ x ≤ x2

We assume values E1 and E2 are fixed, whereas x1 and x2 are

mobile ⇒ the area of the rectangle is constant!
Decision and estimation in information processing: course nr. 3
 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

The derivative of f :
f’(x)
A=E2−E1

(E −E )/(x −x )
2 1 2 1

x
x1 x2

The expression of the derivative f 0 :


0 0 if x < x1 or x > x2
f (x) = E2 −E1 .
x2 −x1 if x1 ≤ x ≤ x2

We assume values E1 and E2 are fixed, whereas x1 and x2 are

mobile ⇒ the area of the rectangle is constant!
Decision and estimation in information processing: course nr. 3
 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

What happens when x2 −→ x1 ?
Function f has a jump (a discontinuity) ’in x1 .
In the derivative of the function there appears a Dirac
impulse:
a “signal” of 0 support.
a “signal” of infinite value.
a “signal” of finite area: E2 − E1 .

f’(x) (E2−E1)δ(x−x1)
f(x)

E2

E
1

x x
x2→ x1 x2→ x1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

What happens when x2 −→ x1 ?
Function f has a jump (a discontinuity) ’in x1 .
In the derivative of the function there appears a Dirac
impulse:
a “signal” of 0 support.
a “signal” of infinite value.
a “signal” of finite area: E2 − E1 .

f’(x) (E2−E1)δ(x−x1)
f(x)

E2

E
1

x x
x2→ x1 x2→ x1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

What happens when x2 −→ x1 ?
Function f has a jump (a discontinuity) ’in x1 .
In the derivative of the function there appears a Dirac
impulse:
a “signal” of 0 support.
a “signal” of infinite value.
a “signal” of finite area: E2 − E1 .

f’(x) (E2−E1)δ(x−x1)
f(x)

E2

E
1

x x
x2→ x1 x2→ x1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

What happens when x2 −→ x1 ?
Function f has a jump (a discontinuity) ’in x1 .
In the derivative of the function there appears a Dirac
impulse:
a “signal” of 0 support.
a “signal” of infinite value.
a “signal” of finite area: E2 − E1 .

f’(x) (E2−E1)δ(x−x1)
f(x)

E2

E
1

x x
x2→ x1 x2→ x1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

What happens when x2 −→ x1 ?
Function f has a jump (a discontinuity) ’in x1 .
In the derivative of the function there appears a Dirac
impulse:
a “signal” of 0 support.
a “signal” of infinite value.
a “signal” of finite area: E2 − E1 .

f’(x) (E2−E1)δ(x−x1)
f(x)

E2

E
1

x x
x2→ x1 x2→ x1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse

What happens when x2 −→ x1 ?
Function f has a jump (a discontinuity) ’in x1 .
In the derivative of the function there appears a Dirac
impulse:
a “signal” of 0 support.
a “signal” of infinite value.
a “signal” of finite area: E2 − E1 .

f’(x) (E2−E1)δ(x−x1)
f(x)

E2

E
1

x x
x2→ x1 x2→ x1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: definition

By definition:

∞ if x = 0
δ(x) = ,
0 elsewhere
with

Z∞
δ(x)dx = 1.
−∞

Conclusion: If a function has a jump at point x0 , in the derivative

of the function, at that point, there will appear a Dirac impulse the
area of which equal the amplitude of the jump!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: definition

By definition:

∞ if x = 0
δ(x) = ,
0 elsewhere
with

Z∞
δ(x)dx = 1.
−∞

Conclusion: If a function has a jump at point x0 , in the derivative

of the function, at that point, there will appear a Dirac impulse the
area of which equal the amplitude of the jump!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: definition

By definition:

∞ if x = 0
δ(x) = ,
0 elsewhere
with

Z∞
δ(x)dx = 1.
−∞

Conclusion: If a function has a jump at point x0 , in the derivative

of the function, at that point, there will appear a Dirac impulse the
area of which equal the amplitude of the jump!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: definition

By definition:

∞ if x = 0
δ(x) = ,
0 elsewhere
with

Z∞
δ(x)dx = 1.
−∞

Conclusion: If a function has a jump at point x0 , in the derivative

of the function, at that point, there will appear a Dirac impulse the
area of which equal the amplitude of the jump!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: definition

By definition:

∞ if x = 0
δ(x) = ,
0 elsewhere
with

Z∞
δ(x)dx = 1.
−∞

Conclusion: If a function has a jump at point x0 , in the derivative

of the function, at that point, there will appear a Dirac impulse the
area of which equal the amplitude of the jump!

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: property

The Dirac impulse has the following property:

Z∞
f (x)δ(x − x0 )dx = f (x0 )
−∞

Proof:
R∞ x0R+
f (x)δ(x − x0 )dx = f (x)δ(x − x0 )dx
−∞ x0 −
xZ0 +

≈ f (x0 ) δ(x − x0 )dx = f (x0 )

&&
x0 −
| {z }
1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: property

The Dirac impulse has the following property:

Z∞
f (x)δ(x − x0 )dx = f (x0 )
−∞

Proof:
R∞ x0R+
f (x)δ(x − x0 )dx = f (x)δ(x − x0 )dx
−∞ x0 −
xZ0 +

≈ f (x0 ) δ(x − x0 )dx = f (x0 )

&&
x0 −
| {z }
1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: property

The Dirac impulse has the following property:

Z∞
f (x)δ(x − x0 )dx = f (x0 )
−∞

Proof:
R∞ x0R+
f (x)δ(x − x0 )dx = f (x)δ(x − x0 )dx
−∞ x0 −
xZ0 +

≈ f (x0 ) δ(x − x0 )dx = f (x0 )

&&
x0 −
| {z }
1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: property

The Dirac impulse has the following property:

Z∞
f (x)δ(x − x0 )dx = f (x0 )
−∞

Proof:
R∞ x0R+
f (x)δ(x − x0 )dx = f (x)δ(x − x0 )dx
−∞ x0 −
xZ0 +

≈ f (x0 ) δ(x − x0 )dx = f (x0 )

&&
x0 −
| {z }
1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: property

The Dirac impulse has the following property:

Z∞
f (x)δ(x − x0 )dx = f (x0 )
−∞

Proof:
R∞ x0R+
f (x)δ(x − x0 )dx = f (x)δ(x − x0 )dx
−∞ x0 −
xZ0 +

≈ f (x0 ) δ(x − x0 )dx = f (x0 )

&&
x0 −
| {z }
1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The Dirac impulse: property

The Dirac impulse has the following property:

Z∞
f (x)δ(x − x0 )dx = f (x0 )
−∞

Proof:
R∞ x0R+
f (x)δ(x − x0 )dx = f (x)δ(x − x0 )dx
−∞ x0 −
xZ0 +

≈ f (x0 ) δ(x − x0 )dx = f (x0 )

&&
x0 −
| {z }
1

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The PDF of ξ

6
X 1
wξ (x) = δ(x − i).
6
i=1

wξ(x)

1/6δ(x−1) 1/6δ(x−6)

1 2 3 4 5 6

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs: conclusions

Presence of a Dirac impulse aδ(x − x0 ) in a PDF:

There is non–null probability that the respective RV takes the
value x0 .
What is the probability?: the area a of the Dirac!
P(ξ = x0 ) = a!
Definition: an RV is called:
continuous if its CDF Fξ is continuous.
discrete if its CDF Fξ is staircase-like.
mixt’a if its CDF Fξ has discontinuities, but is not
staircase-like.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs: conclusions

Presence of a Dirac impulse aδ(x − x0 ) in a PDF:

There is non–null probability that the respective RV takes the
value x0 .
What is the probability?: the area a of the Dirac!
P(ξ = x0 ) = a!
Definition: an RV is called:
continuous if its CDF Fξ is continuous.
discrete if its CDF Fξ is staircase-like.
mixt’a if its CDF Fξ has discontinuities, but is not
staircase-like.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs: conclusions

Presence of a Dirac impulse aδ(x − x0 ) in a PDF:

There is non–null probability that the respective RV takes the
value x0 .
What is the probability?: the area a of the Dirac!
P(ξ = x0 ) = a!
Definition: an RV is called:
continuous if its CDF Fξ is continuous.
discrete if its CDF Fξ is staircase-like.
mixt’a if its CDF Fξ has discontinuities, but is not
staircase-like.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs: conclusions

Presence of a Dirac impulse aδ(x − x0 ) in a PDF:

There is non–null probability that the respective RV takes the
value x0 .
What is the probability?: the area a of the Dirac!
P(ξ = x0 ) = a!
Definition: an RV is called:
continuous if its CDF Fξ is continuous.
discrete if its CDF Fξ is staircase-like.
mixt’a if its CDF Fξ has discontinuities, but is not
staircase-like.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs: conclusions

Presence of a Dirac impulse aδ(x − x0 ) in a PDF:

There is non–null probability that the respective RV takes the
value x0 .
What is the probability?: the area a of the Dirac!
P(ξ = x0 ) = a!
Definition: an RV is called:
continuous if its CDF Fξ is continuous.
discrete if its CDF Fξ is staircase-like.
mixt’a if its CDF Fξ has discontinuities, but is not
staircase-like.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs: conclusions

Presence of a Dirac impulse aδ(x − x0 ) in a PDF:

There is non–null probability that the respective RV takes the
value x0 .
What is the probability?: the area a of the Dirac!
P(ξ = x0 ) = a!
Definition: an RV is called:
continuous if its CDF Fξ is continuous.
discrete if its CDF Fξ is staircase-like.
mixt’a if its CDF Fξ has discontinuities, but is not
staircase-like.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs: conclusions

Presence of a Dirac impulse aδ(x − x0 ) in a PDF:

There is non–null probability that the respective RV takes the
value x0 .
What is the probability?: the area a of the Dirac!
P(ξ = x0 ) = a!
Definition: an RV is called:
continuous if its CDF Fξ is continuous.
discrete if its CDF Fξ is staircase-like.
mixt’a if its CDF Fξ has discontinuities, but is not
staircase-like.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs: conclusions

Presence of a Dirac impulse aδ(x − x0 ) in a PDF:

There is non–null probability that the respective RV takes the
value x0 .
What is the probability?: the area a of the Dirac!
P(ξ = x0 ) = a!
Definition: an RV is called:
continuous if its CDF Fξ is continuous.
discrete if its CDF Fξ is staircase-like.
mixt’a if its CDF Fξ has discontinuities, but is not
staircase-like.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Discrete RVs: conclusions

Presence of a Dirac impulse aδ(x − x0 ) in a PDF:

There is non–null probability that the respective RV takes the
value x0 .
What is the probability?: the area a of the Dirac!
P(ξ = x0 ) = a!
Definition: an RV is called:
continuous if its CDF Fξ is continuous.
discrete if its CDF Fξ is staircase-like.
mixt’a if its CDF Fξ has discontinuities, but is not
staircase-like.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional PDFs: problem posing

Let ξ be a RV and A an event.

We define the conditional CDF of ξ given A as:



P (ξ ≤ x) ∩ A
Fξ|A (x) = P (ξ ≤ x)|A = .
P(A)

We define the conditional PDF of ξ given A as:

dFξ|A (x) P (x < ξ ≤ x + ∆x)|A
wξ|A (x) = = lim .
dx ∆x→0 ∆x

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional PDFs: problem posing

Let ξ be a RV and A an event.

We define the conditional CDF of ξ given A as:



P (ξ ≤ x) ∩ A
Fξ|A (x) = P (ξ ≤ x)|A = .
P(A)

We define the conditional PDF of ξ given A as:

dFξ|A (x) P (x < ξ ≤ x + ∆x)|A
wξ|A (x) = = lim .
dx ∆x→0 ∆x

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional PDFs: problem posing

Let ξ be a RV and A an event.

We define the conditional CDF of ξ given A as:



P (ξ ≤ x) ∩ A
Fξ|A (x) = P (ξ ≤ x)|A = .
P(A)

We define the conditional PDF of ξ given A as:

dFξ|A (x) P (x < ξ ≤ x + ∆x)|A
wξ|A (x) = = lim .
dx ∆x→0 ∆x

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional PDFs: problem posing

Let ξ be a RV and A an event.

We define the conditional CDF of ξ given A as:



P (ξ ≤ x) ∩ A
Fξ|A (x) = P (ξ ≤ x)|A = .
P(A)

We define the conditional PDF of ξ given A as:

dFξ|A (x) P (x < ξ ≤ x + ∆x)|A
wξ|A (x) = = lim .
dx ∆x→0 ∆x

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional PDFs: problem posing

Let ξ be a RV and A an event.

We define the conditional CDF of ξ given A as:



P (ξ ≤ x) ∩ A
Fξ|A (x) = P (ξ ≤ x)|A = .
P(A)

We define the conditional PDF of ξ given A as:

dFξ|A (x) P (x < ξ ≤ x + ∆x)|A
wξ|A (x) = = lim .
dx ∆x→0 ∆x

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

Let wξ (x) the PDF of lifetime at birth:

Let A = {ξ > 60}.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

Let wξ (x) the PDF of lifetime at birth:

Let A = {ξ > 60}.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

Rx
The CDF of ξ: Fξ (x) = −∞ wξ (u)du:

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

The conditional CDF of ξ given
A = {ξ > 60}:

P (ξ≤x)∩(ξ>60) P (ξ≤x)∩(ξ>60)
Fξ|{ξ>60} (x) = P(ξ>60) = 1−Fξ (60)
For x ≤ 60, we have {ξ ≤ x} ∩ {ξ > 60} = ∅ therefore:

Fξ|{ξ>60} (x) = 0.

For x > 60, we have {ξ ≤ x} ∩ {ξ > 60} = {60 < ξ ≤ x}

therefore:
P(60 < ξ ≤ x) Fξ (x) − Fξ (60)
Fξ|{ξ>60} (x) = = .
1 − Fξ (60) 1 − Fξ (60)

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

The conditional CDF of ξ given
A = {ξ > 60}:

P (ξ≤x)∩(ξ>60) P (ξ≤x)∩(ξ>60)
Fξ|{ξ>60} (x) = P(ξ>60) = 1−Fξ (60)
For x ≤ 60, we have {ξ ≤ x} ∩ {ξ > 60} = ∅ therefore:

Fξ|{ξ>60} (x) = 0.

For x > 60, we have {ξ ≤ x} ∩ {ξ > 60} = {60 < ξ ≤ x}

therefore:
P(60 < ξ ≤ x) Fξ (x) − Fξ (60)
Fξ|{ξ>60} (x) = = .
1 − Fξ (60) 1 − Fξ (60)

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

The conditional CDF of ξ given
A = {ξ > 60}:

P (ξ≤x)∩(ξ>60) P (ξ≤x)∩(ξ>60)
Fξ|{ξ>60} (x) = P(ξ>60) = 1−Fξ (60)
For x ≤ 60, we have {ξ ≤ x} ∩ {ξ > 60} = ∅ therefore:

Fξ|{ξ>60} (x) = 0.

For x > 60, we have {ξ ≤ x} ∩ {ξ > 60} = {60 < ξ ≤ x}

therefore:
P(60 < ξ ≤ x) Fξ (x) − Fξ (60)
Fξ|{ξ>60} (x) = = .
1 − Fξ (60) 1 − Fξ (60)

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

The conditional CDF of ξ given
A = {ξ > 60}:

P (ξ≤x)∩(ξ>60) P (ξ≤x)∩(ξ>60)
Fξ|{ξ>60} (x) = P(ξ>60) = 1−Fξ (60)
For x ≤ 60, we have {ξ ≤ x} ∩ {ξ > 60} = ∅ therefore:

Fξ|{ξ>60} (x) = 0.

For x > 60, we have {ξ ≤ x} ∩ {ξ > 60} = {60 < ξ ≤ x}

therefore:
P(60 < ξ ≤ x) Fξ (x) − Fξ (60)
Fξ|{ξ>60} (x) = = .
1 − Fξ (60) 1 − Fξ (60)

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

The conditional CDF of ξ given
A = {ξ > 60}:

P (ξ≤x)∩(ξ>60) P (ξ≤x)∩(ξ>60)
Fξ|{ξ>60} (x) = P(ξ>60) = 1−Fξ (60)
For x ≤ 60, we have {ξ ≤ x} ∩ {ξ > 60} = ∅ therefore:

Fξ|{ξ>60} (x) = 0.

For x > 60, we have {ξ ≤ x} ∩ {ξ > 60} = {60 < ξ ≤ x}

therefore:
P(60 < ξ ≤ x) Fξ (x) − Fξ (60)
Fξ|{ξ>60} (x) = = .
1 − Fξ (60) 1 − Fξ (60)

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

The conditional CDF of ξ given
A = {ξ > 60}:

P (ξ≤x)∩(ξ>60) P (ξ≤x)∩(ξ>60)
Fξ|{ξ>60} (x) = P(ξ>60) = 1−Fξ (60)
For x ≤ 60, we have {ξ ≤ x} ∩ {ξ > 60} = ∅ therefore:

Fξ|{ξ>60} (x) = 0.

For x > 60, we have {ξ ≤ x} ∩ {ξ > 60} = {60 < ξ ≤ x}

therefore:
P(60 < ξ ≤ x) Fξ (x) − Fξ (60)
Fξ|{ξ>60} (x) = = .
1 − Fξ (60) 1 − Fξ (60)

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

The conditional CDF of ξ given
A = {ξ > 60}:

P (ξ≤x)∩(ξ>60) P (ξ≤x)∩(ξ>60)
Fξ|{ξ>60} (x) = P(ξ>60) = 1−Fξ (60)
For x ≤ 60, we have {ξ ≤ x} ∩ {ξ > 60} = ∅ therefore:

Fξ|{ξ>60} (x) = 0.

For x > 60, we have {ξ ≤ x} ∩ {ξ > 60} = {60 < ξ ≤ x}

therefore:
P(60 < ξ ≤ x) Fξ (x) − Fξ (60)
Fξ|{ξ>60} (x) = = .
1 − Fξ (60) 1 − Fξ (60)

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

Fξ (x) ’si Fξ|{ξ>60} (x):

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Conditional CDFs: example

The conditional PDF:
(
0 if x ≤ 60
wξ|{ξ>60} (x) = wξ (x)
1−Fξ (60) if x > 60

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Functions of one RV: problem posing

Let ξ be a RV with known PDF wξ .

Let g : R → R be a known function.
Let RV η = g (ξ).
Problem: compute wη based on wξ and g .
Example: If ξ : N (m, σ), what is the PDF of η = ξ 2 ?

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Functions of one RV: problem posing

Let ξ be a RV with known PDF wξ .

Let g : R → R be a known function.
Let RV η = g (ξ).
Problem: compute wη based on wξ and g .
Example: If ξ : N (m, σ), what is the PDF of η = ξ 2 ?

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Functions of one RV: problem posing

Let ξ be a RV with known PDF wξ .

Let g : R → R be a known function.
Let RV η = g (ξ).
Problem: compute wη based on wξ and g .
Example: If ξ : N (m, σ), what is the PDF of η = ξ 2 ?

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Functions of one RV: problem posing

Let ξ be a RV with known PDF wξ .

Let g : R → R be a known function.
Let RV η = g (ξ).
Problem: compute wη based on wξ and g .
Example: If ξ : N (m, σ), what is the PDF of η = ξ 2 ?

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Functions of one RV: problem posing

Let ξ be a RV with known PDF wξ .

Let g : R → R be a known function.
Let RV η = g (ξ).
Problem: compute wη based on wξ and g .
Example: If ξ : N (m, σ), what is the PDF of η = ξ 2 ?

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Functions of one RV: example

g(x)

y1’

y1

x x x3 x1’
1 2

Fη (y10 ) = P(η ≤ y10 ) = P(ξ ≤ x10 ) = Fξ (x10 )

Fη (y1 ) = P(η ≤ y1 ) = P (ξ ≤ x1 ) ∪ (x2 ≤ ξ ≤ x3 )
= Fξ (x3 ) − Fξ (x2 ) + Fξ (x1 )

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Functions of one RV: example

g(x)

y1’

y1

x x x3 x1’
1 2

Fη (y10 ) = P(η ≤ y10 ) = P(ξ ≤ x10 ) = Fξ (x10 )

Fη (y1 ) = P(η ≤ y1 ) = P (ξ ≤ x1 ) ∪ (x2 ≤ ξ ≤ x3 )
= Fξ (x3 ) − Fξ (x2 ) + Fξ (x1 )

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Theorem

Let ξ be an RV with known PDF wξ .

Let η = g (ξ), with g (x) being a known function.
For ∀y ∈ R with the property that equations g (x) = y has a
finite or countable number of solutions {x1 , x2 , . . .}, the
following holds:
X wξ (xk )
wη (y ) =
|g 0 (xk )|
k

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Theorem

Let ξ be an RV with known PDF wξ .

Let η = g (ξ), with g (x) being a known function.
For ∀y ∈ R with the property that equations g (x) = y has a
finite or countable number of solutions {x1 , x2 , . . .}, the
following holds:
X wξ (xk )
wη (y ) =
|g 0 (xk )|
k

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Theorem

Let ξ be an RV with known PDF wξ .

Let η = g (ξ), with g (x) being a known function.
For ∀y ∈ R with the property that equations g (x) = y has a
finite or countable number of solutions {x1 , x2 , . . .}, the
following holds:
X wξ (xk )
wη (y ) =
|g 0 (xk )|
k

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Theorem

Let ξ be an RV with known PDF wξ .

Let η = g (ξ), with g (x) being a known function.
For ∀y ∈ R with the property that equations g (x) = y has a
finite or countable number of solutions {x1 , x2 , . . .}, the
following holds:
X wξ (xk )
wη (y ) =
|g 0 (xk )|
k

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The number of solutions of equation g (x) = y

The case of finite number of solutions:

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The number of solutions of equation g (x) = y

The case of countable number of solutions:

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

The number of solutions of equation g (x) = y

The case of uncountable number of solutions:

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Let function g (x) have the following (unspecified) shape:

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof
Let y as in the figure. Equation g (x) = y has three solutions
:{x1 , x2 , x3 }.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof
Let y as in the figure. Equation g (x) = y has three solutions
:{x1 , x2 , x3 }.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof
Let y + ∆y as in the figure. Equation g (x) = y + ∆y has three
solutions: {x1 + ∆x1 , x2 + ∆x2 , x3 + ∆x3 }.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof
Let y + ∆y as in the figure. Equation g (x) = y + ∆y has three
solutions: {x1 + ∆x1 , x2 + ∆x2 , x3 + ∆x3 }.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof
We notice that P(y < η ≤ y1 + ∆y ) = P (x1 < ξ ≤

x1 + ∆x1 ) ∪ (x2 + ∆x2 ≤ ξ < x2 ) ∪ (x3 < ξ ≤ x3 + ∆x3 )

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Value of ∆y can be sufficiently small s.t. intervals

[xi , xi + ∆i] do not overlap.
By applying the third axiom of probability:

P(y < η ≤ y1 + ∆y ) = P(x1 < ξ ≤ x1 + ∆x1 )+

+ P(x2 + ∆x2 ≤ ξ < x2 ) + P(x3 < ξ ≤ x3 + ∆x3 )

Let’s not forget:

P(x < ξ ≤ x + ∆x) ≈ wξ (x)∆x!

∆x&&

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Value of ∆y can be sufficiently small s.t. intervals

[xi , xi + ∆i] do not overlap.
By applying the third axiom of probability:

P(y < η ≤ y1 + ∆y ) = P(x1 < ξ ≤ x1 + ∆x1 )+

+ P(x2 + ∆x2 ≤ ξ < x2 ) + P(x3 < ξ ≤ x3 + ∆x3 )

Let’s not forget:

P(x < ξ ≤ x + ∆x) ≈ wξ (x)∆x!

∆x&&

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Value of ∆y can be sufficiently small s.t. intervals

[xi , xi + ∆i] do not overlap.
By applying the third axiom of probability:

P(y < η ≤ y1 + ∆y ) = P(x1 < ξ ≤ x1 + ∆x1 )+

+ P(x2 + ∆x2 ≤ ξ < x2 ) + P(x3 < ξ ≤ x3 + ∆x3 )

Let’s not forget:

P(x < ξ ≤ x + ∆x) ≈ wξ (x)∆x!

∆x&&

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Value of ∆y can be sufficiently small s.t. intervals

[xi , xi + ∆i] do not overlap.
By applying the third axiom of probability:

P(y < η ≤ y1 + ∆y ) = P(x1 < ξ ≤ x1 + ∆x1 )+

+ P(x2 + ∆x2 ≤ ξ < x2 ) + P(x3 < ξ ≤ x3 + ∆x3 )

Let’s not forget:

P(x < ξ ≤ x + ∆x) ≈ wξ (x)∆x!

∆x&&

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Value of ∆y can be sufficiently small s.t. intervals

[xi , xi + ∆i] do not overlap.
By applying the third axiom of probability:

P(y < η ≤ y1 + ∆y ) = P(x1 < ξ ≤ x1 + ∆x1 )+

+ P(x2 + ∆x2 ≤ ξ < x2 ) + P(x3 < ξ ≤ x3 + ∆x3 )

Let’s not forget:

P(x < ξ ≤ x + ∆x) ≈ wξ (x)∆x!

∆x&&

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Hence:

wη (y )∆y1 = wξ (x1 )∆x1 + wξ (x2 ) |∆x2 | + wξ (x3 )∆x3 .

Last question: what is the relation between ∆y and ∆xi ?

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Hence:

wη (y )∆y1 = wξ (x1 )∆x1 + wξ (x2 ) |∆x2 | + wξ (x3 )∆x3 .

Last question: what is the relation between ∆y and ∆xi ?

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Hence:

wη (y )∆y1 = wξ (x1 )∆x1 + wξ (x2 ) |∆x2 | + wξ (x3 )∆x3 .

Last question: what is the relation between ∆y and ∆xi ?

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Hence:

wη (y )∆y1 = wξ (x1 )∆x1 + wξ (x2 ) |∆x2 | + wξ (x3 )∆x3 .

Last question: what is the relation between ∆y and ∆xi ?

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Relation between ∆y and ∆xi

∆y ∆y
Hence: g 0 (xi ) = ∆xi , or, alternatively, ∆xi = g 0 (xi )
Decision and estimation in information processing: course nr. 3
 Discrete RVs
Conditional PDFs
Function of one RV

Relation between ∆y and ∆xi

∆y ∆y
Hence: g 0 (xi ) = ∆xi , or, alternatively, ∆xi = g 0 (xi )
Decision and estimation in information processing: course nr. 3
 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Hence:
wξ (x1 ) wξ (x2 ) wξ (x3 )
wη (y ) = 0
+ 0 + 0 .
g (x1 ) |g (x2 )| g (x3 )

Given that both g 0 (x1 ) and g 0 (x3 ) > 0, we reach the solution:

wξ (x1 ) wξ (x2 ) wξ (x3 )

wη (y ) = 0
+ 0 + 0 .
|g (x1 )| |g (x2 )| |g (x3 )|

Remark: even if we assumed a particular form of graph of

function g , the proof has not lost in generality.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Hence:
wξ (x1 ) wξ (x2 ) wξ (x3 )
wη (y ) = 0
+ 0 + 0 .
g (x1 ) |g (x2 )| g (x3 )

Given that both g 0 (x1 ) and g 0 (x3 ) > 0, we reach the solution:

wξ (x1 ) wξ (x2 ) wξ (x3 )

wη (y ) = 0
+ 0 + 0 .
|g (x1 )| |g (x2 )| |g (x3 )|

Remark: even if we assumed a particular form of graph of

function g , the proof has not lost in generality.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Hence:
wξ (x1 ) wξ (x2 ) wξ (x3 )
wη (y ) = 0
+ 0 + 0 .
g (x1 ) |g (x2 )| g (x3 )

Given that both g 0 (x1 ) and g 0 (x3 ) > 0, we reach the solution:

wξ (x1 ) wξ (x2 ) wξ (x3 )

wη (y ) = 0
+ 0 + 0 .
|g (x1 )| |g (x2 )| |g (x3 )|

Remark: even if we assumed a particular form of graph of

function g , the proof has not lost in generality.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Hence:
wξ (x1 ) wξ (x2 ) wξ (x3 )
wη (y ) = 0
+ 0 + 0 .
g (x1 ) |g (x2 )| g (x3 )

Given that both g 0 (x1 ) and g 0 (x3 ) > 0, we reach the solution:

wξ (x1 ) wξ (x2 ) wξ (x3 )

wη (y ) = 0
+ 0 + 0 .
|g (x1 )| |g (x2 )| |g (x3 )|

Remark: even if we assumed a particular form of graph of

function g , the proof has not lost in generality.

Decision and estimation in information processing: course nr. 3

 Discrete RVs
Conditional PDFs
Function of one RV

Proof

Hence:
wξ (x1 ) wξ (x2 ) wξ (x3 )
wη (y ) = 0
+ 0 + 0 .
g (x1 ) |g (x2 )| g (x3 )

Given that both g 0 (x1 ) and g 0 (x3 ) > 0, we reach the solution:

wξ (x1 ) wξ (x2 ) wξ (x3 )

wη (y ) = 0
+ 0 + 0 .
|g (x1 )| |g (x2 )| |g (x3 )|

Remark: even if we assumed a particular form of graph of

function g , the proof has not lost in generality.

Decision and estimation in information processing: course nr. 3