Curs 7 DEPI - 2va3 - Slides - EN

Decision and estimation in information processing:
course nr. 7
March 29, 2021
Decision and estimation in information processing: course nr. 7

Moments
The regression line
The correlation coefficient
Part I
Moments

Moments
The regression line
Uncentered moments
Joint uncentered moment of order (k, l):

ZZ
(k,l) def k l
mξη = ξ η = x k y l wξη (x, y )dxdy .
R2
The uncentered moment of order (1, 1) is called correlation

between ξ and η:
ZZ
(1,1) not
mξη = ξη = Rξη = xywξη (x, y )dxdy .
R2

Moments
The regression line
Uncentered moments

ZZ
(k,l) def k l
R2

between ξ and η:
ZZ
(1,1) not
R2

Moments
The regression line
Uncentered moments

ZZ
(k,l) def k l
R2

between ξ and η:
ZZ
(1,1) not
R2

Moments
The regression line
Uncentered moments

ZZ
(k,l) def k l
R2

between ξ and η:
ZZ
(1,1) not
R2

Moments
The regression line
Centered moments
The joint centered moment of order (k, l):

ZZ
(k,l) ∆ k l k
Mξη = ξ − ξ (η − η) = x − ξ (y − η)l wξη (x, y )dxdy .
R2
The centered moment of order (1, 1) is called covariance

between ξ and η:
ZZ
not
ξ − ξ (η − η) = Kξη = x − ξ (y − η) wξη (x, y )dxdy .
R2

Moments
The regression line
Centered moments

ZZ
(k,l) ∆ k l k
R2

between ξ and η:
ZZ
not
R2

Moments
The regression line
Centered moments

ZZ
(k,l) ∆ k l k
R2

between ξ and η:
ZZ
not
R2

Moments
The regression line
Centered moments

ZZ
(k,l) ∆ k l k
R2

between ξ and η:
ZZ
not
R2

Moments
The regression line
Relation between correlation and covariance
Kξη = Rξη − ξη
Proof:

Kξη = ξ − ξ (η − η) = ξη − ξη − ξη + ξη = ξη − ξη − ξη + ξη
= ξη − ξη − ξη + ξη = ξη − ξη = Rξη − ξη.

Moments
The regression line
Proof:


Moments
The regression line
Proof:


Moments
The regression line
Proof:


Moments
The regression line
Proof:


Moments
The regression line
Proof:


Moments
The regression line
Proof:


Moments
The regression line
Proof:


Moments
The regression line
Uncorrelatedness
RVs ξ and η are called uncorrelated if their covariance is null:
Kξη = 0
Property: Independence =⇒ uncorrelatedness!

Proff:
If ξ and η are
RRindependent, we have:RR
Rξη = ξη = xywξη (x, y )dxdy = xywξ (x)wη (y )dxdy
R2 R2
Z∞ Z∞
= xwξ (x)dx ywη (y )dy = ξη =⇒ Kξη = 0.
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Uncorrelatedness
Kξη = 0

Proff:
If ξ and η are
R2 R2
Z∞ Z∞
−∞ −∞
| {z }| {z }
ξ η

Moments
The regression line
Relation between independence and uncorrelatedness
Important: Uncorrelatedness =⇒6 independence!

If ξη = ξη, we cannot prove that wξη (x, y ) = wξ (x)wη (y ).
The degree of correlation between ξ and η (numerically
measured by both Kξη and Rξη ) represents the degree of linear
dependence between the two RVs, namely, the extent to
which the dependence between the two RVs can be
approximated by a linear relation between them: η = aξ + b.

Moments
The regression line


Moments
The regression line


Moments
The regression line
Types of dependence between 2 RVs

5 3
3 2
2
1
1
0
0
−1
−2 −1
−3
−4 −2
−5
−3
−6 −4 −2 0 2 4 6 −3 −2 −1 0 1 2 3 4
Linearly dependent RVs independent RVs

(correlated)
1
0.5
−0.5
−1
−1.5 −1 −0.5 0 0.5 1 1.5
RVs dependent according to a non–linear law (dependent, but

uncorrelated)
Moments
The regression line
The regression line
If the cloud of points looks linear: how do we compute the line

that best approximates the cloud of points? (regression line).
16
14
12
10
−2
−4
−10 −5 0 5 10
The mathematical problem: a, b =? s.t. η̂ = aξ + b “best”

approximates η.

Moments
The regression line
The regression line

16
14
12
10
−2
−4
−10 −5 0 5 10

approximates η.

Moments
The regression line
The regression line

16
14
12
10
−2
−4
−10 −5 0 5 10

approximates η.

Moments
The regression line
The regression line
The optimality criterion: we define the mean-squared error

ε = (η − η̂)2 .
The problem to solve: a, b =? such that ε = εmin
The solution:
∂ε
= 0 =⇒ bo
∂b
∂ε
= 0 =⇒ ao
∂a

Moments
The regression line
The regression line

ε = (η − η̂)2 .
The solution:
∂ε
= 0 =⇒ bo
∂b
∂ε
= 0 =⇒ ao
∂a

Moments
The regression line
The regression line

ε = (η − η̂)2 .
The solution:
∂ε
= 0 =⇒ bo
∂b
∂ε
= 0 =⇒ ao
∂a

Moments
The regression line
The regression line

ε = (η − η̂)2 .
The solution:
∂ε
= 0 =⇒ bo
∂b
∂ε
= 0 =⇒ ao
∂a

Moments
The regression line
The regression line

ε = (η − η̂)2 .
The solution:
∂ε
= 0 =⇒ bo
∂b
∂ε
= 0 =⇒ ao
∂a

Moments
The regression line
The regression line
We begin with:
ε = (η − η̂)2 = (η − aξ − b)2 ,
Next:
∂ε ∂(η−aξ−b)2
∂b = ∂b = −2(η − aξ − b) = 0
Wherefrom:
bo = η − aξ

Moments
The regression line
The regression line
We begin with:
ε = (η − η̂)2 = (η − aξ − b)2 ,
Next:
∂ε ∂(η−aξ−b)2
∂b = ∂b = −2(η − aξ − b) = 0
Wherefrom:
bo = η − aξ

Moments
The regression line
The regression line
We begin with:
ε = (η − η̂)2 = (η − aξ − b)2 ,
Next:
∂ε ∂(η−aξ−b)2
∂b = ∂b = −2(η − aξ − b) = 0
Wherefrom:
bo = η − aξ

Moments
The regression line
The regression line
We begin with:
ε = (η − η̂)2 = (η − aξ − b)2 ,
Next:
∂ε ∂(η−aξ−b)2
∂b = ∂b = −2(η − aξ − b) = 0
Wherefrom:
bo = η − aξ

Moments
The regression line
The regression line
We begin with:
ε = (η − η̂)2 = (η − aξ − b)2 ,
Next:
∂ε ∂(η−aξ−b)2
∂b = ∂b = −2(η − aξ − b) = 0
Wherefrom:
bo = η − aξ

Moments
The regression line
The regression line
We begin with:
ε = (η − η̂)2 = (η − aξ − b)2 ,
Next:
∂ε ∂(η−aξ−b)2
∂b = ∂b = −2(η − aξ − b) = 0
Wherefrom:
bo = η − aξ

Moments
The regression line
The regression line
We begin with:
ε = (η − η̂)2 = (η − aξ − b)2 ,
Next:
∂ε ∂(η−aξ−b)2
∂b = ∂b = −2(η − aξ − b) = 0
Wherefrom:
bo = η − aξ

Moments
The regression line
The regression line
By replacing b in the equation of ε, we have:

2
ε = (η − η) − a(ξ − ξ) ,
Then:
2
∂ε ∂ ((η−η)−a(ξ−ξ))
∂a= ∂a = −2 (η − η) − a(ξ − ξ) ξ − ξ
2
= −2 (η − η)(ξ − ξ) − a ξ − ξ = −2 Kξη − aσξ2 = 0,
Hence:
Kξη
ao = .
σξ2

Moments
The regression line
The regression line

2
ε = (η − η) − a(ξ − ξ) ,
Then:
2
∂ε ∂ ((η−η)−a(ξ−ξ))
∂a= ∂a = −2 (η − η) − a(ξ − ξ) ξ − ξ
2
= −2 (η − η)(ξ − ξ) − a ξ − ξ = −2 Kξη − aσξ2 = 0,
Hence:
Kξη
ao = .
σξ2

Moments
The regression line
The regression line

2
ε = (η − η) − a(ξ − ξ) ,
Then:
2
∂ε ∂ ((η−η)−a(ξ−ξ))
∂a= ∂a = −2 (η − η) − a(ξ − ξ) ξ − ξ
2
= −2 (η − η)(ξ − ξ) − a ξ − ξ = −2 Kξη − aσξ2 = 0,
Hence:
Kξη
ao = .
σξ2

Moments
The regression line
The regression line

2
ε = (η − η) − a(ξ − ξ) ,
Then:
2
∂ε ∂ ((η−η)−a(ξ−ξ))
∂a= ∂a = −2 (η − η) − a(ξ − ξ) ξ − ξ
2
= −2 (η − η)(ξ − ξ) − a ξ − ξ = −2 Kξη − aσξ2 = 0,
Hence:
Kξη
ao = .
σξ2

Moments
The regression line
The regression line

2
ε = (η − η) − a(ξ − ξ) ,
Then:
2
∂ε ∂ ((η−η)−a(ξ−ξ))
∂a= ∂a = −2 (η − η) − a(ξ − ξ) ξ − ξ
2
= −2 (η − η)(ξ − ξ) − a ξ − ξ = −2 Kξη − aσξ2 = 0,
Hence:
Kξη
ao = .
σξ2

Moments
The regression line
The regression line

2
ε = (η − η) − a(ξ − ξ) ,
Then:
2
∂ε ∂ ((η−η)−a(ξ−ξ))
∂a= ∂a = −2 (η − η) − a(ξ − ξ) ξ − ξ
2
= −2 (η − η)(ξ − ξ) − a ξ − ξ = −2 Kξη − aσξ2 = 0,
Hence:
Kξη
ao = .
σξ2

Moments
The regression line
The regression line

2
ε = (η − η) − a(ξ − ξ) ,
Then:
2
∂ε ∂ ((η−η)−a(ξ−ξ))
∂a= ∂a = −2 (η − η) − a(ξ − ξ) ξ − ξ
2
= −2 (η − η)(ξ − ξ) − a ξ − ξ = −2 Kξη − aσξ2 = 0,
Hence:
Kξη
ao = .
σξ2

Moments
The regression line
The regression line

2
ε = (η − η) − a(ξ − ξ) ,
Then:
2
∂ε ∂ ((η−η)−a(ξ−ξ))
∂a= ∂a = −2 (η − η) − a(ξ − ξ) ξ − ξ
2
= −2 (η − η)(ξ − ξ) − a ξ − ξ = −2 Kξη − aσξ2 = 0,
Hence:
Kξη
ao = .
σξ2

Moments
The regression line
The regression line

2
ε = (η − η) − a(ξ − ξ) ,
Then:
2
∂ε ∂ ((η−η)−a(ξ−ξ))
∂a= ∂a = −2 (η − η) − a(ξ − ξ) ξ − ξ
2
= −2 (η − η)(ξ − ξ) − a ξ − ξ = −2 Kξη − aσξ2 = 0,
Hence:
Kξη
ao = .
σξ2

Moments
The regression line
The regression line
The final equation of the optimal regression line:

Kξη
η̂o = (ξ − ξ) + η.
σξ2
If Kξη = 0 =⇒ η̂o = η: there is no linear dependence

between η and ξ!
If Kξη 6= 0, we cannot evaluate how good the approximation
is. 4 4
3
ρ =0.8 3
ρξη=0.4
ξη
2 2
1 1
0 0
−1 −1
−2 −2
−3 −3
−4 −4
−4 −3 −2 −1 0 1 2 3 4 5 −5 −4 −3 −2 −1 0 1 2 3 4 5
We must compute the value of the minimal mean-squared

error εmin !
Moments
The regression line
The regression line

Kξη
η̂o = (ξ − ξ) + η.
σξ2

between η and ξ!
is. 4 4
3
ρ =0.8 3
ρξη=0.4
ξη
2 2
1 1
0 0
−1 −1
−2 −2
−3 −3
−4 −4
−4 −3 −2 −1 0 1 2 3 4 5 −5 −4 −3 −2 −1 0 1 2 3 4 5

error εmin !
Moments
The regression line
The regression line

Kξη
η̂o = (ξ − ξ) + η.
σξ2

between η and ξ!
is. 4 4
3
ρ =0.8 3
ρξη=0.4
ξη
2 2
1 1
0 0
−1 −1
−2 −2
−3 −3
−4 −4
−4 −3 −2 −1 0 1 2 3 4 5 −5 −4 −3 −2 −1 0 1 2 3 4 5

error εmin !
Moments
The regression line
The regression line

Kξη
η̂o = (ξ − ξ) + η.
σξ2

between η and ξ!
is. 4 4
3
ρ =0.8 3
ρξη=0.4
ξη
2 2
1 1
0 0
−1 −1
−2 −2
−3 −3
−4 −4
−4 −3 −2 −1 0 1 2 3 4 5 −5 −4 −3 −2 −1 0 1 2 3 4 5

error εmin !
Moments
The regression line
The regression line

Kξη
η̂o = (ξ − ξ) + η.
σξ2

between η and ξ!
is. 4 4
3
ρ =0.8 3
ρξη=0.4
ξη
2 2
1 1
0 0
−1 −1
−2 −2
−3 −3
−4 −4
−4 −3 −2 −1 0 1 2 3 4 5 −5 −4 −3 −2 −1 0 1 2 3 4 5

error εmin !
Moments
The regression line
The minimal mean-squared error
We have:
2
2 Kξη
εmin = (η − η̂o ) = (η − η) − σξ2
(ξ − ξ)
2
Kξη 2
Kξη
= (η − η)2 − 2 (η
σξ2 |
− η)(ξ − ξ) + σξ4 |
ξ −ξ
| {z } {z } {z }
ση2 Kξη σ2
ξ
Kξη 2
2
Kξη 2
Kξη 2
Kξη
= ση2 − 2 σξ2
+ σξ2
= ση2 − σξ2
2
= ση 1 − σξ ση .

Moments
The regression line
We have:
2
2 Kξη
εmin = (η − η̂o ) = (η − η) − σξ2
(ξ − ξ)
2
Kξη 2
Kξη
= (η − η)2 − 2 (η
σξ2 |
− η)(ξ − ξ) + σξ4 |
ξ −ξ
| {z } {z } {z }
ση2 Kξη σ2
ξ
Kξη 2
2
Kξη 2
Kξη 2
Kξη
= ση2 − 2 σξ2
+ σξ2
= ση2 − σξ2
2
= ση 1 − σξ ση .

Moments
The regression line
We have:
2
2 Kξη
εmin = (η − η̂o ) = (η − η) − σξ2
(ξ − ξ)
2
Kξη 2
Kξη
= (η − η)2 − 2 (η
σξ2 |
− η)(ξ − ξ) + σξ4 |
ξ −ξ
| {z } {z } {z }
ση2 Kξη σ2
ξ
Kξη 2
2
Kξη 2
Kξη 2
Kξη
= ση2 − 2 σξ2
+ σξ2
= ση2 − σξ2
2
= ση 1 − σξ ση .

Moments
The regression line
We have:
2
2 Kξη
εmin = (η − η̂o ) = (η − η) − σξ2
(ξ − ξ)
2
Kξη 2
Kξη
= (η − η)2 − 2 (η
σξ2 |
− η)(ξ − ξ) + σξ4 |
ξ −ξ
| {z } {z } {z }
ση2 Kξη σ2
ξ
Kξη 2
2
Kξη 2
Kξη 2
Kξη
= ση2 − 2 σξ2
+ σξ2
= ση2 − σξ2
2
= ση 1 − σξ ση .

Moments
The regression line
We have:
2
2 Kξη
εmin = (η − η̂o ) = (η − η) − σξ2
(ξ − ξ)
2
Kξη 2
Kξη
= (η − η)2 − 2 (η
σξ2 |
− η)(ξ − ξ) + σξ4 |
ξ −ξ
| {z } {z } {z }
ση2 Kξη σ2
ξ
Kξη 2
2
Kξη 2
Kξη 2
Kξη
= ση2 − 2 σξ2
+ σξ2
= ση2 − σξ2
2
= ση 1 − σξ ση .

Moments
The regression line
We have:
2
2 Kξη
εmin = (η − η̂o ) = (η − η) − σξ2
(ξ − ξ)
2
Kξη 2
Kξη
= (η − η)2 − 2 (η
σξ2 |
− η)(ξ − ξ) + σξ4 |
ξ −ξ
| {z } {z } {z }
ση2 Kξη σ2
ξ
Kξη 2
2
Kξη 2
Kξη 2
Kξη
= ση2 − 2 σξ2
+ σξ2
= ση2 − σξ2
2
= ση 1 − σξ ση .

Moments
The regression line
We have:
2
2 Kξη
εmin = (η − η̂o ) = (η − η) − σξ2
(ξ − ξ)
2
Kξη 2
Kξη
= (η − η)2 − 2 (η
σξ2 |
− η)(ξ − ξ) + σξ4 |
ξ −ξ
| {z } {z } {z }
ση2 Kξη σ2
ξ
Kξη 2
2
Kξη 2
Kξη 2
Kξη
= ση2 − 2 σξ2
+ σξ2
= ση2 − σξ2
2
= ση 1 − σξ ση .

Moments
The regression line
We define the correlation coefficient:

Kξη
ρξη =
σξ ση
The minimal mean-squared error can be written as:
εmin = ση2 1 − ρ2ξη

We notice that |ρξη | ≤ 1, as εmin ≥ 0!

|ρξη | % 1 =⇒ εmin & 0.
|ρξη | = 1 =⇒ εmin = 0. Functional dependence
between η and ξ!
Moments
The regression line

Kξη
ρξη =
σξ ση


|ρξη | % 1 =⇒ εmin & 0.
between η and ξ!
Moments
The regression line

Kξη
ρξη =
σξ ση


|ρξη | % 1 =⇒ εmin & 0.
between η and ξ!
Moments
The regression line

Kξη
ρξη =
σξ ση


|ρξη | % 1 =⇒ εmin & 0.
between η and ξ!
Moments
The regression line

Kξη
ρξη =
σξ ση


|ρξη | % 1 =⇒ εmin & 0.
between η and ξ!
Moments
The regression line

Kξη
ρξη =
σξ ση


|ρξη | % 1 =⇒ εmin & 0.
between η and ξ!
Moments
The regression line

Kξη
ρξη =
σξ ση


|ρξη | % 1 =⇒ εmin & 0.
between η and ξ!
Moments
The regression line

Kξη
ρξη =
σξ ση


|ρξη | % 1 =⇒ εmin & 0.
between η and ξ!
Moments
The regression line
ρξη < 0 implies the regression line has negative slope!

4 4 4
3
ρξη=1 3
ρ =0.8 3
ρξη=0.4
ξη
2 2 2
1 1 1
0 0 0
−1 −1 −1
−2 −2 −2
−3 −3 −3
−4 −4 −4
−5 −4 −3 −2 −1 0 1 2 3 4 5 −4 −3 −2 −1 0 1 2 3 4 5 −5 −4 −3 −2 −1 0 1 2 3 4 5
4 4 4
3
ρξη=0 3
ρ =−0.8 3
ρ =−1
ξη ξη
2 2 2
1 1 1
0 0 0
−1 −1 −1
−2 −2 −2
−3 −3 −3
−4 −4 −4
−5 −4 −3 −2 −1 0 1 2 3 4 5 −5 −4 −3 −2 −1 0 1 2 3 4 −5 −4 −3 −2 −1 0 1 2 3 4 5

Moments
The regression line
Correlation, covariance, correlation coefficient
Rξη = ξη: uninformative numerical value per se. Two RVs

with Rξη = 6 could be perfectly correlated or completely
uncorrelated.
Kξη = ξη − ξη: centered correlation. Kξη = 0 =⇒
uncorrelatedness. If Kξη = 6 there is correlation, but we
don’t know how strong it is.
K
ρξη = σξξη
ση : centered and normalized correlation. Yields the
degree of linear dependence on a scale from 0 la 100%.

Moments
The regression line

uncorrelated.
K
ρξη = σξξη

Moments
The regression line

uncorrelated.
K
ρξη = σξξη

Moments
The regression line

uncorrelated.
K
ρξη = σξξη

Moments
The regression line

uncorrelated.
K
ρξη = σξξη

Moments
The regression line

uncorrelated.
K
ρξη = σξξη

Moments
The regression line

uncorrelated.
K
ρξη = σξξη

Joint normal distribution
Part II
Joint normal (Gaussian) distribution

(ξ, η) : N (m1 , m2 , σ1 , σ2 , r ):
"
1 (x − m1 )2
1
wξη (x, y ) = √ exp − −
2πσ1 σ2 1 − r2 σ12
2(1 − r 2 )
!#
(x − m1 )(y − m2 ) (y − m2 )2
−2r +
σ1 σ2 σ22
The distribution’s parameters: m1 , m2 ∈ R, σ1 , σ2 > 0, |r | ≤ 1.

(ξ, η) : N (m1 , m2 , σ1 , σ2 , r ):
"
1 (x − m1 )2
1
wξη (x, y ) = √ exp − −
2πσ1 σ2 1 − r2 σ12
2(1 − r 2 )
!#
(x − m1 )(y − m2 ) (y − m2 )2
−2r +
σ1 σ2 σ22

(ξ, η) : N (m1 , m2 , σ1 , σ2 , r ):
"
1 (x − m1 )2
1
wξη (x, y ) = √ exp − −
2πσ1 σ2 1 − r2 σ12
2(1 − r 2 )
!#
(x − m1 )(y − m2 ) (y − m2 )2
−2r +
σ1 σ2 σ22

R∞ 2

wξ (x) = wξη (x, y )dy = · · · = 1
√
σ1 2π
exp − (x−m
2σ 2
1)
.
−∞ 1
R∞ 2

wξ (y ) = wξη (x, y )dx = · · · = 1
√
σ2 2π
exp − (y −m
2σ 2
2)
.
−∞ 2
If ξ and η are jointly normally distributed, then both ξ and η

are normally (1D) distributed.
Reciprocal
is not necessarily true!!. Ex: ξ : N (m, σ) and
ξ if ξ < σ
η= .
−ξ otherwise

R∞ 2

wξ (x) = wξη (x, y )dy = · · · = 1
√
σ1 2π
exp − (x−m
2σ 2
1)
.
−∞ 1
R∞ 2

wξ (y ) = wξη (x, y )dx = · · · = 1
√
σ2 2π
exp − (y −m
2σ 2
2)
.
−∞ 2

Reciprocal
ξ if ξ < σ
η= .
−ξ otherwise

R∞ 2

wξ (x) = wξη (x, y )dy = · · · = 1
√
σ1 2π
exp − (x−m
2σ 2
1)
.
−∞ 1
R∞ 2

wξ (y ) = wξη (x, y )dx = · · · = 1
√
σ2 2π
exp − (y −m
2σ 2
2)
.
−∞ 2

Reciprocal
ξ if ξ < σ
η= .
−ξ otherwise

R∞ 2

wξ (x) = wξη (x, y )dy = · · · = 1
√
σ1 2π
exp − (x−m
2σ 2
1)
.
−∞ 1
R∞ 2

wξ (y ) = wξη (x, y )dx = · · · = 1
√
σ2 2π
exp − (y −m
2σ 2
2)
.
−∞ 2

Reciprocal
ξ if ξ < σ
η= .
−ξ otherwise

R∞ 2

wξ (x) = wξη (x, y )dy = · · · = 1
√
σ1 2π
exp − (x−m
2σ 2
1)
.
−∞ 1
R∞ 2

wξ (y ) = wξη (x, y )dx = · · · = 1
√
σ2 2π
exp − (y −m
2σ 2
2)
.
−∞ 2

Reciprocal
ξ if ξ < σ
η= .
−ξ otherwise

Who is r ?
Kξη
ρξη = σξ ση = · · · = r!
r=0,3 r = 0,8
If r = 0: 2
(y −m2 )2
h i
wξη (x, y ) = 1
2πσ1 σ2 exp − 21 (x−m
σ2
1)
+ σ22
1
= wξ (x)wη (y ).
In the hypothesis of joint normal distribution, independence
⇔ uncorrelatedness!

Who is r ?
Kξη
ρξη = σξ ση = · · · = r!
r=0,3 r = 0,8
If r = 0: 2
(y −m2 )2
h i
wξη (x, y ) = 1
2πσ1 σ2 exp − 21 (x−m
σ2
1)
+ σ22
1
= wξ (x)wη (y ).

Who is r ?
Kξη
ρξη = σξ ση = · · · = r!
r=0,3 r = 0,8
If r = 0: 2
(y −m2 )2
h i
wξη (x, y ) = 1
2πσ1 σ2 exp − 21 (x−m
σ2
1)
+ σ22
1
= wξ (x)wη (y ).

Who is r ?
Kξη
ρξη = σξ ση = · · · = r!
r=0,3 r = 0,8
If r = 0: 2
(y −m2 )2
h i
wξη (x, y ) = 1
2πσ1 σ2 exp − 21 (x−m
σ2
1)
+ σ22
1
= wξ (x)wη (y ).

Who is r ?
Kξη
ρξη = σξ ση = · · · = r!
r=0,3 r = 0,8
If r = 0: 2
(y −m2 )2
h i
wξη (x, y ) = 1
2πσ1 σ2 exp − 21 (x−m
σ2
1)
+ σ22
1
= wξ (x)wη (y ).

Curs 7 DEPI - 2va3 - Slides - EN

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Curs 7 DEPI - 2va3 - Slides - EN

Uploaded by

Copyright:

Available Formats

Decision and estimation in information processing:

March 29, 2021

Decision and estimation in information processing: course nr. 7

Decision and estimation in information processing: course nr. 7

Joint uncentered moment of order (k, l):

The uncentered moment of order (1, 1) is called correlation

Decision and estimation in information processing: course nr. 7

Joint uncentered moment of order (k, l):

The uncentered moment of order (1, 1) is called correlation

Decision and estimation in information processing: course nr. 7

Joint uncentered moment of order (k, l):

The uncentered moment of order (1, 1) is called correlation

Decision and estimation in information processing: course nr. 7

Joint uncentered moment of order (k, l):

The uncentered moment of order (1, 1) is called correlation

Decision and estimation in information processing: course nr. 7

The joint centered moment of order (k, l):

The centered moment of order (1, 1) is called covariance

Decision and estimation in information processing: course nr. 7

The joint centered moment of order (k, l):

The centered moment of order (1, 1) is called covariance

Decision and estimation in information processing: course nr. 7

The joint centered moment of order (k, l):

The centered moment of order (1, 1) is called covariance

Decision and estimation in information processing: course nr. 7

The joint centered moment of order (k, l):

The centered moment of order (1, 1) is called covariance

Decision and estimation in information processing: course nr. 7

Relation between correlation and covariance

Decision and estimation in information processing: course nr. 7

Relation between correlation and covariance

Decision and estimation in information processing: course nr. 7

Relation between correlation and covariance

Decision and estimation in information processing: course nr. 7

Relation between correlation and covariance

Decision and estimation in information processing: course nr. 7

Relation between correlation and covariance

Decision and estimation in information processing: course nr. 7

Relation between correlation and covariance

Decision and estimation in information processing: course nr. 7

Relation between correlation and covariance

Decision and estimation in information processing: course nr. 7

Relation between correlation and covariance

Decision and estimation in information processing: course nr. 7

RVs ξ and η are called uncorrelated if their covariance is null:

Property: Independence =⇒ uncorrelatedness!

Decision and estimation in information processing: course nr. 7

RVs ξ and η are called uncorrelated if their covariance is null:

Property: Independence =⇒ uncorrelatedness!

Decision and estimation in information processing: course nr. 7

RVs ξ and η are called uncorrelated if their covariance is null:

Property: Independence =⇒ uncorrelatedness!

Decision and estimation in information processing: course nr. 7

RVs ξ and η are called uncorrelated if their covariance is null:

Property: Independence =⇒ uncorrelatedness!

Decision and estimation in information processing: course nr. 7

RVs ξ and η are called uncorrelated if their covariance is null:

Property: Independence =⇒ uncorrelatedness!

Decision and estimation in information processing: course nr. 7

RVs ξ and η are called uncorrelated if their covariance is null:

Property: Independence =⇒ uncorrelatedness!

Decision and estimation in information processing: course nr. 7

RVs ξ and η are called uncorrelated if their covariance is null: