This action might not be possible to undo. Are you sure you want to continue?

Variance, Covariance and Covariance Matrix

The variance for a set of numbers is defined to have a measure how the data points deviate (or vary) from their mean. For a set of numbers: , ,…. , Variance, ∑( ∑ ̅) . ). ( ), the mean of the

square of the deviations from mean, where the mean, ̅

Note that the square root of variance is the standard deviation (

̅̅̅ The above formula can be reduced to the following: ̅ ( ), the mean of square minus the square of mean. By definition, variance is always positive and it has no upper limit. We can likewise, calculate variance for another set of numbers: ∑( ̅) . , ,…. , Variance,

Now we may want to know if there is any connection between the two sets of numbers! For that we devise a formula which may tell us how the variation in one can be related to the same with the other. Let us rewrite the expression (2) in the following way: Following the above, we can construct a formula: or in some other notation, ̅̅̅̅̅ ̅̅̅̅̅ ̅ ̅

̅ ̅, this is COVARIANCE!

If we consider a probability distribution, that is when we consider that the numbers appear according to some probability, the formula for variance is written in the following way: ( ) , where ‘ ’ stands for the expectation value, replaced by the expectation value of the variable . Now, the covariance formula would be written as Cov( Where ( ) and ( ). )= (( )( )) ( ) , ( ), the mean is now

Abhijit Kar Gupta.

Email: kg.abhi@gmail.com

2

By definition, covariance can take any positive or negative value without limit including zero. This is easy to understand if we consider that the covariance is defined in terms of the product )( of two factors, ( ); each of the factors can be positive or negative. To have a normalized measure, one considers the correlation formula: Corr ( )= , it is a normalized definition. Correlation is defined to vary between -1 and +1.

What is the meaning of covariance? If the covariance is positive, both the factors are either positive or both negative. If the covariance is negative, one of the factors is positive while other is negative. In case of zero value, one or both are zero. Now, if we consider two sets of numbers (or two probability distributions for two different events), the covariance value of them tells us how or whether they are related. If the increase in one variable causes in some way the increase in the other, we call they are positively correlated. That means their covariance value is positive. If the increase in one makes another to decrease (or other way), the covariance value will be negative. If the variation in one does not affect the other, the covariance is zero. Now, we can readily see, for two sets of variables (or numbers or probability distributions), we can construct a 2x2 matrix: ( )

It is easy to check that the this matrix, we call this COVARIANCE MATRIX, is a symmetric matrix with and each diagonal elements ( or ) is essentially the covariance with itself (simply the variance)! We can generalize the idea for more than two variables. For 3 variables, we obtain a covariance matrix of order 3x3 and so on. When the value of any off-diagonal element is positive, we call the variables are positively correlated; for higher values we know they are highly (or strongly) correlated. The above is the basis of understanding a covariance matrix.

Abhijit Kar Gupta.

Email: kg.abhi@gmail.com

3

Abhijit Kar Gupta.

Email: kg.abhi@gmail.com

- Path Analysis for Linear Multiple Regression
- From Pick's to Pythagoras
- Correlation
- A Lecture on Teachers' Day
- Tower of Hanoi
- A Short Course on PROBABILITY and SAMPLING
- From Equations of Motions to Mathematical Identities
- Bangla Chhora
- Lectures on Theoretical Mechanics - 8 [Rigid body Rotation]
- Failure of Democracy in India?
- Lectures on Theoretical Mechanics - 7 [Rotating Coordinate System
- Lectures on Theoretical Mechanics - 6
- Solution to Question Paper IV, V.U. 2013
- Properties of Bulk Matter
- Statistical Mechanics TEST Questions
- Lecture in Theoretical Mechanics - 5
- Lect-V Vector Space
- Vector Space - a Set of Lectures
- Lect-VI Vector Space
- How can you create a SUDOKU puzzle?
- Lect-V Vector Space
- Oscillation and Decay - a pictorial tour through Computer Programs
- Lect-IV Vector Space
- Lect-III Vector Space

A basic exposition on variance, covariance and covariance matrix. It can be understood by anyone.

A basic exposition on variance, covariance and covariance matrix. It can be understood by anyone.

- A Short Course on PROBABILITY and SAMPLING
- Lectures on Theoretical Mechanics - 6
- From Equations of Motions to Mathematical Identities
- Lecture on Theoretical Mechanics - 4
- A short course on Statistics, Probability and Applications
- Lectures on Theoretical Mechanics - 7 [Rotating Coordinate System
- Lect-VI Vector Space
- Properties of Bulk Matter
- Lectures on Theoretical Mechanics - 8 [Rigid body Rotation]
- A Lecture on Teachers' Day
- Oscillation and Decay - a pictorial tour through Computer Programs
- Classical Mechanics I - Lect-1
- Lecture in Theoretical Mechanics - 5
- Lect-V Vector Space
- Solution to Question Paper IV, V.U. 2013
- Magic of Squares
- Vector Space - a Set of Lectures
- Statistical Mechanics TEST Questions
- Lectures on Vector Algebra
- Covariance and Correlation.pdf
- Range of Correlation Coefficient
- Gini mean difference
- Homework 3 Solutions
- AI+WallSt
- tirgul34
- Mean Variance portfolio theory
- Prv
- 12 Correlation
- Variance.pdf
- Temporal Stationarity
- Covariance

Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

We've moved you to where you read on your other device.

Get the full title to continue

Get the full title to continue reading from where you left off, or restart the preview.

scribd