Linear Regression

Dr Menaal Kaushal
JR II
Department of S P M
S N Medical College, Agra
1 22-11-2013
 Statistical Analysis can be:
 Univariate: When Only one variable is studied. E.g
Heights of all the IV graders, ages of mothers
delivering at a DH, etc. (Measures of Central
Tendency, Measures of Dispersion)
 Bivariate: When relationship between two variables
are studied. e.g. Relationship between height and
weight of Every Child in the IV grade; relation
between mother’s age & birth weight of her baby, etc.
 Multivariate: When relationship between more than
two variables are studied. E.g Relationship between
height, weight and MAC of every child in the IV grade
2 22-11-2013
 Bivariate Regression

 Linear Regression: When the data is

continuous

 Logistic Regression: When the data is

categorical, e.g. the research question can
be answered as either yes or no category
3 22-11-2013
 Levels (Types) of Data

 Nominal (Categorical) Measures: Are exhaustive

and mutually exclusive (e.g., religion), gender
 Ordinal Measures: All of the above plus can be
rank-ordered (e.g., social class).
 Interval Measures: All of the above plus equal
differences between measurement points
(temperature in ℃ or ℉ ).
 Ratio Measures: All of the above plus a true zero
point (weight, Absolute Temperature in Kelvin).
4 22-11-2013
 Relationship Between Two
Variables
 Association: any relation between variables

 Positive association: above average values of one variable

tend to go with above average values of the other; the scatter
slopes up

 Negative association: above average values of one variable

tend to go with below average values of the other; the scatter
slopes down

 Linear association: roughly, the scatter diagram is clustered

around a straight line. This is Correlation
5 22-11-2013
6 22-11-2013
[‘p-0

7 22-11-2013
8 22-11-2013
The “Football” Bivariate
Normal Scatter Plot

9 22-11-2013
Can you identify any
difference?

10 22-11-2013
How Tightly Clustered
Are these Data?

11 22-11-2013
Calculating the Correlation
Coefficient

12 22-11-2013
So, How to Calculate r

13 22-11-2013
 Formula of Correlation
Coefficient

Lets Simplify:
 Convert the data into Standard units.
 Multiply the corresponding standard unit values
of x and y
 r is the mean of this product
14 22-11-2013
 Properties of Correlation
Coefficient
 The calculations uses only standard units so r is a pure
number with no units

 -1≤ r ≤ 1

 In the extreme cases, r = -1 when the scatter diagram is a

perfect straight line sloping down. If r = 1, the scatter
diagram is a perfect line sloping up

 Switching the variables x and y does not change r. it

remains the same

15 22-11-2013
 Adding a constant to one of the lists just slides the
scatter diagram so r stays the same

 Multiplying one of the lists by a positive constant does

not change standard units so r stays the same

 Multiplying just one (not both) of the lists by a negative

constant switches the signs of the standard units of that
variable, so r has the same absolute value but its sign gets
switched.

16 22-11-2013
Heteroscadastic Curve

17 22-11-2013
 What r can not tell?
 Association is not causation. r does not tell “Why”

 r is only used for linearly correlated variables. It

measures linear association.

 This diagram shows a strong relation

between x& y, but it is not linear. But r

for this diagram comes out to be Zero

18 22-11-2013
 Beware of:

 Outliers

 Tendency for Ecological correlations

19 22-11-2013
Deal with the outliers

20 22-11-2013
Can you find the outlier?

21 22-11-2013
Avoid “Ecological
Correlation”:

Replacing students by averages

can artificially increase
clustering. This is not desirable.

22 22-11-2013
 Regression

 The technique to estimate dependent variable

“y”, for a given value of variable “x” when they

are linearly associated and the correlation

coefficient “r” is known.

23 22-11-2013
Each estimate is at the center of the vertical strip
22-11-2013 24
25 22-11-2013
The slope of the green line= r

26 22-11-2013
 The Equation of Regression
 Estimate of y = r* given x (in Standard units)

 ⇒ estimate of y- µy = r (x- µx)

SDy SDx

 Estimate of y= Slope* (x) + intercept

 (Here Slope= r* SDy / SDx and intercept= µy-slope*x)

27 22-11-2013
 Why call “Regression”
 Sir Francis Galton 1822- 1911: “The Galton Effect”
 “Those who have high values in one variable tend to
be not as high in the second variable”
 A eugenicist, who gave the idea of SD and regression
 “Fathers who are tall, tend to have sons who are not
quite that tall on average”
 All data regresses towards “mediocrity”
 i.e. regresses towards mean
 The Regression Fallacy or Sophomore Slump
28 22-11-2013
29 22-11-2013
Univariate Normal Bivariate Normal

+1 r.m.s.
error
68%
68% r

µx
+1 SD

30 22-11-2013
 Residual Plot

Regardless of the shape of the scatter diagram:

the average of the residuals is Always 0,
There is No linear association between residuals and x.
The residual plot should not show any trend or linear
relation.
Good regression: Residual plot should look like a formless
31 22-11-2013
blob around the horizontal axis
Residual Plot as a Diagnostic
Tool

32 22-11-2013
 Questions??
33 22-11-2013