Professional Documents
Culture Documents
Chapter6 Sampling Regression Method Estimation
Chapter6 Sampling Regression Method Estimation
In ratio method, the conventional estimator sample mean y was improved by multiplying it by a factor
X
where x is an unbiased estimator of the population mean X which is chosen as the population
x
mean of auxiliary variable. Now we consider another idea based on difference.
Yˆ * y ( x X )
which is an unbiased estimator of Y and is any constant. Now find such that the Var (Yˆ * ) is
minimum
Consider a linear regression model y x e where y is the dependent variable, x is the independent
variable and e is the random error component which takes care of the difference arising due to lack of
exact relationship between x and y.
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 1
Note that the value of regression coefficient in a linear regression model y x e of y on x
n
Cov( x, y) S xy
obtained by minimizing e
i 1
2
i based on n data sets ( xi , yi ), i 1, 2,.., n is
Var ( x)
2 . Thus the
Sx
optimum value of is same as the regression coefficient of y on x with a negative sign, i.e.,
.
Yˆreg y ( X x )
which is the regression estimator of Y and the procedure of estimation is called as the regression
method of estimation.
where ( x , y ) is the correlation coefficient between x and y . So Yˆreg would be efficient if x and y
are highly correlated. The estimator Yˆreg is more efficient than Y if ( x , y ) 0 which generally holds.
Yˆreg y 0 ( X x ) .
Bias of Yˆreg :
Now, assuming that the random sample ( xi , yi ), i 1, 2,.., n is drawn by SRSWOR,
E (Yˆreg ) E ( y ) 0 X E ( x )
Y 0 X X
Y
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 2
Variance of Yˆreg
2
Var (Yˆreg ) E Yˆreg E (Yˆreg )
2
E y 0 ( X x ) Y
2
E ( y Y ) 0 ( x X )
E ( y Y ) 2 02 ( x X ) 2 2 0 E ( x X )( y Y )
Var ( y ) 02Var ( x ) 2 0Cov( x , y )
f
SY2 02 S X2 2 0 S XY
n
f
SY2 02 S X2 2 0 S X SY
n
where
N n
f
N
1 N
S X2
N 1 i 1
( X i X )2
1 N
S
2
Y
N 1 i 1
(Yi Y )2
if 02 S X2 20 S XY 0
2S
or 0 S X2 0 2XY 0
SX
2S 2S
or 0 0 and 0 2XY ) 0 0 0 2XY .
SX SX
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 3
Optimal value of
So
Var (Yˆreg )
SY2 2 S X2 2 S X SY 0
S S
Y XY2 .
SX SX
S
The minimum value of the variance of Yˆreg with optimum value of opt Y is
SX
f S2
Varmin (Yˆreg ) SY2 2 Y2 S X2 2 Y S X SY
S
n SX SX
f
SY2 (1 2 ).
n
Since 1 1, so
which always holds true. So the regression estimator is always better than the sample mean under
SRSWOR.
Departure from :
If 0 is the preassigned value of regression coefficient, then
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 4
Estimate of variance
An unbiased sample estimate of Var (Yˆreg ) is
n
(Yˆ ) f
( yi y ) 0 ( xi x )
2
Var
n( n 1) i 1
reg
f
( s y2 02 sx2 2 0 s xy ).
n
Note that the variance of Yˆreg increases as the difference between 0 and opt increases.
sxy
ˆ
sx2
Yˆreeg y ˆ ( X x ).
It is difficult to find the exact expressions of E (Yreg ) and Var (Yˆreg ). So we approximate them using the
Then
E ( 0 ) 0, E ( 1 ) 0,
E ( 2 ) 0, E ( 3 ) 0,
f 2
E ( 02 ) CY ,
n
f
E (12 ) C X2 ,
n
f
E ( 0 1 ) C X CY
n
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 5
and
sxy
Yreg y (X x)
sx2
S XY (1 2 )
Y (1 0 ) (1 X ).
S x2 (1 3 )
S XY
where is the population regression coefficient.
S X2
Assuming 3 1,
Retaining the terms up to second power of 's and ignoring other terms, we have
Bias of Yˆreg
N n
where f and (r , s)th cross-product moment is given by
N
rs E ( x X )r ( y Y )s
So that
21 E ( x X )2 ( y Y )
30 E ( x X )3 .
Thus
f 21 30
E (Yˆreg ) 2 .
n S XY S X
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 6
Also,
E (Yˆreg ) E ( y ) E[ ˆ ( X x )]
Y XE ( ˆ ) E ( ˆ x )
Y E ( x ) E ( ˆ ) E ( ˆ x )
Y Cov( ˆ , x )
Bias (Yˆreg ) E (Yˆreg ) Y Cov( ˆ , x )
MSE of Yˆreg
Retaining the terms of 's up to the second power second and ignoring others, we have
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 7
Comparison of Yˆreg with ratio estimate and SRS sample mean estimate
(i) As MSE (Yˆreg ) VarSRS ( y )(1 2 ) and because 2 1, so Yˆreg is always superior to y .
So regression estimate is always superior to the ratio estimate upto the second order of
approximation.
( xij , yij ), j 1, 2,.., ni , is drawn from ith strata (i = 1,2,..,k) by SRSWOR where xij and yij denote the
jth unit from ith strata on auxiliary and study variables, respectively.
Yˆreg y 0 ( X x )
from each stratum separately, i.e., the regression estimate in the ith stratum is
Yˆreg (i ) yi i ( X i xi ).
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 8
Find the stratified mean as the weighted mean of Yˆreg (i ) i 1, 2,.., k as
k N Yˆ
ˆ
Ysreg i reg ( i )
i 1 N
k
[ wi { yi i ( X i xi )}]
i 1
Sixy Ni
where i 2
, wi .
S ix N
In this approach, the regression estimator is separately obtained in each of the strata and then
combined using the philosophy of the stratified sample. So Yˆsreg is termed as separate regression
estimator,
In this case, all the sample information is combined first and then implemented in regression
- when is pre-assigned as 0
sxy
We consider here the case that is pre-assigned as 0 . Other case when is estimated as ˆ 2
sx
can be dealt with the same approach based on defining various 's and using the approximation theory
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 9
1. Separate regression estimator
Assume is known, say 0 . Then
k
Yˆs reg wi [ yi 0i ( X i xi )]
i 1
k
E (Yˆs reg ) wi E ( yi ) 0i X i E ( xi )
i 1
k
wi [Yi ( X i X i )]
i 1
Y.
2
Var (Yˆs reg ) E Yˆs reg E (Yˆs reg )
2
k k
E wi yi i wi 0i ( X i xi ) Y
i 1 i 1
2
k k
E wi ( yi Y ) wi 0i ( xi X i )
i 1 i 1
k k k
wi2 E ( yi Yi ) 2 wi2 02i E ( xi X i )]2 2 wi2 0i E ( xi X i )( yi Yi )
i 1 i 1 i 1
k k k
wi2Var ( yi ) wi2 02iVar ( xi ) 2 wi2 0i Cov( xi , yi )
i 1 i 1 i 1
k 2
w f 2
( SiY 02i SiX2 2 0i SiXY )]
i i
i 1 ni
(Yˆ ) wi fi ( s 2 2 s 2 2 s )
k 2
Var s reg
i 1 ni
iy oi ix 0 i ixy
and
ˆ
k
wi2 fi 2
Var min (Ys reg )
( siy oi2 six2 ) .
i 1 ni
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 10
2. Combined regression estimator:
Assume is known as 0 . Then
k k
Yˆc reg wi yi 0 ( X wi xi )
i 1 i 1
k k
E Yˆc reg wi E ( yi ) 0 [ X wi E ( xi )]
i 1 i 1
k k
wY
i i 0 [ X wi X i ]
i 1 i 1
Y 0 ( X X )
Y.
Thus Yˆc reg is an unbiased estimator of Y .
i 1 ni
Var (Yˆcreg ) is minimum when
i 1 ni
SiXY
k 2
wi fi 2
i 1 ni
SiX
Since SRSWOR is followed to draw the sample from strata, so using E six2 Six2 , E siy2 Siy2 and
E sixy SiXY , we get the estimate of variance as
(Yˆ ) wi fi ( s 2 2 s 2 2 s )
k 2
Var c reg
i 1 ni
iy o ix 0 i ixy
and
ˆ
k
wi2 fi 2
Var min (Yc reg )
( siy oi2 six2 ) .
i 1 ni
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 11
Comparison of Yˆs reg and Yˆcreg :
Sampling Theory| Chapter 6 | Regression Method of Estimation | Shalabh, IIT Kanpur Page 12