Plane of regression for three variables

1. The equation of the plane of regression of X1 on X2 and X3 is as follows:

X1  b12.3 X 2  b13.2 X 3
If X3 is considered as constant, then the graph of X1 and X2 is a straight line with slope b12.3 and
if X2 is considered as constant then b13.2 is the slope of the line.
According the principle of least squares, we have to determine the constants b12.3 and b13.2 in
such a way that the sum of the squares of residuals is minimum i.e.
U   ( x1  b12.3 x2  b13.2 x3 )2  0
By the principle of maxima and minima, we take the partial derivatives of U with respect to two
constants: Thus,
U U
 0 and 0
b12.3 b13.2
That’s imply,  2( x  b x  b 1 12.3 2 x )( x2 )  0  ( x1  b12.3 x2  b13.2 x3 ) x2  0
13.2 3

 b12.3 x b x x x x
2
2
13.2 2 3 1 2 0 (1)
Similarly, it can be found,
U
 b12.3  x2 x3  b13.2  x32   x1 x3  0 from 0 (2)
b13.2
As we all know that,
1
i2 
N
 ( xi  xi )2 for i=1,2,3.

1

N
 xi 2  xi 2

Since x1, x2, x3 are measured from their means therefore the value means will be 0. As a result,
1
i2 
N
 ( xi )2 for i=1,2,3. (3)

1
And cov( xi , x j ) 
N
 xi x j for i and j=1,2,…,n. (4)

Now, rij 
Cov( xi x j )

x x i j

V ( xi )V ( x j )  i j

It implies that cov( xi , x j )  rij i j (5)

Dividing N of equations (1) and (2) and using the equations (3), (4), and (5), we can write
Equation (1) we can write
b12.3 2 2  b13.2 cov( x2 , x3 )  cov( x1 , x2 )  0 (6)
Which implies,
b12.3 2  b13.2 r23 3  r121  0
And from the equation (2) we can get b12.3r23 2  b13.2 3  r131  0 (7)
Where r12 is the total correlation coefficient of between x1 and x2, r13 is the total correlation
coefficient of between x1 and x3, and r23 is the total correlation coefficient of between x3 and x2.
Solving equations (6) and (7) we can find
r12 1 r23 3
r13 1 3  (r  r r )
b12.3   1 12 13 223 (8)
2 r23 3  2 (1  r23 )
r23 2 3
And similarly we can get,
 1 (r13  r12 r23 )
b13.2  (9)
 3 (1  r232 )
If we write now
1 r12 r13
t  r21 1 r23
r31 r32 1
 1t12 t
and then b12.3   and b13.2   1 13 where tij is the cofactor of the element in the ith row and
 2t11  2t11
th
j column of t matrix.
1. Finally, putting the values of b12.3 and b13.2 in the equation of plane regression we can find
the plane of regression of X1 on X2 and X3 is as follows:
 1 t12  t x x x
x1   x2  1 13 x3  1 t11  2 t12  3 t13  0
 2 t11  3 t11 1 2 3
Similarly, we can find the other plane of regression like:
The plane of regression of x2 on x1 and x3 is
x1 x2 x3
t21  t22  t23  0
1 2 3
And the plane of regression of x3 on x1 and x2 is
x1 x2 x3
t31  t32  t33  0
1 2 3
 Problem 1
From the given data in the following table find out
1. Least square regression equation of X1 on X2 and X3.
2. Estimate the value of X1 for X2  45 and X3  8.
The table is as follows:

X1 1 2 3 4 5
X2 3 4 5 6 7
X3 4 5 6 7 8

Solution: Here X1, X2, X3 are three random variables with their respective means X1 , X 2 , X 3
Then let us consider x1  X1  X 1 , x2  X 2  X 2 , x3  X 3  X 3
The linear regression equation of x1 on x2 and x3 is x1  b12.3 x2  b13.2 x3

From the least square equation solution we have

 1 (r12  r13 r23 )
b12.3 
 3 (1  r232 )
 1 (r13  r12 r23 )
b13.2 
 3 (1  r232 )
The value of 1, 2 , 3, r12, r13, r23 can be obtained through some calculations given in the
following table:

Sl. X1 X2 X3 x1  x 2  x 3  (x )2
1
(x )2
2
(x )2
3
x1x2 x1x3 x2 x3
No
X1  5 X2  6 X3  7

1 1 3 4 -4 -3 -3 16 9 9 12 12 9
2 3 5 5 -2 -1 -2 4 1 4 2 4 2
3 4 6 6 -1 0 -1 1 0 1 0 1 0
4 7 7 9 2 1 2 4 1 4 2 4 2
5 10 9 11 5 3 4 25 9 16 15 20 12
Total 25 30 35 0 0 0 50 20 34 31 41 25

25 30 35
Here, X1   5, X 2   6, X 3  7
5 5 5
1 50
And 12   xi2   10  1  10  3.162
5 5
1 20
 22   x22   4   2  2
5 5
1 34
 32   x32   6.8   3  6.8  2.608
5 5
r12 
x x1 2

31
 0.98
N 1 2 5*3.162*2
41
r13   0.994
5*2.608*3.162
25
r23   0.9596
5*2.608*2
Hence, from the formula we can get
b12.3  0.527andb12.3  0.818
The equation of regression plane of x1 on x2 and x3 is x1  0.527 x2  0.818x3
After substituting the value we can get X1  3.891  0.527 X 2  0.818 X 3
For 2nd part of the problem we have to put the values of the random variables, we get,
X1=26.368

Assignment:
From the given data in the following table find out
1. Least square regression equation of X1 on X2 and X3.
2. Estimate the value of X1 for X2  6 and X3  8.
The table is as follows:
X1 2 6 8 10
X1 4 5 9 12
X1 4 6 10 12