Multiple linear regression

INTRODUCTION: The simple linear regression model is used to determine the effect of a
single independent variable on the dependent variable.

y=β 0 +β 1 x+ε
However, we are often interested in testing whether a dependent variable (y) is related to
more than one independent variable ( x 2 , x3, x4, …) and there by the emphasis is on the
estimation and inferences. The above said process is known as multiple regressions and
indeed, this is commonly done. However, it is possible that the independent variables could
observe each other’s effects. For example, a choice of a restaurant can depend on factors like
cost, convenience and ambience. The cost effect might over rid the convenience effect,
leading to a regression for cost which would not appear very interesting.

One possible solution is to perform a regression with one independent variable, and then test
whether a second independent variable is significant with respect to the residuals from this
regression. Then the second variable is included. Continue in a similar manner to include
other independent variables.

A multiple regression allows the simultaneous testing and modeling of multiple independent
variables.

The model for a multiple regression (called the POPULATION REGRESSION MODEL) takes
the form:
Example

Independent variables Depentent

Year Domin Vertical Import R&D Export Capital Post
ant integrati dependence  intensit competitiven intensity  profit
special on  y  ess  margin
ization 

    (X2)    (X3)     (X4)   (X5)     (X6)   (X7) (y1)
1.27
1974-75 70.58 22.03 18.29 1.48 3.61 4.3
1.39
1975-76 75.39 21.93 18.56 1.7 4.38 4.71
3.2
1976-77 74.43 23.6 16.77 1.81 5.88 4.63
3.12
1977-78 73.51 21.27 15.16 1.77 7.8 4.48
3.19
1978-79 74.78 20.04 15.43 1.96 6.92 4.1
3.28
1979-80 71.73 19.34 16.92 2.62 6.11 4
4.41
1980-81 72.34 18.27 17.92 3.39 5.31 3.75
5.06
1981-82 73.14 18.91 16.06 2.97 5.18 3.64
4.17
1982-83 76.31 17.33 15.93 2.12 5.63 3.85
3.27
1983-84 75.4 17.72 15.52 1.88 4.11 3.9
3.41
1984-85 75.37 18.76 14.23 2.6 4.49 3.91
2.52
1985-86 73.03 18.32 16.03 2.16 3.37 3.9
1.08
1986-87 74.66 17.56 17.35 1.6 2.64 3.96
1.16
1987-88 77.51 16.79 16.44 1.84 2.91 4
2.24
1988-89 77.32 17.76 16.04 1.6 3.4 3.62
3.06
1989-90 77.3 18.14 15.34 1.94 4.38 3.73

Model is: y1 = b0 +b1x1+ b2 x2 + b3 x3 + b4 x4 + b5 x5 + b6 x6 + b7 x7

X is:
(16*7)
1 70.58 22.03 18.29 1.48 3.61 4.3
1 75.39 21.93 18.56 1.7 4.38 4.71
1 74.43 23.6 16.77 1.81 5.88 4.63
1 73.51 21.27 15.16 1.77 7.8 4.48
1 74.78 20.04 15.43 1.96 6.92 4.1
1 71.73 19.34 16.92 2.62 6.11 4
1 72.34 18.27 17.92 3.39 5.31 3.75
1 73.14 18.91 16.06 2.97 5.18 3.64
1 76.31 17.33 15.93 2.12 5.63 3.85
1 75.4 17.72 15.52 1.88 4.11 3.9
1 75.37 18.76 14.23 2.6 4.49 3.91
1 73.03 18.32 16.03 2.16 3.37 3.9
1 74.66 17.56 17.35 1.6 2.64 3.96
1 77.51 16.79 16.44 1.84 2.91 4
1 77.32 17.76 16.04 1.6 3.4 3.62
1 77.3 18.14 15.34 1.94 4.38 3.73

Y is
1.27
(16*1)
1.39
3.2
3.12
3.19
3.28
4.41
5.06
4.17
3.27
3.41
2.52
1.08
1.16
2.24
3.06 Mean(Y): 2.864375
The Model is: y Predicted = -3.0472 +0*x1+ 0.0930465 * x2 +0.2502425 * x3
- 0.101457 * X4+ 1.1340792 * x5 + 0.3784979 * x6 - 2.071527 * x7

  Finding Explained and Unexplained Variations

Explained Variation: Yesti - Ybar: sum(Square(Yesti - Ybar))
UnExplained Variation: Y - Yesti:sum(Square(Y - Yesti))
Total Variation=Explained Variation +Un Explained Variation
   

Explained Unexplained Explained Unexplained Total

Variation Variation Variation Variation Variation
Actual(Y Predicted(Ye Predicted(Ye Actual(Y1) - Predicted(Ye Actual(Y1) - Explained
1) sti) sti) - Mean(Y) Predicted(Ye sti) - Mean(Y) Predicted(Ye Variation
sti) Square sti) Square +
Unexplain
ed
 Variation
1.27 1.314471998 -1.549903 -0.044471998 2.402199316 0.001978 2.404177
1.39 1.401222862 -1.4631521 -0.011222862 2.140814178 0.000126 2.14094
3.2 2.769628153 -0.0947468 0.430371847 0.008976965 0.18522 0.194197
3.12 3.256387332 0.39201233 -0.136387332 0.153673669 0.018602 0.172275
3.19 3.708942087 0.84456709 -0.518942087 0.713293564 0.269301 0.982594
3.28 3.747871945 0.88349695 -0.467871945 0.780566852 0.218904 0.999471
4.41 4.523738777 1.65936378 -0.113738777 2.753488144 0.012937 2.766425
5.06 4.649390316 1.78501532 0.410609684 3.186279678 0.1686 3.35488
4.17 3.333489981 0.46911498 0.836510019 0.220068866 0.699749 0.919818
3.27 2.436937176 -0.4274378 0.833062824 0.182703093 0.693994 0.876697
3.41 3.764927874 0.90055287 -0.354927874 0.810995479 0.125974 0.936969
2.52 2.352273326 -0.5121017 0.167726674 0.262248125 0.028132 0.29038
1.08 1.144152734 -1.7202223 -0.064152734 2.959164646 0.004116 2.96328
1.16 1.600486374 -1.2638886 -0.440486374 1.59741446 0.194028 1.791443
2.24 2.566585 -0.29779 -0.32658 0.088679 0.106657 0.195337
3.06 3.259483 0.395108 -0.19948 0.15611 0.039793 0.195903
b=Inverse(X Transpose * X) *X Transpose*Y

X is:
(16*7)
1 70.58 22.03 18.29 1.48 3.61 4.3
1 75.39 21.93 18.56 1.7 4.38 4.71
1 74.43 23.6 16.77 1.81 5.88 4.63
1 73.51 21.27 15.16 1.77 7.8 4.48
1 74.78 20.04 15.43 1.96 6.92 4.1
1 71.73 19.34 16.92 2.62 6.11 4
1 72.34 18.27 17.92 3.39 5.31 3.75
1 73.14 18.91 16.06 2.97 5.18 3.64
1 76.31 17.33 15.93 2.12 5.63 3.85
1 75.4 17.72 15.52 1.88 4.11 3.9
1 75.37 18.76 14.23 2.6 4.49 3.91
1 73.03 18.32 16.03 2.16 3.37 3.9
1 74.66 17.56 17.35 1.6 2.64 3.96
1 77.51 16.79 16.44 1.84 2.91 4
1 77.32 17.76 16.04 1.6 3.4 3.62
1 77.3 18.14 15.34 1.94 4.38 3.73

Y is
1.27
(16*1)
1.39
3.2
3.12
3.19
3.28
4.41
5.06
4.17
3.27
3.41
2.52
1.08
1.16
2.24
3.06 Mean(Y): 2.864375
X transpose (7*16) =transpose(array) Ctrl+shift+enter

1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
70.5 75.3 74.4 73.5 74.7 71.7 72.3 73.1 76.3 75.3 73.0 74.6 77.5 77.3
75.4 77.3
8 9 3 1 8 3 4 4 1 7 3 6 1 2
22.0 21.9 21.2 20.0 19.3 18.2 18.9 17.3 17.7 18.7 18.3 17.5 16.7 17.7 18.1
23.6
3 3 7 4 4 7 1 3 2 6 2 6 9 6 4
18.2 18.5 16.7 15.1 15.4 16.9 17.9 16.0 15.9 15.5 14.2 16.0 17.3 16.4 16.0 15.3
9 6 7 6 3 2 2 6 3 2 3 3 5 4 4 4
1.48 1.7 1.81 1.77 1.96 2.62 3.39 2.97 2.12 1.88 2.6 2.16 1.6 1.84 1.6 1.94
3.61 4.38 5.88 7.8 6.92 6.11 5.31 5.18 5.63 4.11 4.49 3.37 2.64 2.91 3.4 4.38
4.3 4.71 4.63 4.48 4.1 4 3.75 3.64 3.85 3.9 3.91 3.9 3.96 4 3.62 3.73

X Transpose * X
(7*16)*(16*7)= (7*7)
16 1192.8 307.77 261.99 33.44 76.12 64.48
88986.4 22917.
1192.8 19515.86 2487.3966 5662.9271 4804.8274
9 1
5979.2
307.77 22917.1 5051.255 639.2941 1483.4898 1248.7249
6
19515.8 5051.2
261.99 4311.4599 546.8335 1239.4346 1057.9917
6 6
2487.39 639.29
33.44 546.8335 74.232 162.3257 133.5284
7 4
5662.92 1483.4
76.12 1239.4346 162.3257 394.166 309.0441
7 9
4804.82 1248.7
64.48 1057.9917 133.5284 309.0441 261.533
7 2

minverse(array)

Inverse(X Transpose * X) (7*7)

306.086332 -3.045911241 -1.880342644 -2.49935 -5.47121 -0.33031 2.766747
-3.0459112 0.032857163 0.023196081 0.021459 0.041938 0.004052 -0.07645
-1.8803426 0.023196081 0.087951264 0.000452 0.019016 -0.0198 -0.37063
-2.4993487 0.021458722 0.000451697 0.095178 -0.04736 0.047199 -0.19681
-5.4712146 0.041938295 0.019015793 -0.04736 0.515458 -0.09607 0.529563
-0.3303128 0.004052087 -0.019801711 0.047199 -0.09607 0.073709 -0.12745
2.76674731 -0.076447488 -0.370634077 -0.19681 0.529563 -0.12745 3.172195

Inverse(X Transpose * X) *X Transpose

(7*7)*(7*16)=(7*16)

6.57 - - 4.43 0.40 3.66 - - - 3.46 0.78 6.9 3.6 - - -

 5.9 3.32 0.27 1.55 2.6 2.7 3.5
6066 8.8 0920 0391 2507 1906 4761 903 264
46 3152 8003 4776 237 693 529
78 852 629 126 612 278 271 34 32
1 21 244 6 5 5 9
- - - - - -
0.0 0.0 0.00 0.01 0.00 - 0.0 0.0 0.0
0.07 0.04 0.04 0.02 0.02 0.00 0.0
671 52 3909 6317 7804 0.0 262 549 578
5390 8534 8526 2644 7054 0627 713
3 72 75 951 318 381 59 03 6
5 008 525 362 013 2
- - - - - - - -
0.06 0.0 0.1 0.00 0.09 0.03 0.1 0.0
0.07 0.06 0.01 0.07 0.05 0.0 0.0 0.1
5597 054 31 4416 1427 5691 038 827
8863 1570 7935 6970 6959 389 858 034
29 73 32 687 359 459 42 73
694 997 49 1 664 6 2 6
- - - - - -
0.01 0.0 0.00 0.03 0.11 0.06 - 0.0 0.0
0.0 0.06 0.00 0.05 0.19 0.1 0.0
9986 940 9862 6105 8867 9797 0.0 667 083
14 6787 5303 8793 9877 090 315
03 88 288 872 899 53 008 74 96
79 097 724 319 292 3
- - - - - - - - -
0.1 0.1 0.28 0.15 0.35 0.0 0.1
0.26 0.16 0.16 0.01 0.10 0.06 0.1 0.2 0.0
782 24 4495 6046 1933 356 072
5284 6357 8026 4750 5059 7595 596 353 564
98 79 359 835 088 88 63
5 404 89 24 8 596 7 4 4
- - - - - - - - -
0.09 0.10 0.06 0.03 0.10 0.0 0.0
0.04 0.0 0.0 0.01 0.01 0.14 0.0 0.0 0.0
5837 3395 4857 4635 8261 239 088
1394 238 35 7799 7853 1890 967 204 448
258 033 085 269 91 29 36
1 1 06 422 23 493 1 1 2
- - - - - - - -
0.5 0.43 0.08 0.22 0.45 0.3 0.2 0.6
0.42 0.0 0.25 0.04 0.53 0.00 0.9 0.5
056 4769 2645 3837 9132 248 089 751
9502 74 2241 7546 4635 7105 861 835
81 309 347 845 667 53 5 32
1 29 98 08 985 9 1 6

b=Inverse(X Transpose * X) *X Transpose*Y

(7*16)(16*1)=(7*1)

-
3.047200
3
0.093046
51
0.250242
49
-
0.101456
5
1.134079
22
0.378497
93
 -
2.071527
2
The Model is: y Predicted = -3.0472 + 0.0930465 * x2 +0.2502425 * x3 -
0.101457 * X4+ 1.1340792 * x5 + 0.3784979 * x6 - 2.071527 * x7