Trivariate Regression By Hand

It is not totally unreasonable to conduct a multiple regression analysis by hand, as long as you
have only two predictor variables. Consider the analysis we did in PSYC 6430 (with the program
CorrRegr.sas) predicting attitudes towards animals (AR) from idealism and misanthropy. SAS gave
us the following univariate and bivariate statistics:
Variable N Mean Std Dev Sum Minimum Maximum

ar 154 2.37969 0.53501 366.47169 1.21429 4.21429

ideal 154 3.65024 0.53278 562.13651 2.30000 5.00000
misanth 154 2.32078 0.67346 357.40000 1.00000 4.00000

Pearson Correlation Coefficients, N = 154

Prob > |r| under H0: Rho=0

ar ideal misanth

ar 1.00000 0.05312 0.22064

0.5129 0.0060

ideal 0.05312 1.00000 -0.13975

0.5129 0.0839

Let us first obtain the beta weights for a standardized regression equation, Zˆ y  1Z1   2Z2

ry 1  ry 2 r12 .05312  .22064(0.13975)

1    .0856
1 r 2
12 1  .139752

Look at the numerator of the ratio above. In computing the partial standardized slope for
predicting Y from X1, we remove from the standardized slope for predicting Y from X1 ignoring X2 (ry1)
the overlap among Y, X1, and X2 (-ry2r12).

r y 2  r y 1r12 .22064  .05312( 0.13975)

2    .2326
1 r 2
12 1  .13975 2

Now for the unstandardized equation, Yˆ  a  b1 X 1  b2 X 2

 i sy
bi  a  Y   bX i
si

.0856(.535) .2326(.535)
b1   .086 b2   .185
.53278 .67346

a  2.38  .086(3.65)  .185(2.32)  1.637

The Multiple R2


Copyright 2014, Karl L. Wuensch - All rights reserved.
MultReg-ByHand.docx
 2
r r
2 2
 2r y 1r y 2 r12
.05312  .22064  2(.05312)(.22064)( 0.13975)
2 2
R y212    .0559
y1 y2

1 r 2
12 1  .13975 2
Look at the numerator above. We sum the portion of variance in Y that is explained by X 1
ignoring X2 (ry1) and the portion of variance in Y that is explained by X2 ignoring X1 (ry2) and then we
remove from that the portion of variance that has been counted twice.

Ry212...i ...p    i ryi  ry2yˆ for p  2 (p is number of predictors)

Ry212  .0856(.05312)  .2326(.22064)  .0559

Semipartials

ry 1  ry 2r12 r y 2  r y 1r12
sr1  sr2  (but easier to do by method below)
1  r122 1  r122

sri 2  Ry212...i ...p  Ry212...(i )...p for p  2

sr12  .0559  .22064 2  .0072 , so sr1 = .085

sr22  .0559  .053122  .0531 , so sr2 = .230

Partials

sr1 sr2
pr1  pr2 
1 r 2
y2 1  r y21

sri 2
pri 2  for p  2
1  R y212...(i )...p

Tests of significance of partials

Testing the null that in the population I = 0 is absolutely equivalent to testing the null that the
semipartial or partial correlation coefficient is 0.

1  R y212 1
SE 1 
N  p  1 1  r122


H0: Pop.  = 0  t  df = N - p - 1 (but easier to get with next formula)
SE 

N  p 1
H0: Pop. sr = 0  t  sr df = N - p - 1
1 R 2
 3

154  2  1 154  2  1
t1  .085  1.07 t2  .230  2.91 df = 151
1  .0559 1  .0559

Shrunked (Adjusted) R2
(1  R 2 )(N  1) (1  .0559)(153)
shrunken R  1 
2
 1  .043
N  p 1 151
Please re-read this document about shrunken R2.

ANOVA summary table:

Source SS df MS F p
Regression R2SSY p
Error (1-R2)SSY n-p-1
Total SSY n-1

F tests the null that population R2 = 0

For our example, SSy = (N-1)s2 = 153(.53501)2 = 43.794

SSregr = .0559(43.794) = 2.447
SSerror = 43.794 - 2.447 = 41.347
F(2, 151) = 4.47, p = .013

standard error of estimate  MSE  0.52

Testing the Significance of the R2

It is possible to get the value of F without computing the Mean Squares. Just use this formula:
R n  p  1
2
.0559153  2  1
F . For our data, F   4.47 . Do note that even small rounding
p 1 R 2
 21  .0559
errors in the computation of R2 can have noticeable effects on the value of F here, so obtain the R2
with several decimal point precision.

For a hierarchical analysis,

Ry212...p  ry21  sry221  sry2312    sryp2 12...(p1)

Return to Wuensch’s Stats Lessons Page

Copyright 2014, Karl L. Wuensch - All rights reserved.