136 views

Uploaded by derafik

To Download this page visit : http://downloads.ziddu.com/downloadfile/11543923/MultipleRegression_StatisticalMethodforPsychology.pdf.html

To Download this page visit : http://downloads.ziddu.com/downloadfile/11543923/MultipleRegression_StatisticalMethodforPsychology.pdf.html

Attribution Non-Commercial (BY-NC)

- regression analysis
- learner autonomy and vocabulary learning strategies
- SDM_PPC_Class3_22Jun10
- garcia-berthou
- “The Impact of Oil Price on Remittances: The Case of Morocco” by Farid Makhlouf and Kamal Kasmaoui
- Financial Leverage and Performance of Non-Financial Companies in Nairobi Securities Exchange in Kenya
- Stats
- Group 10 - Curve Fitting.docx
- Regression
- CHAPTER V
- Geographic Distribution of Environmental Factors Influencing Human Skin Coloration.
- MAS-III-Review-Question-Prelim.docx
- My Stata Command Details.pdf
- FORMULATION AND OPTIMIZATION OF LIPOSOMES INCORPORATING CYCLODEXTRIN– CLOPIDOGREL BASED GEL
- Application Project 4
- Measuring the Impact of Foreign Institutional Investments on S&P CNX Nifty - A Pragmatic Study
- Practice 10
- 2. Article Azojete Vol 10 13-23 Waziri
- Regression Model in Matri
- MR Report

You are on page 1of 33

z Howell: Chapter 15

z Last updated: 9/27/04

PY206: Statistical Methods for Psychology Multiple Regression

Multiple regression

z In simple regression we have a single predictor

Yˆ = b0 + b1 X 1

z In multiple regression, several explanatory variables are

employed simultaneously

Yˆ = b0 + b1 X 1 + b2 X 2 + L + bp X p

PY206: Statistical Methods for Psychology Multiple Regression

Example

z Course evaluation: 5-point scale

1. Overall quality of lectures (Overall)

2. Teaching skills of the instructor (Teach)

3. Quality of tests and exams (Exam)

4. Instructor’s knowledge of the subject matter (Knowledge)

5. Student’s expected grade (Grade; F=1, A=5)

6. Enrollment (Enroll)

to predict it based on the other five variables

PY206: Statistical Methods for Psychology Multiple Regression

Example

z Let’s consider the pattern of pairwise correlations

Overall 1 0.804 0.596 0.682 0.301 -0.24

Teach 1 0.72 0.526 0.469 -0.451

Exam 1 0.451 0.61 -0.558

Knowl 1 0.224 -0.128

Grade 1 -0.337

Enroll 1

with Overall

PY206: Statistical Methods for Psychology Multiple Regression

Example

z Some variables have relatively high correlations with

each other

Exam and Teach: r = 0.72

Exam and Grade: r = 0.61

Exam and Enrollment: r = –0.558

z Collinearity

When other variables are included as predictors, Exam will have

little to offer in the way of explaining variability in Overall

PY206: Statistical Methods for Psychology Multiple Regression

Example

z Fitted regression equation:

Yˆ = −1.195 + 0.763Teach + 0.132Exam + 0.498Knowledge

−0.184Grade + 0.0005Enroll

Predicted Y would be higher 0.763 for every Teach unit

Predicted Y would be higher 0.0005 for every Enroll unit

improve Overall by biOverall units

Changing one variable likely will affect other variables because

of the pattern of intercorrelations

PY206: Statistical Methods for Psychology Multiple Regression

Example

z Common interpretation mistake: the relative magnitudes

of the bi provide an index of the relative importance of

the individual predictors

Teach is more important than Enroll because b is larger

function of the standard deviations of the corresponding

variables

Because SD of Enroll is much larger than SD of other variables,

the regression coefficient is small regardless of the importance

of that variable

PY206: Statistical Methods for Psychology Multiple Regression

Example

z Intuitive reason:

For one instructor to have a Teach rating one point higher

is a major accomplishment (range: 2.2 to 4.5)

Having one additional student is a trivial matter

(range: 7 to 800)

PY206: Statistical Methods for Psychology Multiple Regression

z Question of relative importance has several answers…

z Suppose that before applying the multiple regression

equation, we had standardized each of our variables

mean = 0 and SD = 1

to a one-unit difference on other variables

z Solving the regression using standardized variables:

YˆZ = 0.662 Z1 + 0.106Z 2 + 0.325Z 3 − 0.105Z 4 + 0.124 Z 5

PY206: Statistical Methods for Psychology Multiple Regression

Example

necessarily the best indicator of importance, it has a

simple interpretation and provides a rough estimate of

the relative contributions of the variables

Commonly used

PY206: Statistical Methods for Psychology Multiple Regression

Residual variance

z Predictor variables will not predict Y perfectly

z Error: residual variance or residual error

∑ (Y − Yˆ ) 2

2

N − p −1

z Square root of MSresidual is sometimes called standard

error of estimate

PY206: Statistical Methods for Psychology Multiple Regression

Residual variance

z Remember from simple regression that

sY ⋅X =

2 ∑ i

(Y − Yˆ ) 2

=

SS residual

N −2 df

(where p is the number of regressors)

PY206: Statistical Methods for Psychology Multiple Regression

z Correlation coefficient in the context of multiple

regression

Ex.: between Overall and other predictors: 0.869

z What does this mean?

z R can be defined as the correlation between the

criterion (Y) and the best linear combination of the

predictors

R = rYYˆ

Yˆ = b0 + b1 X 1 + b2 X 2 + L + bp X p

PY206: Statistical Methods for Psychology Multiple Regression

SSR SSE

R =

2

= 1−

SSTO SSTO

accountable variation

z R2 = 0.755: we can say that 75.5% of the variation in

Overall quality of the lectures can be predicted on the

basis of the five predictors

11% more than on the basis of Teach alone (best

predictor)

Partitioning of deviations

z Sum of squares that is related to regression (SSR):

∑ (Yi − Y ) = SSYˆ

ˆ 2

∑ (Yi − Yi ) = SSresidual

ˆ 2

∑ i

(Y − Y ) 2

= SSY

PY206: Statistical Methods for Psychology Multiple Regression

z Adding more X variables to a regression model can only increase R2

and never reduce it, because SSE can never become larger with

more X variables and SSTO is always the same

z It is often suggested that R2 be adjusted to take into account the

number of X variables in the model: adjusted coefficient of

multiple determination (“Ajusted R Square”)

z Adjust R2 by dividing each sum of squares by degrees of freedom:

SSE

N − p −1

Ra2 = 1 − SSTO

N −1

PY206: Statistical Methods for Psychology Multiple Regression

z Does the set of variables taken together predict Y at

better-than-chance levels?

( N − p − 1) R 2

F=

p (1 − R 2 )

N – p – 1 degrees of freedom

PY206: Statistical Methods for Psychology Multiple Regression

F distribution

PY206: Statistical Methods for Psychology Multiple Regression

Example

F= = = 27.184

5(0.245) 1.225

PY206: Statistical Methods for Psychology Multiple Regression

z Is there a regression relation between the response

variable Y and the set of X variables?

H 0 : b1 = b2 = L = bp = 0

H a : not all bi equal zero

MSR

F=

MSE

z MSE: mean square error: SSE/df (df = N – p – 1)

Slide 20 Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology Multiple Regression

Partitioning of deviations

Yi − Y Yi − Yˆi Yˆi − Y

Y1 Y1

Yˆ1 Yˆ1

Y Y

Y Yˆ = a + bX Yˆ = a + bX

Yˆ2

Yˆ2

Y2 Y2

PY206: Statistical Methods for Psychology Multiple Regression

z If coefficients are not significantly different from zero,

predictor will serve no use

z To determine this we need to know the standard error of

the bi

Remember that the standard error refers to the variability of the

statistic over repeated sampling

z We can then form a t test on the regression coefficients

b j − b*j

t=

sb j

population and sbj is the standard error of bj

Slide 22 Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology Multiple Regression

z Typically we test H0: bj* = 0

bj

t=

sb j

N – p – 1 = 50 – 5 – 1 = 44 df

We can reject the null hypothesis

Drop Exam from the model?

A test of a predictor is done in the context of all other variables

in the equation

PY206: Statistical Methods for Psychology Multiple Regression

Example

z Note that Exam is actually significantly correlated with

Overall (r = 0.596)

z Nothing useful to contribute once other variables are

included

PY206: Statistical Methods for Psychology Multiple Regression

Sample sizes

z When there is no relation between the criterion and the predictors,

the expected value of R is not zero!

p/(N – 1)

z With 5 predictors and 51 cases: R = 0.1

z Large sample are needed (to push “random” R to 0)

z Power of a statistical test: 1 – β: probability of rejecting the null

hypothesis when it is in fact false and should be rejected

z Example: With 5 predictors, a population correlation of 0.3 would

require N = 187 for 0.8 power

z Reasonable amount of power requires fairly large samples!

PY206: Statistical Methods for Psychology Multiple Regression

Partial correlation

z Coefficients in multiple regression are called partial

regression coefficients

z Example: b1 is coefficient for regression of Y on X1

when we partial out the effect of X2, …, Xp: b01.23…p

When other variables are held constant

z Common mistake: equate b1 in the context of the

other Xi with the simple regression coefficient when

ignoring Xi:

b01.2 ≠ b01

PY206: Statistical Methods for Psychology Multiple Regression

Partial correlation

z Example (somewhat exaggerated)

Y 2 1 4 3 6 5

X1 1 2 3 4 5 6

X2 2 2 4 4 6 6

PY206: Statistical Methods for Psychology Multiple Regression

Example

z For any fixed value of X2, slope of the regression line of

Y on X1 is negative (in fact b01.2 = –1 )

z However, regression of Y on X1 when ignoring X2 is

positive (b01 = 0.829)

are very different!

regression coefficients will be equal in both cases

Slide 28 Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology Multiple Regression

Interpretation

z Suppose we regress Y on X2 and obtain the residual

values Yr = Yi − Yˆi

z Residual values represent part of Y that cannot be

predicted by X2: independent of X2

z Now regress X1 on X2 generating X 1r = X 1i − Xˆ 1i

z Again, residual values represent part of X1 that is

independent of X2

z We now have two sets of residuals: part of Y and part of

X1 that are independent of X2

Partialled X2 out of Y and out of X1

PY206: Statistical Methods for Psychology Multiple Regression

Interpretation

z Now regress Yr on X1r: regression coefficient will be the

partial coefficient b01.2

z Correlation between Yr and X1r is the partial correlation

of Y and X1, with X2 partialled out: r01.2

covYr X1r

bYr X1r = 2

= −1

s X1 r

= b01.2

PY206: Statistical Methods for Psychology Multiple Regression

Example

z Suppose an investigator established a significant

correlation between earned income and success in college

z Should students study more??

z Both variables are probably linked to IQ

People with high IQ likely do well in college and earn

more

Relationship between income and college success

potentially spurious

z How can this issue be addressed?

Partial out effect of IQ

PY206: Statistical Methods for Psychology Multiple Regression

Example

z Compute partial correlation between Income and

Success with IQ partialled out

1. Regress Income on IQ and obtain residuals

2. Regress Success on IQ and obtain residuals

Success that cannot be attributed to IQ

z Question: Can variation in Income-independent-of-IQ

be predicted by variation in Success-independent-of-IQ?

z Correlation between the two variables is the partial

correlation of Income and Success

Partialling out effect of IQ

Slide 32 Luiz Pessoa, Brown University © 2004

PY206: Statistical Methods for Psychology Multiple Regression

Another example

z Example: compute the

partial correlation

between Y and X1: r01.2

cov X1rYr

r= = 0.738

s X1r sYr

- regression analysisUploaded byayu7kaji
- learner autonomy and vocabulary learning strategiesUploaded byEbrahim Azimi
- SDM_PPC_Class3_22Jun10Uploaded byPriyank Shah
- garcia-berthouUploaded byBill Jungers
- “The Impact of Oil Price on Remittances: The Case of Morocco” by Farid Makhlouf and Kamal KasmaouiUploaded byThe International Research Center for Energy and Economic Development (ICEED)
- Financial Leverage and Performance of Non-Financial Companies in Nairobi Securities Exchange in KenyaUploaded byIOSRjournal
- StatsUploaded byLaura Bennett
- Group 10 - Curve Fitting.docxUploaded bykaye
- RegressionUploaded byLaw Jonathan
- CHAPTER VUploaded byEgemen Ertugan
- Geographic Distribution of Environmental Factors Influencing Human Skin Coloration.Uploaded bySimon Henriquez
- MAS-III-Review-Question-Prelim.docxUploaded byJana Lingcay
- My Stata Command Details.pdfUploaded bytaposh
- FORMULATION AND OPTIMIZATION OF LIPOSOMES INCORPORATING CYCLODEXTRIN– CLOPIDOGREL BASED GELUploaded byRAPPORTS DE PHARMACIE
- Application Project 4Uploaded byshags
- Measuring the Impact of Foreign Institutional Investments on S&P CNX Nifty - A Pragmatic StudyUploaded bychitkarashelly
- Practice 10Uploaded byorivalls
- 2. Article Azojete Vol 10 13-23 WaziriUploaded byOyeniyi Samuel Kehinde
- Regression Model in MatriUploaded byRobin Kumar Biswas
- MR ReportUploaded bySubhash Arora
- Hackett; Carrigan 1998 Performance Indicators-Information in Search of a Valid and Reliable Use.pdfUploaded bycastorypolux20762
- 10.1.1.17.5522Uploaded byOfelia Araujo
- A Case Study on the Process Productivity Model for Software PROJECTSUploaded byjgonzalezsanz8914
- 856201203 b 01Uploaded byPeter Reinir Daada
- Evaluating Regression Models Performance.docxUploaded byvinod reddy medapati
- Milan B. Arambašić, Radmila Mandić: Computerized calculation of microbiological potency of antibiotics: Interactive computer program "Potency v. 2" (Test "5 +3" doses): Calculation of microbiological potency of Neomicin and Bacitracin antibiotics in Enbecin® ointment. Boll. Chim. Farm., 145 (1):4, 2005, CD-ROM.Uploaded byMilan B. Arambasic
- ANOVARegressionPCA-InteractiveExamplUploaded byjamalnordin
- monosit benarUploaded bytiya syahrani
- Assignment 01 Nipun Goyal Jinye LuUploaded byNipun Goyal
- ECON 1203_Tut Q Summer 2014-15 Wk 1 (1)Uploaded byJoyce Kim

- Unit-3Uploaded byRajesh
- u06d1 Multiple RegressionUploaded byarhodes777
- Anna University Me-cem 2013 RegulationUploaded byNishad Vadakkedath
- Leadership StratplexUploaded byChetan Gupta
- Accounting for Inter Generational Income PersistenceUploaded bygkbea3103
- SPSS INSTRUCTION – CHAPTER 8Uploaded byEssam Rashid Mohajir
- Low-Order Conditional Independence GraphsUploaded byaajoshi1
- Stationary Time SeriesUploaded byHendry Lukito
- Suicidal Ideation BullyingUploaded byShamini Govinraj
- Determinants of Job Satisfaction Among Industrial WorkersUploaded byNarendra Patankar
- Solving Stepwise Regression ProblemsUploaded byNeetu Gupta
- weatherUploaded bysupvalhere
- gUploaded bychaliljose_386706745
- CorrelationUploaded byKathiresh Nataraj
- Regression Models TMSUploaded byT.M.Sankaran
- en_Tanagra_KMO_Bartlett.pdfUploaded bythe
- JHONSON_DINARDO.pdfUploaded byeeretamalvs
- A Contingency Theory of Socialisation FeldmanUploaded byjmat210
- Multiple RegressionUploaded bySunaina Kuncolienkar
- Neocortex Size as a Constraint on Group Size in Primates (Dunbar)Uploaded byDavid He
- 22. MSMUploaded bysrinilsukumarkhd
- Semi and Partial 2Uploaded byAsmita Gupta
- An Observational Study on the Productivity of Formwork InUploaded byJasmine Tso
- multple.docUploaded byHItesh Khurana
- malhotra17_tif.docUploaded byIsabella Mehnaz
- Pastor StambaughUploaded bymirando93
- Social Media and Depression SymptomsUploaded bymonicamonicc
- Malhotra Mr05 Ppt 17Uploaded byABHISHEK CHAKRABORTY
- Multiple Linear Regression II & Analysis of Variance IUploaded byarenguna
- Manufacturing Overhead Cost Driver Analysis 1990 Journal of Accounting and EconomicsUploaded bylukgv,h