How to describe and summarize relationships between variables: (1) Visualize, describe, and quantify the relationships between

variables or (2) Create mathematical models to

describe the relationships between variables.
Correlation Analysis Regression models
To quantify the relationship between variables, we can use the sample To measure the relationship between variables, we can use the linear model
correlation coefficient (also called the Pearson correlation coefficient). 𝒚 = 𝜷𝟎 + 𝜷𝟏 × 𝒙 + 𝝐
and the regression model without the error 𝜖 term represents the expected value of Y for a given
Test the hypothesis 𝐻0 : 𝜌 = 0 x value and is called the simple regression equation: 𝐸(𝑌|𝑥) = 𝛽0 + 𝛽1 × 𝑥
To assess this regression model, we are customary to use the ordinary least squares method
(OLS). Assumptions of errors for this regression model, although not observable:
• Errors are normally distributed.
• Errors have constant variance.
• Errors are independent of each other.
𝑆𝑆𝑅 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑠𝑙𝑜𝑝𝑒−0
Note that in a bivariate, 𝑅2 ≡ 𝑟 2 = Slope: 𝑡𝑐𝑎𝑙𝑐 = and 𝐻0 : 𝜇1 = 0
𝑆𝑆𝑇 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑒𝑟𝑟𝑜𝑟 𝑜𝑓 𝑠𝑙𝑜𝑝𝑒
𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒𝑑 𝑖𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡−0
Intercept: 𝑡𝑐𝑎𝑙𝑐 = and 𝐻0 : 𝜇0 = 0
𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑒𝑟𝑟𝑜𝑟 𝑜𝑓 𝑖𝑛𝑡𝑒𝑟𝑐𝑒𝑝𝑡

Critical Remember that the critical value

𝑛−2
𝑡𝑐𝑟𝑖𝑡 = 𝑟√
values 1 − 𝑟2 • for two-tailed test: 𝑡𝛼/2 ≡ 𝑇. 𝐼𝑁𝑉. 2𝑇(𝛼, 𝑑𝑓)
• For left-tailed test: 𝑡𝛼 ≡ 𝑇. 𝐼𝑁𝑉(𝛼, 𝑑𝑓)
An equivalent approach is to calculate a critical value for the correlation
• For right-tailed test: 𝑡𝛼 ≡ −𝑇. 𝐼𝑁𝑉(𝛼, 𝑑𝑓)
𝑡𝑐𝑟𝑖𝑡
coefficient 𝑟𝑐𝑟𝑖𝑡 = with 𝑑𝑓 = 𝑛 − 𝑘 − 1, where 𝑘 is the number of independent variables.
2 +𝑛−2
√𝑡𝑐𝑟𝑖𝑡

Confidence 𝑠𝑒 𝑆𝑆𝐸
For slope: 𝑏1 ± 𝑡𝛼 × 𝑠𝑏1 where 𝑠𝑏1 = and 𝑠𝑒 = √𝑀𝑆𝐸 = √
𝑛−𝑘−1
interval 2 √∑𝑛
1 (𝑥𝑖 −𝑥̅ )
2

1 ̅̅̅̅
𝑥2
For intercept: 𝑏0 ± 𝑡𝛼 × 𝑠𝑏0 where 𝑠𝑏0 = 𝑠𝑒 √ + ∑𝑛(𝑥 2
2 𝑛 1 𝑖 −𝑥̅ )

1 (𝑥 −𝑥̅ )2
For mean 𝑦: 𝑦̅𝑖 ± 𝑡𝛼 × 𝑠𝑒 √ + ∑𝑛(𝑥𝑖 2
2 𝑛 1 𝑖 −𝑥̅ )

1 (𝑥 −𝑥̅ )2
For individual 𝑦: 𝑦̅𝑖 ± 𝑡𝛼 × 𝑠𝑒 √1 + + ∑𝑛(𝑥𝑖 2
2 𝑛 1 𝑖 −𝑥̅ )

Chapter 12. Regression models Nguyen Thi Thu Van [This version is dated on 21 Aug, 2021]