Artificial Intelligence
Linear Regression
Pham Viet Cuong
Dept. Control Engineering & Automation, FEEE
Ho Chi Minh City University of Technology
Linear Regression
What is regression analysis?
ü A basic and commonly used predictive analysis
(predictive analysis: the use of data, statistical algorithms and machine
learning techniques to identify the likelihood of future outcomes based
on historical data)
Examples: Sales manager trying to predict next month’s numbers
ü Monthly sales numbers for, e.g., the past three years
ü Weather
ü Competitor’s promotion
ü Rumor of a new and improved model
ü ...
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 2
Linear Regression
Regression analysis: examine relationship between one dependent
variable and one or more independent variables.
ü Can a set of predictor variables predict an outcome (dependent) variable?
ü Which variables matter most?
ü Which can we ignore?
ü How do those factors interact with each other?
ü How certain are we about all of these factors?
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 3
Linear Regression
Simple linear regression Least-square linear regression problem
N
f x w0 w1 x min
w0 ,w1 y w w x
i 1
i 0 1 i
2
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 4
Linear Regression
Simple linear regression Least-square linear regression problem
f x w0 w1 x 1 N
min
w0 ,w1
2 y w w x
i 1
i 0 1 i
2
w *
1
w yw x
*
0
*
1
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 5
Linear Regression
Multiple Linear Regression Loss function
f x w0 w1 x1 w2 x 2 w p x p
w w0 , w1 , w2 , , w p
T
x 1, x , x , , x
1 2 p
f x xw
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 6
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 7
Linear Regression
Example: Simple Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 8
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 9
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 10
Linear Regression
Max error: 9.12g (1.97%)
Mean error: 1.77g (0.83%)
Standard deviation: 2.21g
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 11
Linear Regression
Sources:
1. Yu-Teng Liang and Yih-Chih Chiou. “Machine Vision-Based Automatic Raw
Fish Handling and Weighing System of Taiwan Tilapia.” B.-C. Chien et al.
(Eds.): IEA/AIE 2009, LNAI 5579, pp. 711–720, 2009
2. http://www.mit.edu/~6.s085/notes/lecture3.pdf
3. https://www.statisticssolutions.com/what-is-linear-regression/
4. https://hbr.org/2015/11/a-refresher-on-regression-analysis
5. https://www.surveygizmo.com/resources/blog/regression-analysis/
6. https://machinelearningcoban.com/2016/12/28/linearregression/
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 12
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 13
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 14
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 15
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 16
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 17
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 18
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 19
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 20
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 21
Linear Regression
Pham Viet Cuong - Dept. Control Eng. & Automation, FEEE, HCMUT 22
