Exercise 4: Implementing Multiple Linear Regression
Model using Neural Networks
John Vincent M. Cabugnason
College of Computer Studies and Engineering
Jose Rizal University
Cainta Rizal
johnvincent.cabugnason@my.jru.edu
Abstract
In this paper, we explore the combination of multiple
linear regression and neural networks to create a more
accurate and robust model for predicting the value of a
dependent variable based on multiple independent variables.
We discuss the limitations of multiple linear regression and
how neural networks can help capture non-linear
relationships between variables. We review related
literature, including studies that use these methods to predict
cancer patient survival, student academic performance, and
energy consumption in residential buildings. By combining
these two approaches, we can create a more powerful and
accurate model that can better represent the complexities of
real-world data. This has important implications for fields
like finance, healthcare, and marketing, where accurate
predictions are crucial for decision-making and outcomes.
I. INTRODUCTION
Multiple linear regression is a statistical method
that can be used to predict the value of a dependent variable
based on the values of multiple independent variables.
Neural Network is the Machine Learning that can used to
learn complex relationships between data. By combining
multiple linear regression and neural networks, it is possible
to create a model that is more accurate and robust than
either method alone.
Formula and Calculation of Multiple Linear Regression
Yi=β0+β1xi1+β2xi2+...+βpxip+ϵ
where, for i = n observations:
yi=dependent variable
xi=explanatory variables
β0=y-intercept (constant term)
βp=slope coefficients for each explanatory variable
ϵ=the odel’s error term (also known as the residuals)
In [1], Machine learning has become an essential
tool for extracting regularities in the data and for making
inferences. Neural networks, in particular, give the
flexibility and scalability required to transform complex
datasets into well-structured models with broad
applicability[1]. Understanding deep learning requires
knowledge of the expressive capabilities of neural networks.
In [2], Deep learning allows computational models that are
composed of multiple processing layers to learn
representations of data with multiple levels of abstraction
[2].
In [3], Multiple linear regression (MLR) is a
statistical technique used to model the relationship between
a dependent variable and multiple independent variables.
The problem it aims to solve is how to predict or explain the
variation in the dependent variable based on the values of
the independent variables. According to home.csulb.edu,
many difficulties tend to arise when there are more than five
independent variables in a multiple regression equation. The
issue of two or more independent variables having a high
degree of correlation with one another is one of the most
prevalent. Multicollinearity is the term for this [3].
The motivation for enhancing MLR with neural
networks is to improve the accuracy and predictive power of
the model. MLR has limits when it comes to comprehending
complex data or capturing non-linear relationships, while
being an effective technique for modeling relationships
between variables. By combining MLR with neural
networks, we can create a more powerful model that can
better capture the nuances and complexities of real-world
data.
II. REVIEW OF RELATED LITERATURE
A. Multiple Linear Regression and Neural Networks
in Regression Tasks
Comparison of Artificial Neural Networks and Logistic
Regression for 30-days Survival Prediction of Cancer
Patients
According to Dogu et al. (2020), A machine
learning technique that imitates neural system and brain can
provide better than traditional methods like logistic
regression for survival prediction and create an algorithm by
determining influential factors [4]. In recent years, the
increasing number of cancer patients and long-term
oncological treatments have increased emergency admission
rates in our everyday practice, due to both pharmacological
side effects and clinical symptoms caused by cancer itself.
In [4], Accurate 30-days survival predictions,
directing patients to palliative care at the right time will
preventboth patients and caregivers from suffering and
ensure correct treatments. Survival analyzes should be
performed as accurately as possible. Carefully studying the
impact of patient data, including clinical, laboratory, KPS,
and PPS outcomes, on survival estimation is necessary.
According to several studies, patients with
resectable lung cancer who have high CRP levels had a
worse prognosis. Besides, the CRP level was associated
with reduced serum albumin, resulting in progressive weight
loss, poor performance, the higher mortality rate in cancer
patients
Predicting Student Academic Performance: A
Comparison of Two Meta-Heuristic Algorithms Inspired
by Cuckoo Birds for Training Neural Networks
According to Chen et al. (2014), Predicting student
academic performance has long been an important research
topic. Among the issues of higher education institutions,
questions concerning admissions remain important [5]. The
primary goal of the admissions process is to identify
applicants who will be successful if they are admitted to the
university. High accuracy academic performance predictions
help with admissions decisions and improve educational
services at educational institutions.
In [5], Different approaches have been applied to
predicting student academic performance, including
traditional mathematical models and modern data mining
techniques [5]. These methods described the quantitative
relationships between the inputs and outputs using a series
of mathematical formulas. As a result, it becomes necessary
to put out a model that forecasts student success based on
the outcomes of standardized tests, such as those used for
college admissions and high school diploma examinations,
as well as other important variables. [5] The prediction is
accurate if the error between the predicted and actual values
is within a small range [5].
Figure 1. Pseudocode of the Cuckoo Search (CS).
In [5], Cuckoo search is an optimization algorithm
introduced by Yang and Deb. This algorithm was inspired
by the special lifestyle of the cuckoo species. The aim of the
CS is to maximize the survival rate of the eggs. Each egg in
a nest represents a solution, and a cuckoo egg stands for a
new solution. In this case, the host bird can either throw the
egg away, or abandon the nest to build a new one in a new
place.
Based on the above-mentioned rules, the steps of the CS can
be described as the pseudocode in Figure 1. The algorithm
can be extended when each nest has multiple eggs
representing a set of solutions.
Comparison of multi linear regression and artificial
neural network to predict the energy consumption of
residential buildings
According to Nainwal and Sharman 2023, The
building sector along with its sub-sectors is the world's
leading consumer of all forms of energy and consumes
about 30% of the world's final energy consumption. With
the increase in population, industrialization and urbanization
trend, the energy sector has grown rapidly in the past
decades to cater to the increase in energy demand. To cater
this, many regulatory efforts in the form of energy
efficiency and conservation codes offering guidelines and
measures during and post design have been proposed in
different countries of the world.
Regression is a tool that is frequently used to
explain how dependent factors transform into independent
variables that are inputs into the instrument. According to
Zhao et. al., Regression models gained popularity in the
field of research as a means of addressing the three
problems. Firstly, for predicting the energy consumption
based on climatic parameters. Secondly, for predicting
energy indexes. Thirdly,
for determining the important attributes of energy
consumption, which helps in understanding the thermal
behavior of the building.
In [6], In the recent years, Artificial Neural
Network (ANN) is the most widely used method for
predicting the building‟s energy consumption as it is very
good at dealing with the non-linear problems.
Figure 2. Energy consumption profile by different
categories of appliances.
It is seen in Figure 2 that the maximum energy is
used by the space heating appliances and minimum energy
is used by the appliances for ventilation
III. METHODOLOGY
IV. RESULT
V. ANALYSIS
Hyper-parameters play a crucial role in determining the
performance of a model. You can observe and analyze the
impact of hyper-parameters on your model's performance by
changing them and observing how the model behaves.
Some of the hyper-parameters that you can experiment
with include the learning rate, batch size, number of epochs,
number of layers, number of hidden units, activation
functions, and regularization techniques. By varying these
hyper-parameters, you can determine which values work
best for your dataset and model architecture.
Building and training neural networks can be
computationally intensive, and if you have limited
resources, it may be challenging to experiment with
different hyper-parameters or large datasets.
To overcome these challenges, it's important to
carefully plan the experiments, allocate sufficient resources,
and monitor the model's performance. You can also consider
using automated tools or techniques such as hyper-
parameter tuning or transfer learning to streamline the
process and improve the results.
VI. CONCLUSION
 Neural networks are a type of tool that can help us
understand and analyze complex data. They are designed to
work with data that is not always straightforward or easy to
understand, and they can capture relationships between
different variables that are not always linear or
straightforward. In other words, they can help us make sense
of data that might be too complicated for other tools to
handle. By combining multiple linear regression with neural
networks, we can create a more powerful model that can
better capture the nuances and complexities of real-world
data.
While multiple linear regression is a powerful tool
for analyzing data, it has some limitations when it comes to
handling complex data or relationships that are not
straightforward. In other words, it may not be able to fully
capture the nuances of real-world data. This is where neural
networks come in - they are designed to handle more
complex data and can capture non-linear relationships
between variables. By combining these two approaches, we
can create a more powerful and accurate model that can
better represent the complexities of the real world.
Combining multiple linear regression with neural
networks is a powerful approach to analyzing complex data
and capturing non-linear relationships between variables.
This can have important implications for fields like finance,
healthcare, and marketing, where accurate predictions are
crucial for decision-making and outcomes. By using these
tools together, we can create a more advanced model that
helps us make more informed decisions based on a more
complete picture of the data.
VII. REFERENCES
1. Montavon, G. (2020). Introduction to
Neural Networks. In Machine Learning Meets
Quantum Physics (pp. 37–62). Springer
International Publishing.
2. LeCun, Y., Bengio, Y., & Hinton, G. (2015).
Deep learning. Nature, 521(7553), 436–444.
https://doi.org/10.1038/nature14539
3. (N.d.). Csulb.edu. Retrieved October 3,
2023, from
https://home.csulb.edu/~msaintg/ppa696/696r
egmx.htm#:~:text=Many%20difficulties
%20tend%20to%20arise,This%20is%20called
%20multicollinearity.
4. Arkin, F., Aras, G., & Dogu, E. (2020).
Comparison of artificial neural networks and
logistic regression for 30-days survival
prediction of cancer patients. Acta Informatica
Medica: AIM: Journal of the Society for
Medical Informatics of Bosnia &
Herzegovina: Casopis Drustva Za Medicinsku
Informatiku BiH, 28(2), 108.
https://doi.org/10.5455/aim.2020.28.108-113
5. Chen, J.-F., Hsieh, H.-N., & Do, Q. (2014).
Predicting student academic performance: A
comparison of two meta-heuristic algorithms
inspired by cuckoo birds for training neural
networks. Algorithms, 7(4), 538–553.
https://doi.org/10.3390/a7040538
6. Nainwal, R., & Sharma, A. (2022).
Comparison of multi linear regression and
artificial neural network to predict the energy
consumption of residential buildings. IOP
Conference Series. Earth and Environmental
Science, 1084(1), 012005.
https://doi.org/10.1088/1755-
1315/1084/1/012005