Modeling House Price Prediction using Linear
Regression Analysis
Abstract—House prices increase every year, so there is a need
for a system to predict house prices in the future. House price
prediction can help the developer determine the selling price of
a house and can help the customer to arrange the right time
to purchase a house. There are three factors that influence the
price of a house which include physical conditions, concept and
location. This research aims to predict house prices based on
Iowa City houses with regression analysis. The result obtained is
based on root mean square error.
Index Terms—regression, lasso, ridge, elastic net, prediction,
svm, random forest
I. I NTRODUCTION
Investment is a business activity that most people are
interested in this globalization era. There are several objects
that are often used for investment, for example, gold, stocks
and property. In particular, property investment has increased
significantly since 2011, both on demand and property selling.
One of the increasing of property demand is because of high
population in Africa.
There are several approaches that can be used to deter-
mine the price of the house, one of them is the prediction
analysis.The first approach is a quantitative prediction. A
quantitative approach is an approach that utilizes time-series
data. The time-series approach is to look for the relationship
between current prices and prevailing prices.
The second approach we use is classification as to com-
pare between which performs better between linear regres-
sion or classification techniques namely support vector ma-
chine(SVM) and random forest.In linear regression, determin-
ing coefficients generally using the least square method, but it
takes a long time to get the best formula.
RMSE is used for selection of affect variables in house
prediction, regression is used to determine the optimal coef-
ficient in prediction.Prediction house prices are expected to
help people who plan to buy a house so they can know the
price range in the future, then they can plan their finance
well. In addition, house price predictions are also beneficial
for property investors to know the trend of housing prices in a
certain location. This research is focused in Iowa City based
in Africa.
II. R ELATED W ORK
A. House Price Affecting Factors
There are several factors that affect house prices. In this
research we divided these factors into three main groups,
there are physical condition, concept and location. Physical
conditions are properties possessed by a house that can be
observed by human senses, including the size of the house, the
number of bedrooms, the availability of kitchen and garage,
the availability of the garden, the area of land and buildings,
and the age of the house.
B. Preprocessing
Before feeding data to the algorithms, the data is prepro-
cessed. In this model, initially we import the data set and then
the dataset is explored by finding possible correlation between
depending and independent variables. Since, linear regression
models are sensitive to non-linear data, outliers.
The outliers are removed from TotalBSMTSF and GrLi-
vArea. After this, the missing values are dealt with the help of
encoding. Non-linear data present in the dataset is transformed
to linear data by using log transformation. Some of the features
are integrated to form a new feature. In feature selection, we
select the least number of features which are highly correlated
with the dependent variable. In the final step of preprocessing,
standardisation of data is done. We use cross-validation so the
scaling has to be done independently for training and testing
data.
III. DATASET
The dataset for some of the previous works on housing
price predictions is not large. For example, Bahia only studies
506 samples to develop the model (Bahia , 2013).There is a
limitation in the paper by Pow saying the research does not
have enough historical transaction data. Pow points out using
enough data might increase the performance of predictions
(Pow, Janulewicz,and Liu, 2014).
This project uses a dataset from Kaggle open source
datasets. The dataset consists of 80 explanatory features and
1460 entries of housing sales in Iowa, Africa. It describes
different aspects of housing sales from 2006 to 2010 (Kaggle
Inc.).
The independent variables are date, price, bedrooms, bath-
rooms, sqft living, sqft lot, floors, waterfront, view, condition,
grade, sqft above,sqft basement, yr built, yr renovated, zip-
code, latitude, longitude, sqft living15, and sqft lot15. We can
see that these variables include categorical variables, numerical
variables, and time series variables. The dependent variable is
the sale price of houses from 2006 to 2010 in Iowa,Africa.
IV. R ESEARCH M ETHODOLOGY
A. Regression Analysis
By using scikit-learn, the regression models are imple-
mented. We aim to measure the performance of each model

Fig. 1. Diagram of flow research..
and compare it with other models. For validation purpose we
use cross-validation, different linear regression models used
are
• Ordinary least square
• Lasso
• Ridge
• Elastic Net
• Random Forest
1) Ordinary Least Squares: Least Square Error is a well
known mathematical measure of the performance of a linear
regression model. LSE works by changing the coefficients of
the model in a way that minimize the sum of squares between
the true values and the predicted values. It solves a problem
of the form (1):
2 selected features. Seems they are really good options once the
min ||Xw − y||2 (1)
w
and can be solved analytically by the equation (2)
β̂ = (X T X)−1 X T y
Where X is a matrix of the independents features, y is the
actual response and β̂ the estimated weights w.
2) Ridge: Ridge regression addresses some of the problems
of Ordinary Least Squares by imposing a penalty on the size
of coefficients. The new equation of a penalized residual sum
of square is (3)
2 2
min ||Xw − y||2 + α||w||2
w
Here, α ≥ 0 is called the regularization parameter (L2)
and controls the amount of shrinkage. Higher the values
of alpha, bigger is the penalty and therefore the magnitude
of coefficients are reduced. It is worth to note that we
used RidgeCV which does an implicit leave-one-out cross-
validation to choose the best alpha.
3) Lasso: The Mathematics behind lasso regression is quiet
similar to that of ridge. the only difference is instead of adding
squares of theta, we add the absolute value of w: (4)
1 when comparing to rbf or sigmoid. It is a sign that a linear fit
min ||Xw − y||22 + α||w||1 .
w 2nsamples
Lasso is a linear model that estimates sparse coefficients, i.e., it
reduces the number of variables upon which the given solution
is dependent. It does a kind of feature selection and thus lead
to a less complex final model. For instance, when there are
correlated features it will choose one and set the coefficient
of the other to zero. The regularization parameter alpha (L1)
controls the degree of sparsity of the coefficients estimated
and we employed once more the version of the algorithm that
automatically chooses the best value.
4) Elastic Net: Is a hybrid method that trains with L1 and
L2 as regularize rs. This combination allows for learning a
sparse model where few of the weights are non-zero like
Lasso, while still maintaining the regularization properties of
Ridge. It is useful when there are multiple features which are
correlated with one another. Lasso is likely to pick one of
these at random, while Elastic Net is likely to pick both. The
objective function to minimize is in this case (5)
1 α(1 − ρ)
min ||Xw − y||22 + αρ||w||1 + ||w||22
w 2nsamples 2
(5)
5) Random Forest: Random Forest is one of the most
versatile and precise classifier. It does not need the data being
scaled and can deal with any number of features. In order to
test the random forest algorithm a little further than the others,
we decided to train it using different train sets. First, with a
data set containing all the features and no scaling. Second, in
the reduced train set with scaled features.
As we can see in the results bellow, it did well in both
cases. This test was also interesting to validate our set of
performance using the the full set is not significantly better
than using the restricted set.
After some experimentation, the configuration we chose
(2) was: the number of trees is 500 as any greater value did
not enhance the accuracy, MSE as the function to measure
the quality of a split, max depth with None which means the
nodes are expanded until all leaves are pure or until all leaves
contain less than min samples split samples, max features =
auto to consider all the features when looking for the best split
and bootstrap = True for replacement as it gave better results
during the tests. All the others parameters were maintained in
their default values.
(3)
B. Classification
1) SVM: SVM is a large margin classifier. The rationale
behind having decision boundaries with large margins is that
they tend to have a lower generalization error. Whereas,
models with small margins are more prone to overfitting. We
can control the width of the margin using the regularization
parameter C. Lower values of C give smaller margins and vice
versa.
We have used the linear kernel as it gave far the best result
(4) is well adjusted to the true data and also can explain that no
regularization was necessary. The best value for C was 1. We
kept the other parameters in their default values.
doing more feature engineering, as we did when we create the
TotalSF, or doing more useful transformations such as Box-
Cox or log transformation to other variables to reduce their
variability. There is no set model for house price prediction
depending on the dataset the model accuracy may vary for
this given dataset of Iowa, the minimum RMSE is recorded
for elastic net and maximum RMSE is recorded for random
forest regression.

Fig. 2. Comparsion between RMSE

C. Testing Methods
The model developed in this research will be tested using
Root Mean Square Error (RMSE). RMSE is used to calculate
predicted performance by considering the prediction error of
each data. RMSE formula can be seen here as (6)
r n
1X 2
RM SE = (di − pi ) (6)
n i=1
V. EXPERIMENT AND RESULT
The experimental process examines the parameters used on
particle swarm optimization such as particle test, iteration test,
and also inertia weight combination test.
Result generated from the models are judged over the
training data which takes 80% of the total dataset, we compare
all the models on the basis of RMSE. We take RMSE since
root mean square value helps in normalising the dataset.The
minimum RMSE value recorded for the model is given by
elastic net where the value is
Average RMSE: 0.12379041525319807
All other methods give fairly similar performance. Hence,
it can be seen that regularisation is not a big issue for this
dataset.
When classification model SVM is introduced then the value
of RMSE increased which proves that classification cannot be
used for house price prediction.
VI. C ONCLUSION
Data preprocessing have been proven to be a crucial part of
our work, for instance adressing the non-linearity problem with
log transformation improved the performance dramatically.
Moreover, removing the outliers also yield better results.
Encoding the features according to their type: nominal, or-
dinal, and numerical is also critical to our work. One way of
improving our results is creating an ordinal version of the
location, because, as we know, location is quite important
factor in most housing prices. We can also improve our model