House Price Prediction Using Machine Learning Techniques
Ashray Kakadiya, Khushal Shingala, Shivraj Sharma
California State University, Sacramento
Abstract
Using “Ames Housing dataset” we are
predicting the sales price of homes in Ames, Iowa
taking various machine learning Approaches like
data cleaning, data visualization, feature engineering
and advanced regression techniques like random
forests and gradient boosting. We Used Python as
Scripting language to use various machine learning
libraries like pandas, Numpy, Scikit learn etc. to
implement our predictor model.
1. Introduction
Ask a home buyer to describe their
dream house, and they probably won't begin with the
height of the basement ceiling or the proximity to an
east-west railroad. But this predictor’s dataset
proves that much more influences price negotiations
than the number of bedrooms or a white-picket
fence.
'The most difficult thing in life is to know
yourself'. This quote belongs to Thales of Miletus.
Thales was a Greek philosopher, mathematician and
astronomer. I wouldn't say that knowing your data is
the most difficult thing in data science, but it is
time-consuming. Therefore, it's easy to overlook this
initial step and jump too soon into the water. We
spend most time In understanding the data.
Decision trees leave you with a difficult
decision. A deep tree with lots of leaves will overfit
because each prediction is coming from historical
data from only the few houses at its leaf. But a
shallow tree with few leaves will perform poorly
because it fails to capture as many distinctions in the
raw data. Even today's most sophisticated modeling
techniques face this tension between underfitting
and overfitting. But, many models have clever ideas
that can lead to better performance. So, we first
implemented random forest to see the results. We
used gradient boosting after implementing random
forest which is machine learning technique for
regression and classification problems, which
produces a prediction model in the form of an
ensemble of weak prediction models, typically
decision trees.
With 79 explanatory variables
describing(almost)
Every aspect of residential homes in Ames, Iowa.
Our inspiration is to do Comprehensive data
exploration and staked regression to increase the
accuracy of the model. By doing this project we can
better implement the techniques the we learned in
the class like data cleaning, regression analysis,
ensembling learning, feature engineering etc.
2. Related Work
There are much related work going on with different
prediction models and different type of approaches
used to implement the prediction models like
Xgboost, Gradient boosting, random forests etc. The
reference we used to implement our project used
linear regression to implement his prediction model
by doing data cleaning, data visualization, how
common factors are affecting the prices of the
houses and then followed by linear regression.
The Approach follows:
• imported dependencies , for linear
regression we used sklearn (built in python
library) and import linear regression from it.
• then initialized Linear Regression to a
variable reg.
• set labels (output) as price columns and also
converted dates to 1’s and 0’s so that it
doesn’t influence our data much .
• he again imported another dependency to
split our data into train and test. Made 90%
and 10% as train and test data respectively.
And randomized the splitting the data by
using random_state.
• So now , the train data , test data and labels
for both let us fit our train and test data into
linear regression model.
• After fitting our data to the model we can
check the score of our data ie , prediction. in
this case the prediction is 73%.
The accuracy of the model is much lower to be
useful, to achieve the satisfactory accuracy we
decided to choose random forest and gradient
boosting implementation by doing better data
cleaning, data exploration, data visualization on a
different data set which covered almost every aspect
that is dependent on house price prediction
3. Data Set
searching related dataset that has most of the
variables to predict the house price was hard in this
task. Expected data from the dataset was:
It should to have essential and adequate variables to
frame a composite choice parameter depend on
which results can be acquired.
It should not have a high recurrence of clashing
information. It should be in an available and perfect
configuration on which information preprocessing
could be performed.
3.1 Data Issues
3.1.1 Data Quality
Certain fields need property estimations, certain
qualities of interest, and contain just total
information. A portion of the field estimations of the
basic decision parameters were absent. Due to this a
portion of the information must be included.
Another issue was that of total information.
3.1.2 Performance
Performance containing mistakes and
exceptions. Since the information was not perfect, it
wasn't possible to accomplish the normal output
without expelling mistakes and exceptions. This was
one of the aspects to consider to get expected
outcomes.
3.1.3 Unformatted Text
Some data can be in different format than
expected. So machine can not determine output. For
example, data should be in integer format but there
are some data which are in character format. That
will create some issues while training data and
calculating.
3.1.4 Noisy Data
Particular fields that contain new information
can't be comprehended and translated accurately by
machines, like unstructured content. For example, in
a dataset, signs had numerous unstructured fields.
For example, some had strange symbols that can not
be read by machine.
3.1.5 Inconsistent Data
Containing inconsistencies (an absence of
similarity or comparability between at least two
realities). Recurrence of this sort of information was
high in every one of the fields where one certainty
was spoken to in various ways code names, images,
abbreviation and so on.
3.2 Project Data
The labeled data set consists of 1460 rows
different 79 variables of details for number of
bedrooms, house reviews, specially selected for
house analysis. The reviews is not
binary, meaning the final result will depend on
height of ceilings or proximity of nearest school or
nearest transportation center, number of bedrooms
and bathrooms, like all 79 house price prediction
variables. The 1460 review labeled training set does
not include any of the same house as the 1460
review test set.
train - The labeled training set. This file has a
header row containing all variable name followed by
1460 rows containing locations, bedrooms, and all
79 variables prediction.
test - The test set. This file has a header row
followed by 1460 rows containing an same variables
for testing. Your task is to predict the price for
house for each one.
4. Methods for data processing
These methods can be used to process the data for
optimization of prediction:
4.1 Data cleaning

This can be apply on raw data by
normalization, statistical methods, duplicate
elimination. Data issues specified above, should be
solved to predict the optimized house price. For
example, to improve data quality, we can eliminate
unrequired variables or variables which can
decrease the performance of machine. For noisy or
unformatted data, it can be solved by updating
unformatted variables by changing values in proper
format.
4.2 Data visualization
Let’s look at this problem from a builder’s
perspective, sometimes it’s important for a builder
to see which is the highest selling house type which
enables the builder to make house based on that.
from the visualization 3 bedroom houses are most
commonly sold followed by 4 bedroom. So how is it
useful ? For a builder having this data , He can make
a new building with more 3 and 4 bedroom’s to
attract more buyers.
2. Visualizing the location of the houses based on
latitude and longitude.
So according to the dataset , we have latitude and
longitude on the dataset for each house. We are
going to see the common location and how the
houses are placed.
We use seaborn , and we get his beautiful
visualization. Joinplot function helps us see the
concentration of data and placement of data and can
be really useful. Let us see what we can infer from
this visualization. For latitude between -47.7 and -
48.8 there are many houses , which would mean that
maybe it’s an ideal location isn’t it ? But when we
talk about longitude we can see that concentration is
high between -122.2 to -122.4. Which would mean
that most of the buy’s has been for this particular
location.
seaborn which is another built in python library used
to do data representation
4.3 Gradient boosting
It is a machine learning technique for regression and
classification problems, which produces a prediction
model in the form of an ensemble of weak
prediction models, typically decision trees.
Gradient boosting involves three elements: A loss
function to be optimized. A weak learner to make
predictions. An additive model to add weak learners
to minimize the loss function.
we will be using the sklearn library for it. we create
a variable where we define our gradient boosting
regressor and set parameters to it , here
n_estimator - The number of boosting stages to
perform. We should not set it too high which would
overfit our model.
max_depth - The depth of the tree node.
learning_rate - Rate of learning the data.
loss - loss function to be optimized. ‘ls’ refers to
least squares regression.
minimum sample split - Number of sample to be
split for learning the data.
We then fit our training data into the gradient
boosting model and check for accuracy.
5. What we have done
We gone through following steps to know our data
the best:
1. Understand the problem. We look at each
variable and do a philosophical analysis
about their meaning and importance for this
problem.
in order to have some discipline in our
analysis, we created an Excel sheet with
following colums:
1) Variable,
2) Type(numerical,categorical),
3) Segment(e.g. building, space, location )
4) Expectation(expections regarding the
variable like high, medium, low)
5) Conclusion(importance of the variable)
6)comments(any general comment).
we went through this process and concluded
that the following variables can play an
important role in this problem:
 OverallQual (which is a variable that we
don't like because I don't know how it was
computed; a funny exercise would be to
predict 'OverallQual' using all the other
variables available).
YearBuilt.
TotalBsmtSF.
GrLivArea.
we ended up with two 'building' variables
('OverallQual' and 'YearBuilt') and two
'space' variables ('TotalBsmtSF' and
'GrLivArea'). This might be a little bit
unexpected as it goes against the real estate
mantra that all that matters is 'location,
location and location'. It is possible that this
quick data examination process was a bit
harsh for categorical variables.
2. Univariable study: We just focus on the
dependent variable ('SalePrice') and try to
know a little bit more about it.
-we analyzed the ‘SalesPrice’ variable and
saw the relationship to its categorical
features and numerical variables.
Based on that we concluded that:
1) 'GrLivArea' and 'TotalBsmtSF' seem to
be linearly related with 'SalePrice'. Both
relationships are positive, which means that
as one variable increases, the other also
increases. In the case of 'TotalBsmtSF', we
can see that the slope of the linear
relationship is particularly high.
2) 'OverallQual' and 'YearBuilt' also seem
to be related with 'SalePrice'. The
relationship seems to be stronger in the case
of 'OverallQual', where the box plot shows
how sales prices increase with the overall
quality.
3. Multivariate study : We'll try to understand
how the dependent variable and independent
variables relate.
To explore the universe, we started with
some practical recipes to make sense of our
'plasma soup'((source:
http://umich.edu/~gs265/bigbang.htm)):
• Correlation matrix (heatmap style).
• 'SalePrice' correlation matrix.
• Scatter plots between the most
correlated variables.
At first sight, there are two red colored squares
that get my attention. The first one refers to the
'TotalBsmtSF' and '1stFlrSF' variables, and the
second one refers to the 'GarageX' variables.
Both cases show how significant the correlation
is between these variables.
Another thing that got my attention was the
'SalePrice' correlations. We can see our well-
known 'GrLivArea', 'TotalBsmtSF', and
'OverallQual' are highly correlated, but we can
also see many other variables that should be
taken into account.
mega scatter plot gives us a reasonable idea
about variables relationships.
4. Basic cleaning: We'll clean the dataset and
handle the missing data, outliers and
categorical variables.
variables like 'PoolQC', 'MiscFeature' and
'FireplaceQu' are strong candidates for
outliers, so we happily to deleted them.
Regarding 'MasVnrArea' and
'MasVnrType', we consider that these
variables are not essential. Furthermore,
they have a strong correlation with
'YearBuilt' and 'OverallQual' which are
already considered.
we have one missing observation in
'Electrical'. Since it is just one observation,
we'll delete this observation and keep the
variable
Univariate analysis: The primary
concern here is to establish a threshold that
defines an observation as an outlier. To do
so, we standardized the data. In this context,
we convered data values to have mean of 0
and a standard deviation of 1.
5. Test assumptions: We'll check if our data
meets the assumptions required by most
multivariate techniques.
According to Hair et al. (2013), four
assumptions should be tested:
Normality, Homoscedasticity, Linearity,
Absence of correlated errors
 We are in progress to test this assumptions
6. References
[1] House Prices: Advanced Regression Techniques
(Dataset)
https://www.kaggle.com/c/house-prices-advanced-
regression-techniques/data
[2] . A Beginner's Guide to Recurrent Networks and
LSTMs - Deeplearning4j: Open-source, Distributed
Deep Learning for the JVM
https://deeplearning4j.org/lstm.html
[3] Using Recurrent Neural Networks in DL4J -
Deeplearning4j: Open-source, Distributed Deep
Learning for the JVM
https://deeplearning4j.org/usingrnns
[4] Pythonic Data Cleaning With NumPy and
Pandas
https://realpython.com/python-data-cleaning-numpy-
pandas/
[5] . Create a model to predict house prices using
Python
https://towardsdatascience.com/create-a-model-to-
predict-house-prices-using-python-d34fe8fad88f
[6] A Gentle Introduction to the Gradient Boosting
Algorithm for Machine Learning
https://machinelearningmastery.com/gentle-
introduction-gradient-boosting-algorithm-machine-
learning/
[7] . How to Implement Random Forest From
Scratch in Python
https://machinelearningmastery.com/implement-
random-forest-scratch-python/
[8] Bagging and Random Forest Ensemble
Algorithms for Machine Learning
https://machinelearningmastery.com/bagging-and-
random-forest-ensemble-algorithms-for-machine-
learning/