Professional Documents
Culture Documents
net/publication/327225380
CITATION READS
1 11,948
5 authors, including:
Rajat Garg
4 PUBLICATIONS 20 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Teja Silaparasetti on 25 August 2018.
Abstract—Demand Forecasting, undeniably, is the single most and Operations Planning (S&OP) is an integration process
important component of any organizations Supply Chain. It used in business organization to ensure efficient coordina-
determines the estimated demand for the future and sets the level tion among cross functional units to align company strategy
of preparedness that is required on the supply side to match the
demand. It goes without saying that if an organization doesnt get with Supply Chain planning. Globalization, new opportunities
its forecasting accurate to a reasonable level, the whole supply and Supply Chain Management differentiation compel the
chain gets affected. Understandably, Over/Under forecasting has organization even struggle further as the integration process
deteriorating impact on any organizations Supply Chain and among suppliers, market, and stakeholders become complex
thereby on P and L. Having ascertained the importance of De- and data intensive. Various forecasting models are used to
mand Forecasting, it is only fair to discuss about the forecasting
techniques which are used to predict the future values of demand. predict what the demands on the system will be in the future
The input that goes in and the modeling engine which it goes so that appropriate designs and operating plans can be devised.
through are equally important in generating the correct forecasts Ironically, the basic premise behind generating forecasts is
and determining the Forecast Accuracy. Here, we present a very that they are mostly wrong. Forecasts must include analysis of
unique model that not only pre-processes the input data, but their potential errors. The longer the forecast horizon, the less
also ensembles the output of two parallel advanced forecasting
engines which uses state-of-the-art Machine Learning algorithms accurate the forecast will generally be. Benefits can often be
and Time-Series algorithms to generate future forecasts. Our obtained from transforming a forecasted quantity into a known
technique uses data-driven statistical techniques to clean the data quantity. The information systems play a key role mainly in
of any potential errors or outliers and impute missing values if operative planning and management. It is the requirement
any. Once the forecast is generated, it is post processed with of almost all the comapanies to start run their business
Seasonality and Trend corrections, if required.Since the final
forecast is the result of statistically pre-validated ensemble of without having relevant information from ERP (Enterprise
multiple models, the forecasts are stable and accuracy variation Resource Planning) systems. In consequence of more efficient
is very minimal across periods and forecast horizons. Hence it information use through ERP, the partial planning methods
is better at estimating the future demand than the conventional as MRP II (Manufacturing Resource Planning), Sales and
techniques. Operations Planning (S&OP) or APS (Advanced Planning
Index Terms—Supply Chain, Demand Forecasting, Time Se- and Scheduling) are developed. To maximize the effect of
ries, Machine Learning, Ensemble, Optimization, Cognitive Anal- accessible methods for any internal company processes, the
ysis, Outliers
company must be build on an objective and evaluated demand
forecasts. The choice of optimum forecasting procedures and
I. I NTRODUCTION following use of obtained forecasts may become a competitive
Demand Forecasting is the key activity which more or less advantage. Together with other modern methods it accelerates
controls all other activities of Supply Chain Management. It other company processes, reduces the costs and increases
is the key driving factor in planning and decision making in the value for the customer. The demand forecast determines
Supply Chain Management as well as Enterprise level. All ma- the volume of products, place and time horizon in which
jor firms truly depend on the efficiency of demand forecasting they will be needed. In relation with the demand forecast
to take major decisions such as capacity building, resource it is necessary to deal not only with the quantitative aspect
allocation, expansion and forward or backward integration, of the needs (the volume demanded by customers) but also
etc. Forecasting is a prediction or an estimation of an actual their qualitative aspect (the type of customer’s needs). The
value in a future time period .It is usual that there might be accurate demand forecast is thus important for the produc-
forecast error as actual results differ from the projected value, tion and distribution management but also for e. g. areas
long time horizon has chance of more error. This is important of marketing (distribution of sales forces, communication,
to measure the forecast error for adapting corrective action promotion and planning of new products), finance (current
plan. The development of advanced technology enables the need of money, budgets and calculations), investment designs
Supply Chain Management stakeholders to share real time data (production facilities, workshops and warehouses), research
and information across the network which helps dual benefit and development (innovations) and human resources (structure
to inventory and customer service. This underlying result of and labour force volume planning, training). Demand planning
the process is accuracy of forecast which firmly ensures the represents a set of methodologies and information technologies
successful and sustainable business operations. This is called for the use of demand forecasts in the process of planning.
Collaborative Planning, Forecast, and Replenishment. Sales The aim is to accelerate the flow of raw materials, materials
978-1-5090-6697-1/17/$31.00 ©2017 IEEE 1105
Proceedings of the 2nd International Conference on Inventive Computation Technologies ( ICICT 2017)
IEEE Xplore Compliant - Part Number:CFP17K52-ART, ISBN:978-1-5090-6697-1
and services beginning with the suppliers through transforming • Spinning:manipulating forecasts to obtain the most fa-
to products in the company and to their distribution to their vorable reaction from individuals or departments in the
final consumers. The demand planning process is done to help organization
the business understand its profit potential. Indirectly it sets Demand plans should reflect what we really think what we
the stage for capacity, financing, and stakeholder confidence will sell in the future.
. The implementation of the demand planning enables to
determine the closest possible forecast to the planning horizon
B. Statistical Forecast:
and decide the volume of production, stock and sources
capacity distribution among particular products to maximize The ambition of the business in using the statistical
the profits of the whole company. The key requirement for forecast is to minimize the sales planner work load and
efficient company management is sharing the mutual forecast. in the meantime give a good forecast that will benefit the
However, the research carried in production companies showed organization. Statistical forecasting might not be suitable
individual departments of the company in some cases draw up for every product, but it is proven successful for products
forecasts on their own and thus they base their planning on with a fairly stable sales history and a large number of
different figures. This provokes conflicts among the resulting lighting products fit this description. Statistical forecasting
activities of in-company plans . This kind of situation happens significantly reduces the number of products that will have
in case when the company prefers to approve the financial plan to be touched by every individual sales planner every
which doesn’t correspond with the updated forecast results. week/month.
The forecasting should always be the process which is essen-
tial and determining for other company processes, including • Data pre-processing: Outlier alerts are used as input for
financial planning. The financial plan often represents the the stat forecast manager to do history cleaning.
main motivation source for the company managers as this All individual alerts will be listed, which means that if a
reflects the requirements of the company top management and certain product has more than one outlier, all these events
company’s strategic goals. will be specified. A value is considered an outlier if:
A variety of modeling techniques are available for produc- Value is higher than average +3 ∗ standarddeviation
ing forecasts. Based on data patterns, forecasting horizon, data Value is lower than average 3 ∗ standarddeviation
availability and business requirements the choice of technique • Algorithms: Based on the chosen statistical model, the
differs [20]. Through appropriate combining of the forecasting existing business model will calculate a forecast. Avail-
techniques it is possible to estimate quantitative influences able statistical forecast models are listed as:
of the identified factors and set the demand forecast [14]. 1) Moving Average: The moving average model is used
The most frequently used statistical forecasting method is the to exclude irregularities in the history pattern. The
time series technique. It uses historical data sequenced by model calculates the average of all historical values.
time and projects future demand by the same time sequence All values are weighted equally. It does not take
[24]. POS data is rich in information for building forecast leading zero-values into account.
models. Building a good forecasting model with POS data is 2) SES Model:The SES model will also calculate an
demonstrated in many case studies It is important to realize average, but instead of weighting all historical val-
that demand planning used in practice is not a mere creation ues equally, the SES model will give more weight
of a perfect system to carry out the demand and sales forecast. to the more recent history.Within the SES model
The objective of forecasting is to predict demand whilst the the smoothing factor (alpha) will determine how
aim of demand planners is to shape the demand and produce quickly the forecast reacts to a change in the history
a resource requirements plan [21]. Without the right forecast - pattern. The higher the alpha factor, the more weight
planning system integration it is not possible to use efficiently will be assigned to the most recent history, the
the information provided in the forecasts. quicker the forecast reacts to the recent history.
3) Croston Model: The Croston model is used for
II. E XISTING P ROCEDURE :
products with intermittent demand. It consists of
A. Demand Planning Process: exponential smoothing calculation, together with an
The objective of Demand planning is to make reliable average interval between demands. These 2 inputs
demand plans as base for the S&OP process. Game playing, will then be used in a form of a constant model to
the intentional manipulation of the forecasting process to gain predict future demand.In case no zero values exist in
personal, group, or corporate advantage, is clearly not allowed. the history pattern, Croston model will have exactly
Examples of such games: the same result as the SES model (in case of an
• Sandbagging:underestimating sales in order to set expec- equal alpha factor )
tations lower than the demand actually anticipated 4) Seasonal Linear Regression: Seasonal linear re-
• Hedging:overestimating sales in order to secure addi- gression will automatically determine trend values
tional product or production capacity and seasonal indices. The system will first try to
• Enforcing:maintaining a higher forecast than the antic- find a seasonal pattern. In case the system cannot
ipated sales, in order to keep forecasts in line with the find a seasonal pattern, it will calculate a forecast
organizations sales or financial goals using linear regression.In case it can find a seasonal
978-1-5090-6697-1/17/$31.00 ©2017 IEEE 1106
Proceedings of the 2nd International Conference on Inventive Computation Technologies ( ICICT 2017)
IEEE Xplore Compliant - Part Number:CFP17K52-ART, ISBN:978-1-5090-6697-1
pattern, it will first correct the historical values using C. Data Shaping
calculated seasonal indices. A linear regression will The Data should be aggregated or dis-aggregated to the
calculate a forecast using these corrected historical level in which the forecast should be generated. This helps
values. Last step is to apply the seasonal indices to in making the generalization capability of the forecasting to a
the calculated linear regression. better level. The cost function for the time-series forecasting
5) Double exponential smoothing: Double exponential in this domain differs for different organization.
smoothing uses the same technique as the single Some common cost functions used are:
exponential smoothing, but will also apply a trend. 1. PN P3
As in the single exponential smoothing, an alpha i=1 | j=1 (yp − ya )|
factor will determine the weight to determine how Accuracy = 1 − P3
quickly the forecast reacts to a change in recent j=1 yp
2) Model Selection:: The forecast done for each SKU, is as the outlier and is either brought down/up to
done using the time series and machine learning algorithms. the mean/median based on maximizing the cost
Then each of the SKU is matched with the results given by function of the organization. This basically means
time-series , machine learning and an model with the ensemble that the each SKU will have a different set of the
of the outputs of time-series and machine learning. outlier treatment. The missing value is nothing but
1) Time-Series Algorithms: The actual data is fed into to findout whether the ”sales zero” in a particular
the model consisting of the time-series algorithms. For data is actually the case or not. If the zero sales
different SKU’s different algorithms are selected based is not possible for the SKU, then the missing
on the mse in the validation or testing data. Before value is imputed with mean of the actual data. For
passing the data into the model, it is pre-processed. [2] getting the seasonal pattern of the data, the data is
tranformed into the lags of the months for finding
• Data Pre-processing: The actual sales do ave great
a relationship between the current actual sales. The
impact with the promotions,stockouts and dumping
problem of forecasting is a regression problem. The
etc.These behave as the outlier in the system and
regressors are the lags of the actual data. Here to
need to be removed or treated. The values which
find a seasonality pattern in the data, each yearly
are more than 40% or less than 10% for a particular
data is taken as a lag. The two important constraint
month in a range of 12-month are replaced with
are taken into the consideration:
the mean of the actual sales data or if the sales
for a particular month is above the particular limit – Time Constraint: It takes a lot of time to find the
Inter Quartile Range is treated and imputed with correct hyper-parameter settings.
the accepted sales for that month. Particular limit – Sesonality Pattern: It was found that if the lag of
varies SKU to SKU. Also, the sparse SKU i.e the 12 for monthly data and 53 for weekly data was
actual data containing more number of zeros are taken into the consideration, the accuracy [cost
either passed into croston model [17] [5] else the function] increased.
zero sales data in a month is replaced with mean of • Algorithms: There is a lot of constraints for the
the data. forecasting using machine learning.
• Algorithms: The algorithms which are used in – The hyper-parameters for each SKU for different
the time-series algorithms are Weighted moving algorithms will be different all the time. This
Average, Simple moving average, Autoregressive takes a lot of time to optimized to the required
integrated moving average (ARIMA) and Holts parameters settings.
Winter [22] [19] [9] [7] [25]. For Weighted Moving – Seasonality capturing is very less when used
Average 0.5, 0.3 and 0.2 weights are given starting machine learning algorithms.
from the most recent observation to the 3rd last
The algorithms that are used are Support Vector
observation whereas equal weights are assign to the
Regression [11] [13] [8], Decision Trees Regression
last n (no of dependent variables) observations to
[10], Linear Regression, Ridge Regression [6] and
find out the new forecast. On the basis of residuals
Random Forest Regression [16]. Random Forest
we are finding out the value of n. To find out the
Regression is used for the selection of important
order of the Arima model we have used Arima
features out of the lag training set [12]. Other feature
optimization function ML and CSS and then using
selection techniques can be used which is the future
residuals RMSE and MSE.To find out the value of
implementation and research for our work. As the
α, β and γ of Holt’s Winter, the optimization func-
dataset is a time series dataset, the sequence of
tions L-BFGS-B and Nelder-Mead and for finding
the data or the training examples is important. The
the residuals RMSE and MSE is used. In case the
training and testing sets are divided into the 70,30
model gives 0 in the middle of the forecast [18]
% using the hold-out method. For the optimizing
value is replaced with the mean of the whole vector.
the hyper-parameters, Grid Search and Random
In the validation set, the algorithm which gives the
search methods are used. Bayesian Search and other
minimum mse is selected for the future forecast [3].
evolutionary optimization search methods are the
2) Regression Based Algorithms: There is a bit of differ- future work. The best algorithm which results in
ence in the data that is fed into the models containing the minimum mse in the validation set is selected
machine learning regression algorithms. This regression for the future prediction of the sku. Out of all the
based model contains the following steps to be done to algorithms the one which gives the minimum mse is
take care of the forecasting of sales data. matched again with the ensemble of the algorithms.
• Data Processing: The sales of a particular period Ensemble is done with the averaging of the error in
(day, week, month or year) is matched with the ac- the validation set.
tual data and the values or the sales which contribute
to more than 40% or less than 10% of the total sales model = [algorithm1 , algorithm2 , ..., algorithmn ]
for 7 days(if the data is weekly), 53 weeks(data is
weekly) or 12 months(data is monthly) is treated errori = |yp − ya |
978-1-5090-6697-1/17/$31.00 ©2017 IEEE 1108
Proceedings of the 2nd International Conference on Inventive Computation Technologies ( ICICT 2017)
IEEE Xplore Compliant - Part Number:CFP17K52-ART, ISBN:978-1-5090-6697-1
where yp is the predicted value and ya is actual. used averaging of the time series and regression based
Also,the weights assigned to each model/algorithm model results to give an ensemble forecast which brings
[algoritmi ] is given by: the deviation near to zero or minimum of the deviation
1 of time series output and regression based model output.
weighti = Pnerrori1 Hence, this more enhances the generalization capability
i=1 errori of the model. The weights that is calculated for the two
where i(1, n), n varies in the model set. kinds of models is by calculating the deviation from the
• Future Forecast Generation:Once the forecast is validation period of the data.
done, then the forecast results are validated and
different methodology are taken into the account to model = [modeltime−series , modelmachine−learning ]
see which all process can be taken into the account
for improving the accuracy. The comparison is
1
required to be done at the level at which the forecast
is generated. Identify the Models[algorithm with weighti = Pkdeviation1 i
i=1 deviationi
hyper-parameter settings] producing SKU wise best
results and separate the SKUs as per the models
here, k varies in the model dataset. The model that is
giving best results consistently in the validation
selected in case of the ensembling is through the greedy-
phase. Once the Model for specific variable is
selection. Here the algorithms that gives better results
identified, initiate the run to forecast for future 18
consistently is selected to used in the ensembling of the
months. Seasonality factor is applied to the model
results [15].
if required. This depends on the pattern in which
4) Market Intelligence or Cognitive Approach: After the
the result is produced.
results are generated, the forecasts are plotted against the
history actual and level, trend and seasonality pattern of
the SKU is observed. If the forecast for the SKU doesn’t
Algorithm for the Proposed Model:
follow any pattern with the history sales, it is changed
or tweaked a bit. The following are some of the market
– Input the Data intelligent inputs that are implemented in the model:
– for sku 1:N, where N = total number of SKU for
• Zero Forecast: If the actual sales for a SKU is zero
a market
for the consecutive 2 months, its converted into zero
∗ Data Preprocess: Outlier treatment, Missing forecast for the forecasting periods. This case will
value treatment not be valid for the SKU having sparse data.
∗ Dataset Creation: Convert the Actual data to • Trend Imbalance: If the trend of the forecast doesn’t
12-lag dataset follow the trend of the history of the sales, the trend
∗ Train, Test subset: Holdout method is used and is corrected. If the mean of the initial forecast value
time sequence for the data is maintained [As is 4 times or 0.25 time of the mean of the last 6
the data is time series] months then the forecast value is replaced with the
∗ Feature Selection: Transform the dataset to the last 3 months weighted moving average.
most correlated dataset • Level Correction: There will be some SKU’s which
∗ Training the Algorithms: Pass the Data through are going to have a change in the level due to some
different algorithms. promotional impacts and other external conditions.
∗ Algorithm Selection: Based on the MSE in the This information which is externally present is input
validation/test period the model which results to the model and the level is shifted for the future
in the minimum error is selected. forecast for that many amount of periods which the
∗ Future Prediction: Generate the future forecast market Demand Planner have informed.
3) Ensemble of the Algorithms: The results/forecast gen- • Seasonality Correction: Some of the months will
erated by the time-series and regression based algorithms have lower peaks or depths in the forecast data than
can either be over-forecasting or under-forecasting. To expected. This can be externally corrected before
achieve higher accuracy 100%, the forecast should be the forecast is submitted.
very much near to the actual sales or the error in the • PIPO:Phase-In and Phase-Out SKU’s are clustered
forecasting should minimize. Here, the term that defines and the forecast is changed in that manner. The
the error is deviation. Phase-In products are basically the NPI or New-
Xk Products Introduced and will have a higher or lower
deviation = |yp − ya | sales which are difficult to predict. In case of those
j=1 SKU’s having only one data point, are either used
where k is the time period for the calculation of the Naive Forecast or done the same result as the fore-
deviation. Thus, if the deviation is minimum, the gen- casting output. Mostly Market Demand Planner’s
eralization capability of the model increases. Here, we intelligence helps a lot to take care of this.
978-1-5090-6697-1/17/$31.00 ©2017 IEEE 1109
Proceedings of the 2nd International Conference on Inventive Computation Technologies ( ICICT 2017)
IEEE Xplore Compliant - Part Number:CFP17K52-ART, ISBN:978-1-5090-6697-1
Total number of SKU’s that has been forecasted is 12966. D. Forecast Comparison for different algorithms in the model
Using the model developed by us, the overall average accuracy for different markets:
that has been achieved is 44.14% and the overall accuracy that Market-1:
has been achieved by Incumbent Business Model is 44.72%.
In checking the market-wise accuracy comparison, our model Month Time Series Regression Models Ensemble
is equivalent to Incumbent Business Model. This comparison
is the overall comparison of the total SKU’s for each market April’17 66% 63% 66%
in the month of April-2017. May’17 68% 64% 69%
June’17 65% 63% 66%
B. Forecast Comparison for the month May-2017 using the Market-5:
actual data till March-2017
Market-Name total SKU’s Model FACC I-B-M Month Time Series Regression Models Ensemble
Market-1 2999 54% 45% April’17 60% 58% 62%
Market-2 1164 49% 49% May’17 62% 59% 64%
Market-3 2007 49% 39% June’17 60% 59% 62%
Market-4 1446 56% 58%
Market-5 3265 58% 56% Market-3:
Market-6 3026 47% 43%
Market-7 2515 51% 53% Month Time Series Regression Models Ensemble
Total number of SKU’s that has been forecasted is 16422. April’17 53% 48% 53%
Using the model developed by us, the overall average accuracy May’17 45% 46% 49%
that has been achieved is 52% and the overall accuracy that June’17 49% 47% 51%