A test of B2B sales forecasting methods

A Test of B2B Sales Forecasting Methods

July 2012

Share this white paper!

2012 Nimble Apps Limited

A test of B2B sales forecasting methods

Table of contents
1 Introduction

....................................................................................................................................

2 Scope and methodology ................................................................................................. 2

2.1.Scope ........................................................................................................................................................... 2
2.2.Methodology ............................................................................................................................................. 2

3 Analysis of sample

.................................................................................................................

3.1.Closing dates are always optimistic .............................................................................................. 2

3.2.Losing takes longer than winning ................................................................................................... 3

4 Forecasting methods ........................................................................................................... 3

4.1.Description ............................................................................................................................................... 3
4.2.Results ........................................................................................................................................................ 4

5 Conclusion ........................................................................................................................................ 6

Share this white paper!

A test of B2B sales forecasting methods

1 Introduction
Sales forecasting is a major issue for B2B companies. On one hand, B2B companies often lack the
thousands of data points that statistical forecasting techniques require. But on the other hand, recent
research by Aberdeen Group shows a clear link between forecasting best practices and sales
performance.
The implication is obvious: robust, B2B-specific forecasting methods would change the life of sales
managers.
This white paper describes the test of common and not-so-common B2B forecasting techniques we
recently performed on a sample of SalesClic client data. Our research yields a number of confirmations and
a few surprises.

Share this white paper!

A test of B2B sales forecasting methods

2 Scope and methodology

2.1.Scope
Our sample data - representative of B2B compa-

Measure of forecasting error

nies selling complex products and services was made up as follows:

We measured forecasting errors over the test pe-

12 sales teams

riods using their root mean square, normalized by

The teams were located in the US, UK and Asia

The teams operated in the software, electronic
equipment and financial services industries

the average amount of opportunities in the sample.

3 Analysis of sample

T he teams managed structured sales pipelines

Before discussing the accuracy of the forecasting

(i.e. following stage-by-stage sales processes)

techniques included in the test, it is worth noting

Over the research period, we totaled 144,817

closed opportunities

The sales cycles were from 75 to 250 days

2 interesting patterns in the sample data.

3.1.Closing dates are always optimistic
Initial closing dates are optimistic for 10 teams out

2.2.Methodology
Training and test periods
We divided the historical data of these 12 teams
into training and test periods for the selected
algorithms. The training period is always twice
as long as the test period, with a minimum of 1.5

of 12 in our sample. On average, it takes 22%

longer than initially expected to win an opportunity for the sample teams.
That is worrying in a B2B context, where sales
forecasts are very sensitive to closing dates. For
sales managers and sales operations managers,
monitoring closing dates is a clear priority.

years, an average of 4.5 years and a maximum of

9 years. Training periods contained at least 500
sales opportunities.

Share this white paper!

A test of B2B sales forecasting methods

4 Forecasting methods
120%

Closing date error

4.1.Description

100%

The following describes the 8 forecasting

80%

techniques we tested.
60%

Weighted pipeline #1 is a simple weighted pipe-

40%

line using declared opportunity amounts, closing

Sales cycle length
(days)

20%

0%

50

100

150

200

250

300

dates and closing probabilities

 eighted pipeline #2 uses declared opportunity
W
amounts and closing dates but historical closing

-20%

Figure 1 - Closing date error x Sales cycle length

probabilities
 eighted pipeline #3 uses declared opportunity
W

The bias increases with the length of the sales

amounts, historical opportunity time-to-wins1 and

cycle, as illustrated by figure 1 above. In our

historical closing probabilities

sample, we find a closing date error of 16% for the

Weighted pipeline #4 is a variation of weighted

team with the shortest sales cycle, and of 109%

pipeline #3, using declared opportunity amounts,

for the team with the longest sales cycle.

historical stage durations2,3 and historical closing

probabilities

3.2.Losing takes longer than winning

 e also tested most combinations of the 4

W

In our sample, two-thirds of closed-lost oppor-

weighted pipelines methods mixing declared

tunities are lost after the closing date initially

and historical inputs

expected, and losing an opportunity takes an

average 1.7 times longer than winning one.

The linear predictors assume a linear relationship between stage amounts on day d and
closed-won amount on day d+n

Stagnation in the pipeline does not bode well for

pending opportunities, and B2B companies have
much to gain from detecting stuck opportunities as early as possible.

The decision tree predictors are sophisticated

algorithms using the decision tree technique. Our
trees are grown and pruned on stage amounts

The daily closing rate method assumes that,

until the end of the forecast period, a team will
close the same daily amount as during the last n
days (see example p. 5)

1. The time to win of an opportunity is the average time required to win opportunities that have reached the corresponding
pipeline stage.
2. The duration of a pipeline stage is the average time that opportunities spend in that stage.
3. Time to win for pipeline stage n and the sum of stage durations for stages n to i usually differ
because of early losses, stage jumps and back-and-forth opportunity movements.

A test of B2B sales forecasting methods

We classified these techniques according to 2 criteria that are very relevant to CRM software users:

Whether they use declared or calculated closing dates and closing probabilities
How computer intensive they are
Closing date

Closing probability

Computer intensive

Weighted pipeline #1

Declared

Declared

Low

Weighted pipeline #2

Declared

Calculated

Medium

Weighted pipeline #3

Calculated

Calculated

Medium

Weighted pipeline #4

Calculated

Calculated

Medium

WP combinations

Both

Both

Medium

Linear predictors

Both

None

High

Decision tree predictors

None

None

High

Daily closing rate

None

None

Low

Figure 2 - Classification of forecasting methods

4.2.Results
Judgmental forecasts are not reliable

This means that sales teams are sitting on a

huge amount of forecasting information they

The simple weighted pipeline forecasting tech-

could be using to inform their judgments.

nique (declared amounts, declared closing dates

and declared closing probabilities) is the second

Our research shows that replacing sales rep

worst performing in the sample.

and manager judgment on closing dates and

closing probabilities with historical averages

This research thus confirms what most sales ma-

increases forecast accuracy.

nagers already know: simple weighted pipelines

 eighted pipeline #3 is 7% more accurate than

W

cannot be trusted.
Leveraging historical data helps

weighted pipeline #1

Weighted pipeline #4 is 35% more accurate than

weighted pipeline #1

Traditional CRM software is unable to leverage

properly the historical data of sales teams for
optimization purposes.

Share this white paper!

A test of B2B sales forecasting methods

Sophistication pays... up to a point

And the winner is...

Averaging time-to-wins, durations and closing

As shown by figure 3 (below left), the best fore-

probabilities is a straightforward way to leverage

casting technique on our sample data is a simple

historical data. How do the more sophisticated

but nimble one: the daily closing rate. Here is an

techniques tested here perform?

example of how it works:

Linear predictors dont work very well.

Suppose your team has closed 300K

In particular, they are outperformed by weighted

pipeline #4. This is disappointing but not
surprising since sales pipelines can have widely
different shapes, and linear equations are ill
equipped to deal with such irregularities.

Decision trees perform well. Compared to the

over the past rolling 3 months

That is a daily average of 3.3K

 uppose that you are 30 days
S
into the current quarter (1/3 of the quarter)...

...and that you have closed 50K so far

simple weighted pipeline, they increase forecast

 our historical daily closings forecast for the

Y

accuracy by an average 46%. However, they are

quarter is: 50K + 3.3K x 60 days = 250K

quite hard to implement.

This method improves simple weighted pipeline

Accuracy improvement
Weighted pipeline #1

Reference point

Daily closing rate

53%

Decision tree
predictors4

46%

Weighted pipeline #4

42%

WP combinations

20%

Linear predictors5

11%

Weighted pipeline #3

7%

Weighted pipeline #2

-8%

forecasts by 53%. Forecast accuracy also increases by 20% compared to weighted pipeline #4.

Figure 3 - Performance of forecasting methods

vs. simple weighted pipeline

4. This is the average of 3 decision tree predictors all 3 tightly grouped around this average.
5. This is the average of 6 linear predictors. The best one improves forecast accuracy by 21% over weighted pipeline #1.

A test of B2B sales forecasting methods

5 Conclusion
This research suggests immediate ways for B2B companies to improve sales forecast accuracy.

Measure pipeline dynamics (opportunity time-to-wins, pipeline stage durations,

closing probabilities by pipeline stage) and use that information for forecasting purposes.
 alculate your daily closing rate and use that information for forecasting purposes.
C
Implementing forecasts based on decision trees is also a good idea, although potentially complicated.
For additional progress, we believe that moving from the analysis of a pipelines macro structure (pipeline
stages essentially) to a pipelines micro structure (the behavior of individual opportunities) is required.
Nimble Apps will continue to study and share insights on these topics.

About Nimble Apps Limited

Nimble Apps is the publisher of SalesClic, a simple and powerful solution for visualizing,
analyzing and forecasting your sales pipeline.
SalesClic integrates with Google Apps, Highrise and Salesforce.

Follow SalesClic on
6