A/B Test Kit

Significance
Calculator

Hubspot Marketing Hub

All your marketing tools and data — all under one roof.

Get Started
 How to Use This Template:

In the Calculator tab, plug your result into the

red cells. You'll see the conversion rates and
standard level of error calculate automatically
for you based on the numbers you inputted in
Step 1.

Based on these inputs, you'll see the estimated

range of condidence that the value is
statistically significant based on Z score
confidence intervals. These are then used to
test the P value against the confidence
intervals.

Feel free to look at the equations within the

cells to see how the logic is calculated. Read
cells C38:E41 to the right, then down. The
results of your test (and whether or not they
are significant) will be printed for you here.
For the logic behind the formulas, feel free to
Hub click into the cells.
all under one roof.
 This Template:

b, plug your result into the

the conversion rates and
ror calculate automatically
e numbers you inputted in

uts, you'll see the estimated

e that the value is
ant based on Z score
s. These are then used to
inst the confidence

the equations within the

logic is calculated. Read
e right, then down. The
(and whether or not they
be printed for you here.
d the formulas, feel free to
Step 1: Plug and Chug your Visits and Conver

Visitors Conversions
Variation A 5000 802
Variatio B 5001 801

Your variations' conversion

Step 2:

Conversion Rate Standard Error

Variation A 16.04% 0.52%
Variation B 16.02% 0.52%

Step 3: Significance levels ba

90% Conversion Rate Limits

From To
Variation A 15.18% 16.90%
Variation B 15.16% 16.87%

95% Conversion Rate Limits

From To
Variation A 15.02% 17.06%
Variation B 15.00% 17.03%

Step 4 How confident are we that you

Significant At
Does it pass 90% confidence? NO
Does it pass 95 Confidence? NO

Z= 0.0316

P-value = 0.51

Step 5 Are you test signifiant? F

 Version A converted 0.1%
Read cells to better than
right, then certainVersion
that the B.
changes We are
down 51% your
will improve in is not
Your test Version A
conversion rate. statistically significant.
hug your Visits and Conversion rates from each variation here!

Plug your result into the red cells on the left (D5:E6)

our variations' conversion rates and standard error.

You'll see the conversion rates and standard level of error calculate automatically
for you based on the numbers you inputted in Step 1.

Significance levels based on your inputs

Based on your inputs in Step 1, you'll see the estimated range of condidence that
the value is statistically significant based on Z score confidence intervals. These are
then used to test the P value against the confidence intervals. Feel free to look at the
equations within the cells to see how the logic is calculated.

ow confident are we that your test is significant based?

This step calulates the results. If P passes the 90% and the 95%, you result below
will say the test is statistically significant. If P passes the 90% but not the 95%, the
result will say it is unlikely to be statistically signficant. If the results say it does no
pass either, the test is not statistically significant.

Are you test signifiant? Find the answer here.

Read cells C38:E41 to the right, then down. The results of your test (and whether
or not they are significant) will be printed for you here. For the logic behind the
formulas, feel free to click into the cells.
nd Conversion rates from each variation here!

Plug your result into the red cells on the left (D5:E6)

onversion rates and standard error.

see the conversion rates and standard level of error calculate automatically
for you based on the numbers you inputted in Step 1.

e levels based on your inputs

d on your inputs in Step 1, you'll see the estimated range of condidence that
ue is statistically significant based on Z score confidence intervals. These are
ed to test the P value against the confidence intervals. Feel free to look at the
equations within the cells to see how the logic is calculated.

we that your test is significant based?

tep calulates the results. If P passes the 90% and the 95%, you result below
y the test is statistically significant. If P passes the 90% but not the 95%, the
will say it is unlikely to be statistically signficant. If the results say it does not
pass either, the test is not statistically significant.

gnifiant? Find the answer here.

cells C38:E41 to the right, then down. The results of your test (and whether
t they are significant) will be printed for you here. For the logic behind the
formulas, feel free to click into the cells.