The variant is up 25.9%. What lift can you actually promise?
Every A/B calculator gives you a p-value and a lift. This one also gives you the interval around that lift, which is the half of the result that decides whether the business case survives. Four counts, no ranges, one click.
Marketing Starter Statistics free
After you install, this is the model to open.
Did the B Version Win?
- In your spreadsheet, click the Sortia icon in the strip of icons down the right-hand edge. No strip? Click the arrow at the bottom-right to open it. You can also use Extensions, then Sortia, then Open Sortia.
- Click Start from a template and put that name in the search box.
- Pick the card with that name and click Load this template. It arrives on a new tab with real numbers already in it.
The answer
The variant won, and the interval says how much of a win you can bank on:
- Relative lift
- 25.9%
- p-value
- 0.010 two-tail
- Control vs variant
- 3.66% to 4.61%
- Lift the data supports
- 6% to 46% 95% interval
The point estimate is 25.9%. The 95% interval on the absolute lift runs from 0.225 to 1.672 percentage points, which is a relative lift of anywhere between about 6% and 46%. Anything you promise at 25.9% is the most optimistic reading of your own evidence that is still defensible, so plan on the bottom of the interval and be pleased by the top. Two things no test can see: whether the groups were comparable, and how many times you looked before you stopped.
The model
A two-week split test on a signup page. The A/B Test tool takes no range at all: you type the two visitor counts and the two conversion counts, and it runs a two-proportion test on them.
| Control visitors / signups | 5,840 / 214 |
| Variant visitors / signups | 5,810 / 268 |
| Significance level | 0.05 |
| Annual traffic to the page | 260,000 |
| Verdict, lift and interval | calculated |
Once it is in your sheet
- The model arrives with real numbers in it and runs as it stands, so you can press the button first and understand it second.
- Change the numbers to yours. The sheet marks which cells are inputs and which hold formulas, and most labels carry a note explaining the row.
- Press the run button at the bottom of the panel. It is labeled for the tool you are in, and the result lands on its own tab, with a written reading of it beside the figures.
Never used Google Sheets? Start here goes the whole way, in seven steps, and assumes nothing.
Next question
- How long does this A/B test have to run?How Many Visitors Does This Test Need?
- What does a margin of error of four points cost?How Many People Do We Have to Ask?
- A 7% margin in January. How often is the year a loss?Will the Retainer Book Cover the Year?
- Plan for the average month or the bad one?How to Split Spend When Conversion Is a Guess
- When does a new hire start paying for themselves?Is the Next Hire Worth It?
- A product launch risk register pricing five ways a launch goes wrong against mitigation cost, from a calendar invite to production fixes. Free template.What Could Go Wrong With This Launch?
Every model like this one, and the method behind them: Statistics in Google Sheets.