Free tool
A/B Test Significance Calculator
Drop in visitors and conversions for each variation. Get confidence, p-value, and relative lift. Two-tailed z-test, pooled standard error. The same maths your testing platform uses at the end of the run.
Inputs
Enter totals for each variation. Results update live.
Result
Control rate
2.00%
Variant rate
2.40%
Relative lift
+20.00%
Absolute lift
+0.40 pp
Statistical confidence
94.62%
Not yet significant. Keep running, or redesign the test.
Two-tailed z-test for proportions, pooled standard error. Assumes random assignment, independent samples, and that you pre-committed to sample size. If you’re peeking daily, the confidence number above understates your false-positive rate.
A reminder before you ship the win
Hitting 95% in this calculator is necessary, not sufficient. Three things worth checking before you call a test:
- Sample size was pre-committed. Peeking daily and stopping at the first crossing inflates false positives from 5% to 20 to 30%.
- Test ran for at least two full business cycles. Weekday and weekend traffic behave differently. A three-day test captures one of them.
- Segments are hypothesis generation, not validation. Slicing by device and re-running significance on the slice is a false-positive factory.