Free A/B Test Significance Calculator (No Sign-Up)
Is your A/B test result statistically significant? Enter visitors and conversions for each variant to get the p-value, uplift and winner. Free, no sign-up.
This tool runs entirely in your browser. Toolcrow stores nothing — no input, no results, no tracking.
What "statistical significance" means for an A/B test
When you run an A/B test, one variant almost always ends up with a higher conversion rate. The real question is whether that gap is a genuine difference or just random luck from a small sample. Statistical significance is the answer to that question: it tells you how likely it is that you would see a gap this big by pure chance if the two variants were actually identical.
This calculator runs a two-proportion z-test, the standard method for comparing two conversion rates, and turns it into a plain-English verdict: significant, or not significant yet.
How to use this A/B test calculator
- Enter the number of visitors and conversions for Variant A (your control) and Variant B (the new version you are testing).
- A "conversion" is whatever you count as a success: a sale, a sign-up, a click, a download. Use the same definition for both variants.
- Pick a confidence level. 95% is the standard choice for most tests.
- Read the result. You get each variant's conversion rate, the relative uplift, the p-value, and whether the winner is statistically significant.
Everything is calculated live in your browser as you type. Nothing is uploaded.
Reading the result
- Conversion rate is conversions divided by visitors, shown as a percentage.
- Relative uplift is how much better (or worse) Variant B did compared to Variant A. A +25% uplift means B converted a quarter more often than A.
- p-value is the probability of seeing a difference this large if the two variants were really the same. Lower is stronger. At a 95% confidence level, a p-value below 0.05 counts as significant.
- Confidence is simply 1 minus the p-value, shown as a percentage, so you can see how close a "not significant" result is to the line.
Why significance matters before you pick a winner
Calling a winner too early is the most common A/B testing mistake. Early in a test, conversion rates swing wildly because a handful of visitors move the numbers a lot. A result that looks like a clear win on day one often evaporates by day seven. A significance test protects you from shipping a change that never actually beat the control, and from killing a variant that just needed more traffic.
If your result is not significant yet, you usually have three options: keep the test running to gather more visitors, accept that the two variants perform about the same, or lower the bar (a 90% confidence level is more sensitive, but riskier).
Is my data private?
Yes. This calculator runs 100% in your browser. Your visitor and conversion numbers are never uploaded, stored or logged, so you can safely test results for unreleased pages and private experiments.
Frequently asked questions
What does statistical significance mean in an A/B test?
It means the difference between your two variants is unlikely to be caused by random chance. At a 95% confidence level, a statistically significant result has less than a 5% probability of being a fluke, so you can trust that one variant really did perform differently.
What is a good p-value for an A/B test?
A p-value below 0.05 is the standard threshold and matches a 95% confidence level. The lower the p-value, the stronger the evidence that the difference is real. If you need to be very strict, use a 99% confidence level, which requires a p-value below 0.01.
How many visitors do I need before the result is significant?
There is no single number. It depends on your baseline conversion rate and how big the difference between variants is. Small differences need much larger samples. Keep entering your running totals in this calculator, and it will tell you the moment the result crosses your confidence threshold.
Which method does this calculator use?
It uses a two-proportion z-test with pooled variance and a two-tailed p-value, the standard test for comparing two conversion rates. This is the same core method built into most professional A/B testing tools.
What is the difference between a 90%, 95% and 99% confidence level?
A higher confidence level makes the test stricter. At 99% you need very strong evidence before a result counts as significant, which reduces false positives but needs more data. At 90% the test is more sensitive and calls winners sooner, but with a higher risk of being wrong. 95% is the common middle ground.
Is this A/B test calculator free and private?
Yes. It is completely free, needs no sign-up, and runs entirely in your browser. Your visitor and conversion numbers are never sent to a server or stored anywhere.
Built and hosted with Toolcrow.
Toolcrow turns the little tools you make with AI into working, shareable links, private by default.
Try Toolcrow