How many visitors does an A/B test need?
At a 2% conversion rate, detecting a 20% relative lift (2.0% to 2.4%) takes about 21,106 visitors per version at 95% confidence and 80% power. Smaller lifts and lower conversion rates need far more. The table below gives the number for common cases.
A/B test sample size calculator
About 21,106 visitors per version, 42,212 in total: about 3.7 weeks at 50,000 visitors a month.
Two-sided test, 95% confidence, 80% power, one control and one challenger.
Visitors needed per version
Two-sided test, 95% confidence, 80% power, one control and one challenger. Lift is relative (a 20% lift on 2% is 2.4%).
| Current conversion rate | Detect +10% | Detect +20% | Detect +30% | Detect +50% |
|---|---|---|---|---|
| 1% | 163,092 | 42,691 | 19,824 | 7,747 |
| 2% | 80,679 | 21,106 | 9,795 | 3,823 |
| 3% | 53,208 | 13,911 | 6,452 | 2,515 |
| 5% | 31,231 | 8,155 | 3,778 | 1,468 |
| 10% | 14,749 | 3,839 | 1,772 | 683 |
Formula: n = (zα/2 + zβ)² × (p₁(1−p₁) + p₂(1−p₂)) / (p₂ − p₁)², with zα/2 = 1.96 and zβ = 0.84. Computed by Luria, October 2026; reuse with a link.
How long that takes
Two versions need twice the number above. A store with 50,000 visitors a month on the tested page and a 2% conversion rate needs about 3.7 weeks to detect a 20% lift, and 14.0 weeks for a 10% lift. At 10,000 visitors a month, the 20% case takes about 18.3 weeks.
Run every test for at least one full week, even when the number is reached sooner, so weekdays and weekends are both counted.
How to need fewer visitors
- Test bigger changes. A new offer or headline moves conversion more than a button colour, and big lifts need far fewer visits.
- Test where the traffic is. Product pages and the home page, not a page with 300 visits a month.
- Use a bandit. Moving traffic toward the winner cuts the cost of losing versions while the test runs (bandit vs A/B test).
- Use a closer goal only if it tracks the real one; add-to-cart can mislead.
How Luria handles sample size
Luria will not keep a change before 200 visits per version, and keeps it only when it is 95% sure the new version beats your page. With a large lift that can happen near the floor; with a small one it waits much longer. Its guarantee needs 5,000+ tracked visits on the tested pages over 60 days.
Questions
What is the minimum sample size for an A/B test?
There is no single minimum; it depends on your conversion rate and the lift you want to detect. At a 3% conversion rate and a 20% lift it is about 13,911 visitors per version. A few hundred visits per version only detects very large differences.
Can I stop an A/B test early when it looks significant?
Not with a fixed-sample test: checking every day and stopping at the first significant result inflates false winners. Bayesian and bandit methods are built to be checked as they go.