A/B test sample-size calculator.
Before you split an SMS or WhatsApp audience, work out how many recipients each variant needs so the result is statistically valid — not noise. Enter your baseline conversion rate, the smallest lift worth detecting, your confidence and power, and get the required sample per variant and in total. Everything runs in your browser.
The math is standard and exact; the reliability depends on your assumptions.
This uses the textbook two-proportion sample-size formula. It assumes each recipient converts independently, that you fix the sample size in advance and read the result once at the end (a fixed-horizon test), and that the baseline you enter is representative. We invent no benchmarks — you supply the baseline. Repeatedly checking the test and stopping as soon as it looks significant (“peeking”) inflates false positives well beyond your stated confidence; size the test, then let it run.
Your test
Your current control conversion rate — clicks, replies, purchases, opt-ins, whatever you count as a conversion.
Relative: e.g. +10 means a 10% improvement on the baseline — from 5% to 5.5%.
Confidence 95% → α = 0.05, the tolerated false-positive rate. Higher confidence needs a larger sample. The chance of detecting a real lift of this size. 80% is common; 90% is stricter and needs more recipients.
Two-tailed tests whether the variant differs at all (recommended). One-tailed only tests for improvement and needs a smaller sample.
Including the control. 2 = classic A/B. More variants apply a Bonferroni correction and need more audience.
Total contacts you can enrol per day across all variants. Used to estimate how long the test will take.
Required sample
Two-proportion test · fixed sample size decided in advance · read once at the end · recipients convert independently.
n per variant is computed from the standard two-proportion formula with your exact inputs:
p̄ = (0.0500 + 0.0550) / 2 = 0.0525
z(α) = 1.9600 z(β) = 0.8416
n = (z(α)·√(2·p̄·(1−p̄)) + z(β)·√(p₁(1−p₁)+p₂(1−p₂)))² / (p₂−p₁)²
n per variant = 31,234 ×2 variants = 62,468On peekingThis sample size is only valid if you decide it up front and evaluate the test once, at the end. Checking daily and stopping the moment it looks significant can multiply your real false-positive rate several times over. If you need to monitor mid-flight, use a sequential or Bayesian method designed for it.
Enter a baseline between 0 and 100% and a minimum detectable effect greater than zero to see the required sample.
The short answer
How many people do you need in an A/B test?
Enough that a real difference of the size you care about would show up above the random noise of conversion. The required sample per variant grows when your baseline conversion rate is low, when the lift you want to detect is small, and when you demand higher confidence or power. It falls when the effect is large or you accept more risk. This calculator applies the standard two-proportion sample-size formula so your SMS or WhatsApp test is powered to actually detect the lift — instead of ending inconclusive or, worse, declaring a winner that was really chance.
How the calculation works
You give two conversion rates: your baseline (p₁) and the target you want to be able to detect (p₂), derived from your minimum detectable effect. From your confidence level the calculator finds z(α) — the number of standard deviations that bounds the false-positive rate — and from your power it finds z(β). These plug into the two-proportion formula below to return the minimum recipients per variant. The total is that figure multiplied by the number of variants. z-values are computed on the fly with a rational-approximation inverse-normal function, so any confidence or power works, not just the common presets.
The formula
With p̄ = (p₁ + p₂) / 2, the required sample per variant is:
| Pooled rate | p̄ = (p₁ + p₂) / 2 |
|---|---|
| z(α) two-tailed | Φ⁻¹(1 − α/2) |
| z(α) one-tailed | Φ⁻¹(1 − α) |
| z(β) | Φ⁻¹(power) |
| n per variant | (z(α)·√(2·p̄·(1−p̄)) + z(β)·√(p₁(1−p₁)+p₂(1−p₂)))² / (p₂ − p₁)² |
| Total audience | n × number of variants |
Common z-values
For reference, the presets resolve to these standard critical values (computed, not looked up):
| Level | z(α) two-tailed | z(β) power |
|---|---|---|
| 90% | 1.645 | 1.282 |
| 95% | 1.960 | 1.645 |
| 99% | 2.576 | — |
| 80% power | — | 0.842 |
z-values come from Acklam's rational-approximation inverse of the standard normal CDF (relative error below 1.15×10⁻⁹), so arbitrary confidence and power values are supported.
Frequently asked questions
What is minimum detectable effect (MDE), absolute vs relative?
MDE is the smallest lift you want the test to be able to catch. Absolute is measured in percentage points: from a 5% baseline, +1 pp means detecting a move to 6%. Relative is a percentage of the baseline: +10% relative means detecting a move from 5% to 5.5%. Smaller MDEs are harder to prove and require much larger audiences, so set it to the smallest lift that would actually change your decision.
What do confidence and power actually mean?
Confidence (1 − α) bounds false positives — at 95% confidence you accept a 5% chance of calling a difference real when it is not. Power (1 − β) is the chance of detecting a true lift of your chosen size; at 80% power you would miss a real effect one time in five. Raising either makes the test stricter and increases the required sample.
One-tailed or two-tailed?
Use two-tailed unless you have a strong reason not to. Two-tailed asks “is the variant different?” and protects you if the variant is actually worse. One-tailed asks only “is the variant better?” — it needs a smaller sample but ignores the possibility of harm, which is rarely what you want in a live campaign.
Can I test more than two variants?
Yes. Set the number of variants (including the control). With more than two, the calculator applies a Bonferroni correction — it divides α across the comparisons against the control so the overall false-positive rate stays within your confidence level. That correction is conservative and increases the audience each variant needs.
Why can't I just stop the test once it looks significant?
Because this formula assumes a fixed sample size read once at the end. Every time you peek and could stop, you get another chance to be fooled by noise, so repeated peeking can multiply your true false-positive rate far beyond the 5% you thought you set. Decide the sample up front and let it run. If you must monitor continuously, use a sequential or Bayesian design built for it.
Is this accurate for SMS and WhatsApp campaigns?
The statistics are standard and exact for comparing two conversion proportions, which is exactly what most messaging A/B tests do — comparing click, reply, opt-in or purchase rates between variants. Accuracy depends on your inputs: a representative baseline, independent recipients, and one clean read at the end. The tool invents no benchmarks; you provide the baseline and the lift that matters to you.
Run the winning campaign at scale.
Size the test here, then send it on infrastructure built for deliverability — SMS, WhatsApp and voice across our direct carrier network. Talk to our team about routing, throughput and rates.