Sample Size Confidence Lift Calculator

Estimate how confidence improves with larger sample sizes.

%
%

Quick Facts

Sample
Scale
Larger samples reduce error
Variance
Impact
Lower variance improves confidence
Margin
Focus
Margin shrink drives lift
Decision Metric
Lift
Confidence gain

Your Results

Calculated
New Margin
-
Adjusted margin of error
Margin Lift
-
Percent improvement
Confidence Lift
-
Estimated lift
Sample Delta
-
Added samples

Lift Plan

Your defaults show a strong confidence lift.

What it is and when to use it

This calculator answers a specific planning question: if I grow my sample from its current size to a target size (and optionally improve data quality), how much smaller does my margin of error get, and is that lift worth the extra data collection? It is built around the square-root relationship between sample size and margin of error that governs most survey and A/B test precision.

Use it when deciding whether to extend a survey, run an experiment longer, or add participants to a study. Rather than guessing whether doubling your sample is worth the cost and time, it quantifies the exact percentage improvement in margin of error you'd get, plus how that compares to a "lift goal" you set for the project.

New Margin = Current Margin × √(n1 / n2) × Variance Factor
n1: current sample size.
n2: target (larger) sample size.
Variance Factor: a multiplier below 1.0 if you also expect to reduce measurement noise or variability (1.0 means no change).
Margin Lift: (Current Margin − New Margin) / Current Margin, expressed as a percent reduction in margin of error.

Worked example

Starting with a current sample of 200 and a margin of error of 3.5%, growing the sample to 400 (with no variance change, factor 1.0): New Margin = 3.5 × √(200/400) × 1.0 = 3.5 × √0.5 = 3.5 × 0.7071 ≈ 2.47%. Margin Lift = (3.5 − 2.47) / 3.5 × 100 ≈ 29.29% — matching the calculator's default "Strong Lift" result. Notice that doubling the sample size did not halve the margin of error; it only reduced it by about 29%, because margin of error shrinks with the square root of sample size, not linearly.

Common mistakes and how to interpret the result

  • Expecting doubling the sample to halve the margin of error. Because of the square-root relationship, doubling sample size only reduces margin of error by about 29%; to actually halve the margin, you need roughly 4 times the sample size.
  • Ignoring the variance factor's leverage. Reducing measurement variance (through better instrumentation or a more homogeneous sample) multiplies directly with the sample-size effect, so a 0.8x variance factor combined with a larger sample compounds the improvement rather than just adding to it.
  • Chasing a lift goal without weighing collection cost. A 35%+ "Major Lift" often requires a very large jump in sample size; compare the marginal cost of that additional data collection against how much the extra precision actually matters for your decision.
  • Treating margin lift and statistical significance as the same thing. A smaller margin of error makes your estimate more precise, but it does not by itself tell you whether an observed effect or difference is statistically significant — that depends on the specific hypothesis test being run.

Frequently Asked Questions

Why doesn't doubling the sample size cut the margin of error in half?
Margin of error is proportional to 1 over the square root of sample size, not 1 over sample size directly. Doubling n multiplies the margin by 1/√2 (about 0.707), a 29% reduction, not a 50% reduction. To actually halve the margin of error, sample size needs to increase fourfold.
What does the variance factor represent in practice?
It represents an independent improvement in data quality, separate from sample size, such as using a more precise measurement instrument, reducing sampling bias, or targeting a more homogeneous population. A factor of 0.9 means you expect 10% less variability in the underlying measurements, which reduces the margin of error on top of any sample-size increase.
What is a reasonable lift goal to set?
A lift goal of 10-20% is common for most incremental studies, representing a meaningful but achievable improvement in precision without requiring an extreme sample-size increase. Goals above 35% usually require either a very large jump in sample size or a substantial variance reduction achieved through better measurement, not sample size alone.
Does a bigger sample always justify the cost?
Not necessarily. Because of the square-root relationship, the precision gained per additional respondent shrinks as your sample grows, so going from 200 to 400 gives a bigger relative improvement than going from 2,000 to 2,200 does, even though both add 200 responses. Weigh the marginal lift against the marginal cost of collecting more data before committing to a larger target.

Related calculators