A
-
A/B Test Calculator
Compare two conversion rates or two means with a named frequentist test, effect size, and confidence interval.
At a glance
- Computes
- Two-proportion z-test or Welch t-test, the difference, a confidence interval, and effect size.
- You supply
- Conversions and visitors for both variants, or n, mean, and SD for both.
- Use when
- Two independent groups, one outcome, and a sample size fixed before the test ran.
- Not for
- Stopping a running test the moment it looks significant. Power and Sample Size Calculator
95 percent confidence interval (proportion-point difference); reference at No difference
p-value
Enter values
Two-sided is the default.
B
-
| Metric | Value |
|---|
z = (pB - pA) / sqrt(pPool(1 - pPool)(1/nA + 1/nB))
How?
How this is calculated
Conversion mode uses a two-proportion z-test for the p-value and a Newcombe Wilson interval for B minus A. Mean mode uses Welch's two-sample t-test, so equal variances are not assumed.
Formula: z = (pB - pA) / sqrt(pPool(1 - pPool)(1/nA + 1/nB))
Sources
- statsmodels proportions_ztest. statsmodels. Retrieved .
- statsmodels confint_proportions_2indep. statsmodels. Retrieved .
Method last reviewed