A/B testing

Sample size calculator

Estimate the sample size required for an A/B test based on the minimum effect of interest and the required confidence threshold.

Try it now

Use all our tools for 14 days! No credit card required.

A/B test sample size calculator

  • Works with both binomial and non-binomial metrics
  • Sample size estimation for tests with multiple variants (A/B/N tests)
  • Supports non-inferiority A/B test designs
  • Calculates sample sizes required for analysis of statistics for percent change (lift)
  • Excellent documentation and presentation of calculation results

Sample size calculations done right

Support for non-binomial metrics

Support for non-binomial metrics

On top of binomial metrics such as conversion rates, the calculator supports sample size calculation for tests with a primary KPI which is a non-binomial (continuous) metrics such as average revenue per user, average sessions per user, average session duration, average order value, etc. out of the box.

An easy-to-use interface allows you to upload a file with your non-binomial data and all relevant statistics are calculated from it.

Properly plan A/B/N tests

Tests with more than one variant versus a control need to be analyzed with special methods that account for the multiple comparisons problem that otherwise arises. The appropriate p-value and confidence interval correction is the Dunnett's correction which means that a sample size calculation should take these corrections into account.

This tool does just that so your A/B/N tests have the right sample size requirement and therefore duration to hit your desired power level.

Early stopping of an experiment for efficacy
Non-inferiority tests

Plan non-inferiority tests

If you have planned a test with a non-inferiority hypothesis it needs to be analyzed as such. This significance calculator has native support for non-inferiority tests.

Simply enter the non-inferiority margin used when planning the test and the tool will take care of the rest.

Sample size for lift estimation

Calculating p-values and confidence intervals for percent change (% lift) requires that this is taken into account when considering the required sample size. This tool uses a proprietary formula to make these calculations.

Diagnostic checks
Statistical documentation

Excellent documentation

This tool is documented in detail, with information about the statistical assumptions being displayed below each computation. Inline help information assists in using it correctly and makes the statistical jargon accessible by presenting with terms familiar to professionals engaged in online A/B testing.

A/B testing tools

Take your A/B testing to the next level

Experience the power and utility of our A/B testing hub and all tools during our 14-day free trial. No credit card required.

Start free trial

See what our clients have to say

A tool with real value if you know how to use it. Might seem intimidating at the beginning, but it tries to provide flexibility, accuracy and optimization in your statistical analysis instead of over-simplification.

Read full testimonial

Arnau Vendrell
Co-Founder, iSocialWeb Marketing

At Cro Metrics, we've run tens of thousands of experiments for leading brands. Analytics Toolkit and the AGILE test calculator represent a step above and beyond every one of the other tools we've used.

Read full testimonial

Ryan Lucht
Director of Strategy, Cro Metrics