A/B testing

Sample ratio mismatch calculator

Easily check your A/B testing data for the presence of sample ratio mismatch which is an indicator of issues with grave consequences for a test's trustworthiness.

Try it now

Use all our tools for 14 days! No credit card required.

SRM calculator video demo
Start free trial

Easy-to-use A/B test SRM checker

  • Works for both A/B and A/B/N tests
  • Supports equal and unequal allocation between test groups
  • Robust statistical calculations (Chi-square test of goodness-of-fit)
  • Enter target allocation ratios as proportions or percentages
  • Information to guide you while using the SRM calculator and while interpreting the check result
  • Built with online A/B testing in mind

Use-case example

Detecting a sample ratio mismatch is as easy as entering the number of users observed in each test group and pressing the "Calculate" button. If the target allocation was unequal for some reason (it is typically suboptimal), then you will also need to enter the target ratios as proportions (e.g. 0.2, 0.8) or percentages (e.g. 20%, 80%).

The result of the check is displayed and the presence or lack of an SRM will be indicated based on a p-value threshold of 0.001 (1 in 1,000). An observed p-value lower than 0.001 should be solid ground for suspecting the trustworthiness of the test outcome.

Investigating the cause of the issue can be difficult as an SRM can be an indication of failure(s) at dozens of points. The most general places to look for the cause of the problem are: the randomization process, the variant delivery system, the tracking setup, the data retrieval process.

Example outcome of an SRM check
A/B testing tools

Take your A/B testing to the next level

Experience the power and utility of our A/B testing hub and all tools during our 14-day free trial. No credit card required.

Start free trial

See what our clients have to say

A tool with real value if you know how to use it. Might seem intimidating at the beginning, but it tries to provide flexibility, accuracy and optimization in your statistical analysis instead of over-simplification.

Read full testimonial

Arnau Vendrell
Co-Founder, iSocialWeb Marketing

At Cro Metrics, we've run tens of thousands of experiments for leading brands. Analytics Toolkit and the AGILE test calculator represent a step above and beyond every one of the other tools we've used.

Read full testimonial

Ryan Lucht
Director of Strategy, Cro Metrics