A/B testing
Sample ratio mismatch calculator
Easily check your A/B testing data for the presence of sample ratio mismatch which is an indicator of issues with grave consequences for a test's trustworthiness.
Try it nowUse all our tools for 30 days! No credit card required.
Easy-to-use A/B test SRM checker
- Works for both A/B and A/B/N tests
- Supports equal and unequal allocation between test groups
- Robust statistical calculations (Chi-square test of goodness-of-fit)
- Enter target allocation ratios as proportions or percentages
- Information to guide you while using the SRM calculator and while interpreting the check result
- Built with online A/B testing in mind
Use-case example
Detecting a sample ratio mismatch is as easy as entering the number of users observed in each test group and pressing the "Calculate" button. If the target allocation was unequal for some reason (it is typically suboptimal), then you will also need to enter the target ratios as proportions (e.g. 0.2, 0.8) or percentages (e.g. 20%, 80%).
The result of the check is displayed and the presence or lack of an SRM will be indicated based on a p-value threshold of 0.001 (1 in 1,000). An observed p-value lower than 0.001 should be solid ground for suspecting the trustworthiness of the test outcome.
Investigating the cause of the issue can be difficult as an SRM can be an indication of failure(s) at dozens of points. The most general places to look for the cause of the problem are: the randomization process, the variant delivery system, the tracking setup, the data retrieval process.
Take your A/B testing to the next level
Experience the power and utility of our A/B testing hub and all tools during our 30-day free trial. No credit card required.
Start free trialSee what our clients have to say
Sim Lenz
Director of Experimentation at Conversion.com
Ryan Lucht
Director of Strategy, Cro Metrics