A/B testing hub
A/B testing statistics done right
Address the leading reason for experimentation program failure - poor use of statistical methods.
Up to 80% faster tests. Unrivaled statistical rigor. Experiments planned for optimal business returns.Try it now
Use all our tools for 14 days! No credit card required.
Enable your A/B tests
- 20-80% faster testing through flexible sequential analysis
- Stop early for efficacy or futility (lack of positive effect)
- Tests with optimal business returns using our ROI-optimal planner
- Support for A/B and A/B/N tests
- Analyze binomial and continuous metrics alike
- Support for non-inferiority test designs
- Robust false positive and false negative error control
- Advanced confidence intervals and point estimates
- Automate data reporting and analysis through our APIs
5 reasons to use the A/B Testing Hub
Significantly faster A/B tests
Get an edge on your competitors and amaze your clients with the significantly faster AGILE tests.
Benefit from winners earlier and cut losers as quickly as possible and increase the business returns from your A/B testing program. Depending on the actual difference between the tested variant(s) and control is, the gains from using the sequential evaluation methodology of the A/B testing hub (AGILE) can range from the moderate 20%, to a whopping 80% faster!. Real-life data show an average benefit of 26% for clients of Analytics Toolkit.
Get the benefits of sequential testing without sacrificing the statistical error control which would otherwise be compromised by what is known as peeking. The embedded calculation of statistical significance, confidence intervals, and other statistics are as rigorous as they come.
Evaluate data on the go
Run flexible sequential A/B and A/B/n tests, periodically evaluating the data and making decisions to stop or continue testing.
Have you found yourself having to compromise between running a proper A/B test and pleasing a client or HiPPo who demands to act on the results right this minute? Stopping a classical statistical significance test based on interim data would break the statistical assumptions and invalidate its error guarantees.
No more compromises! Get the flexibility and efficiency your A/B tests deserve with AGILE sequential testing. Periodically evaluate A/B and A/B/n tests with stopping boundaries for efficacy and futility which guide you when to call a test a winner or to cut a non-promising test short.
Robust statistics you can trust
Unrivaled statistical rigor means you can trust your A/B test results. Sequential evaluation tests using AGILE are based on the same methodology which was employed in the COVID vaccine trials - easily the most scrutinized experiments in human history. P-values, confidence intervals, and lift estimates are adjusted uniquely so they account for the sequential nature of the test. The statistical estimates have been checked through millions of simulations.
The A/B testing hub offers an integrated experience of planning and analyzing both single evaluation and sequential evaluation tests. This helps avoid many of the common pitfalls encountered in applying statistics in A/B testing. The instrument chooses the correct statistical methods and adjustments for you and applies them automatically giving you the much-needed confidence in the results.
Plan tests with optimal business impact
Plan ROI-optimal tests and remove the guesswork in choosing how long to run a test and what confidence to require.
Coming up with the right test plan can be a daunting task while the costs associated with getting a test plan wrong can be significant. Test for too long and you run the risk of exposing too many users to a variant which loses money, or to unnecessarily delay the implementation of a winner which means lost business revenue. Test for too short and risk a high probability of a false negative and therefore failure to implement actual winners. Similar issues accompany the choice of the confidence required to call a test’s outcome.
The A/B testing hub helps you plan tests that deliver optimal business returns without going through the typical difficulties. Given relevant business details related to the test at hand the system searches through the space of possible combinations of test durations and confidence levels until it arrives at a nearly optimal one.
Automate data extraction and reporting
There are three ways to get data into this A/B testing statistical calculator. First, you can manually enter summary data. A tedious process for sure, even with file uploads, but it comes with the benefit that it can be used with any testing system.
The second data input method supported by our A/B testing calculator is to use our simple Data API. It can work with any system for storing A/B test data, including custom-built ones, as long as you have the capability to code the API endpoint. It is nicely complitmented by our powerful Reporting API.
Third, if your AB testing delivery software syncs data to MixPanel® our calculator can automatically extract it using the MixPanel® API. This happens in a completely automated fashion, and you will get email notifications when a test is ready to be stopped.
Take your A/B testing to the next level
Experience the power and utility of our A/B testing hub and all tools during our 14-day free trial. No credit card required.Start free trial
See what our clients have to say
Co-Founder, iSocialWeb Marketing
Director of Strategy, Cro Metrics