A/B testing hub

A/B testing hub

A/B testing statistics done right

Address the leading reason for experimentation program failure - poor use of statistical methods.

Up to 80% faster tests. Unrivaled statistical rigor. Experiments planned for optimal business returns.

Try it now

Use all our tools for 30 days! No credit card required.

Enable your A/B tests

  • 20-80% faster testing through flexible sequential analysis
  • Stop early for efficacy or futility (lack of positive effect)
  • Tests with optimal business returns using our ROI-optimal planner
  • Support for A/B and A/B/N tests
  • Analyze binomial and continuous metrics alike
  • Support for non-inferiority test designs
  • Robust false positive and false negative error control
  • Advanced confidence intervals and point estimates
  • Automate data reporting and analysis through our APIs

5 reasons to use the A/B Testing Hub

Test duration improvement by using AGILE

Significantly faster A/B tests

Get an edge on your competitors and amaze your clients with the significantly faster AGILE tests.

Benefit from winners earlier and cut losers as quickly as possible and increase the business returns from your A/B testing program. Depending on the actual difference between the tested variant(s) and control is, the gains from using the sequential evaluation methodology of the A/B testing hub (AGILE) can range from the moderate 20%, to a whopping 80% faster!. Real-life data show an average benefit of 26% for clients of Analytics Toolkit.

Get the benefits of sequential testing without sacrificing the statistical error control which would otherwise be compromised by what is known as peeking. The embedded calculation of statistical significance, confidence intervals, and other statistics are as rigorous as they come.

Evaluate data on the go

Run flexible sequential A/B and A/B/n tests, periodically evaluating the data and making decisions to stop or continue testing.

Have you found yourself having to compromise between running a proper A/B test and pleasing a client or HiPPo who demands to act on the results right this minute? Stopping a classical statistical significance test based on interim data would break the statistical assumptions and invalidate its error guarantees.

No more compromises! Get the flexibility and efficiency your A/B tests deserve with AGILE sequential testing. Periodically evaluate A/B and A/B/n tests with stopping boundaries for efficacy and futility which guide you when to call a test a winner or to cut a non-promising test short.

Early stopping of an experiment for efficacy
Statistical error guarantees

Robust statistics you can trust

Unrivaled statistical rigor means you can trust your A/B test results. Sequential evaluation tests using AGILE are based on the same methodology which was employed in the COVID vaccine trials - easily the most scrutinized experiments in human history. P-values, confidence intervals, and lift estimates are adjusted uniquely so they account for the sequential nature of the test. The statistical estimates have been checked through millions of simulations.

The A/B testing hub offers an integrated experience of planning and analyzing both single evaluation and sequential evaluation tests. This helps avoid many of the common pitfalls encountered in applying statistics in A/B testing. The instrument chooses the correct statistical methods and adjustments for you and applies them automatically giving you the much-needed confidence in the results.

Plan tests with optimal business impact

Plan ROI-optimal tests and remove the guesswork in choosing how long to run a test and what confidence to require.

Coming up with the right test plan can be a daunting task while the costs associated with getting a test plan wrong can be significant. Test for too long and you run the risk of exposing too many users to a variant which loses money, or to unnecessarily delay the implementation of a winner which means lost business revenue. Test for too short and risk a high probability of a false negative and therefore failure to implement actual winners. Similar issues accompany the choice of the confidence required to call a test’s outcome.

The A/B testing hub helps you plan tests that deliver optimal business returns without going through the typical difficulties. Given relevant business details related to the test at hand the system searches through the space of possible combinations of test durations and confidence levels until it arrives at a nearly optimal one.

ROI-optimal A/B test example
Test data sources

Automate data extraction and reporting

There are three ways to get data into this A/B testing statistical calculator. First, you can manually enter summary data. A tedious process for sure, even with file uploads, but it comes with the benefit that it can be used with any testing system.

The second data input method supported by our A/B testing calculator is to use our simple Data API. It can work with any system for storing A/B test data, including custom-built ones, as long as you have the capability to code the API endpoint. It is nicely complitmented by our powerful Reporting API.

Third, if your AB testing delivery software syncs data to MixPanel® our calculator can automatically extract it using the MixPanel® API. This happens in a completely automated fashion, and you will get email notifications when a test is ready to be stopped.

A/B testing tools

Take your A/B testing to the next level

Experience the power and utility of our A/B testing hub and all tools during our 30-day free trial. No credit card required.

Start free trial

See what our clients have to say

A tool with real value if you know how to use it. Might seem intimidating at the beginning, but it tries to provide flexibility, accuracy and optimization in your statistical analysis instead of over-simplification.

Read full testimonial

Arnau Vendrell
Co-Founder, iSocialWeb Marketing

At Cro Metrics, we've run tens of thousands of experiments for leading brands. Analytics Toolkit and the AGILE test calculator represent a step above and beyond every one of the other tools we've used.

Read full testimonial

Ryan Lucht
Director of Strategy, Cro Metrics