AGILE A/B Testing Calculator

An advanced statistical calculator for A/B & MVT tests. Up to 80% faster A/B tests, thanks to the AGILE statistical method! Flexibile data monitoring with control of false positives and false negatives. Ideal for testing in conversion rate optimization, landing page optimization, e-mail marketing, SEO, PPC. Use it now: Sign In or Start Free Trial.

Maximize the ROI of your A/B tests!

Differentiate between noise and data, errors and real results while running tests as quickly as possible! With this A/B testing calculator you get:

  • 20-80% faster A/B & MVT tests versus equivalent fixed sample designs
  • Flexibility to perform interim analyses and evaluations
  • Early stopping for both efficacy & futility (lack of positive effect)
  • Control of the tolerance for false positive and false negatives
  • Automatic retrieval of test data from Google Analytics®
  • No more illusory A/B testing results!

AGILE A/B Testing Calculator

Close [X]



AGILE A/B Testing Calculator - Video Transcript

Wouldn’t you like to run A/B tests several times faster than current best-practice methods allow?

Don’t you want to be flexible in monitoring your tests as they go, without biasing your statistics?

Do you want to identify and stop futile tests quickly so you can focus on more promising ones?

You can now do all the above with the help of the AGILE A/B Testing Calculator.

It is based on the AGILE Statistical Method for conducting A/B tests in Conversion Rate Optimization, Landing Page Optimization, and other areas where scientific experiments can be applied to drive better business results.

The AGILE A/B Testing Calculator allows you to run A/B & multivariate tests 20-80% faster than traditional fixed sample designs and gives you the flexibility to monitor results as they accrue. It is based on proven and time-tested statistical procedures used in medical research.

It can help identify better versions quicker so you can implement them sooner, improving your bottom-line. It also helps detect unpromising tests quickly allowing you to cut revenue loses and the time committed to them.

All the while the tool helps you maintain rigorous statistical control over false positives and false negatives.

The AGILE calculator is designed to be used alongside an existing tool for delivering A/B or MVT tests. The test data is then entered manually or syndicated automatically, for example from your Google Analytics account.

To experience AGILE sign up for a non-obligatory free trial and explore our free white papers and articles on AGILE A/B testing. Start running more efficient A/B tests today!

Close [X]

5 reasons to use the AGILE A/B testing calculator now



#1. Faster A/B and multivariate tests

Get an edge on your competitors and amaze your clients with significantly faster tests. Depending on what the actual difference between your tested variant(s) and control is, the gains can range from the moderate 20%, to a whopping 80%!

You can run tests and make decisions 20-80% faster than traditional best-practice methods, without sacrificing the rigourorous error control.

Comparing the performance of sequential AGILE A/B tests with classical fixed-sample statistical signifiance tests




An AGILE AB test, stopped early for efficacy (winner declared)

An AGILE A/B test, stopped early for efficacy (winner declared)

#2. Flexible monitoring & decision making

Do you have to compromise between running a proper test and pleasing a client or HiPPo who demands to see and act on the results? Like, right now! Stopping a classical statistical significance test based on interim data would result in breaking the statistical assumptions of the test and is thus a big no-no.

No more compromise! Run flexible sequential A/B and A/B/n tests, periodically evaluating the data and making decisions to stop or continue testing. The stopping boundaries for efficacy and futility are based on your desired statistical significance and statistical power so you always have guidance on when to call a test a winner or a loser.





#3. Early stopping for efficacy & futility

Stopping rules (also Decision rules) are what allow sequential monitoring of test results and they guide your decision to stop the test. When running A/B tests, it is often difficult to tell when to stop the test, especially of the results seem unpromissing, but that is not an issue with our statistical calculator.

The efficacy stopping boundary will tell you when to stop a test and declare a statistically significant winner, while the futility stopping boundary will tell you when there is little chance for the test to turn out a winner. Thus AGILE allows you to minimize budgets wasted chasing unpromissing variants and quickly find real winners in order to capitalize on them, improving the ROI of your A/B testing efforts as a whole!

Decision boundaries of a sequential A/B test




Correct and incorrect conclusions (errors) when interpreting A/B test data

Correct and incorrect conclusions (errors) when interpreting A/B test data

#4. False positive & false negative control

We emply best-practice statistical procedures from medical testing to allow flexible monitoring without sacrificing error-control.

Adjustments to the statistical significance are applied for multivariate testing, while sequential procedures adjust for multiple testing. Non-inferiority designs are supported as well.





#5. Automatic A/B test data retrieval

There are two ways to get data into this A/B testing statistical calculator. One is manually entering of summary data. It is a tedious process, even with our bulk input method, but it can be used with any A/B testing system.

Secondly, if your AB testing delivery software syncs data to Google Analytics® (many industry leaders do), our calculator can automatically extract it using the Google Analytics® API. This happens in a completely automated fashion and you will get e-mail notifications when a test is ready to be stopped.

Automatic test data retrieval

Close [X]



AGILE A/B Testing Calculator - Video Transcript

Wouldn’t you like to run A/B tests several times faster than current best-practice methods allow?

Don’t you want to be flexible in monitoring your tests as they go, without biasing your statistics?

Do you want to identify and stop futile tests quickly so you can focus on more promising ones?

You can now do all the above with the help of the AGILE A/B Testing Calculator.

It is based on the AGILE Statistical Method for conducting A/B tests in Conversion Rate Optimization, Landing Page Optimization, and other areas where scientific experiments can be applied to drive better business results.

The AGILE A/B Testing Calculator allows you to run A/B & multivariate tests 20-80% faster than traditional fixed sample designs and gives you the flexibility to monitor results as they accrue. It is based on proven and time-tested statistical procedures used in medical research.

It can help identify better versions quicker so you can implement them sooner, improving your bottom-line. It also helps detect unpromising tests quickly allowing you to cut revenue loses and the time committed to them.

All the while the tool helps you maintain rigorous statistical control over false positives and false negatives.

The AGILE calculator is designed to be used alongside an existing tool for delivering A/B or MVT tests. The test data is then entered manually or syndicated automatically, for example from your Google Analytics account.

To experience AGILE sign up for a non-obligatory free trial and explore our free white papers and articles on AGILE A/B testing. Start running more efficient A/B tests today!

Close [X]



Need more information?

How to use this statistical calculator?

The tool is designed for usage in conversion rate optimization (CRO), landing page optimization (LPO), e-mail marketing, SEO, and PPC experts in mind. The basic flow is such: design a test, run the test and input data as it accrues, and finally analyse results. We support automatic data retrieval of A/B test data from Google Analytics, given that you use custom dimensions to differentiate between users or sessions exposed to the different test variants.

The tool is quite intuitive to use and has plenty of contextual help on each screen. This post on designing an AGILE A/B test is a good starting point for understanding the different parameters and trade-offs in using the calculator's options.

Educational Resources

Check out our Free White Paper, with a detaled description of the AGILE statistical approach to A/B testing, complete with simulations.

Posts on AGILE A/B Testing on our blog. Practical advice on applying the AGILE method, as well as details on the major fundamentals. Make sure to also check our posts on A/B testing and Conversion Optimization in general.



Our clients about us

The value added to our agency by these tools is tremendous. I would say it provides Inflow with the best bang for the buck out of any tool we pay for. [ full testimonial ]

Chris Hickey
Director of Inbound Services, InFlow

From time-saving tools to quick and amazing support (in the rare case) it is a must have. Analytics-Toolkit is truly a must-have for any large analytics team. [ full testimonial ]

Thomas Bosilevac
President and Founder, MashMetrics