## What is an AGILE A/B Test?

An AGILE A/B test is an online controlled experiment conducted following the AGILE method as described in the paper "Efficient A/B Testing in Conversion Rate Optimization: The AGILE Statistical Method". It includes a group sequential testing approach with alpha and beta spending functions that define stopping/monitoring boundaries for efficacy and futility. Among its main advantages is the fact that tests conducted under this method can be stopped early due to overwhelmingly positive effects or very low probability of a positive finding (futility). The ability to monitor tests as they go while maintaining type I error control is a highly sought quality among CRO practitioners and stakeholders alike.

The "AGILE" part comes from the fact that the method based on error-spending functions instead of fixed evaluation points allows for great flexibility in monitoring the data as it gathers.

The method also includes prescriptions for dealing with multiple testing in the form of both multiple comparisons as well as using more than one metric to evaluate the test (usually a key performance indicator, KPI) so that the Family-Wise Error Rate is kept in check. Part of the method is the use of conditionally near-unbiased estimators (confidence intervals, point estimators and p-values).

The easiest way to conduct an AGILE A/B test is through using the AGILE A/B testing calculator

## Related A/B Testing terms

## Articles on AGILE A/B Test

- Improving ROI in A/B Testing: the AGILE AB Testing Approach
- Efficient AB Testing with the AGILE Statistical Method
- 20-80% Faster A/B Tests? Is it real?
- Efficient A/B Testing in Conversion Rate Optimization: The AGILE Statistical Method
- Error Spending in Sequential Testing Explained

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.