What does "AGILE A/B Test" mean?
Definition of AGILE A/B Test in the context of A/B testing (online controlled experiments).
What is an AGILE A/B Test?
An AGILE A/B test is an online controlled experiment conducted following the AGILE method as described in the paper "Efficient A/B Testing in Conversion Rate Optimization: The AGILE Statistical Method". It includes a group sequential testing approach with alpha and beta spending functions that define stopping/monitoring boundaries for efficacy and futility. Among its main advantages is the fact that tests conducted under this method can be stopped early due to overwhelmingly positive effects or very low probability of a positive finding (futility). The ability to monitor tests as they go while maintaining type I error control is a highly sought quality among CRO practitioners and stakeholders alike.
The "AGILE" part comes from the fact that the method based on error-spending functions instead of fixed evaluation points allows for great flexibility in monitoring the data as it gathers.
The method also includes prescriptions for dealing with multiple testing in the form of both multiple comparisons as well as using more than one metric to evaluate the test (usually a key performance indicator, KPI) so that the Family-Wise Error Rate is kept in check. Part of the method is the use of conditionally near-unbiased estimators (confidence intervals, point estimators and p-values).
The easiest way to conduct an AGILE A/B test is through using the AGILE A/B testing calculator
Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.
Articles on AGILE A/B Test
Improving ROI in A/B Testing: the AGILE AB Testing Approach
blog.analytics-toolkit.com
Efficient AB Testing with the AGILE Statistical Method
blog.analytics-toolkit.com
20-80% Faster A/B Tests? Is it real?
blog.analytics-toolkit.com
Efficient A/B Testing in Conversion Rate Optimization: The AGILE Statistical Method
www.analytics-toolkit.com
Error Spending in Sequential Testing Explained
blog.analytics-toolkit.com
Related A/B Testing terms
Test DurationPeekingFutility BoundaryEfficacy BoundarySequential TestingAlpha-SpendingBeta-SpendingSee this in action
A/B Testing Calculator
Statistical Methods in Online A/B Testing
Take your A/B testing program to the next level with the most comprehensive book on user testing statistics in e-commerce.
Learn moreGlossary index by letter
Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.