What does "A/B Test" mean?

Definition of A/B Test in the context of A/B testing (online controlled experiments).

What is an A/B Test?

Aliases: split test

An A/B test, a.k.a. a split test is a simple form of an online controlled experiment in which one user experience is tested versus another. The name comes from the convention of labeling one experience "A" and the other "B". One, usually the default, current or preferred experience is designated as the control (A) and it is then tested against another, hopefully better in some way with randomization being used to assign users/sessions/pageviews/etc. to one or the other.

The A/B test is a simple yet powerful tool for making causal claims in the toolbox of online marketing and user experience specialists so it is no wonder that over the years it has become a bit detached from its strict meaning and is often used to refer to any kind of online experimentation like a Multivariate Test (A/B/n) or even a Factorial Design.

We can distinguish between two main goals of performing an A/B test: controlling the business risk associated with a given proposed change (hypothesis testing) as well as estimating the magnitude and direction of effect (estimation). Often one seeks to achieve both and usually the former is the primary goal and the latter the secondary.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.

Purchase Statistical Methods in Online A/B Testing

Glossary Index by Letter


Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.