What does "Factorial Design" mean?

Definition of Factorial Design in the context of A/B testing (online controlled experiments).

What is a Factorial Design?

A factorial design is a type of online controlled experiment in which the outcome of interest is the contribution of individual factors to a variable of interest (often a KPI) and not higher-order interactions between these factors. A factorial design requires a smaller sample size compared to an equivalent multivariate test if it is not powered to detect higher-order interactions and requires the same sample size if powered for them (see Power Analysis).

In most cases CRO practitioners are interested not only about the effect of certain factors, but also of the combined effect of multiple factors changed together. When that is the case, a properly designed MVT can provide insights both on the performance of the combined changes as well as the contribution of each individual factor. Given that MVTs are much easier to set up and analyze properly they are often the preferred option.

Related A/B Testing terms

Multivariate Test

Articles on Factorial Design

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.

Purchase Statistical Methods in Online A/B Testing

Glossary Index by Letter


Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.