What does "Factorial Design" mean?

Definition of Factorial Design in the context of A/B testing (online controlled experiments).

What is a Factorial Design?

A factorial design is a type of online controlled experiment in which the outcome of interest is the contribution of individual factors to a variable of interest (often a KPI) and not higher-order interactions between these factors. A factorial design requires a smaller sample size compared to an equivalent multivariate test if it is not powered to detect higher-order interactions and requires the same sample size if powered for them (see Power Analysis).

In most cases CRO practitioners are interested not only about the effect of certain factors, but also of the combined effect of multiple factors changed together. When that is the case, a properly designed MVT can provide insights both on the performance of the combined changes as well as the contribution of each individual factor. Given that MVTs are much easier to set up and analyze properly they are often the preferred option.

Related A/B Testing terms

Multivariate Test

Articles on Factorial Design


Glossary Index by Letter

ABCDEFGHIKLMNOPRSTUVZ

Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.