What does "Control Group" mean?

Definition of Control Group in the context of A/B testing (online controlled experiments).

What is a Control Group?

The Control group in an online controlled experiment is a group of (usually randomly assigned) users/sessions/pageviews/etc. who will not be exposed to the experimental treatment(s). The performance of the Test Group(s) is compared to that of the Control group to check for discrepancies large enough to reject the null hypothesis.

In most tests the control and test groups are of equal sizes as it facilitates easier interpretation due to lack of generalizability issues, is easier to compute and is usually an optimal design. In an adaptive sequential design or a multi-armed bandit design the weights of the control and test groups are usually different and vary across the test duration.

Related A/B Testing terms

Test Group

Sample Ratio

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.

Purchase Statistical Methods in Online A/B Testing

Glossary Index by Letter


Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.