What does "Control Group" mean?

Definition of Control Group in the context of A/B testing (online controlled experiments).

What is a Control Group?

The Control group in an online controlled experiment is a group of (usually randomly assigned) users/sessions/pageviews/etc. who will not be exposed to the experimental treatment(s). The performance of the Test Group(s) is compared to that of the Control group to check for discrepancies large enough to reject the null hypothesis.

In most tests the control and test groups are of equal sizes as it facilitates easier interpretation due to lack of generalizability issues, is easier to compute and is usually an optimal design. In an adaptive sequential design or a multi-armed bandit design the weights of the control and test groups are usually different and vary across the test duration.

Related A/B Testing terms

Test Group


Glossary Index by Letter

ABCDEFGHIKLMNOPRSTUVZ

Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.