What does "Dunnett's Correction" mean?

Definition of Dunnett's Correction in the context of A/B testing (online controlled experiments).

What is the Dunnett's Correction?

The Dunnett's Correction has been developed in the 1960s in order to provide a powerful procedure for controlling the Family-Wise Error Rate (FWER) in the case of comparing multiple treatments to a common control group which is the case in a multivariate test. Such a scenario is a case of multiple comparisons / multiple testing. The goal of the correction is to maintain the overall type I error rate which is computed under the null hypothesis that there is no difference between the control group and any of the test groups. In this sense it is an adjustment of the Bonferroni Correction to that specific case.

Dunnett's test is the most-powerful method that maintains control of the FWER in such a scenario as it makes use of the positive dependence between the comparisons.

While the correction is a p-value adjustment after a test is completed, when planning an MVT one needs to perform sample size computations and power analysis by taking the p-value correction into account.

The correction is usually applied as a step-down procedure whereas the sample size calculations require integrating over multivariate normal distributions, often through simulations.

Articles on Dunnett's Correction


Glossary Index by Letter

ABCDEFGHIKLMNOPRSTUVZ

Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.