What does "Sidak Correction" mean?
Definition of Sidak Correction in the context of A/B testing (online controlled experiments).
What is the Sidak Correction?
The Šidák correction is a method for controlling the Family-Wise Error Rate (FWER) in the strong sense and it is only slightly less conservative than the most conservative Bonferroni correction. This test produces a familywise Type I error rate of exactly α when the tests are independent from each other and all null hypotheses are true. For positively-correlated tests the procedure is conservative and it is liberal for negatively-correlated tests.
In practice the Šidák correction a good method when one compares two or more groups across a range of metrics, for example comparing bounce rate, conversion rate, average revenue per user and average site speed and considering each of the comparisons enough to make a call on the test. The correction is usually implemented by using the Šidák Step-Down procedure.
Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.
Related A/B Testing terms
Multiple TestingSee this in action
Statistical Significance Calculator
About the author
Georgi has over twenty years of experience in online marketing, web analytics, statistics, and design of business experiments.
Author of the book "Statistical Methods in Online A/B Testing", white papers on statistical analysis of A/B tests, and a speaker, he has been distinguished as a winner in the Data & Analytics category of the 2024 Experimentation Thought Leadership Awards.
Statistical Methods in Online A/B Testing
Take your A/B testing program to the next level with the most comprehensive book on user testing statistics in e-commerce.
Learn moreGlossary index by letter
Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.