What does "Minimum Effect of Interest" mean?

Definition of Minimum Effect of Interest in the context of A/B testing (online controlled experiments).

What is a Minimum Effect of Interest?

Aliases: MEI, MDE, Minimum Detectable Effect

The minimum effect of interest is the effect size we would be happy/excited to find, usually denoted μ1. Proper estimation of the MEI is vital for setting the sample size of the experiment so that it provides sufficient statistical power at the point of the MEI.

In many cases an optimal can be approximately calculated using Risk-Reward Analysis which includes Power Analysis as a sub-task.

"Minimum detectable effect" (MDE) is a confusing term as it hints that true effects below it would not be detected which is not the case which can easily be established by examining the power function (which has a value of alpha (α) at the point of the null hypothesis closest to the alternative hypothesis). The term "minimum effect of interest" is less confusing as it reflects the fact that it is the minimum effect we want to detect reliably and does not carry with it the possibility of confusing it with a minimum effect size which can produce a statistically significant outcome.

Related A/B Testing terms

Statistical Power

Effect Size

Articles on Minimum Effect of Interest

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.

Purchase Statistical Methods in Online A/B Testing

Glossary Index by Letter


Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.