## What is Uniformly Most Powerful Test?

Aliases: *UMP*

In hypothesis testing using statistical methods a uniformly most powerful (UMP) test is a test of a specified null hypothesis which has the greatest statistical power (1-β) among all possible tests with a specified significance threshold α. The Neymanâ€“Pearson lemma proved the presence of a UMP test for two point hypotheses: the likelihood ratio test, while an extension by the Karlin-Rubin theorem extended the lemma to cover composite hypotheses.

For the lemma to apply the parameter space Θ must be partitioned into two disjoint sets Θ_{0} and Θ_{1} (Θ_{0} ∪ Θ_{1} = Θ, Θ_{0} ∩ Θ_{1} = ∅).

A UMP test does not always exist with a trivial case being a two-sided test in which the alternative lies on both sides of the null. In such a case the uniformly most powerful test of a certain sample size for a value of the parameter on one side of the null is different than that for a value of the parameter on the other side of the null. This is not a major concern in A/B testing where most of the time we perform a one-sided test.

In practice, having a UMP test simply means that one is using the statistical method which makes the most efficient use of the available sample size and there does not exist a more efficient test for this data set.

If you want to read more on A/B testing stats, check out the upcoming book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.