## What is an Alternative Hypothesis?

The alternative hypothesis is usually a **claim** we hope to be true, e.g. the tested variant is performing better than the control group, the treatment is outperforming the current experience by X%, etc. Once we have an alternative we can select the null hypothesis such that the null and alternative exhaust the entire parameter space (Θ_{0} ∪ Θ_{1} = Θ, Θ_{0} ∩ Θ_{1} = ∅).

The alternative hypothesis is usually denoted by H_{1} (H_{2}, etc. in case there are multiple alternatives under consideration).

Most often in A/B tests we set up a superiority alternative such as H_{1}: μ > 0 or H_{0}: μ > ε where ε is some positive discrepancy. However, sometimes the alternative is that of non-inferiority so it is H_{0} > -ε where ε is the magnitude of the non-inferiority margin.

Upon observing a p-value below the significance threshold we selected we can then reject the null hypothesis and **accept the alternative**. One should remember that accepting the alternative means exactly that and not that any particular sub-set or point value from the alternative space is accepted. For example observing a percentage difference in conversion rate of 5% in the treatment group versus the control group and having a p-value less than the significance threshold does not mean that we accept the alternative: "the conversion rate of the treatment group is 5% higher" or "the conversion rate of the treatment group is 5% higher or more" if the p-value was calculated under a null of "the percentage difference between the treatment group and control group is less than or equal to zero". Only if the p-value was calculated under a null hypothesis which compliments these specific alternative hypotheses can one make these claims.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.