What does "Modus Tollens" mean?

What is Modus Tollens?

Modus tollens is a Latin term meaning "mode that denies by denying" or denying the consequent and is a valid argument form / rule of inference. Its logic is that if a statement is true, then so is its contra-positive (opposite). It can be stated formally as: P->Q,¬Q ∴ ¬P.

Modus tollens is important in causal inference and frequentist inference (model-based induction) in particular since it describes the logic of hypothesis testing. In a Null Hypothesis Statistical Test first we form a substantive hypothesis that we then translate into a fully-fledged probabilistic statistical model in order to establish the sample space which allows us to calculate the relevant error probabilities of any statistic. Then, upon observing some data x₀ that is either surprising, giving the sample space, or not, we can make the relevant calls to reject the null hypothesis or to fail to reject it.

The relevance of modus tollens logic in A/B testing is due to our goal of extracting as much as we can about a metric of interest from a particular test, including the relevant error bounds which serve as risk management devices and estimation uncertainty measurements which we can then present to stakeholders to assist in their decision-making process. Employing this argument shifts the burden of proof on whoever is willing to argue for the null hypothesis and thus gives data a central place.

The modus tollens rule of inference can be juxtaposed with an invalid form of inference such as affirming the consequent (a.k.a. converse error, fallacy of the converse or simply confusion of necessity and sufficiency). It takes the form P -> Q, Q ∴ P. It is logically invalid since the fact that P is a necessary condition for Q does not logically entail that observing Q means P is true. For example, the fact that when it rains it is almost always cloudy does not entail that if it is cloudy it is almost always raining as well.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.