What does "Identically Distributed" mean?

What is Identically Distributed?

Alias: ID

Two variables (X,Y) are identically distributed (ID) if they have the same probability distribution. A sufficient condition for this is that CDF(X)=CDF(Y) where CDF stands for Cumulative Distribution Function. A textbook way of describing this would be to write P(x ≤ X) = P(y ≤ Y). Even more formally and in the general case of k random variables (X₁, X₂... X_n) it can be said that they are Identically Distributed if their marginal distributions have the same form: f_k(X_k;θ_k) ≡ f_k(X_k;θ), for all k=1,2...,n.

The necessity for variables to be identically distributed plays a significant role in a classical Null Hypothesis Statistical Test where the full definition of the null hypothesis states, among other things that the two variables are identically distributed.

An immediate consequence of the requirement for identical distributions is that data analyzed under the assumption of ID should have constant mean and variance. If the data has a trend this assumptions is violated and a statistical model with the ID assumption in it becomes misspecified.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.