What does "Sample Space" mean?

What is a Sample Space?

In probability theory the sample space is the set of all possible outcomes of a random phenomenon that could be observed, all possible values of the sample X. It can be a set of real numbers or a higher-dimensional vector space, or even a list of non-numerical values such as {Yes,No} (a user converted or not, bounced or not).

For example, the sample space of any type of conversion rate is [0,1] while that of any kind of revenue-based metric is [0,+∞) (unless negative revenue is possible due to refunds).

The sample space is a key concept in frequentist inference wherein in both hypothesis testing and estimation one needs to consider all possible realizations of variable in order to determine how unexpected a given observation is under the null hypothesis (a p-value) or calculate an interval that would contain the true value with a given probability (coverage of a confidence interval). Framed in terms of the statistical model that most closely resembles the true data-generating mechanism the sample space is simply an enumeration of all possible outcomes from the statistical model.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.