What does "Sample Space" mean?

Definition of Sample Space in the context of A/B testing (online controlled experiments).

What is a Sample Space?

In probability theory the sample space is the set of all possible outcomes of a random phenomenon that could be observed, all possible values of the sample X. It can be a set of real numbers or a higher-dimensional vector space, or even a list of non-numerical values such as {Yes,No} (a user converted or not, bounced or not).

For example, the sample space of any type of conversion rate is [0,1] while that of any kind of revenue-based metric is [0,+∞) (unless negative revenue is possible due to refunds).

The sample space is a key concept in frequentist inference wherein in both hypothesis testing and estimation one needs to consider all possible realizations of variable in order to determine how unexpected a given observation is under the null hypothesis (a p-value) or calculate an interval that would contain the true value with a given probability (coverage of a confidence interval). Framed in terms of the statistical model that most closely resembles the true data-generating mechanism the sample space is simply an enumeration of all possible outcomes from the statistical model.

Related A/B Testing terms

Statistical Model


Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.

Purchase Statistical Methods in Online A/B Testing

Glossary Index by Letter


Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.