What does "Sample Ratio" mean?

What is a Sample Ratio?

In a controlled experiment, the sample ratio is the ratio of allocation of the sample size between the different test groups: control and variants.

For example, if the allocation is equal, the sample ratio will be 1. If 2 times more users are assigned to the variant test group in an A/B test, the ratio will be 0.5, signifying that the users in the control group are 0.5 times the users in the treatment group. If the allocation is reversed, the ratio will be 2, signifying that the control group contains 2 times more users than the variant.

In tests with multiple variants, e.g. A/B/C/D, instead of a single ratio, usually the proportion of the total sample size allocated to each group is specified, e.g. 0.25/0.25/0.25/0.25 with equal allocation (25% of users in each group), or 0.4/0.2/0.2/0.2 with an example unequal allocation scheme. In this example, a ratio of 2 can also be used, since the split between the variants is equal, however this is not the case in a scheme like this one: 0.4/0.1/0.1/0.4. In all cases, the proportions should sum to 1 (100% of users).

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.