What does "Sample Ratio" mean?
Definition of Sample Ratio in the context of A/B testing (online controlled experiments).
What is a Sample Ratio?
In a controlled experiment, the sample ratio is the ratio of allocation of the sample size between the different test groups: control and variants.
For example, if the allocation is equal, the sample ratio will be 1. If 2 times more users are assigned to the variant test group in an A/B test, the ratio will be 0.5, signifying that the users in the control group are 0.5 times the users in the treatment group. If the allocation is reversed, the ratio will be 2, signifying that the control group contains 2 times more users than the variant.
In tests with multiple variants, e.g. A/B/C/D, instead of a single ratio, usually the proportion of the total sample size allocated to each group is specified, e.g. 0.25/0.25/0.25/0.25 with equal allocation (25% of users in each group), or 0.4/0.2/0.2/0.2 with an example unequal allocation scheme. In this example, a ratio of 2 can also be used, since the split between the variants is equal, however this is not the case in a scheme like this one: 0.4/0.1/0.1/0.4. In all cases, the proportions should sum to 1 (100% of users).
Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.
Statistical Methods in Online A/B Testing
Take your A/B testing program to the next level with the most comprehensive book on user testing statistics in e-commerce.Learn more
Glossary index by letter
Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.