What does "p-value Distribution" mean?

Definition of p-value Distribution in the context of A/B testing (online controlled experiments).

What is a p-value Distribution?

Simulating many experiments and plotting the distribution of their p-values is a good way to insure that the experimental setup and statistic used are properly chosen. With a proper setup, the p-value distribution should be flat.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.

Purchase Statistical Methods in Online A/B Testing

Glossary Index by Letter


Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.