What does "z Score" mean?

What is a z Score?

A z Score or a z-statistic is the result of applying a Z-test and it represents a point from a standard normal distribution. The z score is a standardized statistics meaning that the percentage of observation that fall between any two points is known. For example, all values below a z score of 1.96 represent 97.5% of the cumulative probability and all values below 1.28 represent 90% of the cumulative probability.

Z scores are often used in statistical analysis of an A/B test due to the large sample sizes involved which allow the population standard deviation to be estimated with precision using the sample standard deviation. A z score is usually communicated as a p-value by calculating the cumulative distribution function of the Z-distribution, however they may also be presented in raw form, e.g. when communicating an efficacy boundary or a futility boundary due the impossibility in transforming them to p-values reliably in such cases.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.