What does "Chi Squared Test" mean?

Definition of Chi Squared Test in the context of A/B testing (online controlled experiments).

What is a Chi Squared Test?

The Chi-Squared Test is a goodness of fit test that uses the one-tailed Χ2 distribution and is therefore often written as Χ2 test. It is used to determine whether there is a statistically significant difference between the expected frequencies and the observed frequencies in one or more test groups and is applicable when the statistic of interest is chi-squared distributed under the null hypothesis. Usually it is performed using the Fisher's exact test (vs. Pearson's chi-squared test) due to its finite-sample precision versus the asymptotic guarantee otherwise.

A Chi-squared test is generally suitable for contingency tables of categorical data, usually of qualitative nature. If suited to the simplest case of a basic A/B test it should produce results identical to the T-test.

It should be noted that Chi-squared tests are performed under a two-sided hypothesis by default, although p-values can also be calculated for a one-sided hypothesis.

Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.

Purchase Statistical Methods in Online A/B Testing

Statistical Methods in Online A/B Testing

Take your A/B testing program to the next level with the most comprehensive book on user testing statistics in e-commerce.

Learn more

Glossary index by letter

Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.