White papers

Non-Inferiority Designs in A/B Testing

A/B Testing

Non-Inferiority Designs in A/B Testing

The current paper argues that there are many cases where testing for non-inferiority, as opposed to the widespread practice of superiority testing, is both more appropriate and more powerful in the statistical sense, resulting in better decision-making, and in some cases: significantly faster A/B tests.

Learn moreDownload
Efficient A/B Testing in Conversion Rate Optimization: The AGILE Statistical Method

A/B Testing

Efficient A/B Testing in Conversion Rate Optimization: The AGILE Statistical Method

This paper presents AGILE - an improved A/B testing statistical methodology and accompanying software tool that allows for running conversion rate optimization (CRO) experiments that reach conclusions 20% to 80% faster than traditional methods and solutions while providing the same or better statistical guarantees, rules for stopping for futility and other benefits.

Learn moreDownload
Issues with Current Bayesian Approaches to A/B Testing in Conversion Rate Optimization

A/B Testing

Issues with Current Bayesian Approaches to A/B Testing in Conversion Rate Optimization

This paper covers what the author perceives as major issues with the current (as of late 2016) mainstream approaches to statistical design and statistical analysis of A/B testing experiments, mostly as applied in fields of Conversion Rate Optimization (CRO) and Landing Page Optimization (LPO).

Learn moreDownload