Glossary of A/B testing terms
Alphabetical index of 219 terms and abbreviations in A/B testing / online controlled experiments in Conversion Rate Optimization (CRO).
Jump to terms by letter
A
A/A TestA/B TestA/B TestingA/B/n TestAGILE A/B TestAOVARPUASDAbsolute ChangeAbsolute DifferenceActual SignificanceActual p-valueAdaptive Sequential DesignAlpha-SpendingAlpha-spending FunctionAlternative HypothesisArgument from CoincidenceAverageAverage Order ValueAverage Revenue Per UserAverage Sample Sizealpha
B
Bayesian InferenceBernoulli MetricBeta-SpendingBeta-spending FunctionBiasBinomial MetricBonferroni CorrectionBounce Ratebeta
C
CICRCROCarry-Over EffectCausal InferenceChi Squared TestChurn RateComposite HypothesisConcurrent TestsConfidence BoundConfidence IntervalConfidence LevelConfidence LimitConfidence ThresholdConsistent EstimatorContinuous MetricControl GroupConversion RateConversion Rate OptimizationCookie ChurnCostCritical Region
D
E
Effect SizeEfficacy BoundaryEfficient EstimatorError ProbabilityError RateError-Spending FunctionEstimationEstimatorExternal Validity
F
FWERFactorial DesignFalse Discovery RateFalse NegativeFalse Negative RateFalse PositiveFamily-Wise Error RateFrequentist InferenceFutility Boundary
G
H
I
K
L
M
MDEMEIMLEMSEMVTMaximum Likelihood EstimateMeanMean Square ErrorMinimum Detectable EffectMinimum Effect of InterestModus TollensMuMulti-Armed BanditMultiple ComparisonsMultiple TestingMultivariate TestMultivariate Testing
N
NHSTNominal SignificanceNominal p-valueNon-Binomial MetricNon-Inferiority MarginNon-Inferiority TestNon-Parametric TestNormal DistributionNovelty EffectNull HypothesisNull Hypothesis Statistical Test
O
OCEOECObserved SignificanceOne-Sided HypothesisOne-Tailed TestOne-sided TestOnline Controlled ExperimentOptional StoppingOptional StoppingOverall Evaluation CriterionOverpowered Test
P
Parametric TestPeekingPercent ChangePoint HypothesisPower AnalysisPower FunctionPrimary KPIProportionPurchase Ratep-valuep-value Adjustmentp-value Distribution
R
ROIRPURandomizationRandomizerRateRejection RegionRelative ChangeRelative DifferenceRepresentative SampleRepresentativenessReturn on InvestmentRevenue per UserRewardRiskRisk-Reward Analysis
S
SDSEMSRMSample Allocation MismatchSample RatioSample Ratio MismatchSample SizeSample SpaceSeasonalitySecondary KPISegmentationSelection BiasSensitivitySequential MonitoringSequential TestingSerial TestingSeveritySidak CorrectionSigmaSignificance LevelSignificance TestSignificance ThresholdSignificantSplit TestSplit TestingStandard DeviationStandard Error of the MeanStatisticStatistical DesignStatistical ModelStatistical PowerStatistical SignificanceStatistically SignificantSubgroup AnalysisSufficient EstimatorSuperiority TestSurvivorship Bias
T
T-testTest DurationTest GroupTest Of SignificanceThetaTransaction RateTreatmentTwo-Sided HypothesisTwo-Tailed TestTwo-sided TestType I ErrorType II Errort Score