What does "Serial Testing" mean?
Definition of Serial Testing in the context of A/B testing (online controlled experiments).
What is Serial Testing?
Serial testing is basically the practice of running a set of tests one after another instead of as concurrent tests. It is sometimes done due to well-founded concerns about interaction effects while in other cases such concerns are completely unfounded and convey simple misunderstanding of randomization and/or overestimation of the risks involved.
It should be noted that while serial testing completely removes the risk of interaction effects it also takes much longer for tests to run and it results in untested variations: e.g. there might have been a positive relation between treatment B from test 1 and treatment B from test 3 but if they are run in a serial manner and test 1 ends with B losing to the control then the interaction will not be tested at all.
Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.
Related A/B Testing terms
A/B TestOnline Controlled ExperimentAbout the author
Georgi has over twenty years of experience in online marketing, web analytics, statistics, and design of business experiments.
Author of the book "Statistical Methods in Online A/B Testing", white papers on statistical analysis of A/B tests, and a speaker, he has been distinguished as a winner in the Data & Analytics category of the 2024 Experimentation Thought Leadership Awards.
Statistical Methods in Online A/B Testing
Take your A/B testing program to the next level with the most comprehensive book on user testing statistics in e-commerce.
Learn moreGlossary index by letter
Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.