What does "Serial Testing" mean?
Definition of Serial Testing in the context of A/B testing (online controlled experiments).
What is Serial Testing?
Serial testing is basically the practice of running a set of tests one after another instead of as concurrent tests. It is sometimes done due to well-founded concerns about interaction effects while in other cases such concerns are completely unfounded and convey simple misunderstanding of randomization and/or overestimation of the risks involved.
It should be noted that while serial testing completely removes the risk of interaction effects it also takes much longer for tests to run and it results in untested variations: e.g. there might have been a positive relation between treatment B from test 1 and treatment B from test 3 but if they are run in a serial manner and test 1 ends with B losing to the control then the interaction will not be tested at all.
Like this glossary entry? For an in-depth and comprehensive reading on A/B testing stats, check out the book "Statistical Methods in Online A/B Testing" by the author of this glossary, Georgi Georgiev.
Statistical Methods in Online A/B Testing
Take your A/B testing program to the next level with the most comprehensive book on user testing statistics in e-commerce.
Learn moreGlossary index by letter
Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.