What does "Serial Testing" mean?

Definition of Serial Testing in the context of A/B testing (online controlled experiments).

What is Serial Testing?

Serial testing is basically the practice of running a set of tests one after another instead of as concurrent tests. It is sometimes done due to well-founded concerns about interaction effects while in other cases such concerns are completely unfounded and convey simple misunderstanding of randomization and/or overestimation of the risks involved.

It should be noted that while serial testing completely removes the risk of interaction effects it also takes much longer for tests to run and it results in untested variations: e.g. there might have been a positive relation between treatment B from test 1 and treatment B from test 3 but if they are run in a serial manner and test 1 ends with B losing to the control then the interaction will not be tested at all.

Related A/B Testing terms

A/B Test

Online Controlled Experiment


Glossary Index by Letter

ABCDEFGHIKLMNOPRSTUVZ

Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.