What does "Efficient Estimator" mean?

Definition of Efficient Estimator in the context of A/B testing (online controlled experiments).

What is an Efficient Estimator?

An efficient estimator is the "best possible" or "optimal" estimator of a parameter of interest. The definition of "best possible" depends on one's choice of a loss function which quantifies the relative degree of undesirability of estimation errors of different magnitudes. When one compares between a given procedure and a notional "best possible" procedure the efficiency can be expressed as relative finite-sample or asymptotic efficiency (a ratio). The relevance to A/B testing is that the more efficient the estimator, the smaller sample size one requires for an A/B test.

When defined asymptotically an estimator is efficient if the varianceof θ equals the Rao-Cramér lower bound. Since in many cases the lower bound in the Rao–Cramér inequality cannot be attained, an efficient estimator in statistics is frequently chosen based on having minimal variance in the class of all unbiased estimator of the parameter. Thus optimality in practice is defined using the variance or mean square error (MSE). In other words, the optimal estimator deviates as little as possible from the true value (θ*) one is trying to estimate.

If an efficient estimator exists it is also a sufficient estimator and can be obtained by the maximum-likelihood method (see Maximum Likelihood Estimate).


Glossary Index by Letter

ABCDEFGHIKLMNOPRSTUVZ

Select a letter to see all A/B testing terms starting with that letter or visit the Glossary homepage to see all.