Statistics, Completeness in

Completeness is a probabilistic property that bestows a form of uniqueness on a statistic h(X) in the context of a statistical model Mθ(x) and crystallizes the link between sufficient and ancillary statistics.

[Page 871]Ronald A. Fisher’s (1922) path-breaking paper recast Karl Pearson’s descriptive statistics (Yule, 1916) into a model-based statistical induction grounded on the concept of a parametric statistical model whose generic form is:

$$\text{Mθ}\left(\text{x}\right)\text{ = }\left\{\text{f}\left(\text{x; θ}\right)\text{, θ }\in \text{ Θ}\right\}\text{, x }\in \text{ RXn,}\text{for Θ }\subset \text{ Rm, m < n},$$

where R := (−∞, ∞), f(x; θ) denotes the (joint) distribution of the sample X := (X1, …, Xn), RXn the sample space, and Θ the parameter space. The likelihood function is defined by: L(θ; x0) ∝ f(x0; θ), θ ∈ Θ.

Example 1. The simple Normal (N) model is specified by:

$$\text{Xt}~\text{NIID}\left(\text{μ, σ2}\right)\text{, θ := }\left(\text{μ, σ2}\right)\in \text{Θ := R × R+,}\text{xt}\in \text{R, t}\in \text{N := }\left(\text{1, 2, }\mathrm{\dots }\text{, n, }\mathrm{\dots }\right)\text{, μ = E}\left(\text{Xt}\right)\text{,}\text{σ2 = Var}\left(\text{Xt}\right)\text{, fN}\left(\text{x; θ}\right)\text{ = }\left(\text{2πσ2}\right)\text{ −}\text{nexp − 12σ2t = 1nxt − μ2}\text{.}$$

Example 2. The simple Bernoulli (Ber) model is specified by:

$$\text{Xk ~ BerIID}\left(\text{θ, θ}\left(\text{1 − θ}\right)\right)\text{, θ}\in \text{ Θ := }\left(\text{0, 1}\right)\text{,}\text{xk = 0,1, k}\in \text{N, θ = E}\left(\text{Xt}\right)\text{, Var}\left(\text{Xt}\right)\text{ = θ}\left(\text{1 − θ}\right)\text{,}\text{fB}\left(\text{x; θ}\right)\text{ = θnx}\left(\text{1 − θ}\right)\text{n}\left(\text{1 − x}\right)\text{.}$$

where X = (1/n)k = 1n‍Xk.

Frequentist inference revolves around Borel measurable (well-behaved) mappings of the form:

$$\text{h}\left(\text{.}\right)\text{ : RXn}\to \text{RK, n > k}\ge \text{1,}$$

Fisher (1922) named statistics Y = h(X), such as estimators, confidence interval (CI) bounds, test statistics, and predictors, whose sampling distributions are derived via:

$$\text{F}\left(\text{y}\right)\text{ = P}\left(\text{Y }\le \text{ y}\right)\text{ = }\cdots \left\{\text{x: h}\left(\text{x}\right)\le \text{y}\right\}\text{f}\left(\text{x; θ}\right)\text{dx,}\forall \text{y}\in \text{Rk,}$$

f(y; θ) = ∂Fn(y)/∂y, ∀y ∈ Rk, and provide the foundation of model-based frequentist inference. Of particular interest in this context are the sufficient and the ancillary statistics, introduced by Fisher in the early 1920s (see Lehmann & Scholz, 1992).

Sufficient. In the context of a statistical model Mθ(x), a statistic S := S(X) is said to be sufficient for θ if:

$$\text{f}\left(\text{x ∣ s; θ}\right)\text{ = f}\left(\text{x ∣ s}\right)\text{, }\forall \in \text{RXn,}$$

or equivalently (see Cox & Hinkley, 1974, p. 22):

$$\text{f}\left(\text{x; θ}\right)\text{ = f}\left(\text{x ∣ s}\right)\cdot \text{f}\left(\text{s; θ}\right)\text{,}\forall \text{x}\in \text{RXn, }\forall \text{s}\in \text{Rk}\text{.}$$

That is, the distribution of the sample f(x; θ) can be separated into a product of:

(i) a conditional distribution of X given S = s that is free of θ and

(ii) the marginal distribution of S which depends on θ.

The sufficient statistic S is said to induce the family of sampling distributions {f(s; θ), θ ∈ Θ, s ∈ Rk}.

The statistic S is said to be minimal sufficient for θ, if for any other sufficient statistic T := T(X) there exists a function g(.) such that S = g(T(X)); in practice S has the lowest dimension possible.

Ancillary. In the context of a statistical model Mθ(x), a statistic V := V(X) is said to be ancillary for θ, if its sampling distribution:

$$\text{f}\left(\text{v}\right)\text{, v}\in \text{Rvk, n > k}\ge \text{1,}$$

is free of (does not depend on) θ (see Lehmann and Romano, 2005).

Intuitively, an ancillary statistic V contains no information about θ. The statistic V is said to be maximal ancillary for θ, if for any other ancillary statistic U(X) there exists a function g(.) such that U(X) = g(V(X)); a maximal ancillary statistic V has the highest possible dimension k.

As argued by Erich Leo Lehmann and Fritz Scholz,