k-Statistic

An Unbiased Estimator of the Cumulants $\kappa_i$ of a Distribution. The expectation values of the $k$-statistics are therefore given by the corresponding Cumulants

$$\left\langle{k_1}\right\rangle{} = \kappa_1$$ (1)

$$\left\langle{k_2}\right\rangle{} = \kappa_2$$ (2)

$$\left\langle{k_3}\right\rangle{} = \kappa_3$$ (3)

$$\left\langle{k_4}\right\rangle{} = \kappa_4$$ (4)

(Kenney and Keeping 1951, p. 189). For a sample of size, $N$, the first few $k$-statistics are given by

$$k_1 = m_1$$ (5)

$$k_2 = {N\over N-1}m_2$$ (6)

$$k_3 = {N^2\over (N-1)(N-2)}m_3$$ (7)

$$k_4 = {N^2[(N+1)m_4-3(N-1){m_2}^2]\over(N-1)(N-2)(N-3)},$$ (8)

where $m_1$ is the sample Mean, $m_2$ is the sample Variance, and $m_i$ is the sample $i$th Moment about the Mean (Kenney and Keeping 1951, pp. 109-110, 163-165, and 189; Kenney and Keeping 1962). These statistics are obtained from inverting the relationships

$$\left\langle{m_1}\right\rangle{} = \mu$$ (9)

$$\left\langle{m_2}\right\rangle{} = {N-1\over N} \mu_2$$ (10)

$$\left\langle{{m_2}^2}\right\rangle{} = {(N-1)[(N-1)\mu_4+(N^2-2N+3){\mu_2}^2]\over N^3}$$ (11)

$$\left\langle{m_3}\right\rangle{} = {(N-1)(N-2)\over N^2}\mu_3$$ (12)

$$\left\langle{m_4}\right\rangle{} = {(N-1)[(N^2-3N+3)\mu_4+3(2N-3){\mu_2}^2]\over N^3}.$$ (13)

The first moment (sample Mean) is

$$m_1 \equiv \left\langle{x}\right\rangle{}= {1\over N} \sum_{i=1}^N x_i,$$ (14)

and the expectation value is

$$\left\langle{m_1}\right\rangle{} = \left\langle{{1\over N}\sum_{i=1}^N x_i}\right\rangle{} =\mu.$$ (15)

The second Moment (sample Standard Deviation) is

$$m_2 \equiv \left\langle{(x-\mu)^2}\right\rangle{} = \left\langle{x^2}\right\... ...mu\left\langle{x}\right\rangle{}+\mu^2 = \left\langle{x^2}\right\rangle{}-\mu^2$$

$$= {1\over N}\sum_{i=1}^N {x_i}^2-\left({{1\over N} \sum_{i=1}^N {x_i}}\right)^2$$

$$= {1\over N} \sum_{i=1}^N {x_i}^2 -{1\over N^2} \left({\sum_{i=1}^N {x_i}^2+\sum_{\scriptstyle{i,j=1}\atop \scriptstyle i\not=j}^N x_ix_j}\right)$$

$$= {N-1\over N^2} \sum_{i=1}^N {x_i}^2-{1\over N^2} \sum_{\scriptstyle {i,j=1}\atop \scriptstyle i\not= j}^N x_ix_j,$$ (16)

and the expectation value is

$$\left\langle{m_2}\right\rangle{} = {N-1\over N} \left\langle{{1\over N} \sum_{i=1}^N {x_i}^2}\right\... ...{\sum_{\scriptstyle{i,j=1}\atop \scriptstyle i\not= j}^N x_ix_j}\right\rangle{}$$

$$= {N-1\over N} \mu'_2-{N(N-1)\over N^2}\mu^2,$$ (17)

since there are $N(N-1)$ terms $x_ix_j$, using

$$\left\langle{x_ix_j}\right\rangle{}=\left\langle{x_i}\right\... ...langle{x_j}\right\rangle{}=\left\langle{x_i}\right\rangle{}^2,$$ (18)

and where $\mu'_2$ is the Moment about 0. Using the identity

$$\mu'_2=\mu_2+\mu^2$$ (19)

to convert to the Moment $\mu_2$ about the Mean and simplifying then gives

$$\left\langle{m_2}\right\rangle{} = {N-1\over N} \mu_2.$$ (20)

The factor $(N-1)/N$ is known as Bessel's Correction.

The third Moment is

$$m_3 \equiv \left\langle{(x-\mu)^3}\right\rangle{} = \left\langle{x^3-3\mu x^2+3\mu^2x-\mu^3}\right\rangle{}$$

$$= \left\langle{x^3}\right\rangle{}-3\mu\left\langle{x^2}\right\rangle{}+3\mu^2\left\langle{x}\right\rangle{}-\mu^3$$

$$= \left\langle{x^3}\right\rangle{}-3\mu\left\langle{x^2}\right\rangle{}+3\mu^3-\mu^3$$

$$= \left\langle{x^3}\right\rangle{}-3\mu\left\langle{x^2}\right\rangle{}+2\mu^3$$

$$= {1\over N} \sum {x_i}^3 -3\left({{1\over N}\sum x_i}\right)\left({{1\over N} \sum {x_j}^2}\right)+2\left({{1\over N} \sum x_i}\right)^3$$

$$= {1\over N}\sum {x_i}^3-{3\over N^2}\left({\sum x_i}\right)\left({\sum {x_j}^2}\right)+{2\over N^3} \left({\sum x_i}\right)^3.$$ (21)

Now use the identities

$\quad \left({\sum {x_i}^2}\right)\left({\sum x_j}\right)=\sum{x_i}^3+\sum{x_i}^2x_j$	(22)
$\quad \left({\sum x_i}\right)^3=\sum{x_i}^3+3\sum{x_i}^2x_j+6\sum x_ix_jx_k,$	(23)

where it is understood that sums over products of variables exclude equal indices. Plugging in

$$m_3=\left({{1\over N}-{3\over N^2}+{2\over N^3}}\right)\sum{... ...r N^3}}\right)\sum{x_i}^2x_j+6\cdot{2\over N^3}\sum x_ix_jx_k.$$ (24)

The expectation value is then given by

$$\left\langle{m_3}\right\rangle{}= \left({{1\over N}-{3\over ... ...\mu_2'\mu+{12\over N^3} {\textstyle{1\over 6}}N(N-1)(N-2)\mu^3$$ (25)

where $\mu_2'$ is the Moment about 0. Plugging in the identities

$$\mu'_2 = \mu_2+\mu^2$$ (26)

$$\mu'_3 = \mu_3+3\mu_2\mu+\mu^3$$ (27)

and simplifying then gives

$$\left\langle{m_3}\right\rangle{} = {(N-1)(N-2)\over N^2} \mu_3$$ (28)

(Kenney and Keeping 1951, p. 189).

The fourth Moment is

$$m_4 = \left\langle{(x-\mu)^4}\right\rangle{} = \left\langle{x^4-4x^3\mu+6x^2\mu^2-4x\mu^3+\mu^4}\right\rangle{}$$

$$= \left\langle{x^4}\right\rangle{}-4\mu\left\langle{x^3}\right\rangle{}+6\mu^2\left\langle{x^2}\right\rangle{}-3\mu^4$$

$$= {1\over N} \sum {x_i}^4-{4\over N^2} \left({\sum x_i}\right)\left... ... x_i}\right)^2\left({\sum {x_j}^2}\right)-{3\over N^4}\left({\sum x_i}\right)^4$$ (29)

Now use the identities

$\left({\sum x_i}\right)\left({\sum{x_j}^3}\right)=\sum{x_i}^4+\sum{x_i}^3x_j$	(30)
$\left({\sum x_i}\right)^2\left({\sum{x_j}^2}\right)=\sum{x_i}^4+2\sum{x_i}^3x_j+2\sum{x_i}^2{x_j}^2+2\sum{x_i}^2x_jx_k$	(31)
$\left({\sum x_i}\right)^4=\sum{x_i}^4+4\sum{x_i}^3x_j+6\sum{x_i}^2{x_j}^2+12\sum{x_i}^2x_jx_k+24\sum x_ix_jx_kx_l.$	(32)

Plugging in,

$$m_4 = \left({{1\over N}-{4\over N^2}+{6\over N^3}-{3\over N^4}}\right)\sum{x_i}^4$$

$$\phantom{=} +\left({-{4\over N^2}+2\cdot{6\over N^3}-4\cdot{3\over N^4}}\right)\sum{x_i}^3 x_j$$

$$\phantom{=} +\left({2\cdot{6\over N^3}-6\cdot{3\over N^4}}\right)\sum{x_i}^2{x_j}^2$$

$$\phantom{=} +\left({2\cdot{6\over N^3}-12\cdot{3\over N^4}}\right)\sum{x_i}^2x_jx_k$$

$$\phantom{=} -24\cdot{3\over N^4}\sum x_ix_jx_kx_l.$$ (33)

The expectation value is then given by

$$\left\langle{m_4}\right\rangle{} = \left({{1\over N}-{4\over N^2}+{6\over N^3}-{3\over N^4}}\right)N\mu'_4$$

$$\phantom{=} +\left({-{4\over N^2}+{12\over N^3}-{12\over N^4}}\right)N(N-1)\mu'_3\mu$$

$$\phantom{=} +\left({{12\over N^3}-{18\over N^4}}\right){\textstyle{1\over 2}}N(N-1){\mu'_2}^2$$

$$\phantom{=} +\left({{18\over N^3}-{36\over N^4}}\right){\textstyle{1\over 2}}N(N-1)(N-2)\mu_2'\mu^2$$

$$\phantom{=} -{72\over N^4} {\textstyle{1\over 24}} N(N-1)(N-2)(N-3)\mu^4,$$ (34)

where $\mu_i'$ are Moments about 0. Using the identities

$$\mu_2' = \mu_2+\mu^2$$ (35)

$$\mu_3' = \mu_3+3\mu_2\mu+\mu^3$$ (36)

$$\mu_4' = \mu_4+4\mu_3\mu+6\mu_2\mu^2+\mu^4$$ (37)

and simplifying gives

$$\left\langle{m_4}\right\rangle{}={(N-1)[(N^2-3N+3)\mu_4+3(2N-3){\mu_2}^2]\over N^3}$$ (38)

(Kenney and Keeping 1951, p. 189).

The square of the second moment is

$${m_2}^2 = (\left\langle{x^2}\right\rangle{}-\mu^2)^2=\left\langle{x^2}\right\rangle{}^2-2\mu^2\left\langle{x^2}\right\rangle{}+\mu^4$$

$$= \left({{1\over N}\sum{x_i}^2}\right)^2-2\left({{1\over N}\sum x_i}\right)^2\left({{1\over N}\sum{x_i}^2}\right)+\left({{1\over N}\sum x_i}\right)^4$$

$$= {1\over N^2}\left({\sum{x_i}^2}\right)^2-{2\over N^3}\left({\sum x_i}\right)^2\left({\sum{x_j}^2}\right)+{1\over N^4}\left({\sum x_i}\right)^4.$$ (39)

Now use the identities

$\left({\sum{x_i}^2}\right)^2=\sum{x_i}^4+2\sum{x_i}^2{x_j}^2$	(40)
$\left({\sum x_i}\right)^2\left({\sum{x_j}^2}\right)=\sum{x_i}^4+2\sum{x_i}^2{x_j}^2+2\sum{x_i}^3x_j+2\sum{x_i}^2x_jx_k$	(41)
$\left({\sum x_i}\right)^4=\sum{x_i}^4+6\sum{x_i}^2{x_j}^2+4\sum{x_i}^3x_j+12\sum{x_i}^2x_jx_k+24\sum x_ix_jx_kx_l.$	(42)

Plugging in,

$${m_2}^2 = \left({{1\over N^2}-{2\over N^3}+{1\over N^4}}\right)\sum{x_i}^4$$

$$\phantom{=} +\left({2\cdot{1\over N^2}-2\cdot{2\over N^3}+6\cdot{1\over N^4}}\right)\sum{x_i}^2{x_j}^2$$

$$\phantom{=} +\left({-2\cdot{2\over N^3}+4\cdot{1\over N^4}}\right)\sum{x_i}^3x_j$$

$$\phantom{=} +\left({-2\cdot{2\over N^3}+12\cdot{1\over N^4}}\right)\sum{x_i}^2x_jx_k$$

$$\phantom{=} +{24\over N^4}\sum x_ix_jx_kx_l$$ (43)

The expectation value is then given by

$$\left\langle{{m_2}^2}\right\rangle{} = \left({{1\over N^2}-{2\over N^3}+{1\over N^4}}\right)N\mu_4'$$

$$\phantom{=} +\left({{2\over N^2}-{4\over N^3}+{6\over N^4}}\right){\textstyle{1\over 2}}N(N-1)\mu_2'^2$$

$$\phantom{=} +\left({-{4\over N^3}+{4\over N^4}}\right)N(N-1)\mu_3'\mu$$

$$\phantom{=} +\left({-{4\over N^3}+{12\over N^4}}\right){\textstyle{1\over 2}}N(N-1)(N-2)\mu_2'\mu^2$$

$$\phantom{=} +{24\over N^4}{\textstyle{1\over 24}}N(N-1)(N-2)(N-3)\mu^4$$ (44)

where $\mu_i'$ are Moments about 0. Using the identities

$$\mu_2' = \mu_2+\mu^2$$ (45)

$$\mu_3' = \mu_3+3\mu_2\mu+\mu^3$$ (46)

$$\mu_4' = \mu_4+4\mu_3\mu+6\mu_2\mu^2+\mu^4$$ (47)

and simplifying gives

$$\left\langle{{m_2}^2}\right\rangle{}={(N-1)[(N-1)\mu_4+(N^2-2N+3){\mu_2}^2]\over N^3}$$ (48)

(Kenney and Keeping 1951, p. 189).

The Variance of $k_2$ is given by

$${\rm var}(k_2)={\kappa_4\over N}+{2\over(N-1){\kappa_2}^2},$$ (49)

so an unbiased estimator of ${\rm var}(k_2)$ is given by

$$\hat{\rm var}(k_2)={2{k_2}^2N+(N-1)k_4\over N(N+1)}$$ (50)

(Kenney and Keeping 1951, p. 189). The Variance of $k_3$ can be expressed in terms of Cumulants by

$${\rm var}(k_3)={\kappa_6\over N}+{9\kappa_2\kappa_4\over N-1}+{9{\kappa_3}^2\over N-1}+{6{\kappa_2}^3\over N(N-1)(N-2)}.$$ (51)

An Unbiased Estimator for ${\rm var}(k_3)$ is

$$\hat{\rm var}(k_3)={6{k_2}^2N(N-1)\over (N-2)(N+1)(N+3)}$$ (52)

(Kenney and Keeping 1951, p. 190).

Now consider a finite population. Let a sample of $N$ be taken from a population of $M$. Then Unbiased Estimators $M_2$ for the population Mean $\mu$, $M_2$ for the population Variance $\mu_2$, $G_1$ for the population Skewness $\gamma_1$, and $G_2$ for the population Kurtosis $\gamma_2$ are

$$M_1 = \mu$$ (53)

$$M_2 = {M-N\over N(M-1)} \mu_2$$ (54)

$$G_1 = {M-2N\over M-2} \sqrt{M-1\over N(M-N)}\, \gamma_1$$ (55)

$$G_2 = {(M-1)(M^2-6MN+M+6N^2)\gamma_2\over N(M-2)(M-3)(M-N)}-{6M(MN+M-N^2-1)\over N(M-2)(M-3)(M-N)}$$ (56)

(Church 1926, p. 357; Carver 1930; Irwin and Kendall 1944; Kenney and Keeping 1951, p. 143), where $\gamma_1$ is the sample Skewness and $\gamma_2$ is the sample Kurtosis.

See also Gaussian Distribution, Kurtosis, Mean, Moment, Skewness, Variance

References

Carver, H. C. (Ed.). ``Fundamentals of the Theory of Sampling.'' Ann. Math. Stat. 1, 101-121, 1930.

Church, A. E. R. ``On the Means and Squared Standard-Deviations of Small Samples from Any Population.'' Biometrika 18, 321-394, 1926.

Irwin, J. O. and Kendall, M. G. ``Sampling Moments of Moments for a Finite Population.'' Ann. Eugenics 12, 138-142, 1944.

Kenney, J. F. and Keeping, E. S. Mathematics of Statistics, Pt. 2, 2nd ed. Princeton, NJ: Van Nostrand, 1951.

Kenney, J. F. and Keeping, E. S. ``The $k$-Statistics.'' §7.9 in Mathematics of Statistics, Pt. 1, 3rd ed. Princeton, NJ: Van Nostrand, pp. 99-100, 1962.