info prev up next book cdrom email home

Central Limit Theorem

Let $x_1, x_2, \ldots, x_N$ be a set of $N$ Independent random variates and each $x_i$ have an arbitrary probability distribution $P(x_1,\ldots,x_N)$ with Mean $\mu_i$ and a finite Variance ${\sigma_i}^2$. Then the normal form variate

\begin{displaymath}
X_{\rm norm} \equiv {\sum_{i=1}^N x_i - \sum_{i=1}^N \mu_i\over\sqrt{\sum_{i=1}^N {\sigma_i}^2}}
\end{displaymath} (1)

has a limiting distribution which is Normal (Gaussian) with Mean $\mu=0$ and Variance $\sigma^2 = 1$. If conversion to normal form is not performed, then the variate
\begin{displaymath}
X\equiv{1\over N}\sum_{i=1}^N x_i
\end{displaymath} (2)

is Normally Distributed with $\mu_X=\mu_x$ and $\sigma_X={\sigma_x/\sqrt{N}}$. To prove this, consider the Inverse Fourier Transform of $P_X(f)$.
$\displaystyle {\mathcal F}^{-1}[P_X(f)]$ $\textstyle \equiv$ $\displaystyle \int_{-\infty}^\infty e^{2\pi i f X}p(X)\,dX$  
  $\textstyle =$ $\displaystyle \int_{-\infty}^\infty \sum_{n=0}^\infty {(2\pi i f X)^n\over n!} p(X)\,dX$  
  $\textstyle =$ $\displaystyle \sum_{n=0}^\infty {(2\pi i f)^n\over n!} \int_{-\infty}^\infty X^np(X)\,dX$  
  $\textstyle =$ $\displaystyle \sum_{n=0}^\infty {(2\pi i f)^n\over n!} \left\langle{X^n}\right\rangle{}.$ (3)

Now write
$\left\langle{X^n}\right\rangle{} = \left\langle{N^{-n}(x_1+x_2+\ldots+x_N)^n}\right\rangle{}$
$ = \int_{-\infty}^\infty N^{-n}(x_1+\ldots+x_N)^n p(x_1)\cdots p(x_N)\, dx_1\cdots dx_N,$

(4)
so we have
${\mathcal F}^{-1}[P_X(f)] = \sum_{n=0}^\infty {(2\pi i f)^n\over n!}\left\langle{X^n}\right\rangle{}$
$= \sum_{n=0}^\infty {(2\pi i f)^n\over n!} \int_{-\infty}^\infty N^{-n}(x_1+\ldots+x_N)^n p(x_1)\cdots p(x_N)\,dx_1\cdots dx_N$
$= \int_{-\infty}^\infty \sum_{n=0}^\infty \left[{2\pi i f(x_1+\ldots+x_N)\over N}\right]^n {1\over n!} p(x_1)\cdots p(x_N)\,dx_1\cdots dx_N$
$= \int_{-\infty}^\infty e^{2\pi i f(x_1+\ldots+x_N)/N} p(x_1)\cdots p(x_N)\,dx_...
... dx_N \cdots \left[{\int_{-\infty}^\infty e^{2\pi ifx_N/N} p(x_N)\,dx_N}\right]$
$= \left[{\int_{-\infty}^\infty e^{2\pi i f x/N}p(x)\,dx}\right]^N$
$= \left\{{\int_{-\infty}^\infty \left[{1+\left({2\pi i f \over N}\right)x +{1\over 2}\left({2\pi i f \over N}\right)^2 x^2 +\ldots}\right]p(x)\,dx}\right\}^N$
$= \left[{\int_{-\infty}^\infty p(x)\,dx+{2\pi i f\over N}\int_{-\infty}^\infty ...
...)^2\over 2N^2} \int_{-\infty}^\infty x^2p(x)\,dx+{\mathcal O}(N^{-3})}\right]^N$
$= \left[{1+{2\pi i f\over N}\left\langle{x}\right\rangle{}-{(2\pi f)^2\over 2N^2}\left\langle{x^2}\right\rangle{}+{\mathcal O}(N^{-3})}\right]^N$
$= \mathop{\rm exp}\nolimits \left\{{N\ln\left[{1+{2\pi i f\over N}\left\langle{...
...er 2N^2}\left\langle{x^2}\right\rangle{}+{\mathcal O}(N^{-3})}\right]}\right\}.$ (5)
Now expand
\begin{displaymath}
\ln(1+x)=x-{\textstyle{1\over 2}}x^2+{\textstyle{1\over 3}}x^3+\ldots,
\end{displaymath} (6)

so


$\displaystyle {\mathcal F}^{-1}[P_X(f)]$ $\textstyle \approx$ $\displaystyle \mathop{\rm exp}\nolimits \left\{{N\left[{{2\pi if\over N}\left\l...
...ver N^2}\left\langle{x}\right\rangle{}^2 +{\mathcal O}(N^{-3})}\right]}\right\}$  
  $\textstyle =$ $\displaystyle \mathop{\rm exp}\nolimits \left[{2\pi if\left\langle{x}\right\ran...
...angle{}-\left\langle{x}\right\rangle{}^2)\over 2N}+{\mathcal O}(N^{-2})}\right]$  
  $\textstyle \approx$ $\displaystyle \mathop{\rm exp}\nolimits \left[{2\pi if\mu_x-{(2\pi f)^2{\sigma_x}^2\over 2N}}\right],$ (7)

since
$\displaystyle \mu_x$ $\textstyle \equiv$ $\displaystyle \left\langle{x}\right\rangle{}$ (8)
$\displaystyle {\sigma_x}^2$ $\textstyle \equiv$ $\displaystyle \left\langle{x^2}\right\rangle{}-\left\langle{x}\right\rangle{}^2.$ (9)

Taking the Fourier Transform,
$\displaystyle P_X$ $\textstyle \equiv$ $\displaystyle \int_{-\infty}^\infty e^{-2\pi ifx}{\mathcal F}^{-1}[P_X(f)]\,df$  
  $\textstyle =$ $\displaystyle \int_{-\infty}^\infty e^{2\pi if(\mu_x-x)-(2\pi f)^2{\sigma_x}^2/2N}\,df.$ (10)

This is of the form
\begin{displaymath}
\int_{-\infty}^\infty e^{iaf-bf^2}\,df,
\end{displaymath} (11)

where $a\equiv 2\pi(\mu_x-x)$ and $b\equiv (2\pi\sigma_x)^2/2N$. But, from Abramowitz and Stegun (1972, p. 302, equation 7.4.6),
\begin{displaymath}
\int_{-\infty}^\infty e^{iaf-bf^2}\,df = e^{-a^2/4b} \sqrt{\pi\over b}.
\end{displaymath} (12)

Therefore,
$\displaystyle P_X$ $\textstyle =$ $\displaystyle \sqrt{\pi\over{(2\pi\sigma_x)^2\over 2N}} \mathop{\rm exp}\nolimits \left\{{-[{2\pi(\mu_x-x)]^2}\over {4{(2\pi\sigma_x)^2\over 2N}}}\right\}$  
  $\textstyle =$ $\displaystyle \sqrt{2\pi N\over 4\pi^2{\sigma_x}^2} \mathop{\rm exp}\nolimits \left[{-{4\pi^2(\mu_x-x)^2 2N\over 4\cdot 4\pi^2{\sigma_x}^2}}\right]$  
  $\textstyle =$ $\displaystyle {\sqrt{N}\over\sigma_x\sqrt{2\pi}} e^{-(\mu_x-x)^2N/2{\sigma_x}^2}.$ (13)

But $\sigma_X={\sigma_x/\sqrt{N}}$ and $\mu_X=\mu_x$, so
\begin{displaymath}
P_X = {1\over\sigma_X\sqrt{2\pi}} e^{-(\mu_X-x)^2/2{\sigma_X}^2}.
\end{displaymath} (14)


The ``fuzzy'' central limit theorem says that data which are influenced by many small and unrelated random effects are approximately Normally Distributed.

See also Lindeberg Condition, Lindeberg-Feller Central Limit Theorem, Lyapunov Condition


References

Abramowitz, M. and Stegun, C. A. (Eds.). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, 9th printing. New York: Dover, 1972.

Spiegel, M. R. Theory and Problems of Probability and Statistics. New York: McGraw-Hill, pp. 112-113, 1992.

Zabell, S. L. ``Alan Turing and the Central Limit Theorem.'' Amer. Math. Monthly 102, 483-494, 1995.



info prev up next book cdrom email home

© 1996-9 Eric W. Weisstein
1999-05-26