Information Dimension

Define the ``information function'' to be

$$I \equiv -\sum_{i=1}^{N} P_i(\epsilon)\ln[P_i(\epsilon)],$$ (1)

where $P_i(\epsilon)$ is the Natural Measure, or probability that element $i$ is populated, normalized such that

$$\sum_{i=1}^N P_i(\epsilon) = 1.$$ (2)

The information dimension is then defined by

$$d_{\rm inf} \equiv - \lim_{\epsilon\to 0^+} {I\over\ln(\epsilon)}$$

$$= \lim_{\epsilon\to 0^+} \sum_{i=1}^{N} {P_i(\epsilon)\ln[P_i(\epsilon)]\over\ln(\epsilon)}.$$ (3)

If every element is equally likely to be visited, then $P_i(\epsilon)$ is independent of $i$, and

$$\sum_{i=1}^N P_i(\epsilon) = N P_i(\epsilon) = 1,$$ (4)

so

$$P_i(\epsilon) = {1\over N},$$ (5)

and

$$d_{\rm inf} = \lim_{\epsilon\to 0^+} {\sum\limits_{i=1}^N {1\over N}\ln\left({1\over N}\right)\over\ln\epsilon}$$

$$= \lim_{\epsilon\to 0^+} {\ln(N^{-1})\over\ln \epsilon} = -\lim_{\epsilon\to 0^+} {\ln N\over\ln(\epsilon)} = d_{\rm cap},$$ (6)

where $d_{\rm cap}$ is the Capacity Dimension.

See also Correlation Exponent

References

Farmer, J. D. ``Chaotic Attractors of an Infinite-dimensional Dynamical System.'' Physica D 4, 366-393, 1982.

Nayfeh, A. H. and Balachandran, B. Applied Nonlinear Dynamics: Analytical, Computational, and Experimental Methods. New York: Wiley, pp. 545-547, 1995.