In physics, the word entropy has important physical implications as the amount of ``disorder'' of a system. In mathematics, a
more abstract definition is used. The (Shannon) entropy of a variable is defined as

where is the probability that is in the state , and is defined as 0 if . The joint entropy of variables , ..., is then defined by

**References**

Ott, E. ``Entropies.'' §4.5 in *Chaos in Dynamical Systems.*
New York: Cambridge University Press, pp. 138-144, 1993.

© 1996-9

1999-05-25