In this chapter we describe a relatively new result – a consequence of positive dynamical entropy of a process. It concerns the behavior of the return time random variables Rn(x) for large n, the same as treated by the Ornstein–Weiss Return Times Theorem, but in a complementary manner. The theorem has a very interesting interpretation, easy to articulate in a language accessible also to nonspecialists. Yet, as usual on such occasions, one has to be very cautious and not get enticed into pushing the conclusions too far. We begin this chapter with a short historical note concerning the debate on the Law of Series in the colloquial meaning. We explain how the Ergodic Law of Series contributes to this debate. Then we pass to the mathematical proof preceded by introducing a number of ergodic-theoretic tools.

History of the Law of Series

In the colloquial language, a “series” happens when a random event, usually extremely rare, is observed surprisingly often throughout a period of time. Even two repetitions, one shortly after another, are often interpreted as a “series.” The Law of Series is the belief that such series happen more often than they should by “pure chance” (whatever that means). This belief is usually associated with another; that there exists some unexplained force or rule behind this “law.” A number of idioms, such as “run of good luck” or “run of misfortune,” or proverbs like “misfortune never comes alone,” exist in nearly all languages, which confirms that people have been noticing this kind of mystery for a long time.

The topological entropy of a dynamical system is a rather crude measurement of its complexity. In order to thoroughly understand the dynamics it is essential not only to replace the topological entropy by the entropy function µ ↦ h(µ, T) on invariant measures, but also to replace the topological entropy detectable in a given resolution, say h1(T, ε) or h2(T, ε), by some function, say µ ↦ h(µ, T, ε), reflecting the dynamical entropy of each measure detectable in that resolution. Such a function has not been presented in this book yet. As we shall see, there is an essential difference between entropy of a measure with respect to a measurable resolution, i.e., a partition (even if we require that the cells have diameters bounded by ε) and the entropy of a measure with respect to a topological resolution, which we are about to define. The difference is in semicontinuity properties and in the “type of convergence” to the entropy function as the resolution refines. This type of convergence turns out to be the most important feature in the part of entropy theory leading to digitalization and data compression of topological dynamical systems.

The material in this section is based mainly on the paper [Downarowicz, 2005a].

The type of convergence

In this section we will try to understand what it means, for two monotone nets of real-valued functions converging to the same limit function, to converge “the same way,” or to represent the same “type of convergence.”

There are clearly some gaps in the theory of entropy for operators. At least as long as we seek for similarities with the analogous theory for dynamical systems. It can be hoped that these similarities reach further than we know.

Questions on doubly stochastic operators

In the entropy theory of doubly stochastic operators, the fundamental missing issue is a relevant information theory. The notion of operator entropy is created without reference to any reasonable notion of information function. Clearly, it is most desirable that such a function depends on the family of functions J and is defined directly on the phase space X, however, a compromise solution with this function defined on the product X × [0, 1] seems also acceptable. In any case, the static entropy should be the integral of the information function with respect to the appropriate measure (µ or µ × λ, respectively). Needless to say, the notion should coincide with the classical one for a family of characteristic functions of a partition. Of course, the best justification of this notion would be an analog (generalization) of the Shannon–McMillan–Breiman Theorem. Let us verbalize the problem:

Question 13.1.1 Is there a meaningful notion of an information function with respect to a family of functions, such that the static entropy is its integral? Does a generalization of the Shannon–McMillan–Breiman Theorem hold for doubly stochastic operators?

Smooth dynamics is concerned with smooth transformations of Riemannian manifolds. It is one of the most exploited areas of dynamical systems, and many papers and books are devoted to this branch. We refer the reader to the book [Katok and Hasselblatt, 1995] as a primary reference. These studies require background in smooth geometry, hyperbolic dynamics, foliation theory, and many more. Also here the entropy is one of the most important subjects.

As this book is designed to be self-contained, and there is obviously no room to provide all that background, deprived of the basic tools, we will actually be able to do very little. In fact we will prove only one rather elementary fact: an estimate of the measure-theoretic entropy in terms of characteristic exponents, a weaker version of the Margulis–Ruelle estimate of entropy for ergodic measures. Besides that, we will only state several results without a proof: the Pesin Entropy Formula, the Buzzi–Yomdin estimate of the topological tail entropy for Cr maps, and some results and questions concerning symbolic extensions.

Margulis–Ruelle Inequality and Pesin Entropy Formula

Let T : M → M be a C1 transformation of a compact Riemannian manifold M of dimension dim. We refrain from providing the detailed definition of the derivative DxT of T at x, which is a transformation defined on the tangent bundle of M.

Zero-dimensional dynamical systems

Due to the existence of arbitrarily fine covers which are at the same time partitions (by disjoint open sets – we call them “clopen covers”), zero-dimensional dynamical systems allow us to switch between the topological and measuretheoretic dynamical notions easier than any other systems.

In order to understand zero-dimensional dynamical systems it suffices to understand subshifts and their countable joinings.

Definition 7.1.1 By a subshift we mean a topological dynamical system (X, T, S), where X is a closed, shift-invariant subset of ∧S′, ∧ is a finite set (called alphabet), and T denotes the shift map σ restricted to X. Both S′ and S stand for either ℕ0 or ℤ but we require that S ⊂ S′. Shift-invariance of X is understood as σ(X) ⊂ X or σ(X) = X, depending on whether S = ℕ0 or ℤ, respectively.

Note that S′ = ℤ implies that T is injective.

Definition 7.1.2 By a symbolic array system we will mean a topological dynamical system (X, T, S), where X consists of symbolic arrays of the form x = (xk,n)k∈ℕ,n∈S′ (S ⊂ S′ like for subshifts), where for each k, n, xk,n belongs to a finite set ∧k not depending on n or x ∈ X, called the alphabet in row k. The transformation T is the restriction to X of the left shift map on arrays

The above definition implicitly requires that X be closed and invariant under T (recall that the meaning of invariance depends on S).

Given a topological dynamical system (X, T, S) of finite entropy, we are interested in computing the “amount of information” per unit of time transferred in this system. Suppose we want to compute verbatim the vertical data compression, as described in the Introduction (Section 0.3.5): the logarithm of the minimal cardinality of the alphabet allowing the system to be losslessly encoded in real time. Since we work with topological dynamical systems, we want the coding to respect not only the measurable, but also the topological, structure. There is nothing like the Krieger Generator Theorem in topological dynamics. A topological dynamical system of finite entropy (even invertible) need not be conjugate to a subshift over a finite alphabet. Thus, we must first create its lossless digitalization, which has only one possible form: a subshift, in which the original system occurs as a factor. In other words, we are looking for a symbolic extension. Only then can we try to optimize the alphabet. It turns out that such a vertical data compression is not governed by the topological entropy only by a different (possibly much larger) parameter.

What are symbolic extensions?

First of all, notice that if a system has infinite topological entropy, it simply does not have any symbolic extensions, as these have finite topological entropy. We do not accept extensions in form of subshifts over infinite alphabets, even countable, like ℕ0∪ ∞.

This book is designed as a comprehensive lecture on entropy in three major types of dynamics: measure-theoretic, topological and operator. In each case the study is restricted to the most classical case of the action of iterates of a single transformation (or operator) on either a standard probability space or on a compact metric space. We do not venture into studying actions of more general groups, dynamical systems on noncompact spaces or equipped with infinite measures. On the other hand, we do not restrict the generality by adding more structure to our spaces. The most structured systems addressed here in detail are smooth transformations of the compact interval. The primary intention is to create a self-contained course, from the basics through more advanced material to the newest developments. Very few theorems are quoted without a proof, mainly in the chapters or sections marked with an asterisk. These are treated as “nonmandatory” for the understanding of the rest of the book, and can be skipped if the reader chooses. Our facts are stated as generally as possible within the assumed scope, and wherever possible our proofs of classical theorems are different from those found in the most popular textbooks. Several chapters contain very recent results for which this is a textbook debut.

We assume familiarity of the reader with basics of ergodic theory, measure theory, topology and functional analysis. Nevertheless, the most useful facts are recalled either in the main text or in the appendix.