The results presented in this book have been written in the desire that practitioners will use them. We have tried therefore to illustrate the use of the theory in a systematic and accessible way, and so this book concentrates not only on the theory of general space Markov chains, but on the application of that theory in considerable detail.

We will apply the results which we develop across a range of specific applications: typically, after developing a theoretical construct, we apply it to models of increasing complexity in the areas of systems and control theory, both linear and nonlinear, both scalar and vector valued; traditional “applied probability” or operations research models, such as random walks, storage and queueing models, and other regenerative schemes; and models which are in both domains, such as classical and recent time series models.

These are not given merely as “examples” of the theory: in many cases, the application is difficult and deep of itself, whilst applications across such a diversity of areas have often driven the definition of general properties and the links between them. Our goal has been to develop the analysis of applications on a step-by-step basis as the theory becomes richer throughout the book.

To motivate the general concepts, then, and to introduce the various areas of application, we leave until Chapter 3 the normal and necessary foundations of the subject, and first introduce a cross-section of the models for which we shall be developing those foundations.