2 - Continuous-time Markov chains
Published online by Cambridge University Press: 11 November 2010
Summary
Q-matrices and transition matrices
Markov processes specialists like to do it with chains.
(From the series ‘How they do it’.)Definition 2.1.1 A Q-matrix on a finite or countable state space I is a real-valued matrix (qij, i, j ∈ I) with:
non-positive diagonal entries qii ≤ 0, i ∈ I,
non-negative off-diagonal entries qij ≥ 0, i ≠ j, i, j ∈ I,
the row zero-sum condition: −qii = Σj∈I:j≠iqij, i.e. Σjqij = 0 for all i ∈ I.
For i ≠ j, the value qij represents the jump, or transition rate from state i to j. The value −qii = Σj:j≠iqij is denoted by qi (we will see that it represents the total jump, or exit rate from state i). A Q-matrix will be denoted by Q (a common abuse of notation). As in Chapter 1, we will denote by I the unit matrix.
In a general theory of countable continuous-time Markov chains, the row zerosum condition Σj∈I:j≠iqij = −qii presumes that the series Σj:j≠iqij < ∞. However, a substantial part of the theory can be developed when the equality in this condition is relaxed to the upper bound Σjqij ≤ 0, i.e. qi ≥ Σj:j≠iqij for all i ∈ I. Then a Q-matrix satisfying the row zero-sum condition is called conservative; we will omit this term in the present volume, as we will not consider non-conservative Q-matrices.
- Type
- Chapter
- Information
- Probability and Statistics by Example , pp. 185 - 348Publisher: Cambridge University PressPrint publication year: 2008