This appendix gives a brief guide to the probability theory needed at various stages in the book. The following is too brief to be intended as a first exposure to probability, but rather is here to act as a reference. Good introductory books on probability include Bishop, and Duda, Hart and Stork.

Discrete probabilities

Discrete events are the simplest to interpret. For example, what is the probability of

• it raining tomorrow?

• a 6 being thrown on a die?

Probability can be thought of as the chance of a particular event occurring. We limit the range of our probability measure to lie in the range 0 to 1, where

• lower numbers indicate that the event is less likely to occur, 0 indicates it will never occur;

• higher numbers indicate that the event is more likely to occur, 1 indicates that the event will definitely occur.

We like to think that we have a good grasp of both estimating and using probability. For simple cases such as “will it rain tomorrow?” we can do reasonably well. However, as situations get more complicated things are not always so clear. The aim of probability theory is to give us a mathematically sound way of inferring information using probabilities.

Discrete random variables

Let some event have have M possible outcomes. We are interested in the probability of each of these outcomes occurring.