Probability theory has a long, eventful, and still not fully settled history. As pointed out in [63]: ‘For all human history, people have invented methods for coming to terms with the seemingly unpredictable vicissitudes of existence … Oracles, amulets, and incantations belonged to the indispensable techniques for interpreting and influencing the fate of communities and individuals alike … In the place of superstition there was to be calculation — a project aiming at nothing less than the rationalization of fortune. From that moment on, there was no more talk of fortune but instead of this atrophied cousin: chance.’

The only consistent mathematical way to handle chance, or rather probability, is provided by the rules of (Bayesian) probability theory. But what does the notion ‘probability’ really mean? Although it might appear, at flrst sight, as obvious, it actually has different connotations and definitions, which will be discussed in the following sections.

For the sake of a smooth introduction to probability theory, we will forego a closer definition of some technical terms, as long as their colloquial meaning suffices for understanding the concepts. A precise definition of these terms will be given in a later section.

Classical definition of ‘probability’

The first quantitative definition of the term ‘probability’ appears in the work of Blaise Pascal (1623–1662) and Pierre de Fermat (1601–1665). Antoine Gombauld Chevalier de Mere, Sieur de Baussay (1607–1685) pointed out to them that ‘…mathematics does not apply to real life’.