The usefulness of probability distributions was well established by such early mathematicians as Laplace (1749–1827) and Gauss (1777–1855). The idea that frequency distributions could be explained as a practical consequence of the laws of probability applied to everyday matters, seized the imagination of the pioneers of mathematical statistics. Since a probability distribution is by its nature, in most instances, composed of an infinite number of items, and frequency distributions by their nature are composed of a finite number of items, these latter had to be thought of as samples from an underlying theoretical probability distribution. The problem that then arose was how to describe a probability distribution given only a sample from it. The mathematical difficulties of this seemed immense and such steps as were taken needed experimental verification to give the early workers confidence. Thus was born the sampling experiment. A close approximation to a probability distribution was created, samples were taken, combined and transformed in suitable ways and the resulting frequency chart of sampled values compared with the predictions of theory. Although mathematical techniques have developed to levels of sophistication that would astonish earlier workers, the value of sampling experiments in mathematical statistics still remains.

There are two main types of distribution from which samples are required. The first is where the statistical variable takes a continuous form, giving rise to a continuous probability density function; the second is where the statistical variable can take a discrete number of values.