Introduction

What can reasonably be inferred from data gathered in an experiment? This simple question lies at the heart of experimentation, as an experiment can be judged by how much insight can be drawn from data. An experiment may have a broad or narrow focus, and may be designed to:

• challenge a relationship that has an established theoretical basis;

• critically examine a discovery that results from ‘chance’ observations;

• check for drift in an instrument;

• compare analysis of materials carried out in two or more laboratories.

Such general goals give way to specific questions that we hope can be answered by careful analysis of data gathered in well designed experiments. Questions that might be asked include:

• is there a linear relationship between quantities measured in an experiment;

• could the apparent correlation between variables have occurred ‘by chance’;

• does a new manufacturing process produce lenses with focal lengths that are less variable than the old manufacturing process;

• is there agreement between two methods used to determine the concentration of iron in a specimen;

• has the gain of an instrument changed since it was calibrated?

It is usually not possible to answer these questions with a definite ‘yes’ or definite ‘no’. Though we hope data gathered during an experiment will provide evidence as to which reply to favour, we must be satisfied with answers expressed in terms of probability.

Consider a situation in which a manufacturer supplies an instrument containing an amplifier with a gain specified as 1000. Would it be reasonable to conclude that the instrument is faulty or needs recalibrating if the gain determined by a single measurement is 995? It is possible that random errors inherent in the measurement process, as revealed by making repeat measurements of the gain, would be sufficient to explain the discrepancy between the ‘expected’ value of gain of 1000 and the ‘experimental’ value of 995. What we would really like to know is whether, after taking into account the scatter in the values of the gain obtained through repeat measurements, the difference between the value we have reason to expect will occur and those actually obtained through experiment or observation is ‘significant’.