Why do we need analysis of variance?

We have considered already (Chapter 3) how the difference between two sample means may be tested for significance. In practice we are often faced with the need to examine the differences between the means of more than two samples. On meeting this problem for the first time many biologists react by testing the significance of each difference separately by means of the t test. This is neither efficient nor proper.

That it is not proper can be understood by recalling the nature of a test of significance. When we apply such a test at, for example, the 5% level of significance there is a probability of 1:20 of our concluding that the two population means are different when in fact they are equal (i.e. of making a Type 1 error). This is so for each comparison we make and it may be calculated (as 1–0.9521) that for an experiment with 7 treatments, and hence 21 comparisons, there is a probability of 0.66 of making at least one Type 1 error. This is clearly not good enough. It would be possible by using, for instance, the 1% level of significance to reduce the probability but this would greatly increase the risk of rejecting as nonsignificant differences which were real (i.e. of making a Type 2 error). What is needed is a technique which examines the variation within the whole group of sample means.