The current framework of hypothesis testing is largely due to the work of Neyman and Pearson in the late 1920s, early 30s, complementing Fisher's work on estimation. As in estimation, we begin by postulating a statistical model but instead of seeking an estimator of θ in Θ we consider the question whether θ ∈ Θ0 ⊂ Θ or θ ∈ Θ1, = Θ - Θ0 is mostly supported by the observed data. The discussion which follows will proceed in a similar way, though less systematically and formally, to the discussion of estimation. This is due to the complexity of the topic which arises mainly because one is asked to assimilate too many concepts too quickly just to be able to define the problem properly. This difficulty, however, is inherent in testing, if any proper understanding of the topic is to be attempted, and thus unavoidable. Every effort is made to ensure that the formal definitions are supplemented with intuitive explanations and examples. In Sections 14.1 and 14.2 the concepts needed to define a test and some criteria for ‘good’ tests are discussed using a simple example. In Section 14.3 the question of constructing ‘good’ tests is considered. Section 14.4 relates hypothesis testing to confidence estimation, bringing out the duality between the two areas. In Section 14.5 the related topic of prediction is considered.

Testing, definitions and concepts

Let X be a random variable (r.v.) defined on the probability space (S, ℱ, P(·)) and consider the statistical model associated with X:

1. (i) Φ = {f(x;θ), θ∈Θ};

2. (ii) X = (X1, X2, …, Xn)′ is a random sample, from f(x; θ).