Reliability versus validity

Measuring behaviour, like measuring anything else, can be done well or badly. When assessing how well behaviour is measured, two basic issues must be considered: reliability and validity (sometimes expressed as the distinction between ‘good’ measures and ‘right’ measures).

1 Reliability concerns the extent to which measurement is repeatable and consistent; that is, free from random errors. An unbiased measurement consists of two parts: a systematic component, representing the true value of the variable, and a random component due to imperfections in the measurement process (see section 9.K and Martin & Kraemer, 1987). The smaller the error component, the more reliable the measurement. Reliable measures, sometimes referred to as good measures, are those which measure a variable precisely and consistently.

At least four related factors determine how ‘good’ a measure is:

(a) Precision: How free are the measurements from random errors? The degree of precision is represented by the number of significant figures in the measurement. Note that precision and accuracy are not synonymous: accuracy concerns systematic error (bias) and can therefore be regarded as an aspect of validity. A clock may tell the time with great precision (to within a millisecond), yet be inaccurate because it is set to the wrong time.

(b) Sensitivity: Do small changes in the true value invariably lead to changes in the measured value?

(c) Resolution: What is the smallest change in the true value that can be detected?