Having decided which aspects of behaviour to measure and chosen the suitable recording medium, you would be wise to check the quality of your measurements before you proceed to collect a lot of data. Measuring behaviour, like measuring anything else, can be done well or badly. When assessing how well behaviour is measured, two basic issues must be considered: reliability and validity – sometimes expressed as the distinction between ‘good’ measures and ‘right’ measures.

Reliability versus validity

Reliability concerns the extent to which measurement is repeatable and consistent: that is, free from random errors. An unbiased measurement consists of two parts: a systematic component, representing the true value of the variable, and a random component arising from imperfections in the measurement process. The smaller the error component, the more reliable the measurement. Reliable or good measures are those that measure a variable precisely and consistently.

At least four related factors determine how ‘good’ a measure is:

• Precision: How free are the measurements from random errors? The degree of precision is represented by the number of significant figures in the measurement. Note that precision and accuracy are not synonymous: accuracy concerns systematic error (bias) and can therefore be regarded as an aspect of validity (see below). A clock may tell the time with great precision (to within a millisecond), yet be inaccurate because it is set to the wrong time.

• Sensitivity: Are small changes in the true value invariably reflected by changes in the measured value?

• Resolution: What is the smallest change in the true value that can be detected?

• […]