ALTERNATIVE EVALUATION DESIGNS

CBAs of any intervention require comparisons between alternatives: the program or policy that is subject to the CBA is compared to the situation that would exist without the program (the so-called counterfactual), and impacts are measured as differences in outcomes (e. g., in health status or earnings) between the two situations. The term internal validity refers to whether this measured difference can be appropriately attributed to the program being evaluated. Internal validity, in turn, depends on the particular way in which the comparison between the program and the situation without the program is made. There are numerous ways in which this comparison can be made. Researchers usually refer to the specific scheme used for making comparisons in order to measure impacts as an evaluation design.

Diagrams that represent five commonly used evaluation designs,2 as well as brief summaries of the advantages and disadvantages of each of these designs, appear in Table 12-1. In these diagrams, the symbol O represents an outcome measurement point, X represents a treatment point, and R indicates that subjects were assigned randomly to treatment and control groups.

The evaluation designs that are listed in Table 12-1 are not the only ones that exist. There are numerous others. But these designs provide a good sampling of the major alternatives. So that we can make our discussion of them as concrete as possible, we assume that they all pertain to alternative ways in which a program for training the unemployed might be evaluated. In this context, “being in the treatment group” means enrollment in the training program.