The research design of the National Evaluation of Sure Start (NESS) presented many challenges. These challenges were not unique, and as this chapter shows, have been and continue to be faced by others. However, Sure Start Local Programmes (SSLPs) are both a social intervention and a complex one. There is as yet no firm consensus around the best methodologies to use to evaluate the outcomes either of social interventions or of those in which the details of the treatment can vary between individuals and may be unknown to the evaluator. The purpose of this chapter is to review some of the main issues that have arisen in the literature on both social and complex interventions and which NESS confronted. It starts with a brief history of the evaluation of social interventions. It then considers the importance in social and other policy-related interventions of the questions why and how certain interventions work, as well as the standard question of whether they work. Finally it discusses the recent attempts to build a consensus around approaches to the central issues of the social, cultural and political context that may be fundamental in explaining observed outcomes, however those outcomes are measured.

The development of the evaluation of social Programmes

Rigorous methods for evaluating medical treatments have a long history, and were adopted in the US for looking at innovations in education during the 1950s and 1960s. The experimental or quasi-experimental evaluation of social programmes and interventions has a much more recent history. Some have suggested that it dates from 1967 with the New Jersey Negative Income Tax Experiment (Greenberg and Shroder, 2004), whereas others quote examples from earlier in the same decade (Shadish et al, 1991). Certainly, the seminal text (Campbell and Stanley, 1963) dates from the early part of the decade, which makes 1967 an unlikely date. Social programmes are generally defined as those whose intention is to train, educate or otherwise influence behaviour in order to achieve better outcomes for disadvantaged participants and for society more generally (Donaldson, 2003). Some health interventions have social impacts, but these are not necessarily intended, and are not always measured.