Placing Experimental Research in Context

In typical discussions of estimating causality, social scientists who come from a statistics perspective often begin with a review of the experimental approach in an idealized setting that rarely exists, argue that the experimental approach as idealized is not feasible in social science, and then go on to discuss how causality is measured in observational data. For example, Winship and Morgan (1999) begin their otherwise excellent review of the literature in social science on measuring the effects of causes with the statement (p. 660), “sociologists, economists, and political scientists must rely on what is now known as observational data – data that have been generated by something other than a randomized experiment – typically surveys, censuses, or administrative records.” This tendency to bracket off measuring causality in experimental social science from measuring causality in observational data presumes that experiments are “either-or propositions”: a researcher can either conduct an “ideal” experiment, which we argue in this book would not be ideal for many questions in which political scientists are interested, or work with observational data.

Most of experimental social science is not the hypothesized ideal or classical experiment, usually with good reason. The bracketing off prevents a discussion of how causality is measured in experiments as they exist in social science and a realistic comparison of those methods to research with observational data.