Introduction
Brazil has long been ravaged by an undeclared civil war. According to the Citizen Council on Public Security and Criminal Justice, a Mexican think tank, nineteen of the fifty most violent cities in the world are located in Brazil (Consejo Ciudadano para la Seguridad Pública y Justicia Penal 2014).Footnote 1 The 2014 Violence Map survey shows that 56,337 people were murdered in Brazil in 2012 alone, the highest incidence rates of intentional homicides on the planet (Reference WaiselfiszWaiselfisz 2014; United Nations Office on Drugs and Crime 2013). Paradoxically, the sharp rise in lethal violence has occurred during Brazil’s longest period of political openness (Reference AhnenAhnen 2003; Reference PinheiroPinheiro 2000, Reference Pinheiro2001). Murder rates have almost doubled over three decades of democracy, jumping from 15 homicides per 100,000 people in 1985 to roughly 29 per 100,000 in 2012 (Reference WaiselfiszWaiselfisz 2014).Footnote 2
São Paulo has traditionally occupied a key position in Brazil’s violence statistics. It is the country’s richest and most densely populated state, and in the 1990s its homicide rate was roughly 50 percent higher than the national average (Reference Barata and RibeiroBarata and Ribeiro 2000, 120). Some areas of the namesake capital city had even worse numbers. Between 1996 and 1999, the ramshackle districts of Jardim São Luiz and Jardim Ângela had respectively 103 and 116 violent deaths per 100,000 residents (Reference Cardia, Adorno and PoletoCardia, Adorno, and Poleto 2003, 8), figures that placed them among the deadliest neighborhoods on the globe (World Health Organization 2015).
Nevertheless, the state of São Paulo has experienced a drastic reduction in homicides during recent years (Reference CamargoCamargo 2007). The decline is so remarkable that some authors have called it “the great homicide drop” (Reference Goertzel and KahnGoertzel and Kahn 2009). The city of São Paulo, which is currently home to about 11 million inhabitants, provides a telling example. Over a span of only seven years (2000–2007), the number of annual violent deaths in the capital fell from 5,979 to 1,311, a 78 percent decrease.Footnote 3 Significantly, São Paulo became the safest capital in Brazil (Reference WaiselfiszWaiselfisz 2011).
São Paulo’s success should be attributed to local factors. From 1999 onward, the state government created or expanded a number of policies that have arguably contributed to the decrease in criminality. In a move coherent with the basic tenets of the economics of crime (e.g., Reference BeckerBecker 1968; Reference Cornish and ClarkeCornish and Clarke 2014), the administration increased the certainty and the intensity of punishment to discourage potential offenders. Among other measures, the government implemented strict gun control policies (Reference Goertzel and KahnGoertzel and Kahn 2009), raised incarceration rates (Reference SallaSalla 2007), and imposed harsher sentences on those convicted of a crime (Reference Carvalho and FreireCarvalho and Freire 2005).
But whereas several authors acknowledge the effectiveness of these policies, few quantitative studies have gone beyond statistical correlations to justify their arguments. In the case of São Paulo, a major difficulty is separating the state’s particular time trend to that of Brazil. Ideally, one should compare São Paulo to a control case that shares the same characteristics of the existing state, except that it has not been subjected to the specific set of policies implemented by the São Paulo government. This thought exercise, which emulates the logic of a controlled experiment (Reference Angrist and PischkeAngrist and Pischke 2008; Reference Imbens and RubinImbens and Rubin 2015; Reference HollandHolland 1986; Reference Morgan and WinshipMorgan and Winship 2014), would allow practitioners to untangle the effects of homicide reduction programs from other potential confounders.
In this article, I employ the synthetic control method (henceforth SCM) to approximate this experimental ideal and measure the total causal effect of post-1999 public policies on São Paulo homicide rates. The method consists of creating an artificial counterfactual to estimate the impact of a given intervention on a unit of interest. SCM has gained widespread acceptance in many fields, having been successfully applied in political science (Reference Abadie, Diamond and HainmuellerAbadie, Diamond, and Hainmueller 2014; Reference MontalvoMontalvo 2011), economics (Reference Billmeier and NanniciniBillmeier and Nannicini 2013; Reference Coffman and NoyCoffman and Noy 2012; Reference Jinjarak, Noy and ZhengJinjarak, Noy, and Zheng 2013), education studies (Reference HinrichsHinrichs 2012), and public health science (Reference Heim and LurieHeim and Lurie 2014). However, SCM has rarely, if ever, been used to evaluate homicide prevention strategies in São Paulo, despite being a useful tool for this particular type of question. SCM was specifically designed for situations where there is only one treated unit of interest, no readily available counterfactual, and no certainty as to whether the treated and the control units follow parallel trends after the intervention (Reference Abadie and GardeazabalAbadie and Gardeazabal 2003; Reference Abadie, Diamond and HainmuellerAbadie, Diamond, and Hainmueller 2010, Reference Abadie, Diamond and Hainmueller2014). Moreover, SCM also has some of the desirable properties of other popular causal inference tools such as differences-in-differences (Reference Angrist and PischkeAngrist and Pischke 2008; Reference Bertrand, Duflo and MullainathanBertrand, Duflo, and Mullainathan 2004) and matching estimators (Reference Dehejia and WahbaDehejia and Wahba 2002; Reference Ho, Imai, King and StuartHo et al. 2007; Reference RubinRubin 1973; Reference StuartStuart 2010).
I find that from 1999 to 2009, about 20,000 lives were saved in São Paulo. When compared to a synthetic counterfactual, São Paulo’s actual homicide rates were less than 50 percent of what would be expected in the absence of policy implementation (15 versus 32 homicides per 100,000 people). Additional tests confirm the robustness of the results and indicate a 96.3 percent chance of a causal effect in the intervention period.
Theoretical Background
Deterrence, information, and the drop in homicides
A myriad of explanations have been proposed for the fall in homicide rates in São Paulo. Some authors have stressed the importance of long-term factors on local levels of violence. Mello and Schneider (Reference Mello and Schneider2010) claim that the shrinking proportion of males in the fifteen to twenty-five age bracket has led to fewer violent deaths at both state and city levels. Hughes (Reference Hughes2004) argues that São Paulo’s spatial segregation patterns have had a lasting impact on murder rates. Barata and Ribeiro (Reference Barata and Ribeiro2000), in turn, posits that macroeconomic conditions, mainly inequality indicators, are positively correlated with violent crime in São Paulo.
Structural variables have likely been important in reducing violence, but the role of public policies should not be underestimated. The Brazilian Social Democracy Party (Partido da Social Democracia Brasileira, PSDB), which has ruled São Paulo since 1995, has repeatedly asserted its commitment to reducing urban crime throughout the state (Reference BuenoBueno 2014). In 1998, former governor Mário Covas, then running for reelection, set the ambitious goal of “slashing criminality rates in half” during his second term in office (Reference SantosSantos 2008). This commitment was then followed by his vice-governor and successor, Geraldo Alckmin, who has expanded those measures and taken a notoriously tough stance on crime (Reference FeltranFeltran 2012a).
Methods of crime prevention have received considerable attention from the authorities. Firstly, the São Paulo government significantly increased incarceration rates in the past decade (Reference SallaSalla 2007). The state currently holds around 200,000 convicts in prison (35 percent of Brazil’s inmate population) and adds another 15,000 inmates to the official statistics every year (Brasil de Fato 2013). Furthermore, prisoners have also become subject to harsher legal punishments. The São Paulo administration has also been making large use of the Regime Disciplinar Diferenciado (Special Disciplinary Regime), which provides for up to 360 days of solitary confinement for disobeying the law (Reference Carvalho and FreireCarvalho and Freire 2005).
Secondly, the state government has successfully enforced a ban on gun possession in São Paulo. Studies show that this policy has been effective in reducing homicides resulting from both drug-related crimes and domestic disputes (Reference Goertzel and KahnGoertzel and Kahn 2009; Reference Kahn and ZaneticKahn and Zanetic 2005). Furthermore, the impact of the Brazil’s 2003 National Disarmament Act was particularly pronounced in São Paulo. Cerqueira and Mello (Reference Cerqueira and Mello2013) argue that between 2005 and 2007 the enforcement of the anti-firearm legislation was responsible for saving between 2,000 to 2,750 lives in cities with more than half a million inhabitants in the state of São Paulo.
This set of policies is largely in line with the rational choice theory of crime (e.g., Reference BeckerBecker 1968; Reference EhrlichEhrlich 1975; Reference LevittLevitt 1996, Reference Levitt1997; Reference PaternosterPaternoster 2010). The rational choice school posits that criminals are motivated by utilitarian cost-benefit analysis. Individuals calculate what the possible trade-offs are between the benefit of the committing a crime and the risk of being punished for it. Criminal offenders, therefore, only differ from noncriminals in their choices (Reference NaginNagin 2007). To reduce criminality, policy-makers have to ensure that the costs of committing a crime outweigh the eventual utility an individual derives from it.
Deterrence measures have been complemented by investments in police intelligence. In 1999, the state administration created a new system for crime prevention, Infocrim (Reference RissoRisso 2014, 3) The system gathers geo-coded information on homicides and maps the most important “hot spots” of criminal activity in the state. The government has also developed a new photo database, Fotocrim, to speed up the process of facial recognition of criminals (Reference Mello and SchneiderMello and Schneider 2010, 3).
More information improves the effectiveness of police strategies via two mechanisms. First, police forces can be quickly moved to where they are most needed. This reinforces the role of deterrence as it increases the likelihood of punishment for criminals. Second, the system also makes clear what regions are making progress in reducing crime. This allows police chiefs to monitor local personnel and take measures to improve performance if required.Footnote 4
Recent evidence shows that the intelligence system has effectively lowered the crime statistics in São Paulo. Using a spatial differences-in-differences estimator, Cabral (Reference Cabral2016) argues that Infocrim has had a large negative impact on homicide rates in the municipalities where it was implemented. The author also notes that the effect remains important even after accounting for possible displacement effects. As expected, some criminals did take their activities elsewhere after the creation of Infocrim, but this movement has not offset the benefits of the system.
How well have these policies performed over time? The results suggest a favorable outlook. Compared to other Brazilian states, São Paulo is an outlier when it comes to homicide rates. Despite the fact that crimes against property have declined little over the last decades,Footnote 5 the number of violent deaths per 100,000 inhabitants shows a steep downward trend. Figure 1 presents the evolution of homicide rates in São Paulo in comparison with the Brazilian average.
Figure 1 Homicide rates per 100,000 population: São Paulo and Brazil (excluding the state of São Paulo). Data from Instituto de Pesquisa Econômica Aplicada, http://www.ipeadata.gov.br/.
The trends are even more striking if we consider that deterrence policies are still controversial in the literature. Barbarino and Mastrobuoni (Reference Barbarino and Mastrobuoni2014), Buonanno and Raphael (Reference Buonanno and Raphael2013), Levitt (Reference Levitt1996, Reference Levitt2004) and Owens (Reference Owens2009) claim that incapacitation measures effectively reduce crime, but Eck and Maguire (Reference Eck, Maguire, Blumstein and Wallman2006) and Beattie and Mole (Reference Beattie and Mole2007) suggest that increases in police forces and incarceration rates in the United States and in Canada did not lead to expected outcomes.
There is good evidence that incapacitation measures have worked well in São Paulo during the last decade. Gun-related homicides declined about 74 percent from 2001 to 2008 (Reference Peres, Vicentin, Nery, Lima, Souza, Cerda, Cardia and AdornoPeres et al. 2011) while São Paulo experienced an increase of 770 percent in the arrests of repeated murderers (Reference MansoManso 2012, 36). Although there are studies that indicate possible “hardening effects” of imprisonment, that is, longer sentences may increase an individual’s tendency to commit further crimes (e.g., Reference Chen and ShapiroChen and Shapiro 2007; Reference Glaeser, Sacerdote and ScheinkmanGlaeser, Sacerdote, and Scheinkman 1996; Reference Western, Kling and WeimanWestern, Kling, and Weiman 2001), the São Paulo case appears to suggest otherwise. Moreover, violent deaths have decreased in all population strata but especially among males (–74.5%), fifteen-to twenty-four-year-old men (–78.0 percent) and those who live in extreme poverty (–79.3 percent), groups that are generally associated with criminality (Reference Peres, Vicentin, Nery, Lima, Souza, Cerda, Cardia and AdornoPeres et al. 2011).
Nevertheless, it is difficult to know which of the policies have contributed more to this large homicide reduction. Not only do we lack disaggregated data to test preliminary hypotheses, but there may be large interaction effects among different public security measures. Therefore, at the moment it is not possible to disentangle micro-level causes from macro effects. But the aggregated impact of the anticrime policies can be correctly identified if there is no other variable in the causal path leading from the policies mentioned above to our dependent variable (state homicide rates). I argue below that this type of estimation is feasible for the São Paulo case. To back this claim, I suggest that a competing explanation for the homicide drop in São Paulo—the rise of the Primeiro Comando da Capital (PCC)—interferes only with the direct effect of the policies on crime but not with their effect. In this sense, the synthetic control method provides a plausible identification strategy for my question of interest.
Alternative explanation: The emergence of the PCC
A recent hypothesis attributes the decrease in violent deaths in São Paulo to the Primeiro Comando da Capital (PCC) (Reference BiondiBiondi 2010; Reference DiasDias 2009a, Reference Dias2011; Reference FeltranFeltran 2010, Reference Feltran2012a; Reference WillisWillis 2015). The PCC is a prison gang that emerged in the early 1990s as a response to the demands of a growing prison population. The PCC provides personal security and financial assistance to its members and affiliates. The gang’s internal statute clearly declares that “those who are in liberty [must contribute] to the brothers inside prisons [PCC members] through lawyers, money, help to family members and prison outbreak operations” (Folha de São Paulo, 2001).
A group of scholars argues that the PCC significantly contributed to the reduction in violence mainly through the São Paulo prison system. At least since the mid-2000s, these authors argue that the PCC has been able to emerge as an undisputed mediator and solve conflicts between inmates. Dias (Reference Dias2009b, 83) writes that “when unable to constitute a universal source of regulation, the official law leaves gaps which are filled by informal instances—such as the Primeiro Comando da Capital (PCC), in the prisons of São Paulo.” The gang has implemented informal courts that resemble state institutions, and those meetings have progressively replaced other forms of popular justice such as lynchings or the hiring of target killers (Reference Feltran, Azaïs, Kessler and da Silva TellesFeltran 2012b, 3). Moreover, the Comando has developed a series of assertive ways to terrorize inmates. Since the PCC’s threats are credible, the group is able to impose discipline within the São Paulo prison system (Reference BiondiBiondi 2010; Reference DiasDias 2009a).
Paradoxically, the PCC might have also helped to reduce crime in São Paulo by collaborating with street-level police. The Brazilian state does not hold a perfect monopoly of force in many areas of the country (Reference FeltranFeltran 2012a; Reference HughesHughes 2004; Reference PinheiroPinheiro 2000), thus access to local knowledge may prove vital for the success of a given operation. In this regard, the PCC and the state may collude if the situation is beneficial to both, and in this way there is an informal—but potentially unstable—“killing consensus” in the state (Reference WillisWillis 2014, Reference Willis2015).
There has been a vigorous debate over whether the PCC has had a significant impact on violence rates. A few authors see the PCC as the sufficient condition behind the homicide rate decline across São Paulo State (Reference BiondiBiondi 2010; Reference DiasDias 2009a, Reference Dias2011), while others take a more nuanced view of the role of the prison gang (e.g., Reference WillisWillis 2015). But both groups of scholars affirm that, based on their firsthand experience, the PCC is the key explanatory variable behind the drop in murders in São Paulo.
Recent econometric works, however, do not seem to confirm that argument. Marcelo Nery has found no convincing results in favor of the PCC hypothesis using geo-referenced data for São Paulo (Reference GuimarãesGuimarães 2016). Biderman et al. (Reference Biderman, De Lima, De Mello and Schneider2014) use anonymous calls to a crime hotline as a proxy for PCC presence in São Paulo city favelas. The authors find some support for the idea that the gang reduces lethal violence in areas under its control, but PCC presence corresponds to only a minor drop in violent crime. Although the PCC impact is not negligible, the gang is not a sufficient condition for the homicide decline.
Another counterargument to the PCC thesis is that homicides also decreased in areas and groups over which the PCC does not exert control. First, descriptive statistics show that the decline in violent deaths started before the PCC’s expansion period.Footnote 6 Second, the drop in crime was evenly distributed throughout the state: urban and rural areas, small and large cities alike experienced fewer murders.Footnote 7 Finally, as noted above, Peres et al. (Reference Peres, Vicentin, Nery, Lima, Souza, Cerda, Cardia and Adorno2011) point out that violent death rates decreased in all age groups and social classes in the city São Paulo. Hence, cohorts that do not correspond to typical PCC members (such as the elderly or middle-aged females) are also less affected by violence. It seems that the influence of the PCC on physical violence has been overstated. It is unlikely that the PCC, which is underfunded for its size, could have achieved such deep penetration into society and lowered the violence levels across all population groups in the whole state.Footnote 8
Yet the group’s importance cannot be fully dismissed either. Data on PCC-controlled areas are likely to contain measurement errors that may bias the coefficients, thus caution is required before making strong causal claims on this discussion. Despite mounting observational evidence that the PCC may not provide a complete explanation to São Paulo’s lower crime rates, the argument could only be comprehensively tested in a counterfactual case in which the PCC is present and the state policies are not.Footnote 9 Currently available data do not allow us to evaluate such scenario.
Causal paths, moderators, and total effects
A methodological issue remains. If we are to estimate the causal effect of the public measures on the crime rates, how should we proceed? I have noted above that the specific impact of micro-level policies cannot be evaluated due to lack of data. Nonetheless, it is theoretically possible to estimate the total effect of policies on crime.
The difference between direct and total effects can be understood as follows. The direct effect captures the sensitivity of a dependent variable Y to changes in X when this relationship is not mediated by any other variables in the model. Holding all factors constant, the direct effect is a causal chain of length one (Reference SobelSobel 1987, 160) and could be described simply as X → Y. In turn, the total effect can be defined as P (Yx = μ), that is, “the probability that response variable Y would take on the value μ when X is set to x by external intervention” (Reference Pearl, Breese and KollerPearl 2001, 1572). The total effect is the sum of direct and indirect (or mediated) effects.
In our case, gun control, incarceration, and police intelligence have likely had a direct effect on homicides. Combined, these variables constitute a direct aggregate policy effect. The omission of a variable measuring the impact of the PCC could bias such an effect but not interfere with the total policy effect. This point is worthy of further consideration. The total policy effect would be unbiased under the assumption that the PCC is in fact a moderator between the public policies and the homicide rates, even if the gang’s impact over the violence levels is not particularly large.
Although this argument has rarely been posited in such terms, this position is largely supported by the qualitative literature on the PCC. Fieldwork research generally traces the group’s origins and growth to the rising incarceration rates in São Paulo and the need for protection among prisoners (Reference DiasDias 2011; Reference Manso and GodoyManso and Godoy 2014). Like other prison groups, the PCC would only mobilize resources to provide welfare and act as an arbitrator under the condition that the certainty of punishment by the state is high (Reference SkarbekSkarbek 2011; Reference FreireFreire 2014). Had the state not increased the costs associated with crime, the prison gang would not have expanded their reach or even have been created in the first place. Hence the impact of the PCC on street-level violent deaths—if it exists—can be safely assumed to be a moderator effect.
Whereas it would be interesting for researchers to separate these types of effects and isolate the PCC from the other causal outcomes, such estimation is not possible at the state level. However, as these measures were implemented throughout São Paulo State at roughly the same time, their combined effect is computable even though their individual direct effects are not. To do so, it is only necessary to contrast the treated unit (São Paulo) with a counterfactual without the time-assigned treatment (1999 onward) and evaluate the aggregated effect of the public policies.
This analysis can be estimated in a consistent manner with the synthetic control method. In the following sections I describe how the method creates a valid counterfactual case under a certain set of assumptions. The assumptions are: (1) the PCC is an outcome, not a cause, of the crime-targeting policies; (2) the model does not include unnecessary control variables; (3) interpolation bias is not very severe because the cases in the “donor pool” are relatively similar to the treated unit.
Methods
The synthetic control approach provides an adequate solution for two enduring problems in the social sciences: the arbitrary selection of comparative cases and the poor estimation of causal effects when few pretreatment observations are available (Reference Abadie and GardeazabalAbadie and Gardeazabal 2003; Reference Abadie, Diamond and HainmuellerAbadie, Diamond, and Hainmueller 2010). With respect to the first issue, scholars often resort to ambiguous criteria in their choice of control units. This practice ends up casting doubts over the validity of their selected counterfactual (Reference Abadie, Diamond and HainmuellerAbadie, Diamond, and Hainmueller 2011). The synthetic method provides a reliable comparative case by adopting a purely data-driven process in order to select a counterfactual. Also, the researcher can still specify what control cases enter the donor pool. In this sense, qualitative expert knowledge can be incorporated in the estimation via the selection of cases.
Regarding the second issue, the accurate estimation of coefficients from a small number of cases, SCM employs a consistent statistical solution to problems of incorrect data extrapolation and model dependence. SCM can be understood as a combination of matching with differences-in-differences. SCM uses matching as a flexible preprocessing tool to reduce imbalance between treated and control units (Reference Ho, Imai, King and StuartHo et al. 2007; Reference RubinRubin 1973, Reference Rubin2006). But unlike matching, SCM deals with only one treated unit over time. Therefore, the method can also be interpreted as a semi-parametric extension to differences-in-differences estimators in which both treated and control units are not required to follow parallel trends in the whole period (Reference AbadieAbadie 2005). By combining semi-parametric matching with differences-in-differences, SCM provides a rigorous yet versatile method to evaluate time-dependent treatment effects.
SCM starts with the assumption that one case in the sample has received a treatment.Footnote 10 The treatment is defined as a time-delimited event that affects the unit of interest, such as the implementation of a new policy or the outbreak of a conflict. SCM also requires a series of control cases to estimate the models, that is, units that did not receive the treatment during the same period. These cases are often related to the treatment case in some meaningful way, and natural choices for the donor pool are provinces within the same country or states that share important characteristics. These traits can also be more specifically defined and included as quantitative variables in the estimation models.
SCM then selects a few cases from the donor pool to create a new, artificial control for the treated unit of interest. The main goal of SCM is to construct a counterfactual that resembles the treatment unit more closely than any individual control in the donor pool. Cases are combined in a way similar to a weighted average, in which controls that are more similar to the treated unit receive more weight. The weights make explicit the contribution of each separate case to the synthetic control, which also increases the transparency and reliability of the method (Reference Abadie, Diamond and HainmuellerAbadie, Diamond, and Hainmueller 2014). The closer the synthetic control matches the original treated unit before the assignment of the period, the better the quality of the counterfactual.
The method uses an algorithm to minimize the difference between the control cases and the treated unit before the intervention. The authors adopt the mean squared prediction error (MSPE) as a measure of fit (Reference Abadie and GardeazabalAbadie and Gardeazabal 2003). MSPE is simply the difference between the fitted and the observed trends of the treatment case. A small value means that the two lines are highly correlated and the artificial control is a good approximation of the missing counterfactual in the post-intervention period. In our case, the counterfactual would be São Paulo from 1999 to 2010 without the crime-reducing policies.
SCM has an intuitive interpretation. Although numeric summaries and other statistics can be obtained from the model, a simple time series graph is usually enough to assess the results. The causal effect is the difference between the treated and the synthetic cohort. The larger the post-treatment gap, the stronger the treatment impact.
As with all types of observational studies, SCM can also suffer from omitted variable bias. One can never be sure whether all required confounders have been included in a given model. However, the graphical output of the SCM helps diagnose the presence of large disparities between treatment and control cases. If the trends follow similar paths during the control period, it provides some indication, albeit only informally, that omitted variable biases are not driving the output. This bias can also be mitigated with expert knowledge. Econometric studies show that the inclusion of a large number of covariates and post-treatment variables to correct for omitted variables bias can actually worsen the problem (Reference AchenAchen 1992, Reference Achen2002; Reference ClarkeClarke 2005, Reference Clarke2009; Reference PearlPearl 2009). This is particularly true for matching methods. Authors have noted that “over-matching” can lead to severe statistical bias (Reference BaserBaser 2006; Reference Brookhart, Schneeweiss, Rothman, Glynn, Avorn and StürmerBrookhart et al. 2006; Reference Marsh, Hutton and BinksMarsh, Hutton, and Binks 2002). In this regard, the most plausible solution seems to be attention to the trends and sensible selection of control variables. As I discuss below, the covariates included in this article are some of the most robust quantitative predictors of homicides.
Furthermore, placebo tests can be run to test the robustness of the findings. For instance, researchers can include “in-time placebos,” dates under which the treatment did not occur. Results should change only in the period when the treatment starts and not at any other point in time. Moreover, scholars can also add “in-space placebos” to their models. This test consists of adding different members of the donor pools into the models to see if the estimation varies (Reference Abadie, Diamond and HainmuellerAbadie, Diamond, and Hainmueller 2014). Finally, one can also compare the effects of the treatment of interest by creating a distribution of synthetic cohorts, where every unit (treated or not) is matched with a specific synthetic control case. The parameter of interest should still be relevant. The results of all these tests can be seen in the following sections.
Data
I build panel data for the variables Homicide Rate, State GDP per Capita, State GDP Growth, Years of Schooling, Gini Index, Natural Logarithm of Population, and Population Living in Extreme Poverty. These variables are very common in the specialized literature and represent important social and economic factors I wish to control for.Footnote 11
The unit of analysis is State-Year. I have data from all of the twenty-six states plus the capital city (Distrito Federal), ranging from 1990 to 2009. The data for years prior to 1990 are scarce and for years after 2009 have not yet been published. All data used in this article come from the same source, the Instituto de Pesquisa Econômica e Aplicada (IPEA), a government-led research group.Footnote 12
My dependent variable measures the number of homicides per 100,000 inhabitants, which is the most commonly used unit of analysis for lethal violence. This variable was coded by the Brazilian Health Ministry from obituary records, therefore it is less likely than police files to suffer from intentional misrepresentation.
There are six control variables in the models. State GDP per Capita is adjusted in 2010 Brazilian Reals (at the time 1 Brazilian real bought roughly 0.5 US dollars). State GDP Growth is measured in constant 2010 Brazilian reals and varies by percentage points. Years of Schooling describes the average number of years of formal instruction at educational facilities (males and females, twenty-five years old or older). Gini Index is a measure of inequality ranging from 0 to 1, where 0 is the most equal and 1 the most unequal. Natural Logarithm of Population represents yearly projections of the state population. Since Brazil only runs a census every ten years, these projections represent the most accurate data available. I have taken the natural logarithm of this variable to account for size effects. Finally, Population Living in Extreme Poverty describes the percentage of the state population which do not meet the minimum intake of 2,000 calories per day. This is the only variable that I created specifically for this study. It was coded by simply taking the number of individuals classified as extremely poor by the IPEA and dividing this number by the state’s total population.Footnote 13
Analysis
Main model
I construct the synthetic cohort (synthetic São Paulo) by imputing information from all of the Brazilian states plus the Federal District. The synthetic control method outputs a set of weights for states and variables such that the treatment state is approximated optimally by these weighted components. This method not only provides a quantitative way of selecting comparison cases but also gives us a much better baseline to compare with the treatment unit. Synthetic São Paulo is constructed using the six out of the twenty-seven possible cases that received non-zero weights. Table 1 shows that the states that best synthesize São Paulo are Santa Catarina (0.274), Distrito Federal (Brasília) (0.210), Espírito Santo (0.209), Rio de Janeiro (0.169), Roraima (0.137), and Pernambuco, which only accounts for 0.01 of the weights. In this regard the state selection does not appear as a complete surprise. Apart from Roraima, the other members of the federation are richer, more densely populated and better schooled than the country average, thus being indeed similar to São Paulo.
Table 1 Synthetic weights for São Paulo.
Among the independent variables, only three out of six receive substantial weights. Given the data I could obtain, the predictors that receive more weight are Years of Schooling (0.469), State GDP per Capita (0.275), and past Homicide Rate (0.241). The three remaining variables are much less relevant to the model. They are the Population Living in Extreme Poverty (0.009), Gini Index (0.005), and Natural Logarithm of the Population (0.001). Table 2 compares characteristics of São Paulo and its synthetic control prior to policy implementation. We see that synthetic São Paulo has very similar coefficients to those of the treatment unit. Moreover, the synthetic control clearly outperforms the sample means in all of the three relevant predictors. The worst measure is State GDP Growth, whose mean is about 2.6 whereas the figure for São Paulo is roughly 1.3 during that period. However, this outcome does not affect the results since the variables that received zero weight were discarded from the models.
Table 2 Homicide rate predictor means before policy implementation.
The results show that the synthetic control method has successfully created a valid counterfactual to our case of interest. Figure 2 depicts the evolution of the dependent variable for the treatment and synthetic control cases. We can see that São Paulo and synthetic São Paulo have very close homicide rates series for the period ranging from 1990 until 1998. From 1999 onward we observe the trajectories departing sharply from each other. The increase in homicide rates shown in the graph is consistent with previous statistical evidence. It indeed confirms that São Paulo had higher than expected levels of lethal violence, which I noted in the first part of this text.
Figure 2 Trends in homicide rates: São Paulo versus synthetic São Paulo.
Despite the high levels of violence in 1999, when the new crime-reducing program was implemented, the number of homicides consistently declined until 2009. The trend is indeed monotonic and there is not a single peak in homicide rates after the policies have been put into practice. I interpret that as strong evidence in favor of the public policies.
With respect to the size of the effect, in 1998 the homicide rate in São Paulo was around 40 deaths per 100,000 inhabitants. In 2009, the last year for which data are available, the rate dropped to 15, whereas synthetic São Paulo observed above 30 deaths per 100,000. That means a gap of –20 deaths for every 100,000 people in São Paulo in 2009, as can be seen in Figure 3. I estimate that the new policies implemented in São Paulo saved roughly 20,300 lives in the period from 1999 to 2009.Footnote 14 It is important to mention that the homicide rate in São Paulo continues to drop by the year, while the same is not happening in the rest of the country.
Figure 3 Homicide rates gap between São Paulo and synthetic São Paulo.
Robustness checks
To further analyze the findings, I run five robustness tests. I first create an “in-time placebo” synthetic control. This test consists of creating a false starting date for the intervention period to check if one could observe false treatment effects in the pretreatment years (Reference Abadie, Diamond and HainmuellerAbadie, Diamond, and Hainmueller 2014). If that were to be the case, the validity of the main results could be put into question. The result of this placebo test can be seen in Figure 4. When I run the model with 1994 as the year when there was a supposed policy change, the result shows that there is only a minor gap between both lines. In other words, the method does not indicate a definite departure of trends.
Figure 4 Placebo policy implementation in 1994: São Paulo versus synthetic São Paulo.
I also conducted a leave-one-out robustness test. In this test I drop the states composing the synthetic control one at a time. The main goal of this analysis is to evaluate whether a single control state is driving the results. This would suggest that the original synthetic control, which is composed of five states at a time, is probably not a reasonable counterfactual. The results of this analysis can be found in Figure 5. We see that the synthetic control (dashed line) is a reasonable amalgam of cases. Also, because the relative positions of treatment and controls are stable across controls, we observe that no control state is biasing the estimates.
Figure 5 Leave-one-out distribution of the synthetic control for São Paulo.
Figure 6 shows the difference in homicide rates between the treated units and their synthetic controls. Here I estimate a synthetic control case for São Paulo and for each of the other twenty-six Brazilian states. This test assesses whether there is any previously unobserved national or regional trend that explains the original results. We observe that in São Paulo the homicide rate gap increases consistently during the treatment period, whereas the lines for the other states are moving randomly. Several lines fail to show any substantial difference between the state line and that of its synthetic counterfactual case. This indicates the results for São Paulo are unlikely to be a result of broader trends.
Figure 6 Permutation test: Homicide rate gaps in São Paulo and twenty-six control states.
Figure 7 presents the same test displayed in Figure 6, but it uses a stricter threshold for the simulated synthetic controls. The graph features cases in which the mean squared prediction error, a measure of goodness-of-fit, is no higher than twice that of São Paulo. That is, only placebos that have good synthetic matches were selected for the analysis (Reference Abadie, Diamond and HainmuellerAbadie, Diamond, and Hainmueller 2010, 503). In this group, the negative gap for the homicide rate São Paulo is by far the most relevant, providing further evidence for the original findings.
Figure 7 Permutation test: Homicide rate gaps in São Paulo and selected control states.
Last, I estimate another synthetic control using another approach. In Figure 8, I employ a Bayesian structural time-series model to verify the stability of the previous results (Reference Brodersen, Gallusser, Koehler, Remy and ScottBrodersen et al. 2015). This inference procedure is similar to that described in the methods section of this article and it also consists of matching pretreatment values of the unit of interest, São Paulo, to other potential control states. However, in this model only the time trends of the dependent variable are matched. In a sense, this is closer to a traditional differences-in-differences approach, but without the restrictive assumption that the treated and the control cases would follow parallel trends over time (Reference Abadie, Diamond and HainmuellerAbadie, Diamond, and Hainmueller 2010, 494).
Figure 8 Bayesian structural time series model: São Paulo and synthetic São Paulo.
The model shows that in 2009 we should have expected São Paulo to have a homicide rate equal to 32.3 deaths per 100,000, but we observe only 15.2. Thus, the actual rate in São Paulo corresponds to only 47 percent of the expected counterfactual. The method also generates an estimate for the probability of causal effect. The calculations indicate a 96.3 percent chance of a causal impact in the period. In this sense, it is unlikely that the results are a statistical fluke.
Conclusion
As I have demonstrated, when compared to a synthetic control case, homicide rates were drastically reduced in São Paulo. Although it is not possible to estimate the treatment effect of each specific policy implemented between the 1990s and 2010 I suggest that their aggregate impact is surely not negligible. If the estimation strategy employed in this article is correct, the state of São Paulo offers an example that it is feasible to fight crime with targeted policies. This as an encouraging result, as it suggests that governments can make progress in reducing crime with the resources they already have at hand and need not rely exclusively upon structural conditions that are largely beyond their control, such as unemployment, per capita income and inequality. Robustness tests provide further evidence for my findings.
I also argue in favor of the synthetic control method as a tool to evaluate government policies. This approach offers an intuitive way to assess causality claims when there is only a single treated unit and it can be easily applied in a great number of situations. Assuming that there is a reasonable number of potential cases in the donor pool, a synthetic control can be meaningfully compared to the actual case. In this way, the technique allows the researcher to use the potential outcomes framework even in unusual conditions.
Future research can extend the present findings in a number of ways. First, it would be interesting to test whether other criminal activities have been affected by the state government policies I mentioned previously. Since property crimes are pervasive in São Paulo, scholars could evaluate the causal link (or lack thereof) between public policies and the incidence of theft or robberies. Unfortunately, several states in Brazil do not publish time-series data for property crime, so I could not use the synthetic control method for that dependent variable. As more data become available, this will create an interesting opportunity for investigation. Secondly, micro-level studies are needed to clarify the mechanisms behind São Paulo’s homicide reduction, and isolate direct from indirect effects of each individual policies. Due to the shortage of data on targeted policies, qualitative research may explain what the motivations, successes and shortcomings of São Paulo’s recent security measures were. Finally, there are still unresolved questions with regard to the PCC hypothesis, and this is a promising avenue for future academic work. New research could provide insights into how public policies work and, hopefully, help public authorities to design more effective policies against crime.
Additional File
The additional file for this article can be found as follows:
† Supplementary Appendix. Replication R Code (dataset available at https://github.com/danilofreire/homicides-sp-synth). DOI: https://doi.org/10.25222/larr.334.s1
Appendix
This appendix presents a formal presentation of the synthetic control estimator. Let j = 1, …, J + 1 be a series of units in periods t = 1, …, T. In our case, the units are the twenty-seven Brazilian states, including the federal district, and the time period spans from 1990 to 2009. Assuming that the first unit, São Paulo, has been exposed to the treatment, we have J control units to be included in the case studies donor pool, i.e., the twenty-six remaining states. We define treatment as the series of post-1999 government anticrime policies implemented in the São Paulo.
Let
be the homicide rate that would be observed for unit i, São Paulo, at time t with no treatment (1990–1998). Conversely, let
be the observable outcome for unit i at time t had it been subjected to the treatment in periods T 0 + 1 to T (1999–2009). An important assumption is that the treatment has no effect on unit i before the date of intervention, therefore, the values for São Paulo with and without the policy interventions are the same for the pretreatment period (1990–1998). In formal terms,
. The observed outcome is defined by
, where αit is the effect of crime-reducing policies on homicide rates, and Dit is a binary variable that takes the value of 1 if we refer to post-intervention period (after 1999) and 0 otherwise. The goal of this article is to estimate αit, the effect of the “treatment” (homicide reduction policies), for the state of São Paulo for all t ≥ T 0, that is, from 1999 to 2009. However, we cannot observe São Paulo without those policies, as there is no way for the state to have and not have the intervention at the same time. This is what Holland (Reference Holland1986) calls the “fundamental problem of causal inference”: only one of the outcomes of interest is measurable at any given time.
But although we cannot accurately know how São Paulo would be without the treatment, we can approximate it by using a weighted average of the remaining Brazilian states such that
. In this model, δt is an unobserved time-dependent factor common to all cases, Zi is a (1 × r) vector of observed covariates not affected by the policy, θt is a (r × 1) vector of unknown time-specific parameters, λt is a (1 × F) vector of unknown common factors to all states, μt is is a state-specific unobservable variable and ∈it represents unobserved transitory shocks with mean 0 for all units. Basically, what SCM tries to do is to match Zi and the pretreatment Yit of the treated unit so that μt is matched as a result.
To state again clearly, synthetic São Paulo is the weighted average of the other twenty-six Brazilian states. Therefore, it is a (J × 1) vector of weights W = (w 2, …, w J + 1)′ with wj ≥ 0 for j = 2, …, J + 1 and w 2 + … + w J + 1 = 1. Each of the elements included in W represents a specific weighted average of control states, that is, a potential synthetic control for São Paulo. The idea is to select a case that resembles the treated unit as closely as possible. Let X 1 be a (k × 1) vector of pre-1999 predictor variables for São Paulo and let X 0 be a (k × J) matrix containing the predictor variables for the potential control states. Let
be M linear functions of pretreatment outcomes (M ≥ F). One can choose w* such that:
Consequently, as noted by Abadie, Diamond, and Hainmueller (Reference Abadie, Diamond and Hainmueller2010), if T 0 is sufficiently large when compared to the scale of ∈it, an approximately unbiased estimator for α1t the effect of public security policies in São Paulo, can be described by:
for all
, that is, after the intervention period (1999–2009). In practice, W* is chosen non-parametrically as to minimize
, subject to the weight constraints. We consider
where V is a (k × k) symmetric and semi-definite positive matrix with the relative importance of each assigned homicide rate predictor. From various possible ways to choose V, in this article I follow the recommendation of Abadie and Gardeazabal (Reference Abadie and Gardeazabal2003) and choose V* as the value of V that minimizes the root mean squared prediction error (RMSPE) for homicide rates in the entire pretreatment period (1990–1998).
Acknowledgements
I would like to thank André Amaro, Guilherme Arbache, Fábio Barros, Rodolpho Bernabel, Nick Cowen, Guilherme Kerr, Rodrigo Martins, Robert Myles McDonnell, David Skarbek, the members of the Political Economy Group at Caeni/USP, two anonymous reviewers and the editor Aníbal Pérez-Liñán for their valuable comments and suggestions. The usual disclaimer applies. All data, code, and information required to replicate this study are available at the following web address: https://github.com/danilofreire/homicides-sp-synth. The author gratefully acknowledges the support of the National Council for Scientific and Technological Development and the Faculty of Social Science and Public Policy at King’s College London.
Open access
