What Are Behavioural Insights?

Legislative actors advance policy goals, but public servants handle the specifics of policy design and implementation, using research and evidence (French, Reference French2018). Policy success depends on balancing competing considerations, such as financial inputs and policy goals, as well as citizen behaviour. As citizens access public services, they confront technical and cognitive factors that may produce suboptimal decisions (for themselves and/or the government). Citizens can also face cognitive overload (Ly and Soman, Reference Ly and Soman2013), which overwhelms their ability to effectively access public services (McFadden, Reference McFadden2006). For example, consider a municipality that institutes a small fine for violating a new bylaw. The city spends considerable effort designing the optimal fine amount. However, the online portal for paying the fine is clunky and unintuitive. A citizen who wants to make such a payment tries repeatedly, only to get frustrated and abandon the task. The citizen is no better off (having to engage in a more laborious trip to city hall to pay the fine) and the city is no better off (delaying payment and having to engage a more costly and inefficient mode of processing the payment).

One innovative solution to this problem is behavioural public policy (BPP). BPP applies a behavioural lens to the policy process, with special focus on design, development, implementation and evaluation (Loewenstein and Chater, Reference Loewenstein and Chater2017; Oliver, Reference Oliver2017).Footnote ² BPP's theoretical underpinnings are driven by the rationalist and behavioural sciences at the nexus of policy, economic and psychological research. To this, policy makers apply experimental and data-driven methodologies to make design recommendations, improve implementation and evaluate policy effectiveness. BPP focuses on perceived utility and incorporates behaviourally motivated techniques, including BI. These techniques often simplify services and improve outcomes for citizens and government (Chen et al., Reference Chen, Bendle and Soman2017; Ewert, Reference Ewert2020).Footnote ³ The term BI, as we apply it here, refers to the mechanism that underlies policy design and implementation.Footnote ⁴ BI may incorporate incentives (explicit inducements) and what Thaler and Sunstein (Reference Thaler and Sunstein2008: 6) call a “nudge” or “any aspect of the choice architecture that alters people's behavior in a predictable way without forbidding any options or significantly changing their economic incentives.”Footnote ⁵ Common behaviours include applying for public services/benefits, saving money, improving personal health, and adherence with laws or regulations (for example, paying taxes). BI can help encourage or discourage behaviours. BI can help promote pro-self and pro-social behaviours and encourage citizens to avoid “regret loss” from choices based on intuition and emotion rather than deliberation and reasoned analysis (Hagman et al., Reference Hagman, Andersson, Västfjäll and Tinghög2015). Above all, BI avoids rigid commitments that disengage citizens by removing their choices.

BI is unique because it integrates psychology, political and economic theory, and rigorous testing through randomized control trials. BI has its intellectual roots in behavioural research in Daniel Kahneman and Amos Tversky's work in the 1970s and borrows from their research on prospect theory (Reference Kahneman and Tversky1979) and mental shortcuts (Tversky and Kahneman, Reference Tversky and Kahneman1974). With the recognition that BI is not the solution to every policy problem (Richard Thaler (quoted in Nesterak Reference Nesterak2021) argues that “nudge is part of the solution to almost any problem but is not the solution to any problem”) and that policy makers must judiciously decide when to apply BI, the technique has recently become mainstream in public policy.Footnote ⁶ For example, McDavid and Henderson (Reference McDavid and Henderson2021) persuasively argue that the time has come for greater professionalization of BI, drawing insights from the field of evaluation to suggest a path forward. Mainstreaming BI is possible in part due to recent progress on ethics and ethical frameworks (for example, Delaney, Reference Delaney2018; Lepenies and Malecka, Reference Lepenies, Małecka, Poama and Lever2019; Lades and Delaney, Reference Lades and Delaney2022). These have allowed the field to more confidently advance practical discussions of how and when to apply BI (Ewert, Reference Ewert2020: 338).

As BI gains popularity, scholars are investigating its effectiveness, focusing on replication and publication bias. Some BI-style interventions have failed to replicate, even in controlled settings. Prominent examples include priming effects (Cesario, Reference Cesario2014) and inconsistencies in the ego depletion effect (for example, Carter et al., Reference Carter, Kofler, Forster and McCullough2015, Dang et al., Reference Dang, Barker, Baumert, Bentvelzen, Berkman and Buchholz2021). One challenge is the potential for heterogeneous effects across interventions, populations and field settings. Rather than generalize about its overall effectiveness, researchers are increasingly asking when and where BI can be effective (Bryan et al., Reference Bryan, Tipton and Yeager2021). A related concern involves systematic biases in the reporting of BI trials. Consistent with other fields (see, for example, Franco et al., Reference Franco, Malhotra and Simonovits2014; Esarey and Wu, Reference Esarey and Wu2016), meta-analysis shows evidence of publication bias in peer-reviewed behavioural science (Hummel and Maedche, Reference Hummel and Maedche2019). Using internal data from two large BI units in the United States, DellaVigna and Linos (Reference DellaVigna and Linos2022) show publication bias tends to exaggerate the effectiveness of BI interventions. In short, the evidence suggests that BI can be effective, that its effectiveness varies by context, and that academic publications involving BI may overstate the magnitude of these effects.

Situating BI in the Public Policy Literature and Process

The traditional policy cycle includes agenda setting, policy formulation, decision making, implementation and evaluation (Howlett et al., Reference Howlett, Ramesh and Perl2020). Classically, the cycle envisions senior policy makers and their staff at the centre of a top-down policy process (Wegrich and Jann, Reference Jann, Wegrich, Fischer, Miller and Sidney2006). However, the rise of behaviourally informed policy implementation brings a renewed emphasis on citizens as end-users of public services (Pal et al., Reference Pal, Auld and Mallett2021: 235–36). Brock similarly notes the pivot to the “deliverology” model of public service that is meant to transcend “the traditional obstacles to change in the public sector that [include] bureaucratic lethargy, lack of expertise, intransigence and risk aversion” (Reference Brock2021: 230). In one sense, this behavioural turn expands on previous studies of implementation that focus on citizen experiences, such as “street- level” bureaucracy (Brodkin, Reference Brodkin2012; Lipsky, Reference Lipsky2010), “bottom-up” implementation (Sabatier, Reference Sabatier1986; Thomann et al., Reference Thomann, van Engen and Tummers2018) and “backward mapping” (Elmore, Reference Elmore1979). But even here, the perspective is usually that of a policy maker or public manager, rather than the citizen. In short, there is a gap between prominent theories of public policy (which focus on decision making) and questions of citizen engagement (which focus on service-level interactions with government). Underlying this gap is the 40-year decline in institutional trust and confidence in government.Footnote ⁷

With the rapid expansion of the state in the postwar era, citizens are presented with a great number of areas and ways in which government can provide services. But this wealth of choice has come with a problem: undersubscription and unequal access to public services. Low uptake is particularly concerning because it disproportionately impacts the least resourced populations, such as seniors, immigrants, refugees, Indigenous peoples, and youth. Compounding this complexity is the digital divide and the transition to digital government (Longo, Reference Longo2017), particularly in the pandemic and post-pandemic environment. Research by the Citizens First initiative suggests a growing challenge for service delivery in Canada. Their 2020 report uses polling data to show that overall satisfaction with public services is at a two-decade low (Citizen First, 2021).

In this context, BI has proven a timely resource for policy makers. As Howlett notes, “conceptual work on procedural policy instruments [has tended to lag] behind administrative practice” (Howlett, Reference Howlett2013: 119). BI is already firmly present in Canadian public policy making. As a diagnostic tool, BI helps policy makers focus on policy goals, anticipate delivery gaps and increase service uptake. One of the most effective ways to increase uptake is to reduce administrative burden (Herd and Moynihan, Reference Herd and Moynihan2019; Moynihan et al., Reference Moynihan, Herd and Harvey2015). Burdens arise when an individual experiences program or service enrolment or engagement as onerous (Burden et al., Reference Burden, Canon, Mayer and Moynihan2012). Burdens result from costs imposed on the user, such as the hassle of figuring out eligibility requirements (learning costs), potential stigma in accessing public services (psychological costs) and the time/effort required to fill out documentation (compliance costs) (Herd and Moynihan, Reference Herd and Moynihan2019). In this way, BI can be considered a type of procedural policy instrument capable of reducing burdens and narrowing this divide. What appears on the surface as an operational challenge (improving policy uptake) is really an interconnected chain of policy design and service delivery choices that influence how citizens experience government.

Policy makers are increasingly optimistic about the potential for BI to improve public policy and service delivery. In a recent survey of 719 Canadian public servants, Doberstein and Charbonneau (Reference Doberstein and Charbonneau2020) find BI teams (and innovation labs more generally) are now seen as the single most important source of public sector innovation. However, it would be naive to suggest that more innovative service delivery will remedy all problems of citizen engagement. Failing to engage with citizens can be viewed as the product of choices by politicians—the result of purposefully underinvesting in the welfare state. It is therefore no accident when the most underserved are those with the fewest resources to access the system (Banting, Reference Banting, Banting, Hoberg and Simeon1997, Reference Banting2010). In this way, policy design and implementation are deeply political.

Taking into account the literature on administrative burden, policy makers must be careful not to use BI solely as a solution to “middle-class” problems—such as saving for retirement, buying energy-conserving appliances, and so on (Benartzi et al., Reference Benartzi, Beshears, Milkman, Sunstein, Thaler, Shankar, Tucker-Ray, Congdon and Galing2017). Policy makers who apply BI have demonstrated its potential to yield high returns at low cost—but, in many cases, these successes are concentrated among those who are already capable of effectively interacting with government. If BI works well for the middle class but does not reduce administrative burden among those with fewer resources, it will not realize its potential to advance democratic, pro-social outcomes. It is equally important that policy makers understand the potential for heterogeneous results, such as differences by race, gender and income. BI should not be applied without consideration for resource disparities, lest the public view it simply as “policymaking by other means” (Herd and Moynihan, Reference Herd and Moynihan2019).

For these reasons, an ethical lens is crucial when using BI, especially when selecting targeting behaviours and considering downstream impacts. For example, a Canadian government experiment aimed to increase women's enrolment in the military using Facebook ads with different appeals (Privy Council Office, 2017). But MacDonald and Paterson (Reference MacDonald and Paterson2021) argue that the evidence is mixed about whether increasing women's participation in the Armed Forces will increase equity. They cite research on the Royal Canadian Mounted Police (Bastarache, Reference Bastarache2020) to point out that when women enter traditionally male-dominant professions, it may accompany increased reports of sexual assault and harassment. We do not suggest it is inherently unethical to nudge women into the Armed Forces; however, we believe an ethical lens is required both in the scoping process and in post hoc analysis.

Data, Methods and Analysis

Our analysis is a four-pronged environmental scan. An environmental scan is a method to gather information and identify opportunities and risks for future decisions. The method involves various data sources, including both published and unpublished information (Porterfield et al., Reference Porterfield, Hinnant, Kane, Horne, McAleer and Roussel2012). Environmental scans are not exhaustive but rather give a general picture of a field (Wheatley and Hervieux, Reference Wheatley and Hervieux2020). Our environmental scan considers (1) the number and location of existing BI units in government, (2) BI projects from public-facing reports spanning 2015 (the first published results from a publicly reported BI project by government in Canada) to 2022 by policy area, (3) BI projects by methodology and (4) BI projects by rate of success. While we are relying on self-reported government data, and recognize the associated concerns around governments only presenting “successes,” we observe that governments are increasingly following an “open science” model of BI. For example, the government of British Columbia has preregistered several field trials, which limits the degree to which public reporting may represent (only) positive outcomes. Preregistration is an encouraging trend given the concerns over replication and publication bias described earlier in this research note.

Finally, we supplement our environmental scan with interview data conducted with BI personnel across five BI units in three jurisdictions. Semistructured interviews took place remotely between July 2020 and July 2021. The thematic foci of these conversations included project design, responses to project successes and failures, and methodological approaches, as well as partnerships across government units. While we do not present a full thematic analysis of the interview data, we do use the observations that were noted across multiple offices to complement our empirical findings. Since Canada's BI field is comparably small and there are only three stand-alone units at present (though several mini units within government departments), we report interview comments anonymously without geographic or jurisdictional identification.

BI projects

We next address our data collected from public reports published by the three active BI units in Canada (the federal IIU and the BI units in Ontario and British Columbia). Our dataset includes a total of 59 projects across 13 policy areas (see supplementary material for more information on coding). Our data is supplemented by email correspondence and interviews with individuals in five BI units (federally and provincially) to address discrepancies or data limitations in the reports and to gain insight into project efficacy and rollout. Where appropriate, we supplement the data with observations from our interview data. Given the small number of projects, the analysis is necessarily limited. However, we are able to provide top-line information in policy coverage, methodological design, and goal-meeting or project efficacy.

Figure 1 illustrates the number of projects by policy area (full dataset available in supplementary materials). A clear majority of projects (N = 30) were initiated in the areas of health (N = 17) and government operations (N = 13; for example, administrative fines, documentation renewals), though we do see a variety of projects across social and administrative policy areas. Health projects range from the Ontario BI unit's “Supporting Ontario's COVID-19 Response” to helping physicians improve prescribing behaviour to reduce the prescription of opioids. A total of 21 per cent of these projects were initiated through the federal IIU, while the remaining 79 per cent come from provincial offices (68 per cent from Ontario and 11 per cent from British Columbia). Interview participants R1 and R2 both noted the suitability of administrative operations to BI trials because they often entail fewer ethical concerns (compared with projects involving social policy) and because they lend well to experimentation using digital touchpoints (that is, sending notices, online platform-based delivery). During the early days of the COVID-19 pandemic, these factors facilitated the continuation of BI work.

Figure 1 Number of Projects by Policy Area.

We also evaluate the form of these projects, specifically the methodological approaches taken. The majority of publicly released BI projects are field studies (N = 32)—that is, interventions or initiatives that apply BI principles in real-world settings. Another 22 projects focused on user-centred design; four focused on using survey experiments to test BI assumptions. All of these projects are evidence-based responses to policy challenges, with RCTs as the dominant methodological design (49 per cent) and the remainder using survey research or qualitative designs such as focus group analysis and post hoc reviews by experts in the field.Footnote ⁸ Interviews with R2 and R3 yielded a preference for a multi- or mixed-method approach to validate findings with supplementary stakeholder interviews, or to engage priority communities with requests for feedback. Respondents demonstrated a responsiveness to broad critiques of BI as being too quantitatively focused, and they accentuated the importance of consultation with program beneficiaries to reduce the risk of losing the “human” aspect of their goal.

Project efficacy is measured through a very simple metric: whether the BI project yielded a significant result on its stated goal. Importantly, the scale of BI results is moderate. Some projects hope to increase uptake by a modest 3 to 5 per cent; however, given the scale of these interventions, this can translate into tens—or even hundreds—of thousands of citizens. In every interview, practitioners noted the challenges of articulating the significance of what appeared to be modest effects when demands on government resources are so large. However, three respondents also observed that government partners would quickly embrace these findings when they viewed them as sustainable ways to enhance participation at a low cost to the budget unit. Many projects that we tracked are still ongoing and therefore may have only reported preliminary data. However, of those that are registered as complete, 80 per cent were successful at attaining their goal. These outcomes ranged from reducing delinquency on tax collection payments by 35 per cent in British Columbia to increasing applications for the Ontario Electricity Support program for low-income Ontarians.

One might reasonably ask how confident we can be that the BI interventions were the cause of these impacts. We suggest confidence depends on research design. When the project involves an RCT, randomization to treatment and control ensures an unbiased estimate of the impact (for example, intent-to-treat effect, average treatment effect). When we restrict our analysis to only those completed BI projects involving an RCT (N = 25), we find a similar efficacy as the full sample: 80 per cent of BI RCTs in Canada were successful at attaining their goal.

Finally, we use Canada's Government Electronic Directory Services to estimate changes in the number of BI personnel over time. We collect cross-sectional directory data across seven timepoints, spanning 2016 to 2022 (see supplementary material for more information). For each timepoint, we identified employees who either worked in units containing the phrase “behaviour” or those who worked at the IIU. We then manually reviewed each case, excluding observations where the employee did not appear to work in BI. In Figure 2, we tally the number of personnel by year. We find that the number of BI staff has increased tenfold over six years, with an average annual increase of approximately 48 per cent. In 2016, six federal employees worked in BI. By 2022, there were 63.Footnote ⁹ Further, our data allow us to map the spread of BI across departments. For example, in 2016, BI staff worked in just two departments: the Canada Revenue Agency (N = 1) and Privy Council Office (N = 5). By 2022, we find BI staff in 10 departments, including the Public Health Agency of Canada (N = 16), Immigration, Refugees and Citizenship (N = 5), Treasury Board Secretariat (N = 3) and Environment and Climate Change (N = 2). Growth has been especially strong in the Public Health Agency. In 2019, prior to the COVID-19 pandemic, the agency reported just one BI employee. By 2022, the BI team had grown to 16.

Figure 2 Growth of BI staff in the Government of Canada, 2016–2022.

Note: Data collected from the Government Electronic Directory Services.

Going Forward

Behavioural science is changing public policy, and, with it, political science. This research note provides four empirical contributions about BI in Canada. First, we observe that governments are actively investing in BI as a policy-making tool.Footnote ¹⁰ Governments invest in BI to increase program success and more efficiently use financial resources. Second, governments are expanding the scope of BI, moving beyond central agencies to line departments. The number of BI practitioners across government has increased dramatically—by our estimates, a tenfold increase in just six years. Third, our data show that, in general, BI initiatives are working. A critic might observe that governments are only likely to advertise successful projects, leaving failed projects firmly within the private reports of civil servants. However, our data show that governments have acknowledged several instances where BI projects have come up short. In part, this is because the methodological approach to BI embraces failure as a part of design and evaluation. Finally, as demonstrated in our reporting and data analysis, Canadian governments are using BI across policy domains, with a clear emphasis on improving government operations and service delivery, health, and social welfare.

From a comparative perspective, our findings dovetail with recent research on the application and methodology of BI across contexts. Like others (for example, Hummel and Maedche, Reference Hummel and Maedche2019: 52), we find that BI projects in Canada are most common in the domain of health. Similarly, we find a growing acceptance for non-experimental BI work in Canada, which parallels recent research from Australia (for example, Ball and Head, Reference Ball and Head2021: 114–16). Nonetheless, we recognize several limitations of our Canada-focused environmental scan. First, limited data availability mean we cannot report on some projects. For example, we find few instances where Canadian governments discuss using BI to improve tax collection. Yet media reports (Beeby, Reference Beeby2018) and our own data on the number of BI personnel suggest the Canada Revenue Agency is actively using BI to improve tax collection. Additionally, interview data that we have collected indicates numerous projects in progress across multiple policy domains that are not yet ready for disclosure. We argue that even without the full universe of BI data, our study includes many policy initiatives that reflect a particular approach to policy making—one that should be recognized in political science. Additionally, no policy scholar has a complete list of government projects to evaluate, owing to the same data considerations we face here (for example, access to information, privacy, secrecy). A second limitation involves how to define a BI project. As we have noted above, distinguishing between what the government might formally call a BI project and instances where BI tools or approaches have been used informally means that we may have undercounted our observations of BI in government.Footnote ¹¹

In closing, we reflect on Michael Atkinson's 2013 presidential address to the Canadian Political Science Association. Atkinson (Reference Atkinson2013) urged the discipline to embrace its roots in social and behavioural science. He framed this as an empirical and normative calling: to improve governance and, by extension, the lives of Canadians. In many ways, the discipline has taken up the charge. So too have some parts of government. In this note, we reflect on BI as a potentially powerful tool that can help public policy practitioners and scholars live up to Atkinson's ideal and strengthen citizen–government engagement in Canada. This can include external evaluation of BI policy design and implementation on its own merits but could also extend to collaboration between BI practitioners and political scientists, who bring a rich skillset around research questions and methodological design, while simultaneously taking a critical lens to governments’ policy design choices.

This research note highlights the fertile ground that exists to explore a robust research agenda on BI or with BI. Two streams clearly emerge: (1) evaluating the BI projects conducted by government and (2) using BI either with or independent of government as a tool to generate insights around program delivery. We make no claim that BI is the only or best tool to resolve any policy problem. We merely suggest it is an underused tool for resolving policy challenges in novel and cost-effective ways. We hope political scientists will take up this call.