Humboldt thought … A hill whose height remained unknown was an insult to the intelligence and made him uneasy … A riddle, no matter how small, could not be left by the side of the road.
1.1 Introduction
Our innate drive to make sense of things is one of the most powerful forces shaping both individual human cognition and collective societal progress. Consider the huge impetus behind the accumulation and critique of knowledge, which touches on all subjects – whether they be scientific, historical, or cultural – and proceeds at a grand scale to fill every corner of life, from the lectures of academic halls to the chatter of coffee houses. Sometimes knowledge is sought with some immediate objective in mind, but this makes up, on the whole, a surprisingly small part of our intellectual life. The force driving us to identify the causes of the Bolshevik revolution, map the deep oceans or the surface of the moon, chart the history of jazz, and understand the origins of life is powerful enough to drive millions of hours of scholarly activity – often without obvious direct application and even without pay. Daily life, too, is filled with myriad activities that provoke our interest, from exploring new cities, music, or cuisine to tracing our family history, becoming intrigued by gossip at the next table, and following the news. Indeed, these pleasures are so great that vast sectors of human activity are devoted to creating objects whose primary purpose is to stimulate the delights of sense-making: novels, movies, works of art, puzzles, and many more.
Although we generally take our undirected urge to make sense of the world for granted, it may seem strange upon reflection, especially because it frequently does not confer obvious near-term benefits. One might expect that the brutal logic of natural selection would have favored creatures interested only in practical concerns that directly enhance survival and reproduction. One might imagine, too, that societies with a laser-like focus on knowledge with immediate utility, rather than those promoting apparently purposeless inquiry, would be the ones to get ahead. Yet the opposite seems to be true: just “figuring stuff out” often yields unpredictable, but enormous, practical benefits. Indeed, the aimlessness of human curiosity may, paradoxically, be the secret of our species’ success (if it can be called that). This chapter focuses on the rationale for, and nature of, the motivational processes underlying the drive for sense-making: the intrinsic human desire to make sense of the world. We explore why the drive for sense-making is so valuable and, crucially, how particular features of its implementation can at times lead us astray into systematically incorrect beliefs.
In Section 1.2 (“The Drive for Sense-Making”), we start by discussing why sense-making generates a drive, similar to those associated with the primary reinforcers of food, water, sleep, sex, shelter, and air. The essence of our argument is that the drive for sense-making helps us balance the immediate benefits of satisfying tangible wants against the delayed benefits of investing in knowledge about ourselves and the world around us. The task of optimally making such trade-offs, which are incumbent upon all agents capable of self-directed learning, presents a formidable challenge because explicitly forecasting the beneficial consequences of each and every potential cognitive investment is often more trouble than it is worth. For many of the decisions we are faced with every day, such calculations would require a great deal of computational effort and yield inaccurate results, if they are even possible at all.
The drive for sense-making circumvents this problem by directly incentivizing our ability to make sense of the world in the here and now. It operates under the general assumption that “knowledge is power” – that is, that an enriched understanding of the world will benefit us in the future even if we cannot foresee exactly how. In the absence of a drive for sense-making, a limited ability to prospectively evaluate, and hence appreciate, the benefits of cognitively enriching activities would lead us to persistently underinvest in them. In this way, the drive for sense-making fills a critical gap that arises in purely goal-oriented cognition.
An economic framing of this argument reveals that the motivational incentive generated by the sense-making drive is analogous to the monetary incentive generated by a subsidy on knowledge-producing activities. We glean insights from this analogy by discussing why societies do in fact subsidize what is called basic research: “systematic study directed toward greater knowledge or understanding of the fundamental aspects of phenomena and of observable facts without specific applications.”Footnote 1 Analogous to our argument that the drive for sense-making exists to enhance future pay-offs, the quoted definition continues by noting that basic research is “farsighted high payoff research that provides the basis for technological progress.”
While Section 1.2 examines reasons why humans have a drive for sense-making, Section 1.3 (“The Objectives Governing Sense-Making”) examines three different factors that guide the particular form sense-making takes: (1) the practical utility of accurate beliefs for attaining concrete goals, (2) the desire to make sense of the world in a way that feels good, and (3) the impact of computational limitations on the sense-making process, including our limited ability to explicitly predict what information will turn out to be useful.
Of note, only the first of these categories is accounted for by standard rational theories of human behavior. Standard economics treats cognition as strictly a means to material ends. Accordingly, it holds that both cognitive states (e.g., knowledge, understanding, beliefs) and functions (e.g., information acquisition and processing) are only valuable to the degree that they are “instrumental” in helping us achieve concrete goals, such as increasing consumption or reducing labor. According to this view, because a rational agent is better prepared to maximize utility when they have an accurate understanding of their environment (Reference BlackwellBlackwell, 1953), the goal of information acquisition and processing should be to arrive at beliefs that are as accurate (and hence useful) as possible.
Some cognitive scientists, for their part, have recently proposed that correctly predicting the environment is all that matters to agents – essentially inverting the classical economist’s long-standing position by entirely subordinating material objectives to cognitive ones (Reference FristonFriston, 2010). Such “predictive processing” accounts take a different conceptual and mathematical form than rational economic models, but they share the fundamental conclusion that our cognition is exclusively aimed at generating accurate predictions about the future.
These perspectives yield important insights, but they also leave out critical aspects of sense-making. First, theories that exclusively focus on instrumental value (e.g., standard economics) fail to explain why we so fervently pursue activities, such as solving puzzles or reading mystery novels, that seem to yield little instrumental value relative to other readily available uses of our time.Footnote 2 On the other hand, theories that exclusively focus on inferential value (e.g., predictive processing models) do not readily explain the purposeful, goal-oriented nature of much of our cognition: the obvious fact that we do care about eating, sleeping, attracting the attention of potential mates, and achieving innumerable other material objectives. Predictive processing theories also seem to make the implausible prediction that agents should seek a maximally predictable environment and stay there forever (known as the “dark room problem”; see Reference Friston, Thornton and ClarkFriston, Thornton, & Clark, 2012; Reference Sun and FirestoneSun & Firestone, 2020).
Both the standard instrumental and the predictive processing theories of sense-making also leave out the fact that motivation and beliefs frequently interact with one another. In recent years, however, economists have begun to recognize that certain cognitive states and processes seem to be valued in themselves and confer strong motivational significance for agents. Work on “belief-based utility” (Reference Loewenstein and MolnarLoewenstein & Molnar, 2018) has shown that the desire to make sense that feels good plays a significant role in determining how we seek, interpret, and act upon information. In a similar vein, psychologists outside of the predictive processing tradition have long recognized the importance of motivated reasoning in shaping our beliefs (Reference KundaKunda, 1990).
Motivational factors are also crucial for ensuring we make the best use of our limited cognitive resources when gathering and processing information. For example, the motivational signals of flow and curiosity direct us toward the most valuable new information we might gather through reading, observing, discussing, or experimenting (Reference Wojtowicz, Chater and LoewensteinWojtowicz, Chater, & Loewenstein, 2020; Reference Wojtowicz and LoewensteinWojtowicz & Loewenstein, 2020), and the sense of “cognitive dissonance” (Reference FestingerFestinger, 1957) alerts us to inconsistencies in our beliefs that require further analysis and scrutiny. As a result, understanding what interpretation an individual will arrive at requires, at least in part, accounting for the motivational factors that guide our uptake and processing of information.
The standard instrumental and predictive accounts also generally overlook the impact of computational constraints on the sense we can and do make of the world. In particular, these accounts leave out the fact that considering each of the myriad possible interpretations of a given body of information as prescribed by Bayes’ rule is often intractable (Reference JeffreyJeffrey, 2004), even for relatively simple problems (Reference KwisthoutKwisthout, 2011; Reference Van RooijVan Rooij, 2008). Evidence suggests that our cognitive system instead approximates this normative standard by sampling interpretations one at a time (e.g., we see the duck-rabbit as either a duck, or a rabbit, but not both at once; see Figure 1.1). As we will argue, this has huge ramifications for how sense-making operates (Reference ChaterChater, 2019; Reference PashlerPashler, 1999).
Figure 1.1 Three examples of perceptual multistability. Each image can be interpreted in two ways, but we only perceive one interpretation at a time. Evidence suggests that this phenomenon extends to higher cognitive processes such as explanatory inference. See text for details.
Perhaps the most important practical limitation of both the standard instrumental and predictive processing accounts of sense-making, however, is that they fail to explain the troubling predominance of nonnormative belief patterns in society or to provide adequate guidance as to how they might be addressed. Recent developments – such as the precipitous growth of online radicalization, conspiracy theory communities, religious extremism, political polarization, anti-science rhetoric, climate change skepticism, antivaccination sentiment, COVID-19 denial, and hate groups – have heightened concerns about the descriptive adequacy of rational frameworks. Such phenomena are especially puzzling for rational theories given that their growth has coincided with (and, arguably, been fueled by) the rise of the Internet, which enables free and instantaneous access to much of human knowledge. According to a purely rational conception of belief-formation, such a dramatic increase in access to high-quality information should have resulted in a commensurate increase in the accuracy of popular beliefs, contrary to recent events. Finally, Section 1.4 (“Implications”) shows how the alternative perspective we lay out in the preceding sections can be used to better understand these phenomena.
The core argument of this chapter is that analyzing the multiplicity of objectives governing sense-making can help to explain the scientific and practical puzzles that vex current theories. According to our account, instrumental, inferential, and computational factors work together to guide our decisions. The drive for sense-making is primarily directed at maximizing predictive accuracy, but the other above-noted factors – belief-based utility and cognitive efficiency – also shape the sense we make. The interaction of these (sometimes competing) factors gives rise to characteristic patterns of irrationality, which leave us vulnerable to seductive mistruths that are increasingly amplified, both passively (by technologies that spread misinformation with unprecedented speed) and actively (by social movements dedicated to propagating abnormal patterns of beliefs). A comprehensive picture of how sense-making fits into the broader psychology of motivation explains characteristic distortions in our relationship with truth and, in turn, sheds new light on these concerning trends.
1.2 The Drive for Sense-Making
In this section, we develop a functional account of the drive for sense-making that explains its characteristic features by analyzing the cognitive problem it solves. Our account starts with the general observation that many – if not all – motivational states exist to address the boundedness of our rationality (Reference HanochHanoch, 2002; Reference MacLeodMacLeod, 1996; Reference Muramatsu and HanochMuramatsu & Hanoch, 2005; Reference Samuelson and SwinkelsSamuelson & Swinkels, 2006; Reference Sorg, Singh and LewisSorg, Singh, & Lewis, 2010). Immediate drives, feelings, and urges help us make decisions quickly and cheaply by circumventing the need to prospectively calculate the costs and benefits of each potential option explicitly. More specifically, these visceral states circumvent the (often intractable) task of forecasting the consequences of our actions arbitrarily far into an uncertain future (Reference Bechara and DamasioBechara & Damasio, 2005; Reference DamasioDamasio, 2006) by encoding the expected survival value associated with evolutionarily significant behaviors, such as consuming key nutrients, copulating, nurturing offspring, and avoiding bodily harm (Reference CabanacCabanac, 1971; Reference Cosmides, Tooby, Lewis and Haviland-JonesCosmides & Tooby, 2000).
A subset of these states specifically function to shape our information seeking and processing behavior: boredom, flow (Reference Wojtowicz, Chater and LoewensteinWojtowicz et al., 2020), curiosity (Reference Wojtowicz and LoewensteinWojtowicz & Loewenstein, 2020), mental effort (Reference Kurzban, Duckworth, Kable and MyersKurzban et al., 2013; Reference Shenhav, Musslick, Lieder, Kool, Griffiths, Cohen and BotvinickShenhav et al., 2017), and, as we will argue, the drive for sense-making. Although these states are psychologically distinct, they share many theoretical connections and overlap operationally due to the interrelated nature of their underlying functions. Indeed, we have argued elsewhere that curiosity may in fact be a special case of the drive for sense-making (Reference Chater and LoewensteinChater & Loewenstein, 2016; Reference Wojtowicz and LoewensteinWojtowicz & Loewenstein, 2020), and that flow and boredom partly reflect deviations from the amount of cognitive enrichment one has come to expect from similar environments (Reference Wojtowicz, Chater and LoewensteinWojtowicz et al., 2020).
As we have suggested, explicitly appraising the value of an increase in information, knowledge, or understanding is computationally intractable and would exhaust our finite cognitive resources in most situations. In most cases, our models of the external world are so underspecified that they do not provide meaningful answers to the question of how useful a particular piece of information is likely to be. But even if such models were available and could in principle yield well-defined answers, the computational costs of generating accurate predictions would still be prohibitive in most circumstances. This is because explicitly assessing the value of a piece of information or knowledge requires that we consider the many instances where a piece of information or knowledge would be applied. In general, the number of potential futures grows exponentially with the time horizon one considers; because cognitive resources can be applied arbitrarily far in the future, this explosion can be difficult to contend with (Reference BellmanBellman, 1957; Reference SavageSavage, 1972; Reference Sutton and BartoSutton & Barto, 2018). Planning the optimal sequence of information-acquisition behaviors also requires that one anticipate how information gained at each stage will impact the interpretation and usefulness of information gained at all later stages (Reference Meder, Nelson, Jones and RuggeriMeder et al., 2019).
Our hypothesis is that the brain circumvents these computational challenges by directly incentivizing actions that result in increased understanding using a motivational state that we experience as the drive for sense-making (Reference Chater and LoewensteinChater & Loewenstein, 2016). This approach avoids the need to prospectively calculate the potential usefulness of knowledge explicitly because “sense” is quantified using a contemporaneous measure of our ability to explain empirical regularities in the world. This is principally a backward-looking appraisal that operates on fixed data and, critically, does not require us to simulate the exponential number of diverging possible futures where that sense might be applied.
The exact nature of how the brain quantifies sense is still an area of active research, but one hypothesis is that sense measures our ability to compress the information we encounter into explanations. Data can be compressed to the extent that patterns can be found in that data, so the degree of compression achieved provides a natural measure of how well patterns in that data have been uncovered, irrespective of whether those patterns will turn out to help achieve any practical goal. Viewed in this way, the amount of sense we make out of a particular piece of information corresponds to the reduction in representational code length that we can achieve when we discover successively better (i.e., compressive) explanations for it. Sense-making occurs when we strike upon insights or critical pieces of new information that help us to resolve ambiguities or recognize regularities in an existing set of facts, thereby enabling us to compress them further.
As an example, consider the text “GNIKAMESNES.” While this might at first appear to be meaningless, it acquires more sense – especially in the context of this chapter – once we recognize it as “SENSEMAKING” spelled backwards. Under the compression hypothesis, this insight makes sense of the original text precisely because it reduces an unfamiliar and unwieldy jumble of letters to two simple cognitive operations: recalling a familiar word (“sense-making”) plus applying a familiar transformation (left–right transposition), enabling us to cognitively represent, manipulate, encode, and recall the string more efficiently. If, for illustration, we imagine all “units” are equal (whether letters, words, or transpositions), then we can see that spotting this new representation of “GNIKAMESNES” counts as definite progress. For a hypothetical cognitive system that encoded text using such a system – that is, either by storing it letter by letter or by applying a transformation to previously stored text – detecting this pattern would reduce the representational length of “GNIKAMESNES” from eleven to just two units, thus yielding nine “units of sense.”
While more research is needed to determine what form the representations underlying a fully domain-general measure of sense-making might take, a variety of candidates have been proposed that range from the most comprehensive model of computation – programs compiled by a Turing complete language (Reference ChaterChater, 1996; Reference Chater and LoewensteinChater & Loewenstein, 2016; Reference Chater and VitányiChater & Vitányi, 2003) – to less powerful automata capable of expressing more restricted grammars (i.e., ones at a lower level of the Chomsky hierarchy; see Reference Griffiths and TenenbaumGriffiths & Tenenbaum, 2003; Reference SimonSimon, 1972). For now, the question of how these mathematically abstracted computational-level measures might be implemented in the brain is a largely unexplored – but exciting – topic for future research.
According to this perspective, the drive for sense-making is an innate source of motivation that rewards us for each marginal increase (and, perhaps, punishes us for each marginal decrease) in our ability to compress information into efficient representations (Reference Chater and LoewensteinChater & Loewenstein, 2016). While the goal of compressing the information we encounter is certainly valuable for its own sake (e.g., because it enables us to store information more efficiently in the brain), its primary benefit is that it directs our cognitive machinery to actively search for regularities in the phenomena we observe, thus enabling us to better describe, predict, and control the world.Footnote 3
Given that sense-making and the classical drives serve similar psychological functions, they also share many basic characteristics. For example, classical drives consist of both a “carrot” and a “stick”: pleasure when we fulfill the drive’s target behavior and pain when we abstain from it. For example, eating when hungry feels good, but failing to do so for long periods of time becomes highly aversive, especially while in the presence of food. Paralleling these mechanisms, a few studies have shown that curiosity activates the same areas of the brain that process extrinsic rewards (Reference Jepma, Verdonschot, Van Steenbergen, Rombouts and NieuwenhuisJepma et al., 2012; Reference Kang, Hsu, Krajbich, Loewenstein, McClure, Wang and CamererKang et al., 2009), suggesting that sense-making considerations may enter into standard reward calculation as an intrinsic reward (or punishment) signal (Reference Gottlieb, Oudeyer, Lopes and BaranesGottlieb et al., 2013; Reference Kidd and HaydenKidd & Hayden, 2015).
In the case of sense-making, the carrot corresponds to the pleasure we experience when we succeed at uncovering regularities that generate new sense. In moments of profound insight, the sudden rush of sense-making pleasure can be quite intense (Reference GopnikGopnik, 1998), as exemplified by Archimedes’ famous exclamation of “Eureka!” upon discovering the principle of buoyancy. Less acute instances of sense-making pleasure also permeate many aspects of our daily life and range from the delight of discovering the answer to a riddle to the satisfaction of arriving at a mystery novel’s grand reveal. The stick, on the other hand, consists of the unpleasant sense of deprivation we feel when we are faced with a salient lack of understanding, as exemplified by the torment of leaving a riddle unanswered or a mystery novel unfinished. This deprivation is stronger the more apparent the gap in our understanding becomes, and the less easily it can be closed (Reference Golman and LoewensteinGolman & Loewenstein, 2018; Reference LoewensteinLoewenstein, 1994).
The drive for sense-making is related to, and may even entirely subsume, other motivational states that guide how we gather and process information. The most obvious example is curiosity, which shares the same drive-like features (Reference LoewensteinLoewenstein, 1994), solves the same cognitive problem (Reference Wojtowicz and LoewensteinWojtowicz & Loewenstein, 2020), and has overlapping behavioral implications (Reference Chater and LoewensteinChater & Loewenstein, 2016) as sense-making. Other examples include boredom, which redirects our attention away from understimulating activities when more promising opportunities seem to exist in our environment, and flow, which keeps our attention focused on the task at hand when other, better opportunities seem unlikely to exist. Both of these states emerge from a counterfactual comparison between the current and anticipated value of engagement, which is largely determined by the degree of sense-making achieved (Reference Chater and LoewensteinChater & Loewenstein, 2016; Reference Wojtowicz, Chater and LoewensteinWojtowicz et al., 2020). Sense-making is also closely related to our preferences for creating and resolving uncertainty (Reference Ruan, Hsee and LuRuan, Hsee, & Lu, 2018) and may underpin the states of suspense and surprise (Reference Ely, Frankel and KamenicaEly, Frankel, & Kamenica, 2015). Finally, the explanatory values we use to evaluate everything from scientific hypotheses to quick excuses – such as how simple, descriptive, or unifying an account is – are key implements of sense-making and arguably exist to further the same overall inferential objective (Reference Wojtowicz and DeDeoWojtowicz & DeDeo, 2020).
According to our account, the drive for sense-making makes up for our limited ability to appreciate the true long-term value of investing in knowledge. This parallels the way in which governments use subsidies to overcome the inherent tendency of private enterprise to underinvest in knowledge-generating activities. In a social setting, it is virtually free to include, and very difficult to exclude, others from using knowledge once it has been created. Knowledge is therefore an example of what economists refer to as “public goods,” which are chronically undersupplied relative to the socially efficient optimum because potential producers cannot capture the full value they create by investing in them.
Modern societies address this problem through government funding of public universities, scientific institutions, and basic research. Just as the drive for sense-making is necessary to motivate undirected inquiry, this funding is necessary to sustain learning for its own sake, without any immediate expectation of profit. As it turns out, however, such research often lays the groundwork for a variety of unforeseen applications that more than pay for the initial outlay through increased long-term economic growth. Also like sense-making, our inability to predict which types of knowledge will eventually be useful for particular problems means that continued broad investment in basic research often turns out to be the best way of ensuring we eventually solve them. Moreover, heavy-handed attempts to override research curiosity and narrowly optimize the direction of their work often end up backfiring because the process of justifying the value of scientific projects (including, sometimes, their practical value) through grant writing and related activities takes up time that could be used for actual research. In much the same way, forecasting the future value of sense-making uses up the very mental resources one needs to make sense of the world.
The function of the drive for sense-making is also illustrated by an analogy to education. Students perpetually complain that what they learn has no obvious value or relevance to their daily lives or future careers. While out-of-date education is certainly a problem, these critiques are often overstated, especially in young children who have no conception of what adult life is like and consequently cannot accurately gauge the importance of the knowledge and skills they are learning. Indeed, the distinction between education and training nicely captures the difference between the provision of knowledge which has no immediate application and that which is focused on learning an applicable skill. While training is, of course, extremely important, a school and university system focused purely on immediately applicable skills would fail to cultivate the growth of general knowledge that is crucial to long-term development. The main goal of education, therefore, is to provide a broad base of fundamental knowledge that helps students get a sense of the overall “geography” of knowledge in its broadest outlines. As students get older and their particular interests, proclivities, and goals become more clear, a greater degree of specialization is gradually introduced, but education is not, and cannot be, perfectly tailored.
If, as this analysis suggests, the purpose of formal education is to ensure that students acquire skills that are unexpectedly useful (and therefore would not seek out themselves), initiatives to shift the curriculum toward more apparently useful material may miss the point entirely. In much the same way, sense-making drives us to enrich our cognitive capacities in numerous directions, only some of which will turn out to be useful. Like a good teacher, the sense-making drive encourages us to engage in enriching activities, even in the absence of foreseeable benefits. Given that sense-making functions as “nature’s endogenous teacher,” it is not surprising that its derivative states, most notably curiosity, play a critical role in supporting learning, both in and out of the classroom (Reference Deci, Ryan and KatzDeci & Ryan, 1981; Reference LitmanLitman, 2005; Reference Markey, Loewenstein and Linnenbrink-GarciaMarkey & Loewenstein, 2014; Reference Pluck and JohnsonPluck & Johnson, 2011; Reference Wade and KiddWade & Kidd, 2019).
These points are corroborated by research in machine learning, which has shown that intrinsically generated sense-making rewards help to foster robust learning by encouraging structured exploration. Reference Schmidhuber, Meyer and WilsonSchmidhuber (1991, p. 222) points out that these incentives not only instill a desire for an artificial system to improve its understanding of the world, but also “to model its own ignorance, thus showing a rudimentary form of self-introspective behavior.” Reference Lopes, Lang, Toussaint, Oudeyer, Pereira, Burges, Bottou and WeinbergerLopes, Lang, Toussaint, and Oudeyer (2012) further show that such rewards can be generated using heuristic online estimates of learning progress, closely matching our conception of the drive for sense-making as rewarding gains in our ability to compress existing information. In a similar vein, Reference Pathak, Agrawal, Efros and DarrellPathak, Agrawal, Efros, and Darrell (2017) demonstrated the benefits of combining standard reinforcement learning with an “intrinsic curiosity module” that learns to predict which actions might expose the shortcomings in an agent’s model of the environment. They show that adding these predictions to the stream of extrinsic reward feedback an agent receives from the environment speeds up learning; in fact, their agents learn to successfully navigate video games when motivated by intrinsic curiosity alone (see also Reference Burda, Edwards, Pathak, Storkey, Darrell and EfrosBurda et al., 2018).
1.3 The Objectives Governing Sense-Making
In this section, we describe three objectives that shape sense-making, either directly, through motivational signals that orient sense-making, or indirectly, through constraints on the cognitive processes that underlie it.
1.3.1 Instrumental Objectives
The most obvious goal of sense-making is to help people make decisions that reliably lead to desired outcomes. From the perspective of decision theory, a rational agent acting in isolation is better equipped to pursue concrete ends when armed with more accurate beliefs (Reference BlackwellBlackwell, 1953), so the instrumental objective of sense-making often boils down to developing beliefs that are as accurate as possible. Indeed, in extreme cases, holding beliefs that are too divorced from reality (e.g., believing that one knows how to swim when one does not) can be fatal. Consequently, there seem to be strong constraints on the sense-making process: we cannot simply believe whatever we wish, and we labor to justify even our most fanciful beliefs to ourselves and others.
However, the fact that humans are both boundedly rational and highly social adds several important caveats to the truth-orienting function of sense-making, such that inaccurate beliefs may sometimes be advantageous when other psychological factors are taken into account. For example, the autonomic effects of nervousness evolved because they are usually adaptive, but they have the unintended consequence of degrading performance in some circumstances, such as test taking (Reference Zeidner, Weiner and CraigheadZeidner, 2010), public speaking (Reference BeattyBeatty 1988), high-stakes games (Reference Ariely, Gneezy, Loewenstein and MazarAriely, Gneezy, Loewenstein, & Mazar, 2009), athletic performance (Reference KleineKleine, 1990), and sexual function (Reference McCabeMcCabe, 2005). Conditional on being subject to these autonomic forces, overconfidence in our objective abilities might benefit us in situations in which performance anxiety would otherwise hold us back. Systematic cognitive errors can also be beneficial if and when they compensate for other types of errors. As one example, Reference Kahneman and LovalloKahneman and Lovallo (1993) argue that overconfidence can be beneficial to the degree that it compensates for the conservatism and extreme avoidance of risk that would, in its absence, arise from loss aversion.
The beliefs we hold also change the way others regard us. For example, people are more likely to trust the leadership and advice of those who are self-confident. Reference Anderson, Brion, Moore and KennedyAnderson, Brion, Moore, and Kennedy (2012) present a series of studies showing that overconfidence leads other people to view an individual as more competent, and generally enhances their social status. Some strategic interactions, such as the game of chicken, also favor those who can convince others of their irrational commitment to undertake risky actions (Reference ColmanColman, 2003; Reference Rapoport and ChammahRapoport & Chammah, 1966; Reference SchellingSchelling, 1980). While it is always possible, in principle, for a well-calibrated individual to fake confidence, such an act can, in practice, be difficult to sustain. In some domains, the most effective way to convince others of one’s exceptional abilities may be to first convince oneself.
Reference Mercier and SperberMercier and Sperber (2011) go even further, advancing the provocative hypothesis that the principle function of reasoning is to develop arguments that will be convincing to others. Needless to say, it is not always in one’s personal best interest to reason in good faith when the objective is to sway someone else. According to this account, many apparently irrational aspects of cognition are actually driven by the benefits associated with successfully influencing others.
1.3.2 Hedonic Objectives
Although beliefs primarily function to help us achieve desired outcomes, people also care about what happens purely in their own minds. In other words, beliefs are not merely a means to an end, but can also become an end in themselves. While this general phenomenon – known in economics as belief-based utility – is at odds with basic tenets of rational thought (and, as we will describe, can undercut the instrumental function of beliefs outlined in the preceding section), it nevertheless performs an indispensable cognitive function by motivating us to pursue complex goals that would be hard to define without the aid of sense-making.
Evolution has, as we have noted, endowed us with a variety of motivational mechanisms that encode the value of various goals and push us to pursue beneficial actions, most notably the visceral feeling states and hedonic signals associated with classic drives that incentivize us to maintain homeostasis and satisfy various biological imperatives. While the satisfaction of many basic physiological goals can be determined automatically and without conscious awareness (e.g., monitoring the blood for a satisfactory glucose level), measuring progress on other goals, especially social goals, depends on nuanced inferences that must be assessed using higher-level cognitive processing. For instance, we care about considerations such as our standing in the world – whether we are liked and respected by others – and, with especially obvious evolutionary significance, whether we are found attractive by potential mates. Our motivational system induces us to pursue these goals by making certain belief states directly valuable. The pleasure associated with, for example, believing that others view us favorably provides an incentive for us to behave in a fashion that makes it true. This may, in turn, lead to substantial long-term benefits, such as the cooperative support of others, although the specific nature of these benefits will, of course, be difficult to foresee precisely.
Belief-based utility exists to motivate behaviors that bring about desirable situations, but it is an imperfect mechanism for achieving this goal from a purely hedonic point of view. After all, simply believing what makes us feel good, irrespective of reality, would be a more direct route to unlocking the pleasures of the mind. Fortunately for the survival of our species, there seem to be significant limitations on our ability to believe whatever makes us feel good (Reference Loewenstein and MolnarLoewenstein & Molnar, 2018). So, for example, we cannot, by sheer force of will, perceive low teaching feedback scores as high praise – though we may be able to avoid looking at our teaching ratings entirely. However, despite such constraints on our ability to see what we want to see, the motives induced by belief-based utility can, in some instances, distort our relationship with truth and undermine our ability to achieve material goals. For example, an overestimate of our ability might feel good, but it can also lead us to expend time, energy, and money on endeavors where we are overwhelmingly likely to fail.
Exactly how motivational forces influence the direction that sense-making takes is an interesting and underexplored question. The influence of motivational processes on sense-making is undoubtedly aided by the fact that sense-making is, like most cognitive processes, sequential. That means that motivations can influence the direction that information processing takes. As Reference Epley and GilovichEpley and Gilovich (2016, p. 133) note, “People don’t simply believe what they want to believe … People generally reason their way to conclusions they favor, with their preferences influencing the way evidence is gathered, arguments are processed, and memories of past experience are recalled. Each of these processes can be affected in subtle ways by people’s motivations.” “For propositions we want to believe,” Reference GilovichGilovich (2008, pp. 83–84) writes in his classic How We Know What Isn’t So, “we ask only that the evidence not force us to believe otherwise… For propositions we want to resist, however, we ask whether the evidence compels such a distasteful conclusion… For desired conclusions, in other words, it is as if we ask ourselves, ‘Can I believe this?’, but for unpalatable conclusions we ask ‘Must I believe this?’.” Or, as Kunda (1990, pp. 482–483) expressed it, people “draw the desired conclusion only if they can muster up the evidence necessary to support it.”
As the above-quoted passages hint, the processes that people use to achieve sense-making that feels good bear a striking resemblance to the biased processes that scientists use to collect and analyze data in a fashion that supports the conclusions they want to arrive at (c.f. Reference John, Loewenstein and PrelecJohn, Loewenstein, & Prelec, 2012; Reference Simmons, Nelson and SimonsohnSimmons, Nelson, & Simonsohn, 2011) – a set of practices that have come to be known, collectively, as “p-hacking.” Much as scientists may collect just enough information to support their favored hypothesis and no more, people who want to behave selfishly without perceiving themselves as such will avoid collecting new information about the consequences of their actions when their current data supports the conclusion that their actions will not hurt others (Chen, et al., 2020).
One potential consequence of motivated processing is a phenomenon known as belief polarization, which occurs when exposure to the same new piece of evidence causes individuals who hold different beliefs to diverge even further (Reference BatsonBatson, 1975; Reference Liberman and ChaikenLiberman & Chaiken, 1992; Reference Lord, Ross and LepperLord, Ross, & Lepper, 1979). Some have rightly pointed out that this pattern of updating is not necessarily irrational given that it can result from Bayes’ rule in certain circumstances (Reference Cook and LewandowskyCook & Lewandowsky, 2016; Reference Jern, Chang and KempJern, Chang, & Kemp, 2014). However, by the same token, consistency with Bayes’ rule does not, on its own, necessarily preclude the influence of motivational factors. Indeed, the many degrees of freedom available to a mischievous Bayesian – what evidence to consider and how, exactly, to interpret that evidence – provide a variety of opportunities for motivation to influence an otherwise mechanical application of Bayes’ rule (c.f., Reference Rabin and SchragRabin & Schrag, 1999).
In Reference Cook and LewandowskyCook and Lewandowsky (2016), for example, Bayes’ theorem is made to accommodate belief polarization through the addition of variables such as “a pro-market worldview” and “trust in scientists” that influence an agent’s priors and interpretation of evidence about global warming. This, however, only pushes the question of motivational influence up a level: while it may indeed be Bayesian for someone to reject the academic consensus on global warming conditional on the belief that climate science is a communist conspiracy to undermine the free market, this hypercritical approach to evidence may itself not be warranted, especially if it were only adopted to protect a cherished belief.
The question of what normative constraints, if any, can be said to limit how a rational agent constructs their likelihood function and prior is a fascinating, complex, and as-yet unresolved problem; our goal here is simply to point out that the apparent use of Bayesian inference at one level does not necessarily preclude the influence of motivational factors at another. At the same time, it is important to note that post-hoc rationalization and other forms of motivated reasoning are not necessarily irrational once the many practical considerations that constrain cognition have been taken into account. Reference CushmanCushman (2020), for instance, argues that adjusting beliefs to rationalize our actions is a functional mechanism that transfers knowledge from the decision processes that underlie instinct, intuition, and habit to our rational mind. We turn to the influence of cognitive constraints next.
1.3.3 Cognitive Efficiency
Sense-making, like all cognitive processes, is subject to computational constraints. This has the implication that we generally cannot consider every potential interpretation of information in strict accordance with Bayes’ rule (Reference Aragones, Gilboa, Postlewaite and SchmeidlerAragones, Gilboa, Postlewaite, & Schmeidler, 2005; Reference KwisthoutKwisthout, 2011; Reference Van RooijVan Rooij, 2008). Recent work suggests that our brain instead approximates Bayesian inference using a step-by-step process of sampling – that is, we adopt a single working hypothesis and update it over time as we think of alternative hypotheses and gather new data (Reference Bramley, Dayan, Griffiths and LagnadoBramley, Dayan, Griffiths, & Lagnado, 2017; Reference Dasgupta, Schulz and GershmanDasgupta, Schulz, & Gershman, 2017; Reference DayanDayan, 1998; Reference Gershman, Vul and TenenbaumGershman, Vul, & Tenenbaum, 2009; Reference Griffiths, Vul and SanbornGriffiths, Vul, & Sanborn, 2012; Reference Levy, Reali and GriffithsLevy, Reali, & Griffiths, 2009; Reference Sanborn and ChaterSanborn & Chater, 2016; Reference Vul, Goodman, Griffiths and TenenbaumVul, Goodman, Griffiths, & Tenenbaum, 2014; Reference Vul and PashlerVul & Pashler, 2008). Randomly selecting potential hypotheses for consideration is inefficient because most of them would turn out to be nonsensical, so the alternatives we consider are usually local, piecewise modifications of the working hypothesis we already have and are heavily informed by prior beliefs (Reference Bramley, Dayan, Griffiths and LagnadoBramley et al., 2017; Reference Tenenbaum, Griffiths and NiyogiTenenbaum, Griffiths, & Niyogi, 2007). This strategy brings otherwise intractable problems within reach, but it also leads to a number of biases (Reference Chater, Zhu, Spicer, Sundh, León-Villagrá and SanbornChater et al., 2020; Reference Sanborn and ChaterSanborn & Chater, 2016).
The ambiguous visual patterns of Figure 1.1 demonstrate the constraint that we only perceive one interpretation at a time. Each image appears to flip back and forth between two distinct percepts, exhibiting “multistability.” Both interpretations are equally correct, but we experience one – and only one – as the concrete, definitive, and exclusive truth at each point in time. Thus, rather than being able to average over possible perceptual interpretations, we seem limited to sampling interpretations one at a time (Reference Moreno-Bote, Knill and PougetMoreno-Bote, Knill, & Pouget, 2011). In the examples of Figure 1.1, these shifts happen quickly enough that the subjectivity of our perception is made apparent and can be tempered by rational self-reflection, but this is likely the exception, not the rule. When extrapolated to higher-order epistemic (rather than perceptual) beliefs, it is easy to imagine how such illusions of objectivity might exacerbate interpersonal conflict between people who arrive at different conclusions from common information.
Another implication of this cognitive strategy is that sense-making proceeds much like evolution, which “tinkers” by creatively repurposing existing biological machinery to address new environmental challenges rather than redesigning each species, cell-type, organ, physiological process, or behavior from scratch. Reference JacobJacob (1977) illustrates this with the examples of the woodpecker and the aye-aye (a type of lemur), both of which exploit the same ecological niche of extracting insects from small crevices in dead wood. Each has adapted to accomplish this task using features already possessed by their evolutionary forebears: the woodpecker, whose ancestors possessed beaks but lacked hands, developed an elongated beak; the aye-aye, whose ancestors lacked beaks but had hands, developed unusually long and thin fingers. While there may, in principle, be an optimal solution to this ecological problem involving neither beak nor hand, evolution cannot discover it except by way of intermediate forms. Each evolutionary lineage is, in the near term, fated to build on what it has already developed.
In much the same way, reconsidering our entire worldview in light of every new piece of information would be computationally prohibitive: we do not update globally. Instead, we take the bulk of our beliefs as given and re-examine only those that bear most directly on new evidence we encounter. In Otto Neurath’s famous metaphor, knowledge is like a boat that is always on the high seas, never able to rebuild from scratch but always forced to repair using whatever existing materials are to hand (Reference Bramley, Dayan, Griffiths and LagnadoBramley et al., 2017; Reference Cat and ZaltaCat, 2021). In cases when our deeply held beliefs and evidence come into conflict, the latter is usually made to yield. Only rarely do we reflect upon and take stock of “core” beliefs, foundational assumptions, and axiomatic commitments; crises of faith are the exception, not the rule. Such phenomena are, to a degree, already a feature of hierarchical Bayesian models, where priors descending from higher levels can override the bottom-up flow of information (Reference Friston and KiebelFriston & Kiebel, 2009; Reference Tenenbaum, Griffiths and NiyogiTenenbaum et al., 2007; Reference Yuille and KerstenYuille & Kersten, 2006). Note, however, that these dynamics are greatly exacerbated by the process of local updating that makes a sampling approximation to Bayesian inference computationally efficient.
The ambiguous patterns in Figure 1.1 demonstrate that we do not control certain aspects of how our brain performs inference; no matter how hard we try, it is simply impossible to see both interpretations of a multistable image simultaneously. There are, however, other aspects of sense-making that we are able to influence through deliberative choice. To the degree that these choices bear upon our use of scarce cognitive resources, we might expect them to be mediated by motivational signals. In the same way that physical exhaustion exists to force us to reckon with the physiological consequences of continued exertion, some mental states may exist to “price in” the cognitive costs of sense-making operations.
This observation suggests a possible functional reinterpretation of dogmatic thinking, which Reference ChristensenChristensen (1994, p. 69) paraphrases as the epistemological attitude that “I happen to believe it – and that’s all the justification I need for continuing to believe it.” The process of updating beliefs imposes costs that range from the physical (e.g., rewiring neurons) to the practical (e.g., sapping cognitive resources from other important uses). Moreover, if belief updating is a serial, step-by-step process, it will inevitably be both limited and slow (although changing one belief may then, of course, have a cascade of implications for others). This means that the tendency to maintain one’s beliefs in the absence of any reason to do otherwise may be unavoidable – and possibly also normatively justified (a philosophical position that Gilbert Harman calls “general foundationalism”; Reference Harman and LuperHarman, 2003). In the presence of such considerations, a motivational force that pushes back against the free revision of belief might be beneficial, even if it led to epistemic distortions. Such an account might also help to explain why we are more dogmatic about certain types of beliefs than others; for example, a higher real cost of uprooting more fundamental beliefs would explain our greater resistance to questioning them.
1.4 Implications
The fact that sense-making is a drive helps to explain why humans are so enthusiastic about many activities that make no obvious contribution to survival or reproduction – for example, reading fiction, watching films, and solving puzzles that we ourselves create. Sense-making, like other drives, originally emerged to promote biological fitness, but operates even when this function is obviously nonoperative, similarly to when we have sex using birth control or consume “empty calories.” Indeed, forms of entertainment that provide the least amount of informational enrichment (and are therefore arguably the most heavily driven by sense-making, e.g., mystery novels) are often precisely structured to build and release suspense artificially. This characteristic strategy of drive buildup and consummation is seen across the many other drives we cultivate for pleasure. Take, for instance, hunger, which we actively protect by avoiding snacks that will “ruin our appetite,” then tease over many courses before finally indulging. Viewed from this perspective, much of what we call culture appears, indeed, to be a grand collection of machines that produce the pleasure of sense-making through the origination, elaboration, and resolution of fascinating complexities.
All drives are, of course, imperfect regulators. A starving person exposed to unlimited amounts of food will overconsume to the point of sickness or even death; extreme levels of pain and fear can, in some cases, become counterproductive. Sense-making, likewise, is not always perfectly calibrated to the provision of long-term benefits in every situation. The mass appeal of conspiracy theories and pseudoscientific frameworks, as exemplified by the widespread rejection of life-saving vaccines against COVID-19, illustrate sense-making taken beyond the point of functionality. The potential for sense-making to reach dysfunctional levels is also vividly illustrated by delusional schizophrenia, which is marked by a tendency to attribute “too much” coherence to meaningless or inconclusive information while dismissing contrary evidence (DSM-5, 2013; Reference McLean, Mattiske and BalzanMcLean, Mattiske, & Balzan, 2017). Of note, individual differences in conspiracy-mindedness and schizotypal personality disorder are interrelated (Reference Bruder, Haffke, Neave, Nouripanah and ImhoffBruder, Haffke, Neave, Nouripanah, & Imhoff, 2013; Reference Darwin, Neave and HolmesDarwin, Neave, & Holmes, 2011), raising the intriguing possibility that shared cognitive foundations may help explain these excesses of sense-making.
The rapid proliferation of new digital information technologies poses both great promise and great peril for sense-making. On one hand, the “information explosion” occasioned by the rise of the Internet has drastically expanded access to nourishment for sense-making, ranging from the most extensive encyclopedia in history (Reference VoßVoß, 2005) to tens of thousands of digitized books (Reference CoyleCoyle, 2006) and millions of user-generated videos (Reference Cheng, Dale and LiuCheng, Dale, & Liu, 2008). At the same time, social media has also greatly increased our exposure to the sense-making produced by others, leading to a dense cross-fertilization of ideas and the almost instantaneous transmission of new insights between people, leading to a kind of globalization of knowledge.
On the other hand, changes to the topological structure of communication have profoundly disrupted how sense-making flows through society, often in troubling ways. Homophily (the preferential tendency for similar individuals to form network connections; Reference McPherson, Smith-Lovin and CookMcPherson, Smith-Lovin, and Cook (2001)) combined with the sense-making distortions introduced by belief-based utility (as discussed in Section 1.2) has led to concern about the emergence of online “echo chambers”: massive networks of individuals who see and propagate information or explanations that corroborate their existing worldview with little critical feedback (Reference Bakshy, Messing and AdamicBakshy, Messing, & Adamic, 2015; Reference Colleoni, Rozza and ArvidssonColleoni, Rozza, & Arvidsson, 2014; Reference SunsteinSunstein, 2002; but see also the moderating evidence of Reference Dubois and BlankDubois & Blank, 2018; Reference Flaxman, Goel and RaoFlaxman, Goel, & Rao, 2016). These effects are, no doubt, exacerbated by the failure of individuals to take account of just how biased their media diet is (Reference Enke and ZimmermannEnke & Zimmermann, 2019; Reference Eyster and RabinEyster & Rabin, 2014; Reference Pronin, Lin and RossPronin, Lin, & Ross, 2002; Reference Vallone, Ross and LepperVallone, Ross, & Lepper, 1985), worsening the recalcitrance and illusions of objectivity already inherent to our inferential cognition.
Even outside echo chambers, certain of these belief dynamics threaten to reduce diversity in how we interpret the world. Rather than, quite literally, thinking for ourselves, the proliferation of public commentary has made it all too easy to simply adopt the explanations of the people and media we surround ourselves with. This is appealing to each individual in the short run, as doing so yields an immediate sense-making boost with little cognitive investment, but it is potentially disastrous to the health of social discourse as a whole, which depends on the diversity of public opinion.
For all its many benefits, the rapid democratization of information and mass communication has also had the side-effect of destabilizing mechanisms that societies have historically relied upon to filter information and vet explanations. Those who in ages past claimed privileged sense-making authority – most notably academic scholars, religious leaders, journalists, and political officials – are now frequently reduced to shouting their opinions over the din of popular commentary. Increasingly, the most valuable commodity in the marketplace of ideas is not a reputation for careful consideration, but rather the sheer ability to garner attention (Reference Heath and HeathHeath & Heath, 2007). Newly ascendant counternormative belief communities fueled by these dynamics – antivaxxers, climate deniers, flat earthers, conspiracy theorists, and religious extremists – have begun to undermine the ability of social institutions to function properly by out-competing their traditional counterparts when it comes to harnessing public attention, and, with it, opinion.
As the sense-making landscape has been upended, those who seek to influence society’s understanding have adapted their strategies to take advantage of the new opportunities it provides. Technology has created an increasingly sophisticated set of tools that grant the ability to precisely target and massively amplify both the dissemination (Reference GoldfarbGoldfarb, 2014; Reference Kramer, Guillory and HancockKramer, Guillory, & Hancock, 2014) and the suppression (Reference Bamman, O’Connor and SmithBamman, O’Connor, & Smith, 2012) of information. These efforts are becoming increasingly sophisticated now that insights from behavioral and data science are being applied to predict what will engage and persuade us (Reference Matz, Kosinski, Nave and StillwellMatz, Kosinski, Nave, & Stillwell, 2017; Reference Zarouali, Dobber, De Pauw and de VreeseZarouali, Dobber, De Pauw, & de Vreese, 2020).
These intra- and interindividual-level processes that determine the direction of sense-making can have profound consequences for society. As highlighted in George Reference MarshallMarshall’s (2015) insightful treatise Don’t Even Think About It: Why Our Brains Are Wired to Ignore Climate Change, our collective ability to grapple with existential problems facing humanity depends on how we collect and make sense of information. Different nations’ success in combating the coronavirus pandemic has likewise been affected by the sense that citizens have made of the virus and of interventions intended to stem its spread, often in ways that link to wider political attitudes and group affiliations.
The analysis of sense-making’s cognitive foundations that we have pursued in this chapter is only the start of a much broader intellectual project, one which will involve an analysis of how the quirks of our fixed sense-making capacity can be deceived by shifting environmental forces, especially those created by technological advances. As has been illustrated by some of the recent trends that have emerged from this dynamic, the psychological foundations of sense-making have far-reaching consequences for society that we are only just beginning to understand.
