Ideally, we want to resist mis/disinformation but not evidence. If this is so, we need accounts of misinformation and disinformation to match the epistemic normative picture developed so far. This chapter develops a full account of the nature of disinformation. The view, if correct, carries high-stakes upshots, both theoretically and practically. First, it challenges several widely spread theoretical assumptions about disinformation – such as that it is a species of information, a species of misinformation, essentially false or misleading, or essentially intended/aimed/having the function of generating false beliefs in/misleading hearers. Second, it shows that the challenges faced by disinformation tracking in practice go well beyond mere fact checking. I begin with an interdisciplinary scoping of the literature in information science, communication studies, computer science, and philosophy of information to identify several claims constituting disinformation orthodoxy. I then present counterexamples to these claims and motivate my alternative account. Finally, I put forth and develop my account: disinformation as ignorance-generating content.

12.1 Information and Disinformation

Philosophers of information, as well as information and communication scientists, have traditionally focused their efforts in three main directions: offering an analysis of information, a way to measure it, and investigating prospects for analysing epistemic states – such as knowledge and justified belief – in terms of information. Misinformation and disinformation have traditionally occupied the backseat of these research efforts.Footnote ¹ The assumption has mostly been that a unified account of the three is going to become readily available as soon as we figure out what information is. As a result, for the most part, misinformation and disinformation have received dictionary treatment: for whatever the correct analysis of information was taken to be, misinformation and disinformation have either been taken to constitute the false variety thereof (misinformation) or the intentionally false/misleading variety thereof (disinformation) – by theorists endorsing non-factive accounts of disinformation – or, alternatively, something like information minus truth (misinformation) or information minus truth spread with an intention to mislead (disinformation) in the case of theorists endorsing factive accounts of information.

This is surprising in more than one way: first, it is surprising that philosophers of any brand would readily and unreflectively endorse dictionary definitions of pretty much anything – not to mention entities with such high practical stakes associated with them – such as mis/disinformation. Second, it is surprising that not more effort on the side of information, communication, and computer scientists has been spent on identifying a correct account of the nature of disinformation given the increasingly high stakes of issues having to do with the spread of disinformation that threaten our democracies, our trust in expertise, our uptake of health provision, and our social cohesion. We are highly social creatures, dependent on each other for flourishing in all walks of life. Our epistemic endeavours make no exception: due to our physical, geographical, and psychological limitations, most of the information we have is sourced in social interactions. We must inescapably rely on the intellectual labour of others, from those we know and trust well, to those whose epistemic credentials we take for granted online. Given the staggering extent of our epistemic dependence – one that recent technologies have only served to amplify – having a correct account of the nature of mis/disinformation, in order to be able to reliably identify it and escape it, is crucial.

Disinformation is widespread and harmful, epistemically and practically. We are currently facing a global information crisis that the Secretary-General of the World Health Organization (WHO) has declared an ‘infodemic’. Furthermore, crucially, there are two key faces to this crisis: two ways in which disinformation spreads societal ignorance: One concerns the widespread sharing of disinformation (e.g. fake cures, health superstitions, conspiracy theories, political propaganda, etc.), especially online and via social media, which contribute to dangerous and risky political and social behaviour. Separately, though at least as critical to the wider infodemic we face, is the prevalence of disinformation-generated resistance to evidence: even when the relevant information available is reliably sourced and accurate, many information consumers fail to take it on board or otherwise resist or discredit it (Klintman Reference Klintman2019) due to the rising lack of trust and scepticism generated by the ubiquity of disinformation. An important pay-off, then, of a correct analysis of the nature of disinformation is an understanding of how to help build and sustain more resilient trust networks. It is urgent that we gain such answers and insights: according to the 2018 Edelman Trust Barometer, UK public trust in social media and online news has plummeted to below 25 per cent, and trust in government is at a low of 36 per cent. This present crisis in trust of corresponds with a related crisis of distrust, in that the dissemination and uptake of disinformation, particularly on social media, have risen dramatically over the past few years (Lynch Reference Lynch2001, Levinson Reference Levinson2017, Barclay Reference Barclay2022).

12.2 Against Disinformation Orthodoxy

In what follows, I will scope the scientific and philosophical literature, identify three very widely spread – and rarely defended – assumptions about the nature of disinformation, and argue against their credentials.

1. (1) Assumption §1: Disinformation is a species of information (e.g. Shannon Reference Shannon1948, Carnap and Bar-Hillel Reference Carnap and Bar-Hillel1952, Frické Reference Frické1997, Fallis Reference Fallis2009, Reference Fallis2015, Cevolani Reference Cevolani2011, D’Alfonso Reference D’Alfonso2011, Dinneen and Brauner Reference Dinneen and Brauner2015).

These theorists take information to be non-factive and disinformation to be the false and intentionally misleading variety thereof. On accounts like these, information is something like meaning: ‘the cat is on the mat’, on this view, caries the information that the cat is on the mat in virtue of the fact that it means that the cat is on the mat. Disinformation, on this view, consists in spreading ‘the cat is on the mat’ in spite of knowing it to be false and with the intention to mislead.

Why think in this way? Two rationales can be identified in the literature, one practical and one theoretical.

12.3 A Knowledge-First Account of Disinformation

In what follows, I will offer a knowledge-first account of disinformation that aims to vindicate the findings of the previous section.

Traditionally, in epistemology (e.g. Dretske Reference Dretske1981) and philosophy of information alike, the relation between knowledge and information has been conceived on a right-to-left direction of explanation (i.e. several theorists have attempted to analyse knowledge in terms of information). Notably, Fred Dretske thought knowledge was information-caused true belief. More recently, Luciano Floridi’s (Reference Floridi2004) network theory involves an argument for the claim that, should information be embedded within a network of questions and answers, then it is necessary and sufficient for it to count as knowledge. Accounts like these, unsurprisingly, encounter the usual difficulties in analysing knowledge.

The fact that information-based analyses of knowledge remain unsuccessful, however, is not good reason to abandon the theoretical richness of the intuitive tight relation between the two. In extant work (Simion and Kelp Reference Simion, Kelp and Popa-Wyattforthcoming), I have developed a knowledge-based account of information that explores the prospects of the opposite, left-to-right direction of explanation: according to this view, very roughly, a signal s carries the information that p iff it has the capacity to generate knowledge that p.Footnote ⁴ On this account, then, information carries its functional nature up its sleeve, as it were: just like a digestive system is a system with the function to digest and the capacity to do so under normal conditions, information has the function to generate knowledge and the capacity to do so under normal conditions (i.e. given a suitably situated agent).

Against this background, I find it very attractive to think of disinformation as the counterpart of information: roughly, as stuff that has the capacity to generate or increase ignorance (i.e. to fully/partially strip someone of their status as knower, or to block their access to knowledge, or to decrease their closeness to knowledge). Here is the account I want to propose:

• Disinformation as ignorance-generating content (DIGC): X is disinformation in a context C iff X is a content unit communicated at C that has a disposition to generate or increase ignorance at C in normal conditions.

Normal conditions are understood in broadly etiological functionalist terms (e.g. Graham Reference Graham2012, Simion Reference Simion2019b, Reference Simion2021a) as the conditions under which our knowledge-generating cognitive processes have acquired their function of generating knowledge. The view is contextualist in that the same communicated content will act differently depending on contextual factors such as the evidential backgrounds of the audience members, the shared presuppositions, extant social relations, and social norms. Importantly, as with dispositions more generally, said content need not actually generate ignorance in the context – after all, dispositions are sometimes masked.

Now, generating ignorance can be done in a variety of ways – which means that disinformation will come in diverse incarnations. In what follows, I will make an attempt at offering a comprehensive taxonomy of disinformation. (The ambition to exhaustiveness is probably beyond the scope of this chapter, or even of an isolated philosophical project such as mine; however, it will be useful to have a solid taxonomy as a basis for a fully-fledged account of disinformation: at a minimum, any account should be able to incorporate all varieties of disinformation we will have identified.Footnote ⁵) Here it goes:

1. (1) Disinforming via spreading content that has the capacity of generating false belief. The paradigmatic case of this is the traditionally recognised species of disinformation: intentionally spread false assertions with the capacity to generate false beliefs in hearers.

2. (2) Disinforming via misleading defeat. This category of disinformation has the capacity of stripping the audience of held knowledge/being in a position to know via defeating justification.

3. (3) Disinforming via content that has the capacity of inducing epistemic anxiety (Nagel Reference Nagel2010). This category of disinformation has the capacity of stripping the audience of knowledge via belief defeat. The paradigmatic way to do this is via artificially raising the stakes of the context/introducing irrelevant alternatives as being relevant: ‘Are you really sure that you’re sitting at your desk? After all, you might well be a brain in a vat’; or ‘Are you really sure he loves you? After all, he might just be an excellent actor, in which case you will have wasted years of your life.’ The way in which this variety of disinforming works is via falsely implicating that these error possibilities are relevant in the context when in fact they are not. In this, the audience’s body of evidence is changed to include misleading justification defeaters.

4. (4) Confidence-defeating disinformation. This has the capacity to reduce justified confidence via justification/doxastic defeat: you are sure that your name is Anna, but I introduce misleading (justification/doxastic) defeaters, which gets you to lower your confidence. You may remain knowledgeable about p: ‘my name is Anna’ in cases in which the confidence lowering does not bring you below the knowledge threshold. Compatibly, however, your knowledge – or evidential support – concerning the correct likelihood of p is lost: you now take/are justified to take the probability of your name being Anna to be much lower than it actually is.

5. (5) Disinforming via exploiting pragmatic phenomena. Pragmatic phenomena can be easily exploited to the end of disinforming in all of the ways above: true assertions carrying false implicatures will display this capacity to generate false beliefs in the audience. I ask: ‘Is there a gas station anywhere near here? I’m almost out of gas.’ And you reply: ‘Yeah, sure, just one mile in that direction!’, knowing perfectly well that it’s been shut down for years. Another way in which disinformation can be spread via making use of pragmatic phenomena is by introducing false presuppositions. Finally, both justification and doxastic defeat will be achievable via speech acts with true content but problematic pragmatics, even in the absence of generating false implicatures.

What all of these ways of disinforming have in common is that they generate ignorance – by generating either false beliefs, knowledge loss, or a decrease in warranted confidence. One important thing to notice, which was also briefly discussed in the previous section, is that this account, and the associated taxonomy, is strongly audience-involving, in that disinformation has to do with the capacity to have a particular effect – generating ignorance – in the audience. Importantly, though, this capacity will heavily depend on the audience’s background evidence/knowledge: after all, in order to figure out whether a particular piece of communicated content has the disposition to undermine an audience in their capacity as knowers, it is important to know their initial status as knowers. Here is, then, on my view, in more precise terms, what it takes for a signal to carry a particular piece of disinformation for an audience A:

• Agent disinformation: A signal r carries disinformation for an audience A wrt p iff A’s evidential probability that p conditional on r is less than A’s unconditional evidential probability that p and p is true.

What is relevant for agent disinformation with regard to p is the probability that p on the agent’s evidence. And A’s evidence – and, correspondingly, what underlies A’s evidential probability – lies outwith A’s skull: it consists in probability raisers that A is in a position to know. Recall the account defended in Chapter 7:

• Evidence as knowledge indicators: A fact e is evidence for p for S iff S is in a position to know e and P(p/e) > P(p).

In turn, we have seen that, on this account, a fact e being such that I am in a position to know it has to do with the capacity of my properly functioning knowledge-generating capacity to take up e:

• Being in a position to know: S is in a position to know a fact e if S has a cognitive capacity with the function of generating knowledge that can (qualitatively, quantitatively, and environmentally) easily uptake e in cognisers of S’s type.

This completes my account of disinformation. On this account, disinformation is the stuff that undermines one’s status as a knower. It does so via lowering their evidential probability for p – the probability on the p-relevant facts that they are in a position to know – for a true proposition. It can, again, do so by merely communicating to A (semantically, pragmatically, etc.) that not-p when in fact p is the case. Alternatively, it can do so by (partially or fully) defeating A’s justification for p, A’s belief that p is the case, or A’s confidence in p.

One worry that the reader may have at this point goes along the following lines: isn’t the account in danger of over-generating disinformation? After all, I might be wrong about something I tell you through no fault of my own; isn’t it harsh to describe me as thereby spreading disinformation? Furthermore, every true assertion that I make in your presence about p being the case may, for all I know, serve as (to some extent) defeating evidence for a different proposition q, which may well be true. I truthfully tell you it’s raining outside, which, unrelatedly and unbeknownst to me, together with your knowledge about Mary not liking the rain, may function as partial rebutting defeat for ‘Mary is taking a walk’ – which may well, nevertheless, be true. Is it now appropriate to accuse me of having thereby disinformed you? Intuitively, that seems wrong.Footnote ⁶

Here are also a couple of parallel cases from Sandy Goldberg (in conversation): say that S is widely (but falsely) thought to be an inveterate liar, so that whenever S says that p, everyone immediately concludes that ~p. Prior to encountering S, A has a credence of 0.2 in p (this is what A’s evidence supports prior to encountering S’s testimony). S testifies (truly) that p, and A, who, like everyone else in the entire community, takes S to be a liar, drops her credence in p to 0.1. If S’s reputation as a liar is assumed to be common knowledge, such that everyone knows of it and would update accordingly, it seems that S’s true testimony would meet my analysis of agent disinformation. Conversely, one can imagine cases in which S says something false, explicitly aiming to disinform, but others (who know of S’s lying ways) come to true conclusions on the basis of the fact that S said so. On my account, this will not count as a case of disinformation.

Three things about these cases: first, note, once more, that intentions don’t matter for disinforming. As such, restricting disinforming via defeat to intentional/functional cases will not work for the same reasons that created problems for the intention/function condition on disinformation more broadly – we want an account of disinformation to be able to predict that asserters generating doubt about, for example, climate change via spreading defeaters to scientific evidence, even if they do it without any malicious intention, are disinforming the audience.

Second, note that it is independently plausible that, just as any bad deed can be performed blamelessly, one can also disinform blamelessly; if so, given garden variety epistemic and control conditions on blame, any plausible account of disinformation will have to accommodate non-knowledgeable and non-intentional instances of disinformation. Conversely, like with all intentions, intentions to disinform can also fail: one may aim to disinform and fail to do so. Indeed, it would be theoretically strange if an intention to disinform will be analytically successful.

Finally, note that we don’t need to restrict the account in order to accommodate the datum that disinformation attribution, and the accompanying criticism, would sound inappropriate in the cases above. We can use simple Gricean pragmatics to predict as much via the maxim of relevance: since the issue of whether Mary was going for a walk was not under discussion, and nor was it remotely relevant in our conversational context, flat out accusing you of disinforming me when you assert truthfully that it’s raining is pragmatically impermissible (although strictly speaking true with regard to Mary’s actions).

Going back to the account, note that, interestingly, on this view, one and the same piece of communication can, at the same time, be a piece of information and a piece of disinformation: information, as opposed to disinformation, is not context-relative. Content with knowledge-generating potential (i.e. that can generate knowledge in a possible agent) is information. Compatibly, the same piece of content, in a particular context, can be a piece of disinformation insofar as it has a disposition to generate ignorance under normal conditions. I think this is the right result: me telling you that p: 99 per cent of Black people at Club X are staff members is me informing you that p. Me telling you that p in the context of you inquiring as to whether you can give your coat to a particular Black man is a piece of disinformation since it carries a strong disposition (due to the corresponding relevance implicature) to generate the unjustified (and maybe false) belief in you that this particular Black man is a member of staff (Gendler Reference Gendler2011).

Finally, and crucially, my account allows that disinformation for an audience A can exist in the absence of A’s hosting any relevant belief/credence: (partial) defeat of epistemic support that one is in a position to know is enough for disinformation. Even if I (irrationally) don’t believe that vaccines are safe or that climate change is happening to begin with, I am still vulnerable to disinformation in this regard in that I am vulnerable to content that has, under normal conditions, a disposition to defeat epistemic support available to me that vaccines are safe and climate change is happening. In this, disinformation, on my view, can generate ignorance even in the absence of any doxastic attitude – by decreasing closeness to knowledge via defeating available evidence. This, I submit, is a very nice result: in this, the account explains the most dangerous variety of disinformation available out there – disinformation targeting the already epistemically vulnerable.

12.4 Conclusion

Disinformation is not a type of information and disinforming is not a way of informing: while information is content with knowledge-generating potential, disinformation is content with a disposition to generate ignorance under normal conditions in the context at stake. This way of thinking about disinformation, crucially, tells us that it is much more ubiquitous and hard to track than it is currently taken to be in policy and practice: mere fact-checkers just won’t do. Some of the best disinformation detection tools at our disposal will fail to capture most types of disinformation. To give but a few examples (but more research on this is clearly needed): the PHEME project aims to algorithmically detect and categorise rumours in social network structures (such as X (formerly Twitter) and Facebook) and to do so, impressively, in near real time. The rumours are mapped according to four categories, including ‘disinformation, where something untrue is spread with malicious intent’ (Søe Reference Søe2016). Similarly, Kumar and Geethakumari’s project (Reference Kumar and Geethakumari2014) had developed an algorithm that ventures to detect and flag whether a tweet is misinformation or disinformation. In their framework, ‘Misinformation is false or inaccurate information, especially that which is deliberately intended to deceive [and d]isinformation is false information that is intended to mislead, especially propaganda issued by a government organization to a rival power or the media’ (Kumar and Geethakumari Reference Kumar and Geethakumari2014, 3). In Karlova and Fisher’s (Reference Karlova and Fisher2013) diffusion model, disinformation is taken to be deceptive information. Hoaxy is ‘a platform for the collection, detection, and analysis of online misinformation, defined as “false or inaccurate information”’ (Shao et al. Reference Shao, Ciampaglia, Flammini and Menczer2016, 745). Examples targeted, however, include clear cases of disinformation such as rumours, false news, hoaxes, and elaborate conspiracy theories (Shao et al. Reference Shao, Ciampaglia, Flammini and Menczer2016).

It becomes clear that these excellent tools are just the beginning of a much wider effort that is needed in order to capture disinformation in all of its facets rather than mere paradigmatic instances thereof. At a minimum, pragmatic deception mechanisms, as well as evidential probability-lowering potentials, will need to be tracked against an assumed (common) evidential background of the audience.