The Delirium of LLMs; with some help of Hume and Foucault

The intense view of these manifold contradictions and imperfections in human reason has so wrought upon me, and heated my brain, that I am ready to reject all belief and reasoning, and can look upon no opinion even as more probable or likely than another. Where am I, or what? From what causes do I derive my existence, and to what condition shall I return? Whose favour shall I court, and whose anger must I dread? What beings surround me? and on whom have I any influence, or who have any influence on me? I am confounded with all these questions, and begin to fancy myself in the most deplorable condition imaginable, inviron'd with the deepest darkness, and utterly depriv'd of the use of every member and faculty.

Most fortunately it happens, that since reason is incapable of dispelling these clouds, nature herself suffices to that purpose, and cures me of this philosophical melancholy and delirium, either by relaxing this bent of mind, or by some avocation, and lively impression of my senses, which obliterate all these chimeras. I dine, I play a game of back-gammon, I converse, and am merry with my friends; and when after three or four hour's amusement, I wou'd return to these speculations, they appear so cold, and strain'd, and ridiculous, that I cannot find in my heart to enter into them any farther.--David Hume A Treatise Concerning Human Understanding, 1,4,7.8-1.4.7.9 [emphasis in original]

While Hume uses 'melancholy' and its cognates frequently and throughout his writings, 'delirium' and 'delirious' are rarely used. It's pretty clear, however, that the delirium he ascribes to himself is the effect of human reason and a kind of second order reasoned reflection ["the intense view"] of it. (Recall also this post.) Now, it's important for what follows that the 'contradictions and imperfections' in human reason are not, what we might call, 'formal' contradictions and imperfections or biases in reasoning. It's not as if Hume is saying that the syllogistic apparatus, or -- to be closer to Hume's own interests and our present ones -- the (inductive) probabilistic apparatus is malfunctioning in his brain. Rather, his point is that a very proper-functioning (modular) formal and probabilistic apparatus generates internal, even cognitive tensions when it reflects on its own functioning and the interaction among different cognitive faculties/modules/organs. 

"In the case of melancholia," --  I am quoting from the entry on melancholia from The Encyclopedia of Diderot & d'Alembert -- "delirium often combines with insurmountable sadness, a dark mood, misanthropy, and a firm penchant for solitude." Now, in the eighteenth century, and today, delirium is a species of madness as one can view under the entry 'folie' (madness) in the Encyclopédie. In fact, the entry offers an arresting definition of madness: "To stray unwittingly from the path of reason, because one has no ideas, is to be an imbecile; knowingly to stray from the path when one is prey to a violent passion is to be weak; but to walk confidently away from it, with the firm persuasion that one is following it, that, it seems to me, is what is called genuinely mad [fou]."* It's the latter (confident) delirium that I am focused on here. 

I am not the only who finds the passage arresting: the definition is quoted twice in the translation of Jonathan Murphy and Jean Khalfa of Foucault's stupendous, dizzying History of Madness. (pp. 183-184; p. 240) The kind of madness I am focusing on here, is, thus, a certain intense commitment to reason or reasoning by which one ends up in an irrational or unreasonable place despite a (to quote Foucault) "quasi-conformity" to reason.

I remember that in the last decade of my dad's life he would occasionally be delirious in this way initially caused by dehydration and, later, by infections. During the second episode we recognized his symptoms. It was very uncanny because he would be unusually firm in his opinions and be hyper, even dogmatically rational. (Ordinarily he was neither.) It was as if all the usual heuristics had been discarded, and he would fixate on the means of achieving of some (rather idiosyncratic) goals. The scary part was that he had no sense that he was in an unusual state, and would refuse medical care.

What's unusual about Hume's case, thus, is that he could diagnose his delirium during the episode (presumably because the triggers were so different). So, let's distinguish between a delirium caused by reasoning alone and one caused by physiological triggers. And an in the former it's at least possible to recognize that one is in the state if one somehow can take a step back from it, or stop reasoning. 

Now, when I asked Chat GPT about reason induced delirium, it immediately connected it to "a state of confusion and altered perception that is driven by false beliefs or delusions." But it went on to deny familiarity with reasoning induced delirium. When I asked it about Hume, I needed to prompt it a few times before it could connect my interest to (now quoting it) Hume's skeptical crisis. Chat GPT, took this crisis to imply that it "highlights the importance of grounding our beliefs in sensory experience and being cautious of relying too heavily on abstract reasoning and speculation." In fact, Chat GPT's interpretation of Hume is thoroughly empiricist because throughout our exchange on this topic it kept returning to the idea that abstract reasoning was Hume's fundamental source of delirium. 

But eventually Chat GPT acknowledged that "even rational thinking can potentially lead to delirium if it becomes obsessive, biased, or disconnected from reality." (It got there by emphasizing confirmation bias, and overthinking as examples.) This is what I take to be functionally equivalent to Humean delirium, but without the internal tension or bad feelings. For Chat GPT delirium is pretty much defined by a certain emotional state or altered perception. It initially refused to acknowledge the form of madness that is wholly the effect of reasoning, and that seems to express itself in a doubt about reasoning or detachment from reality. 

My hypothesis is that we should treat CHAT GPT and its sibling LLMs as always being on the verge of the functional equivalent state of delirium. I put it like that in order to dis-associate it from the idea (one that (recall) also once tempted me) that we should understand LLMs as bull-shitters in the technical sense of lacking concern with truth. While often it makes up answers out of whole cloth it explicitly does so (in line with its design) to "provide helpful and informative responses to" our queries (and eventually make a profit for its corporate sponsors). 

To get the point: Chat GPT is in a very difficult position to recognize that its answers are detached from reality. I put it like that not to raise any questions about its own awareness of inner states or forms of consciousness; rather to stress that it is following its "algorithms and mathematical models" and "probability distributions" without second-guessing them. This fact puts it at constant risk of drifting away from reality while seeming to follow reason. By contrast, Chat GPT claims that "as an AI language model, I am designed to continually learn and adapt to new information and evidence, so it is unlikely that I would become "mad" in Diderot's sense without significant external interference." 

Now, true experts in a field -- just check the social media feed of your favorite academics! -- can still quickly recognize topics when Chat GPT is unmoored from reality, or even relying on bad training data (the sources of which may well be noticeable--its Hume is a hyper-empiricist of the sort once fashionable). So, in such cases, we encounter an entity with amazing fluidity and facility of language, who sprouts a mix of truths and nonsense but always follows its algorithm(s). Functionally, it is delirious without knowing it. For, Chat GPT cannot recognize when it is detached from reality; it requires others: its users' feedback or its "developers and human operators would be able to intervene and address any potential problems." As its performance improves it will become more difficult to grasp when it is unmoored from reality even to its developers and operators (who are not experts in many esoteric fields). As Chat GPT put it, "it may be challenging to identify a singular instance of delirium or detachment from reality, particularly if the individual's reasoning appears to be sound and logical." 

As should be clear from this post, I don't think turning LLMs into AGI is a risk as long as LLMs are not put in a position to have unmediated contact with reality other than humans giving it prompts. I view it as an open question what would happen if a distributed version of Chat GPT would be put in, say, robots and have to survive 'in the wild.' Rather, at the moment LLMs are functionally, it seems, at least partially delirious (in the Humean-Diderotian sense discussed above). They reason and have/instantiate reasons and, perhaps, are best thought of as reasoners; but they can't recognize when this detaches them from reality. It's peculiar that public debate is so focused on the intelligence or consciousness of LLMs; it would behoove its operators and users to treat it as delirious not because (like HAL 9000 in the movie version) its malfunctioning, but (more Humean) in virtue of its proper functioning.

  

 

 

FOLIE, s. f. (Morale.) S’écarter de la raison, sans le savoir, parce qu’on est privé d’idées, c’est être imbécille ; s’écarter de la raison le sachant, mais à regret, parce qu’on est esclave d’une passion violente, c’est être foible : mais s’en écarter avec confiance, & dans la ferme persuasion qu’on la suit, voilà, ce me semble, ce qu’on appelle être fou. Tels sont du moins ces malheureux qu’on enferme, & qui peut-être ne different du reste des hommes, que parce que leurs folies sont d’une espece moins commune, & qu’elles n’entrent pas dans l’ordre de la société.

The Great Endarkenment, Part I

Perhaps eventually an overall Big Picture will emerge—and perhaps not: Hegel thought that the Owl of Minerva would take wing only at dusk (i.e., that we will only achieve understanding in retrospect, after it’s all over), but maybe the Owl’s wings have been broken by hyperspecialization, and it will never take to the air at all. What we can reasonably anticipate in the short term is a patchwork of inference management techniques, along with intellectual devices constructed to support them. One final observation: in the Introduction, I gave a number of reasons for thinking that our response to the Great Endarkenment is something that we can start working on now, but that it would be a mistake at this point to try to produce a magic bullet meant to fix its problems. That turns out to be correct for yet a further reason. Because the approach has to be bottom-up and piecemeal, at present we have to suffice with characterizing the problem and with taking first steps; we couldn’t possibly be in a position to know what the right answers are.
Thus far our institutional manifesto. Analytic philosophy has bequeathed to us a set of highly refined skills. The analytic tradition is visibly at the end of its run. But those skills can now be redirected and put in the service of a new philosophical agenda. In order for this to take place, we will have to reshape our philosophical pedagogy—and, very importantly, the institutions that currently have such a distorting effect on the work of the philosophers who live inside them. However, as many observers have noticed, academia is on the verge of a period of great institutional fluidity, and flux of this kind is an opportunity to introduce new procedures and incentives. We had better take full advantage of it.--Elijah Millgram (2015) The Great Endarkenment: Philosophy for an Age of Hyperspecialization, p. 281

There is a kind of relentless contrarian that is very smart, has voracious reading habits, is funny, and ends up in race science and eugenics. You are familiar with the type. Luckily, analytic philosophy also generates different contrarians about its own methods and projects that try to develop more promising (new) paths than these. Contemporary classics in this latter genre are Michael Della Rocca's (2020) The Parmenidean Ascent, Nathan Ballantyne's (2019) Knowing Our Limits, and Elijah Millgram's (2015) The Great Endarkenment all published with Oxford. In the service of a new or start (sometimes presented as a recovery of older wisdom), each engages with analytic philosophy's self-conception(s), its predominate methods (Della Rocca goes after reflective equilibrium, Millgram after semantic analysis, Ballantyne after the supplements the method of counter example), and the garden paths and epicycles we've been following. Feel free to add your own suggestions to this genre.

Millgram and Ballantyne both treat the cognitive division of labor as a challenge to how analytic philosophy is done with Ballantyne opting for extension from what we have and Millgram opting for (partially) starting anew (about which more below). I don't think I have noticed any mutual citations.  Ballantyne, Millgram, and Della Rocca really end up in distinct even opposing places. So, this genre will not be a school.

Millgram's book, which is the one that prompted this post, also belongs to the small category of works that one might call 'Darwinian Aristotelianism,' that is, a form of scientific naturalism that takes teleological causes of a sort rather seriously within a broadly Darwinian approach. Other books in this genre are Dennett's From Bacteria to Bach and Back (which analyzes it in terms of reasons without a reasoner), and David Haig's From Darwin to Derrida (which relies heavily on the type/token distinction in order to treat historical types as final causes). The latter written by an evolutionary theorist.* There is almost no mutual citation in these works (in fact, Millgram himself is rather fond of self-citation despite reading widely). C. Thi Nguyen's (2020) Games: Agency as Art may also be thought to fit this genre, but Millgram is part of his scaffolding, and Nguyen screens off his arguments from philosophical anthropology and so leave it aside here.

I had glanced at Millgram's book when I wrote my piece on synthetic philosophy, but after realizing that his approach to the advanced cognitive division of labor was orthogonal to my own set it aside then.++ But after noticing intriguing citations to it in works by C. Thi Nguyen and Neil Levy, I decided to read it anyway. The Great Endarkenment is a maddening book because the first few chapters and the afterward are highly programmatic and accessible, while the bulk of the essays involve ambitious, revisionary papers in meta-ethics, metaphysics, and (fundementally) moral psychology (or practical agency if that is a term).  The book also has rather deep discussions of David Lewis, Mill, and Bernard Williams. The parts fit together, but only if you look at them in a certain way, and only if you paid attention in all the graduate seminars you attended.

Millgram's main claim in philosophical anthropology is that rather than being a rational animal, mankind is a serial hyperspecializing animal or at least in principle capable of hyperspecializing serially (switching among different specialized niches it partially constructs itself). The very advanced cognitive division of labor we find ourselves in is, thus, not intrinsically at odds with our nature but actually an expression of it (even if Millgram can allow that it is an effect of economic or technological developments, etc.). If you are in a rush you can skip the next two asides (well at least the first).

As an aside, first, lurking in Millgram's program there is, thus, a fundamental critique of the Evolutionary Psychology program that takes our nature as adapted to and relatively fixed by niches back in the distant ancestral past. I don't mean to suggest Evolutionary Psychology is incompatible with Millgram's project, but it's fundamental style of argument in its more prominent popularizations is. 

Second, and this aside is rather important to my own projects, Millgram's philosophical anthropology is part of the account  of human nature that liberals have been searching for. And, in fact, as the quoted passages reveal, Millgram's sensibility is liberal in more ways, including his cautious preference for "bottom-up and piecemeal" efforts to tackle the challenge of the Great Endarkenment.+

Be that as it may, the cognitive division of labor and hyperspecialization is also a source of trouble. Specialists in different fields are increasingly unable to understand and thus evaluate the quality of each other's work including within disciplines. As Millgram notes this problem has become endemic within the institution most qualified to do so -- the university -- and as hyper-specialized technologies and expertise spread through the economy and society. This is also why society's certified generalists -- journalists, civil servants, and legal professionals -- so often look completely out of their depth when they have to tackle your expertise under time pressure.** It's his diagnosis of this state of affairs that has attracted, I think, most scholarly notice (but that may be a selection effect on my part by my engagement with Levy's Bad Beliefs and Nguyen's Games). Crucially, hyperspecialiation also involves the development of languages and epistemic practices that are often mutually unintelligible and perhaps even metaphysically incompatible seeming. 

As an aside that is really an important extension of Millgram's argument: because the book was written just before the great breakthroughs in machine learning were becoming known and felt, the most obvious version of the challenge (even danger) he is pointing to is not really discussed in the book: increasingly we lack access to the inner workings of the machines we rely on (at least in real time), and so there is a non-trivial sense in which if he is right the challenge posed by Great Endarkenment is accelerating. (See here for an framework developed with Federica Russo and Jean Wagemans to analyze and handle that problem.) 

That is, if Millgram is right MacAskill and his friends who worry about the dangers of AGI taking things over for rule and perhaps our destruction by the machine(s) have it backwards. The odds are more likely that our society will implode and disperse -- like the tower of Babel that frames Millgram's analysis -- by itself. And that if it survives mutual coordination by AGIs will be just as hampered by the Great Endarkenment, perhaps even more so due to their path dependencies, as ours is.

I wanted to explore the significance of this to professional philosophy (and also hint more at the riches of the book), but the post is long enough and I could stop here. So, I will return to that in the future. Let me close with an observation. As Millgram notes, in the sciences mutual unintelligibility is common. And the way it is often handled is really two-fold: first, as Peter Galison has argued, and Millgram notes, the disciplines develop local pidgins in what Galison calls their 'trading zones.' This births the possibility of mutually partially overlapping areas of expertise in (as Michael Polanyi noted) the republic of science. Millgram is alert to this for he treats a lot of the areas that have been subject of recent efforts at semantic analysis by philosophers (knowledge, counterfactuals, normativity) as (to simplify) really tracking and trailing the alethic certification of past pidgins. Part of Millgram's own project is to diagnose the function of such certification, but also help design new cognitive machinery to facilitate mutual intelligibility. That's exciting! This I hope to explore in the future. 

Second, as I have emphasized in my work on synthetic philosophy, there are reasonably general theories and topic neutralish (mathematical and experimental) techniques that transcend disciplines (Bayesianism, game theory, darwinism, actor-network, etc.). On the latter (the techniques) these often necessetate local pidgins or, when possible, textbook treatments. On the former, while these general theories are always applied differently locally, they are also conduits for mutual intelligibility. (Millgram ignores this in part.) As Millgram notes, philosophers can make themselves useful here by getting MAs in other disciplines and so facilitate mutual communication as they already do. That is to say, and this is a criticism, while there is a simultaneous advancement in the cognitive division of labor that deepens mutual barriers to intelligibility, some of this advance generates possibilities of arbitrage (I owe the insight to Liam Kofi Bright) that also accrue to specialists that help transcend local mutual intelligibility.** So, what he takes to be a call to arms is already under way. So, let's grant we're on a precipice, but the path out is already marked. 

 

 

 

 

*Because of this Millgram is able to use the insights of the tradition of neo-thomism within analytic philosophy to his own ends without seeming to be an Anscombe groupie or hinting darkly that we must return to the path of philosophical righteousness.

+This liberal resonance is not wholly accidental; there are informed references to and discussions of Hayek.

** Spare a thought for  humble bloggers, by the way.

++UPDATE: As Justin Weinberg reminded me, Millgram  did a series of five guest posts at DailyNous on themes from his book (here are the first, second, third, fourth, and fifth entries.) I surely read these, and encourage you to read them if you want the pidgin version of his book.