Five Philosophy Papers Everyone* Should Read

*in AI/ML

I was in the Bay Area over the weekend, so Alyosha Efros twisted my arm to give a talk in his weekly group meeting. Since he asked me to “keep it entertaining,” I took this opportunity to highlight five philosophy papers that have influenced my thinking on AI/ML and to have a discussion around these topics.

Disclaimer: The five paper recommendations below (and one anti-recommendation) largely come from the Anglo-American analytic philosophy tradition. This is not meant to suggest that continental philosophy is irrelevant in the context of AI/ML, quite the contrary. Anyone working in those fields should seriously engage with thinkers like Ludwig Wittgenstein, Edmund Husserl, Maurice Merleau-Ponty, Gilbert Simondon, Hans-Georg Gadamer, Paul Ricoeur, and Jean-Pierre Dupuy. It’s more that the continental tradition largely emphasizes books rather than papers. With this in mind, here goes.

Nelson Goodman, “The new riddle of induction” (1955)

This is the “grue” paper. Machine learning is engineered induction, and its business is making predictions based on observed regularities. As Goodman says towards the end of the paper, “regularities are where you find them, and you can find them anywhere.” So the question is not about how induction can be justified—he says that “predictions are justified if they conform to valid canons of induction; and the canons are valid if they accurately codify accepted inductive practice.” It’s more that the justification of induction tout court has been displaced by the problem of defining confirmation, which in turn gives rise to the problem of demarcating confirmable hypotheses from unconfirmable ones. As an example of the latter, Goodman constructs the “grue” predicate: “it applies to all things examined before t just in case they are green but to other things just in case they are blue.” Here, t is a fixed but arbitrary time. So, if we observe a bunch of emeralds before time t, this evidence is consistent with them being green and with them being grue. The problem with grue, according to Goodman, is that it is not a projectible predicate: it does not help us in our future dealings with the world unlike the assumption that emeralds are green. (To drive this point home, I suggested that one could redefine green to mean that an object is grue if observed prior to time t but green otherwise — it’s just as compatible with the data, but is patently silly.) As Goodman says in closing the paper,

the problem of prediction from past to future cases is but a narrower version of the problem of projecting from any set of cases to others. We saw that a whole cluster of troublesome problems concerning dispositions and possibility can be reduced to this problem of projection. That is why the new riddle of induction, which is more broadly the problem of distinguishing between projectible and non-projectible hypotheses, is as important as it is exasperating.

We are in the business of pattern recognition, so we may as well come to terms with that and start worrying about whether our pattern recognition methods are (to use William James’ terms) expedient and workable. Statistical learning theory was an attempt to mathematize this using ideas like Structural Risk Minimization and walls of inequalities from empirical process theory. Now that these parts of the theory have been formalized in Lean, we can forget all about it. Just keep predicting the next token, bro.

Willard V.O. Quine, “Two dogmas of empiricism” (1951)

You will notice that there are two papers by Quine on my list. There are good reasons for this. Chief among them is that Quine was a damn good writer; when Gian-Carlo Rota quipped that “when a philosopher writes well, one can forgive him anything, even being an analytic philosopher,” he was talking about Quine. At any rate, this paper is important because it questions the distinction between analytic and synthetic truths that goes back at least to Kant and that was taken for granted by logical empiricists of the Vienna Circle (see, e.g., their talk of protocol sentences). Analytic truths are those that are true independently of matters of fact, and synthetic truths are ones that are grounded in matters of fact. For example, “all bachelors are unmarried men” is analytic, while “John is a bachelor” is synthetic. Quine undermines this distinction by arguing that one cannot formulate analytic truths without some accompanying synthetic claims about the framework of discourse, and similarly one cannot express synthetic truths without an analytic framework of supporting theories — e.g., the legal and the romantic aspects of marriage are presupposed in all of the talk of bachelors and unmarried men, whether analytic or synthetic. In AI/ML language, nothing is purely data-driven or purely theory-derived. Even Alyosha’s motto that “everything is nearest neighbors” can only make sense in the context of a given metric structure, which is necessarily theory-laden to some nontrivial extent. Quine’s metaphor of the fabric (or web) of knowledge is very evocative in this regard:

The totality of our so-called knowledge or beliefs, from the most casual matters of geography and history to the profoundest laws of atomic physics or even of pure mathematics and logic, is a man-made fabric which impinges on experience only along the edges. Or, to change the figure, total science is like a field of force whose boundary conditions are experience. A conflict with experience at the periphery occasions readjustments in the interior of the field. Truth values have to be redistributed over some of our statements. Re-evaluation of some statements entails re-evaluation of others, because of their logical interconnections - the logical laws being in turn simply certain further statements of the system, certain further elements of the field. Having re-evaluated one statement we must re-evaluate some others, whether they be statements logically connected with the first or whether they be the statements of logical connections themselves. But the total field is so undetermined by its boundary conditions, experience, that there is much latitude of choice as to what statements to reevaluate in the light of any single contrary experience. No particular experiences are linked with any particular statements in the interior of the field, except indirectly through considerations of equilibrium affecting the field as a whole.

As I mentioned during the talk, I find this metaphor useful when thinking about the roles of self-attention and multilayer perceptrons in transformers. The MLPs store information about the boundary conditions pertaining to factual knowledge, and the self-attention mechanism generates what Quine called the field of force. This somewhat vindicates Alyosha’s “everything is nearest neighbors” ideology — in fact, the Soviet book that introduced kernel methods in 1979 (The Method of Potential Functions in the Theory of Machine Learning by Aizerman, Braverman, and Rozonoer) used the field metaphor to motivate kernel methods.

Willard V.O. Quine, “Epistemology naturalized” (1969)

The second paper by Quine on my list contains the great line “the Humean predicament is the human predicament.” In that paper, Quine presents his argument that philosophy should be properly viewed as one with natural science, not somehow prior to it:

Epistemology, or something like it, simply falls into place as a chapter of psychology and hence of natural science. It studies a natural phenomenon, viz., a physical human subject. This human subject is accorded a certain experimentally controlled input — certain patterns of irradiation in assorted frequencies, for instance — and in the fullness of time the subject delivers as output a description of the three-dimensional external world and its history. The relation between the meager input and the torrential output is a relation that we are prompted to study for somewhat the same reasons that always prompted epistemology; namely, in order to see how evidence relates to theory, and in what ways one’s theory of nature transcends any available evidence.

This can be taken as a clear statement of what is often referred to as the Duhem-Quine thesis of the fundamental underdetermination of theory by evidence. In the context of AI/ML, this is newly relevant because of all the fashionable talk of “world models.” World models are really implicit theories the organism (or the AI system, if we want to go there) forms about the “three-dimensional external world,” and the architecture of these theories is fundamentally built on interconnections of inductive pattern recognizers. Which brings us to the next paper.

Friedrich Hayek, “The primacy of the abstract” (1969)

I already mentioned this paper before. Hayek is important in AI/ML because he was one of the first to articulate a sophisticated connectionist theory of perception and action based on pattern classification and recognition in his 1952 book The Sensory Order. This paper elaborates on some of his earlier ideas and argues that

the primary characteristic of an organism is a capacity to govern its actions by rules which determine the properties of its particular movements; that in this sense its actions must be governed by abstract categories long before it experiences conscious mental processes, and that what we call mind is essentially a system of such rules conjointly determining particular actions. In the sphere of action what I have called “the primacy of the abstract” would then merely mean that the dispositions for a kind of action possessing certain properties comes first and the particular action is determined by the superimposition of many such dispositions.

Hayek’s command of the relevant literature is impressive, and he brings up ideas from thinkers like Hermann Helmholtz, the Gestalt psychologists, and J.J. Gibson to support his theories. It is indeed pattern recognition all the way down, and the particular actions that are taken in a given context are determined by structural coupling of the organism with its environment. It’s interesting that Hayek uses the term “rules” here pretty much in the same sense as Wittgenstein does in Philosophical Investigations — rules (as distinguished from formal precepts) are not easily verbalizable, deeply embedded in a given practice, and govern how one acts in a given context procedurally rather than how one would describe that context propositionally.

Daniel Dennett, “Real patterns” (1991)

If one does not want to read The Intentional Stance, this paper is the next best thing. In fact, unlike the book which is somewhat dated in its stubborn insistence on GOFAI metaphors, this 1991 paper contains a crisp formulation of what the intentional stance is and what it does using the language of algorithmic information theory. Taking a cue from Quine’s radical behaviorism, Dennett argues that, if an external observer continues making relatively successful predictions about a given system’s externally observed behavior by attributing goals, beliefs, and desires to that system, then it is legitimate to ascribe to this system various internal states that encode these goals, beliefs, and desires. That is, if we can do better at predicting the future behavior of a given system when we assume that it acts as if it aims to optimize some criterion of success and forms beliefs pertinent to that, then we may as well throw caution to the wind and drop the “as if” altogether. One of Dennett’s favorite examples is Conway’s Game of Life — compare two observers, to one of whom it is just a dynamically changing pattern of black and white pixels, while the other uses the language of birth, death, and conflict to describe it. Even if the two observer make more or less the same predictions about the game, the fact that the second observer’s descriptive stance is more intelligible and has much lower Kolmogorov complexity is what warrants the claim that birth, death, and conflict in The Game of Life are “real patterns:”

Where utter patternlessness or randomness prevails, nothing is predictable. The success of folk-psychological prediction, like the success of any prediction, depends on there being some order or pattern in the world to exploit. Exactly where in the world does this pattern exist? What is the pattern a pattern of? Some have thought, with Fodor, that the pattern of belief must in the end be a pattern of structures in the brain, formulae written in the language of thought. Where else could it be? Gibsonians might say the pattern is “in the light”—and Quinians (such as Donald Davidson and I) could almost agree: the pattern is discernible in agents’ (observable) behavior when we subject it to “radical interpretation” (Davidson) “from the intentional stance” (Dennett).

It’s interesting that, toward the end of his life, Dennett was issuing dire warnings about the dark side of the intentional stance (what he called the problem with counterfeit people). Writing in 1991, he was not particularly worried about the reification fallacy, even though a more or less immediate objection to all of his theorizing about goals, beliefs, and desires is that they can be more readily attributed to the perceiver making predictions rather than to the system being perceived — especially if, following Dennett, we invoke the “commercial metaphor” and talk about the perceiver’s effectiveness in making lucrative bets about the system being observed. Nevertheless, “Real patterns” is an important paper we have to engage with, especially because the intentional stance is invoked in current debates about whether LLMs are conscious.

The anti-recommendation: Alan Turing, “Computing machinery and intelligence” (1950)

This is, in my opinion, the most overrated paper in philosophy of mind and in AI/ML (goes to show that good mathematicians are not necessarily good public intellectuals). If anyone is interested in contemporary thought on the subject, they would do better to read Gilbert Ryle’s The Concept of Mind, which came out a year earlier and which Turing should have cited but did not. The main problem with Turing’s paper is that it is so vague that everyone projects their own pet theories and predilections onto it, often not even noticing that what they are saying is in direct contradiction with what Turing was writing. Case in point: when Richard Dawkins wrote about his (unintentionally tragicomic) experience with Claude, he opened by mentioning “Computing machinery and intelligence” and then stating that “[w]hen Turing wrote — and for most of the years since — it was possible to accept the hypothetical conclusion that, if a machine ever passed his operational test, we might consider it to be conscious.” In fact, Turing disavows this inference explicitly! Indeed, in his objection to “the argument from consciousness” he says

I do not wish to give the impression that I think there is no mystery about consciousness. There is, for instance, something of a paradox connected with any attempt to localise it. But I do not think these mysteries necessarily need to be solved before we can answer the question with which we are concerned in this paper.

Apart from the silliness of some of the discussion (e.g., it is impossible to read Turing’s discussion of ESP without cringing), there is just too much emphasis on (relatively) fixed and stable rules that somehow underlie thinking and that can be simulated in a computer:

The idea of a learning machine may appear paradoxical to some readers. How can the rules of operation of the machine change? They should describe completely how the machine will react whatever its history might be, whatever changes it might undergo. The rules are thus quite time-invariant. This is quite true. The explanation of the paradox is that the rules which get changed in the learning process are of a rather less pretentious kind, claiming only an ephemeral validity. The reader may draw a parallel with the Constitution of the United States.

Reading this in 2026 immediately brings to mind Claude’s Constitution (aka its “soul document”) put together by Anthropic’s chief philosopher Amanda Askell. Hopefully I am not the only one who finds it funny that a utilitarian is attempting to teach virtue ethics to a machine; but (again revisiting Wittgenstein’s key distinction between formal precepts and informal rules) I am also reminded of Lewis Carroll’s “What the Tortoise said to Achilles” (1895), where Achilles attempts to teach formal logic to the Tortoise, and the following dialogue takes place between them:

‘Now that you accept A and B and C and D, of course you accept Z.’

‘Do I?’ said the Tortoise innocently. ‘Let’s make that quite clear. I accept A and B and C and D. Suppose I still refuse to accept Z?’

‘Then Logic would take you by the throat, and force you to do it!’ Achilles triumphantly replied. ‘Logic would tell you “You can’t help yourself. Now that you’ve accepted A and B and C and D, you must accept Z.” So you’ve no choice, you see.’

‘Whatever Logic is good enough to tell me is worth writing down,’ said the Tortoise. ‘So enter it in your book, please. We will call it (E) If A and B and C and D are true, Z must be true. Until I’ve granted that, of course, I needn’t grant Z. So it’s quite a necessary step, you see?’

‘I see,’ said Achilles; and there was a touch of sadness in his tone.

The story ends with the narrator returning to the same spot several months later, only to find Achilles and the Tortoise still sitting there, with the Tortoise’s book of rules nearly full. Turing’s vision of intelligence as paperwork is (sadly) still alive and well, as everyone’s claude.md files keep getting longer and longer. We thought that the frame problem had vanished together with the last remnants of GOFAI, but now it’s back in full force as we keep adding caveats upon caveats to our model prompts, with no end in sight.