On Computational Positivism

From Indian astronomers to the Bitter Lesson and beyond.

Thanks to my colleague Lav Varshney, I have been exploring the writings of Roddam Narasimha on “computational positivism.” Narasimha was a renowned Indian aerospace engineer and fluid dynamicist with a PhD from Caltech, who also wrote extensively on the history of Indian science and mathematics.

Narasimha’s investigations were motivated by an observation made by Joseph Needham (a British biochemist and a historian of Chinese science and technology):

With the appearance on the scene of intensive studies of mathematics, science, technology and medicine in the great non-European civilizations, debate is likely to sharpen, for the failure of China and India to give rise to distinctively modern science while being ahead of Europe for fourteen previous centuries is going to take some explaining.

Narasimha’s answer to “the Indian half of Needham’s question” is based on the comparative analysis of two styles in science, which he refers to as Greek and Indic. In the context of astronomical calculations, the Greek style is exemplified by Ptolemy (2nd century CE). Ptolemy’s Almagest consists of thirteen books, the first of which is devoted entirely to setting up and justifying his axiomatic system, which was based on Greek metaphysics and cosmology and modeled on Euclid’s geometry, where the theoretical propositions are entailed deductively by the axioms. The Ptolemaic axioms reflected the Greek view of the finite geocentric universe, with the fixed stars located on the periphery and with the planets’ motion confined to concentric crystalline spheres, with the Earth in the center. These axioms reflect the Greek view of spheres and circles as representations of perfection. Proceeding deductively from these axioms, Ptolemy arrived at his theory based on epicycles. The axioms came first, codifying the metaphysical idea of perfection in circular motion. Ptolemy’s astronomy was eventually supplanted by the Newtonian “system of the world.”

By contrast, the Indic style is exemplified by Aryabhata (5th century CE). Unlike Ptolemy, who starts by presenting and justifying his axioms, Aryabhata “begins with lists of numerical parameters that he needs in the 60 or so algorithms described in the book.” The Indic approach is entirely computational—they use epicycles, just like Ptolemy, but there is no justification for them other than computational expediency and a close match to empirical observation. Unlike the Greek system, which is supported by a whole set of metaphysical and ontological commitments, the Indic system is unabashedly instrumentalist. For example, while epicycles are used in both systems, the Indian astronomers have no problem working with epicycles that have time-varying parameters, and the parameters in their algorithms are open to tuning and revision based on observations. There is little room for metaphysical commitments and for logical deduction. Narasimha emphasizes that, while Indian philosophers did use deductive logic, they mostly applied it as a critical tool for poking holes in their opponents’ arguments; the primary mode in Indian science was inferential and inductivist. The basic formula, drg (observation) + ganita (calculation) = siddhanta (effective or established conclusion), reflected the fact, pointed out by Hermann Weyl, that the concept of number was considered logically prior to the concepts of geometry in India. The calculations of Aryabhata, and especially of Nilakantha (14th-15th centuries CE), were a great deal more accurate than those based on Ptolemy’s theory; this situation persisted until about a hundred years after Newton’s Principia.

As Narasimha wrote,

the Indian schools declared that their objective was drîg-ganît-aikya, literally meaning the identity of the seen and the computed. It was understood that an algorithm which gives good agreement with observation at one time may not do so at a later time; it could gradually become weaker (slatha), i.e. discrepancies with observation could emerge. What one had to do at that time was in fact to change or ‘tune’ the algorithm.

He proposed to call this philosophical attitude “computational positivism:”

an approach in which the primary objective is to make computation agree with observation. Physical or geometrical models were not necessarily absent, but were clearly considered secondary, and even irrelevant—what was the point of a model, no matter how beautiful, if it did not result in quantitative agreement with observation? Models need algorithms before predictions can be made, and algorithms may imply models.

…

In pursuit of this computational positivist goal of ‘identity between the seen and the computed’, Indian mathematicians were willing to try mathematical options that would probably have shocked the Greeks. For example, Indian epicycles had time varying parameters, and orbits could even be taken as cut and pasted elliptic arcs (so they were non-dfferentiable at two points), undoubtedly seen as asaty’-opayas, useful fiction. This would not conform to Greek ideas of beauty, but a short algorithm that led to excellent agreement with observations would clearly conform to Indian ideas of effectiveness and elegance.

It is remarkable that the Indian system gave far more accurate predictions compared to the Ptolemaic one until about a hundred years after the publication of Newton’s Principia. Here, there are two important factors at work: The first one is the influence of Descartes’ analytic geometry based on the method of coordinates, which tilted the scales towards the logical primacy of numbers compared to geometry (so that, in particular, raising a numerical value to an integer power did not have to be interpreted geometrically as increasing the spatial dimension); the second one is Newton’s invention of (his version of) differential and integral calculus. Here, the conceptually revolutionary innovation was the following: If we wish to model the motion of a body (earthly or celestial) as a function of time, x(t), we can expand it in an infinite series, as in

\(x(t+h) = x(t) + x'(t)h + \frac{1}{2}x''(t)h^2 + \frac{1}{6}x'''(t)h^3 + \dots\)

Thus, prima facie, in order to deduce where the body will be at time t+h from the knowledge of its position at time t we need to know all the derivatives of x at t. In this context, one could view the Ptolemaic system based on epicycles as a high-order perturbation theory for almost-periodic functions. On the other hand, Newton’s laws of motion (in the differential equation form written down roughly a century later by Euler) entail that, for every time t, the second derivative of x can be expressed as a function of x(t) and x’(t) only. (The first derivative is the velocity, the second derivative is the acceleration, which is proportional to the total force applied to the body, and the force is a function of instantaneous position and velocity only). This was a revolutionary advance, a fundamental simplification both at the level of theory (we still retain universal laws in a parsimonious framework) and at the level of computation!

Earlier, I have referred to Hermann Weyl, who on the one hand did much to introduce the ideas of symmetry into mathematical physics but, on the other hand, was opposed to Hilbert’s formalist approach to mathematics on constructivist grounds. Weyl’s views were an elaboration of the ideas of Heinrich Hertz, who wrote the following in his Principles of Mechanics:

We create internal images or symbols of the external objects and we make them of such a kind, that logically necessary consequences of the symbols are always the symbols of caused consequences of the symbolized objects. A certain concordance must prevail between nature and our mind, or else this demand could not be satisfied. Experience teaches us that the demand can be satisfied and hence that such an agreement actually does exist. When we have succeeded in deriving symbols of this kind by means of previously gathered experiences we can by using them as models in a short time develop the consequences that will occur within the external world only after a long time or as reactions to our own interferences. Thus we become capable of anticipating facts and can direct our present decisions by such knowledge. The symbols we talk of are our concepts of the objects; they agree with them in that one essential respect which is expressed by the demand mentioned above, but it is irrelevant for their purpose that they should have any further resemblance with the objects. We neither know, nor do we have means to find out whether our representations of the objects have anything in common with the objects themselves except that one fundamental relation alone.

The “symbols” Hertz is talking about are the formal elements of a scientific theory (one of his famous quotes is that “Maxwell’s theory is Maxwell’s system of equations”), but the key point is that the deductive entailment relations between symbols of a theory have to correspond to material implication relations between the external objects depicted by these symbols. In other words, if the state of affairs of the world at time t is S(t) and we formally represent it by X(t) in our theory and if our theory gives the deductive entailment of X(t+h) from X(t), then we require X(t+h) to be the formal representation of S(t+h), which is materially implied by S(t) and corroborated by observation. This is not a mere matter of reducing everything to language, as the logical positivists of the Vienna Circle had hoped to do, because there is still the issue of Duhem-Quine undertermination of theory by evidence. As Hertz himself put it,

we cannot decide without ambiguity whether a picture is appropriate or not; as to this, differences of opinion may arise. One picture may be more suitable for one purpose, another for another. Only by gradually testing many pictures can we finally succeed in obtaining the most appropriate.

Here, Hertz puts forth three criteria: (1) logical consistency; (2) empirical correctness; (3) simplicity and distinctness. The first two are self-explanatory; for the third one, this is what Ulrich Majer says in his essay on the relation between Hertz’s views and the metatheoretical ideas of Hilbert, Weyl, and Ramsey:

Simplicity is the requirement that a picture (or theory) entails no superfluous elements where superfluous means roughly that the presence or non-presence of the elements in question have no effect whatsoever on the observable consequences of the theory.

Distinctness is the complementary requirement that a picture (or theory) entails enough elements to represent all the objective relations between the observed phenomena, which really exist; thus, a picture is distinct if no objective relation among the observable phenomena is missing.

From this, one could argue that computational positivism of the kind Narasimha describes is compatible with constructive empiricism of van Fraassen: The main virtue of scientific theories is to “save the phenomena,” and any request for explanation is subordinated to the virtue of empirical adequacy.

In a certain sense, computational positivism in its modern guise bears the same relation to logical positivism of the Vienna Circle as the one the connectionist approach to AI bears to the earlier symbolic approach. It is not at all a stretch, then, to read Rich Sutton’s “Bitter lesson” as a manifesto of computational positivism. In fact, there are two readings of it: the minimalist one (which, I think, everyone can agree on) and the maximalist one (which only the e/acc crowd will embrace fully). On the minimalist reading, we should avoid making metaphysical commitments that go beyond empirical adequacy (“the bitter lesson [is] that building in how we think we think does not work in the long run”) and aim for general-purpose frameworks based on a parsimonious algorithmic approach that (to use a Popperian turn of phrase) has proved its mettle (i.e., has been empirically correct) and carries the Hertzian virtues of simplicity and distinctness. Moreover (and this is the point made by any number of writers on the subject of complex systems), “the actual contents of minds are tremendously, irredeemably complex; we should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries.” Relatively simple meta-strategies, given enough time and data, will outperform fancy frameworks based on higher-level theories of cognition or intelligence. On the maximalist reading, though, Sutton’s Bitter Lesson applies universally not just to engineered systems but to the entire Universe, which is viewed as a vast computational Moloch—in the long run, nothing can beat what David Donoho has called “brutal scaling,” which is at work everywhere and at all levels and in all processes transpiring in the Universe. But here we see that this maximalist reading runs counter to the spirit of computational positivism because it tries to smuggle metaphysics (and really anti-humanist aesthetics to boot) through the backdoor!

Another way to see these things is through what Alistair G.J. MacFarlane, in his 1993 essay on “Information, knowledge, and control,” refers to as the tension between data descriptions and process descriptions. This idea, which MacFarlane credits to Herbert Simon’s The Sciences of the Artificial, refers to the fact that our models of natural and artificial systems come as a mixture of two flavors, based on extensional descriptions (data) and intensional descriptions (process). As he writes,

any computation carried out by an information processor will involve a mixture of data and process. Explanatory processes underpin all our understanding—to understand is to be able to explain. Explanation in essence arises from our ability to replace data by process. The working life of the scientist and engineer is a constant battle to strike a working balance between the management of data and the management of process, and to replace data by process to the largest possible extent.

At the end, MacFarlane advocates a pragmatic, empirical approach to designing next-generation intelligent control systems. While the Viennese logical positivism has largely failed, we would do well to realize that its computational counterpart is not infalliable. Some ideas and abstractions will always play a role—parameter modification by gradient descent, backpropagation, parallelization using GPUs are abstractions after all, and, while they may not conform to some people’s ideas of beauty, they may embody other people’s ideal of effectiveness and elegance. Perhaps, constructive computational empiricism is the way to go?