Still too few people reading JL Austin these days.
Thanks for introducing me to Waisman and "open texture" of language. I actually mildly prefer the translation "porosity of concepts" (working from Waisman's footnote giving the original German). I've been looking for a good way to capture the way natural/genAI language complicates building software. Classical software is 'deterministic', the semantics are 'finite' (need a better word) in the sense that a SQL update does precisely *this* and not *that*, while neuro (I'd like to call it 'quantum' software but CS people confuse quantization with discretization) software is always probabilistic (using softmax as Born rule).
So we need 'transduction' from porous semantics of language into discrete(?) semantics of databases (and other classical CS objects) and then back.
Regarding using softmax as Born rule: I recently decided to re-read Keith van Rijsbergen's _Geometry of Information Retrieval_, where he proposed to represent both documents and queries as vectors in a complex Hilbert space and use the Birkhoff-von Neumann logic of quantum observables as a formal model for interacting with the IR system. I don't think the book fully delivered what it set out to do, but the idea is interesting and somewhat reminiscent of what you are describing as "quantum" software.
Thanks for the Rijsbergen reference, I’ll look it up.
Mostly thinking of forward pass as time evolution of a discrete measure on token space which is “observed” with the softmax-“Born rule”. A crude “wave function” with nonlinear evolution, but some quantum intuition (eg path integral) resonates with our neural model of computation.
I heard Penrose speak at Cornell shortly after “The Emperor’s New Mind” came out and ever since I’ve been thinking that Penrose was right to try linking consciousness to QM, but he was doing it too literal-mindedly. Not entangled particles in a too-hot brain, but something mixing the ideas differently, like using “high, but finite-dimensional QFT”. Use the math in a different way sort of like quasiparticles in condensed matter. (Not actual “elementary” particles, but particle like excitations which can be described with the formal machinery of QFT.) I’ve been watching pieces of this picture slowly come into focus for 35 years now. It’s still kind of indistinct and probably wrong in some basic way. But tantalizing none the less.
David Mumford’s paper on vision and Bayesian inference from about 2003 was another fun milestone. So it’s cool to find my old hero drop a path integral calculation into the mix … 11 yrs ago … how did I miss it that long?
really enjoyed this, especially how you tied it together at the end.
one thought on the idea of a /situation/ especially in contrast to a /state/ -- the discussion of "situations" as a fundamental reserve of human freedom in the anarchist tradition of situationism and especially Tiqqun's "The Cybernetic Hypothesis"
The problem is that while our umwelts are inherently vague, we seem to think we can reduce them to some precisely characterizable basis, which leads to a hard problem (in the sense of Chalmers): how can a non-numerical fuzzy world emerge out of a numerically measurable substrate? I am not sure if the question as posed is solvable or even coherent.
On another note, how did Pospelov have enough access to Navya-Nyaya? The classical Indian theories of inference/anumana aren't formal in the way today's logical systems are, but they are a technical language, which makes me ask if there are refinements of natural language that remain non-formal and are yet strong enough to reason about robots that move with the agility of animals or humans.
Pure speculation on my part, but I am wondering if the Chicago school of mathematical biology initiated by Rashevsky and students of his such as Robert Rosen might be a bridge between the Soviet and the American schools of cybernetics. Rosen penned several lamentations that remind me of Zadeh's list of failures.
The question of Umwelts is interesting. I think in this regard we should take seriously von Neumann's suggestions in _The Computer and the Brain_ for treating the imprecision of the brain dynamics as fundamental, but, beyond this, the question is indeed not coherent. We can come up with various formal models (as Rosen does), but should be aware of their limitations. Certainly Zadeh's approach of encoding "fuzziness" in a rigid formal semantics seems to fall prey to fallacy of misplaced concreteness.
On Pospelov's sources for Navya-Nyāya: He cites the Russian translation of Donald Ingalls' book.
Regarding Rashevsky, Rosen etc. -- I think there might be something to that. Interestingly, like Zadeh, Rosen also tried to capture the essence of living systems in a formal framework based on category theory, and some categorical approaches along the lines of general systems theory were developed by Soviet cyberneticists (like Yuliy Shreider).
Thanks for the link to von Neumann’s Computer and the Brain/Silliman Lectures. I haven’t read them in a while - will have to take a look at his ideas on imprecision in the brain. We think it natural to reason precisely about imprecise things (probability theory, for example) but we don’t have systematic ways of reasoning imprecisely about precise things, especially if we are looking for situations where the imprecise meta-explanation is superior to the precise 1st order system. Is there an English language reference to the Yuliy Shreider style cybernetics? Wikipedia says he was a good friend of Varlam Shalamov whose Kolyma Tales I loved (if that’s the word for such dark writing).
Banger of a post.
Still too few people reading JL Austin these days.
Thanks for introducing me to Waisman and "open texture" of language. I actually mildly prefer the translation "porosity of concepts" (working from Waisman's footnote giving the original German). I've been looking for a good way to capture the way natural/genAI language complicates building software. Classical software is 'deterministic', the semantics are 'finite' (need a better word) in the sense that a SQL update does precisely *this* and not *that*, while neuro (I'd like to call it 'quantum' software but CS people confuse quantization with discretization) software is always probabilistic (using softmax as Born rule).
So we need 'transduction' from porous semantics of language into discrete(?) semantics of databases (and other classical CS objects) and then back.
"Open texture" --> discrete semantics --> "open texture".
Or weakly typed --> strongly typed --> weakly typed.
So being 'weakly typed' relates to 'porosity' and more broadly to the 'leaky abstraction' metaphor.
Also love the William Calvin article, need to read it more carefully.
Regarding using softmax as Born rule: I recently decided to re-read Keith van Rijsbergen's _Geometry of Information Retrieval_, where he proposed to represent both documents and queries as vectors in a complex Hilbert space and use the Birkhoff-von Neumann logic of quantum observables as a formal model for interacting with the IR system. I don't think the book fully delivered what it set out to do, but the idea is interesting and somewhat reminiscent of what you are describing as "quantum" software.
Thanks for the Rijsbergen reference, I’ll look it up.
Mostly thinking of forward pass as time evolution of a discrete measure on token space which is “observed” with the softmax-“Born rule”. A crude “wave function” with nonlinear evolution, but some quantum intuition (eg path integral) resonates with our neural model of computation.
Take a look at David Mumford's write-up on path integrals, it may be relevant: https://www.dam.brown.edu/people/mumford/BookBlogPosts/Feynman%20Path%20Integral.pdf
Fun to see this quantum analogy surfacing in the narrative of a fairly mainstream bit of research on 'reasoning' in models:
https://x.com/tydsh/status/1935206012799303817
I heard Penrose speak at Cornell shortly after “The Emperor’s New Mind” came out and ever since I’ve been thinking that Penrose was right to try linking consciousness to QM, but he was doing it too literal-mindedly. Not entangled particles in a too-hot brain, but something mixing the ideas differently, like using “high, but finite-dimensional QFT”. Use the math in a different way sort of like quasiparticles in condensed matter. (Not actual “elementary” particles, but particle like excitations which can be described with the formal machinery of QFT.) I’ve been watching pieces of this picture slowly come into focus for 35 years now. It’s still kind of indistinct and probably wrong in some basic way. But tantalizing none the less.
David Mumford’s paper on vision and Bayesian inference from about 2003 was another fun milestone. So it’s cool to find my old hero drop a path integral calculation into the mix … 11 yrs ago … how did I miss it that long?
really enjoyed this, especially how you tied it together at the end.
one thought on the idea of a /situation/ especially in contrast to a /state/ -- the discussion of "situations" as a fundamental reserve of human freedom in the anarchist tradition of situationism and especially Tiqqun's "The Cybernetic Hypothesis"
That thought did occur to me, the notion of situation as something actively constructed.
The problem is that while our umwelts are inherently vague, we seem to think we can reduce them to some precisely characterizable basis, which leads to a hard problem (in the sense of Chalmers): how can a non-numerical fuzzy world emerge out of a numerically measurable substrate? I am not sure if the question as posed is solvable or even coherent.
On another note, how did Pospelov have enough access to Navya-Nyaya? The classical Indian theories of inference/anumana aren't formal in the way today's logical systems are, but they are a technical language, which makes me ask if there are refinements of natural language that remain non-formal and are yet strong enough to reason about robots that move with the agility of animals or humans.
Pure speculation on my part, but I am wondering if the Chicago school of mathematical biology initiated by Rashevsky and students of his such as Robert Rosen might be a bridge between the Soviet and the American schools of cybernetics. Rosen penned several lamentations that remind me of Zadeh's list of failures.
The question of Umwelts is interesting. I think in this regard we should take seriously von Neumann's suggestions in _The Computer and the Brain_ for treating the imprecision of the brain dynamics as fundamental, but, beyond this, the question is indeed not coherent. We can come up with various formal models (as Rosen does), but should be aware of their limitations. Certainly Zadeh's approach of encoding "fuzziness" in a rigid formal semantics seems to fall prey to fallacy of misplaced concreteness.
On Pospelov's sources for Navya-Nyāya: He cites the Russian translation of Donald Ingalls' book.
Regarding Rashevsky, Rosen etc. -- I think there might be something to that. Interestingly, like Zadeh, Rosen also tried to capture the essence of living systems in a formal framework based on category theory, and some categorical approaches along the lines of general systems theory were developed by Soviet cyberneticists (like Yuliy Shreider).
Thanks for the link to von Neumann’s Computer and the Brain/Silliman Lectures. I haven’t read them in a while - will have to take a look at his ideas on imprecision in the brain. We think it natural to reason precisely about imprecise things (probability theory, for example) but we don’t have systematic ways of reasoning imprecisely about precise things, especially if we are looking for situations where the imprecise meta-explanation is superior to the precise 1st order system. Is there an English language reference to the Yuliy Shreider style cybernetics? Wikipedia says he was a good friend of Varlam Shalamov whose Kolyma Tales I loved (if that’s the word for such dark writing).