So many cool connections to think about! But I'm a bit confused by this distinction between symbolism and connectionism (maybe in general, but also as you've described it):
> The classical Chinese view of language (and, as it turns out, modern connectionist paradigm in AI as well) interprets strings of names (tokens) as the external interface guiding program execution, where program is understood not propositionally as a proof or a syllogism, but as a parallel and distributed mechanism for pattern recognition, pattern prediction, and action.
Why doesn't the difference just amount to layer(s) of abstraction, i.e. are these two framings actually inconsistent? If such a program exists and will be executed, why can't we probe how it arrives at its outputs, and wouldn't doing that reveal some type of internal representation? Maybe your next post will answer these - I'm very curious to see what "natural signs" means, and why it's inconsistent with "framing the capabilities of LLMs in terms of reasoning, internal representations, and mental states".
Also, I think I am missing something - why the emphasis on parallel+distributed?
The distinction is, roughly, similar to the one between McCulloch-Pitts and Rosenblatt neural nets. In the former, the configurations represent logical propositions, in the latter they are vectors of activations. So, I guess everything really comes down to what we understand as internal representations.
In the symbolic paradigm, we work with the semantic theory of truth (going back to Tarski) -- the processing is serial, consisting of truth-preserving transformations of logical propositions. Representations are what philosophers of mind call "propositional attitudes," and they are meant to mirror the external environment. In the connectionist paradigm, the notion of truth is pragmatic -- it's ok to make mistakes sometimes, and there is no notion of truth-preserving transformations anymore, we are just manipulating vectors and only care about the end-to-end performance. The reason why one might not refer to these vectors as internal representations is that they do not encode propositional attitudes. Mental states are abstractions that arise in a layered architecture of a particular kind (Sunny Auyang speaks of "an open mind emerging from intricate infrastructures").
The notion of natural sign vs. conventional sign comes from semiotics, where three kinds of signs are introduced: symbols, icons, and indices. The idea is that the association of symbols with things is arbitrary and dependent on convention (e.g., why do we use these specific words for various things and not some others?), whereas some signs are associated with things in a stable manner. For example, the association between a map of a given terrain and the terrain itself (these are icons) or the association of animal tracks on the ground with the presence of that animal nearby (these are indices). From this perspective, Olshausen-Fields receptive fields capture natural signs, as opposed to conventional signs, and they are not manipulated as logical propositions.
In some sense, this really is a question about the right layer of abstraction -- seeing how programs with the same denotational semantics (input-output behavior) are realized by different operational semantics (what's going on internally).
Ah this is super helpful. I think my confusion comes from the fact that in neuroscience, (and I guess mechanistic interpretability in AI now) we try to probe an end-to-end system to understand its intermediate "representations", for example by trying to reconstruct input from neural activity. Even if the reconstruction function were very simple, or there were some kind of "mirroring" between the activity and the external world (e.g. a visual object moves in 2D and the neural activity moves the same way on a 2D manifold), these are not "truth-preserving transformations of logical propositions", and would count as natural signs?
I guess in my previous question, I was thinking about a case in which we found that logical propositions were approximated by such a system. But this would still be to some pragmatic end, and not propositional by construction. If the system were able to *exactly* recover the logical propositions, maybe it's then that it becomes a question of abstraction.
So many cool connections to think about! But I'm a bit confused by this distinction between symbolism and connectionism (maybe in general, but also as you've described it):
> The classical Chinese view of language (and, as it turns out, modern connectionist paradigm in AI as well) interprets strings of names (tokens) as the external interface guiding program execution, where program is understood not propositionally as a proof or a syllogism, but as a parallel and distributed mechanism for pattern recognition, pattern prediction, and action.
Why doesn't the difference just amount to layer(s) of abstraction, i.e. are these two framings actually inconsistent? If such a program exists and will be executed, why can't we probe how it arrives at its outputs, and wouldn't doing that reveal some type of internal representation? Maybe your next post will answer these - I'm very curious to see what "natural signs" means, and why it's inconsistent with "framing the capabilities of LLMs in terms of reasoning, internal representations, and mental states".
Also, I think I am missing something - why the emphasis on parallel+distributed?
The distinction is, roughly, similar to the one between McCulloch-Pitts and Rosenblatt neural nets. In the former, the configurations represent logical propositions, in the latter they are vectors of activations. So, I guess everything really comes down to what we understand as internal representations.
In the symbolic paradigm, we work with the semantic theory of truth (going back to Tarski) -- the processing is serial, consisting of truth-preserving transformations of logical propositions. Representations are what philosophers of mind call "propositional attitudes," and they are meant to mirror the external environment. In the connectionist paradigm, the notion of truth is pragmatic -- it's ok to make mistakes sometimes, and there is no notion of truth-preserving transformations anymore, we are just manipulating vectors and only care about the end-to-end performance. The reason why one might not refer to these vectors as internal representations is that they do not encode propositional attitudes. Mental states are abstractions that arise in a layered architecture of a particular kind (Sunny Auyang speaks of "an open mind emerging from intricate infrastructures").
The notion of natural sign vs. conventional sign comes from semiotics, where three kinds of signs are introduced: symbols, icons, and indices. The idea is that the association of symbols with things is arbitrary and dependent on convention (e.g., why do we use these specific words for various things and not some others?), whereas some signs are associated with things in a stable manner. For example, the association between a map of a given terrain and the terrain itself (these are icons) or the association of animal tracks on the ground with the presence of that animal nearby (these are indices). From this perspective, Olshausen-Fields receptive fields capture natural signs, as opposed to conventional signs, and they are not manipulated as logical propositions.
In some sense, this really is a question about the right layer of abstraction -- seeing how programs with the same denotational semantics (input-output behavior) are realized by different operational semantics (what's going on internally).
Ah this is super helpful. I think my confusion comes from the fact that in neuroscience, (and I guess mechanistic interpretability in AI now) we try to probe an end-to-end system to understand its intermediate "representations", for example by trying to reconstruct input from neural activity. Even if the reconstruction function were very simple, or there were some kind of "mirroring" between the activity and the external world (e.g. a visual object moves in 2D and the neural activity moves the same way on a 2D manifold), these are not "truth-preserving transformations of logical propositions", and would count as natural signs?
I guess in my previous question, I was thinking about a case in which we found that logical propositions were approximated by such a system. But this would still be to some pragmatic end, and not propositional by construction. If the system were able to *exactly* recover the logical propositions, maybe it's then that it becomes a question of abstraction.