Does AI Think, and Is It Conscious?

Posted first as a comment on a friend's Facebook post, but worth elevating here, I think. Extended/corrected and corrected:

The anthropomorphizing of (present-day) AI doesn't bother me much. I don't refer to any LLM by name or say "please" and "thank you" and worry about its feelings (or, as a recent report would have it, its pain), but referring to its activity as "thinking" -- heck, we've done that since before I joined the CS community. How many programs printed out "thinking..." while they were computing? The verb "to think" has such a broad set of vernacular meanings that I don't worry about it much.

I have recently been on a low-intensity quest to learn about the current theories of cognition, consciousness, theory of mind, and neuroscience. I am just batting about, no fixed curriculum yet; I have made a little bit of an effort to identify mainstream, up-to-date understanding, but so fair have failed in that. What I really need to do is find a guide on the topic...I have a couple of colleagues who do some aspect of neuroscience, but far from this particular topic. But they might have some pointers. I should ask them.

Decades ago, I saw John Searle talk in person (at Xerox PARC) on The Chinese Room. All the way back then I thought it was bunk. I also thought Roger Penrose's "we don't understand it, therefore, quantum!" approach was unscientific.

(If there is a regular series of talks on this level of major thinkers somewhere near me, I have missed it. I should try harder to find one. Keio has occasional big-name ones at our Mita Campus, but they are usually invited by and paid for by people over there, and students get priority over faculty in attending in medium-sized auditoriums, which is not necessarily a bad thing.)

More recently, I've been reading about things like Global Workspace Theory (GWT); the summaries on places like Wikipedia don't go beyond the metaphor level, so if there is an abstraction, logical or mathematical, that can be used to explain or predict how a large pile of neurons with various chemical and physical stimuli can produce ME, I haven't gotten there yet.

(I have also spent an unfortunate amount of time, entirely out of curiosity rather than any expectation that it tells me anything useful, trying to understand the current revival of panpsychism. Besides being seriously unscientific woo, proponents appear to outright punt on the hard problem of why a recently-dead person, whose atoms are arranged exactly as they were before, no longer has consciousness, or how sleep and anesthesia work; they just say, "It's a product of complex interactions within the system." ....Thanks. I think I knew that. I definitely don't need conscious rocks and electrons to reach that particular conclusion.

I am certainly a physicalist (I think the term is), not a Cartesian dualist; I believe consciousness arises from strictly physical and chemical processes within our own bodies, primarily within the organ we identify as the brain. And as a consequence, since digital computers can ultimately simulate or model any physical system with high fidelity, I see no reason to believe that artificial general intelligence and artificial consciousness are fundamentally impossible. This makes me a computationalist (again, I hope I'm using the vocabulary properly).

It also seems quite likely that other cognitive architectures are possible, and also that something like our own architecture could be reimplemented on a different substrate by teasing out the thinking-important aspects of the neuro-electrical-chemical activities and modeling them as more abstract computation or equations.

But do machines "think" in the sense that we do? Certainly not, it's a different mechanism. Is it still "thinking", in a tight sense of the word? Well, what does the word mean to you? Does it require consciousness, an awareness of self with shifting focus and the ability for spontaneous ides or topics to appear? Does it require creativity?

It seems to me that we most likely haven't gotten there yet with our existing systems, but it's definitely getting harder to tell. My current opinion is that we will have a VERY hard time actually figuring out (and then, as a whole, accepting) when they do cross that threshold.

Any interest in this topic? If so, drop me a note in the comments.

https://en.wikipedia.org/wiki/Models_of_consciousness