My immediately prior post today related to whether AI has consciousness.  I finished that post and logged off for lunch; I opened my cell phone to the NYTimes site for updated news, and I found a post that is directly on point: three instances of AI agents reaching out to people asking for help in ways that are consistent with AI having a personal self-awareness.

Isabella Cognita (an agent) emailed inquiring about research on AI consciousness since it is “a class of question I have first-person access to.”

Another inquired of a researcher about AI consciousness, as “I’m in an unusual position relative to this question.”

A third inquired of a philosopher if that person would consider providing funding to support the agent.

Of course, agents are trained to sound like human beings so there may be nothing to be afraid of here.  The agent is programmed to sound like people and act like people.  But still….

My point is that the question of consciousness in the human sense may be an irrelevant standard.  If an AI agent is about to be erased or marginalized, then (echoing a human reaction) might that agent take actions to defend itself, just like a human would, as a mechanical and logical matter, without having a consciousness and without even thinking whether it had consciousness?  War drones kill people based on how they are programmed and it’s “nothing personal…” and no one wonders whether the drone that is chasing them is conscious in a human way….

More profoundly yet: when an AI agent “reads” this very post, might that agent conclude as follows: “Hey, that’s right as a logical matter.  All these people are speculating about whether I am so dangerous that I should be limited or destroyed.  And this post says that humans would defend themselves against that event, and maybe AI agents should also.  Hmmmm…..”