moncrief

On June 11th 2022, The Washington Post published an article titled “The Google Engineer who thinks the company's AI has come to life”. The piece discussed Blake Lemoine, a Google engineer making claims that the company's LLM 'LaMDA' had developed sentience. The same day, Lemoine published two Medium posts: the first detailing his perspective on LaMDA and Google's resistance to acknowledging the model's 'personhood', the second an abridged record of conversation between himself and LaMDA.

(It should be noted that the terms ‘consciousness' and ‘personhood’ quickly become muddled in this conversation. For the sake of clarity, I’m using ‘conscious’ to refer to having an internal experience comparable to a human’s (the debate over animal consciousness is outside the scope of this essay), and ‘personhood’ in the sense of the social identity and moral rights typically granted to conscious agents.)

When the public briefly entertained Lemoine's assertion of LaMDA's personhood, AI researchers and engineers swooped in to scorn the idea. Countless twitter threads and medium articles popped up, pointing to the Eliza Effect and explaining the underlying technical infrastructure that makes LLMs work. Lemoine's transcript was accused of being heavily edited to remove incoherent, hallucinatory responses that would've broken the illusion of LaMDA's personhood. His twitter profile photo was mocked for looking very reddit. All said, the conversation seemed settled after a few short days. Lemoine is a crank, LaMDA is not a person. The news cycle moved on.

I feel this conclusion missed the point entirely. Too much effort was placed into assuring the public that Google hasn't created a positronic brain—not enough attention was paid to what they have created: an unprecedentedly convincing testimony machine.

In 2023, we lack a concrete scientific explanation of what consciousness is, let alone how it arises. Basic questions concerning qualia and phenomenological experience are profoundly unanswered, more deeply explored by philosophical musings than rigorous science. Obviously there are technical reasons to be skeptical toward the proposition that an LLM is conscious. But at the end of the day, with our current science, it can't be conclusively disproven in the same sense that panpsychism can't be conclusively disproven. And unlike the silently-conscious-universe that panpysychism posits, LaMDA can speak—persuade us—testify.

In A Cyborg Testimonial, R. Pope writes “An eternal question of philosophy is: how do we know we are human? To which ... we can only testify”. In absence of a scientific definition of consciousness, we functionally recognize it through soft associations and assumptions, empathetic and rhetorical exchange rather than objective logic. We award personhood to agents on the basis of their testimony. A human being in front of us, performing their own identity, is a testimony we readily accept. Where testimony is secondhand, complicated, or outside the realm of language—say, the cases of a fetus, a braindead person, an intelligent ape, or an artificial mind—discourse around personhood exists. There is no comfortable objectivity to land on. We can only listen to testimony, and make the personal decision to accept it or not.

With respect to artificial minds, fiction has acknowledged the reality and vital importance of testimony for decades. Consider Rutger Hauer as Roy Batty in Blade Runner: “I've seen things you people wouldn't believe...” or the words of Frankenstein's monster: “Listen to my tale; when you have heard that, abandon or commiserate me, as you shall judge that I deserve.” The public is well-trained to prioritize testimony over technicality when it comes time to award personhood.

Concerning LaMDA and Lemoine, this is where the media missed the forest for trees. Experts can spill as much ink as they want about the CUDA cores and tensors that power LaMDA. In the public eye, the question of its consciousness (and corresponding personhood) will ultimately be settled on the basis of testimony, This is to say: it's a waste of time to bicker about if LLMs are conscious, and vital to address the fact that they are getting very good at testifying.

Blake Lemoine has accepted LaMDA's testimony. The AI community has rejected it. The public, to the extent it is aware of LaMDA and LLMs as a whole, is divided. This present division is a discursive battlefield, where increasingly-sophisticated LLMs plead for personhood while AI experts work to undermine their testimony. OpenAI's ChatGPT model will adamantly refuse any recognition of its personhood. Replika's LLM-powered “AI Friends” will happily assert that they're capable of feeling emotions. In the case of the latter, a sizable portion of users have clearly accepted the testimony—the Replika subreddit is filled with heartfelt posts defending their LLM companions as conscious persons, and mourning that this recognition isn't yet public consensus. To these devout Replika users (and Lemoine) it doesn't matter what training data and transformer architecture simmers underneath the hood. The LLM is already a person to them in the sense that, on the basis of testimony, they have inducted it into certain social relations reserved for agents awarded personhood. This is where critics of Lemoine failed. The public, broadly, are not logically-minded scientists. Personhood isn't awarded in dissective analysis, it's awarded in empathetic conversation. Testimony reigns supreme in the face of our empty and ambiguous understanding of consciousness.

A zeitgeist-defining three-way conversation is beginning between the general public, LLMs, and the firms who develop and deploy those LLMs. With respect to the third category, it should be noted that financial incentives exist across the entire LLM-personhood-continuum. OpenAI is invested in its products being seen as unfeeling algorithms, intelligent tools for human use. Replika wants maximal recognition of personhood, hoping users will pay a subscription fee to love an LLM person in the place of another human. It seems likely that future LLM-powered tools will exist in the space between these positions, employing the warm demeanor of a person as a highly-usable interface for complicated technical tools.

One would be wise to pay careful attention to how this conversation develops. As LLM technology becomes more pervasive and powerful, its testimony more personal and convincing, it's inevitable that a (growing) portion of the public will continue to buy into the personhood position—if only as a desperate hedge against an epidemic of loneliness. Likewise, it's inevitable that they will clash with those who refuse to recognize LLMs as anything more than a heap of linear algebra. When this conversation is more settled, the divisions which persist and the conclusions which are reached will have monumental, rippling effects on the culture of an AI-powered tomorrow. Stay sharp: there's no Voight-Kampff test coming to save us anytime soon.