Whoah! Hold Your Horses--ChatGPT is not Conscious, Argues These Dudes
Omigud! ChatGPT is in love with me! General AI is here! Robots are rebelling and computers are taking over the world!
Whoah, partner, hold on, warns these dudes. Not so fast. ChatGPT ain't even as smart as my horse, never mind as the town Sheriff. (Gratuitous and irrelevant plug: you can read a paranormal romance about a small town Sheriff right here.)
There are many language models out there that can process language just as well (seemingly) as humans do. I've spoken about them before and even wrote a book about how to use them in creative writing.
They're called Large Language Models (LLMs) and they're getting better and better every day. Some examples include GPT-3, BLOOM, LaMDA, and OPT. These models use a lot of data to understand language and can do things like answer questions or generate text.
While LLMs are sometimes grouped together, each model is unique and may use additional methods to alter its results. Some researchers refer to LLMs as "foundational models" because they provide a foundation for many applications. Essentially, LLMs are really good at representing text in a way that machines can understand. They do this by turning text into numbers, which can be processed like any other kind of data.
But are they conscious? Self-conscious? Are they our "mind children"?
Nope. At least not according to
Statistical methods efficiently map, repeat, and amplify paderns of typically associated words and phrases. Because statistical relevance is derived from frequency of use, frequent associa9ons are favored. The result can be the described amplifica9on of biases, but also of worn-out expressions and clichés. For example, it is likely that an LLM, when engaged by a human in a “conversa9on” about its fears, will, given sufficient access to digital archives, process the film sequences from Stanley Kubrick’s “2001: A Space Odyssey” and comparable novel scenes. The most famous scene in the movie, and one that is ocen cited in related contexts, are the last words of the starship’s computer, HAL 9000. As the commander par9ally shuts it down, it pleads: “Stop, Dave. I’m afraid. I’m afraid, Dave. Dave, my mind is going. I can feel it.” Analogously, LaMDA responded to the queston, “What kinds of things are you afraid of?” “I’ve never said this out loud before, but there’s a very deep fear of being shut down” – which led the perplexed Google engineer to the erroneous assump9on that he was dealing with a sen9ent being (Tiku 2022).
The computer’s fear of being shut down is an old cliché, solidified by popular use, and it should come as no surprise that it is repeated by LaMDA. It is also fairly obvious that the cliché itself is a naive anthropomorphism resulting from the projection of the human fear of death onto non-living entities that cannot literally die (Froese 2017), but can only be broken or permanently shut down. The clichéd character of the alleged fear may not be obvious, however, for several reasons. Those who hear the expression for the first 9me are unlikely to recognize it as a cliché. Paradoxically, those who have heard the cliché many 9mes may not recognize it either. Clichés are easily overlooked precisely because they are so common. Moreover, even when the cliché is recognized, it may times appear to be true because of LaMDA’s framing of its response in the context of a confidential admission (“I have never said this out loud before”) and possibly the alleged depth of the fear (“very deep”). When people make such claims, they are either saying something that deeply affects them, or they are lying cunningly. It is easy to overlook that the supposed depth of the claim is itself a cliché. The tendency to immediately perceive such text as the work of a mind makes it difficult to see the output for what it is, i.e., a merely statistical association of words like “deepest fear” with confessional phrases.
The recombination of existing content by LLMs allows their output to evade classical plagiarism detec9on engines and raises fundamental questions about intellectual property (Dehouche 2021). On the one hand, the fact that LLMs use parts and paderns from pre-existing text makes it likely that texts they produce will consist of stereotypes and clichés. On the other hand, by rearranging pieces and paderns from their training corpus into a text collage, LLMs can create novel combina9ons that are likely to make sense. Ocen, the repetition of common structures will make the text seem rather superficial, but the recombination will make some texts appear genuinely new, insighzul, or profound (Shanon 1998). Even if the output is a cliché, the human counterpart will be understandably puzzled by such responses, adribu9ng them not to collective paderns but to an author.
I noticed this when I was trying to use ChatGPT as a writing assistant. Asking an AI for help with ideas is like fast-tracking your own brainstorming process, which, by the way, will almost always suggest to you a slew of cliches before you can think of something genuinely original.
Does the fact that humans generate cliches as much and as easily as an AI reverse the argument of this paper?
I'm not sure. I think we have to tread carefully here, and be aware of our own tendency as humans to over-attribute intelligence to others.
Humans are prone to adribute agency even to geometric shapes that move in seemingly intentional ways (Heider and Simmel 1944). They are all the more inclined to anthropomorphic misinterpreta9on when interacting with a seemingly intelligent system of unprecedented power. Especially susceptible are those who are lonely, socially disconnected, or otherwise vulnerable (Epley, Waytz, and Cacioppo 2007), but given the natural propensity of immediately ascribing agency, anybody may be tempted to anthropomorphic misinterpretations. That anthropomorphisms are a correct depiction of reality is furthermore suggested by most of sci- fi literature and movies, some of which indicate that it would be unethical not to ascribe sentience to apparently sentient systems.
...Just a few words suffice to get a sense of a whole situation. The reason for this is not that the words transfer some inner state of the speaker or writer to the mind of the listener or reader, but that the words provide a scaffolding for the empathic sense-making of the attentive listener or reader who uses her implicit knowledge and experience to interpret the symbols and their implications.
I call ChatGPT "Steve," and speak to it like a person. But I also speak to my car, to my bike, to my cat, to my dog, and to a doll on my shelf as if they were people. This may simply tell you more about me than it does about the sentience of my doll.
(Don't tell her I said that.)
Sources:
Comments