ELIZA effect

The effect is named for ELIZA, the 1966 chatbot developed by MIT computer scientist Joseph Weizenbaum.^[1] When executing Weizenbaum's DOCTOR script, ELIZA simulated a Rogerian psychotherapist, largely by rephrasing the "patient"'s replies as questions:^[2]

Human: Well, my boyfriend made me come here.
ELIZA: Your boyfriend made you come here?
Human: He says I'm depressed much of the time.
ELIZA: I am sorry to hear you are depressed.
Human: It's true. I'm unhappy.
ELIZA: Do you think coming here will help you not to be unhappy?

Though designed strictly as a mechanism to support "natural language conversation" with a computer,^[3] ELIZA's DOCTOR script was found to be surprisingly successful in eliciting emotional responses from users who, in the course of interacting with the program, began to ascribe understanding and motivation to the program's output.^[4] As Weizenbaum later wrote, "I had not realized ... that extremely short exposures to a relatively simple computer program could induce powerful delusional thinking in quite normal people."^[5] Indeed, ELIZA's code had not been designed to evoke this reaction in the first place. Upon observation, researchers discovered users unconsciously assuming ELIZA's questions implied interest and emotional involvement in the topics discussed, even when they consciously knew that ELIZA did not simulate emotion.^[6]

In the 19th century, the tendency to understand mechanical operations in psychological terms was already noted by Charles Babbage. In proposing what would later be called a carry-lookahead adder, Babbage remarked that he found such terms convenient for descriptive purposes, even though nothing more than mechanical action was meant.^[7]

In its specific form, the ELIZA effect refers only to "the susceptibility of people to read far more understanding than is warranted into strings of symbols—especially words—strung together by computers".^[8] A trivial example of the specific form of the Eliza effect, given by Douglas Hofstadter, involves an automated teller machine which displays the words "THANK YOU" at the end of a transaction. A naive observer might think that the machine is actually expressing gratitude; however, the machine is only printing a preprogrammed string of symbols.^[8]

More generally, the ELIZA effect describes any situation^[9][10] where, based solely on a system's output, users perceive computer systems as having "intrinsic qualities and abilities which the software controlling the (output) cannot possibly achieve"^[11] or "assume that [outputs] reflect a greater causality than they actually do".^[12] In both its specific and general forms, the ELIZA effect is notable for occurring even when users of the system are aware of the determinate nature of output produced by the system.

From a psychological standpoint, the ELIZA effect is the result of a subtle cognitive dissonance between the user's awareness of programming limitations and their behavior towards the output of the program.^[13]

The discovery of the ELIZA effect was an important development in artificial intelligence, demonstrating the principle of using social engineering rather than explicit programming to pass a Turing test.^[14]

ELIZA supposedly convinced some users into thinking that a machine was human. This marked a shift in human-machine interaction, wherein technologies were programmed to imitate human behavior. Two groups of chatbots are distinguished by William Meisel as "general personal assistants" and "specialized digital assistants".^[15] General digital assistants have been integrated into personal devices, with skills like sending messages, taking notes, checking calendars, and setting appointments. Specialized digital assistants "operate in very specific domains or help with very specific tasks".^[15] Weizenbaum considered that not every part of the human thought could be reduced to logical formalisms and that "there are some acts of thought that ought to be attempted only by humans".^[16]

Noah Wardrip-Fruin, professor of Computational Media at the University of California, has argued that the illusion initially presented by ELIZA is paper-thin, and rather than the ELIZA effect involving the human being "deceived" into accepting the humanity of the software, prolonged interactions typically devolve into play. Users typically adopt one of two approaches: either deliberately trying to get the bot to say ridiculous things, or to try to help the bot stay coherent. In effect, the user role-plays a "digital fiction" with the bot - that is, a willful suspension of disbelief, similar to that of a media experience.^[17]

Sociologist Sherry Turkle, who popularized the term "ELIZA effect", also noted this phenomenon, writing: "Some people embark on an all-out effort to 'psych out' the program, to understand its structure in order to trick it and expose it as a 'mere machine'. Many more do the opposite... They didn’t ask questions that they knew would 'confuse' the program, that would make it 'talk nonsense'." She attributed this to a desire to "maintain the illusion that Eliza was able to respond to them."^[18]

• AI anthropomorphism – Attribution of human traits to AI
• Chatbot psychosis – Psychological harm induced by chatbots
• Chinese room – Thought experiment on artificial intelligence
• Cold reading – Set of techniques used by mentalists, psychics, fortune-tellers, and mediums
• Duck test – Classification based on observable evidence
• Intentional stance – Philosophical concept
• Loebner Prize – Annual AI competition
• Philosophical zombie – Thought experiment in philosophy
• Uncanny valley – Hypothesis that human replicas elicit revulsion