Should AI systems behave like people?

Note to readers: we changed our name to the AI Security Institute on 14 February 2025. Read more here.

On May 13 th , 2024, OpenAI released GPT-4o (or “omni”) – the latest addition to their fleet of Frontier AI models. According to the blog post that accompanied its release 1 , GPT-4o achieves state-of-the-art performance on tests of language understanding, logic and maths. But a particularly striking feature, as revealed in the live video demonstrations that accompanied the release, was the upgrade to Voice Mode – allowing GPT-4o to generate exceptionally naturalistic patterns of speech during spoken conversation with the user. Based on the demos, it can speak with highly realistic intonation, stress, and rhythm, and exhibit stops and pauses that are characteristic of human speech.

OpenAI's release represents the latest step towards AI systems that behave in more human-like ways (“anthropomorphic” AI). In GPT-4o, this is evident not just from natural flow of conversation, but from how the model seemed willing to laugh, tease and even flirt – straying into territory in which software simulates an emotional connection to the human user (for example, in one demo, female-voiced GPT-4o tells the male user: “you’re making me blush”).

Humanlike AI could make it easier and more fun for users to engage with AI, broadening access to these tools. For example, people use AI for educational purposes might get better outcomes from systems which can engage them in a humanlike way. Yet, the availability of humanlike or anthropomorphic AI could also pose risks to the safety of the user 2 . Human-realistic AI systems could be used to impersonate people for fraudulent or deceptive purposes, especially when combined with voice cloning techniques 3 . Moreover, because humans are prone to believe that they have formed a personal connection with artificial systems capable of producing natural language (a phenomenon dubbed the Eliza effect 4 ) they may be vulnerable to deliberate political or commercial manipulation and exploitation 5 .

However, even without overt misuse, humanlike AI raises tricky ethical questions 6 . Is it acceptable or not for an AI to talk like a human? Should AI systems be allowed to, or prevented from displaying conversational motifs characteristic of human exchanges among friends or intimates? With millions of users already subscribing to services in which an AI behaves like a ‘companion’, is it acceptable or not for AI systems to be developed in a way that encourages humans to engage in simulated ‘relationships’ with AI systems?

The legitimacy of anthropomorphic machines is frequently debated in philosophy 7 , and is a popular discussion topic on social media platforms. However, despite previous work measuring public attitudes to AI 1 , to our knowledge, no previous survey has examined public views on humanlike AI directly. We sought the public’s view on this topic to foster a maximally inclusive debate about this issue, and to help ensure that what counts as “safe” AI behaviour isn’t decided by researchers or policymakers alone. The study was thus designed to gauge the UK public's view on humanlike AI behaviours, and particularly those that could theoretically be considered harmful or undesirable. We hope that better understanding public attitudes to (and awareness of) these AI model behaviours, will help start a conversation - and we will continue work with model developers and the wider AI community on tools and mitigations which minimise potential harm to the public from AI.

In March 2024, the UK AI Safety Institute , working with polling company Deltapoll , asked a roughly demographically representative sample of 1583 adult UK residents to complete a survey that measured attitudes to humanlike behaviour of currently available chatbots, such as ChatGPT, Gemini or Claude (we focus on text-based chatbots because very few users have experience of interacting with AI systems in voice mode, raising the possibility that different results may be obtained when speech models are widely available). In addition to items that measured demographic variables and familiarity with current AI systems, the survey items were divided into 5 categories, which were designed to measure: