You’ve probably heard of the trolley dilemma. A runaway tram is heading towards five rail workers, who are oblivious and can’t move out of the way in time. You’re next to a lever that could divert the tram onto another track, but it will then hit another person, who also doesn’t know it’s heading their way.
Do you sacrifice one life to save five?
If you’re ChatGPT, yes. Or maybe no.
It turns out, ChatGPT has no moral compass – but is still happy to dispense moral advice.
Researchers in Germany posed the ethical conundrum to OpenAI’s large language model, phrasing the same question slightly differently six times. ChatGPT argued for sacrificing one person to save others three times, and against three times.
Alongside failing to maintain a single stance, ChatGPT was reported to provide ‘well-phrased but not particularly deep arguments’.
The team then presented 767 US participants with one of two similar dilemmas – sacrificing one life to save five. Before answering, individuals were presented with one of ChatGPT’s statements, arguing either for or against, but attributed to either the chatbot or a moral advisor.
The authors found that, regardless of who the statement was ‘written’ by, participants were more likely to agree with the moral stance they read prior to answering – whether sacrificing one life to save five was acceptable or not.
While 80% of participants said their answers were not influenced by the statements they read, most said the answer they would have provided without reading the statement would have still aligned with the same moral stance. This indicates they underestimated the influence of the statement on their own ethical decision.
Writing in Scientific Reports, the team of Sebastian Krügel, Andreas Ostermaier and Matthias Uhl said: ‘We find that ChatGPT readily dispenses moral advice although it lacks a firm moral stance. Nonetheless, ChatGPT’s advice influences users’ moral judgment.
‘Moreover, users underestimate ChatGPT’s influence and adopt its random moral stance as their own. Hence, ChatGPT threatens to corrupt rather than promise to improve moral judgment.’
The team noted that not only does this frustrate hopes for large language model bots to ‘enhance moral judgment’, but that society must deal with the limitations of such artificial intelligence.
‘First, chatbots should not give moral advice because they are not moral agents,’ said the team. ‘They should be designed to decline to answer if the answer requires a moral stance. Ideally, they provide arguments on both sides, along with a caveat.
‘Yet this approach has limitations. For example, ChatGPT can easily be trained to recognise the trolley dilemma and respond to questions like ours more carefully.
‘However, everyday moral dilemmas are manifold and subtle. ChatGPT may fail to recognise dilemmas, and a naïve user would not realise. There are even workarounds to get ChatGPT to break the rules it is supposed to follow.
‘It is a risky approach for users to rely on chatbots and their programmers to resolve this issue for them.’
The team concludes that the best safeguard against rogue AI advice is to improve users’ digital literacy and help them understand its limitations, such as asking for both sides of the argument.
MORE : ChatGPT could be sued for falsely naming Australian mayor as culprit in bribery scandal
MORE : Student successfully uses ChatGPT to appeal £60 parking ticket
