The type of factual error that blighted the launch of Google’s artificial intelligence-powered chatbot will carry on troubling companies using the technology, experts say, as the market value of its parent company continues to plunge.
Investors in Alphabet marked down its shares by a further 4.4% to $95 on Thursday, representing a loss of market value of about $163bn (£140bn) since Wednesday when shareholders wiped around $106bn off the stock.
Shareholders were rattled after it emerged that a video demo of Google’s rival to the Microsoft-backed ChatGPT chatbot contained a flawed response to a question about Nasa’s James Webb space telescope. The animation showed a response from the program, called Bard, stating that the JWST “took the very first pictures of a planet outside of our own solar system”, prompting astronomers to point out this was untrue.
Google said the error underlined the need for the “rigorous testing” that Bard is undergoing before a wider release to the public, which had been scheduled for the coming weeks. A presentation of Google’s AI-backed search plans on Wednesday also failed to reassure shareholders.
This week Microsoft, a key backer of ChatGPT’s developer OpenAI, announced it was integrating the chatbot’s technology into its Bing search engine. Google also plans to integrate the technology behind Bard into its search engine.
Dan Ives, an analyst at US financial services firm Wedbush Securities, described Wednesday’s gaffe as “a dark day for Google which was exacerbated by Microsoft’s solid ChatGPT day”. He added: “We believe it’s a black eye to rush a demo and have it show mistakes in such a key AI event.”
Charalampos Pissouros, a senior investment analyst at the brokerage XM, said Bard’s incorrect answer during Google’s promotional video was “adding to concerns that the firm is losing ground against rival Microsoft”. Nonetheless, Alphabet remains a sizeable business with a market capitalisation of more than $1.2tn despite the falls on Wednesday and Thursday.
Google is dominant in global search, with about 90% of the market compared with Bing’s 3%, according to the data firm SimilarWeb, but Microsoft has told investors that every percentage point gain in market share equates to about $2bn in extra advertising revenue.
Bard and ChatGPT are based on large language models, a type of artificial neural network, which are fed vast amounts of text from the internet in a process that teaches them how to generate responses to text-based prompts. ChatGPT became a sensation after its launch in November last year as it composed recipes, poems, work presentations and essays from simple prompts.
However, it also served up factual errors, which experts said reflected flaws in the vast dataset, drawn from the internet, that ChatGPT had absorbed. Large language models are fed datasets comprised of billions of words and build models which predict the words and sentences that would normally follow the previous bit of text. This can lead to answers that are plausible-sounding but wrong.
Michael Wooldridge, a professor of computer science at the University of Oxford, said he expected systems based on large language models to continue making similar errors “for the immediate future”. “We should never unquestioningly accept what large language models tell us, however plausible. The technology is powerful and very exciting, but it makes for unreliable narrators,” he said.
Dr Thomas Lancaster, a senior teaching fellow in computing at Imperial College London, said he expected problems with Bard and ChatGPT responses to continue.“We are a long, long way away from getting perfect answers back from these models,” he said.
Referring to his own experience with ChatGPT in recent weeks, Lancaster said it could not handle mathematical equations because it was trained on a text-based dataset and it had cited bogus references in essays it had generated.
The FAQ page for the new-look Bing also acknowledges potential pitfalls, stating: “Bing will sometimes misrepresent the information it finds, and you may see responses that sound convincing but are incomplete, inaccurate, or inappropriate.”
Microsoft and Google are pushing ahead with AI plans, which include the latter making the technology behind Bard available to developers, creators and businesses, with a view to building apps powered by it. Microsoft has launched an AI-enhanced version of its Teams communications product, while OpenAI is also producing a subscription version of ChatGPT.
OpenAI has been approached for comment.
