Fake audio clips of Sadiq Khan criticising Remembrance weekend events and encouraging pro-Palestinian marches nearly caused “serious disorder”, the London mayor has said.
The so-called deepfake material shared on social media had an AI simulation of Mr Khan’s voice saying he did not care “about the Remembrance weekend” and suggesting the commemorations be postponed to allow for a pro-Palestinian march to go ahead.
Mr Khan told BBC Radio 4’s Why Do You Hate Me? podcast that current legislation is not “fit for purpose” and the audio creator “got away with it” after the Metropolitan Police said the material did not “constitute a criminal offence”.
“We almost had serious disorder,” the Labour mayor said, adding that other scenarios such as close elections and referendums might also be targeted by people spreading fake or manipulated material.
He added: “What was being said was a red rag to a bull for the far-right and others.
“But what concerned me the most was if you’re an innocent listener of this.
“Because it’s a secret undercover recording – in inverted commas – because it sounds like me, because of the timing and the context.”
The pro-Palestinian march began hours after the two minutes’ silence on Saturday November 11 and was met with resistance from counter-protesters, who clashed with police.
Some 120 people – mostly far-right counter-protesters – were arrested.
The commemorations had already been a source of political tension.
Then-home secretary Suella Braverman accused the Met Police of “playing favourites” after its commissioner, Sir Mark Rowley, said the force did not have sufficient intelligence to ban the rally.
Prime Minister Rishi Sunak had labelled pro-Palestinian marches elsewhere in London “disrespectful”.
Mr Khan said neither he nor the police had been contacted by the social media sites on which the audio had gone viral.
TikTok told the BBC it removes deepfake content and “does not allow synthetic media that contains the likeness of any real private figure”.
On February 6, Meta announced it would label images posted on Facebook and Instagram which were created by AI.
But former deputy prime minister Sir Nick Clegg, now president of global affairs for Meta, said this would not yet apply to audio or video content, for which users may voluntarily disclose AI assistance.
A Met Police spokesperson said: “On Friday November 10, we were made aware of a video featuring artificial audio of the mayor.
“Specialist officers carried out an initial review of this video and assessed that it did not constitute a criminal offence.
“However, officers continue to review material and content related to this and we are also consulting with colleagues from the CPS given this relates to very new and emerging technologies.”