ITN has warned of the threat posed by AI-generated deepfake online videos and other misleading content featuring some of its leading journalists.
The broadcaster has also warned that social platforms including Meta, which hosts many of the misleading videos on Facebook and Instagram, can be slow to act once alerted to such content which can be used to promote illegal scams.
The latest example seen by Press Gazette last week featured ITV News presenter Mary Nightingale. The AI-generated video, posted on Meta-owned platform Instagram, appeared to show Nightingale in her own voice telling viewers about a new app purporting to be backed by popstar Adele, also deepfaked in the video, and allegedly helping people win thousands of pounds.
The bulletin ended with the faked Nightingale saying: “If you want to experience even more vivid emotions, and get a chance to win the jackpot, go to the app shop right now. The app is still available but may soon be closed to new players.”
Tami Hoffman, ITN‘s director of news distribution and commercial innovation, told Press Gazette this was the “most sophisticated one” the ITV News, Channel 4 News and 5 News producer has seen so far.
It is not seamless, especially if you know what to look for, Hoffman said. “But if you’re not looking out for it, and if you’re just viewing it on a phone or a small screen as a casual viewer, I think it’s not a bad attempt.”
“From ITN’s perspective, one of the first risks that we identified around AI was to our reputation – that obviously the easier it is to make deepfakes, that puts our reputation at risk, because obviously our journalists would never promote a commercial good like that,” Hoffman said.
“But it’s very easy to make something which looks pretty realistic and I don’t think the platforms are hugely helpful in terms of monitoring this stuff and we’re still not really in a position where the public can easily check whether something is AI-generated or not, so it leaves us really vulnerable.
“It’s bad for our reputation, it’s bad for Mary’s reputation, and it’s really bad for the industry as a whole and just public trust in news.”
Last month a fake BBC article shared on Facebook purported to report on an on-air conversation between ITV News colleagues political editor Robert Peston and UK editor Paul Brand in which Peston endorsed a get-rich-quick investment scheme and was subsequently sued by the Bank of England.
Meanwhile, last year Money Saving Expert Martin Lewis, who hosts The Martin Lewis Money Show for ITV, warned people against falling for a deepfake video share on X/Twitter appearing to show him endorsing a fake investment project by Elon Musk.
“That was exactly the same thing,” Hoffman said, comparing it to Nightingale’s video. “Somebody who trades on his impartiality suddenly being seen to promote a commercial good. He was furious about it, but it took him quite a long time to get that removed.”
BBC News and Sky News have both also seen deepfakes of their presenters shared online. More simple fake videos are also created purporting to show news content with a broadcaster’s logo superimposed.
Instagram, X (formerly Twitter), Youtube and Tiktok are all increasingly seeing these videos appear, Hoffman said, and they can jump from platform to platform.
AI deepfake threat in election year
Aside from commercial “endorsements”, Hoffman said deepfakes like these are a huge worry in a year in which the UK is expected to hold its next general election – with the next US president also being elected in November.
She said: “I think that in an election year, we’re just going to see a lot more of it. I mean, this [latest] one isn’t politically motivated. This is obviously using Mary for commercial purposes. But you can see that people could be using it for whatever it is that they’re trying to flog – whether that’s political views or a commercial product.”
Last month a fake robocall targeted voters in New Hampshire. It impersonated President Joe Biden, who was seemingly heard to be telling them they did not need to vote in the Democratic primary election.
In October a fake BBC News video appeared to show a report that open-source intelligence news outlet Bellingcat had proved Ukraine was smuggling weapons to Hamas. Bellingcat founder Eliot Higgins confirmed this was “100% fake” – but even Hoffman conceded it “looks really convincing”.
Hoffman said that “even in the last six to 12 months you’ve seen these deepfakes are starting to improve and I really worry that some of the glitches which at the moment do signal that it might be AI, that actually that’s going to start to get smoothed out over time”.
ITN: Whole organisation must ‘keep eyes and ears open’
ITN is working to educate staff on how to spot deepfakes “because obviously one of our concerns is not just about our reporters being deepfaked, but about our staff or our journalists being taken in by deepfakes,” Hoffman said.
The company is about to roll out a compulsory AI learning module teaching staff the signs that can show a video has been generated by AI: for example, weird hands or the wrong numbers of fingers, strange skin tones, and something odd in the background.
ITN also has an internal email address where people can flag fakes they see, so the information can be shared across all three of its newsrooms as quickly as possible.
“I think that what we’ve realised with AI both in terms of the opportunities and the challenges is this has to be a bottom-up thing,” Hoffman said. “We’re asking staff across the whole organisation to take responsibility for skilling themselves up, and for keeping their eyes and ears open.”
‘If trust is damaged, that hits all of us’
Hoffman added that there is “really quite collegiate behaviour” between news organisations who are increasingly sharing examples and tips for dealing with the threat.
“There’s actually quite, I think, a good community growing up between the news organisations in the UK about appreciating that actually, when it comes to AI, we probably have more in common with each other than divides us,” she said.
She continued: “If trust in a reputable news organisation is damaged, that hits all of us. It might be a video about the BBC but it impacts ITN as well because it’s polluting the information ecosystem.”
Despite the social platforms having simple reporting processes, “the problem is that once it’s been reported, it’s then got to be reviewed and that can take time,” Hoffman said. “And of course, you don’t really have much time because the damage is done once it’s spread.
“We would love to have the platforms creating kind of rapid response teams. We think there’s a lot more that they could be doing with news organisations to help us…”
Hoffman added that Youtube has been the most responsive and easiest to deal with in relation to these types of videos so far.
Ultimately, Hoffman said: “We are really concerned about this in an election year. We’re doing our best to improve our internal processes, both in terms of flagging stuff, and also in training staff up, but that doesn’t help us when it comes to reputational management.
“We would love the big tech companies and the social platforms to engage more with news organisations about how they can monitor fakes, how they can identify fakes, and then obviously when something comes out how they can get it taken down as quickly as possible.”
Email pged@pressgazette.co.uk to point out mistakes, provide story tips or send in a letter for publication on our "Letters Page" blog
