View all newsletters
Sign up for our free email newsletters

Fighting for quality news media in the digital age.

Study: AI news videos benefit from a human touch

Researchers compared viewer reactions to automated clips with human-created or human-edited videos.

By William Turvill

A new academic study has found that news consumers prefer short-form videos that are produced or edited by journalists to those generated exclusively by artificial intelligence.

For the research, 4,200 people were shown clips from 14 news events involving the likes of Cristiano Ronaldo, Donald Trump and Justin Bieber. For each news story, one video was highly automated, one was partially automated with input from human editors, and the third was fully human-made. Each of the 42 individual videos was watched and rated by 100 people.

Comparing viewer opinions on each clip, the study concluded that “human-made videos have the edge over those created with automation”.

However, the difference between partially-automated videos and human-created clips was not significant in terms of how much they were liked overall, suggesting that “a modest use of automation, with some post-automation human editing, can be well received”.

The researchers found that human videos tended to rate more favourably in terms of “telling respondents things they didn’t know”, “professionalism”, “factfilledness,” being “engaging”, “comprehensiveness”, “story flow” and “understatedness”.

Select and enter your email address Weekly insight into the big strategic issues affecting the future of the news industry. Essential reading for media leaders every Thursday. Your morning brew of news about the world of news from Press Gazette and elsewhere in the media. Sent at around 10am UK time. Our weekly dose of strategic insight about the future of news media aimed at US readers. A fortnightly update from the front-line of news and advertising. Aimed at marketers and those involved in the advertising industry.
  • Business owner/co-owner
  • CEO
  • COO
  • CFO
  • CTO
  • Chairperson
  • Non-Exec Director
  • Other C-Suite
  • Managing Director
  • President/Partner
  • Senior Executive/SVP or Corporate VP or equivalent
  • Director or equivalent
  • Group or Senior Manager
  • Head of Department/Function
  • Manager
  • Non-manager
  • Retired
  • Other
Visit our privacy Policy for more information about our services, how Progressive Media Investments may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications.
Thank you

Thanks for subscribing.

They were “on average rated as more emotive and lighter in tone” and were judged to include less “sensationalism”. Respondents also suggested that human-created videos made better use of audio, music and images.

Content from our partners
Unified solution offers publishers unrivalled print and digital efficiency
How DPG Media invested in print technology to help it focus on digital
How to make sure you are social media ready for job hunting

[Read more: Survey finds half of newsrooms already ‘actively working’ with generative AI]

Dr Neil Thurman of Ludwig-Maximilians-University Munich – who conducted the research along with Dr Sally Stares at City, University of London, and Dr Michael Koliska of Georgetown University – said of highly-automated videos: “They tended to be a little bit repetitive with the images that they used. They didn’t do quite as well matching images to captions and that sort of thing.” He suggested that perceptions of “sensationalism” in the AI videos may have related to the music that accompanied clips.

One of the automated videos, documenting the career of Portuguese footballer Cristiano Ronaldo in 60 seconds, was set to rock music. In an apparent error, it included one still image of Fernando Torres, a different footballer, and another showing England fans in a pub.

The researchers found that human-made videos did not stand out in terms of “story structure”, the “number of relevant facts presented”, “bias/fairness”, “subjectivity/objectivity”, “relatability”, “informativeness”, “clarity” and “accuracy”.

‘You’ve got to keep humans in the loop’

The study used human-made videos from PA Media and compared them with automated and semi-automated offerings made by Wibbitz, which counts Reuters, Conde Nast, USA Today, TMZ and NBC among its clients.

The highly-automated videos for the study were created using a script of text prepared for each news event. The automation tools sourced media including video clips and still images, added background music, and used a variety of pre-built “themes” that set the tempo and appearance of the videos. The semi-automated videos included edits from journalists, who could alter images, text and style.

[Read more: AI and journalism – Six things news publishers need to know]

“Our research shows that, on average, news consumers liked short-form, automated news videos as much as manually made ones, as long as the automation process involved human supervision,” said Thurman.

Koliska said that “one key takeaway of the study is that video automation output may be best when it comes in a hybrid form, meaning a human-machine collaboration. Such hybridity involves more human supervision, ensuring that automated video production maintains quality standards while taking advantage of computers’ strengths, such as speed and scale.”

Thurman told Press Gazette that he believed news publishers now recognise that “you’ve got to keep humans in the loop” when deploying automation.

“I think maybe a few years ago they were thinking more along the lines that AI would do more,” he said. “Now I think they’ve come to realise you need that human supervision and it’s just as fruitful for them to work on technologies that empower perhaps journalists who don’t have the training in some of the traditional video editing tools but still enable them to produce something that’s of professional quality.”

Read the full study here.

Topics in this article :

Email pged@pressgazette.co.uk to point out mistakes, provide story tips or send in a letter for publication on our "Letters Page" blog

Select and enter your email address Weekly insight into the big strategic issues affecting the future of the news industry. Essential reading for media leaders every Thursday. Your morning brew of news about the world of news from Press Gazette and elsewhere in the media. Sent at around 10am UK time. Our weekly dose of strategic insight about the future of news media aimed at US readers. A fortnightly update from the front-line of news and advertising. Aimed at marketers and those involved in the advertising industry.
  • Business owner/co-owner
  • CEO
  • COO
  • CFO
  • CTO
  • Chairperson
  • Non-Exec Director
  • Other C-Suite
  • Managing Director
  • President/Partner
  • Senior Executive/SVP or Corporate VP or equivalent
  • Director or equivalent
  • Group or Senior Manager
  • Head of Department/Function
  • Manager
  • Non-manager
  • Retired
  • Other
Visit our privacy Policy for more information about our services, how Progressive Media Investments may use, process and share your personal data, including information on your rights in respect of your personal data and how you can unsubscribe from future marketing communications.
Thank you

Thanks for subscribing.

Websites in our network