Major publishers are losing visibility in Google search results, data shared with Press Gazette suggests.
In Google‘s top stories carousel – the box that appears within search results when Google detects a news-oriented query – publishers are increasingly competing with specialist and other sites.
Data from news and publishing SEO company Newzdash shows that 36 of 58 (62%) major publishers tracked by Press Gazette lost visibility in top stories between February 2022 and February 2023 in UK searches.
While visibility is not necessarily a measure of traffic, the top stories carousel is a major driver of search traffic to publishers accounting for some 80% to 90% of their search audience, according to SEO expert Barry Adams of Polemic Digital.
"We see diminished visibility in top stories, either by [news publishers] appearing lower in the top stories order or not appearing at all and then being relegated to Google’s news tab. When you click on the 'more stories' link, we're getting fewer opportunities to rank in top stories," says Adams.
Dan Smullen, head of SEO at Mediahuis Ireland but speaking in a personal capacity, says: "Since 2019, when Google opened the floodgates for any publisher to be eligible for Google News and top stories, original news reporting has struggled with copycat publishers regurgitating the news without attribution. Despite its PR statement of 'elevating original reporting in search', the top stories news algorithm is heavily weighted toward freshness; original news reports are not given the visibility they deserve."
Leonie Roderick, audience reach editor at The Times and Sunday Times, says the publisher no longer expects its articles to cut through immediately or be almost automatically given a spot in the top stories box.
"We’ve certainly seen that Google is becoming more selective in terms of what it’s showing. To give an example, just because a competitor is ranking with a live blog, we can’t expect ours to do the same," she says.
According to Smullen, Google is also taking more real estate from publishers with its own features gathering information at the top of search engine results pages (SERPs) when big news events such as the Oscars and World Cup take place.
A Google spokesperson told Press Gazette: "We have not downgraded News in our search results. In fact, each month, people click through from Google Search and Google News results to publishers' websites more than 24 billion times."
A 2022 update to top stories now shows a greater range of sources, which Google says results in less publisher repetition and a greater variety of websites.
Luke Budka, director at B2B agency Definition, says that while in his view it's not clear whether publishers are losing in SERPs, Google has lowered the barrier to entry for inclusion in its news tab and top stories. News outlets are no longer required, for example, to register in the Publisher Centre to appear in Google News or to use its fast-loading Accelerated Mobile Pages format to appear in top stories. "So it makes sense that competition for traditional publishers is heating up," he says.
Adams has termed the downgrading of news in search the "newspocalypse". He dates this to 2018 when a major search update by Google rapidly reduced the amount of space given to news publishers, who have struggled to feature on Google since.
"March 2018 was the first time Google rolled out an algorithm update that appeared to penalise news publishers, specifically tabloids. We saw The Sun, the Mail, the Express, the Star absolutely plummet in search traffic off the back of that. They lost half of their search traffic," says Adams.
While Adams conjectures that core updates - Google algorithm changes big enough to be noticed by web publishers - probably do not deliberately target certain types of websites (he says Google machine learning systems that are used to improve their search results are extremely complex), updates do cause certain pages to fall up or down in search results.
Data from SEO analysis company Sistrix shows how the visibility of major publishers fell sharply in Google search results in 2018. Although its data does not include top stories and Google News, Adams says there are strong correlations between the visibility score and news publishers’ presence in top stories.
How publishers can rank in Google top stories
To rank well in top stories, ensuring that content can be quickly indexed is important. More broadly for search success, Google says a publisher should rank well if it follows its recommendations for establishing credibility and trust. These include becoming an authority on a subject, being transparent and clear on editorial guidelines and configuring pages properly for SEO.
This has resulted in fewer publishers now appearing in searches for things such as brands and celebrities compared to five years ago when publishers could rank well even for content for which they had little authority.
Adams says that Google’s focus on giving users more options for this kind of content by downgrading news leaves publishers more room to focus on "proper journalism".
"Google says we want to focus on giving the best news experience and that SEO content isn't the best user experience so publishers who still think that they need to write that to capture content have been losing out," he says.
However there are times when publishers rank more poorly than they deserve, says Adams.
Mail Online is one publisher he says has lost visibility at times – even in areas where the site has a lot of expertise and authority, for example, the royal family.
"Even when publishers are theoretically doing everything correct by the book as Google wants them to do, they’re still not winning. They're still losing, and it just makes it frustrating," he says.
"The biggest frustration that we have is that publishing clients ask me all the time what do we need to do to dominate a specific topic, a specific area of news and the answer according to Google's guidelines is to write really good authoritative news supporting that on a consistent basis," says Adams. "And it works until it doesn't. You see the rewards of that and then suddenly Google rolls out an update and all of the gains that they've made in the last few years are wiped out."
Publishers are also struggling for visibility in evergreen searches that feature outside of top stories.
Affiliate sites that once appeared on the second or third page of results are now appearing higher up in high-volume traffic searches for things such as the best albums or best artists. These websites, which were previously penalised by Google, have taken on board the search giant’s requirement to show E-E-A-T (experience, expertise, authoritativeness and trustworthiness), and are increasingly competing with news sites.
How can publishers increase their Google search success?
Looking more carefully at the competition – and broadening your idea of who your competitors are – is key, says Roderick of the Times titles.
"The days of solely competing against other national publishers are over. Our competitor set is much broader now. For example, we see more local publishers breaking through for news events that were usually dominated by national titles. And with Google presenting a more diverse set of assets in the SERPs, we are now competing against other types of content producers such as Youtubers and bloggers.
"Perhaps most importantly, Google is also trying to answer a lot more questions itself, which means we stay clear of queries that could easily be answered and instead look to provide more complex analysis or comment pieces to provide value for readers."
Going back to basic SEO principles such as ensuring the correct article structured data is used and focusing on page experience remains critically important, says Budka, but he advises publishers to start thinking carefully about their "experience" in a topic.
"The main thing I'd start thinking about is the recently [December] added 'experience' part of E-E-A-T," he says as Google now penalises news outlets unable to demonstrate enough authority in their reporting.
"This is where, long-term, I suspect publishers will start to see traffic erosion to companies doing the news piece with actual experts, not reporters reporting the views of experts.
"Forward-thinking companies who want a slice of the Google News pie are already doing this – they know if they can produce consistent news content from obviously 'experienced' experts, there’s no reason they can’t rank in Google News above the traditional national publishers."
To increase their chances of cutting through in top stories, Roderick says the Times and Sunday Times think carefully about the purpose of each piece of content.
"[We think] very carefully about what our digital coverage looks like, especially when it comes to big news events. We consider the different keywords that we’d like to rank for, and which search intent we’re trying to serve with each story. In short, every article has to have its own purpose and do its own job. It also allows us to think much more creatively about our commissioning.
"We can target long-tail keywords using analysis, profiles, features or interviews to bag a spot in top stories and get people through the door. As a subscription publisher, it’s something that suits us nicely because we’re always thinking about how we can add value with our journalism, and how we can offer something that won’t be found elsewhere for free," she says.
Investing in other content areas can also pay off.
Sistrix analysis shows that some of the highest-gaining news sites in 2022 grew their search visibility most in areas not traditionally thought of as "news". Newsquest's The Northern Echo for example increased its search visibility by 112%, with the gains largely coming from its lottery results, opening times, recalls and "net-worth" content. Forbes' Advisor affiliate project, which rates financial products such as credit cards, insurance and loans, was the driver of Forbes' increased visibility in 2022.
The highest-performing directories for many leading publishers are often non-news related. Weather is the best-performing content for the BBC, while the Financial Times’ rankings content performs particularly well in search.
One of the biggest things publishers can do is improve the assets they already publish, says Smullen.
"When you look at Google trends, each year recipes are one of the biggest search terms of the year. Most publishers publish recipes from their magazines, mostly from cooking experts, the E-E-A-T that Google is looking for, but in the form of an article and don't take advantage of structured data to help them have better visibility on search.
"In addition, most publishers don't think about evergreen content. They are trapped in the news agenda. And this is where small bloggers can swoop in and rank above big publishers and steal all of that search traffic," he says.
"If publishers want to think about growing their traffic, they also need to consider their 'search audience' has different needs,” adds Smullen, pointing to The New York Times’ yoga hub as a "best in class" example.
Although at The Times and Sunday Times most effort goes towards appearing in top stories, Roderick says, the publisher regularly updates its evergreen content around key pillars such as money, travel and arts.
While traffic to these verticals might be smaller than to breaking news content for many publishers, investing in other verticals can pay off when a publisher takes a hit to its top stories visibility.
Owning your audience through products like newsletters and podcasts is, however, essential for publishers, says Smullen.
"The only audience you own and control is your homepage and newsletter. If big tech moves the goalposts like Facebook did in 2017, some publishers can be completely wiped out.
"Lessons need to be learnt from 2017. A publisher's most important audience is its direct and subscriber audience. For a publisher to survive in the current economic climate, the goal should be to get users to download an app/subscribe to a newsletter or, even better, a paid subscription. Bring users into an ecosystem that the publishers can control."
Email pged@pressgazette.co.uk to point out mistakes, provide story tips or send in a letter for publication on our "Letters Page" blog
