Analysis of nearly 500 articles finds quality news dominates.
News about the coronavirus has saturated the media for the past several weeks and promises to dominate the headlines for months to come. It’s not just world news or health news; business, sports, and even entertainment news is all about the pandemic.
Nobl was curious about the quality of coronavirus coverage, so we turned our AI engine to evaluate the articles on Google News, aggregated from a variety of news sources from March 1 – 20. Our engine uses indicators, such as credibility, bias and sentiment, to evaluate the text of an article and derive a score from 1 – 100. The higher the score, the higher the quality.
How can a high-quality score make a difference in the era of coronavirus? People are making decisions and learning how they can stay healthy based on what they read. They need sources of information that are accurate and reliable. Tools to publish, distribute, and access content are abundant; yet there’s a paucity of tools to evaluate all that information. Users assume that news content aggregators, such as Google News, solve this problem by curating content from a multitude of sources. But the curation process is not transparent and their evaluation process is unclear.
So, we evaluated each coronavirus article featured on the Google News homepage and then filtered that report to look at only articles that contained “coronavirus” or “covid-19” in the title of the article. This resulted in about 490 articles that populated many sections on Google News, including the Headlines section and US, World, Business, Health, Science, Sports and even Entertainment.
Since Google News curates the articles featured on their homepage, we expected the overall quality of the articles to be fairly high. And in fact the average score across all sections was 88.13. The median score was a whopping 97.
The high score is in sharp contrast to the quality of general news articles previously seen in a study just two months earlier. In that analysis of more than 1300 articles appearing in Google News’ headlines, US and World sections, the average score was 83.81.
The average score of Coronavirus articles from the same three sections in March was 91.05. About 62% of the articles came from 16 publishers and the average score for this group was nearly 90 (see table, below). Of this group of publishers, the articles from the Los Angeles Times had the highest average score of 96.75. Articles from Yahoo! had the lowest average score at 76.71.
Coronavirus articles by publisher on Google News (Mar 1-20)
Almost all of the articles evaluated were ‘news’ articles; there were 21 opinion articles (~ 4%).
The scores of articles also varied based on the section of the page to which they were assigned.
Most of the articles on the virus (about 70%) were in the Headlines, US, Health and Business sections. The highest quality articles were in the Business, US, World and Headlines sections. Articles in the Sports and Health sections scored lower, on average.
Caveats
While these initial data points are interesting, more research is needed to better understand why certain publishers had lower scores and why certain categories had lower scores.
Our study is sampling a sample of articles that were selected by Google. While we would assume that Google would select only the best articles for Google News, it is possible that the variance is an artifact of their selection process.
In other words, Google may have selected higher quality articles from some sites and lower quality from other sites. Then again, it’s also possible Google’s selection process was representative of the quality of articles on the site. In that case, it would provide evidence that some publishers produce higher quality content than others. Trustium intends to do another study to address this question in the near future.
Likewise, we can’t definitively conclude that certain categories of articles are higher quality than other categories. This also could be confounded by Google’s selection process or other factors.
Conclusion
The coronavirus content selected by Google News between March 1-20 was dominated by news and not opinion content. This is in contrast to Nobl’s previous analysis of general news on Google News, which contained a higher percentage of opinion and biased articles.
Opinion articles tend to contain bias and are more likely to contain emotion, both of which are hallmarks of more provocative and controversial content. And controversial content is known to attract attention and seduce users to click and share. These behaviors are monetized through advertising and provide profits to content distribution platforms.
Therefore, it’s noteworthy that in this pandemic, Google News is propagating more unbiased, unemotional news content than is seen in its coverage of other news topics.
This could be because the health and medical science community tends to be less politically motivated. Of course it is in everyone’s best interest to get the most accurate information available when dealing with the pandemic, so the demand for quality news remains high despite the lack of sensationalism.
If this global health crisis provides evidence to Google and other purveyors of content that there is a marketplace and demand for non-controversial content, that would certainly be a silver lining.