A Computer’s Hot Take on the 2016 Election

Republican presidential nominee Donald Trump and Democratic presidential nominee Hillary Clinton are introduced during the presidential debate at Hofstra University in Hempstead, N.Y., Monday, Sept. 26, 2016.

Republican presidential nominee Donald Trump and Democratic presidential nominee Hillary Clinton are introduced during the presidential debate at Hofstra University in Hempstead, N.Y., Monday, Sept. 26, 2016. David Goldman/AP File Photo

An artificial intelligence found Bernie Sanders and Hillary Clinton seemed to get “happier” coverage than Donald Trump. But is that evidence of media bias? Not necessarily.

Donald Trump’s message of the week, in case you’ve somehow managed to avoid it, is that the election is rigged by a corrupt (and apparently monolithic) media.

“The election is absolutely being rigged by the dishonest and distorted media pushing Crooked Hillary,” Trump wrote in a tweet on Sunday night.

Trump has said repeatedly he’s been treated unfairly by news organizations. But is that true?

Back in July, I asked the computer scientist Andy Reagan if he could help me design an experiment that might begin to gauge the tone of media coverage about various presidential candidates. I knew Reagan, who is working toward his Ph.D. in mathematics at the University of Vermont, from his fascinating research into how a computer identified the six main arcs in storytelling across literary fiction. For that work, he and his colleagues had developed a database of more than 10,000 frequently-used words, all ranked by how “happy” they were perceived to be. (The happiest word on their list: Laughter. The saddest: Terrorist.)

I wondered if it would be possible to use that list—and Reagan’s computer skills—to assess a huge trove of campaign coverage over time, so we could compare the overall tone of articles about the different candidates. But to do this, we would need a ginormous database of campaign coverage.

Digging into the Nexis newspaper archives, I collected tens of thousands of articles about Bernie Sanders, Hillary Clinton and Donald Trump—whatever was published in more than 50 U.S. newspapers and websites over a 13-month period from July 2015 through August 2016. (I also pulled a year’s worth of campaign coverage about Bill Clinton from his 1992 run for president; more on that in a minute.)

Reagan then took the data and fed it into his computer model, which spit out a complex portrait of the tone of campaign coverage over the past year. We had the beginnings of a sentiment analysis of presidential campaign coverage, one that might suggest the emotional tone of media stories about each candidate.

Trump may feel validated to know that, indeed, we found that articles about him, overall, were the most negative of the bunch—even before the utter insanity of the past several weeks. But there are plenty of caveats.

First, let’s look at the data. A totally neutral article—one with just as many happy words as sad words—would have scored a 5.0 on Reagan’s ranking system. Sanders, Clinton and Trump all scored higher than that—meaning, the collection of articles about each of them were more positive than negative. That’s not surprising, Reagan told me, because natural language is biased toward happy words.

The 18,640 articles about Hillary Clinton that we assessed scored 5.85 on the happiness scale, whereas the 29,019 stories about Trump from that same period ranked 5.78—not quite as happy. Bernie Sanders scored the highest for the positive tone of coverage about him, with a ranking of 5.90 across 7,841 articles. (For context on what these numbers mean, consider the happiest day on Twitter in the past year: Christmas day, according to Reagan’s model, with a score of 6.28.)

It isn’t clear from the number-rankings alone, but the differences we found in coverage are meaningful—if not exactly dramatic. The scores themselves are “only useful in relative terms,” Reagan told me. And they’re most interesting once you dig into the substance of why coverage about one candidate is more positive or negative than the next.

On a word-by-word basis, we can see why coverage of Trump skews more negative—and in many cases, it seems to be because the coverage about him reflects the language he himself uses.

So, for instance, the word “great” appears frequently in articles Trump, whose campaign slogan is “Make America Great Again.” And because “great” is perceived as an exceedingly positive word—it ranks 55th happiest on a list of more than 10,000 words—this helps buoy his score. But there are many other negative words frequently associated with Trump—more so than with Hillary Clinton—that drag it back down, including: “bad,” “racist,” “war,” “no,” “don’t,” “ban” and “illegal.”

This doesn't come as much of a surprise: Some of Trump’s loudest campaign messages have been about his call for a ban on Muslims entering the United States—or, in his words a “complete and total shutdown”—and his belief that illegal immigrants “have to go.”

Tonally, many observers noted a deep contrast in rhetoric between Trump and Clinton at the conventions in July—Trump’s speech was widely perceived as far more pessimistic than Clinton's. Put another way, a sentiment analysis of the candidates’ words—just verbatim transcripts, with no added reporting—might still find a more negative ranking for Trump.

Looking at the words that influence the sentiment score for articles about Hillary Clinton, we find words with positive associations like: “new,” “like,” “win,” “first,” “you,” “united” and “women.” And despite the negative words associated with coverage of the Democratic candidate (“gun,” “attacks,” “fight,” “tax,” “war,” “never,”) the overall picture is more positive for Clinton than it is for Trump.

Coverage of Bernie Sanders ranked “happiest” of all the candidates—and, interestingly, his score was defined by a notably higher prevalence of positive words than his political peers. Once again, as with Trump, you can catch a glimpse of Sanders’s campaign rhetoric in the articles about him—articles that were characterized by only a handful of negative words (“poverty,” and “fight” were among the few) but many positive ones (“win,” “health,” “new,” “victory,” “free,” “young” and “college.”)

There’s a pretty significant contextual problem to taking overall scores at face value, though.

For one thing, there are two words in particular that are weighted as positive, but that represent decidedly negative elements of coverage of the 2016 campaign: “Email” and “women.”

So, while the computer model would tell us that more mentions of “email” would give coverage of any candidate a higher score on the happiness scale—more mentions of “email” is, in actuality, a potential negative for Clinton, given her improper use of a private email account during her time as secretary of state.

Similarly, up until August, there were more mentions of “women” in articles about Clinton than there were in articles about Trump—a finding that contributed to Trump’s lower score on the happiness scale. But the many mentions of “women” in stories about Trump since then, given the repeated accusations of sexual assault he is facing, are decidedly negative in context, despite how a computer model would see it.

There’s a similar contextual absence in the computer’s assessment of tone among stories about Bill Clinton, in 1992. One of the happiest words that appears frequently in stories about him, and a word that helps inflate his overall score, is “flowers.”

But anyone who remembers the 1992 campaign is likely to recognize that the many references to “flowers” were probably references to “Gennifer Flowers,” the woman who came forward during Bill Clinton’s run for presidency and said she’d had an affair with him. Not so positive, after all.

“There are always going to be words that are used in different contexts, and sometimes the difference is subtle,” Reagan told me. This is exactly the reason it’s important to use methods where a computer’s model can be inspected on a word-by-word basis after the fact, he says, rather than a deep learning approach that would be impossible to reverse engineer.

Of course, there’s a larger point still: Journalistic fairness doesn’t come down to how happy each word in the article makes a reader feel—even when the word is weighted properly in context of the events it’s describing. Nor does every candidate merit the same kind of coverage. Often, the tone of a story is negative because it reflects the reality of what’s happening. And no one should need a computer model to recognize that.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.