recommended reading

Why Experts Worry Algorithms Can Do More Harm Than Good


Some tech experts fear the mathematical equations underlying much of today's data analytics technology could ultimately be dangerous for individuals.

In a poll of about 1,300 people who closely observe technology development, about 37 percent concluded algorithms could have more negative societal impacts than positive ones, according to a new Pew report.

"[S]ometimes, the application of algorithms created with good intentions leads to unintended consequences," said the report, which canvassed futurists including Defense One's Patrick Tucker.

» Get the best federal technology news and ideas delivered right to your inbox. Sign up here.

About 38 percent thought positive impacts would outweigh the negatives, and a quarter thought the impact would be evenly split between positive and negative.

The potential misuse of algorithms has garnered increased attention recently. In May, the White House published a report warning about the downsides of big data. For instance, the algorithm that suggests a credit score for an applicant with no credit history might draw on that person's phone bills or social media connections; but it could also create greater disparities between communities by providing a lower credit score to applicants who share attributes with other low-credit customers, such as shopping at the same grocery store.

Especially when algorithms are making decisions that could have major social impacts, "humans are considered to be an 'input' to the process and they are not seen as real, thinking, feeling, changing beings," the report said. Experts agreed this system creates a "flawed, logic-driven society and that as the process evolves—that is, as algorithms begin to write the algorithms—humans may get left out of the loop, letting the robots decide.'"

The Pew report pointed to Microsoft's recent Twitter experiment—in which a bot named Tay was instructed to communicate with other users and to learn from their behavior, but quickly began spouting racist and sexist messages—as another example of algorithms going awry.

A lack of diversity in the tech workforce could be reinforcing biases coded into algorithms, at least one expert thought.

Across industries, "algorithms will be primarily designed by white and Asian men—with data selected by these same privileged actors—for the benefit of consumers like themselves," wrote Justin Reich, executive director at the MIT Teaching Systems Lab. "The harms of new technology will be most experienced by those already disadvantaged in society, where advertising algorithms offer bailbondsman ads that assume readers are criminals, loan applications that penalize people for proxies so correlated with race that they effectively penalize people based on race and similar issues."

One expert suggested countering these challenges by creating a "Food and Drug Administration for Algorithms" that could involve a "pre-market review to prevent—without stifling innovation—the introduction of unacceptably dangerous algorithms into the market," researcher Andrew Tutt said in that report.

Pew conducted the research with Elon University's Imagining the Internet Center.

Threatwatch Alert

Thousands of cyber attacks occur each day

See the latest threats


Close [ x ] More from Nextgov

Thank you for subscribing to newsletters from
We think these reports might interest you:

  • It’s Time for the Federal Government to Embrace Wireless and Mobility

    The United States has turned a corner on the adoption of mobile phones, tablets and other smart devices, outpacing traditional desktop and laptop sales by a wide margin. This issue brief discusses the state of wireless and mobility in federal government and outlines why now is the time to embrace these technologies in government.

  • Featured Content from RSA Conference: Dissed by NIST

    Learn more about the latest draft of the U.S. National Institute of Standards and Technology guidance document on authentication and lifecycle management.

  • A New Security Architecture for Federal Networks

    Federal government networks are under constant attack, and the number of those attacks is increasing. This issue brief discusses today's threats and a new model for the future.

  • Going Agile:Revolutionizing Federal Digital Services Delivery

    Here’s one indication that times have changed: Harriet Tubman is going to be the next face of the twenty dollar bill. Another sign of change? The way in which the federal government arrived at that decision.

  • Software-Defined Networking

    So many demands are being placed on federal information technology networks, which must handle vast amounts of data, accommodate voice and video, and cope with a multitude of highly connected devices while keeping government information secure from cyber threats. This issue brief discusses the state of SDN in the federal government and the path forward.

  • The New IP: Moving Government Agencies Toward the Network of The Future

    Federal IT managers are looking to modernize legacy network infrastructures that are taxed by growing demands from mobile devices, video, vast amounts of data, and more. This issue brief discusses the federal government network landscape, as well as market, financial force drivers for network modernization.


When you download a report, your information may be shared with the underwriters of that document.