AI Algorithms Intended to Root Out Welfare Fraud Often End Up Punishing the Poor Instead

Jonathan Weiss/Shutterstock.com

These systems must be designed in ways that are fair, transparent and accountable to prevent hurting society’s most vulnerable.

President Donald Trump recently suggested there is “tremendous fraud” in government welfare programs.

Although there’s very little evidence to back up his claim, he’s hardly the first politician – conservative or liberal – to vow to crack down on fraud and waste in America’s social safety net.

States – which are charged with distributing and overseeing many federally funded benefits – are taking these fraud accusations seriously. They are increasingly turning to artificial intelligence and other automated systems to determine benefits eligibility and ferret out fraud in a variety of benefits programs, from food stamps and Medicaid to unemployment insurance.

Of course, government agencies should ensure that taxpayer dollars are spent effectively. The problem is these automated decision-making systems are sometimes rife with errors and designed in ways that punish the poor for being poor, leading to tragic results.

As a clinical law professor who has researched safety net programs and has represented low-income clients in public benefits cases for over 20 years, I believe it’s essential these systems are designed in ways that are fair, transparent and accountable to prevent hurting society’s most vulnerable.

Facts about Fraud

First, it’s important to make one thing clear: The evidence suggests incidents of user fraud in government welfare programs are rare.

For instance, the food stamp program, formally called the Supplemental Nutrition Assistance Program, currently serves about 40 million people monthly at an annual cost of US$68 billion. Despite regular denigration of food stamp recipients, less than 1% of benefits go to ineligible households, according to the federal government.

And, of those families, the majority of overpayments result from mistakes by recipients, state workers or computer programmers as they navigate complex regulatory requirements – not any intent to defraud the system.

As for Medicaid, which provides health insurance for low-income people, research has shown that the bulk of fraudulent activity is committed by health care providers – not by the 64 million needy people that use the program.

Within unemployment insurance, the “improper payment” rate for 2019 is 10.6%, which includes payments that should not have been made or that were made in an incorrect amount, but intentional fraud estimates are much lower.

When Algorithms Fail

Nonetheless, many states seem to be adopting systems that assume criminal intent on the part of the needy.

Many states have begun using “sophisticated data mining” techniques to identify fraud in the food stamp program, according to the General Accountability Office. Another report identified 20 states using AI tools in unemployment insurance. And the federal government is providing support to state Medicaid programs to upgrade their decades-old technology with more advanced software.

These types of automated decision-making systems rely on algorithms, or mathematical instructions. Some algorithms use machine learning – a form of artificial intelligence – to replace decisions that would otherwise be made by humans. They analyze large sets of data to recognize patterns or make predictions.

But officials should approach these systems with caution. The results for low-income families with little margin for error can be disastrous.

For instance, in Michigan, a $47 million automated fraud detection system adopted in 2013 made roughly 48,000 fraud accusations against unemployment insurance recipients – a five-fold increase from the prior system. Without any human intervention, the state demanded repayments plus interest and civil penalties of four times the alleged amount owed.

To collect the repayments – some as high as $187,000 – the state garnished wages, levied bank accounts and intercepted tax refunds. The financial stress on the accused resulted in evictions, divorces, destroyed credit scores, homelessness, bankruptcies and even suicide.

As it turns out, a state review later determined that 93% of the fraud determinations were wrong.

How could a computer system fail so badly? The computer was programmed to detect fraud when claimants’ information conflicted with other federal, state and employer records. However, it did not distinguish between fraud and innocent mistakes, it was fed incomplete data, and the computer-generated notices were designed to make people inadvertently admit to fraud.

Michigan is not an outlier. Program-wide algorithmic errors have similarly plagued Medicaid eligibility determinations in states such as Indiana, Arkansas, Idaho and Oregon.

And the issue isn’t just an American one. Many countries such as Australia and the U.K. are embracing these types of systems and encountering similar problems. The United Nations special rapporteur on extreme poverty and human rights issued a report in October that warned governments across the world to “avoid stumbling zombie-like into a digital welfare dystopia” as they automate their social welfare systems.

In a closely watched decision, a court in the Netherlands recently halted a welfare fraud detection system, ruling that it violates human rights. The decision is likely to bring closer scrutiny to these systems worldwide, although Americans have fewer legal protections than their European counterparts.

Algorithms Aren’t Magic

AI won’t magically root out what little fraud there is from the welfare rolls.

Mistakes can happen when software developers translate complex regulatory requirements into code and when they make programming errors. The massive sets of data fed into automated systems inevitably will contain some inaccuracies and omissions. And algorithms can also replicate embedded societal biases and end up discriminating against marginalized groups.

Without a human in the decision-making loop, these mistakes become compounded as they flow through multiple data-sharing systems.

To avoid these problems, state and other governments should ensure the systems they install are transparent in how they function, are accountable for mistakes and don’t incentivize private contractors hired to design them to kick people off the rolls to make more money. States should also make sure representatives from all groups affected are involved in their creation and monitoring.

In my research and legal work, I have found automated fraud detection is too often built on the assumptions that computers are magic and fraud among the poor is endemic. State officials should flip those assumptions and make computers work for the people rather than against them.

Michele Gilman is a venable professor of Law at the University of Baltimore

The ConversationThis article is republished from The Conversation under a Creative Commons license. Read the original article.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.