Firm Offers Tips on Eliminating Bias and Other Risks When Deploying Analytics Models

metamorworks/istockphoto

Government leaders are “appropriately” wary of implementing models they don’t understand, a senior expert explained.

Advanced analytics models perform what people have asked them to do—so the right kind of human oversight system is needed to eliminate bias in them, according to a new report from management consulting firm McKinsey & Company on de-risking public sector technology.

“I hope our research will help leaders understand that models can be made more transparent, more accurate and fairer—and that these risks can be managed,” McKinsey Senior Expert Eric Schweikert told Nextgov Thursday.

Schweikert served as lead author of the report, released Friday. He briefed Nextgov on some of its main takeaways for government insiders.

Multiple McKinsey experts contributed to the work, which discusses “pernicious risks” associated with advanced analytics in the public sector, like bias and discrimination. “Examples are not hard to find: advanced analytics models have been shown to sentence people of color more harshly, erroneously accuse low-income and immigrant families of fraud, and award lower grades to students from less privileged neighborhoods,” they wrote in the report, noting that this might be why many public agencies may be hesitant to deploy such assets at scale. 

“As the complexity of data increases, fewer and fewer people truly understand what models are doing—nor where their bias may lie—which makes that human oversight more difficult. My work often involves helping government agencies kickstart their analytics efforts. I find senior leaders are—appropriately—wary of implementing models they don’t understand,” Schweikert explained. “That’s why bringing transparency and accountability to models is a passion of mine, both by temperament, and in order to help organizations be more comfortable with using analytics models.” 

Artificial intelligence and machine learning models can present more potential risks than others, depending on the complexity of their algorithms. But techniques like “explainable AI” can calculate the contribution of different variables to an answer in an otherwise unexplainable AI model, Schweikert said. People can therefore have debates about them that anyone can follow, regardless of expertise. 

In the report, the experts outline key actions to help mitigate risks of bias and other hazards.

First on that list is to make someone responsible for model risk management. In regulated industries, that responsibility typically sits with a chief risk officer, the report notes, but “in federal agencies that lack robust enterprise-risk-management structures, it could sit with the chief information officer, chief data officer, or the senior-level executive responsible for governance and oversight of technology.”

To Schweikert, the right candidate needs a unique mix of hard-to-find and much-in-demand skills.

“But when you find the right person, it can transform analytics in an agency. Crucially, they need to act as an analytics ‘translator,”’ he said. “That means engaging deeply with the mission of the agency—the senior leaders who are asking the questions—yet understanding enough of the data science to spot problems.” 

Further, he noted, individuals in such roles must be able to balance process and oversight to minimize risk without inadvertently creating obstacles to productivity. They also need to have the interpersonal skills to communicate the transformative value of analytics and collaborate with skeptical colleagues.

Other recommendations listed in the report include: Developing and sharing a clear set of analytical practices and standards, building a model-risk-management infrastructure, considering forming algorithm review panels and appointing an ombudsperson, and strategizing at the enterprise level.

“These are all ways of ensuring the people building the models have considered potential sources of bias and have done everything to eliminate them,” Schweikert said. “Finally, rigorous measurement of bias in the model’s decisions, in production, could underpin the human effort.”

The study points to a recent report that shows 45% of U.S. agencies were “still only experimenting with advanced analytics “and that 12% “were using highly sophisticated techniques.” Schweikert emphasized that point, noting that “a lot of government agencies are just getting started with AI.” In some cases, they’re turning to chatbots or other simple tools. Still, in the related area of machine learning, he said, “there are agencies that have done deep and serious work with good outcomes.”

Diving deeper into the challenges faced by both government and industry in this realm, the senior expert said one of the biggest is bias that’s existed in data for decades. 

“For instance, few people would explicitly use ethnicity as an input variable, but many would use zip code, which—depending on how the model is constructed—could inadvertently become a proxy for ethnicity,” he explained. “And that data also includes historical bias: for instance, home-loan models have had to address the legacy of human-driven redlining that is embedded in loan data.”

On top of that, some algorithms can be so sophisticated that it’s hard to understand how they are making decisions. In those cases, there could be a need to “simplify” the model “to give up a little precision for the sake of explainability,” Schweikert said. And federal entities with less experience with analytics may also lack expertise translating business questions into specific mathematical targets for models.

“[L]et’s say I’m overwhelmed with resumes and want to build an AI model to screen them. But I want to make sure I consider demographic characteristics in the outcome—ensuring there is a diverse representation. That can be translated into the math in a surprising number of ways,” he noted. “The agency needs to have someone who can help broker that conversation between agency leadership and the data science team to deliver the right outcomes.”

Schweikert added, going forward, McKinsey will continue to do extensive research and development on model risk management and responsible analytics.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.