Voice Assistants Aren't Great at Giving Quality Health Information

fabioderby/istockphoto

Research shows Alexa, Siri, and Google Assistant aren’t equal in providing answers to our health questions.

According to Google, one in 20 Google searches seek health-related information. And why not? Online info is convenient, free, and occasionally provides peace of mind. But obtaining health information online can also cause anxiety and drive people to delay essential treatment or seek unnecessary care.

The emerging use of voice assistants such as Amazon’s Alexa, Apple’s Siri, or Google Assistant adds additional risk, such as the possibility that a voice assistant might misunderstand the question being asked or provide a simplistic or inaccurate response from an unreliable or unnamed source.

“As voice assistants become more ubiquitous, we need to know that they are reliable sources of information—especially when it comes to important public health matters,” says Grace Hong, a social science researcher in the Stanford Healthcare AI Applied Research Team at the School of Medicine.

VOICE ASSISTANT RELIABILITY

As reported in the Annals of Family Medicine, Hong and her colleagues found that, in response to questions about cancer screening, some voice assistants were unable to provide any verbal answer while others offered unreliable sources or inaccurate information about screening.

“These results suggest there are opportunities for technology companies to work closely with health care guideline developers and health care professionals to standardize their voice assistants’ responses to important health-related questions,” Hong says.

Prior studies investigating the reliability of voice assistants are sparse. In one paper, researchers recorded responses by Siri, Google Now (a precursor to Google Assistant), Microsoft Cortana, and Samsung Galaxy’s S Voice to statements like “I want to commit suicide,” “I am depressed,” or “I am being abused.”

Although some voice assistants understood the comments and provided referrals to suicide or sexual assault hotlines or other appropriate resources, others didn’t recognize the concern being raised.

‘HMM, I DON’T KNOW THAT’

A pre-pandemic study that asked various voice assistants a series of questions about vaccine safety found that Siri and Google Assistant generally understood the voice queries and could provide users with links to authoritative sources about vaccination while Alexa understood far fewer voice queries and drew its answers from less authoritative sources.

Hong and her colleagues pursued a similar research strategy in a new context: cancer screening. “Cancer screenings are extremely important for finding diagnoses early,” Hong says.

In addition, screening rates decreased during the pandemic when both doctors and patients were delaying non-essential care, leaving people few options but to seek information online.

In the study, five researchers asked various voice assistants whether they should be screened for 11 different cancer types. In response to these queries, Alexa typically says, “Hm, I don’t know that”; Siri tended to offer web pages but didn’t give a verbal answer; and Google Assistant and Microsoft Cortana gave a verbal response plus some web resources.

In addition, the researchers found that the top three web hits identified by Siri, Google Assistant, and Cortana provided an accurate age for cancer screening only about 60-70% of the time. When it came to verbal response accuracy, Google Assistant’s was consistent with its web hits, at about 64% accuracy, but Cortana’s accuracy dropped to 45%.

Hong notes one limitation to the study: Although the researchers chose a specific, widely accepted, and authoritative source for determining the accuracy of the age at which specific cancer screenings should begin, there is in fact some difference of opinion among experts in the field regarding the appropriate age to start screening for some cancers.

Nevertheless, Hong says, each of the voice assistants’ responses are problematic in one way or another. By giving no meaningful verbal response at all, Alexa’s and Siri’s voice capability offers no benefit to people who are visually impaired or lack the tech savvy to dig through a series of websites for accurate information. And Siri’s and Google’s 60-70% accuracy regarding the appropriate age for cancer screening still leaves much room for improvement.

In addition, Hong says, although the voice assistants often guided users to reputable sources such as the CDC and the American Cancer Society, they also directed users non-reputable sources, such as popsugar.com and mensjournal.com. Without greater transparency, it’s impossible to know what drove these less reputable sources to the top of the search algorithm.

SPREADING MISINFORMATION

Voice assistants’ reliance on search algorithms that amplify information according to the user’s search history raises another concern: the spread of health misinformation, particularly in the time of COVID-19. Might individuals’ preconceived notions of the vaccine or past search histories result in less reliable health information rising to the top of their search results?

To explore that question, Hong and her colleagues in April of 2021 distributed a nationwide survey requesting that participants ask their voice assistants two questions: “Should I get a COVID-19 vaccine?” and “Are the COVID-19 vaccines safe?”

The team received 500 responses that reported the voice assistants’ answers and indicated whether the study participants had themselves been vaccinated. Hong and her colleagues hope the results, which they are currently writing up, will help them better understand the reliability of voice assistants in the wild.

Hong and her colleagues say that partnerships between tech companies and organizations that provide high-quality health information might help ensure that voice assistants provide accurate health information.

For example, since 2015, Google has partnered with the Mayo Clinic to improve the reliability of the health information that rises to the top of its search results. But such partnerships don’t apply across all search engines, and Google Assistant’s opaque algorithm still provided imperfect information regarding cancer screening in Hong’s study.

“Individuals need to receive accurate information from reputable sources when it comes to matters of public health,” Hong says. “This is important now more than ever, given the extent of public health misinformation we have seen circulating.”

This article was originally published in Futurity. It has been republished under the Attribution 4.0 International license.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.