The Technology That Will Make It Impossible for You to Believe What You See

Evan El-Amin/Shutterstock.com

With these techniques, it’s difficult to discern between videos of real people and computerized impostors that can be programmed to say anything.

The president was seething.

His problem was with the press, yes, but also with the technology they used. Electronic media had changed everything. People were glued to their screens. “I have never heard or seen such outrageous, vicious, distorted reporting,” he said in a news conference.

The age of television news, Richard Nixon told reporters gathered that day in October 1973, was shaking the confidence of the American people. He didn’t yet know his presidency would reach a calamitous end. When Nixon announced he would resign, in August 1974, he spoke directly into a television camera. The recording remains stunning half-a-century later—mostly because of the historic nature of the moment, but also because of the power of broadcast.

Even in an informational era transformed by the web, video is a gripping format. In the chaos of real-time news, especially, there’s an advantage to being able to see something with your own eyes.

Or, there used to be.

At a time when distrust in journalistic institutions is swelling, technology that further muddies the ability to discern what’s real is rapidly advancing. Convincing Photoshop-esque techniques for video have arrived, and the result is simultaneously terrifying and remarkable.

Computer scientists can now make realistic lip-synched videos—ostensibly putting anyone’s words into another person’s mouth.

The animated gif that you see above? That’s not actually Barack Obama speaking. It’s a synthesized video of Obama, made to appear as though he’s speaking words that were actually inputted from an audio file.

The clip comes from researchers at the University of Washington, who developed an algorithm to take audio of someone talking and turn that into a realistic video of someone speaking those words. In the video below, you can see a side-by-side comparison of the original audio—which came from actual Obama remarks—and the generated video.

Obama was a natural subject for this kind of experiment because there are so many readily available, high-quality video clips of him speaking. In order to make a photo-realistic mouth texture, researchers had to input many, many examples of Obama speaking—layering that data atop a more basic mouth shape. The researchers used what’s called a recurrent neural network to synthesize the mouth shape from the audio. (This kind of system, modeled on the human brain, can take in huge piles of data and find patterns. Recurrent neural networks are also used for facial recognition and speech recognition.) They trained their system using millions of existing video frames. Finally, they smoothed out the footage using compositing techniques applied to real footage of Obama’s head and torso.

The researchers wrote a paper about their technique, and they plan to present their findings at a computer graphics and interactive techniques conference next month.

“The idea is to use the technology for better communication between people,” says Ira Kemelmacher-Shlizerman, a co-author of the paper and an assistant professor in the department of computer science and engineering at the University of Washington. She thinks this technology could be useful for video conferencing—one could generate a realistic video from audio, even when a system’s bandwidth is too low to support video transmission, for example. Eventually, the technique could be used as a form of teleportation in virtual reality and augmented reality, making a convincing avatar of a person appear to be in the same room as a real person, across any distance in space and time.

“We’re not learning just how to give a talking face to Siri, or to use Obama as your GPS navigation, but we’re learning how to capture human personas,” says Supasorn Suwajanakorn, a co-author of the paper. Not surprisingly, several major technology companies have taken notice: Samsung, Google, Facebook, and Intel all chipped in funding for this research. Their interest likely spans the realms of artificial intelligence, augmented reality, and robotics. “I hope we can study and transfer these human qualities to robots and make them more like a person,” Suwajanakorn told me.

Quite clearly, though, the technique could be used to deceive. People are already fooled by doctored photos, impostor accounts on social media, and other sorts of digital mimicry all the time.

Imagine the confusion that might surround a convincing video of the president being made to say something he never actually said. “I do worry,” Kemelmacher-Shlizerman acknowledged. But the good outweighs the bad, she insists. “I believe it’s a breakthrough.”

There are ways for experts to determine whether a video has been faked using this technique. Since researchers still rely on legitimate footage to produce portions of a lip-synched video, like the speaker’s head, it’s possible to identify the original video that was used to create the made-up one.

“So, by creating a database of internet videos, we can detect fake videos by searching through the database and see whether there exists a video with the same head and background,” Suwajanakorn told me. “Another artifact that can be an indication is the blurry mouth [and] teeth region. This may be not noticeable by human eyes, but a program that compares the blurriness of the mouth region to the rest of the video can easily be developed and will work quite reliably.”

It also helps if you have two or more recordings of a person from different views, Suwajanakorn said. That’s much harder to fake. These are useful safeguards, but the technology will still pose challenges as people realize its potential. Not everyone will know how to seek out the databases and programs that allow for careful vetting—or even think to question a realistic-looking video in the first place. And those who share misinformation unintentionally will likely exacerbate the increasing distrust in experts who can help make sense of things

“My thought is that people will not believe videos, just like how we do not believe photos once we’re aware that tools like Photoshop exist,” Suwajanakorn told me. “This could be both good and bad, and we have to move on to a more reliable source of evidence.”

But what does reliability mean when you cannot believe your own eyes? With enough convincing distortions to reality, it becomes very difficult to know what’s real.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.