Artificial Intelligence Just Changed the Future of Information Security

Patrick Tucker/Defense One

At DARPA’s Cyber Grand Challenge, bots showed off their ability to help a world wallowing in vulnerable code.

LAS VEGAS, Nev. — Mayhem ruled the day when seven AIs clashed here last week — a bot named Mayhem that, along with its competitors, proved that machines can now quickly find many types of security vulnerabilities hiding in vast amounts of code.

Sponsored by the Defense Advanced Research Projects Agency, or DARPA, the first-of-its-kind contest sought to explore how artificial intelligence and automation might help find security and design flaws that bad actors use to penetrate computer networks and steal data.

Mayhem, built by the For All Secure team out of Carnegie Mellon University, so outclassed its competition that it won even though it was inoperable for about half of the contest’s 96 270-second rounds. Mayhem pivoted between two autonomous methods of finding bugs and developing ways to exploit them.

Under one method, dubbed symbolic execution, Mayhem tries to figure out how a target program works by systematically replacing sample inputs with classes of inputs. As James King writes in this seminal 1976 paper on the idea: “Each symbolic execution result may be equivalent to a large number of normal test cases. These results can be checked against the programmer’s expectations for correctness either formally or informally.”

But symbolic execution has serious drawbacks when you attempt to use it against complex code. If you are evaluating what a bit of code might do then you have to consider more than one possibility. This leads to something called path explosion. Considering all those paths exhausts memory.

So, the Carnegie Mellon team made its symbolic execution engine smarter than average by helping it prioritize which paths to explore first.

“For example, we have found that if a programmer makes a mistake—not necessarily exploitable—along a path, then it makes sense to prioritize further exploration of the path since it is more likely to eventually lead to an exploitable condition,” Carnegie Mellon professor David Brumley and co-authors wrote in a paper that lays out the basics of their approach.

But the bot also uses a second technique, called guided fuzzing, sort of the Oscar Madison to symbolic execution’s Felix Unger. Where symbolic execution is neat and cerebral, fuzzing is messy. For All Secure’s engine, dubbed Murphy, throws random or invalid data at the target code, and watches to see whether it  crashes, slows down, or exhibits other behavior that suggests a flaw to exploit.

“These two components communicate through a … database by sharing testcases they find ‘interesting,’ based on the coverage they achieve.” Brumley wrotes on the team’s blog. “By using Murphy and Mayhem together, we are able to boost both: the fuzzer is great at quickly finding shallow bugs, but fails on complex cases; Mayhem is good at generating deep paths in a program, but is not always fast enough to explore them all.”

Mayhem wasn’t the only machine that made history at DARPA’s contest, dubbed the Cyber Grand Challenge. Team Shellphish from the University of California at Santa Barbara won a key consolation prize by detecting the so-called Crackaddr bug, a task long thought impossible for a machine-reasoning system. Like Mayhem, Shellphish combined constrained symbolic execution and fuzzing, but it used the former to tell the latter what areas to attack.

“If I make a version of it this long,” said Mike Walker, DARPA program manager, holding his hands about a foot apart to represent a highly abbreviated version of Crackaddr, “I still can’t find a machine that can figure it out. There are papers in 2015 that are like, ‘We still can’t figure this out.’ A machine solved it live in its compiled form in front of a live audience. To the vulnerability community, that was a pretty big deal.”

Cyber Jiu-Jitsu

It will be some time before robots can tackle the most difficult bugs out there, but AI promises to remove the easy ones and drastically improve the ability of humans to find the more difficult ones.

“It’s not hard for computers to analyze a huge number of programs; we just parallelize. But it’s hard for humans; they are expensive to parallelize and scale,” Brumely said. “It’s hard for computers to analyze a large program in depth … Humans seem better right now at getting ‘deeper’ into large programs.”

But there is hope AIs will be able to solve less complicated vulnerability problems much faster, and help the human bug hunters who have little chance of keeping up with the enormous volume of code that will permeate the globe in the decades ahead.

“People are always asking me the replacement question,” meaning: When will an AI replace a human in a given role? he says. “Do you know which person is monitoring a terabyte link?” asked Walker, referring to a data link the size of a million, million bytes. “Nobody. How could we ever look at terabytes?”

Instead, organizations from Apple to the Pentagon are increasingly crowdsourcing their bug hunts. But the military can’t allow the public access to weapons programs such as the F-35 Joint Strike Fighter, which relies on millions of lines of code.

Automated bug hunting in 2016 is in somewhat the same situation as mixed martial arts a quarter-century ago. The sport’s earliest days were arguably its most interesting; each competition promised a never-before-seen clash of styles, approaches and schools of thought.

In the first Ultimate Fighting Championship, held in Denver in 1993, fighters trained to deliver and absorb wild aerial kicks found themselves stunned by Royce Gracie, who used a form of Brazilian jiu-jitsu to pull them to the mat and defeat them on the ground.

A match between two opponents where one had trained in Gracie’s techniques and the other had not was incredibly fast and brilliant to watch. But as Gracie’s influence spread and became mainstream, fighters employing the method lost the surprise element that gave certain victory. A fight between two fighters trained in jiu-jitsu looks very different. Picture two men locked in an embrace, virtually motionless until one creates just enough room for his opponent to execute a key hold.

This is what the Las Vegas match-up means for the future of information security.  We are rapidly leaving the phase of easy takedowns. The AIs will quickly show themselves able to execute all the easy moves faster than any human competitor.

The real match has barely begun.

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.