NASA AI to power first-responder tech

There's more data than ever available to first responders, but finding the right data at the right time poses big challenges. NASA's Jet Propulsion Laboratory might have an answer.

JPL is working on an artificial intelligence solution called Audrey (Assistant for Understanding Data through Reasoning, Extraction, and sYnthesis) that is designed to help local law enforcement, firefighters and other first responders filter the clutter of available information to achieve situational awareness.

"Too much information can overwhelm a person," said Ed Chow, an engineer who leads the Homeland Security project portfolio at JPL's National Space Technology Applications Office. Audrey, he explained, is an outgrowth of NASA's efforts to extract insights from its 500TB and growing Planetary Data System.

It might be easier to explain Audrey by saying what it is not. It is not a supercomputer that relies on power and access to churn through networks of law enforcement databanks. It is not a collection system with a massive hub of stored information. Rather, it is a lightweight piece of cloud-based software that can run on an ordinary personal computer. And Audrey isn't a single system – it's envisioned as a next-generation architecture that operates on a distributed basis.

"We can make intelligent decisions without knowing everything around the world," Chow said.

Chow envisions Audreys being run at the level of individual local first-responder agencies. A local police Audrey might have permission to access databases on criminal offenders, local traffic cameras, maps and other tools. A fire department Audrey could have business and residential information to give firefighters a more detailed picture of a location where a blaze is taking place, including who lives there or any particular hazards posed at industrial sites. Audrey systems could also access data on wearable cameras and sensors used by first responders, to share observations and provide dispatchers and commanders with more intelligence to support tactical decisions.

A cheap detective

Audrey isn't just for first responders.

The idea behind its design is to make AI that is "less brittle," Chow said, less tied to a single use case. Audrey is intended to provide AI solutions across a range of problems, and teach itself based on the data that it has access to, rather than have programmers create AI software for every problem. It has potential applications for NASA's core mission of space exploration, as well as potential military and intelligence community use cases.

"Audrey knows how to do research," Chow said. In a phone interview with FCW, Chow explained that the system is designed to develop and extend its own capabilities, learning about new tools that are available and tapping into data sources to extract the right signal at the right time.

Local first responders are eager to draw insight and awareness from the fire hose of information available to them, whether it's law enforcement data or public facing social media. More information is coming, especially as new first responder and dispatch technology comes online to expand the communications capacity of local emergency response and law enforcement departments.

"When you start talking about Next Gen 911, you will quadruple when you open up the IP pipes, the information coming at you," said Jay English, head of communications center and 911 services at the Association of Public-Safety Communications Officials, and a public safety official in Seminole County, Fla. "We've got to triage the data at some point," English said at a town hall at the August APCO conference in Washington, D.C.

English was part of the law enforcement team that responded to protests in the wake of the killing of Trayvon Martin in Sanford, Fla., in 2012. With the help of federal law enforcement and vendor partners, officials in Seminole County spun up ways to feed social media straight into the local emergency operations center, to get an operational picture of social unrest in the community, using basic keyword and phrase filters.

"We ended up getting more intel out of Facebook and Twitter than we did out of our detectives on the street, and there were hundreds out there, including FBI folks and [other] folks at the federal level," English said.

Parsing out the garbage

Audrey is designed to do more than this basic kind of filtering and triage. It has facial recognition capacity, but also the ability to read and understand documents, do intelligence fusion and synthesis and communicate inferences to users.

"Think of a firefighter responding to a mission, sitting in front of a terminal, and only the information they need is provided to them. By the time they get to scene they're fully aware of people in the building, or chem or biohazards to go after," said John Merrill, director of the Office for Interoperability and Compatibility, First Responders Group at the Science and Technology Directorate of the Department of Homeland Security. "It will parse out all the miscellaneous garbage and push it aside," Merrill told FCW in an interview at the APCO conference.

Audrey also won't be used to vacuum up private information, according to Merrill. Any deployment that includes tapping into public facing social media or other non-government content will be done in accordance with agency privacy rules, Merrill said.

Initially, Chow said, the idea is to provide assistance at the level of the dispatch center or field commander, but eventually there could be a single instance of Audrey running for every first responder in a department, providing customized, targeted intelligence and advice in natural language via an earpiece.

The Audrey capability isn't available yet. At APCO, Merrill told the audience of public safety communications officials that DHS was looking for local partners to test Audrey in the next eight to 16 months, initially on static or archival data. A live Audrey could be running in two years.