DHS Launches Third Biometric Rally to Assess How the Tech Works on Crowds

The Homeland Security Department will soon bring together industry vendors and volunteers willing to share their biometrics to collectively test and accelerate emerging automated biometric technology systems that could one day be used to support its vast operations. 

The agency unveiled plans to host its third Biometric Technology Rally in the fall and is accepting applications from interested participants through April 30. Though past rallies only tested the tools on individuals one by one, this year’s event will gauge the systems’ abilities to solely identify groups of people who opt-in, and avoid bystanders who do not.  

“We're doing this because there's a broad commercial market and we want [companies] to make better technologies for us, but also for other users potentially using these technologies,” Arun Vemury, director of the agency’s Biometric and Identity Technology Center, told Nextgov this week. 

The agency and its subcomponents already use facial recognition and biometrics to help verify certain travelers at various U.S. points of entry (and beyond) and there are ambitious plans in place to expand those technological tools and deployments over the next decade. Vemury explained that through the rallies, federal officials are essentially setting a high throughput use case with specific performance targets and inviting potential partners to innovate and assess how their products work on heaps of volunteers who agree to share their biometric information. The first rally launched in 2018 and tested 11 face and iris recognition systems. In 2019, the agency expanded its participants’ reach to include both camera systems and biometric algorithm developers. Last year, 15 companies were selected from more than 48 submissions to participate. 

While the agency is still inviting providers of both biometric matching algorithms and biometric systems to participate in 2020, this year’s event will not be limited to testing the budding technologies on individuals one at a time. Instead, Homeland Security is shifting to evaluate systems’ performance in much more realistic conditions, such as crowded environments, where they’ll need to make identifications within groups of people and only collect information from those who opt-in. 

“Whether this is at a stadium or an airport, usually people don't wait one at a time to go through—they usually go through as a family or as a group of friends. So one part of it is that we need to make sure that these technologies fit the way people actually behave in real life,” Vemury said. “Another part of this is in these high throughput spaces, you’ll usually have a lot of people. Some people may want to use the biometric system, and some people may not. So we're also evaluating the systems on how well they collect only the people who are in front of the camera and are supposed to be captured—and not capture people who are off to the side, like bystanders.”

The third rally, like those prior, will be conducted in late September and early October at the Maryland Test Facility in Upper Marlboro, Maryland. Vemury said the selected participants will have about six by eight square feet of space to implement their biometric technologies and test them on “several hundred volunteers” who will have gone through an informed consent process to fully grasp how their personal information will be used. 

“They understand what we're going to do with it, that it’s used for research purposes, that it's not going to any sort of government system, and we apply all the protections that are associated with human subjects, protocols or human subject testing,” he said. 

Vemury also noted that the term “bias” has a different meaning to lawyers than it does to computer scientists, but in that light, the agency will be evaluating what it calls “demographic differentials and performance,” or how the systems respond and characterize people based on certain aspects of their looks, such as race. During the tests, the agency will also assess how fast the systems perform, how long it takes people to leave and enter the space, and it’ll also review how the volunteers actually respond to the technologies. 

“So we're not just talking about your match rates,” Vemury said. “We're talking about the considerations you really need to think about if you're truly trying to deploy these technologies well.”

Homeland Security does this work through cooperative research and development agreements, or CRADAs, with the industry participants. Vemury said through testing many companies’ offerings all at once and providing simultaneous feedback informed by real-world scenarios, the agency is able to help steer the development of new, relevant capabilities. At the same time, companies get a chance to see how their systems actually perform in realistic circumstances and also receive data and feedback so that they can refine and enhance their technologies. 

“We've had some companies come in and say that they deployed something that didn't work really well, they made changes to it, and then they were able to win new business somewhere because they made changes from what they learned at the rally,” Vemury said.