Researchers: Navy SEALS' new brain injury test has high false-positive rate

Detecting if a soldier has a concussion caused by roadside bombs has been one the toughest tests clinicians face in Afghanistan and Iraq, but the elite Navy SEALs fighting force believes it has found a better way to screen for brain injuries.

In May 2008, the Defense Department mandated the military services use an application called the Automated Neuropsychological Assessment Metrics tool as the standard way to assess the extent of brain injuries. But in August, the Naval Special Warfare Group signed a contract with ImPACT Applications Inc. of Pittsburgh to use the Immediate Post-Concussion Assessment and Cognitive Testing test. The group said preliminary results indicate the new test improves diagnosis and treatment.

But some university researchers reported the ImPACT test has the highest rate of false positives -- indicating a brain injury when there is none -- of the three tools physicians use in sports medicine, a field that has a long history in diagnosing and treating concussions. The Defense and Veterans Brain Injury Center is evaluating ImPACT for use throughout all three services.

The Naval Special Warfare Group, which includes the SEALS and combat craft crew headquartered in Coronado, Calif., still uses the older Automated Neuropsychological Assessment Metrics test, but it is analyzing the ImPACT test to see if it can augment the older tool, said Lt. Catherine Wallace, a spokeswoman for the Naval Special Warfare Group.

The ImPACT test, unlike its predecessor, can be taken online, even in the remote and hazardous areas where the roughly 2,000 SEALS and combat crew operate. "ImPACT is designed specifically for the detection of concussions; does not require a trained neurologist or psychologist; and because it is Internet-based, can produce nearly instantaneous pre- and post-[traumatic brain injury] comparison results," Wallace said. "We can quickly assess if an operator has suffered a head injury that requires him to be removed from the fight temporarily, or sent to a medical facility for further testing."

The value of the contract with ImPACT applications is $35,000 for one base year with each of four additional option years costing $8,000, Wallace said.

She pointed out the Naval Special Warfare Group also added 17 questions to the ImPACT test to help a corpsman detect post-traumatic stress disorder.

Mickey Lutz, chairman and chief operating officer of Vista Partners, which has contracts to provide Defense with the Automated Neuropsychological Assessment Metrics tool, said an Internet version of the older assessment tool is available. But he cautioned the Internet slows response times, therefore affecting patients' scores. Delays over the Web are about 15 milliseconds, and "concussion management requires millisecond timing measurement," he said. "Also, Web connections can be interrupted or at times not available."

If the SEALS use ImPACT, they need to know it has a high rate of false positives, said Steven Broglio, a professor in the kinesiology and community health department at the University of Illinois at Urbana-Champaign who has experience with brain injuries in sports.

In 2007, Broglio tested ImPACT and two other neurocognitive assessment tools -- the Concussion Sentinel manufactured by CogState Ltd. in Melbourne, Australia, and the Concussion Resolution Index developed by New York-based Headminder Inc. -- on 118 student volunteers, none of whom had a brain injury. ImPACT concluded 38.4 percent of the healthy students were impaired, Broglio reported in a paper. The Concussion Sentinel program reported 21.9 percent false positives, and the Concussion Resolution Index issued 19.2 percent wrong diagnoses for concussions.

The Defense and Veterans Brain Injury Center has been studying the three applications, along with the automated neuropsychological assessment, since September 2008. Its final report is due January 2011. Broglio said Defense must take into account the false positive rates, but until better tests are devised, these tools "are better than nothing."

Dr. James Sterling, a sports medicine concussion specialist on the staff of Texas Health Presbyterian Hospital in Dallas, said ImPACT is the easiest to use and the simplest to administer of all the assessment tests. Sterling, who served as head physician for the 2006 U.S. Winter Olympics, also works with high school athletes in the Dallas area, said the extensive database ImPACT has developed helps assess brain injuries in cases where an athlete had not taken a baseline test that is typically used to compare results.

Officials at ImPACT declined requests for an interview, and referred Nextgov to information on the company's website, which discusses how the test works, to read clinical papers and view a list of clients.

ImPACT references National Football League teams as clients. In 2007, ESPN: The Magazine reported Dr. Mark Lovell, co-founder of ImPACT and its chairman, served as director of neuropsychological testing for the NFL. Christopher Randolph, professor of neurology at Loyola University Medical Center in Chicago and former team neuropsychologist for the Chicago Bears, told ESPN he considered Lovell's dual role a "major conflict of interest, scientifically irresponsible."