USPS, State and the White House, during recent breaches, had golden opportunities to stalk adversaries.
Some recently hacked agencies let attackers stay inside their networks for a bit before booting them out. Likely, there was a method to this madness – and it's called the “honeypot” trap.
While no agency wants to be breached, in some cases, intrusions provide rare glimpses into an adversary's modus operandi – glances critical for preventing further damage.
This fall, the White House, State Department and U.S. Postal Service each deliberately delayed fully squelching malicious activity after suffering a data breach. USPS has acknowledged using the honeypot technique, after detecting an intrusion in September. State officials said significant planning occurred before totally pulling the plug on systems connected to the Internet, after an intrusion weeks before. Obama administration officials declined to comment on White House incident response.
Any agency might use a compromised system as a honeypot, said Jasper Graham, a 15-year National Security Agency veteran, who worked with U.S. Cyber Command and intelligence agencies to contain network attacks.
“They are going to do things in a controlled manner to understand the adversary," he said. "They are by no means going to put sensitive data at risk. There’s no sense in giving up the crown jewels, just to run an experiment.”
Graham said he doesn’t have inside knowledge about the use of honeypots in the three specific cases.
Let the adversaries break out all their sophisticated malware and poke around in government files for a few weeks – then document every technique they use. Sound dangerous? It can be, some former federal investigators say.
The counterintelligence work can only continue for so long. Tim Ryan, a former FBI special agent who oversaw cybercrime cases, said the question any agency asks itself when deciding how long to keep the hacker inside is: “Can we mitigate the risk to people, property, assets without the bad guy finding out that we know about him?”
Sometimes, the answer might not make everyone happy.
Lawmakers at a House hearing last month berated the Postal Service for not telling employees their Social Security numbers were stolen, until two months after unauthorized activity was detected.
“The secret squirrel stuff – we have to figure out how sophisticated these people were and what information they’ve got – that doesn’t fly,” said Stephen Lynch, D-Mass., ranking Democrat on the House Oversight and Government Reform's subcommittee on the federal workforce, which held the hearing.
He added, "If we go with your plan, a U.S. government agency could have the Social Security numbers for all its employees compromised and you’ll decide based on your own interests when the employees will be notified.”
Agency officials said they held off on coming forward until Nov. 10, because communicating the threat to the public or disinfecting systems could do more harm than good.
"It was critically important that the adversary not know that we were watching their activity," said Randy Miskanic, incident commander on the case and the USPS secure digital solutions vice president. "Any premature leak about our remediation steps might have caused this adversary to cover their tracks or take countermeasures that might have further harmed our network."
Ultimately, it was determined that payroll and other personal data on about 800,000 USPS employees was stolen. The weekend of Nov. 8, the Postal Service shut down and then restored certain systems for "a full scale remediation," USPS spokesman David Partenheimer said.
Efforts to suppress abnormal behavior on an unclassified White House network were still ongoing as of Oct. 30, after a breach weeks before. That incident occurred at the same time as the hack that hit State. Nov. 14, the department shut down its main unclassified system to improve security.
How Honeypots Work
Honeypots are good at drawing out never-before-seen "indicators" – or tell-tale signs – of a specific hacking campaign that otherwise would be invisible in another agency’s system.
“If somebody uses a particular, very sophisticated technique to get into one government agency, you want to understand that as best as possible because the fifth stage might be the only thing that’s really detectable,” said Graham, now senior vice president of cyber technologies and analytics for security provider Darktrace. It might be the phase when the attackers, for instance, steal files or copy data.
That intelligence about the hackers’ hallmarks should then be circulated internally within the government, so other agencies can make sure they aren’t under attack, too, he said.
The Social Security Administration, a virtual Candyland for hackers, buys into the honeypot method. In October, the agency began conducting market research on vendors who can produce one that, among other things, automatically alerts staff to trouble and controls connections in a way undiscernible to the attackers.
At some point, an agency might need to notify the public, which requires another cost-benefit analysis.
Comparing busting hackers to taking out a narcotics ring, “every day that drug cartel exists, it is selling and producing more drugs. When do you say, ‘We’re done. The wiretaps are coming down. We’re going to arrest people?’” said Tim Ryan, managing director with security consultancy Kroll's cyber investigations practice. He was not citing any particular hacking case.
“We don’t want to just successfully prosecute very bad crimes,” he added. “We want to prevent as much crime as possible.”
But immediately disclosing a compromise could show the hackers your hand.
If the agencies "just came out on day one and said, 'We have been breached,' then the adversary is going to cease and desist all operations," Graham said. "And if they are in there, and they have the ability to either destroy or remove all their tools, they are going to, and so all of that evidence and all of that understanding just goes out the window."
Another use of the pot: prime the breached system with bogus government files.
”You want to consume his or her time,” Ryan said. “There’s only 24 hours in a day. If you tie up your adversary with a lot of meaningless stuff, that’s a way of neutralizing the adversary.”
For example, the U.S. government could turn the tables on a foreign adversary who tries cyberspying during trade talks.
“Feed them false information that benefits us during the trade negotiation,” Ryan said. “You are using your computer system as a double agent.”
An interesting suggestion – but again he’s not talking about any specific incident.
When Does the Public Find Out?
After fessing up about a hack that occurred months before, how do you explain that time lapse to the public?
“At the very end of the incident, we kind of want to be honest with ourselves and ask, ‘Why did it take us so long to notify and was it justifiable?’” Graham said. “There should be some level of transparency that kind of holds people accountable that says – there are bona fide reasons for doing this.”
For now, all that State officials are saying is there was a lot of deliberation before disconnecting systems.
“We conducted a full inventory of the malicious activity to better ensure the effectiveness of our remediation activities,” a State official told Nextgov. That is more than the department revealed after the email shutdown.
“Our priority is to implement effective security countermeasures,” the official said. “The scheduled outage was part of our remediation strategy and was executed after a thorough planning process.”
White House officials would not comment on their recent breach, but said, in general, many factors are weighed before going public about a hacking attack.
“For example, if law enforcement officials advise that notification could tip off the adversary and cause them to hide deeper in the system, then we often hold off until we can prevent that from happening,” a senior administration official who requested anonymity told Nextgov.
It seems like agencies are just hanging the “Nothing to See Here” sign, but what’s happening internally can take six months to really understand, Graham said.
Agencies usually are not slow to disclose “out of embarrassment,” said Patrick Howard, former chief information security officer at the Nuclear Regulatory Agency. His own agency was hacked multiple times by suspected foreign powers, as reported earlier, but the incidents predated his time in office.
“You are trying to really identify where they are coming from, what their intent is, what they’re after and those are some of the real drivers for not shutting the door on them immediately,” said Howard, now a consultant at Kratos SecureInfo. But “there is some risk involved there and that’s why amateurs are not allowed to do it.”