The DoD Centralized Cyber Command and the Challenges Ahead

June 26, 2009

Major kudos to the Department of Defense for creating the DoD Cyber Command last week. Centralizing the defense of the DoD’s war against cyber crime and creating enforceable standards will be a huge step forward in protecting the IT backbone of our physical defenses.  The challenge is huge, with one report citing over 100 foreign intelligence operations bombarding the DoD network with millions of exploit attempts per day.   There is also a cost factor – human costs as well as dollars – with the DoD recently reporting that they spent over $100M remediating compromised machines over a six month period, and that 1,500 machines had to be taken offline.

We have had the opportunity to work with various elements of the DoD over the past several months, and seen firsthand what will be at least part of the agenda of this new Cyber Command. 

The first challenge is continually ensuring the security readiness of every endpoint machine.  Sounds elemental (which it is) but given the diversity of equipment, software, policies and geographies represented by the DoD, this is no small task.  This includes ensuring that the defensive software and standard configurations and policies are enforced on every machine, every day.  The defensive software must be properly stood up, properly configured, and checked to detect if it has been compromised in any way.  I have not served in the military, but I believe that would fall under the concept of establishing and protecting the perimeter.  While it sounds elemental, it is often the first failure point in a defense in depth strategy.

The second challenge is keeping endpoint machines free of applications that directly or indirectly create vulnerabilities on these machines.  Unauthorized software such as peer-to-peer applications has already been identified as the source of breaches of sensitive data, and even the most seemingly innocuous software must be identified and removed from machines on an ongoing basis.  And applications must be removed surgically – not through uninstall programs – to ensure that there are no malicious payloads left behind.

The third challenge is situational awareness and response to the evolving attack vectors bombarding the DoD networks and machines.  The networks are a focal point and are continuously hardened, but the real fight may be at the endpoint.  The expectation must be that attacks will come at critical times at critical places, and that these attacks will be engineered to evade signature based protections.   The DoD machines must be able to identify such an attack in real-time, analyze the threat level and root cause, and remediate it as quickly as possible.   The remediation must be situational, holistic, and surgical, and return the endpoint machine to working order without rebooting or re-imaging.  All of this would likely happen in an environment where cyber experts will not be at the ready and there will be no time to do analysis and research to write a new signature or a remediation script.

At this point those of you who know our product are likely concluding that this is a thinly veiled marketing message, as these three challenges fit neatly into what our product can deliver.  It is actually the opposite, as these challenges have emerged after numerous conversations with members of the defense and intelligence communities.  After showing what our product can do, the feedback from these meetings is represented in these challenges, so they naturally fall along the lines of our capabilities.  DoD audiences certainly like the idea that Triumfant will enforce configurations and policies on a daily basis, returning every machine to audit readiness every day.   We are already on the job at the Pentagon identifying and removing unauthorized software from approximately 12,000 machines (see the case study).  And the collective eyes at DoD briefings get pretty large when they see our ability to detect, analyze and remediate a malicious attack in three to five minutes without the need for prior knowledge of the attack or the need to write a script or re-image the machine.

What we see consistently in our dealings with the DoD is need, and we see multiple organizations working to address those needs, and we therefore see the benefit of having a centralized command for cyber security.   It is clearly the right strategy, but as in all things, success will be measured by results and not concepts.  There are a lot of moving parts, diverse requirements, and multiple initiatives within any one branch of the DoD, much less the entire DoD structure.  So bringing all of these stakeholders together will be a challenge.  But it is certainly a noble undertaking and one that is critical to the protection of the country, because advanced warfare technology is quickly neutralized without the computer systems we rely on for command and control.


Updating the Worldwide Malware Signature Counter to Keep Pace

June 23, 2009

This morning we updated the Triumfant Worldwide Malware Signature Counter to adjust the count upward and to accelerate the rate at which it increments to keep pace with what Symantec is reporting for their signature count. When we introduced the counter we made every attempt to model the rate of increment to the data presented in the Symantec Global Internet Security Threat Report, and we have been tracking the Symantec signature counts to ensure that the counter is as accurate as possible. 

It should be noted that the counter started the year at roughly 2.6 million and has just passed 4.2 million.  This is noteworthy because the 1.6 million new signatures is the equivalent to the number of new signatures Symantec reported for all of 2008, and we have not yet hit the halfway point in the year.  Given that a graph of the signature counts appears to be geometric rather than linear, we expect the rate of increase to accelerate and raise that delta for 2009 to on or about 4 million signatures. For the second year in a row, the number of new signature for just this year will surpass the previous combined total number of signatures.

When we had the idea for the counter, we were careful to apply some science and statistical analysis to the process because we wanted to be fair and conservative.  The counter was never meant to be about hype – it was built to provide a visual representation of the unsustainable nature of the signature model for defensive software.  That is why we are updating the counter in our attempts for accuracy, and we will also adjust the numbers down if we see that it begins to exceed the reported numbers.   

The point of this exercise remains the same.  Companies and government agencies must look beyond signature based tools for endpoint protection, as the sheer volume of new attacks makes it impossible for these tools to protect organizations from malicious activity.  Many new approaches to endpoint security such as behavioral analysis and heuristics still require previous knowledge of the attack to be really effective.  Triumfant is the one tool on the market that can detect, analyze and remediate a malicious attack without any prior knowledge of the attack.  No waiting for a vendor to create a remediation script or signature.  Remediation is minutes not hours or days.  And as the counter illustrates, every day your organization does not look beyond signature based tools, the problem only grows worse.

I would also note that the counter is not meant as a direct poke at Symantec.  We use their numbers because of our respect for the capabilities of their research team and because they graciously make their numbers public.  Other products that use signatures may have differing counts when it comes to signatures, but the basic problem still exists for those solutions. 

I have heard a lot of complaints from IT security people that say there has not been much new in the way of technology lately.  I would respectfully disagree and would invite you to have a look at the Triumfant solution and get a feel for how it works via a video of our Three Minute Malware Challenge from RSA.  Words don’t do the product justice, so the video will provide much deeper insight.   Then give us a call and let’s talk about what is keeping you up at night and allow us to show you how we can help.


Stopping Stupid Part 2 – There is No Cyber Santa

June 22, 2009

A recently released study by Verisign states that 88% of American Web users are unable to spot a phishing Web site. This was done by showing sample users side-by-side comparisons of legitimate and companion phishing sites and asking the user to point out the malicious site. The study is a sterling example that the CLF problem (carbon based life form) is still the single biggest impediment to cyber security.

I have a bit of a cynical streak and therefore normally do not fall on the side of education to stop things like cyber crime.  But it is clear that Web users need some pragmatic education because it really makes the job of IT security difficult when users willingly walk into malicious activity.  I use the term “Stopping Stupid” in a previous post, but if only one out of every eight people can spot a phishing site, then it is clear that endpoint security education is clearly needed before we can place blame solely on the users.

When my own kids began to surf the net, I was careful to educate them on what they would encounter.  For example, I made sure they knew that there was not some benevolent force on the World Wide Web that existed to give them a free iPod just for visiting their site. We talked about that if something seemed to good to be true, it probably was and that if they ended up on a page they did not expect to see, they should immediately stop. Simple stuff when they were younger progressing to many of the basics defined in the Verisign study now that they are in their teens. 

But it is unreasonable to expect that all people grow up in a house with a cynic working in the IT security market.  Few Web users know what the padlock symbol means or why the colors change in the security status bar.  Many still believe there is a Cyber Santa that really does want them to have a new notebook PC.   We hand people a computer when they show up for work and in most cases, no one shows them the basics of physical security or what to look for when doing simple tasks on the Web.   Then when we see stories such as the latest Nine-Ball mass infection, we wonder how such a thing could happen, but we are at least partially culpable for sending the lambs to the slaughter. 

So as much as it goes against my cynical nature, we in the IT security market must take the steps to educate the army of CLFs that access the Web jungle daily.  It is no fun to tell them there is no Santa Claus, and we will never get 100% on the Verisign test, but we do need to do a better job of at least teaching the basics such as the simple signs of a phishing attack.   We should offer basic education when we hand over their new computer, and there should be constant reminders of the fundamentals.   The bad guys are getting smarter, so we must make our users smarter.  After all, at 88% only one of Santa’s eight reindeer would spot a phishing attack.


A Wide Eyed View of Real Time Malware Detection and Remediation

June 18, 2009

The last several weeks have been full of opportunities to show the Triumfant product to prospects in a variety of markets. When we do a deep technical dive, our CTO and founder Dave Hooks often comes along to do the demonstrations during which I get the opportunity to watch the people at the table to see what gets their attention. Lately I have been seeing some interesting body language: eyes get larger, blink rates increase, people lean across the table open mouthed. Permit me to tell you what prompts these behaviors.

For background, Dave runs two virtual images on his machine – one for the victim endpoint machine and one for the Triumfant server. He starts on the victim machine and executes a piece of malware (or a “critter” as he likes to call them) he collects directly from the Web. Dave will walk the audience through the steps of how Triumfant does its job from the agent recognizing suspicious behavior on the endpoint machine, through the analysis of the detected changes on the Triumfant server, to the creation of the remediation and ultimately the remediation on the endpoint machine – all without human intervention. The audience can see how Triumfant goes through the stages of the detection, analysis and remediation, and actually view each of the granular level changes that it detected. Within minutes the administrator console shows the full analysis of the problem and the synthesized remediation, which can be executed through the click of a button. Dave will start the remediation and return to the endpoint image so the audience can visually see the machine remediated without the user’s intervention, without restarting and without the need to re-image.

In the first part of the demo, Dave uses malware that Triumfant has never seen to show the ability of the product to see malicious attacks for which there is no signature. The audience gets hooked early as Triumfant immediately detects the attack and their interest ratchets up quickly when he shows the depths of the analysis returned by Triumfant and its ability to see the entire breadth of the attack on the endpoint machine. The visuals of seeing the analysis makes the connection that Triumfant is uniquely capable of building a remediation on the fly because if it sees all of the changes, it can fix those changes. Once he shows the created remediation, the wheels really start to turn, and the more experienced folks are clearly connecting the practical dots of what they are seeing and how it can plug myriad gaps in their endpoint protection.

In the second part of the demo, Dave executes a rootkit which quickly burrows into his machine and, as rootkits will do, promptly cloaks itself. Triumfant sees the changes made to the system, begins the analysis and sends the results to the console, noting all of the registry keys that were changed, the files that were installed, and the executable itself. Triumfant builds the required remediation and because it divides the entire remediation into subgroups, Dave takes the opportunity to show off a bit. Instead of killing the entire rootkit and all of the collateral damage at once, Dave executes the part of the remediation that uncloaks the rootkit. Dave switches back to the victim machine so the audience can actually see the rootkit appear in Task Manager in real time as Triumfant uncloaks it. That is when I know to look at the audience, because that is when the eyes get the largest.

With a product like Triumfant that really is quite different from any other product on the market, things like Word, PowerPoint, and Web pages do not do it justice. The demo is really where you can fully grasp the depth of what it is doing and how it can detect and remediate an attack in three minutes. Dave’s “Three Minute Malware Challenge” demo he did at RSA has been captured on video and is available on YouTube so I invite you to see for yourself. Of course, you can always contact me at Triumfant and I will be happy to set up your own demonstration so you can get some of that wide eyed feeling for yourself.


Remediation Without Re-Imaging

June 16, 2009

One of the benefits of the Triumfant solution for endpoint security is that it builds a situational remediation for the problems it detects.  These remediations not only address the source of the problem, but also clean up all the collateral damage of the attack.  This unique ability has an interesting associated benefit of eliminating the need to re-image the infected machine.

The analytics of Triumfant Resolution Manager do a remarkable job of finding the boundaries of an attack and all of the changes to the machine within those boundaries.  Triumfant continually scans endpoint machines for over 200,000 attributes to identify all of the changes in that machine.  When it detects an attack it uses dependency walks on the affected files and temporal analysis to ensure that it sees all of the related ripples.  In this way, Triumfant sees the attack in its complete form and knows everything that was changed in that machine.  While other tools simply kill the offending executable, Triumfant builds a surgical remediation that restores the machine back to the pre-attack condition, changed attribute by changed attribute. No roll-backs or logs, no need to reboot.  And best of all, no need to re-image.

Think about the time and money that would be saved by eliminating the re-imaging process.  I know that in these days of virtual machines the process is not that complicated from a technical point of view, but it is still an intrusion on the user and still uses up valuable people time on the IT staff. 

In my time in the security industry, I have found that most CISOs and IT security managers look for ways to move their teams from the defensive, reactive work of fixing problems to the offensive, proactive work of protecting the organization.  Eliminating the need to write a remediation for an infected machine and further eliminating the need to re-image the machine would seem to be two pretty good places to recover people time from the reactive side of the ledger.


Securing the Software Supply Chain

June 15, 2009

I just finished the draft of a white paper on the software supply chain and how Triumfant addresses some of the problems presented in that chain.  The white paper explores how to protect organizations from the subversion of third party software to create security problems in the form of exploits to be used later for malicious activity, or actual malicious code baked into the software.  The growing global economy, the demand for new applications, and the pressure to get those new applications to market quickly are all factors that are driving the problem.  The research brought into clear view that we are in an interesting conundrum because as security threats become increasingly complex and persistent, we are going the exact opposite way in our development processes and methodologies. 

Think about the gold rush to build iPhone applications – just how much time do you think was spent on securing those applications?  The software being developed today is neither designed nor built to be secure.  Today’s developers have had very little exposure to secure development methodologies, and therefore do not integrate sound security practices into their coding and engineering.  Rapid development, iterative design, and the growing use of mash-ups all point to the fact that there can be presumption that security is baked in.  Combine this lack of security rigor with the overt threats of baking exploits or malware into an application and we have a serious security problem.

So back to the conundrum – as the cyber criminals have become more organized and find new and innovative ways to attack our systems, we are countering by rolling out software across our computer populations that is increasingly less prepared from a security perspective.  After all, how much easier is it for a cyber criminal to subvert application software that is willingly distributed by the targeted organization rather than go through all the problems of infiltrating machines one at a time?

Up to the point where I started this paper, I was focused on the more direct acts of infiltration and had not fully considered the implication of the software supply chain.  I actually was pointed that way by someone steeped in IT security who, after getting the three minute malware challenge demo at RSA, noted that Triumfant was uniquely capable of addressing much of the software supply chain issues because of its change detection capabilities.  After my research I have a better appreciation of the problem and now understand that the software supply chain must be considered in any defense in depth strategy.  And not just the normal processes of testing applications before they are deployed, but the vigilance of testing applications post-deployment.  There was actually a great article in PC World about how DISA continues rigorous testing post-deployment. I would also note that the subject of the software supply chain was noted in the White House Cybersecurity Policy Review.

I will address how Triumfant addresses this problem in a future post and provide the link to the white paper as soon as it is ready for prime time.


A Visceral Lesson in Endpoint Security

June 12, 2009

Elinor Mills of CNET News had a great article yesterday called “Look Ma, I created a botnet!” that had Elinor going through the paces of infecting and controlling a PC through various malware including a botnet.  Two things jumped out at me from her story.

First, I have done a lot of writing about the depth and breadth of the malware menace by using the numbers from the Symantec Global Internet Security Threat Report.  The data from this report is the basis for our Worldwide Malware Signature Counter on the Triumfant Web site.  In her story, Elinor provides some great data from McAfee’s Avert Labs that adds yet another set of sobering statistics to the conversation. According to the article Avert Labs:

“…sees more than 400,000 new zombies a day, 4,000 new pieces of malware a day and 1.5 million malicious sites a month. There were 1.5 million pieces of unique malware last year and McAfee predicts that number will rise to 2.4 million this year.”

Like the Symantec numbers, these figures are staggering, but sometimes I fear that executives that look at security budgets and endpoint protection cannot grasp their meaning.   IT Security is a funny business where success often brings a sense of false security with those not savvy about the depth of the threat.  Somehow in spite of a deluge of sound statistics, those under budgetary pressure allow themselves to fall into the mind trap of “I have not had a major breach, therefore there is no real threat, therefore I am overspending on security”. 

Which brings me to my second point: I wonder if those same executives would think that way if each were able to take the same malware test drive as Elinor.   My guess is that they would walk away with a completely new outlook on the world and be able to better put the statistics like those from Avert Labs into practical context.  Elinor lives in this world continually and reports on massive breaches almost daily and she found the experience “sobering”.  Hats off to the McAfee folks for putting together such an eye opening demonstration – I am sure it has helped them close more than one contract.  But it may serve to do all of us in IT security a collective favor by providing a very visceral lesson to those who doubt the need for endpoint security.


Putting Teeth Into Security Policies Via Continuous Enforcement

June 10, 2009

I came across an article by Kelly Jackson Higgins of Dark Reading yesterday that reported that “Most Employees Disobey Security Policies”.  While the study used in the report was commissioned by IronKey and therefore had a leaning toward policies around removable media, there were plenty of other policy violations cited in the study.

Of the 1,000 employees polled, half said that corporate security policies are ignored by employees and management.  Violations listed included: turning off firewall and other security settings on their machines, social networking, using web based personal email on work machines, and password sharing.  Higgins adds that 70 percent of end users don’t think their organizations have a policy forbidding their turning off security settings (including a host firewall) on their work computers. And 21 percent say they disable those security settings, up from 17 percent two years ago.”

Allow me to add some of my own analysis:

  • I think that half is low because there are likely some that did not even know there were policies. 
  • A telling phrase in the summary is the term “and management” – because if management ignores the rules, then it is a given that the rank and file will go along.  
  • I will refrain from asking why anyone would think that turning off security settings would be a good idea, because we all know that people are still the biggest X-factor when it comes to endpoint protection.  I had a senior security exec at a government agency refer to such things as CLF problems – carbon based life form.

Policies in and of themselves solve nothing.  Without continuous enforcement they are doomed to failure as indicated in this report. And by continuous I mean relentlessly continuous because the enemies – ignorance and incompetence – are equally relentless. 

That is where Triumfant is so well suited for enforcing policies and configurations.  It continuously scans for changes on endpoint machines, and if a detected change violates a policy or configuration, it builds a remediation to set the machine back to compliance.  It does this every day for every machine and continuously meets the challenge of the CLF problem because it matches the problem with equal relentlessness.  If the user changes a setting, it changes it back.  If the user does the same thing the next day, it sets it back.  This can go on for days, but eventually the human normally relents. 

At a minimum, your organization knows that it starts every day with the endpoint population in compliance with security policies.  And at the end of the day, Triumfant will put back all of the slippage, returning you to the same place for the next morning.  That is continuous enforcement and that is what makes policies effective.


Detecting the Work of the Maliciously Intended Insider

June 8, 2009

The recent arrest of a retired State Department worker and his wife accused of spying for Cuba for 30 years brings into focus one of the other great capabilities of Triumfant’s technology.  Because Triumfant can see all of the changes on an endpoint machine as well as the work done to cover up those changes, Triumfant Resolution Manager is uniquely capable of detecting the work of the maliciously intended insider. 

In this case, as well as the case for others like Robert Hanssen, the methods for transferring information was very “old school” and did not represent a deep grasp of technology.  Walter Kendall Myers and his wife, Gwendolyn Steingraber Myers, often relayed information to their Cuban handlers by exchanging shopping carts at the grocery store and Robert Hanssen was arrested after leaving a package under a wooden footbridge in a Northern Virginia park.   Such techniques do not lead one to believe either party was terribly computer savvy, but begs the question of the amount of damage done if they had cyber expertise.  It also, unfortunately, begs the question of what activity is potentially being done by those with cyber expertise that is going undetected.

Maliciously intended insiders are a real threat to organizations because the majority of defensive software and endpoint protection is created to prevent intrusions from outside the organizational walls and based on previous knowledge of the attack.  But malicious insiders work from a position of trust and introduce the human factor that normally takes their work outside of the paths of known attacks.  They may directly pull information from confidential or sensitive sources and directly funnel that information out.  Or they may place maliciously intended programs on machines such as key loggers to collect information.  Or they may make subtle changes to machines to make them vulnerable to the eventual installation of malware.

The common factor in this activity is change.  The maliciously intended insider must make changes, even subtle ones, to an endpoint machine to perform their activity.  Triumfant can of course detect change, as well as detect the attempts to cover up the evidence of change.  The fact is, there is almost no way (not that anyone can tell us, anyway) of making changes to a machine without us being able to see those changes.  And because the work of the insider would change an endpoint machine into a state that would be anomalous in comparison to other machines, Triumfant would not only detect the change, but would flag it as a problem and then do the analysis to look for other changes that it could logically associate with the detected change to provide a complete picture of the activity. 

Only a tool with the depth and breadth of scan scope and the ability to quickly identify changes can perform these functions.  Which narrows the list of tools that fit that requirement to one: Triumfant.  We talk a lot about the ability of Triumfant to see the malicious attacks that other signature based tools miss, and we have also discussed the ability of Triumfant to protect the endpoint environment from acts of ignorance and incompetence by continuously enforcing security policies and configurations.  But the protection of your company or government agency from threats on the inside is also a critical functionality that Triumfant brings to the table.  I would also add that the President’s new cyber czar needs to ensure that this topic is front and center as he or she begins to address the issues in the White House Cyberspace Policy Review.


Action Instead of Rhetoric – It Can and Does Happen

June 5, 2009

The response to Triumfant CEO John Prisco’s less than laudatory reaction to the White House Cybersecurity Policy Review has been interesting to watch.  To John’s credit, he did not fall into line and unilaterally sing the praises of the document or the President’s speech, and his was one of the first voices in the IT security market to express practical concerns over the review.  One of John’s primary concerns was a lack of urgency in regards to taking some real and concrete action sooner rather than later given the depth of our current problems and vulnerabilities.    

One good example of action over rhetoric was made public Wednesday, when the National Institute of Standards and Technology (NIST) announced that they were teaming with the U.S. Department of Commerce’s National Telecommunications and Information Administration (NTIA) and National Institute of Standards and Technology (NIST) to work with the Internet Corporation for Assigned Names and Numbers (ICANN) and VeriSign on an initiative to “enhance the security and stability of the Internet”.  Specifically, the initiative is working to bring a new security technology called Domain Name System Security Extensions (DNSSEC) into use to address known vulnerabilities in the DNS protocol.  The working group plans to deliver an interim approach to DNSSEC by year end and continue to collaborate with U.S. agencies and private sector to further refine the technology going forward.

There is a lot of good in this little announcement.  One, they are addressing – not studying or measuring or debating – a real problem.  Two, this is a collaboration of multiple government entities and the private sector, proving that it can be done without dissolving into Lord of the Flies.  Third, they are moving forward to deliver something sooner rather than later, and will refine as they go.  It appears they have a solid plan with dates and deliverables, and have the proper commitments in place to deliver to that plan. 

I have heard John say this more than once this week and I believe he is dead on right: we have ceded the luxury of debate and we need to move quickly to action.  In regards to U.S. cyber security, the problems we face are deep enough that we don’t need to waste time measuring their depth before we start to fix them.  Action is required and required sooner rather than later, which is why John rightfully asked why the review was announced without a cyber czar selected and ready to get started.  Hats off to NIST and the others behind the DNSSEC initiative, as they are moving forward at a time when more walk and less talk is the order of the day.