I offer Dave’s analysis with the full disclosure that it is based solely on detailed analysis of the attack, and that we had no firsthand exposure to the attack itself.  Dave broke his analysis into four parts: initial detection, diagnosis, knowledge base, and remediation, showing how Triumfant can identify an attack without prior knowledge, diagnose the attacks and correlate all of the changes to the machine associated with the attack, and build a situational and contextual remediation to return the machine to its pre-attack condition.

———-

Analysis of Operation Aurora

Initial Detection

Operation Aurora creates several service keys during three specific steps: execution of the dropper, the first stage of installation, and the second stage of installation.  Some of these keys are subsequently deleted but at least one is persistent.  The appearance of one or more of these keys would trigger the Triumfant agent’s 30 second scan cycle for markers of malicious activity, resulting in the agent requesting permission to execute a fast scan.  The Triumfant server would respond within seconds, green lighting the scan.  The agent would then capture the state of the machine immediately after infection and send the data to the server for analysis within 3 minutes.

Diagnosis

The Triumfant server would receive the snapshot, recognize that is was executed as a result of suspicious behavior, and immediately compare it to the adaptive reference model (the unique context built by our patented analytics).  The result of this comparison would be a set of anomalous files and registry keys.  The fact that the files and keys associated with Operation Aurora have random names would guarantee that they would be perceived as anomalous despite the fact that humans might tend to confuse them with legitimate Windows services.  Further analysis would then be applied to the anomaly set to identify important characteristics and functional impacts.  In this case the salient characteristics would be an anomalous service and a number of anomalous system32 files.

The discovery of an anomalous service would cause the Triumfant server to launch a probe requesting the Triumfant agent to explore the service further.  The probe would contain a list of all of the anomalous attributes found by the server during its analysis.  The Triumfant agent would activate a series of correlation functions to partition the anomalous attributes into related groups.  In this case it would group all of the anomalous attributes related to Operation Aurora.  It would then perform a threat analysis on this group and discover, for example, that it was communicating over the internet.  The results of the correlation and threat analysis would then be sent back to the Triumfant server.

At this point the diagnosis would be complete and the Triumfant server would alert the appropriate personnel that an “Anomalous Application” had been discovered and the data would be available on the console.  It would then be possible for an analyst to view all of the persistent attributes of Operation Aurora as well as the corresponding threat analysis, as well as readily share the data with CIRT and forensics teams.

Knowledge Base

An analyst can save the analysis for an Anomalous Application such as Operation Aurora to the Triumfant database.  This would allow the analysis to be converted into a new recognition filter.  Recognition filters have a number of benefits.  First, they provide a very precise mechanism for storing and sharing knowledge about an incident.  Second, they allow the system to search for any other instances of that particular condition in other environments.  Third, they enable the operator to pre-authorize automatic responses such as remediation should that incident be detected again in the future.

Remediation

If a Triumfant server detected Operation Aurora as an anomalous application, it would have sufficient knowledge of the anomalous attributes to synthesize a remediation response.  This remediation would be custom built to exactly match the attributes of the anomalous application on an attribute by attribute basis.  The ability to create remediations on the fly would enable the Triumfant system to surgically and reliably remove the components of Operation Aurora without reimaging the machine.  It would also enable follow on variants to be addressed without the need for new signatures.

———-

Again, let me state for the record that this is based on Dave’s analysis and not actual “live fire” data of our software responding to an actual attack.  But we are quite confident that Triumfant would have responded as described, detecting the attack and building a situational and contextual remediation.

One line of conversation has been about how APT, specifically the threat component, is not about malware.   Specifically, this addresses the positioning around by the antivirus vendors who continue to perfect their defenses around older attack forms - a process I call “Perfecting the Obsolete” – and look to defend market position by framing APT as malware.  In Mike Cloppert’s blog he notes in a post about the Google incident that the “defense industrial base has been pleading with the AV industry for innovation to address more sophisticated threats and detection resiliency for at least 5 years, likely longer”.  The A/V vendors will continue to characterize APT as malware largely because they have no effective answer, in spite of the wildly inflated claims of their love affair with recently acquired/developed whitelisting tools and prevalence data. 

Eventually the dialogue went to the suggestion (Threatpost blog entry by Scott Crawford and Nick Selby) that APT should be called Advanced Persistent Adversary, ideas similarly expressed by an Andy Jaquith blog post and a great post by Richard Betjlich).  I have no issue with that line of thinking, because I think the nature of the adversary is what has changed IT security so dramatically, not the attacks themselves.  The new adversary is organized, skilled, relentless, opportunistic, and enormously resourceful.  They are also patient, willing to invest time in research and planning to create breaches that get to the information they want without detection and hopefully opening up a long-term conduit for extraction.  Again, it is this change in the adversary that the traditional AV suites have failed to embrace. 

I also agree on the notion that the actual attack process does not have to be exotic or elaborate.  Why spend time going through the trouble of designing and engineering a zero day when there are numerous exploits created and widely deployed through production software?  The time and energy is better spent gaining information about how and what to attack and what to do when successful.  Which of course supports the APA notion: it is the advanced profile and skill of the adversary that makes these attacks so problematic, not the attack. 

Here is where I wade into danger.  The most common thread among of these posts is the notion that there is no one vendor-produced solution for APT (or APA).  Crawford and Selby warn of the vendor “easy button” and Rich Mogull notes on an associated blog entry that “Every vendor who tells me they can ‘solve’ APT instantly ends up on my snake oil list. There isn’t a tool on the market, or even a collection of tools, that can eliminate these attacks.” (emphasis from Mogull). 

We agree.  Let me state for the record that Triumfant is most certainly not a solution for APT nor can we eliminate these attacks.  What we do have is a tool that takes a completely different approach to detection, identifying changes at the granular level to trigger analysis.  And we think this approach has solid applications in detecting APT activities (or the work of the APA).

I will stop here hopefully short of pegging Mogull’s snake oil sensors.  Yes, I am a vendor and yes the company stands to benefit from using APT as a discussion point for selling our product.  But I have talked to enough people here in the DC area engaged daily against the APA that I understand that and effective method of anomaly/change detection is a necessary tool in detecting the work of the APA.  I will go deeper into our detection capabilities specific to APT in a later post.  Until then, I will continue to read the ongoing discussion with interest.

I have been on record as a doubter of these magnificent claims, largely because the tools base a lot of their efficacy on one highly flawed component – the carbon based life form. 

Let me explain.

If you read the vendor’s own materials closely, whitelist and prevalence products cannot block bad things unless you lock down yourr endpoint environment.  I know there are some organizations that have such an environment, but they are certainly not the norm.  So for the rest of the world who are not locked down, these tools can only warn.  And who do they warn you ask?  The end user – the very person who got the machine into trouble in the first place. 

I have a very dear and old friend that told me something that has stuck with me for a very long time:  “remember,” he said with total authority, “half of all people are below average.”  (Note: If you find yourself either thinking too long about that last sentence or find it really insightful, please call your PC support staff and have your admin rights revoked.)  But cynicism is not enough to prove my point.  Luckily, a new study was recently released in the New York Times that provides some real insight into the mind of the end user. 

The article speaks to a study done by software maker Imperva that examined a list of 32 million passwords from RockYou (software for users of social networking sites) that was hacked and subsequently posted on the Web.  Imperva’s research on the data shows that one out of five people use easily hacked passwords such as “123456” and “password”.  I would submit that these types will be the first in line to get to places on the web that are dodgy or fall victim to social engineering.  Gartner analyst John Pescatore has some thoughts about this study from the viewpoint of passwords, but I think the study speaks to the bigger issue of having end users involved with security processes.

I do not think that it is a reach to believe that users who would pay so little mind to their passwords will blithely skate right past any warning from a whitelist or prevalence tool.  Why stop?  I clicked on it, didn’t I? After all, there is a free iPhone waiting on the other side of that warning screen. 

My cynicism is not just genetic – it is founded by years of hard-won experience.  In the 80’s I spent some of my formative years supporting some new wild idea called the Information Center where we placed user friendly (as friendly as any mainframe tool could be) tools into the hands of the end users.  Every Monday morning I spent the first hour of my day resetting scores of passwords of people who simply could not remember their password from the previous Friday.  And I knew easily half had it written on a post-it note on the monitor. 

If you need further proof simply get on any major road during the morning or afternoon commute.  In spite of warnings that texting makes you more dangerous on the road than being intoxicated to twice the legal limit, I spend my drive dodging people who are clearly engaged in critical text conversations.  Shoot, I saw someone this morning with the newspaper opened on their steering wheel.  If these people don’t care about their physical safety, why would we believe that they can be part of the security process on their endpoint computer? 

And there you have my doubts about whitelisting and prevalence tools.  It would be fascinating to do a study on the reaction of users to warnings from such tools to really support my point, and I am confident what the results would show.  After all the proof is all around us every day.  Just ask the 1% of people that use “123456” as their password.

What I really liked about the study was the portion that showed the “detection over time” chart that captured the sliding risk as new attacks are assimilated into antivirus offerings. One of the critical differentiators of Triumfant is the ability to detect malware without any prior knowledge of the attack. This graph shows how much coverage gap exists for a new attack and how long Triumfant would be standing as the first line of defense against the attack.

Posts to our blog that deal with antivirus detection rates such as “Antivirus Detection Rates – It is Clear You Need a Plan B” are consistently the most viewed entries, so I thought this study would prove a popular read.

Conficker made the jump from malware attack to media darling in 2009, finding its way onto the front page and 60 Minutes.  For those of us who work in the general anonymity of IT security, Conficker (aka Downup, Downandup, Conflicker, and Kido) was one of those things that took on a life of its own and rose quickly into the public consciousness.    

To be accurate, Conficker actually surfaced in November of 2008, but its effect really peaked in 2009.  The Conficker Working Group estimates that 9 million to 15 million PCs are infected with the worm.  Costs have been placed at a wide range of numbers with some estimates reaching $9B.  The worm was noteworthy from its use of sophisticated techniques to avoid detection and its ability to morph via commands from a well designed command and control process.  It has been through three iterations, each making it harder to detect and defeat.  It spread rapidly – in May it was reported to be spreading to 50,000 new machines a day - and is widely believed to be the largest worm infection since Slammer in 2003.   

What became almost humorous was the effect it had even when it did not do anything.  When the command and control elements of Conficker would stir, there was rampant speculation as to what it would do.  Conficker appeared to be readying for something big on April 1 and the speculation became somewhat comical as predictions ranged from minor attacks to global Armageddon. Eventually people became to see Conficker in every shadow, with the paranoia coming to a crescendo when Conficker was suspected as the cause when Big Ben stopped just before midnight on March 31.  I wrote then that such blame “makes sense – build a worm, get it distributed to millions of computers worldwide, have it confound the best and brightest of IT security, and then instruct it to stop Big Ben.”

The real lessons of Conficker are many.  First, the worm took advantage of an exploit that Microsoft patched in October of 2008 and many noted that the infection vector was not exceptional, just opportunistic.  The fact that it spread so rapidly and continues to spread illustrates the issues we have in patch management and maintaining the security readiness of endpoint machines.  In spite of all research and recommendations, business and government agencies still take far too long to close well known, dangerous gaps in their security.  Second, the sophistication of the worm, the command and control structure and its evolving nature all are illustrative of the growing sophistication of malicious activity.  Conficker is an attack with the careful engineering of a commercially available application.  Third, the traditional endpoint protections have long been left behind by the growing sophistication of the current attacks.  It took far too long to come up with a viable detection process for Conficker, and even longer to come up with a fix.  Most have to re-image and start over at enormous costs when one considers 9M computers.

Finally, Conficker brought IT security issues to the masses.  Lots of people that never considered the security readiness of their PC began to ask some serious questions.  The size and scope of the infection woke people up to the enormous potential of a mass attack.  And the sophistication was instructive to the public as to how malicious attacks have evolved from the days of the Anna Kournikova virus.   

Conficker is a noteworthy fail because it all started by leveraging a known exploit that savvy malware writers saw as an easy path to a significant infection that will cost the world billions of dollars.  Best of all, Conficker is still out there on a very large number of machines.  Even in its most benign state, the fact remains that someone controls a huge botnet that has the potential to be used for harm.

Simple and logical question, right?  But it led me to think long and hard why attacks happen at a very elemental level.  We in security face this question from the powers that be because they cannot understand that attacks still come even though we have added multiple layers of defense. 

After consideration I came up with three reasons.  For perspective, my reasons are very much endpoint centric and presume the attacks have already made their way through protections on the network level, so this is not a cloud to ground holistic view.  Each reason is based on the assumption that the preceding reason(s) have been fully addressed and the represented gap is closed – each reason stands on its own as a problem.  And I will resist the urge to plug in how Triumfant addresses each gap, but I have noted blog entries that do if you care to read on.

Here are my three reasons:

1. Attacks get through because the machines and the protection software deployed to protect them are not configured to be secure.  The analogy is simple: the most well designed and secure deadbolt lock only secures a door when the deadbolt is engaged.  Too frequently, endpoint protection tools are either improperly installed or improperly configured to perform the tasks for which they are intended, so attacks make it through.  For how Triumfant addresses the configuration management gap see “A New Approach to Configuration Management”.
2. Attacks get through because traditional endpoint protection tools miss known attacks even when there is a signature for that attack and the protection is properly configured.  The failure rate depends on whose statistics you chose to use, but Gartner puts the detection failure rate at two to ten percent while other studies show failure rates exceeding fifty percent.  Given there will be well over 5M signatures by the end of 2009, ten percent is non-trivial.  See “Antivirus Detection Rates – It is Clear You Need a Plan B”.
3. Attacks get through because they have been carefully designed and engineered to evade traditional endpoint protections.  These include zero day attacks, rootkits, targeted attacks and the work of the maliciously intended insider.  Zero day attacks are more generic in nature and broker on the fact that most tools require prior knowledge to detect an attack.  Targeted attacks are designed to specifically infiltrate networks and applications to retrieve sensitive information or financial data.  See “It is Raining and You Will Get Wet”.

I am not saying this is groundbreaking thinking here, but if you put things into this perspective, it clearly defines the gaps in protection and subsequently provides a roadmap of what must be done to protect your endpoints.  Reducing the attack surface is clearly not enough.  Antivirus is not getting it done – even the AV vendors say so.  And the bad guys are relentless in their pursuit to exploit any crack in the defenses. 

So what do you think? Too simple or simply brilliant?

One common theme is the general sense that such an attack was not a particularly hard attack to perform if one had access to a botnet.  Botnets are clearly a widespread issue and while they often pop up as a vehicle for chaos there is not much seemingly being done to prevent them or shut down the existing ones.  For a good primer on how botnets are used in a DoS attack, see Elinor Mill’s post on the subject.

I also found it interesting that on Tuesday the Marines announced that they were banning Twitter, inciting a lot of second guessing on Twitter and the blogosphere.  Two days later the Marines look a little smarter as this attack showed that the Twitter infrastructure is open to compromise.  While I am sensitive that Twitter and Facebook allow our soldiers to communicate with home, I have seen enough evidence to know that these applications also could be used to create problems with our information infrastructure at critical times. 

Yesterday I arrived at our office and began a conversation with one of my colleagues.  Slowly everyone that was in the office filtered in and soon we were all engaged in some very interesting conversations about prospects, customers, our product direction, competitors and the broader market.  It was spontaneous, open, and very refreshing.  I say this because it was a reminder that with all of the ways to communicate –Twitter, Facebook, texting – nothing beats a face-to-face sit down.  I am hoping some folks took the time during the Twitter Eclipse as a time to get re-acquainted with such quaint methods of communication.

• Triumfant CEO John Prisco is quoted in the July 10 post of Byron Acohido’s The Last Watchdog blog regarding the Korean DoS attacks. These attacks have taken an interesting turn as the botnets created by attackers are now literally turning on the infected machines, deleting files and ultimately corrupting the system until it will not boot.  I have read a lot about this attack from many respected members of the IT security community.  Some have assessed the attacks as unsophisticated and poorly executed while others like Acohido and Brian Krebs of Security Fix (which was targeted in the actual attack) are speculating on if is a practice run – a war game – for more targeted attacks down the road.  Either way, it is one of the most interesting story lines since we were all gripped with Conficker fever in the early spring.  I suspect there will be more intrique to come.  If it was a war game, it will be interesting to see how the good guys grade themselves. 
• I posted a blog entry in June about Securing the Software Supply Chain and how Triumfant can help manage that important part of any organization’s security strategy.  The white paper on the subject is now available on the Triumfant web site for your reading pleasure.  Since many defensive products do their monitoring as malicious software is inbound to the machine, attacks imbedded in what appears to be legitimate software may evade protection.  Because Triumfant looks for changes on endpoint machines, it will detect the event where the imbedded malware “wakes up” and begins its malicious activity.
• I recently was away at the beach for a week with my family.  I mention that because I did not tweet or blog about the fact that I was gone as there have been reports that people have been robbed after letting the world know through social media outlets that they would be away from their home for extended periods. Which brings me to two points.  First, never underestimate the speed in which the bad guys will find and exploit new paths – in this case social media – to do their criminal work.  Second, security, whether it is IT security or physical security, requires an element of good old prudent thinking to succeed no matter how much technology is deployed.  Human factor eengineering (or stopping stupid as I call it) has been and will always be the biggest failure point in security.
• Isn’t it time for someone in the Obama Administration to tell us why we do not have a cyber czar yet? I mean really.  I agree with our CEO John Prisco completely and join him in wondering why they would first make the announcement without a person in the spot much less go six weeks after the announcement without a nomination.  The claims of IT Security being a priority are starting to sound very hollow.

It should be noted that the counter started the year at roughly 2.6 million and has just passed 4.2 million.  This is noteworthy because the 1.6 million new signatures is the equivalent to the number of new signatures Symantec reported for all of 2008, and we have not yet hit the halfway point in the year.  Given that a graph of the signature counts appears to be geometric rather than linear, we expect the rate of increase to accelerate and raise that delta for 2009 to on or about 4 million signatures. For the second year in a row, the number of new signature for just this year will surpass the previous combined total number of signatures.

When we had the idea for the counter, we were careful to apply some science and statistical analysis to the process because we wanted to be fair and conservative.  The counter was never meant to be about hype – it was built to provide a visual representation of the unsustainable nature of the signature model for defensive software.  That is why we are updating the counter in our attempts for accuracy, and we will also adjust the numbers down if we see that it begins to exceed the reported numbers.   

The point of this exercise remains the same.  Companies and government agencies must look beyond signature based tools for endpoint protection, as the sheer volume of new attacks makes it impossible for these tools to protect organizations from malicious activity.  Many new approaches to endpoint security such as behavioral analysis and heuristics still require previous knowledge of the attack to be really effective.  Triumfant is the one tool on the market that can detect, analyze and remediate a malicious attack without any prior knowledge of the attack.  No waiting for a vendor to create a remediation script or signature.  Remediation is minutes not hours or days.  And as the counter illustrates, every day your organization does not look beyond signature based tools, the problem only grows worse.

I would also note that the counter is not meant as a direct poke at Symantec.  We use their numbers because of our respect for the capabilities of their research team and because they graciously make their numbers public.  Other products that use signatures may have differing counts when it comes to signatures, but the basic problem still exists for those solutions. 

I have heard a lot of complaints from IT security people that say there has not been much new in the way of technology lately.  I would respectfully disagree and would invite you to have a look at the Triumfant solution and get a feel for how it works via a video of our Three Minute Malware Challenge from RSA.  Words don’t do the product justice, so the video will provide much deeper insight.   Then give us a call and let’s talk about what is keeping you up at night and allow us to show you how we can help.

I have a bit of a cynical streak and therefore normally do not fall on the side of education to stop things like cyber crime.  But it is clear that Web users need some pragmatic education because it really makes the job of IT security difficult when users willingly walk into malicious activity.  I use the term “Stopping Stupid” in a previous post, but if only one out of every eight people can spot a phishing site, then it is clear that endpoint security education is clearly needed before we can place blame solely on the users.

When my own kids began to surf the net, I was careful to educate them on what they would encounter.  For example, I made sure they knew that there was not some benevolent force on the World Wide Web that existed to give them a free iPod just for visiting their site. We talked about that if something seemed to good to be true, it probably was and that if they ended up on a page they did not expect to see, they should immediately stop. Simple stuff when they were younger progressing to many of the basics defined in the Verisign study now that they are in their teens. 

But it is unreasonable to expect that all people grow up in a house with a cynic working in the IT security market.  Few Web users know what the padlock symbol means or why the colors change in the security status bar.  Many still believe there is a Cyber Santa that really does want them to have a new notebook PC.   We hand people a computer when they show up for work and in most cases, no one shows them the basics of physical security or what to look for when doing simple tasks on the Web.   Then when we see stories such as the latest Nine-Ball mass infection, we wonder how such a thing could happen, but we are at least partially culpable for sending the lambs to the slaughter. 

So as much as it goes against my cynical nature, we in the IT security market must take the steps to educate the army of CLFs that access the Web jungle daily.  It is no fun to tell them there is no Santa Claus, and we will never get 100% on the Verisign test, but we do need to do a better job of at least teaching the basics such as the simple signs of a phishing attack.   We should offer basic education when we hand over their new computer, and there should be constant reminders of the fundamentals.   The bad guys are getting smarter, so we must make our users smarter.  After all, at 88% only one of Santa’s eight reindeer would spot a phishing attack.