Have No Fear: Triumfant’s Remediation Capability is Automated, Not Automatic

July 9, 2010

In my previous blog entry I attempted to unravel some of the fear, uncertainty and doubt around our automated remediation capabilities.  After a week of quiet contemplation at the beach and writing a whitepaper on the subject I had an epiphany.  I think the fear comes from a misunderstanding between the words “automated” and “automatic”.  Allow me to explain.

Triumfant has automated the construction of a remediation for any malicious attack or anomalous incident it finds.  The detail is all there in the blog entry.  But in short, because Triumfant knows what has changed and knows the value of every attribute or file before it was changed, it is a logical next step to build a remediation script to return the affected attributes to their pre-attack condition.  No spooky “thinking computer” stuff, just sound logic driven by the capabilities of our granular change detection.

The automated remediation Triumfant creates is not automatic in its execution.  The Triumfant administrator must perform a one-touch confirm of the remediation before it is sent to the agent for execution.  Triumfant does the heavy lifting of analyzing the attack or problem and building a comprehensive remediation script.  It is automated.  There is still the failsafe of human interaction as a confirmation.  It is not automatic.

There does exist easily implemented mechanisms to make remediations automatic for incidents encountered in the past and for configuration and policy enforcement.  In these cases the remediations are known and the customer is comfortable with their effects.  For example we have customers who identify specific applications they want removed from endpoint machines and these remediations are automatic.  But for anything new encountered by Triumfant such as a zero day attack or an Advanced Persistent Threat type attack, the default is the one-touch confirm by the administrator, providing oversight and control.

Why am I writing about this subject?  People seem to have a love/fear relationship with the concept of automated remediation.  For example, at the NIST Security Automation Conference last October in Baltimore I asked the attendees in my presentation two questions:

Q1: Who wants automated remediation?  A: Every hand in the room enthusiastically shot up.

Q2: Who is ready to implement automated remediation?   A: Crickets.

All I can surmise is that security people suffer from what I have dubbed “SkyNet Syndrome” – the lingering effects of watching science fiction movies and television series that ultimately play the “smart computer comes to life and destroys the world” card.  The list is long and distinguished: M5 and Nomad on Star Trek the Original Series, V’Ger on the first Star Trek Movie, Colussus from Colossus: The Forbin Project, Proteus on The Demon Seed, and, of course, SkyNet from The Terminator.

Notice that I did not include W.O.P.R. from WarGames as he was hacked and did not become sentient.  Of course he was hacked because he had poorly configured endpoint protection that could have been easily recognized through change detection and quickly addressed by Triumfant through an automated remediation.  But I digress.


Security Configuration Management – Don’t Fall for the Old Saw of Patch Management

April 16, 2010

Yesterday I attended a customer event for one of the larger IT security firms and one of our partners.  During one of the sessions, another partner gave a presentation on security configuration management that nearly drove me to a Kanye West “grab the microphone” moment.

The partner in question regaled the attendees with a story of automated configuration management that involved long intrusive scans, followed by analysis to identify problems, followed by the issuance of what are essentially patches to correct non-compliant machines.  This process seemed horribly cumbersome and certainly did not meet my definition of automated.  And the remediations for the detected problems were pulled from a list of pre-written remediations in a one-size-fits-all approach.

Worse of all, the process happened infrequently – monthly at best, perhaps quarterly or twice yearly.

The saw blades are a visual of configuration drift over time. The length of the saw teeth represents the amount of drift and the size of the gap between them represents the time between corrections.  If the height of each saw tooth indicates how much configuration drift you will experience with large gaps between configuration corrections, which do you think represents the most secure environment?  The bigger the teeth, the higher the organizational risk.  You want the hacksaw blade.

My negative reaction comes from knowing there is a better way to deliver security configuration management Triumfant will continuously scan for changes to endpoint machines and detect when the machine is in a non-compliant state.  Triumfant’s analytics will evaluate the changes to the machine, create a remediation for that problem, and return the machine to compliance.  Remediations are created on the fly to address specific detected problems on each machine.  The remediations are surgical, contextual, and situational.  The remediation is delivered to the machine and executed by the agent.  All of this can be set to a one-touch confirm from the administrative console or fully automated.  And we will open a touble ticket, populate the ticket, and close it to track the process.

The result – your organization starts every day in an audit ready state.  The maximum drift is 23 hours and 59 minutes.  Not a month, 3 months, or 6 months.  No need for heavy, obtrusive scans, no human intervention needed to write remediation scripts, no additional patching activity.

Folks, patching is a big part of the problem, so why would you get excited about any so-called solution that is essentially a patch management process?  Patching is hard and rarely done well.  Why do you think there is so large a time gap between correction cycles with this technique?  Take a hard look at many of the companies pushing configuration management – they often have their roots in patch management and that is how they address the problem.

Security configuration management effectively reduces the attack surface on each machine, but this is achieved only when the configurations are continuously enforced.  When you can detect and remediate problems every day, you create what we call persistent security readiness.  Don’t settle for old school techniques and large gaps between corrections because monthly or quarterly is not persistent.  There is a better way.


Cyber Czar Announcement Slipped Under the Door – What Does That Say?

December 22, 2009

Today it was announced that Howard Schmidt was appointed to the White House Cybersecurity Coordinator position otherwise known as the cyber czar.  Much has been written in our blog about this position since it was announced, and the timing and approach to the announcement has done nothing to eliminate the concerns previously expressed.  I have much reading to do before I comment directly on Mr. Schmidt, but I do have very strong impressions from the way it has been handled.

The position was originally announced at a press conference in late May on the Friday before Memorial Day.  Not exactly a day and time that you would select for something of importance or to maximize the impact.  Months passed as candidates not only turned down the position, but candidates like Melissa Hathaway left the government for the private sector. 

And now we get an announcement stating the position has been filled on the Tuesday of Christmas week, in a city where most of the government is closed because of a record setting snowstorm.  The announcement gets a mention on the White House blog with a picture of the President shaking hands with his new Cybersecurity Coordinator in what appears to be a hallway.  No press conference, no fanfare, no President standing at the side of the new Czar as a show of support to the position and a commitment to the idea of cyber security. 

I went to the White House press page at 11:00 am EST and there is no formal announcement.  There is news about the Enactment of the Airline Flight Crew Technical Corrections Act, something about agencies cutting spending by $19B, and some nominations that were sent to the Senate, but nothing about this position. 

I am sure there will be Obama acolytes that will line up to applaud the announcement, but I find myself angry today as I weigh what I see from the White House.  I have the somewhat unique perspective of being a marketing person in cyber security.  That means that cyber security is important to me and I have a taste of the threat against our country.  As a marketing person, I see the not so subtle signals being sent by the White House regarding this subject.  Put this in context of the amount of energy thrown at other items such as the Chicago Olympic bid, and it is reasonable to draw the conclusion that the White House is not committed to cyber security and this role.  

I would be a lot angrier if I had not spent the last year working with people at NIST, the NSA, the intelligence community and the DoD who are proactively and energetically looking for new and innovative ways to protect our government from malicious attacks.  Best of all, these groups are working together to share data and knowledge in a way that would make taxpayers happy and proud.  So I’ve seen real, actionable progress taking place long before Mr. Schmidt assumed this role.   

To be clear, I never viewed the cyber czar as some sort of mythical figure that could solve our cyber security problems with a wave of the hand, but I was hopeful that the role would serve as a way to focus attention on the problem and help create a sense of urgency toward progress.  And if that person was seen as having the “full faith and credit” of the President, it would give that person some authority to be something other than a figurehead. But that is exactly how I view this position today – a campaign promised fulfilled in the least effective way possible and with minimal authority, buried in slow press days.

This week we got news that our military drones had been hacked with $26 software, a plea was entered in the Heartland breach by the same person who pulled off the TJX breach (who used a simplistic and well known SQL injection technique to penetrate Heartland), and that Twitter was disabled completely by a DNS attack.  And we get a tepid announcement the Tuesday before Christmas on a position that it took them seven months to fill.  I’d hate to see what type of catastrophic, IT security based incident it would take to make the White House treat the problem with the seriousness it deserves.


Triumfant is Now McAfee Compatible – Our Integration with McAfee ePolicy Orchestrator

October 19, 2009

On October 5 it was announced that Triumfant had achieved McAfee Compatible status in McAfee’s Security Innovation Alliance (SIA).  This status is achieved when a vendor like ourselves is able to demonstrate interoperability between McAfee ePolicy Orchestrator (ePO) and Resolution Manager in testing conducted by McAfee. 

So what does that mean and what level of integration and interoperability exists?  The integration with ePO is being done in phases with the first phase available today.  In the first phase, information from Resolution Manager is available through the ePO console, with drill-downs to Resolution Manager for more details on specific incidents and events.   In the next phase scheduled to be completed by year end, there will be additional integrations:

  • The ability to push out the Triumfant agents from within ePO.
  • Direct integration with the ePO event tables.  Resolution Manager will use database to database integration to move event information from Resolution Manager directly to the ePO event table.   This allows data collected by Triumfant to be a part of the integrated reporting of ePO.
  • Additional drill down capabilities from the ePO console to view to our machine status, configuration, change history, incident history, performance, and diagnosis screens.  These links provide ready access to as much data from Resolution Manager as possible into the ePO console. 

Additional phases are still being scoped and will be shaped by the needs of our customers once they put the integrations to use.  Likely additions include integration of the wealth of asset data collected by Resolution Manager to the ePO asset data stores. 

What does our integration with McAfee mean and why is it important?  We understand that companies are inundated with new interfaces, making integration points for presentation and management – the elusive single pane of glass – very important.  For McAfee customers, ePO is that integration point.  This was very apparent to me at the McAfee FOCUS 09 User Conference as most of the people who came by our booth started the conversation with a query about our ePO integration.  It was obvious that such integration was a gating factor in their decision to looking deeper into our offering. 

We have always positioned our product as a complement to traditional endpoint protection tools like McAfee, and our relationship with McAfee is a natural extension of that philosophy.  It is clear from the McAfee customers that they embrace ePO as a single point of command and control for endpoint protection, so it is an equally natural extension to integrate Resolution Manager into ePO.  I commend McAfee for their desire to build an ecosystem for their customers that includes third party vendor partners, and we are very pleased to have achieved the McAfee Compatible designation.

As always, we are happy to demonstrate our capabilities, including our integration with ePO and I invite you to contact us to set up an online demo.  I think you will see the benefits of our collaboration with McAfee and the value of adding Resolution Manager to the McAfee suite will be readily apparent.


Infection Rates Up 15% in September

September 30, 2009

Another study, another set of results that support that the malware threat is quickly outpacing traditional protections.  Antivirus vendor PandaLabs released a study yesterday showing that the number of infected PCs worldwide increased by 15% in September.   The average number of PCs hit by malware now stands around 59 percent, with the U.S. checking in with a 58 percent infection rate.

The data was pulled from “users that scanned and disinfected their computers with the free Panda ActiveScan online antivirus”.  There is no way to know if those users were already running an AV product on their machine, although that data would have really been instructive.   Given that these users were running free software it is likely that these are consumer users or small businesses and not enterprise customers, and there is no breakdown of user type in the study.    

What the study does show is that the malware problem is getting worse, not better, and that this malware is finding its way to machines.   With a plethora of studies showing AV detection rates at less than 50% (some significantly less) it is safe to assume that even with traditional protections in place a lot is getting through. 

Of course the obvious question is what did the scanning tool not find?  The PandaLabs tools is signature based, meaning it could only find what is already known, leaving any number of already working attacks undiscovered.  These infection numbers would undoubtedly be higher if everything could be seen and counted.

The story is simple.  Lots of bad stuff is getting through traditional protections, and the bad guys are making more bad stuff and making it harder to detect every day.   Traditional protections can only see what is already known in the form of signatures, and even when a signature exists the failure rate is too high.  And we haven’t even begun the discussion of seeing all of the damage from an infection or properly remediating the damage.  The AV vendors continue to trot out new functions and features to try and patch the gap in their offerings, but it is clear you need something more. 

This study supports what I said in an earlier post when I compared traditional protections to an umbrella on a rainy day and noted it is raining and you will get wet.  This study is yet another brick in the wall of support for that thesis, and shows again that you will have to decide just how wet you are willing to get.


Putting Teeth Into Security Policies Via Continuous Enforcement

June 10, 2009

I came across an article by Kelly Jackson Higgins of Dark Reading yesterday that reported that “Most Employees Disobey Security Policies”.  While the study used in the report was commissioned by IronKey and therefore had a leaning toward policies around removable media, there were plenty of other policy violations cited in the study.

Of the 1,000 employees polled, half said that corporate security policies are ignored by employees and management.  Violations listed included: turning off firewall and other security settings on their machines, social networking, using web based personal email on work machines, and password sharing.  Higgins adds that 70 percent of end users don’t think their organizations have a policy forbidding their turning off security settings (including a host firewall) on their work computers. And 21 percent say they disable those security settings, up from 17 percent two years ago.”

Allow me to add some of my own analysis:

  • I think that half is low because there are likely some that did not even know there were policies. 
  • A telling phrase in the summary is the term “and management” – because if management ignores the rules, then it is a given that the rank and file will go along.  
  • I will refrain from asking why anyone would think that turning off security settings would be a good idea, because we all know that people are still the biggest X-factor when it comes to endpoint protection.  I had a senior security exec at a government agency refer to such things as CLF problems – carbon based life form.

Policies in and of themselves solve nothing.  Without continuous enforcement they are doomed to failure as indicated in this report. And by continuous I mean relentlessly continuous because the enemies – ignorance and incompetence – are equally relentless. 

That is where Triumfant is so well suited for enforcing policies and configurations.  It continuously scans for changes on endpoint machines, and if a detected change violates a policy or configuration, it builds a remediation to set the machine back to compliance.  It does this every day for every machine and continuously meets the challenge of the CLF problem because it matches the problem with equal relentlessness.  If the user changes a setting, it changes it back.  If the user does the same thing the next day, it sets it back.  This can go on for days, but eventually the human normally relents. 

At a minimum, your organization knows that it starts every day with the endpoint population in compliance with security policies.  And at the end of the day, Triumfant will put back all of the slippage, returning you to the same place for the next morning.  That is continuous enforcement and that is what makes policies effective.


Exhibit A for Bad Advice – A Questionable Recommendation from the New York Times

May 15, 2009

Yesterday a friend sent me an article in the New York Times asking my opinion on a recommendation made by the author regarding improving performance on home PCs.  In the article Five Controversial Ways to Speed Your PC, author Paul Boutin recommends that users “uninstall your antivirus software” because he perceives the threats are an overhyped and basically scaremongering by his fellow journalists. 

I hope the writer has the guts to come back and tell his readers just how long his machine survived unprotected.  I have seen studies where unprotected PCs have been connected to the Internet and are infected in minutes and part of a botnet in hours.  In my opinion, this recommendation was irresponsible and could cause a lot of people to lose personal data on their home machines. 

But this is just the kind of behavior that I pointed out in my recent post about “Stopping Stupid”.  All of the security software, policies and configurations cannot protect against the human element, especially when it looks to do something like the recommendation for this NY Times article.  Because you know that there are people in the workplace that read the article, decided that their AV software was the reason their machines at work were not as fast as they want, and started the process of disabling or eliminating their AV software on their work PC.  If this were an old horror movie, CISOs and IT techs would be an angry mob on their way to Mr. Boutin’s office with torches and pitchforks. 

That is why security configuration management tools have got to be more than a one-way push of configurations to ensure endpoint security.  These products must have every machine, every day vigilance to verify that the configurations and policies are in place and take the steps to remediate the machines if they are not.  The only way to fight incompetence or ignorance is through relentless repetition.   And since stupid is a free-style art form, signature based tools and pre-written remediation scripts will not get the job done.  The security configuration management tool has to be able to do situational remediation to address problems as they are encountered.

Lots of endpoint protection and configuration management tools may say they do exactly that, but they don’t.  They are pushing scripts.  I suggest you ask for more from your security configuration management tool and make sure you choose one that will stand against the crafty work of the maliciously intended cyber criminal as well as stand in the gap against user incompetence and ignorance.


Eliminating Unauthorized Software – Plugging the Holes in Your Endpoint Security

April 13, 2009

This is the fifth and final in a series of how Triumfant helps plug the holes in your endpoint security defense-in-depth strategy.  In this entry I will address eliminating unauthorized software – making sure that endpoint machines are free from software that can cause vulnerabilities and cause interruptions of service.

As personal computers became a fixture in the workplace, employees began to take liberties with using these machines for personal use.  What started as small steps has now grown significantly, with the amount of personal usage programs sometimes exceeding the applications for business use.  While this usage is often benign, it can cause significant problems: 

-      Some applications can introduce significant vulnerabilities to endpoint machines.  For example, the peer-to-peer programs used for music sharing and gaming have been tied to a multitude of breaches, the most notable recent case being the breach of the Marine One plans.

-      The use of business machines for personal finances may expose personally identifiable information if that machine were breached.

-      Multiple applications may have the cumulative effect of causing other required programs to not run properly.  This gives the IT support staff an incredible headache, as managing infinite permutations of application combinations is simply impossible. 

The growth of unauthorized software on endpoint machines is not trivial.  I cannot tell you the number of times we go to an organization to do an install, and find literally thousands of programs.  As we are fond of saying: you don’t know what you don’t know.  In one case the customer expected to find 150 to 200 applications, and the first inventory found over 9,000.    Organizations expend countless millions in administrative functions to control this software and respond to the problems that result.

Triumfant excels at controlling unauthorized applications, and policies can be readily and easily created that combine whitelist and blacklist techniques to control what can and cannot exist on endpoint machines.  Unauthorized applications can be removed automatically with a one-touch confirmation by the administrator.  The policies are highly flexible, meaning that you can perform the operations with or without notification to the user, and tailor the policies to apply different rules to specific groups.  Because unauthorized software may be a veiled attempt at bringing a malicious payload onto an endpoint machine, Triumfant synthesizes a custom remediation to remove the software instead of simply using the uninstall script.  This ensures that everything added to the machine by the unauthorized software has been removed.  Triumfant customers start every day knowing that every machine is free of unauthorized software – at least until users start the daily process of adding new applications.

We perform this role for the U.S. Army Information Management Support Center (IMCEN), which uses Triumfant Resolution Manager to control non-compliant software.  IMCEN has deployed over 12,000 desktops and tells us that they achieve estimated savings of approximately $8 per desktop, per month.

While it is not as high profile and glamorous as battling exotic malware such as Conficker, controlling unauthorized software on endpoint machines is an important part of any security strategy.  Triumfant performs this task without the need for human intervention, providing organizations the functionality they require without the labor costs.


Triumfant Added to Army IA Approved Product List

April 6, 2009

Today we are announcing that we have been added to the U.S. Army’s Information Assurance Approved Products List (AIA-APL), fresh on the heels of our announcement of our EAL2+ certification.  We are excited about these certifications because they say a lot about the quality, stability and capabilities of our product.  The tests performed by the DoD labs at Ft. Huachuca, Arizona are  rigorous.  We are pleased to have passed successfully.

These are especially important for Triumfant because they reinforce the interest and support we have gotten from the DoD and intelligence communities.  The DoD organizations are drawn to our ability to ensure that endpoint computers meet standard policies and controls on a daily basis.  They like knowing that we can detect problems and remediate these problems automatically, with the practical result of having every machine audit ready at the start of every day.   We have successfully demonstrated the ability to build a body of policies that capture the Army Golden Master and successfully enforce those policies.  We are removing unauthorized software from IMCEN at the Pentagon and they tell us that we save them $8 per computer per month.

For the intelligence community, they are drawn to our ability to detect malicious software without the need for prior knowledge of the attack or a signature.   Why?  Because they spend their days knowing that nearly every attack they see will be new and will never have a signature.  So beyond detection, they love the granular change detection that provides a wealth of knowledge to their incident response teams so they can decompose every attack.  They also like the way this granular change detection allows them to spot the work of maliciously intended insiders, because unless someone finds a way to change a machine without changing registry values and the other most basic elements of a machine, Triumfant will detect those changes.

For an emerging company like Triumfant, these certifications mean that the DoD and Intelligence organizations can feel a lot more sure about making a bet on Triumfant.   They already see the value of what we do.  These certifications allow them to be confident that we can stand up to their requirements and deliver against that promise of value.


Enforcing Security Policies – Plugging the Holes in Your Endpoint Security

March 31, 2009

This is the fourth in a series of how Triumfant helps plug gaps in your endpoint security defense-in-depth strategy.  In this entry I will address the enforcement of security policies – starting every day with every endpoint machine in compliance with organizational or mandated policies.

Your first question may be why I differentiate between configuration management and policy management.  There are many similarities but some subtle differences, particularly when taken in the context of automated remediation.  First, I would offer that configuration management is more granular and deals with specific settings and installed software, where policy management tends to take a wider business orientation.  Of course, the wider business policies must eventually be expressed in the language of granular settings to be implemented.  In the context of remediation, where a configuration may be black and white in regards to remediation, a policy may call for multiple remediation scenarios based on user profiles, geographies, or other criteria.

Let me give you an example.  Triumfant does a great job of removing unauthorized software from endpoint machines, a topic I will detail in my next entry.  A configuration would tell Triumfant Resolution Manager that when it detects a specific program that it should perform an automatic removal of the software.  One cause, one action.  A policy would step in and determine that the software is unauthorized except for a specific group of machines within the broader population (an exception).  For those machines where the software is not authorized, the policy would further define three specific remediation actions for groups of endpoint machines based on the title of the machine owner: notification only for VPs and higher, automatic removal with notification for director and senior director, and automatic removal with no notification for everyone else.

Customers can use a wizard driven interface to capture the policies into Triumfant Resolution Manager so the policies can be broken into the specific pieces and parts that will be monitored.  As with configuration management, Triumfant will detect changes in the endpoint machine that place the machine out of compliance with a given policy, and will synthesize a remediation to correct the problem.  Any deviation from policy is detected and corrected in a 24 hour cycle or less, creating a continuous state of compliance.

The management of specific, granular configuration settings has enormous value, but the world is not always black and white.  In fact, the term “except for” is one that frequently causes lots of complication and special consideration.  Such grey areas can only be expressed in the business language of a policy.  Many tools can only process black or white through their interfaces and require that complex scripts be created by hand to handle more complex logic and exceptions.  The ability to go beyond the black and white and readily accommodate exceptions is yet another differentiator for Triumfant.