Antivirus Detection Rates – Undetected Attacks Are Still Attacks

I came across an article in The Business Times this morning that contained a quote that caught my eye.  The article was called “Singapore a growing platform for cyber attacks on region” which talked about the growing number of cyber attacks originating in Singapore.  In the article there was a definition attributed to Symantec:

“By Symantec’s definition, an attack denotes any malicious activity carried out over a network that has been detected by a firewall, intrusion detection or prevention systems.”

Obviously, the word that stuck out in this definition was “detected”.  Why?  Because I have news for you – malicious activity that goes undetected is also an attack.  In fact, I would say that undetected attacks would be placed in a higher tier of the definition, because Rule One of criminal behavior is Don’t Get Caught.  Attacks that would fall under the characterization of an Advanced Persistent Threat are engineered to evade detection and are very much an attack.

(This reminds me of one of my favorite movie scenes.  In Stripes, Harold Ramis and Bill Murray are sitting in the Army recruitment office and the recruiter asks them if they have “ever been convicted of a felony?”.  Bill Murray’s response: “Convicted?”.)

In fairness to Symantec, I am not sure if this quote from the article was paraphrased or misquoted, and I am not out to pick on Symantec.  What I do want to point out is a huge flaw in how in the industry measures malicious activity.  Let me explain.

Both AV software vendors and internal security groups often report on what was detected.  Makes sense, right?  If you could count undetected attacks they would instantly be now detected.  But according to the Symantec Internet Security Threat Report: “Symantec created 2,895,802 new malicious code signatures in 2009, a 71 percent increase over 2008”.  It therefore makes sense that the number of detected attacks would go up proportionately with the number of identified signatures.  An organization could be doing a worse job year over year detecting attacks but their raw volume of detected attacks would still go up, giving a perception of success.

Executives look at the bulk score and are mollified that the organization is protected.  But if the number of attacks grew by 71%, the number of attacks detected by the organization better track to that same 71% or the organization is losing ground.  If you think it through, that 71% may be deceiving because what Symantec and the other AV vendors don’t tell you is how long your organization was exposed between when the attack actually was first introduced and when they finally detected it and wrote a signature. It could have been six hours, but it could have also been six months.

In short, gauging success from bulk detection numbers is a quick way to obfuscate the real risk to any organization.  But if you are selling a shield that has known flaws, it is a great way to use the steadily growing malware volume to present either software or organizational effectiveness in a successful light.

Because Triumfant uses change detection to identify malicious attacks, we have always been open about our ability to see attacks that are resident prior to our installation.  That being said, we inevitably see anomalies that are artifacts of attacks that have passed through the organization’s shields soon after we are installed.  Once installed, we can readily detect what does make it through the organization’s shields or attacks being done by maliciously intended insiders.  It is eye opening to the organization just how many attacks have and are getting through.

Don’t let yourself be lulled to sleep by bulk detection rate numbers.  A lot of attacks are getting through, so counting detected attacks is potentially a false gauge of success.