Microsoft OneCare fails virus test

Microsoft's Windows Live OneCare security suite has failed a test to detect or stop 37 malicious programs on Windows Vista. Independent security certification body Virus Bulletin tested 15 security suites on Windows Vista against a set of malware from the WildList database. "The tests conducted in our secure labs were against the most significant viruses and worms affecting real-world users," said John Hawes, technical consultant at Virus Bulletin.

"In these days of hourly updates, it is always a surprise and a disappointment to see major products missing them. Computer users deserve to see a better performance than this from security vendors." OneCare detected about 99.91 per cent of the threats, picking up on all macro viruses. But it failed to stop some viruses, file infectors and polymorphic malware which constantly changes in an effort to avoid detection by security software. Microsoft's offering was one of four suites which failed to detect all malware. The others were G-Data AntiVirusKit 2007 v.17.0.6353, McAfee VirusScan Enterprise 8.1i and Norman Virus Control 5.90.

View: The full story
News source: vnunet

Report a problem with article
Previous Story

AMD's RS690T, SB700 notebook chipset: details emerge

Next Story

AMD Creates First 45nm Chips, Says Manufacturing Chief

36 Comments

Commenting is disabled on this article.

[Edited to correct frequency of Virus Bulletin tests. AG]

Hello,

For those who are not familiar with it, Virus Bulletin is the basically the industry trade magazine for the anti-virus industry. Every other month they publish a Comparative Review where they test a range of anti-virus products on a particular platform, such as Linux, Novell NetWare, Microsoft Windows Vista and so forth. The test consists of several criteria, the two main ones being that products must detect all 100% of ItW ("In the Wild" viruses, as reported by the WildList Organization) and not generate any false-positive alarms of viruses on a clean set of files.

The products tested this month were Alwil Avast, CA Anti-Virus, CA eTrust, CAT QuickHeal, ESET NOD32, FortiNet FortiClient, F-Secure Anti-Virus, G-DATA AntiVirusKit, Grisoft AVG, Kaspersky Anti-Virus, McAfee VirusScan, Microsoft Windows Live OneCare, Norman Virus Control, Sophos Anti-Virus and Symantec AntiVirus. Of those products, the ones that did not receive a VB100 this time were G-DATA, McAfee, Microsoft and Norman.

Almost all products which are tested by Virus Bulletin have failed to receive a VB100 award at some point or another, so the fact that a product does so on occasion should be no cause for alarm. Also, it is important to keep in mind that while Microsoft's Live OneCare is based on technology they acquired from GeCAD in 2003, they are still rather new to shipping a comprehensive anti-malware product (the Malicious Software Removal Tool only detects and removes a subset of known malware) so I think it is fair to say the verdict's still out on their Windows Live OneCare product.

From my perspective, where the Virus Bulletin's tests really come into play is now with an individual comparison but with how they allow one to take a neutral and unbiased look at how vendors' products do over time and see if the detection rate is improving, declining or staying about the same. Bear in mind, that there is more to the magazine than product comparisons. Leading-edge research, descriptions of new malware techniques and threats is all discussed. As a matter of fact, the current February 2007 issue has an interesting article on web-borne malware, which should be of interest to anyone who runs an online forum, for example.

If you are interested in this sort of thing, I would strongly suggest looking into getting a subscription. It is not inexpensive (subscriptions start at $175.00 a year for individuals), however, if it is a subject matter in which you have a personal or professional interest, having access to this sort of information is invaluable.

Regards,

Aryeh Goretsky

99.91% is horrible when so many others got 100%. Why are your defending mediocrity? The point of the article is to show Vista is NOT what Bill Gates and others are saying about Vista. Vista should be the standard for others to follow.

99.91% might be considered good if everyone else was in that range but with so many others doing better then Microsoft needs to get back to work. We settle for '2nd best' too many times and the area of security should not be one of them.

Sometimes these posts and their comments remind me of the two grumpy men on the muppet show (Statler & Waldorf):

Boo!
Boooo!
That was the worst thing I ever heard!
It was terrible!
Horrendous!
Well, it wasn't that bad.
Oh yeah?
There were parts of it I liked.
Yeah, I liked a lot of it.
Yeah, it was good, actually.
It was great!
It's wonderful!
Ah, bravo!
More!

It starts with "Man what a ****ty program, it fails the virus test", then "Ah well they're new to the game", finally "Well 99.8% isn't that bad, quite good actually" :)
I like OneCare, it works great for a good price.

Well, since the onboard "security" (ahem ) solutions are a bad joke already (Vista firewall fails *any* leaktests, is easily circumvented and in default setting permits *any* outgoing traffic, and Windows Defender is just a half-assed attempt at Anti-Spyware), I wouldn't trust them farther than I could spit with their OneCare crap.
Heck, even the free Avast is a lot better than OneCare, which you actually have to pay for (but apparently there are enough clueless n00bs who pay for it).

Microsoft's new to the game....though they have intelligent people working on it....no one app can kill everything!

What doesn't surprise me it all is that POS McAfee sells...trust me, from what I and so many others have seen those using McAfee, I'd rather deal with the virus.

xMorpheousx416 said,
Microsoft's new to the game....though they have intelligent people working on it....no one app can kill everything!

What doesn't surprise me it all is that POS McAfee sells...trust me, from what I and so many others have seen those using McAfee, I'd rather deal with the virus.

"New to the game"? Perhaps, but OneCare already passed VB100 certification in June 2006. What happened? They went from industry-leading to last place, one of 4 failed products.

"No one app can kill everything"? Well, 11 of the 15 products did just that. "Not one", indeed! It was eleven that succeeded.

There's only one product I've had any trust in for the past 5 years, BITDEFENDER
Apart from having the best AV detection and an excellent firewall, it's the Only Security Software to Accurately Detect 100% of Malware in Performance Tests. Works well on Vista too of course.

If your AV software doesn't have a VB100% certification, then don't use them!
99.91% is a failure, it only takes one Malware or Virus to cause a big problem.

If that didn't exist, I guess NOD32 or Kaspersky would be my next choice.

Microsoft's attempt wouldn't be in my top 10 that's for sure - and that's before this recent review.

Hang on. Isn't Windows an operating system, when it gets down to the nitty gritty? Sure, they can pack in an AV, and stopping 9,991 of every 10,000 threats is quite nice. But what happened to the concept of using specialist AV stuff AS WELL which is made by companies whose MAIN objective is to make great AV software, rather than a company whose main objective is to make an OS?

This "article" doesn't deserve to be posted on a blog, let alone the front page of Neowin. It is less professional than a common blog. This whole story should be removed from the frontpage.

iconoclast said,
This "article" doesn't deserve to be posted on a blog, let alone the front page of Neowin. It is less professional than a common blog. This whole story should be removed from the frontpage.

So, you disagree with the article? Yeah, that means it shouldn't be posted anywhere. And, you are the NeoWin editor, right?

How do you carry that ego around with you?

I think AV and Malware protection is all a matter of balance. I'm certain "whatever" the winning software was requires maintenence and updates to function well. I'm also certain it eats up CPU cycles and other system resources. I've found that One Care when coupled with common sense strikes up a great balance between maintence, resource use and usability to be the product I recommend to others. Not only does it offer protection from malware and viruses, it also makes sure all of your Microsoft software is up to date, keeps your hard drives degragmented and provides automated backup. You pretty much set it and forget it.

Some people are misunderstanding the 99.91% detection rate. It's just the detection rate; not the removal rate. Just because an AV product can find the virus doesn't mean it can easily remove it. Some threats (especially spyware) are just really difficult to remove.

So when they talk about failures, they're not saying that OneCare failed to detect the threat. They're saying that OneCare failed to stop/disable/neutralize/remove the threat.

Oogle said,
Some people are misunderstanding the 99.91% detection rate. It's just the detection rate; not the removal rate. Just because an AV product can find the virus doesn't mean it can easily remove it. Some threats (especially spyware) are just really difficult to remove.

So when they talk about failures, they're not saying that OneCare failed to detect the threat. They're saying that OneCare failed to stop/disable/neutralize/remove the threat.


How's it supposed to remove something it can't find?

Croquant said,
How's it supposed to remove something it can't find? :rolleyes:

What does that have to do with anything I just said?

Oogle said,

What does that have to do with anything I just said? :rolleyes:

cause you make it appear that removal rate is far more significant than detection rate but if the virus can't be detected it can't be removed.

Jexel said,
cause you make it appear that removal rate is far more significant than detection rate but if the virus can't be detected it can't be removed.

Don't read what you think I wrote. Read what I actually wrote.

Oogle said,

Don't read what you think I wrote. Read what I actually wrote.

True; lets rephrase it; if a piece of software detects 100% of all virus's but only is able to remove 80%, isn't that a whole lot worse than OneCare which, hypothetically might detect 99.91% and remove 100% of virus's detected?

http://www.virusbtn.com/index

Where are the stats at? If you're only going to say who the losers are, why even post it. I'd be more "concerned" about the McAfee failure than I would OneCare, as it's installed on many corporate environments (and it's still the worst of them all).

I would start looking here: http://www.virusbtn.com/news/virus_news/2007/02_02.xml
They offer the Vista VB100 certification list for free (registration req'd.). They offer the full testing information only to subscribers though, and that may mean you must pay.

But that page does say they tested 15 suites. 11 passed with 100% detection. Of the ones that failed, OneCare did the worst.


EDIT: I registered, and was able to also see the history testing of OneCare. It should be noted that OneCare passed VB100 certification in June 2006. So they went from perfect score to apparently the worst.

On the up side, perhaps Microsoft will see these results as a sign to get back in gear and strive for that 100% perfect score again.

yeah this is a load of FUD. If you're going to say "this program failed THIS much" then go into detail about it, you can't really say "oh and these programs failed, too" without showing some more statistics.

Show me the results from this, depicting how the other major AV's did for a real comparison.

Do you work for Microsoft's spin department? 99.91% isn't near failure, indeed. But what do you call it when it is the very worst performance of all the suites tested? Certainly not a "star performer".

What, and they want us to think any of the other major players did any better. Why post test results for one vendor when they MUST have tested others. I think we all know why.... The others did no better, and in all probability they did worse.

They may have been tested, but the problem with this post is even though G-Data, McAfee and Norman (???) have also failed these tests, the original author decided to highlight the fact that OneCare did...

I can't find Virus Bulletin's published results anywhere either...

99.91%??... i am no expert but that to me sounds like a nice %... im guessing there are ones out there that do get 100%??

Yeah, I think 99.91% is pretty good myself!

What I want to know is what did good? The referenced article only mentions those that "failed." I went to the article that was referenced by the article that this article references ( woo! ) and it requires a paid subscription to view, the cheapest of which is a ridiculous $125 a year! The most expensive being $2,000 a year for corporate environments! o_O!

You would think with that kind of subscription level they could at least afford to keep their site fairly snappy... I am apparently mistaken!

Yeah, it doesn't sound bad. And the article listed that three other suites also failed to get 100%. They don't say how many did get 100% (which should have been listed). Also, the BBC report stated that "The security testing group found that Live OneCare missed far more active viruses than any other program tested."

So, I guess 99.91% is a lot better than nothing, but worse than every single other product they tested. Again, a listing of what their full scope of apps were would be nice, but seems to be lacking.