Microsoft: AV-TEST study on Bing malware search links was inaccurate

Earlier this month, German-based security firm AV-TEST released the results of an 18-month study on how many sites from the top search results of Google and Microsoft's Bing are links to malware infections. AV-TEST stated that, while the overall percentage of malware sites in search results were low, Bing brought up five times as many malware links as Google.

Late on Friday, Microsoft's official Bing blog posted up a new entry that disputes the results of the AV-TEST study. Microsoft said they only got the methodology of the study on Wednesday and found out that AV-TEST didn't actually search on the Bing.com website itself. Instead, AV-TEST used a Bing API for their search results so they could download those links for their study.

Microsoft stated:

By using the API instead of the user interface, AV-TEST bypassed our warning system designed to keep customers from being harmed by malware. Bing actually does prevent customers from clicking on malware infected sites by disabling the link on the results page and showing the below message to stop people from going to the site.

The blog adds that Bing warns users about malware infected sites instead of just removing them from search results because they tend to be links to real URLs that have become infected from outside forces. Microsoft says their warnings prevent 94 percent of the clicks to those sites from Bing. Overall, warnings come up in about 0.04 percent of all searches on Bing.

Microsoft concluded their blog post by saying, "In this particular case, we here at Bing are very confident that our methods for malicious link detection and warning make our engine one of the safest on the net."

Source: Bing blog | Image via Microsoft

Report a problem with article
Previous Story

Report: Waze GPS app coming to Windows Phone 8 in June

Next Story

Windows 8.1 9374 shows search charm won't take over screen

22 Comments

Commenting is disabled on this article.

I actually tried this now i.e. searched a malware URL using Bing and it did not bring out the site. It only made references to the site. Impressive...

jimmy_jazz said,
Well done Bing blog, draw attention to a news story from two weeks ago that everyone had forgotten about.

Well, since sites like Neowin are mentioning it repeatedly and in completely unrelated news stories, I think it was prudent for them to point out the flaws in the study.

jimmy_jazz said,
Well done Bing blog, draw attention to a news story from two weeks ago that everyone had forgotten about.

I think having their say on the methodology is more important than trying to keep quiet about it in the hopes that people will forget.

jimmy_jazz said,
Well done Bing blog, draw attention to a news story from two weeks ago that everyone had forgotten about.


I see your point, but I agree it was worth the story as it sets the record straight (to those who care to read anyway).

Really? I HATE MS NOW! Yawn, google and apple act just as bad if not worse, and certain of their fans pump up any negative towards MS even if it's not true.

Ashleighhank said,
what do you expect from microsoft? any negative towards them they defend even if its true

Ooh, a troll on a MS story. Shocking!

I'd expect them to defend a defendable position once all facts were known, which they now are. They are basically saying "this headline is wrong, because they missed out all these crucial bits of info" - and they are crucial, when saying the bing search engine is 5x more likely to bring you malware than google its important to note that the 'findings' were not based on the way you and the average joe searches - but via the API (designed for apps to access the engine and provide their own protection), so this explanation (no excuse) is perfectly viable and its 100% logical and honest.

If the tests were done in the same way on the other sites, it seems fair. Apples to apples so to speak. I mean Google also has malware warning systems on their site. But there is no info on how tests where done on other sites.

I disagree. If the bing user gets warned, he gets warned. The fact that you can bypass this with the API may be helpful in certain situations, the tone of the study was that bing *users* were at risk, and they are not. Any developer mining bing with the API probably knows better.

I honestly believe there's a consorted effort to undermine Microsoft's security gains. Why else would you compare a full fledge security suite, like Norton, and compare it to the stand alone anti-malware MSE?

W32.Backdoor.KillAV.E said,
MSE is a full fledged suite, including the version now built in to Win 8. It's not like Windows Defender on Windows 7 even though it shares the same name on 8.

It isn't a full fledge security suite on it's own. For example, URL and spam protection is done via SmartScreen on IE and Windows 8. MSE doesn't not offer such capabilities. Software suites such as Norton, however, does, even going so far to offer email protection. This is what we mean by "full fledge security suite." MSE deals only with malware.