I found this interesting article from a pointer over at The Inquirer.
A Look at SYSmark 2001 - The Broken Benchmark, by Dean Kent, Real World Technologies
The first question that will likely be asked is why I chose to evaluate this benchmark at all. What with all of the accusations and problems that have been found, it's well known that this is a 'broken' benchmark - right? Well, not necessarily. Now, before passing judgement here, let me explain...
Sometime last year, charges were leveled against BAPCo accusing them of being 'biased' towards Intel, because their SYSmark 2001 benchmark suite seemed to heavily favor the P4. Later, it was revealed by AMD that one of the applications used in SYSmark 2001 used SSE instructions, but only if an Intel processor was installed. This caused the Athlon XP to appear to be at a disadvantage when compared to the P4. The outcry from many was that SYSmark 2001 was therefore an invalid benchmark that should not be used to measure system performance.
Now, as it turns out, the application in question was Windows Media Encoder 7.0, and that it actually had this code in production - meaning, it is exactly what users of WME 7.0 would experience. Though AMD released an unofficial patch to 'fix' this problem, Microsoft elected to wait until the release of version 7.01 to correct it. Therefore, during the period that WME 7.0 was still the only available version, SYSmark 2001 was not putting the Athlon XP at an unfair advantage at all - but what about now?
It's a long article and gets sometimes involved... And dean's conclusion.... In the end, I would have to say that SYSmark 2001 is probably not the best benchmark available for comparing either systems or components, but not for the reasons that seem to be popular. I think that many people have been looking for reasons to discount the results without really understanding how it works, just because it hasn't favored their favorite manufacturer. This is a lousy reason to not like a benchmark, IMO. Perhaps the information provided here will at least give those who don't like the benchmark a truly good reason to not like it.
News source and read the full article: Real World Technologies - A Look at SYSmark 2001
-1 Comments - Add comment