March 1, 2009

performance fraud

Apple's got a pretty consistent reputation for exaggerating their marketing claims right up to the very edge of fraudulent but never over the line enough to get penalized. Safari 4's claims are right there on that line.

This is the JS engine performance benchmark suite that the WebKit/Safari team built, and which most people credit as a reasonable set of tests -- as run by the good folks over at c|net.


I'm not one to get into pissing contests but I'd say that Safari 4's time of 967 milliseconds and Firefox's time of 969 milliseconds are a lot closer than 3x.

The difference in the c|net run of the SunSpider JS test suite is literally 2/1000ths of a second. By my math, that's Safari winning in the JS performance test by approximately 1.002x -- which, if my arithmetic skills haven't deteriorated too far, is a pretty long ways from 3x.

update: Ahhh, I see. Wasn't that clever of Apple. They compared their beta of Safari 4 to the shipping Firefox 3 rather than to Mozilla's beta of Firefox 3.1. See what I mean about Apple marketing? It's really bordering on fraud.

update2: And just to make sure we don't miss the forest for the trees, and to point out the further absurdity of any comparisons between Safari, Firefox, (and Chrome) performance, we shouldn't forget that the real performance slug holding the Web back is Microsoft.

IE 7, which accounts for roughly 47% of Web usage, is simply a joke, coming in about 10x-20x slower than the rest of the released browsers today. IE 6, which still accounts for about 19% of Web usage is even worse. They've done a somewhat better job with IE 8, but with that browser nearly done and shipping very soon, it's a real shame that it will lag by 5x-6x compared to the other browsers shipping in 2009.

Posted by asa at 3:17 PM