Dodgy benchmarks http://jroller.com/page/fate?entry=dodgy_benchmarks
For anyone following JavaBlogs, you've probably skimmed over a bunch of Sri Lankan shirtlifters cooing recently about a highly dodgy benchmark that somehow proves that Axis2 is faster than XFire. These turds are of course the same bunch of clowns who are very busy plopping their tagnuts all over the WS-* racket at Apache. The benchmark is by none other than WSO2; the company that effectively owns Axis2.
What's interesting about this benchmark is that it manages to compare apples with oranges, and yet has its proponents drawing some truly perplexing conclusions.
There are some, ahem, issues with this benchmark. Like any vendor driven benchmark desperate to compete, It manages to compare wildly different things, just to find a case whereby they 'win'. Instead of comparing the stacks fairly, they focussed on comparing the data binding parts. What they proved in fact is that ADB (a proprietary Axis2 data binding framework) is faster than JAXB (data binding specification). OK, not so surprising. If you wanted to be fair and picked the fastest data binding for XFire too, you could test Axis2/ADB against XFire/JiBX (JiBX being a frightfully fast OSS data binding framework). You'd see that XFire shoves a sharp pointy plunger in the general (or maybe even not general) vicinity of the WSO2's development team's collective asshole.
In fact, buried in the so called benchmark, you'd find the results themselves prove that XFire is faster than Axis2 when the comparison is fair and uses the same binding framework. Somehow though, this rather crucial fact is lost in the conclusions.
What's more impressive is that no sooner than the company issues this highly dodgy report, its employees all start blogging about it and bragging that they finally managed to concoct a benchmark whereby they beat their competition.
Eran Chinthaka's (whoever the fuck he is) blog has a particularly hilarious entry, with such gems as 'We never did marketing for Axis2 telling bad things about other stacks'. Well my friend, I certainly am going to tell bad things about your stack, given your smug, snide, and ignorant post. The real icing though has to be '...then I feel Axis2 is even better than the numbers mentioned in the paper'. Huh? What, you, someone with a vested interest in your product, feel that the benchmark that you published and skewed to highlight your product, is not as biased as it should be? That reality, despite not even conforming to the distortion field imposed by your assumptions, somehow goes above and beyond it? I had no idea that the drugs available in Sri Lanka are so superior to those available to the rest of us.
This is of course, another 'open source' company. I thought that with JBoss being bought out and the old cabal being effectively neutered, we'd seen the end of the snide slimey comments by OSS against OSS, but our Sri Lankan friends seem keen to assume the throne of corporate fuckwittery in the name of OSS.
I hope this benchmark is taken as what it is, a company defending its own product against what is obviously a superior OSS solution. Friends don't let friends use Axis2. Do you want to base your webservices infrastructure on a product from a company desperate to take your money, written as a student project, with a bunch of incompetent weasels at the helm trying very hard not to look like used car salesmen? Do you want something that integrates and embeds well withint your existing infrastructure, or do you want something that hijacks it instead?
Still, good thing we have the goodship Apache; that thriving haven of open communities, that final refuge for corporate flotsam and jetsam, that last bastion against whored solutions foisted on an unwilling and uninterested public, eh?