XP SP3 outperforms Vista SP1, but less when running same Office version

A heavily promoted performance test by an evaluation software firm appeared to situate Windows Vista SP1 performance against Windows XP SP3. But the initial workloads were actually different due to the Office software used, testers admitted to BetaNews today.

Devil Mountain Software's test results comparing similar workloads on systems with varying editions of XP and Vista -- including the latest service packs or their equivalents -- show the Vista system performing astonishingly more poorly, by a staggering 144%.

But a breakdown of the team's initial tests reveal that, although they used identical Dell computers, they actually compared Office 2007 performance on Vista to Office 2003 performance on XP.

"All testing was conducted against fresh installations of the respective OS platforms," the CTO of Devil Mountain Software, Craig Barth, told BetaNews this afternoon. "In the case of Vista, we installed the RTM code only -- no updates were installed in order to preserve a pristine image. We then installed Office 2007 and the DMS Clarity Studio and Tracker tools. Once benchmarking was complete, we upgraded the installation with the v.658 build (RC0) of Service Pack 1."

For the XP test, Barth said his team followed a similar methodology, but one which equipped the older operating system with the older applications suite.