Previously in WinInfo, I discussed a couple of tests performed by PC Week and PC Magazine that "corroborated" the controversial Mindcraft study, which showed Windows NT beating the performance of Linux by an order of magnitude. The Mindcraft study was soundly derided by the Linux community, and for good reason apparently, as the NT systems used in the study were heavily tweaked while the Linux systems were not. And of course, Microsoft commissioned the Mindcraft study. Still, I feel that the Mindcraft study was a turning point in the public perception of Linux, which had previously existed in a sort of odd invulnerable bubble of nonsensical critical acclaim, despite the fact that most people crowing its many charms had spent little or no time with it. Of course, the Linux community is working at a heady pace to correct existing problems, and I've always said that Linux represents the most obvious threat to the Windows monopoly that's ever existed. I stand by that claim today, for both the desktop and the server.
When I saw the PC Week and PC Magazine reports, I felt that we had moved firmly into a newer, more realistic assessment of Linux's prowess. Both PC Week and PC Magazine, I determined, had presented realistic appraisals of Linux when compared to Windows NT Server. But my statement that these tests "corroborated" the Mindcraft study is, it seems, a bit off target. Sure, both of these new tests show NT beating Linux for the most part, but they do not show the gap to be as large as the Mindcraft report suggested. And Linux did come out ahead in one interesting area. When I received an intriguing email from Samba's Jeremy Allison that better explains the situation, I figured it would be a good idea to share it with WinInfo.
For those of you who haven't heard of him, Jeremy spent about a month in the PC Week labs to ensure that their Linux installs were tweaked properly to achieve the best possible scores, putting them on equal footing with their NT counterparts. Jeremy is a key player in the Linux community but, as he says, he has no particular axe to grind. He, like myself, simply wants the truth to get out. And the truth, as always, lies somewhere in the middle of opposite and extreme claims.
"Essentially my position is that the PC Week benchmark shows that Samba on Linux needs more work to match Windows NT as a file server for Windows 95/98 clients, but is a better file server than Windows NT Server for Windows NT clients," Jeremy wrote. "The PC Week numbers bear this out."
In fact, you can find the results of the PC Week test pitting Windows NT vs. Linux, Solaris, and Netware on the PC Week Web site here and here.
Jeremy says that PC Week's results are very similar to the results he gets independently in SGI's labs: Windows NT beats Linux and Samba handily when serving Windows 95 clients. However, and perhaps most perplexingly, Windows NT Server performed more slowly than Linux when the client was Windows NT.
"I'm not trying to bash Windows...I also want to get the truth out there so that customers can decide for themselves (obviously I'd like them to choose Samba, but there are many circumstances where I'd happily admit NT is a better choice)," he wrote. "Knowing that NT beats Samba and Linux with Win95 clients has caused a great deal of activity to fix the scaling problems in the Linux kernel so I'm really glad of the real numbers being out there (and I do trust the PC Week figures, they're a very honest and reputable lab)."
Again, thanks to Jeremy Allison for his response.
--Pau
