One thing this year's Transaction Processing Performance Council (TPC) TPC-C benchmarks prove is that while getting to the top is tough, staying there is even tougher. At the Windows Server 2003 launch earlier this year, SQL Server for the first time captured the coveted top nonclustered TPC-C score—the most prized system and database performance mark for demonstrating a platform's scale-up capabilities.
Some people say benchmark configurations don't reflect their real-world systems. But the TPC-C is as fair a performance measure as you can get. The TPC—made up of leading hardware and software vendors such as Hewlett-Packard (HP), IBM, Intel, Microsoft, NEC, Oracle, Sun, Unisys, and others—designed the TPC-C as an independently implemented and measured benchmark of sustained system performance. The benchmark simulates an OLTP order entry and shipping application in which multiple users execute transactions against a database. The database transactions include entering new orders, checking order status, delivering shipments, adjusting stock levels, and recording payments. The transactions per minute C (tpmC) score measures how many new orders you can insert into the system while it performs a given workload of other activities in the background. While the raw tpmC score tells which system can sustain the highest workload, the price/tpmC ranking tells you which platforms are the most cost-effective.
Although SQL Server claimed the top TPC-C clustered score last year, the nonclustered score eluded Microsoft until February 2003. Using Windows Server 2003 Datacenter Edition running SQL Server 2000 Enterprise Edition 64-bit on a 32-way 64-bit NEC Express5800 server, Microsoft posted a record score of 513,034 tpmC. Just 16 days later, IBM grabbed the top spot with a score of 680,613 tpmC, set on a 32-way 64-bit Power 4 eServer pSeries running AIX 5.2 and IBM UDB 8.1. But Microsoft turned around 11 days later with 64-bit SQL Server and Windows 2003 Datacenter once again and reclaimed the lead with a score of 707,102 tpmC, this time using a 64-way 64-bit HP Superdome system. The saga continued 16 days later as IBM ousted Microsoft with a score of 763,868 tpmC. Then, Oracle jumped to the top of the rankings with a score of 824,164 tpmC, running the new Oracle 10g database system on HP-UX and a 64-way Superdome. Obviously, all the major database vendors have their eyes on the nonclustered crown, and the leader in this race will no doubt change again before year's end.
The new 64-bit Itanium2 chip has enabled SQL Server to address enough memory to scale the peaks of the TPC-C ranks and prove that Windows and SQL Server are capable of providing the highest levels of enterprise performance. After SQL Server claimed the top clustered TPC-C score, analysts and rival vendors said the database still hadn't proved real system scalability. One direct result of Microsoft's clustered scores was the division of the TPC-C results into clustered and nonclustered categories. Given SQL Server's rapid ascent up the TPC-C clustered and nonclustered charts, I'm sure the database system will eventually be king of the entire TPC-C hill. But this year's nonclustered TPC-C results have already proved that SQL Server performance rivals that of any other database solution.
