Since the release of Windows Server 2003, competition for recognition as the fastest database platform has been red hot. On the day that Windows 2003 launched (April 24), Microsoft CEO Steve Ballmer announced that Windows 2003 Datacenter Edition in conjunction with Microsoft SQL Server 2000 Enterprise Edition 64-bit had taken the number-one performance spot in the TPC Benchmark C (TPC-C) nonclustered results. Running on a 32-way 64-bit NEC Express5800 system, the Microsoft platform posted a top score of 514,034 transactions per minute (tpmC).
Because UNIX systems had maintained a firm hold on the TPC-C performance pinnacle for the past decade, that achievement was a momentous milestone for Microsoft. But the company soon realized that life at the top is tough—16 days later, IBM ousted Redmond from the top spot with a score of 680,613 tpmC. IBM achieved its score with a 32-way eServer pSeries system that used DB2 Universal Database (UDB) 8.1 as a database platform and AIX 5L V5.2 as the OS. Microsoft wasn't about to let IBM tarnish its Windows 2003 launch that easily, however, and 11 days later, with the help of Hewlett-Packard's (HP's) 64-way HP Superdome system, Microsoft reclaimed the top spot with a TPC-C score of 707,102 tpmC. Currently, SQL Server on an HP Superdome holds the runner-up position behind an OracleHP configuration.
Although this battle clearly is far from over, Windows 2003's climb to the top of the performance heap showed that Windows can scale to match any available system. Understanding the TPC-C can help you understand why it's a valid and fair benchmark.
What Is the TPC-C?
The TPC-C benchmark is designed by the Transaction Processing Performance Council (TPC), an organization consisting of the leading hardware and software vendors. Working together, the members constructed a set of benchmarks for measuring system throughput, which the TPC-C benchmark defines as sustained system performance. The involvement of all the major system vendors helps to ensure that the benchmarks are fair to all participants. Other TPC benchmarks are the TPC-H, which measures ad hoc decision support systems; the TPC-R, which is a decision-support benchmark for business reporting systems; and the TPC-W, which measures transactional Web-based e-commerce systems.
As the most widely known benchmark and the one that's most representative of typical system usage, the TPC-C measures the performance of online transaction processing (OLTP) systems. The benchmark simulates an OLTP order-entry and shipping application that supports multiple users executing transactions against a database. The transactions include entering new orders, checking order status, delivering shipments, adjusting stock levels, and recording payments. Because of advances in database scale-out technologies, the TCP-C is split into clustered (scale-out) and nonclustered (scale-up) results. For those who are interested in knowing more, the TPC describes the TPC-C benchmark in detail at http://www.tpc.org/tpcc/detail.asp.
The performance measurement used in the TPC-C benchmark is the tpmC score. The tpmC is essentially a measurement of how many new orders can be input while the system continues to perform a given workload of other OLTP transactions in the background. The two primary TPC-C metrics are the raw tpmC score, which tells how fast a system is regardless of price, and the price/performance ratio ($/tpmC), which measures how cost-effective a system is. For example, the SQL ServerHP Superdome solution has a $6.49 price/performance value; the OracleHP price/performance value is $8.28. Currently in its fifth version, the TPC-C benchmark continues to evolve to remain representative of the latest application practices.
What Does the TPC-C Mean to You?
Two of the questions that I most commonly hear about the TPC-C are "Does it really represent the fastest systems?" and "Is it relevant to my environment?" Because the TPC-C is a database application performance test, it doesn't necessarily provide insight about file server or Active Directory (AD) performance. However, the TPC-C is a useful yardstick for comparing application platforms, which most Windows Server systems are.
Although no customer's system is exactly like the application that the TPC-C benchmark uses, one valuable piece of information that everyone can gain from the TPC-C is the ultimate upward scalability of a given platform. Microsoft's attainment of the top TPC-C spot proved that Windows and SQL Server can scale up to levels that rival even "big iron" mainframe systems.
