Some people take a rather cynical approach to benchmarking by considering them to be inaccurate representations of what the database platform and hardware can do. They’re cynical because the benchmarkers take every opportunity to give their system every advantage they can. They view it as a sort of cheating since most real-world applications never use the techniques employed to gain high scores on a TPC benchmark. I, on the other hand, don’t look at it so pessimistically. After all, it’s only cheating if a vendor uses these unique and obscure techniques if they were the only one doing it. But quite to the contrary, every database platform that has ever undertaken a TPC test has used these exact same techniques. Since everyone is doing it and everyone is fully documenting the shortcuts they take, it can’t possibly be construed as cheating.
Consider raw partitions. When I used them back in my Oracle days (well over 15 years ago now), raw partitions provided a 5-8% performance boost at most at a high cost of administration and maintainability. I would never want to administrate a database deployed on raw partitions (although I did have to support a couple). Any way, raw partitions aren’t used much in commercial IT these days, but the fact that both Oracle and SQL Server and DB2 can and do use them for their TPC benchmarks puts all of them on an equal footing and makes the competition for highest score completely fair.
So consider the unique SQL Server startup parameters used by Microsoft and HP for their TPC-H benchmarking run: -c -x - E –T834 –T2301 -T661. Hmmm… They’re not immediately familiar. What do they do?
§ -x, Disables keeping of CPU time and cache-hit ratio statistics.
§ -c, Starts SQL Server independently of Windows NT Service Control Manager
§ -E, Increases the number of consecutive extents allocated per file to 4
§ -T2301, Enables more accurate query run-time behavior modeling in the SQL Server query optimizer usually only needed for large data set decision support processing.
§ -T834, Causes the buffer pool to use large pages on 64-bit installations of SQL Servers with 8GB or more of memory. These are allocated at startup and are kept throughout the lifetime of the process.
§ -T661 Turn off the ghost thread
So now I want to know more. For example, what is the ghost thread referenced for switch –T6661?
Interesting stuff, eh? What other questions does this bring up with you?
Cheers,
-Kevin
