Amazingly enough, I'm not going to mention SQL Server for the rest of this column. Instead I'm going to defend the integrity and reputation of database professionals—regardless of their product affiliation. We're all brothers (and sisters) in data and must stick together when the going gets tough.
A weekly trade journal recently published an interview that contained the following comments about building e-commerce sites:
"Q. What kinds of technical challenges are you seeing in your role as CTO of \[xyz.com\]?
A. ...I've spent more time on caching in the last four years than anything else; that's the only way you get performance. And the reason is that databases are so damn slow....
Q. Can you give me an example of how the lack of database performance affects your www site?
A. ...One of the things on our www site that the customer service reps use is the ability to look up users. It's certainly not done using indexes, and that's by default. When we started, the database was fairly small, and it seemed to make a lot of sense. You didn't have to be that accurate. Now it can take 20 seconds if they don't press the little button that says 'Exact Matches Only'. I guess that's the thing a lot of people don't get. They keep thinking that the RAM, disk, and everything else is all free. They think that it's not really a big deal...."
I salute you Mr. CTO for recognizing that "RAM, disk, and everything else" aren't free and aren't the magic answer to your performance problems. But I beg to differ with some of your other inferences. This Q&A exchange highlights one of my biggest frustrations with application development in general and "e-business" (whatever that means) development in particular. The same companies that spend gobs of cash and lavish pre-IPO stock options to hire incredibly talented application architects typically spend paltry sums on database design, maintenance, and optimization and the related human talent. Then these organizations wonder why performance is so slow. Even worse, they blame poor performance on the database instead of accepting that they did a sloppy job of designing their applications to interact with the database.
Let me rephrase the CTO's responses from the perspective of a data-tier architect: "We built a customer look-up process to support our front-line customer service reps. Unfortunately, we did a horrible job. We built a system that requires table scans and massive amounts of I/O rather than using indexes or other high-speed search mechanisms to serve a service rep who's looking for a customer but doesn't have an exact name match. Everything worked great during design and development using fake data, but we forgot that the code might take longer to run when we added real customers to the system. Darn! I sure hate having to work with slow database technology." Perhaps we can recommend a 12-step program to help application folks who are still in denial about their applications' poor interaction with key databases.
One dictionary definition for the word "system" is "a group of interacting, interrelated, or interdependent elements forming a complex whole." We call applications "systems" for a reason. Still, I'm amazed that many organizations treat database integration as an afterthought rather than a key interrelated, interdependent element of a complex whole. You can put an "e" in front of anything you want, but databases are the core of everything "e."
Why don't companies spend more time building database applications that can scale? Beats me, but I'm not complaining. Evil consultants like me can charge lots of money to fix application and database performance problems after the fact. Remember, databases don't kill performance, people do.
