In the Exchange world, Storage Area Networks (SANs) are sometimes maligned by the people who have them, frequently coveted by the people who don't, and often mysterious to the general population. People react this way for multiple reasons, not least of which is the relatively small number of SANs deployed with Exchange (relative, that is, to the total number of Exchange installations). Overlaying these reasons is a fairly thick fog of misunderstanding (and even misinformation) that surrounds the best way to design a SAN for use with Exchange.
Misunderstanding number one is that Exchange will be happy if you create one big volume that spans all the disks in the SAN and put Exchange data on the volume. As with many misunderstandings, this one has a grain of truth to it. In some configurations, Exchange performance will be acceptable when the SAN is shared between Exchange and other applications. However, most of the time doing so is a recipe for substandard performance. The "Exchange Server 2003 Performance and Scalability Guide" at http://www.microsoft.com/technet/prodtechnol/exchange/guides/E2k3Perf_ScalGuide/687c4655-1050-40ae-8545-9fe7030d6d5d.mspx explains how you can estimate the number of concurrent I/O operations per second that Exchange requires per user in different scenarios. You can use that number to easily determine whether a given SAN configuration will cut the mustard.
Misunderstanding number two is related to number one. SAN vendors love to talk about their SAN caches, which tend to be stupendous in size. The problem is that Exchange I/O requests for data in Exchange .edb files are random, which means that the cache might not help nearly as much as the vendor claims. The cache will definitely improve performance for items that are read repeatedly, such as public folder posts, but it won't necessarily help mailbox access times. On a related note, despite what you might have heard, a big cache isn't really a substitute for large numbers of physical disk spindles. The spindle count directly affects how many I/O operations per second are available; for example, EMC claims that in a CLARiiON or Symmetrix array, a 10,000rpm disk should be expected to provide about 130 I/O operations per second, with about 180 I/O operations per second available from a 15,000rpm disk (see http://www.emc.com/techlib/abstract.jsp?id=1490 for EMC's recommendations for sizing and configuring the company's SANs for use with Exchange). Multiply this number by the number of disks and the I/O operations per second data from the Microsoft white paper, and you'll know how many spindles you need.
Misunderstanding number three is probably the worst. Almost every administrator understands that SANs aren't the same as Direct Attached Storage (DAS), but many administrators persist in trying to configure and manage them the way they configure and manage DAS. When you're designing a SAN for a particular Exchange configuration, spend a little extra money and hire an expert to validate your assumptions about performance and the proposed configuration. Microsoft, SAN vendors, and high-end Exchange consulting organizations all have experienced SAN engineers who can help make sure that the configuration you're buying provides adequate performance.
SANs are a terrific technology that can provide a great boost to availability, performance, and redundancy. With the advent of iSCSI, mid-range SAN prices are becoming reasonable for even fairly small organizations. But you have to understand what you're buying and what the technology will--and won't--be able to do.
