I just took a new position with a firm that has a medium-sized Exchange Server SAN. All the disks are 18GB, which surprises me. Wouldn't it make more sense to use larger disks, given that the number of slots in the SAN is fixed?
The use of relatively small disks in RAID sets seems counterintuitive. After all, given the relatively low cost of disk storage, using three 400GB disks instead of dozens of 18GB disks would make more sense, right? As it turns out, there are several good reasons for using smaller disks in large RAID sets.
First is speed, or, more precisely, rotational latency. The number of I/O operations per second (IOPS) that a disk can generate is an important factor in how that disk will perform in a SAN or RAID array. A few years ago, the IOPS-per-disk sweet spot (at least for 10PS per gigabyte) was an 18GB or 36GB disk, so that's what people tended to use. As the data density of disks increased, that sweet spot moved upwards, but your company's SAN was probably put in a place before that trend occurred.
The second reason involves with the number of spindles in the disk set. The total number of IOPS that a disk set can generate is the product of the IOPS per disk and the number of disks. From a pure I/O speed standpoint, you'll get more IOPS—and thus better performance—from seven 18GB disks than you will from one 120GB disk (even though you'll also get about seven times the power usage, heat, and noise).
The third reason is rebuild time. If one disk in a RAID 5 set fails, the array will run slowly during the rebuild process. The bigger the disks in the array, the longer the rebuild time will be.
Finally, we get to what I suspect is the real reason: cost. Once a company builds an array with 18GB disks, it's hard to justify removing them and replacing them unless most or all of them fail, because you must replace them all together.
