The story of Exchange IOPS: How a crusade to make Exchange less of a storage hog enabled a successful cloud service

The history of software is littered with examples of grand strategic changes that never amounted to much when implemented. The crusade to reduce the number of disk I/O operations generated per user per second (IOPS) is an example of how a big bet paid off.

If we look back to Exchange 2003, if you wanted to deploy the software to serve more than a few hundred mailboxes, you had to invest in expensive Storage Area Network (SAN) hardware. SAN technology got a bad reputation in places as being complex and prone to problems if it wasn’t configured just so. That feeling lingers today, but when a SAN was deployed, configured, and managed properly, it is capable of delivering great performance for a range of applications.

Unfortunately, a SAN costs. The storage smarts, high-speed connections, expensive disks built for reliability and robustness, and all the surrounding technology makes a SAN something that is expensive to sell, service, and operate. In short, the need for SAN-quality storage created a dependency that Exchange could live without.

A focused effort to reduce the IOPS footprint required by the Information Store started soon after Exchange 2003 appeared. As depicted in the table below, that effort has been extraordinarily successful in driving the IOPS per user from 1.0 to around 0.05 today (lower in Exchange 2016 if you use lagged database copies). As with all performance data, you have to remember that input parameters exert huge influence over results. In this instance, the data is based on an IOPS profile of 100 messages per day.

(*) The official guidance from Microsoft is that Exchange 2016 and Exchange 2013 deliver the same IOPS performance. However, I checked the lower figure before publication and am happy that this can be achieved and indicates that continued progress is being made. For planning purposes, it's best to use the higher figure, which is the basis used by tools such as Microsoft's Exchange Server Role Sizing Calculator. The redoubtable Ross Smith IV would always prefer you to use 0.07.

Note: SATA or SAS drives can be used with Exchange 2013 or Exchange 2016. Microsoft prefers SAS and says in its discussion about the preferred architecture: “7.2K RPM serially attached SCSI (SAS) disks (while SATA disks are also available, the SAS equivalent provides better IO and a lower annualized failure rate).”

Change has come about through changes in the software at the ESE database layer and the Information Store allied to fundamental architecture changes in the high availability space to deliver the Database Availability Group (DAG). Taken together with the dropping cost of storage and the growing popularity of JBOD, the net result is that the topic of storage for Exchange leads to a radically different discussion today.

Lots of expense is removed from the deployment equation when the need for high-end storage disappears. Driving storage cost so low creates the ability to run cloud services like Exchange Online at price points that would be impossible if SANs were required.

In fact, the crusade to shrink IOPS has not only proven to be the right call, it also made it possible for Microsoft to lay the foundation for the transfer of so much user data to Office 365 over the last five years to serve over 60 million active mailboxes as part of a commercial cloud business that’s now running at an $9.4 billion annual revenue run rate.

There’s no way that Microsoft could afford to provision 50 GB of storage per mailbox and charge the prices they currently offer for Office 365 plans. In fact, 50 GB is just the start, because there’s that expandable archive mailbox as well plus space for Recoverable Items, and so on. Put it this way, a single Office 365 user could easily fill a complete old-style 146 GB enterprise-class SAN drive.

As MVP Paul Cunningham points out in an article, good reasons still exist to use a SAN to provide storage to Exchange, especially if your company takes a storage-centric view rather than an application-centric view of how resources should be deployed. The good thing that arises from the design decision made by the Exchange developers twelve years ago is that choice now exists. SAN or JBOD – it’s up to you.

Follow Tony @12Knocksinna