The benefit of hindsight is a wonderful gift to technologists. It lets us understand how changes that were made in the past laid the foundation for current capabilities. In the case of Microsoft Exchange Server, three types of deployment are now possible: classic on-premises, cloud with Microsoft Office 365, and hybrid, in which some parts of the infrastructure reside on premises and some reside in the cloud.
Exchange didn't arrive at this point by accident. The Exchange development group has done a lot of heavy lifting over the past decade so that customers can enjoy the choices they have today. Looking back, I see seven important areas in which innovation has liberated Exchange from its origins as a LAN-based email server running on Windows NT. Apart from all else, I think that these areas represent the most crucial areas for Exchange administrators to master in the foreseeable future.
The Magnificent Seven
Perhaps it's unfair to focus on just seven areas, when Exchange is now a huge product that spans well over 20 million lines of code. I freely admit that other areas of innovation within Exchange deserve consideration.
For example, there's the elegance of single page patching within a database availability group (DAG). This implementation allows the Information Store to detect page corruptions and then broadcast requests to other database copies to retrieve the data necessary to fix the problem, all while keeping the database online and serving users.
Even so, I'm happy to stay with the group that I've selected. In no particular order of importance, these are my chosen areas of innovation:
1. Remote Procedure Call (RPC) over HTTP
2. Windows PowerShell
3. Autodiscover service
4. Mailbox Replication Service (MRS)
5. Multibrowser interfaces
6. Storage improvements
7. Client Access server
Now, let me explain my logic.
#1: RPC over HTTP: Eliminating VPNs
In the early days of Exchange, remote access was characterized by synchronization woes, slow dial-up connections, and VPNs. Improvements in client technology -- such as the drizzle-mode synchronization that Microsoft Office Outlook 2003 introduced -- and the now-ubiquitous nature of fast wireless connections have eliminated the first two issues. And the need to establish a VPN before connecting to Exchange was firmly whacked on the head by the introduction of RPC over HTTP in Exchange Server 2003.
Now known as Outlook Anywhere, RPC over HTTP was initially difficult to configure. Those who persisted and published the necessary public connectivity points discovered that Outlook could connect easily. Furthermore, it could use the public Internet to transport, safely encapsulated in HTTP packets, the Messaging API (MAPI) remote procedure calls (RPCs) that form the basis of any communication between Outlook and Exchange. Administrators loved the fact that ports 80 and 443 were the only ones that they needed to open in the firewall, especially because these ports are usually open to support other web-based applications.
Since Exchange 2003, Microsoft has gradually improved the configuration and management of this component. Now, Outlook Anywhere is a real strength of the product, so much so that it provides the fundamental underpinning of both Microsoft Business Productivity Online Standard Suite (BPOS) and Office 365.
After all, requiring every customer to create a VPN to access Microsoft Exchange Online would be impossible and too expensive to create and maintain for many small-to-midsized businesses (SMBs). Such a requirement would also create a huge burden for Microsoft, which would need to manage the incoming VPN connections.
Simply put, Outlook Anywhere allows everyone to connect across the Internet. Plus, the $6 per month price point for Office 365 is feasible. That's why RPC over HTTP is #1 on my list.
#2: PowerShell: Delivering a Common Management Platform
PowerShell might seem a strange choice for #2. But its introduction in Exchange Server 2007 and subsequent upgrade to remote PowerShell in Exchange Server 2010 have delivered many benefits. First, PowerShell provides consistency across the management interfaces within Exchange. In other words, you can use the Exchange Control Panel (ECP) to update a mailbox's properties in Office 365, or you can run the Set-Mailbox cmdlet by using PowerShell. Both routes lead to the execution of the same logic.
But my main reason for selecting this component is that the advent of remote PowerShell delivers the ability to manage Exchange Online without needing to log on to the servers on which Exchange runs. Obviously, Microsoft couldn't permit thousands of Office 365 customers access to mailbox or Hub Transport servers. But remote PowerShell allows domain administrators to connect across the Internet and validate their credentials for a tailored session that contains the exact set of cmdlets that they're allowed to run. And because PowerShell forces all paths to the same logic, the user can connect by running Exchange Management Shell (EMS), or through ECP, or through the Microsoft Management Console (MMC)-based Exchange Management Console (EMC): Everything works in the same way. That's why PowerShell is my #2 choice.
#3: Autodiscover: Solving User Pain
Microsoft introduced the Autodiscover service in Exchange 2007 as a solution for the perennial problem in which users had trouble configuring Outlook with the parameters that were necessary to connect to Exchange. The basic difficulty: Server names that make perfect sense to administrators put users into a deep sleep. Apparently, regular users have problems coping with names such as EXSERVER1 or MBXHUB-LONDON.
Microsoft figured out that the issue could be solved by making computers communicate to figure out a user's mailbox location. That's what Autodiscover does, by consulting signposts such as a service connection point (SCP) in Active Directory (AD) to retrieve information about Exchange and to discover a mailbox's current location. Of course, AD is available only inside a firewall, but Autodiscover can use URLs that are published to the public Internet to retrieve what it needs. This capability laid the foundation to allow easy connection to Exchange Online in BPOS and Office 365. Without Autodiscover, administrators would face far more complexity and cost when they configured user PCs to connect to the cloud.
Autodiscover works well for Outlook 2010 and Outlook 2007, the only two PC clients that support the feature. It also works well with the latest ActiveSync clients, Outlook 2011 for Macintosh, and Entourage 2008. Microsoft expanded the information that Autodiscover returns to the client in Exchange 2010. Autodiscover can now retrieve data about alternative mailboxes that Outlook should open (i.e., mailboxes to which the user has been granted full access) and URLs that map Exchange services such as the Offline Address Book (OAB) distribution point, Exchange Web Services, Unified Messaging, and so on. Exchange returns all this information as XML data to Outlook, which refreshes the information every 15 minutes to ensure that it keeps up-to-date with any changes.
You can use Outlook's Test E-mail AutoConfiguration option to gain an insight into the information that Autodiscover retrieves. To run the option, press CTRL and right-click the Outlook icon in the system tray and select Test E-mail AutoConfiguration from the menu. Clear the Use Guessmart and Secure Guessmart Authentication options, enter your email address and your password, and click Test. Outlook goes through the same process that it uses when it populates a user profile for the first time. Click the XML tab to see the data returned by Exchange.
As Figure 1 shows, this data includes details such as the server name and the URLs that are necessary to connect to Exchange Web Services (EwsUrl) and ECP (EcpUrl). In this example, I've connected to my Office 365 mailbox, so the results are those that come back from Office 365.
Figure 1: Results of an Autodiscover test run
It's worth noting that Autodiscover is one reason why Microsoft doesn't support connecting Outlook 2003 clients to Office 365. Although you could manually configure all the server settings in Outlook 2003 to make a connection to a cloud-based mailbox, the automatic nature of profile updates that Autodiscover performs in Outlook 2010 and Outlook 2007 facilitates easy movement of mailboxes between servers. Microsoft continually adds servers as it builds out its Office 365 infrastructure, and then rebalances workload across available servers by transferring mailboxes between databases. If Outlook 2003 clients were connected, users would need to know what to do if their mailboxes were moved. The result might be a support nightmare. Restricting support to Outlook 2010 and Outlook 2007 solves that potential support headache and encourages users to upgrade to software that can take advantage of the most recent server features (e.g., MailTips), so it's a logical win-win situation for Microsoft.
Autodiscover keeps Office 365 administrators from running around to help users more than they already need to do, so it's a good choice for #3.
#4: Mailbox Replication Service: Smooth, Elegant Moves
Large mailboxes are all the rage today. I can't quite understand why, as the vast majority of humans take a long time to fill a 25GB mailbox. Still, the advent of massive mailboxes has made the task of moving them a much more complex business than it used to be.
Until Exchange 2010, administrators had to ask users to log off before their mailboxes could be moved. The mailboxes moved in real time, and nothing could be done until the full and complete mailbox reached its destination. If something happened during the move, you had to restart. Although workable, this solution is ineffective when mailboxes need to move from on-premises servers to the cloud and vice versa. And as I mentioned earlier, Microsoft needs to operate in a state of almost perpetual mailbox moves to rebalance databases as it builds out its Office 365 infrastructure.
Cue the introduction of MRS in Exchange 2010. Mailbox moves now occur in the background, with frequent checkpoints and sufficient intelligence to acknowledge when the receiving server might be under pressure. Although moves cannot be scheduled (a deficiency that Microsoft will surely address in the future), they can be auto-suspended. Such moves copy an initial snapshot of a mailbox's contents, in the background, and then pause. An administrator can check the mailbox move report and resume the move if all seems well. MRS then takes another snapshot of the source mailbox and performs an incremental synchronization before switching pointers in AD to direct the user to the new location.
Best of all, MRS is extremely competent at transferring mailboxes between AD forests, from on-premises deployments to the cloud, and between versions of Exchange (from Exchange 2003 onward). Without the service it's difficult to see how Microsoft could cope with the transfer of millions of mailboxes to Office 365; the manual workload involved in setting up and managing mailbox moves across the Internet would be massive and costly. Instead, many organizations use the auto-suspend technique to move mailboxes in the background, as many as 30 days before a user is finally transferred to Office 365.
And MRS does more than move mailboxes. Microsoft has expanded its responsibilities to manage tasks such as import and export of mailbox data from personal folder stores (PSTs) and mailbox restore requests. Because of its key role in mailbox management, I think MRS deserves its position as #4 on my list.
#5: Multibrowser Support
No self-respecting cloud service can run without offering browser support. In Exchange, browser connectivity began in 1997, when Exchange 5.0 shipped the first version of Outlook Web Access (OWA), now charmingly renamed Outlook Web App (but still OWA). In truth, the first version of OWA wasn't very good. But much has been learned about browser interfaces, standards, and functions since 1997.
Today's OWA boasts a solid and highly functional UI that puts competitors such as Google Gmail to shame. OWA connects with ease to both on-premises and cloud versions of Exchange and supports a rich user experience on Internet Explorer (IE), Google Chrome, Mozilla Firefox, and Apple Safari. Other browsers can connect with the basic version of OWA -- still highly usable, even if missing some of the bells and whistles of its premium counterpart.
An equally important evolution is occurring for browser-based management interfaces. Exchange 2010 introduced ECP -- and promptly confused administrators because some tasks, such as discovery searches or ActiveSync device policy maintenance, can be performed only through ECP. No trace of these tasks appears in the flagship EMC. Equally frustrating, some tasks, such as DAG management, show up only in EMC. For example, DAG management can be performed only through EMC, which boasts other convenient features such as wizard-based execution of complex tasks (e.g., certificate management) and the ability to show administrators the underlying PowerShell code that will be executed to accomplish a task.
However, the important thing to realize is that ECP is in its first iteration. Web-based management is a crucial piece of the evolution of Exchange Online within Office 365. Over time, I expect more and more administrative tasks to show up in ECP, until the need to maintain two management consoles disappears. I predict a time when Microsoft removes EMC from Exchange, leaving us with a single browser-management client that hopefully includes all the EMC features that administrators depend on.
ECP already supports the same range of browsers as OWA, and Exchange 2010 Service Pack 1 (SP1) introduced the ability for users who aren't mail-enabled to use ECP to perform administrative tasks. Because of their key roles for both users and administrators (today and in future cloud services), I give the OWA/ECP combination the #5 slot on my list.
#6: Storage: Cheap and Cheerful Bytes
Storage was the single biggest cost bucket for Exchange 2003 deployments. This can be partially explained by the cost per gigabyte back in 2003 to 2005, but the I/O profile that Exchange generated was more relevant. To be blunt: Exchange 2003 was an I/O hog. The only way to get reasonable performance was to deploy expensive storage that could deliver the required I/O operations per second (IOPS) performance to support the desired number of mailboxes. SAN storage -- expensive to buy and equally expensive to set up and maintain for Exchange -- was often necessary.
Microsoft tweaked the Information Store in Exchange 2007 to improve its I/O profile, with considerable success. Fundamental change then occurred in Exchange 2010, with the first database schema change since Exchange Server 4.0. Many other updates were made to drive down I/O requirements and deliver the ability to run Exchange on cheap DAS and Just a Bunch of Disks (JBOD) arrays. As a result, the I/O requirement of one IOPS per mailbox that existed in Exchange 2003 is now down to circa 0.1 IOPS. Furthermore, Exchange 2010 can process huge mailboxes stuffed with hundreds of thousands of items -- a situation that would have brought Exchange 2003 to its knees.
If Microsoft hadn't invested in the engineering effort to make such a dramatic improvement to database performance, it wouldn't be able to offer 25GB mailboxes in all but the most basic Office 365 kiosk plans, and it would be unable to compete with Gmail. For those reasons, storage improvements are innovation #6.
#7: Client Access Server: Exchange's Black Box
The Client Access server role first appeared in Exchange 2007, as the common endpoint for Internet protocols. The role was then enhanced with the RPC Client Access layer in Exchange 2010, to become the endpoint for MAPI clients. The Exchange 2010 Client Access server also hosts MRS, so it plays an important role in mailbox moves.
The transition from mailbox server to Client Access server is why I think this component is important. Without the Client Access server, Exchange would have been unable to break the connection between mailbox and server that existed prior to Exchange 2010. The high availability gained through DAGs would also have been impossible because we would be unable to switch databases easily and quickly between servers. And without DAGs, Microsoft probably would have been unable to deploy Exchange Online as it has, with multiple database copies split across geographically separate data centers. The upshot is that Microsoft might not be able to offer the financially backed 99.9 percent service level agreement (SLA) that it offers for Office 365.
But there's more. The Client Access server is key to the effective management of incoming client connections. On-premises administrators will be aware of the interaction between firewalls, load balancers, and the Client Access server to manage HTTP, MAPI, IMAP4, POP3, and ActiveSync clients. Between Windows Server 2008 R2 and Exchange 2010, important changes -- such as the ability to combine individual Client Access servers into Client Access server arrays and better handling of session identifiers -- allow servers to handle a greater volume of connections more effectively. This ability is hugely important, given the tens of millions of clients that Office 365 is expected to manage. That's why the Client Access server takes the final position in my list.
I'd be happier with my choice if Microsoft provided functions to allow administrators to understand exactly what the Client Access server is doing at any point. Right now, the Client Access server is often a black box, with connections going in and out with no indication of whether everything is progressing as expected. Perhaps Microsoft will do something to improve the telemetry and feedback from the Client Access server in a future version of Exchange, if only to help the folks who run Office 365.
What's Next?
I won't be offended if you disagree with the logic behind my choice of the magnificent seven. Have fun reordering my list to assign your best idea of the relative importance of each component. The important thing is that Exchange has been gradually evolving over the years, to the point that it can now cope with the demands of three very different kinds of deployment.
I'm not sure that the engineers realized how all these pieces would fit together when they set out to design any one component, but blessed serendipity has brought us to the current situation. I'm quite sure that development isn't complete and that you'll see massive evolution in Exchange over the next few years, as the product heads for its 20th anniversary in 2016. If we revisit the assessment at that time, I think some of the same components will be on the list. The question is which new components will appear over the next four to five years?
