|
Executive Summary: Microsoft Office Outlook uses the Offline Folder file (OST) to synchronize files for offline access but increasing mailbox sizes have led to sluggish performance because OSTs weren't designed to handle such large files. You can compact OSTs, run Scanost.exe, or rebuild OSTs to improve performance; however, Microsoft Office 2007 SP2 addresses the performance problem and provides quicker responses on large mailboxes. An additional fix lets Outlook shut down faster, which helps avoid OST corruption. |
Microsoft Exchange Server administrators don't typically get very excited about a Microsoft Office system service pack, perhaps because it's usually someone else that takes care of updating software on PCs. However, with Office 2007 SP2, Exchange admins—and end users—might have reason to celebrate. Office 2007 SP2 includes fixes for a major performance problem in Microsoft Office Outlook 2007 that has affected Microsoft Exchange Server for years. Even better, because the results of the performance work were so good, Microsoft decided to release an update for Outlook 2007 SP1 in the February 2009 Cumulative Update that you can apply immediately without waiting to roll out the full SP2. Let's take a look at why Outlook performance has suffered on large mailboxes and examine the improvements that Microsoft has made.
The Zen of the OST
MAPI-based Exchange Server clients have always supported the Offline Folder file (OST). Back in the mists of time, early versions of Outlook and the original Microsoft Exchange Viewer used OSTs to synchronize folders for offline access on an on-demand basis. Important folders such as the Inbox were synchronized automatically, but if you wanted to use other folders offline, you had to configure the client to synchronize them whenever the it connected to Exchange. Folders in the OST are replicas of the online master folders in user mailboxes; the synchronization process keeps the two copies aligned. Unlike PSTs, which can be opened by any MAPI client, an OST is always linked to a specific user mailbox and can be opened only by a client that can connect to that mailbox.
Simple synchronization was sufficient in the era of dial-up networking when a 56Kbps connection was something to celebrate. A major change occurred when Microsoft introduced Outlook 2003 alongside Exchange Server 2003 and we entered the era of Cached Exchange Mode. Exchange 2003 introduced features such as drizzle-mode synchronization and a bunch of networking improvements that Outlook exploited to capture a complete copy of user mailboxes in the OST. This change was great from a user perspective because you had a complete copy of your mailbox to work on even when the network wasn't available.
The introduction of Cached Exchange Mode was popular with administrators because it facilitated server consolidation. Previously, clients had to connect to a local Exchange server when suitable communications weren't available to allow online access to servers in centralized data centers. With Cached Exchange Mode, those same clients could connect over extended links and synchronize behind the scenes while users worked with data in the replica folders in the OST. Cached Exchange Mode also isolated clients from temporary network outages, and the Outlook 2003/Exchange 2003 combination was more effective in using network bandwidth so more clients could work across the same link.
Maintaining OST Performance
Cached Exchange Mode remains a tremendously important aspect of Exchange design today. However, the OST file structure wasn't designed to handle the size of mailboxes that are now common. OSTs cope splendidly when only select folders are synchronized from small mailboxes; they're much less effective when you synchronize all the folders from multigigabyte mailboxes. My personal experience is that performance erodes after an OST grows to more than 1.5GB, more than 5,000 items in a folder, or more than 20,000 items in the OST, but this can vary according to the speed of your PC's hard disk (as measured in its ability to transfer data; PCs equipped with solid-state disks are typically the fastest, those with 7,200 RPM standard disks are the next quickest). You know an OST is having problems if the hard disk indicator of your PC stays constantly lit when Outlook opens a new folder or performs other operations that force Outlook to access data in the OST.
An OST is typically between 10 percent and 15 percent larger than its source mailbox because of the way that data is organized in the file and the internal structures that index the data. You can't restore performance simply by deleting items from your online mailbox to reduce the size of the OST and the number of items it holds because Outlook has to perform some processing to compact the OST. This work normally is done when Outlook isn't busy, but you can force Outlook to compact the OST, as Figure 1 shows. To do so in Outlook 2007, go to Tools, Account Settings. On the Data Files tab, select the OST from the list, then select Settings. On the Advanced tab, click Offline Folder File Settings, then click Compact Now. This operation can take from a few seconds to a few minutes to complete depending on the size of the OST.
Even if an OST is as small as it can be, it still might not be very efficient because internal structures within the OST degrade over time. You can run the OST Integrity Check tool (Scanost.exe), which Figure 2 shows, to check and fix any internal problems. The process is roughly equivalent to defragmenting a hard disk or running the Isinteg utility to perform integrity checks on an Exchange database. See the Microsoft article "Scan and repair corrupted Outlook data files" for details about what Scanost does and how to run it.
Microsoft introduced Scanost with Outlook 2007 specifically to deal with OST files. Earlier versions of Outlook include Scanpst.exe that can repair OSTs and PSTs, and I can only conclude that the Scanost program is smarter at dealing with internal OST structures than Scanpst. When Scanost is finished processing, it creates a message in your Deleted Items folder to report its activity. Figure 3 shows a sample message.
If an OST still doesn't perform well after compacting and scanning, the last thing you can try to restore an OST to good health is to rebuild it from scratch. Think of this as the equivalent to running Eseutil to rebuild an Exchange database. You have to exit Outlook and delete the OST—although people who are more cautious might opt to rename it instead, just in case—and then restart Outlook to force the client to recreate the OST.
Just like you wouldn't recommend that Exchange administrators rebuild databases regularly—a task not necessary with Exchange 2003 and later, even if the mythical need persists in the imaginations of many—rebuilding OSTs isn't something to recommend users do regularly. Recreating an OST can take a few hours depending on its size, number of items it contains, and the speed of the network connection to the Exchange server. You definitely don't want to recreate an OST over a slow network link or when the server is under heavy load because these factors slow processing considerably.
The Importance of OST Speed
A rebuilt OST isn't a guarantee of snappy performance; the inherent flaws of the OST structure when coupled to a large mailbox can still condemn you to slowness. That's why it was so important for Microsoft to invest some engineering effort to help OSTs cope with large mailboxes. Microsoft has known for some time that large OSTs perform like slow pigs, and it's surprising that they didn't address the problem in Outlook 2007. The growing size of mailboxes and the increasing number of mailboxes connected to Exchange 2007 and Exchange 2003 servers made the problem more evident.
In addition, Microsoft needs to make it easier for users to have even larger mailboxes in the future so that it can compete with Google and other online vendors that offer free mailboxes in the 5GB to 10GB range. Some organizations already run Exchange servers with very large mailboxes; a 50GB mailbox isn't uncommon for a senior corporate executive, especially those who are targets of legal discovery actions. The slowness of the OST can make using large mailboxes a real pain even if you're using the fastest, most up-to-date hardware.
The good news is that the fixes now available for Outlook 2007 SP1 and in Outlook 2007 SP2 deliver much better performance and responsiveness for OSTs connected to large mailboxes. It’s hard to measure just how good the improvement is because no tools are available for this purpose. My nonscientific tests show that the beta version of Outlook 2007 SP2 spends less time causing the hard disk light to come on and provides a much more responsive performance with mailboxes of up to 10GB—in other words, you definitely experience fewer "stutters" when Outlook opens a large folder or changes views. I didn't attempt to test a mailbox larger than 10GB. I anticipate that the final SP2 code will deliver even better performance after Microsoft removes the debugging code that it usually includes in beta software.
Relieving a Shut Down Problem
Along with speeding OST performance, Outlook 2007 SP2 changes how the program shuts itself down. The details are explained in "Application Shutdown Changes in Outlook 2007 SP2," but basically the problem was that Outlook took too long to shut down after a user requested the application to close. Users would sometimes become frustrated that Outlook didn't shut down as promptly as they expected, which led them to terminate the process using Windows Task Manager.
Forcing Outlook to quit with Task Manager certainly is effective. However, this process can cause Outlook to fail to write cached data to disk properly and so corrupt the OST. The next time Outlook starts, it detects the corruption and fixes it, but while Outlook is scanning for problems and fixing whatever it finds, it's slower to respond to user requests and therefore creates the impression of being a performance hog. The change to force snappier shutdowns makes users happy and stops itchy users wanting to kill processes, so all in all it's a good thing.
Upgrade for Better Performance
It would be nice if every change that Microsoft makes to Outlook had such an obvious and immediately positive effect as the company managed to do in these updates for OST performance. Deploying a new service pack or hotfix for any software can be a painful and costly affair, especially if you lack the ability to distribute the new software and apply it automatically to all the PCs in your organization. It's not always obvious that the costs of such an exercise will result in any measurable benefit.
However, if you have the opportunity to deploy the February 2009 Cumulative Update or Outlook 2007 SP2 in the near future, the enhanced performance will delight any user whose mailbox is larger than 1GB. Given our ability to be human packrats and grow mailboxes to sizes that we never contemplated a few short years ago, better performance and support for large mailboxes could be just the reason you need to justify the upgrade.