Coping with Swelling Mailboxes: Why Outlook 2013 Changes Caching

It’s always fun to try and understand the logic followed by developers when they add features to a new version of a well-established product, especially when they change the behaviour of the product. I wrote about the change in OST caching enabled in Outlook 2013 Preview on September 6 and concluded that the way that Outlook 2013 only cached a year’s email by default had the potential to confuse users.

People generally don’t like it when email vanishes, even when there’s a good reason why this happens, as in the case when Exchange 2010’s Managed Folder Assistant (MFA) diligently processes mailboxes according to retention policies. The “disappearing mail” syndrome can generate many help desk calls along the line of “I know I had that email yesterday and now it’s gone. Please get it back for me”.

Outlook 2013 also changes the internal format of the OST to compress some data fields in an attempt to make the file smaller on disk. You can argue that there is goodness here because large mailboxes lead to larger OSTs and the OST file format has never been the fastest. However, the downside is the need to recreate OSTs to compress data. Easy enough to do when you have ten or so PCs to upgrade; much harder and demanding of network bandwidth when hundreds of PCs start to run Outlook 2013.

Given the obvious problems that can be anticipated when the time comes to deploy the client, why have the Outlook developers decided to change the OST format and the way Outlook caches data?

I’ve done some thinking about this point and can only conclude that it comes about through an implicit recognition that the OST file format is not fit for purpose in an era when fully populated 25GB mailboxes might be the norm.

A period always elapses between allowing users to have a larger mailbox and when they fill the new quota. That period is pretty long when users move from 1GB mailboxes to 25GB. Even the most prolific (or anal retentive) of users will take time to fill a massive mailbox. The older OST format has therefore not come under real strain because it hasn’t been taxed by having to expand much past 10GB or so, a size that a modern laptop can cope with reasonably easily provided it’s equipped with an SSD.

But looking ahead, you can anticipate a time when 10GB is a common mailbox size and 50GB is not unknown. Some tuning is necessary to ensure that the OST internal structures are as efficient as possible. Without a complete redesign to use a new format, tweaking is all that can be done, and that’s what I think has happened in Outlook 2013’s compression of some data fields.

If you can’t redesign the OST file format, it then makes sense to restrict the amount of data that the OST is asked to store. This brings us to the new caching scheme. Users could be asked to archive information (or have Exchange archive items for them through retention policies), but even so it’s predictable that mailboxes will swell over time. Caching by age is a reasonable approach to the problem because it’s probable that users will want to work with their most recent data when offline. Outlook 2013 uses a default cache period of 12 months and a user (or an administrator) can dictate that a different period from 1 month to “all” should be used instead. Apart from anything else, caching only a part of a user’s mailbox makes it easier to move a user to a new PC because less data has to be downloaded to create the OST on that PC.

Another scenario when you might want to restrict the amount of data cached on a PC is when it uses a small SSD, as in the case of a ultrabook. SSDs are great and their speed makes many aspects of PC life much easier than otherwise, but space is at a premium and you might not want to cache the complete 10GB of your mailbox locally. In this scenario, it makes sense to throttle back the cache period to six months or a year so that the items that you reference most often will be held locally. In fact, if this scheme works well, you might then move to the next step by archiving all mail older than the cached period on the basis that you've proven to yourself that you don't need to access it often.

It’s obvious that goodness exists in how Outlook 2013 manages the OST. I understand and appreciate the logic that I think has been used by the Outlook developers as they made these changes. The issues that exist are the potential deployment blocker (the need to download new OSTs for every user who upgrades to Outlook 2013) and the relative lack of communication on the topic. But then again, isn’t discovering the detail of change and its implications one of the joys of working with technology?

Follow Tony @12Knocksinna

Hide comments


  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.