Skip navigation
Exchange 2013's version differences and testing difficulties

Exchange 2013's version differences and testing difficulties

To be fair to the Exchange developers, they are taking great strides to remove version dependencies in the product. By this I mean that Microsoft is changing the product architecture so that you don’t run into instances where an Exchange component cannot communicate with an earlier version because of some change made in the most recent release. For example, in Exchange 2013 all data rendering is now done by the mailbox server rather than being split between the mailbox and CAS. In addition, communications between servers occurs at the protocol layer using a set of well-defined protocols such as EWS, SMTP, and MRS rather than allowing components lower down in the bowels of Exchange make direct connections to each other.

All of this is good and is intended to resolve situations such as when Exchange 2010 mailbox servers could not submit messages to an Exchange 2007 hub transport and vice versa. Or when calendar item processing occurred at several places in the product before the introduction of the Exchange Management Shell (EMS) provided a common business logic layer with which clients could interact.

Over the long term, you should be able to upgrade different Exchange servers without really caring what other servers are in place. For instance, when Exchange 16 debuts, you might be able to leave the Exchange 2013 CAS servers in place while upgrading the mailbox servers to the new software. At least, this is the plan as I understand it.

If everything works, then hopefully we will be liberated from many annoying gotchas that seem to come along to create problems for deployments. Or maybe not, because it is in the detail of implementation rather than the broad strokes of architecture that most problems lurk and await the unwary.

As an example, recently it has become clear that you need to deploy separate search mailboxes in organizations that are in the middle of transitioning from Exchange 2010 to Exchange 2013. One search mailbox is used to hold results extracted from Exchange 2010 mailboxes; the other serves the same purpose for Exchange 2013 mailboxes. This is understandable because of the radically different search architecture used by Exchange 2013, but it would have been so much better if it were possible to conduct a single search across both platforms that resulted in a common set of results. Software, after all, is supposed to do all sorts of magic of this nature.

Another example is in Unified Messaging where the arbitration mailbox (SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}) hosting custom grammar files should also be relocated to Exchange 2013 to make sure that it works properly.

And flaws exist in other areas as well that should really have been picked up during development, especially by those charged with testing. Take administrator auditing for instance. This feature tracks the actions taken by administrators according to the configuration established for the organization and stores the audit items in a system mailbox. But that mailbox must be located on an Exchange 2013 server as otherwise Exchange cannot write the audit items into it. Microsoft has not commented as to why this should be so, but I hazard a guess that it might just be due to attempting to use the latest version of XSO (Exchange System Objects, the internal Exchange API) to write the audit items into a mailbox on a server that doesn’t speak the latest XSO. The upshot is that you get a heap of event 5000 logged on your Exchange 2013 servers as they dutifully report their inability to record audit data. It’s hardly an earthshattering crisis but it is indicative of a certain lack of care and attention to detail during testing that this kind of cross-platform problem was not picked up.

Another recent bug in the same category is when you move a mailbox from one Exchange 2013 CU1 server to another (or between databases on the same Exchange 2013 CU1 server). The bug strikes to remove the extended rule information used to hold the junk mail settings that users can manipulate with Outlook or Outlook Web App, or administrators can set for them using the Set-MailboxJunkEmailConfiguration cmdlet or indeed read using the Get-MailboxJunkEmailConfiguration cmdlet.

Have a look at the two sets of information below. The first is extracted with Get-MailboxJunkEmailConfiguration before I moved my mailbox. You can see that I have some trusted senders and domains and a blocked domain. After the move, these properties are nullified.

Before:

RunspaceId               : ec820eed-7cf2-47af-a6e8-633f8694e6cd
Enabled                  : True
TrustedListsOnly         : False
ContactsTrusted          : False
TrustedSendersAndDomains : {hp.com, Dell.com, Fabrikam.com, Microsoft.com}
BlockedSendersAndDomains : {sexygirls.com}
MailboxOwnerId           : contoso.com/Exchange users/Tony Redmond

After:

RunspaceId               : ec820eed-7cf2-47af-a6e8-633f8694e6cd
Enabled                  : True
TrustedListsOnly         : False
ContactsTrusted          : False
TrustedSendersAndDomains : {}
BlockedSendersAndDomains : {}
MailboxOwnerId           : contoso.com/Exchange users/Tony Redmond

Ouch! I hear that this is a known bug that is fixed in Exchange 2013 CU2. That’s all very well and I am happy that Microsoft has fixed the problem, but I am genuinely surprised that such a bug made it through the massive automated test infrastructure that Microsoft has for Exchange – or even that a human tester didn’t pick it up. I do not think that this is an edge case because people do actually use their junk mail settings as a backstop to block spam that gets past server filters. I don’t think that that enabling administrator auditing in a multi-version Exchange organization is an edge case either.

Perhaps Exchange is now so big and so complex that the number of possible use cases that the product can be put through is so high that we should no longer expect testing to cover more than (say) 95% of use cases. Given the size of the installed base, that last 5% could create a lot of unhappiness and I think such a situation would be unacceptable to Microsoft, if only because of the huge support load (and cost) that it would incur. Testing regimes need to be tweaked and refined to reflect the real world and to make sure that the highest possible percentage of use cases are covered. It’s possible that I am unreasonable to use the instances cited here as examples, but I don’t think so. But I guess you’d expect me to say that…

Follow Tony @12Knocksinna

Hide comments

Comments

  • Allowed HTML tags: <em> <strong> <blockquote> <br> <p>

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Publish