Since Microsoft released Exchange 2013 SP1 on February 25, a number of similar client access errors have been reported in different forums, all of which have the following characteristics:
- User mailboxes are moved from Exchange 2007 or Exchange 2010 to Exchange 2013 SP1 databases.
- The old databases are removed from the organization.
- Users attempt to connect with Outlook Web App (OWA) or an Exchange ActiveSync (EAS) client and an HTTP 500 error is observed.
- The headers in the HTTP response noted in the OWA and EAS logs (in Exchange Server\v15\Logging\HttpProxy\Owa or \EAS) contain the error code: “X-CasErrorCode: DatabaseGuidNotFound.”
Examples of the problems found by customers can be viewed on TechNet and Reddit. Some entertaining and innovative workarounds such as recycling OWA/EAS app pools or deleting the EAS device partnerships from Active Directory have been tried with various degrees of success. However, as explained below, there’s an easy workaround until Microsoft comes up with a final fix.
Microsoft is still investigating but it seems pretty clear that this is a new bug that’s appeared in Exchange 2013 SP1. No software is flawless and it’s unsurprising that bugs surface after a new version is deployed inside customer environments. It can be argued (and with good reason) that this is the kind of bug that should have been discovered by Microsoft’s automated test suite before SP1 shipped, but the same argument can be advanced for any bug.
While they work out the finer details of the problem and consider what has to be done to fix it, Microsoft has just released KB2958434 to acknowledge the bug. The determined root cause is:
“This problem occurs because a user's mailbox database GUID (source) is contained within a client-side cookie and is added to the server cache on the Client Access server (CAS). The HTTP proxy on the CAS tries to locate the mailbox database by using the old database GUID within the cache. Because the old database GUID has been deleted, the attempt fails and returns a DatabaseGuidNotFound error.”
The problem happens when mailboxes are moved from older versions of Exchange to Exchange 2013 SP1, which is the only way that users can be transferred to the new version. The key thing is that the mailbox moves are followed soon afterward by the removal of the source databases. The clue is the DatabaseGuidNotFound error, which tells us that a connection was attempted to a database by reference to its GUID but failed. In this case, the clients want to connect to the old (source) mailbox and can't find it, so you get the failure.
Moving mailboxes between different versions of Exchange is a complex matter. There’s a wide ocean of difference between Exchange 2007 and Exchange 2013 when it comes to internal database structures. Microsoft has built a lot of code to smooth the migration and make sure that everything works properly after the Active Directory bits are finally swapped to point Exchange to the new database. In this instance it looks as if Exchange is attempting to contact the source database for some reason after the user attempts to log on to Exchange 2013.
For now, the solution is simple: don’t remove old databases from Exchange 2007 or Exchange 2010 after you move the mailboxes to Exchange 2013 SP1. Leave them alone and make sure that they remain available until every moved mailbox has successfully logged on to Exchange 2013 SP1. Once that happens, you should be able to remove the old database.
Microsoft is aware of the issue and the support team who featured (twice) in the “Expert Unplugged: Top Support Issues” sessions at the Microsoft Exchange Conference (MEC) in Austin last week is driving hard for a better resolution than simply keeping old databases around waiting for mailboxes to connect to their new home. Stay tuned for further information while this irritating and unexpected bug is stamped out.
Follow Tony @12Knocksinna