\[Editor's note: Email your Exchange and Outlook solutions (under 400 words) to R2R at [email protected] Please include your phone number. We will edit submissions for style, grammar, and length. If we print your contribution, you'll get $100.\]
Recently, a customer received the following Dr. Watson error message on his company's Exchange Server 5.0 Service Pack 2 (SP2) server:
store.exe – Application Error
The instruction at "0x0052ce22" reference memory at "0x00ff9000".
The memory could not be "read".
This error had occurred once before, a few days earlier, right after we had made extensive modifications to a public folder and to the method for replicating it to the organization's other Exchange sites. In an attempt to correct the error, we restored the Information Store from the previous night's full backup.
When the second error occured, we did some research and came across Microsoft Knowledge Base article Q169223 (http://support.microsoft.com/support/kb/articles/169/2/23.asp), which described almost exactly the problem we were experiencing:
Event ID: 4116
Category: Internal Processing
Description: An error was returned from the messaging software the Internet Mail Service uses to process messages on the Exchange Server. ... The message that was being processed has been moved to the "BAD" folder. Use the appropriate utilities found in the SUPPORT directory of your Exchange CD-ROM to view and manipulate these messages.
Event ID: 4023
Description: Unable to start the Internet Mail Service because the work threads could not be initialized.
We believed that the problem was with a bad message in the Internet Mail Service (IMS) IMCDATA/IN queue. However, after analyzing the Application Log further, we discovered that the Private Information Store initialized successfully, but when the Public Information Store tried to initialize, the Dr. Watson error occurred. However, because we hadn't loaded the Exchange Debug symbols properly, we were not generating the complete error in the error log. After we reloaded the symbols and received the complete error message, we could determine that the problem was with the store.exe. I received updated store.exe, mdbmsg.dll, and emsmdb32.dll files from Microsoft Technical Support and placed files in the following locations:
store.exe version 5.0.1461.30c:\exchsrvr\bin
These replacement files weren't part of the post-SP2 STORE hotfix. After we replaced the files and rebooted Exchange Server, the Exchange services started successfully and mail began moving.
When we looked further into the problem, we learned that recent changes to Public Folder replication had caused the Information Store to pass a number that exceeds the storage limit for a variable to the memcpy function, causing the Information Store to experience an exception and shut itself down. The stack dump appears as though the Information Store passed a negative number to memcpy. Knowledge Base article Q182107 (http://support.microsoft.com/support/kb/articles/q182/1/07.asp) explained the problem and noted that the next service pack would contain a fix.