In "Is Your Exchange Server Relay-Secure?" (January 2000), I described how to configure your Exchange server to prevent relaying. The article described how to make the Exchange server return status codes that were in line with the Internet Engineering Task Force's (IETF's) Request for Comments (RFC) 2505, "Anti-Spam Recommendations for SMTP MTAs." If you use the default instructions for installing imsext.dll (which specify using the Internet Mail Service—IMS—relay page's custom routing hook), you might see some unexpected results. Depending on your configuration, your server won't be able to generate the 550 Relaying prohibited message or might respond with 550 Relaying prohibited to all inbound mail.
Microsoft originally intended the external routing program hook to let Exchange smarthost mail for other SMTP systems. When you use this hook to integrate the DLL, it overrides some, but not all, parts of the Exchange code that control routing and relaying. If you select the custom routing program check box on the Routing tab, the routing table and the routing restrictions button become disabled. Custom routing doesn't disable all the routing code, however. The IMS still evaluates any information you've specified in a prior configuration session. So, although the DLL replaces the routing table functionality, routing restrictions are active even though you can't access them for configuration.
If you use the custom routing hook to install the DLL, the system can't respond to a relay attempt with a 550 Relaying prohibited message. If you configure your server as I described in my January article and then use the custom routing hook to install the DLL, your system will reject every inbound message with the 550 message.
You can configure your server to prevent unauthorized relays and make use of the imsext.dll's functionality. If you modify the NonRoutingExtensionDLL Registry key to install the DLL as I describe in the main article, you can make all of these tools work together.
