Are you looking for out-of-the-box Exchange Server functionality that can help you satisfy corporate-policy or regulatory-compliance requirements? In "Exchange 2003 Advanced Journaling," May 2005, InstantDoc ID 45644 and "An Exchange 2003 Journaling Primer," April 2005, InstantDoc ID 45348, I outline the capabilities and operation of Exchange journaling, which you can use to capture messages that users on an Exchange database send or receive. However, Exchange journaling doesn't distinguish between internal and external messages. If you want more granular control over your archiving operations, consider using another native Exchange tool: Exchange Server ArchiveSink.
You can use ArchiveSink to selectively capture and store messages that are routed to specific groups of individuals. Ostensibly, ArchiveSink was created as a diagnostic tool, intended for use in troubleshooting message flow. But Microsoft enhanced the tool in Exchange Server 2003, and that version makes a handy "poor man's" email-archiving tool. Exchange 2003 implements the tool as a transport event sink that you can install on an Exchange 2003 server running Windows 2000 Service Pack 3 (SP3) or later, then configure to capture all or a subset of messages that pass through a specific SMTP virtual server or servers on that system. Email that requires local delivery (i.e., email from one user to another on the same server) still passes through an SMTP virtual server, so this type of email can be archived as well. There are no restrictions on the number of SMTP virtual servers that can participate in the ArchiveSink operation.
Deployment Scenarios
The best place to use ArchiveSink is on Exchange servers that act as SMTP relay servers, either within your organization or at the interface to the Internet or to other companies. A common implementation scenario, which Figure 1 shows, is in a company that communicates with a few partner organizations and that must maintain an audit log of its email exchanges with those partners. The primary organization implements two Exchange SMTP relay servers: one that deals with general-purpose Internet communications and one dedicated to communications with the partner organizations. The primary company then installs ArchiveSink on the second server and enables ArchiveSink logging for all SMTP traffic that the server processes.
Installation
ArchiveSink is part of the Exchange 2003 release to Web (RTW) toolset; you can download the tool at http://www.microsoft.com/exchange/downloads/2003/default.mspx. The download consists of a self-extracting executable (archivesink.exe). When executed, this file creates a directory that contains a .dll file (archivesink.dll), an installation script (archivesink_setup.vbs), some brief documentation (archivesink.doc), and an End User License Agreement (EULA—eula.txt).
Copy everything but the EULA from the extraction folder to the Exchsrvr\bin directory on the Exchange server on which you plan to run ArchiveSink. (At the very least, put archivesink.dll and archivesink_setup.vbs in this directory.) To install ArchiveSink on and bind it to a virtual server, open a DOS command window and target the Exchsrvr\bin directory. Then, run the installation script, using the following syntax (typed on one line):
cscript archivesink_setup.vbs action vs_id \[location\]
where action is install, uninstall, or display; vs_id is the ID of the SMTP virtual server on which you want to install ArchiveSink; and location specifies the directory to which you want the tool to archive messages. The location parameter is optional for the uninstall and display actions but is required for the install action. If the location directory doesn't exist yet, ArchiveSink will create it when the tool first archives a message.
For example, to install ArchiveSink on my default SMTP virtual server, I would issue the command
cscript archivesink_setup.vbs install 1 c:\vs1archive
Note that in this example, I opted to archive messages to the C drive. Don't do this on production systems! Instead, use a dedicated disk or storage volume. ArchiveSink-captured messages can quickly amount to a huge volume of data, depending on how many messages pass through the SMTP virtual server. Before you put the tool into production, analyze your existing traffic patterns, then estimate and provide for adequate storage.
Note that the vs_id parameter in this example is the number 1. To determine the ID of a particular SMTP virtual server, open Exchange System Manager (ESM) and navigate down through your Exchange organization in the following order: the Administrative Groups folder, the folder for the administrative group that holds the SMTP virtual server, the Servers folder, the folder for the Exchange server that hosts the SMTP virtual server, the Protocols folder, then the SMTP folder. Right-click the desired virtual server object and select Properties from the context menu. Go to the Messages tab; the SMTP virtual server ID appears as part of the value in either the Badmail directory or Queue directory text box. The example that Figure 2 shows reveals that the vs_id for my default SMTP virtual server is the number 1 (as part of the value vsi 1). You can also determine the number by looking at the Logging settings on the General tab.
To determine whether the utility has been successfully installed and bound to a particular SMTP virtual server, you can run the installation script with the display action. For example, if I issue the command
cscript archivesink_setup.vbs display 1
I would get the data that Figure 3 shows.
After you install ArchiveSink, you must then enable message archiving by setting the value of the REG_DWORD Enable Message Logging entry (under the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Exchange\ArchiveSink\vs_id\ registry subkey) to 0x00000001 (i.e., enabled). Because the sink is tied to the SMTP protocol, you must restart the IIS Admin service before the registry change will take effect.
After you've enabled archiving, ArchiveSink will write archived messages to the directory that you specified during its installation. The installation process creates two subdirectories in the specified location. The first subdirectory, Smtp Messages, holds archived SMTP messages that pass through the SMTP virtual server. The second subdirectory, Mapi-Gateway Messages, holds messages submitted by Messaging API (MAPI) clients that connect to the Exchange server and picked up by the SMTP virtual server. For each archived message, ArchiveSink creates two files: an .eml file, which is the actual archived message, and an .xml log file that contains the message header information. Both files have the same filename—arch_random number—but different extensions.
Operation
ArchiveSink works by using two transport event sinks: the OnMessageSubmission event and the OnPostCategorize event. The first event, which the Enable PreCat entry (under the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Exchange\ArchiveSink\vs_id\ registry subkey) controls, is enabled (i.e., set to 0x00000001) by default when you install ArchiveSink. The second event, which the Enable PostCat entry (under the same registry subkey) controls, is optional. Generally, you need to enable only one of these two entries, but they aren't mutually exclusive.
By default, when SMTP or the Information Store (IS) submits a message that passes through the SMTP virtual server on which you've installed ArchiveSink, the OnMessageSubmission event is triggered. This event is triggered just once for each message submitted to the Exchange transport system. When this event is triggered, ArchiveSink captures the message to an .eml file and generates a corresponding .xml log file. Figure 4 shows an example of this file, which clearly specifies the message's originator and recipient details as well as a cross-reference to the corresponding .eml file. All recipients, including BCC recipients, are captured and exposed.
You can also enable the OnPostCategorize event by setting the Enable PostCat entry, which is disabled by default, to 0x00000001. When the Exchange categorizer, which determines final routing information, processes a message that's addressed to more than one recipient, the categorizer will bifurcate the message into multiple instances. For example, if I address one message to two recipients—[email protected] and [email protected]—the categorizer creates two instances of the message: one for each recipient. When you enable the OnPostCategorize event, ArchiveSink creates a set of archive files (i.e., an .eml and .xml file) for the message, then creates an additional set for each instance. Thus, my message to two separate and distinct recipients results in six archive files, as Figure 5 shows. The additional files have the filename format arch_random number_postcat.eml and arch_random number_postcat.xml. Note that the files have different names.
The OnPostCategorize event also displays the actual recipients within a distribution list (DL), assuming that the server on which you run ArchiveSink expands the DL. However, the default behavior (i.e., enabling only the OnMessageSubmission event) doesn't display the members of a DL because those members are determined only during categorizer processing. Thus, for more comprehensive and accurate logging, enabling OnPostCategorize event logging is essential.
Additional Configuration
ArchiveSink's functionality and the extent to which it captures messages is governed by its associated registry entries (under the HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Exchange\ArchiveSink\vs_id\ registry subkey), which Figure 6 shows. I've already discussed three of these entries (i.e., Enable Message Logging, Enable Precat, and Enable PostCat); let's take a look at what the other entries do.
By default, ArchiveSink archives report messages such as delivery and nondelivery notifications and read receipts but doesn't archive system messages such as public-folder replication messages. If you want to archive such messages, set the value of the Archive System Messages entry to 0x00000001.
Enabling the Dump P1 entry (i.e., setting the entry to 0x00000001) creates an additional file that contains archived messages' P1 headers. This file has a .p1 file extension (the rest of the filename follows the ArchiveSink naming convention). P1 headers aren't stored in human-readable format, but Microsoft might ask you to capture them for diagnostic purposes. Typically, however, you should leave this entry disabled (which it is by default).
When enabled, the Enable Mapi-Gateway Messages entry instructs the Exchange server running ArchiveSink to capture messages that are submitted from MAPI clients such as Outlook. This entry is enabled by default, but you can disable it (i.e., set the value to 0x00000000) if you want to capture only messages that the server processes in an SMTP relay capacity. Doing so can reduce disk space usage and processing overhead.
The Enable Smtp Messages entry is enabled by default and controls the capture of SMTP messages that the SMTP virtual server relays or receives. If you want to capture only those messages submitted by MAPI clients on a particular server, you can disable this entry (i.e., set the value to 0x00000000).
The Mapi-Gateway Messages and Smtp Messages entries define the locations in which ArchiveSink stores archived MAPI and SMTP messages, respectively. Anytime you make changes to any of the ArchiveSink registry entries, you must restart the IIS Admin service.
Something for Nothing
ArchiveSink provides no native indexing or search capability, and archived data volumes can quickly grow and become unmanageable, especially on heavily trafficked email servers. However, with a little ingenuity, you can implement scripts or utilities to create a searchable index of archived messages and periodically upload archived messages to Microsoft SQL Server, a data-management system, or an archive store.
ArchiveSink functionality merely proves that a message was sent from a mailbox and received by the SMTP virtual server. To meet the complex and comprehensive challenges of full regulatory compliance, you'll typically need to consider a more sophisticated third-party compliance solution. Still, you can achieve a degree of compliance without spending tens of thousands of dollars. Whether that's sufficient for your organization is something that only your company, its compliance department, and its lawyers can decide.
