| Downloads |
|---|
| 25963.zip |
In "Load Testing Exchange 2000," May 2002, InstantDoc ID 24126, I introduce several load-simulation tools for Microsoft Exchange 2000 Server. I explain how load testing Exchange can be a quick and easy way to stress-test new hardware or a network design or to illustrate a proof of concept such as a distributed Web architecture with front-end servers for client connection points and back-end servers for mailbox stores. In that article, I describe how to set up the Microsoft Exchange Load Simulator (LoadSim), which simulates Messaging API (MAPI) clients such as Microsoft Outlook, and how to use Windows 2000's Performance Monitor to capture the results. But as I explain in "Load Testing Exchange 2000," you need to rely on other tools—in particular, the Exchange Stress and Performance (ESP) tool—to test non-MAPI clients and protocols such as POP3, IMAP, SMTP, and HTTP-DAV (which Outlook Web Access—OWA—uses). In fact, you must use ESP rather than LoadSim to test a front-end/back-end architecture, which MAPI doesn't support. After you learn how to analyze your captured LoadSim results, you're ready to take the next step: learning how to use ESP to simulate Internet-protocol clients.
Processing LoadSim Output Files
LoadSim uses several load-generating computers to simulate the effect of hundreds or thousands of Outlook clients. After you run a load simulation against your Exchange mailbox servers, look in the folder in which you installed LoadSim. You should see a file called loadsim.out. (I find that the easiest way to locate the file is to sort the folder contents by date, then look for the newest file. If you can't find the loadsim.out file, run a search for it; I've seen the file's location change depending on whether I run LoadSim manually or through an automated option.) When you select the Archive previous file check box during LoadSim configuration (as I detail in "Load Testing Exchange 2000"), the tool will rename loadsim.out to loadsim.00x (where x is an increasing integer beginning with 1) and generate a new loadsim.out file each time you run a load test. For example, the first time you run a test, LoadSim creates loadsim.out. At the second run, the tool renames that file to loadsim.001 and creates a new loadsim.out file. At the third run, the tool renames loadsim.out to loadsim.002 and creates a new loadsim.out file. (Note that you must clean up these files manually.) Open loadsim.out in a text editor such as Notepad, and you see lines of logged information, similar to the following:
Jan 31 10:34:47: Action 1 Jan 31 10:34:49: Action 2
Notice that LoadSim logs a timestamp for each action. You can use the Lslog utility (lslog.exe), which comes with LoadSim, to process these files and present the results in a table of response times.
To run Lslog, open a command prompt in the LoadSim directory. You can process each client separately to determine the scores and watch for variations (which typically indicate a client-resource bottleneck, such as a lack of memory), or you can merge all your clients' results to get an average score for all clients. The following command merges the results:
c:\loadsim>lslog merge /r cl*.log > all-cl.log
I first copied each client's loadsim.out file to a central network share, renaming the files clxx.log (where xx is a numeric identifier), then ran the merge command to merge the client logs into one file. (You might not need to copy the output files to merge results to a central location—for example, when you're simulating 500 clients from only one machine.) This example uses the /r option to rebase all the clients to the same time (i.e., Jan 01 00:00)—a useful option when your simulation clients' clocks aren't synchronized.
The next step is to examine the merged file (i.e., all-cl.log) to determine the length of time that the test run covered. During the first few hours of a test run, clients are logging on and Exchange is allocating memory; performance isn't optimized until the steady-state period. The final few hours of a test run involve shutting down and cleaning up pending requests. You want to analyze only the steady-state period, which occurs in the middle of the test run. To view the test run's length, enter the following command:
c:\loadsim>lslog times all-cl.log
Next, truncate the file to eliminate the ramp-up and ramp-down times, isolating the steady-state interval:
c:\loadsim>lslog trunc all-cl.log 2 6 > all-cl.out
In this example, I use hours 2 to 6, for a total of 4 hours, which is the average length of the steady-state interval. (If you run only one simulation client, this step will be your starting point.)
The next step, which analyzes the response times and generates the output table, is the most important. The first command processes the merged and truncated LoadSim output file, then pipes the output to a text file. The second command opens that text file in Notepad.
c:\loadsim>lslog answer all-cl.out > all-cl.txt c:\loadsim>notepad all-cl.txt
Analyzing LoadSim Results
When you process the LoadSim results in this manner, the output appears in tabular format, as Figure 1 shows. (All values in the table are in milliseconds.) This table lists the response times for all the actions that occurred during the test simulation. The column of interest is the 95th Pctile column, which shows the 95th-percentile response. The vast majority of MAPI (e.g., Outlook) clients will experience at least this response time from the tested Exchange servers. In the example that Figure 1 shows, the weighted average of all commands' 95th-percentile response time is 157ms, which is quite fast. Microsoft guidelines suggest that the weighted average of LoadSim-measured 95th-percentile responses measure at less than 1 second (expressed as 1000ms). This figure can give you a feel for response-time latency in your environment (e.g., how long it takes to move between your Inbox and your Public Folders). In most environments, this process is slow—often more than several seconds, which can seem like an intolerable delay.
Slight variations occur among LoadSim runs, producing slightly different response times even when you don't change any tuning parameters on the Exchange server. Such variations are normal and happen because of the slight randomization of messages in LoadSim. When I test the difference that a tuning parameter or third-party add-in (e.g., an antivirus product) makes, I always run a minimum of two LoadSim tests. If the results aren't similar, I run a third test to determine which of the first two runs was most accurate. Even then, though, your results might not be truly comparable with each other. For example, your Exchange server might have been queuing messages for delivery during one of the runs. Therefore, you need to use the data you captured with Performance Monitor to confirm your analyses.
Analyzing Performance Monitor Data
In "Load Testing Exchange 2000," I suggest a list of performance objects, counters, and instances to monitor (Table 1 shows this list) and explain how to capture the relevant data. You then can analyze the data at the comfort of your workstation, connecting remotely to the capture files or copying them to the local system.
On a Win2K system, open the Microsoft Management Console (MMC) Performance snap-in to launch Performance Monitor and select the Console Root\System Monitor object in the left-hand pane. Right-click anywhere in the right-hand pane and select Properties to open the System Monitor Properties dialog box. Go to the Source tab, select the Log file option, then click Browse to search for your capture files, which you can open as binary log (*.blg), comma-separated value (*.csv), or tab-separated value (*.tsv) files. (Binary log files can handle logging stops and restarts without beginning a new file, but I prefer .csv files, which are smaller and which you usually can open in Microsoft Excel for further analysis and graphing.) Click Open and select the desired capture file. Click Time Range on the Source tab and define a range that matches the same steady-state period you used in your LoadSim analyses.
Next, go to the System Monitor Properties' Data tab, then click Add to select the counters that you want to analyze. (The only counters available are those you chose to capture during your load test.) After you finish adding counters, click Close, then click OK. Be forewarned that each of these steps can take quite a few seconds when you're working with large files.
You can view the data in several ways. Press Ctrl+G to switch to Graph view, which shows more information per counter; press Ctrl+R to switch to Report view, which shows more counters and averages for the specified time range. Several handy tricks can also give you more viewing options. To save your workspace, right-click anywhere in the right-hand pane, then select Save As. Doing so lets you create an HTML file so that you can view your defined counters and settings as a Web page. To switch from .csv to .blg format or vice versa, right-click anywhere in the right-hand pane and select Save Data As. You then can convert the file from the current format to the other; you can also resample the log file to create a smaller file that uses only a subset of counters (i.e., those counters that you selected on the Data tab) and the steady-state time range. Simply enter a value for X in the Reduce log file size by only writing one out of every X original log records text box.
Before you close Performance Monitor, be sure to save your console settings so that you open the same System Monitor view of the counters and log files. To do so, select File, Save from the menu bar.
Testing Internet-Protocol Clients
To test front-end/back-end configurations or to test clients that use Internet protocols such as POP3, IMAP, SMTP, or HTTP-DAV, use the ESP utility, which you can find in the Microsoft Exchange 2000 Server Resource Kit. To install the tool, run msetup.exe (in \exreskit\tools\esp). The tool's Setup process creates the _ESPUser service account and prompts you to enter a password. ESP uses this account to run multiple ESP load-generating clients (which the tool refers to as hosts) that connect to a central control host—a process that involves changing Distributed COM (DCOM) permissions. However, you need only one ESP host to simulate thousands of Internet-protocol clients. If you plan to run multiple ESP hosts, see the tool's readme.txt file for information about known concerns involving running multiple hosts.
To configure a simulation, select Programs, ESP from the Start Menu. To add an ESP host, select File, Add Host from the menu bar. Enter the host information: typically, the local machine on which you're running ESP (unless you've set up multiple hosts to simulate thousands or millions of simulated clients). Click Add Host, then click Connect Host to open a dialog box. Go to the Available Modules tab, which Figure 2, page 74, shows. Select a protocol from the Installed Modules list, then click Add. If necessary, you can configure general settings (e.g., enable logging) on the Options tab. Click OK to close the dialog box.
The next step is to configure the selected protocol modules with the desired load-testing scripts. Instead of using predefined load profiles as LoadSim does, ESP uses scripted commands. The trick is to modify existing scripts rather than create scripts from scratch. ESP scripts include four primary components: a list of commands, a list of users (you can export the list from Active Directory—AD—or use a random list), a list of messages and attachments for ESP to send, and the contents of those messages.
In the main ESP interface, right-click a protocol module—the examples in this article use DAV to simulate OWA clients—then select Properties to open a dialog box such as the one that Figure 3, page 74, shows. Find Server in the Parameter column, then click the corresponding entry in the Value column and type in the name of the front-end Exchange server that you'll connect to. To test OWA, make sure that the corresponding value for the Authentication Method parameter is BASIC and enable that authentication on your front-end Exchange server. Find the Script File (or Script Path for all modules except DAV) parameter, then click the corresponding value and enter the name of the script that the module will run. To search for the script, double-click the corresponding Value field to display an ellipsis (...) button, then click the button. Because of a minor bug, ESP doesn't show you the script files as you browse. To see a list of files, locate the proper directory, then enter an asterisk (*) and press Enter. You then can see and choose from the files. If you haven't created your scripts yet, simply enter a filename, then save the scripts under that name later.
Repeat these steps to configure each protocol module that you want to use. You don't need to test all the available modules, although if you test IMAP4 and POP3 clients, you should also use the SMTP module because those clients typically use SMTP to send email.
Listing 1 provides a sample script for testing SMTP clients. (I modified this script from samples provided in the ESP documentation. Listing 2 provides another modified sample for testing IMAP4 clients. I suggest that you read the ESP documentation to determine which commands to use to customize the scripts for your environment.) The sample script that Listing 1 shows connects to the Exchange server, then uses the HELO command to identify itself as the user Admin. The script sets the Reply address to [email protected], then uses ESP's looping capability to generate a random number of messages between 100 and 1000.
The script then uses a random number (between 1 and 2000) of users you've created. Alternatively, you can use the RANDLIST(userlist.txt) command, where userlist.txt contains the list of accounts you created for your LoadSim tests, to configure the script to pull names out of an exported AD list. (Whereas LoadSim handles the creation of user accounts in AD, ESP doesn't. You can either populate AD manually or through a third-party utility, or you can use the accounts that you created during your LoadSim tests. This approach might be the easiest; you simply export a list of the accounts to a .csv file, then paste the relevant account information into the userlist.txt file for use with ESP. To further simplify the process, first create an organizational unit—OU—in AD, then move the ESP test accounts to that OU.)
Next, the script directs ESP to pull a random message from the msglist.txt file, which Listing 3, page 75, shows. This file contains a list of MIME messages of varying sizes. (To use the sample script, you need to create all these files.) I use Outlook Express to save a message as a .eml file, then rename the file with a .txt extension. (You could also turn on SMTP archiving and capture .eml files.) Then, delete or add blocks of text to adjust the text file to the desired size. You can also define attachments to send with your test messages. ESP scripts can randomly select messages from msglist.txt.
Finally, the script uses a SLEEP command, followed by a random interval (in milliseconds), to help control the duration of the script's cycles. ESP loops the script, repeating the commands as often as it can and waiting only for the specified sleep cycles, which are the key to simulating real-world client loads. If you don't build in enough sleep latencies, client processor utilization will climb to 100 percent and you'll have trouble controlling the client machine to stop the script. To determine how often to insert a SLEEP command, think about how your users access your Exchange servers. IMAP4 or POP3 clients might poll the Exchange server for new messages every half hour. Users who are actively working at the computer might read their email messages, respond to a few messages, and create a few new messages. You can use these activity intervals to determine the sleep intervals to use in the ESP scripts. OWA clients might remain connected to the Exchange server all day, so your ESP script should connect once and periodically check for new messages during the day. (If you use OWA 2000, which comes with Exchange 2000 Service Pack 2—SP2, you'll need to update your ESP scripts to simulate the functionality of OWA 2000's updated features.)
To run the load simulation, select a host and click Start Module(s) or select a module and click Start Modules(s)—or right-click the module, then select Start. The latter methods give you complete control over how many modules you want to run at the same time. Be aware that once you begin a test, you can't click Stop Module to stop a module until the module icon turns green (indicating that the module is running). If you try to use this button before the module is running, you might need to close and restart the entire ESP client. Unlike LoadSim, ESP doesn't generate any client response measurement, so the only performance statistics you can gather are those you capture through Performance Monitor. (See "Load Testing Exchange 2000" for instructions about how to capture that data.)
Get the Big Picture
Analyzing your LoadSim results provides a good picture of your Exchange servers' performance in relation to MAPI clients such as Outlook. For an analysis of any Internet-protocol clients, the script-driven ESP utility provides similar functionality, although these protocols' lack of a standard profile such as MAPI Messaging Benchmark (MMB) 2 means that you must rely solely on Performance Monitor data captures to determine whether the load that ESP generates accurately represents your desired test load. With LoadSim (and its built-in analysis utilities), ESP, and Performance Monitor, you'll soon have a solid idea of the strengths and weaknesses of your Exchange servers' performance.
