When building an application, performance is an important consideration because the application must support a target number of users. Performance tuning for Web applications is more difficult than for traditional client/server applications because you can’t predict the number of users that your Web site will have to support (unless you use a login or filtering action to lock down your site). To facilitate Web-application-performance testing, Microsoft introduced a new tool called the Web Application Stress tool (formerly code-named Homer).
Microsoft designed the Web Application Stress tool to provide a reasonable simulation of activity against a Web application or static Web server. This tool lets you test any site that you can view with a browser. The tool’s GUI is easy to set up and use and lets you run tests from one workstation or as many workstations as you need to simulate the number of potential users of your Web site. You can also control the load each client places on the application. After you complete a test, the tool provides a report that details the test’s results, including loading page times and data from Performance Monitor counters that you selected to record.
Starting Out
The Microsoft Internet Information Server Resource Kit includes the Web Application Stress tool, and you can download the tool from http://homer.rte.microsoft.com. After you complete the download, run setup.exe to install the Web Application Stress tool. You need Administrator rights to perform the installation.
By default, setup.exe installs the tool in the C:\Program Files\Microsoft Web Application Stress Tool directory and adds an entry for the tool on the Programs menu. The setup program also installs an Access database, was.mdb, which the Web Application Stress tool uses to store test scripts. In addition, setup.exe installs and automatically starts the Web Tool service, which is the Web Application Stress tool service that runs the tests. Workstations must be running the Web Tool service to participate in tests.
Creating and Running Scripts
After you successfully install the Web Application Stress tool, you’re ready to create a script and run tests. You can create test scripts by manually selecting each script element, recording a browser session, importing an IIS log file, selecting files from a directory, or importing a Web Capacity Analysis Tool (WCAT) script. After you create a script, you can use it to run a test at any time.
To test the Web Application Stress tool, I used the tool’s sample files, which are located in the C:\Program Files\Microsoft Web Application Stress Tool\Samples directory. To run a test, I created a new Web application and imported the files from the Samples directory to the application. After you create a sample Web application, you can create and run a test script. However, don’t run the Web Application Stress tool on the same system as the Web application that you’re testing. In an optimal testing configuration, the Web server runs the application you’re testing and a set of workstations run the Web Application Stress tool. This setup lets the clients use their resources rather than the servers’ resources to perform the test. If you test a Web server that is running the Web Application Stress tool, then the test results will be inaccurate because the testing process consumes a significant portion of the server’s resources.
The tool uses multiple clients for testing, and you can select one of the clients to serve as the controller workstation. In addition to running the Web Application Stress tool and the Web Tool service, the controller workstation hosts the was.mdb Access database that contains the test scripts. On the other workstations in the test, you need to run only the tool and the Web Tool service.
To begin a test session, start the Web Application Stress tool on the controller workstation by selecting Start, Programs, Web Application Stress tool. When the Web Application Stress tool starts, the system will present you with the Create new script dialog box, which Screen 1 shows. (This dialog box doesn’t provide the option to import WCAT scripts. You can find this option in the tool's File menu.) For this example, let’s use the Record method to create a script, so click Record.
Next, you need to change several Microsoft Internet Explorer (IE) settings on the controller workstation. Select Properties from the File menu, and click Delete Files on the General tab to clear the browser cache. Click the Connections tab, and change the proxy server settings by clicking LAN Settings. To change the proxy server configuration, enter localhost in the Address text box and 8000 in the Port text box, as Screen 2 shows. After you complete your recording session, you will need to change these settings back to their default values.
Next, select the three check boxes in the Web Application Stress tool’s Browser Recorder Step 1 of 2 dialog box, which Screen 3 shows, then click Next. Click Finish in the Browser Recorder Step 2 of 2 dialog box, and the Web Application Stress tool will start your browser. To begin your recording session, enter the URL for the test site into the browser, navigate through the site, and perform the actions you want the tool to record.
When you’re done performing the actions you want the script to include, switch back to the Web Application Stress tool. The tool will present you with the steps that it recorded as you used the application through the browser, as Screen 4 shows. Click Stop Recording and your test script is complete. The script will have a default name of New Recorded Script. To change this name, double-click the script’s name in the explorer pane, then enter the new name. Create a naming convention for your test scripts so that you can easily identify them.
Before you run the script, change the settings that control the test by clicking the Settings folder in the left pane of the Scripts window, which Screen 5 shows. In the right pane, change the value in the Stress Level (threads) text box to 20. The stress level value is the total number of threads on all test workstations. The stress multiplier value is the number of sockets per thread. Setting the stress multiplier to 1 and the stress level to 20 results in 20 connections to the Web application. The Web Application Stress tool’s documentation suggests that you use between 1 and 100 threads per client workstation. The tool divides the total number of threads that you enter on the controller workstation across all the workstations in the test.
Next, enter the length of time that you want the test to run in the Test Run Time section. In addition, you can select a random delay that the tool will use between each request to the server by selecting the Use random delay dialog box and entering the minimum and maximum delay values.
If you want to add clients to the test, click the Clients folder in the left pane, and double-click the default entry in the right pane. Enter the name of each client workstation that you want to add, then click Add after you’ve entered all the workstation’s names. If the Web Application Stress tool can connect to the Web Tool service on that workstation, the right pane will list the workstation and provide a status of Connected in the right column.
To add Performance Monitor counters to your test script, click the Perf Counters folder in the left pane, and click Add in the right pane. Then enter your server in the Computer text box. You can simultaneously select multiple counters.
Next, you need to define in the Web Application Stress tool users for the Web site you are testing. The tool employs these users to define the browser sessions and authenticate users in the site. To define users, click the Users folder, then double-click Default in the right pane. The system will present you with a Users window that lets you create several users by entering a base username (e.g., User) and the number of users you want to create. When you click Create, the tool adds the users to the Web Application Stress tool database. The only way that I have successfully run tests is by using valid user accounts as Web Application Stress tool users. I usually create a set of users (e.g., User1 through User10) in the Users window, then add these users as local accounts on the Web server I’m testing.
Finally, you must make a minor change to the script’s settings. In the left pane, click the script you’re going to run (e.g., Sample Script). In the right pane, you’ll see the server name and files that the script will access. The default server name is localserver. You must change this name to the target server’s name for the test to work. After you make this change, the test should run successfully.
To run the test after you have completed your configuration, select the name of the script you want to run, then click the Run Script icon on the toolbar. Click Run Test in the resulting dialog box, and the tool will run the test.
The tool will display the Test Status dialog box throughout the test. This dialog box shows the status of the test and a countdown of the time remaining in the test. You can stop the test by clicking Cancel in the Test Status dialog box. When the tool completes a test, the Test Status dialog box closes automatically or displays the information about the test.
Viewing the Report
You can review a test’s results by clicking the Reports icon on the toolbar. After you select this icon, the tool creates a tab at the bottom of the Web Application Stress tool window that lets you easily toggle between the Reports window, which Screen 6 shows, and the Web Application Stress tool window. The Reports window’s left pane lists your test scripts and the results of each test. The window lists test results under the test script name by date and time. This setup lets you quickly find the results of a particular test. You can display a test’s results by expanding the date and time folder for the test.
The right pane of the Reports window provides an overview about the test process. This information is useful because it contains the date and time of the report as well as the version of the Web Application Stress tool. The overview also provides a summary of the test results, including the number of hits, the requests per second, and other vital statistics.
To view the result codes returned during the test, click the Result Codes folder in the left pane. If users couldn’t open pages, the Result Codes folder will show a 404 result code. The Web Application Stress tool’s Help file includes an HTTP Response code reference in the Reports section.
The Page Summary folder displays the individual results for each page. This folder is handy because you can see how many times users hit each page and the timing metrics for each page. The TTFB metric is the average number of milliseconds before a client received the first byte of a page. The TTLB metric is the average number of milliseconds before a client received the last byte of a page. To get specific data about each page, select the Page Data folder, then select a particular page. For details about the Performance Monitor counters that you defined for a test, select the Perf Counters folder.
During testing, the Web Application Stress tool generated a few error messages on the controller workstation that caused the entire test to fail. To restart testing, I had to shut down the Web Application Stress tool, open the Control Panel Services applet, and restart the Web Tool service. After I restarted the Web Tool service, I could restart the Web Application Stress tool and everything worked fine.
Testing Your Options
This example focused on creating scripts by recording a browser session, but you can use any of the methods I previously mentioned to create test scripts. The process of running the test and viewing the results is the same regardless of how you create the test scripts.
