Dynameasure software by Bluecurve lets you test and troubleshoot performance in a client/ server environment. Although this simple statement of purpose implies that Dynameasure is a modest tool for limited applications, Dynameasure is highly flexible for a variety of applications such as server benchmarking, capacity planning, stress testing, and network analysis.
Dynameasure is different from most other performance-testing software because it uses actual client systems to simulate the client load. When you deploy Dynameasure, "motors" running in a client system generate the workload. You fire up the motors from a central control point, and they ride through your network like a band of Hell's Angels looking for trouble. Each physical client system can run from one to 20 motors, so you can, for example, use only 100 client systems to simulate the load of 2000 clients. The motors can coexist with other client-side applications, so you can test while end users are working--this capability can be pretty handy during capacity planning if you want to consider your existing client workload.
Bluecurve specifically geared the current version of Dynameasure toward testing Microsoft's SQL Server environment. The client motors connect to the SQL server via Open Database Connectivity (ODBC) links and use a mix of transactions to interact with a test database. Dynameasure then stores the transaction results in a separate results database. Bluecurve recommends one SQL server to sponsor the test database and a separate one to handle the test results. That way, the I/O load of updating the result database does not affect the I/O load of the test database. You have control over the size of the database and the transaction mix (e.g., the frequency of read, write, and mixed read/write operations). In fact, the amount of control you have over the test environment and transaction behavior is one of the most impressive aspects of Dynameasure.
Future versions of Dynameasure will support other databases such as Oracle and Sybase, file services, messaging engines such as Exchange, and Web servers. Bluecurve is clearly trying to position Dynameasure as the single tool you need to handle enterprise-level benchmarking and performance testing for all your client/ server applications.
How does Dynameasure measure up as an analytical tool? To answer this question, Windows NT Magazine brought Dynameasure into the Lab.
In this article, I'll look at the installation, configuration, and operational aspects of Dynameasure. Subsequent articles will show you the results of tests that use Dynameasure to assess the scaleability of SQL Server on a variety of server platforms.
The Dynameasure Operating Environment
To install Dynameasure, you need to start with at least one operational SQL
Server system (again, Bluecurve recommends two), one client system that you
designate as the test manager (the control point), and at least one client
system to run motors (if necessary, you can concurrently use the test manager
system as a client system and run motors on it). The more client systems the
merrier, but starting small and progressing upward is probably good.
The SQL server can be Intel-based or a Digital Alpha and can run SQL Server 6.0 or 6.5 under Windows NT Server 3.51 or 4.0. The client systems must be Intel-based and can run Windows 95, NT Workstation 3.51 or 4.0, or NT Server 3.51 or 4.0.
I initially tested one SQL Server system with one test manager system and one client system. The specific hardware configurations for these tests were as follows:
- For the SQL Server platform, I ran NT Server 4.0 and SQL Server 6.5 on a Total Peripherals 200MHz Pentium Pro with 132MB of memory.
- For the test manager platform, I ran NT Workstation 4.0 on a Dell OptiPlex GXMT 5133 133MHz Pentium with 32MB of memory
- For the client system, I ran both NT Workstation 4.0 and Windows 95 on a Toshiba Satellite Pro laptop configured with a 75MHz Pentium and 24MB of memory.
A 10 Mbits per second (Mbps) Ethernet LAN interconnected all these systems.
Install and configure your SQL Server system before you install Dynameasure. In particular, be sure to establish your network and user permissions so the test manager and client systems can access the SQL server. Also be sure to install ODBC support for SQL Server. If you are not familiar with SQL Server, I strongly recommend that you have access to someone with SQL Server experience during the installation process--a little humility here will probably save you a lot of grief as you progress through the installation steps.
Once your SQL Server system is in place and you are confident in your client/server network connections, you can install Dynameasure. The installation process involves four phases: manually creating empty Dynameasure databases on the SQL Server, setting up the test manager system, loading test data on the SQL Server, and setting up the client systems (the motors). After completing these four phases, you're ready to begin testing with Dynameasure. (Before you jump into the testing aspect, you can look at the sidebar, "Installing and Configuring Dynameasure ," to get a better idea about the installation steps, which I found to be somewhat quirky.)
Dynameasure in Action
Once you successfully navigate all the twists and turns of Dynameasure's
installation and configuration, the product's operation is clean and crisp. You
begin the test process at the test manager system by invoking the Manager
program, which then prompts you for the Dynameasure access password. After you
enter the correct password, the Manager program automatically launches the
Dispatcher program so you can define or start a test. Screen 1 shows the
Dynameasure Dispatcher running in the foreground with the Dynameasure Manager
running in the background.
When the Dispatcher starts, it establishes contact with the SQL Server system you are using in your test environment. The Dispatcher also checks to see what client-side operator programs are available--which leads to a very important point: To run a test, you must have the Operator program running in each client system you want to use. Each Operator program is responsible for all the motors running on its host system, so the Operator is a key player in the test process.
The interaction among the Dispatcher and the Operator programs is channeled through Dynameasure's control database. Each Operator program checks in when it is started and connected, and the Dispatcher sees check-in messages as they arrive. Available operators then appear in the Clients area of the Dispatcher display. For example, Screen 1 shows that the Dispatcher has detected one Operator, so one client icon appears in the Clients display area.
The Operator also provides useful information about the hosting client. For example, the Operator reports the processor type, operating system, memory size, and the hosting client's CPU utilization. You can view this information through the Dispatcher by selecting the client in the Clients display area and choosing the Machine detail option from the Resources menu. Screen 2 shows a sample Machine Information display. This information is valuable because it lets you judge what kind of load (i.e., how many motors) that specific client can reasonably generate.
Once you have the Dispatcher and the Operators running, your next step is to select the test to run. As Screen 3 shows, Dynameasure includes several OnLine Transaction Processing (OLTP) tests that can simulate a variety of read/write conditions. In fact, Bluecurve designed Dynameasure's OLTP test environment to simulate a mainstream order-entry business application.
To select the tests to run, click Add Test in the Dispatcher window. A list of test specifications will appear. Select the tests you want, and press Add to Queue to enter them into the Dispatcher. Once you place the test in the Dispatcher queue, you can edit the test's characteristics by double-clicking the test entry in the queue. Editing the test specifications is optional, but this option is where you can control the length of the test, the number of motors, the number of steps to perform, and even how to weight the various I/O operations. Screens 4 and 5 show an example of the test information you can edit. After you edit the test, you can save it back into the queue or save it to disk for subsequent use.
When you have placed all your tests in the Dispatcher queue, click Start Queue to begin the test. At that point, the Dispatcher determines what Operators are available and how many motors each Operator will handle. Based on those findings, the Dispatcher will send messages out to the Operator programs and have them start their motors. This phase of test preparation can take some time, depending on how many motors you need to start. From the client perspective, the motors appear on screen as they start. Screen 6 shows a typical client-side view of an Operator and its motors.
The Dispatcher display also changes to reflect the state of the operator and motor environment and the state of the test. For example, Screen 7 shows a test in progress that uses six motors running under one client operator. As the test progresses, the motor indicators change colors--blue for ready, yellow for wait, gray for not-quite-ready, green for in-progress, and red for trouble. The Progress tab on the Dispatcher also changes to reflect the overall status of the test in progress.
You can divide each test into steps. Each step can bring into play a specific number of motors. So, for example, you can define a six-stage test that starts with one motor and proceeds to five, 10, 15, 20, and then 25 motors. This stepwise approach lets you vary the load over time to reflect your user environment. Each step is then further broken down into phases.
A step begins with the Warm-Up phase, which includes a start-up limit to give the motors enough time to get started. The process proceeds through the remainder of the Warm-Up phase, in which motors complete their initialization and get ready for the test. After Warm Up comes the Measure phase, which is where the motors generate the transaction load. On completion of the Measure phase, the motors go into the Cool-Down phase, where they complete any pending operations and then proceed to the Report and Sync phases to report their findings to the control database. The total time you assign to the steps usually determines how much time to allocate to these phases, but you can edit the individual phases.
On completion of the test, you can use the Dynameasure Analyzer program to view the results, as shown in Screen 8. The analyzer lets you view the results of a specific test, or you can overlap test results within a test run to see how different trends look in relationship to one another. For example, you can look at how read-only performance compares to read-write performance. Don't be fooled by how simple the Analyzer looks at first glance--it is a powerful part of Dynameasure that lets you draw meaningful conclusions from your test runs. We'll spend more time on the topic of using the Analyzer in future issues of Windows NT Magazine. Note that you can greatly augment the information you get from Dynameasure tests if you run the NT Performance Monitor (Perfmon) on the SQL Server during the Dynameasure tests. When you use them together, Perfmon gives you an inside-NT view of what is happening to your server, while Dynameasure gives you the outside view of your client/server environment.
Does Dynameasure Measure Up?
Aside from the quirky installation and configuration problems I encountered,
I found Dynameasure to be a sophisticated instrument that you can customize to
address a broad range of tests. This sophistication is, of course, not without a
price: A "foundation" license for Dynameasure costs $29,995 and
supports up to 50 motors. Support for additional motors costs $2495 per set of
25 motors. So to create a test environment that emulates 300 users, you're
looking at a price tag of $54,935.
Is Dynameasure worth that kind of investment? That depends on how important accurate performance data is to you. To give you a better view of how Dynameasure can pay off for you, the Windows NT Magazine Lab will be using Dynameasure to conduct some key benchmarks for publication in the next several issues of the magazine. (For information about why we chose Dynameasure as our Lab's benchmarking tool, see Joel Sloss, "Birth of a Benchmark.") We will use Dynameasure to compare NT 3.51 and 4.0 performance, to assess SQL scaleability, and to measure the relative performance of a variety of server platforms currently on the market. We hope our Lab experience with Dynameasure will give you insights into the potential usefulness of the product in your environment. See you next month--and remember, keep those motors running.
