Load Balancing Down on the Web Farm

Use WLBS to combat Web server delays in your IIS 4.0 clusters

Comedian Rodney Dangerfield's memorable punch line, "I get no respect," could easily be the Web server administrator's motto. Those of us who manage Web servers are expected to have Web pages readily available for download at any time under any network conditions. Although some customers might be patient while standing in line at the local fast-food restaurant or stacking up on the airport runway, they're not as patient when it comes to Web server delays. Queuing browser requests results in users going elsewhere—in other words, you lose business.

One way in which Web server administrators can combat Web server delays is by using clustering technology to provide fault-tolerant Web services. Microsoft entered the Web-clustering arena with Windows NT Load Balancing Service (WLBS), which the company acquired in 1998. With WLBS, you can have as many as 32 servers providing high-availability services such as IIS, FTP, VPN, and Microsoft Proxy Server. Let's explore the ins and outs of WLBS on IIS 4.0.

How WLBS Works
WLBS provides fault tolerance and clustering at the network layer rather than at the application layer. This distinction is important because the clustering software isn't aware of problems that might exist with IIS. Because WLBS operates at the network layer and below, it detects breaks in network connectivity and catastrophic server failures, including loss of power, hardware failures, and blue screens. The failover process in WLBS is reliable and swift. In most cases, packet loss is limited to just one packet. However, WLBS isn't designed to detect an application-layer failure, such as a failing Active Server Pages (ASP) application or a back-end database problem.

WLBS performs its clustering function through a TCP/IP-based message called a heartbeat. Heartbeats are multicast messages that a node (i.e., server) sends once a second to every server in the cluster. When a Web request arrives, the node sends it to every server in the cluster. The servers then use a dynamic algorithm to determine which server should handle the request. Should one of the servers in the cluster fail to respond, the other servers in the cluster take over and redistribute the load from the failed server.

WLBS works best with sites that contain only static Web pages. Web sites that keep stateful connections (e.g., Web sites that rely on or update back-end databases) enjoy only limited support in WLBS. WLBS uses affinity—a concept I cover later in this article—to support connections that require state, such as Secure Sockets Layer (SSL) connections. In addition, WLBS can support multiple virtual IP addresses, providing support for hosting multiple Web sites on one WLBS cluster. Hardware and ongoing support costs change dramatically when you add multiple Web sites on a cluster. If such additions are the case in your environment, examine other clustering technologies, such as hardware-based load balancing, which can be more cost-effective.

Speaking of costs, here's the good news and the bad news. The good news is that WLBS is freely available for download from the WLBS home page at http://www.microsoft.com/ntserver/nts/downloads/winfeatures/wlbs/default.asp. WLBS 2.2 was the latest version at the time of this writing. The bad news is that you can license WLBS only for use with NT Server, Enterprise Edition (NTS/E), which can make for an expensive clustering solution, especially if you're thinking about using more than just a few nodes. (Note that although the licensing agreement allows for the use only of NTS/E, you can install and run WLBS on the much less expensive NT Server 4.0, Standard Edition.) If you plan to run multiple sites and clusters, you might be able to save money by using a hardware-based cluster.

Before you install WLBS, you need to have two servers with comparable hardware. I strongly recommend that you also use identical hardware for each node in your cluster. Using dissimilar hardware and software increases support difficulties. When it comes time to troubleshoot, patch, or upgrade your servers, having similar equipment makes life much easier.

The WLBS documentation states that WLBS requires only one NIC. However, I strongly recommend that you use two NICs because then the network bindings are much easier to understand and configure. With two NICs, one card is dedicated to serving Web requests, and the other card is dedicated to passing heartbeat status messages about the health of that NIC's node in the cluster. Figure 1 shows a diagram of what a two-server WLBS cluster usually looks like. Note that the two public-facing interfaces share the same IP address—172.16.25.1. You can uniquely address servers through their dedicated IP addresses.

Installing WLBS on Your Servers
Now that you understand the basics, let's install and configure WLBS. Extract the WLBS installation files into a temporary directory. WLBS installs as a new network adapter called WLBS virtual NIC. To install WLBS, right-click Network Neighborhood, then select Properties. In the Network dialog box, click the Adapters tab, then click Add. From the Select Network Adapter menu, click Have Disk. In the Insert Disk dialog box, enter the temporary directory that you chose for the installation files. Click OK twice to begin installation. (You'll receive a warning about ensuring that the bindings are correctly mapped.)

In the Windows NT Load Balancing Setup dialog box, you configure a particular node in the cluster. You can configure almost every aspect of the cluster from this dialog box, including identifying which cluster you want to join, defining some of the host parameters, and setting the server load that this node will be handling. In the Cluster parameters portion of the dialog box, which Figure 2 shows, enter the primary address, the primary subnet mask, the full Internet name (i.e., www.your domain.com), the dedicated IP address, and the dedicated subnet mask.

Port rules help WLBS adjust the load algorithm for the node. You can create multiple rules for multiple ports on each node—for example, you can give weight and preference to particular servers that distribute multimedia content. For now, leave the port filters at the default port range of 1 to 65,535.

Earlier, I mentioned affinity. This feature lets WLBS consistently direct users to the same server, which allows stateful sessions such as those that ASP applications and SSL connections create. If your Web application has stateful requirements, you can force subsequent client requests to the same server by selecting the Single option in the Affinity section of the Windows NT Load Balancing Setup dialog box. Sometimes, proxied client requests can show up from multiple-source IP addresses. In this case, you can set affinity to Class C: WLBS will direct all subsequent client requests from the same Class C block of IP addresses (i.e., the first three octets of the IP address are the same) to the same server.

Here's something noteworthy about affinity and WLBS. The WLBS load-sharing algorithm always prefers to send traffic to a default server until that server reaches its load capacity, at which point subsequent servers start to accept client requests. This behavior assists in maintaining state with clients. In my WLBS server farm, my default Web server might generate 30MB to 50MB of logs daily, while the other server in the cluster might generate 6MB or 7MB daily.

When you're satisfied with the port and affinity settings, click Add to save your port rule. Add any other rules you need for your configuration. After you've added your rules, click OK to save the WLBS configuration.

The WLBS configuration is now complete. However, you still need to configure the bindings for the network adapters (which is the most difficult part of configuring WLBS) before you reboot the server so that the new WLBS drivers take effect. I'll show you the two-NIC setup. (For single-NIC setups and other configurations, consult the online Windows Help file.) Here are two golden rules to remember when configuring NICs:

Follow these steps to set the bindings. In the Network dialog box, click the Bindings tab, which Figure 3 shows. From the Show Bindings for drop-down list, select all adapters. Expand all the adapters to show the services bound to each adapter. Figure 3 offers some guidance about how to select the proper bindings. Adapter 1 (the Intel 82559) is the adapter configured with the dedicated IP address. Adapter 2 (the Intel PRO/100+) is configured with the shared virtual IP address. Disable the WLBS driver on the first adapter by selecting WLBS Driver on Adapter 1 and clicking Disable. Next, disable the WINS client on Adapter 2. Click OK, then click Apply to rewrite the bindings. Reboot the server.

Using WLBS
Now that you've successfully installed WLBS on the first server in your cluster, you won't see much evidence that WLBS has been installed unless you start poking around in the Control Panel Network applet. Running

ipconfig /all

from the command line also reveals the multiple IP addresses.

You can perform most routine WLBS administration through the command line. To see the status of a WLBS cluster, run the command

wlbs query

from the command line, as Figure 4 shows. Run the command

wlbs stop

to remove servers from a cluster individually. Similarly, run the command

wlbs start

to reinsert a server into a cluster. These last two commands are helpful when you're performing upgrades. Using these commands, you can withdraw a server from the cluster, upgrade it, then reinsert it into the cluster; all the while, the remaining servers handle the load. (This process is also known as performing a rolling upgrade.) If you need to change affinity settings, you can use the command

wlbs setup

to change settings. The online Help file, which you can access from the Start menu or from the Windows NT Load Balancing Setup dialog box, documents many other command-line settings.

WLBS dynamically adjusts to changing network conditions with a process called convergence. The addition or loss of a server initiates this process. When the cluster fails to receive a heartbeat from an existing cluster server or hears a new heartbeat from a newly inserted server, convergence begins. By sending and receiving heartbeats, the convergence process learns what other servers are available for forming a new cluster to serve clients. Only a few seconds are necessary for the entire convergence process, and clients will continue to be served while convergence is in progress.

WLBS Limitations
I run my WLBS IIS 4.0 cluster without most of the bells and whistles of an NT server infrastructure. Protected by a strict demilitarized zone (DMZ) architecture with restrictive content management, this Web site hasn't suffered a server-related outage since I brought it up more than 2 years ago. Rather than using WINS, I use DNS for all my name resolution. Because I've been diligent in applying patches, recent security problems such as CodeRed and Nimda didn't affect these servers.

If you use the port-filtering features, remember that the filters aren't a substitute for a high-quality stateful inspection firewall. The port filtering that WLBS includes can't filter Internet Control Message Protocol (ICMP) and other non-TCP or UDP protocols. It also bears repeating that WLBS won't monitor IIS, so you need to monitor IIS in some other way to ensure that the service remains up. You might also want to think about how you'll perform content management and keep the Web content synchronized on each server in the cluster. See the sidebar "A Special Note for Switched Environments," page 3, for information about a known problem in using WLBS-based Web servers with switches.

Beyond IIS
You can easily use WLBS to load-balance other Windows services besides IIS. For example, you could build a cluster that offers highly available VPN services, redundant SMTP services, Proxy Server, and other TCP/IP services.

WLBS is an easy way to increase uptime and reliability. The service requires a bit of discipline with change management and content management, and you must keep up with patches and security administration. With that combination, you can achieve a lot of 9s in your uptime numbers.

Next month, I'll dive into WLBS's successor, Network Load Balancing Service (NLBS). I'll also introduce you to a few non-Microsoft load-balancing products that perform functions similar to NLBS.