Maybe it's because I live in Colorado, but when I hear vendors talk about their Windows NT clustering and high-availability products, I think of the medicine shows that once roamed the West. Remember the ones from the classic Westerns? The shows traveled from town to town, selling snake oil and promising it would cure everyone's ills. But what really happens when you sip the magic elixir--or attempt to implement a high-availability solution in your environment?
Clustering isn't a cure-all for your NT availability woes. No amount of faith or sweat will make an incorrect solution solve a problem it wasn't designed to solve. Don't misunderstand me--I've implemented both hardware and software solutions to increase NT's availability, and those solutions work. However, the key to succeeding in your quest for continuous NT computing lies in understanding your needs and each solution's capabilities, and in prototyping to make sure a solution will scale to solve your business problem.
Software and hardware solutions that increase NT's availability are part of, not replacements for, good systems-management practices. These solutions increase system availability and reduce your business' exposure to computer downtime by providing redundancy to your computing environment in the same way RAID or multiple power supplies in a system provide redundancy. NT clusters and related solu-tions only let you recover from or mask failures that cause system outages--you must continue to execute proper backup and disaster-recovery strategies.
In this article, I'll give you my refined definitions of clustering terminology and review availability classifications that help categorize vendor solutions. Then, I'll identify features that can help you select and implement a data mirroring and failover solution that meets your business requirements. Along the way, I'll point out some differences between data mirroring and failover solutions and Microsoft Cluster Server (MSCS).
Defining Clustering
| No amount of faith or sweat will make an incorrect solution solve a problem it wasn't designed to solve. |
My definition isn't much different from the definitions I've heard in my discussions with Microsoft staff. Mark Wood, Windows NT Server, Enterprise Edition's first product manager, defines a cluster as a group of independent systems linked together for higher availability, easier manageability, and greater scalability. Jim Gray, senior researcher in Microsoft's Scaleable Servers Research Group and manager of Microsoft's Bay Area Research Center, describes a cluster as "a collection of independent computers that is as easy to use as a single computer." He further describes clusters as solutions that not only provide failover capabilities but also disperse data and computation among a cluster's members. (To learn more about Jim Gray's clustering vision, see his sidebar "Commodity Cluster Requirements," June 1998.)
My definition of clustering narrows the focus to exclude from the clustering category data-mirroring-with-failover solutions, which do not provide access to common storage resources or support the automatic reentry of a recovered system into a cluster. Access to common storage and the seamless addition and removal of systems in a cluster is key to the distinction I make between NT clustering solutions such as MSCS--which provide seamless interaction between systems--and data mirroring with high availability products--which do not provide seamless interaction. This distinction might appear minor today, but it will become increasingly important as Microsoft expands MSCS beyond its current two-node support.
While I'm defining terms, let's look at the terms active/standby and active/active. These terms apply at the system level to refer to the nodes in a cluster that actively perform work or wait in standby mode to assume the load of a failed cluster node. Active/standby means one node is working and the other is waiting. In active/active implementations, both nodes actively perform independent work.
Active/standby and active/active apply to the system level, but they can also apply to applications. For example, both nodes in an MSCS solution can actively run and offer services; this capability makes MSCS an active/active system-level implementation. At the application level, MSCS supports both active/active and active/standby solutions. For example, SQL Server 6.5 Enterprise Edition and Internet Information Server (IIS) support active/active configurations on MSCS, whereas Exchange 5.5, Enterprise Edition runs only in an active/standby configuration.
A problem arises when you apply any definition of clustering to current NT availability solutions: Most of these solutions address only the increased-availability portion of the clustering triad, the other two elements of which are manageability and scalability. Mark Smith pointed out this shortcoming in "Clusters for Everyone," June 1997, and it's still true a year later. Few increased-availability products for NT offer continued (automated failover and back) access to resources without operator intervention or, worse, system restart. Even fewer products have addressed the manageability and scalability legs of the triad that mini and mainframe clusters have targeted for over a decade. In fact, the real advances in multinode scalability have been limited to database and Internet-related solutions. Does that shortcoming mean increased-availability solutions are bad? Certainly not. However, this situation means you'll probably have to take advantage of each product's strength, work around its limitations, and use a combination of products to meet your NT availability needs.
Availability Classifications
The keys to selecting and implementing the right high-availability solution
are identifying applications that need increased availability, defining the
outage duration and type your business can tolerate, and determining how much
your business is willing to spend for the redundancy necessary to meet your
expectations. Vendors such as Digital Equipment, HP, and NCR place the
single-host availability of NT Server running on Pentium Pro systems at the 99
percent uptime level. For systems that must operate 24 hours a day, 7 days a
week year-round, 99 percent availability translates to about 87 hours of planned
and unplanned downtime per year. Adding RAID data protection to such a system
lets it survive some level of disk failure and raises availability to 99.5
percent, or 44 hours of downtime per year.
Fifty-two planned outages lasting 50 minutes each (44 hours distributed over 52 weeks) is manageable for most sites. For other sites, though, even a few minutes of planned downtime, let alone the threat of outages lasting for days, justify moving beyond the usual commercial availability of a single NT system. These sites are where high-availability (data mirroring with failover) and fault-resilient clustering (data-sharing solutions such as MSCS) products that take NT systems to 99.9 percent (8.8 hours of downtime per year) and 99.99 percent (53 minutes of downtime per year) availability come into play.
High-availability and clustering solutions provide system redundancy and support some level of application restart or resource failover among member systems. These features increase system availability by facilitating the transfer of resource responsibilities to surviving systems. Although the resources remain highly available, the transfer, or failover, takes time, from seconds (for a few file shares) to minutes (5 minutes to 10 minutes for the restart of an application such as Exchange). Some client/server applications, by fluke or design, can survive these momentary transitions. Other applications cannot tolerate any identifiable transfer time. For a more detailed view of system availability, see Chapter 3 of Transaction Processing: Concepts and Techniques, by Jim Gray and Andreas Reuter (Morgan Kaufman Publishers, 1992).
Hardware fault-tolerant products take NT to the 99.99 percent availability level in a different way from fault-resilient clusters. Hardware fault-tolerant solutions (such as Marathon Technologies' Endurance 4000) involve total system redundancy in which all components perform actively during normal operation. This configuration allows continuous processing or compute-through capability for hardware-related failures. Unlike high-availability and fault-resilient cluster solutions, this configuration requires no application restart. Thus, no loss of application state or client connectivity to the hardware fault-tolerant system occurs.
As you move up the scale from 99.9 percent to 99.99 percent availability, successive solutions result in more than incremental increases in cost. This fact is why it's imperative that you identify in dollars what availability is worth to your enterprise.
Data Mirroring with Failover
At the low end of the price and complexity spectrum ($1895 to $3999 for two
nodes), realtime data mirroring between servers lets you increase NT's
availability and provides features that let each server assume the other's
identity in case of failure. Data mirroring with failover between NT systems
isn't new. In fact, it's a high-availability solution I ran across in 1995 and
first used at one of those buried-in-a-mountain government installations. The
air force colonel responsible for the system was concerned about having failover
capability between the system's two new NT servers. He was used to the
fault-resilient capability of the VMS-based clusters he had been relying on, and
he wasn't going to give it up completely when he moved to NT.
What's new in data mirroring is an increase in the number of realtime data-mirroring products. The abundance of solutions has increased competition and driven improved product functionality. From a failover standpoint, product functionality improvement has meant moving away from older active/standby system-level implementations that, in some cases, require a reboot for the standby system to assume the identity of the failed system. Today's solutions can retain their identity while assuming the identities (including NetBIOS names and TCP/IP addresses) of multiple failed servers.
Table 1, page 132, lists some prominent current solutions that combine data mirroring with failover capability. (Many data-mirroring products do not include failover capabilities, and others, such as NT 5.0's IntelliMirror, do not offer realtime capabilities.) Although the advanced features of the solutions in Table 1 are targeted toward NT 4.0, NSI Software's Double-Take for Windows NT and the Qualix Group's OctopusHA+ support NT 3.51 but have reduced failover capabilities with NT 3.51.
The solutions in Table 1 meet the independent OS and application-execution criteria of my definition of clustering. Where they fall short--and why I don't classify them as true clusters--is in their inability to access shared storage. (To learn more about these solutions' features and functionality from a standard clustering perspective, see Jonathan Cragle, "Clustering Software for Your Network," July 1998.) Each node accesses data only on its locally attached storage. This limitation doesn't render these solutions useless in your search for higher NT availability; in fact, the beauty of these solutions is that their hardware and software requirements are not (unlike those of MSCS and other clustering solutions) stringent. For the most part, with these solutions you can use the systems and (with some elbow grease) applications you already have, to build a system-level high-availability solution between two or more than two systems that can run NT. At most, because you are duplicating your data, you must add disk space. Depending on the traffic your systems support and the data mirroring and failover product you choose, you might also need to add network cards to create a private network between the systems you target for data mirroring and failover. Figure 1 illustrates a typical two-node data-mirroring configuration with internal storage.
An advantage of network-based data replication is that it opens the door for wide-area data replication. Wide-area data replication can give you offsite copies of your data, which you can use for centralized backup and which provide some level of disaster tolerance. Some of the products' architectures allow more wide-area data replication than other architectures allow. Therefore, investigate product architecture as you evaluate each solution, and prototype your implementation. Extend your product investigation to the granularity at which a product lets you select data to mirror. Some products let you select specific files to mirror, and other products mirror at the volume level. An additional benefit of network-based data replication is the flexibility that some products' many-to-one mirroring capabilities provide. This flexibility lets you mirror many systems to one failover server, an approach that differs from the paired-server implementation MSCS offers.
The advantage of mirroring data can also be a disadvantage. Because data is mirrored, every data change results in two or more writes to disk. The or more comes into play if a log file holds changes before the changes are written to the target system. Depending on the implementation, the use of log files can mean that one write on a nonmirrored system can result in four writes in a mirrored solution: one write to write the data, one write to the source transaction log, one write to the target transaction log, and one write to the mirrored target data. The additional overhead these writes cause can make mirroring solutions less desirable in write-intensive environments. Another disadvantage of data mirroring is that recovering from a failure requires the remirroring of data. For long outages affecting large volumes of protected data, remirroring can cause lengthy recovery times. Data mirroring solutions try to reduce recovery time by maintaining logs of the changes that have occurred and mirroring only the changes to the recovered system.
Another factor to investigate when you evaluate data mirroring and failover solutions is how a solution supports application failover. The products listed in Table 1 handle the failover of file shares, and OctopusHA+ and Co-StandbyServer handle print shares without a reboot as well. Table 1 also identifies some basic failover features these solutions provide, and it shows that, unlike MSCS, these solutions limit out-of-the-box failover support to the system level. That is, these solutions detect only system failures and fail over all applications, rather than individual applications.
Each solution addresses the intricacies of failing over applications differently from other solutions. Remember that, unlike MSCS, these products are attempting to provide failover support to applications such as Exchange 5.0, which were not written with failover in mind. Beyond file and print services, all these solutions support only active/standby configurations for most server applications. Application failover might be version- and feature-specific and require operator intervention and a reboot at the time of failure. Application failover support is an area in which prototyping can stop you from taking the wrong high-availability path. The last three columns in Table 1 identify some failover features data mirroring and failover solutions provide for major BackOffice applications.
Exchange, with its reliance on primary computer name and account and use of Registry keys, is a particularly difficult application to support. Initially, data mirroring failover solutions for Exchange were little more than workarounds that involved prestaging a failover system for Exchange disaster recovery. The introduction of MSCS, with its support for active/active server configurations with active/standby failover support for Exchange 5.5 Enterprise Edition, and competition among the data mirroring vendors has driven the refinement of data mirroring with failover solutions. All the vendors in Table 1 now automate the entire Exchange failover process (changing the primary computer name, stopping and starting services, patching the Information Store to fix globally unique identifiers--GUIDs--and in some cases removing the computer from or adding it to the domain). Microsoft's planned support for MSCS and Exchange Enterprise Edition versions that support active/active configurations, and perhaps even service partitioning, should drive the data mirroring vendors to even higher ground. Some vendors are already investigating the feasibility of adding many-to-one Exchange failover capabilities.
Taking Stock
I hope you now understand why clustering isn't the only path to increased NT
availability. Although data mirroring doesn't fit my definition of clustering,
nor is it a fault-tolerant solution, it offers features that clustering can't
offer, and it might better meet your NT availability and business requirements.
If you want to use your existing equipment, can tolerate system-level failover
and restart of your existing applications, or are in search of an increased
level of disaster-tolerance, the recent improvements in data mirroring with
failover products more than justify a download from the Internet and a day or so
of evaluation.
