When I was new to Windows NT, I learned that all server changes fall on a hierarchy of difficulty. My teachers drummed into my mind that any change that challenges the order of NT domains is on the most difficult end of the spectrum. They told me that domain controllers can change roles or move to a different domain through either an act of God or a reinstallation of the OS—and that the former solution was usually easier to justify than the latter. As I worked with the OS and eventually taught NT classes, I passed this commandment on to other systems administrators: After you set up a Primary Domain Controller (PDC) or a Backup Domain Controller (BDC), don't try to change its role or its domain. If you try to change a domain controller, I explained, you'll suffer through countless network conflicts and corrupt Security Accounts Manager (SAM) databases.
Considering domain configuration immutable isn't always comfortable. I've seen many NT servers that administrators installed as BDCs then wanted to change to be standalone members of the domain. I've seen other cases in which administrators wanted to remove a BDC from one domain and establish a new domain on the server. And frequently, I've wanted to change the name of an existing domain.
I always assumed that the three tasks were equally difficult, and I know a number of experienced NT administrators who agreed with me. I recently discovered, however, that I was wrong about one of these situations. You can't easily change domain controllers into standalone servers or split domains in two, but you can rename a domain, even a master domain that contains thousands of user accounts and that multiple resource domains trust.
Why Bother? 
My employer, St. Paul Fire and Marine Insurance, is the eighth-largest property and casualty insurance company in the world; corporate headquarters are in St. Paul, Minnesota. A year ago, St. Paul Fire and Marine was the world's fourteenth-largest property and casualty insurer, but in April 1998, the company purchased Baltimore, Maryland-based USF&G. The corporations' merger nearly doubled the number of employees and IS resources that my department is responsible for. Each company maintained more than 100 production NT servers before the merger.
Among the many tasks my department performed to integrate the two organizations was reconciling the companies' NT domain structures, which held accounts for all the companies' PC users. The companies agreed to consolidate systems administration and data security functions in St. Paul, so the IS department decided to create a single master domain structure to serve the new organization's 12,000 users.
| Renaming a master domain gives administrators pause, with good reason. But the renaming process eliminates slow, manual tasks that migrating to a new domain requires. |
Of the two companies' domains, St. Paul Fire and Marine's master domain contained the most user accounts. All of St. Paul Fire and Marine's more than 8000 user accounts resided within the company's master domain. Most of the company's NT servers were members of one of 21 resource domains that trusted the master domain. Six standalone servers were members of the master domain, and three BDCs served as the master domain's Remote Access Service (RAS) servers. Of the five domain controllers in the master domain, two (including the PDC) ran NT Server 4.0 with Service Pack 3 (SP3), and three ran NT Server 3.51 with Service Pack 5 (SP5).
For reasons my colleagues and I forgot long ago, the master domain had the name CHQ_COD_PROD. Needless to say, this name didn't reflect the master domain's role in the merged company's NT structure, and it was a bit ugly. In addition, the name CHQ_COD_PROD was a time bomb waiting until the company converts to Windows 2000 (Win2K—formerly NT 5.0) to explode. The Win2K naming structure complies with the X.400 standard, which does not permit use of the underscore character (_).
Biting the Bullet
When the merged Enterprise Networking Services (ENS) team's integration discussions turned to domain structures, we unanimously agreed to replace the existing CHQ_COD_PROD domain with a new master domain named SFM that would support all the merged companies' users and resource domains. We all assumed that SFM would be a new domain because we knew that altering an existing domain's name was impossible.
After we decided to create a new SFM domain, we looked for ways to automate the process of migrating the existing master domain's user accounts and those accounts' permissions to access resources in the 21 resource domains. Microsoft tools offered little help. Microsoft Windows NT Server 4.0 Resource Kit's ADDUSERS utility transfers user accounts between domains, but it re-creates only the accounts' basic attributes—username, full name, description, and group memberships. (For information about the utility, see Mark Minasi, "AddUsers," May 1998.) These attributes only scratched the surface of CHQ_COD_PROD users' permissions, and our Help desk staff shuddered at the thought of helping so many users set new passwords.
None of the third-party tools we examined offered much help, either. We didn't find a tool that let us analyze the permissions that administrators can grant to users and groups in trusting domains, and no utilities consistently retained account passwords.
After our initial research, the ENS team agreed that SFM and CHQ_COD_PROD would have to operate in parallel for a time, during which we would migrate users in small groups. This procedure would not be attractive. Network administrators would have to gather data from servers in the resource domains to discover which permissions each CHQ_COD_PROD user account had, re-create the account in SFM, and grant the same permissions to the new account. The move would inconvenience users because administrators would need to switch users' workstations' logon domains, and the users' SFM accounts would initially have blank passwords. We estimated that switching all the accounts would take 4000 person hours and that we couldn't complete the process in less than 10 weeks.
Around the time that we came to this conclusion, I was using Microsoft's TechNet to research a different problem, and a provocative document title caught my eye: "Unable to Change Domain Name of Windows NT BDC." (The document is available at http://support.microsoft.com/support/kb/articles/q139/4/71.asp?FR=0.) When I examined the article, I was surprised to find that it suggested that you could change a domain's name. I read additional material within TechNet and began to realize that I had been foolish not to consider changing CHQ_COD_PROD's name. The fact that I wasn't alone was little consolation.
Microsoft Support Engineer Eric Fitzgerald, who assisted us in our domain renaming project, told me that Microsoft hadn't formally supported the procedure of renaming domains until 1997, but that changing domain names had always been possible. Fitzgerald, who wrote the Microsoft article "Renaming a Domain: Process and Side Effects" (http://support.microsoft.com/support/kb/articles/q178/0/09.asp?FR=0), cited corporate mergers and X.400 compatibility as reasons for Microsoft's support for renaming domains.
How It Works
As everyone who has taken a few NT classes or worked with NT knows, the security ID (SID) number is the cornerstone of NT security. NT assigns SIDs to users, groups, shares, servers, and domains. Internally, NT uses SIDs almost exclusively, although users and administrators generally identify system resources through the resources' convenient (to humans) alphanumeric symbol—their name. When resources' names change, their SIDs remain constant. You can change a domain's name because NT maintains domains' security relationships through SIDs, not alphanumeric names. For example, when you add a user account that resides in a master domain to a local group on a server in a domain that trusts the master domain, NT adds only the user account's SID to the server's SAM database. When you use User Manager to examine a list of user accounts in the server's local group, User Manager looks up the account's alphanumeric name for your sake, but NT doesn't care what that name is.
If you want to test NT's reliance on SIDs without renaming a domain, experiment with a trust relationship. On a server in a resource domain, grant permissions or group membership to a user account in the master domain. Then, break the two domains' trust relationship, and examine the properties of the object you changed the user's permissions on or the group you added the user to. The server will remember the account's permissions, but until you reestablish the trust relationship, utilities will list the account's name as Account Unknown. The resource domain's server can't look up the account's name, but the account's SID remains on the server. When you reestablish the domains' trust, NT will restore all account names on the server.
You create essentially the same situation when you rename a master domain because you break trust relationships during the renaming process. Renaming a domain that contains crucial servers and maintains vital trust relationships isn't simple, but you can accomplish the goal with careful planning.
Convincing Everyone
If you suggest to your colleagues that your company rename a domain, expect the response "No way." When I first raised the idea to my manager, an experienced NT engineer, I heard those words. Members of our IS group, other managers, and some of our more savvy users echoed the reaction. However, when those doubters compared the renaming procedure's advantages with the alternative of creating a new domain and moving all user accounts to the domain, they found good reason to take a second, open-minded look at the proposal. I used Table 1 to compare our IS department's choices. The factors that Table 1 lists justified renaming the domain to our data security department, which performed much of the migration's groundwork.
Another advantage of renaming the master domain was the process' seamless effect on NT workstations. About 200 workstations ran NT in our resource domains; those machines didn't require manual intervention because the mechanism that links NT workstations to a domain and through the domain to other trusted domains uses SIDs, not names. After we renamed the master domain, NT automatically updated the drop-down list of logon domains on our resource domains' NT workstations.
Despite the advantages of renaming a domain, you must perform a few of the process' tasks manually. You must re-create the domain's trust relationships, change the default domain in users' logon boxes, and edit your service accounts. My department discovered that service accounts are a notable exception to NT's standard use of SIDs. NT stores accounts that services use to log on to the system as text entries in each server's Registry. As a result, service accounts don't automatically change to reflect a domain's new name. Fortunately, most NT services use the server's System account, so we had to change service accounts' text files on only a handful of systems. Before renaming CHQ_COD_PROD, we ran Server Manager on all our production NT servers, looking for services that used an account in CHQ_COD_PROD, and noted those that needed attention during the name-change process.
Staff Concerns
Because of the preponderance of reasons for renaming St. Paul Fire and Marine's master domain, our technical planning team quickly geared up for the name change. We acknowledged, however, that the domain renaming process had drawbacks.
One common concern was that the process might fail insidiously. For example, if the master domain's SAM database became corrupt, the problem might not be fully apparent until users had worked in the renamed domain for a day or two. In addition, if NT security became corrupt after we renamed the master domain, we would have trouble undoing the name change. Changing the domain's name back to CHQ_COD_PROD might not repair whatever damage the initial name change caused in the SAM database.
The aspect of the renaming procedure that generated the most concern was that the process involves a sudden, networkwide change. As Brad TerEick, manager of our ENS department, observed, St. Paul Fire and Marine had never implemented a procedure that affected all 8000 of our users in 1 day. Unlike migrating to a new domain, renaming a domain doesn't permit the tried-and-true practice of operating new and old systems simultaneously. Renaming a master domain is an all-or-nothing process, not a project for the timid.
Finally, our research revealed that renaming a domain isn't always formulaic in environments that use Microsoft's BackOffice products. We discovered several problems during the CHQ_COD_PROD renaming. For more information about the problems BackOffice causes and the methods for resolving them, see Sean Daily, "How to Rename Your NT Domain," page 113.
Testing, Planning, and Preparing
After we determined that we wanted to rename the company's master domain, we set three primary goals that we wanted to accomplish before the renaming. We wanted to verify that the process worked, determine a way to change nearly 8000 Windows workstations' logon prompts to reflect the new master domain name, and devise a means for undoing the change in case a rollback became necessary. In addition, assuming we wouldn't encounter any roadblocks, we needed to notify affected users and construct a full conversion plan.
The plan to rename CHQ_COD_PROD reached a crucial juncture when, 5 weeks before the planned date for the change, we tested the basic procedure. We were fortunate to have a server lab for Year 2000 (Y2K) compatibility testing that simulates our NT domain and workstation environment. The Y2K lab includes only 14 NT servers, but those servers duplicate key domains, infrastructure, and NT-based applications, including the Microsoft products SNA Server, SQL Server, Internet Information Server (IIS), and Proxy Server; Lotus Notes; Windows Internet Naming Service (WINS); Domain Name System (DNS); and Dynamic Host Configuration Protocol (DHCP). The number of servers in the lab is far fewer than the number of machines in our production environment, but the lab includes all our active user accounts and groups—more than 10,000 objects. We followed Microsoft's procedures to change the name of the lab network's version of CHQ_COD_PROD to SFM. This initial test worked without a hitch, and it took only about 20 minutes. Many of the test network's CHQ_COD_PROD accounts and groups had permissions within resource domains, and all those permissions remained intact after the renaming. The test boosted our confidence tremendously.
Another key to feeling comfortable about renaming the domain was holding regular conferences, not only among the network administrators who performed the renaming procedure but also among Help desk and client/server application support staff. We also used our intranet and email to notify all our NT users that the renaming was approaching. Our final step to ensure the support staff's peace of mind was arranging a schedule for testing workstations and applications after we completed the name change but before most users came to work.
We considered several options for altering the logon box's default domain name on Windows 95 and Windows 3.1 workstations, including using logon scripts and system policies. We even considered an intricate scheme that would use our software distribution system to pass out a time-activated batch file, but we recoiled at the thought of how difficult removing that change might be. Eventually, we settled on an almost laughably simple method. We gave users instructions that explained that when they reached their system's network logon box the day after our name change, they needed to fill in their username and password as usual, but then tab to the next field, Login From, and replace CHQ_COD_PROD with SFM. Our Help desk, which had to deal with user confusion, chose this method and suggested that we perform the domain renaming on a weeknight to keep these instructions fresh in users' minds.
Finding a way to reverse the procedure if unexpected problems occurred was another technical challenge. After quickly rejecting the idea that we could simply change the domain's name back to CHQ_COD_PROD, we set up a new BDC for CHQ_COD_PROD. Just before the renaming, we synchronized this BDC with the domain, then shut it down and set it aside. If we needed to reverse the domain renaming process, we had the option of shutting down the corrupt SFM servers, restarting this BDC as the PDC for CHQ_COD_PROD, and rebuilding the original master domain from it. With these pieces in place, we set the date of the domain renaming to Tuesday, July 14, 1998.
D-Day
After months of preparation, renaming the master domain was almost anticlimactic. During the day of July 14, we coordinated the people who were taking part in the procedure, but we didn't begin renaming the domain until 7:00 p.m. We finished our changes to the server around 11:15 p.m., and we finished testing the renamed domain by 1:45 a.m.
We followed Microsoft's prescribed process carefully, maintaining open telephone connections between the two sites that contained CHQ_COD_PROD domain controllers. At Microsoft's suggestion, we frequently forced WINS replication to ensure that NT propagated our new server entries throughout the network. This cautious attitude paid off. During the 2 days that followed the domain renaming, our Help desk recorded fewer than 150 support calls regarding the name change, almost all of which related to user errors entering the new domain name. When the dust settled, our new SFM master domain was ready to merge with USF&G's domain.
The process of renaming a master domain gives experienced NT administrators pause, with good reason. Don't alter domains without considering the change carefully and using caution. The process of renaming St. Paul Fire and Marine's master domain involved a lot of planning and hands-on support, but we justified that effort by eliminating slow, manual tasks that might have delayed our domain merger.