In "Software Update Services, Part 1" (November 2002, http://www.secadministrator.com, InstantDoc ID 26767), I introduced you to the basic components of Microsoft Software Update Services (SUS) and showed you how to set up SUS to automatically deploy crucial updates to computers on your network. In this article, I examine scalability and stability concerns—how to test updates before you deploy them in your production environment, handle large numbers of Automatic Updates (AU) clients and limited bandwidth, deploy different AU lists to multiple computers, and track update installation activity.
Rolling Out SUS to Your Production Environment
Most people agree that before you install a new update on computers, you should first test it in your production environment. Typically, you should install a new fix with a limited rollout on some noncrucial computers and observe those computers for a week or so. At the same time, you should read the Windows & .NET Magazine UPDATE newsletters (http://www.winnetmag.com/email) to learn about problems early adopters have encountered. You can then roll out the update to your larger production environment.
You can use two simple methods to implement the rollout process with SUS. The low-tech method involves simply downloading and installing the update manually on your test systems. When you're ready to roll out the update to your production environment, approve the update on your SUS server, as Part 1 describes. This method is appropriate for small networks, but SUS also supports a more controlled and automated method for coordinating testing and deployment of crucial updates.
To use this more sophisticated method, you need to set up two SUS servers—let's call them SUSTest and SUSProd. Configure your production computers to pull their updates from SUSProd by editing an appropriate Group Policy Object (GPO) that you apply to all your production computers; maneuvering to Computer Configuration, Administrative Templates, Windows Components, Windows Update; and entering SUSProd as the Set intranet update service for detecting updates option. Next, create another GPO, link it to your test computers, and enter SUSTest as the Set intranet update service for detecting updates option. To test an update, simply approve it on SUSTest and all your test computers will install it. After you're satisfied that the update is safe for wider deployment, log on to SUSProd and approve the update.
To reduce the likelihood of errors (e.g., accidentally approving the update in SUSProd before you approve it in SUSTest), you can create separate user accounts such as SUSTestAdmin and SUSProdAdmin and place each user in the local Administrators group on the corresponding SUS server. Let's hope that Microsoft will enhance SUS so that you can manage both your test and production environments from one SUS server and that you can someday use SUS to require that an update be approved in the test environment before you can approve it in production. But, for now, the dual SUS server method isn't a bad solution. If you need to approve different updates for different types of computers (e.g., servers, workstations, domain controllers—DCs) in your production environment, you need to set up different SUS servers for each of type of system and configure those systems to pull updates from their corresponding SUS servers.
Understanding Bandwidth Capacity
Pulling updates across the network can eat up bandwidth. Fortunately, Microsoft thought about this problem when it designed SUS. When you implement SUS, you must consider two aspects of bandwidth capacity. First, when SUS servers synchronize their databases of available updates with Microsoft Windows Update servers, your Internet connection's bandwidth is affected. Second, when AU clients download those updates—either from your SUS server or directly from a Windows Update server—the bandwidth of your internal network and Internet connection is affected. You have options to control both situations. SUS always downloads the catalog (i.e., a list of available updates and their descriptions), but you can choose whether it downloads the update packages to the SUS server. To configure this option, open Microsoft Internet Explorer (IE), go to an SUS Web site such as http://sustest/susadmin, and select the Set Options link. Under the Select where you want to store updates setting, you can specify the Maintain the updates on a Microsoft Windows Update server setting and enter a folder name under Save the updates to a local folder.
Specifying the Maintain the updates on a Microsoft Windows Update server setting lets you avoid the initial hit of a 500MB download, but in the long run, you'll almost certainly save Internet bandwidth by specifying the Save the updates to a local folder setting. After the initial synchronization, you have to download future updates only once from the Internet, regardless of how many computers you need to update or how many SUS servers you have. Because such a scenario can create a disk space problem, I recommend using the Maintain the updates on a Microsoft Windows Update server setting only when you must support many different locales but just a few computers per locale.
Microsoft says that SUS requires about 500MB of disk space per locale. But what if you have different types of systems or multiple SUS servers that support testing and production rollouts? Having each SUS server download the same updates from Microsoft's servers consumes a lot of Internet bandwidth. Instead, use the Select which server to synchronize content from setting to configure an SUS server to synchronize catalogs and updates from another internal SUS server. For example, you can configure SUSProd to use the Synchronize directly from the Microsoft Windows Update servers option, then configure SUSTest to use the Synchronize from a local Software Update Services server option and enter SUSProd as the server. You should configure your test SUS server to get updates from your production server instead of vice versa so that your production environment isn't dependent on your test environment, which would introduce a potential point of failure. Even if your test SUS server experiences problems, you can still approve appropriate updates in the production environment and roll them out. When your AU clients notice a new update, they start to download it, either from your local SUS server or from a Windows Update server. AU clients use the Background Intelligent Transfer Service (BITS) to throttle downloads so that the downloads don't interfere with the computer's other network activities. You can learn more about BITS at http://msdn.microsoft.com/library/default.asp?url=/library/en-us/bits/bits/bits_start_page.asp. Windows XP includes BITS 1.0, and XP Service Pack 1 (SP1) and Windows 2000 SP3 includes BITS 1.2. If you choose to install only the AU client on Win2K computers (as Part 1 describes), SUS will also install BITS.
Using Multiple SUS Servers
Multiple SUS servers can support large numbers of clients and conserve bandwidth between sites. Microsoft says you can support 15,000 clients with one SUS server running on a Pentium III 700MHz processor and 512MB of RAM. For more than 15,000 clients, you might need multiple SUS servers. You need to use different GPOs to divide your clients and direct the correct number of clients to each SUS server.
For example, suppose your organization has headquarters in New York and has large offices in San Francisco and London. If you don't want every client in San Francisco and London to drag updates over the WAN from the New York SUS server, you can set up SUS servers at each of the other sites. You can then link a different GPO to each site that directs clients at that site to pull updates from the local SUS server and configure the San Francisco and London SUS servers to synchronize content with the New York SUS server. Each update would then traverse the WAN links only once before being distributed locally. If you want the administrators at each site to decide which updates to deploy locally, they can simply approve updates accordingly on their local SUS servers. But if you want to centrally control which updates you approve, you can configure Synchronize from a local Software Update Services server with the New York SUS server name, then enable Synchronize list of approved items updated from this location (replace mode)—as Figure 1 shows—on the San Francisco and London SUS servers. Then, when you approve an update on the New York server, the other SUS servers will also mark it approved. If the San Francisco and London offices' Internet connections aren't as busy as the WAN link to New York, the local SUS servers can download the approved update list from New York and download the sizable updates from the Internet. However, if you enable Synchronize from a local Software Update Services server, you must also select Save the updates to a local folder. This limitation isn't a problem because after you perform the initial synchronization, updates will trickle in as they're released.
Monitoring Activity
After you set up your SUS infrastructure, you need to monitor its activity, make sure new updates are downloaded from Microsoft's servers, and track which computers have successfully installed the new updates. You can track synchronization activity by selecting View synchronization log under Other Options. As Figure 2 shows, the Synchronization Log lists each synchronization event, whether automatic or manual, and which updates, if any, SUS downloaded. The synchronization log also alerts you to any problems the SUS server encounters when it gets updates from a Windows Update server or from your internal server. Next, you can verify your approval decisions with the View approval log option. As Figure 3 shows, the approval log documents each time the approval list is changed, when, and by whom, as well as the list of approved updates. You can consult the Win2K System log for more information about SUS activity and troubleshooting.
Figure 4 shows an example of the SUS event log. For a complete list of events, see Appendix B: "Software Update Services Event Log Messages" in the "Software Update Services Deployment" white paper (http://www.microsoft.com/windows2000/windowsupdate/sus/susdeployment.asp). SUS events contain the most detailed event description, complete with advice about user action. To learn about all the possible events that SUS might log, see Appendix B in the deployment guide.
To track the installation of updates on AU clients, configure the updates to report their activity to the Microsoft IIS log on an IIS server you specify. (IIS logs typically reside in \winnt\system32\logfiles\w3svc1.) To configure AU clients to report activity, fill in the Set the intranet statistics server GPO setting under Computer Configuration, Administrative Templates, Windows Components, Windows Update, as Part 1 explains. Typically, you should specify the same SUS server to receive log updates. If you specify a different IIS server, you must copy wutrack.bin from the Vroot folder on the SUS Web site to the Vroot folder on the other IIS server. After you configure the AU clients, they start sending a record of their update activities to the IIS log on that server by making URL requests to wutrack.bin. Unfortunately, the data's format is nearly impossible to read and is evidently designed for analysis by a program, not a human. This flaw is serious because reporting is crucial to managing update deployment. You can use HFNetChk, but this tool requires much more network traffic and time to scan your network than if SUS had a reliable reporting mechanism.
Web Figure 1 (http://www.secadministrator.com, InstantDoc ID 27069) lists initialization (&A=n), self-update (&A=s), and detection (&A=d) that 10.0.0.41 reported to SUS server 10.0.0.68. The information in the wutrack.bin log is cryptic, but the deployment guide makes an effort to explain the log entries in Appendix C: "Client Status Logging." The log contains records other than wutrack.bin messages. Appendix C also shows how to configure IIS to log only wutrack.bin messages by turning off logging at the Web site level and enabling logging only on the wutrack.bin file.
You can monitor update activity by using the \%windir%\windows update.log file on each of your clients. This log file contains an easy-to-read record of update information. As Web Figure 2 (http://www.secadministrator.com, InstantDoc ID 27069) shows, the computer queried a Windows Update server for new updates, downloaded them, installed them, and rebooted. If the systems administrator configured the client to pull updates from an internal SUS server, that configuration would be reflected in the log.
Handling User Intervention
You probably want to prevent users from interfering with SUS updates and from downloading unapproved updates from a Windows Update server. You can use group policy settings to control these actions. Enable the Remove access to use all Windows Update features setting (under User Configuration, Administrative Templates, Windows Components, Windows Update) in a GPO, and any XP computers that apply the GPO will disable the AU folder on the Control Panel System applet. However, this setting doesn't affect users' ability to operate the XP Keep your computer up-to-date with Windows Update task on the Help and Support Center tool and has no effect on Win2K computers. To lock down these options, enable Remove links and access to Windows Update under User Configuration, Administrative Templates, Windows Components, Start Menu, Taskbar.
Receiving Email Notification
If you want to receive email notification whenever Microsoft issues a crucial update for your products, you can register at http://go.microsoft.com/fwlink/?linkid=6931. If you subscribe to this service, you won't have to log on to your SUS server every day to determine whether to approve new updates. The email service will prompt to you for approval when necessary. The difference between this alert service and the security bulletin service is that the security bulletin service includes notification of all security concerns for all Microsoft products. The SUS email notification service covers only security concerns that affect SUS-supported products (XP, Win2K, and IE 5.0 and later) and problems for which Microsoft has issued a crucial update that you can install using SUS. The advantage to subscribing to both services is that you can track which updates SUS will handle and which ones you'll handle through other means.
SUS isn't a total solution for keeping systems up-to-date; it doesn't address other Microsoft Office or Microsoft BackOffice products. But the combination of SUS and group policy's Software Installation features lets you automatically keep the OS up-to-date with service packs and hotfixes and eliminates most of the work involved in patching your computers. Keep in mind that after you approve an update and SUS installs it, SUS doesn't include a way to uninstall the update. You must write a short startup script in group policy under Computer Configuration\Windows Settings\Scripts (Startup/Shutdown) that uninstalls the update and deploy the script to affected computers.
