Would you like your Microsoft IIS Web servers to run more reliably without interruption? Would you like applications that can declare themselves unfit to live and automatically restart without administrative intervention? Would you like clients to be able to upload a large number of files, resume if the transfer fails, and not consume all your bandwidth? Would you like Web applications that meet all these goals and don't exceed 50 percent of the CPU? If you've used IIS 5.0 and IIS 4.0, you might be surprised when I tell you that IIS 6.0 moves you closer to meeting these goals.
Microsoft.com's Results
At Microsoft TechEd in Dallas in June 2003, Casey Jacobs, group manager for Microsoft.com, and I gave a presentation. Casey presented a few statistics about Microsoft.com, which is the largest IIS 6.0 deployment to date:
- 99.8 percent uptime (according to Keynote Systems, which tests Web site performance)
- 950 servers and three data centers
- 80 Internet sites serving 1000 Microsoft SQL Server databases and thousands of Web applications
- 25.5 million home page views in March 2003
- more than 75,000 requests per second
- more than 300,000 concurrent users
- 20 system engineers, 12 DBAs, and 16 staff members who provide 24 * 7 support
Better than 99 percent uptime on 950 servers supported by 20 system engineers is impressive, especially considering that this information comes directly from the people in the field, not from the marketing department. The Microsoft people achieved these numbers with a mixture of IIS 6.0 and IIS 5.0 servers.
What was Microsoft.com's experience in reliability in moving from Windows 2000 Server to Windows Server 2003? Microsoft Developer Network (MSDN) had a 100 percent increase in uptime, as Table 1, page 42, shows. Some of the OS restarts were required reboots for hotfixes and service packs, which Microsoft is working to significantly reduce.
So how did these improvements in Microsoft.com reliability come about? Some cool features in the Windows 2003 server family and IIS 6.0 make such improvements possible.
Isolation of WWW Service Administration and Monitoring
One of the biggest changes that directly affects reliability involves how IIS 6.0 works internally. To have a basis for comparison, let's take a quick look at the IIS 5.0 architecture. In IIS 5.0, the primary process that runs the core IIS functions is inetinfo.exe. Inetinfo also hosts Web applications that you set to run in Low (In Process) Application Protection in the Microsoft Management Console (MMC) Internet Information Services console. You can also run Web applications "out of process" by setting Application Protection to Medium (Pooled) or High (Isolated), in which case they run in a process named dllhost.exe. Inetinfo has other significant duties such as routing incoming requests from Winsock to the proper out-of-process Web applications and hosting the IIS Admin Service, which communicates with the metabase (i.e., the configuration store of settings for IIS Web sites and directories). As a result, tasks related to the configuration and internal administration of the Web server run in the same process as Web applications. As a result, a poorly written application can throw an exception (i.e., crash) or otherwise halt Inetinfo and bring the entire Web server, including the out-of-process applications, to a halt.
In IIS 6.0, the WWW Service Administration and Monitoring component runs in a separate process from Web applications so that a crashed or hung Web application can't cause WWW Service Administration and Monitoring to fail. This separation lets WWW Service Administration and Monitoring monitor the Web server and keep it running—WWW Service Administration and Monitoring can even restart applications that fail to respond to "Are you live?" requests from the component.
Application Pools
IIS 6.0 Web applications run in a worker process called w3wp.exe, the replacement for IIS 5.0's Dllhost (and IIS 4.0's wam.exe). However, in IIS 6.0, you can give a worker process a name and manage it as an item called an application pool. You can easily assign a Web site, directory, or virtual directory to any application pool by using that item's Properties dialog box, as Figure 1 shows.
Keep in mind that each application pool is isolated from the other pools so that if one pool fails or is recycled, only the applications in that pool are affected; other application pools keep delivering content. This feature is perhaps the heart of IIS 6.0's reliability improvements. You can organize your applications in pools that best suit your requirements. For example, you might group applications that run reliably into one pool and isolate unreliable or untrusted applications into separate pools to insulate the reliable applications from the untrusted applications' wayward behavior.
WWW Service Administration and Monitoring automatically checks that an application pool is responsive every 30 seconds (by default) or at an interval that you can define on the Health tab of the application pool's Properties dialog box. If the pool fails to respond, WWW Service Administration and Monitoring recycles, or restarts, the pool without administrator action, which greatly improves reliability. In addition, you can use the settings on the Recycling tab, which Figure 2 shows, to configure when that pool will recycle (i.e., stop and start). For example, by default, application pools recycle every 1740 minutes (i.e., 29 hours). This value doesn't make any sense to me as a default, so I set the recycling to occur at a specific time of day, say 3:00 a.m. local time. Scheduled recycling can be useful for applications that need to be periodically reset.
The Recycling tab also lets you monitor a worker process's memory consumption and recycle the application when it exceeds a certain threshold. The ability to automatically recycle applications that are overconsuming resources will help a great many IIS servers that must deliver applications that inexplicably consume too much memory over time. For more information about IIS 6.0's recycling process, see the sidebar "How Recycling Works."
When you configure recycling for your application pools, you will likely want to set IIS 6.0 to record recycling events in the Event Viewer when they occur. By default, only memory- and time-based recycle events are recorded. Look in the IIS 6.0 Help files for the LogEventOnRecycle metabase property. This property is a bitmask value in which each bit turns recording of a specific event on or off. You can use the following command to turn on recording of all the events:
cscript SystemDrive\InetpubAdminScripts\adsutil.vbs set W3SVC/AppPools/ LogEventOnRecycle
Self-Monitoring Applications
The ability to configure application pools so that they'll be recycled when specified events occur is useful, but the ability to build health checks right into an application and, if necessary, have that application request recycling is even more useful. Any IIS 6.0 Internet Server API (ISAPI) extension, including those that you author, can use the new HSE_REQ_REPORT_UNHEALTHY function to invoke application recycling. This capability enables you to build health checks directly into your ISAPI extensions instead of relying exclusively on the server. For more information about this function, go to http://msdn.microsoft.com/library/default.asp?url=/library/en-us/iissdk/iis/extensions_ssf_hse_req_report_unhealthy.asp.
Active Server Pages (ASP) and ASP.NET in IIS 6.0 are both self-monitoring and will under certain circumstances invoke HSE_REQ_REPORT UNHEALTHY. ASP, for example, monitors how long the ASP script engine takes to return from a processing request. If the time exceeds the ASP Script Timeout setting in the Internet Information Services console, IIS tells the script engine to stop processing. If the script engine fails to return from the stop request, ASP abandons the thread. Internally, ASP keeps track of the number of abandoned threads. If enough threads are abandoned, asp.dll requests recycling.
CPU Affinity
Application pools have another capability that can be useful on multiple-CPU systems: CPU affinity. CPU affinity lets you designate which CPUs to use for your Web applications and can be useful if you're contractually obligated to designate a CPU for a particular client's application. To establish affinity between an application pool and a CPU, you have to set two metabase properties: SMPAffinitized and SMPProcessorAffinityMask.
You use the SMPAffinitized property to turn on affinity for an application pool. In Adsutil, you can use the following command to set this property:
cscript SystemDrive\InetpubAdminScripts\adsutil.vbs set W3SVC/AppPools/ ApplicationPoolName/ SMPAffinitized TRUE
where ApplicationPoolName is the name of the pool with which you want to establish affinity. When SMPAffinitized is set to the default value of 0 (FALSE, using Adsutil), the application pool can run on any processor.
The SMPProcessorAffinityMask property is a bitmask provided as a hexadecimal number that represents the CPU you want to assign to the pool. You can also assign a pool to run on more than one processor, if your system has multiple processors. The following command configures a pool to run on processors 0 and 2:
cscript SystemDrive\InetpubAdminScripts\adsutil.vbs set W3SVC/AppPools/ ApplicationPoolName/ SMPProcessorAffinityMask 0x5
The Internet Information Services (IIS) 6.0 Resource Kit (http://www.microsoft.com/downloads/details.aspx?familyid=80a1b6e6-829e-49b7-8c02-333d9c148e69&displaylang=en) has a table of processor hex values in Chapter 3, "Running IIS 6.0 as an Application Server."
Web Gardens
IIS 6.0 can perform yet another trick in relation to application pools—its Web garden feature lets you designate that an application pool will contain more than one worker process. When so configured, IIS 6.0 will create an instance of w3wp.exe for each request up to the number of worker processes you specified for the pool. For example, if you set an application pool to have three worker processes, after three requests, you'll see three instances of W3wp running on your server, all delivering the same content. IIS 6.0 makes subsequent connections in a round-robin fashion to the worker processes in the Web garden. Although Web gardens don't have session affinity per se, IIS 6.0 routes all the traffic from one connection to the same worker process. The worker process stores the session information, Secure Sockets Layer (SSL) secret key, and authentication information until the connection is terminated or the worker process is recycled.
Web gardens aren't widely used, partly, I think, because IIS 6.0 performance and reliability are generally good to excellent without them and partly because people don't know what kind of applications are likely to benefit from using them. I've been having trouble finding examples of real-world installations in which this feature clearly benefits the application. If you know of any, please write to me. Nevertheless, here are a couple of examples.
A Web garden might be useful when your application uses a component that has a built-in restriction. For example, if you have a database connection object that accepts only 10 connections, you can enable n worker processes to provide 10 * n connections.
Another and more likely scenario is a database connection "blocking" an application. The application makes a request to the database that takes some time to render. The thread (i.e., the working unit for executing code on the CPU) that makes the request is now tied up waiting on that request and isn't available for any other task. Thus, the available pool of worker threads is reduced by 1. If requests come in faster than the database can process them, your application can become "thread starved." This scenario would benefit from a Web garden because each process in the Web garden has its own dedicated thread pool.
Note that ASP.NET also has Web garden and processor affinity features that are completely different from the IIS 6.0 implementation. You can find the settings for these features in the processModel element of the machine.config XML file, the primary configuration file for ASP.NET applications. IIS 6.0 completely ignores the machine.config settings for Web garden and processor affinity; only IIS 5.x uses them. For more information about ASP.NET and IIS 6.0, see "How ASP.NET Works with IIS 6.0" (http://www.microsoft.com/technet/treeview/default.asp?url=/technet/prodtechnol/iis/iis6/proddocs/resguide/iisrg_arc_dkvi.asp).
Windows System Resource Manager
You can lose all the benefits of process isolation that application pools offer if your applications overconsume system resources. Sure, you can configure an application to recycle when too much memory is used, but wouldn't you rather be able to allocate a maximum amount of RAM for the application to use? Also, if your application is CPU-intensive, IIS 6.0 has little in terms of built-in capabilities to help you because its built-in CPU monitoring does little more than turn the application on and off. As a user-mode application, IIS 6.0 just doesn't have a lot of rights to manage what goes on in kernel mode and therefore can't really self-regulate these matters.
To solve these problems, Microsoft provides a significant new service called the Windows System Resource Manager. WSRM runs only on Windows 2003, Enterprise Edition and Windows 2003, Datacenter Edition, but you can manage it from any Windows 2003 server. With WSRM installed, you can configure a process (including services) so that it won't use more than a specified amount of memory or CPU capacity. In addition, you can assign applications to specific CPUs. Thus, you can designate applications that must have high availability to specific CPUs and assign other processes and services to run on the remaining CPUs--without using IIS 6.0's processor affinity feature. Performance monitoring, the ability to enforce limits based on date and time, the ability to include and exclude users, and logging functions are also built into WSRM. If you need to reliably enforce memory or CPU use on your server, WSRM is the way to go. For more information about WSRM, see http://www.microsoft.com/windowsserver2003/downloads/wsrm.mspx.
BITS
Almost completely unheralded is a new feature called Background Intelligent Transfer Service (BITS)--what I call a "dribble service." Aside from offering the entertainment value of seeing your colleagues look at you in disbelief when you say "dribble service," BITS has the practical aim of conserving bandwidth, thereby increasing availability under high loads.
When installed, BITS manages downloads from and uploads to the server by using "spare bandwidth" to deliver content to the client. BITS also tracks requested downloads and lets you queue and resume requests. As Figure 3 shows, you can configure several parameters, including maximum file size, how frequently to purge the queue of incomplete jobs, and whether to enable limited support for server farms. If you've downloaded large files from MSDN, you'll recognize the client-side window that shows the status of your download activity.
The BITS service isn't installed by default when you install IIS. You can install BITS on Windows 2003 servers by opening the Control Panel and selecting Add/Remove Programs, Windows Components, Application Server, Internet Information Services (IIS), Background Intelligent Transfer Service Server Extensions. To download a version of BITS that runs on Win2K Server with IIS 5.0, go to http://www.microsoft.com/downloads/details.aspx?familyid=17967848-be86-4cd6-891c-ec8241611ad4&displaylang=en.
Be sure to check out the Help files for important details about security and virtual directories that are BITS-enabled. BITS bypasses IIS 6.0 security settings and lets you upload files to a folder that has the IIS-based Write permission disabled. Normal NTFS security permissions are enforced. As a bonus, BITS has a software development kit (SDK) that you can download at http://www.microsoft.com/downloads/details.aspx?familyid=ad9fb937-62f9-4b9f-993b-f754f968b8a6&displaylang=en. BITS works with SSL, is fully programmable, and uses Windows authentication and NTFS, so perhaps it's the badly needed replacement for FTP to publish files to an IIS server without using Microsoft FrontPage Server Extensions.
Newly Reliable
As I've discussed in this article, IIS 6.0 features
- isolated applications that you can insulate from poorly behaving Web applications
- automatic recycling of nonresponsive applications by an independent service
- automatic recycling of worker processes based on programmable events or configurable parameters
- assignment of application pools to designated CPUs
- limited application consumption of memory and CPU time
- managed file uploads and downloads that don't overconsume bandwidth
These capabilities plus others that I haven't mentioned due to space constraints (such as Quality of Service--QoS--policies to restrict bandwidth for IIS) make a compelling case that IIS 6.0 is far more reliable than any previous version of the IIS server. Add IIS 6.0's performance and security improvements and the universally good reports I've heard from those who've deployed IIS 6.0, and you have several good reasons to move to the Web server's latest version.
