Monitoring Your Systems with ISA Server

Consider using ISA Server's robust built-in monitoring and reporting functionality

Many third-party companies have built their success upon extending monitoring and reporting functionality for mainstream vendor products. NetIQ's WebTrends, Hewlett-Packard's HP OpenView, and other products extend the feature set of Microsoft's enterprise products. With Microsoft Internet Security and Acceleration (ISA) Server 2000, however, Microsoft has included substantial built-in monitoring and reporting functionality. Third-party reporting and monitoring tools are expensive add-ons, and you might find the internal monitoring capabilities of ISA Server robust and compelling enough to serve your needs. ISA Server's monitoring features comprise built-in and customizable elements that include alerts, logging, and reporting. You can also view realtime information through the Microsoft Management Console (MMC) ISA Management snap-in or through extended performance monitor counters. For information about configuring ISA Server, see my Windows 2000 Magazine article "ISA Server: Your Network's Lifeguard," http://www.win 2000mag.com, InstantDoc ID 22251.

Although I discuss ISA Server Enterprise Edition installed in an array configuration, much of the monitoring functionality is the same when you install ISA Server in a standalone configuration. The array configuration provides centralized management of multiple ISA servers and integration into Active Directory (AD), features that a standalone installation doesn't support. Also, the examples I include assume that you've installed ISA Server in Integrated mode, which combines the functions of both the Firewall and Caching services. (You can install ISA Server in Caching, Firewall, or Integrated modes. For more information about these modes, see Sean Daily's Windows 2000 Magazine article "Microsoft's Stellar ISA Server," http://www.win2000mag.com, InstantDoc ID 15477.) To take advantage of ODBC logging, you'll need an ODBC-compliant database installed. I describe the ODBC functionality based on using Microsoft SQL Server 2000 and Microsoft Access configurations.

ISA Server Monitoring includes alerts, logs, and report jobs. You access and configure these functions under the Monitoring Configuration node in the ISA Management snap-in. ISA Server monitors for alert conditions on a per-array basis, generates logs for each server, and assembles reports that aggregate the data for the entire array.

Alert! Alert!
Trigger-driven alerts notify you when a significant event occurs on an ISA server. The preinstalled alerts include everything from service failures to detected intrusions. To create a new alert, expand the Monitoring Configuration node, right-click the Alerts folder, select New, then click Alert. Name the alert and select whether a specific ISA server or any server in the array will trigger it. Next, select the event and specify the conditions that trigger the alert. Microsoft includes 45 predefined events, such as Service started, Service shutdown, and Intrusion detected. Conditions are properties of a specific event that let you further define the alert. For example, the Intrusion detected event lets you specify the type of intrusion (e.g., all-port-scan attack, Land attack) as an additional condition.

The ability to use conditions to further specify events is particularly useful in classifying alerts based on your security requirements. For the default installation, Microsoft lumps all intrusion detections into one generically defined alert that writes to the event log. However, I recommend that you create individual alerts and specify particular intrusion detections.

After you create an alert, you choose which actions you want to invoke when events trigger the alert. ISA Server can write to the event log, send an email message, run a program, and start or stop a service. You don't need to install the SMTP service to support sending SMTP messages through an alert. Also, you can run any program under a specified account (the account must have Logon as a batch job privilege). ISA Server supports starting or stopping only the Firewall, Web Proxy, and Scheduled Content Download services. However, you can create a batch file that Net Starts and Net Stops additional services. To do so, select the Program check box, which appears in the Actions tab of the event's Properties dialog box, and enter the name of the batch file in the input box to run the batch file.

After you configure an alert, you can open its Properties dialog box to review its settings. In the General tab, you can enable or disable the alert. In the Events tab, you can further define when and how the alert actions should occur. Figure 1 shows some of the fine-tuning options you can choose to configure an alert. Suppose that after you configure a port-scanning alert, the alert fills your event log with notifications. You can then reconfigure the alert to reissue notifications only after the first alert has been manually reset.

Logging Who Went Where
ISA Server logging is adequate; in addition to proprietary ISA Server log format, ISA Server logging supports World Wide Web Consortium (W3C) extended log-file format logging. (Those of you familiar with Microsoft Internet Information Services—IIS—5.0 logging will be pleased that ISA Server continues to support W3C extended log-file format logging functionality.) ISA Server supports logging to a text file or to an ODBC-compliant database, and you can control which fields and what content from the fields is logged. You can configure three different logs independently: the Firewall Service log, the Web Proxy log, and the Packet Filter log. In addition, you can include or exclude any combination of information from the Firewall service, the Web Proxy service, and packet-filtering functions.

The Firewall Service log shows the Firewall client's activity. This log details activity across the ISA Server—including client username, agent, and IP address; application (or service) identification, destination name, IP address, and port; and transactional information such as processing time, cache information, and rule information.

The Web Proxy log details Web-based client activity and includes Web-related fields such as browser type, HTTP commands (e.g., GET), filenames for each object retrieved (e.g., an image, a Web page), content type, and the rule that allowed access.

Comparing the Web Proxy log with the Firewall Service log will help you understand Web Proxy service and Firewall service activity. Figure 2, page 9, shows the information each log contains for the same set of transactions. Notice that where the Web Proxy log lists anonymous connections that the Web Proxy service allowed, the Firewall log captured the username. To include Web Proxy users in the log, you must use the ISA Management snap-in to enable Web Proxy authentication. To do so, right-click the array name and select Properties. Click the Outgoing Web Requests tab and select the Ask unauthenticated users for identification check box. If ISA Server can't obtain authentication automatically, the server will prompt the user for credentials. The Firewall client logs user authentication information automatically. However, because you can install the Firewall client software only on Win32 client computers, only Win32 platforms support it.

The Packet Filter log differs from the Firewall Service log in that the former shows traffic at the packet rather than the application level. The Packet Filter log shows date and time, source and destination addresses, the packet's port, whether the packet was blocked or allowed, TCP flags, and, optionally, the packet's actual payload (main content). The payload is represented in hexadecimal format, so know your IP packet architecture and hex translation. This log is useful for troubleshooting why packets are dropped when they hit your firewall. By default, ISA Server doesn't log packets that are allowed across the firewall. To log the allowed packets, open the ISA Management snap-in, expand the Access Policy node, right-click IP Packet Filters, then select Properties. In the IP Packet Filters Properties dialog box, click the Packet Filters tab and select the Log packets from 'Allow' filters check box. ISA Server supports logging to either a text file or an ODBC-compliant database.

Logging to a Text File
As I mentioned previously, ISA Server file logging supports both W3C extended file format and a proprietary format. You can specify whether to create a new file daily, weekly, monthly, or yearly. Your log files can grow extremely large depending on the frequency of your logging as well as which fields you select to log. ISA Server supports log-file compression, and you can specify the maximum number of log files before replacing old logs that fall outside the setting with new logs. (In this case, the oldest log file is deleted when the new log file is created.) To enable compression, select the Compress log files check box in the Options dialog box of the log file's Properties dialog box. Enabling log-file compression in ISA Server simply turns on native NTFS file compression for the individual log files (the log files must be on an NTFS partition), and you don't need to indicate this choice separately in NTFS; it's made for you automatically. Log-file compression is enabled by default, regardless of the log-file folder's compression state. Table 1 shows the naming convention used for these logs and highlights differences between logging formats.

By default, the text log files reside in the %programfiles%microsoft isa serverisalogs folder. However, you can specify an alternate folder location. If you specify a relative path, each server will log to its own ISA Server installation directory or an alternate location, such as %systemdrive%. If you specify an absolute path, the logs will be stored at that location.

Logging to an ODBC Database
Writing ISA Server logs to an ODBC database is a handy method for accessing the data from custom reporting engines built with Active Server Pages (ASP) applications, ADO, or similar technologies. Microsoft includes three data files—fwsrv.sql, pf.sql, and w3 proxy.sql—to create the correct tables and fields. These files reside in the installation directory (by default, %programfiles%microsoft isa server) or in the ISA directory on the ISA Server installation CD-ROM. Note that you must create a database to hold the tables before you run these scripts.

Follow these steps to enable ODBC logging with ISA Server (the steps refer to SQL Server 2000). Because the steps are quite basic, I recommend making adjustments as necessary to suit your company's SQL architecture or database security requirements. You might have to make minor changes to support other databases such as Access.

I found configuring ODBC logging a bit tricky and not as intuitive as the Help files indicate it will be. If you have problems, such as one of the services failing to start, consider using these techniques:

After you've successfully logged to the database, you can configure a database client such as Access for rapid front-end access to the data. You'll find it handy to link an Access table to the SQL data source and execute short queries to retrieve data. For example, to retrieve the previous 30 minutes of activity, enter this query:

SELECT *  FROM  dbo_FirewallLogWHERE  (((Day([logDate]))=Day(Now())) AND ((Hour([logTime]))>Hour(Now())-1) AND ((Minute([logTime])) >Minute(Now())-30) AND ((Month([logDate]))=Month(Now())))ORDER BY dbo_FirewallLog.logDate DESC,         dbo_FirewallLog.logTime DESC;

When you choose whether to log through ODBC or to a text file, be sure to consider the loading and performance impact on your ISA Server.

ICSA Endorsement: Running with the Big Dogs Takes Its Toll
Independent security laboratory ICSA Labs endorsed ISA Server in February 2001 as a certified firewall under criteria 3.01. Many people consider ICSA Labs to be the main and most objective body for determining the standards for security-related products. Although this certification will likely draw more attention to ISA Server as a true enterprise firewall, ICSA Labs identified logging as an area that needed change. Microsoft provided two hotfixes and a script that alter the default behavior of ISA Server to satisfy the ICSA criteria. You can find these files and script on the ICSA Web site (http://www.icsalabs.com/html/communities/firewalls/certification/ vendors/microsoft/index.shtml). Usually, ISA Server applies packet filters only to the external interface.

These changes reconfigure ISA Server Packet Filtering to block and log traffic on the internal interface, dramatically affecting how ISA Server fundamentally operates. Operating ISA Server in this configuration results in substantial drawbacks—in my opinion, almost crippling the product. The configuration disables or severely limits ISA Server functionality, including firewall clients, array functionality, Web proxy listeners, and authentication to domain controllers (DCs). With this patch, you must use SecureNAT, and your ISA server must be a standalone server that isn't a part of a domain.

Reporting for Duty
Through ISA Server's fairly intuitive interface, you can generate reports on a flexible schedule. ISA Server logs generate the reports, which are functional and clearly organized although not as comprehensive as those that third-party providers have developed.

Microsoft's inclusion of autogenerated HTML-based reports is a useful addition to the ISA Server firewall package—especially considering that competitive products cost several thousands of dollars per firewall. The report function uses the ISA Server text log files. (Note that if you change to ODBC logging, ISA Server will continue to generate your scheduled reports, but those reports will be blank or incorrect.)

ISA Server reports can display information for five different time segments—daily, weekly, monthly, yearly, or for a custom period. You can also schedule recurring reports. One drawback is that you can't create a report for the current day and view the report immediately. (If you create a report for today, you must wait until the following day to view it.) However, you can create a report for any previous day's data and view it immediately.

Microsoft categorizes the reports into Summary, Web Usage, Application Usage, Traffic & Utilization, and Security. Figure 4 shows a Web browser -accessed sample report. Because the reports aren't template driven, adding to or changing the appearance or content isn't simple. The reports are fairly basic and offer few extended functions or parsing features such as DNS lookup or protocol name and number translation. If you need more flexible reports, consider a third-party reporting tool such as WebTrends' Firewall Suite, which now includes support for ISA Server reporting on general firewall and outgoing Web access. Such third-party reporting software products highlight aggregating log data and providing more customization options for report presentation.

Need It Realtime?
ISA Server provides realtime reporting for active sessions, current alerts, and running services. These options reside under the Monitoring node of the ISA Management snap-in. ISA Server also installs additional performance counters, as Figure 5 shows. ISA Server includes five performance objects to monitor Bandwidth Control, Cache, Firewall Service, Packet Filter, and Web Proxy Service.

These objects contain the ISA Server performance counters, which offer a gold mine of realtime information about ISA Server's health. Use these built-in tools to monitor ISA Server in realtime or to monitor ISA Server over a period of time to create a baseline, then watch for any performance degradation that load changes or perhaps a maintenance problem might cause.

Also, use the performance counters to determine the remaining capacity of an array member and to help pinpoint input and output bottlenecks. Looking at disk or CPU performance will give an indication of whether you should add more disk space, more processors, or another server to address a bottleneck. For example, if your logs and cache reside on the same disk as your ISA Server installation, use Performance Monitor to identify whether disk throughput affects end users.

Flexible and comprehensive monitoring differentiates great products. ISA Server provides a good suite of monitoring tools and functionality. And with this Microsoft .NET Framework product, Microsoft has left the door open for others to develop what isn't included and improve what is.