The NSS Group, which I've written about before, analyzes hardware-based security solutions and produces in-depth reports that cover the products' performance and capabilities. The group uses specialized equipment in a controlled environment for its analysis and has reviewed high-performance Intrusion Prevention Systems (IPSs), Intrusion Detection Systems (IDSs), and application firewalls. You can find previous articles about the group's reports at our Web site.
Recently, The NSS Group released two new reports: Unified Threat Management (UTM) and IPS Group Test (Edition 3). The UTM report is the group's first report on products in this category. It looks at unified security solutions that include firewall, VPN, IDS/IPS, antivirus, antispam, URL filtering, and content filtering components. Of the six vendors that agreed to take part in the tests, two had products that weren't ready in time and two others had products that failed the group's stringent testing. The end result was that only two products passed the overall UTM tests: Fortinet FortiGate-3600 and Internet Security Systems (ISS) Proventia M50.
IPS Group Test (Edition 3) is The NSS Group's third report on IPSs. The group reported that products in this classification have improved since last year in terms of performance, stating that "whereas last year we were seeing top speeds of 1-2Gbps, this year we are starting to see devices that can go well beyond that limit and which are looking over-engineered for Gigabit environments." Even so, four of the twelve products submitted by vendors for testing failed the overall tests, leaving eight products to receive an NSS Approved rating. Those products are Cisco IPS-4255 5.0(3), Cisco IPS-4240 5.0(3), Intoto IntruPro 3.0, Juniper Networks IDP 600F 3.1, NFR Sentivist Smart Sensor 100C, Radware DefensePro-3000 2.43, Symantec SNS 7160 18.104.22.168, and Westline Athena Aegis IPS 510L 2.1.
The NSS Group also said that because product performance has improved significantly, the group will begin testing multigigabit Ethernet IPSs. Ten companies are signed up for the tests, which begin this month. That report should be interesting to those of you who must deal with super-high-speed networks.
The NSS Group's reports reveal a lot about the performance characteristics of particular products (which of course is a huge aid in buying decisions) and about how to test such products. For example, the group uses specialized hardware and software from Spirent Communications to generate and measure high volumes of network traffic. The group also uses tools that might be common in your own environments, such as Cisco Systems Catalyst switches. And you might be interested to know that the group uses Tcpreplay (at the first URL below) and Tomahawk (at the second URL below), both of which are open-source tools that you can easily obtain.
Tcpreplay lets you replay previously captured traffic and modify packets. Tomahawk also lets you replay network traffic and generates large volumes of traffic for stress testing. You could use both tools to test the effectiveness of your particular IPS or IDS.
Overall, I think you'll find the new reports very interesting and valuable, particularly if you're evaluating new high-end security solutions. You can read the full reports online (at the URL below) and purchase copies in PDF format or as printed and bound reports.