NSS Labs to publish firewall test report

Security testing company NSS Labs announced today that it would be publishing a test report on firewalls next week. Company CEO Rick Moy told journalists at NetEvents in Barcelona that the new report on the testing of traditional firewalls, some of which have been around for 25 years, would help companies decide how to select the most cost-effective security.

"We're asking if traditional firewalls are still doing what they're expected to do," said Moy. "Many of them don't even though they're effectively commodities, which meant we could break into the network that a couple of them were supposed to be protecting. Everybody had a problem."

Moy said that vendors tested included Cisco, CheckPoint, Juniper, Fortinet, and Palo Alto.

Moy also highlighted a report, published in January 2011, on a group test of network intrusion prevention products. Moy said that the products under test had improved since the last test in 2009. he said that on average 62 percent of exploits were blocked compared to 45 percent in the earlier test. Products tested came from Cisco, Juniper, IBM, McAfee, and Stonesoft. The best results was achieved by CheckPoint, which blocked 97 percent of exploits. "However, you can always do better if you configure the system properly," said Moy.

Moy said that the test measured the cost-effectiveness of the products as well as their ability to block exploits. "We invited vendors in to tune their products and we measured the time they took. We gave them two weeks and a man-month per product," he said. "We also measured the total cost of ownership including metrics such as maintenance fees and purchase price. Using the labour time and cost per month you can calculate TCO and allow buyers to make value judgements."

Moy sounded a note of warning about some next-generation firewalls that offer a wide range of features. "Unified threat management products failed previously because they did everything, but nothing well," said Moy. "If you buy a new firewall it had better be at least as good as the last one you bought. Firewall capability and intrusion prevention are the absolute minimum but you but can't have everything."

"Our testing roadmap is twice as big as last year," said Moy.