|
A study based on the testing of thousands of security products over 20 years finds that most require several rounds of testing before achieving certification
By Thomas Claburn, InformationWeek, November 17, 2009, 1230 hrs
Seventy-eight percent of security products do not perform as intended when first tested and typically require at least two rounds of further testing to achieve certification, claims a report released recently.
The ‘ICSA Labs Product Assurance Report’ comes from ICSA Labs, a division of Verizon Business. The company offers vendor-neutral certification and testing of security products.
The report was produced in conjunction with the Verizon Business Data Breach Investigations research team and was based on the testing of thousands of security products over the past 20 years.
The report aims to make buyers aware that "all is not as it seems in the world of security products" and to make vendors aware of common pitfalls in the certification process.
George Japak, MD, ICSA Labs and a co-author of the report, said in a statement that vendors should view certification failures as opportunities to improve their products.
Only 4 percent of products pass on the first round. Nonetheless, most vendors address shortcomings in their products and resubmit them, which is why 82 percent of products submitted for certification eventually receive it.
That 82 percent figure represents an average that includes anti-virus, network firewall, Web app firewall, network IPS, IPSec VPN, SSL VPN, and Custom Testing products. In most of these categories, the percentage of products eventually receiving certification ranged from 80 percent and 100 percent. But one category, network IPS, represented an outlier: Only 29 percent of network IPS products ever attained certification.
The report says the category covers "a complex technology with difficult testing requirements" and notes that many vendors, unable to pass the rigorous tests, dropped out of the certification process.
The primary reason for these certification failures is that the products tested don't do what they're supposed to do.
For an anti-virus product, that means failing to block viruses and for an IPS (intrusion prevention system) that means failing to block malicious network traffic.
Failure to properly log data represented the second most common reason for certification failure.
Security problems represent the third most common reason for certification failure. These are seen in 44 percent of security products.
"One of the more ironic examples we've ever come across was a Web application firewall that turned up numerous vulnerabilities within its Web administration interface," the report states. "Cross-site scripting, SQL injection, and buffer overflow vulnerabilities and unencrypted admin interfaces are some of the common security issues identified within the Custom Testing engagements, Web Application Firewalls, and Network Firewalls programs."
|