Despite slow progress, NetSecOpen — a group of network-security companies and hardware testing organizations — aims to have its testing and benchmark standards in place by later this year.
The group published the latest version of its network-security testing standard for next-generation firewall technology in May to gather feedback as the group moves toward a final version. The end result will be a consensus method for testing and benchmarking network-security appliances that allows comparisons of different vendors’ devices even if they are evaluated by different third parties, says Brian Monkman, executive director of NetSecOpen.
“What we’re working on accomplishing here is something that’s never been done — setting up standard test requirements that can be executed by multiple labs using different test tools and getting comparable results,” he says. “It’s something analogous to when the miles per gallon … had different approaches and … they tested things differently and so they forced the creation of a standard. That’s kind of what we’re doing here.”
Established in 2017, NetSecOpen aims to ease the tension between product makers and test labs, which have occasionally become rancorous. Members include large network-security firms — including Cisco Systems, Fortinet, Palo Alto Networks, and WatchGuard — as well as testing equipment makers, such as Spirent and Ixia, and evaluators such as the European Advanced Networking Test Center (EANTC) and the University of New Hampshire InterOperability Laboratory (UNH-IOL).
While the latest standards document is published as part of the Internet Engineering Task Force (IETF) process, the eventual guidelines will not be an Internet standard to which equipment makers must adhere, but a common approach to testing methodology and configurations that improve the reproducibility and transparency of resulting tests.
The current testing standards for firewalls published by the IETF (RFC3511) are 20 years old, and the technology has changed dramatically, NetSecOpen stated in its draft (RFC9411).
“Security function implementations have evolved and diversified into intrusion detection and prevention, threat management, analysis of encrypted traffic, and more,” the draft stated. “In an industry of growing importance, well-defined and reproducible key performance indicators (KPIs) are increasingly needed to enable fair and reasonable comparisons of network security functions.”
Real-World Test Cases
The NetSecOpen tests aim to use real-world data to pit the latest network-security appliances against realistic network loads and security threats. The attack traffic test set, for example, brings together common vulnerabilities that have been used by attackers in the past decade.
The NetSecOpen draft recommends specific test architectures, traffic mixes between IPv4 and IPv6, and enabled security features. However, other aspects of testing include required elements, such as the capabilities of emulated browsers, attack traffic that targets a specific subset of known exploitable vulnerabilities, and tests of a variety of throughput performances, such as application traffic, HTTPS requests, and quick UDP Internet connections (QUIC) protocol requests.
Network-security firm Palo Alto Network, a founding member of NetSecOpen, actively collaborates with NetSecOpen to “create the tests and actively participating in testing our firewalls using those tests,” says Samaresh Nair, director of product line management at Palo Alto Networks.
“The testing process is … standardized with accredited test houses,” he says. “Customers can use it to evaluate various products with standardized results tested similarly.”
The vulnerabilities test sets are in the process of being updated, because the Cybersecurity and Infrastructure Security Agency (CISA) demonstrated that smaller, noncritical vulnerabilities can be strung together into effective attacks. The organizations had previously dismissed many of those vulnerabilities as a lesser threat, but attack chain data CISA collected show that attackers will adapt.
“There’s definitely a class of CVEs out there that we, in the past, would have ignored, and we need to pay attention to those simply because vulnerabilities are being strung together,” Monkman says. “That’s going to be really the biggest challenge that we have, because the CISA KEV vulnerability list might grow.”
Cloud Up Next
In addition to new mixes of vulnerabilities — such as focusing on sets of threats such as those that currently target the education and healthcare sectors — NetSecOpen is looking to include detection of command-and-control channels used by attackers, as well as ways of preventing infection and lateral movement.
Testing the security of cloud environments — such as distributed cloud firewalls and Web application firewalls — is also on the future blueprint, says Chris Brown, technical manager at UNH-IOL, which joined NetSecOpen in 2019.
“Cloud would not change NetSecOPEN’s mission for well-defined, open, and transparent standards, but rather expand the products currently tested,” Brown says. “In the foreseeable future, network perimeter defense will still be necessary despite the many benefits of cloud computing.”
