Penetration testing is a critical cybersecurity and compliance tool, but it’s one that is greatly misunderstood. With so many different implementations and uses of pen tests, the term can be more confusing than illuminating.
The differing opinions begin with the best ways to leverage pen testing. Should it be used broadly to identify a wide range of security-posture issues and improperly retained sensitive data (including problems that neither security nor IT teams had even considered)? Or should it be used primarily to detect particular weaknesses before an attacker does? Or is it best used far more narrowly—to look for the existence of previously known types of security holes (possibly even via solely automated testing)?
The Larger Cybersecurity Strategy
The biggest strategic issue speaks to cybersecurity strategies well beyond the act itself of pen testing: What should an enterprise do with the results of a pen test? Some argue that they should be used solely to identify problems that are then quickly negated. Others argue that pen-test results are best used as a road map to the root causes of the holes that were discovered (in other words, that the results should give hints about how to fix much more fundamental problems).
It’s critical to explore using pen testing wherever any company data is touched. This includes IoT/IIoT, cloud environments, hosted apps, partner environments (contracts permitting), and even modern fleet vehicles (which are quietly retaining a tremendous amount of data).
“CISOs should use pen testing to match what they perceive as their risk assessment,” said Matt Miller, principal security engineer at Triaxiom Security. “What are the top threats they are fighting? What are the risks that they are worried about the most?”
Therefore, when getting started with pen testing, it’s important to define a scope—even if only a preliminary/initial scope.
“It’s an issue of prescriptive versus non-prescriptive,” said Taylor Smith, a penetration tester at Pivot Point Security. “We’ve worked with businesses that are trying to tackle compliance but don’t know where to start.”
Still, the prospect of pen testing can be overwhelming to already heavily burdened CISOs.
“Given how stressed and overworked security departments are today, CISOs can barely keep up with reactive tasks, putting out fires that seem to crop up hourly. To ask them to be proactive and to actively seek out problems that haven’t blown up yet, that can seem very unattractive,” said Tom Brennan, executive director for the Americas region of the Council of Registered Ethical Security Testers (CREST). “But it’s precisely what they need to do. They need to get ahead of attacks and to fix holes and backdoors before the bad guys discover them.”
Smith argues that pen testing is often given insufficient consideration by many CISOs—despite its decades-long history—because it represents a specialty typically outside of a CISO’s reach.
“Pen testing is valuable because it serves as both an adversary simulation [and] a pulse of the current state of assets—like networks, web apps, and beyond,” Smith said. “I think CISOs often misunderstand how this data can be useful and may miss the forest for the trees by focusing on specific results—rather than looking long term, seeking what caused those results.”
Whose Pen Test Is It Anyway?
When strategizing a pen-test approach, one of the considerations is identifying who will perform the tests: internal or external talent. Internal talent is often far more familiar with existing systems, but that is both a plus and a minus. That internal expertise can sometimes blind a tester to things in the same way that writers can often miss their own typos because their brain knows what they were trying to say.
An internal tester may also have difficulty navigating corporate political issues that a third-party tester would have no problem sidestepping or ignoring. Let’s say that an internal tester discovers a flaw due to sloppy coding. That code may have been written by a specific manager that the internal tester needs a favor from next week. A third-party pen tester is unlikely to have such a conflict of interest.
On the flip side, that internal tester will likely have an easier time tracking down a root cause than a third-party tester would because that internal tester knows the environment’s history.
“Internal talent and familiarity with systems can be excellent for digging deeply into potential issues and having a constant vigil on the state of security,” Smith said. “But the third-party element can help simulate realistic threats and identify shortfalls that internal resources may not be able to. Ideally, the inside and the outside working in tandem is ideal but not always realistic concerning budget or execution.”
Tony UcedaVélez, CEO of VerSprite and co-creator of the Process for Attack Simulation and Threat Analysis (PASTA) threat-modeling methodology, echoed the sentiment that combining internal and external talent is a good idea for pen testing.
“Both are definitely important, but using external resources can bring a lot of unbiased expertise to be shared with your internal groups,” UcedaVélez said. “We are seeing more and more purple-teaming efforts being demanded because of this reason.”
The Role of Automation
Combining automation and pen testing is another dicey issue. Unlike the internal versus third-party issue, automation is almost universally embraced as a powerful—and cost-effective—supplement to manual pen testing. Problems arise, however, when CISOs and CIOs try to conduct pen testing solely through automation.
Smith’s position is that automation in pen testing must be encouraged but sharply limited.
“Automated tools are a huge boon for penetration testers. I don’t know any professional who doesn’t leverage them. The time saved doing scans is incredible and has evolved even just over the last few years,” said Smith. “But there is definitely a line and I would draw it very aggressively. We are far, far away from the concept of a meaningful automated pen test.”
UcedaVélez agrees but sees automation as a fine start to the process.
“Automated tools are good for the initial stages of a pen test, recon, discovery, and vulnerability assessment. But manual testing efforts are best for testing specific abuse cases such as those affecting application logic or context-based business flows,” he said. “Manual also works best for exploitation and post-exploitation tasks that can show the real impact of the findings and how they substantiate the attack patterns analyzed.”
“The manual element is crucial to really peeling back the layers of discovery and exploitation in ways that scripts and scanners and automated exploit tasks simply can’t,” added Smith. “Being able to tailor to the situation, adapt, grow knowledge very quickly and on the fly—those are all things that I’ve yet to see an automated tool really perform well without the human element.”
At its simplest level, an automated pen test can only find what it has been programmed to find. A human pen tester can look around and explore what looks off. Automation will rarely—not never, but rarely—find a hole that no one anticipated.
“One of the psychological challenges for pen testing–especially for enterprise CISOs–also happens to be arguably its greatest strength,” said Brennan. “It is the fact that pen testing, done properly, is proactive, not reactive.”
