Failure of classical vulnerability scanning, penetration testing, and Red Team Exercises

In this section, we will focus on the limitations of traditional/classical vulnerability scanning, penetration testing, and Red Teaming Exercises. Let's now discuss the actual meaning of these three methodologies in simple terms and look at their limitations:

• Vulnerability scanning (Vscan): This is a process of identifying vulnerabilities or security loopholes in a system or network. Limitations with Vscan are only potential vulnerabilities, which might include lots of false positives, and to the business owner, there is no clear vision on whether these are relevant risks or not.
• Penetration testing (Pentest): This is a process of safely exploiting vulnerabilities without much impact to the existing network or business. There are a fewer number of false positives since the testers will try and simulate the exploit. Limitations with Pentest are only the current known publicly available exploits and mostly these are project-focused tests. In Pentest, we often hear Yay! Got Root, but we never question What's next ?. This could be due to various reasons such as that the project limits you to report the high risk issues immediately to the client or that the client is interested only in one segment of the network and wants you to compromise.
• Red Team Exercises (RTE): This is a process of evaluating the effectiveness of an organization to defend cyber threats and improve its security; during RTE, we notice multiple ways of achieving project goals, such as the complete coverage of the activities with the defined project goal, including phishing and wireless, drop box, and physical penetration testing. Limitations with RTE are that they are time bound, with predefined scenarios, and they have an assumed versus real environment.

Often, all three different testing methodologies refer to the terms hack or compromise. We will hack your network and show you where its weaknesses are, but wait, does the client or business owner understand the terms hack or compromise? How do we measure hack or compromise? What are the criteria? When do we know that hack or compromise is complete. All these questions point to only one thing--what's the primary goal?

The primary goal of penetration testing/RTE is to determine the risk, differentiating the risk rating from the scanner and a business risk value of each asset along with the brand image of the organization. It's not about how much they have, rather it's about how much they are exposed. A threat found does not really constitute a risk and need not be demonstrated. For example, a Cross-Site Scripting (XSS) on a brochure website may not have significant impact on the business; however, a client might accept the risk to put in a mitigation plan using Web Application Firewall (WAF) to prevent the XSS attacks.