The 2018 independent web application security scanners benchmark results have been published. How did Netsparker fare when compared to the other web vulnerability scanners?
In short, Netsparker was:
- The only scanner that identified all the vulnerabilities
- One of the only two scanners that did reported zero false positives
None of the other scanners in the comparison performed as well as Netsparker. If you'd like to find out more information, including results, read this post which explains how the tests were conducted and displays the results of each individual test.
Table of Content
- What is the Web Application Security Scanner (DAST) Benchmark?
- The Benchmark Results – Global Results
- The Benchmark Results – Individual Tests Results
- Are Web Security Scanner Comparisons Useful & Realistic?
- Past Comparisons Between Automated Web Application Security Scanners
What is the Web Application Security Scanner (DAST) Benchmark?
It is a test that compares the features, coverage, vulnerability detection rate and accuracy of automated web application security scanners, also known as web vulnerability scanners or Dynamic Application Security Testing (DAST) solutions.
Individual tests were conducted by the independent information Security Researcher and Analyst, Shay Chen. Shay has been conducting benchmark tests and improving the platform since 2010. So far he has released six (2010, 2011, 2012, 2013/2014, 2015, 2017/2018). His work is considered the de facto comparisons results by the application security industry.
How Are Tests Performed?
Shay Chen and his team built The Web Application Vulnerability Scanner Evaluation Project (WAVSEP), a testbed that they scan to see how every scanner performs. The WAVSEP is an open source project and new tests are incorporated every year. You can download it from the WAVSEP GitHub repository.
This year Shay and his team went a step further. They have been installing and integrating DAST solutions in real-life enterprise SSDLC (Secure Software Development Lifecycle) processes to get a better understanding of how they can expand the WAVSEP testbed and test the scanners. The have implemented automated vulnerability scanners in financial, hi-tech and telecom organizations. As Shay himself explains:
Some of these experiences led us to develop test cases aimed to inspect issues in proclaimed features that we noticed didn't work as expected in actual implementations, and some to the creation of comparison categories that are apparently crucial for real-world implementations.
The Negative Impact of False Positives
Shay and his team also talked about the importance of accurate scan results in the report, after their first-hand experience with scanners in real-life environments. Quoting from the official benchmark results:
Weeding out a reasonable amount of false positives during a pentest is not ideal, but could be performed with relative ease. However, thousands upon thousands of false positives in enterprise SSDLC periodic scan scenarios can take their toll.
False positives occur in scan results to the detriment of the web application security industry. So much so, that large organizations, that have hundreds or even thousands of web applications, limit their efforts to a handful of mission-critical websites and ignore the rest. I was quite shocked to learn this, though it is unsurprising because many hacks and data leaks that happen every year.
False Positives Make Scaling Up Web Security Impossible
If a solution reports false positives, it is impossible – unless you have an army of people – to scale up your efforts and secure all your web applications. Even if you have the budget for such an undertaking, there is still the troublesome problem of human error.
This is why we developed Netsparker's proprietary Proof-Based Scanning, technology that automatically verifies detected vulnerabilities – proving they are real flaws, and not false positives. The benefits of such technology are plentiful, and since the scan results are accurate, you can easily scale up your efforts. In a real-life environment, with thousands of web applications, you can start the vulnerability triage process and fix them within a matter of hours.
Evaluation Criteria
In the 2017/2018 benchmark tests, Shay and his team included several previously uncovered aspects of scanners and new tests to check the detection capabilities of previously uncovered vulnerabilities. This included OS Command Injection, and repurposing XSS via RFI tests that can also be used for Server Side Request Forgery (SSRF) evaluation.
The Benchmark Results – Global Results
How Many Vulnerabilities Did the Scanners Detect?
This matrix lists what percentage of all vulnerabilities each web application security scanner identified. Missing data or scores are represented with 'N/A'.
| Netsparker | WebInspect | AppSpider | Acunetix | Burp Suite | AppScan | |
|---|---|---|---|---|---|---|
| OS Command Injection (New) | 100 | N/A | 99.11 | 78.57 | 93.3 | N/A |
| Remote File Inclusion/SSRF (New) | 100 | 100 | 82.67 | 64.22 | 74.67 | N/A |
| Path Traversal | 100 | 91.18 | 81.61 | 94.12 | 78.31 | 100 |
| SQL Injection | 100 | 98.46 | 95.39 | 100 | 97 | 100 |
| Reflective XSS | 100 | 100 | 100 | 100 | 97 | 100 |
| Unvalidated Redirect | 100 | 95.51 | 100 | 100 | 76.67 | 36.67 |
| Average % | 100.0 | 97.0 | 93.1 | 89.5 | 86.2 | 84.2 |
Clearly, Netsparker beats the competition in terms of vulnerability detection. It was the only scanner to identify all the security issues, followed by HP WebInspect at 97% and Rapid7 AppSpider at 93.1%.
Note: Missing data or scores were the result of lack of support (in some cases even a lack of response) from some vendors. Only the tests for which scanners had a result were used to calculate the global average.
How Many False Positives Were Reported?
This matrix lists what percentages of all false positives each web application security scanner identified.
| Netsparker | AppSpider | WebInspect | AppScan | Acunetix | Burp Suite | |
|---|---|---|---|---|---|---|
| OS Command Injection (NEW) | 0 | 0 | 0 | 0 | 0 | 0 |
| Remote File Inclusion / SSRF (NEW) | 0 | 0 | 0 | 0 | 0 | 16.67 |
| Path Traversal | 0 | 0 | 0 | 0 | 0 | 12.5 |
| SQL Injection | 0 | 0 | 0 | 0 | 0 | 0 |
| Reflective XSS | 0 | 0 | 0 | 0 | 0 | 0 |
| Unvalidated Redirect | 0 | 0 | 11 | 11 | 11 | 0 |
| Total % | 0.0 | 0.0 | 1.8 | 1.8 | 1.8 | 4.9 |
Netsparker and Rapid7 AppSpider were the only solutions that reported zero false positives, while Burp Suite was the one that reported the most false positives.
Graph with Global Detection & False Positives Rates
This graph is a visual representation of the global results, illustrating both the vulnerability detection and false positives rates side by side for each vendor.
The Benchmark Results – Individual Tests Results
OS Command Injection Detection
The OS Command Injection vulnerability tests is one of the new tests. Netsparker was the only scanner to detect all the vulnerability instances in the test.
Remote File Inclusion / SSRF
This was also one of the new tests included in the WAVSEP benchmarking tests. Netsparker and WebInspect were the only two scanners that detected all the vulnerabilities in this test. AppSpider followed with 82.67%, and then Burp Suite with 74.67%. Though Burp Suite also had 16.67% false positives.
Path Traversal
This time Netsparker and Appscan led the field, both of which detecting all the Path Traversal vulnerabilities.Acunetix WVS and HP WebInspect came third and fourth, followed by AppSpider. Burp Suite was the scanner that detected the least at 78.31% and also reported 12.5% false positives.
SQL Injection
This is one of the classic tests; the SQL injection vulnerability. In this test Netsparker, Acunetix WVS and Appscan detected all the vulnerabilities. HP WebInspect followed with 98.46%. None of the scanners reported any false positives in this test.
Reflective Cross-site Scripting (XSS)
All scanners but Burp Suite detected all the cross-site scripting vulnerabilities.
Unvalidated Redirect
In the unvalidated redirect vulnerability tests three of the scanners, WebInspect, Acunetix and AppScan reported vulnerabilities. AppScan also performed very poorly with a detection rate of only 36.67%. On the other hand, Netsparker, AppSpider and Acunetix detected all the vulnerabilities.
Are Web Security Scanner Comparisons Useful & Realistic?
As a rule of thumb, nothing beats a live environment test. In fact, at Netsparker we always encourage potential customers to test our web security solution by scanning a staging copy of their web applications, as explained in How to Evaluate Web Application Security Scanners.
It's impossible to test all the scanners available on the market. So, these comparisons are incredibly useful because they highlight who the market leaders are – those scanners that can detect the most vulnerabilities and generate accurate results.
Which is the Best Web Application Security Scanner?
The best web vulnerability scanner is the one that detects the most vulnerabilities in your web applications, is easiest to use and can help you automate most of your work. Finding vulnerabilities in a web application is not just about the duration of the scan, but how long it takes to setup the scan (pre-scan) and verify the results (post scan). Therefore, when you evaluated solutions, you should ensure that automated vulnerability confirmation is part of the equation.
Read Shay Chen’s full report: Evaluation of Web Application Vulnerability Scanners in Modern Pentest/SSDLC Usage Scenarios.
Can Netsparker Identify Security Flaws in Your Web Applications and APIs?
The best way to find out is to download a demo and launch a vulnerability scan. Netsparker is very easy to use and most of the pre-scan configuration is automated. All you need to do is specify the URL and credentials (to scan password protected websites), and launch the scan.
Past Comparisons Between Automated Web Application Security Scanners
See the previous results for the comparisons between the 2015 web application security scanners and 2013-2014 web application security scanners.
The post 2018 Web Vulnerability Scanners Comparison – Netsparker Confirmed a Market Leader appeared first on Security Boulevard.
from 2018 Web Vulnerability Scanners Comparison – Netsparker Confirmed a Market Leader
No comments:
Post a Comment