Penetration Tests vs. Vulnerability Scans: The Difference
| certifications | security - Matt McClure

Penetration Tests vs. Vulnerability Scans: The Difference

Everyone involved in IT wants (or at least should want) to run as tight of ship as possible security-wise, from the CTO all the way down to the junior developers and help desk engineers.  But we all know security problems will come up from time to time, whether it’s via unpatched software, misconfigured servers, or hacky code jobs. Because sometimes you just don’t know what you don’t know, it’s always a good idea to get an outside party to poke and prod your infrastructure, looking for any weaknesses.

Working with any security vendor always starts with one key question: what kind of engagement do you want to run, a penetration test or a vulnerability scan? They sound pretty similar right?  Until you get a quote and see that one might cost 10 or 20 times more!  So, which do you need, or do you need both done? How often?  What’s the difference?  Does your company have certain certifications or regulations that will require either?

With so many questions to explore, it’s time for the ultimate infosec showdown: Pentest vs Vuln Scan.

In this Corner: Penetration Test

Penetration tests are very involved engagements conducted over the course of days or even weeks, the length is greatly controlled by the scope. Simply put, the more IPs or web apps you ask to be tested, the longer the pentesters will require to conduct a thorough test.  They will use a wide variety of tools to conduct the tests depending on the circumstances and services they find on your network.

Speaking of the pentesters, these will be infosec pros with a great breadth and depth of knowledge and experience across network services and web app architecture, frameworks, and development languages. They will generally be credentialed with difficult and expensive certifications from the likes of Offensive Security and SANS.

In other words, a good pentest firm hires top tier infosec talent. These pros will be devoted full time to completing your engagement in the time scoped out. As a professional service, their cost to you will reflect this, so expect pentests to be very expensive.

Speaking of scope, every pentest will start with a scoping out phase. This is where you meet with the team conducting the test and work out every possible detail. What hosts or apps are you wanting to be tested? Anything else is explicitly off limits to avoid liability. Is there a dev or testing environment they should test against? Does it 100% accurately mimic what is in production? Or are you trying to test a new release before it goes live?

What dates will the testing occur? What times of day are acceptable? You don’t want an attack accidently taking critical production services offline during the work day, but be prepared to pay extra for off hour testing. Is there a WAF or IDS in place to protect against attacks?  Should the pentesters try to run their testing with these security precautions in place (a more real world scenario but testing is slower) or should their traffic be whitelisted from the start (allowing for a more thorough test in the same amount of time)? Will the test involve attempting to access an application with no login credentials? Or with user-level credentials, attempting to elevate to admin-level?

This seems like a lot of details to prepare for, but it’s really just the basics. A test could involve social engineering, trying to manipulate employees into installing backdoor software or divulging passwords. Another test could involve physical testing, where a team attempts access to one or more office buildings, getting direct access to the network without having to compromise an internet-exposed service. This level of testing can be incredibly expensive and is less common, but is a fair representation of what an actual bad guy might do to pwn you.

Once scoping is done the test is scheduled and conducted. The final deliverable to you, the customer, is the report. This will outline all findings, generally starting with an executive summary (or management who don’t care about technical details, they just need a one-pager with highlights and some pretty graphs) then getting into the nitty-gritty about any findings.  They should provide details about any and all issues found, how they were or potentially could be exploited to gain network access, the potential severity of exploitation, and recommended remediations.

In the Other Corner: Vulnerability Scan

Compared to a pentest, a vulnerability scan is a cake walk.  Generally, you use one tool, feed it the targets you want scanned, and hit start. The result is after some hours will be a list of discovered open ports, application vulnerabilities, and common misconfigurations found on the targets.

There are several vuln scan tools, but two are regarded as supreme: OpenVAS and Nessus.  OpenVAS is an open source tool that can be installed on any Linux host or downloaded as a virtual appliance. Though open source software, it has been maintained for over a decade with updates, so you can trust that it will find the latest vulnerabilities that might show up on your unpatched systems.

The other option, Nessus, comes in a few flavors. One version, Nessus Essentials, is free but is limited to scanning 16 IPs at once. This version replaces Nessus Home in 2019, which was a significant change as Home was limited to scanning within your private LAN for non-commercial use only. Essentials can be used commercially and while there are a few other minor differences, it’s still a powerful product with a nice price tag.

While infosec firms will perform vulnerability scans, there’s no reason for a company to not do their own scanning with either product. They’ll both run easily on a $20/month AWS or DigitalOcean instance, but the oversight into your exposed services will be invaluable (be careful with cloud providers’ policies however). Schedule monthly or even weekly scans to discover problems before the bad guys do. Scan any newly deployed applications or devices as soon as they come online. Find issues with insecure or soon-to-expire certificates on sites.

No IT department has an excuse to not perform regular vulnerability scans of their external-facing systems.

The Final Verdict: It Depends

So, why bother paying for a pentest? Vulnerability scans run by free software can alert on all these issues, why would you pay a firm in the thousands or tens of thousands to run a pentest?  This is where the purpose behind the two types of tests diverge. Pentests bring the human factor to play. Vulnerability scans go for the low hanging fruit. Both are useful.

Imagine a bad actor scanning huge blocks of IPs looking for easily exploited servers. Maybe an out of date IIS server or an FTP server accidently setup with anonymous access enabled.  These are the kind of issues that can be easy to miss in a busy sysadmin’s day to day, until they show up on a monthly vuln scan. After that they’re easy fixes, but the oversight of the network just wasn’t there before.

Now imagine a targeted attack against the web app you run for your customers. A hacker might spend days carefully crafting SQL injections or an XML External Entities attack. These things are much more nuanced and require time to fully explore the app before they could be found and exploited. A vuln scanner probably won’t find them, but a pentester probably will.  They won’t be easy fixes, but cleaning up after a data breach won’t be either.  This is the value a pentest brings.

Another factor to explore: you might be required to perform one or the other.  For example, if you need a SOC 2 certification, the industry standard for tech service organizations, the auditors are probably going to want to see a clean pentest report.

Another example is if you are a business accepting credit card data online then you must be PCI compliant. What do you do with the card data (store it, transport it to a card processor, etc.) effects on the level of compliance required, but regular third-party vulnerability scans are generally required to renew your compliance. Otherwise you’ll have some angry banks refusing to process payments or a boatload of regulatory fines if your systems contribute to a card data breach.

Final Thoughts: Pen Testing vs Vuln Scanning

Both pentests and vuln scans are important and a good infosec firm or consultant will help you navigate the waters of which you need for your business. There’s no reason every IT department can’t roll their own vuln scan system to eliminate the low hanging fruit.  Or you might need to go deeper with an annual pentest. It’s all money well spent to keep your systems secure, the hackers at bay, and your company off of the evening news reporting on yet another preventable data breach.



Ultimate Security Cert Guide

A 62-page guide to every Palo Alto, Offensive Security, (ISC)2, Check Point, CompTIA, and Cisco certification, and how they fit into your career.

I have read and understood the privacy policy, and am able to consent to it.