PCI Compliance and Pen Testing: Fundamental Keys to Success

PCI Compliance is no longer a checkbox exercise—it’s a critical component of protecting cardholder data in today’s evolving threat landscape. And with PCI DSS 4.0 introducing new requirements and complexities, many organizations are navigating fresh challenges when it comes to compliance and Penetration Testing.

In this expert-led webinar, Tevora’s cybersecurity leaders—Clayton Riness, Kevin Dick, and Mikayla Bartell—break down the biggest areas of confusion around PCI 4.0 and how Penetration Testing plays a crucial role in ensuring both compliance and security. Drawing from real-world experience, they’ll highlight practical insights and strategies to help organizations stay ahead of emerging requirements.

Key Takeaways:

• Common areas of confusion with PCI 4.0 requirements
• How PCI Compliance and Penetration Testing reinforce each other
• Ways to align Penetration Testing with PCI strategy for stronger outcomes
• Frequent issues Pen Tests uncover that impact compliance New and emerging tactics in Penetration Testing tied to PCI 4.0 updates

Whether you’re updating your current program or preparing for your first PCI 4.0 assessment, this session will give you actionable insights to strengthen your compliance efforts.

Thank you all for joining. This is the Tevora PCI compliance and penetration testing fundamentals, key, fundamental keys to success panel discussion. I’m Clayton Riness. I’m Principal Consultant here at Tevora, been here for almost 14 years, and really oversee a lot of our technical delivery that we do, including penetration testing, incident response, cloud security and the like, as well as PCI compliance, given that it’s very prescriptive. And I’ve also been a QSA for about 10 years or so, and it’s my pleasure to introduce Michaela Bartel, who’s our manager of payments. She’s been here for six years.

Hey everyone, Thanks all for joining. Really appreciate it. As Clayton said, I’m the manager over on the payments team. I’m also one of the Q essays at Tevora. Handle a lot of our PCI DSS assessments, whether it’s for service providers, merchants, whether it’s for a rock and AOC or an SAQ and AOC. So here to provide context and answer any questions about PCI on terms of, you know, penetration testing and any changes that have happened recently. Thanks.

We have Kevin Dick, who’s Senior Director of our threat practice, been here for 13 years.

Senior pen test here at Tevora, we do a lot of pen tests on my team, but obviously PCI requires pen testing, so good portion of those tests are associated with PCI compliance. Looking forward to discussing PCI and how it relates to pen testing and general questions on pen testing. Excited to be here.

We really thought it appropriate to bring these two groups together. Obviously, we’ve got the penetration testing group and then the audit slash compliance. I think as it relates to PCI, there’s definitely some room to demystify things. Michaela, you’ve got an excellent handle on the standard and how it works under 4.0 but, we hear a lot like clients like, I need a pen test for my PCI compliance. There’s obviously a lot to unpack there. Could you give us the 1000-foot view of what’s actually necessary currently under PCI?

Depending on the environment that’s in place, so there’s a couple different penetration tests that might be applicable for PCI. There is the internal pen test, the external pen test, and then there’s also segmentation penetration testing, which is optional. Depending on your scope, you might have that internal or external penetration test that’s required. If you choose to implement segmentation in order to limit your PCI scope, then you’d also need that segmentation penetration testing. That penetration testing is required as part of the test securities of systems and networks regularly, aka requirement 11 for PCI, DSS standards. What that generally involves is that for the external pen test, the pen testing would need to look at the look at the external perimeter of trusted networks. Look at critical systems that are connected to the CDE that have external facing capabilities. That’s what the external is going to focus on. For the internal, as you can tell from the name, it looks more at internal networks and systems. You have testing from both inside the CDE and into the CDE from trusted and untrusted systems. And then segmentation is going to be focused on validating any segmentation controls that are in place, making sure that they’re actually enforcing that segmentation so that it’s validated. And then you can get that reduced scope for PCI. All of these have a little bit of a different function to them, but for penetration testing overall, a lot of it will be similar in that, PCI requires certain methodology to be followed. This is a penetration test compared to just a vulnerability scan. Where a vulnerability scan is just going to identify vulnerabilities, whereas a penetration test is more of that manual, more advanced testing to see what can actually be exploited. Are there any vulnerabilities that we can combine together to result in a higher critical or what have you exploit your finding. In general, that is what is required for PCI, DSS from a high-level view.

Recently the standard did change to actually ask for penetration testing from within the CDE, whereas prior and we by CDE, we mean car, whole data environment, just so we’re clear, where cards are stored, processed or transmitted. But recently that was changed. You must test from within the CDE. It used to be able to have sort of a gooey center in your environment. That’s no longer the case. I know, Kevin, you’ve done some tests where this has been interesting. What are your thoughts on how much has that really affected how the pen test goes down based on those new standard changes?

It was an interesting change. When there are changes like that, a lot of times we find it’s because the council was seeing breaches as a result of those things not being looked at. I think a lot of these changes are motivated by real events. We were kind of expecting that once we started digging into testing from directly in the CD, we would find, like you said, that gooey center. It’s been about a year now, and actually, kind of to our surprise, it makes sense, but we’ve been seeing that actually, those environments haven’t been yielding that many new findings, which makes sense when you think about it. Your CD that’s going to be, in a way, your crown jewels, there’s a lot of focus on the CD. I think what we’ve seen is that maybe our customers in particular are just great, but we’ve seen that they’ve been doing a really good job at keeping that locked down. Even though we now are doing that extra internally, you’re already giving us some access to your environment, but now you are really giving us some internal, deep access to go right into the CD. We’re seeing that those environments have actually been pretty resilient. As far as findings, we haven’t seen much of a load additional as far as findings for remediation. Where it can throw a wrench in things is getting access to the CD can be somewhat difficult. Because if you segmented your environment, you don’t want to make it easy for attackers to get directly in the CD. As pen testers, we’re essentially authorized simulated attackers. That has probably been the bigger challenge. It’s logistically of, how do you get our guys into the CDE where we can test and start looking at things from that perspective? As far as the actual findings, as far as the actual things that we’re seeing, there hasn’t been too much of a gooey center. It’s been pretty hardened once we got a look at the CD, which makes sense, but it was a little bit of a surprise for me. Just because I’ve seen in the past, when the council has come with new requirements, it’s because they were seeing a trend of like this thing wasn’t getting looked at and was causing breaches. I think we’re at a place where, luckily, not too many findings, just a little bit more of a logistical hurdle than you’d have to deal with in the past, as far as getting us access.

You mentioned sort of segmentation and how that affects scope. That’s sort of the S word, as far as I’m concerned. When it comes to PCI, everyone throws out segmentation, but there is a pretty clear definition of what. Segmentation is, does it mean no access? Does it mean just well defined access? Michaela, what’s your take on that?

For segmentation under PCI, DSS, really the core component of it is just making sure that any networks that we want to be segmented from the CDE, the car data environment, so those out of scope networks, just making sure that those out of scope networks, if they were to be breached or have some kind of event occur, they can’t impact the security of the CDE that card data environment. It doesn’t mean that there aren’t any connections allowed between out-of-scope networks and in-scope networks. It just means that if an out-of-scope network were to be compromised, it doesn’t impact an in-scope network. That’s where you can look at network and logical controls that are in place to enforce that segmentation so there can be connections allowed. It just can’t be security impacting at the end of the day. How sensitive is that in scope issue in defining the scope and how well do you think clients actually do that on their own. I know we have a lot of discussions around what is and not in scope. How much of a factor do you think that is in deciding what to pen test?

It can be difficult sometimes, because this is one of those areas where, in order to be able to cover all the different environments, in scope technologies that might be used the PCI, SSC, the standard Security Council has written this in really kind of generic, high-level language. When we’re looking at; is it security impacted or not? Are these segmentation controls working or not? It really comes down to having that penetration testing. If that’s something where, we already have a client that is doing their every six or 12 months doing that segmentation testing, they haven’t had a lot of changes. They have a really strong change management process. We see with a lot of those clients that they continue to have an easy pass, or there are minor fixes needed when they’re doing their segmentation penetration test, and otherwise it’s just coming out as a past result with other clients where there’s been significant changes, or maybe there’s not as strong of a change management process that’s involved. We see that when they get to their next penetration test to test that segmentation, there can be some surprises sometimes. I would say it depends on how many changes are happening in the environment, and then overall, how much of a strong control we have and when those changes are made, what the security impact might be, because sometimes those aren’t found until that penetration testing, when we’re actually getting to the nitty gritty details, it comes to segmentation. Kevin, what are how you actually test that it’s important to know that we’re not talking about exploitation. Is that correct when you’re doing just a straight segmentation test?

For the baseline segmentation test you do biannually, if you’re a service provider, annually, if you’re not a service provider, it’s really kind of a sanity check network port scan from a sampling. Usually just one is sufficient. Sometimes it can be a couple, depending on the nature of your network structure, to make sure that the out-of-scope networks cannot talk to the CD. You’re basically making sure if I’m, a standard employee, I’m not going to have access to where all the cardholder data is stored. Like Michaela mentioned, the purpose of that is just to make sure that, you cannot access, you can say that your CD is separate for the purposes of PCI scoping from the rest of your network. Now, that’s starting to change, because obviously segmentation there. There are elements of logical segmentation beyond just network segmentation, and those are some things we explored during the internal network pen test and some of the application testing, but the PCI is actually specifically called out. Now you do need to you need to do logical segmentation tests in addition to the network segmentation tests, which can affect folks that are specifically that requirements for if you’re hosting multi-tenant web applications. Now web application pen test that we do that’s going to cover that. In fact, all the segmentation testing is essentially a subset of the things we would do on a penetration test anyway. You know, for the network side, the port scanning, for the website, the application, logic testing, authorization, authentication testing that goes into that logic testing. It is worth noting there is an increase in kind of ramp up in requirements there. I think it is actually pretty heavy of an increase if you are falling into that multi-tenant bucket. You may find yourself now with more scrutiny on some of the segmentation than there has been in the past. I think that’s a general theme for the council. In general, is that they’re wanting to be careful of folks saying, we have this very clean PCI environment, and meanwhile, everything else is on fire. Then It’s really easy to make sure that that is segmented for from the rest of it. And part of what we see on the pen test, too, outside of the segmentation check, is, you do see that broader exploitation when we are doing the pure pen test, and that that is kind of a good. A kind of synergistic relationship with the pen test where, we’re able to see maybe on this thing. On the out-of-scope networks, where we’re starting from, we’re finding some ways we can pivot in. You may have good network segmentation in place, but maybe there’s steps one through three where we can get around that segmentation. On the pure segmentation test, yes, no exploitation. But on the pen test, sometimes that can kind of take that segmentation check a little bit further.

I was going to say, just to add on that too, just to clarify, it’s in the language that we’re using here. With PCI, we kind of have these three different groups. We have the CDE, that card data environment, so that’s going to be everything that actually directly interacts with credit card information or is on the same network as something else that directly interacts with credit card information, whether it’s, you know, processing, transmitting or storing. That would be your CDE, that’s kind of your core of your PCI scope. Then you also have your in scope, which is a broader group that includes the CDE, but also includes any networks or systems that connect into the CDE. Maybe you have some admin workstations that have remote access for some of your CDE servers. That would be something that’s included in your in scope, that’s part of that broader group, as well as any kind of security impacting systems and networks as well. If you have some authentication servers that you use for your identity access management tools for the CDE, those would be considered security impacted. They’ll be part of your in scope, broader group there. And then, of course, your last one being the simplest, your out-of-scope network. So those are the things that are segmented from the CDE completely and might have some connections into the in scope that broader group, not the CDE, but otherwise, shouldn’t be able to impact the security of the CDE at all. So just wanted to add in some clarifications there too. Just clarify the language.

That’s definitely a gray area. Folks are very quick to say that’s not in scope. That’s not in scope, when they really mean it’s not part of the CDE, but it is in scope for their PCI assessment, because it’s critical to the controls that they’re upholding. That’s definitely a key part, along those lines too. Kevin, you mentioned sort of the complexity on the web apps, and web apps being an important tag vector. There was recently some guidance that was published around how to use iframes and how you redirect into a payment channel. I know we have a lot of clients that use web apps, and they’re very much want to de scope to use their entire web application stack, because I’m using an iframe or I’m using a downstream API for my service provider to be able to transact. How much of an impact do you think that guidance has changed what we’re looking at from a PCI scoping standpoint,

I’m not sure if it’s changed the scoping too much as far as how and when we can do scope, but definitely for the ones that are in scope, it does, does ramp up the testing that you need to do, since you have to do some of that segmentation, logical testing by annually. I think it made to your point. Clayton put more of a focus on, can we get these things de scoped by using things like iframes? And whether that’s discoverable or not, sometimes you know that obviously QA and somebody’s going to take a close look at that and make sure that that is something that makes sense. Certainly those are strategies we’ve seen employed to not open up and blow up the scope of the application beyond just the payment processing portions.

Michaela, has that come up for you at all? I know that was it definitely is causing some heartburn for some folks and not being able to desculpt the web applications due to this new guidance.

It has been a change for some clients here, where a lot of times with the penetration testing they’re coming into it with more of a focus on the networking side, rather than from the application and logical side. It’s definitely important to point out, that’s going to be included still, as well, as Kevin mentioned before, with all our multi-tenant service provider customers, right? Those clients that are considered a multi-tenant service provider, they also have to make sure that they’re doing penetration testing to show that there is segmentation between their different customer environments that they have in place. If they’ve got an application that’s used by multiple customers, we also need to check that one customer’s access into the application doesn’t grant them access into other customers data that might also be used with that application. There’s some involvement there too, with the multi-tenant service provider changes that have come with 4.0, I would say overall, it really just depends on how they classify under PCI. There is more of that emphasis on that logical access now for PCI and 4.0 when it comes to cloud. And I think that’s an important distinction here, too, when we’re talking about what to test and how cloud infrastructure works, because there’s a lot of serverless environments, things are using lambda functions in AWS, and there’s a multi tenancy component there, and they’re using the key management system to encrypt those records. And it gets it gets interesting from a testing standpoint, because what is the Tax Service actually look like. Kevin, do you have thoughts on how folks should be thinking about that if they’re using a lot of cloud infrastructure.

I think you touched on a problem in general, where, if you’re looking at the scoping guidance for PCI, if not even, how do you scope the pen desk, but how do you scope the rock? How do you scope the whole engagement? It seems very geared towards on prem. When you get into micro segmentation, when you get into cloud, first infrastructure as a service, or serverless, where, like you said, you’re doing lambda API gateway. For example, in the AWS, there’s a lot of flexibility in how you determine that scope and kind of morph it into the PC eyes, kind of buckets there. For pen testing in general, I usually recommend, you just cast a wide net make sure you’re kind of covering those things, because it’s easier to go back and say, this is not really in scope or not fixed for PCI, but at least we know about it, versus you get into a situation where it wasn’t tested, and now you need to have a more thorough test as a follow up. Especially on things, if you’re looking at cloud infrastructure, and there’s a lot of external facing systems like lambda functions tied to API gateway. It doesn’t hurt to just throw those all in there just see what an attacker would find. And then if it ends up that, these things aren’t really related to the PCI scope, you can always, it’s good probably to know about the findings there anyway, just for general security. But then, those can be marked as such, so it doesn’t impact the timeliness of the rock coming out and with AWS as well. There are other challenges, like ephemeral IPs, for the internal network, you can give us, an IP range or a list of IPs. If these are easy to instances, or if they’re an ELB, maybe those IPs are going to change. You do want to try and find more static identifiers, like host names that are going to be long lived, or going to something like dumping the AWS kind of records of what’s currently in EC two right now, just before the testing starts. You have a kind of refresh list to look at. Of course, on the pen desk, we do rely on some Osint as well, right where we’re looking at, what’s out there on the public Internet. If an attacker, we’re just targeting your company, because that’s can also inform a lot of what is actually can happen the real world.

Sorry, the acronyms. It gets kind of crazy with the acronym. So, yeah, thanks. Thanks for that reminder. So Osint is open-source intelligence gathering, and it’s just kind of a fancy word for Googling, in a way, but it does go beyond just googling. You can imagine, if you’re an attacker, because ultimately, what a pen test is a little bit of an attack simulation. If you’re a real attacker, you’re not going to have a scope doc. You’re not going to have a defined, this is our rock, this is our PCI. You’re just going to have the attackers wanting to get into your company. They want to get access to the card data. How are they going to approach that? There’s a lot of information about your systems and services out there on the public Internet, about your perimeter, ranging from, Google, Bing, standard search engines, indexing websites, indexing pages, but also purpose-built things like Shodan, since SEO more specific to AWS websites like gray hot warfare, that index s3 buckets. Those can be very useful if you’re trying to kind of look at a target and find what you can get. And one step more for cloud I know I’ve kind of been rolling on this tangent for a little while here, but an option too, to go deeper on Cloud testing, not always necessary for PCI, but we do this a lot of times for bespoke cloud infrastructure. Pen test is, do the testing with the more open box approach, where you actually give credentials, audit roles to your AWS is your GCP environments, and AWS is Amazon’s GCP would be Google’s kind of infrastructure, and maybe some Oracle as well. And with that, we as testers, we can take a more, audit approach, where we have that auditor role, we can dump all the resources that are in there to get a very thorough look of what is actually in this infrastructure on the cloud, which can let us get a little deeper. That goes back to we’re probably casting a wider net than we need to cast, but that can make sure go through, and that’s more going into just generally, looking for more security issues long times go beyond the requirements of PCI, but that can be a good way to cover those environments, especially if you have a lot of ephemeral IPs that are changing quite frequently, or if you’re not quite sure of the handle on your cloud scope.

I think you mentioned something that’s interesting. We like to push a little bit on the edges of the environment. I think it’s important as an assessor to not always assume where the edge of the CDE or your environment is. We always want to be probing that a little bit. That includes some of what we’re doing on the pen test. Then when it comes to scoping, we certainly do that when it comes to, the assessment work that Michaela and team are doing. You mentioned the cloud infrastructure and being able to go in with creds, because there’s really no internal footprint. What are your thoughts, Michaela, if you had something that was totally serverless, totally cloud based, how would you scope things from a pen test perspective?

In general, for the PCI scope that then impacts the penetration scope. So, you know, if we have any, it kind of goes back to the kind of high-level definition here. If we have, say, VPCs under some AWS accounts that are transmitting or processing. Saying or storing credit card information. They’re automatically considered part of the CDE there. As Kevin mentioned, there’s a couple of different techniques or strategies to go about how to handle when there’s not a traditional kind of on prem environment where we have those static IPs for certain servers that are always live. Overall, it’s still going to be in scope for PCI. If something is considered part of your CDE, it’s still going to be something that you need to cover in the penetration test. Same thing for if it’s in that broader in scope environment. Even if things might change, those things still need to be covered. It really comes down to what makes sense for the client in that environment, depending on the different technologies that you’re using. How long certain things might live in your environment, it ends up just being a conversation between the pen tester and then the QSA, if you’re using us for both at Tevora, and then for you. Then we can figure out a solution that makes the most sense for testing that environment and still covering everything that PCI needs from the penetration test. I know we consume a lot of other people’s penetration tests as part of a PCI work that we do. We’re not always the pen tester. It’s great when we are, because we know the standard that’s being upheld. But what are you really looking for? If you’re looking at someone else’s penetration test report for a particular environment? Is there a certain minimum standard that you’re looking for? What things would pass the sniff test for you as being a competent, good pen test?

PCI calls out some specific things that we look for. One thing is we see that the methodology is documented. We have to know who was doing the pen test, what the scope of the pen test was, different techniques and strategies that were used during that pen test, there’s some open-ended language in PCI for what specific kind of attacks need to be simulated or what kind of techniques are used. It really just goes down to making sure that we’re following any kind of current exploits or vulnerabilities that are out there for those kind of environments, making sure that we’re covering any kind of best practices as well, and just making sure that the penetration tester themselves are independent from the organization as well as are knowledgeable enough to do this pen test. If you get a pen test back from someone and it has no findings, I would say that’s less of an indication that you have a really awesome environment, and more of an indication that probably a lot of things just weren’t found that should have been found. You always want to see those findings back, even if they don’t end up being high or critical or maybe they’re outside of your PCI scope. The other thing that we look for as well in others penetration tests, that Tevora is just making sure that the scope is correct too. As Kevin mentioned, it’s actually great. We love it as QSAs from the PCI perspective, when the pen test maybe pushes a little bit past what we would think is the boundaries of the in-scope environment. Because, PCI, it can be complicated. There’s a lot of different ways that you have to go down to figure out what your scope is. Sometimes there can be, some surprises during a PCI assessment for, new clients, or when we’re setting up a new service or a new environment, there’s been a major change. There can be some surprises around what is actually in scope and what’s not. We do see it that there are times that we, the client has assumed that something is out of scope when it actually ends up being in scope. Having that kind of a little bit of additional coverage during the penetration test initially does help cover for those situations, in case it does get expanded from what was previously the understanding. I would say overall, those are the main things that we look for, especially making sure that it makes sense for what they tested right different environments are going to have different techniques that you want to use, different strategies, and just making sure that’s full and comprehensive. So overall, that’s what we would look for. Like I said, it kind of varies across but those are the high-level items that we’re looking for in each penetration test report that we’re getting.

Kevin, the guidance given is not for any destructive testing whatsoever, correct? It’s important to note, it’s, this is a logic based test. Can you expand on that?

I think for, pen testing in general, even outside of PCI, we don’t want to copy the cause of an incident. We’re really there to identify things that then you can fix to prevent an incident. And part of security isn’t just confidentiality and integrity, it’s going to be availability. We do want to tread carefully, and the council doesn’t ask us to do denial service, do load testing, certainly, things that we’re going to do that would be abnormal traffic. Certainly, doing a port scan is going to generate a lot of network traffic from a single IP that you would not usually see, but it’s something that we want to make sure it stayed within bounds. Oftentimes we’re doing things that are potentially dangerous. It’s to serve a purpose of finding an issue, but by default, we’re not doing those unless we check in and get approval. Probably the most aggressive thing, or riskiest thing that we would do during the course of a pen test would be password spraying. Nine, which is essentially trying to guess valid passwords. As you may know, if you guess too many incorrect passwords, you’re going to lock out accounts, and that can be disruptive or just a pain the butt for your help desk team. But the reason why we would still want to pursue that and try to do a safe rate where possible is it is a vector we see used in the wild quite frequently where attackers come in and they guess. You know, spring, summer 2025, estimation mark, you’d be surprised at how many that can potentially capture. It’s a blind spot we don’t want to ignore. But we’re not going in there for the purpose of seeing the resiliency of the infrastructure. We’re not going in like, can we delete this database? Can we overload this network? Though, certainly, just my course of doing the pen test, you’re going to be doing abnormal activity, but it’s one of those things. Try and keep it safe, keep it low, make sure we’re always improving, to see where we can optimize that and make sure we’re not causing any kind of disruption or any kind of, headache for the team. Because really, our goal is to find things that could cause headaches so you can fix them before they do.

What would you tell a client that maybe is asking a little bit about? Michaela, you want to expand on that?

I was just going to add a quick note too, just it’s sort of related here. Even for one of the biggest changes right for these multi-tenant service providers on that new classification under PCI, DSS, even the PCI guidance points out for that testing that needs to be done to be done to see that logical separation between customer environments. It specifically calls out that a temporary, a mockup environment can be used. We don’t need to actually get access into your customers different environments. We can use that temporary mockup environment that is set up similarly to test that logical separation as well.

That’s a great point. I think we do that quite a bit. I mean, we’re testing pre-production environments. Because it’s Code Complete, it’s configured the same way as a production environment, therefore it it’s applicable, is that, is that the understanding generally?

I mean, for the network testing, we’re usually looking at the real PCI environment. Just because of the nature of we do have to look at what that production network looks like. For a lot of the application testing, certainly testing a UAT and a non-prod, a dev environment is a great way to do that and setting up, not using a real customer, but let’s set up a Tevora b tenant that we can test in as a mock up that’s using that same code for the same logical isolation. It’s a great way to test, ultimately, especially on the application side, it’s a bit of a UAT exercise. Sometimes I do, maybe getting on a soapbox. He’ll feel like security bugs are unnecessarily very separate from other kind of bugs in an application, when ultimately it’s kind of just a category of bug. And so doing testing in a UAT environment makes a lot of sense, just like you would test for the functionality of the app, testing it for the security of it in an environment that has the same code is a good way to do that. Now, the only exception to that would be, obviously, if you don’t have a UAT environment or a lower environment for the application that you’d be looking at there, or if the code is significantly different than from what you’re trying to certify in the production environment for PCI, then it’s not always feasible. I would say the vast majority of cases, doing the application in a lower environment, perfectly acceptable. But for the network test, that’s a little bit different, where we do need to kind of look at the implementation as is of like, what is your real network perimeter, right to make sure that that is looking good.

What would you say to clients that are worried about testing during business hours? This is sort of related, obviously, to the, we’re not doing the download service type attacks. I know you built a lot of guardrails around having standards and but also allowing a little bit of the art of the pen test. Can you expand on how you’ve been able to structure things that we don’t bring everything down, but we’re giving a good test?

Being here for 13 years, there’s definitely some rules that are written in blood on our test safety guidelines. Not that we’ve caused major disruptions, but there can be, like, little things or things that could be maybe major annoyances. If, like, your help desk has to reset 100 passwords, that’s going to be a pain in the butt. So, we’ve gotten pretty good, and I think most firms as well, cricket are avoiding those common pitfalls. Another thing is just generally keeping in mind the safety of, if there is exploitation that you want to pursue, checking in first, getting buy in, making sure everyone’s aboard, of what’s going to happen if it’s something that has any risk at all. Now, there’s some things that don’t have any risk that, logging into the system, or collecting this hash, or cracking offline, especially you’re not even interacting with the system, very safe to do. But we do have a couple, guardrails where we say, this is the time, where we ask, do we want to pursue or just report on this? Then, different kind of hours for testing. For most folks, I recommend during business hours actually, because sometimes counter ironically, it can be more of an issue if something’s happening after hours, because then your team is getting woken up. Maybe they’re not available. They’re coming in the next day. Now it’s off cycle where people aren’t synced up. I’ve seen that honestly cause more of a problem usually. Usually, I don’t recommend off the bat, but some teams, they are geared towards that. I would say, depend on the client a little bit whether they’re used to doing after hours maintenance cycles and whether, whether it’s something they would prefer. Usually doing during normal hours is the safer bet. These people, pen test, usually not much going on, as far as risk of disruption. And obviously, pen test is a new thing happening environment, so we’re more than happy to triage issues of like, what’s going on here? I’d say most of the time it’s something unrelated to the pen test. But of course, those things can happen, and something we make sure to constantly improve and not in our processes. If there is something that does occur, but it’s gotten pretty rare. I will say, it was a bigger issue much longer ago, when you still had a lot of really ancient switches, ancient networking equipment that would take a dump if you were doing Port Scans. That’s pretty rare. Now we’re still pretty careful, especially if we’re PCI, you can have a lot of retail locations. Those can be on satellite WAN links, so their internet link is going over a very slow connection. Those you do have to tread more carefully than maybe you would in another kind of standard corporate environment, in a single office building or something like that. Things are pretty safe as far as the pen test goes. Michaela, I know PCI does allow merchants and service providers to do their own penetration testing, provided they meet the standard. Do you see that much at all, or people using a third party most of the time?

I would say most of the time we see that, a third party is used. You can have someone internal to your organization be the pen tester. There just also still has to be that organizational independence. What that means is, if that pen tester that you’re using, is checking out your network and applications and looking for any kind of potential findings they need to in your own organization, not be part of, those teams that those findings would relate to, not be reporting to somebody that is handling that, basically setting them up so that they don’t have any reason to lie about the findings. There’s nothing that they’re going to try to hide, to cover their own backside, or their team’s backside, right? They are completely separate from that process of having to remediate and resolve those findings. They’re just mostly interested in identifying and then, as a result, having your security hardened from that. It is something where you can do it with an internal person. They also need to be qualified, so they would need to have some sort of experience or certifications behind them, education to enable them to do this penetration test. At the end of the day, Kevin would be able to speak a lot more on it than I would, but from a high level, that penetration test, it is different from that vulnerability scan, like we mentioned. It’s not just scanning for vulnerabilities and then seeing if you can use those vulnerabilities. It’s using other tools, techniques, knowledge, using combination of different things that are played to create potentially, a critical finding out of what was just low vulnerabilities, potentially. As long as they have that ability to do so, and they have that organizational independence, there’s not that independence within the organization. A lot of times they don’t have someone on staff that has that knowledge and expertise to be able to do that pen test. That’s where it is usually a great benefit to most of the clients that we work with to use a third party to do that penetration testing. Then you have someone that is usually doing penetration testing as their full-time roles and responsibilities at their own job. It’s not something that they’re just picking up once every six or 12 months as kind of a side responsibility to their main responsibilities.

Probably not a lot of good casual penetration testers out there. You got to really know and do it a lot to be good at it. What would you say to clients? This is really for both of you. What would you say to clients that say, my environment hasn’t really changed much since last year. I don’t need that rigorous of a pen test. Is that a viable excuse to do sort of a, less rigorous pen test at all. Or what are your thoughts there?

I’ll start from the PCI perspective. I would say for PCI, there is that requirement in there. If you have significant changes, you would need to do another penetration test otherwise, or just set up on that normal schedule where, if you’re a merchant every 12 months, if you’re a service provider every six months. To someone that says, we haven’t really had any big changes. It’s basically the same environment as before, I would say you’re still getting a lot of benefit of this penetration testing, even beyond just compliance for PCI, DSS, because really, penetration testing can also identify some potential insider threats that you may not be knowledgeable about, or, more importantly, a lot of times, it can identify any kind of gaps in your change management process. If something was missed, there was a change that was put in place, and something that lessened your security was put in place without that being understood or maybe caught or given as much attention as maybe usually it would elicit, that’s something that the penetration testing can catch. It’s not just about any changes that you’ve had to your environment, but also just making sure that the understanding of the environment is still correct, our assumptions that we have about how well those systems and networks are hardened is still correct. There’s not anything additional that has been kind of swept under the rug or accidentally not caught.

In addition to finding those unknowns, if you don’t think there’s changes, but maybe there has been a year, is enough time where there’s typically new techniques that we’re employing as pen testers that maybe we didn’t know about or were newly researched or discovered, over the years. I would say the tools that I use today are completely different than I was using five years ago, or the kind of findings we’re having. It is a little bit of that, that rat race, or even if your environment stayed static, you may have new issues that can be exploited just by the nature of tooling getting more advanced, or new types of issues being discovered. This one’s getting involved now. In these certificate services, that was a big one in the 2020s where not as much scrutiny on that. Now, a lot more scrutiny on that. We’re finding a lot more things there, maybe with more advanced tooling, more advanced techniques, than we would if we looked at it five years ago. Those things kind of incrementally improve every year. Maybe a year is not always the perfect cadence, but it tends to be the right confluence of events, where you’ve had enough potential changes in the environment, and you’ve had enough iteration and improvement on the pen testing side of things, where it’s worth doing that new check. I think that’s why the industry as a whole, but more specifically, the PCI Council has settled on annual being a good cadence to do that deep dive, pen test.

You got to keep up with the arms race. I think we’ve been talking a lot about penetration testing, but there are other aspects that are required for PCI, including ASV scanning, maybe we can define that and vulnerability scanning. There’s a lot to unpack there. Michaela, could you step us through what that actually looks like in practice for clients?

With the vulnerability scanning, it’s very easy to kind of mix it in with the penetration test and think of it as one thing. With those vulnerability scans, just like with the penetration tests for vulnerability scans, you have internal, and you have external. The internal scans there actually have been some changes. The 4.0 there’s been a what was previously a best practice. So effectively optional requirement that now went into effect is now required after March 31 so effectively starting April. On the internal vulnerability side, that change is basically required now that we’re using authenticated scanning, where you’re able to if there’s some kind of technical constraint where authenticated scanning doesn’t make sense, it can’t be implemented, that’s fine. That’s an exception there where you don’t have to do that. But for any technologies that can use authenticated scanning that is now required. That is one change on the internal for the internal vulnerability scans, it’s identifying any of those internal vulnerabilities, as you would imagine. Then we’ve also got the external side, so your external vulnerabilities identifying the external vulnerabilities with the external vulnerability scan, that’s where you have to have an ASP scan and approved scanning vendor that is doing that external scan. So ASV scans for PCI are different than just your general external scan. With ASV scans, those approved scanning vendors have to go through their own assessment and follow certain processes and protocols under PCI DSS for how they’re conducting those scans. Those scans also check for some things that external vulnerability scans usually would not so if there’s something that is not compliant with PCI DSS, specifically, those ASV scans are going to flag that as something that needs to be fixed, and thus results in a failed scan, whereas an external vulnerability scan may not even check for that, or it might just say, this is a low vulnerability. But ASV scans do keep in mind the PCI, DSS requirements and whether it be compliant with it. In either of those, the internal or those external vulnerability scans, you do have to do those quarterly. The PCI console has become a lot more strict about the timing with the 4.0 changes. So before, they would always just use the word quarterly, and they’ve, more recently in the last couple of years, defined quarterly as between 90 to 92 days. You have a two day window. They also do give the option for your frequency, that if you perform it on the same day of the month every three months, that’s fine as well. If you always do your testing on the fifth and you’re doing it every three months, no longer than that, then that’s fine as well. Either getting within that 90 to 92 duty range or doing it on the same date, but those are some of the changes with the internal vulnerability scans, and then that is how the internal and external vulnerability scans are under PCI. Again, these are completely separate. You’ve got your penetration testing and then you’ve got your vulnerability scans that you need. Having vulnerability scans in your environment doesn’t mean that you get to skip out on the penetration test.

You need to have both for PCI compliance. Yeah, they’re mutually exclusive. What would you tell a client that misses a month or a quarter for vuln scanning? Have they blown their PCI compliance? How do we handle that?

The nice thing is, this has actually always been a point of contention with the PCI console for both QSAs and other clients and entities. Because at the end of the day you can’t time travel. The technology is not there, not yet at least. With those scans, if you do miss it where you know you’re not within that 90-92 day, or you always do on the fifth a month, but then suddenly you did it on the 16th, or something, one month. You have a couple of different options here. One of the options is kind of the traditional route that was available in previous versions of PCI as well, where we can do what’s called a compensating control. The compensating control, it basically looks at, you didn’t meet the exact language of the PCI requirement. That exact scan frequency interval but did you meet the intent of the requirement. For vulnerability scans, generally, the intent is, are you identifying vulnerabilities and then following a PCI compliant process to remediate or address or resolve those vulnerabilities within the right amount of time? If you do have something else that’s in place that might not be your exact internal or external scan provider during that time, where you miss the interval, potentially, we can use that as a compensating control, or if you have other things in mind for your environment that might even limit the ability of those vulnerabilities being relevant or being exploited. That’s something that we can take a look at. Compensating controls are never guaranteed. There are always going to be a discussion between you and your QSA to determine what makes sense for that situation. That’s the first option. The second option, which I think is loved by everybody, is that they’ve officially released guidance more recently, that if you have a process under PCI where you miss the time interval, as long as you can say what that exceptional circumstance was that led you to miss it, and you can prove that you’re now correcting and following that corrected process so that it shouldn’t happen again, then you can still meet PCI compliance through that way. If you miss a vulnerability scan, for instance, because maybe you changed vendors and the schedule didn’t save correctly, and so it didn’t run when you expected it to. We’re going to take a look, see who’s responsible for checking that, make sure that there’s a process in place that that shouldn’t occur again, and then making sure that your schedule is now set up where you shouldn’t miss that time interval in the future. We’ve got these automatic scans going off of that schedule. That would be one example of how you could show an exceptional circumstance and how you’ve corrected the process.

We’ve got quite a few questions, which I think we want to pivot to. There’s one around vulnerability scanning. Let’s start there. Since we’re talking about vulnerability scanning, the question is, how are vulnerability scans considered when the scan is only performed by agents installed on systems, for example, using a solution such as Tanium, but not still performing a wide IP range scan.

For that sort of situation, I’ll say first, if it’s used for internal scanning, and say we can’t use authenticated credentials, because it just doesn’t make just doesn’t make sense for that situation. Those agent-based scans, the agent that they’re using, would still have the elevated permissions of some sort of privileged or admin user on those systems that can be something that still meets the intent of that requirement. Those agent based scans, as long as they have those elevated privileges that should still meet the requirement, and then otherwise, I would say, for any of those agent based scans, we would just need to make sure that all the systems that need to be covered, that are in scope, that need to be covered by that vulnerability scan are included. If we have something per system that can be fine, as long as we just see that all of the CDE, the in-scope environment is being covered by those vulnerability scans, even if they’re per system.

Makes sense. There’s another scoping question here. What guidance does PCI provide for scoping a service provider audit for a company that doesn’t have a CDE, such as a cloud or co located hosting provider, but their customers may process card holder data on their own servers?

This is an interesting circumstance that happens sometimes, especially with service providers where you don’t yourselves have a CDE. You’re not actually transmitting, processing, storing any kind of credit card information or interacting with it, but you have customers where maybe you provide them like a managed network service, and so you would then be part of your customers in scope environment, right? Because you can impact the security of their CDE through that network management or the service that you provide. It comes down to understanding for your customers what that relationship is between you and them, what you have access into, what you have control over. That’s something where service providers, part of PCI, is providing that responsibility documentation that tells your customer, we set up these accounts for you, but maybe the customer is actually responsible for enforcing the password rotation and complexity things like that, so that your customer understands where their responsibilities are, versus your responsibilities for PCI compliance. Having that responsibility documentation is huge. To understand what needs to go into that where your responsibilities actually are, it really comes down to looking at your customer environments and seeing, if you yourselves, your own environment were to be compromised, or you had insider threats from your admins, how much of an impact can you have on your customer’s environment? If you can have an impact on their security, then it’s going to be in scope for PCI there. Say you have, one segmented network where you use all of your systems, that’s where the access is, where you be able to remotely access your customers environments, and then you have a completely other segmented network that is just used for your own internal purposes. It has nothing to do with your customers. That might be something where, as long as you can show that that segmentation is in place between those two networks, that first one that connects with your customers environments remotely, that would be in scope there for PCI, but that second one, as long as it’s segmented, shouldn’t be in scope for PCI. There are things like that to consider as well per environment.

That’s great. Next question here is, at what size or type of organization do you see organizations have an ISA on staff, and what department are they in? We’re defining what ISA is,

You can think of an ISA is basically, almost effectively kind of a QSA just for internal use there. Depending on if you’re a merchant, the number of transactions that you do through credit cards, you may be required to use a QSA for your PCI assessments. Or if you’re under a certain amount, for different payment brands, it’s all different. You might be able to use an ISA instead. For those that can use an ISA, it really just depends on the preference of that organization. Some organizations they want to be able to just outsource their PCI assessment to be validated by a QSA, even if they’re not required to just so they don’t have to worry about getting anything incorrect. Then their personnel can focus on, their specialties, what they’re great at. Then other organizations have more of an appetite to bring that into their own organization and have an ISA on staff, and it might be part of a compliance team, usually, that maybe handle other things, like SOC assessments, or really depends on the organization, but yeah, I would say as a whole, it really just depends on the organization’s appetite for outsourcing versus having internal personnel do those responsibilities. After you get through, whether you’re allowed to use an ISA or if you have to use a QSA, depending on your merchant eligibility. For service providers, most payment brands are going to require you to use a QSA depending on basically your impact to your customers environments, or how many customers you have, sometimes as well. Unfortunately, it is varied between all the different payment channels. Sometimes they’ll reach out specifically to you. Sometimes they expect you to agree to a process that includes this. It really just depends on your service that you’re providing.

It’s good to work with ISAs. Sometimes they make assumptions about scope, and then you sort of having to walk that back, but they definitely play a critical role in a lot of organizations. Sometimes we’re working with them even if you do have an ISA. Sometimes, ISA, sometimes we are also the QSA, and we’re working with them very closely.

Yeah, exactly that. We have multiple clients where they do have an ISA on staff that we work with, and so, of course, we list them as part of the PCI assessment. We also have a lot of clients that use us for advisory services where we’re not the QSA that’s going through their PCI assessment, but their ISA is using us for that advisory. They’re asking us questions around scope, how to interpret certain requirements, what kind of evidence needs to be collected, all that sort of thing. We have services to support on both sides, whether you know you want to use this as your QSA or whether you’re just looking to ask some quick questions and get that process rolling internally.

Sounds great. Kevin, I think this last question is for you. Someone’s asking if there is an EDR solution, or managed EDR solution that is better than the rest, or one that you would not like to go up against, or what’s your you know which one’s causing you the most heartache. I think you’re welcome to play. Like a vendor here at this point.

I feel bad taking sides because they’re pitchforks, but I mean, most common that we see is CrowdStrike, probably, and I think that’s probably well earned, because they do a really good job. One of the interesting things, and maybe why I think they’re so successful, is they do bring not just, our endpoint has good detection capabilities and good insight, but also close the loop on, actually alerting and doing a good job of making sure those alerts get to your IT team, whether it’s just the DUI of the, cross track Falcon interface kind of getting you those emails, but also the Overwatch team, they tend to practically reach out and send you emails to the support context, like something’s going on. You guys need to go take a look at this. On the pent up side. We can get around CrowdStrike send a one, pretty much any of the EDRs out there, Tanium with our payloads, but they do a good job at restricting what behavior we can do without trying to throw up alarm bells, not a PCI pen test. That’s a little bit more open box. I mean, it’s for the person, PCI, we’re not so worried about never being seen, but for a real attacker, obviously, that’s key. They want to make sure they stay under the radar. Right now, the controls, EDR, when we talk to our IR team, tends to be that first trip wire that usually ends up going off. For our pen test team as well, does throw a bunch of wrenches in there, back in the day, let’s say like 2013/2015, oftentimes we would find an issue we could really easily just run Metasploit, exploit that issue, get them interpreter, shell. That kind of thing is not happening anymore if you have EDR in your systems, the EDR is going to flag that thing immediately. It’s going to burn it down. It’s going to quarantine it. We still can get payloads deployed, but we have a lot of times we’re having to customize that exploit, and it’s probably more work to get the payload working than the entire rest of the pen test. Oftentimes those are things that are kind of preventing that from being realistically pursuable during the course of, you’re trying to turn it into a full-blown Red Team, which I don’t want to blow up the budget on that for, for your PCI pen test. A lot of times, we’re leaning more network factors. How can we live off the current host? How can we avoid EDR altogether? How can we, just interact with systems over the network, versus having to drop payload, versus having to run exploits to do certain things? Of course, still reporting and identifying attacker, given a couple months, they might be able to get something around the CDR, but here are the things that we’re going to pursue instead. We see a lot of CrowdStrike can definitely give them a shout out. They do a good job at, I would say, notifying our customers that something’s going wrong, but all those protections out there are good. I think really, the key thing that you want to rely on is make sure those alerts are actually going somewhere. I’ve seen a lot of cases where we were throwing up alarm bells, but they didn’t end up going to anyone. You don’t want that to happen in a real attack.

Good advice. Any closing thoughts here before we wrap? Kevin?

You know one thing, and this is maybe a little bit tangential for a closing thought, but going back to the question on it, on vulnerability scanning from agents, one thing you do need to keep in mind that and Michaela touched on this as well, is that you have two requirements in PCI. You have your internal volume scans, which are just, you’re kind of running volume scans on regular basis. Those agents meet that need. You do also have something separate, called an ASV scan, and that has to be done by a very specific authorized provider, and so they may not be counting as ASV, and usually that’s against your external perimeter. I’ve seen that confused be a little bit confusing, because you have your ASV scan and your vulnerability scans like, what’s the difference in these scans? They are a few different things. And the ASV scan, I think it stands for authorized scanning vendor. I could be wrong there, but that is basically something that is very strict on. It has to be a specific qualified ASV that’s doing that against your external network. That’s where you, in addition to your agent-based scans, you may need to also do an ASV contract. I think there’s pretty reasonable, cheap contracts out there do ASV. I know we’ve worked with Qualis, tenable, pretty much any vulnerability scanning solution is going to have an ASV arm that they support in service of this PCI requirement.

I’ll add on the tangent real quick, too, for those ASV those approved scanning vendors, that’s also something that it’s a list of approved scanning vendors on the PCI SSCs website. That’s something where you can go and look up the full list too. If you’re already using a scanning vendor and you’re not sure if they offer ASV scans, they should be listed on there if they do, or if you’re exploring different options, you should be able to find them on there as well.

Then maybe for Mark. Going back to your initial question of a closing thought, I would say, when you’re looking at the pen test, it helps to look at it as not just a burden that you have to do for PCI, but something that’s helping you get a look at your environment. When we do the pen test, we try not to only be checking the box for compliance but trying to help find this puts you at risk for ransomware. This puts you at risk for data. Application. We have a lot of customers that maybe there’s not a huge amount of budget for security. You have to be very judicious. Now you spend that. And if you’re getting a pen desk for PCI, I would say, don’t be afraid to ask to put in some of your out of scope or other networks, because that can tell you a lot about your environment and give you things you have to fix. Well, we can keep that separate pretty easily from your actual, you need to go fix this for PCI. Even again, those findings, they may not affect your compliance at all, but it might be something you want to know about. Or, like Michaela mentioned, it’s going to put the QSA a little bit easier if they see that scope was a little broader. If they do need to expand it, because there maybe was some initial confusion on the scope. It’s complex there. There could be things that maybe you thought were out of scope that really should be in scope. Then your kind of covered there. Especially for the external side, just throw that wider net. Look at the external, we’re not really, I know some firms may be charging exactly by IP. We’re trying to get the order of magnitude. We’re not really just, you have 10 more IPs. We’re charging incrementally, 10 more effort, obviously, if there’s like, 1000 more things to look at, maybe that’s a different discussion. Especially on the external side, casting that wider net and making sure that you’re looking at kind of a holistic view can be pretty helpful, and then helps you just get the value out of, yes, you’re getting this to meet your compliance requirement, but hopefully it also is adding some value. That’s always our goal, that we’re not just doing it, just to do it, but it is, beyond just the PCI compliance giving you good information that helps you understand your environment and hopefully address and fix issues before an attacker comes in and takes advantage of them.

No great stuff. I know we’re at time here. Kevin Michaela, thank you very much for your time, and thank you all for joining. If you need more information, feel free to look us up at the Tevora website, or you can always find us on LinkedIn.