Security Sandbox 10.28.22
Ep 20 | 10.28.22

A How-To Guide on Perfecting Your Pentesting Program


Amanda Fennell: Thanks for tuning in. If you enjoy today's episode, please rate and review us wherever you get your podcasts. Welcome to "Security Sandbox." I'm Amanda Fennell, chief security officer and chief information officer at Relativity, where we help the legal and compliance world solve complex data problems securely. And that takes a lot of creativity. One of the best things about a sandbox is you can explore and try anything. When good tech meets well-trained, empowered employees, your business is more secure. This season, we're exploring ways to elevate the strongest link in your security chain, people, through a creative use of technology, process and training. Grab your shovel, and let's dig in.

Marcin Swiety: In today's episode, our sandbox heads across the globe for a special takeover episode by yours truly, Marcin Swiety, and my colleague and security expert, Bart Czyz, for a dialed-in conversation with offensive security and pentester-extraordinaire Julio Cesar Fort on the intricacies of modern-day pentesting and how you can elevate the everyday skills of your team members to make them excel in offensive security techniques. So in the words of Amanda, let's grab a Red Bull, pull up a seat, and let's dive in. 

Marcin Swiety: So it's very, very nice for me to host both of you because I have been working with you, Bart, for quite a while now, known you, Julio, for a couple of years now, and I know both of you have offensive security techniques and skillset and the entire area dear to your hearts. And I also know for sure that both of you have different perspectives. And actually, let's start with that. So, Bart, let's start with maybe going directly to the chase, like, cutting to the chase. What is a successful pentest? 

Bart Czyz: Sure. Thanks, Marcin. For me, as a blue teamer, a successful pentest is the one that actually results in improving a security posture of a company. This is how I - how my perspective looks like. I don't really care about, for example, how far a pentest goes. Are pentesters able to achieve the crown jewels (ph) like active directory domain, you know what I mean? From my perspective, it's more important to actually make the security posture better as a result of pentest engagement. 

Marcin Swiety: Cool, Bart. So I think at the beginning, we should also talk about pentests because that's also - might be something that we might understand differently. So for some of you, probably, pentests might be just trying to get into the system, trying to verify what our defenses are, right? But for others, might be something much more like looking at what you just said, Bart, looking at the entire portfolio, entire process, like, making sure that the company is more resilient to security threats and attacks. So, Julio, when we think about pentesting as a definition of what it actually is, how would you define it from your perspective? 

Julio Cesar Fort: Yeah, I also agree, like, more or less to, actually, what Bart said. So actually, allow me just, like, to go back a little bit. So, like, the whole term of, like, pentesting and red teaming and so on, it has been changing a lot for, like, for the past, like, 20 years or so. I remember - so when I started working professionally with the cybersecurity - so it's been a while now - don't want to tell my age, but it's been, like, a little over, like, 15 years. And I remember that back then, the pentest was what a lot of people nowadays call an adversary simulation. So even the term kept - keeps changing. So a pentest, before, was like ominous, possible hacking to a company. There was no such thing as much of a scope or sort of, like, limitation. And then pentests became something more like a - little bit more like an audit, you know, like a web application pentest, a mobile application pentest and red team became the thing that - you know, like, this adversary simulation, this come and hack my organization kind of thing. And nowadays, even red team is actually - anything that's offensive security-related, it's now being labeled or being under this big umbrella of red team. 

Julio Cesar Fort: So, like, even the whole thing with terms and so on, they are changing a bit. But yeah, like, to actually answer the question, like, I think what Bart just said, so the whole thing with success, I agree with him. It's actually bringing value to - it's not, the pentest just going to be I hacked your company and here it is, but actually bringing value to the team to kind of, hey, these are the stats. These are the weaknesses. This is how we can improve. These are, maybe, recurring patterns of vulnerability that we have seen. So this means that probably your team has to upscale with some training, or maybe, you know, just go and look into your infrastructure internally if these factors of vulnerability are still there as well, and so on. So essentially, a successful pentest is the one that not necessarily just finds, like, individual vulnerabilities alone, but, you know, it's like capable of clearly illustrating impact of those issues as a chain, not in isolation. And more - also importantly, if it catches attention of senior leadership to the needs of further investment in the security program as well, I think, it's also super important for a good pentest, like, together, like, not only helping the blue team, but also bringing awareness of more, like, senior leadership about the other things the security team are doing. 

Marcin Swiety: Oh, so this is - I love that you both actually went a little bit further than just saying a pentest can be a simple scan from OpenVAS or other tools because I also seen that in my life, that somebody called that type of engagement a penetration test. But you caught a very, very important note, Julio, on making sure that higher parts of our companies, decision-making bodies or entities, are also aware. So it brings awareness. I know that in our industry, there is always a saying that nothing brings better awareness to our decision-makers than a breach, right? We've seen that with multiple companies. We have number of posting - job postings right after a security incident. But did you find, like - both of you, Bart and Julio, have you found a successful pentest to bring that awareness instead of a real-time and realistic breach that should have happened to bring that awareness too? Bart, was that kind of the same vehicle? Was it successful? 

Bart Czyz: So I believe I did see that. So in many cases, when you receive a report from a penetration test, you not only get the list of vulnerabilities, you get more of a list of priorities, what you should be focusing on. And it really helps to, like, speed up some of the decisions made by the leadership team. And it's very important. Actually, a report can be a huge leverage when you need a buy-in to get some - whether be it visibility, like, buying some new tools to increase the visibility of your team, of your security program. And I find that it actually is the case for many penetration tests. 

Marcin Swiety: Yeah. But, Julio, on the other hand, you're doing this commercially, right? Your company, Blaze, is doing amazing job with offensive security efforts across the globe. And you probably had situations when you are handing over the report, and you probably, you know, wonder how it's going to progress the security posture and all around. Do you ever have concerns that your work, that you've put into your heart and time and focus and skills and amazing expertise, might just be a paper that is going to lay in the drawer? 

Julio Cesar Fort: Yeah. So it actually happens more often than not, unfortunately, actually. Because I think that there are, like, two big drivers - right? - when it comes to cybersecurity, like, in an organization, maybe three if you want to stretch it a bit. So one of them is actually - the one that's the main drive is engineering. So I think that's what you guys are doing at Relativity, that you do have a strong, like, security program, like, product security team and so on. And then a lot of the engineering is driving security and vice versa. 

Julio Cesar Fort: And - but I would say that, from a commercial standpoint, unfortunately, in my opinion, at least, a lot of the driver is actually compliance, especially with SOC 2 becoming very ubiquitous across, like, Europe and the states and other regulations like GDPR and so on that are kind of pushing organizations to, like, perform more sort of, like, assurance - cybersecurity assurance services, like, such as penetration testing. And a lot of it is actually driven by compliance, and a lot of the compliance was - it's just a kind of a tick box. They want to check that box and then please the auditor, whenever he comes. And they don't really care so much whether the results are good or not. Sometimes they actually care the results because they want to look good in front of a business partner, like, for example, when you have to do third-party security assessments. And so there is also sometimes when they actually care about what the results are going to be. But unfortunately, I would say that half the time, people just want that paper. They just want that rubber stamp from an auditor and off you go. And this is kind of, like, a bummer in many, many aspects. 

Julio Cesar Fort: But one of the things that I have seen that actually brings a lot of awareness and actually changed things, it was that a couple of times when we were actually hired by internal audit teams - internal audit, they usually have the ears of the board of the C levels. And depending on the results and how you also - not just the results, not just, like, you know, a series of vulnerabilities there, but kind of the whole storytelling, the whole, you know, like, proper illustration of the impact. And I have seen changes, like, serious changes. Like, within a year, a lot of things have changed. A lot of extra budget appeared for the security team, some of the right tooling in place, some of the right mitigations in place. And - but I think internal audit is a very powerful driver to actually change things. In many cases, you have security leaders trying to speak with the same loud voice to the board, in many cases not really heard. But when a chief auditor comes, I was like, hey, these are the cyber risks that we expose it to. In many cases, CFO has the buy-in, as well. And that really - it's a very powerful change, like, far more sometimes these going via the route of CISO and the security team. 

Marcin Swiety: Thanks, Julio. This is - so this is something very interesting that you touched. So it seems like in order for a pentest to be successful, you also need to have a strong partner on the other end, right? So it's not only how well we perform the pentest as a, you know, pentesters or offensive security experts, but also, we need to have the partner on the other end to take the results and run with it, right? So let's, like, talk about the human factor of both of those sides. How would you tell, Julio - what is the best composition of skills that you would be looking for if you would be selecting your pentest team to, you know, a certain engagement? What should you be looking for? 

Julio Cesar Fort: That's a good one. That's a pretty good question. So it really kind of - it depends. So it really depends on what the scope of the engagement will be like. So what are the actual needs of the customers? So, for example, if a customer comes to say, like, oh, we have, like, a new consumer mobile app that we want to release to the market, it actually makes no - it would make no sense to get, like, a very good guide with active directory in it, you know? So finding - understanding the needs of the client, sitting together to figure out what the scope is like, what track modeling is like, and then, OK, I'll figure out, like, who, like, within the team actually has this proper skills. I think this is the best way to do it - not every time it's possible, but we always try to. 

Julio Cesar Fort: Like, for instance, we're getting, like, a lot - more and more requests of people performing, like, Web3 kind of - they want, like, Web3 type of pentesting and, like, block - things related to blockchain and so on. And not everybody is yet well versed with this technology. So we really have to find the right people for that. Like, makes no sense to just get, you know, a guy who is good at pentesting of, you know, like, networks or, like, a Windows or Mac, even. Just makes absolutely no sense for that kind of scope. So that's - basically understand the need and figure out who's the best for the job. 

Marcin Swiety: And I have very controversial question. I will transfer it to you, Bart, in a minute. But, Julio, this is a very something that is dear to my heart. Do you feel a pentester needs to be good at client interfacing activities? Is it internally important part of the job? 

Julio Cesar Fort: No, definitely. Like, in the end of the day, there's actually a lot of (inaudible) on my team. I think not everybody here is, but in the end of the day, this is professional services, just like anything else. This is like accounting or just like being a lawyer or just like, really, like anything else. It's like consulting, more than anything else. It's true that some guys, they want to, you know, be, like, left alone, like, with the hoodies in the basement doing their thing, and that's fine. But in the end of the day, I would say that people that truly progress in their careers are the ones that are not only technically capable and gifted but also the ones that put effort to be good at client facing. And I think - but this is actually tough to learn. 

Julio Cesar Fort: I know some - I have seen, like, some companies that have got - like, for instance, got their guys, their geeks, and they're like, OK, I'm going to get all of you guys to become, like, proper consultants. So actually paying, training for them to become, you know, to know how to speak in public, how to interface with clients and things like that. And I think it might work. But I truly believe that, yeah, if you want to properly progress in a career, knowing how to write really well, write good reports, explain things properly and be client facing or at least try to make an effort, it's actually super important in my opinion. 

Marcin Swiety: Yeah. I couldn't agree more, Julio. I remember my days of penetration testing for a number of different industries. And this is something that I thought is going to be the side part of my job, the client part. But also, you know, over a couple of years in that industry, I really felt like the contacts are important. I think all side of it is important - like, that's the meat. But how you handle the meat, how you provide it and how you describe what's in there, what's it about, it's even more important because eventually, you want to - for them to find a value in what you just did with a pentest. OK. So, Bart, going to you - and you probably know the reason why. You're our blue team expert. That's how you also introduce yourself to this podcast. But you also know that we are kind of savvy on making sure that our blue team is staying on top of the red side of things. And do you believe this is - like, the offensive side is helping in becoming the true defender of our fortress? 

Bart Czyz: Oh, definitely. I believe, like - I believe in - every security team or every security program should have at least one person that is very offensive security forward. And I believe that whether it's a fully - whether your responsibilities are fully blue team - like, you work in a SOC, or you're a forensic analyst, or you're an incident responder - you have to know the basics of offensive security 'cause as a defender, you can't defend against something that you don't know nothing about. So definitely, I agree that it's very important to have the skills or at least have an idea of what an offensive security or an adversary can do to your organization. It is very - it's, like, crucial. 

Marcin Swiety: So how do you keep up with, you know, this evolving - its industry on its own, right? Pentesting in security field - it's huge enough that you can spend literally entire career and still needing more and craving for more. So how do you make sure that you and your team keep up with the evolving threat landscape? 

Bart Czyz: Pay attention to all the available threat intelligence. See what's used in the world, what the new techniques are, and try to emulate them in your environment. So it's kind of an internal red team assessment or internal adversary assessment or so-called adversary evaluation. I think it's a great way to not only improve your skill set - like, offensive security skill set - but it's one of the best ways to improve your defensive posture. You know exactly what visibility you're going to have whenever a true adversary attacks your organization. An adversary attacking your organization is - it's not a question if it happens, but when it happens. So you have to stay on top. 

Marcin Swiety: And do you feel like there is difference in what you would get, skillwise, from people that are commercially focused on penetration testing versus the internal, you know, cyber/pentest/purple/offensive-focused person? 

Bart Czyz: Definitely. Mainly because when you do these internal assessments, you're missing the - you can't get very, like, objective. So whenever you take someone from - like, an external person, they have this very fresh perspective, and they look at your environment in a very different way. And this is, I believe, also required if you want to improve your posture. And... 

Marcin Swiety: Yeah. It feels like creativeness with that is - it's important, right? Being creative about... 

Bart Czyz: Yeah. 

Marcin Swiety: ...How you would approach hacking this company, breaching the perimeter or getting into the crown jewels - there are probably certain different ways how, internally, you would think that would happen. But we often find that, you know, external parties come up with even crazier scenarios. 

Bart Czyz: Exactly. 

Marcin Swiety: So as we are, you know, going through the discussion of, you know, commercial and internal, do you believe there is - both of those should work together? Like, what I'm trying to get is we oftentimes talk about, should cyber team know about pentests being performed, right? That's often the thing that pops up because on some point, you want to test, also, their preparedness and their robustness and readiness to respond. But on the other hand, should they block? If a customer comes to you with that question, should I notify our blue team about your services being performed? What do you respond, usually? 

Julio Cesar Fort: Yeah. So it's a good question. It's something that has happened in the past. But what I usually advise is that - for them, like, to actually understand what they want from the assessment itself. If they want - just like you mentioned, if they want to test the capabilities of defense and reaction and response and also communication from the cyberdefense team that they have, that's fair enough to just keep it quiet. Only very few people in the organization are briefed, and that's pretty much it. That's OK. 

Julio Cesar Fort: But in many cases, this just not really required. Like, in many cases it's, like, let's say your everyday, like, average assessment. And I think that, yeah, as many people, like, briefed as possible, the better for the assessment. As many information we can get, like, say, documentation, access to engineers and so on, also, like, even better. So this gives us - so essentially, you come - like, a third party come in with this fresh perspective that Bart just talked about, but with the same advantages as the internal team that has access to the engineers, has access to documentation, maybe source code snippets, and so on. So I think you combine best of both worlds in this situation. But then again, if you want to test preparedness and readiness and whatnot, yeah, I agree that keeping just, like, a few stakeholders aware of what's going on, that's a fair point. 

Marcin Swiety: OK. So I will make a pivot now to last question that I have for both of you, and I want for your really honest answers. So how do you keep your team - your offensive team - engaged and excited about pentests? Are you promising that they will find something? Are you hyping, you know, the hunt for bucks, for breach, for crown jewels? 

Julio Cesar Fort: That's actually something that - how to keep the - like, the team engaged, and so on - I think one of the things is probably try to come up with also, like, different sort of engagements because I would say the excitement is still there for a lot of people. And also when it comes to - like, perhaps, maybe not so much, you know, when we're talking about, like, a compliance, like, checkbox pentest, but especially if they're focused on new technologies or, for example, customers that say that, yeah, this is, like, going to be, like, long engagement of a couple of weeks open, we have crown jewels and objectives, and this is going to be fun to kind of really use your - all your skills, your tool sets and go for it. 

Julio Cesar Fort: Usually, these kind of engagements are actually very excited for everyone, and engagements that have new technologies involved, that they force, at least, the ones that are more technically inclined, and so on, to learn new things and to challenge themselves as well kind of, that they say that, OK, I might not find anything in this assessment if I don't learn this and this other technologies, like, say, for example, just learning to GraphQL. Like, OK, I know nothing about GraphQL, but I have to brush up on this topic and learn new things. And I think this is when people get excited, is when - especially when they encounter new technology, new things that they haven't - that they don't see on a daily basis. And I think that might be a little less excitement when it's your - just your regular no compliance checkbox kind of scanning or pentesting. I don't think this actually makes people super happy unless they're, like, starting their career. So when they're starting their careers, I think anything is new. Anything's exciting. But, like, yeah, for people that - with a bit more experience, this would not really be enough to keep them excited and engaged. 

Marcin Swiety: Cool, Julio. So what I got is rotate, make it fresh, allow for people to grow and adjust the task to skill and expertise level. So, Bart, over to you - how do you keep our team excited about offensive tasks? 

Bart Czyz: Yeah. So I luckily work with people that are very enthusiastic about everything they do, offensive security included. But from my perspective, I mostly oversee internal assessments. So whenever there's a finding, I try to be very - like, I try to celebrate every small win so that people who work on these offensive security assessments, they feel like their input is very valued, and there actually are some follow-up actions. So whatever they provide, it doesn't go to dev, no. It's - you know, it's all - every time, it's actioned. And I also try to be very supportive whenever I can, whenever there is a new idea, maybe some new program or, you know, any idea that can get us to the point where we are more secure than we were yesterday. I try to be very supportive, and I think that so far, it's been working. 

Marcin Swiety: Cool. Thanks, Bart. So to sum up our - today's episode, the key takeaways that really resonated through and through our - today's meeting is pentesting is not about just checking the box. It's about bringing current and future value to your team and company. Also, a well-constructed pentest report can give you huge leverage for getting exec buy-in. Internal audits, in particular, are powerful in driving that change. And, of course, good pentester and penetration testing experts need to be client-facing. How do you deliver news and provide solutions is just as important as the results of the report. 

Marcin Swiety: And thank you for today's appearance with me. But I also prefer something as a quote to close out this episode. So I would like to share a quote that encapsulates how and why penetration testing is so much more like a science pursuit, to not only looking for bugs in your security defenses, but also in gaining a deeper understanding of them so you can better predict and plan for them in the future. So I'll end up with Brian Schmidt, a Nobel Prize winner for his discovery of dark energy, who said, science is not, despite how it's often portrayed, about absolute truths. It's about developing an understanding of the world, making predictions and then testing those predictions. Thank you both for indulging with me with that amazing discussion. I was super hyped about having both of you in this room. And thank you. Have a wonderful day. 

Bart Czyz: Thank you. It was a pleasure. 

Julio Cesar Fort: Likewise. Thanks. 

Amanda Fennell: Thanks for digging into these topics with us today. We hope you got some valuable insights from the episode. Please share your comments. Give us a rating. We'd love to hear from you. 

Unidentified Person: "Security Sandbox" is produced by Relativity. Our theme music was created by Monarch. Find us wherever you listen to your podcasts, or visit for more episodes.