A Role for Testing

NSS Labs started in Europe, and then came to US as a security research and testing company. “Our mission is to provide transparency to the buyers so they know what they're getting,” Chief Executive Officer Vikram Phatak told us. “Think of it as Consumer Reports for enterprise cyber security.”

We spoke with both Phatak and Brian Soldato, NSS Labs’ Senior Director of Cloud Management. They said they've been doing point-in-time testing for some time, but that they're also strong proponents of live-testing. They’re working to extend from point-in-time testing to continuous validation. Attacks change minute-by-minute, which means that the future clearly lies in continuous validation.

They’ve received a generally positive reception by the market, they say. The tendency in that market is to want to learn more and fix the problems testing uncovers. “A few get defensive. But we're transparent. We hold ourselves to a standard publicly. A practice manager who knows he'll be exposed for all the world to see tends to want to get it right.”

NSS Labs’ approach is to conduct group testing for each individual vendor, providing the market with basic transparency. Vendors get their results for free. “We're not in competition with security vendors. Our job is to be the umpire. Adversaries are doing enough damage without us holding back information. Our customers are the enterprises who need the data, not the vendors, and our objective is to provide the most accurate information possible. We want to get it right—if someone thinks we've missed something, we want to know so we can get it right.”

They’ve recently published a breach detection report. They tested nine products that aim to tell you what got past your security devices. “We got together about 600 nasty attacks that we used to test the nine products.” All the major attack vectors were represented. You can find an account of some recent testing here .

NSS Labs is interested in feedback. They’ve made a free version of their CAWS service generally available. They make their money through enterprises, so if you want an API or automation, that you’ll have to pay for. But if you’re just interested in giving the system a whirl, they invite you to do so. “Please come and use it, and give us feedback.”

Updated 9/9/2016, 16:56 EDT