An editorial piece by Jaded Security on the verification problem in security consulting, written against the backdrop of the 2011 Unveillance / Anonymous controversy and the broader question it raised about provenance in the security services market.

Every time there is a major breach, the same question comes up: who is to blame? And every time, the answer is the same unsatisfying mess of shared responsibility that nobody wants to hear.

The Unveillance situation is worth a closer look. Karim Hijazi, CEO of Unveillance, claimed that Anonymous tried to extort him. Anonymous claimed Hijazi was using his “security company” as a front for less legitimate activities. The truth is probably somewhere in between, and the full story may never be known.

But here is what is interesting: the security industry is full of companies that nobody has heard of, run by people nobody can verify, offering services that nobody can evaluate. How many boutique security firms are actually what they claim to be?

It is not uncommon to meet operators who talk a big game about their offensive capabilities. Some are legitimate researchers. Some are former intelligence community. And some are, let us say, less clearly defined in their provenance.

The verification problem

When an organisation hires a penetration testing firm, how does it verify the firm’s credentials? Certifications — which are not reliable indicators of competence on their own. References — which can be fabricated. The firm’s website — which proves nothing. A pre-engagement conversation to assess apparent knowledge — which is subjective.

There is no equivalent of a medical board for penetration testers. No licensing authority. No malpractice framework. Anyone can hang a shingle and call themselves a security consultant. Some of those people are very good at sounding impressive while delivering nothing of value.

The accountability gap

When a breach happens, we blame the victim organisation. We blame the attackers. We blame the software vendors. But we rarely blame the security consultants who gave the organisation a clean bill of health six months before the breach.

We rarely ask: who did the last pen test? What did they find? What did they miss? Were they actually qualified to do the work? Or were they a two-person shop with impressive certifications and a nice website who ran an automated scanner and wrote a report?

The security industry has an accountability problem. Until that gets solved, organisations are going to keep getting breached despite spending millions on security services from firms they cannot properly evaluate.