Compliance · Testing

Your register says that control works. Your pentest just proved it does not.

A control marked verified and a Critical finding against the same control can coexist for months without anybody noticing. Not because either is wrong — because they live in different systems and nothing compares them.

SemperWise Security Engineering 7 min read Compliance
Each one assumes the one above it is finished 1 Data map 2 Notice & consent 3 Rights 4 Breach 5 Retention 6 Children's data 7 Processors 8 Safeguards 9 SDF duties start not here Map first — starting at two means doing two twice

Here is a situation that is far more common than it should be, and almost never detected.

In the compliance platform, control A.8.24 — Use of cryptography is marked verified. There is evidence attached: a key policy export, collected in February, with an annual expiry. An auditor sampled it in March and was satisfied. As far as the register is concerned, encryption is a control this organisation operates.

In the testing platform, a finding from an assessment in June: TLS 1.0 accepted on a public endpoint, severity High, mapped by the scanner to ISO 27001 A.8.24. It sits in the findings register, assigned to an engineer, alongside two hundred others.

Both records are accurate. Both are maintained by people doing their jobs properly. And they say opposite things about the same control, in two systems that have never been introduced.

Why nothing catches this

The structural reason is simple, and it is not anybody's fault.

Compliance platforms hold claims

Vanta, Sprinto, Scrut and the rest are registers of assertions with evidence attached. They are good at that. What they do not have is findings, because none of them employ people who test — so there is nothing in the system capable of contradicting a claim.

Testing platforms hold findings

Astra, Cobalt and every scanner produce findings, often mapped to framework clauses. What they do not have is your control register — they do not know which frameworks you track, which controls you have claimed, or what state you claimed they were in. The clause reference is decoration.

The mapping is decorative on both sides

A finding tagged "ISO 27001 A.8.24" looks like integration. It is a string. Nothing resolves it against your register, so nothing can notice that the register says the opposite.

And a person would have to go looking

Reconciling two hundred findings against ninety-three controls by hand, monthly, is not work anybody does. It is not that teams are careless — it is that the task is unreasonable, so it does not happen.

The result is a specific, quiet failure: the organisation holds a defensible certificate and an undefended control at the same time, and finds out which one was load-bearing during an incident.

What it takes to detect

Not much, as it turns out — but it requires holding both halves in one place, which is the part almost nobody does.

  1. Resolve the reference, do not display it.

    When a finding arrives carrying "ISO 27001 A.8.24", look that clause up in the register of the organisation the finding belongs to. If they do not track ISO 27001, drop it — it is noise to them. If they do, you now have a control, its state, its owner and its evidence.

  2. Compare severity against claimed state.

    A High or Critical finding against a control marked verified or implemented is a contradiction. A Low against the same control is not — informational findings against working controls are normal and flagging them would train everyone to ignore the flag.

  3. Say so where both audiences look.

    On the finding, so the engineer knows this one also breaks a compliance claim. On the assessment summary, so the security lead sees "this test contradicted three controls you had marked as working". On the control, so the compliance lead sees it before an auditor does.

  4. Do not auto-correct the register.

    The temptation is to downgrade the control automatically. Resist it: the finding might be out of scope for the control as you defined it, or already remediated, or a false positive. Flag it and make a person decide. A register that changes itself is a register nobody trusts.

The same argument, applied to certificates

There is a second version of this problem that is easier to see once you have noticed the first.

The standard deliverable at the end of a penetration test is a PDF certificate, which the client forwards to their customers as proof they were tested. It states what was true on the day it was generated. Then findings get reopened, a retest fails, a fix regresses — and the PDF says exactly what it said in March, because a document cannot know.

The fix is the same shape: stop storing the conclusion, and derive it. Make the certificate a URL whose remediation status is computed when somebody opens it. The same link that proved a clean result in March shows an honest picture in November. It is a harder thing to sell and a much better thing to own — a recipient who can re-check a link is being offered evidence, and a recipient handed a PDF is being offered a claim.

And to the frameworks themselves

Once findings resolve against a real control register, a third thing becomes possible that is worth more than either: the register stops being per-framework.

Reduce every clause of every framework to what it is about — "restricting who can reach what" is one subject, whether ISO 27001 calls it A.5.15, SOC 2 calls it CC6.1, RBI calls it Annex1.8 or IRDAI calls it ACC.1 — and a finding lands on all of them at once. So does the evidence. So does the gap. An organisation running five frameworks stops maintaining five registers of the same work, and a subject with nothing verified anywhere shows up as one hole rather than the same hole repeated five times.

That is also what makes a second framework cheap. Not a marketing claim about coverage — a mechanical consequence of having mapped the clauses to subjects instead of to each other.

The test to run this week

Take your last penetration test. Pick the three most serious findings. For each one, open your control register and look up the clauses that finding was mapped to.

If any of them are marked verified or implemented, you have found an instance of this. It is not a crisis and it does not mean anybody was careless — it means the two records were never introduced. But it is worth knowing which of your controls are currently claiming to work while your own testing says otherwise, and it is considerably better to find that out yourself than during the conversation that follows an incident.

SemperWise Security Engineering

Cryptography & Infrastructure Research

Works the applied side of security architecture — cryptography, cloud infrastructure and secure design — and writes up what the engagement work is teaching us.

Next step

Talk to us about your next assessment.

A 30-minute call, then a scope document with what is in, what is out and what it costs. No obligation, and no charge for the conversation.