Your Security Score Is Lying to You

A security score can be built from individually correct checks and still tell you something false. No bug required — just four structural ways the compression from findings to number loses exactly the information you needed. We know because our own scanner did most of these to us before we fixed it, scored our own repository 24/100, and taught us what an honest number has to survive.

Any real repository accumulates code that should look dangerous: deliberately vulnerable test fixtures, security tooling whose rule definitions match their own patterns, vendored examples, a training app kept around for onboarding. A scanner that scores these as production findings produces a number dominated by code no attacker can reach — which is exactly how our repository earned 24/100 from its own fixtures.

The wrong fix is a quiet ignore-list, because a finding suppressed is a finding nobody can audit. The honest mechanism is scope classification: findings outside product code stay recorded, listed and readable — they just stop deciding the number. A credential committed to a fixtures directory is still a committed credential; it is a different fact from one in your checkout handler, and the score should know the difference.

Checks that award points for a control existing rather than working inflate every score they touch. The canonical example is a Content-Security-Policy carrying 'unsafe-inline': present, green on every checker, and useless against the attack the header exists to stop. The same pattern hides in an HSTS header with a five-minute max-age, a DMARC record at p=none forever, cookie flags on the cookie that does not matter. A score that reads presence gives you the grade for owning a fire extinguisher, empty.

Scores that weight findings by CVSS alone rank a never-exploited critical above a medium being actively harvested. Severity answers how bad if it happens; a score meant to order your week also needs how likely — exploit probability, whether the vulnerable path is even reachable, whether the asset faces the internet. Fold those in, or the number optimises for the wrong queue.

Ninety-nine perfect checks and one exposed database can average to a comfortable 90. Any score that lets volume of passing checks dilute a single catastrophic finding is a dashboard designed to reassure. An honest aggregate is asymmetric: one critical should drag the number to somewhere that ruins a morning, and the score should recover only when the finding does.

If your current tool's score seems either flattering or absurd, the diagnostic is the first row of the table: ask it which findings are deciding the number. A free scan here shows you both layers — the score, and every finding with the evidence that produced it, marked by whether it counts.