The Green Dashboard Problem

Most board packs I have seen in the last decade contain a version of the same page. Controls listed down one side, status along the other, and on a normal quarter the page is largely green. Patching is green. Backups are green. Access management is green. Incident response is green, exercised earlier in the year, with the actions closed.

The page gets a short discussion, if it gets one at all, and the agenda moves on. This is not a failure of attention. Boards are entitled to rely on summarised assurance; that is what the page is for. The problem is quieter than inattention. The page is read as a statement about what the organisation can withstand, when what it mostly records is what the organisation has done. Those are different claims, and they rest on different kinds of evidence. The distance between them is what I have come to call the confidence gap, and after years of sitting on both sides of these reports, writing them and consuming them, I think it is among the least examined risks in institutional security.

What a green status actually asserts

It is worth being pedantic about what earns a green square, because the pedantry is the point.

A control exists. A policy has been issued and attested. A tool has been deployed across the estate. A backup job runs on schedule and reports success. An auditor has sampled the evidence and found it in order. All of this is real and all of it is verifiable, and none of it should be dismissed. But each of these facts describes the presence of a control, and presence is not the same thing as capability. Whether the backups can actually restore the estate, whether the response plan works when invoked by tired people during a live incident rather than walked through in a scheduled exercise, whether the recovery assumptions hold when the failure is total rather than partial: these questions are only ever answered by testing, under conditions that bear some resemblance to the conditions that matter.

In practice, the dashboard reports that controls exist, and the room hears that capability exists. Nobody has lied at any point in the chain. By the time the information reaches the board, a statement about control activity can easily be interpreted as a statement about capability, and there is rarely anyone whose job it is to notice.

The Maersk case

The clearest public illustration is still the NotPetya attack on A.P. Møller-Maersk in June 2017, which I use here because the company has been unusually candid about it.

Maersk was a well-governed business with mature controls, and it had backups of its applications, its data and its servers. Its working assumption, which I have no doubt was documented and audit-approved, was that if systems were lost they could be restored. When NotPetya destroyed some 45,000 computers and 4,000 servers in an afternoon, that assumption failed in a way nobody had planned for: the malware also took every live copy of the directory service on which every restoration depended. The backups had survived, but the thing that made them usable had not. Recovery ultimately turned on a single machine that happened to be offline when the attack spread, which is to say it turned substantially on luck. The company's chairman later put the cost at between 250 and 300 million dollars and described the rebuild, reasonably, as heroic.

Told as a malware story, the lesson is about patching and network segmentation, and those lessons were learned quickly. Told as an evidence story, it is more unsettling. The belief that the company could recover was sincere and reasonable, and it had passed every form of assurance the institution applied to it. What it had never been subjected to was a test against total loss, because total loss had not happened to anyone before. The controls were present; the capability, against that scenario, was not; and nothing in the assurance machinery was capable of telling the two apart, because on paper they look identical.

That, at institutional scale, is the confidence gap: the distance between the confidence an organisation places in its cyber position and the evidence that would justify it.

Why the gap persists

The gap is not usually a product of negligence. It arises because confidence and evidence accumulate in different ways.

Confidence builds easily and from many sources. Certifications are renewed, audits are passed, budgets are spent and tools deployed, and each of these artefacts adds a little weight to the institution's sense of its own position. Seniority adds more: a claim made by a credible senior voice in a well-run meeting tends to acquire standing that the underlying evidence never earned, a pattern I have watched repeatedly and, if I am honest, contributed to on occasion. Time adds the rest. Every quarter in which nothing goes wrong is informally booked as confirmation that the position is sound, when it may confirm only that the position has not yet been tested.

Evidence, by contrast, is expensive. It comes from exercises that are permitted to fail, run against scenarios nobody enjoys contemplating, at a cost that appears in this year's numbers to guard against an event that may never arrive. An organisation that is not deliberately generating evidence will tend, over time, to generate confidence instead. I would qualify this: some institutions, particularly in heavily regulated sectors, do test genuinely and often. But even there, the standard assurance machinery leans heavily toward verifying existence. An audit can confirm that the backup policy is followed. It is rarely designed to confirm that the estate can be rebuilt from nothing in ten days, and both findings arrive at the board in the same colour.

What follows from this

The remedy is not additional tooling; the organisations that feature in the public case record were not short of tools. It is a discipline of questioning that has to be applied while things are still quiet. What does the organisation actually know, as distinct from what it believes? When was the evidence behind a given assurance last generated, and under what conditions? And which of its assumptions has never been tested at all?

A green status that can cite a recent, realistic test is a statement about capability. One that cannot is a statement about activity, and a board does not need technical depth to ask which of the two it is looking at. In my experience the question is rarely resented. It is simply rarely asked.

Where this argument comes from

This article is the first in a series drawn from a theory I published this summer, the Institutional Cyber Judgement Doctrine, which asks how organisations can know that the confidence they place in their cyber position is justified, and which separates four things institutions habitually blur: controls, capability, judgement and alignment.

I should also be clear about the evidential status of the idea. The confidence gap is at this stage a theoretical construct with a research and validation agenda attached, not a validated instrument. The public case record is consistent with the construct; it does not yet constitute proof of it. The full paper, including the propositions and the conditions under which each could be weakened, is available as a preprint on SSRN, published openly so that it can be challenged early rather than deferred to prematurely.

The next article in this series considers what happens when the gap is left open over time, under the heading of judgement debt. In the meantime, the practical suggestion I would leave with any executive reader is modest: take one green item from your most trusted report and establish when that assessment was last earned by a test, rather than carried forward by an absence of incident. The answer is usually instructive.

Dr Joseph Ikhalia is the author of the Institutional Cyber Judgement Doctrine, published by Real Cyber Nation, an independent cyber capability and judgement assessment institute founded by the author. The full preprint is available on SSRN.