Security Metrics Programme — Guide

Cybersecurity Metrics & Reporting Programme

Most security reports measure what the tools emit: alerts, blocked emails, patches applied. Nobody decides anything differently because of them, and nobody notices when a number is last month’s figure carried forward. This pack builds a small set of metrics that each answer a question someone has to decide on, checks every figure before it is reported, and turns a worsening indicator into a decision on time.

Is this for you?

This pack is for you if:

• your monthly report has more numbers than decisions;
• a figure has reached the board that turned out to be wrong or out of date;
• nobody could say who owns a metric, or what happens when it goes red.

Three audiences, three cadences

Every value is shown with its target or thresholds, its direction since last period and its status in words. Never colour alone, never a combined index, and never a score out of 100.

Two kinds of metric

A key performance indicator (KPI) says whether a process does what it should, against a target. A key risk indicator (KRI) warns that a risk may be moving, against thresholds set from that risk’s appetite and tolerance:

An indicator amber for three periods in a row is treated as red, unless the risk behind it has an approved treatment plan that is on schedule, recorded by the risk owner each period. If the plan slips, it is treated as red. Where another CISO Times pack already defines a measure, this pack uses that definition unchanged: of the catalogue’s 40 metrics, 33 come from the packs that own them.

How it runs in practice

Take a distributor with 900 staff. Its September figures are supplied by 5 October and checked by 7 October. Two figures fail their checks and are corrected before anything is reported: the supplier register export was August’s file used again, and the last exercise had been entered as the date of a real incident. Phishing compromises reach two in three months, so that indicator goes amber; the risk owner is told the same day and reviews the risk on 13 October. The executive committee on 14 October agrees an action to close the second-factor gap and recommends the board’s funding decision for the quarter. The indicator for risks outside appetite has been amber every month since July, but both treatment plans are on schedule, so it is not treated as red. The board report is built from the same checked figures.

Start with these three

1. Security Metric Definition & Measurement Method: how a metric is defined, checked, changed and retired.
2. Security KPI & KRI Catalogue: 40 metrics, with owners and thresholds, to choose your set from.
3. Metrics Data Collection & Quality Workbook: the monthly checks every figure passes before it is reported.

Your first month

What to watch

Report these to executive management each quarter:

• metrics reported with an approved definition card;
• figures that failed a quality check, and what was done with each;
• indicators amber or red, and whether each escalation was on time;
• metrics retired because they informed no decision.

Everything in the pack

Adapting it

• Small organisation: start with the few metrics your board report needs. One person may supply and interpret, but never supply and check the same figure.
• Regulated entity: NIS2 asks for procedures to assess whether your risk-management measures work, and the metrics record is part of that evidence.
• IT run by an outside provider: most inputs come from the provider. Name the supplier in each card and check the contract gives you the data by the day you need it.

Where it maps

• ISO/IEC 27001:2022: Clause 9.1.
• NIST CSF 2.0: GV.OV-03.
• NIS2: Article 21(2)(f).