Security Metric Definition & Measurement Method

Purpose

This methodology is how [[Organisation Name]] turns security activity into numbers that people can rely on and act on. Every metric is specified once, on a definition card. Its inputs come from a named source, supplied by one person and checked by another before anyone sees the figure. It is calculated the same way every period, shown with its target or thresholds, and retired when it stops informing a decision.

The aim is consistency between periods. A figure this month must mean what it meant last month, so that a change in the number is a change in the world, and never a change in how it was counted.

It sets the rules MR-01 to MR-12. Every other document in the Cybersecurity Metrics & Reporting Programme pack cites them by number: the Security KPI & KRI Catalogue lists the metrics, the Metric Definition Card Template holds each one's specification, the Metrics Data Collection & Quality Workbook runs the checks, and the Key Risk Indicator Threshold & Escalation Model sets the KRI thresholds and escalation.

The rules

These 12 rules are the method. The sections that follow explain each one; the rest of the pack quotes them by number.

MR-01 Every metric must answer a stated question tied to a decision that a named audience takes. A metric that informs no decision is not collected.

MR-02 Every metric must have an approved definition card before its first value is reported. The card is the only definition, and every document that shows the metric uses it.

MR-03 Every metric must have one owner, who interprets it and answers for it, and one data supplier for each input. Each is a person in a named role, never a team or a tool.

MR-04 Where another pack already defines a measure, the catalogue must use that definition and target unchanged. A change is made in the pack that owns it, never in the catalogue.

MR-05 Every input must pass the completeness, freshness and plausibility checks before a figure is reported. A figure that fails is corrected, reported with a printed caveat, or withheld. It is never reported silently, and never replaced by last period's value.

MR-06 The person who validates an input must not be the person who supplied it.

MR-07 Every KPI must have a target, and every KRI green, amber and red thresholds. A KRI's thresholds are set from the appetite and tolerance of the risk it indicates (P05) and approved by that risk's owner.

MR-08 A KRI that enters amber or red must be escalated as the escalation table sets, counting from the date its value is validated. A KRI amber for [[3]] periods in a row is escalated as red, unless its linked risk has an approved treatment plan that is on schedule (P05 RM-06, RM-07), recorded by the risk owner each period. If the plan slips, the KRI is escalated as red.

MR-09 A change to a metric's definition, source or thresholds must be recorded on its card with the date. Earlier values are restated on the new basis, or the break is marked and no trend is drawn across it.

MR-10 Metrics must be reported at the cadence of their audience: the security and IT leadership team monthly, the executive committee monthly, and the board quarterly through the P03 board report.

MR-11 Every metric must be reviewed once a year against the metric review questions. A metric that informed no decision in the last [[12]] months is retired, and its card kept with the reason.

MR-12 Every value must be shown with its target or thresholds, its direction since the last period and its status in words (P03 BR-04). Never colour alone, never a combined index, and never a score out of 100.

Principles

Most questions about why a metric is defined, checked or shown as it is are answered by one of these.

1. A metric exists for a decision. If nobody would decide anything differently when the number moves, it is not collected (MR-01). Interesting is not enough.
2. Define once, use everywhere. The card is the only definition. The dashboard, the committee report and the board report show the same number because they read the same card (MR-02).
3. The pack that runs the control owns the measure. Vulnerability, exception, risk, supplier, access, incident and exercise measures keep the words their own packs give them (MR-04). This pack adds only what it needs to run them.
4. Nobody checks their own homework. The person who supplies an input never validates it (MR-06).
5. Missing is never zero, and old is never new. A missing input is recorded as missing. Last period's value is never carried forward (MR-05).
6. Words beside every colour. Every value is shown with its target or thresholds, its direction and its status in words (MR-12).
7. Change the card before the number. A new definition, source or threshold is recorded first, and the series is restated or its break marked (MR-09).
8. Fewer, better metrics. A metric that informs no decision for a year is retired (MR-11). The catalogue should grow only as fast as it is pruned.

Inputs

The metric owner gathers these before writing a card. The Metric Definition Card Template has a field for each.

Scales and criteria

Four fixed lists make the method. They change only when this methodology is reviewed, never for one metric or one period.

Kinds of metric

Every metric is one of two kinds, and each is judged differently (MR-07).

• Key performance indicator (KPI). A metric that shows whether a control or process is working as intended, judged against a target. Its status is one of the four status words below.
• Key risk indicator (KRI). A metric that gives early warning that a risk in the risk register may be moving, judged against thresholds set from that risk's appetite and tolerance. Its status is one of the three zones below.
• Leading: an indicator that moves before harm is done, such as weaknesses past their fix deadline. Lagging: an indicator that moves only after harm is done, such as incidents. Useful, but on its own too late to warn. Each audience should see some of each.
• Metric or measure. The word P03 and the other packs use for a metric. In this pack the two mean the same.

Status words for a performance indicator

A KPI's status uses the words of the Board Reporting Narrative Model, so the dashboard, the committee report and the board report agree (MR-12, P03 BR-04).

A metric showing No target set does not reach any audience until a target is set (MR-07).

Zones for a risk indicator

A KRI's zone says what its value suggests about the linked risk, in the P03 position words. The colour always sits beside the words.

The zone says what the value suggests about the linked risk. It never changes the risk's position on its own: only a reassessment in the risk register does that (P05).

Reporting levels

Metrics must be reported at the cadence of their audience: the security and IT leadership team monthly, the executive committee monthly, and the board quarterly through the P03 board report.

Every metric in the catalogue goes to the operational level. A metric reaches the executive committee only if the committee decides on it, and the board only through the eight board measures the Board Metric Selection Catalogue selects.

Decision logic

The steps

1. Name the question and the decision the metric serves, and who takes it (MR-01). If another pack already defines the measure, use its definition and target unchanged (MR-04).
2. Write and approve the definition card before the first value is reported (MR-02): the four roles (MR-03, MR-06), and the target or the thresholds (MR-07).
3. Each period, collect and check every input for completeness, freshness and plausibility before any figure is used (MR-05).
4. Calculate the value, its direction and its status or zone, and write the commentary (MR-12).
5. Escalate any KRI in amber or red (MR-08).
6. Report to each audience at its cadence (MR-10).
7. Record every change to the card, and restate or mark the break (MR-09).
8. Review every metric once a year, and retire those that informed no decision (MR-11).

Specifying a metric

A metric answers a question for a decision

Every metric must answer a stated question tied to a decision that a named audience takes. A metric that informs no decision is not collected.

Write the question in the words the audience would use, and the decision as something a named person does: fund, reassess, escalate, change a plan, accept. If the decision cannot be written, the metric is not needed.

The Board Metric Selection Catalogue names 5 ways a metric misleads: Vanity, Gaming, Lagging, No target and Lacks context. The known limitations field on every card says which apply, and the Metric Definition Card Template gives each one's meaning.

The definition card

Every metric must have an approved definition card before its first value is reported. The card is the only definition, and every document that shows the metric uses it. The card has 25 fields, each serving a rule. The Metric Definition Card Template gives the blank card, guidance for each field and two completed cards.

• A card is approved by the metric owner and the head of the audience it goes to. A KRI's thresholds are also approved by the owner of the linked risk (MR-07).
• A metric with an empty field is not reported until the field is filled. Write "None" where a field does not apply, such as thresholds for a KPI.
• Cards are numbered with the catalogue ID (MC-01 to MC-40), and kept with the Security KPI & KRI Catalogue.

Reusing other packs' definitions

Where another pack already defines a measure, the catalogue must use that definition and target unchanged. A change is made in the pack that owns it, never in the catalogue. The catalogue imports 33 measures from 8 packs and adds 7 of its own where no pack covers the need.

• Not imported: P08's certification readiness measures (ISM-01 to ISM-04), which end with the project; P03's board measures, which the catalogue feeds and never copies.
• What "unchanged" means. The card's name, definition and target are copied word for word from the owning pack. P11 adds only the type, theme, audience, roles, the board measure it feeds and, for a KRI, the linked risk.
• To change an imported measure, change it in the pack that owns it, through that pack's review. Then copy the new words to the card, and record the change and its date (MR-09).
• This pack's own metrics (MC-34 to MC-40) are marked as P11's. Before adding one, search every pack's headline measures and the eight board measures. If one fits, import it.

Sourcing and ownership

Every metric must have one owner, who interprets it and answers for it, and one data supplier for each input. Each is a person in a named role, never a team or a tool. The person who validates an input must not be the person who supplied it. Four roles cover every metric:

The Metric Ownership Responsibility Matrix is checked against these rules:

• Exactly one Supplies per input, one Validates, one Interprets and one Reports per audience.
• Supplies and Validates are never the same person (MR-06).
• Interprets is a person in a named role who can explain the number to the audience, never a team or a tool (MR-03).
• A metric with an empty cell is not reported until the cell is filled (MR-02).

Naming the source

• Name the system, register or workbook each input comes from, and the report or query within it. "The scanner" is not a source; "the weekly authenticated scan report, all sites" is.
• Where two sources hold the same facts, choose one and name it. The other becomes the cross-check for the plausibility test (DQ-03).
• Where a provider supplies an input, the data supplier is the person in your organisation who receives it and answers for it. A provider is never the validator of its own figures.
• Record the extract date with every input. Freshness is checked on the date, never on whether the value looks current.

Calculation

The formula

• A ratio states its numerator and its denominator, and is shown as a count out of a count ("9 of 14"), with a percentage beside it only where the denominator is large. A percentage alone hides how few items there were.
• A count states what one item is: one incident case, one supplier, one account. It states the window: at the period end, or over the last three months.
• A time states its start and end events, the unit, and whether calendar or working days.
• Better when says whether a higher or lower value is better. Direction words follow from it.
• Every value is as at the period end: the last day of the calendar month, or of the quarter for a quarterly metric.

Direction and status

Every value must be shown with its target or thresholds, its direction since the last period and its status in words (P03 BR-04). Never colour alone, never a combined index, and never a score out of 100.

1. Compare the value with the last period's value at the same cadence. Say Better, Worse or Same, with the earlier value and its period: "Better (was 8, last month)". Across a break in series there is no direction: say "Not comparable" (MR-09).
2. For a KPI, compare the value with its target, then its tolerance, and give the status in words.
3. For a KRI, compare the value with its thresholds and give the zone in words.
4. The metric owner writes one or two sentences of commentary: why it moved, and what is being done. No commentary is needed for a value on target that did not change.

Cadence changes the direction. In the EXAMPLE, MC-17 Tier 1 suppliers assessed was 9 of 14 at 30 Sep 2026. The monthly reports compare it with August (8): Better (was 8, last month). The board report compares it with the last quarter end (6): Better (was 6, last quarter). Both are right; each says which period it compares with.

Setting a risk indicator's thresholds

Every KPI must have a target, and every KRI green, amber and red thresholds. A KRI's thresholds are set from the appetite and tolerance of the risk it indicates (P05) and approved by that risk's owner. The Key Risk Indicator Threshold & Escalation Model sets out the full method; in short:

1. Start from the linked risk in the risk register: its residual impact and likelihood, and its category's appetite level (P05).
2. Ask the risk owner: at what value of this indicator would you score the likelihood one step higher? At what value would you also score the impact higher?
3. Work out the residual score each answer would give, and its position against appetite and tolerance (P05).
4. The amber threshold is the first value at which the position would be Outside appetite, within tolerance. The red threshold is the first value at which it would be Outside tolerance.
5. Where the category's appetite and tolerance are the same band (the Open level), there is no amber zone: the KRI goes from green to red.
6. Where the linked risk is already outside appetite, the KRI starts in amber. Green is then the treatment's target, and red still marks tolerance.
7. The risk owner approves the thresholds, and they are written on the card with the reasoning. They are reviewed whenever the appetite statement changes (MR-09).

EXAMPLE: MC-34 phishing compromises, linked to R-09 staff account taken over through phishing.

Checking the data

Every input must pass the completeness, freshness and plausibility checks before a figure is reported. A figure that fails is corrected, reported with a printed caveat, or withheld. It is never reported silently, and never replaced by last period's value. Every input passes three checks, run by the validator:

A figure that fails a check has one of four outcomes, and only these:

How the checks run

1. The supplier puts each input in the Metrics Data Collection & Quality Workbook with its extract date, within 3 working days of the period end.
2. The validator runs the three checks on each input by working day 5, and records each as passed or failed, with a note.
3. A failed input goes back to the supplier to fix. The checks run again on the fixed input.
4. If it cannot be fixed in time, the metric owner decides: report it with a caveat that says what is wrong and which way the error runs, or withhold it as Not available with the reason.
5. Every failure is counted in MC-38, figures that failed a quality check, and shown to the executive committee (Executive Committee Security Report Deck, the data quality slide).

Escalating a risk indicator

A KRI that enters amber or red must be escalated as the escalation table sets, counting from the date its value is validated. A KRI amber for [[3]] periods in a row is escalated as red, unless its linked risk has an approved treatment plan that is on schedule (P05 RM-06, RM-07), recorded by the risk owner each period. If the plan slips, the KRI is escalated as red. The Key Risk Indicator Threshold & Escalation Model sets out who is told and what follows:

• A KRI entering amber or red is a trigger for the linked risk's reassessment in the Information Security Risk Register (P05): "a control that has failed, or is found not to work as the assessment assumed".
• Only a reassessment moves the risk. The zone never does on its own.
• Amber that persists. A KRI amber for 3 periods in a row is escalated as red, unless its linked risk has an approved treatment plan that is on schedule (P05 RM-06, RM-07), recorded by the risk owner each period. The treatment plan has at least one open action, and every open action is before its due date and not reported at risk by its owner. An open action passes its due date or is reported at risk, or the plan's funding or approval is refused. The KRI is then escalated as red in that period. In the EXAMPLE, MC-13 has been amber for 3 months while R-01 and R-02 are brought back within appetite; both plans were recorded on schedule on 7 Oct 2026, so it is not escalated as red.
• Where a reassessment puts a risk past tolerance, the board or its equivalent is told within 5 working days, out of cycle, as P03 requires.

Changing a metric without breaking the series

A change to a metric's definition, source or thresholds must be recorded on its card with the date. Earlier values are restated on the new basis, or the break is marked and no trend is drawn across it.

• Marking a break. Record the date and the reason on the card. On every chart and table, draw a line at the break, and never join values across it. Say "not comparable" in place of a direction for the first period after it.
• Restating. Recalculate earlier values from the source on the new basis, record on the card which periods were restated, and show only the restated series.

Two illustrations on the EXAMPLE

A definition change that can be restated. Suppose the distributor counted phishing compromises over one month in place of three. The case log keeps each incident's detection date, so earlier values can be restated:

The restated series is shown alone, and the change and the restated periods go on the card. The thresholds would need setting again for the new window (MR-07).

A threshold change that cannot be restated. Suppose the board moved People and insider from Cautious to Averse (appetite Low, tolerance Medium). R-09, at Medium, would then be outside appetite, within tolerance, so MC-34 would start in amber and green would become the treatment's target (the threshold method). The counts stay comparable; the zones before and after do not, so the break is marked on the zone line.

Cadence by audience

Each month runs to the same cycle, in working days after the period end. These are EXAMPLE defaults: set your own, keeping the order. The dates are the EXAMPLE's September 2026 cycle.

In the last month of a quarter the same figures feed the board report. The Board & Executive Reporting Calendar (P03) works back from the board meeting: data returned by 5 Oct 2026 and figures checked by 8 Oct 2026 for the board on 29 Oct 2026. The monthly cycle has the inputs in by 5 Oct 2026 and the checked figures by 7 Oct 2026, in time for both. The board and the executive committee therefore see the same numbers, as P03 BR-08 requires: "All figures come from one data collection workbook, checked and signed off before the deck is built."

• A metric shown to two audiences uses the same value and the same card at both. Only the level of detail and the commentary differ.
• A metric its owning pack defines over a quarter (MC-25, MC-26 and MC-28) is reported in the last month of each quarter only, at every level, and compared with the last quarter.
• A metric worked out from this pack's own logs (MC-38 and MC-39) has no data supplier and is not collected. MC-39 is reported one cycle in arrears: a KRI's risk review can finish after the month's report is sent, so each report counts the escalations started in the month before.

Reviewing and retiring metrics

Every metric must be reviewed once a year against the metric review questions. A metric that informed no decision in the last [[12]] months is retired, and its card kept with the reason. Each metric is reviewed against six questions:

The answers decide the outcome:

• Retiring. Stop collecting the metric, mark the card Retired with the date and the reason, and keep it. If the metric is ever proposed again, the card shows why it was dropped.
• Replacing. Where a dropped metric pointed at a real need, a better metric for the same decision may replace it, with a new card and a new ID. It is never the old metric renamed.
• The review is recorded in the Metrics Programme Maturity Self-Check, which also checks the programme as a whole.

Worked examples

The examples are from the EXAMPLE organisation used throughout this library: a wholesale distributor with about 900 staff, three warehouses and an online ordering service that takes 60% of its orders. Its September 2026 cycle ran as the cadence section shows: inputs by 5 Oct 2026, checks by 7 Oct 2026, the executive committee on 14 Oct 2026. Two inputs failed their checks and were corrected before anything was reported, and one KRI went amber. Every value below is worked out from the other packs' EXAMPLE facts by the rules above.

Example 1: a stale supplier register export (QF-01)

MC-17 Tier 1 suppliers assessed is imported unchanged from P06 (TPM-01): "Tier 1 suppliers with an assessment within the last 12 months, out of all Tier 1 suppliers." Target: All Tier 1 by [[year end]]. It feeds the board measure BM-05.

Example 2: a live incident counted as an exercise (QF-02)

MC-28 Months since the plan was last exercised is imported unchanged from P09 (IMM-04): "Whole months from the last exercise of the plan (not a live incident) to the end of the quarter." Target: 12 or fewer.

What the two examples show. Neither error would have been caught by looking at the number. The first needed the extract date; the second needed the definition and a cross-check with another pack. Both failures went into MC-38 for September: 2 failed; 2 corrected, 0 reported with a caveat or withheld, status On target.

Example 3: the phishing indicator goes amber

MC-34 Phishing compromises is one of this pack's own metrics: "Incidents in the last 3 calendar months whose root cause was a phishing email: an account taken over, or malware run from an attachment or link. Counted from the incident case log by the date detected." Its thresholds are set from R-09 (see "Setting a risk indicator's thresholds").

What it shows. The KRI warned; the risk owner judged. Two compromises in three months crossed the amber threshold, and the review found both contained within hours, so R-09 stayed within appetite. The committee still agreed an action, and the KRI stays on its report until it is green again.

Example 4: reviewing the metrics used before

When it adopted this method, on 26 Jun 2026, the distributor reviewed the 6 metrics its IT team had been reporting, each against MQ-1 to MQ-6 (MR-11):

What it shows. Three were dropped because they informed no decision (MQ-1), whatever else they did well. A dropped metric may be replaced by a better one for the same need, with its own card: "Backup jobs completed successfully" by MC-35. Metrics that informed decisions but had gaps were fixed, here by moving each to the catalogue metric that answers the same question, with its card: "Access reviews completed" to MC-22 and "Security training completion" to MC-36. One was kept as it was.

How to defend the method to an auditor

An auditor will usually test that the organisation has decided what needs to be monitored and measured, the methods that make the results valid, when monitoring and measuring take place, who does them, and when and by whom the results are analysed and evaluated; and that it keeps evidence of the results (ISO/IEC 27001 clause 9.1). The definition card answers each of these for each metric.

Evidence to have ready

• This methodology, approved, and the catalogue of metrics in use.
• An approved definition card for every reported metric, with its change log (MR-02, MR-09).
• The Metric Ownership Responsibility Matrix, with no empty cells and no supplier who validates their own input (MR-03, MR-06).
• For each of the last [[three]] periods, the Metrics Data Collection & Quality Workbook: every input with its extract date, each check passed or failed, and every outcome (MR-05).
• The escalation record for every KRI in amber or red, with dates (MR-08).
• The last annual metric review, with each metric's answers and outcome (MR-11).

The re-derivation test

Invite the auditor to pick [[3]] figures from the last executive committee report. For each, take the inputs from the collection workbook and the formula from the card, and recalculate the value, the direction and the status or zone. Each should match the report. Then check the extract dates against the period end, and that the validator was not the supplier. Run this test yourself each quarter, so the first time is not in front of the auditor.

Questions auditors commonly ask

Limitations

No measurement method is complete. These are this one's known weaknesses, and how they are managed.

Calibration and review

Calibrating validators

1. Before their first cycle, and every year after, each validator works through the cases below on their own.
2. Compare with the expected outcome and discuss each difference. A pattern of differences shows a check or an outcome that needs clearer wording: change it here, and record the change below.
3. Where the validator and the metric owner disagree on reporting with a caveat or withholding, the metric owner decides and the reason is recorded.

Validator calibration cases

Every month

The Head of Information Security reports four measures of the programme itself to executive management with the other metrics: MC-37 metrics reported with an approved card (target: all); MC-38 figures that failed a quality check (target: none reported with a caveat or withheld); MC-39 KRI escalations made on time (target: all); MC-40 metrics reviewed in the last 12 months (target: all). The Security KPI & KRI Catalogue defines each. In the EXAMPLE for September 2026, MC-38 was 2 failed; 2 corrected, 0 reported with a caveat or withheld, and MC-39 was None due.

Treat these as signals that the method needs attention: the same input failing a check two periods running, any figure reported with a caveat or withheld two periods running, and a KPI or KRI that changes status or zone when the owner's commentary cannot say why.

Review

This methodology is reviewed at least every 12 months, and also when the appetite statement changes, when a pack whose measures are imported changes them, or when the monthly signals show the same problem twice. The review is approved by executive management. A change to the status words, the zones or the checks affects every metric: record it, and mark the break where results before and after are not comparable (MR-09).

Calibration record

Related documents

Adapting this template

Framework references

These references show where this document supports an external framework. They indicate relevance only and do not reproduce the text of any standard. Editions referenced: ISO/IEC 27001:2022 incl. Amd 1:2024; NIST CSF 2.0; Directive (EU) 2022/2555 (NIS2) and Implementing Regulation (EU) 2024/2690; Regulation (EU) 2022/2554 (DORA).

Definitions