Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

Metrics Data Collection & Quality Workbook

Collects metric inputs per period with completeness, freshness and plausibility checks before anything is reported.

Available soon

Format
Excel
Size
93 KB
Length
12 sheets
Version
1.1
Updated

What's inside

What changed

  1. 1.1

    MC-35 shows the ordering platform's monthly restore check (P10 EA-04): 0 of 4.

  2. 1.0

    Published.

Changes across this pack

Preview

The workbook's sheets as you will receive them, with its example rows and the values its formulas calculate. Highlighted [[text]] is for you to replace; rows marked EXAMPLE are for you to delete. Wide sheets scroll sideways. The cover, document control and changelog sheets are in the file.

Instructions

How to use this workbook

StepWhat to do
1Save a copy of this workbook for each month, named for it (for example 'Metrics 2026-09'). Keep every copy: together they show what was checked, what failed and what was reported.
2Cycle: enter the period and its last day, and the date you are checking as the as-at date. The timetable counts 6 steps in working days after the period end, from the Security Metric Definition & Measurement Method. Enter your public holidays so they are skipped. Type the as-at date each time you check; do not use a formula that changes it.
3Metric Checks: one row per metric you collect, with the quality-check fields from its definition card (Metric Definition Card Template): the supplier and the validator (never the same person, MR-06), the source, what one item in scope is, the scope count, the plausible range, and anything you cross-check it against.
4Inputs: by the supply deadline (3 working days after the period end), each data supplier enters one row per metric: the extract date, the number of items with a value, the value, and last period's value from the previous copy. A missing input is left blank, never entered as zero (DQ-01).
5By the checks deadline (5 working days), the validator answers 'Follows the definition', enters any cross-check value and whether a large change was explained, and adds a note. The three checks and the result calculate. A share is entered as a number from 0 to 1.
6An input that fails is never overwritten. Record the failure on the Quality Log, then either add a new row with the corrected input and the next submission number (the checks run again), or record that the metric owner agreed to report it with a caveat, or that it is withheld (MR-05).
7Results: each metric's result. Ready to report goes to the Operational Security Metrics Dashboard. A held metric is shown there as "Not available", with the reason: never last period's value. Copy the MC-38 line into the dashboard.
8To clear the EXAMPLE, delete the contents of the 'Row' column and of every yellow column on Metric Checks, Inputs and the Quality Log, and the period on Cycle.

Legend

Yellow cells are inputs. Everything else is calculated or fixed — do not overwrite formulas.

EXAMPLERows marked EXAMPLE in the first column show how a completed row looks. Delete them before approval.

Rule MR-05: "Every input must pass the completeness, freshness and plausibility checks before a figure is reported. A figure that fails is corrected, reported with a printed caveat, or withheld. It is never reported silently, and never replaced by last period's value." Rule MR-06: "The person who validates an input must not be the person who supplied it."

DQ-01 Completeness: Every input the card names is supplied for every item in scope: every system, site, supplier or risk the card lists. A missing input is recorded as missing, never as zero.

DQ-02 Freshness: Each input is taken on or after the period end, and no later than [[3]] working days after it. An older extract is replaced before use, never adjusted.

DQ-03 Plausibility: The value follows the card's definition, sits within the range the card sets, and agrees with related figures, including another pack's count of the same things. A change from last period larger than the card allows is explained by the supplier before it is accepted.

EXAMPLE: a wholesale distributor with about 900 staff, three warehouses and an online ordering service that takes 60% of its orders. Its September 2026 cycle, checked as at 7 Oct 2026. Two inputs failed and were corrected before anything was reported. QF-01 (freshness): The supplier register export was dated 31 Aug 2026: August's file, used again. The pallet network (SUP-009) was assessed on 15 Sep, after the old export. The fresh export counts it. QF-02 (plausibility): The last exercise was entered as 12 Aug 2026: the ransomware incident, INC-2026-014. A live incident is never an exercise (P09 IMM-04). The date must be a held exercise in the P10 programme: EX-2026-01 on 18 Mar 2026.

Not collected here: MC-38, derived from the Quality Log of the collection workbook: this workbook's own result, the figures that failed a check this period. Not supplied, so not checked for freshness. MC-39, derived from the escalation log, and reported one cycle in arrears: the risk owner's review can fall after the month's report is sent. The September 2026 report shows August's escalations. Not supplied, so not checked for freshness.

Tailoring — small organisation: one person may supply most inputs. The validator must still be someone else; a finance or operations manager who can read the source is enough. Keep the three checks: they take minutes once the card sets the scope count and range.

Tailoring — regulated entity (NIS2, DORA): the checks are the method that ensures valid results (Implementing Regulation (EU) 2024/2690, Annex 7.2). Keep each month's copy, with its Quality Log, as the record of how the figures were produced.

Tailoring — IT run by a service provider: the provider will supply several inputs. Put the extract deadline and the definitions in the service agreement, and validate its figures yourself: a provider never validates its own input (MR-06).

Cycle

Cycle — the period, the as-at date and the timetable

Yellow cells are yours. The deadlines count working days after the period end; the as-at date is typed, so a saved copy reads the same on any day.

SettingValue
PeriodSeptember 2026EXAMPLE — your month
Period end (the last day of the month)30 Sep 2026
As-at date (the day you check)7 Oct 2026EXAMPLE — the day the checks were finished
Supply deadline (calc)5 Oct 2026
Checks deadline (calc)7 Oct 2026
Ready to report? (from Results, calc)Yes: every metric is ready to report

Timetable — working days after the period end

StepWhoWorking days after period endDue date (calc)Completed onStatus (calc)
Inputs supplied in the collection workbook, each with its extract dateData supplier35 Oct 20265 Oct 2026Done
Quality checks run; each input accepted, corrected or rejected (MR-05, MR-06)Validator57 Oct 20267 Oct 2026Done
KRI zones and KPI statuses set; commentary written; escalations started (MR-08)Metric owner68 Oct 2026Not yet due
Operational dashboard reviewed by the security and IT leadership teamReporter79 Oct 2026Not yet due
Executive committee report sentReporter812 Oct 2026Not yet due
Executive committee meetingReporter1014 Oct 2026Not yet due

Working days are Monday to Friday, less the non-working days below. In the last month of a quarter the same figures feed the board report: these deadlines fall before its data-return and sign-off dates.

Non-working days (public holidays)

DateHoliday

[[e.g. a public holiday in the first two weeks of the month]]

Metric Checks

One row per metric you collect: the quality-check fields of its definition card (Metric Definition Card Template). Yellow columns are yours. The 17 EXAMPLE rows are the example organisation's.

RowMetric IDMetric (calc)Type (calc)Data supplier (role)Validator (role)Data sourceOne item in scope isScope countPlausible fromPlausible toLargest change accepted without explanationCross-check againstCard check (calc)
EXAMPLEMC-13Risks outside appetite and past toleranceKRIAssessorHead of Information SecurityP05 registerTop risk in the risk register12012OK
EXAMPLEMC-14Treatment actions overdueKPIHead of RiskAssessorP05 treatment actionsTreatment action in the risk treatment plan404OK
EXAMPLEMC-15Acceptances past reviewKPIAssessorHead of Information SecurityP05 acceptancesRisk acceptance101OK
EXAMPLEMC-17Tier 1 suppliers assessedKPIProcurement ManagerAssessorP06 supplier registerTier 1 supplier in the supplier register14014OK
EXAMPLEMC-19High findings past deadlineKPIAssessorHead of Information SecurityP06 findingsFinding raised with a supplier404OK
EXAMPLEMC-20ConcentrationKRIProcurement ManagerAssessorP06 supplier registerTier 1 supplier in the supplier register, with the critical services it supports14014OK
EXAMPLEMC-22Removals confirmed on timeKPISystem administratorCampaign coordinatorP07 campaign UAR-2026-Q3Revoke or Modify decision in the access review69069OK
EXAMPLEMC-24Process weaknesses openKPICampaign coordinatorHead of Information SecurityP07 campaign UAR-2026-Q3Process weakness sent to a process owner202OK
EXAMPLEMC-25Significant incidents and time to containKPIIncident managerHead of Information SecurityP09 case logCase in the incident case log for the quarter505OK
EXAMPLEMC-26Reports made on timeKPIIncident managerData Protection OfficerP09 reportsReport to an authority or individual that fell due404OK
EXAMPLEMC-27Review actions overdueKPIIncident managerHead of Information SecurityP09 review actionsPost-incident review action606OK
EXAMPLEMC-28Months since the plan was last exercisedKPIIncident managerHead of Information SecurityP09 and P10 exercise programmeExercise in the exercise programme60The last held exercise in the exercise programme (never a live incident)OK
EXAMPLEMC-29Audiences exercised within their intervalKPIExercise leadExercise sponsorP10 exercise programmeExercise audience (AU-01 to AU-04)404OK
EXAMPLEMC-31Exercise actions overdueKPIExercise leadExercise sponsorP10 improvement actionsImprovement action from an exercise606OK
EXAMPLEMC-34Phishing compromisesKRIIncident managerHead of Information SecurityP09 case logCase in the incident case log for the last 3 months505OK
EXAMPLEMC-35Critical systems without a checked restoreKRIIT Operations ManagerInformation Security ManagerBackup restore recordsSystem that supports a critical service404OK
EXAMPLEMC-36Staff security training completionKPIHR Learning LeadInformation Security ManagerTraining records (P03 BM-08)Member of staff in post90001OK

Inputs

One row per metric and submission for September 2026. Suppliers fill the first yellow columns; the validator fills 'Follows the definition' to the note. The three checks and the result calculate. EXAMPLE rows: the example's September inputs.

RowMetric IDMetric (calc)SubmissionSupplied byValidated byExtract dateItems with a valueValueLast period's valueFollows the definitionCross-check valueChange explainedValidated onValidator's noteScope count (calc)Plausible from (calc)Plausible to (calc)Largest change (calc)Completeness (calc)Freshness (calc)Plausibility (calc)Plausibility reason (calc)Roles, MR-06 (calc)Result (calc)Latest submission (calc)
EXAMPLEMC-13Risks outside appetite and past tolerance1AssessorHead of Information Security30 Sep 20261223Yes7 Oct 202612012PassPassPassOKReady to reportYes
EXAMPLEMC-14Treatment actions overdue1Head of RiskAssessor30 Sep 202640Yes7 Oct 2026404PassPassPassOKReady to reportYes
EXAMPLEMC-15Acceptances past review1AssessorHead of Information Security30 Sep 202610Yes7 Oct 2026101PassPassPassOKReady to reportYes
EXAMPLEMC-17Tier 1 suppliers assessed1Procurement ManagerAssessor31 Aug 20261488Yes7 Oct 2026QF-01: The supplier register export was dated 31 Aug 2026: August's file, used again.14014PassFailPassOKHeldNo
EXAMPLEMC-17Tier 1 suppliers assessed2Procurement ManagerAssessor1 Oct 20261498Yes7 Oct 2026QF-01 corrected: The pallet network (SUP-009) was assessed on 15 Sep, after the old export. The fresh export counts it.14014PassPassPassOKReady to reportYes
EXAMPLEMC-19High findings past deadline1AssessorHead of Information Security30 Sep 2026411Yes7 Oct 2026404PassPassPassOKReady to reportYes
EXAMPLEMC-20Concentration1Procurement ManagerAssessor30 Sep 2026146Yes7 Oct 202614014PassPassPassOKReady to reportYes
EXAMPLEMC-22Removals confirmed on time1System administratorCampaign coordinator30 Sep 20266965Yes7 Oct 202669069PassPassPassOKReady to reportYes
EXAMPLEMC-24Process weaknesses open1Campaign coordinatorHead of Information Security30 Sep 202620Yes7 Oct 2026202PassPassPassOKReady to reportYes
EXAMPLEMC-25Significant incidents and time to contain1Incident managerHead of Information Security30 Sep 202651Yes7 Oct 2026505PassPassPassOKReady to reportYes
EXAMPLEMC-26Reports made on time1Incident managerData Protection Officer30 Sep 202644Yes7 Oct 2026404PassPassPassOKReady to reportYes
EXAMPLEMC-27Review actions overdue1Incident managerHead of Information Security30 Sep 202661Yes7 Oct 2026606PassPassPassOKReady to reportYes
EXAMPLEMC-28Months since the plan was last exercised1Incident managerHead of Information Security30 Sep 202661No67 Oct 2026QF-02: The last exercise was entered as 12 Aug 2026: the ransomware incident, INC-2026-014.60PassPassFailDoes not follow the definitionOKHeldNo
EXAMPLEMC-28Months since the plan was last exercised2Incident managerHead of Information Security30 Sep 202666Yes67 Oct 2026QF-02 corrected: A live incident is never an exercise (P09 IMM-04). The date must be a held exercise in the P10 programme: EX-2026-01 on 18 Mar 2026.60PassPassPassOKReady to reportYes
EXAMPLEMC-29Audiences exercised within their interval1Exercise leadExercise sponsor30 Sep 202642Yes7 Oct 2026404PassPassPassOKReady to reportYes
EXAMPLEMC-31Exercise actions overdue1Exercise leadExercise sponsor30 Sep 202661Yes7 Oct 2026606PassPassPassOKReady to reportYes
EXAMPLEMC-34Phishing compromises1Incident managerHead of Information Security30 Sep 2026521Yes7 Oct 2026505PassPassPassOKReady to reportYes
EXAMPLEMC-35Critical systems without a checked restore1IT Operations ManagerInformation Security Manager30 Sep 202640Yes7 Oct 2026404PassPassPassOKReady to reportYes
EXAMPLEMC-36Staff security training completion1HR Learning LeadInformation Security Manager30 Sep 20269000.92Yes7 Oct 202690001PassPassPassOKReady to reportYes

Quality Log

Every input that failed a check, and what happened to it (MR-05). Outcomes: Corrected; Reported with a caveat; Withheld. Last period's value is never carried forward.

RowFailure IDMetric IDMetric (calc)Check failedWhat was foundValue suppliedCorrected valueWhy, and the correctionOutcomeCaveat printed beside the valueCheck (calc)
EXAMPLEQF-01MC-17Tier 1 suppliers assessedDQ-02 FreshnessThe supplier register export was dated 31 Aug 2026: August's file, used again.89The pallet network (SUP-009) was assessed on 15 Sep, after the old export. The fresh export counts it.CorrectedOK
EXAMPLEQF-02MC-28Months since the plan was last exercisedDQ-03 PlausibilityThe last exercise was entered as 12 Aug 2026: the ransomware incident, INC-2026-014.16A live incident is never an exercise (P09 IMM-04). The date must be a held exercise in the P10 programme: EX-2026-01 on 18 Mar 2026.CorrectedOK

Results

Results — what can be reported this period

Everything here is calculated. A metric is ready to report when its latest submission passed every check; a held metric goes to the Operational Security Metrics Dashboard as "Not available", with the reason.

Summary for the period

CountMetrics
Metrics with a card on Metric Checks17
Ready to report17
Ready to report, with the caveat0
Held, not yet checked, or not supplied0
MC-38 Figures that failed a quality check2
of which corrected2
of which reported with a caveat0
of which withheld0
MC-38 for the dashboard2 failed; 2 corrected, 0 reported with a caveat or withheld
Ready to report?Yes: every metric is ready to report

Result per metric

Metric IDMetricSubmissionsSubmissions heldLatest submissionOutcomeResultValue to report
MC-13Risks outside appetite and past tolerance10Ready to reportPassedReady to report2
MC-14Treatment actions overdue10Ready to reportPassedReady to report0
MC-15Acceptances past review10Ready to reportPassedReady to report0
MC-17Tier 1 suppliers assessed21Ready to reportCorrectedReady to report9
MC-19High findings past deadline10Ready to reportPassedReady to report1
MC-20Concentration10Ready to reportPassedReady to report6
MC-22Removals confirmed on time10Ready to reportPassedReady to report65
MC-24Process weaknesses open10Ready to reportPassedReady to report0
MC-25Significant incidents and time to contain10Ready to reportPassedReady to report1
MC-26Reports made on time10Ready to reportPassedReady to report4
MC-27Review actions overdue10Ready to reportPassedReady to report1
MC-28Months since the plan was last exercised21Ready to reportCorrectedReady to report6
MC-29Audiences exercised within their interval10Ready to reportPassedReady to report2
MC-31Exercise actions overdue10Ready to reportPassedReady to report1
MC-34Phishing compromises10Ready to reportPassedReady to report2
MC-35Critical systems without a checked restore10Ready to reportPassedReady to report0
MC-36Staff security training completion10Ready to reportPassedReady to report0.92

Value to report is the latest submission's value, in the metric's unit. "Corrected": a check failed and the corrected input passed. Held: the metric owner decides on the Quality Log whether to report it with a caveat or withhold it.

Lists

CatIDCatNameCatTypeYesNoCheckFailedQualityOutcome
MC-01Scan coverageKPIYesDQ-01 CompletenessCorrected
MC-02Mean time to remediate, by priorityKPINoDQ-02 FreshnessReported with a caveat
MC-03Deadline adherenceKPIDQ-03 PlausibilityWithheld
MC-04Overdue exposureKRI
MC-05Open exceptions by bandKPI
MC-06Expired exceptionsKPI
MC-07Chronic exceptionsKPI
MC-08Unmitigated SoD conflictsKRI
MC-09Documents past review dateKPI
MC-10Documents in force without approvalKPI
MC-11Acknowledgement coverageKPI
MC-12Overdue acknowledgementsKPI
MC-13Risks outside appetite and past toleranceKRI
MC-14Treatment actions overdueKPI
MC-15Acceptances past reviewKPI
MC-16Risks not reviewed this quarterKPI
MC-17Tier 1 suppliers assessedKPI
MC-18Reassessments overdueKPI
MC-19High findings past deadlineKPI
MC-20ConcentrationKRI
MC-21Reviews on timeKPI
MC-22Removals confirmed on timeKPI
MC-23Access that should not have existedKRI
MC-24Process weaknesses openKPI
MC-25Significant incidents and time to containKPI
MC-26Reports made on timeKPI
MC-27Review actions overdueKPI
MC-28Months since the plan was last exercisedKPI
MC-29Audiences exercised within their intervalKPI
MC-30Objectives met at the last exerciseKPI
MC-31Exercise actions overdueKPI
MC-32Re-tests outstandingKPI
MC-33Critical suppliers exercisedKPI
MC-34Phishing compromisesKRI
MC-35Critical systems without a checked restoreKRI
MC-36Staff security training completionKPI
MC-37Metrics reported with an approved cardKPI
MC-38Figures that failed a quality checkKPI
MC-39KRI escalations made on timeKPI
MC-40Metrics reviewed in the last 12 monthsKPI

Columns A to C are the metrics of the Security KPI & KRI Catalogue (MC-01 to MC-40). Yellow rows: add your own metrics with an ID, name and type.

Definitions

Definitions

TermMeaning in this workbook
MetricA number defined once, on a definition card, that answers one question for one audience: what is counted, how, from which source, by whom and how often.
ValueThe figure a metric gives for one period, as at the period end.
Data qualityWhether an input can be relied on. This pack checks three things: completeness, freshness and plausibility.
DQ-01 CompletenessEvery input the card names is there, for the whole scope the card sets. Test: The count of items with a value equals the scope count on the card.
DQ-02 FreshnessThe input is as at the period end, taken within the time the card allows. Test: The extract date is recorded, and falls between the period end and the supply deadline.
DQ-03 PlausibilityThe value could be true: it follows the definition, sits in a believable range and agrees with related figures. Test: Definition, range, cross-check and change against last period, each passed or failed with a note.
Data supplierThe person who provides an input to a metric each period, from a named source.
ValidatorThe person who runs the quality checks on an input and accepts or rejects it. Never the person who supplied it.
Metric ownerThe person who interprets a metric, writes the commentary and answers for it. One per metric.
Period endThe last day of the month the figures describe. Every input is as at this date.
Supply deadlineThe last day an input may be taken and supplied: 3 working days after the period end. An extract taken before the period end, or after this date, fails freshness.
As-at dateThe day you check the workbook, typed in. The timetable's status is judged at this date.
Scope countThe number of items the card says a metric covers, such as top risks or Tier 1 suppliers. Completeness compares the items with a value with it.
Cross-check valueThe same thing counted from another source, such as another pack's register. A value that disagrees with it fails plausibility.
Submission1 for the first input of a metric in the period; 2, 3 for each corrected input. The checks judge every submission; the latest one counts.
Outcome — PassedEvery check passed. The figure is reported.
Outcome — CorrectedA check failed. The supplier fixed the input, and the checks passed on the fixed input before the figure was reported.
Outcome — Reported with a caveatA check failed and the input could not be fixed in time. The metric owner agreed to report it, and the caveat is printed beside the value.
Outcome — WithheldA check failed and the figure could mislead. It is shown as "Not available" with the reason. Last period's value is never carried forward.
CaveatA short note printed beside a value that failed a check and could not be fixed in time, saying what is wrong and which way the error runs.
HeldNot reported as it stands: a check failed, or roles break MR-06. The Quality Log records whether it is corrected, reported with a caveat or withheld.
MR-nn, DQ-nn, MC-nnRules, data quality checks and metrics defined in the Security Metric Definition & Measurement Method and the Security KPI & KRI Catalogue.
(calc)A column or cell the workbook calculates. Do not type or paste over it.
EXAMPLEA worked example: a wholesale distributor with about 900 staff, three warehouses and an online ordering service that takes 60% of its orders, September 2026. Delete before use.

Framework References

Framework references

These references indicate relevance only and do not reproduce the text of any standard.

FrameworkReferenceSupported by
ISO/IEC 27001:2022Clause 9.1 — Monitoring, measurement, analysis and evaluationThe workbook as a whole: what is measured, who supplies and who checks it, when, and the checks that make results valid
NIST CSF 2.0GV.OV-03 — “Organizational cybersecurity risk management performance is evaluated and reviewed for adjustments needed”Results: only figures that passed their checks are used to evaluate performance
NIS2 — Directive (EU) 2022/2555Article 21(2)(f) — “policies and procedures to assess the effectiveness of cybersecurity risk-management measures”The checks as part of the procedures to assess whether measures are effective
NIS2 — Implementing Regulation (EU) 2024/2690Annex, point 7.2 — taking into account the risk assessment and past significant incidents, determine what is monitored and measured, the methods “to ensure valid results”, when it is measured, who measures, when the results are analysed and evaluated, and who analyses and evaluates themMetric Checks and Inputs: what is measured, who measures, when, and the methods that ensure valid results

Editions referenced: ISO/IEC 27001:2022 incl. Amd 1:2024; NIST CSF 2.0; Directive (EU) 2022/2555 (NIS2) and Implementing Regulation (EU) 2024/2690; Regulation (EU) 2022/2554 (DORA)

Prepared by Zero One Cybersecurity · Published by CISO Times

CISO Times Resource Licence — Internal Use