Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

If…, then how…

If a re-assessment shows a lower level than last time, then what do we do?

5 decision pointsThe pack behind this answer: Security Maturity & Internal Assessment

Add to board memo

The short answer

Recommendation
Check first that the two results can be compared. If they can, the lower level is the result and the fall is a finding: find out why, agree a new action one level up from the level found, re-order the plan, and report the fall with its reason.
Reasoning
Results compare only like with like: the same criteria, scope, evidence window and sampling. Any change is a break in series, and no fall is shown across it (rule MA-15). Without a break, the lower level stands: do not keep the earlier one, average the two or explain the fall away. A fall can be real decay, or a sign that the earlier level rested on evidence that should not have been accepted. Either way the practice is planned from where it is now.
Considerations
  • If you cannot show that nothing changed, mark a break. Still ask what changed in the practice, so a real fall does not hide behind the break.
  • Where the earlier level was too generous, the practice did not fall. Put the case in the calibration record so assessors judge the same evidence the same way next time.
  • An improvement action whose re-assessment finds a lower level is Not achieved, and a second Not achieved goes to the approver: the board or a board risk committee it has delegated to, the management board in a two-tier structure, or executive management where there is no board.
  • A re-assessment by someone who owns or runs the practice is self-assessed and cannot confirm a level above Defined.

The decision, step by step

Select any step for the reasoning behind it and where the pack covers it.

  • Start
  • Step
  • Decision
  • Outcome
StartA re-assessment shows a lower level than last timeDecisionWere the criteria, scope, window and sampling the same?StepMark a break in series beside each result it affectsOutcomeTreat the new level as a fresh baseline against the targetStepTreat the fall as a finding: the lower level is the resultDecisionWas this the re-assessment of an improvement action?StepSet the action to Not achieved, with the level foundDecisionIs this the action's second Not achieved?StepTake it to the approverDecisionDid the practice decay, or was the last level too generous?StepRecord it as a calibration problem for the assessorsStepFind what lapsed and who now runs the practiceStepAgree a new action one level up from the level foundStepRe-order the plan: a larger gap may now take priorityDecisionDid the fall follow a significant incident or major change?StepRe-assess the rest of the domain nowOutcomeReport the fall, its reason and the new action

Keyboard: Tab to a step, arrow keys move along the flow, Enter pins its explanation, + and − zoom, 0 fits the diagram, Esc closes. The text version below has the same content.

Where a step quotes a number, it is the default in the CISO Times pack. Your own standard may set a different one.

Read as text

The same flow written out: each step, its reasoning and where the pack covers it. 17 steps.

  1. Start

    A re-assessment shows a lower level than last time

    It may come from the next full assessment, from re-assessing a domain after an incident, a major change or a new obligation, or from re-assessing a practice when its improvement action was marked ready. A fall can be real decay, or a sign that the earlier level was too generous. Either way it is reported with its reason, never smoothed over or hidden in a total.

    In the pack

  2. Decision

    Were the criteria, scope, window and sampling the same?

    Results compare only like with like. A change to a practice's evidence criteria, a domain added or removed, a different evidence window or a different sample size is a break in series. If you cannot show that nothing changed, take the cautious branch.

    In the pack

  3. Step

    Mark a break in series beside each result it affects

    Say in the report that the result is not comparable, and why, beside each result the change touches. Show no change in level across the break. This is how a changed metric is marked too: the break is recorded, and no trend is drawn across it.

    In the pack

  4. Outcome

    Treat the new level as a fresh baseline against the target

    The new result is still the practice's level. If it is below the domain's target it is a finding and needs an action, ordered with the others. Compare it with later results on the same basis. If you took this branch because you could not tell whether anything changed, it is still worth asking what changed in the practice: a real fall should not hide behind a break.

    In the pack

    See also

  5. Step

    Treat the fall as a finding: the lower level is the result

    A fall without a break in series is a finding. The lower level is the result: record it as the level found, and do not keep the earlier one or average the two. The practice's level, and its domain's, follow this re-assessment. A report that explains a fall away is sent back.

    In the pack

  6. Decision

    Was this the re-assessment of an improvement action?

    When an owner marks an action ready, someone other than the owner checks the practice against the level the action aimed for. A lower level than before means the action did not reach its aim.

    In the pack

  7. Step

    Set the action to Not achieved, with the level found

    The action is not closed. Record who re-assessed it, when, and the level found, and set it to Not achieved. The practice's level follows this re-assessment. Then agree what is missing and a new due date, or record the approver's decision to stop.

    In the pack

  8. Decision

    Is this the action's second Not achieved?

    One miss can be timing: a practice that has only just changed may not yet have left the records its level needs. The pack takes an action that reaches Not achieved twice to the approver.

    In the pack

  9. Step

    Take it to the approver

    The pack has the Head of Information Security escalate an action that reaches Not achieved twice to the approver your governance sets: the board, or a board risk committee it has delegated to; the management board in a two-tier structure; or executive management where there is no board. Bring the reason and a proposal: more effort, a new date, or stopping.

    In the pack

  10. Decision

    Did the practice decay, or was the last level too generous?

    Ask what changed: a person who left, a tool replaced, a process dropped under pressure. Or the earlier result rested on evidence that was accepted and should not have been. If you cannot tell, treat it as decay, which needs the practice fixed.

    In the pack

  11. Step

    Record it as a calibration problem for the assessors

    If last time's evidence should not have supported the level, the practice did not fall; the earlier result was wrong. Put the case in the calibration record, so assessors judge the same evidence the same way next time. The new level still stands.

    In the pack

    See also

  12. Step

    Find what lapsed and who now runs the practice

    A practice that depended on one person, or that was dropped when the team was busy, stops leaving the records its level needs. Find the change behind it, so the new action fixes the cause and not only the records.

    In the pack

  13. Step

    Agree a new action one level up from the level found

    Plan from where the practice is now, not from where it was. One level per action; a target two levels away needs two steps.

    In the pack

  14. Step

    Re-order the plan: a larger gap may now take priority

    If the domain's level falls, its gap grows, and the action may move up the order. It goes first if a risk outside appetite, or an obligation recorded as not met, rests on the practice.

    In the pack

    See also

  15. Decision

    Did the fall follow a significant incident or major change?

    An incident or a change to the systems or the organisation can touch more than the one practice re-assessed. If you are unsure whether one lies behind the fall, take the cautious branch.

    In the pack

  16. Step

    Re-assess the rest of the domain now

    Do not wait for the next full assessment. Re-assess the domains the incident or change touched, on the same criteria and with the same independence as the full assessment, and re-plan their actions.

    In the pack

  17. Outcome

    Report the fall, its reason and the new action

    Put it in the next quarterly progress report to the sponsor, and in the next assessment report's changes since the last one. Do not hide it in a total: there is no total. Levels are reported in words, by domain.

    In the pack

Practical checklist

  1. Check that the criteria, scope, evidence window and sampling were the same.
  2. Mark a break in series beside each affected result if anything changed.
  3. Record the lower level as the result, and set any action being re-assessed to Not achieved.
  4. Ask whether the practice decayed or the earlier level was too generous, treating not known as decay.
  5. Agree a new action one level up from the level found, fixing the cause and not only the records.
  6. Re-order the plan, since a larger gap may now take priority.
  7. Re-assess the rest of the domain now if the fall followed a significant incident or major change.
  8. Report the fall, its reason and the new action in the next quarterly progress report.

Evidence to have ready

  • Both results with their criteria, scope, window and sampling, or the break in series marked beside them.
  • The re-assessor, the date and the level found, with the assessor's note.
  • The new action, its owner and due date, and its place in the re-ordered plan.
  • The quarterly report and the next assessment report showing the fall and its reason.

Assumptions

  • Both results come from assessments run on the same written maturity model, with each result's evidence and the assessor's note kept.
  • The person who re-assessed the practice does not own or run it. If they do, the result is self-assessed and cannot confirm a level above Defined.
  • You keep a findings and actions tracker and report progress to the sponsor each quarter.

Not covered here