Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

When do we…

When should we commission an independent assessment instead of self-assessing?

8 decision pointsThe pack behind this answer: Security Maturity & Internal Assessment

Add to board memo

The short answer

Recommendation
Run the assessment internally by default, with no one rating a practice they own or run. Commission an independent one, instead of or as well as the internal one, when any of the five triggers applies, and record the sponsor's decision and reason either way.
Reasoning
A colleague from another function knows how the organisation works and has no stake in the answer, so an internal assessment is enough for most cycles. The question is whether someone will rely on the result in a way an internal view cannot support, or whether the internal view has stopped being trustworthy. The five triggers cover both: outside reliance, a request for assurance from outside security, too many self-assessed results, doubted or contradicted levels, and a major change needing an outside view. Any one is enough.
Considerations
  • A maturity assessment is not an internal audit, a certification or the independent review a law may require (rule MA-19). Where a law asks for one, run it to that law's own conditions; the assessment can only feed it.
  • If someone needs evidence that specific controls operated, a maturity rating will not answer them. Have those controls tested independently as well.
  • An outside assessor does not move responsibility. The sponsor still approves the scope and targets, and the approver the improvement plan.
  • Who asks for outside assurance depends on governance: the board or a risk committee it has delegated to, the management board in a two-tier structure, or executive management where there is no board.

The decision, step by step

Select any step for the reasoning behind it and where the pack covers it.

  • Start
  • Step
  • Decision
  • Outcome
StartAn assessment cycle is being plannedDecisionDoes someone need evidence that specific controls work?StepHave those controls tested independently as wellStepThe assessment lead checks the five triggersDecisionDoes a regulator, certifier or contract need or rely on the result?DecisionIs what they need a legal audit or independent review?OutcomeRun that audit or review to the law's own conditionsDecisionHas the governing body asked for assurance from outside security?DecisionWould over a third be self-assessed, or does one person run all?DecisionHave levels stalled under doubt, or did an incident contradict them?DecisionDoes a major change need an outside view before investing?OutcomeRun it internally, with independent assessorsDecisionDoes the sponsor commission it?StepBind the assessor to your model and termsOutcomeCommission it; responsibility stays with youOutcomeRecord why not, and state the limits in the report

Keyboard: Tab to a step, arrow keys move along the flow, Enter pins its explanation, + and − zoom, 0 fits the diagram, Esc closes. The text version below has the same content.

Where a step quotes a number, it is the default in the CISO Times pack. Your own standard may set a different one.

Read as text

The same flow written out: each step, its reasoning and where the pack covers it. 16 steps.

  1. Start

    An assessment cycle is being planned

    Most cycles can be run internally. A colleague from another function knows how the organisation works, can ask for a record and wait for it, and has no stake in the answer. The question is whether someone will rely on the result in a way an internal view cannot support, or whether the internal view has stopped being trustworthy.

    In the pack

  2. Decision

    Does someone need evidence that specific controls work?

    A maturity assessment rates how reliably whole practices work, on four levels. It does not show that one control operated throughout a period. If an auditor, a customer or the governing body asks for that, a maturity rating will not answer them, whoever carries it out. If you are not sure what they need, ask, and answer yes until you know.

    In the pack

  3. Step

    Have those controls tested independently as well

    Add the controls to your control testing plan, each tested by someone who owns and performs none of them. An owner's check on their own control is a self-check: it does not count as a test, and the control stays untested. The test results can feed this assessment as evidence. Then carry on with the triggers: the assessment still needs its own decision.

    In the pack

    See also

  4. Step

    The assessment lead checks the five triggers

    The lead checks the triggers while planning the cycle, before assessors are assigned, and the sponsor decides. Any one trigger is enough. The first two are about who will rely on the result, the last three about whether an internal result can be trusted.

    In the pack

  5. Decision

    Does a regulator, certifier or contract need or rely on the result?

    Yes when a regulator, a certification body or a customer contract requires an independent assessment or audit, or will rely on the result. Read what the contract or the regulator actually asks for. If you do not know whether anyone will rely on the result, answer yes and find out.

    In the pack

  6. Decision

    Is what they need a legal audit or independent review?

    Some laws ask for their own audit or review, with their own rules on independence and competence: the internal audit of the ICT risk management framework that DORA requires of financial entities, or the independent review the NIS2 Implementing Regulation asks of the providers it names. A maturity assessment is not that audit or review, and is never presented as one; it can feed it. If you are not sure, answer yes and ask counsel.

    In the pack

  7. Decision

    Has the governing body asked for assurance from outside security?

    Depending on how you are governed, that is the board or a risk committee it has delegated to, the management board in a two-tier structure, or executive management where there is no board. If they want a view that does not come from the security function, an internal assessment led from security will not give it to them.

    In the pack

  8. Decision

    Would over a third be self-assessed, or does one person run all?

    Count the practices for which no one independent can be found, so the owner would rate their own work. Those results are marked self-assessed and capped at Defined. More than a third of the results self-assessed, or one person running every practice, is a trigger. If you have not yet assigned assessors, do that first.

    In the pack

    See also

  9. Decision

    Have levels stalled under doubt, or did an incident contradict them?

    Yes when the last two internal assessments gave the same levels and management doubts them, or when a significant incident contradicted them. An outside assessor who uses the same model keeps the result comparable with the internal ones.

    In the pack

  10. Decision

    Does a major change need an outside view before investing?

    Yes when a major change is planned, such as a merger, a new regulated service or a move to a new platform, and the investment decision needs an outside view of what the programme can carry.

    In the pack

  11. Outcome

    Run it internally, with independent assessors

    No trigger applies, so an internal assessment is the right choice this cycle. Keep the independence rule: no one rates a practice they own, do, manage day to day or wrote the procedure for. The Head of Information Security owns the programme and rates no practice they own or run. Note in the plan that the triggers were checked.

    In the pack

    See also

  12. Decision

    Does the sponsor commission it?

    A trigger applies, and the sponsor decides whether to commission an independent assessment instead of the internal one, or as well as it. A partial one is possible: an outside assessor can rate only the practices no one inside can rate independently. Record the decision and the reason in the assessment plan either way.

    In the pack

  13. Step

    Bind the assessor to your model and terms

    Agree in writing before fieldwork: they have not designed, run or advised on the practices they rate in the last 12 months (the pack's default); they rate against the same model and rules, or the result is a break in series; they give levels in words, with no overall score or percentage; they state how each result was evidenced and keep their samples; they sign a confidentiality agreement; and they show their competence.

    In the pack

  14. Outcome

    Commission it; responsibility stays with you

    Using an outside assessor does not move responsibility. The sponsor still approves the scope and the targets before scoring, and the approver, who depending on how you are governed is the board or a board risk committee it has delegated to, the management board in a two-tier structure, or executive management where there is no board, still approves the improvement plan. The result is a maturity profile, not an audit or a certificate, and the report says so.

    In the pack

  15. Outcome

    Record why not, and state the limits in the report

    Not commissioning one when a trigger applies is allowed, but the plan records the decision and its reason. The report's limits section then states which results are self-assessed and what the result does not say, so whoever relies on it knows how far to.

    In the pack

Practical checklist

  1. Ask whoever will rely on the result what they need, and route any control-level evidence to independent control testing.
  2. Check the five triggers while planning the cycle, before assessors are assigned.
  3. Confirm with counsel or compliance whether a law requires its own audit or review, and run that to its own conditions.
  4. Count how many results would be self-assessed; more than a third is a trigger.
  5. Put the decision to the sponsor, and record it and its reason in the assessment plan.
  6. Bind any outside assessor in writing to your model, levels in words, stated confidence, confidentiality and competence.
  7. Where a trigger applies and no one is commissioned, state the limits in the report.

Evidence to have ready

  • The assessment plan, showing the triggers checked, the sponsor's decision and its reason.
  • The written terms with any outside assessor, including their independence from the practices they rate.
  • The count of self-assessed results in the report.
  • The report's limits section, stating what the result does not say.

Assumptions

  • Someone in management sponsors the assessment and can commission outside work, and someone who runs none of the practices can lead it.
  • You rate practices against a written maturity model with four levels in words, such as the CISO Times maturity pack's, so an outside assessor can use the same model and the results stay comparable.
  • Whether a law asks you for an internal audit or an independent review depends on your sector and size. Confirm that with counsel or compliance before relying on an assessment for it.

Not covered here