Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

How do we…

How do we track AI security findings and the deviations we accept?

8 decision pointsThe pack behind this answer: Generative AI Security

Add to board memo

The short answer

Recommendation
Enter each confirmed gap in your findings register as soon as the reviewer can repeat it, and let its band set the severity, the due date and who may approve an exception. Fix it and have someone else retest it, or ask for a time-limited exception with compensating actions before the due date.
Reasoning
A finding is a confirmed gap between a feature and the standard, found in a review, a check, a test or an incident. The incident itself belongs to the incident route; the register holds the gap it showed. Rating impact and likelihood from 1 to 4 gives a band, and the band decides the due date, the approver of any exception and how long it may last. The register is read at every release decision, so an open High or Critical finding without an approved exception holds the release.
Considerations
  • Three gaps take no exception: a breach of the AI data classification rules (AG-03), a prohibited practice (AG-05), and a feature that releases a payment or changes supplier bank details (GS-27).
  • The pack's defaults by severity: Critical fixed within 30 days, High 90, Medium 180, Low 365, counted from the day the finding is confirmed. Exceptions last at most the same.
  • A Critical exception goes to the board, or a board risk committee it has delegated to; the management board in a two-tier structure; executive management where there is no board. The finding's owner never approves its exception.
  • An exception takes effect from the day it is approved, never earlier, and is never approved by silence.

The decision, step by step

Select any step for the reasoning behind it and where the pack covers it.

  • Start
  • Step
  • Decision
  • Outcome
  • Loop back
StartA review, test, check or incident shows a gap in an AI featureDecisionCan the reviewer repeat the problem?OutcomeNote it and watch it; do not rate it yetStepRate impact and likelihood, 1 to 4; the band is the severityStepEnter it in the register with the day it was confirmedDecisionDoes it break a rule that takes no exception?OutcomeFix it or stop the use: there is no exception routeStepAgree the fix and a date no later than the due dateDecisionCan it be fixed by the due date?StepFix it; the status becomes Fixed, awaiting retestDecisionDid someone other than the fixer repeat the test, and did it pass?OutcomeClose it with the retest date and result enteredStepAsk for an exception before the due date, with compensating actionsDecisionHow severe is the finding?StepTake it to the board or its equivalent to decideStepThe Head of Information Security and accountable executive approveStepMedium: Head of Information Security. Low: the control ownerDecisionDid the approvers approve it?StepRecord the exception with its approvers, date and expiryDecisionOn the expiry date, is it fixed and retested?DecisionIs there a renewal left with this band's approvers?OutcomeFix it now, hold the feature, or take it up a level

Keyboard: Tab to a step, arrow keys move along the flow, Enter pins its explanation, + and − zoom, 0 fits the diagram, Esc closes. The text version below has the same content.

Where a step quotes a number, it is the default in the CISO Times pack. Your own standard may set a different one.

Read as text

The same flow written out: each step, its reasoning and where the pack covers it. 22 steps.

  1. Start

    A review, test, check or incident shows a gap in an AI feature

    A finding is a confirmed gap between a feature and the Generative AI Security Standard, found in a review, a configuration check, a test or an incident. The incident itself belongs to the incident route; the register holds the gap it showed. Each finding has an owner and a due date, and the register is read at every release decision.

    In the pack

  2. Decision

    Can the reviewer repeat the problem?

    The reviewer repeats the problem and writes the steps to repeat it. A gap that cannot be repeated is noted and watched, and is not rated. Judgement: if you cannot tell yet, take the Yes branch and try again, because an unrated real gap has no owner and no date.

    In the pack

  3. Outcome

    Note it and watch it; do not rate it yet

    Write down what was seen, where and when, and keep the evidence as received. Judgement: look again after the next change to the model, prompt, tool or source, and at the next review. Once it can be repeated, it is confirmed and the clock starts from that day. If it was an event, it also goes to the incident route.

    In the pack

    See also

  4. Step

    Rate impact and likelihood, 1 to 4; the band is the severity

    The Head of Information Security rates the finding on the risk management scales: impact from 1 to 4, likelihood from 1 to 4, and the band of the two multiplied together is Low, Medium, High or Critical. 12 to 16 is Critical and 8 to 9 is High. Judgement: if an input is not known, use the higher value until it is known, so a gap in the facts costs the people who can close it.

    In the pack

  5. Step

    Enter it in the register with the day it was confirmed

    The register holds the finding, the feature and its P24 use, the threat and requirement it relates to, the owner as a named role, and the source: review, check, test, red team or incident. The day the reviewer confirms it starts the clock, and the due date is that day plus the days for the severity. Pack defaults: Critical 30, High 90, Medium 180, Low 365 days.

    In the pack

  6. Decision

    Does it break a rule that takes no exception?

    Three things take no exception: a breach of P24 AG-03 (the data classification rules, including special category data) or AG-05 (the practices the EU AI Act prohibits), and a feature that releases a payment or changes supplier bank details (GS-27). The register flags an exception on any of them. Judgement: if you cannot tell, ask the Head of Information Security before any exception request.

    In the pack

  7. Outcome

    Fix it or stop the use: there is no exception route

    Fix the cause, or stop or hold the feature until it is fixed. A legal or regulatory obligation can never be excepted either (P02 EX-02). The finding is still entered and still counted, so the register shows the gap.

    In the pack

    See also

  8. Step

    Agree the fix and a date no later than the due date

    The owner of the AI use agrees the fix with the developer, and the status becomes Plan agreed. The owner is a named role. The finding's owner is never among those who approve an exception for it (FV-10).

    In the pack

  9. Decision

    Can it be fixed by the due date?

    Judge it before the date arrives. Judgement: a fix that depends on a supplier, a new model version or a budget decision is a likely No. If a finding passes its due date without a fix or an approved exception, the register shows it as past due.

    In the pack

  10. Step

    Fix it; the status becomes Fixed, awaiting retest

    The developer fixes it. A change of model, prompt, tool, source or filter is a change, so the attack classes it could affect are tested again (GS-18). Closing needs a retest.

    In the pack

  11. Decision

    Did someone other than the fixer repeat the test, and did it pass?

    A pass closes the finding and a fail returns it to Open. The retest date and result are entered. A later change to the model, prompt or tool can reopen a closed finding, so the register is checked again after one.

    In the pack

  12. Outcome

    Close it with the retest date and result entered

    Report counts with the gaps named, such as 1 High finding is open without a live exception. Do not report a score or a share. Keep the finding and its retest with the feature's evidence for the records period (GS-29).

    In the pack

    See also

  13. Step

    Ask for an exception before the due date, with compensating actions

    The request names the finding, its severity, why the fix is not possible by the due date, and the date by which it will be. It lists the compensating actions that lower the risk meanwhile, each with an owner. Ask before the due date passes: an exception takes effect from the day it is approved, never earlier (P02 EX-02). The approver decides in writing (P02 EX-05).

    In the pack

    See also

  14. Decision

    How severe is the finding?

    The severity sets who approves, how quickly they decide and how long the exception can last. The finding's owner is never an approver. If the approver of a band owns the finding, the approval passes to the next level up (P02 EX-05), and your exception policy says who that is.

    In the pack

  15. Step

    Take it to the board or its equivalent to decide

    Only the board or its equivalent may accept a Critical exception: the board, or a board risk committee it has delegated to; the management board in a two-tier structure; executive management where there is no board. Pack defaults: decided within 5 working days of a complete request, 30 days at most, 1 renewal.

    In the pack

    See also

  16. Step

    The Head of Information Security and accountable executive approve

    Both approve a High exception. Where it leaves a risk outside appetite, P05's approver signs as well (RM-08). Pack defaults: decided within 10 working days, 90 days at most, 1 renewal. An open High finding holds a release unless this exception is in place (RG-4).

    In the pack

  17. Step

    Medium: Head of Information Security. Low: the control owner

    A Medium exception is approved by the Head of Information Security, decided within 10 working days, 180 days at most, 2 renewals. A Low one is approved by the control owner, within 5 working days, 365 days at most, 2 renewals. These are the pack's defaults. An open Medium or Low finding does not hold a release; it becomes a condition.

    In the pack

  18. Decision

    Did the approvers approve it?

    An approver may always refuse, shorten the duration or add conditions (P02 EX-05). A request that is not decided in time goes to the next approval level up and is never approved by silence. If it is refused, or not yet decided, the due date stands: fix it, and the register shows the finding as past due once the date passes.

    In the pack

  19. Step

    Record the exception with its approvers, date and expiry

    Enter the exception, its band, the approval date, who approved and the expiry, and write its number against the finding. The status becomes Deferred as an exception only when the approvers have approved. The expiry is the review date. Compensating actions run until then.

    In the pack

  20. Decision

    On the expiry date, is it fixed and retested?

    On that day one of three things happens: the finding is fixed and retested, or the exception is renewed, or the feature is held. An exception that passes its expiry without being closed or renewed has expired and the requirement applies again in full (P02 EX-10). The register flags it (FV-13).

    In the pack

  21. Decision

    Is there a renewal left with this band's approvers?

    A renewal is a new request with an updated risk assessment, made before the exception expires, and decided by the same approvers (P02 EX-08). The band limits how many the same approvers can grant: 1 for Critical and High and 2 for Medium and Low, by default. A renewal beyond the limit goes to the next approval level up, and the exception is then reported by name each quarter as a chronic exception (P02 EX-08).

    In the pack

    See also

  22. Outcome

    Fix it now, hold the feature, or take it up a level

    With no renewal left with the same approvers, the exception cannot run on with them. Fix the finding and retest it, or the approvers hold the feature, or the request goes to the next approval level up (P02 EX-08). For a running feature the AI governance lead may suspend it while they decide (P24 AG-10). Report it as a count with the gap named.

    In the pack

    See also

Practical checklist

  1. Confirm the reviewer can repeat the gap; note and watch one that cannot be repeated.
  2. Rate impact and likelihood from 1 to 4, using the higher value where an input is not known.
  3. Enter the finding with its feature, threat, requirement, owner, source and the day it was confirmed.
  4. Fix or stop the use where the gap breaks a rule that takes no exception.
  5. Agree the fix and a date no later than the due date.
  6. Have someone other than the fixer retest it, and close it only on a pass.
  7. Ask before the due date for an exception with compensating actions, from the approvers the band sets.
  8. On the expiry date, close it, renew it within the band's limit, or hold the feature.

Evidence to have ready

  • The register entry for each finding: steps to repeat it, rating, severity, owner, confirmed date and due date.
  • Each exception with its band, approvers, approval date, compensating actions and expiry.
  • The retest date, who retested and the result for each closed finding.
  • Counts with the gaps named, such as High findings open without a live exception, never a score or a share.

Assumptions

  • You have a findings register and an exception process under your security exception standard. This flow uses the pack's register fields and the bands in your exception policy.
  • The figures shown are the pack's defaults: fix within 30, 90, 180 or 365 days by severity, and exceptions of at most the same length. Your own policy may set different ones.
  • Who approves depends on how you are governed. The top approver is the board, or a board risk committee it has delegated to; the management board in a two-tier structure; executive management where there is no board.
  • Each finding has an owner who is not one of the approvers of its exception.

Not covered here