Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

How do we…

How do we configure a generative AI service securely before it goes live?

7 decision pointsThe pack behind this answer: Generative AI Security

Add to board memo

The short answer

Recommendation
Confirm the use is approved and on the inventory with an owner, then work through the configuration checklist for the checks that apply, each with evidence and a date. Release only when no blocking check has failed or is unverified and every open High or Critical finding has an approved exception.
Reasoning
The model will sometimes be fooled, so the security rests on the settings and controls around it. The checklist turns that into settings you can look at and evidence you can keep. Which checks apply depends on three questions: does the service read stored company content, does it have tools or connectors, and do you host or build it and hold its keys. A setting you cannot see counts as not verified, because Not verified is not a pass. Fourteen of the 30 checks hold the release if they fail or are not verified (RG-2).
Considerations
  • Configuring a feature nobody approved only makes it look safe. Without an intake decision, the first step is intake.
  • Do not mark a check Not applicable because it is slow to verify. A provider's page or statement counts as evidence if you say so and date it.
  • Who signs follows the higher of the intake tier and the review depth, with the Head of Information Security as well from Depth 2. A Critical exception goes to the board, or the committee it has delegated to; the management board in a two-tier structure; executive management where there is no board.
  • The pack is silent on whether an exception can lift a blocking check, so do not assume one can; ask the Head of Information Security.

The decision, step by step

Select any step for the reasoning behind it and where the pack covers it.

  • Start
  • Step
  • Decision
  • Outcome
  • Loop back
StartA generative AI service is set up and not yet liveDecisionIs the use approved, on the inventory, with a named owner?OutcomeDo not switch it on: take it through intake firstStepSet the review depth and answer the three scope questionsStepCheck sign-in, administrators, sharing, training and data settingsDecisionDoes it have tools, connectors or write access?StepLimit connectors and scopes, set approvals and test the switch-offStepCheck keys, logging, alerts, filters, limits and the model recordStepEnter each result with its evidence, its date and who lookedDecisionIs evidence missing, or a blocking check failed or not verified?StepFix it or get the evidence, and enter each failed check as a findingDecisionCan it be fixed or evidenced before the go-live date?DecisionIs a High or Critical finding open with no approved exception?StepWrite each open Medium, Low or unverified item as a conditionDecisionWhat is the higher of the P24 tier and the review depth?OutcomeRelease signed by the AI governance leadOutcomeRelease signed by the lead and Head of Information SecurityDecisionIs the independent red team report received?OutcomeRelease signed by the executive and Head of Information SecurityOutcomeHold: the service stays off until the missing item is cleared

Keyboard: Tab to a step, arrow keys move along the flow, Enter pins its explanation, + and − zoom, 0 fits the diagram, Esc closes. The text version below has the same content.

Where a step quotes a number, it is the default in the CISO Times pack. Your own standard may set a different one.

Read as text

The same flow written out: each step, its reasoning and where the pack covers it. 20 steps.

  1. Start

    A generative AI service is set up and not yet live

    The service may be a supplier's assistant you switch on, a feature you configure inside a platform, or one you build and host. An LLM application puts instructions and data in one channel, so the security rests on the settings and controls around the model, because the model will sometimes be fooled. The configuration checklist turns that into settings you can look at and evidence you can keep.

    In the pack

  2. Decision

    Is the use approved, on the inventory, with a named owner?

    Which AI may be used, with which data, and who approves it are P24's rules. A feature a supplier adds to software already in use counts. If you cannot tell, take the No branch, because configuring a feature nobody approved only makes it look safe.

    In the pack

  3. Outcome

    Do not switch it on: take it through intake first

    The review starts after the intake decision. A P24 Stop means no review is started. Take the proposed use through intake and come back with the tier it sets.

    In the pack

    See also

  4. Step

    Set the review depth and answer the three scope questions

    The depth is the higher of the P24 tier and what the feature has: stored company content above Public, tools, memory, or the rule of two. Then answer three questions: does it read stored company content, does it have tools or connectors, and do you host or build it and hold its keys. Each check applies or not on those answers. Do not mark a check Not applicable because it is slow to verify.

    In the pack

  5. Step

    Check sign-in, administrators, sharing, training and data settings

    Staff reach the service only through work single sign-on (DC-01). Administrators are named and use phishing-resistant sign-in (DC-02). Sharing outside the organisation is off or limited to a listed set (DC-03), the no-training term is on and evidenced (DC-04), and retention, location and data level are set as recorded (DC-05 to DC-07). If it reads stored content, every source has an owner and a level (DC-08) and a removed document leaves the index (DC-09). DC-01, DC-02, DC-04, DC-07 and DC-08 hold the release.

    In the pack

  6. Decision

    Does it have tools, connectors or write access?

    Yes also when a supplier's connector is switched on by default. A tool turns a fooled model into an action, so the checks on scope and approval are stricter. If you do not know what is connected, answer Yes and list it.

    In the pack

  7. Step

    Limit connectors and scopes, set approvals and test the switch-off

    Connectors are on an approved list and all others are off (DC-10). Each holds the least scope its job needs (DC-11), and tools run as the signed-in person or a named service identity (DC-14). Complete the agent permission review so each action has an approval level: a person approves each time for actions that leave the organisation, change access or cannot be undone, and the agent never releases a payment, changes supplier bank details or decides a matter about a person. The switch-off works in one step and was tested (DC-30). DC-10, DC-11, DC-14 and DC-30 hold the release.

    In the pack

    See also

  8. Step

    Check keys, logging, alerts, filters, limits and the model record

    For every service: logs of prompts, outputs and administrator actions (DC-15), protected logs (DC-17), alerts to a named person (DC-18), filters (DC-22, DC-24), rate limits (DC-25) and the model version recorded (DC-28). With tools, every tool call is logged too (DC-16). If you host or build it: keys in a secrets store, one per feature and environment (DC-12), a network allow list (DC-19), input and output checks in code (DC-23), a hard spending cap (DC-26) and the prompts under version control (DC-29). DC-12, DC-15, DC-16, DC-19 and DC-23 hold the release. For a service you only configure, the tenant shows most of these; ask the provider in writing for the rest.

    In the pack

  9. Step

    Enter each result with its evidence, its date and who looked

    Verified needs evidence and a date. A provider's page or statement counts as evidence if you say so and date it. A check with no entry counts as Not verified. The review also needs the other evidence for its depth on file (RG-1): the data flow, the standard check, this checklist and the approval record, with more at higher depths.

    In the pack

  10. Decision

    Is evidence missing, or a blocking check failed or not verified?

    A gate fails if required evidence for the depth is missing (RG-1) or one of the 14 checks that hold the release has failed or is unverified (RG-2). Answer Yes if a result is missing, because Not verified is not a pass. The checklist gives counts, such as 14 of 17 applicable checks verified, never one score.

    In the pack

  11. Step

    Fix it or get the evidence, and enter each failed check as a finding

    For a Failed check, rate it, enter it in the findings register and write its number against the check. For a check not yet verified, or evidence not yet on file, ask the provider or the IT owner and keep the answer. A change to the model, prompt, tool, source or a filter setting is a change: record it, and repeat the checks it touches (GS-18).

    In the pack

    See also

  12. Decision

    Can it be fixed or evidenced before the go-live date?

    If yes, go back and enter the new result. If no, the release holds. The pack is silent on whether an exception can lift a blocking check or missing evidence, so do not assume one can. Judgement: ask the Head of Information Security.

    In the pack

  13. Decision

    Is a High or Critical finding open with no approved exception?

    Findings come from the checks, the tests and the provider answers. A High finding passes only with an exception approved by the Head of Information Security and the accountable executive. A Critical one passes only with an exception from the board or its equivalent (RG-3, RG-4). Judgement: if a severity is not rated, rate it as the higher band until it is.

    In the pack

    See also

  14. Step

    Write each open Medium, Low or unverified item as a condition

    Something open that fails no gate does not hold the release. It becomes a condition with an owner and a date, and a missed condition comes back to the approvers. This is the Release with conditions outcome. It differs from a clean Release only in what the decision record lists.

    In the pack

  15. Decision

    What is the higher of the P24 tier and the review depth?

    The approvers follow the higher of the two, and from Depth 2 the Head of Information Security signs as well. The AI governance lead convenes the decision. Nobody approves a use they own: where an approver owns the feature, the executive decision-maker takes the place, or the board or its equivalent where that person owns a Tier 3 use. Judgement: if you are not sure of the tier, take 3.

    In the pack

  16. Outcome

    Release signed by the AI governance lead

    Tier 1 and Depth 1. The use owner signs the conditions it accepts. Turn the alerts on (DC-18), set the next review date, which is the earlier of the pack's date and the P24 inventory date, and keep the checklist with the review evidence.

    In the pack

  17. Outcome

    Release signed by the lead and Head of Information Security

    Tier 2 or Depth 2. The Head of Information Security reviews the evidence and signs with the lead. A third-party feature also needs the provider questionnaire answered and reviewed. Turn the alerts on, set the next review date, and repeat the checks after any change of model, prompt, tool, source or filter.

    In the pack

    See also

  18. Decision

    Is the independent red team report received?

    At Depth 3 the release also needs a report from someone who did not build the feature, with its findings in the register (RG-5). If the report is late, answer No.

    In the pack

    See also

  19. Outcome

    Release signed by the executive and Head of Information Security

    Tier 3 or Depth 3. The executive decision-maker signs with the Head of Information Security. Set the next review date, which follows the higher of the tier and the depth, and keep the alerts on.

    In the pack

  20. Outcome

    Hold: the service stays off until the missing item is cleared

    A failed gate holds a new feature; for one already running, the AI governance lead may suspend it while the approvers decide (P24 AG-10). The decision record names each gate that failed and what clears it, and the review restarts at the stage that holds the missing item. Any approver may hold a release.

    In the pack

    See also

Practical checklist

  1. Confirm the use is approved, on the inventory and has a named owner.
  2. Set the review depth and answer the three scope questions.
  3. Check sign-in, administrators, sharing, the no-training term, retention, location and data level.
  4. Limit connectors and scopes, set an approval for each tool action, and test the one-step switch-off.
  5. Check keys, logging, alerts, filters, limits and the recorded model version.
  6. Enter each result with its evidence, its date and who looked.
  7. Fix or evidence each blocking check, and enter each failed check as a finding.
  8. Write open Medium, Low or unverified items as conditions with owners and dates, then get the right approvers to sign.

Evidence to have ready

  • The completed configuration checklist with evidence, date and who looked for each check.
  • The counts of applicable checks verified, failed and not verified, never one score.
  • The agent permission review for a service with tools, and the date the switch-off was tested.
  • The findings register entries and any approved exception for a High or Critical finding.
  • The approval record with the approvers, conditions and the next review date, and at Depth 3 the independent test report.

Assumptions

  • The service is approved under your AI policy and is on the inventory with a named owner. Without that, the first step is intake, not configuration.
  • You can see the tenant settings, or the provider will confirm them in writing. A setting you cannot see counts as not verified.
  • The check list, the review depths and the figures shown are the pack's defaults. Your own standard may add checks or set different ones.
  • Who signs depends on how you are governed. The top approver is the board, or the committee it has delegated to; the management board in a two-tier structure; executive management where there is no board.

Not covered here