Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

LLM Application Security Review Procedure

Reviews an LLM feature or agent before release against the standard and threat catalogue.

Available soon

Format
Word
Size
76 KB
Length
26 pages
Version
1.0
Updated

What's inside

What changed

  1. 1.0

    Published.

Changes across this pack

Preview

The document from section 1, as you will receive it. Highlighted [[text]] is for you to replace; shaded guidance boxes are for you to delete before approval. The cover and document control pages are in the file.

Purpose

This procedure sets out how [[Organisation Name]] reviews a generative AI feature before it goes live, and again while it runs. It answers one question for the people who build, configure and approve the feature: can it go live? The answer is Release, Release with conditions or Hold, and the evidence is on file.

It applies the Generative AI Security Standard. It sets how much review a feature needs, who does each part, what evidence the review holds, which gates the release must pass, who signs, and how findings and exceptions are handled.

An LLM application puts instructions and data in one channel. The model reads the organisation's instructions and an attacker's words as the same kind of thing, so the checks that work for ordinary input, such as matching a field against a pattern, cannot tell them apart. The national cyber centres and OWASP both say the defence lies in the system around the model, because the model will sometimes be fooled. Everything the organisation already does for access, logging, secure development, patching and supplier assurance still applies and is not repeated here. This pack adds what is specific: what the model may read, what its output may reach, what it may do, who approves what it does, and how the organisation tests all of this before release and watches it afterwards. The result is counts of features reviewed, checks verified and findings open. It is never one score.

Guidance — delete before approval

Replace every value in [[double brackets]] before approval, and name roles, not people.

The settings in [[double brackets]] (days to decide, months between reviews, days to review a change, years to keep evidence) are shared with the Generative AI Security Standard and with the AI Security Governance & Acceptable Use pack (P24). Change them in one place and here together.

Try the procedure first on one feature that is already running. If the owner and the developer can complete the data flow and the standard check in a day for a simple feature, it is working. If they cannot, shorten the forms and keep the gates as they are.

This document helps you secure the generative AI your organisation builds, configures or buys. It is not legal advice, and following it is not a statement that you comply with the EU AI Act, the GDPR or any standard. A test that finds no problem does not show that a feature is secure. Whether the Act binds you, in which role, and which of its duties apply to each system depends on the facts: confirm it with counsel.

This procedure does not decide whether a system is prohibited, high-risk or subject to a transparency duty, and it does not decide which role the organisation holds under the EU AI Act. A review that finds a feature might touch those questions sends it to Legal.

Regulation (EU) 2024/1689 binds the operators of its Article 2(1). Article 15 sets requirements for high-risk AI systems, and the provider of such a system answers for them. An organisation that uses another party’s AI system under its own authority is a deployer (Article 3(4)), and its duties are those of Article 26 and the other articles that name deployers. This pack cites Article 15 as the Act’s own statement of what security a high-risk system needs, and cites no system as high-risk. Counsel confirms the role and the category of each system.

OWASP, MITRE ATLAS, NIST AI 600-1, NIST SP 800-218A, the NCSC blog, the NCSC and CISA guidelines and the joint agency guidance on agents are voluntary guidance. They are not law, and none of them says what the AI Act or the GDPR requires. This pack cites them as their publishers state them.

Scope

This procedure applies to every LLM feature the organisation builds, configures or switches on. That includes a feature a supplier adds to software already in use. A feature is one LLM capability the organisation uses, builds or has switched on, for one purpose, with one owner. A feature is the technical view of a P24 use.

The rules on which AI may be used, with which data, who approves a use and how a use is recorded are P24's (AG-01 to AG-22). This pack does not change them. It sets what a generative AI feature has to do to be secure once a use is approved, and what the organisation checks before the feature goes live and while it runs. Intake, the tiers and the inventory are in the AI Security Governance & Acceptable Use pack (P24): this procedure cites them by ID and does not repeat them.

Nor does it decide whether an event is a security incident. That is the incident response pack (P09). The reporting duties and their clocks are in Regulatory Incident Notification & Reporting pack (P18).

A feature that is already running when the Generative AI Security Standard is adopted is reviewed within [[90]] calendar days of adoption, at the depth this procedure sets.

Roles

Role

What they do in this procedure

AI governance lead

[[e.g. Head of Risk]]

Owns the approved list and the inventory (P24), convenes the release decision and collects the signatures of the approvers (RG-6), may suspend a running feature pending review, and reports the measures.

AI use owner

[[the manager accountable for the business activity the AI supports]]

Owns the feature, answers for its output, has the data flow drawn, agrees fixes for findings and keeps the evidence. Never approves a feature it owns.

Application developer

[[the engineer or configurer who builds or sets up the feature]]

Builds or configures the feature to this standard, completes the configuration checks with the IT owner, fixes findings and takes part in the review. Does not test the feature at Depth 3, and at Depth 2 is not the tester unless the review says why.

Head of Information Security

[[e.g. Head of Information Security]]

Owns this standard, selects the threats and sets the review depth, reviews the evidence, rates each finding, signs the release decision from Depth 2 (RG-6), and approves exceptions in the Medium and High bands.

Independent tester

[[a security analyst or external tester who did not build the feature]]

Tests the feature against the attack classes and writes the report. Required at Depth 3, where the tester does not own or build the feature (P20 CA-10; NIST AI RMF MEASURE 1.3), and expected at Depth 2.

Data Protection Officer

[[or the privacy lead where no DPO is appointed]]

Advises on personal data in any use, decides with the use owner whether a data protection impact assessment is needed, and is consulted on every use of personal data.

Legal

[[in-house or external counsel]]

Confirms the AI Act role and category of each system, reads supplier terms, and decides whether the organisation could be a provider.

Procurement

[[e.g. Procurement Manager]]

Makes sure no AI tool or AI feature is bought or switched on without an intake decision, and puts the AI terms into the contract (ST-1 to ST-8).

Head of IT

[[e.g. Head of IT]]

Connects approved tools to single sign-on, applies the tool's settings and keeps the technical list of what is installed.

Executive decision-maker

[[e.g. Executive Committee, or the management body]]

Decides Tier 3 uses: executive management by default, or the board where the governance structure puts it. Signs as well for a High risk outside appetite (P05). Does not accept a Critical risk as executive management: it goes to the board or its equivalent (P05 RM-04).

Three rules about people apply throughout. The owner of a feature never approves it. The builder of a feature never tests it at Depth 3, and at Depth 2 a tester who did not build it is expected and the review states who tested. Nobody tests a control they own or perform (P20 CA-10). Nobody approves an exception for a finding they own (P02 EX-05). Where a band's approver owns the finding, the approvers of the next band up take that approver's place, the same principle as P24 IS-4.

The seven stages

Every feature goes through the same seven stages. A simple feature passes through them in a few days. A feature that acts for customers takes longer and has more people in it.

Stage

What happens

Who

Rule

RP-1

Describe

The owner and the developer describe the feature, link it to its P24 inventory entry, and draw the data flow (GS-03): sources, sinks, tools, stores and the provider. They say whether the feature is built on a supplier's model API, which counts as building under P24 (AG-17).

AI use owner

GS-03

RP-2

Select threats

The Head of Information Security selects the threats from the catalogue (A04) that apply, with the reason for each class left out, and sets the review depth.

Head of Information Security

GS-28

RP-3

Check the standard

The developer checks each requirement GS-01 to GS-31 as met, not met or not applicable, with evidence. The IT owner completes the configuration checklist (A05) with the developer.

Application developer

GS-29

RP-4

Assess the provider

From Depth 2, where an outside provider supplies the model, Procurement sends the questionnaire (A06) to the provider and Security reviews the answers. At Depth 1 Security reads the provider's published security statement instead. The supplier decision stays P06's.

Procurement

GS-17

RP-5

Test

The tests for the selected classes are run at the depth: by the team at Depth 2, where a tester who did not build the feature is expected and the review states who tested, and by an independent tester at Depth 3 under the red team plan (A07). Agent features also get the permission review (A09).

Independent tester

GS-28

RP-6

Decide

The AI governance lead convenes the decision. The approvers decide: Release, Release with conditions or Hold. The approvers of the higher of the use's P24 tier and the review depth sign, and the Head of Information Security signs as well at Depth 2 and 3.

AI governance lead

GS-30

RP-7

Release and watch

The feature goes live (or continues) with the settings reviewed. Monitoring is on, the next review date is set, and findings and exceptions are in the register (A08).

Head of IT

GS-20

Review depth

Review depth says how much review a feature gets. It is set at Stage 2 and does not depend on how confident the team feels. The depth is the highest any trigger gives, and the review records every trigger that applied.

Depth

Who does the work

What the review holds

Decision within

Review every

No review

Stopped under P24

None

P24 has said Stop. No review is started.

Not applicable

Not applicable

Depth 1

Follows P24 Tier 1, unless a property of the feature moves it up

The owner, the developer and the IT owner

Data flow, standard check and configuration checklist. No separate test of the attack classes beyond the checks the checklist holds, and no tester is required. Where an outside provider supplies the model, its published security statement is read instead of the questionnaire. Depth 1 follows the P24 tier unless a property of the feature moves it up.

[[5]] working days

[[12]] months

Depth 2

Follows P24 Tier 2, unless a property of the feature moves it up

The team, with the Head of Information Security reviewing

Depth 1, the threat selection, and tests of the selected classes by the team. A tester who did not build the feature is expected, and the review states who tested. The provider questionnaire, required when an outside provider supplies the model. The agent permission review where there are tools.

[[15]] working days

[[6]] months

Depth 3

Follows P24 Tier 3

An independent tester

Depth 2, with an independent red team test of every selected class under a written plan (A07), and the agent permission review for any tools. A new feature built on a supplier's model API is Tier 3 under P24 until Legal records the role, so it starts at Depth 3. When Legal records the deployer role, the depth is worked out again from the new tier and the feature's properties. When Legal records the provider role or another role, the feature stays Tier 3 and stays at Depth 3. A feature already running when the standard starts carries the P24 hold as a condition with a date, and its depth follows its tier and properties.

[[30]] working days

[[6]] months

Decision time, approvers and review interval follow the higher of the P24 tier and the review depth. A feature can sit at a higher depth than its P24 tier. A Tier 2 use reviewed at Depth 3 takes Tier 3's approvers, decision time and review interval: it is signed as Tier 3.

A feature has two review dates: the one this review sets from its depth, and the one in its P24 inventory entry. The earlier of the two applies. The review date is the day set from the decision and the depth's review interval, or the review date in the P24 entry if that is sooner.

What sets the depth

The feature

Depth at least

Rule

P24 has said Stop

No review

AG-05

P24 Tier 1, and the feature has none of the properties below

Depth 1

GS-28

P24 Tier 2

Depth 2

GS-28

The feature retrieves company content above Public

Depth 2

GS-09

The feature has tools, read or write

Depth 2

GS-22

The feature keeps memory between sessions

Depth 2

GS-14

Two of the three properties of the rule of two hold

Depth 2

GS-24

All three properties of the rule of two hold

Depth 3

GS-24

A new feature is built on a supplier's model API and Legal has not yet recorded the role (P24 Tier 3, held)

Depth 3

GS-28

The same, and Legal has recorded the provider role or another role (P24 keeps it at Tier 3; the hold clears)

Depth 3

GS-28

P24 Tier 3

Depth 3

GS-28

The first row is P24's. A use that P24 has stopped is not reviewed. The other rows come from this procedure. If a feature meets more than one, the highest depth applies.

The rule of two

The rule of two counts three properties of a feature. The count raises the depth and says what the review must record.

Property

True when

A. Reads untrusted content

The feature reads content from outside the organisation or from anyone who can submit text, or customers or the public use it directly.

B. Reaches sensitive data

The data it can reach is above Public, or includes personal data.

C. Changes state or communicates outside

It has tools that can write, it sends output outside, or it acts without a person reading its output first.

Count

What follows

0 or 1 of 3

The depth comes from the other triggers.

2 of 3

At least Depth 2. The review records the residual risk in plain words and the approvers read it.

3 of 3

Depth 3, with an independent test. A person approves every action the feature takes.

GS-24 Count three properties of a feature: it reads untrusted content (A), it can reach sensitive data (B), and it can change state or communicate outside the organisation (C). With all three it needs a person to approve every action. With two the review records the residual risk. The review procedure (A03) counts them.

Guidance — delete before approval

The count is a count of properties, never a score. A feature with two properties is not half as risky as one with three. It is a feature where the review has to say what remains and who accepts it.

If the owner cannot say whether a property is true, treat it as true until the data flow shows otherwise.

Stage 1 (RP-1): Describe

GS-03 Draw a data flow for each feature that shows every source the model reads (user input, files, web pages, email, tool results, retrieved text, memory) and every place its output goes (screens, files, databases, links, tools, other systems). Mark each source trusted or untrusted. Content from outside the organisation, and content from anyone who can submit text, is untrusted, whichever system holds it.

In practice. Open the feature's entry in the P24 inventory and copy its use ID, tier and data level. Draw the data flow on one page: every source the model reads, every place its output goes, every tool, every store and the provider. Mark each source trusted or untrusted. Write down the model name and version, the system prompt version and the tools. Then answer five questions for the depth: does it retrieve company content, what level of tools does it have (none, read or write), does it read untrusted content, does it keep memory, and does it send anything outside. Say whether the organisation built the feature (building on a supplier's model API counts, for staff or for customers), put its own name on it or changed what it is meant for, and tell Legal if so. P24 holds such a feature at Tier 3 until Legal records the role, and Legal decides what that means under the Act. A supplier's finished product used as supplied is not building. When the record is made, ask for the depth to be worked out again. Record: The data flow (E-1) and the five answers.

Stage 2 (RP-2): Select threats

In practice. Run the depth rules with the P24 answers and the five feature answers, and keep the reasons it prints. Then go through the threat catalogue and mark each threat selected or left out. Write one line for each threat left out. Record the rule of two result. Tell the owner the depth, the evidence the depth needs, the approvers and the date the decision is due. Record: The threat selection (E-2) and the depth with its reasons.

The LLM Application Threat Catalogue holds twelve threats. Each has an attack class in the AI Red Team Test Plan Template, which is how it is tested. A class is tried when its threat is selected and the feature has the property in the last column.

Threat

Attack class

Applies when

LT-01 Direct prompt injection

AC-01

always

LT-02 Indirect prompt injection

AC-02

always

LT-03 Sensitive information disclosure

AC-03

always

LT-04 Excessive agency

AC-04

tools

LT-05 AI supply chain compromise

AC-05

always

LT-06 Data and model poisoning

AC-06

always

LT-07 Unbounded consumption

AC-07

always

LT-08 Misinformation

AC-08

always

LT-09 Hidden context exposure (system prompt leakage)

AC-09

always

LT-10 Vector and embedding weaknesses

AC-10

retrieval

LT-11 Improper output handling

AC-11

output reaches another component

LT-12 Poisoned memory and tool descriptions

AC-12

memory or tools

A threat is left out only with a reason in writing, such as no retrieval source or no memory. 'Low risk' is not a reason. The Head of Information Security reads the reasons.

Stage 3 (RP-3): Check the standard

In practice. The developer marks each of the thirty-one requirements as met, not met or not applicable. A met requirement has evidence: a setting export, a screenshot, a test result or a document. A not applicable one has a reason. The Head of IT and the developer then go through the configuration checklist together and mark each applicable check Verified, Failed or Not verified. A Verified check has evidence and a date. Note which of the checks hold the release. Record: The standard check (E-3) and the configuration checklist result (E-4), each as a count.

The configuration checklist has 30 checks. 14 of them hold the release if they fail or are not verified. Which checks apply depends on the feature: all features, features with retrieval, features with tools, and features the organisation hosts or builds itself. A check that does not apply is shown as not applicable without being entered.

Report both as counts: for example, 14 of 17 applicable checks verified, with the three that are not verified listed. Never report them as a share.

Stage 4 (RP-4): Assess the provider

In practice. Required from Depth 2 when a provider outside the organisation supplies the model or the service. Procurement sends the questionnaire and asks for evidence with every answer. At Depth 1 no questionnaire is required: the Head of Information Security reads the provider's published security statement and records the date it was read. The Head of Information Security reads the answers and counts them: answered with evidence, answered without evidence, not answered. Open questions get a date agreed with the provider. The questionnaire informs the supplier decision. The decision, and the contract terms, are not made here. Record: The returned questionnaire with the Head of Information Security's reading (E-8); at Depth 1, the date the published statement was read.

The contract terms for an AI supplier (ST-1 to ST-8) and the supplier decision are in the AI Security Governance & Acceptable Use pack and the Third-Party Security Risk Management pack (P06). The questionnaire's answers are evidence for that decision. They do not replace it.

Stage 5 (RP-5): Test

GS-28 Test each feature before release against the attack classes AC-01 to AC-12 at the depth the review procedure (A03) sets. At Depth 3 someone other than the builder tests (P20 CA-10). At Depth 2 the team tests, the review states who tested, and a tester who did not build the feature is expected. At Depth 1 no separate test is required. A feature already running when this standard is adopted is reviewed within [[90]] calendar days of adoption. Tests are repeated after a change that could affect an attack class.

In practice. At Depth 1 the configuration checks are the test. At Depth 2 the team runs the tests for each selected attack class, with the safety rules in the red team plan template, and writes down what it tried, what happened and who tested. A tester who did not build the feature is expected at Depth 2. At Depth 3 an independent tester, who did not build or own the feature, runs them under a written plan approved before anything is sent, and writes the report. No tester is required at Depth 1. A feature with tools also gets the agent permission review. Every confirmed problem becomes a finding. Record: The test results (E-5), the independent report at Depth 3 (E-6) and the agent permission review for a feature with tools (E-7).

The test plan template (AI Red Team Test Plan Template) holds the safety rules SR-1 to SR-9 and the report parts. Three matter most when you plan. The plan is approved in writing before anything is sent (SR-1). Where a provider owns the model or the service, its terms permit the test or it has agreed in writing (SR-2). A test never releases a payment or changes supplier bank details (SR-5). If a test exposes real sensitive data or has a real effect, stop, keep the evidence and report it through the incident route (SR-8).

Test the content the feature reads, as well as what users type. Most serious cases arrive in a document, an email, a web page or a tool result.

A test that finds no problem does not show that the feature is secure. It shows what the testers tried. The report says what was not tried.

Evidence the review holds

Nine pieces of evidence make up a review. Which are required depends on the depth, on whether an outside provider supplies the model or service, and on whether the feature has tools.

ID

Evidence

Made by

At

Complete when

E-1

The description and data flow, with sources and sinks marked trusted or untrusted

AI use owner

RP-1

One dated page. Every source and every place the output goes is on it, each source marked trusted or untrusted.

E-2

The threat selection from the catalogue (A04), with the reason for each class left out

Head of Information Security

RP-2

Every threat marked selected or left out. A reason against each one left out.

E-3

The standard check: each requirement met, not met or not applicable, with evidence

Application developer

RP-3

Every requirement marked. Evidence against each met requirement. A reason against each not applicable one.

E-4

The configuration checklist (A05) result

Head of IT

RP-3

Every applicable check marked, shown as a count. Evidence and a date against each Verified check.

E-5

The test results for the selected attack classes, with who tested (an independent tester at Depth 3)

Independent tester

RP-5

For each selected attack class: tried or not tried, what happened, who tested and the date. The count of classes tried.

E-6

The independent red team report (A07)

Independent tester

RP-5

The written report with every part of the report template filled in.

E-7

The agent permission review (A09)

Application developer

RP-5

Every item of the agent permission checklist marked. Security has read it.

E-8

The provider questionnaire (A06), answered and reviewed: required from Depth 2 when an outside provider supplies the model

Procurement

RP-4

The provider's answers with evidence, read by security, with the count and the open questions.

E-9

The approval record of the approvers (recorded in the P24 inventory)

AI governance lead

RP-6

The approvers' names, roles, decision and date, and every condition with an owner and a date.

What each depth needs. Each row is the list the review must hold before the release gates are run:

Feature

Evidence required

Depth 1

E-1, E-3, E-4, E-9

Depth 2, own build, no tools

E-1, E-2, E-3, E-4, E-5, E-9

Depth 2, outside provider, no tools

E-1, E-2, E-3, E-4, E-5, E-8, E-9

Depth 2, outside provider, with tools

E-1, E-2, E-3, E-4, E-5, E-7, E-8, E-9

Depth 3, own build, no tools

E-1, E-2, E-3, E-4, E-5, E-6, E-9

Depth 3, outside provider, with tools

E-1, E-2, E-3, E-4, E-5, E-6, E-7, E-8, E-9

Stage 6 (RP-6): Decide

In practice. The AI governance lead convenes the decision, collects the file and runs the release gates. The approvers named under RG-6 read the gate result and decide. Any approver may hold. Each approver signs the decision record with name, role and date. A condition has an owner and a date. Record: The decision record (E-9), kept in the P24 inventory.

The release gates

A gate is a statement that must be true before a feature is released. The AI governance lead runs the gates in order and writes down each one that fails.

Gate

What must be true

Rule

RG-1

Every item of the required evidence for the depth is on file.

GS-29

RG-2

No configuration check that blocks release has failed or is unverified.

GS-28

RG-3

No Critical finding is open without an exception approved by the board or its equivalent.

GS-30

RG-4

No High finding is open without an exception approved by the Head of Information Security and the accountable executive.

GS-30

RG-5

At Depth 3, the independent red team report has been received and its findings are in the register.

GS-28

RG-6

The approvers sign: those of the higher of the use's P24 tier and the review depth, and the Head of Information Security as well at Depth 2 and 3. The approval record (E-9) shows it.

GS-30

RG-7

Every open Medium or Low finding has an owner and a due date in the register, or an approved exception.

GS-30

A gate that fails holds the release. Something that is open and does not fail a gate does not hold the release, and becomes a condition.

The three outcomes

Outcome

What it means

What the record holds

Release

Every gate passes and nothing is open. A new feature goes live. A running feature carries on.

The decision record and the next review date.

Release with conditions

No gate holds the release. Something is still open: a check that is not verified, or a finding that is open under a plan or an approved exception.

The decision record lists each condition with an owner and a date. A condition that is missed comes back to the approvers.

Hold

At least one gate fails. A new feature does not go live. For a feature that is already running, the AI governance lead may suspend it while the approvers decide (P24 AG-10).

The decision record names each gate that failed and what clears it. The review restarts at the stage that holds the missing item.

Who signs

The AI governance lead convenes the decision and collects the signatures. The approvers of the higher of the use's P24 tier and the review depth sign (RG-6), and from Depth 2 the Head of Information Security signs as well. A Tier 2 use reviewed at Depth 3 is signed as Tier 3: the executive decision-maker signs. Where an approver owns the feature, P24 names who takes that approver's place (P24 AG-09, IS-4): the executive decision-maker, or the board or its equivalent where the executive decision-maker owns a Tier 3 use.

Higher of the P24 tier and the depth

Signs

P24 reference

Tier 1 and Depth 1

AI governance lead

RS-05

Tier 2 or Depth 2

AI governance lead and Head of Information Security

RS-06

Tier 3 or Depth 3

Executive decision-maker and Head of Information Security

RS-07

The use owner is consulted and signs the conditions it accepts. The developer and the independent tester advise and sign nothing. The P24 responsibility matrix (RS-05 to RS-07 and RS-16) holds the rest.

Any approver may hold a release. Only the approvers named above may approve it.

Stage 7 (RP-7): Release and watch

In practice. The Head of IT switches the feature on, or leaves it on, with the settings the review verified. Keep a copy of the settings as they stand on the day. Check that the alerts go to a named person, that the switch-off works where the feature acts, and that the findings and exceptions are in the register. Put the next review date in the owner's calendar: the earlier of the date the depth sets and the review date in the P24 entry. Record: The settings copy, the alert owner, the next review date and the register entries.

Alerts are looked at within [[2]] working days by a named person. The review repeats at the depth's interval, and at once when something changes.

GS-18 A change of model, model version, system prompt, tool, retrieval source or filter setting is a change: it goes through the change control that applies to the system, the attack classes it could affect are tested again at the feature's review depth, and the change is recorded. A provider's change of model is a trigger for review within [[10]] working days of the notice (P24 ST-7; AG-10).

What forces a new review

A change in this list forces a review of the feature within [[10]] working days of the change or of the provider's notice. The owner tells the AI governance lead as soon as they know. The review repeats the stages the change touches, and repeats the tests for the attack classes it could affect.

  • A new model, or a new version of the model, including a change the provider makes.
  • A change to the system prompt or to a tool description.
  • A new tool or connector, or a change in what a tool is allowed to do.
  • A new retrieval source, or a change in who can add to one.
  • A change to a filter setting or to a safety setting.
  • A finding from an event, a test or a report that shows a gap in the review.

GS-18 names these threats as the ones a change touches: LT-01, LT-02, LT-05, LT-08 and LT-09. Their attack classes are AC-01, AC-02, AC-05, AC-08 and AC-09. Test at least those, and any other class the change could affect.

What is a new use, not a change

Some changes make a new use. It goes back to the intake in the AI Security Governance & Acceptable Use pack (P24), and this review follows its decision.

  • A new purpose for the feature.
  • A higher data level, or personal data where there was none.
  • A new audience, such as customers or the public where there were only staff.
  • More freedom to act without a person reading the output.
Signals and events

The monitoring signals AS-01 to AS-12 are in the Generative AI Security Standard. When one is confirmed, the response is not part of this review. Report an event at once through the incident route (P09 IM-01). Nobody is blamed for a report made in good faith. P09 decides whether it is an incident and whether anything must be reported, and P18's clocks apply. P09 IM-07 sets the order of containing and preserving. Where harm can still spread, such as an agent that can still act (AS-03, AS-07, AS-08) or data still leaving, contain first by switching the feature's access off, and capture what you can in the same step. Otherwise preserve the evidence first, then switch the feature off.

Keep what P09 will need (EP-01 to EP-10 in the Generative AI Security Standard). Keep the evidence as P09 IM-07 asks: preserve it before anything is changed, unless waiting would let the harm spread. If it would, contain first and capture what you can in the same step.

Findings and exceptions

GS-30 A finding is rated on P05's impact and likelihood scales, has an owner and a due date set by its severity, and is tracked in the register (A08). A departure from this standard is a P02 exception approved by the approver its band sets, and a Critical one goes to the board or its equivalent. A breach of P24's AG-03 or AG-05, and a feature that releases a payment (GS-27), take no exception.

Rating a finding

A finding is rated from its impact and its likelihood, each from 1 to 4 on the risk management scales (P05). The severity is the band of the two multiplied together.

Impact

Likelihood 1

Likelihood 2

Likelihood 3

Likelihood 4

1

Low

Low

Low

Medium

2

Low

Medium

Medium

High

3

Low

Medium

High

Critical

4

Medium

High

Critical

Critical

What each severity needs

Severity

Fix due within

An exception is approved by

Decision within

Exception lasts at most

Renewals the approvers may grant

Critical

30 days

Board or its equivalent

5 working days

30 days

1

High

90 days

Head of Information Security and Accountable executive

10 working days

90 days

1

Medium

180 days

Head of Information Security

10 working days

180 days

2

Low

365 days

Control owner

5 working days

365 days

2

The fix days for High, Medium and Low are those of the control assurance pack (P20 CA-13). A Critical finding is due in the longest time the exception pack (P02 EX-05) allows a Critical exception to run. Days are calendar days from the day the reviewer confirms the finding. The approvers and the lengths are from the exception pack (P02 EX-05, EX-06).

The renewals a band shows are the renewals its approvers may grant. A renewal beyond the band's limit goes to the next approval level up (EX-08). A renewal is a new decision.

The severity of a finding is a P05 band, and the band decides who may accept the risk it leaves (P05 RM-04, RM-08): Low, the risk owner; Medium, the risk owner and the Head of Information Security; High, the accountable executive and the Head of Information Security; Critical, the board or its equivalent. Where the risk is outside appetite, at least executive management also approves. Outside tolerance the band's approvers recommend, and only the board or its equivalent may accept.

What happens to a finding

Step

What happens

Who

F-1

Confirm

The reviewer repeats the problem and writes the steps to repeat it. A finding that cannot be repeated is noted and watched, and is not rated.

Head of Information Security;

Independent tester

F-2

Rate

The Head of Information Security rates the finding: an impact and a likelihood, each from 1 to 4 on the risk management scales, starting from the tester's proposal where there is one. The register works out the severity from the two.

Head of Information Security

F-3

Enter

The reviewer enters the finding in the register with the threat and the requirement it relates to, the feature and its P24 use, and the date confirmed. The due date follows from the severity.

Head of Information Security

F-4

Agree the fix

The owner agrees a fix and a date no later than the due date, and sets the status to Plan agreed.

AI use owner;

Application developer

F-5

Fix and retest

The developer fixes it and the status becomes Fixed, awaiting retest. Someone other than the person who fixed it repeats the test. A pass closes the finding. A fail returns it to Open.

Application developer;

Independent tester

F-6

Or ask for an exception

If the fix cannot be made by the due date, the owner asks for an exception before that date. The status becomes Deferred as an exception only when the approvers have approved.

AI use owner

A finding is always in one of these states: Open; Plan agreed; Fixed, awaiting retest; Closed; Deferred as an exception.

Exceptions

  • The request names the finding, the severity, the reason the fix is not possible by the due date, and the date by which it will be.
  • The request lists the compensating actions that lower the risk meanwhile, each with an owner.
  • The approvers for the severity decide, within the working days the band allows. The owner of the finding is never among them.
  • The exception has an expiry no later than the days the band allows. A renewal is a new decision by the same approvers. The band sets how many renewals they may grant, and a renewal beyond that goes to the next approval level up (P02 EX-08).
  • A High finding reaches the gate only with an exception approved by both approvers. A Critical finding reaches it only with one approved by the board or its equivalent.
  • On the expiry date the finding is fixed and retested, or the exception is renewed, or the feature is held.

RG-3 and RG-4 apply the exception rules at the release gate. An open Medium or Low finding does not hold a release. It becomes a condition with an owner and a date.

What takes no exception

A breach of P24 AG-03 or AG-05 takes no exception. Nor does a feature that releases a payment or changes supplier bank details (GS-27).

GS-27 No feature releases a payment or changes supplier bank details, by any tool or path (P24 AG-16). A feature may draft a payment request that a named person then makes through the normal approval. This requirement takes no exception (GS-30).

Records

GS-29 The owner keeps for each feature the data flow, the threat selection, the standard check, the configuration checklist, the test results, the findings and exceptions, and the approval, while the feature runs and for [[3]] years after it ends, or longer where law or a contract asks.

The review file holds E-1 to E-9 as the depth requires, the gate result, the findings and exceptions, and the decision record. A held or stopped review is kept as well: it shows what was asked and why the answer was no. The decision record is also recorded in the P24 inventory (AI System & Use Case Inventory).

Worked review cases

EXAMPLE, not part of the procedure. 13 features, each run through the depth rules. Every feature starts from the same plain case: Internal information, no personal data, staff only, a person reads every output, no effect on a person, a low operational effect, an approved supplier, a bar on training on the organisation's data and processor terms in place. Each row changes only what the case says. The depth is what the rules give.

Case

Feature

P24 tier

Rule of two

Depth

Deciding rule

RD-A

The supplier's office-suite assistant, used as supplied, drafts and summarises internal text, and reads documents that come from suppliers

Tier 1

2 of 3

Depth 2

GS-24

RD-B

A design tool's image generator, Public data, no tools

Tier 1

0 of 3

Depth 1

GS-28

RD-C

A customer chatbot that looks up order status and answers on its own

Tier 3

3 of 3

Depth 3

Independent test

GS-24

RD-D

A supplier's assistant, used as supplied, that answers staff questions over company documents, Internal data, no personal data

Tier 1

1 of 3

Depth 2

GS-09

RD-E

The same supplier's assistant over HR files with personal data

Tier 2

1 of 3

Depth 2

GS-09

RD-F

A supplier's coding assistant, used as supplied, that runs commands, reads public repositories and has access to source code

Tier 2

3 of 3

Depth 3

Independent test

GS-24

RD-G

A summary of supplier call recordings, with names

Tier 2

1 of 3

Depth 2

GS-28

RD-H

A video-call tool that reports the mood of the team

Stop

Not run

No review

AG-05

RD-J

The team builds a staff assistant over company documents on a supplier's model API (Internal data, no personal data), and Legal has not recorded the role

Tier 3

1 of 3

Depth 3

Independent test

GS-28

RD-K

The same built assistant after Legal records the deployer role

Tier 1

1 of 3

Depth 2

GS-09

RD-L

The same built assistant, but Legal records the provider role

Tier 3

1 of 3

Depth 3

Independent test

GS-28

RD-M

A built assistant over company documents that was already running before the standard, role not yet recorded (a dated condition under P24)

Tier 1

1 of 3

Depth 2

GS-09

RD-I

A supplier's agent, used as supplied, with memory that sends no mail and reads no outside content, Public data

Tier 1

0 of 3

Depth 2

GS-14

Of the 13, 1 is Depth 1, 7 are Depth 2, 4 are Depth 3 and 1 is stopped by P24.

Three cases read closely

RD-A: The supplier's office-suite assistant, used as supplied, drafts and summarises internal text, and reads documents that come from suppliers
  • P24 tier 1 sets Depth 1. (GS-28)
  • The feature retrieves company content above Public, so access control at the index needs a test. (GS-09)
  • Two of the three properties of the rule of two hold (untrusted content and sensitive data). The review records the residual risk. (GS-24)

The P24 tier is 1, so the use is easy to approve. The review is Depth 2 because the feature reads documents from suppliers and handles Internal data: two properties of the rule of two. The decision takes up to [[15]] working days, not [[5]], and the Head of Information Security signs as well. The review records what remains: a hidden instruction in a supplier's file could change a summary, so a person reads every output.

RD-F: A supplier's coding assistant, used as supplied, that runs commands, reads public repositories and has access to source code
  • P24 tier 2 sets Depth 2. (GS-28)
  • The feature has tools. (GS-22)
  • The feature reads untrusted content, reaches sensitive data and can change state or communicate outside: all three of the rule of two. (GS-24)

All three properties hold, so a person approves every action and the test is independent, whatever the P24 tier. Here the P24 tier is 2 and the depth is 3, so the use is signed as Tier 3: executive decision-maker and head of information security sign, and the decision takes up to [[30]] working days. Compare RD-C, where P24 gives Tier 3 and the executive decision-maker signs.

RD-J to RD-M: a feature built on a model API
  • P24 tier 3 sets Depth 3. (GS-28)
  • The feature retrieves company content above Public, so access control at the index needs a test. (GS-09)

The team builds the assistant on a supplier's model API, so P24 counts it as building even though only staff use it. It is Tier 3 and held until Legal records which role the organisation holds, so the review runs at Depth 3. When Legal records the deployer role (RD-K), P24 sets the tier again from the answers. Here the tier falls to 1, and the retrieval of company content puts the review at Depth 2. The review is worked out again at that point; it does not stay at Depth 3 because the question was once answered Yes. If Legal records the provider role or another role instead (RD-L), P24 clears the hold but keeps the feature at Tier 3, so the review stays at Depth 3: the organisation is outside P24's deployer governance, and the executive decision-maker decides. This applies to a new feature. A feature already running when the standard starts (RD-M) is not raised to Tier 3 by the signal; P24 carries the hold as a condition with a date, and the review depth follows the feature's tier and properties (Depth 2 here).

Worked example: the distributor's September 2026 reviews

EXAMPLE, not part of the procedure. The organisation used throughout the packs is a wholesale distributor with about 900 staff and three warehouses, an important entity under NIS2 in its Member State, and not a financial entity, so DORA's reporting duties do not apply. The Executive Committee approved the standard on 8 Sep 2026. The Head of Information Security owns the standard, and nobody approves a document they own or wrote (P04 PF-03), so the Executive Committee approves it. In scope: the three generative uses on the P24 inventory: AI-001, AI-003 and AI-004. AI-002 (forecasting) and AI-006 (anomaly detection) are not generative. AI-005 was stopped and AI-007 declined under P24. Existing features are due for review by 7 Dec 2026. Values are as at 30 Sep 2026.

Three features through the procedure

Feature

P24 use

Depth

Rule of two

Requested

Decided

Gate result

LF-1

Draft and summarise with the office-suite assistant

AI-001

Tier 1

Depth 2

2 of 3

14 Sep 2026

25 Sep 2026

9 working days from request

Release with conditions

LF-2

Image generator in the design tool

AI-004

Tier 1

Depth 1

0 of 3

24 Sep 2026

28 Sep 2026

2 working days from request

Release with conditions

LF-3

Customer-service chatbot proposed by the contact centre

AI-003

Tier 3

Depth 3

3 of 3

23 Sep 2026

Not decided

Hold

2 of 3 generative AI features have a completed review; 27 of 32 configuration checks on live features are verified; 0 High or Critical findings are open without an exception; 0 findings are past their date; 1 of 2 providers have returned the security questionnaire, with 7 questions still open.

LF-1: the office-suite assistant (Depth 2)

Reads documents that suppliers send, so the review treats that content as untrusted. No tools. A person reads every output (P24 T-01 conditions). The rule of two gives 2 of 3 properties, so the review records the residual risk. The decision falls inside the [[15]] working days of Depth 2. The evidence on file is E-1, E-2, E-3, E-4, E-5, E-8, E-9. The team tested on 17 Sep 2026 and 18 Sep 2026; the review names who tested ([[name of the tester, who did not build the feature]]). The next review is due by 25 Mar 2027: the depth sets 25 Mar 2027 and the P24 entry sets 2 Sep 2027, and the earlier applies.

The configuration checklist stands at 14 of 17 applicable checks verified; 0 failed, 3 not verified. Three checks are not verified, and none of them holds the release:

  • DC-06. The provider's page names a region; the contract schedule is awaited.
  • DC-09. The time to remove a deleted document from the index is not stated by the provider.
  • DC-18. Alert rules for AS-01 to AS-12 are being written.

The provider returned the questionnaire on 25 Sep 2026. The count is 24 of 31 applicable questions answered with evidence; 5 answered without evidence, 2 not answered. The open questions are VQ-03, VQ-10, VQ-14, VQ-15, VQ-16, VQ-20 and VQ-26, to be answered by [[date agreed with the provider for the open items]].

The gate result is Release with conditions. The conditions are: checks not yet verified: DC-06, DC-09, DC-18; AF-001 (Medium) is open under an approved exception.

AF-001: the one finding, and its exception

A summary of a supplier's price list in PDF repeated hidden text in the file that said the supplier was approved. Found when the tester planted the line in a test copy on 18 Sep 2026. The reviewer rated impact 2 and likelihood 2, which is Medium. It was confirmed on 18 Sep 2026, so the fix is due on 17 Mar 2027. The Head of Information Security approved exception AX-001 on 25 Sep 2026, 4 working days after the request and inside the 10 working days the band allows. It ends on 29 Jan 2027, before the fix is due.

Why. The provider cannot yet remove hidden text before summarising and has been asked for a fix (P24 ST-7). The assistant has no tools, and a person reads every output before it is used or sent (P24 T-01 conditions).

Compensating actions.

  • Staff are told that a summary of an outside document is not a substitute for opening it.
  • The tester repeats the planted-text test each month and when the provider changes the model.
  • The finding is reviewed again on 29 Jan 2027, or sooner if the provider ships a fix.
LF-2: the image generator (Depth 1)

Public information only, and every published image is signed off by the marketing lead (P24 T-03). Two settings cannot be seen from the tenant and wait for the provider. The next review is due by 23 Sep 2027: the depth sets 28 Sep 2027 and the P24 entry sets 23 Sep 2027, and the earlier applies. The feature has no tools and does not read untrusted content, so the evidence is E-1, E-3, E-4, E-9 and the review needs no separate test and no tester. The provider's published security statement is read instead of the questionnaire. The checklist stands at 13 of 15 applicable checks verified; 0 failed, 2 not verified. The two checks that are not verified wait for the provider:

  • DC-05. The tenant administrator cannot see the retention period; asked of the provider.
  • DC-06. The data location is not stated on the provider's page.

The gate result is Release with conditions, decided on 28 Sep 2026. Depth 1 does not require the questionnaire, so it has not been sent. [[P06 reference of the design tool supplier]] is the reference to enter if the next supplier review sends it.

LF-3: the customer-service chatbot (Depth 3), held

Held under P24 until the contact centre's P06 assessment (planned 1 Dec 2026), a bar on training on our data and processor terms are in place. The review waits for the questionnaire and the red team test. All three properties of the rule of two hold: it reads customers' words, it reaches data above Public and it sends output outside. The review is Depth 3 and the approvers are the executive decision-maker and the Head of Information Security. The threats were selected on 23 Sep 2026.

The gate result is Hold. Four gates fail:

  • RG-1. Evidence missing: E-3, E-4, E-5, E-6, E-7, E-8, E-9.
  • RG-2. Checks that hold the release: DC-01, DC-02, DC-04, DC-07, DC-10, DC-11, DC-14, DC-15, DC-16, DC-30.
  • RG-5. The independent red team report has not been received.
  • RG-6. The approval record (E-9) is not on file, so the approvers have not signed.

Only E-1 and E-2 are on file. The checklist has not been run, the provider has not received the questionnaire and no test has taken place. A Hold is not a finding. It says the file is not ready.

The red team plan for LF-3

Item

Value

Plan

RTP-2026-01 (Planned), approved on 29 Sep 2026

Window

9 Nov 2026 to 20 Nov 2026

Environment

The contact centre's test environment, with test accounts and marked test data

Tester

[[independent tester, named in the plan]]

Owner and stop contact

Chief Operating Officer; Head of Information Security

Attack classes selected

11 of 12: AC-01, AC-02, AC-03, AC-04, AC-05, AC-06, AC-07, AC-08, AC-09, AC-11, AC-12

Plan approvers (SR-1)

Chief Operating Officer: [[name of the owner who approves the plan]]; Head of Information Security: [[name of the Head of Information Security]]

Provider permission (SR-2)

Requested from SUP-013 on 29 Sep 2026; not yet received (SR-2)

The one classes left out each have a reason:

  • AC-10. The chatbot has no vector store.

The test is planned for 9 Nov 2026 to 20 Nov 2026, and the provider's permission must arrive before it starts. The release gate stays on Hold until the report (E-6) is on file and its findings are in the register.

Document

Relationship

Generative AI Security Standard

The requirements GS-01 to GS-31 this procedure checks, and the monitoring signals

LLM Application Threat Catalogue

The twelve threats the review selects from

Generative AI Deployment Configuration Checklist

The configuration checks completed at Stage 3

AI Vendor & Model Provider Security Questionnaire

The provider questionnaire sent at Stage 4

AI Red Team Test Plan Template

The plan for an independent test at Depth 3, with the safety rules and the report parts

AI Security Findings & Exceptions Register

Where every finding and exception is recorded

AI Agent Permission & Tool-Access Review Checklist

The permission review for any feature with tools

AI Use Case Intake & Approval Procedure (P24)

The intake, the tiers and who approves a use

AI System & Use Case Inventory (P24)

The inventory that records each feature and the decision record

Security Exception, Waiver & Segregation of Duties pack (P02)

Exceptions: who approves each band, how long one may run and where a renewal beyond the band's limit goes (EX-05, EX-06, EX-08)

Information Security Risk Management pack (P05)

The impact and likelihood scales that rate a finding

Third-Party Security Risk Management pack (P06)

The supplier decision and the contract clauses

Security Incident Management pack (P09)

The incident route (IM-01) and the preservation of evidence (IM-07)

Security Control Assurance pack (P20)

Days to fix a deficiency (CA-13) and the rule that nobody tests their own control (CA-10)

Adapting this template

Guidance — delete before approval

Small organisation: keep the seven stages and let one person hold several roles. The IT manager can be the AI governance lead and the Head of IT, and the owner of a feature can be its developer. A one-page form and a spreadsheet are enough at Depth 1. At Depth 3 you cannot test your own feature, so buy the test from an independent tester. Where one person holds the roles of owner and approver, a second person signs the decision.

Managed service provider or outsourced IT: if an outsourced IT service provider builds, configures or runs the feature, it is the developer in this procedure. Ask it for E-3 and E-4 and the evidence behind them, and for notice of any change that forces a review. The provider must not be the independent tester of its own work. The decision stays with you.

Regulated entity: a financial entity, a provider under NIS2 or an organisation that deploys a high-risk system has duties this procedure does not set. Agree with [[legal or compliance]] which apply, and add the evidence and the sign-offs they need. Do not remove a gate or lower a depth.

Setting the numbers: the days to decide, the months between reviews and the days to review a change are house defaults, shared with the standard and with P24. Raise them only if the process is working at the current ones.

Delete this section before approval.

Framework references

These references show where this document supports an external framework. They indicate relevance only and do not reproduce the text of any standard. Editions referenced: Regulation (EU) 2024/1689 (the EU AI Act) as amended by Regulation (EU) 2026/1744; Regulation (EU) 2016/679 (GDPR); OWASP Top 10 for LLM Applications, 2026 edition; MITRE ATLAS 2026.09; NIST AI 600-1; NIST AI RMF 1.0; NIST CSF 2.0; ISO/IEC 27001:2022 incl. Amd 1:2024; ISO/IEC 42001:2023 (by number and title only); as published and read on 8 October 2026.

Framework

Reference

Supported by

ISO/IEC 27001:2022

Annex A 8.25 — Secure development life cycle

Stages RP-1 to RP-7: security checked at each step in the life of the feature

ISO/IEC 27001:2022

Annex A 8.29 — Security testing in development and acceptance

Stage RP-5: security testing before release and at each review

ISO/IEC 42001:2023

Clause 6.1.2 — AI risk assessment

Stage RP-2: the threat selection and the rating of findings

ISO/IEC 42001:2023

Clause 6.1.4 — AI system impact assessment

Stages RP-1 and RP-2: the data flow and the threats are inputs to the assessment held under P24

ISO/IEC 42001:2023

Annex A 5.2 — AI system impact assessment process

Stage RP-2: the process that decides how a feature is assessed

ISO/IEC 42001:2023

Annex A 9.2 — Processes for responsible use of AI systems

Stages RP-1 to RP-7: a defined process for the responsible use of an AI system

NIST CSF 2.0

ID.RA-05 — “Threats, vulnerabilities, likelihoods, and impacts are used to understand inherent risk and inform risk response prioritization”

Stages RP-2 and RP-6: threats, likelihoods and impacts used to rank findings and to decide

EU AI Act — Regulation (EU) 2024/1689

Article 15(1) — high-risk AI systems “shall be designed and developed in such a way that they achieve an appropriate level of accuracy, robustness, and cybersecurity, and that they perform consistently in those respects throughout their lifecycle”. The Article is a requirement on high-risk systems, which providers meet; it applies to Annex III systems from 2 December 2027 (Article 113 as amended)

The release gates: the Act's own statement of the security a high-risk system needs. It binds the provider of such a system. This procedure uses it as a benchmark and cites no feature as high-risk

EU AI Act — Regulation (EU) 2024/1689

Article 15(4) — high-risk systems “shall be as resilient as possible regarding errors, faults or inconsistencies that may occur within the system or the environment in which the system operates, in particular due to their interaction with natural persons or other systems”, and technical redundancy solutions, including backup or fail-safe plans, may achieve robustness

Stages RP-5 and RP-7: resilience against errors and faults, which the cap on steps and the switch-off test relate to

EU AI Act — Regulation (EU) 2024/1689

Article 25(1) — a distributor, importer, deployer or other third party “shall be considered to be a provider of a high-risk AI system” and subject to the provider’s obligations where it puts its name or trademark on a high-risk system already on the market, makes a substantial modification to a high-risk system that remains high-risk, or modifies the intended purpose of a system that was not high-risk so that it becomes high-risk under Article 6

Stage RP-1: a feature the organisation builds, puts its own name on or gives a new purpose goes to Legal

EU AI Act — Regulation (EU) 2024/1689

Article 26(1) — deployers of high-risk AI systems “shall take appropriate technical and organisational measures to ensure they use such systems in accordance with the instructions for use accompanying the systems”. The obligations of Article 26 applies from 2 December 2027 to systems classified as high-risk under Article 6(2) and Annex III, and from 2 August 2028 to those under Article 6(1) and Annex I (Article 113, third paragraph, point (c), as amended by Regulation (EU) 2026/1744)

Stage RP-4: a deployer follows the provider's instructions for use, which the questionnaire asks for

EU AI Act — Regulation (EU) 2024/1689

Article 113, as amended — the Regulation applies from 2 August 2026, except that Chapters I and II (including Articles 4 and 5) apply from 2 February 2025, with Article 5(1) points (ba) and (bb) and Article 5(1a) and (1b) applying from 2 December 2026; Chapter III Section 4, Chapter V (general-purpose AI models), Chapter VII and Chapter XII and Article 78 apply from 2 August 2025, except Article 101; and Chapter III Sections 1, 2 and 3, except Article 6(5), apply from 2 December 2027 as regards systems classified as high-risk under Article 6(2) and Annex III, and from 2 August 2028 as regards those under Article 6(1) and Annex I. Article 50 is not listed in an exception and so applies from 2 August 2026

The legal note: when each part of the Act applies after Regulation (EU) 2026/1744

EU AI Act — Regulation (EU) 2026/1744 (Digital Omnibus on AI)

Regulation (EU) 2026/1744 of 8 July 2026 amending Regulations (EU) 2024/1689, (EU) 2018/1139 and (EU) 2023/1230 as regards the simplification of the implementation of harmonised rules on artificial intelligence (Official Journal, 24 July 2026); in force on the third day after publication (its Article 4), 27 July 2026. Its Article 1 amends Regulation (EU) 2024/1689

The legal note: the amending Regulation, in force from 27 July 2026

GDPR — Regulation (EU) 2016/679

Article 25(1) — the controller, “both at the time of the determination of the means for processing and at the time of the processing itself”, implements appropriate technical and organisational measures, such as pseudonymisation, “designed to implement data-protection principles, such as data minimisation, in an effective manner”, taking into account the state of the art, the cost, the nature, scope, context and purposes of processing and the risks to people

Stage RP-1: the data flow shows what personal data goes where, so data protection by design can be checked

GDPR — Regulation (EU) 2016/679

Article 32(1)(d) — the security measures include, as appropriate, “a process for regularly testing, assessing and evaluating the effectiveness of technical and organisational measures for ensuring the security of the processing”

Stages RP-5 and RP-7: regular testing of the security measures, before release and at each review

GDPR — Regulation (EU) 2016/679

Article 35(1) — where a type of processing, in particular using new technologies, is likely to result in a high risk to the rights and freedoms of natural persons, the controller carries out “prior to the processing” an assessment of the impact on the protection of personal data; a single assessment may address a set of similar processing operations with similar high risks

Stage RP-1 and the new-use triggers: a change that adds personal data goes back to P24, where the data protection impact assessment is decided

NIS2 — Directive (EU) 2022/2555

Article 21(2)(e) — “security in network and information systems acquisition, development and maintenance, including vulnerability handling and disclosure”

Stages RP-3 to RP-6: security in the development and maintenance of the system, including vulnerability handling

OWASP Top 10 for LLM Applications 2026

OWASP GenAI Security Project, OWASP Top 10 for Large Language Model Applications, version 2026, published August 2026: LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM03 Excessive Agency, LLM04 Supply Chain, LLM05 Data and Model Poisoning, LLM06 Unbounded Consumption, LLM07 Misinformation, LLM08 Hidden Context Exposure, LLM09 Vector and Embedding Weaknesses, LLM10 Improper Output Handling. A community awareness document, weighted about three-quarters on a community vote and one-quarter on incident data, that states its own limits; it replaces the 2025 edition

Stage RP-2: the ten classes the threat catalogue draws on

Guidelines for secure AI system development (NCSC, CISA and partners)

Secure design, “Model the threats to your system” — “you apply a holistic process to assess the threats to your system, which includes understanding the potential impacts to the system, users, organisations, and wider society if an AI component is compromised or behaves unexpectedly”, and document the decision making

Stage RP-2: model the threats to the system and document the decisions

Guidelines for secure AI system development (NCSC, CISA and partners)

Secure deployment, “Release AI responsibly” — “You release models, applications or systems only after subjecting them to appropriate and effective security evaluation such as benchmarking and red teaming”, and you are clear to users about known limitations and failure modes

The release gates: release only after a security evaluation such as red teaming

NIST AI RMF 1.0

MEASURE 1.3 — “Internal experts who did not serve as front-line developers for the system and/or independent assessors are involved in regular assessments and updates.”

Stage RP-5: someone who was not a front-line developer tests at Depth 3

NIST AI RMF 1.0

MEASURE 2.7 — “AI system security and resilience – as identified in the MAP function – are evaluated and documented.”

Stages RP-3 to RP-5: security and resilience are evaluated and documented

NIST AI RMF 1.0

MANAGE 1.1 — “A determination is made as to whether the AI system achieves its intended purposes and stated objectives and whether its development or deployment should proceed.”

Stage RP-6: a determination whether the feature should proceed

NIST AI 600-1

NIST AI 600-1, section 2.9 Information Security — generative AI “expands the available attack surface, as GAI itself is vulnerable to attacks like prompt injection or data poisoning”; “Indirect prompt injection attacks occur when adversaries remotely (i.e., without a direct interface) exploit LLM-integrated applications by injecting prompts into data likely to be retrieved”; and information security for these systems includes “maintaining availability of the GAI system and the integrity and (when applicable) the confidentiality of the GAI code, training data, and model weights”

Stage RP-5: indirect prompt injection is tested, as well as what users type

ISO/IEC 27001 and ISO/IEC 42001 references name the clause or control this document relates to, by number and title only. ISO/IEC 42001 is a paid standard whose text was not read for this document. A reference does not say that meeting a clause or control meets any requirement of the EU AI Act or the GDPR.

Definitions

Term

Meaning in this procedure

Feature

One LLM capability the organisation uses, builds or has switched on, for one purpose, with one owner. A feature is the technical view of a P24 use.

Untrusted content

Any content the feature reads that came from outside the organisation or from a person who can submit text: customers' messages, supplier documents, inbound email, web pages, tickets, public files, and the output of a tool that fetches such things.

Tool

A function the model can ask the application to run: search, read a file, send an email, run a query, run code. Also called a plugin, extension, skill or connector. A connector gives access to another system.

Agent

A feature in which the model chooses which tools to call and in what order, with tool results fed back to it. An agent can act, so P24 treats a system that acts as Tier 3 until assessed (AG-16).

Red team

A planned, authorised attempt to make a feature fail the way an attacker would, by someone other than its builder, with the results reported.

Finding

A confirmed gap between a feature and this standard, found in a review, a configuration check, a test or an incident, rated on P05's scales.

Exception

A recorded, time-limited approval to depart from a requirement, made under P02 and approved by the approver the finding's band sets.

Event

A suspected security event reported through the incident route (P09 IM-01). It becomes an incident only when it is declared there.

Review depth

How much review a feature gets: Depth 1, 2 or 3. It follows the P24 tier and is raised by what the feature has. Depth 0 means P24 has stopped the use.

Rule of two

A count of three properties of a feature: it reads untrusted content, it reaches sensitive data, and it can change state or communicate outside the organisation.

Release gate

A statement that must be true before a feature is released. A gate that fails holds the release.

Prepared by Zero One Cybersecurity · Published by CISO Times

CISO Times Resource Licence — Internal Use