How do we decide which decisions an incident exercise should test?

Start

An exercise is due and no scenario is written yet

An exercise is worth running only if it tests the decisions the incident response plan asks people to take: who isolates, who declares, who reports and by when, who speaks outside, who decides on a ransom. So the objectives come first and the scenario is written to test them. A scenario written first tests whatever it happens to reach.

• Then: Confirm which audience is due and what it decides

Step

Confirm which audience is due and what it decides

Each audience takes different decisions. The board, or its equivalent, meets the decisions only it takes: a ransom demand, a public statement, stopping a core service for days, spending outside budget. The executive and crisis team carries the yearly exercise of the plan. The incident response team runs the first hours. Technical responders are tested on isolation, restores and the out-of-hours route, in a technical tabletop or in a drill that does the step for real.

• Then: Does the plan say who takes each decision, and by what rule?

Decision

Does the plan say who takes each decision, and by what rule?

Every objective is judged against a rule or plan step, so the rule has to exist first. Check the plan, the severity matrix, the playbooks and the communication templates for the decisions this audience takes. If a decision has no owner or no rule, or you cannot tell, the exercise has nothing to rate it against.

• Yes: Is a re-test due, or was a decision left untested last time?
• No, or not known: Write the missing step first, in an unrated workshop

Step

Write the missing step first, in an unrated workshop

Where the plan has no rule for a decision, the pack writes the missing step first, in a workshop that is not rated. Rating people against a rule nobody wrote tests nothing. Then set the objective against the new step.

• Then: Does the plan say who takes each decision, and by what rule? (back to an earlier step)

Decision

Is a re-test due, or was a decision left untested last time?

An objective rated Partly met or Not met must be re-tested once its improvement action is done, within 6 months in the pack's default. An objective the last exercise never reached was rated Not tested; it is carried forward and never counted as met. If you cannot tell what is due, the pack re-tests every objective not Met last time.

• Yes, or not known: Carry those decisions forward, in the same category
• No: Choose the scenario from a top risk or a recent incident

Step

Carry those decisions forward, in the same category

A re-test is written as the same decision, in the same category, adapted to the new scenario. It also steers the scenario: where several qualify, a re-test due comes first in the pack's order, and the pack picks a scenario with an objective in the same category that tests the same decision.

• Then: Choose the scenario from a top risk or a recent incident

Step

Choose the scenario from a top risk or a recent incident

Each scenario must come from a top risk in your risk register or a recent incident, and the same audience never gets the same scenario twice running. Where several qualify, the pack takes the first that applies: a re-test due; a top risk outside appetite or tolerance; a Severity 1 or 2 incident, or a near miss on a top risk, in the last 12 months; a top risk this audience has never exercised; the top risk it exercised longest ago.

• Then: Draft three to six objectives across the eight categories

Step

Draft three to six objectives across the eight categories

The categories are fixed: detection and escalation, containment decisions, reporting duties, communication, recovery priorities, decision rights and governance, supplier coordination, evidence and records. Keeping them fixed is what lets this year's result be compared with last year's. Spread the objectives across them. In the pack's defaults, three suit a 90-minute session and six is the most a 3-hour tabletop can test well.

• Then: Test who may contain the harm without waiting

Step

Test who may contain the harm without waiting

The containment category asks whether the right person stopped the harm spreading, in time, without waiting for permission they did not need. In the pack's incident policy, the incident manager may stop a customer-facing service where waiting would let the harm spread. The pack's own worked example rated this Partly met: isolation came 40 minutes after declaration, after a debate about a permission the plan did not require.

• Then: Include decisions for when a person or access is missing

Step

Include decisions for when a person or access is missing

Plans tend to assume their people and tools are there. Complication injects make something the plan relies on fail: a backup, a person, a tool, a log. The pack's scenarios send the incident manager home after 26 hours, leave the only other global administrator needing a code from a phone that belongs to an engineer on holiday, and put the Data Protection Officer on leave as the notification clock runs. The decision to test is who takes over, and what the fallback is.

• Then: Include a critical supplier's failure where one carries the service

Step

Include a critical supplier's failure where one carries the service

Where a critical supplier runs part of the service, its failure tests decisions of its own: do you know what depends on it, and do you use the contract to get its help? The pack's supplier outage scenario tests both, and whether you declare your own incident and set its severity on your own impact. Critical suppliers with a part in the plan join an exercise that involves their service.

• Then: Does each name a decision, who takes it, the rule and what Met is?

Decision

Does each name a decision, who takes it, the rule and what Met is?

An objective has five parts: its category, the decision, who is entitled to take it, the rule or plan step it is judged against, and a success statement saying what Met looks like and, where it matters, by when. If any part is missing, an observer cannot rate it and every rating becomes an opinion.

• Yes: Has the sponsor approved them before the scenario is written?
• No, or not sure: Rewrite it as a decision someone can be rated on

Step

Rewrite it as a decision someone can be rated on

'Participants understand the plan' tests knowledge. 'Test our ransomware response' is a topic. 'Communicate effectively' cannot be rated. 'Improve our GDPR awareness' is training. Each becomes an objective once it names the decision, who takes it and the rule: for example, the incident manager decides to isolate the affected systems without waiting for the executive.

• Then: Does each name a decision, who takes it, the rule and what Met is? (back to an earlier step)

Decision

Has the sponsor approved them before the scenario is written?

The sponsor, the executive who asks for the exercise, approves what is to be tested before anyone writes the story. The approval, dated before the scenario, is also what shows an auditor the exercise was a real test. In the pack's default timeline it comes 40 working days before the exercise.

• Yes: Write injects so every objective is tested at least once
• No, or not known: Hold the scenario until the objectives are approved

Outcome

Hold the scenario until the objectives are approved

An exercise without approved objectives cannot be rated, and the pack's final check postpones it. If the sponsor has not approved them 5 working days after they were due, in the pack's default, the pack takes it to executive management for a new date or a new sponsor.

Step

Write injects so every objective is tested at least once

Every inject forces at least one decision, and the facilitator's card names the decision, the expected good answer and the rule. A good scenario uses at least three of the five kinds: development, outside contact, complication, pressure, deadline. In the pack's defaults, no more than one inject every 10 to 15 minutes, and at most one complication per phase: a scenario where everything fails tests nothing.

• Then: Does every objective have an inject that forces its decision?

Decision

Does every objective have an inject that forces its decision?

Before the scenario is approved, map each objective to the injects that test it, and check they are spread across the phases. An objective with no inject will be rated Not tested and carried to the next exercise.

• Yes: Keep what Met looks like with the facilitator and observers
• No, or not sure: Write injects so every objective is tested at least once (back to an earlier step)

Outcome

Keep what Met looks like with the facilitator and observers

Participants hear each objective and the decision it tests at the pre-brief. They never see the injects or the success statements in advance: a room that knows the answer plays to it. Have someone who will not take part check the scenario for realism, then run it.