If a service cannot be back within its recovery time, how do we keep the business running?

Start

A critical service will not be back within its recovery time

Some services will not be back in time, whatever the plan says, and after a destructive attack most will not. You may learn it from a test or the objectives checks, or in the middle of an outage. Either way the business keeps running on a workaround agreed in advance.

• Then: Is the service down now?

Decision

Is the service down now?

If it is down, or expected to be, go straight to the crisis branch: there is no time to design a workaround from scratch. If you found the shortfall in a test, a dependency review or the cyber recovery time, you have time to prepare the workaround properly.

• Yes, or expected to be: The crisis lead invokes the plan; the workaround starts at once
• No, found in planning or a test: Record the shortfall as an objective gap

Step

Record the shortfall as an objective gap

A failed objectives check, or an objective a test missed, is an objective gap. By the pack's default its owner agrees the change that closes it within 10 working days, with a due date, and the service's risk owner is told so the risk is reassessed. The workaround carries the service until the change is made.

• Then: Would it be back within its maximum tolerable downtime?

Decision

Would it be back within its maximum tolerable downtime?

Compare the time a test took, and the time after a destructive attack, with the maximum tolerable downtime (MTD). Beyond the recovery time but within the MTD, the service needs its workaround. Beyond the MTD, it cannot recover in time. In the pack's example, dispatch would take 17 hours after an attack against an 8-hour RTO and a 24-hour MTD.

• Yes: The business owner records the workaround and who starts it
• No, or not known: Plan a partial and a long-term option as well

Step

Plan a partial and a long-term option as well

A service that would be back only after its MTD cannot recover in time as things stand: harm reaches Severe before it returns. The gap closes only when its checks pass or the business owner re-agrees the objective on a new analysis. Until then, plan which part of the service comes back first, and what you would use if the primary recovery is not feasible soon: another platform, supplier or site, or a lasting manual process.

• Then: The business owner records the workaround and who starts it

Step

The business owner records the workaround and who starts it

The business owner writes and accepts it. It says how much of the service it keeps, how long it can run before harm reaches Severe, who starts it and who runs it, what it needs, and how it ends. For example, dispatch from printed pick lists of the next day's orders, printed each evening. If the owner will be leading the crisis, name someone else to start it in the first hour.

• Then: Would the workaround still run if an attack took the systems?

Decision

Would the workaround still run if an attack took the systems?

What the workaround needs are dependencies too. A workaround that relies on a system the attack also takes down is no workaround: phones on the same network, or a system that waits for the same foundations. If you cannot tell, take the cautious branch.

• Yes: Has it been tried with the people who would run it?
• No, or not known: Change it so it needs nothing the attack would take

Step

Change it so it needs nothing the attack would take

Move what it depends on outside the systems at risk, as the pack's example does with pick lists printed in advance each evening. Then record what the workaround needs as dependencies of the service, so the register shows them.

• Then: Has it been tried with the people who would run it?

Decision

Has it been tried with the people who would run it?

Only a trial shows that the phones, the forms and the people are there. Try it in a drill or on a quiet day. A trial does not count as a recovery test; it is the only way to know the workaround works. In the pack's example, four of five workarounds were untried.

• Yes: Keep it in the plan with its limit and last trial date
• No: Record it as untried in the plan and book a trial

Outcome

Record it as untried in the plan and book a trial

Marking it untried stops anyone relying on it before it has been shown to work. Book a trial, perhaps alongside the service's next recovery test, so the plan can show how long the workaround really lasts.

Outcome

Keep it in the plan with its limit and last trial date

Record when it was last tried, how, and what was learned, and how long it keeps harm below Severe. Keep a copy of the plan where it can be reached when the main systems are down. Review it with the plan after every test and real recovery.

Step

The crisis lead invokes the plan; the workaround starts at once

The crisis lead invokes the service's continuity plan and logs the time; the recovery time counts from the decision to recover. The workaround starts at invocation, before anyone knows how slow recovery will be. If an attack is the cause or may be, nothing is restored or reconnected until the incident manager agrees.

• Then: Is the business owner free to start it in the first hour?

Decision

Is the business owner free to start it in the first hour?

No one should hold two roles that must act in the same hour. A business owner who is also the crisis lead, or holds another first-hour role, cannot start the workaround as well. In the pack's example, online ordering's owner leads the crisis, so a manager they named starts its workaround.

• Yes: Run it, and keep every paper record for reconciliation
• No, or not known: The person named for the first hour starts it

Step

The person named for the first hour starts it

Use the name in the plan. If nobody was named, or that person cannot be reached, someone must still start it: the service's deputy business owner is the natural choice, and your plan should say so. Record in the crisis log who acted. Starting the workaround cannot wait until the crisis lead is free.

• Then: Run it, and keep every paper record for reconciliation

Step

Run it, and keep every paper record for reconciliation

Tell the people who run it how long it may need to last. A workaround that keeps data on paper adds to the reconciliation after recovery, so keep the paper. Plan on an untried workaround lasting less than its plan says.

• Then: Will the workaround last until the service returns?

Decision

Will the workaround last until the service returns?

Compare how long the workaround can keep harm below Severe with the recovery's target time. Recheck at every crisis team update. If it is untried, or you cannot tell, take the cautious branch.

• Yes: Switch back on the plan's criteria; reconcile; record the gap
• No, or not known: The business owner tells the crisis lead, who may change the order

Step

The business owner tells the crisis lead, who may change the order

When a workaround nears its limit, its business owner tells the crisis lead. The crisis lead sets the recovery order and may move this service earlier, recording why. The order normally runs foundations first, then services by recovery time, shortest first.

• Then: Is it still heading past its maximum tolerable downtime?

Decision

Is it still heading past its maximum tolerable downtime?

After any change to the order, check again whether the service and its workaround together will keep harm below Severe until it returns. If not, or you cannot tell, escalate.

• No: Bring back the most important part first
• Yes, or not known: Add people, supplier help or partial service; tell the approver

Outcome

Add people, supplier help or partial service; tell the approver

The crisis lead decides what else can be done: more people, a supplier's help, a partial service. The approver is told: the board, or a board risk committee it has delegated to; the management board in a two-tier structure; executive management where there is no board. Record each decision in the crisis log.

Step

Bring back the most important part first

Part of a service can return before the whole: one site first, reading orders before taking them, one warehouse at reduced volume. Each part passes the return-to-service gates for the systems it uses, the business owner confirms what users can and cannot do, and the crisis log records it with its own time.

• Then: Switch back on the plan's criteria; reconcile; record the gap

Outcome

Switch back on the plan's criteria; reconcile; record the gap

The workaround ends when the service runs normally again and the business owner has confirmed it works. Enter and reconcile the backlog it built up. A recovery time a real recovery missed is an objective gap, and the plan is reviewed with what was learned.