Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

How do we…

How do we choose which restore tests to run, and how often, for each system?

6 decision pointsThe pack behind this answer: Backup & Restore Assurance

Add to board memo

The short answer

Recommendation
Give each data set the restore test types its kind needs, and repeat each as often as its class sets. By the pack's defaults, critical data has a file restore every 3 months and each higher type every 12.
Reasoning
Each type proves something the others do not. A file restore shows the copy is whole, the keys work and the data opens. Only a system restore, a full service test or a rebuild shows how long recovery takes, and only a rebuild measures recovery after a destructive attack. The class sets how often each type repeats, so the heaviest schedule falls on the data the organisation depends on most.
Considerations
  • By the pack's defaults, important data has a file restore every 6 months and a system or tenant restore every 24, and other data a file restore every 12.
  • A new data set does not wait a full interval: its first test of a higher type is due within 6 months of listing.
  • A tabletop, a drill or a provider failover that restores none of your backups is not a restore test.
  • If you cannot meet the interval for critical data, raise a finding and fix the cause. Do not lengthen the interval.

The decision, step by step

Select any step for the reasoning behind it and where the pack covers it.

  • Start
  • Step
  • Decision
  • Outcome
StartYou are planning restore tests for the yearStepList each data set with its class, kind and last checked restoreDecisionDoes a provider hold the data, with no copy of ours?OutcomeRely on the provider's own recovery test and ask for its resultsDecisionWhat kind of data set is it?StepPlan a file restore (TT-1) and a full system restore (TT-2)StepPlan a file restore (TT-1) and a tenant restore (TT-5)StepPlan a file restore (TT-1) and a clean-room rebuild (TT-4)DecisionHow much does the organisation depend on it?StepSet 3 months for file restores and 12 for higher typesStepSet 6 months for file restores and 24 for system or tenant testsStepSet 12 months for file restores; no higher type is requiredDecisionHas each type it needs already passed its integrity check?StepDate a first test of each type not yet passedStepDate the next test from the last restore that passedDecisionIs it Critical data that a critical service needs?StepAdd a full service test (TT-3) and plan it once with recovery testingStepAlternate the copy tested between online and immutable or offlineDecisionDoes every planned test restore data and measure it?OutcomeTake out exercises and drills: they are not restore testsOutcomeBook each test with its data owner and keep the plan current

Keyboard: Tab to a step, arrow keys move along the flow, Enter pins its explanation, + and − zoom, 0 fits the diagram, Esc closes. The text version below has the same content.

Where a step quotes a number, it is the default in the CISO Times pack. Your own standard may set a different one.

Read as text

The same flow written out: each step, its reasoning and where the pack covers it. 21 steps.

  1. Start

    You are planning restore tests for the year

    Each type of restore test proves something the others do not. A file restore shows the copy is whole. Only a system restore, a full service test or a rebuild shows how long recovery takes. The resilience coordinator and the backup administrator plan the year with the backup owner, once a year and again whenever a system or data set is added.

    In the pack

  2. Step

    List each data set with its class, kind and last checked restore

    Start from the coverage matrix. Note each data set's class, its kind and the date of the last restore that passed its integrity check. Critical means loss would stop a critical service or the foundations could not be rebuilt. Important means Major harm on your risk register's scale with no critical service stopping. Other is the rest, and the data owner agrees the class.

    In the pack

    See also

  3. Decision

    Does a provider hold the data, with no copy of ours?

    Where a provider holds the data and you keep no backup of your own, there is no copy of yours to restore. If you cannot tell whether you hold an independent copy, answer yes and settle it with the provider.

    In the pack

  4. Outcome

    Rely on the provider's own recovery test and ask for its results

    Use the provider's own recovery test, with its results shared and checked against the service's objectives. Plan the date with the recovery programme, which schedules it. Record a contract that gives no restore time, or a longer one than the objective, as a failed check.

    In the pack

    See also

  5. Decision

    What kind of data set is it?

    The kind sets the types. A system is one you run, such as a server, a platform and its database. A tenant is your own area in a provider's cloud or software-as-a-service system. The foundations are the directory, network configuration, backup catalogue and build images. If you cannot tell, treat it as a system you run.

    In the pack

  6. Step

    Plan a file restore (TT-1) and a full system restore (TT-2)

    A file or data set restore shows the copy is whole, the keys work and the data opens, and it is cheap enough to run often. A full system restore starts the whole system in a spare environment and records the time, so it is the first test that shows how long the system takes. Both need a separate place to restore into.

    In the pack

  7. Step

    Plan a file restore (TT-1) and a tenant restore (TT-5)

    A tenant restore returns mailboxes, files or ledgers from your own copy or the provider's into a test tenant, with the provider's help where needed. It shows what the provider can and cannot restore, including what it leaves out. The file restore shows the exported data is whole.

    In the pack

  8. Step

    Plan a file restore (TT-1) and a clean-room rebuild (TT-4)

    The rebuild restores the foundations, and the systems that depend on them, in an isolated recovery environment from trusted build sources. It shows recovery works when the attacker holds the production tools and accounts, and it is the only test that measures the cyber recovery time. The foundations are always Critical, and the recovery programme's clean-room gates apply.

    In the pack

  9. Decision

    How much does the organisation depend on it?

    The class sets how often each type is repeated. Ask the data owner, who agrees it. If the class is not agreed, take the Critical row until it is, because a lighter schedule on an important data set is the costlier mistake.

    In the pack

  10. Step

    Set 3 months for file restores and 12 for higher types

    By the pack's defaults a Critical data set has a file or data set restore (TT-1) at least every 3 months. The system (TT-2), full service (TT-3), rebuild (TT-4) and tenant (TT-5) tests are due at least every 12 months. A data set with no passed restore inside its period fails the coverage check and is a finding, so fix the cause rather than lengthen the interval.

    In the pack

  11. Step

    Set 6 months for file restores and 24 for system or tenant tests

    By the pack's defaults an Important data set has a file or data set restore at least every 6 months. A system or tenant restore is due at least every 24 months. The full service and rebuild tests are not required for this class.

    In the pack

  12. Step

    Set 12 months for file restores; no higher type is required

    By the pack's defaults an Other data set has a file or data set restore at least every 12 months. No system, service, rebuild or tenant test is required for this class.

    In the pack

  13. Decision

    Has each type it needs already passed its integrity check?

    Check type by type. A test of a higher type counts for the lower ones it includes, if it passed, but a file restore does not stand in for a system restore. A restore whose integrity check failed is a finding and does not count. If the history is not known, answer no and plan a first test.

    In the pack

  14. Step

    Date a first test of each type not yet passed

    A new data set does not wait for the first full interval. Date the first file restore within its class interval from the day the data set was listed, and the first test of any higher type within 6 months of that day. Until a restore passes, the copy is unverified.

    In the pack

  15. Step

    Date the next test from the last restore that passed

    The next test of a type falls one interval after the last restore that passed its integrity check. A passed system restore also serves as the file restore for the same data set. Work the dates from the coverage matrix, not from the date the job last ran.

    In the pack

  16. Decision

    Is it Critical data that a critical service needs?

    For a critical service, the full service test (TT-3) is the recovery test your recovery policy sets, and one record serves both. Important and Other data never need one, and the foundations are tested by rebuild, so answer no for those. If you cannot tell which services depend on a Critical data set, answer yes.

    In the pack

  17. Step

    Add a full service test (TT-3) and plan it once with recovery testing

    By the pack's default each critical service is tested at least every 12 months and after a significant change. System, end-to-end and rebuild tests are never less frequent than the recovery policy sets, and one test serves both where it meets both. Its date follows the recovery programme, including any peak-season freeze. The pack's example freezes November and December.

    In the pack

    See also

  18. Step

    Alternate the copy tested between online and immutable or offline

    For a critical data set, alternate between the online copy and the immutable or offline copy from one test to the next, so both are shown to restore. Restores into a separate environment (TT-1 and TT-5) touch no live service, so a peak-season freeze does not stop them. Spread the first year's tests across the year, so the problems surface while there is time to fix them.

    In the pack

  19. Decision

    Does every planned test restore data and measure it?

    A restore test restores data and measures it. A tabletop, a drill or any other exercise tests decisions around a table, and a provider failover that restores none of your backups restores nothing of yours. If you are unsure, answer no and check the test's record.

    In the pack

  20. Outcome

    Take out exercises and drills: they are not restore tests

    Remove each exercise, tabletop or drill from the restore plan, and plan a real restore in its place. The exercise programme keeps its own schedule. The recovery policy says the same: an exercise is not a recovery test. A provider's failover that restores none of your backups counts for the recovery programme and not for a restore check here.

    In the pack

  21. Outcome

    Book each test with its data owner and keep the plan current

    Plan one row for each data set a test restores, with its date, type and what will be restored, and check each falls before it is due. Book the data owner and a deputy for the check. Add a new data set to the matrix before it goes live and replan. Report what is due and overdue as counts with names.

    In the pack

Practical checklist

  1. List each data set with its class, its kind and the date of its last restore that passed its integrity check.
  2. Use the provider's own recovery test for data a provider holds with no copy of yours.
  3. Pick the types: file and system restore for a system you run, file and tenant restore for a tenant, file restore and rebuild for the foundations.
  4. Set how often each type repeats from the data set's class.
  5. Date a first test of each type not yet passed, or the next one from the last restore that passed.
  6. Add a full service test for critical data a critical service needs, planned once with its recovery test.
  7. Alternate the copy tested between online and immutable or offline for critical data.
  8. Take out any exercise or drill, and book each test with its data owner and a deputy.

Evidence to have ready

  • The year's test plan: one row per data set with its date, type and what will be restored.
  • The date of the last passed restore of each type for each data set.
  • The provider's recovery test results, checked against the service's objectives, for data it holds.
  • Counts of tests due and overdue, with each one named.

Assumptions

  • You have a coverage matrix listing each data set with its class and kind, or you will build one first (“How do we decide what to back up, how often and for how long?”).
  • Each critical service has agreed recovery time and data-loss targets and a recovery programme that schedules its tests.
  • The intervals are the pack's defaults, which your own backup standard may change.
  • The backup owner, the backup administrator and the resilience coordinator can plan the year together.

Not covered here