Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

How do we…

How do we plan a year of control testing around audits and reviews?

8 decision pointsThe pack behind this answer: Security Control Assurance

Add to board memo

The short answer

Recommendation
Rate each control first, because the rating sets how often it is tested. Then place the tests that audits and reviews rely on so they are final before each cut-off, spread the rest across the year by due date, and have the sponsor approve the plan before the year starts.
Reasoning
Without a plan, testing drifts towards the month before an audit, which tells the auditor that testing is done for the audit and leaves no time to act on what the tests find. A control that failed its last test is Higher until a retest passes, and a key control is never Lower (rule CA-15). Tests that an audit or review relies on are final well before it starts, and any change to the approved plan is recorded (rule CA-16).
Considerations
  • The cycles are the pack's defaults: at least every 6, 12 or 24 months for Higher, Standard or Lower risk, a design walkthrough every 12 months, and a cut-off 6 weeks before each audit or review.
  • Apart from a failed last test, a Higher rating needs a reason from your registers: a risk recorded as outside appetite or an obligation recorded as not met. A control someone merely thinks important is Standard.
  • The pack sets no testing capacity figure. Use your own testers' days.
  • Delivery is reported to the approver as counts: the board or a board risk committee it has delegated to, the management board in a two-tier structure, or executive management where there is no board.

The decision, step by step

Select any step for the reasoning behind it and where the pack covers it.

  • Start
  • Step
  • Decision
  • Outcome
  • Loop back
StartA plan year of control testing is about to startStepList each control with its owner, frequency and key statusDecisionDid the control fail its last test?DecisionIs it a key control?DecisionDoes a recorded outside-appetite risk or unmet obligation rest on it?StepHigher: test at least every 6 months, with a larger sampleStepStandard: test at least every 12 monthsStepLower: operating test every 24 months, design every 12DecisionHas the control had an operating test concluded before?StepDue within one cycle of the first plan year, or of its own startStepDue within its cycle from the end of its last tested periodStepList the year's audits and reviews, each with its cut-offDecisionWill an audit or review rely on this control's test?StepPlace it first, so it is final before the cut-offStepSpread the rest across the year, by due dateDecisionDoes any month need more hours than your testers have?StepMove a test, add a tester, or add daysDecisionHas the sponsor approved the plan before the year starts?OutcomeTake the plan to the sponsor for a dated approvalDecisionOnce the year starts, must a test move, be added or be dropped?OutcomeRecord the change, its reason and the sponsor's approvalOutcomeRun the plan and report delivery against it in counts

Keyboard: Tab to a step, arrow keys move along the flow, Enter pins its explanation, + and − zoom, 0 fits the diagram, Esc closes. The text version below has the same content.

Where a step quotes a number, it is the default in the CISO Times pack. Your own standard may set a different one.

Read as text

The same flow written out: each step, its reasoning and where the pack covers it. 22 steps.

  1. Start

    A plan year of control testing is about to start

    A plan decides, a year ahead, which control is tested in which month and by whom. Without one, testing drifts towards the month before an audit, which tells the auditor that testing is done for the audit and leaves no time to act on what the tests find. The plan starts from the control inventory and the year's audits and reviews.

    In the pack

  2. Step

    List each control with its owner, frequency and key status

    Copy each control from your inventory into the plan: its name, how often it runs, how many times a year, its owner and whether it is key. A key control is one whose failure on its own would let a risk on the register happen or go unnoticed, or one you name as your evidence for an obligation. Then rate each control, one at a time.

    In the pack

  3. Decision

    Did the control fail its last test?

    Start here, because a failure overrides everything else. A control that failed its last test is Higher until a retest passes, whatever it would otherwise be. An owner's check on their own control is a self-check, not a test, so it neither passes nor fails a control.

    In the pack

  4. Decision

    Is it a key control?

    Ask two questions: would its failure on its own let a risk on the register happen or go unnoticed, and do you name it as your evidence for an obligation? Yes to either makes it key, and a key control is never Lower. If you are not sure, treat it as key.

    In the pack

  5. Decision

    Does a recorded outside-appetite risk or unmet obligation rest on it?

    Look in your registers, not your instincts: a risk recorded as outside appetite in the risk register, or an obligation your organisation has recorded as not met. A control someone merely thinks is important is Standard. If the registers do not say, it is Standard for now; ask the risk owner whether the register is current.

    In the pack

    See also

  6. Step

    Higher: test at least every 6 months, with a larger sample

    Write the reason beside the rating. A Higher control appears twice in the plan year, six months apart, and takes the larger sample. For a control that failed, plan the retest only once the fixed control has run at least twice, or once for an annual or automated control, and enter when it began to run.

    In the pack

  7. Step

    Standard: test at least every 12 months

    Any other key control is Standard and appears once in the plan year. The rating changes when its reason changes, for example when a risk moves outside appetite, and the next test date moves with it.

    In the pack

  8. Step

    Lower: operating test every 24 months, design every 12

    A control that is not key is Lower. Its operating test falls in one year of its two, so choose this plan year or the next for each, about half in each. Its design is still walked through every 12 months and whenever the control changes.

    In the pack

  9. Decision

    Has the control had an operating test concluded before?

    Only a test concluded Effective or Not effective counts. A self-check, a test concluded Not tested, or a design found deficient leaves the control untested. Treat a control as never tested if you cannot find its workpaper.

    In the pack

  10. Step

    Due within one cycle of the first plan year, or of its own start

    A control never tested is due within one cycle of the start of the first plan year, or of the day it started if that is later. Plan a design walkthrough before its first operating test. A control that has not yet run in the period is concluded Not tested, so place the test after its first occurrence.

    In the pack

  11. Step

    Due within its cycle from the end of its last tested period

    The next operating test is due within the control's cycle from the end of its last testing period. Each period starts where the last one ended and is never longer than the cycle, so every month of the control's life falls in some test.

    In the pack

  12. Step

    List the year's audits and reviews, each with its cut-off

    List the audits and reviews that will rely on test results: a certification or surveillance audit, a supervisory review, a customer's annual supplier review, the next maturity assessment. Each one's cut-off is 6 weeks before it starts, the pack's default, which you may lengthen. Add the board meetings the results go to.

    In the pack

  13. Decision

    Will an audit or review rely on this control's test?

    Mark each control an audit or review relies on, and which one. If you do not know yet whether a customer or auditor will look at a control, treat it as relied on and ask them.

    In the pack

  14. Step

    Place it first, so it is final before the cut-off

    Final means the reviewer has signed the workpaper, and the reviewer has up to 10 working days after the last test step, the pack's default. So choose a test month that ends early enough to leave that review time before the cut-off. In the pack's example, a review on 14 June has its cut-off on 3 May, and its five tests fall in February and March.

    In the pack

  15. Step

    Spread the rest across the year, by due date

    Place every other test in a month before it falls due, and spread them so no month is crowded. Name a tester who owns and performs none of the controls they test, and a reviewer who neither tested nor owns the control.

    In the pack

    See also

  16. Decision

    Does any month need more hours than your testers have?

    Estimate each test's hours from set-up and write-up time plus time per sampled item, and compare each month with the days your testers can give. The pack sets no capacity figure: its numbers here are an example, so use your own.

    In the pack

  17. Step

    Move a test, add a tester, or add days

    A month over capacity needs a test moved, a second tester, or more days. A test an audit relies on stays before its cut-off, and no test moves past the date its control falls due.

    In the pack

  18. Decision

    Has the sponsor approved the plan before the year starts?

    The control assurance lead proposes the plan and the assurance sponsor approves it, dated, before the plan year starts. If you are not sure the approval was recorded, treat the plan as unapproved.

    In the pack

  19. Outcome

    Take the plan to the sponsor for a dated approval

    Present the plan with the controls it tests, the audits and reviews it works to, and the months it fills. Record the approval with the plan. The sponsor also receives the dashboard each quarter, including tests delivered against the plan.

    In the pack

  20. Decision

    Once the year starts, must a test move, be added or be dropped?

    Plans change: a tester is ill, a significant incident or change calls for an extra test. Any change after approval is recorded with its new date, its reason and who approved it.

    In the pack

  21. Outcome

    Record the change, its reason and the sponsor's approval

    Write the change into your plan with its date, reason and approver. A test an audit relies on stays before the audit's cut-off. The dashboard shows each slip with its new date.

    In the pack

  22. Outcome

    Run the plan and report delivery against it in counts

    Report tests delivered against the plan, and controls tested within their cycle, as counts with every deficiency named; never a percentage. The approver, depending on how you are governed the board or a board risk committee it has delegated to, the management board in a two-tier structure, or executive management where there is no board, receives the results.

    In the pack

Practical checklist

  1. List each control with its owner, frequency and key status.
  2. Rate each control, starting with whether it failed its last test.
  3. Set each next test date from its cycle, or within one cycle for a control never tested.
  4. List the year's audits and reviews with their cut-offs, and mark the controls each relies on.
  5. Place relied-on tests early enough for the reviewer to sign before the cut-off.
  6. Spread the other tests across the year, naming independent testers and reviewers.
  7. Check each month's hours against your testers' days, and move tests where needed.
  8. Have the sponsor approve the plan, dated, before the year starts, and record every later change.

Evidence to have ready

  • The approved plan, with the sponsor's dated approval, the controls, their ratings and reasons, and their test months.
  • The calendar of audits and reviews with their cut-offs, and the controls each relies on.
  • Each change to the plan with its new date, reason and approver.
  • The quarterly report of tests delivered against the plan, in counts, with every deficiency named.

Assumptions

  • You keep an inventory of the controls you test, each written as who does what, how often and the record it leaves, with its owner and whether it is a key control.
  • You have a risk register with an approved risk appetite, and a record of any obligation your organisation has recorded as not met, so a Higher rating can name its reason.
  • Someone in management sponsors control testing, and you can find testers and reviewers who own and perform none of the controls they test.

Not covered here