Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

How do we…

How do we prove to an auditor that the review really happened?

5 decision pointsThe pack behind this answer: User Access Review

Add to board memo

The short answer

Recommendation
Keep an evidence file for each campaign, collected as it runs, and give the auditor a chain they can follow: from a decision, to its ticket, to a fresh extract showing the access gone inside its window. Run that test yourself after every campaign.
Reasoning
An auditor usually tests that access is reviewed at a set interval by people who can judge it, that privileged access gets closer attention, that decisions are acted on, and that all of it is evidenced. The extracts and decision records are the review: without them it cannot be shown to have happened, and nothing assembled later replaces them. A method the auditor can re-perform is one they can rely on.
Considerations
  • Do not rebuild missing records after the event. Tell the auditor plainly what is missing and for which systems, and fix the next campaign.
  • A closed ticket is not proof of removal; the fresh extract is. Late removals are confirmed like the rest and listed with their reason.
  • The pack's defaults keep the file for 3 years and campaign reports for 5, and sample 5 decisions of each kind. Your own method, records schedule, a regulator or a contract may set others.
  • The extracts and decision records list people and their access, so delete them when the retention period ends.

The decision, step by step

Select any step for the reasoning behind it and where the pack covers it.

  • Start
  • Step
  • Decision
  • Outcome
StartAn auditor asks for proof that access was reviewedStepCollect evidence as the campaign runs, not at the endDecisionDo you hold the scope list and every extract as taken?DecisionDoes every entry have a named decision with a date?OutcomeSay plainly what cannot be shown, and fix the next campaignDecisionIs every Revoke and Modify confirmed on a fresh extract?StepConfirm the rest, and date each item still pendingDecisionAre the findings, weaknesses and quality check on file?StepRecord them before the campaign is signed offStepSign the campaign closed, then report its outcomeStepRe-perform the test yourself after every campaignDecisionDid every sample trace from decision to fresh extract?OutcomeExplain each break, and fix the step that caused itOutcomeHand over the file and invite the auditor to re-perform it

Keyboard: Tab to a step, arrow keys move along the flow, Enter pins its explanation, + and − zoom, 0 fits the diagram, Esc closes. The text version below has the same content.

Where a step quotes a number, it is the default in the CISO Times pack. Your own standard may set a different one.

Read as text

The same flow written out: each step, its reasoning and where the pack covers it. 14 steps.

  1. Start

    An auditor asks for proof that access was reviewed

    Auditors usually test four things: access is reviewed at a set interval by people who can judge it, privileged access gets closer attention, decisions are acted on, and all of it is evidenced. An auditor or regulator asking for evidence should start nothing new: the campaign's evidence file answers it.

    In the pack

  2. Step

    Collect evidence as the campaign runs, not at the end

    The pack's checklist has 13 items in four phases: scope and extracts, decisions, removals, and findings and close. Save each as the campaign produces it, in one folder per campaign, with a location someone else can find. Ask a supplier for dated files, not statements that something was done.

    In the pack

  3. Decision

    Do you hold the scope list and every extract as taken?

    The scope list shows which systems should have been reviewed, as it stood on the extract date. Each extract must be the file as received, named with the system, date and who ran it, with its count reconciled to the system's own. The reconciliation is the proof the list was complete.

    In the pack

  4. Decision

    Does every entry have a named decision with a date?

    Entries decided must add up to entries in the extract, with none blank. Blanks counted as Keep are the most common audit finding. Include the Cannot decide resolutions, the reviewer assignments showing nobody reviewed their own access, and the second review of every privileged, generic and service account.

    In the pack

  5. Outcome

    Say plainly what cannot be shown, and fix the next campaign

    Without the extracts and the decision records, the review cannot be shown to have happened, and nothing assembled later replaces them. Do not rebuild them after the event. Tell the auditor what is missing and for which systems, and make sure the next campaign keeps them from the start.

    In the pack

  6. Decision

    Is every Revoke and Modify confirmed on a fresh extract?

    Each change needs a ticket with the date raised and the date done, and a fresh extract showing the access gone. A closed ticket is not proof. Late removals are confirmed like the rest and listed with their reason.

    In the pack

  7. Step

    Confirm the rest, and date each item still pending

    Take the fresh extract for anything unconfirmed. If the campaign must close first, sign off the evidence file with each pending item named and dated, and finish them in your removals tracker.

    In the pack

  8. Decision

    Are the findings, weaknesses and quality check on file?

    Findings show the review fed back into the joiner, mover and leaver process. Each process weakness needs its owner, fix and date. The quality check shows decisions were actually made, not approved in bulk. A segregation-of-duties conflict kept needs its approved exception on file.

    In the pack

  9. Step

    Record them before the campaign is signed off

    The campaign does not close while a quality signal is unresolved. Record what each check found, even when nothing was seen, so a reader can tell it was run.

    In the pack

  10. Step

    Sign the campaign closed, then report its outcome

    Each system owner signs off their system's results, and the Head of Information Security then signs the campaign closed. The outcome report follows, within 5 working days by the pack's default, and is the single document an auditor reads first: what the campaign removed, the headline measures and the weaknesses found. Keep the file for the retention period (the pack's default is 3 years, and 5 years for campaign reports).

    In the pack

  11. Step

    Re-perform the test yourself after every campaign

    Pick 5 Revoke or Modify decisions from a closed campaign. Follow each to its ticket and to the fresh extract that shows the access gone, and check the date against the window. Then pick 5 Keep decisions on privileged accounts and ask each named owner why the access is needed.

    In the pack

  12. Decision

    Did every sample trace from decision to fresh extract?

    A break is a decision with no ticket, a ticket with no confirming extract, a removal outside its window with no explanation, or a privileged Keep the owner cannot justify.

    In the pack

  13. Outcome

    Explain each break, and fix the step that caused it

    A late removal is counted in the on-time removal measure and named with its cause in the report. A missing record points to the campaign step that should have produced it. Fix that step in your procedure before the next campaign, and say so when the auditor asks what changed.

    In the pack

  14. Outcome

    Hand over the file and invite the auditor to re-perform it

    Have ready the approved method, the inventory of in-scope systems, and for the last two campaigns the evidence file, quality check, findings tracker and campaign report. Invite the auditor to pick their own samples: a method they can re-perform is one they can rely on.

    In the pack

    See also

Practical checklist

  1. Save each item as the campaign produces it, in one folder per campaign that someone else can find.
  2. Keep the scope list as it stood on the extract date, and every extract as received with its count reconciled.
  3. Keep every decision with a name and a date, the Cannot decide resolutions and the second review of privileged, generic and service accounts.
  4. Confirm every Revoke and Modify on a fresh extract, and name and date any item still pending at sign-off.
  5. File the findings, process weaknesses and quality check before the campaign is signed off.
  6. Have each system owner sign off their results and the Head of Information Security sign the campaign closed.
  7. Trace 5 Revoke or Modify decisions to their tickets and fresh extracts, and ask the owners of 5 privileged Keeps why each is needed.
  8. Fix the campaign step behind any break before the next campaign.

Evidence to have ready

  • The approved review method, and the inventory of in-scope systems with owners and frequencies.
  • For the last two campaigns: the evidence file for each system, the quality check, the findings tracker and the campaign report.
  • The removal tickets, and the fresh extracts that confirm each removal.
  • Each break your own re-performance test found, with its cause and the step you changed.

Assumptions

  • You run access reviews as campaigns, with one extract date, one decision deadline and one close date.
  • The sample sizes and retention period shown are the defaults in the CISO Times access review pack. Your own method or records schedule may set different ones.

Not covered here