What evidence is good enough to show a control operated?

Start

You need records to show a control ran as designed

An operating test checks, on a sample the tester chooses, that the control ran as designed throughout the testing period. Each sampled occurrence passes only when every attribute does, and each attribute needs a record that meets the evidence standard. The standard does not change with the size of the organisation: fewer occurrences mean smaller populations, not weaker records.

• Then: Take the population yourself from the system of record

Step

Take the population yourself from the system of record

Run the report yourself or watch it run, and record the system, the report, when it ran, the filter and the count. The system of record is where the trigger is recorded, apart from the control: the HR leaving dates for leavers, the scanner's detections for vulnerabilities. Then agree the count to a second source the control does not feed, such as payroll. For a monthly or quarterly control, a period with no record is an exception, not a smaller population.

• Then: Does the count agree with a second source, every difference explained?

Decision

Does the count agree with a second source, every difference explained?

A sample proves nothing about occurrences missing from the list it was drawn from. A list the owner filtered can leave out the ones that failed. Add any occurrence the second source shows and the list lacks, and record why anything was removed. If you cannot show the list is complete, take the cautious branch.

• Yes: Read what acceptable evidence looks like for this kind of control
• No, or not known: Do not sample from it: find another source or tell the lead

Outcome

Do not sample from it: find another source or tell the lead

A list the tester cannot check is not a population. Look for another source first. A population found to be incomplete is itself an exception: the control's record does not show every occurrence. If no source exists, tell the control assurance lead, who decides whether the control has a design deficiency because it leaves no record that can be tested. If the owner will not give access to the system of record, the lead raises it with the owner's manager, then the assurance sponsor.

Step

Read what acceptable evidence looks like for this kind of control

Before you look at any record, read what counts and what does not for the control's kind. An approval must be dated before the action and given by someone allowed to approve. A review needs a decision for each item, not minutes that say "noted". A deadline needs each date from its own system. A register needs its change history, not today's export. Then ask the owner only for the records of the occurrences you sampled.

• Then: Is it a record, an observation or a re-performance, not an answer?

Decision

Is it a record, an observation or a re-performance, not an answer?

Inspection reads a record the control left. Observation watches it being done, and shows nothing about other occasions. Re-performance does it again from the same inputs, the strongest evidence and the slowest. What the owner tells you goes in the note: it explains the record and never stands on its own.

• Yes: Does it show this attribute, for this sampled occurrence?
• No, only what someone said: Ask for a better record, or take it from the system yourself

Decision

Does it show this attribute, for this sampled occurrence?

Read the record against the attribute. A policy, a procedure or a template shows what should happen; it is not evidence that it did. Check the "Not acceptable" examples for the control's kind first.

• Yes: Is it dated inside the testing period, with a time where needed?
• No, or not known: Ask for a better record, or take it from the system yourself

Decision

Is it dated inside the testing period, with a time where needed?

The date must be the occurrence's and fall inside the testing period. Where a deadline is tested, take each date from its own system, and the time where the deadline is in hours or the same day. A record made after the occurrence, or corrected during the test, does not show the control ran on time.

• Yes: Does it show who acted, and were they allowed to?
• No, or not known: Ask for a better record, or take it from the system yourself

Decision

Does it show who acted, and were they allowed to?

Find the name or account of the performer or approver, and check that person was allowed to act: the approver the band or tier names, and never the requester.

• Yes: Did you take it from the system, or watch it being taken?
• No, or not known: Ask for a better record, or take it from the system yourself

Decision

Did you take it from the system, or watch it being taken?

Run the report yourself, watch it run, or receive it straight from the system, in the form the system produced. A file the owner prepared counts only if you can check it against the system. A screenshot shows the whole window, the system, the date and the account; one the owner sends later cannot show when, where or by whom it was taken. For a setting, add its change record.

• Yes: Does the acceptable record show the attribute passed?
• No, or not known: Ask for a better record, or take it from the system yourself

Step

Ask for a better record, or take it from the system yourself

A record that falls short of any test is not acceptable. Ask for one that meets it, or extract it yourself. Do not let anyone create or correct a record during the test.

• Then: Can you get a record that meets the standard?

Decision

Can you get a record that meets the standard?

If a better record exists, check it against the same tests. If none can be found, the occurrence cannot be shown to pass.

• Yes: Is it a record, an observation or a re-performance, not an answer? (back to an earlier step)
• No: Does other evidence show it ran, with only its record short?

Decision

Does other evidence show it ran, with only its record short?

Sometimes the control clearly ran but left a weak record, such as an approval with no date. The attribute still fails. Note on the workpaper that the shortfall is in the record, because it decides the severity. A control that did not run is never treated this way. If you cannot tell, take the cautious branch.

• Yes: Fail the attribute, noting the record fell short: it may rate Low
• No, or not known: Record an exception: the occurrence cannot be shown to pass

Outcome

Fail the attribute, noting the record fell short: it may rate Low

The occurrence is an exception. If the test ends Not effective, the assurance lead may rate the deficiency Low, whatever else applies, because the control ran and only its record fell short of the standard. A Low deficiency is fixed within 365 days by the pack's default.

Outcome

Record an exception: the occurrence cannot be shown to pass

Where a record cannot be found, the attribute fails and the occurrence is an exception on the workpaper. What the owner says happened does not change that. Whether the control then fails depends on the rest of the sample.

Decision

Does the acceptable record show the attribute passed?

Acceptable evidence and a passed attribute are different questions. A record can meet every test and show that the control failed, such as an account history showing a leaver's access removed weeks late. That is still this control's exception, even if another control caught it.

• Yes: File it with the workpaper where the owner cannot change it
• No: Record it as an exception, with the record that shows it

Outcome

Record it as an exception, with the record that shows it

Keep the record: it is acceptable evidence of an exception. Record Fail for the attribute with the record's reference and date, file the record with the workpaper, and handle the exception by the method's rules.

Step

File it with the workpaper where the owner cannot change it

File every record before the reviewer signs, in a restricted folder, a document system with version history or an audit tool. The owner may read it; editing is not allowed. A workpaper the owner could edit is no evidence that the control operated. Keep it for 3 years by the pack's default, or longer where law or a contract asks, and take only the personal data the attributes need.

• Then: Acceptable: record Pass and let the reviewer check it

Outcome

Acceptable: record Pass and let the reviewer check it

Record Pass with the record's reference and date. The reviewer checks that every result rests on evidence that meets the standard before signing. Evidence that meets it can be shown to an auditor; it does not make the test an audit, and a test result is not a statement that you comply with any law or standard.