Independent thinking. Informed defence.

CISO Times

Intelligence for the people behind the defence.

The CISO Decision Brief

If…, then how…

If a VPN appliance may be compromised, do we patch, rebuild or replace it?

5 decision pointsThe pack behind this answer: Security Incident Management

Add to board memo

The short answer

Recommendation
Keep the appliance's logs, then check it for signs of use. Any sign, or no check yet, makes it an incident: contain it, patch in place only if you know the route in and every change made, otherwise rebuild from a clean image, and replace it if the vendor can no longer supply fixes or a trusted image.
Reasoning
A patch closes the way in. It does not remove what an attacker left on the appliance or in the network behind it. The pack's ransomware playbook puts a system back on its patched version only when the route in and every change are known; applied here by analogy, anything less means a rebuild. The pack has no appliance-specific procedure and does not cover replacement, so that call is your own judgement with the vendor and your IT provider. Reconnect only when the plan's recovery criteria are met.
Considerations
  • With no sign of compromise after a real check, an exploited flaw on an internet-facing appliance is Priority 1 in the pack's defaults: a compensating control within 72 hours and the fix within 7 days.
  • Being unable to check, for example because the logs do not go back far enough, counts as a sign.
  • A VPN may be how sites, on-call engineers or suppliers reach systems others depend on. Bring in those service owners before cutting it off, and keep containing meanwhile.
  • Change every credential on the appliance or used through it: its administrator accounts, the service account it uses to reach your directory, and the accounts of users who signed in during the exposure.

The decision, step by step

Select any step for the reasoning behind it and where the pack covers it.

  • Start
  • Step
  • Decision
  • Outcome

Keyboard: Tab to a step, arrow keys move along the flow, Enter pins its explanation, + and − zoom, 0 fits the diagram, Esc closes. The text version below has the same content.

Where a step quotes a number, it is the default in the CISO Times pack. Your own standard may set a different one.

Read as text

The same flow written out: each step, its reasoning and where the pack covers it. 17 steps.
  1. Start

    A VPN appliance may be compromised

    The trigger is usually a vendor advisory or a national CSIRT warning that a flaw in your appliance is being exploited, or an alert on the appliance itself. A remote-access appliance flaw added to the CISA Known Exploited Vulnerabilities catalogue is the pack's own example of an exploited weakness. The first question is not how to patch. It is whether the weakness has already been used against you.

    In the pack

  2. Step

    Keep the logs, then check the appliance for signs of use

    Ask your monitoring provider and IT provider to preserve the appliance's logs and extend their retention before anything is changed. Then look for the indicators the vendor or CSIRT has published, accounts or settings nobody made, sign-ins from odd places, and unusual connections from the appliance into the network. Record which accounts signed in through it, and from where.

    In the pack

  3. Decision

    Is there any sign it has been used against you?

    This asks about your own appliance, not the world. Any credible indicator counts, and so does being unable to check, for example because the logs do not go back far enough. The triage procedure's rule is that when you are not sure, you declare and set severity by the worst it could be.

    In the pack

  4. Outcome

    Patch it as an exploited flaw, then look back

    With no sign of compromise after a real check, this is a vulnerability to fix fast. A VPN gateway is internet-facing, and an exploited flaw on it is Priority 1 in the pack's defaults: treat it as a potential incident, notify the standard owner at once, put a compensating control in within 72 hours and install the fix within 7 days. After patching, look back over the whole exposed period; that look back is our advice, not a pack step. Any sign found then makes it an incident, and this flow starts again at the first question.

    In the pack

  5. Step

    Declare an incident and preserve evidence

    A weakness in an internet-facing system used to get in is the pack's exploited-vulnerability category, and the incident plan's own phases are its response. Export the appliance's configuration and logs, and keep a copy of any image the vendor's tools can take, before anyone patches, resets or rebuilds it. A rebuilt appliance cannot tell you how the attacker got in.

  6. Decision

    Would cutting it off affect safety or essential operations?

    A VPN may be how remote sites, on-call engineers or suppliers reach systems that others depend on. In the plan, the incident manager contains at once with the least disruptive action that stops the harm, and brings in the service owner and specialists before any action that would stop a service people depend on for safety or essential operations. Stopping a customer-facing service is the executive incident lead's call unless waiting would let the harm spread. If such operations depend on the appliance, or you cannot tell, answer yes.

    In the pack

  7. Step

    Contain around it; bring in service owners before cutting it off

    Keep containing meanwhile, the least disruptive way: close the access paths the attacker is using and disable affected accounts. Before cutting off the appliance, the incident manager brings in the owners of the services that depend on it and specialists, with outside responders where you lack the skills, and decides with them how far to go, for example moving users to another route first. Record who decided and why in the decision log.

    In the pack

  8. Step

    Isolate it, preserving evidence first unless harm is spreading

    The plan's rule: preserve evidence first unless the harm is spreading; if it is spreading, isolate first and record what was lost. The incident manager may isolate a system without waiting for an executive, using the least disruptive containment that stops the harm, and tells the executive incident lead at the next update. Apply the vendor's mitigation or patch as part of containment if you wish. It does not end the incident.

    In the pack

  9. Step

    Change every credential on it or used through it

    The plan says to assume the attacker had more than you have found. For an appliance that means its own administrator accounts, the service account it uses to reach your directory, and the accounts of users who signed in through it during the exposure. Check the network behind it for new accounts, scheduled tasks and remote access tools. The pack does not list appliance secrets such as certificates or keys; replace any that were stored on it.

    In the pack

  10. Decision

    Can the vendor still supply fixes and a trusted image?

    Patching and rebuilding both depend on the vendor. An appliance past its end of support, or one where the vendor cannot give you a clean image and a fixed version, cannot be made trustworthy again. The pack does not cover appliance replacement, so this is your own judgement with the vendor and your IT provider.

  11. Outcome

    Replace it, and do not reconnect the old one

    Keep the old appliance off the network, labelled and stored as evidence until the incident manager agrees it may be wiped. Set up the replacement from a clean configuration, not a copy of the old one, with new credentials. It goes into service only when the plan's recovery criteria are met.

    In the pack

  12. Decision

    Do you know the route in and every change the attacker made?

    This is the test the pack's ransomware playbook sets for each system before choosing between restoring and rebuilding. Applied here, it decides whether patching in place is safe. Yes only when the investigation has found how the attacker got in and has accounted for every change on the appliance. If you are unsure, the answer is no.

    In the pack

  13. Step

    Patch it in place, remove the changes and check it

    The ransomware playbook's rule for this answer, applied here by analogy: put the system right on its patched version, change its credentials and check it before reconnecting. For an appliance, install the fixed version, undo every change the investigation found and apply the new credentials. The patch closes the way in. It is not proof that the attacker has gone: that proof comes from the investigation and from the recovery criteria.

    In the pack

  14. Step

    Rebuild it from a clean image with a fixed version

    In the pack, rebuild means wiping it, installing again from a clean image and then restoring only data. For an appliance, that means re-creating its configuration rather than copying back a file the attacker could have changed. Where the attacker had administrator rights, the ransomware playbook rebuilds the systems it controlled, starting with the sign-in directory; applied here by analogy, check whether the appliance's access reached that far. The plan says not to rebuild before the evidence is safe, so take a full image first if it is not.

    In the pack

  15. Decision

    Are all five recovery criteria met?

    The plan returns a system to service only when: the route in is understood and closed; any image used was checked for integrity; it is patched, scanned and monitored with credentials changed; its owner has tested it; and the monitoring period has started. A patched appliance with no investigation behind it fails the first criterion.

    In the pack

  16. Outcome

    Keep it off the network until the gap is closed

    Whatever criterion is missing goes back to the phase that owns it: the investigation for the route in, the technical responders for credentials and monitoring, the owner for testing. Keep users on the containment route agreed earlier in the meantime.

    In the pack

  17. Outcome

    Reconnect it under closer monitoring

    Bring it back under closer watching. In the pack's defaults the monitoring period is 7 calendar days for Severity 1 and 2 incidents. The incident closes when the criteria are met, the monitoring period has passed without the attacker returning, and the case log is complete.

    In the pack

Practical checklist

  1. Ask your providers to preserve the appliance's logs and extend their retention before anything changes.
  2. Check for the published indicators, accounts or settings nobody made, odd sign-ins and unusual connections into the network.
  3. If there is any sign, or you cannot check, declare an incident and export the configuration and logs.
  4. Contain the least disruptive way, bringing in dependent service owners before cutting the appliance off.
  5. Change every credential on it or used through it, and check the network behind it for new accounts and tools.
  6. Patch in place only if you know the route in and every change; otherwise rebuild from a clean image with a fixed version.
  7. Replace it if the vendor cannot supply fixes or a trusted image, and keep the old one off the network as evidence.
  8. Reconnect only when all five recovery criteria are met, then watch it through the monitoring period.

Evidence to have ready

  • The preserved logs, the exported configuration and any image taken before changes.
  • The check for signs of use: what was looked at and how far back the logs went.
  • The decision to patch, rebuild or replace, with who decided and why.
  • The credential changes with dates, and each recovery criterion met before reconnection.

Assumptions

  • The appliance gives staff, suppliers or sites remote access to your network, so taking it offline has a business cost.
  • The response steps are the defaults in the CISO Times incident management pack, and the priorities and deadlines are the defaults in its vulnerability pack. Your own plan and standard may set different ones.
  • The pack has no appliance-specific procedure. Where this flow applies the pack's general rules to an appliance, it says so.

Not covered here