How do we set up patch deployment rings and maintenance windows?

Start

Patches need a safe order and agreed times

Rings and windows exist so a faulty patch reaches a few systems first and a working one reaches every system before its deadline. Both halves matter: a cautious plan that verifies late breaks the deadline, and a fast one with no pilot breaks the business. Record the plan in one place, with each system owner's agreement, and review it every cycle.

• Then: Give every system a patch owner, class, ring or group, and window

Step

Give every system a patch owner, class, ring or group, and window

The asset class decides how a patch is delivered and tested: end-user devices, servers, network and security appliances, business applications, and services a provider patches. It never changes the deadline. A system that fits two classes takes the class of the part being patched, so a business application's operating system is patched as a server and the application in its own window.

• Then: What kind of system is it?

Decision

What kind of system is it?

End-user devices go in rings and update through device management. Servers and appliances go in groups, each with a window. A cloud or software service the provider patches is the provider's job under its contract; you patch only your own side.

• End-user device: Build a pilot ring that covers every model and application
• Server or appliance: Does it have a redundant partner?
• Service a provider patches: Record it as provider-patched; check its notices every month

Outcome

Record it as provider-patched; check its notices every month

Record "patched by provider" in the inventory with the contract reference, and check the provider's security notices every month. Patch your own clients, agents and connectors in the cycle. The contract should carry a vulnerability management clause. Where its patch periods are longer than your own deadlines, the provider stays on its contract and you cover the gap on your side: for an exploited flaw, your own mitigation within 72 hours in the pack's defaults.

Step

Build a pilot ring that covers every model and application

The pilot is the test for end-user devices, so it holds the people and devices most likely to show a fault early: the IT team's own devices, the test servers, and in the pack's default 2 pilot users from each department and site, about 5% of devices. It must cover every hardware model and every business application in use. Pilot users work normally and report anything that stops working.

• Then: Spread the early ring; everyone else is the broad ring

Step

Spread the early ring; everyone else is the broad ring

The early ring is about 25% of devices in the pack's default, spread across every site and department and never more than half of any one team, so a fault there still leaves every team working. Every other device is the broad ring. Each ring starts only when the one before passed its rollback criteria, and users may postpone a restart only until the forced restart date. With fewer than 50 end-user devices, the pack allows two rings: the full pilot, then every other device.

• Then: Agree rollback criteria for every ring and group

Decision

Does it have a redundant partner?

A redundant server pair backs each other up: domain controllers, clustered nodes, load-balanced web servers. A high-availability appliance pair is a pair of firewalls or remote access gateways. The two are planned differently.

• Server pair: Put the two servers in different groups and windows
• Appliance pair: Keep an appliance pair in one window, standby first
• No partner: Place it in a group with its own window

Step

Put the two servers in different groups and windows

One member is always running, and a faulty patch reaches only one. The cost is that the second member is verified only after the later window, so a patch whose deadline falls before then has to be expedited. Check this when you design the windows, not when a patch arrives.

• Then: Must a supplier certify each patch before it is installed?

Step

Keep an appliance pair in one window, standby first

Both members of a firewall or gateway pair go in the same group and window, one at a time: the standby first, the other only after the first is back in service and has passed its service check. A pair never runs two firmware versions for longer than the window.

• Then: Must a supplier certify each patch before it is installed?

Step

Place it in a group with its own window

Every server belongs to one group and every group to one window. The first group holds the servers whose loss nobody notices for a night, and is patched only after the pilot ring and the test servers pass their rollback criteria. A server with no partner that cannot stop for its window goes through the options for systems that cannot be restarted.

• Then: Must a supplier certify each patch before it is installed?

Decision

Must a supplier certify each patch before it is installed?

Some suppliers support a system only if they have certified each operating system patch first, so installing an uncertified one would end their support. The pack's example is a warehouse management system. Ask the supplier, and check the contract for the certification date it sets. If you do not know, plan it as a certifying system.

• Yes, or not known: Give it its own group, with a window after certification
• No: Agree each window with its system owner, internet-facing first

Step

Give it its own group, with a window after certification

The group's window falls after the certification date the contract sets, and the supplier's engineer may need to attend. If certification can come after the deadline, agree with the supplier to certify within a set time of the release day; the pack's example asks for 5 working days, before the second window.

• Then: Agree each window with its system owner, internet-facing first

Step

Agree each window with its system owner, internet-facing first

Every server and appliance needs a window at least once a month, agreed by its owner. Internet-facing systems go in the earliest group their redundancy allows, because their deadlines are shortest. The pack's defaults count from the release day: Window A on the Wednesday of the following week, 8 days after, and Window B on the Sunday after that, 12 days after.

• Then: Would a Priority 2 patch be verified inside its deadline?

Decision

Would a Priority 2 patch be verified inside its deadline?

Test each window against the tightest routine deadline. A patch released on the release day is usually first detected the next day and, at Priority 2, due 14 days after that. The window's day plus 1 working day to verify must fall on or before that date. The pack's plan workbook calculates this check for every system.

• No, or not known: Move the window earlier, or add one
• Yes: Agree rollback criteria for every ring and group

Step

Move the window earlier, or add one

A window that verifies after a Priority 2 deadline turns every such patch into an expedited one. Bring the window forward, or agree a second one with the owner. A short planned outage at the quietest hour usually costs less than an exception.

• Then: Would a Priority 2 patch be verified inside its deadline? (back to an earlier step)

Step

Agree rollback criteria for every ring and group

Agree them with the pilot users and system owners before the first deployment, and record who agreed and when. The pack's defaults: a pilot device fails to start, connect or sign in; a tested application fails; more than 2% of a ring fails to install after two attempts; a server's service check fails 30 minutes after restart and cannot be fixed in the window; or the vendor withdraws the patch or publishes a known issue. Take snapshots before the window.

• Then: Does any check in the plan still fail?

Decision

Does any check in the plan still fail?

Before the cycle starts, every system should pass its checks: pairs split correctly, internet-facing systems in the earliest window, every window inside a Priority 2 deadline, and criteria agreed before each first deployment. Fix every failing check first. If you have not checked, treat it as failing.

• Yes, or not checked: fix it: Give every system a patch owner, class, ring or group, and window (back to an earlier step)
• No: Keep the plan with each owner's agreement; review it every cycle

Outcome

Keep the plan with each owner's agreement; review it every cycle

Review the plan at each cycle's preparation step, and whenever a system is added, a pair is broken or a supplier changes its certification dates. Keep each version as the record. An auditor asks to see the rings, groups and windows, and the rollback criteria agreed for each, approved by the system owners.