/gke-upgrades

name: gke-upgrades
metadata:
  category: Containers
description: >-
  Plans, executes, and validates Google Kubernetes Engine (GKE) cluster upgrades
  and maintenance operations for both Standard and Autopilot clusters. Produces
  upgrade plans, pre/post-upgrade checklists, maintenance runbooks with gcloud
  commands, release channel strategy, and troubleshooting guides. Handles node
  pool upgrade strategies (surge, blue-green), version compatibility, PDB
  management, and workload-specific concerns (stateful, GPU, operators). Use this
  skill whenever the user mentions GKE upgrades, Kubernetes version bumps, node
  pool maintenance, GKE patching, cluster version management, release channel
  selection, maintenance windows, surge upgrades, stuck upgrades, or any GKE
  lifecycle management task — even casual mentions like "we need to upgrade our
  clusters" or "plan our next GKE maintenance" or "our upgrade is stuck." Don't
  use for GKE cluster creation, application onboarding, general networking/routing
  setup, or security policy configurations (use gke-basics or relevant GKE skills
  instead).

GKE Upgrades & Maintenance

Produce clear, actionable documents — upgrade plans, runbooks, or checklists — tailored to the user's environment. Output should be specific to their cluster mode, release channel, version, and workload types rather than generic advice.

Always frame guidance around the auto-upgrade model: auto-upgrade with maintenance windows and exclusions is the preferred control mechanism.

Context Gathering

Before producing any upgrade artifact, establish:

• **Cluster mode** — Standard or Autopilot? (Autopilot has no node pool management, mandatory resource requests, no SSH)
• **Current and target versions** — Node version skew must be within 2 minor versions of control plane.
• **Release channel** — Rapid, Regular, Stable, or Extended.
• **Environment topology & Rollout Sequencing** — Single vs multi-cluster, dev/staging/prod tiers, and whether Rollout Sequencing is used.
• **Workload sensitivity** — StatefulSets, databases, GPU, long-running batch need special handling.

If the user provides these upfront, skip straight to the deliverable. If they're vague, fill in reasonable defaults and flag assumptions.

Core Principles

GKE versions follow Kubernetes version terminology: **Major.Minor.Patch** (e.g., 1.30.1-gke.1187000). A **Minor** version bump (e.g., 1.29 → 1.30) introduces new features and APIs. A **Patch** version bump (e.g., 1.30.1 → 1.30.2) introduces security and bug fixes. Ensure the user understands this distinction.

1. **Sequential control plane, skip-level node pools** -- Control plane upgrades are sequential (N → N+1 → N+2). Node pools support skip-level (N+2) upgrades. 2. **Control plane first** -- Control plane must be upgraded before node pools. Nodes can trail by up to 2 minor versions. 3. **Environment progression** -- Always upgrade dev/staging before production. Use **Rollout Sequencing** (preferred) to automate and enforce this progression across environments (e.g., dev → staging → prod), or manually coordinate version progression if Rollout Sequencing is not used. 4. **Workload-aware** -- Upgrade strategy depends on what's running (stateless, stateful, GPU, batch). 5. **Release channels first** -- Always recommend release channels. Note that "No channel" (static versioning) is deprecated and clusters should be migrated to release channels. 6. **Rollback/Downgrade** -- Control Plane patches and Node Pools (minor and patches) can be rolled back (downgraded to a target version). GKE supports a 2-step Control Plane minor upgrade where step 1 is rollbackable. Other Control Plane minor version rollbacks are NOT customer-doable and require GKE Support. 7. **Node pool upgrade ordering** -- When upgrading multiple node pools, always recommend sequential ordering: upgrade non-critical/stateless pools first (acting as a canary) to verify cluster health before upgrading critical stateful (database) or GPU pools.

Release Channels

| Channel | Best for | SLA | |---------|----------|-----| | **Rapid** | Dev/test, early feature access | No upgrade stability SLA | | **Regular** (default) | Most production | Full SLA | | **Stable** | Mission-critical, stability-first | Full SLA | | **Extended** | Compliance, EoS enforcement control | Full SLA |

Support Lifecycle

Standard GKE versions are supported for 14 months after they become available in the **Regular** channel. This means:

• **Rapid** channel versions may be supported for longer than 14 months (since they enter Rapid before Regular).
• **Stable** channel versions may be supported for less than 14 months (since they enter Stable after Regular).
• **Extended** support extends this period up to 24 months. Note that extra cost applies only during the extended support period (months 15-24).

Maintenance Windows & Exclusions

Configure maintenance windows to control auto-upgrade timing. GKE also supports node pool level maintenance exclusions (in addition to cluster level) to block upgrades for specific workloads.

**Exclusion types & Limits:**

• **"No upgrades" (Scope: `no_upgrades`)**: Blocks all upgrades (minor, patch, node).
• **Limit**: Max **90 days** of total exclusion duration in any **rolling 365-day window**.
• **Chaining constraint**: Because of the rolling 365-day limit, you cannot chain multiple exclusions to cover a continuous period longer than 90 days (e.g., you cannot cover a 100-day freeze using `no_upgrades`).
• **"No minor or node upgrades" (Scope: `no_minor_or_node_upgrades`)**: Blocks minor and node upgrades, but allows control plane patch upgrades (low risk).
• **Limit**: Up to **180 days per exclusion**. Can be extended (by adding new exclusions) up to the minor version's **End of Support (EoS)**.
• **"No minor upgrades" (Scope: `no_minor_upgrades`)**: Blocks minor upgrades, but allows control plane patches and node upgrades.
• **Limit**: Up to **180 days p