/gke-upgrades

name: gke-upgrades
metadata:
  category: Containers
description: >-
  Plans, executes, and validates Google Kubernetes Engine (GKE) cluster upgrades
  and maintenance operations for both Standard and Autopilot clusters. Produces
  upgrade plans, pre/post-upgrade checklists, maintenance runbooks with gcloud
  commands, release channel strategy, and troubleshooting guides. Handles node
  pool upgrade strategies (surge, blue-green), version compatibility, PDB
  management, and workload-specific concerns (stateful, GPU, operators). Use this
  skill whenever the user mentions GKE upgrades, Kubernetes version bumps, node
  pool maintenance, GKE patching, cluster version management, release channel
  selection, maintenance windows, surge upgrades, stuck upgrades, or any GKE
  lifecycle management task — even casual mentions like "we need to upgrade our
  clusters" or "plan our next GKE maintenance" or "our upgrade is stuck." Don't
  use for GKE cluster creation, application onboarding, general networking/routing
  setup, or security policy configurations (use gke-basics or relevant GKE skills
  instead).

GKE Upgrades & Maintenance

Produce clear, actionable documents — upgrade plans, runbooks, or checklists — tailored to the user's environment. Output should be specific to their cluster mode, release channel, version, and workload types rather than generic advice.

Always frame guidance around the auto-upgrade model: auto-upgrade with maintenance windows and exclusions is the preferred control mechanism.

Context Gathering

Before producing any upgrade artifact, establish:

• **Cluster mode** — Standard or Autopilot? (Autopilot has no node pool management, mandatory resource requests, no SSH)
• **Current and target versions** — Node version skew must be within 2 minor versions of control plane.
• **Release channel** — Rapid, Regular, Stable, or Extended.
• **Environment topology & Rollout Sequencing** — Single vs multi-cluster, dev/staging/prod tiers, and whether Rollout Sequencing is used.
• **Workload sensitivity** — StatefulSets, databases, GPU, long-running batch need special handling.

If the user provides these upfront, skip straight to the deliverable. If they're vague, fill in reasonable defaults and flag assumptions.

Core Principles

GKE versions follow Kubernetes version terminology: **Major.Minor.Patch** (e.g., 1.30.1-gke.1187000). A **Minor** version bump (e.g., 1.29 → 1.30) introduces new features and APIs. A **Patch** version bump (e.g., 1.30.1 → 1.30.2) introduces security and bug fixes. Ensure the user understands this distinction.

1. **Sequential control plane, skip-level node pools** -- Control plane upgrades are sequential (N → N+1 → N+2). Node pools support skip-level (N+2) upgrades. 2. **Control plane first** -- Control plane must be upgraded before node pools. Nodes can trail by up to 2 minor versions. 3. **Environment progression** -- Always upgrade dev/staging before production. Use **Rollout Sequencing** (preferred) to automate and enforce this progression across environments (e.g., dev → staging → prod), or manually coordinate version progression if Rollout Sequencing is not used. 4. **Workload-aware** -- Upgrade strategy depends on what's running (stateless, stateful, GPU, batch). 5. **Release channels first** -- Always recommend release channels. Note that "No channel" (static versioning) is deprecated and clusters should be migrated to release channels. 6. **Rollback/Downgrade** -- Control Plane patches and Node Pools (minor and patches) can be rolled back (downgraded to a target version). GKE supports a 2-step Control Plane minor upgrade where step 1 is rollbackable. Other Control Plane minor version rollbacks are NOT customer-doable and require GKE Support. 7. **Node pool upgrade ordering** -- When upgrading multiple node pools, always recommend sequential ordering: upgrade non-critical/stateless pools first (acting as a canary) to verify cluster health before upgrading critical stateful (database) or GPU pools.

Release Channels

| Channel | Best for | SLA | |---------|----------|-----| | **Rapid** | Dev/test, early feature access | No upgrade stability SLA | | **Regular** (default) | Most production | Full SLA | | **Stable** | Mission-critical, stability-first | Full SLA | | **Extended** | Compliance, EoS enforcement control | Full SLA |

Support Lifecycle

Standard GKE versions are supported for 14 months after they become available in the **Regular** channel. This means:

• **Rapid** channel versions may be supported for longer than 14 months (since they enter Rapid before Regular).
• **Stable** channel versions may be supported for less than 14 months (since they enter Stable after Regular).
• **Extended** support extends this period up to 24 months. Note that extra cost applies only during the extended support period (months 15-24).

Current Capabilities

• **Extended channel math**: 14 months of standard support + ~10 months of extended support ≈ 24 months total per minor version. Even on Extended, forced upgrades still occur: if you take no action, GKE auto-upgrades the cluster at End of Support — averaging a minor version bump roughly every 4 months, the same cadence as other channels (features just arrive later).
• **Upgrade reliability (KubeCon NA 2025)**: Google reports a 99.99% upgrade success rate across GKE control planes and nodes, with safe rollback and skip-version upgrade support positioned to let teams upgrade less often (e.g., annually instead of quarterly). Pilot skip-version upgrades in non-production clusters first.
• **Autoscaled blue-green node upgrades** (Preview): a blue-green variant that scales the green pool on demand instead of pre-provisioning a full duplicate pool — for disruption-sensitive workloads that cannot reserve 2x capacity.
• **Scheduled cluster upgrade notifications** (Preview): opt in to be notified ahead of scheduled minor upgrades and wire the notifications into alerting.
• **Graceful term