/google-cloud-solution-guided-gke-ai-migration

name: google-cloud-solution-guided-gke-ai-migration
metadata:
  category: MultiProductSolutions
description: >-
    Guides the migration of existing AI workloads (Cloud Run, Gemini API, Gemini Enterprise Agent Platform) to self-hosted GKE inference using gcloud and kubectl. Use when the user has an existing AI inference workload (on Cloud Run, the Gemini API, Gemini Enterprise Agent Platform, or a custom VM) and wants to move it to self-hosted inference on GKE, or asks follow-up questions during such a migration (hardware sizing, model staging, manifest generation, validation, traffic cutover). DO NOT use for brand new GKE inference deployments with no existing workload to migrate (use gke-inference instead). DO NOT use if the user intends to automate the migration via the Gemini Cloud Assist MCP server.

Migrate AI Workloads to GKE Inference

This skill guides agents through the end-to-end process of migrating existing AI inference workloads (e.g., from Cloud Run, Gemini API, Gemini Enterprise Agent Platform) to self-hosted inference on Google Kubernetes Engine (GKE). The agent will act as an interactive architect, using a structured 4-phase workflow to discover requirements, design a Google Cloud-native solution, execute the implementation using `gcloud` and `kubectl`, and validate the deployment.

Gemini Cloud Assist MCP off-ramp

This skill covers manual, architect-guided migration only. Automated migration is the job of the Gemini Cloud Assist MCP server. Route between them as follows:

• **The user asks to USE Gemini Cloud Assist or MCP automation for this migration** (e.g., "use the Cloud Assist MCP server to do this"): stop the manual workflow and respond with the 4 required points below.
• **The user mentions MCP only in passing, or explicitly declines it** (e.g., "no MCP, let's do this manually"): proceed with the manual workflow. Do not stop and do not ask about MCP.
• **The user does not mention MCP at all:** proceed directly to the active phase. In your first discovery response only, add one sentence noting that an automated alternative exists via the Gemini Cloud Assist MCP server and the user can switch to it at any time. Do not wait for an answer before beginning discovery.

**When stopping for an MCP request, your response MUST include these 4 points:**

1. **STOP the manual workflow & clarify scope:** State that `google-cloud-solution-guided-gke-ai-migration` is strictly intended for manual, architect-guided migration using native CLIs (`gcloud` and `kubectl`), and that this manual skill workflow is being stopped. 2. **Explain MCP capabilities:** Explain that the Gemini Cloud Assist MCP server assists in automated infrastructure analysis (`gemini_cloud_assist:ask_cloud_assist`) or direct Google Cloud resource mutation (`gemini_cloud_assist:invoke_operation`). 3. **Link to MCP Documentation:** Provide a valid hyperlink to the [Gemini Cloud Assist MCP Documentation](https://docs.cloud.google.com/cloud-assist/configure-mcp). 4. **Link to Intent to Infrastructure Codelab:** Provide a valid hyperlink to the [Intent to Infrastructure Codelab](https://github.com/GoogleCloudPlatform/next-26-keynotes/tree/main/devkey/intent-to-infrastructure) for guidance on setting up the MCP server.

Scope Check: New Deployments vs. Migrations

**This skill is specifically intended for migrating existing AI workloads (from Cloud Run, Gemini API, Agent Platform, or other platforms) to GKE.**

If the user wants to deploy a new AI model server from scratch on GKE (and does NOT have an existing deployment to migrate), **STOP** and recommend using the **`gke-inference`** skill instead. Explain that `google-cloud-solution-guided-gke-ai-migration` focuses on migration workflows (discovering existing Cloud Run/Agent Platform configurations, traffic cutover, etc.), while `gke-inference` is optimized for fresh GKE AI model server deployments using AI Profiles and golden path manifests.

Core Architectural Principles (The "Golden Path")

When designing the solution, always default to the latest GKE AI best practices:

• **Execution:**
• **Execution policy (who runs commands):**
• **Phase 1 (Discovery):** after the user grants permission, execute read-only `gcloud` inspection commands (`list`, `describe`) directly and summarize the results.
• **Phases 2-4:** write manifests to disk, then present the exact `gcloud` and `kubectl` commands for the user to run. Do not execute mutating commands (`apply`, `create`, `delete`, cluster or IAM changes) unless the user explicitly asks you to run them, in which case execute them and report each command's actual output.
• **Informational and troubleshooting questions:** answer with markdown guidance, manifests, and recommended commands only; execute nothing.
• Favor raw Kubernetes manifests, native CLIs (`gcloud` for infrastructure, `kubectl` for workloads), and opinionated templates.
• Save YAML files to the user's current directory and apply them using `kubectl`.
• Write ad-hoc scripts (e.g., for VRAM calculation) only if absolutely necessary.
• **Node Provisioning:**
• Utilize **Custom Compute Classes (CCC)** to maximize accelerator obtainability (e.g., dynamically choosing spot vs. on-demand or specific GPU profiles).
• Use GKE's managed GPU driver installations.
• Select appropriate node topologies: use a single node in a static pool for a simple job, or multiple nodes with LWS/CCC for larger jobs.
• **Inference Stack & Versioning:**
• Default to **vLLM** (`vllm/vllm-openai`) as the standard LLM serving engine. If migrating from Vertex AI, the user may opt to retain the Vertex AI Model Garden image (e.g., `pytorch-vllm-serve`), which is permissible.
• **Explicit Entrypoint Override:** Regardless of the chosen image, the vLLM Deployment MUST explicitly set `command: ["python", "-m", "vllm.entrypoints.openai.api_server"]` to bypass potentially problematic entrypoint scripts (like `gcs_download_launcher.sh` in Vertex A