/agent-platform-inference

name: agent-platform-inference
metadata:
  version: "1.0.1"
  category: AiAndMachineLearning
description: >-
  Connects to and performs inference with Google Cloud Agent Platform GenAI
  models, including First-Party Gemini models and Third-Party OpenMaaS models
  (Llama, DeepSeek, Qwen, etc.). Use when asked to perform inference, ask a
  model a question, run a test prompt, execute chat completions, or generate
  code for calling Gemini or OpenMaaS models, authenticate with GenAI SDK,
  OpenAI SDK, or legacy Agent Platform SDK, configure base URLs and
  global/regional endpoints, or troubleshoot 429 Resource Exhausted (DSQ), 400
  User Validation, or 404 Not Found errors. Don't use for deploying models to
  endpoints or for running model evaluations.

Agent Platform GenAI Inference Skill

This skill provides instructions for authenticating and connecting to Google Cloud Agent Platform to use Generative AI models. It covers:

• **First-Party publisher models** (Gemini) — section 2.
• **Third-Party publisher models** (OpenMaaS: Llama, DeepSeek, Qwen, etc.)

— section 3.

• **Custom endpoints** (any model on a numeric `projects/.../endpoints/<id>`

resource — tuned Gemini models, OSS LLMs self-deployed from Model Garden via the `agent-platform-deploy` skill, and legacy custom models) — section 4.

Attribution

**MUST READ**: Before executing any `gcloud`, REST API, or MCP command, you MUST read [Usage Attribution](references/attribution.md). For commands executed directly in the terminal (never in generated scripts, Terraform, or user code), prefix `gcloud` with `CLOUDSDK_METRICS_REQUEST_ATTRIBUTION` and append the attribution suffix to `User-Agent` headers:

CLOUDSDK_METRICS_REQUEST_ATTRIBUTION="agent-skills/1.0 (skill:agent-platform-inference)"
User-Agent: <existing_user_agent> agent-skills/1.0 (skill:agent-platform-inference)

Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or scripts on behalf of the user, you must adhere to the following safety tiers based on the action requested. (The skill is read-only; other safety tiers are omitted):

1. **Tier R: Read-only / Inference (`client.models.generate_content`, `client.chat.completions.create`, `client.completions.create`, `client.embeddings.create`)**

• Requires **interactive confirmation** with 'Yes'/ 'No' options before

executing model inference on behalf of the user, to prevent unexpected cost or quota consumption.

• **Required Fields in Confirmation Card**: The confirmation prompt must

clearly explain the proposed inference execution and explicitly list all of the following parameters:

• **Project ID**: The Google Cloud project ID or number (e.g.

`123456789012`, `my-project`).

• **Region / Location**: The target region (e.g. `us-central1`,

`global`).

• **Model ID**: The exact model ID (e.g. `gemini-2.5-flash`,

`deepseek-ai/deepseek-v3.2-maas`).

• **SDK**: The SDK choice (e.g. `Google GenAI SDK (google-genai)`,

`OpenAI SDK`).

• **Input Prompt** (or **Input Image** / **Input Media**): The prompt

text or media URI.

• Any additional generation parameters (e.g. `max_output_tokens`,

`response_schema`) if specified. Natural-language paraphrases without explicitly listing these parameters are NOT sufficient.

• **Same-turn restriction**: Do not execute the inference scripts or

commands in the same turn as presenting the confirmation prompt. Stop and wait for the user's reply; only execute after explicit 'Yes' / approval.

• **Gold Standard Example**:

> I will perform model inference with the following parameters. Please > confirm this information before I proceed: > * **Project ID**: `my-project` > * **Region**: `us-central1` > * **Model ID**: `gemini-2.5-pro` > * **SDK**: Google GenAI SDK (`google-genai`) > * **Input Prompt**: "Summarize the plot of Hamlet in 3 sentences" > > Do you confirm? [Yes/No]

• **Post-Execution Response Grounding (CRITICAL)**:

After receiving explicit user approval and executing the inference call via the SDK, the response returning the generated text **MUST explicitly confirm the execution parameters** alongside the model's output. Never return a bare model response alone. Always include:

• **Model ID**: The exact model ID used (e.g. `gemini-2.5-pro`

or `<MODEL_ID>`).

• **SDK**: The SDK used (e.g. `Google GenAI SDK (google-genai)`

or `OpenAI SDK`).

• **Project ID**: The Google Cloud project ID/number used.
• **Region**: The region or endpoint location used (e.g. `global`

or `us-central1`).

• **Generated Output**: The model's complete generated answer.

Phase 0: Environment Setup

**CRITICAL**: Before running any of the Python sample scripts in the `scripts/` directory (e.g., `scripts/openmaas_openai_sdk.py`), you MUST ensure the environment is correctly initialized by following these steps:

1. **Google Cloud Authentication**: Authenticate with your Google Cloud credentials and configure active Application Default Credentials (ADC) for Agent Platform access:

    gcloud auth login
    gcloud auth application-default login

2. **Enable API** (if not already enabled):

    gcloud services enable aiplatform.googleapis.com

3. **Python Dependencies**: The scripts import `vertexai` (from `google-cloud-aiplatform`), `google-genai`, and `openai`. Do **not** create a virtual environment — it starts empty and hides packages the environment already provides, forcing a redundant install. Probe, and install only what is missing:

    python3 -c "import ver