/agent-platform-inference

name: agent-platform-inference
metadata:
  category: AiAndMachineLearning
description: >-
  Connects to and performs inference with Google Cloud Agent Platform GenAI
  models, including First-Party Gemini models and Third-Party OpenMaaS models
  (Llama, DeepSeek, Qwen, etc.). Use when asked to perform inference, ask a
  model a question, run a test prompt, execute chat completions, or generate
  code for calling Gemini or OpenMaaS models, authenticate with GenAI SDK,
  OpenAI SDK, or legacy Agent Platform SDK, configure base URLs and
  global/regional endpoints, or troubleshoot 429 Resource Exhausted (DSQ), 400
  User Validation, or 404 Not Found errors. Don't use for deploying models to
  endpoints or for running model evaluations.

Agent Platform GenAI Inference Skill

This skill provides instructions for authenticating and connecting to Google Cloud Agent Platform to use Generative AI models. It covers:

• **First-Party publisher models** (Gemini) — section 2.
• **Third-Party publisher models** (OpenMaaS: Llama, DeepSeek, Qwen, etc.)

— section 3.

• **Custom endpoints** (any model on a numeric `projects/.../endpoints/<id>`

resource — tuned Gemini models, OSS LLMs self-deployed from Model Garden via the `agent-platform-deploy` skill, and legacy custom models) — section 4.

Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or scripts on behalf of the user, you must adhere to the following safety tiers based on the action requested. (The skill is read-only; other safety tiers are omitted):

1. **Tier R: Read-only / Inference (`client.models.generate_content`, `client.chat.completions.create`, `client.completions.create`, `client.embeddings.create`)**

• Requires **interactive confirmation** with 'Yes'/ 'No' options before

executing model inference on behalf of the user, to prevent unexpected cost or quota consumption.

• **Required Fields in Confirmation Card**: The confirmation prompt must

clearly explain the proposed inference execution and explicitly list all of the following parameters:

• **Project ID**: The Google Cloud project ID or number (e.g.

`123456789012`, `my-project`).

• **Region / Location**: The target region (e.g. `us-central1`,

`global`).

• **Model ID**: The exact model ID (e.g. `gemini-2.5-flash`,

`deepseek-ai/deepseek-v3.2-maas`).

• **SDK**: The SDK choice (e.g. `Google GenAI SDK (google-genai)`,

`OpenAI SDK`).

• **Input Prompt** (or **Input Image** / **Input Media**): The prompt

text or media URI.

• Any additional generation parameters (e.g. `max_output_tokens`,

`response_schema`) if specified. Natural-language paraphrases without explicitly listing these parameters are NOT sufficient.

• **Same-turn restriction**: Do not execute the inference scripts or

commands in the same turn as presenting the confirmation prompt. Stop and wait for the user's reply; only execute after explicit 'Yes' / approval.

• **Gold Standard Example**:

> I will perform model inference with the following parameters. Please > confirm this information before I proceed: > * **Project ID**: `my-project` > * **Region**: `us-central1` > * **Model ID**: `gemini-2.5-pro` > * **SDK**: Google GenAI SDK (`google-genai`) > * **Input Prompt**: "Summarize the plot of Hamlet in 3 sentences" > > Do you confirm? [Yes/No]

Phase 0: Environment Setup

**CRITICAL**: Before running any of the Python sample scripts in the `scripts/` directory (e.g., `scripts/openmaas_openai_sdk.py`), you MUST ensure the environment is correctly initialized by following these steps:

1. **Google Cloud Authentication**: Authenticate with your Google Cloud credentials and configure active Application Default Credentials (ADC) for Agent Platform access:

    gcloud auth login
    gcloud auth application-default login

2. **Enable API** (if not already enabled):

    gcloud services enable aiplatform.googleapis.com

3. **Python Dependencies**: The scripts import `vertexai` (from `google-cloud-aiplatform`), `google-genai`, and `openai`. Do **not** create a virtual environment — it starts empty and hides packages the environment already provides, forcing a redundant install. Probe, and install only what is missing:

    python3 -c "import vertexai, google.genai, openai" \
      || pip install -r scripts/requirements.txt

`scripts/requirements.txt` is a fallback for an environment that does not already provide these SDKs; do not install it on top of a working environment.

4. **Verify Setup (Optional)**: Run all sample scripts at once to verify the environment is working end-to-end:

    ./scripts/verify_all.sh

5. **Execution**: Run the scripts with a plain `python3 scripts/...`. There is no environment to activate first.

> [!IMPORTANT] **CRITICAL: Model IDs & Availability** * **Gemini Models**: See > [Gemini Models][gemini-models-docs] for valid Model IDs and Regions. * > **OpenMaaS Models**: See > [Use Open Models on Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models) > for Llama, DeepSeek, Qwen, etc. * **Incomplete Lists**: The Model IDs listed > in this skill are **examples only** and may be incomplete or outdated. * > **Action**: Always verify the Model ID and Region using the links above before > generating code. > > \[gemini-models-docs]: > https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate >

Parameter Grounding & Clarification Protocol (CRITICAL)

Before preparing code or presenting a Tier R confirmation card, you MUST ensure all necessary parameters are ground