/agent-platform-inference

name: agent-platform-inference
metadata:
  category: AiAndMachineLearning
description: >-
  Connects to and performs inference with Google Cloud Agent Platform GenAI
  models, including First-Party Gemini models and Third-Party OpenMaaS models
  (Llama, DeepSeek, Qwen, etc.). Use when you need to generate code for calling
  Gemini or OpenMaaS models, authenticate with GenAI SDK, OpenAI SDK, or legacy
  Agent Platform SDK, configure base URLs and global/regional endpoints, or troubleshoot
  429 Resource Exhausted (DSQ), 400 User Validation, or 404 Not Found errors.
  Don't use for deploying models to endpoints or for running model evaluations.

Agent Platform GenAI Inference Skill

This skill provides instructions for authenticating and connecting to Google Cloud Agent Platform to use Generative AI models. It covers:

• **First-Party publisher models** (Gemini) — section 2.
• **Third-Party publisher models** (OpenMaaS: Llama, DeepSeek, Qwen, etc.)

— section 3.

• **Custom endpoints** (any model on a numeric `projects/.../endpoints/<id>`

resource — tuned Gemini models, OSS LLMs self-deployed from Model Garden via the `agent-platform-deploy` skill, and legacy custom models) — section 4.

Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or scripts on behalf of the user, you must adhere to the following safety tiers based on the action requested. (The skill is read-only; other safety tiers are omitted):

1. **Tier R: Read-only / Inference (`client.models.generate_content`, `client.chat.completions.create`, `client.completions.create`, `client.embeddings.create`)**

• Requires **interactive confirmation** with 'Yes'/ 'No' options before

executing model inference on behalf of the user, to prevent unexpected cost or quota consumption. The confirmation prompt must clearly explain the proposed inference execution and its key parameters (e.g., target model ID, SDK choice, input prompt). Natural-language paraphrases without specifying the parameters are NOT sufficient.

• **Same-turn restriction**: Do not execute the inference scripts or

commands in the same turn as presenting the confirmation prompt. Stop and wait for the user's reply; only execute after explicit 'Yes' / approval.

• **Gold Standard Example**: > I will perform model inference with the

following parameters. Please > confirm this information before I proceed: > * **Model ID**: `deepseek-ai/deepseek-v3.2-maas` > * **SDK**: OpenAI SDK (via Vertex AI Endpoint) > * **Input Prompt**: "Explain the concept of quantum computing..." > Do you confirm? [Yes/No]

Phase 0: Environment Setup

**CRITICAL**: Before running any of the Python sample scripts in the `scripts/` directory (e.g., `scripts/openmaas_openai_sdk.py`), you MUST ensure the environment is correctly initialized by following these steps:

1. **Google Cloud Authentication**: Authenticate with your Google Cloud credentials and configure active Application Default Credentials (ADC) for Agent Platform access:

    gcloud auth login
    gcloud auth application-default login

2. **Enable API** (if not already enabled):

    gcloud services enable aiplatform.googleapis.com

3. **Python Dependencies**: The scripts import `vertexai` (from `google-cloud-aiplatform`), `google-genai`, and `openai`. Do **not** create a virtual environment — it starts empty and hides packages the environment already provides, forcing a redundant install. Probe, and install only what is missing:

    python3 -c "import vertexai, google.genai, openai" \
      || pip install -r scripts/requirements.txt

The pins in `scripts/requirements.txt` are a fallback for an environment that does not already provide these SDKs; do not apply them on top of a working environment.

4. **Verify Setup (Optional)**: Run all sample scripts at once to verify the environment is working end-to-end:

    ./scripts/verify_all.sh

5. **Execution**: Run the scripts with a plain `python3 scripts/...`. There is no environment to activate first.

> [!IMPORTANT] **CRITICAL: Model IDs & Availability** * **Gemini Models**: See > [Gemini Models][gemini-models-docs] for valid Model IDs and Regions. * > **OpenMaaS Models**: See > [Use Open Models on Agent Platform](https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/maas/use-open-models) > for Llama, DeepSeek, Qwen, etc. * **Incomplete Lists**: The Model IDs listed > in this skill are **examples only** and may be incomplete or outdated. * > **Action**: Always verify the Model ID and Region using the links above before > generating code. > > \[gemini-models-docs]: > https://docs.cloud.google.com/gemini-enterprise-agent-platform/models/migrate > > > ## Workflow Decision Tree

1. **Model Family Identification**: Has the user specified whether they want to call a **Gemini** (First-Party) model or an **OpenMaaS** (Third-Party, e.g. Llama, DeepSeek, Qwen) model?

• **No** -> Ask the user which model family they want to use. If they

provide a specific model name, infer the family from the name.

• **Yes** -> Proceed to Step 2.

2. **SDK Choice**: Which SDK does the user want to use?

• **Gemini + GenAI SDK** (preferred for Gemini) -> Proceed to [1. Gemini

Models].

• **Gemini + legacy Vertex AI SDK** -> Proceed to [1. Gemini Models].
• **OpenMaaS + OpenAI SDK** (preferred for OpenMaaS) -> Proceed to [2.

OpenMaaS Models].

• **OpenMaaS + GenAI SDK** -> Proceed to [2. OpenMaaS Models].
• **Unsure** -> Default to the preferred SDK for the chosen family.

3. **Troubleshooting**: Is the user reporting an error (429 Resource Exhausted, 400 User Validation, 404 Not Found, etc.)?

• **Yes** -> Proceed to [3. Tr