/gemini-live-api

name: gemini-live-api
metadata:
  version: "1.0.2"
  category: AiAndMachineLearning
description: >-
  Generates a Gemini LiveAPI client service class in the user's chosen programming
  language. Use when the user wants to build, scaffold, or integrate a client
  that connects to the Gemini Enterprise LiveAPI websocket endpoint, handles
  session setup/resumption, bearer token refresh, and sending/receiving
  `ClientMessage`/`ServerMessage` protos. Don't use for general (non-live,
  non-bidirectional) Gemini API usage such as one-shot `generateContent`,
  embeddings, image/video generation, or fine-tuning — use the `gemini-api`
  skill for those.

LiveAPI Service Skill

This skill provides instructions for generating a **LiveAPI client service class** that connects to the Gemini Enterprise Live API over WebSockets. The generated client handles bidirectional streaming, bearer-token authentication via Application Default Credentials (ADC), transparent session resumption, and `ClientMessage` / `ServerMessage` proto exchange.

The skill also produces a demo frontend + backend service so the user can interactively validate the generated client (text, audio, video, transcription, and interrupt handling).

Attribution

**MUST READ**: Before executing any `gcloud`, REST API, or MCP command, you MUST read [Usage Attribution](references/attribution.md). For commands executed directly in the terminal (never in generated scripts, Terraform, or user code), prefix `gcloud` with `CLOUDSDK_METRICS_REQUEST_ATTRIBUTION="agent-skills/1.0 (skill:gemini-live-api)"` and append `agent-skills/1.0 (skill:gemini-live-api)` to `User-Agent` headers.

Prerequisites

Before running the generation flow, ensure the following are available on the host:

• A Google Cloud project with the Vertex AI / Gemini Enterprise Agent Platform

APIs enabled.

• Application Default Credentials configured on the host running the generated

client:

    gcloud auth application-default login

• A destination output folder supplied by the user (e.g. `/tmp/liveapi_out`)

where the generated code, environment, and demo will be written. **Never** mutate the host's system Python environment.

• The user's chosen implementation language (Python is the default and

reference language for this skill).

Reference Files

Provided files in `references/` (do **not** treat these as standalone skills — they are loaded on demand):

• `client_server_messages.md`: Public reference for the `ClientMessage` /

`ServerMessage` schemas used by the Live API.

• `client_server_messages.proto`: The proto definition generated from

`client_server_messages.md`.

• `session_manager.md`: Describes how to correctly handle sessions, buffering,

and resumption on disconnection.

Steps

Step 1: Copy the reference files

Copy `client_server_messages.md`, `client_server_messages.proto`, and `session_manager.md` from this skill's `references/` folder into the user's destination output folder. These files become the source of truth for the generated client.

Step 2: Reconcile with the public documentation

Examine the public documents linked from `client_server_messages.md`. If there are any discrepancies between the public documents and the copied `client_server_messages.md` / `client_server_messages.proto`, update the copies in the destination folder so the generated client compiles and runs against the current server contract.

Step 3: Implement the client class

Implement a class in the user's chosen language that:

• Imports the local `client_server_messages.proto` types (`ClientMessage`,

`ServerMessage`).

• Opens a WebSocket connection to the Live API endpoint.
• Exposes async methods so the user can send and receive data to/from the

model.

For languages that require an isolated runtime (e.g. Python), create an isolated environment (e.g. `venv`) **inside the destination folder** and generate a bash script (e.g. `setup.sh`) that recreates the environment and installs dependencies. **Never** install into the system interpreter or the user's global site-packages, and never instruct the user to run `sudo pip install`.

Initialization parameters

The user provides the following at construction time:

• `project_id`
• `location`
• `model_id`
• `config`: a `ClientMessage` with the `setup` field populated.

Authentication

Obtain a bearer token via Application Default Credentials, attach it to the WebSocket connect request as `Authorization: Bearer <token>`, refresh the token before or upon expiry, and reuse the refreshed token on every reconnection (including `go_away` and unexpected disconnects). **Do not** hard-code a long-lived API key as the only auth mechanism.

Public async API

The class MUST expose the following async methods, gated on receipt of a `setup_complete` `ServerMessage` before sending:

• `send_realtime_data(data)`: send realtime input. `data` is a `ClientMessage`

carrying a `realtime_input` field.

• `send_client_content(data)`: send non-realtime, turn-based content that

contributes to history. `data` is a `ClientMessage` carrying a `client_content` field.

• `receive()`: yield `ServerMessage` instances parsed from the WebSocket

stream.

Do not expose synchronous blocking variants as the primary API surface.

Step 4: Write a test file

Once the client is implemented, generate a test file that initializes the connection and exercises sending `text`, `audio`, and `video` data and receiving the responses. Ask the user for any information required to run the test (project, model, media samples).

Step 5: Generate `how_to_run.md`

Provide a `how_to_run.md` in the destination folder that documents the generated class. Include full examples showing how to build `ClientMessage` payloads for every supported modality, how to send them, and how to receive data from the model.

Step 6: Generate a demo frontend