/agent-platform-alert-configuration

name: agent-platform-alert-configuration
metadata:
  category: AiAndMachineLearning
description: >-
  Configures best-practice alerting policies for AI agents using OpenTelemetry
  (OTel) metrics. Use when analyzing, writing, or deploying alerting policies
  to monitor agent latency, error rates, token usage, and quality metrics.
  NOTE: Reliability, Cost, Safety, and Security alerts use generic OTel metrics
  and work across runtimes (e.g., Cloud Run, Vertex AI). Quality alerts rely
  on Vertex AI Online Monitors and are strictly bound to Vertex AI deployments.
allowed-tools: terraform gcloud python

Agent Platform Alert Configuration

Critical Steps

1. Safety & Confirmation Tiers (CRITICAL)

Before executing any commands or writing configurations on behalf of the user, you MUST adhere to the following safety tiers based on the action requested:

1. **Tier R: Read-only (`check_telemetry.py` / `gather_agent_info.py`)**

• **Rule**: No confirmation needed. You may execute these scripts

immediately to inspect telemetry status or gather agent configuration details. 2. **Tier B: Billing & Resource Creation (`create_online_monitor.py` / provisioning)**

• **Rule**: **Explicit User Confirmation Required**. These actions incur

additional billing charges and create cloud resources. The agent MUST ALWAYS warn the user explicitly about the potential extra billing costs of BOTH the Online Monitor (specifically mentioning **LLM evaluations**) and Telemetry (specifically mentioning **Cloud Trace/Logging export**). You MUST STOP and ask for explicit approval before proceeding with provisioning or providing setup commands.

2. Prerequisites & Dependencies

Agent Telemetry

• **Disclaimer**: For Reliability, Cost, Safety, and Security alerts to

function, the underlying agent MUST be instrumented to emit OpenTelemetry (OTel) metrics. If the agent does not emit these metrics, the alerting policies will have no data stream to evaluate.

Python Environment

Before executing any python script in this skill you MUST install the required dependencies in your environment. Run this command first:

pip install -r scripts/requirements.txt

3. Input Assumptions

• **Explicit Project Adherence**: You must ONLY configure alerts, query

telemetry, or interact with the Google Cloud Project(s) explicitly provided by the user in the prompt. Do NOT assume or use other projects from your environment or history unless the user explicitly directs you to do so.

• **Sequential File Transformations**: If the user explicitly asks to copy a

file and then modify it, you MUST perform these actions sequentially (copy first, then modify) rather than writing the final content directly.

4. Execution Steps

1. **Mandatory Prerequisite Execution Protocol (SEQUENTIAL)**: Before generating or writing ANY configuration, you MUST execute these steps in order: 1. **Step 1: Streamlined Discovery (Mandatory)**: Run `gather_agent_info.py` to automatically identify agent runtime, check telemetry, metric scopes, linked datasets, and more. This script covers most of the manual checks listed in subsequent steps.

• Command: `python3 scripts/gather_agent_info.py --project-id

{project_id} --agent-name {agent_name}`

• **Note**: If this script **fails**, returns **partial data**, or

doesn't produce everything you need, you MUST satisfy requirements by running the manual fallback steps listed in Step 2 and then perform Step 3 below. If Step 1 succeeds and provides all info, **SKIP** to Step 3 (Pre-existing Policies Check). 2. **Step 2: Metric Scope Check (Fallback)**: Run this ONLY if Step 1 failed to determine the metric scope.

• **Action A (CLI)**: Run `gcloud beta monitoring metrics-scopes list

projects/{project_id}`. If a scoping project is returned, you MUST deploy policies there.

• **Action B (Code Scan)**: Search Terraform configurations for

`google_monitoring_monitored_project` resources to extract the scoping project.

• **Action C (Fallback)**: If ambiguous, ASK the user: "Are you using

a multi-project Cloud Monitoring Metric Scope? If so, what is the scoping project ID?" 3. **Step 3: Pre-existing Policies Check**: Avoid duplicates.

• **Action**: Scan the target directory to see if aggregated policies

already exist targeting the same metrics (grouped by `reasoning_engine_id` or `gen_ai_agent_name`). Use `scan_duplicates.py` to verify. 2. **Alert Policy Type Resource Files**: You MUST list and read files under `references/` with names ending in `_alert_policies.md` to learn how to configure alert policies based on type. By default you should configure all of the following alert types UNLESS the user requests to generate explicit alert policies and/or types. Follow their tables of content to help you find the reference sections you need to read:

Alert Type | Reference File :-------------- | :------------- **Reliability** | [reliability_alert_policies.md](references/reliability_alert_policies.md) **Quality** | [quality_alert_policies.md](references/quality_alert_policies.md) **Cost** | [cost_alert_policies.md](references/cost_alert_policies.md) **Safety** | [safety_alert_policies.md](references/safety_alert_policies.md) **Security** | [security_alert_policies.md](references/security_alert_policies.md)

5. Outputs & Formats

• **Always configure the supported alerting policies** for the target agent:
• **For Reliability Monitoring**: You MUST configure exactly five alerting

policies: 1. **Latency** (anom