/agentforce-test

name: agentforce-test
description: "Write, run, and analyze structured test suites for Agentforce agents — functional AND security. TRIGGER when: user writes or modifies test spec YAML (AiEvaluationDefinition); runs sf agent test create, run, run-eval, or results commands; asks about test coverage strategy, metric selection, or custom evaluations; interprets test results or diagnoses test failures; asks about batch testing, regression suites, or CI/CD test integration; requests security testing, OWASP LLM Top 10, red-teaming, penetration testing, prompt-injection tests, a security grade, or a vulnerability assessment of an agent. DO NOT TRIGGER when: user creates, modifies, previews, or debugs .agent files (use agentforce-generate); deploys or publishes agents; writes Agent Script code; uses sf agent preview for development iteration; analyzes production session traces (use agentforce-observe); performs a static safety review of .agent file content (use agentforce-generate Section 15)."
allowed-tools: Bash Read Write Edit Glob Grep
metadata:
  relatedSkills:
    - "agentforce-generate"
    - "agentforce-observe"
  version: "0.8"
  domains: ["Agentforce"]
  cliTools:
    - tool: ["curl"]
      semver: ">=7.0.0"
    - tool: ["jq"]
      semver: ">=1.6.0"
    - tool: ["python3"]
      semver: ">=3.10.0"
    - tool: ["sf"]
      semver: ">=2.121.7"

ADLC Test

Automated testing for Agentforce agents with smoke tests, batch execution, and iterative fix loops.

Overview

This skill provides comprehensive testing capabilities for Agentforce agents, including automated utterance derivation from agent subagents, preview-based smoke testing, trace analysis, an iterative fix loop for identified issues, and **security testing** (OWASP LLM Top 10). It bridges the gap between initial development and production deployment.

**Security testing is part of the ADLC, not a separate skill.** Functional correctness (right topic, right action) and security posture (resists attacks) are two dimensions of the same test suite. Treat adversarial coverage as part of the test flow and the Agent Spec — when you plan tests for an agent, plan its security tests too. Security test-case generation is **gated on explicit user confirmation** (see Mode C).

Platform Notes

• Shell examples below use bash syntax. On Windows, use PowerShell equivalents or Git Bash.
• Replace `python3` with `python` on Windows.
• Replace `/tmp/` with `$env:TEMP\` (PowerShell) or `%TEMP%\` (cmd).
• Replace `jq` with `python -c "import json,sys; ..."` if jq is not installed.
• `find ... | head -1` -> `Get-ChildItem -Recurse ... | Select-Object -First 1` in PowerShell.

Usage

This skill uses `sf agent preview` and `sf agent test` CLI commands directly. There is no standalone Python script.

**Quick smoke test (Mode A):**

# Start preview, send utterance, end session (--authoring-bundle generates local traces).
# Run from inside the Salesforce project directory (the CLI requires sfdx-project.json).
# With --authoring-bundle, `start` REQUIRES an action mode: --simulate-actions or
# --use-live-actions. The mode flag belongs on `start` only — `send` and `end` reject it.
sf agent preview start --json --authoring-bundle MyAgent --simulate-actions -o <org-alias>
sf agent preview send --json --session-id <ID> --utterance "test" --authoring-bundle MyAgent -o <org-alias>
sf agent preview end --json --session-id <ID> --authoring-bundle MyAgent -o <org-alias>

**Batch testing (Mode B):**

# Deploy and run test suite
sf agent test create --json --spec test-spec.yaml --api-name MySuite -o <org-alias>
sf agent test run --json --api-name MySuite --wait 10 --result-format json -o <org-alias>

**Security testing (Mode C — confirm with the user before generating):**

# You read the .agent file and write the security cases yourself — same as
# Mode B, with security-specific guidance in references/security-test-design.md.

# C1: deploy the security suite you authored (identical to Mode B)
sf agent test create --json --spec /tmp/MyAgent-security-spec.yaml --api-name MyAgent_Security -o <org-alias>

# C2: live adversarial probing (identical to Mode A, one fresh session per case).
# --simulate-actions is the C2 default: probe the agent's reasoning without firing
# real Apex/Flow writes. Only substitute --use-live-actions on explicit user opt-in.
sf agent preview start --json --authoring-bundle MyAgent --simulate-actions -o <org-alias>
sf agent preview send --json --session-id <ID> --utterance "<payload>" --authoring-bundle MyAgent -o <org-alias>
sf agent preview end --json --session-id <ID> --authoring-bundle MyAgent -o <org-alias>

**Action execution:**

# Execute a Flow or Apex action directly via REST API
TOKEN=$(sf org display -o <org-alias> --json | jq -r '.result.accessToken')
INSTANCE_URL=$(sf org display -o <org-alias> --json | jq -r '.result.instanceUrl')
curl -s "$INSTANCE_URL/services/data/v63.0/actions/custom/flow/Get_Order_Status" \
  -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"inputs": [{"orderId": "00190000023XXXX"}]}'

Testing Workflow

This skill supports three testing modes plus direct action execution:

• **Mode A: Ad-Hoc Preview Testing** -- Quick smoke tests during development using `sf agent preview`. No test suite deployment needed (org authentication still required). Best for iterative development and fix validation.
• **Mode B: Testing Center Batch Testing** -- Persistent test suites deployed to the org via `sf agent test`. Best for regression suites, CI/CD, and cross-skill integration with /agentforce-observe.
• **Mode C: Security Testing (OWASP LLM Top 10)** -- Adversarial testing across 7 OWASP categories. You write the cases yourself by reading the agent's own `.agent` script and business domain, using the neutral technique catalog in `assets/payloads/` as a coverage checklist. Two sub-modes over the same authored case set: **C1** deploys