Skip to content
Development
Agent

annotations

Annotations are named labels/scores attached to spans, traces, or experiment runs by humans, code, or LLM judges.

From plugin
phoenix
11k7 skills7 agents
Install
$ npx -y skills add arize-ai/phoenix --agent claude-code

How it fires

How this agent gets triggered: by you, by Claude, or both.

  • Fires itselfAuto-invocation. Claude auto-loads it when your prompt matches the work.Auto-invocation is when the right skill fires by itself at the right moment, driven by a FLOW.md router and a hook, instead of you invoking it by name. It is the difference between a skill being installed and a skill actually getting used.Read the full definition →
  • You can call itInvoke it directly when you want it.

Context preview

The summary Claude sees to decide when to auto-load this agent.

Annotations are named labels/scores attached to spans, traces, or experiment runs by humans, code, or LLM judges.

Agent definition

annotations.md

Annotations

Annotations are named labels/scores attached to spans, traces, or experiment runs by humans, code, or LLM judges.

Fields

`SpanAnnotation` and `TraceAnnotation` share these fields:

  • `name`, `label`, `score`, `explanation`
  • `annotatorKind` (`AnnotatorKind` enum: human / LLM / code)
  • `metadata`, `identifier`, `createdAt`, `updatedAt`

`SpanAnnotation` additionally has `spanId: GlobalID!` and `span`; `TraceAnnotation` has `trace` (but no `traceId` global-id field).

`ExperimentRunAnnotation` has the same scalar shape plus `error: String`, and uses `startTime`/`endTime` instead of `createdAt`/`updatedAt`.

Reading annotations

  • Per span: `Span.spanAnnotations { name label score explanation annotatorKind }`.
  • Project-wide discovery and rollups: `Project.spanAnnotationNames`, `Project.spanAnnotationSummary`, `Project.traceAnnotationsNames`, `Project.traceAnnotationSummary` — use these to learn which annotation names exist before drilling in.
  • In a span `filterCondition`, reference annotations as `annotations['<name>'].label` / `.score` (or the legacy `evals['<name>']`).

Example

Spans that an LLM judge labelled as hallucinated, with the annotation detail:

query Hallucinations($id: ID!) {
  node(id: $id) {
    ... on Project {
      spans(first: 20, filterCondition: "annotations['Hallucination'].label == 'hallucinated'") {
        edges {
          node {
            spanId
            spanAnnotations { name label score explanation }
          }
        }
      }
    }
  }
}
Read more
Ships withphoenix

AI Observability & Evaluation

Get the whole plugin