/ai-observability

name: ai-observability
description: >
  Use when adding Spring AI-specific model observations, token usage, latency, externally
  configured cost attribution, advisor telemetry, or protected prompt and completion logging.
  Use production-observability for general service metrics, health, logs, and OTLP setup.

AI Observability

Dependencies

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
</dependency>

Spring AI Built-in Observability

Spring AI 1.0+ includes built-in Micrometer instrumentation:

spring:
  ai:
    chat:
      observations:
        log-prompt: false      # GA renamed include-prompt → log-prompt. true only in local dev (PII)
        log-completion: false  # GA renamed include-completion → log-completion
management:
  metrics:
    tags:
      application: order-service
  endpoints:
    web:
      exposure:
        include: health,prometheus,metrics

Auto-generated metrics (OpenTelemetry GenAI semantic conventions):

• `gen_ai.client.operation` — model call latency, tagged with provider and model
• `gen_ai.client.token.usage` — token counts (input/output/total)
• `spring.ai.chat.client` — ChatClient-level operation timer/span

Custom AI Metrics

@Component
@RequiredArgsConstructor
public class AiMetrics {

    private final MeterRegistry meterRegistry;

    private final Timer.Builder promptTimer = Timer.builder("ai.prompt.latency")
        .description("LLM prompt latency");

    private final Counter.Builder tokenCounter = Counter.builder("ai.tokens.used")
        .description("Total tokens consumed");

    public <T> T track(String operation, String model, Supplier<T> call) {
        return Timer.builder("ai.prompt.latency")
            .tag("operation", operation)
            .tag("model", model)
            .register(meterRegistry)
            .record(call); // record(Supplier) — recordCallable throws checked Exception
    }

    public void recordTokens(String operation, String model, int inputTokens, int outputTokens) {
        Counter.builder("ai.tokens.used")
            .tag("operation", operation)
            .tag("model", model)
            .tag("type", "input")
            .register(meterRegistry)
            .increment(inputTokens);

        Counter.builder("ai.tokens.used")
            .tag("operation", operation)
            .tag("model", model)
            .tag("type", "output")
            .register(meterRegistry)
            .increment(outputTokens);
    }
}

Prompt/Response Logging Advisor

GA replaced the whole advisor API: `CallAroundAdvisor` → `CallAdvisor`, `AdvisedRequest` → `ChatClientRequest`, `AdvisedResponse` → `ChatClientResponse`, and `Usage.getGenerationTokens()` → `getCompletionTokens()`. Agents reliably generate the old one — it does not compile on 1.0.

@Component
public class AiAuditAdvisor implements CallAdvisor {

    private static final Logger log = LoggerFactory.getLogger(AiAuditAdvisor.class);

    @Override
    public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
        String requestId = UUID.randomUUID().toString();
        long start = System.currentTimeMillis();

        log.info("[AI-AUDIT] requestId={} promptLength={}",
            requestId, request.prompt().getUserMessage().getText().length());

        try {
            ChatClientResponse response = chain.nextCall(request);
            long latency = System.currentTimeMillis() - start;

            ChatResponse chatResponse = response.chatResponse();
            if (chatResponse != null && chatResponse.getMetadata() != null) {
                Usage usage = chatResponse.getMetadata().getUsage();
                log.info("[AI-AUDIT] requestId={} latencyMs={} inputTokens={} outputTokens={}",
                    requestId, latency,
                    usage.getPromptTokens(), usage.getCompletionTokens()); // GA: not getGenerationTokens()
            }
            return response;
        } catch (Exception e) {
            log.error("[AI-AUDIT] requestId={} FAILED after {}ms", requestId,
                System.currentTimeMillis() - start, e);
            throw e;
        }
    }

    @Override
    public String getName() { return "AiAuditAdvisor"; }

    @Override
    public int getOrder() { return Ordered.LOWEST_PRECEDENCE; }
}

Cost attribution

• Keep provider prices in externally managed configuration with an effective date and currency.
• Key prices by the exact provider model identifier returned in usage metadata.
• Reject an unknown model instead of silently applying a default price.
• Preserve the raw token usage so historical costs can be recalculated after pricing changes.
• Prefer provider billing exports for invoices; application estimates are operational signals only.

Structured AI Audit Log (DB)

The [audit advisor template](templates/AiAuditAdvisor.java) treats telemetry as best-effort. Catch submission failures around the call to the separate async bean, as well as persistence failures inside it: a full or stopped executor can reject a task before its method runs. Use a bounded executor and monitor rejected audit tasks; do not use caller-runs when response latency must remain independent of audit storage. Store sanitized error types, not raw provider exception messages. Mandatory compliance audit records need durable delivery, not this best-effort design.

See [Spring task execution and rejection policies](https://docs.spring.io/spring-framework/reference/integration/scheduling.html).

@Entity
@Table(name = "ai_audit_log")
public class AiAuditLog {
    @Id @GeneratedValue(strategy = GenerationType.UUID)
    private UUID id;
    private String operation;
    private String model;
    private int inputTokens;
    private int outputTokens;
    private d