opik-compare
Run a candidate against the baseline over an Opik test suite and read the numbers back — which cases broke, which got fixed, the per-metric deltas, worst rows,…
Add Opik tracing to an existing app and verify a real trace lands. Installs the Opik package, detects the language and LLM framework, adds the minimum tracing, runs a safe representative path, confirms a trace in Opik, and returns the trace link. Use for "instrument my code",
$ npx -y skills add comet-ml/opik-mcp --skill opik-instrument --agent claude-codeHow it fires
How this skill gets triggered: by you, by Claude, or both.
/opik-instrumentContext preview
The summary Claude sees to decide when to auto-load this skill.
Add Opik tracing to an existing app and verify a real trace lands. Installs the Opik package, detects the language and LLM framework, adds the minimum tracing, runs a safe representative path, confirms a trace in Opik, and returns the trace link. Use for "instrument my code",
name: opik-instrument description: Add Opik tracing to an existing app and verify a real trace lands. Installs the Opik package, detects the language and LLM framework, adds the minimum tracing, runs a safe representative path, confirms a trace in Opik, and returns the trace link. Use for "instrument my code", "add opik tracing", "add observability", "trace my agent". Not for building a new app from scratch, or a review-only pass with no code changes. compatibility: Tested with Claude Code; works with any Agent Skills-compatible host (Cursor, VS Code Copilot, Codex). Requires a Python or TypeScript project. Install the `opik` skill alongside this one — it holds the shared SDK and integration references; without it, this skill falls back to the public docs. allowed-tools: - Read - Edit - Write - Grep - Glob - Bash metadata: last_updated: "2026-08-05" source_commit: "2.0.0" argument-hint: "[optional: file or directory path]"
**Definition of done:** a representative, safely-executed path produces a trace that is confirmed in Opik and a direct trace link is returned. If verification can't be completed safely or autonomously, stop at the **first** genuine blocker and return **exactly one** concrete next step. Code edits alone are not success.
Operate: **opinionated in execution, conservative in code changes, automatic in routine decisions, uncompromising about verifying value — but never by running something unsafe.**
The entry point is just `/opik-instrument` (optionally `/opik-instrument <path>`). Infer everything else; treat these only as **optional overrides** the user may pass, never as required setup:
Never turn inference into a questionnaire. Ask only when you hit a genuine, non-inferable blocker (see **Blockers**).
Python (`*.py`, `pyproject.toml`) or TypeScript (`*.ts`, `package.json`). Identify the LLM framework from imports and pick its integration:
| Import | Integration | |---|---| | `openai` / `anthropic` | `track_openai` / `track_anthropic` | | `langchain` / `langgraph` | `OpikTracer` callback | | `crewai` / `dspy` / google-genai / bedrock / `llama_index` / `litellm` | `track_crewai` / `OpikCallback` / `track_genai` / `track_bedrock` / `LlamaIndexCallbackHandler` / `OpikLogger` | | TS: `opik-openai` / `opik-vercel` / `opik-langchain` | `trackOpenAI` / `OpikExporter` / `OpikCallbackHandler` |
Full list: read `../opik/references/integrations.md` (the `opik` skill, installed beside this one). If that file isn't there, read <https://www.comet.com/docs/opik/integrations/overview> — never settle for manual spans on a framework that has a native integration just because the reference was unreachable. If the project is **already instrumented**, audit and add only what's missing — do not re-instrument.
Decision policy, in order: 1. Prefer the **framework-native integration** for provider LLM spans. 2. Add manual `@opik.track` spans only for orchestration/tools the integration doesn't cover (`type="tool"` / `"llm"` / `"guardrail"`). A bare `@opik.track` produces the default span type, **`general`** — the right choice for an entrypoint/orchestrator. 3. Never instrument the same operation twice (no `@opik.track(type="llm")` on top of `track_openai`). 4. Mark **one entrypoint per independently-runnable agent/service** — not necessarily one per repo. 5. Decorator order relative to framework decorators (e.g. `@app.route`) is **framework-dependent** — verify per framework; do not assume a universal order. 6. Scripts: flush at the end (`opik.flush_tracker()` / `await client.flush()`). LiteLLM inside `@opik.track`: pass `metadata={"opik": {"current_span_data": get_current_span_data()}}` or traces orphan.
Make the **smallest change** that lets one representative path emit a trace.
Add **only** the required Opik package(s) via the repo's detected package manager (pip / uv / poetry / npm / pnpm / yarn), through normal project conventions. **Preserve the lockfile**; do not run generic upgrades; do not install globally; treat unusual lifecycle scripts cautiously. Surface it as a change (e.g. "added `opik` to `pyproject.toml`"). If the environment blocks installation → **Blocker** with the one exact command.
Infer a safe command — prefer an existing **test, example, or dev script**, then a bounded single-request entrypoint. **Never** run anything that looks like production or does irreversible/expensive work (writes, emails, purchases, mass API calls). If no safe path is inferable → **Blocker** ("which dev command safely exercises this agent?"). Print the command, then run it.
If the run needs an **LLM provider credential** (e.g. `OPENAI_API_KEY`) and it's absent, that's a Blocker — the app can't produce a trace. Note some SDK clients raise at **construction** (module load), before any span runs, so there is **no partial trace** to wait on: return blocked with the one next step, don't wait on a flush that never happened.
The official Model Context Protocol (MCP) server for Opik, the open-source LLM observability and evaluation platform, built by Comet.
Repo: comet-ml/opik-mcp
Run a candidate against the baseline over an Opik test suite and read the numbers back — which cases broke, which got fixed, the per-metric deltas, worst rows,…
Surface the Opik traces worth a developer's attention, ranked by signal — Diagnostics issues first, then errors, failed tool calls, latency, regressions, and…
Build an LLM evaluation and run it against the app, returning an Opik experiment with scores and its link. Picks a test suite with judge assertions or a…
Root-cause a specific Opik trace, or a pattern across traces, and return a grounded explanation. Uses the hosted Opik MCP when it is connected, and falls back…
Take a judge live on production traffic — create an Opik online evaluation rule (LLM-as-judge or Python metric) on a project with sampling, filters, variable…
Improve a prompt with the Opik Agent Optimizer — resolve the prompt, a dataset, and a metric, pick the algorithm, run a bounded optimization, check the gain on…