/plotting-agent

name: plotting-agent
description: Step 2 of the PaperOrchestra pipeline (arXiv:2604.05018). Execute the visualization plan from outline.json — render plots and conceptual diagrams from experimental_log.md and idea.md, optionally refine via VLM critique loop, and produce context-aware captions. Runs in parallel with the literature-review-agent. TRIGGER when the orchestrator delegates Step 2 or when the user asks to "generate the figures for my paper" or "render the plots from this experiment log".

Plotting Agent (Step 2)

Faithful implementation of the Plotting Agent from PaperOrchestra (Song et al., 2026, arXiv:2604.05018, §4 Step 2 and App. F.1 p.45).

**Cost: ~20–30 LLM calls.** The paper uses PaperBanana (Zhu et al., 2026) as the default backbone with a closed-loop VLM-critique refinement. This skill expresses that loop in host-agent terms: you (the host agent) generate matplotlib code with your own LLM, render via your Bash/Python tool, optionally critique the rendered PNG with your vision model, redraw, and finally caption.

Inputs

• `workspace/outline.json` — specifically the `plotting_plan` array
• `workspace/inputs/idea.md` and `workspace/inputs/experimental_log.md` —

the source data

• `workspace/inputs/figures/` — optional pre-existing figures (`PlotOn` mode)

Outputs

• `workspace/figures/<figure_id>.png` — one PNG per `plotting_plan` entry

(300 DPI, sized to the requested aspect ratio)

• `workspace/figures/captions.json` — `{figure_id: caption_text}` map

Workflow

Per figure (executed independently per `figure_id`)

1. **Read the figure spec** from `outline.json`:

   {
     "figure_id": "fig_main_results",
     "title": "Main Results on Dataset X",
     "plot_type": "plot",
     "data_source": "experimental_log.md",
     "objective": "Visual summary (Grouped Bar Chart) demonstrating ...",
     "aspect_ratio": "5:4"
   }

2. **Few-shot retrieval (visual planning)**: pick the matching pattern from `references/chart-patterns.md` (for `plot_type=="plot"`) or `references/diagram-patterns.md` (for `plot_type=="diagram"`).

3. **Extract data**: parse `idea.md` and/or `experimental_log.md` (`data_source` field tells you which) to obtain the numeric values or conceptual entities the figure needs. For `experimental_log.md`, the `## 2. Raw Numeric Data` section contains markdown tables.

4. **Render**:

**If `PAPERBANANA_PATH` is set** — use the PaperBanana backbone (Zhu et al., 2026). It runs a Retriever → Planner → Stylist → Visualizer → Critic loop and is especially good for `plot_type == "diagram"`. See `references/paperbanana-cookbook.md` for setup (needs a Gemini API key).

   python skills/plotting-agent/scripts/paperbanana_render.py \
       --figure-id <figure_id> \
       --caption   "<objective from figure spec>" \
       --content-file workspace/inputs/idea.md \
       --task      <diagram|plot> \
       --aspect-ratio <aspect_ratio> \
       --out       workspace/figures/<figure_id>.png

**Otherwise** — write a matplotlib script and run it via your Bash tool, or use the bundled helper:

   python skills/plotting-agent/scripts/render_matplotlib.py \
       --spec spec.json \
       --out workspace/figures/<figure_id>.png

The script must apply the academic style from `chart-patterns.md`, use the correct pixel size from `aspect-ratios.md`, save at 300 DPI, and call `plt.close()` after `savefig`.

5. **VLM critique loop (optional, only if your host has vision)**:

• Reload the rendered PNG as a multimodal input to your LLM.
• Critique it against the figure's `objective` from the outline. Look for:

visual artifacts, mislabeled axes, illegible text, color clashes, misleading scaling, missing legend, overlapping labels.

• If problems are found, regenerate the matplotlib script with corrections

and re-render. Cap at 3 critique iterations per figure.

• This is the closed-loop refinement step the paper inherits from

PaperBanana. See `references/plotting-pipeline.md` for the full loop description.

• **If your host has no vision input, skip this step entirely.** The

figure will still render correctly, just without iterative refinement.

6. **Generate the caption** using the verbatim Caption Generation prompt at `references/caption-prompt.md`. Inputs to the caption prompt:

• `task_name` — the section the figure belongs to (e.g., "Methodology",

"Experiments")

• `raw_content` — the surrounding section text (or content_bullets from

the section_plan if the section isn't drafted yet)

• `description` — the `objective` field from the figure spec
• `figure_desc` — a 1-sentence description of what the rendered figure

actually shows (from your VLM critique pass, or from the script's plan if no vision)

Write the caption to `workspace/figures/captions.json` keyed by `figure_id`. **Captions must NOT contain `Figure N:` or `Caption N:` prefixes** — the LaTeX template handles numbering. Plain text only, no markdown.

Conceptual diagrams

For `plot_type == "diagram"`, prefer PaperBanana when available — its Retriever grounds the Planner in real published paper diagrams. If `PAPERBANANA_PATH` is unset, follow `references/diagram-patterns.md`. Patterns include block diagrams, system overviews, flowcharts, and algorithm-as-graph. The bundled helper:

python skills/plotting-agent/scripts/render_diagram.py \
    --spec diagram_spec.json \
    --out workspace/figures/<figure_id>.png

handles the simple cases (boxes-and-arrows). For complex Fig-1-style overview diagrams, write matplotlib patches code yourself.

Hard rules

• **300 DPI** for every figure. Lower DPI gets rejected at the LaTeX compile

step on conference templates.

• **Aspect ratio is exact**. The figure spec's `aspect_ratio` is one of 12

enumerated strings. Use the pixel targets in `references/aspect-ratios.md`