/comfyui-core

name: comfyui-core
description: Core ComfyUI knowledge covering workflow format, node types, pipeline patterns, and MCP tool usage
globs:
  - "**/*.json"

ComfyUI Core Knowledge

Workflow JSON Format (API Format)

ComfyUI workflows are JSON objects mapping string node IDs to node definitions:

{
  "1": {
    "class_type": "CheckpointLoaderSimple",
    "inputs": { "ckpt_name": "sd_xl_base_1.0.safetensors" },
    "_meta": { "title": "Load Checkpoint" }
  },
  "2": {
    "class_type": "CLIPTextEncode",
    "inputs": { "text": "a cat", "clip": ["1", 1] },
    "_meta": { "title": "Positive Prompt" }
  }
}

Key Rules

• Node IDs are strings of integers (`"1"`, `"2"`, etc.)
• `class_type` is the exact Python class name of the node
• `inputs` contains both widget values (scalars) and connections (arrays)
• Connections use the format `["sourceNodeId", outputIndex]`, a 2-element array where:
• the first element is the string node ID of the source node
• the second element is the integer index into the source node's `output` list (0-based)
• `_meta` is optional and used for display titles only

Connection Examples

"model": ["1", 0]       // Connect to node 1's first output (MODEL)
"clip": ["1", 1]        // Connect to node 1's second output (CLIP)
"vae": ["1", 2]         // Connect to node 1's third output (VAE)
"positive": ["2", 0]    // Connect to node 2's first output (CONDITIONING)
"samples": ["5", 0]     // Connect to node 5's first output (LATENT)
"images": ["6", 0]      // Connect to node 6's first output (IMAGE)

Important: API Format vs Web UI Format

• API format (for execution/analysis) is `{ "1": { class_type, inputs }, "2": { ... } }`. It is compact and used by `enqueue_workflow`, `create_workflow (action:"validate")`, `create_workflow (action:"modify")`, etc.
• Web UI format (for saving and frontend editing) is `{ "nodes": [...], "links": [...] }`. It includes layout positions, sizes, groups, and visual metadata so ComfyUI's canvas can open and edit it
• Execution tools expect and return API format
• Save in Web UI format so saved workflows stay readable and editable in the ComfyUI frontend. A raw API-format save is not canvas-editable. It "exists" in the library but loads blank in the canvas, which strands users and tempts agents into creating yet another new workflow instead of reopening the old one. Because of this, `save_workflow` auto-converts API-format input to Web UI format with a generated layout. Prefer passing real Web UI format (from `get_workflow(action="get", filename=…, format="ui")`), since a generated layout loses the original node positions and groups <!-- API-vs-UI save-format clarification adapted from 1696762169/comfyui-mcp@3da56c9 -->
• `get_workflow` defaults to `format="api"` for analysis/execution; use `format="ui"` when loading a workflow to re-save or edit in the canvas
• Muted/bypassed nodes are preserved with `_meta.mode: "muted"`. They are inactive but visible for understanding the workflow
• Get/Set virtual wire nodes are preserved with `_meta.title` and `Constant` key for tracing data flow

Workflow Library Tools

• `get_workflow(action="analyze", filename=…)` is the first call for understanding any saved workflow. It returns a structured text summary with sections, node IDs, key settings, virtual wires, and connection graph. No raw JSON, just what you need to reason about the workflow. Supports views: summary (default), overview (mermaid), detail (section mermaid), list, flat.
• `get_workflow (action:"list")` lists all saved workflows in ComfyUI's user library
• `get_workflow(action="get", filename=…)` loads raw workflow JSON. Only use it when you need the actual JSON for `enqueue_workflow`, `create_workflow (action:"modify")`, or `save_workflow`. Use `action="analyze"` instead for understanding. When the JSON is headed back to `save_workflow`, request `format="ui"` so the workflow stays editable in the frontend.
• `save_workflow(action="save", filename=…, workflow=…)` saves a workflow to the user library. Pass Web UI format (`{ nodes, links }`) so it keeps its real layout in ComfyUI's canvas. API-format graphs are accepted and auto-converted to Web UI format (with a generated layout) precisely because a raw API-format save is not canvas-editable; the frontend cannot open it. When re-saving an existing workflow, load it with `get_workflow(action="get", filename=…, format="ui")` and edit that, so positions and groups survive.

Data Types

ComfyUI nodes pass typed data through connections:

| Type | Description | Common Source | |------|-------------|---------------| | `MODEL` | Diffusion model weights | CheckpointLoaderSimple (output 0) | | `CLIP` | Text encoder | CheckpointLoaderSimple (output 1) | | `VAE` | Variational autoencoder | CheckpointLoaderSimple (output 2) | | `CONDITIONING` | Encoded text prompt | CLIPTextEncode (output 0) | | `LATENT` | Latent space tensor | EmptyLatentImage, KSampler, VAEEncode | | `IMAGE` | Pixel image tensor (BHWC) | VAEDecode, LoadImage, SaveImage | | `MASK` | Single-channel mask | LoadImage (output 1) | | `UPSCALE_MODEL` | Upscaling model | UpscaleModelLoader |

Standard Pipeline Patterns

Text-to-Image (txt2img)

CheckpointLoaderSimple → MODEL, CLIP, VAE
  ├─ CLIP → CLIPTextEncode (positive) → CONDITIONING
  ├─ CLIP → CLIPTextEncode (negative) → CONDITIONING
  │
EmptyLatentImage → LATENT
  │
KSampler (model, positive, negative, latent_image) → LATENT
  │
VAEDecode (samples, vae) → IMAGE
  │
SaveImage (images)

Node IDs typically: 1=Checkpoint, 2=Positive, 3=Negative, 4=EmptyLatent, 5=KSampler, 6=VAEDecode, 7=SaveImage

Image-to-Image (img2img)

Same as txt2img but replace `EmptyLatentImage` with:

LoadImage → IMAGE
VAEEncode (pixels, vae) → LATENT → KSampler.latent_image

Set `KSampler.denoise` to 0.5 to 0.8 (lower = closer to input image).

Upscale

LoadImage → IMAGE
UpscaleModelLoader → UPSCALE_MODEL
ImageUpscaleWithModel