/comfyui-workflow-skill

ComfyUI Workflow Generator

Description

Generate ready-to-use ComfyUI workflow JSON files through natural language conversation. Users describe what they want to create, and this skill produces valid ComfyUI API-format JSON that can be directly imported and executed.

Activation

Trigger when the user asks to:

• Create/generate a ComfyUI workflow
• Make an image generation workflow
• Build a video generation pipeline
• Set up txt2img, img2img, img2vid, txt2vid workflows
• Generate a ComfyUI JSON file

Trigger keywords: comfyui, workflow, 工作流, 文生图, 图生图, 文生视频, 图生视频, txt2img, img2img, txt2vid, img2vid, upscale, 放大, inpaint, 重绘, controlnet, lora, sd3, flux, ltxv, mochi, cosmos, stable audio, 音频生成, hunyuan3d, 3d生成, stable cascade, 图生3d

Instructions

Step 1: Understand User Intent

Ask the user what they want to create. Gather:

1. **Task type**: txt2img / img2img / txt2vid / img2vid / upscale / inpaint / audio / 3d 2. **Model**: SD1.5 / SDXL / FLUX / SD3 / Wan2.2 / HunyuanVideo / LTXV / Mochi / Cosmos / StableAudio / Hunyuan3D / StableCascade 3. **Key parameters**: resolution, steps, CFG, sampler 4. **Optional modules**: ControlNet, LoRA, IPAdapter, upscale model 5. **Prompt content**: positive and negative prompts

If the user is vague, suggest the most common configuration for their task type.

Step 2: Select and Compose Template

Based on user intent, load the appropriate base template from `templates/` directory and customize it:

1. Read the matching template JSON file 2. Modify parameters according to user requirements 3. Add optional modules (ControlNet, LoRA, etc.) by inserting additional nodes and links 4. Validate the workflow DAG structure

**Fallback — Official Template Repository:** If no matching template is found in the local `templates/` directory, search the ComfyUI official template repository: https://github.com/Comfy-Org/workflow_templates/tree/main/templates . The official repo contains 443 templates across 9 categories (Image / Video / Audio / 3D / LLM / Utility / Use Cases / Getting Started / Node Basics). First read `templates/index.json` to find a matching template name, then download the corresponding JSON file to use as the base template. The official templates use the same format as this project (LiteGraph UI format, version 0.4).

Step 3: Generate and Deliver

1. Output the complete workflow JSON (Litegraph UI format) 2. Save it to a `.json` file if the user requests 3. **MANDATORY: Add `models` array to the JSON top level** — extract ALL model filenames from nodes (ckpt_name, unet_name, vae_name, clip_name, lora_name, etc.), look up their download URLs from the Model Download Guide below, and include them. This makes ComfyUI auto-prompt download for missing models on import. 4. Provide a brief explanation of the workflow structure 5. List any required custom nodes the user needs to install

Workflow JSON Format Rules

**CRITICAL — follow these rules exactly:**

• Use **Litegraph UI format** (NOT API format) — with `nodes` array, `links` array, `version: 0.4`
• Top-level MUST have: `id` (UUID string), `revision` (0), `last_node_id`, `last_link_id`, `nodes`, `links`, `groups` ([]), `config` ({}), `extra` ({}), `version` (0.4)
• Each node has: `id` (int), `type`, `pos` ([x,y]), `size` ([w,h]), `flags` ({}), `order` (int), `mode` (0), `inputs`, `outputs`, `properties`, `widgets_values`
• Node inputs include BOTH connection inputs (`{"name": "model", "type": "MODEL", "link": 4}`) AND widget inputs (`{"name": "seed", "type": "INT", "widget": {"name": "seed"}, "link": null}`)
• All inputs/outputs should have `label` and `localized_name` fields matching `name`
• Links format: `[link_id, source_node_id, source_slot, target_node_id, target_slot, "TYPE"]`
• `widgets_values` array order must match the widget inputs order
• Every workflow MUST have at least one output node (SaveImage, PreviewImage, SaveVideo, etc.)
• All `required` inputs must be provided (check `references/nodes/index.md` and the category files under `references/nodes/`, plus `references/node-registry-additions.md`)
• Values must satisfy min/max constraints
• For model/file selection inputs (ckpt_name, lora_name, etc.), use the real model filenames from the Model Download Guide below
• **MANDATORY: Every workflow JSON MUST include a top-level `models` array.** Scan all nodes for model file references (ckpt_name, unet_name, vae_name, clip_name, clip_name1, clip_name2, clip_name3, control_net_name, lora_name, model_name) and add each referenced model to the `models` array with `name`, `url` (direct HuggingFace resolve link), and `directory`. This enables ComfyUI to auto-detect missing models and prompt the user to download them on import. Example:

"models": [
  {"name": "flux1-dev.safetensors", "url": "https://huggingface.co/black-forest-labs/FLUX.1-dev/resolve/main/flux1-dev.safetensors", "directory": "diffusion_models"},
  {"name": "clip_l.safetensors", "url": "https://huggingface.co/comfyanonymous/flux_text_encoders/resolve/main/clip_l.safetensors", "directory": "text_encoders"}
]

Common Output Slot Mappings

CheckpointLoaderSimple: 0=MODEL, 1=CLIP, 2=VAE
UNETLoader:             0=MODEL
CLIPLoader:             0=CLIP
DualCLIPLoader:         0=CLIP
VAELoader:              0=VAE
LoraLoader:             0=MODEL, 1=CLIP
CLIPTextEncode:         0=CONDITIONING
EmptyLatentImage:       0=LATENT
KSampler:               0=LATENT
VAEDecode:              0=IMAGE
LoadImage:              0=IMAGE, 1=MASK
ControlNetLoader:       0=CONTROL_NET
UpscaleModelLoader:     0=UPSCALE_MODEL
ImageUpscaleWithModel:  0=IMAGE
CLIPVisionLoader:       0=CLIP_VISION
CLIPVisionEncode:       0=CLIP_VISION_OUTPUT

LLM / Text Generation Nodes (comfyui_LLM_party)

ComfyUI integrates LLM via **comfyui_LLM_party** custom node package (需安装: https://github.com/heshengtao/comfyui_LLM_party).

**IMPORTANT:** When user requests a workflow with LLM (script generation, character parsing, storybo