/model-compatibility

name: model-compatibility
description: Model family compatibility matrix covering loaders, resolutions, samplers, CFG, VAE, ControlNet, and LoRA compatibility for SD 1.5, SDXL, Flux, SD3, and video models
globs:
  - "**/*.json"

ComfyUI Model Compatibility Matrix

Stable Diffusion 1.5 (SD 1.5)

Overview

The original widely-adopted Stable Diffusion model. Huge ecosystem of fine-tunes, LoRAs, ControlNets, and embeddings. Still the most compatible and lightweight model family.

Configuration

| Parameter | Value | |-----------|-------| | **Loader** | `CheckpointLoaderSimple` | | **Native Resolution** | 512x512 | | **Supported Resolutions** | 512x512, 512x768, 768x512, 768x768 (some fine-tunes) | | **VAE** | Built-in or external (`vae-ft-mse-840000-ema-pruned.safetensors`) | | **CLIP** | Single CLIP-L (output index 1 from checkpoint) | | **Text Encoder Node** | `CLIPTextEncode` | | **CFG Range** | 7-12 (typical: 7.5) | | **Negative Prompt** | Yes — very important for quality | | **Steps** | 20-30 (standard samplers) | | **Sampler** | All standard samplers: `euler`, `euler_ancestral`, `dpmpp_2m`, `dpmpp_sde`, `ddim` | | **Scheduler** | `normal`, `karras` | | **Denoise** | 1.0 (txt2img), 0.5-0.8 (img2img) | | **VRAM (FP16)** | ~2-3GB |

Workflow Pattern

CheckpointLoaderSimple → MODEL(0), CLIP(1), VAE(2)
  CLIP(1) → CLIPTextEncode (positive) → CONDITIONING
  CLIP(1) → CLIPTextEncode (negative) → CONDITIONING
EmptyLatentImage (width=512, height=512) → LATENT
KSampler (cfg=7.5, steps=20, sampler="euler", scheduler="normal") → LATENT
VAEDecode → IMAGE
SaveImage

VAE Notes

• Most SD 1.5 checkpoints have a built-in VAE, but it's often mediocre
• **Recommended**: Use external `vae-ft-mse-840000-ema-pruned.safetensors` for better color accuracy
• Load via `VAELoader` node and connect to `VAEDecode`
• FP16 VAE can produce NaN on some images. FP32 VAE is more stable

ControlNet Compatibility

SD 1.5 has the largest ControlNet ecosystem:

| ControlNet | Model File Pattern | Notes | |------------|-------------------|-------| | Canny | `control_v11p_sd15_canny` | Edge detection | | Depth | `control_v11f1p_sd15_depth` | Depth map | | OpenPose | `control_v11p_sd15_openpose` | Skeleton/pose | | Scribble | `control_v11p_sd15_scribble` | Hand-drawn lines | | Lineart | `control_v11p_sd15_lineart` | Clean lines | | Softedge | `control_v11p_sd15_softedge` | Soft edges (HED) | | Normal | `control_v11p_sd15_normalbae` | Normal maps | | Seg | `control_v11p_sd15_seg` | Semantic segmentation | | Tile | `control_v11f1e_sd15_tile` | Tile/upscale guidance | | Inpaint | `control_v11p_sd15_inpaint` | Inpainting guidance | | IP-Adapter | `ip-adapter_sd15` | Image prompt |

LoRA Compatibility

• SD 1.5 LoRAs ONLY work with SD 1.5 base models
• Format: `.safetensors` in `models/loras/`
• Loader: `LoraLoader` node, which connects between checkpoint and CLIPTextEncode
• Strength range: 0.5-1.0 (higher can cause artifacts)

---

SDXL (Stable Diffusion XL)

Overview

Major upgrade from SD 1.5 with dual CLIP encoders, higher native resolution, and better prompt understanding. Includes Turbo and Lightning variants for fast generation.

Configuration — SDXL 1.0 (Base)

| Parameter | Value | |-----------|-------| | **Loader** | `CheckpointLoaderSimple` | | **Native Resolution** | 1024x1024 | | **Supported Resolutions** | 1024x1024, 832x1216, 1216x832, 896x1152, 1152x896, 768x1344, 1344x768 | | **VAE** | Built-in (SDXL has good integrated VAE) | | **CLIP** | Dual CLIP: CLIP-L + CLIP-G | | **Text Encoder Node** | `CLIPTextEncode` (unified) or `CLIPTextEncodeSDXL` (separate G/L) | | **CFG Range** | 5-10 (typical: 7.0) | | **Negative Prompt** | Yes — moderately important | | **Steps** | 20-40 | | **Sampler** | `euler`, `euler_ancestral`, `dpmpp_2m`, `dpmpp_sde` | | **Scheduler** | `normal`, `karras` | | **Denoise** | 1.0 (txt2img), 0.5-0.8 (img2img) | | **VRAM (FP16)** | ~6-7GB |

Configuration — SDXL Turbo

| Parameter | Value | |-----------|-------| | **Loader** | `CheckpointLoaderSimple` | | **Resolution** | 512x512 (optimized for lower res) | | **CFG** | 1.0-2.0 | | **Steps** | 1-4 | | **Sampler** | `euler_ancestral` | | **Scheduler** | `normal` | | **Negative Prompt** | Minimal or empty | | **Denoise** | 1.0 |

Configuration — SDXL Lightning

| Parameter | Value | |-----------|-------| | **Loader** | `CheckpointLoaderSimple` + `LoraLoader` (Lightning LoRA) | | **Resolution** | 1024x1024 | | **CFG** | 1.0-2.0 | | **Steps** | 4-8 (match the Lightning variant: 2-step, 4-step, 8-step) | | **Sampler** | `euler` | | **Scheduler** | `sgm_uniform` | | **Negative Prompt** | Empty or minimal | | **Special** | Requires matching Lightning LoRA for the step count |

SDXL Refiner

The optional SDXL refiner model does a second pass to improve fine details:

CheckpointLoaderSimple (base) → KSampler (steps=25, start=0, end=20)
CheckpointLoaderSimple (refiner) → KSampler (steps=25, start=20, end=25)

• The refiner uses `KSamplerAdvanced` with `start_at_step` and `end_at_step`
• Typically run the base for 80% of steps, refiner for the last 20%
• Refiner checkpoint: `sd_xl_refiner_1.0.safetensors`

Workflow Pattern

CheckpointLoaderSimple → MODEL(0), CLIP(1), VAE(2)
  CLIP(1) → CLIPTextEncode (positive) → CONDITIONING
  CLIP(1) → CLIPTextEncode (negative) → CONDITIONING
EmptyLatentImage (width=1024, height=1024) → LATENT
KSampler (cfg=7.0, steps=25, sampler="dpmpp_2m", scheduler="karras") → LATENT
VAEDecode → IMAGE
SaveImage

ControlNet Compatibility

SDXL ControlNets are separate from SD 1.5 ControlNets:

| ControlNet | Model File Pattern | Notes | |------------|-------------------|-------| | Canny | `control-lora-canny-rank256` or `diffusers_xl_canny` | Often LoRA-based | | Depth | `control-lora-depth-rank256` or `diffusers_xl_depth` | | | T2I-Adapter | `t2i-adapter-*-sdxl` | Lighter alternative to ControlNet | | IP-Adapter | `ip-adapter_sdxl` | Image prompt