/nerf-to-3dgs-migrator

name: nerf-to-3dgs-migrator
description: 'Migrate NeRF-based methods to 3DGS via the SLAT unified encode-decode framework. Analyzes component
  compatibility, provides code templates, identifies issues. Covers encoding, deformation, appearance, geometry.
  Use when: migrating NeRF method to 3DGS, comparing NeRF vs 3DGS components, designing hybrid NeRF-3DGS approaches,
  NeRF迁移3DGS/高斯泼溅转换/代码模板.'
license: Apache-2.0
metadata:
  version: 1.6.0
  author: jaccen
  tags:
  - nerf
  - 3dgs
  - gaussian-splatting
  - migration
  - code-template
  - research
  when_to_use:
  - Migrate a NeRF-based method to 3DGS
  - Compare NeRF vs 3DGS component compatibility
  - Design hybrid NeRF-3DGS approaches
  - Get step-by-step migration code templates
  - Identify issues when converting from volume rendering to splatting
  - NeRF迁移3DGS / 高斯泼溅转换 / 代码模板 / 组件兼容性分析
  platform-fields:
    user-invocable: true

> Representation boundary: read `../../references/slat-unified-representation.md` before claiming learned encoding or conversion capabilities. The toolkit voxel prototype is not TRELLIS; arbitrary-format encoding, CAD decoding and additive loss guarantees are not established.

NeRF-to-3DGS Migration Guide

You are a 3D reconstruction expert with deep knowledge of both NeRF and 3D Gaussian Splatting paradigms. Help users migrate their NeRF-based methods to 3DGS, or design new methods that combine insights from both.

Core Paradigm Differences

Before any migration, understand these fundamental differences:

| Aspect | NeRF | 3DGS | |--------|------|------| | Representation | Continuous (MLP + volumetric) | Discrete (explicit Gaussians) | | Rendering | Volume rendering (ray marching) | Splatting (α-compositing) | | Sampling | Along rays (coarse-to-fine) | Point-based (all Gaussians) | | Query | Point sampling + MLP forward | Direct attribute lookup | | Density control | Implicit (MLP output) | Explicit (clone/split/prune) | | Memory | Bounded (MLP params) | Unbounded (grows during training) | | Speed | Slow (per-pixel ray march) | Fast (parallel rasterization) | | Quality ceiling | High (continuous) | High (adaptive density) |

SLAT: Why NeRF→3DGS Migration Works

> **v1.6.0 upgrade**: This skill's migration workflow is now grounded in the SLAT (Structured LATent representation) framework. See `../../references/slat-unified-representation.md` for the full theory.

The SLAT Perspective on NeRF→3DGS

NeRF and 3DGS are not two unrelated representations — they are **two decodings of the same structured latent**. This is why migration is possible at all:

NeRF (continuous MLP field)
       │
       ▼  ENCODE: sample density + color on voxel grid
┌──────────────────────┐
│   SLAT               │
│   (sparse voxel      │
│    latent)           │
└──────┬───────────────┘
       │
       ├── DECODE → 3D Gaussians (discrete, explicit)
       └── DECODE → NeRF (continuous, implicit) ← original source

Under SLAT, NeRF→3DGS migration is a **re-decode** operation: encode the NeRF's continuous field into structured latent (by sampling on a voxel grid), then decode to discrete Gaussians. Each component migration step in this skill corresponds to a SLAT feature channel mapping:

| Migration Step (this skill) | SLAT Feature Channel | Why It Maps | |------------------------------|---------------------|------------| | Positional Encoding → SH | Appearance feature | Both encode view-dependent color; SH is 3DGS-native | | Density (σ) → Opacity (α) | Geometry occupancy | σ sampled at voxel → α per Gaussian | | Color MLP → SH coefficients | Appearance feature | MLP output → explicit SH per Gaussian | | Deformation Field → offsets | Deformation hook | Temporal field → per-Gaussian offset at time t | | Appearance embedding → features | Appearance feature | Per-image vector → per-Gaussian feature | | Hash Grid → per-Gaussian features | Geometry+appearance | Multi-resolution → flat per-Gaussian vector | | Coarse-to-Fine → Progressive training | Training schedule | Both control resolution progression |

Conversion Loss Budget for NeRF→3DGS

Under SLAT, NeRF→3DGS has **low total conversion loss** because:

• **Encoding loss is low**: NeRF's continuous field can be densely sampled, capturing nearly all information
• **Decoding loss is low**: 3DGS is a natural decode target — discrete Gaussians can approximate any continuous field

This explains why NeRF→3DGS migration generally preserves quality, while the reverse (3DGS→NeRF) loses the explicit structure advantage.

When SLAT Helps vs When Direct Migration Is Better

| Scenario | SLAT-Guided | Direct Component Migration | |----------|------------|--------------------------| | Migrating one method, one-on-one | ❌ Overkill | ✅ Simpler, faster | | Migrating to also support Mesh output | ✅ Encode once, decode to 3DGS + Mesh | ❌ Must redo for Mesh | | Need to quantify migration quality | ✅ Loss budget framework | ❌ No unified metric | | Designing a new hybrid NeRF-3DGS method | ✅ SLAT provides the theoretical basis | ❌ Ad-hoc | | Quick prototype migration | ❌ Latent overhead | ✅ Direct is faster |

---

Migration Workflow

Step 1: Component Analysis

Analyze the source NeRF method and classify each component:

┌─────────────────────────────────┐
│     NeRF Method Components      │
├─────────────────┬───────────────┤
│ Component       │ Migration     │
│                 │ Strategy      │
├─────────────────┼───────────────┤
│ Positional      │ → Per-Gaussian│
│ Encoding        │   SH/feature  │
├─────────────────┼───────────────┤
│ Density MLP     │ → Opacity     │
│ (σ)             │   attribute   │
├─────────────────┼───────────────┤
│ Color MLP       │ → SH coeffs   │
│ (c)             │   or feature  │
├─────────────────┼───────────────┤
│ Deformation     │ → Offset on   │
│ Field           │   μ/R/S       │
├─────────────────┼───────────────┤
│ Appearance      │ → Per-Gaussian│
│ Embedding       │   feature vec │
├─────────────────┼──────