/realitykit

name: realitykit
description: "Build iOS augmented reality and 3D experiences with RealityKit and ARKit. Use when adding RealityView content, loading entities or USDZ models, anchoring objects to planes or world positions, distinguishing entity hit tests from ARKit real-world raycasts, handling AR camera availability, world tracking, scene updates, or RealityKit entity gestures and interactions."

RealityKit

Build AR experiences on iOS using RealityKit for rendering and ARKit for world tracking. Covers `RealityView`, entity management, raycasting, scene understanding, and gesture-based interactions. Targets Swift 6.3 / iOS 26+.

Contents

• [Setup](#setup)
• [RealityView Basics](#realityview-basics)
• [Loading and Creating Entities](#loading-and-creating-entities)
• [Anchoring and Placement](#anchoring-and-placement)
• [Raycasting](#raycasting)
• [Gestures and Interaction](#gestures-and-interaction)
• [Scene Understanding](#scene-understanding)
• [Common Mistakes](#common-mistakes)
• [Review Checklist](#review-checklist)
• [References](#references)

Setup

Project Configuration

1. Add `NSCameraUsageDescription` to Info.plist 2. On iOS, `RealityViewCameraContent` displays an AR camera view by default (iOS 18+, macOS 15+); use `.virtual` camera mode for explicit non-AR fallback 3. No entitlement is required for basic AR. If AR is core to the app, add the `arkit` required-device capability; otherwise gate AR UI with `isSupported`.

Device Requirements

Check the exact AR configuration's `isSupported` value before presenting AR UI.

import ARKit

guard ARWorldTrackingConfiguration.isSupported else {
    showUnsupportedDeviceMessage()
    return
}

Key Types

| Type | Platform | Role | |---|---|---| | `RealityView` | iOS 18+, visionOS 1+ | SwiftUI view that hosts RealityKit content | | `RealityViewCameraContent` | iOS 18+, macOS 15+ | Content displayed through an AR camera view on iOS, non-AR on macOS | | `Entity` | All | Base class for all scene objects | | `ModelEntity` | All | Entity with a visible 3D model | | `AnchorEntity` | All | Tethers entities to a real-world anchor |

RealityView Basics

`RealityView` is the SwiftUI entry point for RealityKit. `RealityViewCameraContent` is the iOS/macOS content type. On iOS, it uses an AR camera view by default and can use `content.camera = .virtual` for non-AR mode when requested or when AR/camera access is unavailable.

import ARKit
import SwiftUI
import RealityKit

struct ARExperienceView: View {
    var body: some View {
        RealityView { (content: RealityViewCameraContent) in
            if !ARWorldTrackingConfiguration.isSupported {
                content.camera = .virtual
            }

            let sphere = ModelEntity(
                mesh: .generateSphere(radius: 0.05),
                materials: [SimpleMaterial(
                    color: .blue,
                    isMetallic: true
                )]
            )
            sphere.position = [0, 0, -0.5]  // 50cm in front of camera
            content.add(sphere)
        }
    }
}

Make and Update Pattern

Use the `update` closure to respond to SwiftUI state changes:

struct PlacementView: View {
    @State private var modelColor: UIColor = .red

    var body: some View {
        RealityView { content in
            let box = ModelEntity(
                mesh: .generateBox(size: 0.1),
                materials: [SimpleMaterial(
                    color: .red,
                    isMetallic: false
                )]
            )
            box.name = "colorBox"
            box.position = [0, 0, -0.5]
            content.add(box)
        } update: { content in
            if let box = content.entities.first(
                where: { $0.name == "colorBox" }
            ) as? ModelEntity {
                box.model?.materials = [SimpleMaterial(
                    color: modelColor,
                    isMetallic: false
                )]
            }
        }

        Button("Change Color") {
            modelColor = modelColor == .red ? .green : .red
        }
    }
}

Loading and Creating Entities

Loading from USDZ Files

Load 3D models asynchronously to avoid blocking the main thread:

RealityView { content in
    if let robot = try? await ModelEntity(named: "robot") {
        robot.position = [0, -0.2, -0.8]
        robot.scale = [0.01, 0.01, 0.01]
        content.add(robot)
    }
}

Adding Components

Entities use an ECS (Entity Component System) architecture. Add components to give entities behavior:

let box = ModelEntity(
    mesh: .generateBox(size: 0.1),
    materials: [SimpleMaterial(color: .red, isMetallic: false)]
)

// Make it respond to physics
box.components.set(PhysicsBodyComponent(
    massProperties: .default,
    material: .default,
    mode: .dynamic
))

// Add collision shape for interaction
box.components.set(CollisionComponent(
    shapes: [.generateBox(size: [0.1, 0.1, 0.1])]
))

// Enable input targeting for gestures
box.components.set(InputTargetComponent())

Anchoring and Placement

AnchorEntity

Use `AnchorEntity` to anchor content to detected surfaces or world positions:

RealityView { content in
    // Anchor to a horizontal surface
    let floorAnchor = AnchorEntity(.plane(
        .horizontal,
        classification: .floor,
        minimumBounds: [0.2, 0.2]
    ))

    let model = ModelEntity(
        mesh: .generateBox(size: 0.1),
        materials: [SimpleMaterial(color: .orange, isMetallic: false)]
    )
    floorAnchor.addChild(model)
    content.add(floorAnchor)
}

Anchor Targets

| Target | Description | |---|---| | `.plane(.horizontal, ...)` | Horizontal surfaces (floors, tables) | | `.plane(.vertical, ...)` | Vertical surfaces (walls) | | `.plane(.any, ...)` | Any detected plane | | `.world(transform:)` | Fixed world-space position |

Raycasting

Keep RealityKit scene queries separate from ARKit