/vision-framework

name: vision-framework
description: "Implement computer vision features including text recognition (OCR), face detection, barcode scanning, image segmentation, object tracking, and document scanning in iOS apps. Covers both the modern Swift-native Vision API (iOS 18+) and legacy VNRequest patterns, VisionKit DataScannerViewController for live camera scanning, and CoreMLRequest/VNCoreMLRequest for custom model inference. Use when adding OCR, barcode scanning, face detection, or custom Core ML model inference with Vision."

Vision Framework

Detect text, faces, barcodes, objects, and body poses in images and video using on-device computer vision. Prefer the modern iOS 18+ request APIs and load the legacy reference only when the deployment target requires it.

See [references/vision-requests.md](references/vision-requests.md) for complete code patterns and [references/visionkit-scanner.md](references/visionkit-scanner.md) for DataScannerViewController integration.

Contents

• [Two API Generations](#two-api-generations)
• [Request Pattern (Modern API)](#request-pattern-modern-api)
• [Text Recognition (OCR)](#text-recognition-ocr)
• [Face Detection](#face-detection)
• [Barcode Detection](#barcode-detection)
• [Document Scanning (iOS 26+)](#document-scanning-ios-26)
• [Image Segmentation](#image-segmentation)
• [Object Tracking](#object-tracking)
• [Other Request Types](#other-request-types)
• [Core ML Integration](#core-ml-integration)
• [VisionKit: DataScannerViewController](#visionkit-datascannerviewcontroller)
• [Common Mistakes](#common-mistakes)
• [Review Checklist](#review-checklist)
• [References](#references)

Two API Generations

Vision has two distinct API layers. Prefer the modern API for new code: Swift-native request types plus `try await request.perform(on:)`. Keep `VN*`, `VNImageRequestHandler`, `VNSequenceRequestHandler`, completion handlers, and legacy `CGRect` helpers inside explicit legacy fallback sections or files.

| Aspect | Modern (iOS 18+) | Legacy | |---|---|---| | Pattern | `let result = try await request.perform(on: image)` | `VNImageRequestHandler` + completion handler | | Request types | Swift types — structs and classes (`RecognizeTextRequest`, `DetectFaceRectanglesRequest`) | ObjC classes (`VNRecognizeTextRequest`, `VNDetectFaceRectanglesRequest`) | | Concurrency | Native async/await | Completion handlers or synchronous `perform` | | Observations | Typed return values | Cast `results` from `[Any]` | | Availability | iOS 18+ / macOS 15+ | iOS 11+ |

The modern API uses the `ImageProcessingRequest` protocol. Each request type has a `perform(on:orientation:)` method that accepts `CGImage`, `CIImage`, `CVPixelBuffer`, `CMSampleBuffer`, `Data`, or `URL`. Most requests are structs; stateful requests such as `GeneratePersonSegmentationRequest`, `TrackObjectRequest`, `TrackRectangleRequest`, and `DetectTrajectoriesRequest` are final classes.

Request Pattern (Modern API)

All modern Vision requests follow the same pattern: create a request, call `perform(on:)`, and handle the typed result.

import Vision

func recognizeText(in image: CGImage) async throws -> [String] {
    var request = RecognizeTextRequest()
    request.recognitionLevel = .accurate
    request.recognitionLanguages = [Locale.Language(identifier: "en-US")]

    let observations = try await request.perform(on: image)
    return observations.compactMap { observation in
        observation.topCandidates(1).first?.string
    }
}

Legacy Pattern (Pre-iOS 18)

For pre-iOS 18 targets, use the corresponding `VNRequest` with `VNImageRequestHandler` or `VNSequenceRequestHandler`. Load [references/vision-requests.md](references/vision-requests.md) for complete legacy request and handler patterns.

Text Recognition (OCR)

Modern: RecognizeTextRequest (iOS 18+)

var request = RecognizeTextRequest()
request.recognitionLevel = .accurate       // .fast for real-time
request.recognitionLanguages = [
    Locale.Language(identifier: "en-US"),
    Locale.Language(identifier: "fr-FR"),
]
request.usesLanguageCorrection = true
request.customWords = ["SwiftUI", "Xcode"] // domain-specific terms

let observations = try await request.perform(on: cgImage)
for observation in observations {
    guard let candidate = observation.topCandidates(1).first else { continue }
    let text = candidate.string
    let confidence = candidate.confidence  // 0.0 ... 1.0
    let bounds = observation.boundingBox   // NormalizedRect
}

Legacy: VNRecognizeTextRequest

The legacy request uses string language identifiers and the handler pattern in the reference; both generations support accurate and fast recognition levels.

Face Detection

Detect face rectangles, landmarks (eyes, nose, mouth), and capture quality.

// Modern API
let faceRequest = DetectFaceRectanglesRequest()
let faces = try await faceRequest.perform(on: cgImage)

for face in faces {
    let boundingBox = face.boundingBox   // NormalizedRect
    let roll = face.roll                 // Measurement<UnitAngle>
    let yaw = face.yaw                  // Measurement<UnitAngle>
}

// Landmarks (eyes, nose, mouth contours)
var landmarkRequest = DetectFaceLandmarksRequest()
let landmarkFaces = try await landmarkRequest.perform(on: cgImage)
for face in landmarkFaces {
    let landmarks = face.landmarks
    let leftEye = landmarks?.leftEye.points
    let nose = landmarks?.nose.points
}

Coordinate System

Vision uses a normalized coordinate system with origin at the bottom-left. Convert to UIKit (top-left origin) before display:

import Vision

func imageRectForDisplay(_ rect: NormalizedRect, imageSize: CGSize) -> CGRect {
    rect.toImageCoordinates(imageSize, origin: .upperLeft)
}

Barcode Detection

Detect 1D and 2D barcodes including QR codes.

var request = DetectBarcodesRequest()
let symbologies: [BarcodeSymbology] = [.qr, .ean13, .code128, .pdf417]
request.symbologies = symbologies