/avfoundation-media-pipeline-workflow

name: avfoundation-media-pipeline-workflow
description: Guide AVFoundation media-pipeline implementation and repair, including AVCaptureSession, capture queues, AVPlayer, AVAsset async loading, AVAssetReader, AVAssetWriter, export, transcode, sample-buffer append back-pressure, and capture or playback ownership boundaries. Use when fixing or modernizing Apple media capture, playback, loading, reader, writer, or export code.

AVFoundation Media Pipeline Workflow

Purpose

Guide AVFoundation capture, playback, asset loading, reader, writer, export, and sample-buffer pipeline design and repair. This skill owns media-pipeline shape, typed AVFoundation surface choice, and back-pressure policy, while leaving app audio-session policy, engine graphs, Core Media timing internals, and Xcode execution to their owning skills.

When To Use

• Use this skill for `AVCaptureSession`, capture inputs and outputs, capture queues, `AVPlayer`, `AVPlayerItem`, `AVAsset`, `AVURLAsset`, `AVAsyncProperty`, async asset loading, `AVAssetReader`, `AVAssetWriter`, export, transcode, and sample-buffer append loops.
• Use this skill when existing code blocks the main thread, uses deprecated AVAsset synchronous properties, misuses `loadValuesAsynchronously(forKeys:)`, appends sample buffers without back-pressure, confuses capture authorization with microphone permission, or mixes capture, UI, writer, and player ownership in one object.
• Recommend `avfaudio-session-workflow` when the primary problem is app audio intent, routes, interruptions, or microphone permission.
• Recommend `coremedia-timing-samplebuffer-workflow` when the primary issue is sample-buffer timestamps, format descriptions, timebases, clocks, or synchronization.
• Recommend `camera-capture-depth-workflow` when the primary issue is camera discovery, device controls, formats, rotation, photo capture, depth, calibration, synchronized camera outputs, or computational capture.

Single-Path Workflow

1. Classify the media pipeline:

• capture setup
• player or playback state
• asset inspection or async loading
• reader or writer loop
• export or transcode
• sample-buffer append or back-pressure
• repair of an existing implementation

2. Apply the Apple docs gate:

• read current AVFoundation documentation for the pipeline surface
• state the documented behavior relied on before recommending changes
• apply `../../shared/references/apple-media-type-ownership.md` before introducing custom media wrappers, raw numeric timing, strings, or dictionaries

3. Choose the pipeline owner and queues:

• capture-session configuration owner
• serial capture session queue
• player item and UI update boundary
• async asset loading boundary
• reader and writer queue
• cancellation and teardown path

4. Repair common failure modes:

• `AVCaptureSession.startRunning()` on the main queue
• deprecated synchronous AVAsset property reads in Swift clients
• callback-based asset loading left in otherwise async Swift code
• writer loops that ignore `isReadyForMoreMediaData`
• raw strings, dictionaries, or custom structs replacing `AVAsset`, `AVAssetTrack`, `AVPlayerItem`, `AVCaptureConnection`, `AVAssetReaderOutput`, `AVAssetWriterInput`, `CMSampleBuffer`, or writer receiver types without a documented boundary reason
• capture authorization, recording permission, and `Info.plist` gates collapsed together
• sample-buffer timing issues that need Core Media handoff

5. Return one recommendation with:

• pipeline class
• documented Apple behavior relied on
• owner, queue, and async-loading plan
• back-pressure and cancellation plan
• repair findings
• validation and handoff expectation

Inputs

• `request`: optional free-text media task.
• `pipeline_goal`: optional goal such as `capture`, `playback`, `asset-loading`, `reader`, `writer`, `export`, `transcode`, `sample-buffer-append`, or `repair`.
• `platform_context`: optional platform emphasis such as `ios`, `macos`, or `mixed-apple`.
• Defaults:
• docs-first guidance always applies
• prefer Apple and Swift media types unless `../../shared/references/apple-media-type-ownership.md` identifies a concrete escape hatch
• prefer Swift concurrency asset loading for modern Swift clients
• keep blocking session and media work off the main queue

Outputs

• `status`
• `success`: a media pipeline recommendation or repair path is ready
• `handoff`: another Apple Dev skill owns the next step
• `blocked`: pipeline ownership or runtime evidence is too unclear
• `path_type`
• `primary`: current AVFoundation docs cover the recommendation
• `fallback`: a lower-level timing or audio repair workflow is needed
• `output`
• resolved pipeline class
• documented Apple behavior relied on
• owner, queue, loading, and back-pressure plan
• repair findings
• validation and handoff expectation

Guards and Stop Conditions

• Do not run capture sessions, media loading, export, or writer loops on the main queue when docs identify the operation as blocking or asynchronous.
• Do not replace AVFoundation or Core Media pipeline types with custom wrappers, strings, dictionaries, or raw numeric timing unless the conversion boundary and lost media information are explicit.
• Do not claim capture, playback, export, camera, microphone, or route behavior is verified without runtime evidence.
• Do not silently absorb Core Media timestamp, sample-buffer readiness, or timebase repair.
• Do not collapse UI state, capture configuration, writer loops, and player ownership into one object unless the code is deliberately tiny and no persistent boundary is needed.
• Stop with `blocked` when the requested behavior depends on unavailable camera, microphone, file, route, or device evidence.

Fallbacks and Handoffs

• Recommend `avfaudio-session-workflow` for app audio intent, microphone permission, interruptions, routes, or capture-session audio-session in