arkit-visionos-developer_skill

Overview

This skill helps you build and debug ARKit features for visionOS, focusing on ARKitSession setup, provider selection, authorization, anchor processing, and RealityKit integration. It distills patterns and checks to run reliable, long-lived AR experiences in Full Space and to bridge ARKit data into RealityKit content.

How this skill works

The skill guides creating and managing a long-lived ARKitSession and configuring data providers (world tracking, hand tracking, plane detection, scene reconstruction, etc.). It shows how to request and observe authorization, consume provider anchorUpdates streams, normalize anchor IDs into your model, and map anchors to RealityKit entities while monitoring session events and lifecycle changes.

When to use it

• Implementing immersive AR features on visionOS that require spatial tracking or hand input.
• Integrating ARKit sensor streams with RealityKit rendering and entity lifecycle.
• Debugging provider authorization, missing anchors, or unexpected provider behavior.
• Building room-scale or object-aware experiences that need plane detection or scene reconstruction.
• Accessing camera frames or environment light estimates for custom processing.

Best practices

• Keep a strong, long-lived reference to ARKitSession; call run(_) and stop explicitly on teardown.
• Request provider-required authorizations (e.g., world sensing, hand tracking) before running the session; handle denied states gracefully.
• Use providers that match feature needs and reuse them across the session rather than recreating frequently.
• Consume anchorUpdates streams and reconcile added/updated/removed anchors into a normalized model with stable IDs.
• Bridge to RealityKit by mapping anchors to RealityKit entities and use ARKitAnchorComponent only to inspect backing ARKit data.
• Run ARKit work in Full Space and avoid blocking the main actor while awaiting provider data.

Example use cases

• Set up world tracking and plane detection to place persistent virtual furniture in a full-space visionOS app.
• Combine hand tracking and image tracking providers to enable gesture-driven interactions with identified images.
• Use scene reconstruction to generate a mesh for physics interactions while syncing anchors to RealityKit entities.
• Diagnose missing anchor updates by checking provider authorization, session events, and whether the experience is in Full Space.
• Stream camera frames to a custom processor via CameraFrameProvider while keeping RealityKit rendering authoritative.