realitykit-visionos-developer_skill

Overview

This skill helps developers build, debug, and optimize RealityKit scenes for visionOS. It focuses on entity/component selection, system patterns, async asset loading, and integration with SwiftUI via RealityView. Use it to resolve ECS behavior, rendering, physics, audio, animation, and input issues on visionOS.

How this skill works

The skill inspects RealityKit scene design and recommends the right components and systems for each requirement. It guides component registration, main-actor transform management, async asset loading patterns, and custom System implementation for per-frame behavior. It also highlights common pitfalls and performance optimizations specific to visionOS and RealityView integration.

When to use it

• Implementing or refactoring RealityKit entity/component setups for rendering, interaction, physics, or audio
• Debugging ECS behavior such as missing component state, query mismatches, or registration errors
• Optimizing scene performance with LOD, AdaptiveResolutionComponent, or batching (MeshInstancesComponent)
• Creating custom per-frame behavior using System, EntityQuery, and SystemDependency
• Bridging SwiftUI and RealityKit using RealityView and ViewAttachmentComponent
• Troubleshooting rendering issues like z-fighting, shadow artifacts, or incorrect image-based lighting

Best practices

• Prefer composition: attach Components to Entities instead of subclassing; register custom components once with Component.registerComponent()
• Keep transforms and component updates on the main actor to avoid race conditions and undefined behavior
• Load assets asynchronously (Entity(contentsOf:)/Entity(named:)) and handle errors; avoid blocking the main actor
• Use EntityQuery + QueryPredicate inside custom System.update(context:) for consistent per-frame processing
• Use AdaptiveResolutionComponent, MeshInstancesComponent, and efficient material choices to reduce GPU cost
• Use ViewAttachmentComponent for SwiftUI overlays in 3D; avoid relying on RealityView attachments closure for persistent UI

Example use cases

• Create an interactive product viewer: ModelComponent + ManipulationComponent + SpatialAudioComponent for tactile and audio feedback
• Optimize a dense forest scene: MeshInstancesComponent + AdaptiveResolutionComponent to render many trees efficiently
• Implement NPC behavior: custom System querying AnimationLibraryComponent and PhysicsBodyComponent to drive AI movement
• Build a mixed UI: embed SwiftUI controls with ViewAttachmentComponent on entities for in-world panels
• Debug rendering: add ModelDebugOptionsComponent and ModelSortGroupComponent to diagnose z-fighting and bounding issues