cosmos-policy_skill

Overview

This skill evaluates NVIDIA Cosmos Policy on the LIBERO and RoboCasa simulation benchmarks and documents headless GPU evaluation, profiling, and cluster launch steps. It provides reproducible commands, environment requirements, and checklists to run smoke and full-benchmark evaluations. Use it to validate inference correctness, measure latency/throughput, and scale evaluations on GPU nodes.

How this skill works

The skill wraps the public Cosmos Policy evaluation entrypoints for LIBERO and RoboCasa and provides concrete uv run commands, environment exports, and dependency sync steps. It explains how to configure EGL headless rendering, set cache paths, and run smoke or full evaluation runs while capturing logs and JSON results. It also includes troubleshooting guidance for common runtime and asset issues and recommended benchmark reference numbers.

When to use it

• Running a smoke or full LIBERO benchmark evaluation on a GPU node
• Evaluating RoboCasa tasks or scaling to multi-task runs in simulation
• Setting up headless EGL rendering on a cluster or blank machine
• Profiling model inference latency and throughput for Cosmos Policy checkpoints
• Validating reproductions against published success-rate baselines

Best practices

• Always export CUDA_VISIBLE_DEVICES, MUJOCO_EGL_DEVICE_ID, MUJOCO_GL=egl, and PYOPENGL_PLATFORM=egl together on headless nodes
• Prefer the supported Docker/container runtime if host Python shows binary or loader errors
• Keep cache directories (HF and transformers) consistent across setup and eval to avoid mismatched downloads
• Fix task name, object split, seed, and num_trials_per_task when comparing runs for reproducibility
• Run a quick smoke eval before scaling to 50-trial full benchmarks to validate configs and assets

Example use cases

• Run a LIBERO smoke eval with the provided uv run command to confirm checkpoint and config wiring
• Benchmark inference latency on an A40/A100 using the LIBERO 1-trial or RoboCasa 2-trial smoke jobs
• Deploy a cluster launch from a blank machine: clone repo, sync dependencies, export EGL and cache envs, then run evals
• Validate RoboCasa asset installation by running the robocasa setup scripts and a single-task smoke evaluation
• Collect and parse JSON logs from the local_log_dir to extract Success rate and other per-task metrics