qwen3-tts-profile_skill

Overview

This skill profiles and benchmarks the qwen3-tts-rs inference pipeline inside a CUDA-enabled Docker container. It runs e2e_bench with chrome tracing, flamegraph, or Nsight Systems to produce trace.json, stage timings, and JSON results for performance analysis. Use it to identify GPU→CPU syncs, per-stage hotspots, and Time-to-First-Audio (TTFA) metrics.

How this skill works

The skill launches the qwen3-tts runtime inside a Docker image with the CUDA toolchain and runs the e2e_bench binary under different profiling modes. It can emit a Chrome trace (trace.json) showing span hierarchies, produce per-stage timing reports without profiling features, stream TTFA metrics, and export JSON results. It also includes a GPU sync audit to list all to_vec1() GPU→CPU synchronization points.

When to use it

• You need actionable perf data for qwen3-tts inference on CUDA GPUs
• Investigating why generation dominates end-to-end latency
• Measuring Time-to-First-Audio for streaming playback
• Validating optimization changes (reduced GPU→CPU syncs, batching)
• Collecting baseline metrics across model variants or hardware

Best practices

• Run inside the provided CUDA Docker image to ensure the correct Rust and CUDA toolchain are available
• Start with Chrome Trace mode to inspect span hierarchies and GPU sync events
• Use per-stage timing for quick regression checks without profiling overhead
• Audit GPU→CPU syncs (to_vec1) and aim to batch or eliminate them to reduce round-trips
• Collect results.json for automated comparisons and store traces for perf visualizations

Example use cases

• Generate trace.json via chrome tracing to inspect gpu_sync events and identify CPU stalls between talker_step and code_predictor
• Run per-stage timing to collect RTF, Tok/s, and memory for short/medium/long text workloads
• Enable streaming to measure TTFA and validate interactive playback constraints
• Run the GPU sync audit script to enumerate and prioritize to_vec1() calls for batching optimizations
• Compare baseline numbers across model variants (1.7B-CustomVoice, 1.7B-Base, 1.7B-VoiceDesign) on target hardware