audiocraft-audio-generation_skill

Overview

This skill provides a concise guide to using AudioCraft for neural audio generation, covering MusicGen (text-to-music), AudioGen (text-to-sound), and EnCodec (neural audio codec). It explains model choices, core workflows, and common generation parameters so you can quickly produce music, sound effects, stereo audio, and melody-conditioned outputs. The content focuses on practical examples, installation, and performance tips for real projects.

How this skill works

AudioCraft encodes text prompts with a text encoder (T5) and autoregressively generates discrete audio tokens with a transformer decoder. An EnCodec decoder converts tokens back to high-fidelity waveforms, supporting stereo and configurable durations. Models come in multiple sizes and specializations (melody, style, stereo) and expose generation parameters such as duration, top_k, temperature, and classifier-free guidance (cfg_coef).

When to use it

• Generate music from text descriptions (ambient, pop, orchestral, etc.)
• Create sound effects and environmental audio for games or media
• Produce melody-conditioned music using an input audio motif
• Generate stereo outputs or perform style-conditioned transfers from reference audio
• Quick prototyping with small models, higher quality with medium/large models

Best practices

• Choose model size based on budget and latency: small for quick tests, medium/large for production quality
• Set duration and sampling rate correctly (MusicGen 32 kHz, AudioGen 16 kHz) and resample inputs when needed
• Use batch generation for efficiency when creating multiple clips in one run
• Tune top_k, temperature, and cfg_coef for the balance between creativity and prompt adherence
• Clear GPU cache and consider half precision or shorter durations to reduce memory usage

Example use cases

• Text-to-music demo app: generate short tracks from user prompts with MusicGen-small for low latency
• Sound design pipeline: batch-create SFX (thunder, footsteps, doors) with AudioGen and save 16 kHz WAVs
• Melody-conditioned composition: feed a melody WAV to musicgen-melody to produce arranged tracks
• Style transfer: use musicgen-style with a reference clip to generate tracks in a matching style
• Audio continuation: provide an intro audio file to continue or expand musical ideas programmatically