phy-video-bgm_skill

Overview

This skill analyzes a video's mood and automatically generates matching background music, then mixes the BGM into the video. It uses Gemini for video analysis and fal.ai Lyria2 to produce short, production-ready music stems, with optional speed and volume adjustments. The pipeline returns a final video and a separate BGM file for review.

How this skill works

The skill uploads the video to Gemini for a focused 'music supervisor' analysis that outputs a concise MUSIC_PROMPT. That prompt (with constraints appended) is sent to Lyria2 to generate a ~32s WAV BGM. FFmpeg is used to optionally change playback speed, strip original audio, trim and fade the generated BGM, boost its level, and mux it into the final video. All intermediate files are saved alongside the original.

When to use it

• You need a quick, AI-driven soundtrack that matches a video's emotional arc.
• You want to remove noisy or unwanted original audio and replace it with polished BGM.
• You need to preview both the mixed video and the raw BGM separately.
• You want to test different music styles or speed variants for the same footage.
• You require automated music generation for promo clips, social reels, or product ads.

Best practices

• Provide a clear, high-quality source video; Gemini needs visual context to recommend style.
• Use the default volume multiplier (5.0) because Lyria2 outputs are quiet by design.
• Prefer short, concrete style overrides if you skip Gemini (instruments, BPM, mood).
• Always strip original audio first to avoid masking or competing tracks.
• Apply modest speed changes (±10–20%) to avoid unnatural motion when changing tempo.

Example use cases

• Add a tasteful, brand-aligned BGM to a 30–60s product promo before publishing.
• Create multiple music variants (lo-fi, cinematic, upbeat) for split testing ads.
• Speed up footage for a social reel and automatically re-time and blend suitable BGM.
• Replace noisy location audio in a lifestyle shoot with a clean, mood-matched bed.