/audiocraft

description: PyTorch library for audio generation including text-to-music (MusicGen)
  and text-to-sound (AudioGen). Use when you need to generate music from text descriptions,
  create sound effects, or perform melody-conditioned music generation.
version: 1.0.0
author: Orchestra Research
license: MIT
dependencies:
- audiocraft
- torch>=2.0.0
- transformers>=4.30.0
metadata:
  catalog:
    tags:
    - Multimodal
    - Audio Generation
    - Text-to-Music
    - Text-to-Audio
    - MusicGen

Kheish Compatibility

This skill is repo-local and stays inactive until explicitly activated.

When the original instructions refer to legacy tool names, use these Kheish mappings:

• `terminal` => `bash`
• `web_extract` => `web_fetch`, plus `web_search` when discovery is needed
• `search_files` => `grep_search` and `glob_search`
• `browser_*` tools require a browser-capable surfaced tool or MCP; if none is available, use the closest available surface and say so explicitly

When the instructions mention local helper files, resolve them from `${KHEISH_SKILL_DIR}`.

AudioCraft: Audio Generation

Comprehensive guide to using Meta's AudioCraft for text-to-music and text-to-audio generation with MusicGen, AudioGen, and EnCodec.

When to use AudioCraft

**Use AudioCraft when:**

• Need to generate music from text descriptions
• Creating sound effects and environmental audio
• Building music generation applications
• Need melody-conditioned music generation
• Want stereo audio output
• Require controllable music generation with style transfer

**Key features:**

• **MusicGen**: Text-to-music generation with melody conditioning
• **AudioGen**: Text-to-sound effects generation
• **EnCodec**: High-fidelity neural audio codec
• **Multiple model sizes**: Small (300M) to Large (3.3B)
• **Stereo support**: Full stereo audio generation
• **Style conditioning**: MusicGen-Style for reference-based generation

**Use alternatives instead:**

• **Stable Audio**: For longer commercial music generation
• **Bark**: For text-to-speech with music/sound effects
• **Riffusion**: For spectogram-based music generation
• **OpenAI Jukebox**: For raw audio generation with lyrics

Quick start

Installation

# From PyPI
pip install audiocraft

# From GitHub (latest)
pip install git+https://github.com/facebookresearch/audiocraft.git

# Or use HuggingFace Transformers
pip install transformers torch torchaudio

Basic text-to-music (AudioCraft)

import torchaudio
from audiocraft.models import MusicGen

# Load model
model = MusicGen.get_pretrained('facebook/musicgen-small')

# Set generation parameters
model.set_generation_params(
    duration=8,  # seconds
    top_k=250,
    temperature=1.0
)

# Generate from text
descriptions = ["happy upbeat electronic dance music with synths"]
wav = model.generate(descriptions)

# Save audio
torchaudio.save("output.wav", wav[0].cpu(), sample_rate=32000)

Using HuggingFace Transformers

from transformers import AutoProcessor, MusicgenForConditionalGeneration
import scipy

# Load model and processor
processor = AutoProcessor.from_pretrained("facebook/musicgen-small")
model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-small")
model.to("cuda")

# Generate music
inputs = processor(
    text=["80s pop track with bassy drums and synth"],
    padding=True,
    return_tensors="pt"
).to("cuda")

audio_values = model.generate(
    **inputs,
    do_sample=True,
    guidance_scale=3,
    max_new_tokens=256
)

# Save
sampling_rate = model.config.audio_encoder.sampling_rate
scipy.io.wavfile.write("output.wav", rate=sampling_rate, data=audio_values[0, 0].cpu().numpy())

Text-to-sound with AudioGen

from audiocraft.models import AudioGen

# Load AudioGen
model = AudioGen.get_pretrained('facebook/audiogen-medium')

model.set_generation_params(duration=5)

# Generate sound effects
descriptions = ["dog barking in a park with birds chirping"]
wav = model.generate(descriptions)

torchaudio.save("sound.wav", wav[0].cpu(), sample_rate=16000)

Core concepts

Architecture overview

AudioCraft Architecture:
┌──────────────────────────────────────────────────────────────┐
│                    Text Encoder (T5)                          │
│                         │                                     │
│                    Text Embeddings                            │
└────────────────────────┬─────────────────────────────────────┘
                         │
┌────────────────────────▼─────────────────────────────────────┐
│              Transformer Decoder (LM)                         │
│     Auto-regressively generates audio tokens                  │
│     Using efficient token interleaving patterns               │
└────────────────────────┬─────────────────────────────────────┘
                         │
┌────────────────────────▼─────────────────────────────────────┐
│                EnCodec Audio Decoder                          │
│        Converts tokens back to audio waveform                 │
└──────────────────────────────────────────────────────────────┘

Model variants

| Model | Size | Description | Use Case | |-------|------|-------------|----------| | `musicgen-small` | 300M | Text-to-music | Quick generation | | `musicgen-medium` | 1.5B | Text-to-music | Balanced | | `musicgen-large` | 3.3B | Text-to-music | Best quality | | `musicgen-melody` | 1.5B | Text + melody | Melody conditioning | | `musicgen-melody-large` | 3.3B | Text + melody | Best melody | | `musicgen-stereo-*` | Varies | Stereo output | Stereo generation | | `musicgen-style` | 1.5B | Style transfer | Reference-based | | `audiogen-medium` | 1.5B | Text-to-sound | Sound effects |

Generation parameters

| Parameter | Default | Description | |-----------|---------|-------------| | `duration` | 8.0 | Length in seconds (1-120) | | `top_k` | 250 | Top-k sampling | | `top_p` | 0.0 | Nucleus sampling (0 = disabled) | | `temperatur