Key Takeaways
Start in 8 Lines
import torch, open_clip as oc, soundfile as sf
hub = "hf-hub:laion/clap2-medium-audioxl-s1B"
m, prep = oc.create_model_from_pretrained(hub)
w, sr = sf.read("dog.wav", dtype="float32")
x = prep(m.audio.cfg.audio_seq_len)((torch.from_numpy(w.T), sr))
a = m.encode_audio({k: v[None] for k, v in x.items()})
t = m.encode_text(oc.get_tokenizer(hub)(["a dog"]))
print(torch.cosine_similarity(a, t))Need Another Size? Swap One ID
hub = "hf-hub:laion/clap2-medium-audioxl-s1B"hub = "hf-hub:laion/clap2-xxs-audioxl-s1B"# sizes: xxs, xs, small, medium, largeThe LAION-Audio-XL Dataset
We collect 1.48B audio samples from YouTube, Common Crawl, web collections, internet datasets and 316 academic datasets, clean and deduplicate them, select for coverage with a 3,718-class taxonomy, and generate short and long captions for clips that lack them.
How Each Clip Is Captioned
Expert models annotate every clip, and its predicted category decides which specialists run. The captioning model hears the audio, reads the annotations as raw JSON and writes a short and a long caption. Hover or tap an expert or a caption to trace its path.
Model Architecture
A dual encoder in the CLAP family. The audio tower turns a clip of any length up to 30 s into one vector, the text tower does the same for any text, and the cosine similarity between the two vectors says how well they match: for search, zero-shot classification or as features.
Scaling Laws
We train five model sizes from scratch, each on 1M to 1,024M samples seen, and fit how the error on MAEB-30 falls with training compute. MAEB-30 is the Massive Audio Embedding Benchmark: 30 tasks across speech, music and environmental sound.
Benchmark Results
LAION-5B and OpenCLIP gave vision open web-scale data and reproducible scaling laws. LAION-Audio-XL and CLAP2 bring the same to audio, with the best MAEB-30 score of any model trained from scratch.
| Model | Params | Mean | Speech | Music | Sound |
|---|
Usage
Use CLAP2 and LAION-Audio-XL with only a few lines of code.
# pip install "open_clip_torch>=4.2.1" soundfile
import torch, open_clip
from open_clip.audio.io import load_audio
hub = "hf-hub:laion/clap2-medium-audioxl-s1B"
model, preprocess = open_clip.create_model_from_pretrained(hub)
tokenizer = open_clip.get_tokenizer(hub)
audio = preprocess()(load_audio("dog.wav")) # any sample rate and channel count
texts = ["A dog barks.", "A door slams shut.", "People are applauding."]
with torch.no_grad():
logits, _ = model.get_logits({k: v[None] for k, v in audio.items()}, tokenizer(texts))
print({t: round(p, 3) for t, p in zip(texts, logits.softmax(-1)[0].tolist())})from torch.nn.utils.rnn import pad_sequence
items = [preprocess()(load_audio(f)) for f in ["dog.wav", "applause.wav"]]
# clips differ in length: pad to the longest, the padding is masked
batch = {k: pad_sequence([x[k] for x in items], batch_first=True) for k in items[0]}
with torch.no_grad():
logits, _ = model.get_logits(batch, tokenizer(texts))
print(logits.softmax(-1)) # one row per clipfrom datasets import load_dataset
name = "laion/laion-audio-xl"
ds = load_dataset(name, streaming=True)
for s in ds["train"].take(3):
print(s["short_caption"], s["duration"])def is_music(s):
return s["tax_l1"] == "Music"
music = ds["train"].filter(is_music)
for s in music.take(3):
print(s["pred_tempo_bpm"], s["music_key"])BibTeX
@misc{wijngaard2026laionaudioxl,
title = {{LAION-Audio-XL}: A Web-Scale Corpus and Scaling Studies for Contrastive Language-Audio Pretraining},
author = {Wijngaard, Gijs and F{\"o}rster, Felix and Wightman, Ross and Nezhurina, Marianna
and Schuhmann, Christoph and Jitsev, Jenia and Koepke, A. Sophia},
year = {2026},
note = {Preprint},
}