MOSS-8B LGT best-of-N — scaling & Sidon study

Across all 300 base cells, the best-of-100 take beats the reference it was cloned from in 100% of cases (uplift +0.808 reward). The cloned reference speaks different words than the eval script, so its invWER is low by construction — best-of-N reliably recovers an on-script, high-quality take.

By language

German scores higher on reward (stronger blend/genuineness) while English scores higher on invWER — the ASR transcribes English more cleanly, but the VoiceCLAP heads rate the German takes as more genuine.

Sidon speech restoration (w2v-BERT 2.0 + DAC, 16→48 kHz) was applied to a deterministic 50% of groups before scoring — 12,923 clips restored and scored paired against their raw versions. The verdict: restoration does not help best-of-N selection.

On the best-of-N pick per cell, Sidon reward is -0.019 vs raw (Sidon wins in only 44.7% of 132 cells). Restoration slightly lowers VoiceCLAP blend similarity while marginally raising genuineness — the two nearly cancel, leaving reward flat. For this pipeline, Sidon is a cost with no scoring benefit; its value would be in output audio quality, not selection.

Measured on the real run, per stage. ASR dominates end-to-end cost — the VoiceCLAP scoring heads are effectively free next to Parakeet decoding.

Expected best-of-k reward as a function of how many of the 1000 takes you sample (bootstrap over random subsets). Reward keeps climbing all the way to 1000 — with clear diminishing returns, but no plateau before the ceiling.

Going 100 → 1000 takes lifts best-of-k reward by +0.138 (+17.5%). The marginal gain shrinks from +0.049 (50→100) to +0.010 (900→1000), but 95% of the 100→1000 gain is not reached until k = 1000 — so within this budget, more samples still pay off. The knee is around k ≈ 300–500, where you capture most of the gain at a quarter of the compute.

Three DramaBox emotional prompts — fear/terror, bliss, vulnerable love — 3 sentences each, EN + DE, cloned onto each named voice. Best-of-100 per (voice × emotion × language).

By reference voice

By emotion prompt

Samantha clones best (reward 1.05, invWER 0.95); goblin is hardest (0.71). Across voices the tender-love prompt scores highest and bliss lowest — the model lands intimate delivery more reliably than sustained elation.