From an acting recording
to a training example.
A timed GENERAL/SCRIPT instruction describes the performance. Twelve audio-codebook targets represent the sound. This additive release pairs both for every German/English acting recording.
What is being prepared?
This is data preparation for Humaneness Voice Small, a voice-generation model. It is separate from Humaneness Ears, the encoder used to estimate scores and locate vocal events. Preparing these examples does not report a new TTS training run or demonstrate a quality gain.
| Instruction source | Role |
|---|---|
| MOSS voice-captioner | Whole-recording GENERAL description |
| MOSS burst-captioner | Free-form descriptions of selected intervals, using the full audio and both endpoints |
| Humaneness Ears Medium + procedural rules | Event proposals, reference-relative emotion/style cues and local delivery descriptions |
| Parakeet v3 | Recognized words and alignment endpoints |
| MOSS Audio Tokenizer v2 | Discrete audio targets; a different model from the two captioners |
Existing, complete captions and acoustic predictions are reused. No new generative caption-model pass was needed. The original MP3s, annotation variants, generation prompts and exact voice IDs remain available. New local cue wording follows the frozen listening-demo format and retains all raw scores in the sidecar.
One clip, two representations
The full GENERAL/SCRIPT is the model instruction. The SCRIPT also occupies the native Text field. The frame budget is T, while the audio targets have shape [T,12]. The native processor creates 13 channels—one text/continuation channel and twelve codebooks—and masks instruction/reference/padding positions from loss.
Audio token contract
| Property | Value and meaning |
|---|---|
| Codec | MOSS Audio Tokenizer v2; pinned revision in the export lock |
| Timing | 48 kHz; one frame per 3,840 samples, or 80 ms |
| Depth | First 12 RVQ codebooks, matching Humaneness Voice Small |
| Storage | Little-endian int16 NumPy [T,12]; indices 0–1023 |
| Tail handling | Explicit zero-padding of less than one frame; exact true length saved for trimming decoded audio |
| Compute | Persistent workers, four CPU threads/GPU, batch eight, 4.8-second streaming chunks, FP32 weights/quantizer and BF16 compute |
Mono is duplicated into two identical channels for the stereo codec. No recording ends are dropped and no loudness normalization is added. Original MP3 bytes remain unchanged in the source archives; codec reconstruction is lossy. STOP and PAD are created by the training processor rather than stored among physical code indices.
Scenario-level partitions
| Split | Clips | Hours | Challenge groups |
|---|---|---|---|
| Train | 36,755 | 300.58 | 18,386 |
| Validation | 794 | 6.50 | 397 |
| Test | 878 | 7.05 | 439 |
Seeded hashing aims for 96% training, 2% validation and 2% test. The 50 earlier listening-demo scenarios are additionally held out for test. English/German counterparts stay together. These are scenario partitions; the 30 voices appear across splits. Related scenario semantics may still overlap, and the released recordings are already public. These preparation splits do not establish independent evaluation for the component annotation models.
Contents and verification
14,157,137 frames contain 169,885,644 codebook indices. The canonical captions retain 312,968 selected event intervals and 2,152,571 recognized words. The additive TAR payload is 6.33 GB; original audio is joined by ID and SHA-256 rather than duplicated.
The encoder validated shapes/ranges and audio durations. Each caption passed independent word/event/timing checks. A 128-clip pilot passed, including four codec reconstruction canaries. Native packing was checked with and without reference audio, confirming T+1 target steps and excluding reference STOP labels from supervision. One empty ASR output is explicitly marked, preserving provenance rather than replacing it with an intended script.
Full source coverage, unchanged source metadata and absence of scenario split leakage were audited before release. Mechanical validity does not imply correct emotions or event timing. The earlier 50-example audio-based automated review describes the format's perceptual limitations.
Measured preparation time
| Measurement | Result | Scope |
|---|---|---|
| Full production job | 15 min 18 s | 4 nodes × 4 GH200 GPUs; four CPU threads per GPU |
| Production allocation | 4.08 GPU-hours | Includes startup and idle allocation |
| Including both pilots | 4.38 GPU-hours | Includes the initial tail-padding failure and successful corrected pilot |
| End-to-end preparation throughput | 41.86 clips/s | All 16 GPUs together, including job startup; excludes publication |
These measurements encode audio and recompose/compress already-complete cached annotations. They do not measure fresh MOSS caption generation or TTS training. Allocated GPU-hours are resource accounting, not a monetary invoice. Read the exact run record.
Acting coverage
| Source subset | Successful clips |
|---|---|
| Eric Morris Inspired | 8,060 |
| Existing Challenge Inspired | 14,780 |
| Extreme Physical | 1,196 |
| Haiku Batch | 2,000 |
| Laions Got Talent | 556 |
| Situation Inspired | 11,435 |
| Voicenet Extension | 400 |
New annotations, documentation and code are CC BY 4.0. Component-model licenses remain their own; the codec retains Apache 2.0. See the credits and source licenses.