Voice-Acting Best-of-10 — can sampling 10 takes beat the reference voice?
What is this?
We took 20 reference voice clips (10 German, 10 English) from a large crowd-sourced expressive-speech corpus — each clip carries a distinct emotion (anger, grief, ecstasy, fear, …) and a natural-language voice caption. For every reference we wrote a fresh, emotionally-matched 3-sentence script + delivery instruction, then asked our fine-tuned MOSS-TTS 8 billion-parameter speech model to voice-clone the reference and perform that script 10 times (10 independent takes, varying the random seed). That is 199 generated clips across 20 groups.
Every take — and every original reference — is then scored by two independent VoiceCLAP reward heads: a blend quality head (0–10) and a genuineness head (0–6, “how real / human does this sound”, VoiceCLAP-commercial). Word-error-rate (WER) from Parakeet-tdt-0.6b-v3 gives invWER = max(0, 1−WER) to gate garbled takes.
Uplift analysis — best-of-10 vs the original reference
In plain words
The model is cloning each reference voice. The question the boss asked: if we generate 10 takes and keep the best one, can that best take actually sound more genuine and better-blended than the real human reference clip it was copied from? A positive Δ (uplift) means yes — the synthetic best-of-10 out-scores the original on that reward. This matters because it says the reward-guided best-of-N loop is not just imitating the reference, it is improving on it in terms of what these quality/genuineness heads reward.
Best-achievable uplift (below) compares the reference to the single best-scoring take on each axis (the ceiling of what best-of-10 can grab). We also report the combined-reward pick — the one take you would actually ship if you rank by the default combined reward — and how often it beats the reference.
subset
n
GENUINENESS (0–6)
BLEND (0–10)
ref μ
best μ
mean Δ
% beat
ref μ
best μ
mean Δ
% beat
All 20
20
1.09
1.99
+0.91
100%
2.89
5.96
+3.07
100%
English
10
1.06
1.95
+0.89
100%
2.76
5.60
+2.83
100%
German
10
1.12
2.04
+0.92
100%
3.02
6.33
+3.31
100%
If you must pick ONE take by the combined reward (the realistic ship-one scenario), how does that single pick compare to the reference?
subset
n
mean Δ genu
% beat genu
mean Δ blend
% beat blend
All 20
20
+0.76
95%
+2.39
85%
English
10
+0.79
90%
+2.54
100%
German
10
+0.74
100%
+2.23
70%
Per-group best-of-10 uplift
ref
lang
emotion
ref genu
best genu
Δgenu
ref blend
best blend
Δblend
de_00
🇩🇪 DE
astonishment surprise amazement an
0.84
1.93
+1.09
0.34
6.66
+6.31
de_01
🇩🇪 DE
yearning longing and craving with
1.00
2.01
+1.01
4.23
7.49
+3.26
de_02
🇩🇪 DE
fear dread apprehension and horror
0.56
2.32
+1.75
1.12
6.31
+5.19
de_03
🇩🇪 DE
sexual lust and carnal desire
1.09
1.80
+0.71
5.14
7.00
+1.86
de_04
🇩🇪 DE
hope enthusiasm optimism anticipat
1.80
2.22
+0.43
2.31
7.19
+4.88
de_05
🇩🇪 DE
interest fascination curiosity and
0.84
1.85
+1.01
5.09
6.03
+0.94
de_06
🇩🇪 DE
happiness excitement joy exhilarat
1.35
1.77
+0.42
1.34
5.09
+3.75
de_07
🇩🇪 DE
confusion bewilderment disorientat
1.89
2.68
+0.79
6.99
7.74
+0.75
de_08
🇩🇪 DE
concentration focus engrossment an
0.49
1.62
+1.14
2.11
3.74
+1.62
de_09
🇩🇪 DE
teasing bantering and playful mock
1.30
2.16
+0.86
1.48
6.06
+4.58
en_00
🇺🇸 EN
physical pain suffering torment an
0.96
1.89
+0.93
1.49
4.62
+3.14
en_01
🇺🇸 EN
relief solace comfort and liberati
0.71
1.91
+1.21
2.63
7.36
+4.73
en_02
🇺🇸 EN
thankfulness gratitude appreciatio
1.69
2.06
+0.37
3.91
6.31
+2.40
en_03
🇺🇸 EN
numbness detachment insensitivity
1.04
2.09
+1.05
1.31
4.66
+3.35
en_04
🇺🇸 EN
contentment relaxation peacefulnes
0.75
2.13
+1.38
4.13
4.67
+0.54
en_05
🇺🇸 EN
astonishment surprise amazement an
0.84
1.84
+1.00
1.71
5.96
+4.24
en_06
🇺🇸 EN
sadness sorrow grief melancholy an
1.21
2.07
+0.86
3.14
7.38
+4.24
en_07
🇺🇸 EN
ecstasy pleasure bliss rapture and
1.69
1.72
+0.03
4.30
4.77
+0.47
en_08
🇺🇸 EN
anger rage fury hatred and annoyan
0.77
1.80
+1.03
3.29
4.14
+0.85
en_09
🇺🇸 EN
helplessness powerlessness despera
0.90
1.96
+1.05
1.71
6.08
+4.37
Re-sort every group (descending) by:
German references 🇩🇪
de_00🇩🇪 DEastonishment surprise amazement and shock
Original reference voice (public reference set)
ref blend 0.34 · ref genu 0.84 · invWER 0.97
Instruction: Fassungsloses Staunen, das in ungläubigen Schock umschlägt. Die Stimme stockt, wird atemlos und beinahe flüsternd vor Überwältigung. High Quality Recording.
Script (spoken / WER ref): Das kann doch nicht wahr sein, die Zahlen stimmen einfach nicht mit allem überein, was wir je geglaubt haben. Ich starre auf den Bildschirm und mein Herz rast, weil das alles verändert. Mein Gott, wir haben es tatsächlich gefunden.
de_01🇩🇪 DEyearning longing and craving with wistfulness
Original reference voice (public reference set)
ref blend 4.23 · ref genu 1.00 · invWER 1.00
Instruction: Sehnsüchtig, wehmütig und nostalgisch. Eine Frau, die leise und reflektiert über eine verlorene Vergangenheit spricht, mit einem Kloß im Hals. High Quality Recording.
Script (spoken / WER ref): Manchmal sehne ich mich so sehr nach diesen Sommern zurück, dass es körperlich wehtut. Ich würde alles geben, um noch einmal durch diese alten Gassen zu gehen und deine Hand zu halten. Aber die Zeit lässt sich nicht zurückholen, und das ist das Schwerste daran.
Reference caption: A medium-quality recording of a woman speaking in German. Her tone is nostalgic and reflective, suggesting she is thinking about a past experience. The audio quality is decent, but not pristine.
Instruction: Wachsende Angst, Grauen und beklommene Vorahnung. Die Stimme eines Mannes, zunächst gefasst, dann zittrig und gehetzt, als würde etwas Unheimliches näher kommen. High Quality Recording.
Script (spoken / WER ref): Hörst du das auch, dieses Kratzen hinter der Wand, das einfach nicht aufhören will? Ich habe das Licht ausgemacht und plötzlich war da dieser Atem, ganz nah an meinem Nacken. Bitte sag mir, dass wir hier nicht allein sind, bitte.
Reference caption: A medium-quality recording of a man speaking in a neutral tone. The audio quality is decent, and there is no background noise.
Instruction: Verlangend, sinnlich und voller unterdrückter Begierde. Eine tiefe, samtige Stimme, die langsam und heiß flüstert, mit hörbarem Verlangen. High Quality Recording.
Script (spoken / WER ref): Die Art, wie du mich ansiehst, lässt mich alles um uns herum vergessen. Komm näher, ich will spüren, wie dein Atem meinen Hals streift. Ich habe den ganzen Tag an nichts anderes gedacht als an diesen Moment.
de_04🇩🇪 DEhope enthusiasm optimism anticipation and courage
Original reference voice (public reference set)
ref blend 2.31 · ref genu 1.80 · invWER 0.91
Instruction: Hoffnungsvoll, entschlossen und voller mutigem Optimismus. Eine Frau, die kämpferisch und mitreißend spricht, mit einem nervösen Lachen zwischendurch. High Quality Recording.
Script (spoken / WER ref): Egal wie steil dieser Berg noch wird, ich werde ihn erklimmen, und nichts wird mich aufhalten. Ich spüre, wie meine Beine brennen, aber in meinem Herzen brennt etwas viel Stärkeres. Heute überschreite ich meine Grenzen und beweise mir, dass ich es schaffen kann.
Reference caption: A medium-quality recording of a woman speaking with determination and slight amusement. She sounds confident and focused. She is describing running a marathon. There is a nervous giggle present. The recording quality is decent, and there is no noticeable background noise.
de_05🇩🇪 DEinterest fascination curiosity and intrigue
Original reference voice (public reference set)
ref blend 5.09 · ref genu 0.84 · invWER 1.00
Instruction: Fasziniert, neugierig und gebannt. Eine Stimme voller wacher Neugier, die leiser und eindringlicher wird, je tiefer sie in ein Rätsel eintaucht. High Quality Recording.
Script (spoken / WER ref): Fasziniert beuge ich mich näher heran, denn dieses Symbol habe ich noch nie zuvor gesehen. Je länger ich es betrachte, desto mehr Fragen türmen sich in meinem Kopf auf. Was, wenn hinter dieser einen Tür alles verborgen liegt, wonach wir jemals gesucht haben?
de_06🇩🇪 DEhappiness excitement joy exhilaration delight jubilation and bliss
Original reference voice (public reference set)
ref blend 1.34 · ref genu 1.35 · invWER 1.00
Instruction: Überschäumende Freude, Jubel und ekstatische Begeisterung. Eine Frau, die vor Glück fast platzt, mit einem Lachen in der Stimme und sprudelnder Energie. High Quality Recording.
Script (spoken / WER ref): Wir haben es geschafft, wir haben es tatsächlich geschafft, ich kann es kaum glauben! Alles, wofür wir so hart gearbeitet haben, wird endlich wahr, und ich könnte vor Freude tanzen. Das ist der schönste Tag meines ganzen Lebens!
Reference caption: A high-quality recording of a woman speaking. She sounds excited, enthusiastic, and a bit astonished. The recording quality is excellent with no background noise.
de_07🇩🇪 DEconfusion bewilderment disorientation and perplexity
Original reference voice (public reference set)
ref blend 6.99 · ref genu 1.89 · invWER 0.90
Instruction: Verwirrt, orientierungslos und ratlos. Eine Stimme, die stockt, sich verhaspelt und nach Worten sucht, überfordert von zu vielen widersprüchlichen Gedanken. High Quality Recording.
Script (spoken / WER ref): Moment, das ergibt doch überhaupt keinen Sinn, wie kann das denn plötzlich alles anders sein? Ich, äh, ich weiß gar nicht mehr, wo oben und unten ist, alles dreht sich. Kannst du mir das bitte noch einmal erklären, denn ich verstehe gerade wirklich gar nichts?
de_08🇩🇪 DEconcentration focus engrossment and absorption
Original reference voice (public reference set)
ref blend 2.11 · ref genu 0.49 · invWER 1.00
Instruction: Hoch konzentriert, analytisch und völlig versunken. Eine Frau, die ruhig, präzise und fokussiert spricht, ganz in ihre Aufgabe vertieft. High Quality Recording.
Script (spoken / WER ref): Wenn ich die Muster übereinanderlege, dann zeichnet sich hier eine klare Struktur ab. Ich muss nur diese eine Variable isolieren, dann fügt sich das ganze Bild zusammen. Fast geschafft, nur noch dieser letzte Datensatz, und dann verstehen wir endlich, was wirklich passiert ist.
Reference caption: A high-quality recording of a female speaker. She sounds focused and analytical, describing the process of analyzing social data and finding patterns. The tone is serious and interested.
Instruction: Neckend, spöttisch und schelmisch amüsiert. Eine Frau, die sarkastisch und verschmitzt frotzelt, mit einem unterdrückten Lachen. High Quality Recording.
Script (spoken / WER ref): Ach, dein Meisterwerk aus Papier ist ja wirklich beeindruckend abgehoben, direkt vom Tisch auf den Boden. Nein, wirklich, ein wahres Wunder der Ingenieurskunst hast du da vollbracht. Vielleicht versuchst du es nächstes Mal einfach mit einem echten Flugzeug, hm?
Reference caption: A medium-quality recording of a woman speaking in German. She sounds sarcastic and amused. The audio quality is decent. There is a sound effect of laughter at the end.
en_00🇺🇸 ENphysical pain suffering torment and agony
Original reference voice (public reference set)
ref blend 1.49 · ref genu 0.96 · invWER 0.97
Instruction: Raw physical agony and torment. A man's voice cracking under unbearable pain, breath ragged, every word wrung out through clenched teeth. High Quality Recording.
Script (spoken / WER ref): It hurts, God, it hurts more than anything I have ever felt in my entire life. Every single breath feels like broken glass tearing through my chest. Please, someone, make it stop, I can't take this anymore.
Reference caption: A high-quality recording of a man speaking with intense sadness and despair. He describes feeling trapped in a nightmare and experiencing unbearable agony. The voice is filled with anguish.
Instruction: Profound relief and release, tension melting into comfort. A woman's voice, weary then softening, letting out a long shaky breath of liberation. High Quality Recording.
Script (spoken / WER ref): It's finally over, after all these endless days, the nightmare is finally over. I feel the weight lift off my shoulders and I can breathe again, really breathe. We made it through, and nothing else matters right now.
Reference caption: A medium-quality recording of a woman speaking, possibly reflecting on a past event. She sounds thoughtful and slightly weary. The recording quality is decent, with no noticeable background noise.
en_02🇺🇸 ENthankfulness gratitude appreciation and gratefulness
Original reference voice (public reference set)
ref blend 3.91 · ref genu 1.69 · invWER 0.98
Instruction: Warm, sincere gratitude straight from the heart. A woman speaking with genuine, heartfelt thankfulness, her voice soft and slightly overwhelmed with appreciation. High Quality Recording.
Script (spoken / WER ref): I honestly don't know how I would have gotten through any of this without you by my side. You believed in me when I couldn't even believe in myself, and that means the world to me. From the bottom of my heart, thank you for everything.
Reference caption: A high-quality recording of a woman expressing gratitude and appreciation for help with a marketing campaign. Her tone is sincere and heartfelt. The recording quality is excellent with no background noise. She sounds like an adult, conveying strong positive emotions.
en_03🇺🇸 ENnumbness detachment insensitivity and apathy
Original reference voice (public reference set)
ref blend 1.31 · ref genu 1.04 · invWER 1.00
Instruction: Emotionally numb, detached and flat. A hollow, affectless voice drained of feeling, speaking as if from behind a pane of glass. High Quality Recording.
Script (spoken / WER ref): They keep asking me how I feel about it, and honestly, I feel nothing at all. It's like watching my own life happen to someone else on a screen far away. I should probably care about this, but I just can't seem to find the part of me that used to.
en_04🇺🇸 ENcontentment relaxation peacefulness calmness satisfaction and serenity
Original reference voice (public reference set)
ref blend 4.13 · ref genu 0.75 · invWER 1.00
Instruction: Deep contentment, serenity and quiet peace. A woman speaking slowly and softly, utterly at ease, savoring a moment of stillness. High Quality Recording.
Script (spoken / WER ref): Standing here in the warm evening light, I feel completely and perfectly at peace. All the noise of the world just fades away, and there is only this gentle, quiet stillness. For the first time in so long, I feel like I am exactly where I am meant to be.
Reference caption: A high-quality recording of a woman speaking with a calm and reflective tone. She is describing a feeling of belonging while surrounded by ancient artifacts. There is no background noise.
en_05🇺🇸 ENastonishment surprise amazement and shock
Original reference voice (public reference set)
ref blend 1.71 · ref genu 0.84 · invWER 0.98
Instruction: Stunned disbelief tipping into amused, sarcastic astonishment. A woman recounting something unbelievable, voice rising with incredulity and a sharp little laugh. High Quality Recording.
Script (spoken / WER ref): Wait, so you're telling me you did all of that and thought no one would ever notice? I genuinely cannot believe what I am hearing right now, this is absolutely insane. Honestly, of all the ridiculous things, this one takes the cake, unbelievable.
Reference caption: A high-quality recording of a woman speaking with a tone of disbelief, amusement, and a hint of sarcasm. She is recounting something unbelievable. There is a brief chuckle and a final "unbelievable" at the end.
en_06🇺🇸 ENsadness sorrow grief melancholy and heartache
Original reference voice (public reference set)
ref blend 3.14 · ref genu 1.21 · invWER 0.88
Instruction: Quiet sorrow and aching heartache. A woman's voice trembling with grief and loss, thick with unshed tears as she remembers what is gone. High Quality Recording.
Script (spoken / WER ref): We built a whole life together, brick by brick, dream by dream. And now I walk through these empty rooms and all I can hear is the echo of everything we used to be. I miss you so much that some mornings I can barely get out of bed.
Reference caption: A medium-quality recording of a woman speaking with a slightly sad and reflective tone. She sounds like she is remembering something. The audio quality is decent, but not perfect.
en_07🇺🇸 ENecstasy pleasure bliss rapture and beatitude
Original reference voice (public reference set)
ref blend 4.30 · ref genu 1.69 · invWER 0.91
Instruction: Blissful, breathy ecstasy and rapturous delight. A woman's soft, dreamy voice, sighing with pure pleasure as she relives a perfect memory. High Quality Recording.
Script (spoken / WER ref): Oh, that moment was pure, absolute bliss, like every nerve in my body was singing. I closed my eyes and just let the wave of happiness wash completely over me. I never wanted it to end, I wanted to stay in that feeling forever.
Reference caption: A high-quality recording of a woman speaking with a soft, slightly breathy voice. She sounds nostalgic and content, describing a memory. The recording is clear and without background noise.
Instruction: Explosive rage, fury and seething hatred. A voice sharp and venomous with contempt, spitting each accusation through gritted teeth. High Quality Recording.
Script (spoken / WER ref): I cannot even stand to be in the same room as you right now, do you understand me? After everything you did, after all your lies, you have the nerve to stand there and smile at me. Get out of my sight before I say something I will truly regret.
Reference caption: This is a high-quality recording of a person expressing extreme anger, disgust, and resentment. The tone is very harsh and accusatory. There are no sound effects or background noise.
en_09🇺🇸 ENhelplessness powerlessness desperation and submission
Original reference voice (public reference set)
ref blend 1.71 · ref genu 0.90 · invWER 0.94
Instruction: Helpless desperation and powerless despair, on the verge of tears. A woman's voice thin and breaking, sinking under the weight of hopelessness. High Quality Recording.
Script (spoken / WER ref): I don't know how to keep doing this anymore, I feel like I'm disappearing a little more every single day. No matter how hard I try, nothing ever changes, and I am so tired of fighting a battle I can't win. Please, somebody, just tell me what I'm supposed to do.
Reference caption: A high-quality recording of a woman speaking with a very sad and distressed tone. She sounds like she is on the verge of tears and is expressing feelings of loss and fading away. The recording is clear with no background noise.