Japanese anime refs — cross-lingual voice cloning

Japanese anime reference audio → German & English

2026-08-03 · MOSS Voice Acting v2 · reference speakers sampled from joujiboi/japanese-anime-speech-v2 · speaker similarity via speechbrain ECAPA-TDNN

This page covers 1 of 20 references. The first full generation job timed out (batch size too small — fixed) and the rerun is queued. The page regenerates from whatever exists, so it will fill in without any rework.

The Japanese voice transfers partially — and the LoRA dose is what breaks it.

Mean speaker similarity between the Japanese reference and its German/English generations is 0.416 (n=64), against two measured anchors: a reference against itself scores 1.000, and two different Japanese speakers from this corpus score 0.105. So the generations sit 35 % of the way from 'unrelated speaker' to 'identical speaker' — clearly above chance, well below a clone.

But the average hides the real story. Speaker similarity falls monotonically as the emotion LoRA is merged in harder:

emotion	LoRA	merge dose	mean speaker similarity	n
angry	Anger	0.50	0.569	16
neutral	none	1.00	0.621	16
sad	Sadness	1.00	0.500	16
amused	Amusement	1.50	-0.025	16

emotion

LoRA

merge dose

mean speaker similarity

angry

Anger

0.50

0.569

neutral

none

1.00

0.621

sad

Sadness

1.00

0.500

amused

Amusement

1.50

-0.025

Identity survives at dose 0 and is destroyed at dose 1.5. With no LoRA the similarity is 0.62 — genuinely recognisable. At dose 0.5 it is 0.57, at 1.0 it is 0.50, and at 1.5 it collapses to −0.03, i.e. below the unrelated-speaker floor: the output voice has no relationship to the reference at all.

Caveat that matters: emotion and dose are perfectly confounded here. Each emotion was generated at exactly one dose, so this run cannot separate "the Amusement adapter harms identity" from "dose 1.5 harms identity". A dose sweep holding the adapter fixed is the experiment that settles it — which is exactly the 25/50/75/100 merge-scale test now planned.

Speaker similarity by language and emotion

prompt language	mean speaker similarity	n
German	0.374	32
English	0.459	32

emotion	mean speaker similarity	n
neutral	0.621	16
angry	0.569	16
sad	0.500	16
amused	-0.025	16

English carries the reference voice better than German (0.459 vs 0.374) — a consistent gap across every emotion, though smaller than the dose effect above. The emotion column is not really an emotion effect: it is the dose ladder seen from the other side, since each emotion here ran at one fixed dose.

Per-emotion listening pages

Each page shows, per reference, the top 3 of 8 candidates for German and English side by side, with the Japanese reference player repeated for direct A/B comparison.

Are the 20 references actually different speakers? Yes.

This was a constraint during sampling, not an afterthought: candidates were clustered and selected with a maximum pairwise cosine of 0.3 over 300 clusters drawn from 3,573 candidate clips.

mean pairwise cosine between the chosen references	0.105
maximum pairwise cosine	0.289
reference vs itself (encoder sanity check)	1.000

mean pairwise cosine between the chosen references

0.105

maximum pairwise cosine

0.289

reference vs itself (encoder sanity check)

1.000

The full matrix — diagonal is self-similarity, everything off-diagonal is a different speaker:

1.00

-0.11

0.00

0.02

0.07

0.06

0.12

0.01

0.17

0.08

0.09

0.21

0.01

0.07

0.14

0.20

0.27

0.29

0.06

0.29

-0.11

1.00

0.05

0.09

0.10

0.11

0.03

0.17

0.07

0.10

0.11

0.24

0.25

0.11

0.04

0.10

0.00

0.11

0.19

0.03

0.00

0.05

1.00

0.01

0.08

-0.10

0.13

0.08

0.17

0.13

-0.03

0.17

0.12

-0.05

0.23

0.06

0.01

0.20

0.11

0.17

0.02

0.09

0.01

1.00

0.11

0.08

-0.07

0.14

0.06

0.20

0.17

0.09

0.17

0.11

0.20

0.03

0.01

0.14

0.22

0.07

0.10

0.08

0.11

1.00

-0.04

0.08

0.14

-0.08

0.18

0.07

0.17

0.04

0.25

0.07

0.14

-0.00

0.02

0.27

0.01

0.06

0.11

-0.10

0.08

-0.04

1.00

0.06

0.07

0.12

0.06

0.22

0.12

0.05

-0.03

0.05

0.07

0.12

0.03

0.12

0.03

0.13

-0.07

0.08

0.06

1.00

0.02

0.20

0.04

0.21

0.11

0.21

0.03

0.22

0.28

0.24

0.23

0.04

0.14

0.01

0.17

0.08

0.14

0.07

0.02

1.00

-0.02

0.17

0.08

0.04

0.07

-0.06

-0.03

-0.04

0.04

0.09

0.24

0.01

0.17

0.07

0.17

0.06

-0.08

0.12

0.20

-0.02

1.00

0.11

-0.01

-0.00

0.12

0.05

0.27

0.18

0.28

0.27

0.13

0.24

0.08

0.10

0.13

0.20

0.18

0.06

0.04

0.17

0.11

1.00

0.06

0.15

0.20

0.23

-0.05

-0.01

0.17

0.10

0.07

0.09

0.11

-0.03

0.17

0.07

0.22

0.21

0.08

-0.01

0.06

1.00

0.16

0.14

-0.02

0.01

0.13

0.27

0.19

0.06

0.16

0.21

0.24

0.17

0.09

0.17

0.12

0.11

0.04

-0.00

0.15

0.16

1.00

0.17

0.11

0.23

0.04

0.11

0.24

0.29

0.09

0.01

0.25

0.12

0.17

0.04

0.05

0.21

0.07

0.12

0.15

0.14

0.17

1.00

-0.03

0.09

0.04

0.09

0.21

0.07

0.09

0.07

0.11

-0.05

0.11

0.25

-0.03

0.03

-0.06

0.05

0.20

-0.02

0.11

-0.03

1.00

0.09

0.11

0.08

0.13

0.02

0.13

0.14

0.04

0.23

0.20

0.07

-0.03

0.22

-0.03

0.27

0.23

0.01

0.23

0.09

1.00

0.14

0.22

0.16

0.23

-0.05

0.20

0.10

0.06

0.03

0.14

0.05

0.28

-0.04

0.18

-0.05

0.13

0.04

0.11

0.14

1.00

0.19

-0.01

0.10

0.27

0.00

0.01

-0.00

0.07

0.24

0.04

0.28

-0.01

0.27

0.11

0.09

0.08

0.22

0.19

1.00

0.25

0.00

0.26

0.29

0.11

0.20

0.14

0.02

0.12

0.23

0.09

0.27

0.17

0.19

0.24

0.21

0.13

0.16

-0.01

0.25

1.00

0.14

0.21

0.06

0.19

0.11

0.22

0.27

0.03

0.04

0.24

0.13

0.10

0.06

0.29

0.07

0.02

0.23

-0.01

0.00

0.14

1.00

-0.00

0.29

0.03

0.17

0.07

0.01

0.12

0.14

0.01

0.24

0.07

0.16

0.09

0.13

-0.05

0.10

0.26

0.21

-0.00

1.00

The 20 reference clips

ref00 · 8.9 s · s00-0010241

善い行いが良い結果をもたらす。情けは人のためならず、というわけじゃな

ref01 · 4.5 s · s08-5fd_mna19610418c (not yet generated)

やっべー、なんだそれ！超青春してるなー！

ref02 · 3.9 s · s24-fem_hot (not yet generated)

失礼いたします。お相手様がお見えになりました

ref03 · 8.1 s · s32-NIK_C03 (not yet generated)

そう言うなよ。大崎のじーさんに呼ばれたら、逆らうことなんてできねーし

ref04 · 3.6 s · s08-5fd_mzh19610302a (not yet generated)

待て坊主、ワシにも何か言わんか

ref05 · 3.5 s · s00-0010774 (not yet generated)

ああああすりん先生！？いつの間に！？

ref06 · 7.0 s · s32-NOA1c16026 (not yet generated)

やっ、やめて。本当に俺、恥ずかしくて変になるから

ref07 · 4.0 s · s08-5fd_mzh19610222d (not yet generated)

さ、入って入って。遠慮はいらないわよ

ref08 · 3.1 s · s16-BC_21_kensuke_19a (not yet generated)

あまりの恥ずかしさに腹を切るやもしれません

ref09 · 3.1 s · s32-nka0073 (not yet generated)

我らの願いは聞けぬ、と？

ref10 · 3.3 s · s00-0010390 (not yet generated)

こ、こほん。何でもありません

ref11 · 4.2 s · s32-nonaka102_010_ref (not yet generated)

傷つきたくない。諦めなきゃいけない

ref12 · 3.7 s · s08-5fd_mna19610326b (not yet generated)

はいはい、彼氏の分はどうするんだい？

ref13 · 5.4 s · s32-nkb0041 (not yet generated)

なっ、何と苛烈な！これではまるで歯が立たぬ！

ref14 · 8.3 s · s16-BH_00_common_06a (not yet generated)

こちらが避ければ、何事かを企んでいる、と邪推されますからなぁ…それがしの方でいなしましょうか？

ref15 · 6.6 s · s00-0011358 (not yet generated)

タッチで数を増やしていき、最終的に全員がゾンビなれば我らの勝ち

ref16 · 3.4 s · s00-0010828 (not yet generated)

ふぅ。ようやく到着しました

ref17 · 5.4 s · s00-0012077 (not yet generated)

いざという時は闇ルートで。くくくっ

ref18 · 4.0 s · s32-natpapa212 (not yet generated)

あんなに可愛くて、楽しそうな笑顔は久々に見た気がするよ

ref19 · 3.3 s · s08-5fd_mzh19610304a (not yet generated)

おはよーございます。もう動けるんですね