13 configurations + real-audio ceiling · n=32 per cell · judged by
gemini-3-flash on the same 0/1/2 prompt used to filter the training data ·
456 clips, 0 unparsed
Headline: the task is saturated, and the LoRA is not clearly needed.
The base model with no LoRA scores 1.750/2. Real human Mediathek sports
commentary scores 1.775/2 — statistically the same thing. So the ceiling for this
metric is ≈1.8 and the base model is already at it. There is very little headroom for an
adapter to occupy.
The best configurations reach 1.969, i.e. slightly above real recordings. Two cells
clear p<0.05 raw, but with 12 comparisons against base the Bonferroni threshold is
p<0.0042 and none survive it. Treat the ranking below as suggestive, not
established.
configuration
mean judge
sd
rated 2
vs base
p (perm)
after correction
n
r64_e1
1.969
0.174
97 %
+0.219
0.027
p<0.05 raw
32
r64_e8
1.969
0.174
97 %
+0.219
0.027
p<0.05 raw
32
r32_e1
1.938
0.242
94 %
+0.188
0.081
—
32
r32_e8
1.938
0.348
97 %
+0.188
0.120
—
32
r32_e2
1.906
0.384
94 %
+0.156
0.232
—
32
r16_e2
1.875
0.331
88 %
+0.125
0.337
—
32
r16_e8
1.875
0.331
88 %
+0.125
0.334
—
32
r32_e3
1.875
0.331
88 %
+0.125
0.336
—
32
r16_e1
1.844
0.441
88 %
+0.094
0.579
—
32
r16_e3
1.844
0.363
84 %
+0.094
0.535
—
32
r64_e2
1.844
0.507
91 %
+0.094
0.601
—
32
r64_e3
1.812
0.527
88 %
+0.062
0.801
—
32
REAL Mediathek
1.775
0.524
82 %
40
base (no LoRA)
1.750
0.433
75 %
32
Validation loss did not predict this — for the third time on this stack.
The training log said epoch 1 was best and epoch 8 badly overfit (r64: 4.4237 → 5.4369, a 0.905
regression). The listener disagrees: r64_e1 and r64_e8 score
identically at 1.969. Whatever epoch 8 lost in validation loss, it did not lose in how the
audio actually sounds. Rank on generation-side metrics, not on loss.
Why a 0–2 scale was the wrong instrument here. Every cell lands between
1.75 and 1.97, and 75–97 % of clips get the top score. The scale is compressed against its
ceiling, so it cannot separate configurations even where real differences exist. A follow-up
should use pairwise A/B preference against base rather than an absolute rating — that stays
sensitive when everything is already "clearly sports commentary".
What is solid from this line of work is the data filter, not the LoRA.
The literal WER × quality formula put 602 of 1,000 candidates at exactly score
0 (60 % had WER 0.00) and therefore selected the half with worse transcription. Replacing
it with a Gemini listening filter kept 820 of 1,000 at the top rating, and a blind judge
scored the corrected selection higher on 3 of 5 dimensions (commentator-likeness +0.43 p=0.024,
euphoria +0.63 p=0.003, broadcast-likeness +0.53 p=0.009). That result is properly powered and
stands.
Listen — 4 clips per configuration
Ordered by mean judge score. The blue block is the no-LoRA base; the green block is
real human commentary. These are the first 4 clips of each cell, not the best —
picking the best would flatter every row equally and tell you nothing.
r64_e1 mean judge 1.969 · 97 % rated 2 · n=32
judge 2/2 · 11.9 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 11.5 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 7.3 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 10.2 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r64_e8 mean judge 1.969 · 97 % rated 2 · n=32
judge 2/2 · 9.4 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 9.4 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 9.0 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 10.2 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r32_e1 mean judge 1.938 · 94 % rated 2 · n=32
judge 2/2 · 12.2 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 7.2 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 8.6 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 9.4 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r32_e8 mean judge 1.938 · 97 % rated 2 · n=32
judge 2/2 · 7.7 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 12.2 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 7.9 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 11.7 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r32_e2 mean judge 1.906 · 94 % rated 2 · n=32
judge 2/2 · 12.1 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 11.8 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 6.9 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 11.9 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r16_e2 mean judge 1.875 · 88 % rated 2 · n=32
judge 2/2 · 10.5 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 13.0 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 7.3 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 8.6 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r16_e8 mean judge 1.875 · 88 % rated 2 · n=32
judge 2/2 · 11.9 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 8.8 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 6.7 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 8.1 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r32_e3 mean judge 1.875 · 88 % rated 2 · n=32
judge 2/2 · 7.5 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 12.0 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 9.8 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 11.7 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r16_e1 mean judge 1.844 · 88 % rated 2 · n=32
judge 2/2 · 12.0 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 12.2 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 7.3 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 11.0 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r16_e3 mean judge 1.844 · 84 % rated 2 · n=32
judge 2/2 · 12.2 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 8.0 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 8.5 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 8.3 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r64_e2 mean judge 1.844 · 91 % rated 2 · n=32
judge 2/2 · 9.4 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 9.5 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 9.0 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 10.2 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
r64_e3 mean judge 1.812 · 88 % rated 2 · n=32
judge 2/2 · 9.5 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 2/2 · 9.4 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 16.0 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 1/2 · 10.2 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!
REAL Mediathek commentary (ceiling control) mean judge 1.775 · 82 % rated 2 · n=40
judge 2/2 · 11.4 s · athletics
Hat sie noch mal einen Konter? Nein, hat sie nicht! Es gibt zwei Medaillen! Es gibt zwei Medaillen für die deutschen Diskuswerferinnen: Claudine Vieter und Christin Pudens. Aber welche Farbe
judge 1/2 · 16.0 s · athletics
Ja, und gerade man hofft und hofft und hofft, aber man weiß auch ganz genau, naja, auch so eine 216 in Köln kann ganz schön wehtun, ja, auch wenn man sagt, ja, das war ganz locker und ich ha
judge 2/2 · 12.7 s · athletics
Wir haben es gesagt, Final Litzkämpfer war davon befreit. Überraschend hier im Halbfinale Patricia van der Weken, die Luxemburgerin, aber hat sich in diesem Jahr wirklich deutlich gesteigert
judge 0/2 · 11.9 s · badminton
Ja, den Aufschlag von Matthias Christiansen können die Holländer aktuell ganz gut kontrollieren. Sie schaffen es da immer wieder selbst zu Punkten, indem sie eben die Aufschlagssituation für
base — no LoRA (control) mean judge 1.750 · 75 % rated 2 · n=32
judge 2/2 · 11.9 s · football
He's through, he's one on one — and he's buried it! In the last minute of the final! This entire stadium has lost its mind!
judge 1/2 · 8.9 s · athletics
She's flying down the home straight! Look at the clock, look at the clock — world record! I cannot believe what I have just watched!
judge 2/2 · 11.8 s · boxing
He's caught him flush and he's down! The champion is on the canvas in the final round! It's all over! It is all over!
judge 2/2 · 9.4 s · tennis
That's championship point saved, and another one! She has broken back from two sets down! This crowd is on its feet and it will not sit down!