The sentence-streaming mode used to admit every intelligible voice. More seriously, native TYGER BARGE events and literal Stop previews could bypass speaker confirmation entirely. Lowering one global similarity threshold would have increased background interruptions. This update introduces session-local conversation ownership and acoustic admission before both conversation turns and playback cancellation.
| Behavior | Previous sentence-stream mode | Current default |
|---|---|---|
| Native interruption | Direct cancellation on a TYGER proposal | Current reply ID, fresh speech, active-person ECAPA match, bot-echo/rival margins and available native-hop student features. Nonlexical interruption requires stronger agreement. |
| A different background voice | Could cancel playback or be assigned the next turn | Ignored while the conversation is focused; no new profile is created from a rejected turn. |
| Changing intonation | New identity below 0.38 cosine; adaptation needed 0.70 | Owner continuity at 0.32, with rival margin. Confirmed moderate variants can be retained from 0.48; the original anchor stays fixed. |
| Another person wants to join | Every voice could join at any time | Say Hey Robbie while he is listening, release the focus, disable the focus, or wait for 3 minutes of inactivity. |
| Background speech continues after the user stops | Global speech activity could keep the pause open | Rolling 1.28-s ECAPA checks can hold the owner's pause clock when a clearly different voice continues. Final ASR excludes the rejected tail where possible. |
Stay with the current conversation partner is on by default near the main controls. Its adjustable partner cosine starts at 0.32, independently of the 0.38 new-voice label threshold. Lower values tolerate more variation but can merge similar voices. The separate 0.55 floor for a nonlexical TYGER interruption avoids treating moderate acoustic similarity as permission to cancel. The label threshold and fast-cue legacy settings are separate controls.
The first clear voice can begin without an introduction. An explicit self-correction such as “Ich bin auch Christoph” links another acoustic mode to the named person. Up to eight variants per person are retained; weak matches never train the reference. Names, embeddings and audio buffers are RAM-only and cleared when the session ends. Embeddings are approximate identity cues, not authentication. A manual Stop button always works independently of voice matching.
The frozen Nemotron encoder and existing V4 student already supply 320-ms speaker features. The update reuses those features and compares them to prototypes from accepted turns. Longer 3.04-s ECAPA anchors improve the voice reference. A separate 1.28-s recent window, sampled every 640 ms on CPU, checks whether current activity belongs to the partner. This reuses already heard audio and does not wait for a future three-second recording.
Literal Stop requests use the same guard. Weak short-word ECAPA evidence can be supported by temporally consistent native-hop features, with rival/echo separation retained. Very strong recent student evidence can trigger a second, longer ECAPA view. Merely keeping one's mouth open, making noise or matching the previous speaker's long context is not sufficient. Checks are single-flight and revalidate the reply ID before cancelling, so a late result cannot stop a newer answer.
Offline implementation contracts: 398; passed=True. Public HTTPS CPU-browser test: True. Public synthetic voice diagnostics: {'background_rejected': 10, 'background_total': 10, 'emotion_accepted': 6, 'emotion_total': 12, 'legacy_038_emotion_accepted': 5, 'short_command_correct': 5, 'short_command_total': 6}.
Warm CPU ECAPA extraction: median 56.22 ms across 12 short reference probes. This is inference compute, not the time to hear the utterance or perform ASR.
| Real short-command guard | Expected owner | Playback cancelled | Teacher cosine | Fast cosine | Guard wall time |
|---|---|---|---|---|---|
| christoph-stop | True | True | 0.345 | 0.3943 | 40.52 ms |
| christoph-wait | True | True | 0.2823 | 0.3239 | 47.49 ms |
| christoph-halt | True | True | 0.4768 | 0.8851 | 44.92 ms |
| christoph-pause-clear | True | True | 0.4041 | 0.6153 | 43.57 ms |
| robert-stop | False | True | 0.4052 | 0.6949 | 43.96 ms |
| background-stop | False | False | 0.35 | 0.4659 | 40.69 ms |
These are deliberately small, correlated synthetic probes, not a false-acceptance-rate estimate for people. In the difficult unknown-voice test, the synthetic Robert Stop can resemble the Christoph anchor in both ECAPA and the feature student. This remains an observed false acceptance; it is not hidden by the functional test result. Registering Robert as a distinct known speaker supplies the rival margin and rejects that probe. Similar or overlapping voices cannot be reliably separated by a threshold alone.
The emotion-profile test is also mixed: the Mira anchor retained all six selected variants, whereas the fixed Idris source did not match six synthesized profile variants at 0.32. A nominal dataset identity does not guarantee that synthesis preserved its acoustic voice. Use actual session speech as the anchor, speak a clear sentence when joining, and explicitly correct a changed voice when necessary. No general speaker accuracy or physical-room guarantee is claimed.
Rolling ownership can stop another voice from extending a pause after a clearly recognized speaker switch. It cannot cleanly unmix two people speaking simultaneously. Strong overlap, a very short unfamiliar Stop, microphone/device artifacts or a failed embedding can remain uncertain. The system then keeps the conversation rather than inventing a new identity; use the manual Stop when needed. No extra GPU model or heavy library was added.
Controls, reload persistence, chat/metrics clearing, desktop adjacency and mobile overflow are checked in the same script. Only public authored synthetic audio is submitted; no private microphone recording or enrolled voiceprint is published.
conversation_focus.py is the pure ownership policy. sentence_session.py owns async evidence, pause filtering, bounded variants and guarded cancellation. server.py routes native proposals and settings through it. Both director prompts preserve the current speaker/thread without repetitive name questions. The existing camera, avatar, TTS and TYGER network remain available.
python run_unit_checks.py python prepare_focus_fixtures.py python validate_conversation_focus.py uv run --no-project --with playwright --python /path/to/venv/bin/python python browser_conversation_focus.py
The audio diagnostics expect the documented existing public-profile asset cache. Source export contains code and result receipts, not the fixture audio or session embeddings. The browser test needs the configured running ASR/TTS/local-LLM services and one free ASR session.
{
"diagnostics": {
"passed": true,
"physical_microphone_used": false,
"source": "Existing public synthetic reference profiles; no private voiceprints",
"cases": [
{
"voice": "Mira",
"condition": "AGEV | extremely_low | de",
"audio_seconds": 3.04,
"legacy_038_accepts": true,
"accepted": true,
"speaker_id": "speaker-1",
"similarity": 0.3921,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Active conversation partner"
},
{
"voice": "Mira",
"condition": "AGEV | moderately_high | de",
"audio_seconds": 3.04,
"legacy_038_accepts": true,
"accepted": true,
"speaker_id": "speaker-1",
"similarity": 0.5016,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Active conversation partner"
},
{
"voice": "Mira",
"condition": "AGEV | moderately_low | de",
"audio_seconds": 3.04,
"legacy_038_accepts": true,
"accepted": true,
"speaker_id": "speaker-1",
"similarity": 0.4334,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Active conversation partner"
},
{
"voice": "Mira",
"condition": "AGEV | very_high | de",
"audio_seconds": 3.04,
"legacy_038_accepts": true,
"accepted": true,
"speaker_id": "speaker-1",
"similarity": 0.5912,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Active conversation partner"
},
{
"voice": "Mira",
"condition": "AROU | extremely_low | de",
"audio_seconds": 3.04,
"legacy_038_accepts": true,
"accepted": true,
"speaker_id": "speaker-1",
"similarity": 0.572,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Active conversation partner"
},
{
"voice": "Mira",
"condition": "AROU | moderately_high | de",
"audio_seconds": 3.04,
"legacy_038_accepts": false,
"accepted": true,
"speaker_id": "speaker-1",
"similarity": 0.3379,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Active conversation partner"
},
{
"voice": "Mira",
"condition": "background Idris",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.095,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Mira",
"condition": "background Selma",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.2173,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Mira",
"condition": "background Anton",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.0234,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Mira",
"condition": "background Robin",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.2567,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Mira",
"condition": "background Osku",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.1607,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "AGEV | extremely_low | de",
"audio_seconds": 3.04,
"legacy_038_accepts": false,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.0162,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "AGEV | moderately_high | de",
"audio_seconds": 3.04,
"legacy_038_accepts": false,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.2734,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "AGEV | moderately_low | de",
"audio_seconds": 3.04,
"legacy_038_accepts": false,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.2308,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "AGEV | very_high | de",
"audio_seconds": 3.04,
"legacy_038_accepts": false,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.291,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "AROU | extremely_low | de",
"audio_seconds": 3.04,
"legacy_038_accepts": false,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.2149,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "AROU | moderately_high | de",
"audio_seconds": 3.04,
"legacy_038_accepts": false,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.315,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "background Selma",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": -0.0245,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "background Mira",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.095,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "background Anton",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.2611,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "background Robin",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.0307,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"voice": "Idris",
"condition": "background Osku",
"expected_background": true,
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.2452,
"rival_similarity": -1.0,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
}
],
"guard_cases": [
{
"case": "owner",
"cancelled": true,
"expected": true,
"wall_ms": 46.32
},
{
"case": "background",
"cancelled": false,
"expected": false,
"wall_ms": 40.9
},
{
"case": "silence",
"cancelled": false,
"expected": false,
"wall_ms": 0.65
}
],
"limitations": "Small proxy test, not a room/overlap or human evaluation. Emotion synthesis may itself change identity.",
"teacher_cpu_ms": {
"median": 56.22,
"n": 12
},
"short_command_probes": [
{
"case": "christoph-stop",
"expected_owner": true,
"cancelled": true,
"wall_ms": 40.52,
"teacher_similarity": 0.345,
"fast_similarity": 0.3943,
"fast_consistency": 1.0,
"fast_frames": 2,
"reason": "Active conversation partner"
},
{
"case": "christoph-wait",
"expected_owner": true,
"cancelled": true,
"wall_ms": 47.49,
"teacher_similarity": 0.2823,
"fast_similarity": 0.3239,
"fast_consistency": 0.6666666666666666,
"fast_frames": 3,
"reason": "Explicit Stop with temporally consistent voice features"
},
{
"case": "christoph-halt",
"expected_owner": true,
"cancelled": true,
"wall_ms": 44.92,
"teacher_similarity": 0.4768,
"fast_similarity": 0.8851,
"fast_consistency": 1.0,
"fast_frames": 6,
"reason": "Active conversation partner"
},
{
"case": "christoph-pause-clear",
"expected_owner": true,
"cancelled": true,
"wall_ms": 43.57,
"teacher_similarity": 0.4041,
"fast_similarity": 0.6153,
"fast_consistency": 1.0,
"fast_frames": 2,
"reason": "Active conversation partner"
},
{
"case": "robert-stop",
"expected_owner": false,
"cancelled": true,
"wall_ms": 43.96,
"teacher_similarity": 0.4052,
"fast_similarity": 0.6949,
"fast_consistency": 1.0,
"fast_frames": 4,
"reason": "Active conversation partner"
},
{
"case": "background-stop",
"expected_owner": false,
"cancelled": false,
"wall_ms": 40.69,
"teacher_similarity": 0.35,
"fast_similarity": 0.4659,
"fast_consistency": 0.5,
"fast_frames": 4,
"reason": "Unstable recent speaker identity; background cannot cancel the reply"
}
],
"summary": {
"background_rejected": 10,
"background_total": 10,
"emotion_accepted": 6,
"emotion_total": 12,
"legacy_038_emotion_accepted": 5,
"short_command_correct": 5,
"short_command_total": 6
},
"short_commands_all_correct": false,
"known_rival_check": [
{
"case": "robert-stop",
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.4052,
"rival_similarity": 0.8945,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
},
{
"case": "background-stop",
"accepted": false,
"speaker_id": "speaker-1",
"similarity": 0.35,
"rival_similarity": 0.8762,
"bot_similarity": -1.0,
"reason": "Background, echo or uncertain voice; keeping the current conversation"
}
],
"finished": 1793011602.768617
},
"browser": {
"passed": true,
"physical_microphone_used": false,
"started": 1793011765.7405212,
"cases": [
{
"case": "wake-free-compatible introduction",
"speaker_id": "speaker-1",
"passed": true
},
{
"case": "different voice while listening does not take over",
"passed": true
},
{
"case": "original speaker returns with a different sentence",
"passed": true
},
{
"case": "background voice during a real long TTS reply",
"passed": true
},
{
"case": "current speaker spoken Stop cancels real streamed TTS",
"passed": true
},
{
"case": "manual Stop still cancels immediately",
"passed": true
}
],
"controls_work": true,
"desktop_layout_preserved": true,
"clear_resets_context_and_metrics": true,
"focus_preferences_persist": true,
"mobile_no_overflow": true,
"errors": [],
"finished": 1793011813.5003715
},
"unit": {
"finished": 1793011755.8150213,
"modules": [
"test_audio_input",
"test_asr_priority",
"test_gpu_guard",
"test_enrollment_cues",
"test_recovery_rules",
"test_stop_commands",
"test_live_decisions",
"test_laya_policy",
"test_conversation_prompt",
"test_conversation_director",
"test_shared_turn",
"test_speaker_names",
"test_speaker_session",
"test_speaker_turns",
"test_identity",
"test_speaker_teacher",
"test_bridge",
"test_performance",
"speaker_v2.test_predictor",
"speaker_v4.test_v4",
"speaker_v5.test_v5",
"speaker_live.test_playback",
"test_open_speakers",
"test_knowledge_tools",
"test_audio_transport",
"test_stop_feedback",
"test_stop_voice",
"test_release_scan",
"test_avatar_actions",
"test_avatar_commands",
"test_avatar_personality",
"test_bolt",
"test_pause_endpoint",
"test_asr_choice",
"test_llm_backends",
"test_visual_identity",
"test_conversation_focus"
],
"tests": 398,
"failures": 0,
"errors": 0,
"skipped": 0,
"passed": true,
"network_or_physical_microphone_used": false
}
}Generated 2026-10-26 10:52 UTC.