Previously, recognized punctuation could send the first sentence immediately, even while the user continued speaking. The fallback pause timer also treated sufficient sound energy as activity, allowing background noise to hold a turn open.
The input path now waits for a configurable neural speech pause: 400 ms by default, adjustable from 200 to 2,000 ms with the prominent Speech pause before replying slider. Consecutive sentences before that pause form one LLM request. Punctuation remains a preview. Output sentence-to-TTS streaming is unchanged.
| Scenario | Pause setting | Gap between sentences | Added noise | Passed | Submission after final clip end |
|---|---|---|---|---|---|
| two_sentences | 400 ms | 0.12 s | False | True | 441 ms |
| two_sentences_with_noise | 400 ms | 0.12 s | True | True | 427 ms |
| longer_setting_holds | 1000 ms | 0.5 s | True | True | 1015 ms |
| two_separate_turns | 300 ms | 1.2 s | False | True | See event record |
| short_separate_turns | 300 ms | 0.6 s | False | True | See event record |
| noise_only | 400 ms | 0.12 s | True | True | See event record |
These checks send authored synthetic German microphone audio in real time through the actual websocket, Nemotron ASR, local LLM and TTS. Noise is continuous Gaussian noise plus a 120-Hz hum, loud enough to exceed the former energy gate. Short gaps must retain both sentences in one request; longer gaps must produce two turns. Noise alone must produce none. Each committed event must report at least the selected neural quiet duration. Transport acknowledgments do not simulate a physical loudspeaker.
HTTPS Chrome browser passed: True. It checks the visible 400-ms default, saves 950 ms, reloads, confirms server restoration, changes live to 1,000 ms, speaks two sentences with a 500-ms gap and continuous noise, receives one complete turn and real TTS audio, then changes live to 300 ms. CPU browser, authored fake microphone; no physical camera or private microphone recording.
Unit checks: 349; passed: True. Dedicated regressions cover native endpoint/punctuation buffering, grouped multi-sentence submission, short farewell, resume-during-ASR races, early speech candidates, bounded settings, immediate Stop and avoiding old words in the next turn’s pre-roll.
A 1.29-MB Silero ONNX model classifies 32-ms audio windows on one CPU inference thread, retaining recurrent state only for this session. Two positive frames establish speech; probability hysteresis prevents flicker. The same microphone packets and timestamp-aligned playback reference are reused. Inference runs off the event loop. Mean measured detector work per 32-ms frame: 0.126 ms. This is actual local CPU inference, not a vendor speed claim.
Native ASR and TYGER remain unchanged. The final transcript check uses a bounded full-window ASR pass; if speech resumes while that pass is running, its old pause cannot submit a turn. An accepted turn resets its input segment so subsequent pre-roll cannot re-submit old words. Explicit Stop while Robby speaks bypasses the pause delay. The browser persists only the selected setting; voice memory remains session RAM.
The selected pause is a minimum quiet interval according to a neural detector, not exact ground-truth word timing. Model onset/tail uncertainty, packet scheduling and ASR work add latency. The table measures time after the authored clip end; it is not physical mouth-to-speaker latency. Background speech can still be detected as speech: this detector is not source separation or a guarantee against TV voices. Strong aligned bot echo is suppressed for the pause clock, but real-room echo needs its own test. Passing these synthetic noise cases does not cover every environment.
A request already decoding in the language model cannot be edited in place. Speech before the configured pause is grouped; speech after a submitted turn becomes the next ordered request. True mutable-context input streaming would require a different generation/control protocol. The older strict speaker-gated mode retains its own endpoint policy.
Reproduce with python run_unit_checks.py, python validate_speech_pause.py, then the Playwright browser_speech_pause_check.py. Do not run audio tests concurrently: the demo has one ASR session slot. Model weights are fetched only at bootstrap/startup and hash-checked; live microphone inference does not download files.
Model revision: 066409f3d9db45a30d21dfcbf9147826ffdd7052. SHA256: 7ed98ddbad84ccac4cd0aeb3099049280713df825c610a8ed34543318f1b2c49. Upstream MIT license and interface: Silero VAD. No extra GPU model is loaded.
{
"audio": {
"model_revision": "066409f3d9db45a30d21dfcbf9147826ffdd7052",
"sha256": "7ed98ddbad84ccac4cd0aeb3099049280713df825c610a8ed34543318f1b2c49",
"physical_microphone": false,
"rows": [
{
"case": "two_sentences",
"pause_ms": 400,
"gap_s": 0.12,
"noise": false,
"passed": true,
"events": [
{
"type": "asr_settings",
"speech_pause_ms": 400,
"min_ms": 200,
"max_ms": 2000,
"vad": "Silero ONNX CPU · neural speech activity",
"submission": "All pending sentences after a speech pause",
"received_ms": 77.12
},
{
"type": "asr_sentence",
"text": "Was können wir heute nachmittags gemeinsam machen? Bitte schlage mir ein lustiges Spiel vor.",
"audio_s": 5.76,
"queued": false,
"reason": "neural speech pause",
"speech_pause_ms": 400,
"observed_quiet_ms": 464,
"received_ms": 6160.74
},
{
"type": "endpoint",
"transcript": "Was können wir heute nachmittags gemeinsam machen? Bitte schlage mir ein lustiges Spiel vor.",
"speaker_id": "speaker-1",
"speaker_name": "Speaker 1",
"by": "asr_complete_sentence",
"timing": {
"speaker_assignment_ms": 69.381,
"speaker_embedding_wait_ms": 68.935,
"speaker_lookup_ms": 0.446,
"speaker_prefetch_used": false,
"speaker_prefetch_compute_ms": null,
"sentence_queue_ms": 69.55,
"sentence_audio_s": 5.76
},
"received_ms": 6230.22
}
],
"submission_after_clip_end_ms": 440.7399999999998
},
{
"case": "two_sentences_with_noise",
"pause_ms": 400,
"gap_s": 0.12,
"noise": true,
"passed": true,
"events": [
{
"type": "asr_settings",
"speech_pause_ms": 400,
"min_ms": 200,
"max_ms": 2000,
"vad": "Silero ONNX CPU · neural speech activity",
"submission": "All pending sentences after a speech pause",
"received_ms": 57.56
},
{
"type": "asr_sentence",
"text": "Was können wir heute nachmittags gemeinsam machen? Bitte schlage mir einen lustiges Spiel vor.",
"audio_s": 5.76,
"queued": false,
"reason": "neural speech pause",
"speech_pause_ms": 400,
"observed_quiet_ms": 464,
"received_ms": 6146.68
},
{
"type": "endpoint",
"transcript": "Was können wir heute nachmittags gemeinsam machen? Bitte schlage mir einen lustiges Spiel vor.",
"speaker_id": "speaker-1",
"speaker_name": "Speaker 1",
"by": "asr_complete_sentence",
"timing": {
"speaker_assignment_ms": 61.585,
"speaker_embedding_wait_ms": 61.5,
"speaker_lookup_ms": 0.086,
"speaker_prefetch_used": false,
"speaker_prefetch_compute_ms": null,
"sentence_queue_ms": 61.71,
"sentence_audio_s": 5.76
},
"received_ms": 6208.45
}
],
"submission_after_clip_end_ms": 426.6800000000003
},
{
"case": "longer_setting_holds",
"pause_ms": 1000,
"gap_s": 0.5,
"noise": true,
"passed": true,
"events": [
{
"type": "asr_settings",
"speech_pause_ms": 1000,
"min_ms": 200,
"max_ms": 2000,
"vad": "Silero ONNX CPU · neural speech activity",
"submission": "All pending sentences after a speech pause",
"received_ms": 58.0
},
{
"type": "asr_sentence",
"text": "Was können wir heute nachmittags gemeinsam machen? Bitte schlage mir ein lustiges Spiel vor.",
"audio_s": 7.04,
"queued": false,
"reason": "neural speech pause",
"speech_pause_ms": 1000,
"observed_quiet_ms": 1040,
"received_ms": 7115.26
},
{
"type": "endpoint",
"transcript": "Was können wir heute nachmittags gemeinsam machen? Bitte schlage mir ein lustiges Spiel vor.",
"speaker_id": "speaker-1",
"speaker_name": "Speaker 1",
"by": "asr_complete_sentence",
"timing": {
"speaker_assignment_ms": 0.086,
"speaker_embedding_wait_ms": 0.002,
"speaker_lookup_ms": 0.084,
"speaker_prefetch_used": true,
"speaker_prefetch_compute_ms": 55.84,
"sentence_queue_ms": 0.23,
"sentence_audio_s": 7.04
},
"received_ms": 7115.46
}
],
"submission_after_clip_end_ms": 1015.2600000000002
},
{
"case": "two_separate_turns",
"pause_ms": 300,
"gap_s": 1.2,
"noise": false,
"passed": true,
"events": [
{
"type": "asr_settings",
"speech_pause_ms": 300,
"min_ms": 200,
"max_ms": 2000,
"vad": "Silero ONNX CPU · neural speech activity",
"submission": "All pending sentences after a speech pause",
"received_ms": 59.02
},
{
"type": "asr_sentence",
"text": "Was können wir heute nachmittags gemeinsam machen?",
"audio_s": 3.52,
"queued": false,
"reason": "neural speech pause",
"speech_pause_ms": 300,
"observed_quiet_ms": 360,
"received_ms": 3719.79
},
{
"type": "endpoint",
"transcript": "Was können wir heute nachmittags gemeinsam machen?",
"speaker_id": "speaker-1",
"speaker_name": "Speaker 1",
"by": "asr_complete_sentence",
"timing": {
"speaker_assignment_ms": 0.074,
"speaker_embedding_wait_ms": 0.002,
"speaker_lookup_ms": 0.072,
"speaker_prefetch_used": true,
"speaker_prefetch_compute_ms": 52.39,
"sentence_queue_ms": 0.18,
"sentence_audio_s": 3.52
},
"received_ms": 3719.95
},
{
"type": "barge",
"yield": 0.905,
"received_ms": 4809.9
},
{
"type": "asr_sentence",
"text": "Bitte schlage mir ein lustiges Spiel vor.",
"audio_s": 6.72,
"queued": false,
"reason": "neural speech pause",
"speech_pause_ms": 300,
"observed_quiet_ms": 348,
"received_ms": 7095.15
},
{
"type": "endpoint",
"transcript": "Bitte schlage mir ein lustiges Spiel vor.",
"speaker_id": "speaker-1",
"speaker_name": "Speaker 1",
"by": "asr_complete_sentence",
"timing": {
"speaker_assignment_ms": 0.1,
"speaker_embedding_wait_ms": 0.002,
"speaker_lookup_ms": 0.098,
"speaker_prefetch_used": true,
"speaker_prefetch_compute_ms": 51.4,
"sentence_queue_ms": 0.21,
"sentence_audio_s": 6.72
},
"received_ms": 7095.42
}
]
},
{
"case": "short_separate_turns",
"pause_ms": 300,
"gap_s": 0.6,
"noise": false,
"passed": true,
"events": [
{
"type": "asr_settings",
"speech_pause_ms": 300,
"min_ms": 200,
"max_ms": 2000,
"vad": "Silero ONNX CPU · neural speech activity",
"submission": "All pending sentences after a speech pause",
"received_ms": 60.31
},
{
"type": "asr_sentence",
"text": "Was können wir heute nachmittags gemeinsam machen?",
"audio_s": 3.52,
"queued": false,
"reason": "neural speech pause",
"speech_pause_ms": 300,
"observed_quiet_ms": 360,
"received_ms": 3719.19
},
{
"type": "endpoint",
"transcript": "Was können wir heute nachmittags gemeinsam machen?",
"speaker_id": "speaker-1",
"speaker_name": "Speaker 1",
"by": "asr_complete_sentence",
"timing": {
"speaker_assignment_ms": 0.084,
"speaker_embedding_wait_ms": 0.002,
"speaker_lookup_ms": 0.082,
"speaker_prefetch_used": true,
"speaker_prefetch_compute_ms": 51.3,
"sentence_queue_ms": 0.2,
"sentence_audio_s": 3.52
},
"received_ms": 3719.38
},
{
"type": "barge",
"yield": 0.946,
"received_ms": 4808.86
},
{
"type": "asr_sentence",
"text": "Bitte schlage mir ein lustiges Spiel vor.",
"audio_s": 6.4,
"queued": false,
"reason": "neural speech pause",
"speech_pause_ms": 300,
"observed_quiet_ms": 344,
"received_ms": 6506.81
},
{
"type": "endpoint",
"transcript": "Bitte schlage mir ein lustiges Spiel vor.",
"speaker_id": "speaker-1",
"speaker_name": "Speaker 1",
"by": "asr_complete_sentence",
"timing": {
"speaker_assignment_ms": 0.064,
"speaker_embedding_wait_ms": 0.001,
"speaker_lookup_ms": 0.062,
"speaker_prefetch_used": true,
"speaker_prefetch_compute_ms": 55.17,
"sentence_queue_ms": 0.13,
"sentence_audio_s": 6.4
},
"received_ms": 6506.96
}
]
},
{
"case": "noise_only",
"pause_ms": 400,
"gap_s": 0.12,
"noise": true,
"passed": true,
"events": [
{
"type": "asr_settings",
"speech_pause_ms": 400,
"min_ms": 200,
"max_ms": 2000,
"vad": "Silero ONNX CPU · neural speech activity",
"submission": "All pending sentences after a speech pause",
"received_ms": 55.49
}
]
}
],
"vad_ms_per_32ms_frame": 0.1264776844177803,
"vad_total_wall_ms": 17.66989380121231,
"passed": true,
"finished": 1792950209.731149
},
"browser": {
"passed": true,
"physical_microphone": false,
"url": "https://arrive-locations-parade-research.trycloudflare.com",
"started": 1792950232.3863406,
"default_and_persistence": true,
"live_settings_ack": true,
"events": [
{
"type": "asr_settings",
"speech_pause_ms": 950,
"min_ms": 200,
"max_ms": 2000,
"vad": "Silero ONNX CPU · neural speech activity",
"submission": "All pending sentences after a speech pause",
"at": 644.7999992370605
},
{
"type": "asr_settings",
"speech_pause_ms": 1000,
"min_ms": 200,
"max_ms": 2000,
"vad": "Silero ONNX CPU · neural speech activity",
"submission": "All pending sentences after a speech pause",
"at": 707.9000015258789
},
{
"type": "asr_settings",
"speech_pause_ms": 1000,
"min_ms": 200,
"max_ms": 2000,
"vad": "Silero ONNX CPU · neural speech activity",
"submission": "All pending sentences after a speech pause",
"at": 711.2000007629395
},
{
"type": "asr_sentence",
"text": "Was können wir heute nachmittags gemeinsam machen? Bitte schlage mir einen lustiges Spiel vor.",
"audio_s": 9.6,
"queued": false,
"reason": "neural speech pause",
"speech_pause_ms": 1000,
"observed_quiet_ms": 1112,
"at": 12658.100002288818
},
{
"type": "endpoint",
"transcript": "Was können wir heute nachmittags gemeinsam machen? Bitte schlage mir einen lustiges Spiel vor.",
"speaker_id": "speaker-1",
"speaker_name": "Speaker 1",
"by": "asr_complete_sentence",
"timing": {
"speaker_assignment_ms": 0.056,
"speaker_embedding_wait_ms": 0.001,
"speaker_lookup_ms": 0.055,
"speaker_prefetch_used": true,
"speaker_prefetch_compute_ms": 59.24,
"sentence_queue_ms": 0.12,
"sentence_audio_s": 9.6
},
"at": 12658.79999923706
}
],
"errors": [],
"audio_chunks": 1,
"live_change_to_300": true,
"finished": 1792950248.267443
},
"unit": {
"finished": 1792950146.6347888,
"modules": [
"test_audio_input",
"test_asr_priority",
"test_gpu_guard",
"test_enrollment_cues",
"test_recovery_rules",
"test_stop_commands",
"test_live_decisions",
"test_laya_policy",
"test_conversation_prompt",
"test_conversation_director",
"test_shared_turn",
"test_speaker_names",
"test_speaker_session",
"test_speaker_turns",
"test_identity",
"test_speaker_teacher",
"test_bridge",
"test_performance",
"speaker_v2.test_predictor",
"speaker_v4.test_v4",
"speaker_v5.test_v5",
"speaker_live.test_playback",
"test_open_speakers",
"test_knowledge_tools",
"test_audio_transport",
"test_stop_feedback",
"test_stop_voice",
"test_release_scan",
"test_avatar_actions",
"test_avatar_personality",
"test_bolt",
"test_pause_endpoint",
"test_visual_identity"
],
"tests": 349,
"failures": 0,
"errors": 0,
"skipped": 0,
"passed": true,
"network_or_physical_microphone_used": false
}
}Generated 2026-10-25 17:45 UTC.