Automatic listening and eye contact

Open Robby · Wake mode report · Implementation and reproduction

What changes

On a new or migrated browser, Robby starts its microphone and camera automatically when server readiness and browser permissions allow. It waits for “Hey Robby”, “Hi Robbie”, “Hallo Robbi” and equivalent forms. Camera landmarks drive position-based gaze following in a browser CPU worker. Webcam Face ID and face-based speech filtering remain off.

The current speaker can follow up for ten seconds after Robby finishes actual playback, even after a long answer. Uncertain/different voices still need the wake phrase. Turn Robby off stops microphone, camera and playback; the off choice persists. Microphone, camera and autoplay permission decisions remain under browser control.

Executed checks

CheckResultScope
Old browser settings migratedPASSMicrophone and camera start without clicking Start Talking; fake device.
Spoken “Hi Robbie”PASSASR: ['Hi Robbie, wie geht es dir?']; 15 PCM chunks; Main reply: nemotron-bolt · local
Ten-second follow-upPASSActual playback-end event renews the window; unit test simulates a 30-second reply.
Camera + gazePASSCPU camera worker 8 frames; synthetic eye positions moved gaze both ways.
Turn off and persistPASSMicrophone, camera and playback stop; next reload respects OFF.
Camera permission deniedPASSMicrophone continues without the camera.
Microphone permission deniedPASSManual-enable guidance is displayed.
Native autoplay policyPASSManual gesture required: True; microphone active: True.
Offline regression584 passedIncludes 20 greeting/name combinations and a 30-second playback clock.

Runtime fingerprint for the browser run: 526ba38680793ecd8161f7cf2889e1c54c0bb72f552881bc15550a774c5d70ef; autoplay policy test: 526ba38680793ecd8161f7cf2889e1c54c0bb72f552881bc15550a774c5d70ef. Each receipt carries its own fingerprint. The local Nemotron model was explicitly selected for the test; the user’s configured LLM default was not changed.

Limits and reproduction

Microphone and camera were browser fake devices. Gaze-direction logic used synthetic eye landmarks; it is not a human eye-contact accuracy measurement. Recognition and speaker verification are approximate, especially for short words or overlap. Some browsers require an explicit microphone/camera grant and an activation click before sound may play; Robby shows the enable control when blocked.

python run_unit_checks.py
python generate_auto_wake_fixture.py
python browser_auto_wake.py --url https://YOUR-HOST
python browser_auto_wake_policy.py --url https://YOUR-HOST

Run browser checks sequentially: the live ASR service has one session slot. The first script makes real ASR/LLM/TTS requests using authored synthetic audio, not a physical microphone.

Full browser receipt · Autoplay-policy receipt