Component relationships
Voice loop, queues and state
A VoiceLoop object composes microphone capture, Silero VAD, faster-whisper recognition, the dialogue client, tool executor and selectable speech engines. Separate worker threads exchange utterances and speech through queues. Listening starts disabled and is explicitly enabled from the tray or configured mute shortcut.
Warm-up loads recognition and the selected voice, initializes mail access and prepares the local address index. Recognition runs locally; dialogue and voice backends determine which text requests use network services.
Interruption and speech handoff
A detected new utterance sets an interruption event and stops the active speaker. Recognition and dialogue paths observe that event, and the speech worker discards obsolete queued sentences. Playback uses an abortable output path.
The dialogue client accumulates response sentences during streaming. A completed non-tool response supplies the speech queue; tool rounds feed structured results back to the model instead. This is sentence buffering after response completion, not token-to-audio streaming.
Tools, drafts and the persisted send policy
Tool definitions provide names, argument schemas, functions and optional prechecks. The executor parses arguments, verifies required values, returns structured failures and records the call/result. Sending requires a draft registered by the compose-email tool; creating a draft itself saves an unsent Outlook item. Contact lookup and the daily calendar have distinct tools.
The prototype uses a model classifier for spoken confirmation. A positive confirmation enables persisted automatic-send mode; later sends read that policy. The set-send-policy tool can restore confirmation mode. Approval is therefore a saved mode, rather than a fresh prompt before every send.
Configuration and credentials
The pywebview settings page uses a Python bridge for configuration, device/model/voice catalogues and the key registry. A local SQLite registry holds credentials separately from settings; the GUI displays masked values.
Dialogue configuration and the active key are reread for a turn, while speech settings are reapplied before handling it. The global mute shortcut is installed at application startup, so changing that shortcut applies on the next launch. The actual GUI also retains the saved dialogue model if provider discovery fails.
Session evidence and prototype limits
Session JSONL records include user/reply text, tool arguments/results, usage, estimated token cost, errors and timing fields. The history cap trims at message boundaries while preserving valid tool-call/result sequences. These records are local artifacts, not a separate visual analytics workspace.
The field named stt_to_first_token_ms currently marks the first sentence queued after a completed answer, rather than the true first streamed token. No timing or recognition-quality benchmark is inferred from the settings captures. Wake-word and speaker-check defaults do not establish an implemented workflow.