The Life of an Utterance¶

In a nutshell

This page follows a single spoken utterance on its whole journey through OpenVoiceOS, from the instant sound reaches the microphone to the moment you hear a reply. Along the way the system notices the wake word, records what you say, turns it into text, works out what you meant, does the task, and speaks back. It's a guided tour of the assembly line that handles everything you say to it. New to the terms here? Start with the Glossary, or see the Architecture Overview for the bigger picture.

📐 Formal specification

This whole journey is specified end-to-end. Audio capture, STT, and utterance dispatch come from OVOS-AUDIO-IN-1 — Audio Input Service. The utterance lifecycle, matching, and dispatch come from OVOS-PIPELINE-1 — Utterance Lifecycle & Pipeline. The enrichment and rewrite points along the way come from OVOS-TRANSFORM-1 — Transformer Plugins. Dialog, TTS, and playback come from OVOS-AUDIO-1 — Audio Output Service. See also the spec index. Spec topic names are canonical below. The legacy name is noted once where current code still emits it.

This guide gives a technical, step-by-step walkthrough of how an utterance is processed by OpenVoiceOS, from the moment sound hits the microphone to the final spoken response.

The sequence diagram below traces the same eight stages across the services and bus events involved:

sequenceDiagram
    participant Mic as Mic
    participant Listener as Listener
    participant Bus as Bus
    participant Core as Core (IntentSvc)
    participant Skill as Skill
    participant Audio as ovos-audio
    participant Speakers as Speakers/GUI

    Mic->>Listener: raw audio stream
    Listener->>Listener: wake word + VAD<br/>detect
    Listener->>Bus: ovos.listener.<br/>record.started
    Listener->>Listener: STT transcription
    Listener->>Bus: ovos.listener.<br/>record.ended
    Listener->>Bus: ovos.utterance.handle
    Bus->>Core: ovos.utterance.handle
    Core->>Core: utterance + metadata<br/>transformers
    Core->>Core: pipeline match<br/>(stop → converse → OCP →<br/>padatious → adapt → fallback)
    Core->>Bus: ovos.intent.matched
    Core->>Bus: ovos.intent.<br/>handler.start
    Bus->>Skill: skill_id:intent_name
    Skill->>Skill: intent handler logic
    Skill->>Bus: ovos.utterance.speak
    Core->>Bus: ovos.intent.<br/>handler.complete
    Bus->>Audio: ovos.utterance.speak
    Audio->>Audio: dialog transformer +<br/>TTS + tts-transformer
    Audio->>Speakers: play WAV /<br/>update GUI
    Core->>Bus: ovos.utterance.handled

Diagram: a sequence from the microphone capturing raw audio through wake-word detection, STT, and the pipeline match (stop, converse, OCP, padatious, adapt, fallback) in ovos-core, to a skill's intent handler, TTS playback in ovos-audio, and the final ovos.utterance.handled event.

1. Capture and Wake-word Detection¶

Service: ovos-dinkum-listener (or similar) Input: Raw audio from the microphone plugin.

The listener service is always active, monitoring a stream of audio.

VAD Plugin: Continuously checks if someone is speaking.
Wake-word Plugin: Monitors the audio stream for the configured wake word (e.g., "Hey Mycroft").
Trigger: Once the wake word is detected, the listener begins recording the subsequent audio as a potential utterance.

2. Speech-to-Text (STT)¶

Service: ovos-dinkum-listener Output: ovos.utterance.handle (legacy: recognizer_loop:utterance) (messagebus)

Once the user stops speaking (detected by the VAD plugin), the recorded audio buffer is sent to the STT Plugin. Capture is bracketed by the listening-lifecycle signals ovos.listener.record.started / ovos.listener.record.ended (OVOS-AUDIO-IN-1 §6; legacy: recognizer_loop:record_begin / record_end).

The STT engine (e.g., Whisper, Google, Vosk) transcribes the audio into text. Before STT, the raw audio first passes through the audio-transformer chain (OVOS-TRANSFORM-1 §3.1).
The listener emits an ovos.utterance.handle message. This is the lifecycle entry point of OVOS-PIPELINE-1 §9.1, and it contains the transcription candidates in data.utterances.

3. Utterance and Metadata Transformation¶

Service: ovos-core (Intent Service) — the orchestrator Bus Event: ovos.utterance.handle (legacy: recognizer_loop:utterance)

The IntentService within ovos-core picks up the transcription. Before matching it to an intent, it passes it through two of the six Transformer chains (OVOS-TRANSFORM-1):

Utterance Transformers (§3.2): These can normalize the text (e.g., "42" -> "forty-two"), fix common STT errors, or expand abbreviations.
Metadata Transformers (§3.3): These can enrich the message context with information like the user's emotion or the current environmental noise level.

The entry message may carry no authoritative lang. This happens when the producer did not know the content language for certain, a common case for STT output. In that case, the orchestrator resolves the utterance's language once, from session evidence (user preference, lang-detect signals), and passes that resolved tag to every pipeline plugin's match call for this utterance. Pipeline plugins may refine the tag they receive (a multilingual matcher may detect a different content language), but they must not re-derive it independently from session evidence. A single resolution point keeps the whole match round matching in the same language, instead of letting the result depend, unpredictably, on which pipeline plugin happens to run first.

4. Intent Pipeline Matching¶

Service: ovos-core (Intent Service) Process: Ordered evaluation of matchers.

The (potentially modified) utterance is now evaluated against the Intent Pipeline. The orchestrator calls each pipeline plugin's match(utterances, lang, session) in order and takes the first that returns a Match. This is first-match-wins, with no cross-plugin confidence scoring (OVOS-PIPELINE-1 §6.2). The sequence diagram above collapses this to one step (stop → converse → OCP → padatious → adapt → fallback). That shorthand names each matcher once. The pipeline is configurable, and the list below is the full expansion of the actual default order. See Pipelines Overview for the authoritative list and how to customize it. A single matcher (e.g. Adapt, Padatious) is often registered several times at decreasing internal-confidence tiers (high → medium → low). This is why those names appear interleaved through the list:

Stop: "stop" / "cancel" is checked first so the assistant can always be interrupted.
Converse: Active skills are given a chance to intercept the utterance (e.g., for multi-turn questions).
Common Play (OCP): If the utterance sounds like a media request (e.g., "Play some jazz"), it's routed to OCP.
Padatious: Example-based matching for natural-language phrasings.
Adapt: Keyword/rule matching for direct commands.
Fallback: As a last resort, fallback skills (like LLM-based solvers) attempt to handle the utterance.

(Other matchers such as Model2Vec and, if installed, Common Query for general-knowledge questions, slot into this order too — see Pipelines for the full default and how to customize it.)

Each match call is invoked directly inside a try/except. If a matcher raises, the orchestrator logs it and moves on to the next matcher as a no-match. A matcher that merely hangs (never returns, never raises) is a different story: keep matchers fast.

Operational limits

Limit	Detail
Default pipeline order	See Pipelines Overview for the full default and how to customize it
Per-match timeout	None. A hung matcher blocks the pipeline until it returns
Handler timeout	5 minutes by default (`intents.handler_timeout`). Bounds the skill handler invoked after a match, not the match calls themselves
`websocket.shared_connection: true` (default)	`ovos-core` runs on a single shared bus connection, and that same thread services the bus. A hung matcher stalls the whole service
Self-blocking `match()` calls	A matcher that does its own bus round-trip inside `match()` (e.g. a stop plugin gathering `stop.pong` replies) waits on the same thread that would deliver the replies. Such reply-collection waits (e.g. the stop pipeline's ~0.5 s pong wait) usually expire without receiving any replies, because the thread that would deliver them is the one doing the waiting. Only `websocket.shared_connection: false` gives skills their own connections and threads, and even then the orchestrator and pipeline plugins stay on the one shared bus

5. Skill Execution¶

Service: A specific Skill (running in ovos-core) Bus Event: ovos.intent.matched, {skill_id}.activate, and the specific intent dispatch message.

Once a match is found, the orchestrator post-processes it through the intent-transformer chain (OVOS-TRANSFORM-1 §3.4), emits ovos.intent.matched (§9.2), then dispatches to the winning skill on <skill_id>:<intent_name>. The orchestrator wraps that invocation in the handler-lifecycle trio ovos.intent.handler.start → …complete / …error (§8). start goes out immediately before the dispatch. The trio does not close when a function returns a value: the dispatch is a bus emit, so the orchestrator closes the trio when the framework's done-signal (mycroft.skill.handler.complete / .error) comes back, or after a timeout if no signal arrives — five minutes by default, intents.handler_timeout. A handler that never signals done therefore ends as a timeout error, five minutes later. See Intent Service for the exact mechanism, and dig_for_message for how the skill's own code gets hold of the dispatch message it is answering.

When the done-signal arrives, the orchestrator syncs the round's working session with whatever the handler mutated — intent_context entry by entry (a null value removes an entry), response_mode, and any active_handlers removals; fields the handler does not own are never pulled in. That synced session, not the dispatch-time one, is what …complete / …error and the closing ovos.utterance.handled carry, and it is authoritative for that round's context decay (OVOS-SESSION-2 §2.6). This sync happens whether or not the handler emitted anything itself. One known gap: writes a handler makes from inside converse() are not yet synced this way.

A handful of intent names are reserved. A Match bearing one of them is a continuation or termination of an already-active skill's participation, not a fresh activation. So the dispatch does not push the skill onto session.active_handlers again. That suppression depends on which pipeline plugin produced the match. ovos-core checks whether the producing pipeline_id is one of the reserved-name pipeline roles (the converse, fallback, and common-query plugins) via _produces_reserved_name(pipeline_id), and skips the session.active_handlers push for those, per OVOS-PIPELINE-1 §7.3. Stop matches suppress the push through a different mechanism: the stop plugin sets IntentHandlerMatch.suppress_activation on the match itself (see Pipeline Plugin Authoring).

The skill's intent handler is triggered.
The skill performs its logic (e.g., querying an API, controlling a device).
If the skill needs to respond, it calls self.speak() or self.speak_dialog().

6. Text-to-Speech (TTS)¶

Service: ovos-audio Bus Event: ovos.utterance.speak (legacy: speak)

The skill emits an ovos.utterance.speak message containing the response text. This is the natural-language response exit point of the lifecycle (OVOS-PIPELINE-1 §9.6).

The ovos-audio service receives the message and runs the text through the dialog-transformer chain (OVOS-TRANSFORM-1 §3.5) before synthesis.
It sends the (transformed) text to the TTS Plugin (e.g., Piper, Mimic, Coqui) to generate a WAV file, then runs the audio through the tts-transformer chain (OVOS-TRANSFORM-1 §3.6). This dialog → TTS → tts-transformer → playback path is specified by OVOS-AUDIO-1.
It also requests Visemes (for lip-sync) from a G2P Plugin.

7. Audio Playback and GUI Updates¶

Service: ovos-audio and ovos-gui Output: Sound from speakers and visuals on screen.

Playback: ovos-audio plays the generated WAV file through the configured audio output (e.g., ALSA, PulseAudio).
GUI: If the skill provided a UI (via self.gui.show_page()), the ovos-gui service renders the QML/HTML view on the screen, often synchronized with the spoken response. ⚠️ The current ("legacy") GUI is deprecated. There is no generally usable OVOS GUI (a replacement is in progress). The spoken response still works regardless.

8. Session Wrap-up¶

Service: ovos-core (Session Manager)

The lifecycle closes with exactly one ovos.utterance.handled event (OVOS-PIPELINE-1 §9.5). This is the universal end-marker that fires whether an intent matched, a fallback answered, or nothing claimed the utterance. Any intent_context entry may carry an optional per-entry turns_remaining field, a declared turn-budget stored alongside the time-based expiry (see Session); entries written through set_context carry an expires_at stamp and decay normally (see Conversational Context).

The end-marker carries the round's working session, not the snapshot the dispatch message arrived with. The orchestrator binds that working session to the dispatch message at intake. An in-process skill handler and the orchestrator then mutate one shared object for the whole round. A handler's intent_context writes are therefore visible on the end-marker, and a named session's context decay reaches the wire instead of being stranded in the orchestrator.

If the skill requested a follow-up question (e.g., expect_response=True / listen=True), the reply is spoken and the listener is reactivated directly into recording. The wake word is bypassed, so the cycle resumes at Step 2 (recording → STT), not the wake-word gate of Step 1, with the current Session context preserved.