The mechanism
Speech interaction has a time structure that a completed text prompt hides. Audio arrives in frames, a recognizer may revise partial transcripts, and endpointing decides when a turn is complete enough for the next stage. A response can also be interrupted after generation or playback has begun. Speech recognition quality and turn-taking behavior therefore need different measurements. Whisper supplies primary context for learned audio-to-transcript modeling; it does not by itself define every streaming, interruption or tool-authorization behavior of a voice assistant. The worked cases here use authored timing and transcription errors. They do not run a recognizer, synthesize speech or execute a real-time audio pipeline.
- Audio and transcript events retain timestamps and distinguish partial hypotheses from finalized text.
- Endpointing or an explicit user signal determines when the application advances the turn.
- Model and tool execution use the intended finalized command while handling later interruption as a separate event.
- Recognition error, response delay and external action correctness are evaluated separately.
Science and assumptions
Speech is a timed interaction · explicit relationship
Word error rate is substitutions plus deletions plus insertions divided by reference word count.
Symbols and units
- S, D, I
- Aligned substitution, deletion and insertion counts
word edits
- Nref
- Number of words in the reference transcript
positive word count
- WER
- Edit ratio, which can exceed one with many insertions
dimensionless
On an authored 20-word reference, two substitutions, one deletion and one insertion give word error rate (2+1+1)/20 = 0.20. Separately, after speech ends, a serial pipeline waits 300 ms for endpointing, 100 ms for final recognition, 200 ms for a policy response and 150 ms before playback: 750 ms before audible output. Overlap could change that sum, but it is excluded here. An early partial transcript saying “reserve two” that later changes to “reserve none” cannot be treated as an irrevocable authorization merely because recognition has started.
Assumptions A fixed reference, alignment convention and tokenization are required. WER is not a probability and does not measure tool correctness, meaning preservation or turn latency.
Engineering example
On an authored 20-word reference, two substitutions, one deletion and one insertion give word error rate (2+1+1)/20 = 0.20. Separately, after speech ends, a serial pipeline waits 300 ms for endpointing, 100 ms for final recognition, 200 ms for a policy response and 150 ms before playback: 750 ms before audible output. Overlap could change that sum, but it is excluded here. An early partial transcript saying “reserve two” that later changes to “reserve none” cannot be treated as an irrevocable authorization merely because recognition has started.
Distinctions and tradeoffs
A partial transcript is not necessarily a completed or authorized user command.
Short endpoint delays improve responsiveness but can prematurely split turns; longer waits preserve context while increasing perceived latency.
Voice agents must coordinate learned speech representations with temporal control and tool boundaries.
Specifications and interfaces
- Mechanism boundary
- When is a spoken request complete enough to act on?
A fixed reference, alignment convention and tokenization are required. WER is not a probability and does not measure tool correctness, meaning preservation or turn latency.
- Evidence type
- Authored worked example
Worked explanation with a bounded numeric or state example. This lesson does not claim an executing implementation of an external model or platform.
Interfaces
- Declared input and state Input
- Audio and transcript events retain timestamps and distinguish partial hypotheses from finalized text.
- Checked outcome Output
- Separate transcription error, turn latency and action authority in a timed speech interaction.
Check your understanding
With 2 substitutions, 1 deletion and 1 insertion against 20 reference words, what is WER?
- 10%
- 20%
- 80%
Answer and reasoning
20%
The three edit counts sum to four, divided by twenty reference words.
Evidence and scope
This is a worked explanation, not an executing external platform, model or robot. The quantities and fixtures are authored. Source scope is recorded in the automation learning review manifest; no source benchmark or private system behavior is claimed. A fixed reference, alignment convention and tokenization are required. WER is not a probability and does not measure tool correctness, meaning preservation or turn latency.
The worked example explains this mechanism. General numerical labs are a reference; they do not execute this lesson’s full mechanism.