{
  "schemaVersion": 1,
  "id": "atlas-lesson-streaming-speech-turns",
  "title": "Speech is a timed interaction",
  "category": "learning",
  "format": "Atlas lesson",
  "description": "Audio frames, transcription revisions, endpointing, interruption and tool confirmation.",
  "canonicalUrl": "https://semiaifoundry.com/learn/atlas/streaming-speech-turns/",
  "publishedDate": null,
  "revisionDate": "2026-10-07",
  "author": null,
  "topic": "Models become evaluated applications",
  "series": null,
  "license": null,
  "rightsUrl": "https://semiaifoundry.com/ai-atlas/licenses.html",
  "version": "1.3.0",
  "status": "Published educational lesson",
  "evidenceScope": "This is a worked explanation, not an executing external platform, model or robot. The quantities and fixtures are authored. Source scope is recorded in the automation learning review manifest; no source benchmark or private system behavior is claimed. A fixed reference, alignment convention and tokenization are required. WER is not a probability and does not measure tool correctness, meaning preservation or turn latency.",
  "contentCoverage": "published-lesson",
  "artifactUrl": "https://semiaifoundry.com/ai-atlas/lesson-data/streaming-speech-turns-abe0eddb9c20f463.json",
  "interactiveUrl": "https://semiaifoundry.com/ai-atlas/#streaming-speech-turns",
  "related": [],
  "supersedes": [],
  "supersededBy": [],
  "contentHash": "c6761dcdefeb8f7aff44cb1334f90019d380be1984e9272dca9af9c2fe75e30e",
  "contentLength": 6148,
  "jsonUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-streaming-speech-turns.json",
  "markdownUrl": "https://semiaifoundry.com/agents/publications/atlas-lesson-streaming-speech-turns.md",
  "sections": [
    {
      "id": "section-1",
      "title": "Speech is a timed interaction",
      "start": 53,
      "end": 444
    },
    {
      "id": "section-2",
      "title": "The mechanism",
      "start": 444,
      "end": 1608
    },
    {
      "id": "section-3",
      "title": "Science and assumptions",
      "start": 1608,
      "end": 1636
    },
    {
      "id": "section-4",
      "title": "Speech is a timed interaction · explicit relationship",
      "start": 1636,
      "end": 2829
    },
    {
      "id": "section-5",
      "title": "Engineering example",
      "start": 2829,
      "end": 3406
    },
    {
      "id": "section-6",
      "title": "Distinctions and tradeoffs",
      "start": 3406,
      "end": 3767
    },
    {
      "id": "section-7",
      "title": "Specifications and interfaces",
      "start": 3767,
      "end": 4238
    },
    {
      "id": "section-8",
      "title": "Interfaces",
      "start": 4238,
      "end": 4509
    },
    {
      "id": "section-9",
      "title": "Check your understanding",
      "start": 4509,
      "end": 4538
    },
    {
      "id": "section-10",
      "title": "With 2 substitutions, 1 deletion and 1 insertion against 20 reference words, what is WER?",
      "start": 4538,
      "end": 4756
    },
    {
      "id": "section-11",
      "title": "Evidence and scope",
      "start": 4756,
      "end": 5340
    },
    {
      "id": "section-12",
      "title": "Sources",
      "start": 5340,
      "end": 5801
    },
    {
      "id": "section-13",
      "title": "Continue exploring",
      "start": 5801,
      "end": 5824
    },
    {
      "id": "section-14",
      "title": "Foundations",
      "start": 5824,
      "end": 6045
    },
    {
      "id": "section-15",
      "title": "Related lessons",
      "start": 6045,
      "end": 6148
    }
  ],
  "citations": [
    {
      "title": "Explore in Atlas",
      "url": "https://semiaifoundry.com/ai-atlas/#streaming-speech-turns"
    },
    {
      "title": "Robust Speech Recognition via Large-Scale Weak Supervision",
      "url": "https://arxiv.org/abs/2212.04356v1"
    },
    {
      "title": "ROS 2 Jazzy: Understanding actions",
      "url": "https://raw.githubusercontent.com/ros2/ros2_documentation/jazzy/source/Concepts/Basic/About-Actions.rst"
    },
    {
      "title": "Published lesson data",
      "url": "https://semiaifoundry.com/ai-atlas/lesson-data/streaming-speech-turns-abe0eddb9c20f463.json"
    },
    {
      "title": "Atlas releases and licensing",
      "url": "https://semiaifoundry.com/ai-atlas/licenses.html"
    },
    {
      "title": "Human review, interruption and cancellation",
      "url": "https://semiaifoundry.com/learn/atlas/human-review-cancellation/"
    },
    {
      "title": "Evaluation & the AGI frontier",
      "url": "https://semiaifoundry.com/learn/atlas/evaluation/"
    },
    {
      "title": "Tools, memory & agent loop",
      "url": "https://semiaifoundry.com/learn/atlas/tool-loop/"
    }
  ],
  "recordHash": "73f48270bd0b767c9a5c4c8ffc30964def5345831accd7d3cce528ab5dde4c32",
  "contentFormat": "text/markdown",
  "content": "Models become evaluated applications · Atlas v1.3.0\n\n# Speech is a timed interaction\n\nAudio frames, transcription revisions, endpointing, interruption and tool confirmation.\n\n[Explore in Atlas](https://semiaifoundry.com/ai-atlas/#streaming-speech-turns)Logical state, task evidence and declared units; no physical assembly scale\n\nLearning objectiveSeparate transcription error, turn latency and action authority in a timed speech interaction.\n\n## The mechanism\n\nSpeech interaction has a time structure that a completed text prompt hides. Audio arrives in frames, a recognizer may revise partial transcripts, and endpointing decides when a turn is complete enough for the next stage. A response can also be interrupted after generation or playback has begun. Speech recognition quality and turn-taking behavior therefore need different measurements. Whisper supplies primary context for learned audio-to-transcript modeling; it does not by itself define every streaming, interruption or tool-authorization behavior of a voice assistant. The worked cases here use authored timing and transcription errors. They do not run a recognizer, synthesize speech or execute a real-time audio pipeline.\n\n1.  Audio and transcript events retain timestamps and distinguish partial hypotheses from finalized text.\n2.  Endpointing or an explicit user signal determines when the application advances the turn.\n3.  Model and tool execution use the intended finalized command while handling later interruption as a separate event.\n4.  Recognition error, response delay and external action correctness are evaluated separately.\n\n## Science and assumptions\n\n### Speech is a timed interaction · explicit relationship\n\n$$\n\\operatorname{WER}=\\frac{S+D+I}{N_{\\mathrm{ref}}}\n$$\n\nWord error rate is substitutions plus deletions plus insertions divided by reference word count.\n\nSymbols and units\n\nS, D, I\n\nAligned substitution, deletion and insertion counts\n\nword edits\n\nNref\n\nNumber of words in the reference transcript\n\npositive word count\n\nWER\n\nEdit ratio, which can exceed one with many insertions\n\ndimensionless\n\nOn an authored 20-word reference, two substitutions, one deletion and one insertion give word error rate (2+1+1)/20 = 0.20. Separately, after speech ends, a serial pipeline waits 300 ms for endpointing, 100 ms for final recognition, 200 ms for a policy response and 150 ms before playback: 750 ms before audible output. Overlap could change that sum, but it is excluded here. An early partial transcript saying “reserve two” that later changes to “reserve none” cannot be treated as an irrevocable authorization merely because recognition has started.\n\nAssumptions A fixed reference, alignment convention and tokenization are required. WER is not a probability and does not measure tool correctness, meaning preservation or turn latency.\n\n## Engineering example\n\nOn an authored 20-word reference, two substitutions, one deletion and one insertion give word error rate (2+1+1)/20 = 0.20. Separately, after speech ends, a serial pipeline waits 300 ms for endpointing, 100 ms for final recognition, 200 ms for a policy response and 150 ms before playback: 750 ms before audible output. Overlap could change that sum, but it is excluded here. An early partial transcript saying “reserve two” that later changes to “reserve none” cannot be treated as an irrevocable authorization merely because recognition has started.\n\n## Distinctions and tradeoffs\n\nA partial transcript is not necessarily a completed or authorized user command.\n\nShort endpoint delays improve responsiveness but can prematurely split turns; longer waits preserve context while increasing perceived latency.\n\nVoice agents must coordinate learned speech representations with temporal control and tool boundaries.\n\n## Specifications and interfaces\n\nMechanism boundary\n\nWhen is a spoken request complete enough to act on?\n\nA fixed reference, alignment convention and tokenization are required. WER is not a probability and does not measure tool correctness, meaning preservation or turn latency.\n\nEvidence type\n\nAuthored worked example\n\nWorked explanation with a bounded numeric or state example. This lesson does not claim an executing implementation of an external model or platform.\n\n### Interfaces\n\nDeclared input and state Input\n\nAudio and transcript events retain timestamps and distinguish partial hypotheses from finalized text.\n\nChecked outcome Output\n\nSeparate transcription error, turn latency and action authority in a timed speech interaction.\n\n## Check your understanding\n\n### With 2 substitutions, 1 deletion and 1 insertion against 20 reference words, what is WER?\n\n1.  10%\n2.  20%\n3.  80%\n\nAnswer and reasoning\n\n20%\n\nThe three edit counts sum to four, divided by twenty reference words.\n\n## Evidence and scope\n\nThis is a worked explanation, not an executing external platform, model or robot. The quantities and fixtures are authored. Source scope is recorded in the automation learning review manifest; no source benchmark or private system behavior is claimed. A fixed reference, alignment convention and tokenization are required. WER is not a probability and does not measure tool correctness, meaning preservation or turn latency.\n\nThe worked example explains this mechanism. General numerical labs are a reference; they do not execute this lesson’s full mechanism.\n\n## Sources\n\n-   [Robust Speech Recognition via Large-Scale Weak Supervision](https://arxiv.org/abs/2212.04356v1)\n-   [ROS 2 Jazzy: Understanding actions](https://raw.githubusercontent.com/ros2/ros2_documentation/jazzy/source/Concepts/Basic/About-Actions.rst)\n\n[Published lesson data](https://semiaifoundry.com/ai-atlas/lesson-data/streaming-speech-turns-abe0eddb9c20f463.json) · [Atlas releases and licensing](https://semiaifoundry.com/ai-atlas/licenses.html)\n\n## Continue exploring\n\n### Foundations\n\n-   [Human review, interruption and cancellation](https://semiaifoundry.com/learn/atlas/human-review-cancellation/)\n-   [Evaluation & the AGI frontier](https://semiaifoundry.com/learn/atlas/evaluation/)\n\n### Related lessons\n\n-   [Tools, memory & agent loop](https://semiaifoundry.com/learn/atlas/tool-loop/)"
}