The mechanism
An audio model receives sampled waveforms or derived features, while a transcript decoder predicts a sequence of text tokens. Sample rate, window length and hop determine the time representation; transcript labels need not align one-to-one with frames. Whisper uses a particular log-Mel representation and sequence-to-sequence task format. That is different from a CTC alignment objective or an audio-generation objective. Evaluation must also declare text normalization: case, punctuation and tokenization can change edit counts. The example below computes geometry and word errors without claiming to execute speech recognition.
- Retain waveform provenance, sample rate and task labels.
- Compute frame boundaries under a stated padding/window convention.
- Distinguish audio frames, text tokens and reference words.
- Score substitutions, deletions and insertions against a fixed normalized reference, with speaker/noise slices separate.
Science and assumptions
Audio representation, alignment and transcript loss · worked relation
Relate audio samples, time frames and transcript errors without mixing their denominators.
Symbols and units
- F
- Unpadded full-window frame count
frames
- N
- Waveform length at declared sample rate
samples
- W,H
- Window length and hop
samples
- S,D,I
- Substitution deletion and insertion counts
word edits
- N_ref
- Reference transcript length
words
- WER
- Word error ratio
dimensionless
A one-second waveform sampled at 16000 Hz contains 16000 samples. With a 400-sample window, 160-sample hop and no padding, frame count is 1+floor((16000−400)/160)=98. A centered/padded library can differ. Reference “turn left now” and hypothesis “turn right now please” have one substitution and one insertion over three reference words, giving WER=2/3. Insertions can make WER exceed 100%; it is not a bounded accuracy percentage.
Assumptions N at least W, no padding or centering for frame arithmetic; fixed whitespace-normalized reference words for WER. No audio data is uploaded or speech model run by this lesson.
Engineering example
A one-second waveform sampled at 16000 Hz contains 16000 samples. With a 400-sample window, 160-sample hop and no padding, frame count is 1+floor((16000−400)/160)=98. A centered/padded library can differ. Reference “turn left now” and hypothesis “turn right now please” have one substitution and one insertion over three reference words, giving WER=2/3. Insertions can make WER exceed 100%; it is not a bounded accuracy percentage.
Distinctions and tradeoffs
It is a mistake to conclude: There are three correctly generated audio frames.
Feature compression reduces sequence length but discards waveform detail. Transcript metrics can miss semantic severity and conceal speaker or noise failures.
Teach modality-specific representation, objective and evaluation choices. Relate audio samples, time frames and transcript errors without mixing their denominators.
Specifications and interfaces
- Learning mode
- worked example
The executable decoder has 423 parameters; worked extensions do not imply an additional simulator.
- Worked contract
- Relate audio samples, time frames and transcript errors without mixing their denominators.
N at least W, no padding or centering for frame arithmetic; fixed whitespace-normalized reference words for WER. No audio data is uploaded or speech model run by this lesson.
Interfaces
- Inputs and state Input
- Retain waveform provenance, sample rate and task labels.
- Result and check Output
- Relate audio samples, time frames and transcript errors without mixing their denominators.
Check your understanding
Why is the WER denominator three in the example?
- There are three reference words, irrespective of hypothesis length.
- There are three correctly generated audio frames.
- WER always divides by the number of hypothesis words.
Answer and reasoning
There are three reference words, irrespective of hypothesis length.
There are three reference words, irrespective of hypothesis length. N at least W, no padding or centering for frame arithmetic; fixed whitespace-normalized reference words for WER. No audio data is uploaded or speech model run by this lesson.
Evidence and scope
Mode: worked example. N at least W, no padding or centering for frame arithmetic; fixed whitespace-normalized reference words for WER. No audio data is uploaded or speech model run by this lesson. Numerical examples are authored calculations, except explicitly identified automated core fixtures. Primary sources support the scoped mechanism; no human specialist review or learner outcome is implied.
Model lifecycle studio: worked example. N at least W, no padding or centering for frame arithmetic; fixed whitespace-normalized reference words for WER. No audio data is uploaded or speech model run by this lesson.