| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
LengthsCapableSequential and LengthsCapableChain only forwarded the lengths argument to layers whose forward() declares a parameter literally named 'lengths'. The HuggingFace integration lobes (wav2vec2, HuBERT, WavLM, ...) name it 'wav_lens', so in inference pipelines such as EncoderASR the encoder silently received no lengths: no attention mask was built and self-attention attended over the zero-padding, degrading batched transcriptions relative to per-utterance inference (speechbrain#2986). The containers now detect the actual parameter name ('lengths' or 'wav_lens' via the new callchains.lengths_arg_name helper) and pass the lengths under that name. The public lengths_arg_exists helper keeps its exact previous semantics, so the augmenter and enhancement call sites are unaffected; the takes_lengths attributes are kept in sync and now also report wav_lens-style layers. Fixes speechbrain#2986 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
| Back | FazBrowse Home | New Git URL |
Motivation
Reported in #2986: batched inference through EncoderASR produces different transcripts than per-utterance inference, with divergences appearing early in the shorter (padded) utterances — not just in the tail.
Root cause: LengthsCapableSequential (and the lighter LengthsCapableChain) only forward lengths to layers whose forward() declares a parameter literally named lengths (speechbrain/utils/callchains.py::lengths_arg_exists). The HuggingFace integration lobes all name it wav_lens (forward(self, wav, wav_lens=None)), so in the standard encoder: !new:speechbrain.nnet.containers.LengthsCapableSequential inference setup, the wav2vec2/HuBERT/WavLM module is silently called without lengths → make_padding_masks(src, wav_len=None) returns None → the HF model runs with attention_mask=None, and self-attention attends over the zero-padding, shifting every frame of the shorter items in the batch.
Note this is also a train/inference mismatch: training recipes typically call the lobe directly (self.modules.wav2vec2(wavs, wav_lens)), so the attention mask is applied during training but silently dropped at batched inference time.
Changes
Backward compatibility
The issue reporter validated the equivalent fix end-to-end via a user-side wrapper: the WER delta between per-utterance and batched inference dropped to ~1e-3 over 300 samples (the residual is the convolutional front-end seeing the padding, inherent to padded batching). This PR makes that wrapper unnecessary.
Test Plan
Fixes #2986.
Co-written by a human (@oliver0006) and Claude AI (Fable 5) working together.