FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

FocalCodec [NeurIPS 2025] by lucadellalib · Pull Request #3000 · speechbrain/speechbrain · GitHub

/ speechbrain Public

FocalCodec [NeurIPS 2025] - #3000

Merged
pplantinga merged 8 commits into
speechbrain:developfrom
lucadellalib:focalcodec
Nov 24, 2025
ConversationCommits (8)ChecksFiles changed
Merged

FocalCodec [NeurIPS 2025]#3000
pplantinga merged 8 commits into
speechbrain:developfrom
lucadellalib:focalcodec

Conversation

lucadellalib commented Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Add FocalCodec training recipe.

mravanelli requested changes Nov 21, 2025
View reviewed changes

mravanelli left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Great Job @lucadellalib! I did a quick code inspection and shared some comments about the docstrings.
Other Comments:

  • Regarding the extra-depenency, please take a look at our policy here. This code should be compliant with that. @pplantinga can advise
  • I'm not sure about having "metrics" as a local folder. I think we might need to resuse the same metrics in other recipes, for instance the streamable focalcodec and the extension of FocalCoded to LibriLight. Maybe we can put it in SpeechBrain/metrics?. Any advise @pplantinga ?

Comment thread recipes/LibriTTS/focalcodec/train_decoder.py


class Generation(sb.Brain):
def fit_batch(self, batch):

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

For every method, we typically add some short description about its functionality for better clarity (See for instance this). It is even more important here as some methods are not standard.

Comment thread recipes/LibriTTS/focalcodec/train_quantizer.py
return super()._fit_valid(valid_set, epoch, enable)

@torch.no_grad()
def evaluate_batch(self, batch, stage):

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Make sure every method has a short description.

Comment thread recipes/LibriTTS/focalcodec/utils.py Outdated


def prepare_recipe(hparams, run_opts):
# Dataset preparation

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Add docstring

Comment thread recipes/LibriTTS/focalcodec/utils.py Outdated
audio_backend="soundfile",
**kwargs,
):
"""This function prepares the datasets to be used in the brain class.

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Improve the docstring by explaining all the parameters. They are a lot in this case, but that can improve clarity and usability.

Comment thread recipes/LibriTTS/focalcodec/utils.py Outdated
provides = ["sig"]

def audio_pipeline_train(wav):
original_sample_rate = sb.dataio.dataio.read_audio_info(wav).sample_rate

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Make sure all the functions have a short docstring

Comment thread recipes/LibriTTS/focalcodec/metrics/dwer.py


class DWER(MetricStats):
def __init__(

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Add docstring with a working example

Comment thread recipes/LibriTTS/focalcodec/metrics/speaker_similarity.py


class SpkSimWavLM(MetricStats):
def __init__(

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Add a docstring with a working example (such that we can test all with our doc tests)

Comment thread recipes/LibriTTS/focalcodec/metrics/utmos.py


class UTMOS(MetricStats):
def __init__(self, sample_rate, model=None):

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Add docstring with example

Comment thread speechbrain/lobes/models/HifiGAN.py Outdated


class HingeGLoss(nn.Module):
"""Hinge Generator Loss

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Add the example to the docstring

Comment thread speechbrain/lobes/models/HifiGAN.py Outdated


class HingeDLoss(nn.Module):
"""Hinge Discriminator Loss

mravanelli Nov 21, 2025

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Add example

pplantinga commented Nov 21, 2025

Copy link
Copy Markdown
Collaborator
  • Regarding the extra-depenency, please take a look at our policy here. This code should be compliant with that. @pplantinga can advise

I think this PR follows the policy: for recipes, the only requirement is an extra-requirements.txt file with all non-speechbrain dependencies. The one thing I might suggest is adding transformers to the extra requirements, as this is (mostly) moved to integrations now, not a core dependency.

  • I'm not sure about having "metrics" as a local folder. I think we might need to resuse the same metrics in other recipes, for instance the streamable focalcodec and the extension of FocalCoded to LibriLight. Maybe we can put it in SpeechBrain/metrics?. Any advise @pplantinga ?

Perhaps we can leave it here for now, and move it if we do end up using it for other recipes. The principle of YAGNI (you ain't gonna need it) might apply here, let's keep it as straightforward as possible and not plan too far ahead.

mravanelli commented Nov 22, 2025

Copy link
Copy Markdown
Collaborator

Thank you for your comments @pplantinga! Do you have other comments or suggestions?

mravanelli commented Nov 22, 2025

Copy link
Copy Markdown
Collaborator

@Adel-Moumen, do you also have some comments and suggestions here?

mravanelli added the enhancement New feature or request label Nov 22, 2025

mravanelli commented Nov 22, 2025

Copy link
Copy Markdown
Collaborator

I tested the recipe and the recipe tests. All seems to work properly.
A couple of small points:

  • We need to upload the logs to Dropbox. I will follow up on that privately.
  • In both Yaml file, train-clean-360 and train-other-500. I would suggest uncommenting that by default.

mravanelli commented Nov 24, 2025 •
edited
Loading

Copy link
Copy Markdown
Collaborator

This PR LGTM now. I think we can go ahead and merge it, unless @pplantinga or @Adel-Moumen have further comments. Great Job @lucadellalib!

pplantinga approved these changes Nov 24, 2025
View reviewed changes

pplantinga left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

LGTM

pplantinga added this to the v1.1.0 milestone Nov 24, 2025
pplantinga added the recipes Changes to recipes only (add/edit) label Nov 24, 2025
pplantinga merged commit 637f0a5 into speechbrain:develop Nov 24, 2025
5 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

pplantinga pplantinga approved these changes
mravanelli mravanelli approved these changes
Adel-Moumen Awaiting requested review from Adel-Moumen

Assignees

lucadellalib

Labels

enhancement New feature or request recipes Changes to recipes only (add/edit)

Projects

None yet

Milestone

v1.1.0

Development

Successfully merging this pull request may close these issues.

3 participants

Back | FazBrowse Home | New Git URL