Commit 1db6d58
committed
Fix typos and add ignores from past 6 months
1 parent e40ee88 commit 1db6d58
65 files changed
Lines changed: 178 additions & 178 deletions
File tree
- recipes
- AISHELL-1/ASR/transformer/hparams
- CVSS
- S2ST
- CommonVoice
- ASR/CTC
- LM
- DNS
- enhancement
- noisyspeech_synthesizer
- IEMOCAP
- quantization
- IWSLT22_lowresource/AST/transformer
- hparams
- LJSpeech/TTS/vocoder/hifi_gan_unit
- LibriSpeech/ASR
- CTC/hparams
- transformer
- LibriTTS/TTS/mstacotron2
- MultiWOZ/response_generation
- gpt
- llama2
- TIMIT/ASR/transducer
- Tedlium2
- ASR/transformer
- hparams
- Tokenizer/hparams
- VoxCeleb/SpeakerRec
- speechbrain
- augment
- decoders
- inference
- k2_integration
- lobes/models
- huggingface_transformers
- transformer
- utils
- templates/speech_recognition/ASR
- tests/unittests
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -146,7 +146,7 @@ The results will be saved in the `output_folder` specified in the YAML file. | |||
| 146 | 146 | ## 🎙️ Speech/Audio Processing | |
| 147 | 147 | | Tasks | Datasets | Technologies/Models | | |
| 148 | 148 | | ------------- |-------------| -----| | |
| 149 | - | Speech Recognition | [AISHELL-1](https://github.com/speechbrain/speechbrain/tree/develop/recipes/AISHELL-1), [CommonVoice](https://github.com/speechbrain/speechbrain/tree/develop/recipes/CommonVoice), [DVoice](https://github.com/speechbrain/speechbrain/tree/develop/recipes/DVoice), [KsponSpeech](https://github.com/speechbrain/speechbrain/tree/develop/recipes/KsponSpeech), [LibriSpeech](https://github.com/speechbrain/speechbrain/tree/develop/recipes/LibriSpeech), [MEDIA](https://github.com/speechbrain/speechbrain/tree/develop/recipes/MEDIA), [RescueSpeech](https://github.com/speechbrain/speechbrain/tree/develop/recipes/RescueSpeech), [Switchboard](https://github.com/speechbrain/speechbrain/tree/develop/recipes/Switchboard), [TIMIT](https://github.com/speechbrain/speechbrain/tree/develop/recipes/TIMIT), [Tedlium2](https://github.com/speechbrain/speechbrain/tree/develop/recipes/Tedlium2), [Voicebank](https://github.com/speechbrain/speechbrain/tree/develop/recipes/Voicebank) | [CTC](https://www.cs.toronto.edu/~graves/icml_2006.pdf), [Tranducers](https://arxiv.org/pdf/1211.3711.pdf?origin=publication_detail), [Transformers](https://arxiv.org/abs/1706.03762), [Seq2Seq](http://zhaoshuaijiang.com/file/Hybrid_CTC_Attention_Architecture_for_End-to-End_Speech_Recognition.pdf), [Beamsearch techniques for CTC](https://arxiv.org/pdf/1911.01629.pdf),[seq2seq](https://arxiv.org/abs/1904.02619.pdf),[transducers](https://www.merl.com/publications/docs/TR2017-190.pdf)), [Rescoring](https://arxiv.org/pdf/1612.02695.pdf), [Conformer](https://arxiv.org/abs/2005.08100), [Branchformer](https://arxiv.org/abs/2207.02971), [Hyperconformer](https://arxiv.org/abs/2305.18281), [Kaldi2-FST](https://github.com/k2-fsa/k2) | | ||
| 149 | + | Speech Recognition | [AISHELL-1](https://github.com/speechbrain/speechbrain/tree/develop/recipes/AISHELL-1), [CommonVoice](https://github.com/speechbrain/speechbrain/tree/develop/recipes/CommonVoice), [DVoice](https://github.com/speechbrain/speechbrain/tree/develop/recipes/DVoice), [KsponSpeech](https://github.com/speechbrain/speechbrain/tree/develop/recipes/KsponSpeech), [LibriSpeech](https://github.com/speechbrain/speechbrain/tree/develop/recipes/LibriSpeech), [MEDIA](https://github.com/speechbrain/speechbrain/tree/develop/recipes/MEDIA), [RescueSpeech](https://github.com/speechbrain/speechbrain/tree/develop/recipes/RescueSpeech), [Switchboard](https://github.com/speechbrain/speechbrain/tree/develop/recipes/Switchboard), [TIMIT](https://github.com/speechbrain/speechbrain/tree/develop/recipes/TIMIT), [Tedlium2](https://github.com/speechbrain/speechbrain/tree/develop/recipes/Tedlium2), [Voicebank](https://github.com/speechbrain/speechbrain/tree/develop/recipes/Voicebank) | [CTC](https://www.cs.toronto.edu/~graves/icml_2006.pdf), [Transducers](https://arxiv.org/pdf/1211.3711.pdf?origin=publication_detail), [Transformers](https://arxiv.org/abs/1706.03762), [Seq2Seq](http://zhaoshuaijiang.com/file/Hybrid_CTC_Attention_Architecture_for_End-to-End_Speech_Recognition.pdf), [Beamsearch techniques for CTC](https://arxiv.org/pdf/1911.01629.pdf),[seq2seq](https://arxiv.org/abs/1904.02619.pdf),[transducers](https://www.merl.com/publications/docs/TR2017-190.pdf)), [Rescoring](https://arxiv.org/pdf/1612.02695.pdf), [Conformer](https://arxiv.org/abs/2005.08100), [Branchformer](https://arxiv.org/abs/2207.02971), [Hyperconformer](https://arxiv.org/abs/2305.18281), [Kaldi2-FST](https://github.com/k2-fsa/k2) | | ||
| 150 | 150 | | Speaker Recognition | [VoxCeleb](https://github.com/speechbrain/speechbrain/tree/develop/recipes/VoxCeleb) | [ECAPA-TDNN](https://arxiv.org/abs/2005.07143), [ResNET](https://arxiv.org/pdf/1910.12592.pdf), [Xvectors](https://www.danielpovey.com/files/2018_icassp_xvectors.pdf), [PLDA](https://ieeexplore.ieee.org/document/6639151), [Score Normalization](https://www.sciencedirect.com/science/article/abs/pii/S1051200499903603) | | |
| 151 | 151 | | Speech Separation | [WSJ0Mix](https://github.com/speechbrain/speechbrain/tree/develop/recipes/WSJ0Mix), [LibriMix](https://github.com/speechbrain/speechbrain/tree/develop/recipes/LibriMix), [WHAM!](https://github.com/speechbrain/speechbrain/tree/develop/recipes/WHAMandWHAMR), [WHAMR!](https://github.com/speechbrain/speechbrain/tree/develop/recipes/WHAMandWHAMR), [Aishell1Mix](https://github.com/speechbrain/speechbrain/tree/develop/recipes/Aishell1Mix), [BinauralWSJ0Mix](https://github.com/speechbrain/speechbrain/tree/develop/recipes/BinauralWSJ0Mix) | [SepFormer](https://arxiv.org/abs/2010.13154), [RESepFormer](https://arxiv.org/abs/2206.09507), [SkiM](https://arxiv.org/abs/2201.10800), [DualPath RNN](https://arxiv.org/abs/1910.06379), [ConvTasNET](https://arxiv.org/abs/1809.07454) | | |
| 152 | 152 | | Speech Enhancement | [DNS](https://github.com/speechbrain/speechbrain/tree/develop/recipes/DNS), [Voicebank](https://github.com/speechbrain/speechbrain/tree/develop/recipes/Voicebank) | [SepFormer](https://arxiv.org/abs/2010.13154), [MetricGAN](https://arxiv.org/abs/1905.04874), [MetricGAN-U](https://arxiv.org/abs/2110.05866), [SEGAN](https://arxiv.org/abs/1703.09452), [spectral masking](http://staff.ustc.edu.cn/~jundu/Publications/publications/Trans2015_Xu.pdf), [time masking](http://staff.ustc.edu.cn/~jundu/Publications/publications/Trans2015_Xu.pdf) | | |
@@ -160,10 +160,10 @@ The results will be saved in the `output_folder` specified in the YAML file. | |||
| 160 | 160 | | Voice Activity Detection | [LibriParty](https://github.com/speechbrain/speechbrain/tree/develop/recipes/LibriParty) | [CRDNN](https://arxiv.org/abs/2106.04624) | | |
| 161 | 161 | | Sound Classification | [ESC50](https://github.com/speechbrain/speechbrain/tree/develop/recipes/ESC50), [UrbanSound](https://github.com/speechbrain/speechbrain/tree/develop/recipes/UrbanSound8k) | [CNN14](https://github.com/ranchlai/sound_classification), [ECAPA-TDNN](https://arxiv.org/abs/2005.07143) | | |
| 162 | 162 | | Self-Supervised Learning | [CommonVoice](https://github.com/speechbrain/speechbrain/tree/develop/recipes/CommonVoice), [LibriSpeech](https://github.com/speechbrain/speechbrain/tree/develop/recipes/LibriSpeech) | [wav2vec2](https://arxiv.org/abs/2006.11477) | | |
| 163 | - | Interpretabiliy | [ESC50](https://github.com/speechbrain/speechbrain/tree/develop/recipes/ESC50) | [Learning-to-Interpret (L2I)](https://proceedings.neurips.cc/paper_files/paper/2022/file/e53280d73dd5389e820f4a6250365b0e-Paper-Conference.pdf), [Non-Negative Matrix Factorization (NMF)](https://proceedings.neurips.cc/paper_files/paper/2022/file/e53280d73dd5389e820f4a6250365b0e-Paper-Conference.pdf), [PIQ](https://arxiv.org/abs/2303.12659) | | ||
| 163 | + | Interpretability | [ESC50](https://github.com/speechbrain/speechbrain/tree/develop/recipes/ESC50) | [Learning-to-Interpret (L2I)](https://proceedings.neurips.cc/paper_files/paper/2022/file/e53280d73dd5389e820f4a6250365b0e-Paper-Conference.pdf), [Non-Negative Matrix Factorization (NMF)](https://proceedings.neurips.cc/paper_files/paper/2022/file/e53280d73dd5389e820f4a6250365b0e-Paper-Conference.pdf), [PIQ](https://arxiv.org/abs/2303.12659) | | ||
| 164 | 164 | | Speech Generation | [AudioMNIST](https://github.com/speechbrain/speechbrain/tree/develop/recipes/AudioMNIST) | [Diffusion](https://arxiv.org/abs/2006.11239), [Latent Diffusion](https://arxiv.org/abs/2112.10752) | | |
| 165 | 165 | | Metric Learning | [REAL-M](https://github.com/speechbrain/speechbrain/tree/develop/recipes/REAL-M/sisnr-estimation), [Voicebank](https://github.com/speechbrain/speechbrain/tree/develop/recipes/Voicebank) | [Blind SNR-Estimation](https://arxiv.org/abs/2002.08909), [PESQ Learning](https://arxiv.org/abs/2110.05866) | | |
| 166 | - | Allignment | [TIMIT](https://github.com/speechbrain/speechbrain/tree/develop/recipes/TIMIT) | [CTC](https://www.cs.toronto.edu/~graves/icml_2006.pdf), [Viterbi](https://www.cs.cmu.edu/~cga/behavior/rabiner1.pdf), [Forward Forward](https://www.cs.cmu.edu/~cga/behavior/rabiner1.pdf) | | ||
| 166 | + | Alignment | [TIMIT](https://github.com/speechbrain/speechbrain/tree/develop/recipes/TIMIT) | [CTC](https://www.cs.toronto.edu/~graves/icml_2006.pdf), [Viterbi](https://www.cs.cmu.edu/~cga/behavior/rabiner1.pdf), [Forward Forward](https://www.cs.cmu.edu/~cga/behavior/rabiner1.pdf) | | ||
| 167 | 167 | | Diarization | [AMI](https://github.com/speechbrain/speechbrain/tree/develop/recipes/AMI) | [ECAPA-TDNN](https://arxiv.org/abs/2005.07143), [X-vectors](https://www.danielpovey.com/files/2018_icassp_xvectors.pdf), [Spectral Clustering](http://www.ifp.illinois.edu/~hning2/papers/Ning_spectral.pdf) | | |
| 168 | 168 | ||
| 169 | 169 | ## 📝 Text Processing | |
Lines changed: 2 additions & 2 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -279,8 +279,8 @@ fea_augment: !new:speechbrain.augment.augmenter.Augmenter | |||
| 279 | 279 | concat_original: True | |
| 280 | 280 | min_augmentations: 1 | |
| 281 | 281 | max_augmentations: 1 | |
| 282 | - augment_start_index: !ref <batch_size> # This leaves unchanges original inputs | ||
| 283 | - concat_end_index: !ref <batch_size> # This leaves unchanges original inputs | ||
| 282 | + augment_start_index: !ref <batch_size> # This leaves original inputs unchanged | ||
| 283 | + concat_end_index: !ref <batch_size> # This leaves original inputs unchanged | ||
| 284 | 284 | augment_prob: 1.0 | |
| 285 | 285 | augmentations: [ | |
| 286 | 286 | !ref <time_drop>, | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -1,6 +1,6 @@ | |||
| 1 | 1 | # Speech-to-Speech Translation (with CVSS) | |
| 2 | 2 | This folder contains the recipe for training a speech-to-unit translation (S2UT) model using a pre-trained Wav2Vec 2.0 encoder and a transformer decoder on the CVSS dataset. | |
| 3 | - The implementation is based on [Textless Speech-to-Speech Translation](https://arxiv.org/abs/2112.08352) and [Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentatio](https://arxiv.org/abs/2204.02967) papers. | ||
| 3 | + The implementation is based on [Textless Speech-to-Speech Translation](https://arxiv.org/abs/2112.08352) and [Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation](https://arxiv.org/abs/2204.02967) papers. | ||
| 4 | 4 | ||
| 5 | 5 | ## Dataset | |
| 6 | 6 | [CVSS](https://github.com/google-research-datasets/cvss) is a massively multilingual-to-English speech-to-speech translation corpus. It covers pairs from 21 languages into English. CVSS is derived from the Common Voice speech corpus and the CoVoST 2 speech-to-text translation corpus. | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -37,7 +37,7 @@ | |||
| 37 | 37 | "test": "test", | |
| 38 | 38 | } | |
| 39 | 39 | ||
| 40 | - # Number of samples for the small evalution subset | ||
| 40 | + # Number of samples for the small evaluation subset | ||
| 41 | 41 | SMALL_EVAL_SIZE = 1000 | |
| 42 | 42 | ||
| 43 | 43 | log_format = "[%(asctime)s] [%(levelname)s]: %(message)s" | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -5,7 +5,7 @@ This folder contains scripts necessary to run an ASR experiment with the CommonV | |||
| 5 | 5 | python train.py hparams/{hparam_file}.yaml | |
| 6 | 6 | ||
| 7 | 7 | To use an n-gram Language Model (LM) for decoding, follow these steps: | |
| 8 | - 1. Uncomment the line `kenlm_model_path: none` in the `test_beam_serch` entry in the yaml file. | ||
| 8 | + 1. Uncomment the line `kenlm_model_path: none` in the `test_beam_search` entry in the yaml file. | ||
| 9 | 9 | 2. Set a path to an ARPA or bin file containing the n-gram LM. | |
| 10 | 10 | ||
| 11 | 11 | For training an n-gram LM in ARPA (or bin) format, refer to the LM recipe in recipes/CommonVoice/LM. | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -1,5 +1,5 @@ | |||
| 1 | 1 | ||
| 2 | - # Traing KenLM | ||
| 2 | + # Training KenLM | ||
| 3 | 3 | This folder contains recipes for training the kenLM-gram model for the CommonVoice Dataset. | |
| 4 | 4 | Using Wav2Vec2 in combination with a language model can yield a significant improvement, especially when the model is fine-tuned on small speech datasets. This is a guide to explain how one can create an n-gram language model and combine it with an existing fine-tuned Wav2Vec2. | |
| 5 | 5 | ||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -70,7 +70,7 @@ def csv2text(): | |||
| 70 | 70 | ) | |
| 71 | 71 | csv2text() | |
| 72 | 72 | ||
| 73 | - logger.info(f"Start tarining {hparams['ngram']}-gram kenlm model.") | ||
| 73 | + logger.info(f"Start training {hparams['ngram']}-gram kenlm model.") | ||
| 74 | 74 | tmp_ngram_file = "ngram.arpa" | |
| 75 | 75 | cmd = f'lmplz -o {hparams["ngram"]} <"{hparams["text_file"]}" > "{tmp_ngram_file}"' | |
| 76 | 76 | os.system(cmd) | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -167,7 +167,7 @@ | |||
| 167 | 167 | "https://dns4public.blob.core.windows.net/dns4archive/datasets_fullband" | |
| 168 | 168 | ) | |
| 169 | 169 | ||
| 170 | - # Impulse reponse and Blind testset | ||
| 170 | + # Impulse response and Blind testset | ||
| 171 | 171 | OTHER_URLS = { | |
| 172 | 172 | "impulse_responses": [ | |
| 173 | 173 | "https://www.openslr.org/resources/26/sim_rir_16k.zip", | |
@@ -308,7 +308,7 @@ def download_file( | |||
| 308 | 308 | Split name of the file being downloaded | |
| 309 | 309 | e.g. read_speech | |
| 310 | 310 | filename : str | |
| 311 | - Fielname of the file being downloaded | ||
| 311 | + Filename of the file being downloaded | ||
| 312 | 312 | resume_byte_pos: (int, optional) | |
| 313 | 313 | Starting byte position for resuming the download. | |
| 314 | 314 | Default is None, which means a fresh download. | |
@@ -572,7 +572,7 @@ def validate_file(download_url, download_path): | |||
| 572 | 572 | else: | |
| 573 | 573 | prepare_download() | |
| 574 | 574 | ||
| 575 | - # Modfy contents inside RIR_simple_table.csv | ||
| 575 | + # Modify contents inside RIR_simple_table.csv | ||
| 576 | 576 | file_path = os.path.join( | |
| 577 | 577 | DECOMPRESSED_PATH, "impulse_responses", "RIR_table_simple.csv" | |
| 578 | 578 | ) | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -8,8 +8,8 @@ For data download and prepration, please refer to the `README.md` in `recipes/DN | |||
| 8 | 8 | python train.py hparams/sepformer-dns-16k.yaml --data_folder <path/to/synthesized_shards_data> --baseline_noisy_shards_folder <path/to/baseline_dev_shards_data> | |
| 9 | 9 | ``` | |
| 10 | 10 | ## **DNSMOS Evaluation on baseline-testclips** | |
| 11 | - *Reference: [Offical repo](https://github.com/microsoft/DNS-Challenge/tree/master/DNSMOS) <br>* | ||
| 12 | - Download the evalution models from [Offical repo](https://github.com/microsoft/DNS-Challenge/tree/master/DNSMOS) and save it under `DNSMOS`. Then, to run DNSMOS evalution on the baseline-testclips saved in the above step. | ||
| 11 | + *Reference: [Official repo](https://github.com/microsoft/DNS-Challenge/tree/master/DNSMOS) <br>* | ||
| 12 | + Download the evaluation models from [Official repo](https://github.com/microsoft/DNS-Challenge/tree/master/DNSMOS) and save it under `DNSMOS`. Then, to run DNSMOS evaluation on the baseline-testclips saved in the above step. | ||
| 13 | 13 | ``` | |
| 14 | 14 | # Model=SepFormer | |
| 15 | 15 | python dnsmos_local.py -t results/sepformer-enhancement-16k/1234/save/baseline_audio_results/enhanced_testclips/ -o dnsmos_enhance.csv | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -277,14 +277,14 @@ def wss(ref_wav, deg_wav, srate): | |||
| 277 | 277 | distortion = [] | |
| 278 | 278 | ||
| 279 | 279 | for frame_count in range(num_frames): | |
| 280 | - # (1) Get the Frames for the test and reference speeech. | ||
| 280 | + # (1) Get the Frames for the test and reference speech. | ||
| 281 | 281 | # Multiply by Hanning window. | |
| 282 | 282 | clean_frame = clean_speech[start : start + winlength] | |
| 283 | 283 | processed_frame = processed_speech[start : start + winlength] | |
| 284 | 284 | clean_frame = clean_frame * window | |
| 285 | 285 | processed_frame = processed_frame * window | |
| 286 | 286 | ||
| 287 | - # (2) Compuet Power Spectrum of clean and processed | ||
| 287 | + # (2) Compute Power Spectrum of clean and processed | ||
| 288 | 288 | clean_spec = np.abs(np.fft.fft(clean_frame, n_fft)) ** 2 | |
| 289 | 289 | processed_spec = np.abs(np.fft.fft(processed_frame, n_fft)) ** 2 | |
| 290 | 290 | clean_energy = [None] * num_crit | |
@@ -340,8 +340,8 @@ def wss(ref_wav, deg_wav, srate): | |||
| 340 | 340 | n -= 1 | |
| 341 | 341 | processed_loc_peak.append(processed_energy[n + 1]) | |
| 342 | 342 | ||
| 343 | - # (6) Compuet the WSS Measure for this frame. This includes | ||
| 344 | - # determination of the weighting functino | ||
| 343 | + # (6) Compute the WSS Measure for this frame. This includes | ||
| 344 | + # determination of the weighting function | ||
| 345 | 345 | dBMax_clean = max(clean_energy) | |
| 346 | 346 | dBMax_processed = max(processed_energy) | |
| 347 | 347 | ||
@@ -379,7 +379,7 @@ def wss(ref_wav, deg_wav, srate): | |||
| 379 | 379 | ) | |
| 380 | 380 | ||
| 381 | 381 | # this normalization is not part of Klatt's paper, but helps | |
| 382 | - # to normalize the meaasure. Here we scale the measure by the sum of the | ||
| 382 | + # to normalize the measure. Here we scale the measure by the sum of the | ||
| 383 | 383 | # weights | |
| 384 | 384 | distortion[frame_count] = distortion[frame_count] / np.sum(W) | |
| 385 | 385 | start += int(skiprate) | |
@@ -415,7 +415,7 @@ def llr(ref_wav, deg_wav, srate): | |||
| 415 | 415 | distortion = [] | |
| 416 | 416 | ||
| 417 | 417 | for frame_count in range(num_frames): | |
| 418 | - # (1) Get the Frames for the test and reference speeech. | ||
| 418 | + # (1) Get the Frames for the test and reference speech. | ||
| 419 | 419 | # Multiply by Hanning window. | |
| 420 | 420 | clean_frame = clean_speech[start : start + winlength] | |
| 421 | 421 | processed_frame = processed_speech[start : start + winlength] | |
0 commit comments