{"id":21299983,"url":"https://github.com/yanghaha0908/fasthubert","last_synced_at":"2025-10-29T11:25:56.678Z","repository":{"id":196433731,"uuid":"660521114","full_name":"yanghaha0908/FastHuBERT","owner":"yanghaha0908","description":"Official implementation for Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning","archived":false,"fork":false,"pushed_at":"2024-11-20T10:54:25.000Z","size":3195,"stargazers_count":86,"open_issues_count":0,"forks_count":4,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-02T07:07:56.170Z","etag":null,"topics":["efficiency-optimization","self-supervised-learning","speech-recognition"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yanghaha0908.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-30T07:39:26.000Z","updated_at":"2025-03-10T07:30:07.000Z","dependencies_parsed_at":null,"dependency_job_id":"3c106f95-f7fe-4b58-b81e-5304b4aee8a3","html_url":"https://github.com/yanghaha0908/FastHuBERT","commit_stats":null,"previous_names":["yanghaha0908/fasthubert"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanghaha0908%2FFastHuBERT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanghaha0908%2FFastHuBERT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanghaha0908%2FFastHuBERT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yanghaha0908%2FFastHuBERT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yanghaha0908","download_url":"https://codeload.github.com/yanghaha0908/FastHuBERT/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247999860,"owners_count":21031046,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["efficiency-optimization","self-supervised-learning","speech-recognition"],"created_at":"2024-11-21T15:06:35.595Z","updated_at":"2025-10-29T11:25:56.590Z","avatar_url":"https://github.com/yanghaha0908.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# __Fast-HuBERT__\n## Paper\n[https://arxiv.org/abs/2309.13860](https://arxiv.org/abs/2309.13860)\n## Intro \nExisting speech-based SSL models face a common dilemma in terms of computational cost, which might hinder their potential application and in-depth academic research. Fast-HuBERT is proposed to improve the pretraining efficiency.\n\nFast-HuBERT optimizes the front-end inputs, loss computation and also aggregates other SOTA techniques, including [ILS](https://github.com/microsoft/UniSpeech) and [MonoBERT](https://arxiv.org/abs/2306.08920).\n\nFast-HuBERT can be trained in 1.1 days with 8 V100 GPUs on Librispeech benchmark, without performance drop compared to the original implementation, resulting in 5.2X speedup. \n\n\n## Model \n![Alt text](model.png)\n\n\n## Requirements and Installation\n```\ngit clone https://github.com/pytorch/fairseq\ncd fairseq\npip install --editable ./\ngit clone https://github.com/yanghaha0908/FastHuBERT\n```\n\n## Wandb log\nPretrain:\n[https://wandb.ai/yanghaha/FastHuBERT?workspace=user-yanghaha](https://wandb.ai/yanghaha/FastHuBERT?workspace=user-yanghaha)  \n\u003c!-- Finetune:\n[S1]() \u0026ensp;[S2]()\u0026ensp; [S3]()\u0026ensp; [S4]()\u0026ensp; [S5]()\u0026ensp; [S6]()\u0026ensp; [S7]()\u0026ensp; [S8]() --\u003e\n\n## Checkpoint for S8 system\nPretrain: [checkpoint280.pt](https://drive.google.com/file/d/1luGaCzw9_LnbWKF0veztGTqgBNokmNqb/view?usp=drive_link)  \nFinetune: [fasthubert_s8.pt](https://drive.google.com/file/d/1nm8p4ouIYs46LYZ846FNMNKfrOIclD63/view?usp=drive_link)  \nDecoding-related files are uploaded to the `decode` folder.\n\n\n## Data preparation\n\u003c!-- Refer to [this](https://github.com/chenxie95/fairseq-asr/blob/main/examples/speech_recognition_sjtu/prep_data_npyfile.py) to extract Fbank features.  private repo 这行需要写吗?--\u003e\n\nRefer to [this](https://github.com/facebookresearch/fairseq/tree/main/examples/hubert#data-preparation) to prepare data.  \n\nFbank extraction code is under [utils](https://github.com/yanghaha0908/FastHuBERT/tree/master/utils). Prepare the info file like `librispeech_example.info`, and execute `run.sh`.\n\n\n## Pre-train a Fast-HuBERT model\n\n```\n$ python fairseq_cli/hydra_train.py \\\n    --config-dir /path/to/FastHuBERT/config/pretrain \\\n    --config-name fasthubert_base_lirbispeech \\\n    common.user_dir=/path/to/FastHuBERT \\\n    task.data=/path/to/data \\\n    task.label_dir=/path/to/labels \\\n    task.labels=[\"phn\"] \\\n    model.label_rate=50 \\\n```\n\n\n## Fine-tune a Fast-HuBERT model with a CTC loss\n\n```\n$ python fairseq_cli/hydra_train.py \\\n    --config-dir /path/to/FastHuBERT/config/finetune \\\n    --config-name base_100h \\\n    common.user_dir=/path/to/FastHuBERT \\\n    task.data=/path/to/data \n    task.label_dir=/path/to/transcriptions \\\n    model.w2v_path=/path/to/checkpoint \\\n```\n\n\n## Decode a FastHuBERT model\n\n### Viterbi decoding\n```\n$ python examples/speech_recognition/new/infer.py \\\n    --config-dir examples/hubert/config/decode \\\n    --config-name infer_viterbi \\\n    common.user_dir=/path/to/FastHuBERT \\\n    common_eval.path=/path/to/checkpoint \\\n    task.data=/path/to/data \\\n    task.normalize=[true|false] \\\n    task._name=fasthubert_pretraining \\\n    dataset.gen_subset=test \\\n```\n\n### KenLM / Fairseq-LM decoding\n```\n$ python examples/speech_recognition/new/infer.py \\\n    --config-dir examples/hubert/config/decode \\\n    --config-name infer_kenlm \\\n    common.user_dir=/path/to/FastHuBERT \\\n    common_eval.path=/path/to/checkpoint \\\n    task.data=/path/to/data \\\n    task.normalize=[true|false] \\\n    task._name=fasthubert_pretraining \\\n    dataset.gen_subset=test \\\n    decoding.lmweight=2 decoding.wordscore=-1 decoding.silweight=0 \\\n    decoding.beam=1500 \\\n    decoding.lexicon=/path/to/lexicon \\\n    decoding.lmpath=/path/to/arpa \\\n```\n\n\n\n## Citation\n```\n@article{yang2023fast,  \n  title={Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning},  \n  author={Yang, Guanrou and Ma, Ziyang and Zheng, Zhisheng and Song, Yakun and Niu, Zhikang and Chen, Xie},  \n  journal={arXiv preprint arXiv:2309.13860},  \n  year={2023}\n}\n```\n\u003cbr\u003e\nContact: yangguanrou@sjtu.edu.cn","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyanghaha0908%2Ffasthubert","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyanghaha0908%2Ffasthubert","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyanghaha0908%2Ffasthubert/lists"}