{"id":18533677,"url":"https://github.com/lonepatient/mobilebert_pytorch","last_synced_at":"2025-07-14T01:14:19.240Z","repository":{"id":106807400,"uuid":"264956600","full_name":"lonePatient/MobileBert_PyTorch","owner":"lonePatient","description":"MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices","archived":false,"fork":false,"pushed_at":"2020-05-19T00:14:51.000Z","size":234,"stargazers_count":67,"open_issues_count":3,"forks_count":11,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-10T09:46:07.238Z","etag":null,"topics":["bert","glue","mobilebert","pytorch","text-classification"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lonePatient.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-18T14:00:23.000Z","updated_at":"2025-02-28T13:00:12.000Z","dependencies_parsed_at":"2023-07-23T22:15:39.666Z","dependency_job_id":null,"html_url":"https://github.com/lonePatient/MobileBert_PyTorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/lonePatient/MobileBert_PyTorch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lonePatient%2FMobileBert_PyTorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lonePatient%2FMobileBert_PyTorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lonePatient%2FMobileBert_PyTorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lonePatient%2FMobileBert_PyTorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lonePatient","download_url":"https://codeload.github.com/lonePatient/MobileBert_PyTorch/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lonePatient%2FMobileBert_PyTorch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265228987,"owners_count":23731092,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bert","glue","mobilebert","pytorch","text-classification"],"created_at":"2024-11-06T19:12:37.101Z","updated_at":"2025-07-14T01:14:19.230Z","avatar_url":"https://github.com/lonePatient.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## MobileBERT_pytorch\n\nThis repository contains a PyTorch implementation of the **MobileBERT** model from the paper \n\n[MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices](https://arxiv.org/pdf/2004.02984.pdf)\n\nby Zhiqing Sun1∗, Hongkun Yu2, Xiaodan Song....\n\n## Dependencies\n\n- pytorch=1.10\n- cuda=9.0\n- cudnn=7.5\n- scikit-learn\n- sentencepiece\n- tokenizers\n\n## Download Pre-trained Models of English\n\nOfficial download links: [google mobilebert](https://github.com/google-research/google-research/tree/master/mobilebert)\n\n## Fine-tuning\n\n１. Place `config.json` and `vocab.txt` into the `prev_trained_model/mobilebert` directory.\nexample:\n```text\n├── prev_trained_model\n|  └── mobilebert\n|  |  └── pytorch_model.bin\n|  |  └── config.json\n|  |  └── vocab.txt\n```\n2．convert mobilebert tf checkpoint to pytorch\n```python\npython convert_mobilebert_tf_checkpoint_to_pytorch.py \\\n    --tf_checkpoint_path=./prev_trained_model/mobilebert \\\n    --mobilebert_config_file=./prev_trained_model/mobilebert/config.json \\\n    --pytorch_dump_path=./prev_trained_model/mobilebert/pytorch_model.bin\n```\nThe [General Language Understanding Evaluation (GLUE) benchmark](https://gluebenchmark.com/) is a collection of nine sentence- or sentence-pair language understanding tasks for evaluating and analyzing natural language understanding systems.\n\nBefore running anyone of these GLUE tasks you should download the [GLUE data](https://gluebenchmark.com/tasks) by running [this script](https://gist.github.com/W4ngatang/60c2bdb54d156a41194446737ce03e2e) and unpack it to some directory $DATA_DIR.\n\n3．run `sh scripts/run_classifier_sst2.sh`to fine tuning mobilebert model\n\n## Result\n\nPerformance of MobileBert on GLUE benchmark results using a single-model setup on **dev**:\n\n|  | Cola| Sst-2| Sts-b|\n| :------- | :---------: | :---------: | :---------: |\n| metric | matthews_corrcoef |accuracy | pearson |\n| mobilebert | 0.5837 | 0.922 | 0.8839 |\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flonepatient%2Fmobilebert_pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flonepatient%2Fmobilebert_pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flonepatient%2Fmobilebert_pytorch/lists"}