{"id":19054701,"url":"https://github.com/ssbuild/pre-trained-models","last_synced_at":"2026-01-30T03:49:42.064Z","repository":{"id":117215257,"uuid":"423355743","full_name":"ssbuild/Pre-trained-Models","owner":"ssbuild","description":null,"archived":false,"fork":false,"pushed_at":"2021-11-01T06:08:50.000Z","size":2898,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-01-02T11:11:41.954Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ssbuild.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-01T06:08:40.000Z","updated_at":"2021-11-01T06:08:54.000Z","dependencies_parsed_at":null,"dependency_job_id":"583b877e-2e8c-4fff-8073-d7c7822e3040","html_url":"https://github.com/ssbuild/Pre-trained-Models","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbuild%2FPre-trained-Models","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbuild%2FPre-trained-Models/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbuild%2FPre-trained-Models/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ssbuild%2FPre-trained-Models/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ssbuild","download_url":"https://codeload.github.com/ssbuild/Pre-trained-Models/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240110300,"owners_count":19749274,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T23:39:26.416Z","updated_at":"2026-01-30T03:49:37.023Z","avatar_url":"https://github.com/ssbuild.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# PTMs: Pre-trained-Models in NLP\n### NLP预训练模型的全面总结(持续更新中)\n\n\n\n## 置顶\n\n知乎文章1:  [全面总结！PTMs：NLP预训练模型](https://zhuanlan.zhihu.com/p/115014536)  ➡️➡️ [图片下载](https://github.com/loujie0822/Pre-trained-Models/blob/master/resources/PTMs.jpg)\n\n知乎文章2：[nlp中的预训练语言模型总结](https://zhuanlan.zhihu.com/p/76912493)\n\n知乎文章3：[nlp中的词向量对比](https://zhuanlan.zhihu.com/p/56382372)\n\n\u003cimg src=\"resources/PTMs.jpg\" style=\"zoom:20%;\" /\u003e\n\n## 1、论文汇总：\n\nPTMs-Papers:\n\n1. https://github.com/thunlp/PLMpapers\n2. https://github.com/tomohideshibata/BERT-related-papers\n3. https://github.com/cedrickchee/awesome-bert-nlp\n4. https://bertlang.unibocconi.it/\n5. https://github.com/jessevig/bertviz\n\n## 2. PTMs单模型解读\n\n1. 自监督学习：[Self-Supervised Learning 入门介绍](https://zhuanlan.zhihu.com/p/108625273)\n2. 自监督学习：[Self-supervised Learning 再次入门](https://zhuanlan.zhihu.com/p/108906502)\n3. 词向量总结：[nlp中的词向量对比：word2vec/glove/fastText/elmo/GPT/bert](https://zhuanlan.zhihu.com/p/56382372)\n4. 词向量总结：[从Word Embedding到Bert模型—自然语言处理中的预训练技术发展史](https://zhuanlan.zhihu.com/p/49271699)\n5. ELMo解读：[关于ELMo的若干问题整理记录](https://zhuanlan.zhihu.com/p/82602015)\n6. BERT解读： [Bert时代的创新：Bert应用模式比较及其它](https://zhuanlan.zhihu.com/p/65470719)\n7. XLNET解读：[XLNet:运行机制及和Bert的异同比较](https://zhuanlan.zhihu.com/p/70257427)\n8. XLNET解读：[XLnet：比Bert更强大的预训练模型](https://zhuanlan.zhihu.com/p/71759544)\n9. RoBERTa解读：[RoBERT: 没错，我就是能更强——更大数据规模和仔细调参下的最优BERT](https://zhuanlan.zhihu.com/p/75629127)\n10. 预训练语言模型总结：[nlp中的预训练语言模型总结(单向模型、BERT系列模型、XLNet)](https://zhuanlan.zhihu.com/p/76912493)\n11. 预训练语言模型总结：[8篇论文梳理BERT相关模型进展与反思](https://zhuanlan.zhihu.com/p/81157740)\n12. ELECTRA解读: [ELECTRA: 超越BERT, 19年最佳NLP预训练模型](https://zhuanlan.zhihu.com/p/89763176)\n13. 模型压缩 LayerDrop:[结构剪枝：要个4层的BERT有多难？](https://zhuanlan.zhihu.com/p/93207254)\n14. 模型压缩 BERT-of-Theseus:[bert-of-theseus，一个非常亲民的bert压缩方法](https://zhuanlan.zhihu.com/p/112787764)\n15. 模型压缩 TinyBERT:[比 Bert 体积更小速度更快的 TinyBERT](https://zhuanlan.zhihu.com/p/94359189)\n16. 模型压缩总结：[BERT 瘦身之路：Distillation，Quantization，Pruning](https://zhuanlan.zhihu.com/p/86900556)\n\n（持续更新中...）\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fssbuild%2Fpre-trained-models","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fssbuild%2Fpre-trained-models","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fssbuild%2Fpre-trained-models/lists"}