{"id":13641356,"url":"https://github.com/zyds/transformers-code","last_synced_at":"2025-05-15T04:08:18.294Z","repository":{"id":163528662,"uuid":"639001213","full_name":"zyds/transformers-code","owner":"zyds","description":"手把手带你实战 Huggingface Transformers 课程视频同步更新在B站与YouTube","archived":false,"fork":false,"pushed_at":"2024-07-15T16:29:29.000Z","size":80231,"stargazers_count":2731,"open_issues_count":3,"forks_count":376,"subscribers_count":20,"default_branch":"master","last_synced_at":"2025-04-11T15:57:05.789Z","etag":null,"topics":["huggingface","peft","transformers"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zyds.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-05-10T14:46:52.000Z","updated_at":"2025-04-11T07:41:32.000Z","dependencies_parsed_at":"2024-11-20T11:29:09.054Z","dependency_job_id":null,"html_url":"https://github.com/zyds/transformers-code","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zyds%2Ftransformers-code","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zyds%2Ftransformers-code/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zyds%2Ftransformers-code/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zyds%2Ftransformers-code/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zyds","download_url":"https://codeload.github.com/zyds/transformers-code/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254270656,"owners_count":22042860,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["huggingface","peft","transformers"],"created_at":"2024-08-02T01:01:20.071Z","updated_at":"2025-05-15T04:08:13.283Z","avatar_url":"https://github.com/zyds.png","language":"Jupyter Notebook","readme":"![手把手带你实战Transformers](./imgs/1.png)\n\n# 简介\n\n手把手带你实战Transformers课程的代码仓库\n\n## 代码适配\n\n- torch==2.2.1+cu118\n\n- transformers==4.42.4\n\n- peft==0.11.1\n\n- datasets==2.20.0\n\n- accelerate==0.32.1\n\n- bitsandbytes==0.43.1\n\n- faiss-cpu==1.7.4\n\n- tensorboard==2.14.0\n\n# 课程规划\n\n- 基础入门篇：Transformers入门，从环境安装到各个基础组件的介绍，包括Pipeline、Tokenizer、Model、Datasets、Evaluate、Trainer，并通过一个最基本的文本分类实例将各个模块进行串讲\n\n- 实战演练篇：Transformers实战，通过丰富的实战案例对Transformers在NLP任务中的解决方案进行介绍，包括命名实体识别、机器阅读理解、多项选择、文本相似度、检索式对话机器人、掩码语言模型、因果语言模型、摘要生成、生成式对话机器人\n\n- 高效微调篇：Transformers模型高效微调，以PEFT库为核心，介绍各种常用的参数高效微调方法的原理与实战，包括BitFit、Prompt-tuning、P-tuning、Prefix-Tuning、Lora和IA3\n\n- 低精度训练篇：Transformers模型低精度训练，基于bitsandbytes库，进行模型的低精度训练，包括LlaMA2-7B和ChatGLM2-6B两个模型的多个不同精度训练的实战演练，包括半精度训练、8bit训练、4bit训练（QLoRA）\n\n- 分布式训练篇：Transformers模型分布式训练，基于accelerate库讲解transformers模型的分布式训练解决方案，介绍分布式训练的基本原理以及accelerate库的基本使用方式，包括与Deepspeed框架的集成\n\n- 对齐训练篇: ...\n\n- 性能优化篇: ...\n\n- 系统演示篇: ...\n\n\n# 课程地址\n\n课程视频发布在B站与YouTube，代码与视频会逐步进行更新，目前课程主要更新在B站，YouTube后续会持续更新\n\n- [Bilibili](https://www.bilibili.com/video/BV1ma4y1g791)\n\n- [YouTube](https://www.youtube.com/@lunatic-zzz)\n\n## Transformers 基础入门篇 (已更新完成)\n\n- 01- 基础知识与环境安装\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1ma4y1g791) | [YouTube](https://www.youtube.com/watch?v=ddCfxkCh-O8)\n\n- 02 基础组件之 Pipeline | \n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1ta4y1g7bq) | [YouTube](https://www.youtube.com/watch?v=Xeu3qFTP9qY\u0026t=7s)\n\n- 03 基础组件之 Tokenizer\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1NX4y1177c) | [YouTube](https://www.youtube.com/watch?v=G4JmQu-VWrU)\n\n- 04 基础组件之 Model(上) 基本使用\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1KM4y1q7Js) | [YouTube](https://www.youtube.com/watch?v=xK-6VcLqa94)\n\n- 04 基础组件之 Model(下) BERT文本分类代码实例\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV18T411t7h6) | [YouTube](https://www.youtube.com/watch?v=nkwOQQDCDvc)\n\n- 05 基础组件之 Datasets\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1Ph4y1b76w) | [YouTube](https://www.youtube.com/watch?v=LRhcUjbSOEk)\n\n- 06 基础组件之 Evaluate\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1uk4y1W7tK) | [YouTube](https://www.youtube.com/watch?v=tpE2bleqk6A)\n\n- 07 基础组件之 Trainer\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1KX4y1a7Jk) | [YouTube](https://www.youtube.com/watch?v=YzS-BvHeSGE)\n\n## Transformers 实战演练篇 (已更新完成)\n\n- 08 基于 Transformers的 NLP解决方案\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV18N411C71F) | [YouTube](https://www.youtube.com/watch?v=WRBPd86T1Fc)\n\n- 09 实战演练之 命名实体识别\n   \n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1gW4y197CT) | [YouTube](https://www.youtube.com/watch?v=3xQR-7sly_I)\n\n- 10 实战演练之 机器阅读理解（上，过长截断策略）\n   \n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1rs4y1k7FX) | [YouTube](https://www.youtube.com/watch?v=-rzKZIpELOk)\n\n- 10 实战演练之 机器阅读理解（下，滑动窗口策略）\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1uN411D7oy) | [YouTube](https://www.youtube.com/watch?v=oTlpbISOkaE)\n\n- 11 实战演练之 多项选择 \n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1FM4y1E77w) | [YouTube](https://www.youtube.com/watch?v=xHM1PjIihJs)\n\n- 12 实战演练之 文本相似度（上，基于交互策略） \n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1Tm4y1J7EF) | [YouTube](https://www.youtube.com/watch?v=SElN5_LqZls)\n\n- 12 实战演练之 文本相似度（下，基于匹配策略） \n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV13P411C7UD) | [YouTube](https://www.youtube.com/watch?v=7zxNXBBDqwA)\n\n- 13 实战演练之 检索式对话机器人\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1Lh4y117KJ) | [YouTube](https://www.youtube.com/watch?v=gHOUoqqXb8I)\n\n- 14 实战演练之 预训练模型\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1B44y1c7x2) | [YouTube](https://www.youtube.com/watch?v=jHRo2qgtE7Y)\n\n- 15 实战演练篇之 文本摘要（上，基于T5模型）\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1Kp4y137ar) | [YouTube](https://www.youtube.com/watch?v=5AusJJbpWaA)\n\n- 15 实战演练篇之 文本摘要（下，基于GLM模型）\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1CF411y7hw) | [YouTube](https://www.youtube.com/watch?v=BK2wUNZZbRg)\n\n- 16 实战演练篇之 生成式对话机器人（基于Bloom）\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV11r4y197Ht) | [YouTube](https://www.youtube.com/watch?v=McE0XUG5Gw4)\n\n## Transformers 参数高效微调篇 (已更新完成)\n\n- 17 参数高效微调与BitFit实战\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1Xu4y1k7Ls) | [YouTube](https://www.youtube.com/watch?v=ynBE40yVTSk)\n\n- 18 Prompt-Tuning 原理与实战\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1Fu4y1C7tJ) | [YouTube](https://www.youtube.com/watch?v=aAbVsm6tWIM)\n\n- 19 P-Tuning 原理与实战\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV17V411N7Ld) | [YouTube](https://www.youtube.com/watch?v=xNC12IhNuw4)\n\n- 20 Prefix-Tuning 原理与实战\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1Ru411g7Qa) | [YouTube](https://www.youtube.com/watch?v=EYd-sJHXCio)\n\n- 21 LoRA 原理与实战\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV13w411y7fq) | [YouTube](https://www.youtube.com/watch?v=-xVJtu9pyoA)\n\n- 22 IA3 原理与实战\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1Y8411k7yD) | [YouTube](https://www.youtube.com/watch?v=WOrHqOkMqxY)\n\n- 23 PEFT 进阶操作\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1YH4y1o7rg) | [YouTube](https://www.youtube.com/watch?v=KJljAinRXs8)\n   \n\n## Transformers 低精度训练篇（已更新完成）\n\n- 24 低精度训练与模型下载\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1y34y1M7t1) | [YouTube](https://www.youtube.com/watch?v=mWiXtVs9ZzY)\n\n- 25 半精度模型训练（上，基于LLaMA2的半精度模型训练）\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1CB4y1R78v) | [YouTube](https://www.youtube.com/watch?v=Is4T8u1Astk)\n\n- 25 半精度模型训练（下，基于ChatGLM3的半精度模型训练）\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1aw411M7Cv) | [YouTube](https://www.youtube.com/watch?v=8SmlpNuY_pU)\n\n- 26 量化与8bit模型训练\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1EN411g7Yn) | [YouTube](https://www.youtube.com/watch?v=XKImkaWv7-Y)\n\n- 27 4bit量化与QLoRA模型训练\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1DQ4y1t7e8) | [YouTube](https://www.youtube.com/watch?v=CY0jTExZlKE)\n\n## Transformers 分布式训练篇（已更新完成）\n\n- 28 分布式训练基础与环境配置\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1cK4y1z7Mv) | [YouTube](https://www.youtube.com/watch?v=eNOoIlUCX6Q)\n\n- 29 Data Parallel原理与应用\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1qN4y1n7iG) | [YouTube](https://www.youtube.com/watch?v=WiRpMjHL79s)\n\n- 30 Distributed Data Parallel原理与应用\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1wS421w7ug) | [YouTube](https://www.youtube.com/watch?v=hoa-AIE_yxk)\n\n- 31 Accelerate 分布式训练入门\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV12Z421t74R) | [YouTube](https://www.youtube.com/watch?v=eDaT_bBoiJ4)\n\n- 32 Accelerate 使用进阶（上）\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1vq421F7Cf) | [YouTube](https://www.youtube.com/watch?v=IhpuxmYoKgI)\n\n- 32 Accelerate 使用进阶（下）\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1Lp421975B) | [YouTube](https://www.youtube.com/watch?v=WmZ94u9QDME)\n\n- 33 Accelerate + Deepspeed\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1hb421E7WY) | [YouTube](https://www.youtube.com/watch?v=Vegqv1PDboY)\n\n## Transformers 番外技能篇\n\n- 基于Optuna的Transformers模型自动调参\n\n   - 视频地址：[Bilibili](https://www.bilibili.com/video/BV1NN4y1S7i8) | [YouTube](https://www.youtube.com/watch?v=ugiAW2ukZZw)\n\n# Star History\n\n[![Star History Chart](https://api.star-history.com/svg?repos=zyds/transformers-code\u0026type=Date)](https://star-history.com/#zyds/transformers-code\u0026Date)\n\n\n# 请作者喝杯奶茶\n\n![](./imgs/wx.jpg)","funding_links":[],"categories":["Summary","Transformer库与优化"],"sub_categories":["大语言对话模型及数据"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzyds%2Ftransformers-code","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzyds%2Ftransformers-code","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzyds%2Ftransformers-code/lists"}