{"id":13595045,"url":"https://github.com/Tongjilibo/bert4torch","last_synced_at":"2025-04-09T10:32:48.815Z","repository":{"id":37463402,"uuid":"469136195","full_name":"Tongjilibo/bert4torch","owner":"Tongjilibo","description":"An elegent pytorch implement of transformers","archived":false,"fork":false,"pushed_at":"2025-04-01T15:52:30.000Z","size":11232,"stargazers_count":1284,"open_issues_count":27,"forks_count":164,"subscribers_count":15,"default_branch":"master","last_synced_at":"2025-04-08T10:52:38.860Z","etag":null,"topics":["belle","bert","bert4keras","bert4torch","chatglm","large-language-models","llama","llm","named-entity-recognition","nlp","pytorch","relation-extraction","seq2seq","text-classification","transformers"],"latest_commit_sha":null,"homepage":"https://bert4torch.readthedocs.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Tongjilibo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-12T16:23:44.000Z","updated_at":"2025-04-07T07:36:26.000Z","dependencies_parsed_at":"2022-07-14T08:08:56.266Z","dependency_job_id":"bab07c8e-01f3-43ad-8bbb-6026469d52b0","html_url":"https://github.com/Tongjilibo/bert4torch","commit_stats":{"total_commits":1217,"total_committers":10,"mean_commits":121.7,"dds":"0.21117502054231718","last_synced_commit":"ebd53d61c28295c9c3f9cda948b1a75c0ee5473b"},"previous_names":[],"tags_count":49,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tongjilibo%2Fbert4torch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tongjilibo%2Fbert4torch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tongjilibo%2Fbert4torch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tongjilibo%2Fbert4torch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Tongjilibo","download_url":"https://codeload.github.com/Tongjilibo/bert4torch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248020593,"owners_count":21034459,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["belle","bert","bert4keras","bert4torch","chatglm","large-language-models","llama","llm","named-entity-recognition","nlp","pytorch","relation-extraction","seq2seq","text-classification","transformers"],"created_at":"2024-08-01T16:01:43.008Z","updated_at":"2025-04-09T10:32:43.797Z","avatar_url":"https://github.com/Tongjilibo.png","language":"Python","funding_links":[],"categories":["Python","Transformer库与优化"],"sub_categories":[],"readme":"![bert4torch](./docs/pics/bert4torch.png)\r\n\r\n[![licence](https://img.shields.io/github/license/Tongjilibo/bert4torch.svg?maxAge=3600)](https://github.com/Tongjilibo/bert4torch/blob/master/LICENSE)\r\n[![GitHub release](https://img.shields.io/github/release/Tongjilibo/bert4torch.svg?maxAge=3600)](https://github.com/Tongjilibo/bert4torch/releases)\r\n[![PyPI](https://img.shields.io/pypi/v/bert4torch?label=pypi%20package)](https://pypi.org/project/bert4torch/)\r\n[![PyPI - Downloads](https://img.shields.io/pypi/dm/bert4torch)](https://pypistats.org/packages/bert4torch)\r\n[![GitHub stars](https://img.shields.io/github/stars/Tongjilibo/bert4torch?style=social)](https://github.com/Tongjilibo/bert4torch)\r\n[![GitHub Issues](https://img.shields.io/github/issues/Tongjilibo/bert4torch.svg)](https://github.com/Tongjilibo/bert4torch/issues)\r\n[![contributions welcome](https://img.shields.io/badge/contributions-welcome-brightgreen.svg?style=flat)](https://github.com/Tongjilibo/bert4torch/issues)\r\n[![Generic badge](https://img.shields.io/badge/wechat-join-green.svg?logo=wechat)](https://github.com/Tongjilibo/bert4torch/blob/master/docs/pics/wechat_group.jpg)\r\n\r\n[Documentation](https://bert4torch.readthedocs.io) |\r\n[Torch4keras](https://github.com/Tongjilibo/torch4keras) |\r\n[Examples](https://github.com/Tongjilibo/bert4torch/blob/master/examples) |\r\n[build_MiniLLM_from_scratch](https://github.com/Tongjilibo/build_MiniLLM_from_scratch) |\r\n[bert4vector](https://github.com/Tongjilibo/bert4vector)\r\n\r\n\r\n## 目录\r\n- [目录](#目录)\r\n- [1. 下载安装](#1-下载安装)\r\n- [2. 功能](#2-功能)\r\n- [3. 快速上手](#3-快速上手)\r\n  - [3.1 上手教程](#31-上手教程)\r\n  - [3.2 命令行快速部署大模型服务](#32-命令行快速部署大模型服务)\r\n- [4. 版本和更新历史](#4-版本和更新历史)\r\n  - [4.1 版本历史](#41-版本历史)\r\n  - [4.2 更新历史](#42-更新历史)\r\n- [5. 预训练权重](#5-预训练权重)\r\n- [6. 鸣谢](#6-鸣谢)\r\n- [7. 引用](#7-引用)\r\n- [8. 其他](#8-其他)\r\n  \r\n\r\n## 1. 下载安装\r\n\r\n安装稳定版\r\n\r\n```shell\r\npip install bert4torch\r\n```\r\n\r\n安装最新版\r\n\r\n```shell\r\npip install git+https://github.com/Tongjilibo/bert4torch\r\n```\r\n\r\n- **注意事项**：pip包的发布慢于git上的开发版本，git clone**注意引用路径**，注意权重是否需要转换\r\n- **测试用例**：`git clone https://github.com/Tongjilibo/bert4torch`，修改example中的预训练模型文件路径和数据路径即可启动脚本\r\n- **自行训练**：针对自己的数据，修改相应的数据处理代码块\r\n- **开发环境**：原使用`torch==1.10`版本进行开发，现已切换到`torch2.0`开发，如其他版本遇到不适配，欢迎反馈\r\n\r\n## 2. 功能\r\n- **LLM模型**: 加载chatglm、llama、 baichuan、ziya、bloom等开源大模型权重进行推理和微调，命令行一行部署大模型\r\n- **核心功能**：加载bert、roberta、albert、xlnet、nezha、bart、RoFormer、RoFormer_V2、ELECTRA、GPT、GPT2、T5、GAU-alpha、ERNIE等预训练权重继续进行finetune、并支持在bert基础上灵活定义自己模型\r\n- [**丰富示例**](https://github.com/Tongjilibo/bert4torch/blob/master/examples/)：包含[llm](https://github.com/Tongjilibo/bert4torch/blob/master/examples/llm)、[pretrain](https://github.com/Tongjilibo/bert4torch/blob/master/examples/pretrain)、[sentence_classfication](https://github.com/Tongjilibo/bert4torch/blob/master/examples/sentence_classfication)、[sentence_embedding](https://github.com/Tongjilibo/bert4torch/tree/master/examples/sentence_embedding)、[sequence_labeling](https://github.com/Tongjilibo/bert4torch/blob/master/examples/sequence_labeling)、[relation_extraction](https://github.com/Tongjilibo/bert4torch/blob/master/examples/relation_extraction)、[seq2seq](https://github.com/Tongjilibo/bert4torch/blob/master/examples/seq2seq)、[serving](https://github.com/Tongjilibo/bert4torch/blob/master/examples/serving/)等多种解决方案\r\n- **实验验证**：已在公开数据集实验验证，使用如下[examples数据集](https://github.com/Tongjilibo/bert4torch/blob/master/data/README.md)和[实验指标](https://github.com/Tongjilibo/bert4torch/blob/master/examples/Experiments.md)\r\n- **易用trick**：集成了常见的[trick](https://github.com/Tongjilibo/bert4torch/blob/master/examples/training_trick)，即插即用\r\n- **其他特性**：[加载transformers库模型](https://github.com/Tongjilibo/bert4torch/blob/master//tutorials/tutorials_load_transformers_model.py)一起使用；调用方式简洁高效；有训练进度条动态展示；配合torchinfo打印参数量；默认Logger和Tensorboard简便记录训练过程；自定义fit过程，满足高阶需求\r\n- **训练过程**：\r\n\r\n  ![训练过程](./docs/pics/training_process.gif)\r\n\r\n|          功能                | bert4torch |  transformers | 备注 |\r\n|-----------------------------|------------|----------------|--------|\r\n|训练进度条                     | ✅         |      ✅        |进度条打印loss和定义的metrics|\r\n|分布式训练dp/ddp               | ✅         |      ✅        |torch自带dp/ddp|\r\n|各类callbacks                 | ✅         |      ✅        |日志/tensorboard/earlystop/wandb等|\r\n|大模型推理，stream/batch输出    | ✅         |      ✅        |各个模型是通用的，无需单独维护脚本|\r\n|大模型微调                     | ✅         |      ✅        |lora依赖peft库，pv2自带|\r\n|丰富tricks                    | ✅         |      ❌        |对抗训练等tricks即插即用|\r\n|代码简洁易懂，自定义空间大        | ✅         |      ❌        |代码复用度高, keras代码训练风格|\r\n|仓库的维护能力/影响力/使用量/兼容性| ❌         |      ✅        |目前仓库个人维护|\r\n|一键部署大模型                   |\r\n\r\n## 3. 快速上手\r\n\r\n### 3.1 上手教程\r\n- [Quick-Start](https://bert4torch.readthedocs.io/en/latest//Quick-Start.html)\r\n- [快速上手教程](https://github.com/Tongjilibo/bert4torch/blob/master//tutorials/README.md)，[教程示例](https://github.com/Tongjilibo/bert4torch/blob/master//tutorials)，[实战示例](https://github.com/Tongjilibo/bert4torch/blob/master/examples)\r\n- [bert4torch介绍(知乎)](https://zhuanlan.zhihu.com/p/486329434)，[bert4torch快速上手(知乎)](https://zhuanlan.zhihu.com/p/508890807)，[bert4torch又双叒叕更新啦(知乎)](https://zhuanlan.zhihu.com/p/560885427?)\r\n\r\n### 3.2 命令行快速部署大模型服务\r\n- 本地 / 联网加载\r\n  ```shell\r\n  # 联网下载全部文件\r\n  bert4torch-llm-server --checkpoint_path Qwen2-0.5B-Instruct\r\n\r\n  # 加载本地大模型，联网下载bert4torch_config.json\r\n  bert4torch-llm-server --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct --config_path Qwen/Qwen2-0.5B-Instruct\r\n\r\n  # 加载本地大模型，且bert4torch_config.json已经下载并放于同名目录下\r\n  bert4torch-llm-server --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct\r\n  ```\r\n- 命令行 / gradio网页 / openai_api\r\n  ```shell\r\n  # 命令行\r\n  bert4torch-llm-server --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct --mode cli\r\n\r\n  # gradio网页\r\n  bert4torch-llm-server --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct --mode gradio\r\n\r\n  # openai_api\r\n  bert4torch-llm-server --checkpoint_path /data/pretrain_ckpt/Qwen/Qwen2-0.5B-Instruct --mode openai\r\n  ```\r\n- 命令行聊天示例\r\n  ![命令行聊天](./docs/pics/cli_chat.gif)\r\n\r\n## 4. 版本和更新历史\r\n### 4.1 版本历史\r\n|更新日期| bert4torch | torch4keras | 版本说明 |\r\n|------| ---------------- | ----------------- |----------- |\r\n|20240928| 0.5.4          | 0.2.7 | 【新功能】增加deepseek系列、MiniCPM、MiniCPMV、llama3.2、Qwen2.5；支持device_map=auto;【修复】修复batch_generate和n\u003e1的bug|\r\n|20240814| 0.5.3          | 0.2.6 | 【新功能】增加llama3.1/Yi1.5；自动选择从hfmirror下载；支持命令行参数`bert4torch-llm-server`|\r\n|20240801| 0.5.2          | 0.2.5 | 【新功能】chatglm/qwen系列支持function call调用, 增加internlm2系列；【小优化】简化pipeline中chat demo的调用，generate的终止token元素允许为列表, 统一rope_scaling参数名，增加rope衍生类；【bug】修复flash_attn2的推理bug, 修复bart的tie_word_embedding的bug|\r\n\r\n[更多版本](https://github.com/Tongjilibo/bert4torch/blob/master/docs/Update.md)\r\n\r\n### 4.2 更新历史\r\n\r\n[更多历史](https://github.com/Tongjilibo/bert4torch/blob/master/docs/History.md)\r\n\r\n## 5. 预训练权重\r\n- 预训练模型支持多种代码加载方式\r\n  ```python\r\n  from bert4torch.models import build_transformer_model\r\n\r\n  # 1. 仅指定config_path: 从头初始化模型结构, 不加载预训练模型\r\n  model = build_transformer_model('./model/bert4torch_config.json')\r\n\r\n  # 2. 仅指定checkpoint_path: \r\n  ## 2.1 文件夹路径: 自动寻找路径下的*.bin/*.safetensors权重文件 + 需把bert4torch_config.json下载并放于该目录下\r\n  model = build_transformer_model(checkpoint_path='./model')\r\n\r\n  ## 2.2 文件路径/列表: 文件路径即权重路径/列表, bert4torch_config.json会从同级目录下寻找\r\n  model = build_transformer_model(checkpoint_path='./pytorch_model.bin')\r\n\r\n  ## 2.3 model_name: hf上预训练权重名称, 会自动下载hf权重以及bert4torch_config.json文件\r\n  model = build_transformer_model(checkpoint_path='bert-base-chinese')\r\n\r\n  # 3. 同时指定config_path和checkpoint_path(本地路径名或model_name排列组合): \r\n  #    本地路径从本地加载，pretrained_model_name会联网下载\r\n  config_path = './model/bert4torch_config.json'  # 或'bert-base-chinese'\r\n  checkpoint_path = './model/pytorch_model.bin'  # 或'bert-base-chinese'\r\n  model = build_transformer_model(config_path, checkpoint_path)\r\n  ```\r\n\r\n- 预训练权重链接和bert4torch_config.json\r\n\r\n| 模型分类| 模型名称 | 权重来源| 权重链接/checkpoint_path | config_path|\r\n| ----- | ----- | ----- | ----- | ----- |\r\n| bert| bert-base-chinese| google-bert | [`bert-base-chinese`](https://huggingface.co/bert-base-chinese) | [`bert-base-chinese`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/bert-base-chinese/bert4torch_config.json)|\r\n|     | [chinese_L-12_H-768_A-12](https://github.com/google-research/bert) | 谷歌 | [tf权重](https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip)\u003cbr\u003e[`Tongjilibo/bert-chinese_L-12_H-768_A-12`](https://huggingface.co/Tongjilibo/bert-chinese_L-12_H-768_A-12) | |\r\n|     | [chinese-bert-wwm-ext](https://github.com/ymcui/Chinese-BERT-wwm) | HFL | [`hfl/chinese-bert-wwm-ext`](https://huggingface.co/hfl/chinese-bert-wwm-ext)| [`hfl/chinese-bert-wwm-ext`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/hfl/chinese-bert-wwm-ext/bert4torch_config.json) |\r\n|     | bert-base-multilingual-cased| google-bert | [`bert-base-multilingual-cased`](https://huggingface.co/bert-base-multilingual-cased) | [`bert-base-multilingual-cased`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/bert-base-multilingual-cased/bert4torch_config.json) |\r\n|     | [MacBERT](https://github.com/ymcui/MacBERT) | HFL| [`hfl/chinese-macbert-base`](https://huggingface.co/hfl/chinese-macbert-base)\u003cbr\u003e[`hfl/chinese-macbert-large`](https://huggingface.co/hfl/chinese-macbert-large) |[`hfl/chinese-macbert-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/hfl/chinese-macbert-base/bert4torch_config.json)\u003cbr\u003e[`hfl/chinese-macbert-large`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/hfl/chinese-macbert-large/bert4torch_config.json)|\r\n|     | [WoBERT](https://github.com/ZhuiyiTechnology/WoBERT)| 追一科技|[`junnyu/wobert_chinese_base`](https://huggingface.co/junnyu/wobert_chinese_base)，[`junnyu/wobert_chinese_plus_base`](https://huggingface.co/junnyu/wobert_chinese_plus_base) |[`junnyu/wobert_chinese_base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/junnyu/wobert_chinese_base/bert4torch_config.json)\u003cbr\u003e[`junnyu/wobert_chinese_plus_base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/junnyu/wobert_chinese_plus_base/bert4torch_config.json)|\r\n|roberta|[chinese-roberta-wwm-ext](https://github.com/ymcui/Chinese-BERT-wwm) | HFL | [`hfl/chinese-roberta-wwm-ext`](https://huggingface.co/hfl/chinese-roberta-wwm-ext)\u003cbr\u003e[`hfl/chinese-roberta-wwm-ext-large`](https://huggingface.co/hfl/chinese-roberta-wwm-ext-large)\u003cbr\u003e(large的mlm权重是随机初始化) |[`hfl/chinese-roberta-wwm-ext`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/hfl/chinese-roberta-wwm-ext/bert4torch_config.json)\u003cbr\u003e[`hfl/chinese-roberta-wwm-ext-large`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/hfl/chinese-roberta-wwm-ext-large/bert4torch_config.json) |\r\n|     |[roberta-small/tiny](https://github.com/ZhuiyiTechnology/pretrained-models)| 追一科技| [`Tongjilibo/chinese_roberta_L-4_H-312_A-12`](https://huggingface.co/Tongjilibo/chinese_roberta_L-4_H-312_A-12)\u003cbr\u003e[`Tongjilibo/chinese_roberta_L-6_H-384_A-12`](https://huggingface.co/Tongjilibo/chinese_roberta_L-6_H-384_A-12) | |\r\n|     |[roberta-base](https://github.com/facebookresearch/fairseq/tree/main/examples/roberta)| FacebookAI | [`roberta-base`](https://huggingface.co/roberta-base) | [`roberta-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/roberta-base/bert4torch_config.json) |\r\n|     | [guwenbert](https://github.com/Ethan-yt/guwenbert) | ethanyt |[`ethanyt/guwenbert-base`](https://huggingface.co/ethanyt/guwenbert-base) | [`ethanyt/guwenbert-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/ethanyt/guwenbert-base/bert4torch_config.json)|\r\n| albert|[albert_zh](https://github.com/brightmart/albert_zh)\u003cbr\u003e[albert_pytorch](https://github.com/lonePatient/albert_pytorch)| brightmart| [`voidful/albert_chinese_tiny`](https://huggingface.co/voidful/albert_chinese_tiny)\u003cbr\u003e[`voidful/albert_chinese_small`](https://huggingface.co/voidful/albert_chinese_small)\u003cbr\u003e[`voidful/albert_chinese_base`](https://huggingface.co/voidful/albert_chinese_base)\u003cbr\u003e[`voidful/albert_chinese_large`](https://huggingface.co/voidful/albert_chinese_large)\u003cbr\u003e[`voidful/albert_chinese_xlarge`](https://huggingface.co/voidful/albert_chinese_xlarge)\u003cbr\u003e[`voidful/albert_chinese_xxlarge`](https://huggingface.co/voidful/albert_chinese_xxlarge) | [`voidful/albert_chinese_tiny`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/voidful/albert_chinese_tiny/bert4torch_config.json)\u003cbr\u003e[`voidful/albert_chinese_small`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/voidful/albert_chinese_small/bert4torch_config.json)\u003cbr\u003e[`voidful/albert_chinese_base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/voidful/albert_chinese_base/bert4torch_config.json)\u003cbr\u003e[`voidful/albert_chinese_large`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/voidful/albert_chinese_large/bert4torch_config.json)\u003cbr\u003e[`voidful/albert_chinese_xlarge`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/voidful/albert_chinese_xlarge/bert4torch_config.json)\u003cbr\u003e[`voidful/albert_chinese_xxlarge`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/voidful/albert_chinese_xxlarge/bert4torch_config.json)|\r\n| nezha|[NEZHA](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/NEZHA-PyTorch)\u003cbr\u003e[NeZha_Chinese_PyTorch](https://github.com/lonePatient/NeZha_Chinese_PyTorch) | huawei_noah| [`sijunhe/nezha-cn-base`](https://huggingface.co/sijunhe/nezha-cn-base)\u003cbr\u003e[`sijunhe/nezha-cn-large`](https://huggingface.co/sijunhe/nezha-cn-large)\u003cbr\u003e[`sijunhe/nezha-base-wwm`](https://huggingface.co/sijunhe/nezha-base-wwm)\u003cbr\u003e[`sijunhe/nezha-large-wwm`](https://huggingface.co/sijunhe/nezha-large-wwm) | [`sijunhe/nezha-cn-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/sijunhe/nezha-cn-base/bert4torch_config.json)\u003cbr\u003e[`sijunhe/nezha-cn-large`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/sijunhe/nezha-cn-large/bert4torch_config.json)\u003cbr\u003e[`sijunhe/nezha-base-wwm`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/sijunhe/nezha-base-wwm/bert4torch_config.json)\u003cbr\u003e[`sijunhe/nezha-large-wwm`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/sijunhe/nezha-large-wwm/bert4torch_config.json)|\r\n|      |[nezha_gpt_dialog](https://github.com/bojone/nezha_gpt_dialog)| bojone| [`Tongjilibo/nezha_gpt_dialog`](https://huggingface.co/Tongjilibo/nezha_gpt_dialog) | |\r\n| xlnet|[Chinese-XLNet](https://github.com/ymcui/Chinese-XLNet) | HFL | [`hfl/chinese-xlnet-base`](https://huggingface.co/hfl/chinese-xlnet-base) | [`hfl/chinese-xlnet-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/hfl/chinese-xlnet-base/bert4torch_config.json)|\r\n||[tranformer_xl](https://github.com/kimiyoung/transformer-xl)|huggingface|[`transfo-xl/transfo-xl-wt103`](https://huggingface.co/transfo-xl/transfo-xl-wt103)|[`transfo-xl/transfo-xl-wt103`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/transfo-xl/transfo-xl-wt103/bert4torch_config.json)|\r\n|deberta| [Erlangshen-DeBERTa-v2](https://github.com/IDEA-CCNL/Fengshenbang-LM) | IDEA | [`IDEA-CCNL/Erlangshen-DeBERTa-v2-97M-Chinese`](https://huggingface.co/IDEA-CCNL/Erlangshen-DeBERTa-v2-97M-Chinese)\u003cbr\u003e[`IDEA-CCNL/Erlangshen-DeBERTa-v2-320M-Chinese`](https://huggingface.co/IDEA-CCNL/Erlangshen-DeBERTa-v2-320M-Chinese)\u003cbr\u003e[`IDEA-CCNL/Erlangshen-DeBERTa-v2-710M-Chinese`](https://huggingface.co/IDEA-CCNL/Erlangshen-DeBERTa-v2-710M-Chinese) |[`IDEA-CCNL/Erlangshen-DeBERTa-v2-97M-Chinese`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/IDEA-CCNL/Erlangshen-DeBERTa-v2-97M-Chinese/bert4torch_config.json)\u003cbr\u003e[`IDEA-CCNL/Erlangshen-DeBERTa-v2-320M-Chinese`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/IDEA-CCNL/Erlangshen-DeBERTa-v2-320M-Chinese/bert4torch_config.json)\u003cbr\u003e[`IDEA-CCNL/Erlangshen-DeBERTa-v2-710M-Chinese`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/IDEA-CCNL/Erlangshen-DeBERTa-v2-710M-Chinese/bert4torch_config.json) |\r\n| electra|[Chinese-ELECTRA](https://github.com/ymcui/Chinese-ELECTRA) | HFL |[`hfl/chinese-electra-base-discriminator`](https://huggingface.co/hfl/chinese-electra-base-discriminator) |[`hfl/chinese-electra-base-discriminator`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/hfl/chinese-electra-base-discriminator/bert4torch_config.json)|\r\n| ernie|[ernie](https://github.com/PaddlePaddle/ERNIE) | 百度文心| [`nghuyong/ernie-1.0-base-zh`](https://huggingface.co/nghuyong/ernie-1.0-base-zh)\u003cbr\u003e[`nghuyong/ernie-3.0-base-zh`](https://huggingface.co/nghuyong/ernie-3.0-base-zh)| [`nghuyong/ernie-1.0-base-zh`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/nghuyong/ernie-1.0-base-zh/bert4torch_config.json)\u003cbr\u003e[`nghuyong/ernie-3.0-base-zh`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/nghuyong/ernie-3.0-base-zh/bert4torch_config.json)|\r\n| roformer|[roformer](https://github.com/ZhuiyiTechnology/roformer)| 追一科技| [`junnyu/roformer_chinese_base`](https://huggingface.co/junnyu/roformer_chinese_base) |[`junnyu/roformer_chinese_base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/junnyu/roformer_chinese_base/bert4torch_config.json) |\r\n|         |[roformer_v2](https://github.com/ZhuiyiTechnology/roformer-v2) | 追一科技| [`junnyu/roformer_v2_chinese_char_base`](https://huggingface.co/junnyu/roformer_v2_chinese_char_base)|[`junnyu/roformer_v2_chinese_char_base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/junnyu/roformer_v2_chinese_char_base/bert4torch_config.json) |\r\n| simbert|[simbert](https://github.com/ZhuiyiTechnology/simbert) | 追一科技|[`Tongjilibo/simbert-chinese-base`](https://huggingface.co/Tongjilibo/simbert-chinese-base)\u003cbr\u003e[`Tongjilibo/simbert-chinese-small`](https://huggingface.co/Tongjilibo/simbert-chinese-small)\u003cbr\u003e[`Tongjilibo/simbert-chinese-tiny`](https://huggingface.co/Tongjilibo/simbert-chinese-tiny) | |\r\n|        |[simbert_v2/roformer-sim](https://github.com/ZhuiyiTechnology/roformer-sim) | 追一科技|[`junnyu/roformer_chinese_sim_char_base`](https://huggingface.co/junnyu/roformer_chinese_sim_char_base)，[`junnyu/roformer_chinese_sim_char_ft_base`](https://huggingface.co/junnyu/roformer_chinese_sim_char_ft_base)，[`junnyu/roformer_chinese_sim_char_small`](https://huggingface.co/junnyu/roformer_chinese_sim_char_small)，[`junnyu/roformer_chinese_sim_char_ft_small`](https://huggingface.co/junnyu/roformer_chinese_sim_char_ft_small) | [`junnyu/roformer_chinese_sim_char_base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/junnyu/roformer_chinese_sim_char_base/bert4torch_config.json)\u003cbr\u003e[`junnyu/roformer_chinese_sim_char_ft_base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/junnyu/roformer_chinese_sim_char_ft_base/bert4torch_config.json)\u003cbr\u003e[`junnyu/roformer_chinese_sim_char_small`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/junnyu/roformer_chinese_sim_char_small/bert4torch_config.json)\u003cbr\u003e[`junnyu/roformer_chinese_sim_char_ft_small`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/junnyu/roformer_chinese_sim_char_ft_small/bert4torch_config.json) |\r\n| gau|[GAU-alpha](https://github.com/ZhuiyiTechnology/GAU-alpha) | 追一科技| [`Tongjilibo/chinese_GAU-alpha-char_L-24_H-768`](https://huggingface.co/Tongjilibo/chinese_GAU-alpha-char_L-24_H-768) | |\r\n| uie| [uie](https://github.com/universal-ie/UIE)\u003cbr\u003e[uie_pytorch](https://github.com/HUSTAI/uie_pytorch) | 百度| [`Tongjilibo/uie-base`](https://huggingface.co/Tongjilibo/uie-base) | |\r\n| gpt |[CDial-GPT](https://github.com/thu-coai/CDial-GPT)| thu-coai| [`thu-coai/CDial-GPT_LCCC-base`](https://huggingface.co/thu-coai/CDial-GPT_LCCC-base)\u003cbr\u003e[`thu-coai/CDial-GPT_LCCC-large`](https://huggingface.co/thu-coai/CDial-GPT_LCCC-large) | [`thu-coai/CDial-GPT_LCCC-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/thu-coai/CDial-GPT_LCCC-base/bert4torch_config.json)\u003cbr\u003e[`thu-coai/CDial-GPT_LCCC-large`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/thu-coai/CDial-GPT_LCCC-large/bert4torch_config.json) |\r\n|     | [cmp_lm(26亿)](https://github.com/TsinghuaAI/CPM-1-Generate)|清华 | [`TsinghuaAI/CPM-Generate`](https://huggingface.co/TsinghuaAI/CPM-Generate) | [`\tTsinghuaAI/CPM-Generate`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/TsinghuaAI/CPM-Generate/bert4torch_config.json) |\r\n|     |[nezha_gen](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/NEZHA-Gen-TensorFlow)|huawei_noah|[`Tongjilibo/chinese_nezha_gpt_L-12_H-768_A-12`](https://huggingface.co/Tongjilibo/chinese_nezha_gpt_L-12_H-768_A-12)|\r\n|     | [gpt2-chinese-cluecorpussmall](https://github.com/dbiir/UER-py/wiki/Modelzoo) |UER | [`uer/gpt2-chinese-cluecorpussmall`](https://huggingface.co/uer/gpt2-chinese-cluecorpussmall) | [`uer/gpt2-chinese-cluecorpussmall`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/uer/gpt2-chinese-cluecorpussmall/bert4torch_config.json)|\r\n|     | [gpt2-ml](https://github.com/imcaspar/gpt2-ml) |imcaspar | [torch](https://github.com/ghosthamlet/gpt2-ml-torch)\u003cbr\u003e[BaiduYun(84dh)](https://pan.baidu.com/s/16tL4Bmoh6jPy0cOND0YyeA) | [`gpt2-ml_15g_corpus`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/gpt2-ml_15g_corpus/bert4torch_config.json)\u003cbr\u003e[`gpt2-ml_30g_corpus`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/gpt2-ml_30g_corpus/bert4torch_config.json) |\r\n| bart| [bart_base_chinese](https://github.com/fastnlp/CPT) |复旦fnlp| [`fnlp/bart-base-chinese`](https://huggingface.co/fnlp/bart-base-chinese)\u003cbr\u003e[v1.0](https://huggingface.co/fnlp/bart-base-chinese/tree/v1.0)| [`fnlp/bart-base-chinese`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/fnlp/bart-base-chinese/bert4torch_config.json)\u003cbr\u003e[`fnlp/bart-base-chinese-v1.0`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/fnlp/bart-base-chinese-v1.0/bert4torch_config.json) |\r\n| t5  | [t5](https://github.com/dbiir/UER-py/wiki/Modelzoo) | UER | [`uer/t5-small-chinese-cluecorpussmall`](https://huggingface.co/uer/t5-small-chinese-cluecorpussmall)\u003cbr\u003e[`uer/t5-base-chinese-cluecorpussmall`](https://huggingface.co/uer/t5-base-chinese-cluecorpussmall) | [`uer/t5-base-chinese-cluecorpussmall`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/uer/t5-base-chinese-cluecorpussmall/bert4torch_config.json)\u003cbr\u003e[`uer/t5-small-chinese-cluecorpussmall`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/uer/t5-small-chinese-cluecorpussmall/bert4torch_config.json)|\r\n|     | mt5 | 谷歌| [`google/mt5-base`](https://huggingface.co/google/mt5-base)| [`google/mt5-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/google/mt5-base/bert4torch_config.json)|\r\n|     | [t5_pegasus](https://github.com/ZhuiyiTechnology/t5-pegasus)| 追一科技| [`Tongjilibo/chinese_t5_pegasus_small`](https://huggingface.co/Tongjilibo/chinese_t5_pegasus_small)\u003cbr\u003e[`Tongjilibo/chinese_t5_pegasus_base`](https://huggingface.co/Tongjilibo/chinese_t5_pegasus_base)| |\r\n|     | [chatyuan](https://github.com/clue-ai/ChatYuan)| clue-ai |[`ClueAI/ChatYuan-large-v1`](https://huggingface.co/ClueAI/ChatYuan-large-v1)\u003cbr\u003e[`ClueAI/ChatYuan-large-v2`](https://huggingface.co/ClueAI/ChatYuan-large-v2)| [`ClueAI/ChatYuan-large-v1`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/ClueAI/ChatYuan-large-v1/bert4torch_config.json)\u003cbr\u003e[`ClueAI/ChatYuan-large-v2`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/ClueAI/ChatYuan-large-v2/bert4torch_config.json)|\r\n|     | [PromptCLUE](https://github.com/clue-ai/PromptCLUE)| clue-ai | [`ClueAI/PromptCLUE-base`](https://huggingface.co/ClueAI/PromptCLUE-base) | [`ClueAI/PromptCLUE-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/ClueAI/PromptCLUE-base/bert4torch_config.json)|\r\n| chatglm   | [chatglm-6b](https://github.com/THUDM/ChatGLM-6B) | THUDM | [`THUDM/chatglm-6b`](https://huggingface.co/THUDM/chatglm-6b)\u003cbr\u003e[`THUDM/chatglm-6b-int8`](https://huggingface.co/THUDM/chatglm-6b-int8)\u003cbr\u003e[`THUDM/chatglm-6b-int4`](https://huggingface.co/THUDM/chatglm-6b-int4)\u003cbr\u003e[v0.1.0](https://huggingface.co/THUDM/chatglm-6b/tree/v0.1.0) | [`THUDM/chatglm-6b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/chatglm-6b/bert4torch_config.json)\u003cbr\u003e[`THUDM/chatglm-6b-int8`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/chatglm-6b-int8/bert4torch_config.json)\u003cbr\u003e[`THUDM/chatglm-6b-int4`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/chatglm-6b-int4/bert4torch_config.json)\u003cbr\u003e[`THUDM/chatglm-6b-v0.1.0`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/chatglm-6b-v0.1.0/bert4torch_config.json) |\r\n|       |[chatglm2-6b](https://github.com/THUDM/ChatGLM2-6B) | THUDM | [`THUDM/chatglm2-6b`](https://huggingface.co/THUDM/chatglm2-6b)\u003cbr\u003e[`THUDM/chatglm2-6b-int4`](https://huggingface.co/THUDM/chatglm2-6b-int4)\u003cbr\u003e[`THUDM/chatglm2-6b-32k`](https://huggingface.co/THUDM/chatglm2-6b-32k) | [`THUDM/chatglm2-6b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/chatglm2-6b/bert4torch_config.json)\u003cbr\u003e[`THUDM/chatglm2-6b-int4`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/chatglm2-6b-int4/bert4torch_config.json)\u003cbr\u003e[`THUDM/chatglm2-6b-32k`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/chatglm2-6b-32k/bert4torch_config.json) |\r\n|       |[chatglm3-6b](https://github.com/THUDM/ChatGLM3) | THUDM | [`THUDM/chatglm3-6b`](https://huggingface.co/THUDM/chatglm3-6b)\u003cbr\u003e[`THUDM/chatglm3-6b-32k`](https://huggingface.co/THUDM/chatglm3-6b-32k) | [`THUDM/chatglm3-6b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/chatglm3-6b/bert4torch_config.json)\u003cbr\u003e[`THUDM/chatglm3-6b-32k`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/chatglm3-6b-32k/bert4torch_config.json) |\r\n|       |[glm4-9b](https://github.com/THUDM/GLM-4) | THUDM | [`THUDM/glm-4-9b`](https://huggingface.co/THUDM/glm-4-9b)\u003cbr\u003e[`THUDM/glm-4-9b-chat`](https://huggingface.co/THUDM/glm-4-9b-chat)\u003cbr\u003e[`THUDM/glm-4-9b-chat-1m`](https://huggingface.co/THUDM/glm-4-9b-chat-1m) | [`THUDM/glm-4-9b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/glm-4-9b/bert4torch_config.json)\u003cbr\u003e[`THUDM/glm-4-9b-chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/glm-4-9b-chat/bert4torch_config.json)\u003cbr\u003e[`THUDM/glm-4-9b-chat-1m`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/THUDM/glm-4-9b-chat-1m/bert4torch_config.json)|\r\n| llama | [llama](https://github.com/facebookresearch/llama) | meta|  | [`meta-llama/llama-7b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/llama-7b/bert4torch_config.json)\u003cbr\u003e[`meta-llama/llama-13b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/llama-13b/bert4torch_config.json)|\r\n|       | [llama-2](https://github.com/facebookresearch/llama) | meta| [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf)\u003cbr\u003e[meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)\u003cbr\u003e[meta-llama/Llama-2-13b-hf](https://huggingface.co/meta-llama/Llama-2-13b-hf)\u003cbr\u003e[meta-llama/Llama-2-13b-chat-hf](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf) | [`meta-llama/Llama-2-7b-hf`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Llama-2-7b-hf/bert4torch_config.json)\u003cbr\u003e[`meta-llama/Llama-2-7b-chat-hf`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Llama-2-7b-chat-hf/bert4torch_config.json)\u003cbr\u003e[`meta-llama/Llama-2-13b-hf`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Llama-2-13b-hf/bert4torch_config.json)\u003cbr\u003e[`meta-llama/Llama-2-13b-chat-hf`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Llama-2-13b-chat-hf/bert4torch_config.json)|\r\n|       | [llama-3](https://github.com/meta-llama/llama3) | meta| [`meta-llama/Meta-Llama-3-8B`](https://huggingface.co/meta-llama/Meta-Llama-3-8B)\u003cbr\u003e[`meta-llama/Meta-Llama-3-8B-Instruct`](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) | [`meta-llama/Meta-Llama-3-8B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Meta-Llama-3-8B/bert4torch_config.json)\u003cbr\u003e[`meta-llama/Meta-Llama-3-8B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Meta-Llama-3-8B-Instruct/bert4torch_config.json)|\r\n|       | [llama-3.1](https://github.com/meta-llama/llama-models) | meta | [`meta-llama/Meta-Llama-3.1-8B`](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B)\u003cbr\u003e[`meta-llama/Meta-Llama-3.1-8B-Instruct`](https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct) | [`meta-llama/Meta-Llama-3.1-8B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Meta-Llama-3.1-8B/bert4torch_config.json)\u003cbr\u003e[`meta-llama/Meta-Llama-3.1-8B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Meta-Llama-3.1-8B-Instruct/bert4torch_config.json) |\r\n|       | [llama-3.2](https://github.com/meta-llama/llama-models) | meta | [`meta-llama/Llama-3.2-1B`](https://huggingface.co/meta-llama/Llama-3.2-1B)\u003cbr\u003e[`meta-llama/Llama-3.2-1B-Instruct`](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)\u003cbr\u003e[`meta-llama/Llama-3.2-3B`](https://huggingface.co/meta-llama/Llama-3.2-3B)\u003cbr\u003e[`meta-llama/Llama-3.2-3B-Instruct`](https://huggingface.co/meta-llama/Llama-3.2-3B-Instruct) | [`meta-llama/Llama-3.2-1B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Llama-3.2-1B/bert4torch_config.json)\u003cbr\u003e[`meta-llama/Llama-3.2-1B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Llama-3.2-1B-Instruct/bert4torch_config.json)\u003cbr\u003e[`meta-llama/Llama-3.2-3B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Llama-3.2-3B/bert4torch_config.json)\u003cbr\u003e[`meta-llama/Llama-3.2-3B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/meta-llama/Llama-3.2-3B-Instruct/bert4torch_config.json) |\r\n|       | [Chinese-LLaMA-Alpaca](https://github.com/ymcui/Chinese-LLaMA-Alpaca)|HFL| |[`hfl/chinese_alpaca_plus_7b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/hfl/chinese_alpaca_plus_7b/bert4torch_config.json)\u003cbr\u003e[`hfl/chinese_llama_plus_7b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/hfl/chinese_llama_plus_7b/bert4torch_config.json)|\r\n|       | [Chinese-LLaMA-Alpaca-2](https://github.com/ymcui/Chinese-LLaMA-Alpaca-2)|HFL| |待添加|\r\n|       | [Chinese-LLaMA-Alpaca-3](https://github.com/ymcui/Chinese-LLaMA-Alpaca-3)|HFL| |待添加|\r\n|       | [Belle_llama](https://github.com/LianjiaTech/BELLE)| LianjiaTech|[BelleGroup/BELLE-LLaMA-7B-2M-enc](https://huggingface.co/BelleGroup/BELLE-LLaMA-7B-2M-enc) | [合成说明](https://github.com/LianjiaTech/BELLE/tree/main/models)、[`BelleGroup/BELLE-LLaMA-7B-2M-enc`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/BelleGroup/BELLE-LLaMA-7B-2M-enc)|\r\n|       | [Ziya](https://github.com/IDEA-CCNL/Fengshenbang-LM) | IDEA-CCNL | [IDEA-CCNL/Ziya-LLaMA-13B-v1](https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-13B-v1)\u003cbr\u003e[IDEA-CCNL/Ziya-LLaMA-13B-v1.1](https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-13B-v1.1)\u003cbr\u003e[IDEA-CCNL/Ziya-LLaMA-13B-Pretrain-v1](https://huggingface.co/IDEA-CCNL/Ziya-LLaMA-13B-Pretrain-v1) | [`IDEA-CCNL/Ziya-LLaMA-13B-v1`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/IDEA-CCNL/Ziya-LLaMA-13B-v1/bert4torch_config.json)\u003cbr\u003e[`IDEA-CCNL/Ziya-LLaMA-13B-v1.1`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/IDEA-CCNL/Ziya-LLaMA-13B-v1.1/bert4torch_config.json) |\r\n|       | [vicuna](https://github.com/lm-sys/FastChat) | lmsys| [`lmsys/vicuna-7b-v1.5`](https://huggingface.co/lmsys/vicuna-7b-v1.5) | [`lmsys/vicuna-7b-v1.5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/lmsys/vicuna-7b-v1.5/bert4torch_config.json)|\r\n|Baichuan| [Baichuan](https://github.com/baichuan-inc/Baichuan) | baichuan-inc | [`baichuan-inc/Baichuan-7B`](https://huggingface.co/baichuan-inc/Baichuan-7B)\u003cbr\u003e[`baichuan-inc/Baichuan-13B-Base`](https://huggingface.co/baichuan-inc/Baichuan-13B-Base)\u003cbr\u003e[`baichuan-inc/Baichuan-13B-Chat`](https://huggingface.co/baichuan-inc/Baichuan-13B-Chat) | [`baichuan-inc/Baichuan-7B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/baichuan-inc/Baichuan-7B/bert4torch_config.json)\u003cbr\u003e[`baichuan-inc/Baichuan-13B-Base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/baichuan-inc/Baichuan-13B-Base/bert4torch_config.json)\u003cbr\u003e[`baichuan-inc/Baichuan-13B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/baichuan-inc/Baichuan-13B-Chat/bert4torch_config.json) |\r\n|       | [Baichuan2](https://github.com/baichuan-inc/Baichuan2) | baichuan-inc | [`baichuan-inc/Baichuan2-7B-Base`](https://huggingface.co/baichuan-inc/Baichuan2-7B-Base)\u003cbr\u003e[`baichuan-inc/Baichuan2-7B-Chat`](https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat)\u003cbr\u003e[`baichuan-inc/Baichuan2-13B-Base`](https://huggingface.co/baichuan-inc/Baichuan2-13B-Base)\u003cbr\u003e[`baichuan-inc/Baichuan2-13B-Chat`](https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat) | [`baichuan-inc/Baichuan2-7B-Base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/baichuan-inc/Baichuan2-7B-Base/bert4torch_config.json)\u003cbr\u003e[`baichuan-inc/Baichuan2-7B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/baichuan-inc/Baichuan2-7B-Chat/bert4torch_config.json)\u003cbr\u003e[`baichuan-inc/Baichuan2-13B-Base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/baichuan-inc/Baichuan2-13B-Base/bert4torch_config.json)\u003cbr\u003e[`baichuan-inc/Baichuan2-13B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/baichuan-inc/Baichuan2-13B-Chat/bert4torch_config.json) |\r\n|  Yi  | [Yi](https://github.com/01-ai/Yi) | 01-ai| [`01-ai/Yi-6B`](https://huggingface.co/01-ai/Yi-6B)\u003cbr\u003e[`01-ai/Yi-6B-200K`](https://huggingface.co/01-ai/Yi-6B-200K)\u003cbr\u003e[`01-ai/Yi-9B`](https://huggingface.co/01-ai/Yi-9B)\u003cbr\u003e[`01-ai/Yi-9B-200K`](https://huggingface.co/01-ai/Yi-9B-200K) | [`01-ai/Yi-6B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-6B/bert4torch_config.json)\u003cbr\u003e[`01-ai/Yi-6B-200K`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-6B-200K/bert4torch_config.json)\u003cbr\u003e[`01-ai/Yi-9B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-9B/bert4torch_config.json)\u003cbr\u003e[`01-ai/Yi-9B-200K`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-9B-200K/bert4torch_config.json) |\r\n|       | [Yi-1.5](https://github.com/01-ai/Yi-1.5) | 01-ai| [`01-ai/Yi-1.5-6B`](https://huggingface.co/01-ai/Yi-1.5-6B)\u003cbr\u003e[`01-ai/Yi-1.5-6B-Chat`](https://huggingface.co/01-ai/Yi-1.5-6B-Chat)\u003cbr\u003e[`01-ai/Yi-1.5-9B`](https://huggingface.co/01-ai/Yi-1.5-9B)\u003cbr\u003e[`01-ai/Yi-1.5-9B-32K`](https://huggingface.co/01-ai/Yi-1.5-9B-32K)\u003cbr\u003e[`01-ai/Yi-1.5-9B-Chat`](https://huggingface.co/01-ai/Yi-1.5-9B-Chat)\u003cbr\u003e[`01-ai/Yi-1.5-9B-Chat-16K`](https://huggingface.co/01-ai/Yi-1.5-9B-Chat-16K) | [`01-ai/Yi-1.5-6B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-1.5-6B/bert4torch_config.json)\u003cbr\u003e[`01-ai/Yi-1.5-6B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-1.5-6B-Chat/bert4torch_config.json)\u003cbr\u003e[`01-ai/Yi-1.5-9B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-1.5-9B/bert4torch_config.json)\u003cbr\u003e[`01-ai/Yi-1.5-9B-32K`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-1.5-9B-32K/bert4torch_config.json)\u003cbr\u003e[`01-ai/Yi-1.5-9B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-1.5-9B-Chat)\u003cbr\u003e[`01-ai/Yi-1.5-9B-Chat-16K`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/01-ai/Yi-1.5-9B-Chat-16K/bert4torch_config.json) |\r\n| bloom |[bloom](https://github.com/bigscience-workshop/xmtf) | bigscience | [`bigscience/bloom-560m`](https://huggingface.co/bigscience/bloom-560m)\u003cbr\u003e[`bigscience/bloomz-560m`](https://huggingface.co/bigscience/bloomz-560m) | [`bigscience/bloom-560m`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/bigscience/bloom-560m/bert4torch_config.json)\u003cbr\u003e[`bigscience/bloomz-560m`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/bigscience/bloomz-560m/bert4torch_config.json) |\r\n| Qwen  |[Qwen](https://github.com/QwenLM/Qwen) | 阿里云 | [`Qwen/Qwen-1_8B`](https://huggingface.co/Qwen/Qwen-1_8B)\u003cbr\u003e[`Qwen/Qwen-1_8B-Chat`](https://huggingface.co/Qwen/Qwen-1_8B-Chat)\u003cbr\u003e[`Qwen/Qwen-7B`](https://huggingface.co/Qwen/Qwen-7B)\u003cbr\u003e[`Qwen/Qwen-7B-Chat`](https://huggingface.co/Qwen/Qwen-7B-Chat)\u003cbr\u003e[`Qwen/Qwen-14B`](https://huggingface.co/Qwen/Qwen-14B)\u003cbr\u003e[`Qwen/Qwen-14B-Chat`](https://huggingface.co/Qwen/Qwen-14B-Chat) | [`Qwen/Qwen-1_8B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen-1_8B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen-1_8B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen-1_8B-Chat/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen-7B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen-7B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen-7B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen-7B-Chat/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen-14B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen-14B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen-14B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen-14B-Chat/bert4torch_config.json) |\r\n||[Qwen1.5](https://github.com/QwenLM/Qwen1.5)|阿里云|[`Qwen/Qwen1.5-0.5B`](https://huggingface.co/Qwen/Qwen1.5-0.5B)\u003cbr\u003e[`Qwen/Qwen1.5-0.5B-Chat`](https://huggingface.co/Qwen/Qwen1.5-0.5B-Chat)\u003cbr\u003e[`Qwen/Qwen1.5-1.8B`](https://huggingface.co/Qwen/Qwen1.5-1.8B)\u003cbr\u003e[`Qwen/Qwen1.5-1.8B-Chat`](https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat)\u003cbr\u003e[`Qwen/Qwen1.5-7B`](https://huggingface.co/Qwen/Qwen1.5-7B)\u003cbr\u003e[`Qwen/Qwen1.5-7B-Chat`](https://huggingface.co/Qwen/Qwen1.5-7B-Chat)\u003cbr\u003e[`Qwen/Qwen1.5-14B`](https://huggingface.co/Qwen/Qwen1.5-14B)\u003cbr\u003e[`Qwen/Qwen1.5-14B-Chat`](https://huggingface.co/Qwen/Qwen1.5-14B-Chat) | [`Qwen/Qwen1.5-0.5B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen1.5-0.5B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen1.5-0.5B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen1.5-0.5B-Chat/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen1.5-1.8B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen1.5-1.8B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen1.5-1.8B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen1.5-1.8B-Chat/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen1.5-7B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen1.5-7B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen1.5-7B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen1.5-7B-Chat/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen1.5-14B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen1.5-14B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen1.5-14B-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen1.5-14B-Chat/bert4torch_config.json)|\r\n||[Qwen2](https://github.com/QwenLM/Qwen2)|阿里云|[`Qwen/Qwen2-0.5B`](https://huggingface.co/Qwen/Qwen2-0.5B)\u003cbr\u003e[`Qwen/Qwen2-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2-0.5B-Instruct)\u003cbr\u003e[`Qwen/Qwen2-1.5B`](https://huggingface.co/Qwen/Qwen2-1.5B)\u003cbr\u003e[`Qwen/Qwen2-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2-1.5B-Instruct)\u003cbr\u003e[`Qwen/Qwen2-7B`](https://huggingface.co/Qwen/Qwen2-7B)\u003cbr\u003e[`Qwen/Qwen2-7B-Instruct`](https://huggingface.co/Qwen/Qwen2-7B-Instruct) | [`Qwen/Qwen2-0.5B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2-0.5B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2-0.5B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2-0.5B-Instruct/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2-1.5B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2-1.5B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2-1.5B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2-1.5B-Instruct/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2-7B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2-7B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2-7B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2-7B-Instruct/bert4torch_config.json)|\r\n||[Qwen2-VL](https://github.com/QwenLM/Qwen2-VL)|阿里云|[`Qwen/Qwen2-VL-2B-Instruct`](https://huggingface.co/Qwen/Qwen2-VL-2B-Instruct)\u003cbr\u003e[`Qwen/Qwen2-VL-7B-Instruct`](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct) | [`Qwen/Qwen2-VL-2B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2-VL-2B-Instruct/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2-VL-7B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2-VL-7B-Instruct/bert4torch_config.json)|\r\n||[Qwen2.5](https://github.com/QwenLM/Qwen2.5)|阿里云|[`Qwen/Qwen2.5-0.5B`](https://huggingface.co/Qwen/Qwen2.5-0.5B)\u003cbr\u003e[`Qwen/Qwen2.5-0.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)\u003cbr\u003e[`Qwen/Qwen2.5-1.5B`](https://huggingface.co/Qwen/Qwen2.5-1.5B)\u003cbr\u003e[`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)\u003cbr\u003e[`Qwen/Qwen2.5-3B`](https://huggingface.co/Qwen/Qwen2.5-3B)\u003cbr\u003e[`Qwen/Qwen2.5-3B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-3B-Instruct)\u003cbr\u003e[`Qwen/Qwen2.5-7B`](https://huggingface.co/Qwen/Qwen2.5-7B)\u003cbr\u003e[`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct)\u003cbr\u003e[`Qwen/Qwen2.5-14B`](https://huggingface.co/Qwen/Qwen2.5-14B)\u003cbr\u003e[`Qwen/Qwen2.5-14B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct) | [`Qwen/Qwen2.5-0.5B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-0.5B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2.5-0.5B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-0.5B-Instruct/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2.5-1.5B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-1.5B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2.5-1.5B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-1.5B-Instruct/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2.5-3B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-3B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2.5-3B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-3B-Instruct/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2.5-7B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-7B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-7B-Instruct/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2.5-14B`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-14B/bert4torch_config.json)\u003cbr\u003e[`Qwen/Qwen2.5-14B-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/Qwen/Qwen2.5-14B-Instruct/bert4torch_config.json)|\r\n| InternLM|[InternLM](https://github.com/InternLM/InternLM) | 上海人工智能实验室 | [`internlm/internlm-7b`](https://huggingface.co/internlm/internlm-7b)\u003cbr\u003e[`internlm/internlm-chat-7b`](https://huggingface.co/internlm/internlm-chat-7b) | [`internlm/internlm-7b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/internlm/internlm-7b/bert4torch_config.json)\u003cbr\u003e[`internlm/internlm-chat-7b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/internlm/internlm-chat-7b/bert4torch_config.json)|\r\n|            |[InternLM2](https://huggingface.co/collections/internlm/internlm2-65b0ce04970888799707893c) | 上海人工智能实验室 | [`internlm/internlm2-1_8b`](https://huggingface.co/internlm/internlm2-1_8b)\u003cbr\u003e[`internlm/internlm2-chat-1_8b`](https://huggingface.co/internlm/internlm2-chat-1_8b)\u003cbr\u003e[`internlm/internlm2-7b`](https://huggingface.co/internlm/internlm2-7b)\u003cbr\u003e[`internlm/internlm2-chat-7b`](https://huggingface.co/internlm/internlm2-chat-7b)\u003cbr\u003e[`internlm/internlm2-20b`](https://huggingface.co/internlm/internlm2-20b)\u003cbr\u003e[`internlm/internlm2-chat-20b`](https://huggingface.co/internlm/internlm2-chat-20b) | [`internlm/internlm2-1_8b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/internlm/internlm2-1_8b/bert4torch_config.json)\u003cbr\u003e[`internlm/internlm2-chat-1_8b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/internlm/internlm2-chat-1_8b/bert4torch_config.json)\u003cbr\u003e[`internlm/internlm2-7b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/internlm/internlm2-7b/bert4torch_config.json)\u003cbr\u003e[`internlm/internlm2-chat-7b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/internlm/internlm2-chat-7b/bert4torch_config.json)|\r\n|            |[InternLM2.5](https://huggingface.co/collections/internlm/internlm25-66853f32717072d17581bc13) | 上海人工智能实验室 | [`internlm/internlm2_5-7b`](https://huggingface.co/internlm/internlm2_5-7b)\u003cbr\u003e[`internlm/internlm2_5-7b-chat`](https://huggingface.co/internlm/internlm2_5-7b-chat)\u003cbr\u003e[`internlm/internlm2_5-7b-chat-1m`](https://huggingface.co/internlm/internlm2_5-7b-chat-1m) | [`internlm/internlm2_5-7b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/internlm/internlm2_5-7b/bert4torch_config.json)\u003cbr\u003e[`internlm/internlm2_5-7b-chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/internlm/internlm2_5-7b-chat/bert4torch_config.json)\u003cbr\u003e[`internlm/internlm2_5-7b-chat-1m`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/internlm/internlm2_5-7b-chat-1m/bert4torch_config.json) |\r\n| Falcon|[Falcon](https://huggingface.co/tiiuae) | tiiuae | [`tiiuae/falcon-rw-1b`](https://huggingface.co/tiiuae/falcon-rw-1b)\u003cbr\u003e[`tiiuae/falcon-7b`](https://huggingface.co/tiiuae/falcon-7b)\u003cbr\u003e[`tiiuae/falcon-7b-instruct`](https://huggingface.co/tiiuae/falcon-7b-instruct) | [`tiiuae/falcon-rw-1b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/tiiuae/falcon-rw-1b/bert4torch_config.json)\u003cbr\u003e[`tiiuae/falcon-7b`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/tiiuae/falcon-7b/bert4torch_config.json)\u003cbr\u003e[`tiiuae/falcon-7b-instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/tiiuae/falcon-7b-instruct/bert4torch_config.json) |\r\n|  DeepSeek  |[DeepSeek-MoE](https://github.com/deepseek-ai/DeepSeek-MoE)|深度求索| [`deepseek-ai/deepseek-moe-16b-base`](https://huggingface.co/deepseek-ai/deepseek-moe-16b-base)\u003cbr\u003e[`deepseek-ai/deepseek-moe-16b-chat`](https://huggingface.co/deepseek-ai/deepseek-moe-16b-chat) | [`deepseek-ai/deepseek-moe-16b-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-moe-16b-base/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/deepseek-moe-16b-chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-moe-16b-chat/bert4torch_config.json) |\r\n|          |[DeepSeek-LLM](https://github.com/deepseek-ai/DeepSeek-LLM)|深度求索|[`deepseek-ai/deepseek-llm-7b-base`](https://huggingface.co/deepseek-ai/deepseek-llm-7b-base)\u003cbr\u003e[`deepseek-ai/deepseek-llm-7b-chat`](https://huggingface.co/deepseek-ai/deepseek-llm-7b-chat) |[`deepseek-ai/deepseek-llm-7b-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-llm-7b-base/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/deepseek-llm-7b-chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-llm-7b-chat/bert4torch_config.json)|\r\n|          |[DeepSeek-V2](https://github.com/deepseek-ai/DeepSeek-V2)|深度求索|[`deepseek-ai/DeepSeek-V2-Lite`](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite)\u003cbr\u003e[`deepseek-ai/DeepSeek-V2-Lite-Chat`](https://huggingface.co/deepseek-ai/DeepSeek-V2-Lite-Chat)|[`deepseek-ai/DeepSeek-V2-Lite`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/DeepSeek-V2-Lite/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/DeepSeek-V2-Lite-Chat`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/DeepSeek-V2-Lite-Chat/bert4torch_config.json)|\r\n|          |[DeepSeek-Coder](https://github.com/deepseek-ai/DeepSeek-Coder)|深度求索|[`deepseek-ai/deepseek-coder-1.3b-base`](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-base)\u003cbr\u003e[`deepseek-ai/deepseek-coder-1.3b-instruct`](https://huggingface.co/deepseek-ai/deepseek-coder-1.3b-instruct)\u003cbr\u003e[`deepseek-ai/deepseek-coder-6.7b-base`](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base)\u003cbr\u003e[`deepseek-ai/deepseek-coder-6.7b-instruct`](https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct)\u003cbr\u003e[`deepseek-ai/deepseek-coder-7b-base-v1.5`](https://huggingface.co/deepseek-ai/deepseek-coder-7b-base-v1.5)\u003cbr\u003e[`deepseek-ai/deepseek-coder-7b-instruct-v1.5`](https://huggingface.co/deepseek-ai/deepseek-coder-7b-instruct-v1.5) | [`deepseek-ai/deepseek-coder-1.3b-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-coder-1.3b-base/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/deepseek-coder-1.3b-instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-coder-1.3b-instruct/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/deepseek-coder-6.7b-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-coder-6.7b-base/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/deepseek-coder-6.7b-instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-coder-6.7b-instruct/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/deepseek-coder-7b-base-v1.5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-coder-7b-base-v1.5/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/deepseek-coder-7b-instruct-v1.5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-coder-7b-instruct-v1.5/bert4torch_config.json)|\r\n|          |[DeepSeek-Coder-V2](https://github.com/deepseek-ai/DeepSeek-Coder-V2)|深度求索|[`deepseek-ai/DeepSeek-Coder-V2-Lite-Base`](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Base)\u003cbr\u003e[`deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct`](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct) | [`deepseek-ai/DeepSeek-Coder-V2-Lite-Base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/DeepSeek-Coder-V2-Lite-Base/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct/bert4torch_config.json)|\r\n|          |[DeepSeek-Math](https://github.com/deepseek-ai/DeepSeek-Math)|深度求索|[`deepseek-ai/deepseek-math-7b-base`](https://huggingface.co/deepseek-ai/deepseek-ai/deepseek-math-7b-base)\u003cbr\u003e[`deepseek-ai/deepseek-math-7b-instruct`](https://huggingface.co/deepseek-ai/deepseek-math-7b-instruct)\u003cbr\u003e[`deepseek-ai/deepseek-math-7b-rl`](https://huggingface.co/deepseek-ai/deepseek-math-7b-rl) | [`deepseek-ai/deepseek-math-7b-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-math-7b-base/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/deepseek-math-7b-instruct`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-math-7b-instruct/bert4torch_config.json)\u003cbr\u003e[`deepseek-ai/deepseek-math-7b-rl`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/deepseek-ai/deepseek-math-7b-rl/bert4torch_config.json)|\r\n| MiniCPM  |[MiniCPM](https://github.com/OpenBMB/MiniCPM)|OpenBMB|[`openbmb/MiniCPM-2B-sft-bf16`](https://huggingface.co/openbmb/MiniCPM-2B-sft-bf16)\u003cbr\u003e[`openbmb/MiniCPM-2B-dpo-bf16`](https://huggingface.co/openbmb/MiniCPM-2B-dpo-bf16)\u003cbr\u003e[`openbmb/MiniCPM-2B-128k`](https://huggingface.co/openbmb/MiniCPM-2B-128k)\u003cbr\u003e[`openbmb/MiniCPM-1B-sft-bf16`](https://huggingface.co/openbmb/MiniCPM-1B-sft-bf16) | [`openbmb/MiniCPM-2B-sft-bf16`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/openbmb/MiniCPM-2B-sft-bf16/bert4torch_config.json)\u003cbr\u003e[`openbmb/MiniCPM-2B-dpo-bf16`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/openbmb/MiniCPM-2B-dpo-bf16/bert4torch_config.json)\u003cbr\u003e[`openbmb/MiniCPM-2B-128k`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/openbmb/MiniCPM-2B-128k/bert4torch_config.json)\u003cbr\u003e[`openbmb/MiniCPM-1B-sft-bf16`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/openbmb/MiniCPM-1B-sft-bf16/bert4torch_config.json) |\r\n|          |[MiniCPM-V](https://github.com/OpenBMB/MiniCPM-V)|OpenBMB|[`openbmb/MiniCPM-V-2_6`](https://huggingface.co/openbmb/MiniCPM-V-2_6)\u003cbr\u003e[`openbmb/MiniCPM-Llama3-V-2_5`](https://huggingface.co/openbmb/MiniCPM-Llama3-V-2_5) | [`openbmb/MiniCPM-V-2_6`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/openbmb/MiniCPM-V-2_6/bert4torch_config.json)\u003cbr\u003e[`openbmb/MiniCPM-Llama3-V-2_5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/openbmb/MiniCPM-Llama3-V-2_5/bert4torch_config.json)|\r\n| embedding| [text2vec-base-chinese](https://github.com/shibing624/text2vec) |shibing624| [`shibing624/text2vec-base-chinese`](https://huggingface.co/shibing624/text2vec-base-chinese) |[`shibing624/text2vec-base-chinese`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/shibing624/text2vec-base-chinese/bert4torch_config.json) |\r\n|          | [m3e](https://github.com/wangyuxinwhy/uniem) |moka-ai| [`moka-ai/m3e-base`](https://huggingface.co/moka-ai/m3e-base) |[`moka-ai/m3e-base`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/moka-ai/m3e-base/bert4torch_config.json)|\r\n|          | bge |BAAI| [`BAAI/bge-large-en-v1.5`](https://huggingface.co/BAAI/bge-large-en-v1.5)\u003cbr\u003e[`BAAI/bge-large-zh-v1.5`](https://huggingface.co/BAAI/bge-large-zh-v1.5)\u003cbr\u003e[`BAAI/bge-base-en-v1.5`](https://huggingface.co/BAAI/bge-base-en-v1.5)\u003cbr\u003e[`BAAI/bge-base-zh-v1.5`](https://huggingface.co/BAAI/bge-base-zh-v1.5)\u003cbr\u003e[`BAAI/bge-small-en-v1.5`](https://huggingface.co/BAAI/bge-small-en-v1.5)\u003cbr\u003e[`BAAI/bge-small-zh-v1.5`](https://huggingface.co/BAAI/bge-small-zh-v1.5) | [`BAAI/bge-large-en-v1.5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/BAAI/bge-large-en-v1.5/bert4torch_config.json)\u003cbr\u003e[`BAAI/bge-large-zh-v1.5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/BAAI/bge-large-zh-v1.5/bert4torch_config.json)\u003cbr\u003e[`BAAI/bge-base-en-v1.5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/BAAI/bge-base-en-v1.5/bert4torch_config.json)\u003cbr\u003e[`BAAI/bge-base-zh-v1.5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/BAAI/bge-base-zh-v1.5/bert4torch_config.json)\u003cbr\u003e[`BAAI/bge-small-en-v1.5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/BAAI/bge-small-en-v1.5/bert4torch_config.json)\u003cbr\u003e[`BAAI/bge-small-zh-v1.5`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/BAAI/bge-small-zh-v1.5/bert4torch_config.json)|\r\n|          | gte |thenlper| [`thenlper/gte-large-zh`](https://huggingface.co/thenlper/gte-large-zh)\u003cbr\u003e[`thenlper/gte-base-zh`](https://huggingface.co/thenlper/gte-base-zh) | [`thenlper/gte-base-zh`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/thenlper/gte-base-zh/bert4torch_config.json)\u003cbr\u003e[`thenlper/gte-large-zh`](https://huggingface.co/Tongjilibo/bert4torch_config/blob/main/thenlper/gte-large-zh/bert4torch_config.json)|\r\n\r\n*注：\r\n1. `高亮格式`(如`bert-base-chinese`)的表示可直接`build_transformer_model()`联网下载\r\n2. 国内镜像网站加速下载\r\n   - `HF_ENDPOINT=https://hf-mirror.com python your_script.py`\r\n   - `export HF_ENDPOINT=https://hf-mirror.com`后再执行python代码\r\n   - 在python代码开头如下设置\r\n    ```python\r\n    import os\r\n    os.environ['HF_ENDPOINT'] = \"https://hf-mirror.com\"\r\n    ```\r\n\r\n## 6. 鸣谢\r\n\r\n- 感谢苏神实现的[bert4keras](https://github.com/bojone/bert4keras)，本实现有不少地方参考了bert4keras的源码，在此衷心感谢大佬的无私奉献;\r\n- 其次感谢项目[bert4pytorch](https://github.com/MuQiuJun-AI/bert4pytorch)，也是在该项目的指引下给了我用pytorch来复现bert4keras的想法和思路。\r\n\r\n## 7. 引用\r\n\r\n```\r\n@misc{bert4torch,\r\n  title={bert4torch},\r\n  author={Bo Li},\r\n  year={2022},\r\n  howpublished={\\url{https://github.com/Tongjilibo/bert4torch}},\r\n}\r\n```\r\n\r\n## 8. 其他\r\n\r\n- Wechat \u0026 Star History Chart\r\n- 微信群人数超过200个（有邀请限制），可添加个人微信拉群\r\n\r\n\u003ctable border=\"0\"\u003e\r\n  \u003ctbody\u003e\r\n    \u003ctr align=\"center\" \u003e\r\n      \u003ctd\u003e\r\n         \u003ca href=\"https://github.com/Tongjilibo\"\u003e\u003cimg width=\"200\" height=\"250\" src=\"./docs/pics/wechat.jpg\" alt=\"pic\"\u003e\u003c/a\u003e\u003cbr\u003e\r\n         \u003ca href=\"https://github.com/Tongjilibo\"\u003e微信号\u003c/a\u003e \r\n      \u003c/td\u003e\r\n      \u003ctd\u003e\r\n         \u003ca href=\"https://github.com/Tongjilibo\"\u003e\u003cimg width=\"190\" height=\"250\" src=\"./docs/pics/wechat_group.jpg\" alt=\"pic\"\u003e\u003c/a\u003e\u003cbr\u003e\r\n         \u003ca href=\"https://github.com/Tongjilibo\"\u003e微信群\u003c/a\u003e \r\n      \u003c/td\u003e\r\n      \u003ctd\u003e\r\n         \u003ca href=\"https://star-history.com/#Tongjilibo/bert4torch\u0026Date\"\u003e\u003cimg width=\"400\" height=\"250\" src=\"https://api.star-history.com/svg?repos=Tongjilibo/bert4torch\u0026type=Date\" alt=\"pic\"\u003e\u003c/a\u003e\u003cbr\u003e\r\n         \u003ca href=\"https://star-history.com/#Tongjilibo/bert4torch\u0026Date\"\u003eStar History Chart\u003c/a\u003e \r\n      \u003c/td\u003e    \r\n      \u003c/tr\u003e\r\n  \u003c/tbody\u003e\r\n\u003c/table\u003e\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTongjilibo%2Fbert4torch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTongjilibo%2Fbert4torch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTongjilibo%2Fbert4torch/lists"}