https://github.com/ssbuild/poetry_training
诗歌模型预训练
https://github.com/ssbuild/poetry_training
Last synced: about 1 year ago
JSON representation
诗歌模型预训练
- Host: GitHub
- URL: https://github.com/ssbuild/poetry_training
- Owner: ssbuild
- Created: 2023-02-15T13:01:17.000Z (over 3 years ago)
- Default Branch: master
- Last Pushed: 2023-04-28T08:14:27.000Z (about 3 years ago)
- Last Synced: 2025-04-18T12:17:56.675Z (about 1 year ago)
- Language: Python
- Size: 144 KB
- Stars: 7
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.MD
Awesome Lists containing this project
README
## 安装
- pip install -U deep_training >= 0.1.0
- (PaLM,LaMDA 修改默认激活函数)最小版本要求 0.1.0
## 更新详情
- [deep_training](https://github.com/ssbuild/deep_training)
## 深度学习常规任务例子
- [pytorch-task-example](https://github.com/ssbuild/pytorch-task-example)
- [tf-task-example](https://github.com/ssbuild/tf-task-example)
## 支持模型
gpt2
laMDA
LLaMA
PaLM
t5
unilm
t5decoder
## 导出onnx模型 通常只需要三步
第一步，参数配置 convert_onnx = True
第二步 加载权重例子
model = MyTransformer.load_from_checkpoint('./best.pt', config=config, model_args=model_args,
training_args=training_args)
第三步 #导出onnx模型
model.convert_to_onnx('./best.onnx')
## 多卡训练策略 strategy , 通常只需要一步
修改参数配置 devices = N
# Available names: bagua, colossalai, ddp, ddp_find_unused_parameters_false, ddp_fork,
# ddp_fork_find_unused_parameters_false, ddp_fully_sharded,
# ddp_notebook, ddp_notebook_find_unused_parameters_false, ddp_sharded,
# ddp_sharded_find_unused_parameters_false, ddp_sharded_spawn,
# ddp_sharded_spawn_find_unused_parameters_false,
# ddp_spawn, ddp_spawn_find_unused_parameters_false,
# deepspeed, deepspeed_stage_1, deepspeed_stage_2, deepspeed_stage_2_offload,
# deepspeed_stage_3, deepspeed_stage_3_offload, deepspeed_stage_3_offload_nvme,
# dp, fsdp, fsdp_native, fsdp_native_full_shard_offload, horovod, hpu_parallel,
# hpu_single, ipu_strategy, single_device, single_tpu, tpu_spawn, tpu_spawn_debug"
## 愿景
创建一个模型工厂, 轻量且高效的训练程序，让训练模型更容易,更轻松上手。
## 交流
QQ交流群：185144988