Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jianzhnie/open-chatgpt
The open source implementation of ChatGPT, Alpaca, Vicuna and RLHF Pipeline. 从0开始实现一个ChatGPT.
https://github.com/jianzhnie/open-chatgpt
chatgpt gpt llama llm lora peft ppo rlhf stanford-alpaca
Last synced: 3 days ago
JSON representation
The open source implementation of ChatGPT, Alpaca, Vicuna and RLHF Pipeline. 从0开始实现一个ChatGPT.
- Host: GitHub
- URL: https://github.com/jianzhnie/open-chatgpt
- Owner: jianzhnie
- License: apache-2.0
- Created: 2023-03-08T03:56:10.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-05-22T08:54:48.000Z (8 months ago)
- Last Synced: 2024-10-29T10:00:38.257Z (3 months ago)
- Topics: chatgpt, gpt, llama, llm, lora, peft, ppo, rlhf, stanford-alpaca
- Language: Python
- Homepage: https://jianzhnie.github.io/llmtech
- Size: 751 KB
- Stars: 175
- Watchers: 12
- Forks: 33
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-human-in-the-loop - Github - jianzhnie/open-chatgpt
README
# Open-ChatGPT: An open-source implementation of ChatGPT
[](https://github.com/jianzhnie/open-chatgpt/blob/main/LICENSE)
[](https://www.python.org/downloads/release/python-390/)
[](https://github.com/psf/black)
## Introduction
`Open-ChatGPT` is a open-source library that allows you to train a hyper-personalized ChatGPT-like ai model using your own data and the least amount of compute possible.
`Open-ChatGPT` is a general system framework for enabling an end-to-end training experience for ChatGPT-like models. It can automatically take your favorite pre-trained large language models though an OpenAI InstructGPT style three stages to produce your very own high-quality ChatGPT-style model.
We have Impleamented RLHF (Reinforcement Learning with Human Feedback) powered by transformer library and DeepsSpeed. It supports distributed training and offloading, which can fit extremly large models.
If you like the project, please show your support by [leaving a star ⭐](https://github.com/jianzhnie/open-chatgpt/stargazers).
## News
- \[2023/04\] We released **RLHF(Reinforcement Learning with Human Feedback) Pipeline** .
- \[2023/03\] We released the code **OpenChatGPT: An Open-Source libraray to train ChatBot like ChatGPT**.
## Install
```bash
git clone https://github.com/jianzhnie/open-chatgpt.git
pip install -r requirements.txt
```
**PEFT**
- If you would like to use LORA along with other parameter-efficient methods, please install [peft](https://github.com/huggingface/peft) as an additional dependency.
**DeepSpeed**
- If you want to accelerate LLM training using techniques such as pipeline parallelism, gradient checkpointing, and tensor fusion. Please install [DeepSpeed](https://github.com/microsoft/DeepSpeed).
## Contributing
Our purpose is to make this repo even better. If you are interested in contributing, please refer to HERE for instructions in contribution.
## License
`Open-ChatGPT` is released under the Apache 2.0 license.
## Acknowledgements
We appreciate the work by many open-source contributors, especially:
- [Hugging Face](https://huggingface.co/)
- [Vicuna](https://github.com/lm-sys/FastChat/)
## Citation
Please cite the repo if you use the data or code in this repo.
```
@misc{open-chatgpt,
author = {jianzhnie},
title = {Open-ChatGPT, An open-source implementation of ChatGPT},
year = {2023},
publisher = {GitHub},
journal = {GitHub repository},
howpublished = {\url{https://github.com/jianzhnie/open-chatgpt}},
}
```