https://github.com/deep-diver/llm-serve

This repository provides a framework to serve LLM(Large Language Model) based applications such as Chatbot.
https://github.com/deep-diver/llm-serve

Last synced: about 5 hours ago
JSON representation

This repository provides a framework to serve LLM(Large Language Model) based applications such as Chatbot.

• Host: GitHub
• URL: https://github.com/deep-diver/llm-serve
• Owner: deep-diver
• License: apache-2.0
• Created: 2023-04-18T07:43:16.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2023-04-20T04:56:02.000Z (over 1 year ago)
• Last Synced: 2024-04-21T17:24:34.483Z (6 months ago)
• Language: Python
• Size: 8.51 MB
• Stars: 17
• Watchers: 2
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# 💬 🚀 Serving LLM based Chatbot

This repository provides a generalized serving framework to serve LLM based application(currently aiming Chatbot). This is a fork project of [Alpaca-LoRA-Serve](https://github.com/deep-diver/Alpaca-LoRA-Serve). Alpaca-LoRA-Serve was initially written heavily dependent on Alpaca model, so it was not so easy to extend to support other models. In this project, I am going to refactor the code base a lot to make it concise and clean with the help of the other library that I built([Ping Pong or Bong Bong](https://github.com/deep-diver/PingPong))

## Screenshots

![](https://raw.githubusercontent.com/deep-diver/LLM-Serve/main/assets/preview.png)

## Todo

- [ ] Structure project

- [ ] Support Alpaca based application 

- [ ] Support GPT-Alpaca based application

- [ ] Support Flan based application

- [ ] Support Dolly based application

- [ ] Support backend only option (w/ FastAPI)

- [ ] Dockerfile